Ship on-device AI with Apple Foundation Models + SwiftUI

Meta description: Wire Apple’s on-device Foundation Models to SwiftUI: streaming with AsyncSequence, memory constraints, and latency tradeoffs for iOS 18.4+.

---

TL;DR

Apple’s Foundation Models framework (iOS 18.4+) gives you private, on-device inference through a clean Swift API. The session-based design, AsyncSequence streaming, and built-in Prompt Guard make it production-ready — but memory constraints on A17/M-series chips are real. Knowing when to stay on-device versus fall back to server inference is the architectural decision that separates shipped apps from demos.

---

The session API: less surface area than you expect

Foundation Models exposes inference through LanguageModelSession — a session-scoped object that handles context, system prompts, and response generation. The API surface is deliberately minimal:

import FoundationModels

let session = LanguageModelSession(
    instructions: "You are a concise summarization assistant."
)

let response = try await session.respond(
    to: "Summarize this note in one sentence.",
    options: .init(temperature: 0.7)
)
print(response.content)

Inference never leaves the device. No network call, no data leaving the user’s phone. For health, finance, and productivity apps handling sensitive personal data, this eliminates an entire class of privacy risk and removes the server inference cost line from your P&L.

---

Streaming to SwiftUI with AsyncSequence

Synchronous respond(to:) works for short completions. For anything the user reads in real time, you want streaming. Foundation Models exposes this via AsyncSequence, which composes cleanly with SwiftUI’s task modifier:

@State private var output = ""

var body: some View {
    ScrollView { Text(output).padding() }
        .task {
            let session = LanguageModelSession()
            for try await partial in session.streamResponse(to: prompt) {
                output += partial.text
            }
        }
}

No third-party dependencies, no Combine pipelines, no manual dispatch queue management. Most teams reach for Combine out of habit — AsyncSequence + task is the right abstraction for this pattern in modern Swift concurrency, and it’s already there.

---

Memory tiers and the on-device budget

On-device inference is not free. Apple’s model sits at approximately 3 billion parameters (per WWDC 2025 session “Explore the Foundation Models framework”). Neural Engine headroom and unified memory vary significantly across the supported lineup:

Chip Tier	Neural Engine Cores	Unified Memory (typical)	Context Headroom
A17 Pro	16-core	8 GB	~4 GB usable for ML workloads
A18 / A18 Pro	16-core (2nd gen)	8 GB	Higher throughput, lower thermal pressure
M1 / M2 (iPad, Mac)	16–32 core	8–16 GB	Largest; lowest fallback risk
A16 and below	Not supported	—	Always route to server

Note: A16 and below lack Apple Intelligence support per Apple’s published hardware requirements. Verify the exact availability check symbol — LanguageModelSession.isSupported or equivalent — against the current SDK documentation before shipping, as this API surface was evolving through the iOS 18.x cycle.

Gate on hardware availability before committing to an inference path, and build your server fallback as a first-class path — not an afterthought:

func runInference(prompt: String) async throws -> String {
    // Verify exact symbol name against current SDK docs
    guard LanguageModelSession.isSupported else {
        return try await serverInference(prompt: prompt)
    }

    let session = LanguageModelSession(
        instructions: "You are a concise assistant."
    )
    let response = try await session.respond(to: prompt)
    return response.content
}

func serverInference(prompt: String) async throws -> String {
    // Route to your API endpoint for unsupported devices
    var request = URLRequest(url: serverEndpointURL)
    request.httpMethod = "POST"
    request.httpBody = try JSONEncoder().encode(["prompt": prompt])
    let (data, _) = try await URLSession.shared.data(for: request)
    return try JSONDecoder().decode(InferenceResponse.self, from: data).text
}

In my experience building production systems, teams that skip representative prompt testing across device tiers discover their fallback paths in production — sometimes during App Store review. Build a small evaluation suite of realistic prompts early, run it on each hardware tier, and watch end-to-end behavior. Evaluation is part of shipping, not a QA phase after the fact.

---

Prompt Guard: the safety layer you cannot disable

Prompt Guard runs automatically on every inference call, screening for prompt injection and policy violations before the model processes input. You cannot opt out. Apple has not published precise per-device latency figures for this layer, so treat any specific numbers in circulation with skepticism until you measure on your target hardware. The practical implication: don’t build flows that depend on sub-100ms first-token latency on the initial call in a session. For conversational UI patterns — summarization, journaling, contextual recommendations — the overhead is invisible to users in practice.

---

On-device vs. CoreML: the honest tradeoff

Dimension	Foundation Models	CoreML Custom Model
Integration effort	Low (< 10 lines)	High (training, conversion, versioning)
General language quality	High	Depends on training data
Latency control	Limited	Full control
Privacy	On-device by default	On-device, but your pipeline
Model updates	Apple-managed	You own the lifecycle

Foundation Models wins on time-to-production for general language tasks. CoreML wins when you need domain-specific accuracy or deterministic sub-50ms inference. Don’t reach for CoreML because it feels more “engineered” — the integration cost is real and ongoing.

---

3 things to internalize before you ship

1. Gate on hardware availability from day one and ship the fallback path alongside it. Device capability is a first-class architectural input, not an edge case to handle later. The code above shows exactly where that seam lives.

2. Use AsyncSequence + task for all streaming UI. It’s the idiomatic Swift concurrency pattern and requires zero additional dependencies in a SwiftUI app.

3. Evaluate on representative prompts across device tiers before submitting. Discovering fallback behavior in production is preventable. Run your prompt suite on A17 Pro, M-series, and an unsupported device before you hit App Store review.

---

Tags: #ios #swift #swiftui #mobile #architecture