OOmnistu

Blog / Mobile App Development

Mobile App Development

On-device AI vs cloud AI: choosing where your app's model runs

Apple's Foundation Models framework and Google's ML Kit GenAI run models on the phone. Privacy, latency, cost and capability trade-offs against cloud inference.

OOmnistu
Reviewed 2026-11-19 · 3 min read

Mobile AI features now come in two shapes: models that run on the phone itself, and models your app calls over the network. Both Apple and Google shipped first-party on-device frameworks - Apple's Foundation Models framework and Google's ML Kit GenAI APIs - so the choice is no longer between "cloud AI" and "no AI". It is a real architecture decision with different privacy, cost and capability profiles.

This guide compares the two approaches using Apple's and Google's own documentation, checked on 19 November 2026, so you can pick deliberately instead of by default.

What on-device AI offers now

Apple's Foundation Models framework, introduced at WWDC25, gives apps direct access to the on-device large language model behind Apple Intelligence through a Swift API. The framework covers guided generation (emitting Swift data structures instead of free text), streaming for responsive interfaces, tool calling so the model can use your app's data sources, and stateful sessions for context management.

Google's ML Kit GenAI APIs run on Gemini Nano through AICore, an Android system service that manages on-device foundation models and accelerates inference while keeping user data on the device. ML Kit offers two levels: task-specific APIs that need no prompt engineering (summarization, proofreading, rewriting in different tones, image description, speech recognition and translation, text-to-speech, and embeddings that pair with AndroidX AppSearch for meaning-based search), and direct model access through the GenerativeModel class for custom prompts, structured output, multi-turn chat and agentic workflows via AgentRunner.

The trade-offs, stated plainly

A decision rule that holds up

Prefer on-device when the task is short-form, privacy-sensitive, latency-sensitive or useful offline: summarizing a user's own messages, proofreading, describing photos, translating speech. Prefer cloud when the task needs broad knowledge, long context, or your own data - retrieval over your documents still means a server, and RAG explained for founders covers that architecture. Many production features will end up hybrid: try on-device first, fall back to cloud for what the local model cannot do, and tell the user when their content leaves the device.

Whichever side you start on, keep the model behind your own interface. The vendors iterate their frameworks quickly, and a thin internal abstraction is cheaper than a rewrite when capabilities shift. If you are still at the "should this feature exist" stage, build, buy or embed is the earlier decision.

Sources

Drafted with AI assistance from these sources. Device support matrices and framework capabilities change with each OS release; verify against current Apple and Google documentation before committing to either approach for a specific feature.