First notes on bringing AI into Android apps
A mobile engineer's map of the territory — on-device models, cloud APIs, and where each one makes sense.
This is a sample post — replace it with your own. It’s here to show how code, lists and headings look.
When people say “add AI to the app”, they usually mean one of two very different things.
Option 1: call a model in the cloud
The simplest path. Your app sends a request, a large model does the work, and you stream the answer back.
suspend fun summarize(text: String): String = withContext(Dispatchers.IO) {
val response = api.complete(
prompt = "Summarize in two sentences:\n$text",
maxTokens = 200,
)
response.text.trim()
}Good for: quality, big context, fast iteration. Trade-offs: latency, cost per call, and the user’s data leaves the device.
Option 2: run a small model on the device
Smaller models can now run on modern phones for tasks like classification, transcription and short text generation.
| Cloud | On-device | |
|---|---|---|
| Quality | Highest | Good for narrow tasks |
| Latency | Network-bound | Instant |
| Privacy | Data leaves device | Stays local |
| Cost | Per request | Free after download |
What I want to explore next
- How to design UX around a response that streams instead of loads.
- Falling back gracefully between on-device and cloud.
- Testing features whose output isn’t deterministic.
More soon.