Wiring Android's WorkManager to a Quantized On-Device LLM for Background Summarization: Constraints, Expedited Tasks, and the Memory Ceiling That Determines Your Model Tier
TL;DR
Running a quantized LLM inside WorkManager is not a drop-in job — it’s an architectural commitment. The memory ceiling on mid-range devices (~2 GB addressable per process) forces you to choose between INT4 and INT8 quantization before writing a single line of scheduling code. Doze mode will kill long-running inference unless you promote to a foreground service. Chained work sequences are the right primitive for chunked summarization. Get the model tier decision right first; everything else follows.
Why background LLM summarization is different
WorkManager’s execution model assumes Workers complete in seconds to low minutes — assumptions that break under 200 ms-per-token LLM inference. The failure modes are silent: OOM kills mid-inference, Doze-mode constraint violations that defer work by hours, and foreground-service promotion that confuses users with persistent notifications.
These examples assume llama.cpp via Android JNI as the inference runtime — the same path used by projects like llama.android and MLC LLM — though the scheduling patterns apply equally to MediaPipe’s LLM Inference API.
A typical mid-range device (Snapdragon 6 Gen 1, 6 GB RAM) allocates roughly 3.5 GB to the system and background processes, leaving your app process with a practical ceiling around 1.8–2.2 GB before the OOM killer becomes aggressive. That ceiling determines your entire model strategy.
The memory ceiling: INT4 vs INT8
| Model Size (params) | INT8 RAM (approx) | INT4 RAM (approx) | Safe on 6 GB device? |
|---|---|---|---|
| 1B | ~1.0 GB | ~0.6 GB | Both tiers |
| 3B | ~3.0 GB | ~1.7 GB | INT4 only |
| 7B | ~7.0 GB | ~4.0 GB | Neither — INT4 footprint (~4 GB) exceeds available RAM on most 6 GB devices even with foreground promotion; treat 7B as server-side territory |
| 1.5B (Phi-2 class) | ~1.5 GB | ~0.9 GB | Both with headroom |
For background Workers without foreground promotion, target sub-1B INT4 or sub-1.5B INT4 models. Anything larger requires a foreground service, and anything at 7B or above should not be attempted on-device on current mid-range hardware. If you find yourself trying to rationalize a 7B model on a 6 GB device, that’s a signal to move the inference server-side, not to keep tuning constraints.
Scheduling the work: constraints that actually matter
WorkManager’s Constraints API gives you knobs that directly map to inference viability:
val inferenceConstraints = Constraints.Builder()
.setRequiresBatteryNotLow(true) // inference drains battery fast
.setRequiredNetworkType(NetworkType.NOT_REQUIRED)
.build()
val summarizeRequest = OneTimeWorkRequestBuilder<SummarizationWorker>()
.setConstraints(inferenceConstraints)
.setExpedited(OutOfQuotaPolicy.RUN_AS_NON_EXPEDITED_WORK_REQUEST)
.setInputData(workDataOf("chunk_index" to 0, "total_chunks" to 3))
.build()The setExpedited call is critical for user-triggered summarization. Expedited tasks bypass Doze-mode deferral and execute within minutes rather than hours, but they require a getForegroundInfo() override on your Worker — WorkManager will call it on older API levels to attach a notification.
Chained work for chunked summarization
What most teams get wrong about long-document summarization: they try to load the entire document in one Worker and blow the memory budget or hit the 10-minute execution window. The correct pattern is a chain of chunk Workers feeding a reduce Worker.
val chunkWorkers = (0 until totalChunks).map { index ->
OneTimeWorkRequestBuilder<ChunkSummarizeWorker>()
.setInputData(workDataOf("chunk" to index))
.build()
}
val reduceRequest = OneTimeWorkRequestBuilder<ReduceSummaryWorker>().build()
WorkManager.getInstance(context)
.beginWith(chunkWorkers) // parallel fan-out
.then(reduceRequest) // serial reduce
.enqueue()Each ChunkSummarizeWorker loads the model, runs inference on a ~500-token window, unloads, and writes its partial summary to the output Data map. The ReduceSummaryWorker concatenates partials and optionally runs a second-pass summary. Model load/unload per chunk is expensive (~200–400 ms for INT4 1B models on a Snapdragon 6 Gen 1), but it keeps peak RSS below the OOM threshold.
Foreground service promotion for larger models
If your product requires a 3B model, you must promote the Worker to a foreground service. On CoroutineWorker, that means implementing getForegroundInfo():
override suspend fun getForegroundInfo(): ForegroundInfo {
val notification = buildSummarizationNotification()
return ForegroundInfo(NOTIFICATION_ID, notification,
ServiceInfo.FOREGROUND_SERVICE_TYPE_SHORT_SERVICE)
}FOREGROUND_SERVICE_TYPE_SHORT_SERVICE (API 34+) gives you up to 3 minutes of guaranteed execution without requiring a declared use-case permission — a practical sweet spot for 3B INT4 inference on a chunked document.
Conclusion
Three decisions determine whether this architecture ships or collapses in production.
First, select your model tier before writing scheduling code. Profile peak RSS on your minimum-spec device under load. If it exceeds 1.8 GB, you need either a smaller model or foreground service promotion — not a WorkManager tuning pass.
Second, use chained Workers for any document over ~1,500 tokens. Fan-out chunk Workers running in parallel, reduce with a serial terminal Worker. This keeps individual Worker memory bounded and works within WorkManager’s execution window.
Third, always set setExpedited for user-initiated summarization. Doze-mode deferral on background work can push execution by hours. Expedited tasks with a foreground notification keep latency in the seconds-to-minutes range and give users visible feedback that inference is running.