Here is the rewritten article:

---

CameraX + TFLite depth maps under 35ms on Android

---

TL;DR

You can run monocular depth estimation at 30fps on a mid-range Android device by combining CameraX’s ImageAnalysis with a quantized MiDaS or Depth Anything v2 model, the XNNPACK delegate, and careful memory layout decisions. The bottleneck is almost never the model — it is the YUV-to-tensor conversion you wrote on day one.

---

Why bother now

Hugging Face hosts over 3 million models. Nvidia just spent $12.9 billion to acquire the platform. On-device inference is moving from research demo to product requirement — the models are there, the question is whether your Android pipeline can keep up.

Depth estimation from a monocular camera is one of the harder inference tasks to ship on mobile. Get it right and you unlock AR occlusion, real-time scene reconstruction, accessibility features. Get it wrong and you ship a thermal throttle machine that kills battery in six minutes.

---

The pipeline at a glance

CameraX ImageAnalysis
    └── ImageProxy (YUV_420_888)
        └── YUV → float32 tensor (CPU, reusable ByteBuffer)
            └── TFLite Interpreter + XNNPACK Delegate
                └── Depth map tensor [1, H, W, 1]
                    └── Min-max normalization (per-frame)
                        └── Bitmap / RenderScript output

Every stage has a failure mode. Most teams optimize the model and ignore the buffer. That’s the wrong order.

---

YUV-to-float tensor conversion: the real bottleneck

CameraX delivers frames as YUV_420_888. Your model wants a [1, 384, 384, 3] float32 tensor. The naive path — decode to Bitmap, then iterate pixels — costs 18–22ms on a Pixel 6 before the model even loads.

The correct path uses a pre-allocated ByteBuffer mapped directly from the Y, U, and V planes:

val yBuffer = image.planes[0].buffer
val uBuffer = image.planes[1].buffer
val vBuffer = image.planes[2].buffer

// Pre-allocated once, reused per frame
val inputTensor = ByteBuffer.allocateDirect(1 * 384 * 384 * 3 * 4)
    .order(ByteOrder.nativeOrder())

convertYuvToFloat(yBuffer, uBuffer, vBuffer, inputTensor)

Write convertYuvToFloat in Kotlin with manual plane stride handling. On a Snapdragon 8 Gen 1, this drops conversion from ~20ms to ~4ms. That single change is worth more than switching model architectures.

---

Model selection and quantization

Model	Size (FP32)	Size (INT8)	Latency (Pixel 7, XNNPACK)	mRel Err
MiDaS v2.1 Small	82 MB	21 MB	28ms	0.148
Depth Anything v2 Small	97 MB	25 MB	31ms	0.121
Depth Anything v2 Base	390 MB	98 MB	68ms	0.091

INT8 quantization via TFLite’s post-training pipeline cuts model size by ~75% and latency by 30–40%, with less than 4% relative accuracy degradation on standard benchmarks. Depth Anything v2 Small hits the 35ms target with headroom. Base doesn’t — not at 30fps.

---

XNNPACK delegate: thread affinity tuning

The default XNNPACK configuration spawns threads equal to device core count. On a big.LITTLE architecture, that means work scheduled on efficiency cores for latency-sensitive frames.

Pin threads to performance cores explicitly:

val options = Interpreter.Options().apply {
    addDelegate(
        XNNPackDelegate(
            XNNPackDelegate.Options().apply {
                numThreads = 4 // Match big-core count, not total
            }
        )
    )
    setNumThreads(4)
}

In my experience with heterogeneous SoCs, over-threading is a common trap. Four threads on big cores consistently outperforms eight threads across all cores for sustained inference.

---

Output normalization for per-frame consistency

Raw depth model output is inverse relative depth — values are not temporally stable. Apply per-frame min-max normalization before rendering:

val min = output.minOrNull() ?: 0f
val max = output.maxOrNull() ?: 1f
val range = (max - min).coerceAtLeast(1e-6f)
val normalized = output.map { (it - min) / range }

For smoother video, apply exponential moving average across frames with α = 0.85. This suppresses flickering without introducing perceptible lag.

---

Memory layout: staying under 500MB

Three allocations that matter, all done once at initialization:

• Input tensor: 1 × 384 × 384 × 3 × 4 bytes = ~1.7 MB
• Output tensor: 1 × 384 × 384 × 4 bytes = ~0.6 MB
• Model weights (INT8): ~21–25 MB, pinned

Never allocate inside the ImageAnalysis.Analyzer callback. Garbage collection during frame delivery is the single most common cause of jank in production.

---

Takeaways

1. Rewrite YUV conversion first. Pre-allocate your ByteBuffer and handle plane strides manually. This alone can recover 15ms per frame.
2. Use INT8 quantization and Depth Anything v2 Small. It hits 31ms on modern hardware with acceptable accuracy for most use cases.
3. Pin XNNPACK to your device’s big-core count. Measure with android.os.SystemClock.elapsedRealtimeNanos() under sustained load, not in isolation — thermal behavior changes everything.

---

#android #mobile #kotlin #architecture #jetpackcompose