eBPF for mobile API observability: zero-touch tracing

Meta description: Use eBPF TC hooks and uprobes on OpenSSL to trace Android OkHttp requests end-to-end at the kernel level — no code changes, no instrumentation overhead.

---

TL;DR

eBPF lets you intercept mobile API traffic at the kernel and TLS layer on your backend — correlating Android OkHttp trace IDs into full end-to-end flame graphs — without touching a single line of application code.

---

The observability gap nobody talks about

In my experience building production systems, the hardest latency to debug is the kind you can’t see. A mobile user reports slowness. Your APM dashboard shows green. Your backend p99 looks fine. So where did the 800ms go?

The answer is usually scattered across four places: the Android HTTP stack, the network, TLS negotiation, and server-side queueing — none of which your application-level tracing stitches together by default.

Most teams get mobile observability wrong the same way: they instrument the app layer and the backend layer independently, then try to correlate them with trace IDs passed through HTTP headers. This works — until it doesn’t. SDK versions drift. OkHttp interceptors get removed in refactors. Sampling rates diverge. And TLS makes the network layer completely opaque.

eBPF changes this entirely.

---

What eBPF actually gives you

eBPF (Extended Berkeley Packet Filter) allows you to run sandboxed programs in the Linux kernel — attached to network interfaces, system calls, or userspace function probes — with near-zero overhead and no application changes.

For mobile API observability, three attachment points matter:

Attachment Point	What It Captures	Overhead
XDP (eXpress Data Path)	Raw packets at NIC driver level	Minimal — pre-kernel stack
TC (Traffic Control) hooks	Ingress/egress at socket buffer level	Low — post-routing
uprobes on OpenSSL/BoringSSL	Plaintext data post-TLS handshake	Low — userspace probe

XDP is your fastest path for packet-level metadata. TC hooks give you structured socket buffer access. But for TLS traffic — which is everything from Android — uprobes on the TLS library are where the real work happens.

---

Cracking TLS without breaking it

Android’s HTTP stack (OkHttp backed by BoringSSL) encrypts everything before it hits the wire. On your backend, if you’re running an Nginx or Envoy TLS terminator, the kernel never sees plaintext.

The approach: attach a uprobe to SSL_write and SSL_read in your backend’s OpenSSL or BoringSSL shared library. These fire after the handshake, at the point where plaintext data is passed into the TLS engine. Your eBPF program reads from the buffer at that moment.

// Simplified uprobe attachment concept
SEC("uprobe/SSL_write")
int trace_ssl_write(struct pt_regs *ctx) {
    int fd    = (int)PT_REGS_PARM1(ctx);
    void *buf = (void *)PT_REGS_PARM2(ctx);
    int num   = (int)PT_REGS_PARM3(ctx);

    // Read HTTP headers from buf, extract trace ID
    // Emit correlation event to ring buffer
    return 0;
}

From here, you extract the traceparent or X-Request-ID header injected by OkHttp, and you have your correlation anchor — captured at the kernel boundary with no middleware involved.

---

Correlating Android OkHttp to backend flame graphs

Let me walk you through the architecture.

On the Android side, OkHttp’s Interceptor interface injects a W3C traceparent header on every outbound request. Standard distributed tracing — nothing exotic.

On the backend, the eBPF pipeline works as follows:

1. The TC ingress hook captures the incoming TCP segment, records arrival timestamp, source IP, and connection tuple.
2. The uprobe on SSL_read captures the decrypted HTTP request headers and extracts traceparent.
3. kprobes on socket operations track kernel queueing time before your application thread picks up the request.
4. All events flow into a perf ring buffer, consumed by a userspace collector — typically a Go or Rust daemon — that assembles the full trace timeline.

The result: a flame graph showing network transit time, TLS handshake duration, kernel queueing latency, and application processing time, all correlated to the exact Android request that triggered it. No SDK changes, no backend code changes.

---

Overhead in practice

eBPF programs run in the kernel with JIT compilation and verifier-enforced safety. Typical overhead for TC hook and uprobe combinations on high-throughput services runs in the single-digit microsecond range per request — well below the noise floor of any latency budget that matters at mobile scale.

Contrast this with application-level APM agents, which introduce middleware, heap allocations, and serialization overhead on every request path. For latency-sensitive mobile APIs, that difference compounds at p99 and above.

---

Where to start

If you’re running OpenSSL or BoringSSL on your backend, you already have everything you need. Tools like bpftrace let you prototype uprobe attachment in minutes before committing to a production pipeline — start there, before building anything.

Before you touch the eBPF pipeline at all, inject W3C traceparent headers in your OkHttp interceptor. It costs nothing and gives your future observability infrastructure a correlation anchor to work from.

When you do build the collector, deploy it as a sidecar rather than a library bundled into your application. Keeping eBPF programs and ring buffer consumers out of your application process means you can update your observability stack independently — and it survives application restarts without drama.

---

#android #mobile #backend #api #architecture