Redis Beyond Caching: Sorted Sets for Leaderboards, Streams for Event Sourcing, and the Lua Scripting Patterns That Replace Three Microservices With One Redis Instance

TL;DR

Redis is not a cache. It’s a data structure server. Sorted sets give you real-time leaderboards with O(log N) updates. Redis Streams work as a lightweight Kafka alternative with consumer groups and acknowledgment semantics. Atomic Lua scripting lets you perform complex multi-key operations without race conditions, often replacing dedicated coordination services entirely. I’ve seen a single well-configured Redis instance absorb the responsibilities of three separate microservices in production.

---

What most teams get wrong about Redis

Most engineering teams bolt Redis onto their stack as a TTL-based key-value cache in front of PostgreSQL. That covers roughly 10% of what Redis actually does. Redis is a programmable, in-memory data structure server with first-class support for sorted sets, streams, hyperloglogs, bitmaps, and atomic scripting. Once you start treating it as a primary data layer for specific workloads, entire services become unnecessary.

Here are the three patterns that have saved me the most headaches.

---

Sorted sets: real-time leaderboards in O(log N)

The ZSET doesn’t get enough credit. Every insert, update, and rank lookup runs at O(log N) against a skip list internally.

ZADD leaderboard 1500 "player:42"
ZADD leaderboard 1620 "player:17"
ZINCRBY leaderboard 30 "player:42"
ZREVRANK leaderboard "player:42"    -- returns 0 (top rank)
ZREVRANGE leaderboard 0 9 WITHSCORES -- top 10

Compare that to the relational alternative:

Operation	PostgreSQL	Redis Sorted Set
Update score	UPDATE + re-index: O(N log N) sort or indexed query	ZINCRBY: O(log N)
Get rank	SELECT COUNT(*) WHERE score > x: full scan or materialized view	ZREVRANK: O(log N)
Top-K query	ORDER BY score DESC LIMIT K: index scan	ZREVRANGE 0 K: O(log N + K)
Concurrent writers	Row-level locks, potential deadlocks	Single-threaded, no locks needed

At 1 million players, ZREVRANK returns in under 1ms. The PostgreSQL equivalent with a B-tree index on score still requires a range count. I’ve measured consistent sub-millisecond p99 latencies on sorted sets with 5M+ members in production. That’s not a benchmark game; it just stays flat.

---

Redis Streams: lightweight event sourcing without Kafka

Redis Streams (XADD, XREAD, XREADGROUP) give you an append-only log with consumer groups, message acknowledgment, and pending entry tracking.

Feature	Apache Kafka	Redis Streams
Setup complexity	ZooKeeper/KRaft + brokers + schema registry	Single Redis instance or cluster
Consumer groups	Yes	Yes (XREADGROUP)
Message acknowledgment	Offset commit	XACK per message
Retention	Time/size-based	Time/size-based (MAXLEN, MINID)
Throughput ceiling	1M+ msg/s per partition	~200K msg/s per stream (single instance)
Operational overhead	High (JVM tuning, partition rebalancing)	Low

-- Producer: append event
XADD orders:events * action "placed" order_id "ord-991" total "89.99"

-- Consumer group setup
XGROUP CREATE orders:events fulfillment-svc $ MKSTREAM

-- Consumer: read and acknowledge
XREADGROUP GROUP fulfillment-svc worker-1 COUNT 10 BLOCK 2000 STREAMS orders:events >
XACK orders:events fulfillment-svc 1684012345678-0

For systems processing under 200K events per second (which honestly covers most startups and mid-scale SaaS products), Redis Streams eliminate the entire Kafka operational burden. You get consumer groups, pending entry lists for retry logic (XPENDING), and XCLAIM for rebalancing dead consumers. That’s a complete event sourcing backbone without a single JVM process.

The tradeoff is real, though. Kafka wins when you need multi-datacenter replication or million-message-per-second partitions. Redis Streams aren’t trying to be Kafka. They’re trying to be the 80% solution that saves you from running Kafka when you don’t need it.

---

Lua scripting: atomic multi-key operations that kill race conditions

This is where things get interesting. A Lua script executes atomically on the Redis server. No other command runs between your script’s operations. This eliminates distributed locks, saga orchestrators, and retry middleware for many common patterns.

Consider a rate limiter with a sliding window:

-- KEYS[1] = rate limit key, ARGV[1] = window (sec), ARGV[2] = max requests, ARGV[3] = now
local key = KEYS[1]
local window = tonumber(ARGV[1])
local max_req = tonumber(ARGV[2])
local now = tonumber(ARGV[3])

redis.call('ZREMRANGEBYSCORE', key, 0, now - window * 1000)
local count = redis.call('ZCARD', key)
if count < max_req then
    redis.call('ZADD', key, now, now .. '-' .. math.random(1000000))
    redis.call('PEXPIRE', key, window * 1000)
    return 1
end
return 0

Without Lua, this pattern requires a distributed lock (Redlock or a separate service) to prevent TOCTOU races between ZCARD and ZADD. With Lua, it’s a single atomic call via EVALSHA. I used this pattern to replace a dedicated rate-limiting microservice: its API gateway sidecar, its own Redis instance, its deployment pipeline. Gone. Twelve lines of Lua loaded into the existing Redis instance.

What Lua scripts replace

Without Lua	With Lua
Distributed lock service (Redlock, ZooKeeper)	Atomic script execution
Saga orchestrator for multi-key writes	Single EVALSHA call
Retry/idempotency middleware	redis.call sequences with conditional logic
Separate rate-limiter microservice	Lua script on existing Redis

---

So what should you actually do?

First, audit your cache-only Redis usage. If you’re only using GET/SET/EXPIRE, you’re ignoring 90% of what’s available. Look for ranking, counting, or time-series workloads that sorted sets handle natively.

Second, evaluate Redis Streams before reaching for Kafka. If your event throughput is under 200K messages per second, Redis Streams give you consumer groups and acknowledgment semantics at a fraction of the operational cost.

Third, move coordination logic into Lua scripts. Any multi-step Redis operation that currently relies on application-level locking is a candidate. Rate limiting, inventory reservation, token bucket algorithms. If it touches multiple keys and needs atomicity, a Lua script eliminates both the race condition and the extra service.

Redis is not your cache layer. It’s a programmable data engine. I think most teams would be surprised how much of their architecture simplifies once they start treating it that way.