The Reliability Envelope I Didn't Finish
An honest accounting of what this job queue guarantees, what it doesn't, and the config settings that do nothing at all.
#backend
13 posts
Tracing a Job That Outlives Its Request
Parent and child is a claim about containment. Queued work isn't contained by anything, so it needs a different relationship.
The Load Test That Was Supposed to Fail
173,000 requests with zero errors told me nothing useful. The test that broke its own thresholds told me everything.
What 'Processing' Means When the Worker Is Gone
A status column records what a job was doing. It cannot record whether anyone is still doing it.
A Timeout Doesn't Stop the Work
Deadlines protect the worker, not the job. Once blocking code is involved, the difference stops being academic.
Idempotency Is a Race Until the Database Says Otherwise
Check-then-insert looks correct and isn't. The only reliable fix is to stop checking and let a constraint decide.
Retries, Backoff, and Why Jitter Is Not Optional
Retrying is easy. Deciding when to retry, and making sure everyone doesn't decide the same thing, is the actual work.
Three Priority Tiers in One Number
A single Redis sorted set gives strict priority and FIFO inside each tier. Then I checked the arithmetic.
Two Stores, One Job, and the Gap Between Them
Splitting durable truth from fast dispatch buys a lot. It also creates a dual write I never made atomic.
Why I Didn't Just Use Celery
A job queue is a promise that work you accepted will eventually happen. Making that promise honestly is the hard part.
The Server Forgets Everything and That's Fine
Rooms live in memory and get thrown away. The client is the persistence layer, which changes what reconnecting means.
What Happens When One Client Can't Keep Up
Every broadcast loop contains an assumption that send() works. Backpressure is what you build when it doesn't.
Writing a Binary WebSocket Protocol by Hand
What actually goes over the wire in a collaborative canvas, and why the ready made library didn't fit.