Build Relay Lab reliability dashboard
This commit is contained in:
@@ -0,0 +1,33 @@
|
||||
# Relay Lab
|
||||
|
||||
A local reliability engineering demo with a SQLite-backed queue, two competing
|
||||
workers, and a live dashboard. The consumer is simulated. The database
|
||||
transactions, retries, and lease recovery are real.
|
||||
|
||||
Requires Python 3.10 or newer. From this directory:
|
||||
|
||||
```sh
|
||||
python3 app.py
|
||||
```
|
||||
|
||||
Open http://127.0.0.1:4176 and select a scenario. Use `--db PATH` to choose a
|
||||
database and `--port PORT` to choose a local port.
|
||||
|
||||
## Failure scenarios
|
||||
|
||||
- **Transient failure** retries after two and four seconds before succeeding.
|
||||
- **Worker interruption** abandons the first lease. Another claim recovers the
|
||||
job once that lease expires.
|
||||
- **Permanent failure** reaches the dead-letter state after three attempts.
|
||||
- **Resubmit** reuses an idempotency key and returns the same job ID. Reusing
|
||||
that key for a different scenario returns HTTP 409.
|
||||
|
||||
## Delivery guarantees
|
||||
|
||||
Job creation and its first event commit together. Workers serialize claims with
|
||||
a SQLite write transaction and use unique lease tokens to reject stale results.
|
||||
Retry timing, terminal state, and event history persist across restarts.
|
||||
|
||||
A worker can lose its lease after the consumer has processed an event and
|
||||
before the result is saved. A real consumer would need to deduplicate by job ID.
|
||||
This lab binds to loopback and is intended for a trusted local environment.
|
||||
Reference in New Issue
Block a user