| OS and kernel | Linux 6.17.0-1022-azure |
|---|---|
| CPU model | AMD EPYC 9V45 96-Core Processor |
| Host logical cores | 4 |
| Effective cores | 4 |
| Affinity cores | 4 |
| Allowed affinity | 0-3 |
| Pinned CPU | 0 |
| CPU quota | |
| Host RAM | MemTotal: 16373448 kB |
| Available RAM | MemAvailable: 15240160 kB |
| Memory limit | |
| Test DB filesystem | type=ext4 total_kib=151263856 available_kib=86859916 |
| Rust | rustc 1.101.0-nightly (8006091eb 2026-10-17) |
| SQLite | 3.53.2 |
| Engine revision | 4176f6bc67c9ca444257183358ff83b8c943325d |
| Dirty source | false |
Scanner driver: scheduler
One CPU; file-backed SQLite with real migrations, WAL and synchronous=NORMAL. Plain custody and the production scanner run against a deterministic scripted daemon while independent SQLite and HTTP status readers plus an admin writer issue background work. A separate connection holds a SQLite write lock for a fixed interval each tick. The capacity sweep uses immediate RPC responses; the fault points below add scheduled RPC delay and failures, and a longer blocked writer followed by a drain period. A blocked writer is a contention proxy, not a real disk fault. Capacity below is observational for this workload and machine.
Observed capacity bracket: at least 128 tenants sustainable; no overload observed. Workload: open orders per tenant as recorded in run.json plus one tenant with 300 open orders, one transaction per block, one new block per tick, 2 database readers, 2 HTTP status readers, 1 admin writer, a 25 ms SQLite write lock per tick, 5 second tick budget, at most three blocks of oldest lag. Only this machine and schedule were tested.
| Tenants | Status | Measured time | Tick p50/p95/p99 | Timer delay | DB reads/writes | HTTP reads | Max DB read/write latency | Max queue wait read/write | Max query read/write | Max HTTP latency | WAL pending pages | Lag after drain | Raw data |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 32 | sustainable | 165 ms | 41/42/42 ms | 1756 µs | 72/11 | 38 | 1169/34337 µs | n/a/33605 µs | n/a/33594 µs | 2588 µs | 0 | 0 | JSON |
| 64 | sustainable | 186 ms | 46/47/47 ms | 1930 µs | 82/19 | 44 | 898/33861 µs | n/a/32906 µs | n/a/33610 µs | 1323 µs | 0 | 0 | JSON |
| 128 | sustainable | 233 ms | 58/59/59 ms | 2052 µs | 103/29 | 54 | 2267/33816 µs | n/a/32795 µs | n/a/33562 µs | 1527 µs | 0 | 0 | JSON |
Blue: network high-water; orange: slowest tenant cursor. The gap is work still to catch up.
| Fault | Result | Fault detail | Final slowest cursor | Raw data |
|---|---|---|---|---|
| transient daemon RPC failures under SQLite contention | recovered | 3 injected RPC failures | 8 | JSON |
| blocked SQLite writer, then recovery | recovered | 1500 ms per tick for two ticks | 8 | JSON |
| single custody scan slot under concurrent tenant scans | recovered | 128 scans; max slot wait 102507 µs | 8 | JSON |
The custody fault uses one scan slot with a scripted 5 ms service delay while retaining real plain-custody matching. Actual slow disk commands and process-kill recovery are not measured by this fixture. The RPC fault point uses deterministic transient failures and a drain period; the capacity sweep has no RPC delay.