Storage benchmark · AWS
Shared-storage performance for the context layer.
The context engine TemporalStore stores memory as an append-structured block store, then keeps one durable copy on the MatrixObject shared object store Enterprise so serving nodes stay stateless. On AWS, MatrixObject beat EFS on the shared-storage write path: sync write p99 6.25 ms vs 29.45 ms, and async flush batch p99 6.70 ms vs 236.10 ms.
When context memory is the product, the storage tier under it decides two things: how fast a new memory becomes durable, and how quickly you can add serving capacity when traffic spikes. In shared-storage mode, TemporalStore separates those concerns — compute and storage are disaggregated, the durable data lives once in MatrixObject, and datanodes become stateless compute you can add, remove, or replace without moving bytes.
What we measured
- Concurrent throughput — sustained writes/sec and reads/sec under mixed load.
- Latency — p50 / p95 / p99 for reads and writes while the other side is active.
- Horizontal scaling — throughput as stateless datanodes are added over the shared store, with no data movement.
- Durability — no data loss under the shared-store write policy (sync vs async).
Engine Rust TemporalStore (append-structured block store; no LSM write-amp)
Shared store Rust MatrixObject object store (one durable copy; disaggregated)
Mode Shared-storage; stateless datanodes serve over the object store
Workload Concurrent reads + writes (per-shard ingest + query together)
Environment AWS us-west-2; c7i.large workers and MatrixObject node
Comparison AWS EFS shared storage vs MatrixObject shared object store
Validation 8,000 shared-store ops, 256-way concurrency, 3 samples/backend
Code version TemporalStore / MatrixObject commit 61ffcbeAWS results: MatrixObject vs EFS
Three-run averages. Lower is better for latency and wall-clock; higher is better for throughput. Replay lag stayed at zero for both.
| Metric | EFS shared storage | MatrixObject shared store | Result |
|---|---|---|---|
| Sync storage write p50 | 9.85 ms | 3.56 ms | 63.9% lower latency |
| Sync storage write p99 | 29.45 ms | 6.25 ms | 78.8% lower latency |
| Async durable write p99 | 11.80 ms | 0.33 ms | 97.2% lower latency |
| Async flush batch p99 | 236.10 ms | 6.70 ms | 97.2% lower latency |
| Concurrent sync write p99 | 3,284.83 ms | 134.12 ms | 95.9% lower latency |
| Concurrent sync write throughput | 120.29 ops/s | 143.90 ops/s | 19.6% higher throughput |
| Concurrent sync read throughput | 27,246.44 ops/s | 29,424.09 ops/s | 8.0% higher throughput |
| Concurrent sync read p99 | 1.60 ms | 1.55 ms | 3.3% lower latency |
| Concurrent sync lag after replay | 0 ops | 0 ops | consistent replay |
| Wall clock | 2,514.75 s | 339.06 s | 86.5% lower elapsed time |
Latest AWS report generated 2026-08-10 at 18:48 Pacific for image
matrixobject-rust:opt-61ffcbe-20260811-0137.
Shared sync vs Raft replication
Raft wins the narrow small-write latency path in this run. The old Raft throughput rows are harness-limited and need a saturation rerun, because the reported ops/s values do not line up with the 1-2 ms write latency. MatrixObject's concurrent read/write rows remain the scale-out shared-store result.
| Metric | Raft data-node replication | MatrixObject shared sync | Result |
|---|---|---|---|
| Write p50 | 1.10 ms | 8.92 ms | Raft lower on small write latency |
| Write p99 | 2.12 ms | 38.64 ms | Raft lower on small write latency |
| Secondary visibility p99 | 0.85 ms | 0.34 ms | 60.1% lower visibility latency |
| Write throughput | 1.85 ops/s harness-limited; rerun needed | 64.23 ops/s | not a clean saturation comparison |
| Read throughput | 0.31 ops/s harness-limited; rerun needed | 64.23 ops/s | not a clean saturation comparison |
| Concurrent write throughput | 1.85 ops/s harness-limited; rerun needed | 143.90 ops/s | MatrixObject scale number retained; Raft needs rerun |
| Concurrent read throughput | 0.31 ops/s harness-limited; rerun needed | 29,424.09 ops/s | MatrixObject scale number retained; Raft needs rerun |
Why shared storage scales context serving
One durable copy, stateless compute, and an append-structured write path.
Because durable data lives once in MatrixObject, adding serving capacity is just adding stateless compute. Combined with TemporalStore's append-structured block store — batched WAL, deferred fsync — that is what keeps concurrent read + write throughput high on AWS.
The read path is strong relative to EFS here (29,424.09 vs 27,246.44 ops/s): serving context is not where the shared store earns its keep. The win is on the durable write path and on wall-clock — the AWS validation finished in 339.06 s vs 2,514.75 s, an 86.5% reduction, with zero replay lag on both backends.