Freshness Benchmark

Vector database freshness.

How quickly do inserts, replacements, and deletions become visible?

60 trials · 19,350 changes · 3 databasesDownload CSV ↓How it works →
2. Choose a change

Time until the change is visible

Milliseconds050100150pgvector: 35.70 ms, 95% interval 34.60 to 36.61 ms35.70pgvectorQdrant: 72.88 ms, 95% interval 72.11 to 105.24 ms72.88QdrantWeaviate: 34.05 ms, 95% interval 32.88 to 35.35 ms34.05Weaviate
Lower bars mean faster observations. Select a bar for its exact values.Median trial p95 · linear scale starts at zero
How to read this graph

Both a direct read and a search result have shown the new state. Lower is faster.

Times are medians of five trial p95s. The interval shows uncertainty in that estimate. Help me read these numbers →

Wikipedia passages and real questions. Ten changes and ten background queries per second.

pgvector

Version 0.8.7 · 5 trials

35.70milliseconds

95% interval: 34.60–36.61 ms

Changes confirmed
100.0%
Search recall
99.4%
Validity checks
All passed
More metrics and individual trials
Repeated confirmation, p95
97.73 ms
Background query, p95
15.80 ms
Confirmed within 100 ms
97.6%
Relevance score (nDCG@10)
0.79

Trial 1 →Trial 5 →Trial 2 →Trial 3 →Trial 4 →

Qdrant

Version 1.20.0 · 5 trials

72.88milliseconds

95% interval: 72.11–105.24 ms

Changes confirmed
100.0%
Search recall
98.9%
Validity checks
All passed
More metrics and individual trials
Repeated confirmation, p95
222.88 ms
Background query, p95
12.26 ms
Confirmed within 100 ms
0.0%
Relevance score (nDCG@10)
0.79

Trial 3 →Trial 5 →Trial 4 →Trial 2 →Trial 1 →

Weaviate

Version 1.40.3 · 5 trials

34.05milliseconds

95% interval: 32.88–35.35 ms

Changes confirmed
100.0%
Search recall
99.1%
Validity checks
All passed
More metrics and individual trials
Repeated confirmation, p95
106.31 ms
Background query, p95
16.00 ms
Confirmed within 100 ms
78.6%
Relevance score (nDCG@10)
0.79

Trial 4 →Trial 3 →Trial 2 →Trial 1 →Trial 5 →

Times describe confirmed changes. Reliability includes all offered changes.

Single-host reference · Apple M4 · 2 CPUs / 3 GiB per database. Scope and limitations →