How Vector Databases Actually Differ (and How to Choose)

"Which vector database is fastest?" is the question everyone asks and the one that helps least. Pinecone, Weaviate, Qdrant, Milvus, pgvector — most of them run the same core search algorithm, so raw speed on a benchmark dataset rarely decides anything. The differences that actually bite you are operational: how the index fits in memory, how it filters, what it costs at scale. Here's how to think about those.

What they (mostly) share

Under the hood, the serious vector databases all do approximate nearest-neighbor (ANN) search: they trade a little recall for a lot of speed, with a tunable knob controlling where on that curve you sit. But "ANN" is a family, and which member a database uses decides its memory and cost profile more than its branding does:

• HNSW (Hierarchical Navigable Small World graphs) is the default for most engines. Query cost grows logarithmically with corpus size, so it's fast and accurate — but the graph lives in RAM, and that memory bill is what eventually hurts.
• IVF-PQ (inverted file + product quantization) clusters vectors and stores them compressed, so it's far smaller and SSD-friendly. Cheaper at huge scale, a little slower and lossier.
• DiskANN is a graph index designed to live on SSD, indexing a billion vectors on a single node with modest RAM — the option when the dataset can't fit in memory at any reasonable price.

Because the core index is shared, vendor benchmarks tend to converge — and to be configured to flatter whoever published them. The real distinctions live around the index, not in it.

The axes that actually differ

1. Who operates it

This is usually the decision. Pinecone is fully managed and cloud-only: no infrastructure, no tuning, fast to stand up — and no self-hosting escape hatch. Qdrant and Weaviate are open-source with managed cloud options, so you can prototype on someone else's servers and later run it yourself for control or cost. pgvector is just an extension to the Postgres you probably already run. The trade is the classic one: managed convenience versus control and portability.

2. How much of the index you control

Managed services hide the knobs. Pinecone, for instance, doesn't expose HNSW parameters — you get the recall/latency point they chose. Self-hosted engines let you tune the index — graph connectivity (M), build effort (ef_construction), and search breadth (ef_search) — to move along the recall/latency curve for your workload. If you have a hard recall target, that control matters; if you don't, it's one less thing to get wrong.

3. How it filters

Real RAG queries are rarely "nearest vectors" — they're "nearest vectors where tenant = X and date > Y." How a database combines filtering with vector search is one of the biggest practical differences:

• Pre-filtering applies your metadata constraints first, then searches only the surviving set. Keeps highly-filtered queries fast and correct, but a naive implementation can mean scanning a lot of candidates.
• Post-filtering searches first, then drops non-matching results — and can fall off a "recall cliff" when a filter is selective, because the top-k vectors may all get filtered away, leaving you with too few (or zero) results.

The hard case is a graph index plus a selective filter: walking an HNSW graph where most neighbors fail the predicate can wander off the part of the graph that holds your answers. This is an active research area — approaches like ACORN (predicate-agnostic filtered search, 2024) and Filtered-DiskANN fold the filter into traversal itself rather than bolting it on. If your app leans on tight metadata filters (multi-tenant SaaS, permission scoping), test this path specifically with your selectivity. It's where naive setups quietly lose results.

4. Quantization: the real cost lever

A raw 768-dimensional float32 embedding is ~3 KB; ten million of them is ~30 GB of RAM before you index anything. Quantization is how you make that affordable, and it's where engines genuinely differ. The idea (product quantization, Jégou et al., 2011) is to store a compressed approximation of each vector and rescore the top candidates against the originals. In practice you'll see three tiers:

• Scalar quantization (float32 → int8): ~4× smaller, tiny accuracy loss, often a speed bump from int8 SIMD. The safe default.
• Product quantization: far higher compression (up to ~64× depending on settings), more accuracy cost, tunable.
• Binary quantization (one bit per dimension): ~32× smaller and very fast, but only viable for embeddings that tolerate it — verify recall before trusting it.

The pattern that makes this work is search-then-rescore: filter with the compressed vectors, then re-rank the survivors against full-precision copies. Qdrant exposes all three modes directly; managed services may quantize for you with less visibility. Either way, this — not the index name — is usually what decides your memory bill.

5. Hybrid (dense + keyword) search

Pure vector search misses exact terms — product codes, names, rare jargon — that keyword search nails. Hybrid search fuses dense vectors with sparse/keyword signals in one query. Weaviate and Qdrant support this natively; with Pinecone or pgvector you often assemble it yourself. If your corpus is full of identifiers and acronyms, hybrid is not a nice-to-have.

6. Freshness, updates, and deletes

Benchmarks measure static indexes; production data churns. Graph indexes like HNSW handle inserts well but deletes poorly — removed vectors usually become tombstones, and reclaiming that space means periodic rebuilds. If your corpus updates constantly (a live document store, frequently re-embedded content), ask how the engine handles deletes and how expensive a rebuild is. It rarely shows up in a leaderboard and frequently shows up in your on-call rotation.

7. Cost and the scale cliff

Managed pricing is wonderful until it isn't. A common pattern: start on Pinecone for speed of development, then migrate to self-hosted Qdrant or Weaviate once volume (tens of millions of vectors) or the monthly bill makes the convenience tax too steep. Knowing that migration may come, keep your embedding and ingestion code vendor-agnostic from day one.

A rough decision guide

Don't trust the benchmark — run your own

Whatever you pick, measure on your data, at your recall target, with your filters. Public ANN benchmarks (such as the long-running ANN-Benchmarks project) are useful for sanity, but recall and latency depend heavily on dimensionality, dataset distribution, and filter selectivity — none of which match your app by default. A thirty-minute test on a representative slice, with the filters and quantization you'll actually run, tells you more than any comparison table, including this one.

Sources

• Malkov & Yashunin, "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs" (2016) — arXiv:1603.09320
• Jégou, Douze & Schmid, "Product Quantization for Nearest Neighbor Search" (IEEE TPAMI, 2011) — ieeexplore.ieee.org/document/5432202
• Subramanya et al., "DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node" (NeurIPS, 2019) — microsoft.com/research
• Patel et al., "ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data" (2024) — arXiv:2403.04871
• ANN-Benchmarks — reproducible benchmarks for approximate nearest-neighbor libraries — ann-benchmarks.com