Vector Databases

Purpose-built for the exact operation RAG and semantic search depend on: given a query vector, find the most similar vectors among millions or billions, fast. RAG already covers the pipeline this sits inside (chunking, embedding, retrieval, generation) — this page is about the database mechanics underneath the "retrieve" step specifically.

What a Vector Database Actually Stores and Searches

Every item (a document chunk, an image, a product) is represented as an embedding — a high-dimensional vector (see Linear Algebra) produced by a model trained so that semantically similar items end up close together in that vector space. A vector database indexes these vectors so that, given a new query vector, it can efficiently find the nearest ones — Approximate Nearest Neighbor (ANN) search, since exact nearest-neighbor search over millions of high-dimensional vectors is too slow for real-time use.

Distance Metrics

Cosine similarity: measures the angle between vectors, ignoring magnitude — the standard choice for text embeddings, where direction encodes meaning.
Dot product: similar to cosine but sensitive to magnitude too — used when the embedding model was specifically trained with dot-product similarity in mind.
Euclidean distance: straight-line distance — more common for non-text embeddings (e.g. some image or audio embeddings).

Which metric to use is typically dictated by how the embedding model itself was trained, not a free choice at query time — but which points end up "nearest" genuinely depends on the metric, not just the data:

Distance metric

Each point is a toy 2D stand-in for a high-dimensional embedding — click to reposition the query.

Nearest 3 to the query under cosine distance: doc7, doc10, doc1. Click anywhere on the canvas to move the query point, or switch metrics with the same query position — the ranking changes because each metric answers a genuinely different question ("closest in a straight line" isn't "most similar in direction," and isn't "highest raw dot product" either).

Indexing Algorithms

Flat (brute-force / exact): no index at all — every query does a full linear scan comparing against every stored vector. Guarantees the mathematically correct nearest neighbors (100% recall) at the cost of scaling linearly with dataset size, which is exactly why it stops being viable past a few tens of thousands of vectors. Every ANN algorithm below exists specifically to avoid this scan while staying close to its accuracy — it's also the baseline every recall benchmark measures against, including the "exact scan" reference point in the tradeoff chart further down this page.
HNSW (Hierarchical Navigable Small World): builds a multi-layer graph structure where search starts at a coarse top layer and progressively narrows down — the most widely used ANN algorithm today, offering an excellent speed/accuracy tradeoff.
IVF (Inverted File Index): clusters vectors into buckets ("cells") ahead of time; at query time, only search the most relevant few buckets instead of the entire dataset.
Product Quantization (PQ): compresses vectors by splitting them into sub-vectors and quantizing each independently, trading some accuracy for a large reduction in memory footprint — often combined with IVF (as "IVF-PQ") for very large-scale deployments (billions of vectors).

Index	Recall	Query Speed	Memory	Build Time	Best For
Flat	100% (exact)	Slowest — linear in dataset size	Lowest per vector (raw vectors only, no index structure)	None — nothing to build	Small datasets, or as the ground-truth baseline other indexes are measured against
IVF	High, tunable via how many clusters are probed	Fast	Low-moderate	Fast — just clustering	Large datasets with reasonably well-clustered embeddings
HNSW	Very high, tunable via search width	Fastest at a given high-recall target	Higher — graph edges add real overhead per vector	Slower — graph construction is the expensive step	Latency-sensitive production search at real scale
IVF-PQ	Lower — quantization is itself lossy	Fast	Lowest — vectors are compressed, not just clustered	Moderate	Billion-scale datasets where memory, not latency, is the binding constraint

This is a genuine tradeoff table, not a "pick the best row" — Flat is the right choice below a few tens of thousands of vectors (an index adds complexity for no real benefit at that scale), and the ANN rows trade some of Flat's guaranteed correctness for the speed and memory headroom that make billion-vector search feasible at all.

How HNSW Search Actually Works

The layer structure is the whole trick: every vector lives in the bottom layer, but only a shrinking fraction of vectors also exist in each layer above it — so the top layer is a sparse "highway" connecting distant regions of the space, and each layer down adds finer, more local connections. A search starts at a fixed entry point in the top layer, greedily walks toward the query (moving to whichever neighbor is closer, one hop at a time) until no neighbor helps anymore, then drops down one layer and repeats from there — narrowing in on the answer the same way you'd navigate a highway system down to city streets down to the exact address, rather than checking every address in the country:

Step 1 of 7

Start at the entry point (node 2) in the top layer.

The Recall/Speed/Memory Tradeoff

"Approximate" is a real, tunable knob, not a fixed penalty — searching more candidates (widening how much of the graph a query explores before stopping) increases the odds of finding the true nearest neighbors (recall) at the cost of latency, with diminishing returns as you approach exhaustive search:

ef (candidates examined per search)40

Recall: 68%Latency: 38ms

At ef=40: ~68% recall in ~38ms. Exact (brute-force) search over 10M vectors guarantees 100% recall but costs ~850ms per query -- ANN trades a small, tunable amount of recall for roughly 10-100x lower latency, which is the entire reason approximate search exists at this scale.

Product Quantization sits on the memory axis of this same tradeoff: compressing vectors trades some accuracy for dramatically less RAM per vector, which is what makes indexing billions of vectors in memory feasible at all.

ChromaDB and Beyond

Deployment	Embedded, in-process (like SQLite) or a lightweight local server
Indexing	HNSW under the hood
Best for	Prototyping, local development, smaller-scale RAG apps that don't need a separately-managed service
Tradeoff	Not built for multi-billion-vector scale or heavy concurrent write load -- optimized for getting started fast, not for being the backbone of a large production system.

pgvector is covered in more depth in PostgreSQL — pgvector and the AI Stack; this table is about picking between it and a dedicated vector database, not how to use it.

The right pick depends on scale, whether you want a managed service vs. self-hosted, and whether you'd rather keep vectors in the same database as your other structured data — see PostgreSQL — pgvector and the AI Stack for that last option in depth.

Hybrid Search

Pure vector similarity can miss exact matches a keyword search would catch instantly (product IDs, exact names). Hybrid search combines vector similarity with traditional keyword/metadata filtering, merging both result sets — covered in more depth in RAG — Dense vs. Sparse Retrieval, since this is precisely the retrieval step every RAG pipeline depends on.

Next: Graph Databases — for when relationships between entities matter more than similarity.