Embeddings and Vector Search in Production

11 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

An embedding model maps text (or images, code) to a vector so that similar meanings land close together. Search becomes "find the nearest vectors to the query vector".
Exact nearest-neighbour search compares the query with every vector, which is too slow at millions of items. Approximate indexes (HNSW graphs, IVF clusters) find most of the true neighbours in milliseconds.
Production vector search is a set of trade-offs you tune and measure: recall against exact search, latency, memory, and build and update cost. Quantisation cuts memory by 4 to 32 times at a small recall cost.
Embeddings are tied to the model that produced them. Changing models means re-embedding everything and running both indexes during the switch.
Often the best vector database is the one you already run: Postgres with pgvector, or your search engine's vector fields, until scale or features force a dedicated system.

How it works

What an embedding is

An embedding model is a neural network trained so that texts with related meaning produce vectors pointing in similar directions. Similarity is measured with cosine similarity (the angle between vectors) or the dot product; when vectors are normalised to length 1, the two are the same. Dimensions range from a few hundred to a few thousand; pgvector, for instance, can index vector columns only up to 2,000 dimensions and halfvec up to 4,000 (pgvector README). Models differ in languages, domains and maximum input length; public benchmarks such as MTEB compare them, but your own retrieval eval decides.

Exact versus approximate search

Exact (brute-force) search is simple and perfectly accurate: compute the similarity of the query with every vector and keep the top k. At 100,000 vectors it takes milliseconds; at 100 million it is far too slow per query. Approximate nearest neighbour (ANN) indexes trade a little recall for orders of magnitude of speed. Faiss, the reference library, implements IVF, product quantisation and HNSW, and handles sets too big for RAM (Faiss):

HNSW (Hierarchical Navigable Small World) builds a layered graph where each vector links to nearby vectors. Search starts at a top layer with few nodes, greedily moves toward the query, and descends. Each node's top layer is chosen at random, rarer for higher layers, and the authors claim logarithmic scaling of search (Malkov and Yashunin). Parameters such as M (links per node) and ef_search (candidates explored) trade memory and latency for recall; pgvector defaults to m = 16, ef_construction = 64 and hnsw.ef_search = 40 (pgvector README).
IVF (inverted file) clusters vectors into, say, 4,096 groups; a query searches only the nearest few clusters (nprobe). It uses less memory than HNSW and builds faster, but recall depends on probing enough clusters. pgvector suggests lists of rows/1000 up to a million rows and sqrt(rows) beyond, probes starting near sqrt(lists) (the default is 1), and building the index only after the table holds data (pgvector README).

Measuring recall

Recall at k for an ANN index means: of the true top k (from exact search), how many did the index return? Measure it on a sample of real queries by running both exact and approximate search. Aim for a target, often 0.95 or higher, and tune parameters to the cheapest setting that meets it. Remember that index recall is only part of the story: if the embedding model itself ranks the wrong passages highly, a perfect index returns the wrong passages perfectly.

Quantisation

Storing vectors as 32-bit floats costs 4 bytes per dimension. Scalar quantisation stores each dimension as an 8-bit integer (4 times smaller); product quantisation compresses further by encoding groups of dimensions as codebook entries; binary quantisation keeps one bit per dimension (32 times smaller). A common pattern is to search compressed vectors to get candidates, then rescore the top few hundred with full-precision vectors. In one MTEB retrieval test, int8 kept about 97 percent of full-precision NDCG@10 (about 99 percent with rescoring) and binary kept about 92.5 percent (96.45 percent with rescoring), with exact-search speedups of roughly 3.7 and 25 times (Hugging Face). Treat those as one model's results and measure your own.

Operating it

Updates and deletes. HNSW handles inserts well, but deletes are often tombstones that degrade the graph until rebuilt. Plan periodic rebuilds.
Filtering. "Nearest vectors where tenant = X and language = en" is hard: filtering after search can return too few results, filtering before can be slow. Check how your engine handles it, especially for multi-tenant permission filters. In pgvector the filter runs after the index scan, so with a 10 percent match rate and ef_search = 40 a query returns about 4 rows on average. Version 0.8.0 (October 2024) added iterative index scans (hnsw.iterative_scan) that keep scanning until enough rows pass the filter, capped by hnsw.max_scan_tuples (default 20,000); partial indexes and partitioning cover selective filters (pgvector README, changelog).
Model changes. New embedding model, new vector space: re-embed the corpus, build a new index, run both, compare on evals, switch reads, delete the old one. It is an expand-and-contract migration. Research is exploring shortcuts: Drift-Adapter trains a small mapping from new-model queries into the old space and reports recovering 95 to 99 percent of full re-embedding recall, though these are the authors' self-reported results (arXiv:2509.23471).

The same shapes as any system

Vector search is the finding-things-fast shape with a new distance function. Every indexing lesson applies: build cost versus query cost, memory versus speed, freshness versus rebuild effort.

Down the stack

Layer 1The contract

A vector search service promises top-k results for a query vector, with filters, at a latency target (for example p99 under 50 ms), with recall at 10 above a target measured against exact search, and with new or updated items searchable within a freshness window.

Layer 2The mechanism

HNSW stores, for each vector, a list of neighbour IDs per layer. Insertion picks a random top layer for the new node (with exponentially decreasing probability for higher layers), then connects it to its nearest neighbours found by searching from the entry point. Search keeps a priority queue of ef_search best candidates, expanding neighbours until no closer candidate is found. IVF trains k-means centroids on a sample, assigns each vector to its nearest centroid's list, and at query time scans the lists of the nprobe nearest centroids. Postgres's pgvector implements both as index types inside the database, so vector search can join with ordinary tables and use the same backups and transactions.

Layer 3The machine

Similarity is arithmetic: a dot product of two 1,024-dimension float vectors is 1,024 multiply-adds, which modern CPUs do 8 to 16 at a time with SIMD instructions (AVX2, AVX-512, NEON). The real bottleneck is memory: HNSW search jumps between nodes stored at random addresses, so each hop risks a cache miss costing around 100 ns from main memory, and a search touching a few thousand nodes spends most of its time waiting for memory. That is why keeping the index in RAM matters so much, and why quantisation, by shrinking vectors, speeds up search as well as saving space.

Why it's built this way

Meaning-based similarity needs a vector representation and a distance function.
Exact search scales linearly with the corpus, too slow at large sizes.
Graph and cluster indexes cut the work per query at a small recall cost.
Memory capacity and bandwidth dominate cost, so compression pays off twice.
Vectors depend on the model that made them, so model changes are migrations.

Diagram

flowchart TD
  T["Text chunk"] --> E["Embedding model"]
  E --> V["Vector (1,024 floats)"]
  V --> Q8["Quantise to int8"]
  Q8 --> H["HNSW graph in RAM"]
  QY["Query text"] --> E2["Same embedding model"]
  E2 --> QV["Query vector"]
  QV --> H
  H --> C["Top 200 candidates"]
  C --> RS["Rescore with full vectors"]
  RS --> K["Top 10"]

Worked example

A marketplace wants semantic search over 30 million product listings with 768-dimension embeddings. Raw float32 vectors take 30 million times 768 times 4 bytes, about 92 GB, before HNSW links (with M = 16, roughly another 30 million times 32 neighbour IDs times 4 bytes, about 4 GB per layer-0 level). That needs large-memory machines and replicas.

They measure on 2,000 real queries with exact search as ground truth. HNSW with float32 and ef_search = 100 gives recall at 10 of 0.97 at p99 18 ms. Scalar int8 quantisation cuts vector memory to about 23 GB; recall drops to 0.94. Rescoring the top 100 candidates with float32 vectors stored on SSD restores recall to 0.97 with p99 26 ms. Filters on category and country are applied inside the index search. Total memory per replica falls from about 100 GB to about 30 GB, so they run three replicas on smaller machines instead of two large ones. When they later switch embedding models, they build the new index alongside the old, A/B test click-through for two weeks, then switch.

In the wild

Cloudflare, 2025: R2 outage takes the Vectorize vector database with it. On 6 February an operator disabled the R2 gateway for 59 minutes (08:14 to 09:13 UTC). Vectorize persists its data in R2, so all inserts, upserts and deletes failed and about 75 percent of queries failed, the rest served from cache. It recovered once R2 did. A vector index is a derived store with a hard dependency on its storage layer (Cloudflare postmortem).
pgvector, documented behaviour: filtered ANN queries return too few rows. Because the filter runs after the approximate scan, a selective WHERE can leave a query with a handful of results. Iterative scans in 0.8.0 were added to address this (pgvector changelog).
Research, 2025: embedding upgrades as a cost problem. The Drift-Adapter paper frames a model swap as re-encoding the whole corpus and rebuilding the ANN index, and measures a query-side adapter as an alternative (arXiv:2509.23471).

Trade-offs

Choice	Strengths	Weaknesses	Choose when
Brute-force exact search	Perfect recall, no index	Linear cost	Under about 100k vectors, or as ground truth
HNSW	High recall, low latency	Memory-heavy, slow deletes	Most interactive search
IVF	Less memory, fast builds	Recall depends on probes	Very large, batch-built corpora
Quantisation plus rescoring	Much less memory, near-full recall	Extra step and tuning	Large corpora, memory-bound
pgvector in existing Postgres	One system, joins, transactions	Scale limits per node	Up to millions of vectors
Dedicated vector database	Scale, advanced filtering	Another system to run	Hundreds of millions, heavy filters

Pitfalls in production

Mixing vectors from different embedding models in one index. Fix: one model per index, versioned.
No recall measurement, only latency. Fix: compare against exact search on real queries.
Post-filtering that returns too few results for selective filters. Fix: engines with filtered search, or partition by tenant.
Indexes degraded by many deletes. Fix: scheduled rebuilds and monitoring of recall over time.
Embedding documents and queries differently (prefixes, truncation). Fix: follow the model's instructions for queries versus documents.

Leadership lens

Start with the simplest store that meets scale (often pgvector); move only when measurements force it.
Treat embedding model choice and changes as migrations with evals, dual indexes and rollback.
Put recall at k, latency and memory on one dashboard; tune for the cheapest setting that meets targets.
Budget for memory: it is usually the dominant cost of vector search at scale.

Check yourself

Why can a perfectly accurate vector index still return irrelevant results?
Show answer
The index finds the nearest vectors exactly, but if the embedding model places irrelevant passages near the query, nearness does not mean relevance.
What does ef_search control in HNSW?
Show answer
How many candidates the search explores; higher values improve recall at the cost of latency.
Why does quantisation also make search faster, not just smaller?
Show answer
Search is memory-bound, so smaller vectors mean fewer bytes moved and more of the index in cache.
Why can't you add vectors from a new embedding model to an existing index?
Show answer
Each model defines its own vector space; distances between vectors from different models are meaningless.

Sources and further study

Primary

Malkov and Yashunin, HNSW (2016): layered proximity graphs with random layer assignment.
Drift-Adapter (2025): cheaper embedding-model upgrades, with self-reported results.

Systems docs

pgvector README: HNSW and IVFFlat parameters, dimension limits, filtering and iterative scans.
pgvector changelog: what 0.8.0 changed.
Faiss: IVF, PQ, HNSW and GPU indexes.
Hugging Face, Embedding quantization: memory, speed and retained quality, measured.

Books

Chip Huyen, AI Engineering (O'Reilly, 2025), ch. 6 "RAG and Agents", Retrieval Algorithms and Retrieval Optimization (ToC).

Other curricula

Azure Architecture Center, RAG solution design: treats the embedding model and search index configuration as separate phases to choose and evaluate.

Free resources

Efficient and robust approximate nearest neighbor search using HNSW (Malkov and Yashunin): the HNSW paper.
HNSW explained (Pinecone): an illustrated walk through the graph and its parameters.
Faiss: Meta's library for similarity search, with documentation on IVF, PQ and HNSW.
Embedding quantization (Hugging Face): scalar and binary quantisation with rescoring, measured.
ANN-Benchmarks: recall versus speed comparisons of ANN libraries.
MTEB leaderboard (Hugging Face): comparing embedding models across tasks.

Open in the interactive app: practice, quiz, mock interview →