home
›
cheatsheets
›
ai_engineering
›
embeddings_vector_databases
Free to read Every card and diagram, and Quiz me
2
Free account Star cards. Your Quiz me marks are kept, so next time you drill just what you missed
3
Pro All 53 answers on this sheet, My sheet across every topic, and PDFs
A free account keeps your stars and Quiz me marks. Pro adds all 53 answers on this sheet, My sheet across every topic, and PDFs.
/
Read
Quiz me
Comfortable
Compact
0
Click any code to copy it
Done 0 /0
To review 0
On this sheet
01 Embeddings & similarity02 Embedding models03 ANN indexes04 Vector databases05 Memory & compression06 Chunking07 Filters & multitenancy08 Hybrid search & reranking09 Measure & debug retrieval10 Production & scale
The model An embedding turns text into a list of numbers. Vectors that sit close together have similar meanings. Embed queries and documents with the same model and version . Mixing two models raises no error. Exact search scores every vector. An ANN index like HNSW skips most of them and trades a little recall for a lot of speed.Similarity scores are relative . A cosine of 0.82 means nothing alone. It only ranks results within one query and one model. Vectors are derived data . The source text is the truth. Re-embed what changes, and everything when you change models. Documents and queries go through the same model. The index returns the nearest ids, and the reranker reorders that shortlist.
Embeddings & similarity what a vector means, and how to compare two What an embedding is Say what an embedding is A list of numbers a model makes from text. Similar meanings land close together.
Search with embeddings Embed each document once and keep its text. Return the ones nearest the query.
Tell static from contextual embeddings Word2Vec gives a word one fixed vector. BERT-style models use the whole sentence.
Use embeddings beyond search Clustering, classification, deduplication, recommendations and outliers.
Trap Similarity tracks topic, not truth. "The build passed" and "The build failed" land close together.
Semantic vs keyword search Match a paraphrase that shares no words Semantic search. "car won't start" finds "vehicle ignition failure".
Match a rare ID, name or error code BM25. It scores shared words and favors rare ones.
Trap An embedding model only knows its training vocabulary. BM25 indexes a new product code at once.
Cosine, dot product or L2 Angle Length
a @ b / (norm(a) * norm(b)) Cosine similarity. Higher is closer. Compare direction only
a @ b Dot product. Higher is closer, and longer vectors score higher. Compare direction and length
norm(a - b) Euclidean or L2 distance. Lower is closer. Measure the straight-line gap
Tip Normalize every vector to length 1 and all three give the same ranking.
Normalize and pick the metric Scale a vector to length 1 v = v / np.linalg.norm(v)
Pick the metric Use the one the model was trained with. Its model card says.
Know when length carries meaning DPR and other dot-product models use it. Normalizing throws it away.
Trap Normalize at insert but add later vectors raw, and ranking degrades with no error.
Embedding models pick one, and feed it the way it was trained Hosted API or open weights Call a hosted embedding model OpenAI, Cohere, Voyage AI, Google Vertex AI or Amazon Titan. You pay per token.
Run an open-weight model yourself SentenceTransformer("BAAI/bge-small-en-v1.5").encode(["hi"]) BGE, E5, GTE, Nomic, Jina or Qwen3-Embedding. Your text stays with you.
Tip A provider can retire a hosted model. Then you re-embed the whole corpus.
Choose an embedding model Narrow the field Filter on hosting, license, context length, languages, dimensions and price.
Use the MTEB leaderboard Shortlist from it. Gaps under about one point are noise.
Pick between finalists Score 50 to 100 labeled queries from real users on your own corpus.
Fine-tune a model on your data When recall fails on your jargon and you have a few thousand pairs. Then re-embed everything.
Tip Try a reranker before fine-tuning. It needs no re-embedding.
Trap Top models are tuned for MTEB, and its retrieval tasks lean English and web-like.
Embed queries and documents right Mix up the query and document models Rankings go near-random. Each model has its own vector space.
Prefix E5 queries and passages Start queries with query: and documents with passage:.
Prefix other asymmetric models Nomic uses search_query: and search_document: . BGE adds an instruction to queries only.
Tip Store the model name and version with each vector. Check it before you search.
Trap Both mistakes fail quietly. text-embedding-ada-002 and text-embedding-3-small both return 1,536 numbers.
How embedding models learn Say how text embedding models train Contrastive learning. It pulls matching pairs together and pushes the rest apart.
Get negatives for training Other items in the batch are free negatives. Hard ones look similar but are wrong.
Search images with a text query CLIP or SigLIP. They train images and captions into one space.
Trap Text-to-image scores don't compare with image-to-image scores. Calibrate before you merge them.
ANN indexes find the nearest vectors without scanning them all Exact or approximate search Find the k nearest vectors exactly top_k = np.argsort(-(corpus @ query))[:k] Always the true top k. Fine for small collections.
Search millions of vectors Use an ANN index. It skips most vectors, so it can miss a few.
Say why a B-tree index can't help It sorts on one value. Closeness depends on every dimension at once.
Say what a vector database adds An ANN index, filters in the same query, and storage sized for vectors.
Trap Plain SQL scores every row to sort by distance. Fine at 10,000 rows, painful at 10 million.
HNSW entry nearest HNSW search Search hops closer on each layer, then drops down. The bottom layer holds every vector.Raise recall at query time Raise ef_search, keeping it at least k. Queries get slower.
Set the links per node M, often 16, and about 2M on layer 0. More links raise recall and memory.
Build a better graph Raise ef_construction. Builds slow down, but queries don't.
Hit a recall target within a latency budget Freeze M. Step ef_search up and take the smallest value that meets both.
Trap M and ef_construction are fixed at build time. If recall plateaus below target, rebuild with a larger M.
IVF and DiskANN Build an IVF index with 4,096 clusters quantizer = faiss.IndexFlatL2(768) index = faiss.IndexIVFFlat(quantizer, 768, 4096) index.train(sample_vectors)
Raise IVF recall index.nprobe = 16 Scan more of the nearest clusters per query.
Choose IVF over HNSW When memory or build cost dominates at huge scale, usually as IVFPQ.
Serve more vectors than RAM holds DiskANN. It keeps the graph and full vectors on SSD, and PQ copies in RAM.
Trap IVF never scans a true neighbor that sits in a cluster you didn't probe.
Vector databases pick where the vectors live Semantic search in pgvector Enable pgvector CREATE EXTENSION IF NOT EXISTS vector;
Add a vector column ALTER TABLE docs ADD COLUMN embedding vector(1536);
Index it for cosine search CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops);
Return the 10 nearest rows SELECT id FROM docs ORDER BY embedding <=> $1 LIMIT 10;
Raise recall for this session SET hnsw.ef_search = 100;
Get enough rows from a filtered query (0.8+) SET hnsw.iterative_scan = relaxed_order;
Tip Only ORDER BY ... LIMIT can use the index. A distance threshold in WHERE can't.
Trap Match the operator to the index, like <=> with vector_cosine_ops . Otherwise Postgres skips the index.
pgvector or a dedicated store Keep vectors next to your rows Use pgvector. The row and its vector commit together, so search never serves deleted rows.
Move to a dedicated vector store At hundreds of millions of vectors, thousands of tenants or tight latency.
Tip Don't move before a limit actually blocks you.
Trap Postgres scales up, not out. The whole index sits on one primary.
Compare vector databases Run no infrastructure at all Pinecone. It's managed only, with a serverless tier.
Self-host with heavy metadata filters Qdrant. Filterable HNSW with payload indexes.
Get keyword and vector search in one system Weaviate, or Elasticsearch and OpenSearch if you already run them.
Reach billions of vectors Milvus or Zilliz. Distributed, with many index types.
Meet strict data residency rules An in-region managed service, or BYOC so the data stays in your cloud.
Tip Most engines run HNSW, so recall converges. Benchmark the shortlist on your own vectors and filters.
Trap Each Lucene segment builds its own HNSW graph, so Elasticsearch gets costly at billions of vectors.
Memory & compression fit more vectors in less RAM Size vectors and the index Weigh more dimensions Quality gains shrink fast, but storage and compute grow in a straight line.
Size the raw vectors N * dims * 4 # bytes at float32 1,536 dimensions take about 6.1 GB per million vectors.
Size the HNSW links N * 2 * M * 4 # int32 ids on layer 0 10M vectors at M = 16 take about 1.3 GB of links.
Tip Add 10 to 30 percent for upper layers, ids, payloads and allocator slack.
Trap Quantization shrinks only the vectors. The graph links stay, so M sets a memory floor.
Shorten vectors with Matryoshka Say what a Matryoshka embedding is A vector whose leading slice, like the first 256 numbers, still works.
Truncate a vector yourself short = full[:512] / np.linalg.norm(full[:512]) OpenAI's text-embedding-3 models take a dimensions parameter too.
Search cheap, then rank precisely Shortlist with short vectors, then rerank with the full ones.
Tip Cutting 3,072 dimensions to 512 shrinks the index six-fold.
Trap Re-normalize after slicing. A slice isn't unit length, so dot-product search breaks.
Quantize vectors Shrink vectors 4x Scalar quantization to int8. Recall drops about 1 to 2 percent.
Shrink vectors 32x Binary quantization. One sign bit per dimension, compared by Hamming distance.
Win back recall after binary quantization Oversample 3 to 5 times, then rescore with the full vectors.
Compress hardest Product quantization. Codebook ids replace sub-vectors, and it needs training.
Trap Binary quantization suits high-dimensional models. At 384 dims the bits carry too little signal.
Chunking split documents so each vector holds one idea Chunk size and overlap Say why you chunk before embedding Models truncate long input, and one vector for a whole manual blurs every topic.
Start with a chunk size and overlap 200 to 500 tokens, with 10 to 20 percent overlap.
Choose where to split At headings and paragraphs first, then sentences.
Weigh small chunks against large Small ones match sharply but can split an answer. Large ones blur the vector.
Pick the size for your data Sweep sizes and overlaps on real queries, and score recall@k.
Tip Retrieve a larger k and rerank. Results then depend much less on chunk size.
Trap More overlap fills the top k with near-duplicates and grows storage.
Give chunks their document context Have an LLM place each chunk in its document Contextual retrieval. One LLM call per chunk writes context to prepend.
Embed the whole document, then split Late chunking. Pool each chunk after the model reads the whole document.
Match small, return big Embed small chunks, but hand the LLM the parent section.
Filters & multitenancy search only what the caller may see Filter by metadata Limit a search to one tenant Store fields like tenant_id beside each vector, and pass a filter with the query.
Search first, then filter Post-filtering. Fast, but it can return fewer than k.
Filter first, then search Pre-filtering. Exact, but it may brute-force scan the matches.
Filter during the search Modern engines check the filter while walking the index, and scan when few rows match.
Fix recall when a filter matches few rows Scan exactly below a match-count threshold, or partition on that field.
Tip Index every field you filter on. An unindexed filter falls back to a full scan.
Trap Filters change recall. Measure quality with your real filters on.
Multitenancy and access control Serve many tiny tenants One collection with a tenant_id filter. That's logical isolation only.
Serve thousands of tenants A namespace or partition per tenant. Give a big or regulated tenant its own index.
Stop users seeing each other's chunks Filter on the caller's permissions inside the search, not after it.
Tip Inversion attacks rebuild short chunks from their vectors. Treat embeddings as personal data.
Trap Take the tenant from the verified token, never from the request body.
Hybrid search & reranking merge keyword and vector hits, then put the best first Hybrid search with RRF Catch exact codes and paraphrases Run BM25 and dense search, then merge the two ranked lists.
Merge lists with Reciprocal Rank Fusion 1 / (k + rank) Sum it over every list the document is in, with k = 60 and ranks from 1.
Sharpen or flatten the top ranks A small k like 10 sharpens the top ranks. 60 or more flattens them.
Tune fusion at scale Cache each retriever's top 1,000, then grid-search k, weights and depth.
Go past BM25 and one vector per chunk SPLADE learns sparse term weights. ColBERT keeps a vector per token.
Tip Raw scores don't merge. BM25 is unbounded, while cosine sits in a fixed range.
Trap RRF throws away confidence. A huge win and a narrow win score the same.
Rerank with a cross-encoder Embed query and document separately A bi-encoder. Document vectors are precomputed, so a query is one embed and one search.
Score every document with a cross-encoder Too slow. It runs one model pass per document, per query.
Rerank the top candidates Rescore the top 100 with the cross-encoder and keep 10. Widen until quality stops improving.
Decide whether a hit is relevant Cut on the cross-encoder's score, or stay near the top score. Never a fixed cosine.
Tip Scores aren't calibrated. Every query gets a top hit, even when nothing answers it.
Trap A reranker only reorders. It can't bring back what stage one missed.
Remove duplicates, add variety Drop duplicate hits Drop exact copies by hash and near-copies above a cosine of about 0.95.
Diversify results with MMR score = lam * rel - (1 - lam) * div div is the highest similarity to an earlier pick.
Tip Fetch about 50 candidates, then pick k. Tune lam on an eval set, where 1.0 is pure relevance.
Measure & debug retrieval prove search works, and find what broke Recall@k, MRR and nDCG Compute recall@k for one query Relevant documents in the top k, over all relevant ones. 3 of 4 in the top 10 is 0.75.
Reward the right hit near the top MRR or nDCG. Recall@k ignores order inside the top k.
Grade the index, not the model ANN recall. Compare the index's top k with a brute-force top k.
Trap Retrieval recall and ANN recall are different numbers, and both can break. Measure each.
Build a retrieval eval set Get labeled queries Start from real query logs. Fill gaps with an LLM that writes a question per chunk.
Report the results Recall@k, MRR and nDCG@10, saved per query so you see what broke.
Stop regressions Version it with the corpus, and gate config changes on it in CI.
Tip Score retrieval apart from generation, so you know which stage failed.
Debug worse retrieval Check whether the index is at fault Rerun the queries with brute-force kNN. If exact is bad too, the index isn't the problem.
Check for a silent model change Re-embed a pinned document and compare it with the stored vector.
Check the data and the queries Count rows, zero vectors and failed ingests. Look for new kinds of queries.
Tip Replay a frozen query set, so you can tell a worse system from harder queries.
Trap Growth alone lowers recall. A fixed ef_search finds fewer true neighbors as the corpus grows.
Production & scale keep vectors fresh, fast and cheap Keep vectors in sync with the source Re-embed only what changed Hash each chunk, and re-embed only the chunks whose hash changed.
Stop stale chunks serving old text Delete by doc_id, then insert, reusing vectors whose hash matches.
Catch dropped change events Run a periodic job that compares source ids with index ids.
Embed 100 million documents that keep changing A backfill lane and a live change lane share one idempotent upsert.
Tip Give the backfill its own API quota. Otherwise a days-long re-embed starves live updates.
Trap An edit that changes the chunk count leaves old chunk ids behind. They keep serving stale text.
Switch to a new embedding model Add a column for the new model ALTER TABLE docs ADD COLUMN embedding_v2 vector(1536); Keep the old one serving. Build the new index CONCURRENTLY.
Fill the new column Re-embed in batches, and dual-write every change to both columns.
Switch reads safely Compare on a golden query set, ramp up behind a flag, and keep a rollback.
Tip Budget for double storage while both indexes run.
Trap Scores don't carry across models. Re-tune cutoffs, RRF weights and ef_search.
Scale out and keep the index healthy Hold more vectors, or serve more queries Shard to split vectors across nodes. Add replicas for throughput and failover.
Stop every query hitting every shard Route by tenant, region or time, not by hash. Watch for hot shards.
Stop recall decaying under heavy updates Alert on the share of deleted vectors, compact early, and rebuild blue-green.
Tip Deleted vectors stay as tombstones that searches still walk. Latency rises and recall falls.
Trap Your p99 is your slowest shard. Hedge requests or cap the fan-out.
Cut the cost of vector search Stop paying for the same rebuild twice Version the index, so each model or chunking change migrates once.
Cut index memory Quantize, or truncate Matryoshka dimensions. Together they often save 4x.
Cut the cost per query Cache hot query vectors. Rerank the top 50, not the top 500.
Tip The biggest waste is paying to embed text that didn't change.
Nothing matches. Try a shorter word, or clear the search.
Practice Guided practice One question at a time, with the ones you miss coming back. Practice 56 interview questions Full answers, from beginner to expert.
Found this useful? Pass it on.
Share on LinkedIn
Share on X
Copy link
This sheet stays free. With Pro , you can:
Every answer on this sheet. Read the full answer to all 53 questions it links, not just the 6 free ones.My sheet. The cards you star and the ones you miss in Quiz me, from every sheet, on one page to quiz and print.PDFs. Download this sheet and My sheet, laid out for A4, with every question a link to its answer.The rest of Pro. Every answer in the question library, and 1,000 AI credits a month.
