Embeddings & Vector Databases.

An embedding is a vector, a list of numbers that a model produces from a piece of content. The model places similar meanings close together in that space, so the distance between two vectors stands in for similarity of meaning. A typical text model returns a few hundred to a few thousand numbers per input.

curl https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "text-embedding-3-small", "input": "How do I reset my password?"}'
# -> {"data": [{"embedding": [0.021, -0.043, 0.017, ...]}]}

For search, you embed each document once at write time and store the vector. You embed the query the same way, then return the closest document vectors. Wording no longer has to match, so a query about resetting a password can retrieve "recover account access". Keep the original text beside each vector, because the search hands back ids and scores, and your app needs the text itself.

Keyword search like BM25 ranks documents by the words they share with the query, while semantic search ranks them by meaning. The keyword score counts how often the query's words appear in a document, and it favors rare words. Semantic search compares embeddings, vectors that capture meaning, so a query can match a passage that shares no words with it.

A search for "car won't start" can return "vehicle ignition failure": close in meaning, zero shared terms.

BM25 needs no model at query time. It indexes new words the moment you add them. An embedding model only knows the vocabulary it was trained on, so a fresh product code means little to it. Most production systems run both.

All three rank by closeness, but they differ in how they treat a vector's magnitude, its length. Cosine similarity looks only at the angle, dot product also rewards longer vectors, and Euclidean distance measures the straight-line gap.

They produce the same ranking when every stored vector has the same norm, which in practice means embeddings normalized to length 1. Then cosine equals the dot product, and this identity ties in the third:

|a - b|^2 = 2 - 2 * (a·b)    # for unit vectors

Distance falls as the dot product rises, so the order matches.

Once the stored norms match, the query's own norm never changes the order, because it scales every candidate's dot product equally. If document norms vary, dot product can push a long, loosely related vector above a short, on-topic one.

A vector database stores embeddings and returns the vectors closest to a query vector. It does this with an approximate nearest neighbor (ANN) index, which finds close vectors without scoring every row.

A relational database indexes for exact matches and ranges. A B-tree keeps values in one sorted order. Similarity has no such order, because closeness depends on every dimension at once. So plain SQL scores every row against the query.

-- no ordinary index helps: every row gets a distance
SELECT id FROM docs
ORDER BY embedding <-> :query_vector
LIMIT 10;

That is fine for 10,000 rows and painful for 10 million. A vector database adds the ANN index, metadata filters in the same query, and storage sized for large vectors. Extensions like pgvector bring those indexes into Postgres, which blurs the line.

The dimensionality of an embedding is how many numbers each vector holds. The model fixes it, so you rarely choose it yourself. No single number means anything on its own. Meaning comes from all the numbers together.

More dimensions give the model room to record finer distinctions, so quality usually improves. The gains shrink fast, while the costs grow in a straight line. Doubling the dimensions doubles storage and index memory, and roughly doubles the work per distance comparison. Stored as float32, each dimension takes 4 bytes:

• 384 dimensions: 1,536 bytes, about 1.5 GB per million vectors
• 1,536 dimensions: 6,144 bytes, about 6.1 GB
• 3,072 dimensions: 12,288 bytes, about 12.3 GB

A small, well-trained model often beats a weaker model with more dimensions. Judge models on your own data, not on dimension count.

k-nearest-neighbor search, or k-NN, takes a query vector and returns the k stored vectors closest to it under a distance metric. It ranks by closeness, it does not look for an exact match.

Exact search compares the query against every stored vector. It always returns the true top k, but the work grows linearly with the collection size.

scores = corpus @ query          # touches every vector, every query
top_k = np.argsort(-scores)[:k]

Approximate nearest neighbor (ANN) search skips most of the collection by using an index built ahead of time. It is much faster, but it can miss some true neighbors.

Exact search is fine for small collections. Once you reach millions of vectors, ANN is the practical default.

Cosine similarity measures the angle between two vectors. It divides their dot product by the product of their lengths, producing a high score when the vectors point in similar directions.

It is common because direction often captures semantic information while vector magnitude may be unrelated to relevance. If all vectors are normalized to length one, cosine similarity and dot product produce the same ranking and can be computed efficiently.

Use the metric recommended for the embedding model. Some models are trained for dot product or Euclidean distance, and changing the metric can hurt quality. A high cosine score is not a universal proof of relevance, so select thresholds with labeled data.