RAG (Retrieval-Augmented Generation) · question

Question 2 of 56

Walk me through the canonical stages of a RAG pipeline.

beginner

← All RAG (Retrieval-Augmented Generation) questions

Re-explain

A RAG pipeline has two halves. An offline indexing pass runs whenever documents change. An online query pass runs on every request.

The offline pass runs these stages in order:

Load and parse the documents.
Chunk them.
Embed each chunk.
Write the vectors, plus metadata, to the index.

The online pass runs these stages in order:

Rewrite the query.
Embed it.
Retrieve the top-k chunks.
Rerank them.
Build the prompt.
Generate the answer and return it with citations.

Two points beginners miss. The same embedding model must serve both halves, or the query vector and the stored vectors do not share one space. And the pipeline is a chain: if any stage before generation drops the right chunk, the model cannot recover it.

Reranking is optional. Retrieval is tuned for speed over a huge index, reranking for precision over a short list.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·

What should the new diagram focus on?

Guided practice for RAG (Retrieval-Augmented Generation) One question at a time. Answer out loud, get graded, see what you missed.

See it - interactive diagram

The two passes of a RAG pipeline

Offline indexing

Load + chunkParses and splits docs whenever they change.

Embedding modelchunks

Embedding modelUses the same model as the query side.

Vector indexvectors + metadata

Online query

Rewrite queryRephrases the question so search finds more.

Embedding modelrewritten query

Embedding modelUses the same model that embedded the docs.

Retrievequery vector

RetrieveFast search over the whole index. A chunk it misses never reaches the model.

Vector indextop-k search
Optional reranktop-k chunks

Optional rerankA slower, more precise model re-sorts the short list. Without it, the retrieved chunks go straight into the prompt.

Build promptreordered chunks

Build promptPuts the question and the chosen chunks together.

Generateprompt

GenerateWrites the answer, with citations.

Embedding model vectors + metadata

Retrieve top-k search

Where passes meet

Vector indexThe only place the two passes share data.

Load + chunk Embedding model (chunks, inside Offline indexing)
Rewrite query Embedding model (rewritten query, inside Online query)
Embedding model Retrieve (query vector, inside Online query)
Retrieve Optional rerank (top-k chunks, inside Online query)
Optional rerank Build prompt (reordered chunks, inside Online query)
Build prompt Generate (prompt, inside Online query)

The passes meet only at the index, and both embed with the same model. If an early stage drops the right chunk, the LLM never sees it.

Other questions have diagrams like this too. Sign in to ask for one where it's missing - it's free.

Create a free account

How well did you know this?

AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens How interviewers actually use this question

Likely follow-ups

Why rerank at all? Why not just make first-stage retrieval more accurate? Retrieval embeds query and chunk separately, so it scales. A cross-encoder reads each pair together, more precise but too slow for the full index.
In a multi-turn chat, why not embed the user's latest message as it is? Follow-ups lean on earlier turns, like 'what about the second one?', so rewrite them into a standalone query from the chat history before embedding.
The assistant says it found nothing, but the document exists. Where do you look first? Walk the chain in order. Was it parsed, chunked intact, in the top-k, kept by the reranker, in the prompt? Check each stage's output for that document.
How do you keep the index correct when documents are edited or deleted? Track each chunk's source document so an edit re-chunks and replaces its vectors and a delete removes them, rather than only appending.
You switch to a new embedding model. What has to happen to the existing index? Old vectors and new query vectors don't share a space, so re-embed the whole corpus, ideally into a parallel index you validate before cutover.

What you can say

I split a RAG pipeline into two halves. An offline pass indexes documents whenever they change, and an online pass runs on every request.
Offline, you load and parse the documents, chunk them, embed each chunk, and write the vectors with their metadata to the index.
Online, each request goes through query rewriting, embedding the query, retrieving the top-k chunks, reranking, building the prompt, and generating an answer with citations.
Both halves have to use the same embedding model. Otherwise the query vector and the stored vectors aren't in the same space, and the similarity scores mean nothing.
I treat reranking as optional. Retrieval is tuned for speed over the whole index, and reranking for precision over the short list it returns.
The main weakness is that it's a chain. If any stage before generation drops the right chunk, the model has no way to recover it.

Weak answers to avoid

Says 'retrieve, then generate' and stops That skips parsing, chunking, embedding, reranking and prompt building. Name both halves and each stage in order so the interviewer sees where things can break.
Blurs indexing time and query time Chunking and embedding the corpus happen offline when documents change, not on every request. Keep the two passes separate and say what triggers each.
Misses the shared embedding model If the query is embedded with a different model than the index, similarity scores are meaningless. Say that both halves must use the same model.
Blames the LLM for every bad answer The pipeline is a chain. If parsing, chunking or retrieval drops the right chunk, no model can recover it, so trace each stage before tuning the prompt.

Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Want a quick review of the fundamentals? See the RAG (Retrieval-Augmented Generation) cheatsheet.

← Back to all RAG (Retrieval-Augmented Generation) questions

Pro · $10/mo

50 of 56 RAG (Retrieval-Augmented Generation) answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

Full answers + code
AI explanations, simpler or deeper
1,000 AI credits / month
Cancel anytime

Get Pro → See plans & pricing →