RAG end to end

Chapter 11 · answer from your documents, honestly

The hour

• Retrieve → augment → generate
• Grounding lives in the prompt, not in hope
• Cite sources; refuse when unsupported
• RAG's ceiling is retrieval's ceiling

The three steps

• Retrieve — top-k chunks by embedding similarity
• Augment — paste them into the prompt as context
• Generate — answer only from that context

Knowledge the model was never trained on, no fine-tuning.

Retrieve

KB_VECS = embed([c["text"] for c in KB])

def retrieve(question, k=3):
    qv = embed([question])[0]
    scored = sorted(zip(KB, KB_VECS),
        key=lambda cv: cosine(qv, cv[1]), reverse=True)
    return [chunk for chunk, _ in scored[:k]]

Generate — the rules do the work

instructions=(
  "Answer using ONLY the context below. "
  "Cite sources by their [id]. "
  "If the context lacks the answer, say "
  "'I don't have that in the docs.'\n\n"
  f"Context:\n{context}")

Drop these rules and the model answers from memory — confidently, wrongly.

The honest third answer

Q: How long do refunds take?      → cited answer
Q: Can I return an opened blender? → cited answer
Q: What's your CEO's name?        → "I don't have that in the docs."

Refusal is a feature, not a bug.

Put it to work — three apps

• Document Q&A — cited answers, retrieved ids shown
• Grounded refuse — narrow corpus, refuses out-of-scope
• Cited summary — synthesize several notes, with sources

Same loop; the corpus and instruction change.

The debugging rule

RAG quality is retrieval quality.

Wrong answer? Look at what was retrieved before you blame the model. Usually the chunk it needed never made the top-k.

Takeaway

Retrieve, augment, generate — and force the model onto the context: answer only from it, cite it, refuse without it. Next: vision, where the document is an image.