Vector store and retrieval

Chapter 10 · the search loop, given a home

The hour

• Last week's loop → a reusable store
• add() + search(): the interface that scales
• Metadata filtering: narrow, then rank
• Where the toy ends and pgvector begins

Why not just a loop?

Two things break at scale:

• You re-embed vectors you already have
• Every query rescans every document

A vector store fixes both.

add() + search()

class VectorStore:
    def add(self, docs):
        vecs = self._embed([d["text"] for d in docs])
        for d, v in zip(docs, vecs):
            self._items.append({**d, "vec": v})

    def search(self, query, k=3, where=None):
        qv = self._embed([query])[0]
        pool = [it for it in self._items
                if not where or matches(it, where)]
        return top_k_by_cosine(qv, pool, k)

Two design choices

• Takes an embed function → same class for OpenAI + Gemini
• Filters by metadata before ranking

"Like this query, but only billing, only this customer, only since March."

Where the toy ends

Still a linear scan. A real database (pgvector, Qdrant, Pinecone) adds:

• An approximate-nearest-neighbor index
• Persistence
• Metadata filters at scale

Same add / search interface.

Put it to work — three views

• Document retriever — top passages for a question (RAG's front half)
• Metadata filter — billing: narrows, then rank
• Recommender — hand it an item, get neighbors

One shared store.py; only the corpus + query differ.

The rule that grows teeth

The embedding model is part of your schema.

Change it → every stored vector is meaningless → re-embed the whole corpus. Version it.

Takeaway

A vector store is add() + search() with vectors kept beside the text. You now have RAG's retrieval half. Next: hand the passages to a model and answer from them, with citations.