Embeddings

What this session covers

Everything so far has treated the model as something you talk to. This week it does something quieter, and for most of the production systems I've built, more useful: it turns text into numbers. An embedding is a vector — a few hundred floats — positioned in space so that text meaning similar things lands near each other. "Refund my order" and "how do I get my money back" share no words, and their embeddings are still neighbors. Once meaning is geometry, "related" stops being a fuzzy idea and becomes a distance. You can measure it, sort by it, put a threshold on it.

That primitive carries the next three chapters. Semantic search, vector databases, retrieval-augmented generation all sit on top of it, and it earns its keep on its own for search, classification, clustering, and dedup. The mechanics are small. One API call embeds a batch, and cosine similarity is four lines of arithmetic. The judgment is in what you do with the distances, and that's the part nobody hands you.

OpenAI

The example embeds six sentences and one query, then ranks the sentences by cosine similarity to the query. Watch what it does: "a friendly canine companion" pulls the dog sentences to the top and drops the stock-market and photosynthesis lines to the bottom. Zero shared keywords. Meaning did all of it.

# One API call embeds a whole batch — cheaper and faster than one call per
# string. Each result is a vector (a list of floats); same model, same length
# every time, which is what makes them comparable.
def embed(texts: list[str]) -> list[list[float]]:
    response = client.embeddings.create(model=_MODEL, input=texts)
    return [item.embedding for item in response.data]

One call embeds the whole batch. Do that, not one call per string — it's cheaper, it's faster, and every vector comes back the same length so they compare directly. Then the only math you need:

# Cosine similarity: the cosine of the angle between two vectors. 1.0 is the
# same direction (as similar as it gets), 0.0 is unrelated. Magnitude doesn't
# matter, only direction — which is why it beats raw distance for text.
def cosine(a: list[float], b: list[float]) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    na = math.sqrt(sum(x * x for x in a))
    nb = math.sqrt(sum(y * y for y in b))
    return dot / (na * nb) if na and nb else 0.0

Cosine similarity is the cosine of the angle between two vectors. 1.0 means same direction, as related as it gets; 0.0 means unrelated. It ignores magnitude and looks only at direction, which is what you want for text. A long document and a short query about the same thing should still count as close, and cosine makes that happen for free.

Gemini

Gemini's embedding model returns an embeddings list with a .values vector per input. Swap the embed call and everything downstream is identical — the cosine function, the ranking, all of it. Once you have vectors, the provider is out of the picture.

# Gemini returns an `embeddings` list, one entry per input, each with a
# `.values` vector. Batch the inputs in one call, same as OpenAI.
def embed(texts: list[str]) -> list[list[float]]:
    response = client.models.embed_content(model=_MODEL, contents=texts)
    return [e.values for e in response.embeddings]

Learn this one early, because it bites people in production: embeddings from different models don't compare. A vector from OpenAI's model and a vector from Gemini's live in different spaces. You can't cosine them against each other, and the day you switch embedding models is the day you re-embed the whole corpus. Pick one per index and stay put.

Put it to work

Three docker-compose apps under code/showcase/<slug>/, same drill: bash bootstrap-secrets.sh, docker compose up --build, http://localhost:3000. Not one of them calls a chat model. Embeddings plus arithmetic is the whole engine, which is exactly why they're fast and cheap.

Showcase 1 — Semantic search

A small help center searched by meaning. The corpus is embedded once and cached; each query is one embedding call, then cosine against every doc. "How do I get my money back" returns the refund policy it shares no words with. This is retrieval in miniature. The cosine loop here becomes a vector database next chapter, but the idea underneath doesn't move.

Showcase 2 — Label classifier

Zero-shot classification with no training. Each category is a one-sentence description, embedded once; an incoming message is embedded and assigned to the nearest description. Adding a category means writing a sentence, and the whole "retraining" step is instant. I'll say it plainly: this is a strong baseline, and it's the honest thing to try before you reach for a fine-tune.

Showcase 3 — Dedup finder

Paste a list and find the pairs that mean the same thing despite different wording. Every line is embedded, all pairs compared by cosine, the closest surfaced. Exact-match dedup can't see "I can't log in" and "I'm locked out of my account" as the same issue. Embeddings can. The all-pairs comparison is O(n²), fine for a text box and hopeless for a million rows, which is the exact problem a vector database exists to solve.

All three run the same backend/main.py off PROVIDER, so Gemini is one environment variable away. And all three rest on the same two operations: embed a batch, compare by cosine. Everything past that is application logic.

Run it

The README in this folder has the Python version, the install line, the two environment variables, and the exact commands for the basic examples and each showcase. Keys come from the untracked .env at the course root. Cosine similarity is pure standard library, so the only things you install are the SDKs.

Takeaways

Embeddings turn text into geometry, and geometry you can compute with. That's the whole unlock. Two operations carry a surprising amount of production software: embed a batch, compare by cosine. A few things keep you out of trouble. Batch your embeddings. Never mix models inside one index, because different models mean different spaces. And treat similarity as a continuous score, not a yes/no — you pick the threshold, and the right threshold is something you find by measuring, not by guessing. This chapter did all its searching with a Python loop over a handful of documents. Next week that loop hits its ceiling and we hand the job to a vector database built to run it over millions.