Embeddings

Chapter 9 · turning text into geometry

The hour

• Text → vector (a point in meaning-space)
• Cosine similarity: "related" as a number
• Search, classify, dedup — no chat model needed
• The foundation for search, vector DBs, RAG

The idea

"Refund my order" and "how do I get my money back"

share no words — but their embeddings are neighbors. Meaning becomes distance.

Embed a batch

def embed(texts):
    r = client.embeddings.create(
        model="text-embedding-3-small", input=texts)
    return [item.embedding for item in r.data]

One call per batch, not per string. Same model → same length → comparable.

Cosine similarity

def cosine(a, b):
    dot = sum(x*y for x, y in zip(a, b))
    na = math.sqrt(sum(x*x for x in a))
    nb = math.sqrt(sum(y*y for y in b))
    return dot / (na*nb) if na and nb else 0.0

1.0 = same direction. 0.0 = unrelated. Magnitude ignored.

Query "a friendly canine companion"

0.61  A dog is a loyal and affectionate pet.
0.58  Golden retrievers are gentle family companions.
0.11  The stock market fell sharply on Tuesday.
0.09  Photosynthesis converts sunlight into energy.

Zero shared keywords. Meaning did the ranking.

Put it to work — three apps

• Semantic search — rank a corpus by meaning
• Label classifier — zero-shot, no training
• Dedup finder — near-duplicates by meaning

No chat model. Embed + cosine is the whole engine.

Two rules

• Never mix embedding models in one index (different models = different spaces)
• Similarity is a score, not a yes/no — you pick the threshold, empirically

Takeaway

Embed a batch, compare by cosine. Those two operations carry an enormous amount of production software. Next: the Python loop meets a vector database.