Needle finder

A wall of release notes and ops notes is already in the prompt; ask for one buried fact and the model pulls it out — the "needle in a haystack" test long-context models are measured on. Try "who was on call for incident 4419?" or "what flag disables telemetry?".

README Try it Source

Showcase — Needle finder

Run

bash bootstrap-secrets.sh              # reads ../../../../.env, writes secrets/
docker compose up --build              # default: PROVIDER=openai

Open http://localhost:3000. Gemini: PROVIDER=gemini docker compose up --build.

What's where

backend/haystack.py — the long document (shared by both providers).
backend/ai_openai.py / backend/ai_gemini.py — answer a question from the whole haystack.
frontend/app/page.tsx — question box, exact-detail answer.

Stop

docker compose down

Run locally

Download the project as a ZIP and run it with Docker. Brings up a FastAPI backend + Next.js frontend on localhost:3000.

Download needle-finder.zip

unzip needle-finder.zip
cd needle-finder
bash bootstrap-secrets.sh   # one-time: pulls API keys into ./secrets
docker compose up --build   # default provider: openai
# or:  PROVIDER=gemini docker compose up --build

Type some input, pick a provider, and run the same code shown in Source against the live API. Sign-in required.

Input

Provider

Output

The same modules the Run button hits. The whole project (frontend, Dockerfile, compose) is in the ZIP under README.

`backend/ai_openai.py`

"""Showcase 3 (OpenAI): find a needle in a long document.

A wall of release notes and ops notes goes into the prompt; you ask for one
buried fact and the model pulls it out. This is the "needle in a haystack" test
that long-context models are measured on — no retrieval, the whole document is
just there.
"""
from openai import OpenAI

from haystack import HAYSTACK

_client = OpenAI()

_MODEL = "gpt-5.4-nano"


def run(question: str) -> str:
    q = question.strip()
    if not q:
        return "Ask about a fact in the document (e.g. 'who was on call for incident 4419?')."
    response = _client.responses.create(
        model=_MODEL,
        instructions=(
            "Answer the question using ONLY the document below. Quote the exact "
            "detail. If it isn't in the document, say so.\n\n" + HAYSTACK
        ),
        input=[{"role": "user", "content": q}],
    )
    return response.output_text

`backend/ai_gemini.py`

"""Showcase 3 (Gemini): find a needle in a long document.

Same haystack, same buried-fact lookup, on Gemini.
"""
import os

from google import genai
from google.genai import types

from haystack import HAYSTACK

_client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

_MODEL = "gemini-3.1-flash-lite"


def run(question: str) -> str:
    q = question.strip()
    if not q:
        return "Ask about a fact in the document (e.g. 'who was on call for incident 4419?')."
    response = _client.models.generate_content(
        model=_MODEL,
        contents=[types.Content(role="user", parts=[types.Part(text=q)])],
        config=types.GenerateContentConfig(
            system_instruction=(
                "Answer the question using ONLY the document below. Quote the exact "
                "detail. If it isn't in the document, say so.\n\n" + HAYSTACK
            ),
        ),
    )
    return response.text or ""