Ahorro con cache

Un prefijo estático grande va al frente de cada pregunta. Pregunta, luego pregunta otra vez — la segunda llamada reporta la mayor parte de su input como CACHED, facturado barato. Así es como un system prompt grande o un contexto largo se mantiene accesible.

README Pruébalo Código

Showcase — Ahorro con cache

Córrelo

bash bootstrap-secrets.sh              # reads ../../../../.env, writes secrets/
docker compose up --build              # default: PROVIDER=openai

Abre http://localhost:3000. Gemini: PROVIDER=gemini docker compose up --build.

Qué hay aquí

backend/ai_openai.py / backend/ai_gemini.py — prefijo fijo grande, reporta el conteo de tokens cacheados.
frontend/app/page.tsx — pregunta dos veces y observa saltar los tokens cacheados.

Detenlo

docker compose down

Ejecútalo en tu máquina

Descarga el proyecto como ZIP y córrelo con Docker. Levanta un backend FastAPI y un frontend Next.js en localhost:3000.

Descargar cache-saver.zip

unzip cache-saver.zip
cd cache-saver
bash bootstrap-secrets.sh   # one-time: pulls API keys into ./secrets
docker compose up --build   # default provider: openai
# or:  PROVIDER=gemini docker compose up --build

Escribe algo, elige un proveedor y ejecuta el mismo código de Código contra la API real. Requiere iniciar sesión.

Entrada

Proveedor

Output

Los mismos módulos que ejecuta el botón Run. El proyecto completo (frontend, Dockerfile, compose) está en el ZIP, pestaña README.

`backend/ai_openai.py`

"""Showcase 2 (OpenAI): watch prompt caching pay off.

A large static reference prefix sits in front of every question. Ask something,
then ask again: the second call reports most of its input tokens as CACHED, and
cached input is billed at a fraction of the price. This is how you make a
long-context or big-system-prompt app affordable — the fixed part is nearly free
after the first hit.
"""
from openai import OpenAI

_client = OpenAI()

_MODEL = "gpt-5.4-nano"

# The big fixed prefix — repeated to be large enough for caching to engage.
_CONTEXT = "Company handbook.\n" + (
    "Employees accrue 15 PTO days; sick leave is separate at 8 days. Remote work "
    "is allowed up to 3 days a week with approval. Expenses are reimbursed within "
    "30 days with a receipt. Parental leave is 12 weeks paid. " * 200
)


def run(question: str) -> str:
    q = question.strip() or "Summarize the handbook."
    response = _client.responses.create(
        model=_MODEL, instructions=_CONTEXT,
        input=[{"role": "user", "content": q}],
    )
    u = response.usage
    cached = getattr(getattr(u, "input_tokens_details", None), "cached_tokens", 0) or 0
    pct = (100 * cached / u.input_tokens) if u.input_tokens else 0
    return (f"{response.output_text}\n\n"
            f"--- metrics ---\n"
            f"input tokens:  {u.input_tokens}\n"
            f"cached tokens: {cached}  ({pct:.0f}% of input)\n"
            f"output tokens: {u.output_tokens}\n"
            f"Ask again — cached should jump and the effective cost drops.")

`backend/ai_gemini.py`

"""Showcase 2 (Gemini): watch prompt caching pay off.

Same large fixed prefix; Gemini reports cached_content_token_count on repeat
calls for its caching-capable models.
"""
import os

from google import genai
from google.genai import types

_client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

_MODEL = "gemini-3.1-flash-lite"

_CONTEXT = "Company handbook.\n" + (
    "Employees accrue 15 PTO days; sick leave is separate at 8 days. Remote work "
    "is allowed up to 3 days a week with approval. Expenses are reimbursed within "
    "30 days with a receipt. Parental leave is 12 weeks paid. " * 200
)


def run(question: str) -> str:
    q = question.strip() or "Summarize the handbook."
    response = _client.models.generate_content(
        model=_MODEL,
        contents=[types.Content(role="user", parts=[types.Part(text=q)])],
        config=types.GenerateContentConfig(system_instruction=_CONTEXT),
    )
    u = response.usage_metadata
    cached = getattr(u, "cached_content_token_count", 0) or 0
    pct = (100 * cached / u.prompt_token_count) if u.prompt_token_count else 0
    return (f"{response.text or ''}\n\n"
            f"--- metrics ---\n"
            f"input tokens:  {u.prompt_token_count}\n"
            f"cached tokens: {cached}  ({pct:.0f}% of input)\n"
            f"output tokens: {u.candidates_token_count or 0}\n"
            f"Ask again — cached should jump and the effective cost drops.")