Prompt caching

Prompt caching is a provider-side optimization for repeated prompt prefixes. When the beginning of your request stays the same across calls, the upstream provider can reuse work it has already done for that prefix instead of recomputing it from scratch.

For users, the practical benefit is simple:

• Cheaper requests. Supported routes can bill repeated prompt-prefix tokens at a lower cached-input rate instead of the normal input rate.
• Faster responses. Reusing the prompt prefix reduces repeated preprocessing work, which can improve latency on follow-up turns and repeated workflows.

This is most useful when you send the same long system prompt, tool schema, document context, or agent scaffold across multiple requests.

How Tresor handles it

Tresor enables prompt caching automatically on routes that support it. You do not need a separate endpoint, a different SDK, or a special request format to benefit.

If a route advertises prompt-caching support, Tresor will use it by default for chat completions on that route. If a route does not advertise support, the request runs normally with standard pricing and behavior.

Customer isolation

Tresor isolates prompt caching by customer on routes where the provider exposes a native per-request cache key, and it can extend that isolation to selected automatic-cache providers when the deployment enables router-owned prompt-marker fallback. In that fallback mode, Tresor injects a deterministic customer-scoped marker into the forwarded prompt prefix so cache reuse stays inside that customer boundary.

Current provider modes on prompt-caching-capable routes:

NearAI and Chutes use the Tresor-injected path on current Tresor-managed deployments, which default prompt-marker fallback on. Self-managed deployments must enable that router setting if they want the same customer isolation behavior.

Important limitation

Prompt caching is not available on every route.

Today it is limited to supported chat routes. Some providers and model routes expose prompt caching, others do not. That means the best way to think about it is as a route capability, not as a global property of the API.

How to find supported routes

There are two easy ways to discover prompt-caching routes:

1. Filter the live pricing catalog

On the live pricing page, open the route catalog and filter for Prompt caching. Routes that support it also show a Cached price alongside their standard input and output prices.

• Live pricing catalog: https://tresor.co/pricing#models

2. Inspect GET /v1/models?detail=true

The models endpoint exposes prompt-caching support directly in route metadata. Look for:

• tresor.prompt_caching
• cache-aware pricing fields such as cache_input_per_mtok
• or, on providers that split cache pricing, cache_read_input_per_mtok and cache_write_input_per_mtok

Example:

{
  "id": "eu/privatemode/gpt-oss-120b",
  "object": "model",
  "owned_by": "tresor",
  "tresor": {
    "model_key": "gpt-oss-120b",
    "region": "eu",
    "provider": "privatemode",
    "type": "chat",
    "pricing": {
      "input_per_mtok": 0.43,
      "cache_input_per_mtok": 0.04,
      "output_per_mtok": 1.7,
      "currency": "eur"
    },
    "prompt_caching": {
      "supported": true,
      "verified": true,
      "default_enabled": true,
      "billing_mode": "cached_aggregate"
    }
  }
}

If you need the live source of truth for route support, use the models endpoint or the live pricing catalog instead of assuming every route for a model key behaves the same way.

What to expect in practice

• Repeated prefixes help; completely different prompts do not.
• Savings depend on the concrete route and provider pricing.
• Automatic support does not mean universal support. Check the route before you rely on cache-specific pricing.

See also

• Models
• Chat completions reference
• Routing failover