Prompt caching - RedPill AI

Prompt caching happens at the provider and needs nothing in the request. When a provider serves part of a prompt from its cache, the response reports it in usage, and the cached tokens are billed at pricing.input_cache_read when the model lists one, otherwise at the normal prompt rate.

"usage": {
  "prompt_tokens": 1905,
  "prompt_tokens_details": {"cached_tokens": 1792},
  "completion_tokens": 64
}

To benefit, keep the stable part of a prompt, such as the system message and reference material, at the start, and vary only the end. OpenAI models cache prefixes of 1,024 tokens or more on repeated requests. Which models expose a cache rate is visible in the catalog: look for input_cache_read in pricing.

Billing

Models