Developer Documentation

Everything you need to integrate CachePilot in under 5 minutes. One line of code. No SDK. No config files.

Quick Start

CachePilot is a caching proxy that sits between your app and your AI provider. Change one string in your code and every duplicate request returns from cache at zero provider cost.

Step 1: Get your API key

Sign up at cachepilot.dev/signup to get your sk-cache-xxx-yyy key. Or skip signup entirely — use auto-provisioning (see below).

Step 2: Change your base URL

Replace your provider's base URL with CachePilot's. Everything else stays the same — your SDK, auth, streaming, response format.

python

from openai import OpenAI

client = OpenAI(
    base_url="https://cachepilot.koaw.workers.dev/v1",
    api_key="sk-cache-xxx-yyy"  # Your CachePilot key
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)
          

javascript

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://cachepilot.koaw.workers.dev/v1',
  apiKey: 'sk-cache-xxx-yyy'  // Your CachePilot key
});

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello!' }]
});
          

curl

curl https://cachepilot.koaw.workers.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-cache-xxx-yyy" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
          

go

client := openai.NewClient(
  "sk-cache-xxx-yyy",
  "https://cachepilot.koaw.workers.dev/v1",
)

resp, err := client.Chat.Completions.New(ctx,
  openai.ChatCompletionNewParams{
    Model: "gpt-4o",
    Messages: []openai.ChatCompletionMessageUnionParam{...},
  },
)
          

Step 3: Verify it works

Check the response headers. A cache hit returns immediately with X-CachePilot-Status: HIT and zero provider cost. A miss forwards to your provider normally.

response headers

# Cache HIT — response served from cache, $0 provider cost
X-CachePilot-Status: HIT
X-CachePilot-Savings: $0.0042

# Cache MISS — forwarded to provider, full response cached for next time
X-CachePilot-Status: MISS
X-CachePilot-Savings: $0.0000
        

That's it. Every subsequent duplicate request returns from cache automatically. No configuration, no tuning, no cache invalidation to manage.

Auto-Provisioning

Skip signup entirely. Send your provider API key in the x-provider-key header and CachePilot auto-creates an account, returning your sk-cache-xxx key in the response header.

curl — auto-provision

# First request: no CachePilot key needed — just your provider key
curl https://cachepilot.koaw.workers.dev/v1/chat/completions \
  -H "x-provider-key: sk-your-openai-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hello"}]}'

# Response header contains your new CachePilot key:
# X-CachePilot-Key: sk-cache-22b8ea87-6ae2c11c

# Use that key for all subsequent requests (no x-provider-key needed)
curl https://cachepilot.koaw.workers.dev/v1/chat/completions \
  -H "Authorization: Bearer sk-cache-22b8ea87-6ae2c11c" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hello"}]}'
        

Auto-provisioned keys are stored in your account and can be viewed in the dashboard. First $20 of savings are free.

Authentication

CachePilot supports two authentication methods:

Method	Header	When to use
CachePilot key	`Authorization: Bearer sk-cache-xxx`	Standard usage — all requests after signup or auto-provision
Provider key	`x-provider-key: sk-xxx`	Auto-provisioning — first request only

When using x-provider-key, the provider key is passed through to the upstream API and never stored. CachePilot only uses it for the single request.

API Reference

Base URL

https://cachepilot.koaw.workers.dev/v1

Supported Endpoints

Endpoint	Method	Description
`/v1/chat/completions`	POST	Chat completions (OpenAI-compatible)

CachePilot is compatible with any client that speaks the OpenAI chat completions API — including Anthropic (via OpenAI compatibility), OpenRouter, and custom providers.

Request & Response Headers

Request Headers

Header	Required	Description
`Authorization`	Yes*	`Bearer sk-cache-xxx` — your CachePilot API key
`x-provider-key`	No	Your provider API key for auto-provisioning. Pass on first request only.
`x-cachepilot-mode`	No	`passthrough` — bypass all caching for this request

* Either Authorization or x-provider-key is required.

Response Headers

Header	Description
`X-CachePilot-Key`	Your CachePilot API key (returned on auto-provision requests)
`X-CachePilot-Status`	`HIT` (cached) or `MISS` (forwarded to provider)
`X-CachePilot-Savings`	Dollar amount saved on this request (e.g., `$0.0042`)

Caching Behavior

What gets cached

Exact duplicate requests — same model, same messages, same parameters
Normalized matches — whitespace differences, key ordering, and formatting variations are normalized before hashing
Non-streaming responses — full response bodies cached at multiple layers

What doesn't get cached

Streaming responses — forwarded directly to preserve SSE format
Passthrough requests — set x-cachepilot-mode: passthrough to bypass caching entirely
Requests with temperature > 0 — non-deterministic responses shouldn't be shared across users

Cache layers

Layer	Speed	TTL	Description
L0 — In-memory	<1ms	30s	Hot responses in process memory, LRU eviction, max 500 entries
L1 — Edge cache	<5ms	1hr	Cloudflare Cache API, shared across edge locations
L2 — KV store	<50ms	24hr	Cloudflare KV, globally distributed, tenant-isolated

Normalization

Cache keys are generated from a normalized representation of the request:

Model name lowercased and trimmed
Message content whitespace collapsed (multiple spaces → single space)
Object key ordering standardized
All parameters except stream included in the hash

Session-Aware Caching

CachePilot detects multi-turn conversations automatically. When a request contains a prefix of previously seen messages, it recognizes this as a session continuation.

How it works

Each request's message sequence is hashed
If the current messages are a strict superset of a previous request, the system identifies it as a continuation
The stable prefix gets Anthropic-compatible cache_control headers injected automatically
Only new messages are processed at full price; the prefix is served from cache

Anthropic users benefit most. Claude's native prompt caching charges only 10% of input cost for cached prefixes. CachePilot injects the cache_control markers automatically — no code changes needed.

Pricing

CachePilot charges 20% of your calculated savings. No subscription. No minimum. No setup fee.

Scenario	Provider cost (no cache)	Your cost (with cache)	CachePilot fee
60% cache hit rate	$100	$40	$12 (20% of $60 saved)
80% cache hit rate	$100	$20	$16 (20% of $80 saved)
0% cache hit rate	$100	$100	$0 (nothing saved)

First $20 of savings are free. No trial period — if you save less than $20 total, you never pay a cent. After that, 20% of cumulative savings.

View your real-time savings breakdown in the dashboard.