↓ Skip to main content
HJP’s Evening Stroll
HJP’s Evening Stroll
  1. Posts/

Build Your Own OpenRouter Smart Gateway

·508 words·2 mins· loading · loading · ·
Cloudflare OpenRouter Serverless Tutorial AI Tools
HJP
Author
HJP
Always learning, always growing.
Table of Contents

This is a summary of a longer Chinese tutorial on building an OpenRouter proxy gateway on Cloudflare Workers. It shares only the method — no deployed endpoints or credentials.

Why build your own gateway
#

The same OpenRouter model is served by several Providers, each with different pricing, implicit-cache support, and stability. OpenRouter’s default scheduling rotates between them, which hurts KV Cache hit rate on long sessions. A thin gateway lets you take control.

What it does
#

  1. Auto-pick the best Provider — on a 3-hour Cron it fetches each model’s endpoints, scores them by throughput / (effective_input_price × 0.4 + output_price × 0.6), and locks a single Provider for a lock period (measured cache hit rate up to ~98%).
  2. Anti-abuse auth — clients send a custom CUSTOM_ACCESS_KEY; the gateway swaps it for the real OPENROUTER_API_KEY upstream.
  3. Auto-failover — 429s retry with exponential backoff (2s/4s/8s) without switching; lag/timeout/5xx trips a circuit breaker after N consecutive failures, cools the Provider for 10 min, and re-locks immediately.

It is multi-model: any model is scored and locked on first request, each model with its own lock and breaker.

Architecture
#

Cloudflare Worker + KV + Cron, fully OpenAI-compatible (/v1/chat/completions). Cron writes candidates; active requests renew the live lock so long jobs never switch upstream mid-run.

Key design points:

  • Locking: candidate + active renewal. Cron only writes candidates; the active lock is renewed in place and only promoted to a new candidate after idle expiry.
  • Scoring: effective input price = hit_rate × cache_read + (1 − hit_rate) × prompt; providers with implicit caching get a CACHE_BONUS multiplier (default 2).
  • Time-of-day pricing: OpenRouter’s overrides (e.g. official DeepSeek off-peak half price) are honored.
  • 429 vs lag: 429 retries in place (switching loses cache); real lag trips the breaker.
  • KV layout (per model): BEST_PROVIDER, PROVIDER_CANDIDATE, PROVIDER_FAILURE, PROVIDER_COOLDOWN, STATS; plus a global MODELS_SEEN.

Deploy (outline)
#

  1. npm i -D wrangler; npx wrangler login.
  2. npx wrangler kv namespace create PROVIDER_KV and put the id in wrangler.toml.
  3. npx wrangler secret put OPENROUTER_API_KEY and npx wrangler secret put CUSTOM_ACCESS_KEY.
  4. Write wrangler.toml (route custom_domain = true, Cron 0 */3 * * *, [vars] for TARGET_MODEL/HIT_RATE/CACHE_BONUS/MIN_UPTIME/SLOW_THRESHOLD/FAIL_LIMIT/COOLDOWN_SECONDS).
  5. npx wrangler deploy; verify with POST /__cron, GET /stats, and a real POST /v1/chat/completions.

The full, secret-free src/index.js (Worker logic) and the complete wrangler.toml are in the Chinese version.

Client setup
#

Point any OpenAI-compatible client at it:

  • Base URL: https://<your-gateway-host>
  • API Key: your CUSTOM_ACCESS_KEY (not the OpenRouter key)
  • Model: any OpenRouter model id, e.g. deepseek/deepseek-v4-flash-0731 — switching models needs no config change.

Gotchas
#

  • wrangler 4.x uses kv namespace create (space, not colon).
  • routes must come before [triggers] in wrangler.toml.
  • OpenRouter’s /models/{id}/endpoints rejects %2F-encoded slashes — use the raw /. This is the #1 cause of 404s and failed scoring.
  • Some official endpoints cost ~9× third-party output price; lower CACHE_BONUS (e.g. 1.2) if output-heavy jobs find third parties cheaper.

Full Chinese version: 自建 OpenRouter 智能网关：自动选最优 Provider、防盗刷、自动容错

Related