A production-grade gateway that routes, governs, and observes every call your product makes to a language model. 38 providers, 255 models, under 5ms overhead, with hash-chained audit and budgets that hold.
38
Providers
OpenAI · Anthropic · Groq · Gemini · Bedrock · Vertex · 32 more
255
Models
Frontier and open-source, one interface
<5ms
Overhead
Measured on 1 vCPU at 350 req/s
62%
Cache Win
Typical savings on repetitive RAG workloads
Capabilities compared line-by-line against TrueFoundry, Portkey, LiteLLM, and Cloudflare AI Gateway. Eight features none of the field presently offers.
Weighted, cost-optimised, latency-aware, geo-aware routing with automatic fallback chains across 38 providers.
Vector-similarity lookup against prior completions. Configurable thresholds. Up to 62% savings on repetitive RAG workloads.
Every completion is appended to an immutable ledger. One-click verification, shaped for EU AI Act Article 50.
HIPAA, PCI-DSS, GDPR, EU AI Act — bundled policy sets applied in one action with guardrails and redaction.
Per-session budgets, loop detection, and trust scoring. The gateway meters agent workloads autonomously.
Replay your audit log against alternative routing policies. See exact cost, latency, and quality deltas before changing config.
Quick Start
MIT-licensed, self-hosted, no managed lock-in. Bring your own API keys. No telemetry unless you configure it. OpenAI-compatible — any client that speaks to OpenAI speaks to it unchanged.
Open the Consolefig. 1 — single command deployment
$ docker run -p 8080:8080 \
-e AIGW_ADMIN_TOKEN=$(openssl rand -hex 32) \
-e OPENAI_API_KEY=sk-... \
-e ANTHROPIC_API_KEY=sk-ant-... \
ghcr.io/shrinidhisuresha/aigateway:latest
→ Listening on :8080
→ Console http://localhost:8080/
→ OpenAPI http://localhost:8080/docsOne endpoint, every model, full control. Deploy in minutes and start routing your first request today.