Why I built a governance-first AI gateway
There is a specific kind of frustration that comes from working in a regulated environment during an AI boom. You watch two things happen at once. On one side, individual contributors quietly start pasting work into consumer chat tools because the official answer is taking too long. No audit trail, no DLP, no idea what data left the building. On the other side, the platform team writes a procurement RFP for an enterprise AI governance platform that will, optimistically, be in production eighteen months later. Neither of those is responsible AI adoption. They are the two failure modes of waiting for the perfect tool.
Synapse AI Gateway is what I wished existed in the middle. A small, open-source, single-container thing a team can stand up in an afternoon, that enforces governance technically — on every request, before the model sees anything.
It is on GitHub: synapse-ai-gateway/synapse-ai-gateway. Apache 2.0. No enterprise tier.
The insight
The thing I kept coming back to is that governance, as most organizations practise it, is essentially a wiki page. A team gets approved to build an customer support assistant. There is a deck. The deck says what data is allowed in, what data is not, what model they can use, and who is accountable. The deck gets approved. Then the deck is forgotten, and the team builds whatever they are going to build.
That is not governance. That is documentation. The two are not the same thing.
If you want governance, you need to put the controls in the path of the request. The request itself has to be checked, every time, by something that cannot be bypassed by changing the application code. That sounds obvious when you say it out loud, but most of the AI tooling I looked at treated governance as a feature on top of a routing tool. Routing was the design centre. Governance was a plugin.
I wanted to flip that. Governance is the design centre. Everything else is in service of it.
The shape of the answer
The architectural primitive is this: every API key issued by the gateway is bound, at the moment of issuance, to a system prompt, a model allowlist, a team identity, and rate limits. That binding is not a configuration the team can override from the application. It is part of the credential. When the team makes a request, the gateway uses the key to look up the bound system prompt and injects it. The team's system message in the OpenAI payload is appended after, not in place of.
Why does this matter? Because it means use-case approval is technically enforced, not policy-enforced. A team that received a key to build an customer support assistant cannot turn around and use that key to build a code review bot. The system prompt will not let them. The model allowlist will not let them. If they need a code review bot, they need a new key, which means a new approval. The gateway scales the approval process — anyone can build, but within the guardrails the organization set.
Around that primitive, four more layers sit on every request:
The bound system prompt is injected and the model allowlist is checked. Invalid key or unapproved model returns 403 immediately.
A built-in regex DLP engine scans the incoming prompt. Three outcomes are configurable per category: block, redact, alert. No external service involved.
Hybrid routing decides between on-premises (Ollama, vLLM) and cloud (OpenAI, Anthropic, Azure, Google). The team's data classification on the key is what decides — sensitive data never leaves the perimeter.
An append-only PostgreSQL audit log captures every request, including rejected ones. SHA-256 hashes of prompts and responses, never plaintext. Token count, latency, DLP outcome, HTTP status, team identity.
The fifth layer is response-side: DLP scan on the way back out, plus anomaly heuristics that webhook into a security pipeline.
A design decision worth dwelling on
The hash-not-plaintext choice in the audit log felt important to get right. The temptation, especially in a regulated environment, is to log everything — the prompt, the response, the timestamps — so you can reconstruct what happened. The problem is that "everything" includes whatever the employee typed, which often includes things the employee did not realise they typed. Customer names. Medical details. Things that should not exist in a database that the security team can query.
Hashes give you most of what you actually need. You can prove that a specific prompt was sent. You can hash-match across users and across time to detect reuse or exfiltration patterns. You cannot recover the plaintext, and that is the point. The privacy guarantee is by design, not by policy.
Quick start
I tried hard to make the first-five-minutes experience match the promise:
git clone https://github.com/synapse-ai-gateway/synapse-ai-gateway
cd synapse-ai-gateway
docker compose up -d
Every setting has a working default — that is the whole quick start for a local trial. Admin console at http://localhost:5173, gateway at http://localhost:8080. The full stack runs at ~113 MB at idle (backend 73 MB + postgres 32 MB + frontend 8 MB). The test suite has 97 tests at around 88% line coverage. The CI runs Bandit and Trivy on every push. GDPR, HIPAA, and PCI-DSS policy packs ship out of the box — one-click apply with pre-configured DLP patterns. Per-team spend attribution with budget alerts is built in.
What this is not
This is not a replacement for LiteLLM if you are routing millions of requests per day. It is not a replacement for an enterprise governance platform if you need SOC 2 attestation and 24/7 support. It is not magic. The controls are real, but the policy decisions — what counts as sensitive in your jurisdiction, which models you are willing to call — are still yours to make.
What it is, is the governance layer a small team can deploy before they are ready for enterprise tooling. That is the part of the curve I wanted to address.
If your organization is staring at the gap between shadow AI and the eighteen-month RFP, this is meant for you. The repo has issues open for additional DLP pattern packs, additional backend adapters, and a Helm chart. Contributions welcome.