Security & Governance

DifficultyBeginner

Putting a gateway in front of your models means it is now the place identity, authorization, and spend are decided. Three jobs are worth separating, because they fail differently:

1. Identity — which caller is this?
2. Tenancy — which tenant does the caller belong to? This is what rate limits, budgets, and usage attribution key off.
3. Authorization — what is this caller allowed to reach, and what may it do once it gets there?

Order matters. A gateway without authentication has no tenants, which means rate limits and budgets have nothing to attach to and guardrail policy has no one to apply to. Configure identity first; everything in this family depends on it.

The three layers

Authentication & RBACIssue scoped credentials, map them to tenants, and restrict which models and operations each client may reach. Documented against Oculis.

Guardrails & safetyPII redaction, prompt-injection classification, structural fencing, and choosing fail-open or fail-closed.

Rate limits & token budgetsPer-tenant request rates, token ceilings, and spend caps — plus what to do when a tenant hits one.

Why authentication lives under Oculis

Credential formats, scope vocabularies, and key-management commands are specific to the gateway you run — there is no portable keys issue. That page documents Oculis’s implementation. The principles below are the part that transfers.

Principles that outlive the gateway you picked

Separate what a credential may do from what it may reach. Scopes govern operations; route, model, or resource grants govern reachability. Enforcing only one of them is the common mistake — a key scoped to send inference requests can still reach your most expensive model unless reachability is restricted too.

401 is an identity problem, 403 is a permissions problem. Keep them distinct in both responses and logs. Collapsing them into a single “denied” makes production debugging guesswork, and it hides the difference between a rotated credential and a missing grant. Expired credentials belong in the 401 bucket: expiry is an identity failure, not an authorization one.

API keys are bearer tokens. Anything that can read a request header can replay them. Terminate TLS in front of the gateway and never log the Authorization header — your gateway may redact it, but an upstream proxy or sidecar might not.

Tenant identity must be immutable. Limits, budgets, and usage attribution key off the tenant. If tenancy is derived from something mutable — an email address, a display name — then a user who changes it silently becomes a brand-new tenant with a fresh budget. Bind to an immutable organization ID.

Fail closed, and keep a break-glass path. If an external identity provider is unreachable and the key cache has expired, rejecting is the correct behavior. That is also exactly when you cannot log in to fix it, so keep a second credential type that does not depend on the provider being up.

Revocation is eventually consistent. Replicas cache credential lookups, so a revoked key keeps working until the cache expires. Immediate fleet-wide revocation means a shared store and a lookup on every request. That is a real latency cost, and it is a deliberate choice rather than a default.

Never expose the metrics endpoint publicly

Gateway metrics reveal tenant names, request volumes, spend, and which models you run. The scrape path is usually exempt from authentication so Prometheus can reach it — bind it to an internal interface or put it behind network policy. It must not sit on a public load balancer.

Next steps

Authentication & RBACStart here — the rest of this family has nothing to attach to without it.

Authentication failuresDiagnosing 401 and 403 in production.