High-throughput API gateway & routing
A gateway layer that authenticates, limits, validates and routes every request, with latency budgets set per route and observability built in from the first deployment.
Flow
- TLS terminates at the edge; partner traffic presents client certificates.
- Tokens are verified locally against cached signing keys; scopes are mapped to routes.
- Per-client and per-route token buckets enforce quotas before any upstream work.
- Requests are validated against the OpenAPI schema and size limits.
- The router applies weights, timeouts and bounded retries with jitter to healthy upstreams.
- Each stage emits spans and RED metrics; security-relevant decisions go to an append-only audit log.
Design decisions
- Local or remote token validation?
- Local JWT verification with short-lived tokens and cached keys keeps p99 low; introspection is reserved for high-risk routes.
- Where to enforce rate limits?
- At the gateway with a shared counter store, so limits hold across replicas and before expensive upstream calls.
- Retries
- Only for idempotent methods or requests carrying an idempotency key, with a per-request retry budget to avoid amplification.
Deliverables
- Gateway configuration as code
- Route catalogue with p99 budgets and limits
- Load-test scripts and report
- Operations runbook