API infrastructure & backend engineering

English

Reference scenarios

Architectural reference scenarios & implementation patterns

Eight recurring infrastructure problems and the patterns we use to address them. Each scenario describes the failure mode, the engineering approach and how success is measured.

NoteThe following reference scenarios illustrate standard architectural patterns, system design methodologies, and target engineering benchmarks for resolving high-concurrency infrastructure bottlenecks.

01

High-throughput edge routing & connection pooling

API gateways & edge infrastructure
Diagram: High-throughput edge routing & connection pooling

Architectural bottleneck

Sudden concurrent webhook bursts push p99 latency far beyond budget. Each request opens a new upstream connection, so TCP and TLS handshakes, ephemeral-port pressure and blocking upstream handling pile up exactly when traffic peaks.

Engineering approach

  1. Terminate client connections at an asynchronous edge proxy and keep warm keep-alive pools to each upstream, sized from peak concurrency.
  2. Use HTTP/2 multiplexing towards upstreams that support it, so bursts share connections instead of opening new ones.
  3. Route on headers without buffering bodies; cap body size per route and stream payloads upstream.
  4. Bound queues and set per-route timeouts so overload is shed early with 503 and Retry-After instead of stalling.

How success is measured

  • p95/p99 edge latency per route within the agreed budget during synthetic burst tests
  • Upstream connection reuse ratio and new-handshake rate during bursts
  • No dropped or reset connections at the target burst size
  • Queueing time at the edge reported separately from upstream time
02

Incremental monolith decomposition (strangler fig)

Legacy system modernisation
Diagram: Incremental monolith decomposition (strangler fig)

Architectural bottleneck

A tightly coupled PHP monolith mixes business logic with database access. Every deployment touches everything, a regression in one module can take down the whole system, and no part can be scaled on its own.

Engineering approach

  1. Place a routing facade in front of the monolith so individual routes can be switched without client changes.
  2. Capture current behaviour with characterisation tests before extracting a capability.
  3. Extract one bounded domain at a time into its own container, keeping session and data contracts compatible during the transition.
  4. Shadow, then shift traffic gradually per route, and delete legacy code once the new path has proven itself.

How success is measured

  • Deployment frequency of each extracted service, independent of the monolith
  • Independent horizontal scaling of extracted containers under load
  • Zero-downtime cut-over per migrated route, verified by shadow-comparison results
  • Legacy code and tables actually removed
03

Stateless stream-based XML & structured payload parsing

High-volume data pipelines
Diagram: Stateless stream-based XML & structured payload parsing

Architectural bottleneck

DOM-based parsing of large, deeply nested XML and ISO 20022 payloads loads entire documents into memory. Under sustained throughput, workers hit memory limits, CPU saturates and one malformed file can stall a whole batch.

Engineering approach

  1. Stream documents with a hardened pull parser (no DTDs, no external entities) and process one record at a time.
  2. Validate each record against the schema version it declares, using pre-compiled validators.
  3. Deduplicate with a short-lived distributed lock keyed by message identifier, for example SET NX with a TTL in Redis.
  4. Normalise to a canonical model, publish downstream and quarantine rejects with a machine-readable report.

How success is measured

  • Stable memory per worker regardless of file size
  • Sustained messages per second at the target volume
  • Deterministic validation: the same input always gives the same accept or reject
  • Duplicate-suppression rate and quarantine volume per sender
04

Zero-trust workload identity & sender-constrained tokens

Service-to-service security
Diagram: Zero-trust workload identity & sender-constrained tokens

Architectural bottleneck

Internal services trust the network and authenticate with static API keys or long-lived JWTs. A leaked token can be replayed, and one compromised workload can move laterally to any internal API.

Engineering approach

  1. Issue short-lived X.509 workload identities (SPIFFE IDs) to every service and rotate them automatically.
  2. Enforce mutual TLS for all east-west traffic, moving from permissive to strict mode namespace by namespace.
  3. Bind access tokens to client keys with DPoP at the gateway, with a shared replay cache for proof identifiers.
  4. Add deny-by-default authorisation policies per workload identity.

How success is measured

  • Share of east-west traffic carried over mutual TLS (target: all of it)
  • Certificate rotation interval and renewal failure rate
  • Rejection of unbound, expired or replayed tokens in automated tests
  • Authorisation-policy denials reviewed and explained
05

Adaptive edge rate limiting & behavioural anomaly filtering

Traffic control & edge resilience
Diagram: Adaptive edge rate limiting & behavioural anomaly filtering

Architectural bottleneck

Automated scraping, credential stuffing and bursty non-human traffic reach the origin unfiltered. Database workers saturate, legitimate users see timeouts and capacity is spent on requests that should never have arrived.

Engineering approach

  1. Fingerprint requests at the edge (client credential, network, behaviour) and keep counters in a shared in-memory store.
  2. Apply token buckets for burst tolerance and sliding windows for strict per-period limits, keyed per client, account and route.
  3. Return 429 with Retry-After for over-limit traffic and a step-up challenge for suspicious but uncertain traffic.
  4. Serve the same content to every visitor; the filter only decides whether a request is let through.

How success is measured

  • Ratio of requests dropped at the edge to requests passed to the origin during surges
  • Rate-limit evaluation overhead at p99 (sub-millisecond target)
  • Origin database CPU stability during traffic spikes
  • False-positive rate, measured from challenge completions and support requests
06

Multi-region active-active failover & automated recovery

High availability & operational resilience
Diagram: Multi-region active-active failover & automated recovery

Architectural bottleneck

Recovery depends on someone noticing an outage and changing DNS by hand. During a regional partition or zone degradation, recovery time stretches far beyond what the business can accept.

Engineering approach

  1. Run both regions active, each sized to carry full peak load alone.
  2. Replicate state asynchronously and monitor replication lag continuously against the agreed RPO.
  3. Probe both regions with synthetic transactions and steer traffic automatically through health-checked DNS or anycast routing.
  4. Rehearse regional loss in chaos drills and keep executable runbooks for promoting stateful components.

How success is measured

  • Replication lag kept below the RPO threshold, with alerts on breach
  • End-to-end failover time in drills compared with the RTO
  • Error rate seen by synthetic probes during failover
  • Runbook steps executed without improvisation
07

Transactional outbox & idempotent event streaming

Distributed data consistency
Diagram: Transactional outbox & idempotent event streaming

Architectural bottleneck

Services write to PostgreSQL and publish to a broker in two separate steps. A crash or broker outage between the two loses events or announces changes that never committed, and downstream state drifts.

Engineering approach

  1. Write the state change and an outbox row in the same database transaction.
  2. Publish outbox rows with an asynchronous relay (polling or change data capture), partitioned by aggregate key to keep per-entity order.
  3. Make every consumer idempotent with an inbox table of processed event IDs.
  4. Send repeatedly failing events to a dead-letter topic with enough context to replay them.

How success is measured

  • No drift between state and events after simulated broker outages
  • Relay lag, measured as the age of the oldest unpublished row
  • Duplicate deliveries skipped by consumers, counted and alerted
  • Dead-letter volume and time to resolution
08

Latency-sensitive autoscaling & multi-locale edge caching

Cloud orchestration & performance
Diagram: Latency-sensitive autoscaling & multi-locale edge caching

Architectural bottleneck

CPU-threshold autoscalers react only after requests are already queueing. In latency-critical, multi-language deployments, sudden ramps raise time to first byte before new pods are ready, and cache keys without locale miss or serve the wrong language.

Engineering approach

  1. Scale on request-queue depth and p95 latency with event-driven autoscaling (KEDA) instead of CPU alone.
  2. Keep spare node capacity so new pods schedule immediately, and gate readiness on warm-up.
  3. Include the locale explicitly in edge cache keys for all six supported languages, with consistent Vary rules.
  4. Remove tight CPU limits on latency-critical pods to avoid throttling.

How success is measured

  • TTFB stability (p95) during synthetic traffic ramps
  • Time from queue growth to ready pods
  • Edge cache hit ratio per locale
  • CPU throttling on latency-critical pods (target: none)

Have a system that needs to scale or be secured?

Tell us about the architecture, the bottleneck or the deadline. We reply by email to agree a time for a scoping session.

Book a consultation