Architecture
Design Back Pressure
Back pressure propagates overload signals upstream to prevent cascading failures. When a downstream service is overloaded, upstream services reduce their request rate rather than overwhelming the system.
- Problem — Overload causes cascading failures
- Solution — Signal propagation to reduce load
- Goal — System operates at sustainable throughput
Back pressure is the immune system of distributed architectures: it detects overload and triggers protective responses.
What Is Back Pressure?
DfBack Pressure
Back pressure is a feedback mechanism where a downstream component signals an upstream component to reduce its request rate. This prevents queue buildup, memory exhaustion, and eventual system collapse. Back pressure can be explicit (HTTP 429, 503) or implicit (queue full, slow consumer).
Without back pressure, a system under load will: (1) Build unbounded queues, (2) Exhaust memory, (3) Start dropping requests, (4) Cascade failures to dependent services. Back pressure is the circuit breaker's companion.
Back Pressure Mechanisms
Strategies
1. Queue-Based Back Pressure
Queue Threshold
Here,
- =Maximum queue capacity
- =Current queue depth
2. Rate Limiting
DfToken Bucket Rate Limiting
Tokens are added to a bucket at a fixed rate. Each request consumes one token. If no tokens are available, the request is rejected. This smooths traffic bursts while allowing sustained throughput.
Token Bucket
Here,
- =Maximum tokens (burst size)
- =Tokens per second (sustained rate)
3. Adaptive Load Shedding
Adaptive load shedding drops requests when system health degrades. Metrics like CPU, memory, latency, and error rate trigger shedding. Priority-based shedding protects critical paths.
4. Timeout-Based Back Pressure
Timeout Propagation
Here,
- =Maximum acceptable latency
- =Observed response time
Health Check Signals
| Signal | Threshold | Action |
|---|---|---|
| CPU > 80% | High | Reduce request rate |
| Memory > 85% | Critical | Shed load aggressively |
| Latency P99 > 500ms | Warning | Queue and slow down |
| Error rate > 5% | Critical | Circuit breaker open |
Reactive Streams
DfReactive Streams (Back Pressure Protocol)
Reactive Streams define a standard for asynchronous stream processing with back pressure. The subscriber tells the publisher how many elements it can handle via request(n). This prevents the publisher from overwhelming the subscriber.
Practice Exercises
- Design: Implement back pressure for a microservice with 3 downstream dependencies at different capacities.
- Monitoring: Design a health check system that detects overload in < 5 seconds.
- Priority: Design priority-based load shedding where payment requests are never shed.
- Recovery: How do you gracefully resume normal load after back pressure is triggered?
Key Takeaways:
- Back pressure propagates overload signals upstream to prevent cascading failures
- Strategies: queue thresholds, rate limiting, adaptive shedding, timeout propagation
- Token bucket provides smooth rate limiting with burst allowance
- Reactive Streams define a standard for back pressure in async systems
- Health metrics (CPU, memory, latency) drive adaptive back pressure decisions
What to Learn Next
-> Circuit Breaker Preventing cascade failures.
-> Retry Patterns Resilient retry mechanisms.
-> Idempotency Handling duplicate requests.
-> Saga Pattern Distributed transactions.
-> Load Balancing Distribution algorithms.
-> Design Netflix Resilient microservices.