API Rate Limiting & Throttling

Why Rate Limiting?

Without rate limiting:

• One client can overwhelm your API - A single client can send unlimited requests, consuming all resources
• DDoS attacks possible - Attackers can easily overwhelm your service with excessive requests
• Unfair resource usage - Aggressive clients consume resources meant for all users
• Backend services crash - Overload can cause services to fail, affecting all users

With rate limiting:

• Fair usage for all clients - Each client gets equal access within defined limits
• Protection from abuse - Prevents malicious or buggy clients from causing harm
• Backend services protected - Services remain available even under high load
• Predictable costs - Resource usage stays within expected bounds

---

Real-World Examples

Rate limiting is used everywhere in production systems. Here are examples from popular services:

GitHub API

Rate Limits:

• Authenticated requests: 5,000 requests/hour
• Unauthenticated requests: 60 requests/hour
• Search API: 30 requests/minute

Implementation: GitHub uses a token bucket algorithm with different limits for authenticated vs unauthenticated users. When you exceed the limit, you receive a 403 Forbidden response with headers indicating when the limit resets:

HTTP/1.1 403 Forbidden
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1609459200
X-RateLimit-Used: 5000

Why these limits? GitHub needs to protect their infrastructure while allowing legitimate developers to build applications. The higher limit for authenticated users encourages API key usage, which helps GitHub identify and manage traffic better.

Twitter API (X)

Rate Limits (v2 API):

• Tweet lookup: 300 requests/15 minutes per user
• User lookup: 300 requests/15 minutes per user
• Search: 180 requests/15 minutes per user
• Post tweet: 300 requests/15 minutes per user

Implementation: Twitter uses a sliding window algorithm. Different endpoints have different limits based on resource cost. More expensive operations (like search) have lower limits.

Real scenario: A social media analytics tool needs to fetch tweets for 1,000 users. With a 300 requests/15 minutes limit, it would take at least 50 minutes to complete, requiring careful request scheduling and rate limit tracking.

AWS API Gateway

Rate Limits:

• Default: 10,000 requests/second per region
• Burst: 5,000 requests/second
• Per-key: Configurable (e.g., 100 requests/second per API key)

Implementation: AWS uses a token bucket with burst capacity. The burst allows short spikes above the steady-state rate, which is perfect for handling traffic patterns with occasional peaks.

Use case: An e-commerce site during Black Friday. Normal traffic is 1,000 requests/second, but during flash sales, traffic spikes to 5,000 requests/second for 30 seconds. The burst capacity handles these spikes without rejecting requests.

Stripe API

Rate Limits:

• Test mode: 25 requests/second
• Live mode: 100 requests/second per API key
• Idempotency: Separate limits for idempotent requests

Implementation: Stripe uses a combination of token bucket and sliding window. They also implement idempotency keys to prevent duplicate charges, which have separate rate limits.

Critical use case: Payment processing. Stripe must prevent both abuse and accidental duplicate charges. Rate limiting protects their infrastructure while idempotency keys protect customers from double-charging.

Google Maps API

Rate Limits:

• Geocoding: 40 requests/second
• Directions: 40 requests/second
• Places: 100 requests/second
• Quota: 40,000 requests/day (free tier)

Implementation: Google uses both per-second rate limits and daily quotas. This dual approach prevents both short-term abuse and long-term overuse.

Example: A delivery app needs to geocode addresses. With 40 requests/second, it can process 2,400 addresses per minute. For a delivery service handling 10,000 orders/day, this requires careful batching and caching of geocoded addresses.

Reddit API

Rate Limits:

• OAuth requests: 60 requests/minute
• Unauthenticated: 30 requests/minute
• Per-user: Tracks usage per OAuth token

Implementation: Reddit uses a fixed window algorithm with per-user tracking. This prevents individual users from overwhelming the API while allowing fair distribution across all users.

Real-world impact: A Reddit bot that posts comments needs to respect the 60 requests/minute limit. Posting too quickly results in temporary bans, requiring exponential backoff and retry logic.

Cloudflare

Rate Limiting Rules:

• Custom rules: Configurable (e.g., 100 requests/minute per IP)
• Challenge pages: CAPTCHA after limit exceeded
• Bypass rules: For trusted IPs or authenticated users

Implementation: Cloudflare uses distributed rate limiting across their global network. Rules are evaluated at edge locations, providing protection before traffic reaches origin servers.

DDoS protection: During a DDoS attack, Cloudflare’s rate limiting automatically blocks excessive requests from individual IPs while allowing legitimate traffic through. This protects origin servers from being overwhelmed.

Netflix API (Historical)

Rate Limits (before shutdown):

• Per application: 5,000 requests/hour
• Per user: 20 requests/second

Why it mattered: Netflix’s API was used by third-party applications to access movie metadata. Rate limiting prevented abuse while allowing legitimate developers to build applications. The API was eventually shut down in favor of direct partnerships, but rate limiting was crucial during its operation.

---

Key Takeaways from Real Examples

1. Different limits for different use cases - Authenticated users get higher limits than anonymous users
2. Burst capacity matters - Token bucket allows handling traffic spikes
3. Per-user vs per-IP - Most services track by user/API key, not just IP
4. Headers communicate limits - Standard headers like X-RateLimit-* help clients manage usage
5. Tiered limits - Free vs paid tiers have different limits
6. Endpoint-specific limits - Expensive operations have lower limits

---

Rate Limiting Algorithms

1. Token Bucket (Most Popular)

Bucket holds tokens. Tokens refill at fixed rate. Request consumes token.

Characteristics:

• Allows bursts (up to bucket capacity) - Can handle sudden spikes in traffic up to bucket size
• Smooths to average rate over time - Over long periods, rate averages to refill rate
• Simple to implement - Straightforward algorithm with clear logic
• Most popular algorithm - Widely used and well-understood

Algorithm:

1. Initialize bucket with capacity tokens
2. Refill tokens at refill_rate per second
3. When request arrives:
   • If tokens available → consume token, allow request
   • If no tokens → reject request (429)

Token Bucket Implementation

💡 Tip: Click dropdown to switch between languages

Python

Python

"token_bucket.py

import time
from threading import Lock
from typing import Optional

class TokenBucket:
    """Token bucket rate limiter"""

    def __init__(self, capacity: int, refill_rate: float):
        """
        Args:
            capacity: Maximum tokens in bucket
            refill_rate: Tokens added per second
        """
        self.capacity = capacity
        self.refill_rate = refill_rate
        self.tokens = capacity
        self.last_refill = time.time()
        self.lock = Lock()

    def acquire(self, tokens: int = 1) -> bool:
        """Try to acquire tokens. Returns True if successful."""
        with self.lock:
            self._refill()

            if self.tokens >= tokens:
                self.tokens -= tokens
                return True
            else:
                return False

    def _refill(self):
        """Refill tokens based on elapsed time"""
        now = time.time()
        elapsed = now - self.last_refill

        # Calculate tokens to add
        tokens_to_add = elapsed * self.refill_rate

        # Add tokens (don't exceed capacity)
        self.tokens = min(self.capacity, self.tokens + tokens_to_add)
        self.last_refill = now

    def get_available_tokens(self) -> int:
        """Get current number of available tokens"""
        with self.lock:
            self._refill()
            return int(self.tokens)

# Usage
rate_limiter = TokenBucket(capacity=10, refill_rate=2.0)  # 10 tokens, refill 2/sec

def handle_request():
    if rate_limiter.acquire():
        # Process request
        return "Request processed"
    else:
        return "Rate limit exceeded", 429

TypeScript

"token-bucket.ts

class TokenBucket {
    private capacity: number;
    private refillRate: number;
    private tokens: number;
    private lastRefill: number;
    private lock: boolean = false;

    constructor(capacity: number, refillRate: number) {
        this.capacity = capacity;
        this.refillRate = refillRate;
        this.tokens = capacity;
        this.lastRefill = Date.now();
    }

    acquire(tokensToConsume: number = 1): boolean {
        while (this.lock) {
            // Wait for lock
        }
        this.lock = true;

        this.refill();

        if (this.tokens >= tokensToConsume) {
            this.tokens -= tokensToConsume;
            this.lock = false;
            return true;
        } else {
            this.lock = false;
            return false;
        }
    }

    private refill(): void {
        const now = Date.now();
        const elapsed = (now - this.lastRefill) / 1000; // Convert to seconds

        const tokensToAdd = elapsed * this.refillRate;
        this.tokens = Math.min(this.capacity, this.tokens + tokensToAdd);
        this.lastRefill = now;
    }

    getAvailableTokens(): number {
        return Math.floor(this.tokens);
    }
}

// Usage
const rateLimiter = new TokenBucket(10, 2.0); // 10 tokens, refill 2/sec

function handleRequest(): string | [string, number] {
    if (rateLimiter.acquire()) {
        return "Request processed";
    } else {
        return ["Rate limit exceeded", 429];
    }
}

C++

"token_bucket.cpp

#include <chrono>
#include <mutex>
#include <algorithm>

class TokenBucket {
private:
    int capacity;
    double refillRate;
    double tokens;
    std::chrono::steady_clock::time_point lastRefill;
    std::mutex mutex;

public:
    TokenBucket(int capacity, double refillRate)
        : capacity(capacity), refillRate(refillRate), tokens(capacity),
          lastRefill(std::chrono::steady_clock::now()) {}

    bool acquire(int tokensToConsume = 1) {
        std::lock_guard<std::mutex> lock(mutex);
        refill();

        if (tokens >= tokensToConsume) {
            tokens -= tokensToConsume;
            return true;
        } else {
            return false;
        }
    }

private:
    void refill() {
        auto now = std::chrono::steady_clock::now();
        auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
            now - lastRefill
        ).count() / 1000.0;

        double tokensToAdd = elapsed * refillRate;
        tokens = std::min(static_cast<double>(capacity), tokens + tokensToAdd);
        lastRefill = now;
    }

public:
    int getAvailableTokens() {
        std::lock_guard<std::mutex> lock(mutex);
        refill();
        return static_cast<int>(tokens);
    }
};

// Usage
TokenBucket rateLimiter(10, 2.0); // 10 tokens, refill 2/sec

std::string handleRequest() {
    if (rateLimiter.acquire()) {
        return "Request processed";
    } else {
        return "Rate limit exceeded"; // Return 429 status
    }
}

C#

"TokenBucket.cs

using System;
using System.Threading;

public class TokenBucket {
    private readonly int capacity;
    private readonly double refillRate;
    private double tokens;
    private DateTime lastRefill;
    private readonly object lockObject = new object();

    public TokenBucket(int capacity, double refillRate) {
        this.capacity = capacity;
        this.refillRate = refillRate;
        this.tokens = capacity;
        this.lastRefill = DateTime.UtcNow;
    }

    public bool Acquire(int tokensToConsume = 1) {
        lock (lockObject) {
            Refill();

            if (tokens >= tokensToConsume) {
                tokens -= tokensToConsume;
                return true;
            } else {
                return false;
            }
        }
    }

    private void Refill() {
        var now = DateTime.UtcNow;
        var elapsed = (now - lastRefill).TotalSeconds;

        var tokensToAdd = elapsed * refillRate;
        tokens = Math.Min(capacity, tokens + tokensToAdd);
        lastRefill = now;
    }

    public int GetAvailableTokens() {
        lock (lockObject) {
            Refill();
            return (int)tokens;
        }
    }
}

// Usage
var rateLimiter = new TokenBucket(10, 2.0); // 10 tokens, refill 2/sec

(string response, int statusCode) HandleRequest() {
    if (rateLimiter.Acquire()) {
        return ("Request processed", 200);
    } else {
        return ("Rate limit exceeded", 429);
    }
}

Java

"TokenBucket.java

import java.util.concurrent.locks.ReentrantLock;

public class TokenBucket {
    private final int capacity;
    private final double refillRate;
    private double tokens;
    private long lastRefill;
    private final ReentrantLock lock = new ReentrantLock();

    public TokenBucket(int capacity, double refillRate) {
        this.capacity = capacity;
        this.refillRate = refillRate;
        this.tokens = capacity;
        this.lastRefill = System.currentTimeMillis();
    }

    public boolean acquire(int tokensToConsume) {
        lock.lock();
        try {
            refill();

            if (tokens >= tokensToConsume) {
                tokens -= tokensToConsume;
                return true;
            } else {
                return false;
            }
        } finally {
            lock.unlock();
        }
    }

    private void refill() {
        long now = System.currentTimeMillis();
        double elapsed = (now - lastRefill) / 1000.0;  // Convert to seconds

        // Calculate tokens to add
        double tokensToAdd = elapsed * refillRate;

        // Add tokens (don't exceed capacity)
        tokens = Math.min(capacity, tokens + tokensToAdd);
        lastRefill = now;
    }

    public int getAvailableTokens() {
        lock.lock();
        try {
            refill();
            return (int) tokens;
        } finally {
            lock.unlock();
        }
    }
}

// Usage
TokenBucket rateLimiter = new TokenBucket(10, 2.0);  // 10 tokens, refill 2/sec

public String handleRequest() {
    if (rateLimiter.acquire(1)) {
        // Process request
        return "Request processed";
    } else {
        return "Rate limit exceeded";  // Return 429
    }
}

2. Leaky Bucket

Bucket holds requests. Requests leak out at fixed rate. If full, reject.

Characteristics:

• Smooths traffic to constant rate - Requests are processed at a steady, predictable rate
• No bursts allowed - Cannot handle sudden spikes in traffic like token bucket
• Predictable output rate - Output rate is constant, making it easier to plan capacity
• Less flexible than token bucket - Cannot accommodate bursts, which may be needed for some use cases

Algorithm:

1. Initialize bucket with capacity (queue size)
2. Process requests at leak_rate per second
3. When request arrives:
   • If bucket has space → add to bucket
   • If bucket full → reject (429)

Leaky Bucket Implementation

💡 Tip: Click dropdown to switch between languages

Python

Python

"leaky_bucket.py

import time
import queue
from threading import Lock, Thread
from typing import Optional

class LeakyBucket:
    """Leaky bucket rate limiter"""

    def __init__(self, capacity: int, leak_rate: float):
        """
        Args:
            capacity: Maximum requests in bucket
            leak_rate: Requests processed per second
        """
        self.capacity = capacity
        self.leak_rate = leak_rate
        self.bucket = queue.Queue(maxsize=capacity)
        self.lock = Lock()
        self.processing = False

    def start_processing(self):
        """Start processing requests"""
        if not self.processing:
            self.processing = True
            Thread(target=self._process_requests, daemon=True).start()

    def add_request(self, request) -> bool:
        """Try to add request to bucket. Returns True if successful."""
        try:
            self.bucket.put_nowait(request)
            return True
        except queue.Full:
            return False

    def _process_requests(self):
        """Process requests at leak rate"""
        interval = 1.0 / self.leak_rate  # Time between requests

        while self.processing:
            try:
                request = self.bucket.get(timeout=interval)
                # Process request
                self._handle_request(request)
            except queue.Empty:
                continue

    def _handle_request(self, request):
        """Handle processed request"""
        # Override in subclass or pass handler
        pass

TypeScript

"leaky-bucket.ts

import { EventEmitter } from 'events';

class LeakyBucket extends EventEmitter {
    private capacity: number;
    private leakRate: number;
    private bucket: any[] = [];
    private processing: boolean = false;
    private interval: NodeJS.Timeout | null = null;

    constructor(capacity: number, leakRate: number) {
        super();
        this.capacity = capacity;
        this.leakRate = leakRate;
    }

    startProcessing(): void {
        if (!this.processing) {
            this.processing = true;
            const intervalMs = (1.0 / this.leakRate) * 1000;
            this.interval = setInterval(() => {
                this.processRequest();
            }, intervalMs);
        }
    }

    addRequest(request: any): boolean {
        if (this.bucket.length < this.capacity) {
            this.bucket.push(request);
            return true;
        } else {
            return false;
        }
    }

    private processRequest(): void {
        if (this.bucket.length > 0) {
            const request = this.bucket.shift();
            this.handleRequest(request);
        }
    }

    private handleRequest(request: any): void {
        this.emit('request', request);
    }

    stop(): void {
        this.processing = false;
        if (this.interval) {
            clearInterval(this.interval);
        }
    }
}

// Usage
const bucket = new LeakyBucket(10, 2.0);
bucket.startProcessing();

function handleRequest(): boolean {
    return bucket.addRequest({ id: Date.now() });
}

C++

"leaky_bucket.cpp

#include <queue>
#include <thread>
#include <chrono>
#include <mutex>

class LeakyBucket {
private:
    int capacity;
    double leakRate;
    std::queue<void*> bucket;
    std::mutex mutex;
    bool processing;
    std::thread processorThread;

    void processRequests() {
        auto interval = std::chrono::milliseconds(
            static_cast<int>((1.0 / leakRate) * 1000)
        );

        while (processing) {
            std::this_thread::sleep_for(interval);

            std::lock_guard<std::mutex> lock(mutex);
            if (!bucket.empty()) {
                void* request = bucket.front();
                bucket.pop();
                handleRequest(request);
            }
        }
    }

    virtual void handleRequest(void* request) {}

public:
    LeakyBucket(int capacity, double leakRate)
        : capacity(capacity), leakRate(leakRate), processing(false) {}

    bool addRequest(void* request) {
        std::lock_guard<std::mutex> lock(mutex);
        if (bucket.size() < capacity) {
            bucket.push(request);
            return true;
        } else {
            return false;
        }
    }

    void start() {
        if (!processing) {
            processing = true;
            processorThread = std::thread(&LeakyBucket::processRequests, this);
        }
    }

    void stop() {
        processing = false;
        if (processorThread.joinable()) {
            processorThread.join();
        }
    }

    ~LeakyBucket() {
        stop();
    }
};

C#

"LeakyBucket.cs

using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

public class LeakyBucket {
    private readonly int capacity;
    private readonly double leakRate;
    private readonly Queue<object> bucket;
    private readonly object lockObject = new object();
    private bool processing;
    private CancellationTokenSource cancellationTokenSource;

    public LeakyBucket(int capacity, double leakRate) {
        this.capacity = capacity;
        this.leakRate = leakRate;
        this.bucket = new Queue<object>();
    }

    public void StartProcessing() {
        if (!processing) {
            processing = true;
            cancellationTokenSource = new CancellationTokenSource();
            Task.Run(() => ProcessRequests(cancellationTokenSource.Token));
        }
    }

    public bool AddRequest(object request) {
        lock (lockObject) {
            if (bucket.Count < capacity) {
                bucket.Enqueue(request);
                return true;
            } else {
                return false;
            }
        }
    }

    private async Task ProcessRequests(CancellationToken cancellationToken) {
        var interval = TimeSpan.FromSeconds(1.0 / leakRate);

        while (!cancellationToken.IsCancellationRequested) {
            await Task.Delay(interval, cancellationToken);

            lock (lockObject) {
                if (bucket.Count > 0) {
                    var request = bucket.Dequeue();
                    HandleRequest(request);
                }
            }
        }
    }

    protected virtual void HandleRequest(object request) {}

    public void Stop() {
        processing = false;
        cancellationTokenSource?.Cancel();
    }
}

// Usage
var bucket = new LeakyBucket(10, 2.0);
bucket.StartProcessing();

bool HandleRequest() {
    return bucket.AddRequest(new { Id = DateTime.UtcNow.Ticks });
}

Java

"LeakyBucket.java

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

public class LeakyBucket {
    private final int capacity;
    private final double leakRate;
    private final BlockingQueue<Request> bucket;
    private final ScheduledExecutorService scheduler;

    public LeakyBucket(int capacity, double leakRate) {
        this.capacity = capacity;
        this.leakRate = leakRate;
        this.bucket = new LinkedBlockingQueue<>(capacity);
        this.scheduler = Executors.newScheduledThreadPool(1);

        // Start processing
        startProcessing();
    }

    public boolean addRequest(Request request) {
        // Try to add request to bucket
        return bucket.offer(request);
    }

    private void startProcessing() {
        long interval = (long) (1000 / leakRate);  // Milliseconds between requests

        scheduler.scheduleAtFixedRate(() -> {
            Request request = bucket.poll();
            if (request != null) {
                handleRequest(request);
            }
        }, 0, interval, TimeUnit.MILLISECONDS);
    }

    private void handleRequest(Request request) {
        // Process request
    }
}

3. Sliding Window

Tracks requests in sliding time window. More accurate than fixed window.

Characteristics:

• More accurate than fixed window - Provides more precise rate limiting without boundary bursts
• No burst at boundaries - Eliminates the burst problem that occurs in fixed window at window boundaries
• More memory intensive (stores timestamps) - Requires storing timestamps for each request, increasing memory usage
• More complex to implement - Requires more sophisticated logic to track sliding windows

Sliding Window Implementation

💡 Tip: Click dropdown to switch between languages

Python

Python

"sliding_window.py

import time
from collections import deque
from threading import Lock

class SlidingWindowRateLimiter:
    """Sliding window rate limiter"""

    def __init__(self, limit: int, window_seconds: int):
        """
        Args:
            limit: Maximum requests allowed
            window_seconds: Time window in seconds
        """
        self.limit = limit
        self.window_seconds = window_seconds
        self.requests = deque()  # Store request timestamps
        self.lock = Lock()

    def is_allowed(self) -> bool:
        """Check if request is allowed"""
        with self.lock:
            now = time.time()

            # Remove old requests outside window
            while self.requests and self.requests[0] < now - self.window_seconds:
                self.requests.popleft()

            # Check if under limit
            if len(self.requests) < self.limit:
                self.requests.append(now)
                return True
            else:
                return False

    def get_remaining_requests(self) -> int:
        """Get remaining requests in current window"""
        with self.lock:
            now = time.time()

            # Remove old requests
            while self.requests and self.requests[0] < now - self.window_seconds:
                self.requests.popleft()

            return max(0, self.limit - len(self.requests))

Java

"SlidingWindowRateLimiter.java

import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.concurrent.locks.ReentrantLock;

public class SlidingWindowRateLimiter {
    private final int limit;
    private final long windowMillis;
    private final ConcurrentLinkedDeque<Long> requests;
    private final ReentrantLock lock = new ReentrantLock();

    public SlidingWindowRateLimiter(int limit, int windowSeconds) {
        this.limit = limit;
        this.windowMillis = windowSeconds * 1000L;
        this.requests = new ConcurrentLinkedDeque<>();
    }

    public boolean isAllowed() {
        lock.lock();
        try {
            long now = System.currentTimeMillis();

            // Remove old requests outside window
            while (!requests.isEmpty() && requests.peekFirst() < now - windowMillis) {
                requests.pollFirst();
            }

            // Check if under limit
            if (requests.size() < limit) {
                requests.addLast(now);
                return true;
            } else {
                return false;
            }
        } finally {
            lock.unlock();
        }
    }

    public int getRemainingRequests() {
        lock.lock();
        try {
            long now = System.currentTimeMillis();

            // Remove old requests
            while (!requests.isEmpty() && requests.peekFirst() < now - windowMillis) {
                requests.pollFirst();
            }

            return Math.max(0, limit - requests.size());
        } finally {
            lock.unlock();
        }
    }
}

TypeScript

"sliding-window.ts

class SlidingWindowRateLimiter {
    private limit: number;
    private windowSeconds: number;
    private requests: number[] = [];
    private lock: boolean = false;

    constructor(limit: number, windowSeconds: number) {
        this.limit = limit;
        this.windowSeconds = windowSeconds;
    }

    isAllowed(): boolean {
        while (this.lock) {
            // Wait for lock
        }
        this.lock = true;

        const now = Date.now() / 1000;

        // Remove old requests outside window
        this.requests = this.requests.filter(
            timestamp => timestamp > now - this.windowSeconds
        );

        // Check if under limit
        if (this.requests.length < this.limit) {
            this.requests.push(now);
            this.lock = false;
            return true;
        } else {
            this.lock = false;
            return false;
        }
    }

    getRemainingRequests(): number {
        const now = Date.now() / 1000;
        this.requests = this.requests.filter(
            timestamp => timestamp > now - this.windowSeconds
        );
        return Math.max(0, this.limit - this.requests.length);
    }
}

// Usage
const limiter = new SlidingWindowRateLimiter(100, 60); // 100 requests per 60 seconds

function handleRequest(): boolean {
    return limiter.isAllowed();
}

C++

"sliding_window.cpp

#include <deque>
#include <chrono>
#include <mutex>
#include <algorithm>

class SlidingWindowRateLimiter {
private:
    int limit;
    int windowSeconds;
    std::deque<std::chrono::steady_clock::time_point> requests;
    std::mutex mutex;

public:
    SlidingWindowRateLimiter(int limit, int windowSeconds)
        : limit(limit), windowSeconds(windowSeconds) {}

    bool isAllowed() {
        std::lock_guard<std::mutex> lock(mutex);
        auto now = std::chrono::steady_clock::now();
        auto windowStart = now - std::chrono::seconds(windowSeconds);

        // Remove old requests outside window
        while (!requests.empty() && requests.front() < windowStart) {
            requests.pop_front();
        }

        // Check if under limit
        if (requests.size() < limit) {
            requests.push_back(now);
            return true;
        } else {
            return false;
        }
    }

    int getRemainingRequests() {
        std::lock_guard<std::mutex> lock(mutex);
        auto now = std::chrono::steady_clock::now();
        auto windowStart = now - std::chrono::seconds(windowSeconds);

        while (!requests.empty() && requests.front() < windowStart) {
            requests.pop_front();
        }

        return std::max(0, limit - static_cast<int>(requests.size()));
    }
};

C#

"SlidingWindowRateLimiter.cs

using System;
using System.Collections.Generic;
using System.Linq;

public class SlidingWindowRateLimiter {
    private readonly int limit;
    private readonly int windowSeconds;
    private readonly List<DateTime> requests;
    private readonly object lockObject = new object();

    public SlidingWindowRateLimiter(int limit, int windowSeconds) {
        this.limit = limit;
        this.windowSeconds = windowSeconds;
        this.requests = new List<DateTime>();
    }

    public bool IsAllowed() {
        lock (lockObject) {
            var now = DateTime.UtcNow;
            var windowStart = now.AddSeconds(-windowSeconds);

            // Remove old requests outside window
            requests.RemoveAll(timestamp => timestamp < windowStart);

            // Check if under limit
            if (requests.Count < limit) {
                requests.Add(now);
                return true;
            } else {
                return false;
            }
        }
    }

    public int GetRemainingRequests() {
        lock (lockObject) {
            var now = DateTime.UtcNow;
            var windowStart = now.AddSeconds(-windowSeconds);

            requests.RemoveAll(timestamp => timestamp < windowStart);
            return Math.Max(0, limit - requests.Count);
        }
    }
}

// Usage
var limiter = new SlidingWindowRateLimiter(100, 60); // 100 requests per 60 seconds

bool HandleRequest() {
    return limiter.IsAllowed();
}

4. Fixed Window

Divides time into fixed windows. Simple but allows bursts.

Characteristics:

• Simple to implement - Straightforward logic with minimal complexity
• Low memory usage - Only needs to store a counter per window, very efficient
• Allows bursts at boundaries - Can allow double the limit when windows reset (e.g., 100 requests at 10:00:59 and 10:01:00)
• Less accurate - Less precise than sliding window due to boundary burst issue

Fixed Window Implementation

💡 Tip: Click dropdown to switch between languages

Python

Python

"fixed_window.py

import time
from threading import Lock

class FixedWindowRateLimiter:
    """Fixed window rate limiter"""

    def __init__(self, limit: int, window_seconds: int):
        """
        Args:
            limit: Maximum requests per window
            window_seconds: Window size in seconds
        """
        self.limit = limit
        self.window_seconds = window_seconds
        self.count = 0
        self.window_start = time.time()
        self.lock = Lock()

    def is_allowed(self) -> bool:
        """Check if request is allowed"""
        with self.lock:
            now = time.time()

            # Check if window expired
            if now - self.window_start >= self.window_seconds:
                # Reset window
                self.count = 0
                self.window_start = now

            # Check limit
            if self.count < self.limit:
                self.count += 1
                return True
            else:
                return False

Java

"FixedWindowRateLimiter.java

import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.locks.ReentrantLock;

public class FixedWindowRateLimiter {
    private final int limit;
    private final long windowMillis;
    private final AtomicInteger count = new AtomicInteger(0);
    private final AtomicLong windowStart = new AtomicLong(System.currentTimeMillis());
    private final ReentrantLock lock = new ReentrantLock();

    public FixedWindowRateLimiter(int limit, int windowSeconds) {
        this.limit = limit;
        this.windowMillis = windowSeconds * 1000L;
    }

    public boolean isAllowed() {
        lock.lock();
        try {
            long now = System.currentTimeMillis();

            // Check if window expired
            if (now - windowStart.get() >= windowMillis) {
                // Reset window
                count.set(0);
                windowStart.set(now);
            }

            // Check limit
            if (count.get() < limit) {
                count.incrementAndGet();
                return true;
            } else {
                return false;
            }
        } finally {
            lock.unlock();
        }
    }
}

---

Distributed Rate Limiting

For multiple servers, use Redis:

💡 Tip: Click dropdown to switch between languages

Python

Python

"distributed_rate_limiter.py

import redis
import time

class DistributedTokenBucket:
    """Distributed token bucket using Redis"""

    def __init__(self, redis_client: redis.Redis, key_prefix: str,
                 capacity: int, refill_rate: float):
        self.redis = redis_client
        self.key_prefix = key_prefix
        self.capacity = capacity
        self.refill_rate = refill_rate

    def is_allowed(self, identifier: str) -> bool:
        """Check if request is allowed for identifier"""
        key = f"{self.key_prefix}:{identifier}"
        now = time.time()

        # Use Lua script for atomic operations
        lua_script = """
        local key = KEYS[1]
        local capacity = tonumber(ARGV[1])
        local refill_rate = tonumber(ARGV[2])
        local now = tonumber(ARGV[3])

        local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
        local tokens = tonumber(bucket[1]) or capacity
        local last_refill = tonumber(bucket[2]) or now

        -- Refill tokens
        local elapsed = now - last_refill
        local tokens_to_add = elapsed * refill_rate
        tokens = math.min(capacity, tokens + tokens_to_add)

        -- Check if can consume token
        if tokens >= 1 then
            tokens = tokens - 1
            redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
            redis.call('EXPIRE', key, 3600)  -- Expire after 1 hour
            return 1
        else
            redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
            redis.call('EXPIRE', key, 3600)
            return 0
        end
        """

        result = self.redis.eval(lua_script, 1, key,
                                 self.capacity, self.refill_rate, now)
        return bool(result)

Java

"DistributedRateLimiter.java

import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;

public class DistributedTokenBucket {
    private final JedisPool jedisPool;
    private final String keyPrefix;
    private final int capacity;
    private final double refillRate;

    private static final String LUA_SCRIPT =
        "local key = KEYS[1]\n" +
        "local capacity = tonumber(ARGV[1])\n" +
        "local refill_rate = tonumber(ARGV[2])\n" +
        "local now = tonumber(ARGV[3])\n" +
        "local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')\n" +
        "local tokens = tonumber(bucket[1]) or capacity\n" +
        "local last_refill = tonumber(bucket[2]) or now\n" +
        "local elapsed = now - last_refill\n" +
        "local tokens_to_add = elapsed * refill_rate\n" +
        "tokens = math.min(capacity, tokens + tokens_to_add)\n" +
        "if tokens >= 1 then\n" +
        "  tokens = tokens - 1\n" +
        "  redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)\n" +
        "  redis.call('EXPIRE', key, 3600)\n" +
        "  return 1\n" +
        "else\n" +
        "  redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)\n" +
        "  redis.call('EXPIRE', key, 3600)\n" +
        "  return 0\n" +
        "end";

    public boolean isAllowed(String identifier) {
        try (Jedis jedis = jedisPool.getResource()) {
            String key = keyPrefix + ":" + identifier;
            long now = System.currentTimeMillis() / 1000;

            Object result = jedis.eval(LUA_SCRIPT,
                Collections.singletonList(key),
                Arrays.asList(
                    String.valueOf(capacity),
                    String.valueOf(refillRate),
                    String.valueOf(now)
                ));

            return ((Long) result) == 1L;
        }
    }
}

TypeScript

"distributed-rate-limiter.ts

import Redis from 'ioredis';

class DistributedTokenBucket {
    private redis: Redis;
    private keyPrefix: string;
    private capacity: number;
    private refillRate: number;

    constructor(
        redis: Redis,
        keyPrefix: string,
        capacity: number,
        refillRate: number
    ) {
        this.redis = redis;
        this.keyPrefix = keyPrefix;
        this.capacity = capacity;
        this.refillRate = refillRate;
    }

    async isAllowed(identifier: string): Promise<boolean> {
        const key = `${this.keyPrefix}:${identifier}`;
        const now = Date.now() / 1000;

        // Use Lua script for atomic operations
        const luaScript = `
            local key = KEYS[1]
            local capacity = tonumber(ARGV[1])
            local refill_rate = tonumber(ARGV[2])
            local now = tonumber(ARGV[3])

            local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
            local tokens = tonumber(bucket[1]) or capacity
            local last_refill = tonumber(bucket[2]) or now

            -- Refill tokens
            local elapsed = now - last_refill
            local tokens_to_add = elapsed * refill_rate
            tokens = math.min(capacity, tokens + tokens_to_add)

            -- Check if can consume token
            if tokens >= 1 then
                tokens = tokens - 1
                redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
                redis.call('EXPIRE', key, 3600)
                return 1
            else
                redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
                redis.call('EXPIRE', key, 3600)
                return 0
            end
        `;

        const result = await this.redis.eval(
            luaScript,
            1,
            key,
            this.capacity.toString(),
            this.refillRate.toString(),
            now.toString()
        );

        return result === 1;
    }
}

// Usage
import Redis from 'ioredis';
const redis = new Redis();
const limiter = new DistributedTokenBucket(redis, 'rate_limit', 10, 2.0);

async function handleRequest(identifier: string): Promise<boolean> {
    return await limiter.isAllowed(identifier);
}

C++

"distributed_rate_limiter.cpp

#include <hiredis/hiredis.h>
#include <string>
#include <sstream>

class DistributedTokenBucket {
private:
    redisContext* redis;
    std::string keyPrefix;
    int capacity;
    double refillRate;

    std::string getLuaScript() {
        return R"(
            local key = KEYS[1]
            local capacity = tonumber(ARGV[1])
            local refill_rate = tonumber(ARGV[2])
            local now = tonumber(ARGV[3])

            local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
            local tokens = tonumber(bucket[1]) or capacity
            local last_refill = tonumber(bucket[2]) or now

            local elapsed = now - last_refill
            local tokens_to_add = elapsed * refill_rate
            tokens = math.min(capacity, tokens + tokens_to_add)

            if tokens >= 1 then
                tokens = tokens - 1
                redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
                redis.call('EXPIRE', key, 3600)
                return 1
            else
                redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
                redis.call('EXPIRE', key, 3600)
                return 0
            end
        )";
    }

public:
    DistributedTokenBucket(
        redisContext* redis,
        const std::string& keyPrefix,
        int capacity,
        double refillRate
    ) : redis(redis), keyPrefix(keyPrefix), capacity(capacity), refillRate(refillRate) {}

    bool isAllowed(const std::string& identifier) {
        std::string key = keyPrefix + ":" + identifier;
        auto now = std::chrono::system_clock::now();
        auto seconds = std::chrono::duration_cast<std::chrono::seconds>(
            now.time_since_epoch()
        ).count();

        std::stringstream ss;
        ss << capacity << " " << refillRate << " " << seconds;
        std::string args = ss.str();

        redisReply* reply = (redisReply*)redisCommand(
            redis,
            "EVAL %s 1 %s %s",
            getLuaScript().c_str(),
            key.c_str(),
            args.c_str()
        );

        bool result = false;
        if (reply && reply->type == REDIS_REPLY_INTEGER) {
            result = reply->integer == 1;
        }

        freeReplyObject(reply);
        return result;
    }
};

C#

"DistributedRateLimiter.cs

using StackExchange.Redis;
using System;

public class DistributedTokenBucket {
    private readonly IDatabase redis;
    private readonly string keyPrefix;
    private readonly int capacity;
    private readonly double refillRate;

    private const string LuaScript = @"
        local key = KEYS[1]
        local capacity = tonumber(ARGV[1])
        local refill_rate = tonumber(ARGV[2])
        local now = tonumber(ARGV[3])

        local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
        local tokens = tonumber(bucket[1]) or capacity
        local last_refill = tonumber(bucket[2]) or now

        local elapsed = now - last_refill
        local tokens_to_add = elapsed * refill_rate
        tokens = math.min(capacity, tokens + tokens_to_add)

        if tokens >= 1 then
            tokens = tokens - 1
            redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
            redis.call('EXPIRE', key, 3600)
            return 1
        else
            redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
            redis.call('EXPIRE', key, 3600)
            return 0
        end
    ";

    public DistributedTokenBucket(
        IDatabase redis,
        string keyPrefix,
        int capacity,
        double refillRate
    ) {
        this.redis = redis;
        this.keyPrefix = keyPrefix;
        this.capacity = capacity;
        this.refillRate = refillRate;
    }

    public async Task<bool> IsAllowedAsync(string identifier) {
        var key = $"{keyPrefix}:{identifier}";
        var now = DateTimeOffset.UtcNow.ToUnixTimeSeconds();

        var result = await redis.ScriptEvaluateAsync(
            LuaScript,
            new RedisKey[] { key },
            new RedisValue[] { capacity, refillRate, now }
        );

        return (int)result == 1;
    }
}

// Usage
var redis = ConnectionMultiplexer.Connect("localhost").GetDatabase();
var limiter = new DistributedTokenBucket(redis, "rate_limit", 10, 2.0);

async Task<bool> HandleRequestAsync(string identifier) {
    return await limiter.IsAllowedAsync(identifier);
}

---

Rate Limiting Strategies

Rate limiting can be applied at different levels depending on your use case. Here are common strategies with real-world examples:

By Client IP

Use case: Public APIs where you don’t have user authentication, or as a first line of defense.

Example: A public weather API limits each IP to 100 requests/hour. This prevents a single user from scraping all weather data while allowing legitimate usage.

rate_limiter = TokenBucket(capacity=100, refill_rate=10)
client_ip = request.remote_addr

if not rate_limiter.is_allowed(client_ip):
    return "Rate limit exceeded", 429

By User ID

Use case: Authenticated APIs where you want to limit per-user usage, regardless of which device or IP they use.

Example: A social media API limits each authenticated user to 1,000 posts/day. This prevents spam while allowing legitimate users to post from multiple devices (phone, tablet, desktop).

Real-world scenario: A user tries to post 1,500 times in one day. After 1,000 posts, all subsequent requests return 429 until the next day. This protects the platform from spam while being fair to legitimate users.

By API Key

Use case: Third-party integrations where each application gets its own API key with specific limits.

Example: A payment processing API provides each merchant with an API key. Free tier merchants get 1,000 transactions/month, while enterprise merchants get 100,000 transactions/month.

Real-world scenario: An e-commerce platform integrates with a payment API. They receive an API key with a 10,000 requests/day limit. During peak shopping season, they might hit this limit and need to upgrade their plan or implement request queuing.

Tiered Limits

Use case: SaaS products with multiple pricing tiers. Each tier gets different rate limits as part of the subscription.

Example: A cloud storage API offers three tiers:

• Free: 100 API calls/hour
• Pro: 1,000 API calls/hour
• Enterprise: 10,000 API calls/hour

Real-world scenario: A file backup application uses the API. Free users can sync files 100 times per hour, which is sufficient for personal use. Pro users (developers) get 1,000 calls/hour, enough for automated backups. Enterprise customers get 10,000 calls/hour for large-scale operations.

Business value: Tiered limits encourage upgrades. Users hitting free tier limits often upgrade to Pro, increasing revenue.

By Endpoint/Resource

Use case: Different API endpoints have different costs. Expensive operations get lower limits.

Example: A machine learning API:

• Image classification: 100 requests/minute (cheap, fast)
• Video processing: 10 requests/minute (expensive, slow)
• Model training: 1 request/hour (very expensive, takes time)

Real-world scenario: A video editing app uses the ML API. Users can classify images quickly (100/min), but video processing is limited to 10/min to prevent resource exhaustion. Model training requests are queued and processed one at a time.

# Different limits per endpoint
endpoint_limits = {
    '/api/classify-image': TokenBucket(100, 100/60),  # 100/min
    '/api/process-video': TokenBucket(10, 10/60),     # 10/min
    '/api/train-model': TokenBucket(1, 1/3600),       # 1/hour
}

endpoint = request.path
if not endpoint_limits[endpoint].is_allowed(user_id):
    return "Rate limit exceeded", 429

By Geographic Region

Use case: Global APIs that want to distribute load or comply with regional regulations.

Example: A content delivery API:

• US/EU: 1,000 requests/second
• Asia: 500 requests/second (smaller infrastructure)
• Other regions: 100 requests/second

Real-world scenario: A global news aggregator API serves different regions. Traffic from high-traffic regions (US/EU) gets higher limits, while emerging markets get lower limits initially, scaling up as infrastructure grows.

# Regional limits
regional_limits = {
    'us': TokenBucket(1000, 1000),
    'eu': TokenBucket(1000, 1000),
    'asia': TokenBucket(500, 500),
    'other': TokenBucket(100, 100),
}

region = get_region_from_ip(request.remote_addr)
if not regional_limits[region].is_allowed(request.remote_addr):
    return "Rate limit exceeded", 429

Dynamic Rate Limiting

Use case: Adjust limits based on system load or user behavior.

Example: During normal load, users get 100 requests/minute. During high load, limits drop to 50 requests/minute to protect the system. Trusted users (good history) get 150 requests/minute.

Real-world scenario: A ride-sharing API dynamically adjusts limits. During rush hour (high load), all users get reduced limits. During off-peak hours, limits increase. Users with good payment history get higher limits.

def get_dynamic_limit(user_id):
    base_limit = 100  # requests/minute

    # Adjust based on system load
    current_load = get_system_load()
    if current_load > 0.8:  # High load
        base_limit = base_limit * 0.5  # Reduce to 50

    # Adjust based on user trust
    user_trust_score = get_user_trust_score(user_id)
    if user_trust_score > 0.9:  # Trusted user
        base_limit = base_limit * 1.5  # Increase to 75-150

    return TokenBucket(int(base_limit), base_limit/60)

limiter = get_dynamic_limit(user_id)
if not limiter.is_allowed(user_id):
    return "Rate limit exceeded", 429

---

HTTP Response Headers

Inform clients about rate limits:

HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1640995200

When limit exceeded:

HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1640995200
Retry-After: 60

---

Algorithm Comparison

Algorithm	Bursts	Accuracy	Memory	Complexity
Token Bucket	Yes	High	Low	Medium
Leaky Bucket	No	High	Medium	Medium
Sliding Window	No	Very High	High	High
Fixed Window	Yes	Medium	Low	Low

Recommendation: Use Token Bucket for most cases. It’s simple, accurate, and allows bursts.

---

Key Takeaways

🪣 Token Bucket: Most Popular

Token bucket allows bursts, smooths to average rate. Most widely used algorithm.

📊 Sliding Window: Most Accurate

Sliding window is most accurate but uses more memory. Use when accuracy is critical.

🌐 Distributed: Use Redis

For multiple servers, use Redis with Lua scripts for atomic operations.

🔢 Return 429

When rate limit exceeded, return HTTP 429 with Retry-After header. Inform clients about limits.

---

Next Steps

• Review API Gateway - rate limiting is often implemented in gateways
• Learn Caching - cache rate limit data for performance
• Understand Distributed Systems - distributed rate limiting challenges