API Gateway Pattern

The Microservices Problem

When you have multiple microservices, clients need to know about each one:

Problems:

• Clients need to know all services - Mobile apps must hardcode multiple service URLs, making updates difficult
• Multiple round trips - Fetching related data requires multiple requests, increasing latency
• Cross-cutting concerns duplicated (auth, rate limiting) - Each service implements its own authentication and rate limiting logic
• Hard to change service locations - Moving services requires updating all clients

Solution: API Gateway - A single entry point that routes requests, handles cross-cutting concerns, and aggregates responses.

---

What is an API Gateway?

API Gateway is a single entry point that sits between clients and microservices. It acts as a reverse proxy that routes client requests to appropriate backend services, handles cross-cutting concerns like authentication and rate limiting, and can aggregate responses from multiple services.

The Core Concept

Instead of clients directly calling multiple microservices, they make requests to a single API Gateway. The gateway handles routing, security, monitoring, and other cross-cutting concerns, allowing backend services to focus on business logic. This pattern simplifies client code, centralizes security, and provides a unified interface to your microservices architecture.

Key Responsibilities

1. Request Routing - Route to correct service
2. Authentication - Verify who you are
3. Authorization - Check permissions
4. Rate Limiting - Control request volume
5. Load Balancing - Distribute load
6. Request Aggregation - Combine multiple calls
7. Protocol Translation - REST to gRPC, etc.
8. Monitoring - Logging, metrics

---

Core Functions

1. Request Routing

Route requests to appropriate services:

Routing Examples:

Request	Route To
GET /api/users/123	User Service
POST /api/orders	Order Service
GET /api/products	Product Service
POST /api/payments	Payment Service

2. Authentication & Authorization

Centralized security:

Benefits:

• Services don’t need auth logic - Backend services can focus on business logic without implementing authentication
• Consistent security - All requests go through the same security checks, ensuring uniform enforcement
• Token validation once - Validate tokens at the gateway instead of in every service, improving performance

3. Rate Limiting

Control request volume per client:

4. Request Aggregation

Combine multiple service calls:

Without Gateway (Multiple Requests):

Client → User Service (get user)
Client → Order Service (get orders)
Client → Product Service (get products)
= 3 round trips

With Gateway (Aggregation):

Client → Gateway → [User, Order, Product Services] → Gateway → Client
= 1 round trip

---

Implementation Example

Simple API Gateway

💡 Tip: Click dropdown to switch between languages

Python

Python

"api_gateway.py

from flask import Flask, request, jsonify
import requests
from functools import wraps
from typing import Dict, Optional

app = Flask(__name__)

# Service registry
SERVICES = {
    'users': 'http://user-service:8001',
    'orders': 'http://order-service:8002',
    'products': 'http://product-service:8003',
}

# Rate limiting (simplified)
request_counts = {}

def rate_limit(max_requests=100, window=60):
    """Rate limiting decorator"""
    def decorator(f):
        @wraps(f)
        def wrapper(*args, **kwargs):
            client_id = request.remote_addr

            # Reset counter if window expired
            if client_id not in request_counts:
                request_counts[client_id] = {'count': 0, 'reset_at': time.time() + window}

            # Check limit
            if request_counts[client_id]['count'] >= max_requests:
                return jsonify({'error': 'Rate limit exceeded'}), 429

            # Increment counter
            request_counts[client_id]['count'] += 1

            return f(*args, **kwargs)
        return wrapper
    return decorator

def authenticate():
    """Simple authentication"""
    token = request.headers.get('Authorization')
    if not token or not token.startswith('Bearer '):
        return None

    # Validate token (simplified)
    token_value = token.replace('Bearer ', '')
    # In production, validate with auth service
    return {'user_id': '123', 'role': 'user'}

def route_to_service(service_name: str, path: str):
    """Route request to appropriate service"""
    service_url = SERVICES.get(service_name)
    if not service_url:
        return None, 404

    # Forward request
    url = f"{service_url}{path}"
    method = request.method
    headers = dict(request.headers)

    # Remove host header (service will set its own)
    headers.pop('Host', None)

    response = requests.request(
        method=method,
        url=url,
        headers=headers,
        data=request.get_data(),
        params=request.args
    )

    return response.json(), response.status_code

@app.route('/api/users/<path:path>', methods=['GET', 'POST', 'PUT', 'DELETE'])
@rate_limit(max_requests=100)
def users_proxy(path):
    """Proxy requests to user service"""
    user = authenticate()
    if not user:
        return jsonify({'error': 'Unauthorized'}), 401

    return route_to_service('users', f'/users/{path}')

@app.route('/api/orders/<path:path>', methods=['GET', 'POST', 'PUT', 'DELETE'])
@rate_limit(max_requests=50)
def orders_proxy(path):
    """Proxy requests to order service"""
    user = authenticate()
    if not user:
        return jsonify({'error': 'Unauthorized'}), 401

    return route_to_service('orders', f'/orders/{path}')

@app.route('/api/user-dashboard/<user_id>')
@rate_limit(max_requests=20)
def user_dashboard(user_id):
    """Aggregate multiple service calls"""
    user = authenticate()
    if not user:
        return jsonify({'error': 'Unauthorized'}), 401

    # Call multiple services in parallel
    import concurrent.futures

    with concurrent.futures.ThreadPoolExecutor() as executor:
        user_future = executor.submit(
            requests.get,
            f"{SERVICES['users']}/users/{user_id}"
        )
        orders_future = executor.submit(
            requests.get,
            f"{SERVICES['orders']}/orders?userId={user_id}"
        )

    user_data = user_future.result().json()
    orders_data = orders_future.result().json()

    # Aggregate results
    return jsonify({
        'user': user_data,
        'orders': orders_data
    })

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8000)

Java

"ApiGateway.java

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.*;
import reactor.core.publisher.Mono;

@RestController
public class ApiGateway {

    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
            .route("user-service", r -> r
                .path("/api/users/**")
                .filters(f -> f
                    .requestRateLimiter(config -> config.setRateLimiter(rateLimiter()))
                    .addRequestHeader("X-User-Id", extractUserId())
                )
                .uri("http://user-service:8001"))
            .route("order-service", r -> r
                .path("/api/orders/**")
                .filters(f -> f
                    .requestRateLimiter(config -> config.setRateLimiter(rateLimiter()))
                )
                .uri("http://order-service:8002"))
            .build();
    }

    @GetMapping("/api/user-dashboard/{userId}")
    public Mono<DashboardResponse> getUserDashboard(@PathVariable String userId) {
        // Aggregate multiple service calls
        Mono<User> user = webClient.get()
            .uri("http://user-service:8001/users/{userId}", userId)
            .retrieve()
            .bodyToMono(User.class);

        Mono<List<Order>> orders = webClient.get()
            .uri("http://order-service:8002/orders?userId={userId}", userId)
            .retrieve()
            .bodyToFlux(Order.class)
            .collectList();

        return Mono.zip(user, orders)
            .map(tuple -> new DashboardResponse(tuple.getT1(), tuple.getT2()));
    }

    private RateLimiter rateLimiter() {
        return RateLimiter.of("default", RateLimiterConfig.of(100, Duration.ofMinutes(1)));
    }
}

---

API Gateway Patterns

Pattern 1: Simple Proxy

Just route requests:

Client → Gateway → Service

Use when:

• Simple routing needed
• No aggregation
• Services handle their own concerns

Pattern 2: Backend for Frontend (BFF)

Gateway tailored for specific client:

Pattern 3: Aggregating Gateway

Aggregates multiple service calls:

Client → Gateway → [Service1, Service2, Service3] → Gateway → Client

Use when:

• Need to reduce round trips
• Client needs data from multiple services
• Services are independent

---

Benefits & Drawbacks

Benefits

• Single Entry Point - Clients only know gateway, simplifying client code and deployment
• Centralized Concerns - Auth, rate limiting in one place, reducing duplication and ensuring consistency
• Hides Complexity - Services can change without affecting clients, enabling independent deployment
• Request Aggregation - Reduce round trips by combining multiple service calls into one request
• Protocol Translation - REST to gRPC, etc., allowing services to use different protocols
• Easier Monitoring - Centralized logging/metrics provide unified observability

Drawbacks

• Single Point of Failure - Mitigate with multiple instances and load balancing
• Potential Bottleneck - Can become overloaded if not properly scaled, requiring careful capacity planning
• Additional Latency - Extra hop adds network latency, though usually minimal
• More Complexity - Another component to manage, deploy, and monitor
• Can Become Monolithic - If too much logic added, gateway can become a bottleneck and violate microservices principles

Gateway Anti-Pattern

Don’t put business logic in the gateway! It should only handle cross-cutting concerns. Business logic belongs in services.

---

Popular API Gateway Solutions

Solution	Type	Best For
Kong	Open source	General purpose, plugin ecosystem
AWS API Gateway	Managed	AWS ecosystem
Zuul	Open source	Netflix stack, Spring Cloud
Envoy	Open source	Service mesh, high performance
NGINX	Open source	Simple routing, high performance

---

LLD Connection: Designing Gateways

At the code level, gateways translate to routing logic, middleware, and aggregation patterns.

Gateway Class Design

💡 Tip: Click dropdown to switch between languages

Python

Python

"gateway_design.py

from abc import ABC, abstractmethod
from typing import Dict, Any, List

class Gateway(ABC):
    """Base gateway interface"""

    @abstractmethod
    def route(self, request: Dict[str, Any]) -> Dict[str, Any]:
        """Route request to appropriate service"""
        pass

    @abstractmethod
    def authenticate(self, request: Dict[str, Any]) -> Optional[Dict[str, Any]]:
        """Authenticate request"""
        pass

class APIGateway(Gateway):
    def __init__(self):
        self.routes = {}
        self.middleware = []

    def add_route(self, path: str, service: str):
        """Add routing rule"""
        self.routes[path] = service

    def add_middleware(self, middleware):
        """Add middleware (auth, rate limiting, etc.)"""
        self.middleware.append(middleware)

    def route(self, request: Dict[str, Any]) -> Dict[str, Any]:
        """Route with middleware"""
        # Apply middleware
        for mw in self.middleware:
            request = mw.process(request)
            if request.get('error'):
                return request

        # Route to service
        path = request['path']
        service = self.find_service(path)

        if not service:
            return {'error': 'Service not found', 'status': 404}

        # Forward to service
        return self.forward_to_service(service, request)

    def aggregate(self, requests: List[Dict[str, Any]]) -> Dict[str, Any]:
        """Aggregate multiple service calls"""
        results = {}

        for req in requests:
            service = self.find_service(req['path'])
            result = self.forward_to_service(service, req)
            results[req['path']] = result

        return results

Java

"GatewayDesign.java

import java.util.List;
import java.util.Map;
import java.util.Optional;

interface Gateway {
    Map<String, Object> route(Map<String, Object> request);
    Optional<Map<String, Object>> authenticate(Map<String, Object> request);
}

class APIGateway implements Gateway {
    private final Map<String, String> routes;
    private final List<Middleware> middleware;

    public APIGateway() {
        this.routes = new HashMap<>();
        this.middleware = new ArrayList<>();
    }

    public void addRoute(String path, String service) {
        routes.put(path, service);
    }

    public void addMiddleware(Middleware middleware) {
        this.middleware.add(middleware);
    }

    public Map<String, Object> route(Map<String, Object> request) {
        // Apply middleware
        for (Middleware mw : middleware) {
            request = mw.process(request);
            if (request.containsKey("error")) {
                return request;
            }
        }

        // Route to service
        String path = (String) request.get("path");
        String service = findService(path);

        if (service == null) {
            return Map.of("error", "Service not found", "status", 404);
        }

        // Forward to service
        return forwardToService(service, request);
    }

    public Map<String, Object> aggregate(List<Map<String, Object>> requests) {
        Map<String, Object> results = new HashMap<>();

        for (Map<String, Object> req : requests) {
            String service = findService((String) req.get("path"));
            Map<String, Object> result = forwardToService(service, req);
            results.put((String) req.get("path"), result);
        }

        return results;
    }
}

TypeScript

"gateway-design.ts

interface Gateway {
    route(request: Request): Promise<Response>;
    authenticate(request: Request): Promise<User | null>;
}

class APIGateway implements Gateway {
    private routes: Map<string, string> = new Map();
    private middleware: Middleware[] = [];

    addRoute(path: string, service: string): void {
        this.routes.set(path, service);
    }

    addMiddleware(middleware: Middleware): void {
        this.middleware.push(middleware);
    }

    async route(request: Request): Promise<Response> {
        // Apply middleware
        for (const mw of this.middleware) {
            const result = await mw.process(request);
            if (result.error) {
                return new Response(JSON.stringify(result), { status: result.status });
            }
            request = result.request || request;
        }

        // Route to service
        const path = new URL(request.url).pathname;
        const service = this.findService(path);

        if (!service) {
            return new Response(JSON.stringify({ error: 'Service not found' }), { status: 404 });
        }

        // Forward to service
        return this.forwardToService(service, request);
    }

    async aggregate(requests: Request[]): Promise<Record<string, any>> {
        const results: Record<string, any> = {};

        await Promise.all(requests.map(async (req) => {
            const path = new URL(req.url).pathname;
            const service = this.findService(path);
            if (service) {
                const response = await this.forwardToService(service, req);
                results[path] = await response.json();
            }
        }));

        return results;
    }

    private findService(path: string): string | null {
        for (const [routePath, service] of this.routes) {
            if (path.startsWith(routePath)) {
                return service;
            }
        }
        return null;
    }

    private async forwardToService(service: string, request: Request): Promise<Response> {
        const url = `${service}${new URL(request.url).pathname}`;
        return fetch(url, {
            method: request.method,
            headers: request.headers,
            body: request.body,
        });
    }
}

C++

"gateway_design.cpp

#include <map>
#include <string>
#include <vector>
#include <memory>

class Gateway {
public:
    virtual ~Gateway() = default;
    virtual std::map<std::string, std::string> route(
        const std::map<std::string, std::string>& request
    ) = 0;
    virtual std::optional<std::map<std::string, std::string>> authenticate(
        const std::map<std::string, std::string>& request
    ) = 0;
};

class APIGateway : public Gateway {
private:
    std::map<std::string, std::string> routes;
    std::vector<std::shared_ptr<Middleware>> middleware;

public:
    void addRoute(const std::string& path, const std::string& service) {
        routes[path] = service;
    }

    void addMiddleware(std::shared_ptr<Middleware> mw) {
        middleware.push_back(mw);
    }

    std::map<std::string, std::string> route(
        const std::map<std::string, std::string>& request
    ) override {
        auto processedRequest = request;

        // Apply middleware
        for (const auto& mw : middleware) {
            processedRequest = mw->process(processedRequest);
            if (processedRequest.find("error") != processedRequest.end()) {
                return processedRequest;
            }
        }

        // Route to service
        std::string path = processedRequest.at("path");
        std::string service = findService(path);

        if (service.empty()) {
            return {{"error", "Service not found"}, {"status", "404"}};
        }

        // Forward to service
        return forwardToService(service, processedRequest);
    }

private:
    std::string findService(const std::string& path) {
        for (const auto& [routePath, service] : routes) {
            if (path.find(routePath) == 0) {
                return service;
            }
        }
        return "";
    }

    std::map<std::string, std::string> forwardToService(
        const std::string& service,
        const std::map<std::string, std::string>& request
    ) {
        // Forward request to service
        // Simplified implementation
        return {{"status", "200"}};
    }
};

C#

"GatewayDesign.cs

public interface IGateway {
    Task<Dictionary<string, object>> RouteAsync(Dictionary<string, object> request);
    Task<Dictionary<string, object>?> AuthenticateAsync(Dictionary<string, object> request);
}

public class APIGateway : IGateway {
    private readonly Dictionary<string, string> routes;
    private readonly List<IMiddleware> middleware;

    public APIGateway() {
        this.routes = new Dictionary<string, string>();
        this.middleware = new List<IMiddleware>();
    }

    public void AddRoute(string path, string service) {
        routes[path] = service;
    }

    public void AddMiddleware(IMiddleware middleware) {
        this.middleware.Add(middleware);
    }

    public async Task<Dictionary<string, object>> RouteAsync(Dictionary<string, object> request) {
        // Apply middleware
        foreach (var mw in middleware) {
            request = await mw.ProcessAsync(request);
            if (request.ContainsKey("error")) {
                return request;
            }
        }

        // Route to service
        string path = (string)request["path"];
        string service = FindService(path);

        if (service == null) {
            return new Dictionary<string, object> {
                { "error", "Service not found" },
                { "status", 404 }
            };
        }

        // Forward to service
        return await ForwardToServiceAsync(service, request);
    }

    public async Task<Dictionary<string, object>> AggregateAsync(
        List<Dictionary<string, object>> requests
    ) {
        var results = new Dictionary<string, object>();

        var tasks = requests.Select(async req => {
            string path = (string)req["path"];
            string service = FindService(path);
            if (service != null) {
                var result = await ForwardToServiceAsync(service, req);
                results[path] = result;
            }
        });

        await Task.WhenAll(tasks);
        return results;
    }

    private string? FindService(string path) {
        foreach (var (routePath, service) in routes) {
            if (path.StartsWith(routePath)) {
                return service;
            }
        }
        return null;
    }

    private async Task<Dictionary<string, object>> ForwardToServiceAsync(
        string service,
        Dictionary<string, object> request
    ) {
        // Forward request to service
        // Simplified implementation
        return new Dictionary<string, object> { { "status", 200 } };
    }
}

---

Key Takeaways

Single Entry Point

API Gateway provides one door for all clients. Hides microservices complexity.

Request Aggregation

Combine multiple service calls into one request. Reduces round trips significantly.

Centralized Security

Handle authentication, authorization, and rate limiting in one place. Services stay simple.

Load Balancing

Gateway can distribute load across service instances. Services don’t need to know about each other.

---

Next Steps

• Learn Backend for Frontend - client-specific API gateways
• Understand Real-time Communication - WebSockets, SSE, long polling
• Master Rate Limiting - controlling API usage