API Gateway Pattern

The Microservices Problem

When you have multiple microservices, clients need to know about each one:

Problems:

• ❌ Clients need to know all services
• ❌ Multiple round trips
• ❌ Cross-cutting concerns duplicated (auth, rate limiting)
• ❌ Hard to change service locations

Solution: API Gateway - Single entry point!

---

What is an API Gateway?

API Gateway is a single entry point that sits between clients and microservices.

Key Responsibilities

1. Request Routing - Route to correct service
2. Authentication - Verify who you are
3. Authorization - Check permissions
4. Rate Limiting - Control request volume
5. Load Balancing - Distribute load
6. Request Aggregation - Combine multiple calls
7. Protocol Translation - REST to gRPC, etc.
8. Monitoring - Logging, metrics

---

Core Functions

1. Request Routing

Route requests to appropriate services:

Routing Examples:

Request	Route To
GET /api/users/123	User Service
POST /api/orders	Order Service
GET /api/products	Product Service
POST /api/payments	Payment Service

2. Authentication & Authorization

Centralized security:

Benefits:

• ✅ Services don’t need auth logic
• ✅ Consistent security
• ✅ Token validation once

3. Rate Limiting

Control request volume per client:

4. Request Aggregation

Combine multiple service calls:

Without Gateway (Multiple Requests):

Client → User Service (get user)
Client → Order Service (get orders)
Client → Product Service (get products)
= 3 round trips

With Gateway (Aggregation):

Client → Gateway → [User, Order, Product Services] → Gateway → Client
= 1 round trip

---

Implementation Example

Simple API Gateway

Python

"api_gateway.py

from flask import Flask, request, jsonify
import requests
from functools import wraps
from typing import Dict, Optional

app = Flask(__name__)

# Service registry
SERVICES = {
    'users': 'http://user-service:8001',
    'orders': 'http://order-service:8002',
    'products': 'http://product-service:8003',
}

# Rate limiting (simplified)
request_counts = {}

def rate_limit(max_requests=100, window=60):
    """Rate limiting decorator"""
    def decorator(f):
        @wraps(f)
        def wrapper(*args, **kwargs):
            client_id = request.remote_addr

            # Reset counter if window expired
            if client_id not in request_counts:
                request_counts[client_id] = {'count': 0, 'reset_at': time.time() + window}

            # Check limit
            if request_counts[client_id]['count'] >= max_requests:
                return jsonify({'error': 'Rate limit exceeded'}), 429

            # Increment counter
            request_counts[client_id]['count'] += 1

            return f(*args, **kwargs)
        return wrapper
    return decorator

def authenticate():
    """Simple authentication"""
    token = request.headers.get('Authorization')
    if not token or not token.startswith('Bearer '):
        return None

    # Validate token (simplified)
    token_value = token.replace('Bearer ', '')
    # In production, validate with auth service
    return {'user_id': '123', 'role': 'user'}

def route_to_service(service_name: str, path: str):
    """Route request to appropriate service"""
    service_url = SERVICES.get(service_name)
    if not service_url:
        return None, 404

    # Forward request
    url = f"{service_url}{path}"
    method = request.method
    headers = dict(request.headers)

    # Remove host header (service will set its own)
    headers.pop('Host', None)

    response = requests.request(
        method=method,
        url=url,
        headers=headers,
        data=request.get_data(),
        params=request.args
    )

    return response.json(), response.status_code

@app.route('/api/users/<path:path>', methods=['GET', 'POST', 'PUT', 'DELETE'])
@rate_limit(max_requests=100)
def users_proxy(path):
    """Proxy requests to user service"""
    user = authenticate()
    if not user:
        return jsonify({'error': 'Unauthorized'}), 401

    return route_to_service('users', f'/users/{path}')

@app.route('/api/orders/<path:path>', methods=['GET', 'POST', 'PUT', 'DELETE'])
@rate_limit(max_requests=50)
def orders_proxy(path):
    """Proxy requests to order service"""
    user = authenticate()
    if not user:
        return jsonify({'error': 'Unauthorized'}), 401

    return route_to_service('orders', f'/orders/{path}')

@app.route('/api/user-dashboard/<user_id>')
@rate_limit(max_requests=20)
def user_dashboard(user_id):
    """Aggregate multiple service calls"""
    user = authenticate()
    if not user:
        return jsonify({'error': 'Unauthorized'}), 401

    # Call multiple services in parallel
    import concurrent.futures

    with concurrent.futures.ThreadPoolExecutor() as executor:
        user_future = executor.submit(
            requests.get,
            f"{SERVICES['users']}/users/{user_id}"
        )
        orders_future = executor.submit(
            requests.get,
            f"{SERVICES['orders']}/orders?userId={user_id}"
        )

    user_data = user_future.result().json()
    orders_data = orders_future.result().json()

    # Aggregate results
    return jsonify({
        'user': user_data,
        'orders': orders_data
    })

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8000)

Java

"ApiGateway.java

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.*;
import reactor.core.publisher.Mono;

@RestController
public class ApiGateway {

    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
            .route("user-service", r -> r
                .path("/api/users/**")
                .filters(f -> f
                    .requestRateLimiter(config -> config.setRateLimiter(rateLimiter()))
                    .addRequestHeader("X-User-Id", extractUserId())
                )
                .uri("http://user-service:8001"))
            .route("order-service", r -> r
                .path("/api/orders/**")
                .filters(f -> f
                    .requestRateLimiter(config -> config.setRateLimiter(rateLimiter()))
                )
                .uri("http://order-service:8002"))
            .build();
    }

    @GetMapping("/api/user-dashboard/{userId}")
    public Mono<DashboardResponse> getUserDashboard(@PathVariable String userId) {
        // Aggregate multiple service calls
        Mono<User> user = webClient.get()
            .uri("http://user-service:8001/users/{userId}", userId)
            .retrieve()
            .bodyToMono(User.class);

        Mono<List<Order>> orders = webClient.get()
            .uri("http://order-service:8002/orders?userId={userId}", userId)
            .retrieve()
            .bodyToFlux(Order.class)
            .collectList();

        return Mono.zip(user, orders)
            .map(tuple -> new DashboardResponse(tuple.getT1(), tuple.getT2()));
    }

    private RateLimiter rateLimiter() {
        return RateLimiter.of("default", RateLimiterConfig.of(100, Duration.ofMinutes(1)));
    }
}

---

API Gateway Patterns

Pattern 1: Simple Proxy

Just route requests:

Client → Gateway → Service

Use when:

• Simple routing needed
• No aggregation
• Services handle their own concerns

Pattern 2: Backend for Frontend (BFF)

Gateway tailored for specific client:

Pattern 3: Aggregating Gateway

Aggregates multiple service calls:

Client → Gateway → [Service1, Service2, Service3] → Gateway → Client

Use when:

• Need to reduce round trips
• Client needs data from multiple services
• Services are independent

---

Benefits & Drawbacks

Benefits

• ✅ Single Entry Point - Clients only know gateway
• ✅ Centralized Concerns - Auth, rate limiting in one place
• ✅ Hides Complexity - Services can change without affecting clients
• ✅ Request Aggregation - Reduce round trips
• ✅ Protocol Translation - REST to gRPC, etc.
• ✅ Easier Monitoring - Centralized logging/metrics

Drawbacks

• ❌ Single Point of Failure - Mitigate with multiple instances
• ❌ Potential Bottleneck - Can become overloaded
• ❌ Additional Latency - Extra hop
• ❌ More Complexity - Another component to manage
• ❌ Can Become Monolithic - If too much logic added

Gateway Anti-Pattern

Don’t put business logic in the gateway! It should only handle cross-cutting concerns. Business logic belongs in services.

---

Popular API Gateway Solutions

Solution	Type	Best For
Kong	Open source	General purpose, plugin ecosystem
AWS API Gateway	Managed	AWS ecosystem
Zuul	Open source	Netflix stack, Spring Cloud
Envoy	Open source	Service mesh, high performance
NGINX	Open source	Simple routing, high performance

---

LLD Connection: Designing Gateways

At the code level, gateways translate to routing logic, middleware, and aggregation patterns.

Gateway Class Design

Python

"gateway_design.py

from abc import ABC, abstractmethod
from typing import Dict, Any, List

class Gateway(ABC):
    """Base gateway interface"""

    @abstractmethod
    def route(self, request: Dict[str, Any]) -> Dict[str, Any]:
        """Route request to appropriate service"""
        pass

    @abstractmethod
    def authenticate(self, request: Dict[str, Any]) -> Optional[Dict[str, Any]]:
        """Authenticate request"""
        pass

class APIGateway(Gateway):
    def __init__(self):
        self.routes = {}
        self.middleware = []

    def add_route(self, path: str, service: str):
        """Add routing rule"""
        self.routes[path] = service

    def add_middleware(self, middleware):
        """Add middleware (auth, rate limiting, etc.)"""
        self.middleware.append(middleware)

    def route(self, request: Dict[str, Any]) -> Dict[str, Any]:
        """Route with middleware"""
        # Apply middleware
        for mw in self.middleware:
            request = mw.process(request)
            if request.get('error'):
                return request

        # Route to service
        path = request['path']
        service = self.find_service(path)

        if not service:
            return {'error': 'Service not found', 'status': 404}

        # Forward to service
        return self.forward_to_service(service, request)

    def aggregate(self, requests: List[Dict[str, Any]]) -> Dict[str, Any]:
        """Aggregate multiple service calls"""
        results = {}

        for req in requests:
            service = self.find_service(req['path'])
            result = self.forward_to_service(service, req)
            results[req['path']] = result

        return results

Java

"GatewayDesign.java

import java.util.List;
import java.util.Map;
import java.util.Optional;

interface Gateway {
    Map<String, Object> route(Map<String, Object> request);
    Optional<Map<String, Object>> authenticate(Map<String, Object> request);
}

class APIGateway implements Gateway {
    private final Map<String, String> routes;
    private final List<Middleware> middleware;

    public APIGateway() {
        this.routes = new HashMap<>();
        this.middleware = new ArrayList<>();
    }

    public void addRoute(String path, String service) {
        routes.put(path, service);
    }

    public void addMiddleware(Middleware middleware) {
        this.middleware.add(middleware);
    }

    public Map<String, Object> route(Map<String, Object> request) {
        // Apply middleware
        for (Middleware mw : middleware) {
            request = mw.process(request);
            if (request.containsKey("error")) {
                return request;
            }
        }

        // Route to service
        String path = (String) request.get("path");
        String service = findService(path);

        if (service == null) {
            return Map.of("error", "Service not found", "status", 404);
        }

        // Forward to service
        return forwardToService(service, request);
    }

    public Map<String, Object> aggregate(List<Map<String, Object>> requests) {
        Map<String, Object> results = new HashMap<>();

        for (Map<String, Object> req : requests) {
            String service = findService((String) req.get("path"));
            Map<String, Object> result = forwardToService(service, req);
            results.put((String) req.get("path"), result);
        }

        return results;
    }
}

---

Key Takeaways

🚪 Single Entry Point

API Gateway provides one door for all clients. Hides microservices complexity.

🔄 Request Aggregation

Combine multiple service calls into one request. Reduces round trips significantly.

🛡️ Centralized Security

Handle authentication, authorization, and rate limiting in one place. Services stay simple.

⚖️ Load Balancing

Gateway can distribute load across service instances. Services don’t need to know about each other.

---

Next Steps

• Learn Backend for Frontend - client-specific API gateways
• Understand Real-time Communication - WebSockets, SSE, long polling
• Master Rate Limiting - controlling API usage