Caching Fundamentals

What is Caching?

Imagine you’re a librarian. Every time someone asks for a book, you could walk to the massive warehouse (database) to find it. Or, you could keep the 100 most popular books on a cart right next to you (cache). When someone asks for a popular book, you grab it instantly. That’s caching.

Caching is storing frequently accessed data in fast storage (usually memory) to avoid slow operations like database queries or external API calls.

Real-World Impact

A typical database query takes 10-100ms. A cache lookup takes 0.1-1ms. That’s 100x faster. At scale, this means:

• 10,000 requests/second → 10,000 database queries/second (expensive!)
• 10,000 requests/second → 100 cache hits + 10 database queries/second (cheap!)

The difference is massive.

Why Caching Matters

Problem	How Caching Solves It
Slow Database Queries	Cache stores results, avoiding repeated queries
High Database Load	Reduces database requests by 90%+
Expensive External APIs	Cache API responses, avoid rate limits
Repeated Computations	Cache expensive calculation results
Geographic Latency	Cache data closer to users (CDN)

---

The Four Fundamental Patterns

There are four main ways to integrate caching into your application. Each has different trade-offs.

Pattern 1: Cache-Aside (Lazy Loading)

The most common pattern. Your application manages the cache directly.

How it works:

1. Application checks cache first
2. If cache hit → return data immediately
3. If cache miss → fetch from database
4. Store result in cache for next time
5. Return data to user

Simple Analogy

Cache-aside is like checking your desk drawer before going to the filing cabinet. You control when to check the drawer and when to update it.

When to use:

• Most common use case - Widely used pattern that fits most scenarios
• You want full control over cache logic - Application manages cache directly
• Different data needs different caching strategies - Flexibility to customize per data type
• Cache and database can be separate systems - No tight coupling required

Trade-offs:

• Simple to understand and implement - Straightforward pattern with clear logic
• Flexible - you control everything - Full control over cache behavior
• Application code must handle cache logic - More code to write and maintain
• Risk of cache stampede - Multiple requests can hit database simultaneously on cache miss

Pattern 2: Read-Through

The cache acts as a proxy. Your application only talks to the cache; the cache handles database access.

How it works:

1. Application requests data from cache
2. Cache checks if data exists
3. If miss, cache automatically fetches from database
4. Cache stores the result
5. Cache returns data to application

When to use:

• You want simpler application code - Cache library handles complexity
• Cache library handles database complexity - Less code to write
• Consistent caching behavior across application - Standardized approach
• Good for read-heavy workloads - Optimized for frequent reads

Trade-offs:

• Simpler application code - Less boilerplate to write
• Cache handles all complexity - Library manages everything
• Less flexible than cache-aside - Limited customization options
• Cache library must support your database - Dependency on library features

Pattern 3: Write-Through

Writes go to both cache and database simultaneously. Ensures they stay in sync.

How it works:

1. Application writes data
2. Cache updates immediately
3. Cache writes to database simultaneously
4. Wait for both to complete
5. Return success

Performance Cost

Write-through has higher latency because you wait for both cache AND database writes. But you get strong consistency - cache and database are always in sync.

When to use:

• Strong consistency required - Cache and database must always match
• Can’t afford stale cache data - Data accuracy is critical
• Write latency is acceptable - Performance trade-off is acceptable
• Critical data that must be accurate - Financial, inventory, pricing data

Trade-offs:

• Strong consistency - Cache and database always synchronized
• Cache and database always match - No stale data risk
• Higher write latency - Must wait for both cache and database writes
• Database becomes bottleneck for writes - All writes go through database

Pattern 4: Write-Behind (Write-Back)

Write to cache immediately, database write happens later. Fastest writes, but risky.

How it works:

1. Application writes data
2. Cache updates immediately
3. Return success to application (fast!)
4. Queue database write for later
5. Background process writes to database

Data Loss Risk

If cache fails before database write completes, data is lost! Use write-behind only when you can tolerate some data loss (like analytics, logs, or non-critical updates).

When to use:

• Write performance is critical - Need maximum write speed
• Can tolerate eventual consistency - Temporary inconsistency acceptable
• Non-critical data - Analytics, logs, metrics where some loss is acceptable
• High write volume - Need to handle many writes efficiently

Trade-offs:

• Lowest write latency - Writes complete immediately
• High write throughput - Can handle many writes per second
• Risk of data loss - Data lost if cache fails before database write
• Eventual consistency - Cache and database may differ temporarily

---

LLD Connection: Implementing Cache Patterns

At the code level, caching patterns translate to decorator patterns and repository abstractions.

Cache Decorator Pattern

The decorator pattern is perfect for adding caching to existing repositories:

💡 Tip: Click dropdown to switch between languages

Python

Python

cache_decorator.py

from functools import wraps
from typing import Callable, Any
import time

class CacheDecorator:
    def __init__(self, cache: dict, ttl: int = 300):
        self.cache = cache
        self.ttl = ttl  # Time to live in seconds

    def __call__(self, func: Callable) -> Callable:
        @wraps(func)
        def wrapper(*args, **kwargs):
            # Create cache key from function args
            cache_key = f"{func.__name__}:{args}:{kwargs}"

            # Check cache (cache-aside pattern)
            if cache_key in self.cache:
                cached_data, timestamp = self.cache[cache_key]
                if time.time() - timestamp < self.ttl:
                    return cached_data

            # Cache miss - fetch from source
            result = func(*args, **kwargs)

            # Store in cache
            self.cache[cache_key] = (result, time.time())
            return result

        return wrapper

# Usage
cache = {}
@CacheDecorator(cache, ttl=300)
def get_user(user_id: int):
    # Simulate database query
    return {"id": user_id, "name": "John"}

Java

CacheDecorator.java

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;

public class CacheDecorator<T, R> {
    private final Map<String, CacheEntry<R>> cache;
    private final long ttlMillis;

    public CacheDecorator(long ttlSeconds) {
        this.cache = new ConcurrentHashMap<>();
        this.ttlMillis = ttlSeconds * 1000;
    }

    public R apply(String key, Function<T, R> function, T input) {
        // Check cache (cache-aside pattern)
        CacheEntry<R> entry = cache.get(key);
        if (entry != null && !entry.isExpired()) {
            return entry.value;
        }

        // Cache miss - fetch from source
        R result = function.apply(input);

        // Store in cache
        cache.put(key, new CacheEntry<>(result, System.currentTimeMillis()));
        return result;
    }

    private static class CacheEntry<R> {
        final R value;
        final long timestamp;

        CacheEntry(R value, long timestamp) {
            this.value = value;
            this.timestamp = timestamp;
        }

        boolean isExpired() {
            return System.currentTimeMillis() - timestamp >
                   CacheDecorator.this.ttlMillis;
        }
    }
}

TypeScript

"cache-decorator.ts

interface CacheEntry<T> {
    value: T;
    timestamp: number;
}

class CacheDecorator<T, R> {
    private cache: Map<string, CacheEntry<R>>;
    private ttl: number;

    constructor(ttlSeconds: number) {
        this.cache = new Map();
        this.ttl = ttlSeconds * 1000;
    }

    apply(key: string, fn: (input: T) => R, input: T): R {
        const entry = this.cache.get(key);
        if (entry && !this.isExpired(entry)) {
            return entry.value;
        }

        const result = fn(input);
        this.cache.set(key, {
            value: result,
            timestamp: Date.now()
        });

        return result;
    }

    private isExpired(entry: CacheEntry<R>): boolean {
        return Date.now() - entry.timestamp > this.ttl;
    }
}

C++

"cache_decorator.cpp

#include <unordered_map>
#include <chrono>
#include <string>
#include <functional>

template<typename T, typename R>
class CacheDecorator {
private:
    struct CacheEntry {
        R value;
        std::chrono::steady_clock::time_point timestamp;
    };

    std::unordered_map<std::string, CacheEntry> cache;
    std::chrono::milliseconds ttl;

    bool isExpired(const CacheEntry& entry) const {
        auto now = std::chrono::steady_clock::now();
        return (now - entry.timestamp) > ttl;
    }

public:
    CacheDecorator(int ttlSeconds)
        : ttl(std::chrono::seconds(ttlSeconds)) {}

    R apply(const std::string& key, std::function<R(T)> fn, T input) {
        auto it = cache.find(key);
        if (it != cache.end() && !isExpired(it->second)) {
            return it->second.value;
        }

        R result = fn(input);
        cache[key] = {
            result,
            std::chrono::steady_clock::now()
        };

        return result;
    }
};

C#

"CacheDecorator.cs

using System;
using System.Collections.Generic;

public class CacheDecorator<T, R> {
    private readonly Dictionary<string, CacheEntry<R>> cache;
    private readonly TimeSpan ttl;

    private class CacheEntry<TValue> {
        public TValue Value { get; }
        public DateTime Timestamp { get; }

        public CacheEntry(TValue value, DateTime timestamp) {
            Value = value;
            Timestamp = timestamp;
        }
    }

    public CacheDecorator(int ttlSeconds) {
        this.cache = new Dictionary<string, CacheEntry<R>>();
        this.ttl = TimeSpan.FromSeconds(ttlSeconds);
    }

    public R Apply(string key, Func<T, R> fn, T input) {
        if (cache.TryGetValue(key, out var entry) && !IsExpired(entry)) {
            return entry.Value;
        }

        var result = fn(input);
        cache[key] = new CacheEntry<R>(result, DateTime.UtcNow);

        return result;
    }

    private bool IsExpired(CacheEntry<R> entry) {
        return DateTime.UtcNow - entry.Timestamp > ttl;
    }
}

Repository with Caching

A more complete example showing cache-aside in a repository:

💡 Tip: Click dropdown to switch between languages

Python

Python

user_repository.py

from abc import ABC, abstractmethod
from typing import Optional

class UserRepository(ABC):
    @abstractmethod
    def get_user(self, user_id: int) -> Optional[dict]:
        pass

class DatabaseUserRepository(UserRepository):
    def get_user(self, user_id: int) -> Optional[dict]:
        # Simulate database query
        return {"id": user_id, "name": "John"}

class CachedUserRepository(UserRepository):
    def __init__(self, db_repo: UserRepository, cache: dict):
        self.db_repo = db_repo
        self.cache = cache

    def get_user(self, user_id: int) -> Optional[dict]:
        # Cache-aside pattern
        cache_key = f"user:{user_id}"

        # Check cache first
        if cache_key in self.cache:
            return self.cache[cache_key]

        # Cache miss - fetch from DB
        user = self.db_repo.get_user(user_id)

        # Store in cache
        if user:
            self.cache[cache_key] = user

        return user

Java

UserRepository.java

import java.util.Optional;
import java.util.Map;

public interface UserRepository {
    Optional<User> getUser(int userId);
}

class DatabaseUserRepository implements UserRepository {
    public Optional<User> getUser(int userId) {
        // Simulate database query
        return Optional.of(new User(userId, "John"));
    }
}

class CachedUserRepository implements UserRepository {
    private final UserRepository dbRepo;
    private final Map<String, User> cache;

    public CachedUserRepository(UserRepository dbRepo, Map<String, User> cache) {
        this.dbRepo = dbRepo;
        this.cache = cache;
    }

    public Optional<User> getUser(int userId) {
        // Cache-aside pattern
        String cacheKey = "user:" + userId;

        // Check cache first
        if (cache.containsKey(cacheKey)) {
            return Optional.of(cache.get(cacheKey));
        }

        // Cache miss - fetch from DB
        Optional<User> user = dbRepo.getUser(userId);

        // Store in cache
        user.ifPresent(u -> cache.put(cacheKey, u));

        return user;
    }
}

TypeScript

"user-repository.ts

interface User {
    id: number;
    name: string;
}

interface UserRepository {
    getUser(userId: number): User | null;
}

class DatabaseUserRepository implements UserRepository {
    getUser(userId: number): User | null {
        // Simulate database query
        return { id: userId, name: "John" };
    }
}

class CachedUserRepository implements UserRepository {
    private dbRepo: UserRepository;
    private cache: Map<string, User>;

    constructor(dbRepo: UserRepository, cache: Map<string, User>) {
        this.dbRepo = dbRepo;
        this.cache = cache;
    }

    getUser(userId: number): User | null {
        const cacheKey = `user:${userId}`;

        // Check cache first
        if (this.cache.has(cacheKey)) {
            return this.cache.get(cacheKey)!;
        }

        // Cache miss - fetch from DB
        const user = this.dbRepo.getUser(userId);

        // Store in cache
        if (user) {
            this.cache.set(cacheKey, user);
        }

        return user;
    }
}

C++

"user_repository.cpp

#include <optional>
#include <unordered_map>
#include <string>

struct User {
    int id;
    std::string name;
};

class UserRepository {
public:
    virtual std::optional<User> getUser(int userId) = 0;
};

class DatabaseUserRepository : public UserRepository {
public:
    std::optional<User> getUser(int userId) override {
        // Simulate database query
        return User{userId, "John"};
    }
};

class CachedUserRepository : public UserRepository {
private:
    UserRepository* dbRepo;
    std::unordered_map<std::string, User> cache;

public:
    CachedUserRepository(UserRepository* dbRepo) : dbRepo(dbRepo) {}

    std::optional<User> getUser(int userId) override {
        std::string cacheKey = "user:" + std::to_string(userId);

        // Check cache first
        auto it = cache.find(cacheKey);
        if (it != cache.end()) {
            return it->second;
        }

        // Cache miss - fetch from DB
        auto user = dbRepo->getUser(userId);

        // Store in cache
        if (user.has_value()) {
            cache[cacheKey] = user.value();
        }

        return user;
    }
};

C#

"UserRepository.cs

using System;
using System.Collections.Generic;

public class User {
    public int Id { get; set; }
    public string Name { get; set; }
}

public interface IUserRepository {
    User GetUser(int userId);
}

public class DatabaseUserRepository : IUserRepository {
    public User GetUser(int userId) {
        // Simulate database query
        return new User { Id = userId, Name = "John" };
    }
}

public class CachedUserRepository : IUserRepository {
    private readonly IUserRepository dbRepo;
    private readonly Dictionary<string, User> cache;

    public CachedUserRepository(IUserRepository dbRepo) {
        this.dbRepo = dbRepo;
        this.cache = new Dictionary<string, User>();
    }

    public User GetUser(int userId) {
        var cacheKey = $"user:{userId}";

        // Check cache first
        if (cache.TryGetValue(cacheKey, out var cachedUser)) {
            return cachedUser;
        }

        // Cache miss - fetch from DB
        var user = dbRepo.GetUser(userId);

        // Store in cache
        if (user != null) {
            cache[cacheKey] = user;
        }

        return user;
    }
}

---

Real-World Examples

Understanding how major companies implement caching patterns helps illustrate when to use each approach:

Cache-Aside: Facebook’s News Feed

The Challenge: Facebook’s news feed serves personalized content to billions of users. Each user’s feed is unique, requiring complex queries across multiple data sources.

The Solution: Facebook uses cache-aside pattern extensively:

• Application checks Memcached first for pre-computed feed data
• On cache miss, queries multiple services (posts, likes, comments, friends)
• Stores computed feed in cache for future requests
• Cache TTL: 2-5 minutes (balance between freshness and load)

Why Cache-Aside? Different users need different caching strategies. Some users have high engagement (shorter TTL), others are casual (longer TTL). Cache-aside gives Facebook flexibility to customize per user.

Impact: Reduces database load by 90%+. A typical feed request that would take 500ms from database takes 5ms from cache.

Read-Through: Amazon Product Catalog

The Challenge: Amazon’s product catalog is accessed millions of times per second. Product data changes infrequently but needs to be fast.

The Solution: Amazon uses read-through caching:

• Application only talks to cache layer (DynamoDB with caching)
• Cache automatically fetches from database on miss
• Cache library handles all database complexity
• Products cached for hours (they change rarely)

Why Read-Through? Simpler application code. Product service doesn’t need to know about cache - it just reads products. Cache layer handles everything.

Impact: Product pages load in 50ms instead of 200ms. During Prime Day, caching handles 10x normal traffic without database overload.

Write-Through: Financial Trading Systems

The Challenge: Trading platforms need real-time, accurate prices. Stale data means wrong trades, which costs money.

The Solution: Trading systems use write-through:

• Price update goes to both cache and database simultaneously
• Cache always has latest price
• Database is source of truth
• No stale data risk

Why Write-Through? Strong consistency is critical. A 1-second delay showing wrong price could mean millions in losses. Write-through ensures cache and database always match.

Example: A stock price update from $100 to $105:

• Write-through: Cache = $105, DB = $105 (consistent)
• Cache-aside: Cache might still show $100 until next read (stale)

Write-Behind: Twitter’s Tweet Analytics

The Challenge: Twitter generates billions of tweets per day. Each tweet needs analytics (views, likes, retweets) tracked, but write performance is critical.

The Solution: Twitter uses write-behind for analytics:

• Tweet creation writes to cache immediately (fast response)
• Analytics (view counts, engagement) written to cache first
• Database writes happen asynchronously in batches
• Some data loss acceptable (analytics are approximate anyway)

Why Write-Behind? Write performance is critical. Users expect instant tweet posting. Analytics can be eventually consistent - losing a few view counts is acceptable.

Impact: Tweet creation latency: 10ms (cache write) vs 100ms (database write). During viral events, write-behind handles 100x normal write volume.

Netflix: Hybrid Approach

The Challenge: Netflix serves video metadata (titles, descriptions, ratings) globally. Data changes rarely but needs to be fast worldwide.

The Solution: Netflix uses multiple patterns:

• Read-Through for video metadata (rarely changes, cache for days)
• Cache-Aside for user recommendations (personalized, cache per user)
• Write-Through for watch history (critical data, must be consistent)

Why Multiple Patterns? Different data has different requirements. Metadata can be stale, recommendations are personalized, watch history must be accurate.

Impact: 95% of requests served from cache. Global latency reduced from 200ms to 20ms average.

---

Key Takeaways from Real Examples

1. Cache-Aside for flexibility - When you need different strategies per use case
2. Read-Through for simplicity - When you want simpler application code
3. Write-Through for consistency - When data accuracy is critical
4. Write-Behind for performance - When write speed matters more than consistency
5. Hybrid approaches - Most companies use multiple patterns for different data types

---

Pattern Comparison

Pattern	Read Latency	Write Latency	Consistency	Complexity	Use Case
Cache-Aside	Low (cache hit)	Low	Eventual	Medium	Most applications
Read-Through	Low (cache hit)	Low	Eventual	Low	Read-heavy apps
Write-Through	Low (cache hit)	High (waits for DB)	Strong	Medium	Critical data
Write-Behind	Low (cache hit)	Very Low	Eventual	High	High write volume

---

Key Takeaways

🎯 Cache-Aside is King

Most applications use cache-aside. It’s flexible, understandable, and gives you control.

⚡ Speed Matters

Cache lookups are 100x faster than database queries. At scale, this difference is massive.

🔄 Consistency Trade-offs

Faster writes (write-behind) = weaker consistency. Stronger consistency (write-through) = slower writes.

🏗️ Decorator Pattern

Use decorator pattern in code to add caching transparently to existing repositories.

---

Next Steps

• Learn about Cache Eviction Policies - what happens when cache is full?
• Understand Distributed Caching - caching across multiple servers
• Master Cache Invalidation - the hardest problem in computer science