Caching Fundamentals

What is Caching?

Imagine you’re a librarian. Every time someone asks for a book, you could walk to the massive warehouse (database) to find it. Or, you could keep the 100 most popular books on a cart right next to you (cache). When someone asks for a popular book, you grab it instantly. That’s caching.

Caching is storing frequently accessed data in fast storage (usually memory) to avoid slow operations like database queries or external API calls.

Real-World Impact

A typical database query takes 10-100ms. A cache lookup takes 0.1-1ms. That’s 100x faster. At scale, this means:

• 10,000 requests/second → 10,000 database queries/second (expensive!)
• 10,000 requests/second → 100 cache hits + 10 database queries/second (cheap!)

The difference is massive.

Why Caching Matters

Problem	How Caching Solves It
Slow Database Queries	Cache stores results, avoiding repeated queries
High Database Load	Reduces database requests by 90%+
Expensive External APIs	Cache API responses, avoid rate limits
Repeated Computations	Cache expensive calculation results
Geographic Latency	Cache data closer to users (CDN)

---

The Four Fundamental Patterns

There are four main ways to integrate caching into your application. Each has different trade-offs.

Pattern 1: Cache-Aside (Lazy Loading)

The most common pattern. Your application manages the cache directly.

How it works:

1. Application checks cache first
2. If cache hit → return data immediately
3. If cache miss → fetch from database
4. Store result in cache for next time
5. Return data to user

Simple Analogy

Cache-aside is like checking your desk drawer before going to the filing cabinet. You control when to check the drawer and when to update it.

When to use:

• ✅ Most common use case
• ✅ You want full control over cache logic
• ✅ Different data needs different caching strategies
• ✅ Cache and database can be separate systems

Trade-offs:

• ✅ Simple to understand and implement
• ✅ Flexible - you control everything
• ❌ Application code must handle cache logic
• ❌ Risk of cache stampede (many requests on miss)

Pattern 2: Read-Through

The cache acts as a proxy. Your application only talks to the cache; the cache handles database access.

How it works:

1. Application requests data from cache
2. Cache checks if data exists
3. If miss, cache automatically fetches from database
4. Cache stores the result
5. Cache returns data to application

When to use:

• ✅ You want simpler application code
• ✅ Cache library handles database complexity
• ✅ Consistent caching behavior across application
• ✅ Good for read-heavy workloads

Trade-offs:

• ✅ Simpler application code
• ✅ Cache handles all complexity
• ❌ Less flexible than cache-aside
• ❌ Cache library must support your database

Pattern 3: Write-Through

Writes go to both cache and database simultaneously. Ensures they stay in sync.

How it works:

1. Application writes data
2. Cache updates immediately
3. Cache writes to database simultaneously
4. Wait for both to complete
5. Return success

Performance Cost

Write-through has higher latency because you wait for both cache AND database writes. But you get strong consistency - cache and database are always in sync.

When to use:

• ✅ Strong consistency required
• ✅ Can’t afford stale cache data
• ✅ Write latency is acceptable
• ✅ Critical data that must be accurate

Trade-offs:

• ✅ Strong consistency
• ✅ Cache and database always match
• ❌ Higher write latency (waits for DB)
• ❌ Database becomes bottleneck for writes

Pattern 4: Write-Behind (Write-Back)

Write to cache immediately, database write happens later. Fastest writes, but risky.

How it works:

1. Application writes data
2. Cache updates immediately
3. Return success to application (fast!)
4. Queue database write for later
5. Background process writes to database

Data Loss Risk

If cache fails before database write completes, data is lost! Use write-behind only when you can tolerate some data loss (like analytics, logs, or non-critical updates).

When to use:

• ✅ Write performance is critical
• ✅ Can tolerate eventual consistency
• ✅ Non-critical data (analytics, logs)
• ✅ High write volume

Trade-offs:

• ✅ Lowest write latency
• ✅ High write throughput
• ❌ Risk of data loss
• ❌ Eventual consistency (cache and DB may differ temporarily)

---

LLD Connection: Implementing Cache Patterns

At the code level, caching patterns translate to decorator patterns and repository abstractions.

Cache Decorator Pattern

The decorator pattern is perfect for adding caching to existing repositories:

Python

cache_decorator.py

from functools import wraps
from typing import Callable, Any
import time

class CacheDecorator:
    def __init__(self, cache: dict, ttl: int = 300):
        self.cache = cache
        self.ttl = ttl  # Time to live in seconds

    def __call__(self, func: Callable) -> Callable:
        @wraps(func)
        def wrapper(*args, **kwargs):
            # Create cache key from function args
            cache_key = f"{func.__name__}:{args}:{kwargs}"

            # Check cache (cache-aside pattern)
            if cache_key in self.cache:
                cached_data, timestamp = self.cache[cache_key]
                if time.time() - timestamp < self.ttl:
                    return cached_data

            # Cache miss - fetch from source
            result = func(*args, **kwargs)

            # Store in cache
            self.cache[cache_key] = (result, time.time())
            return result

        return wrapper

# Usage
cache = {}
@CacheDecorator(cache, ttl=300)
def get_user(user_id: int):
    # Simulate database query
    return {"id": user_id, "name": "John"}

Java

CacheDecorator.java

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;

public class CacheDecorator<T, R> {
    private final Map<String, CacheEntry<R>> cache;
    private final long ttlMillis;

    public CacheDecorator(long ttlSeconds) {
        this.cache = new ConcurrentHashMap<>();
        this.ttlMillis = ttlSeconds * 1000;
    }

    public R apply(String key, Function<T, R> function, T input) {
        // Check cache (cache-aside pattern)
        CacheEntry<R> entry = cache.get(key);
        if (entry != null && !entry.isExpired()) {
            return entry.value;
        }

        // Cache miss - fetch from source
        R result = function.apply(input);

        // Store in cache
        cache.put(key, new CacheEntry<>(result, System.currentTimeMillis()));
        return result;
    }

    private static class CacheEntry<R> {
        final R value;
        final long timestamp;

        CacheEntry(R value, long timestamp) {
            this.value = value;
            this.timestamp = timestamp;
        }

        boolean isExpired() {
            return System.currentTimeMillis() - timestamp >
                   CacheDecorator.this.ttlMillis;
        }
    }
}

Repository with Caching

A more complete example showing cache-aside in a repository:

Python

user_repository.py

from abc import ABC, abstractmethod
from typing import Optional

class UserRepository(ABC):
    @abstractmethod
    def get_user(self, user_id: int) -> Optional[dict]:
        pass

class DatabaseUserRepository(UserRepository):
    def get_user(self, user_id: int) -> Optional[dict]:
        # Simulate database query
        return {"id": user_id, "name": "John"}

class CachedUserRepository(UserRepository):
    def __init__(self, db_repo: UserRepository, cache: dict):
        self.db_repo = db_repo
        self.cache = cache

    def get_user(self, user_id: int) -> Optional[dict]:
        # Cache-aside pattern
        cache_key = f"user:{user_id}"

        # Check cache first
        if cache_key in self.cache:
            return self.cache[cache_key]

        # Cache miss - fetch from DB
        user = self.db_repo.get_user(user_id)

        # Store in cache
        if user:
            self.cache[cache_key] = user

        return user

Java

UserRepository.java

import java.util.Optional;
import java.util.Map;

public interface UserRepository {
    Optional<User> getUser(int userId);
}

class DatabaseUserRepository implements UserRepository {
    public Optional<User> getUser(int userId) {
        // Simulate database query
        return Optional.of(new User(userId, "John"));
    }
}

class CachedUserRepository implements UserRepository {
    private final UserRepository dbRepo;
    private final Map<String, User> cache;

    public CachedUserRepository(UserRepository dbRepo, Map<String, User> cache) {
        this.dbRepo = dbRepo;
        this.cache = cache;
    }

    public Optional<User> getUser(int userId) {
        // Cache-aside pattern
        String cacheKey = "user:" + userId;

        // Check cache first
        if (cache.containsKey(cacheKey)) {
            return Optional.of(cache.get(cacheKey));
        }

        // Cache miss - fetch from DB
        Optional<User> user = dbRepo.getUser(userId);

        // Store in cache
        user.ifPresent(u -> cache.put(cacheKey, u));

        return user;
    }
}

---

Pattern Comparison

Pattern	Read Latency	Write Latency	Consistency	Complexity	Use Case
Cache-Aside	Low (cache hit)	Low	Eventual	Medium	Most applications
Read-Through	Low (cache hit)	Low	Eventual	Low	Read-heavy apps
Write-Through	Low (cache hit)	High (waits for DB)	Strong	Medium	Critical data
Write-Behind	Low (cache hit)	Very Low	Eventual	High	High write volume

---

Key Takeaways

🎯 Cache-Aside is King

Most applications use cache-aside. It’s flexible, understandable, and gives you control.

⚡ Speed Matters

Cache lookups are 100x faster than database queries. At scale, this difference is massive.

🔄 Consistency Trade-offs

Faster writes (write-behind) = weaker consistency. Stronger consistency (write-through) = slower writes.

🏗️ Decorator Pattern

Use decorator pattern in code to add caching transparently to existing repositories.

---

Next Steps

• Learn about Cache Eviction Policies - what happens when cache is full?
• Understand Distributed Caching - caching across multiple servers
• Master Cache Invalidation - the hardest problem in computer science