Design a Feed Generator
What is the Feed Generator Problem?
Section titled “What is the Feed Generator Problem?”Design a feed generator system that aggregates content from multiple sources, ranks posts based on various algorithms, and delivers personalized feeds to users. The system should support different ranking strategies, caching mechanisms, real-time feed updates, and content filtering. The design should be extensible, performant, and handle high volumes of concurrent requests.
In this problem, you’ll design a service that builds a custom view for every user, balancing freshness with relevance while handling millions of concurrent content updates.
Problem Overview
Section titled “Problem Overview”Design a discovery engine that collects posts from a user’s network, ranks them based on interest and quality, and serves a smooth, personalized experience.
Core Requirements
Section titled “Core Requirements”Functional Requirements:
- Aggregation: Collect content from followed users, pages, and groups.
- Personalization: Rank posts using multiple strategies (Latest, Popular, Relevant).
- Filtering: Remove duplicate, low-quality, or sensitive content.
- Real-time Updates: Notify users of new relevant content as it arrives.
- Multi-content Support: Handle text, images, videos, and shared links.
- Social Metrics: Integrate engagement (likes, comments) into the ranking logic.
Non-Functional Requirements:
- Ultra-Low Latency: Feeds must load in milliseconds even with millions of posts.
- Consistency: Ensure users don’t see the same post twice in a single session.
- Scalability: Handle massive fan-out (one celebrity post going to millions of followers).
- Availability: The system must remain fast even during traffic spikes.
What’s Expected?
Section titled “What’s Expected?”1. System Architecture
Section titled “1. System Architecture”The generator sits between the ContentStore and the UserFeed, coordinating between the Ranker and Cache.
2. Key Classes to Design
Section titled “2. Key Classes to Design”classDiagram
class FeedGenerator {
-RankingStrategy ranking
-List~Filter~ filters
-FeedCache cache
+generateFeed(user)
+onNewPost(post)
}
class RankingStrategy {
<<interface>>
+score(post, user) double
}
class Post {
-String id
-User author
-ContentType type
-EngagementStats stats
}
class FeedCache {
-Map~UserId, List~Post~~ storage
+invalidate(user)
+update(user, post)
}
FeedGenerator --> RankingStrategy
FeedGenerator --> FeedCache
Post --> ContentType
System Flow
Section titled “System Flow”Personalized Ranking Flow
Section titled “Personalized Ranking Flow”
Key Design Challenges
Section titled “Key Design Challenges”1. Ranking Latency
Section titled “1. Ranking Latency”Sorting 10,000 posts for every “refresh” click is too slow.
Solution: Use Asynchronous Pre-computation. When a post is created, the system “pushes” it into the pre-computed feeds of active followers. For inactive users, the system only computes the feed “lazily” when they log in.
2. The “Celebrity” Fan-out Problem
Section titled “2. The “Celebrity” Fan-out Problem”When a user with 50 million followers posts, updating 50 million caches simultaneously will crash the system.
Solution: Hybrid Model (Push + Pull). Push posts from “Normal” users to their few followers. For “Celebrities,” don’t push. Instead, when a follower of a celebrity loads their feed, the system “pulls” the celebrity’s recent posts and merges them into the ranking on-the-fly.
3. Dynamic Ranking Rules
Section titled “3. Dynamic Ranking Rules”Business wants to promote “Video” content this week and “Breaking News” next week.
Solution: Use the Strategy Pattern combined with a Weighted Scorer. The RankingStrategy iterates through a list of Scorer components (RecencyScorer, EngagementScorer, TypeScorer). You can change the “weight” of each scorer via a configuration file without changing the code.
What You’ll Learn
Section titled “What You’ll Learn”By solving this problem, you’ll master:
- ✅ Feed Optimization - Balancing Push vs. Pull models.
- ✅ Personalization Logic - Building flexible weighted ranking systems.
- ✅ Cache Invalidation - Managing user-specific snapshots at scale.
- ✅ Event-Driven Architecture - Handling massive data fan-outs gracefully.
View Complete Solution & Practice
Section titled “View Complete Solution & Practice”Ready to see the full implementation? Open the interactive playground to access:
- 🎯 Step-by-step guidance through the 8-step LLD approach
- 📊 Interactive UML builder to visualize your design
- 💻 Complete Code Solutions in Python, Java, C++, TypeScript, JavaScript, C#
- 🤖 AI-powered review of your design and code
Related Problems
Section titled “Related Problems”After mastering the Feed Generator, try these similar problems:
- Search Index - Building high-speed retrieval systems.
- Notification Service - Handling massive broadcast events.
- Chat Room Manager - Managing real-time message streams.