System Design Sketches

Functional requirements: ------------------------ - Follow people - See followers and falowees - Create posts (text, image, url, @mentions) - Like posts - View user feed - View home feed Non-functional requirements: ---------------------------- - Add post <500ms - View feed <500ms - Post in feed with ~2 mins delay is ok - Billions of users - Unlimited followers Core entities: -------------- - User - Follow - Post API (RPC-like): --------------- - POST /getFollowers {userId} - POST /getFollowees {userId} - POST /followUser {userId} - POST /createPost {text} - POST /likePost {postId} - POST /getHomeFeed {userId} - POST /getUserFeed {userId} 

Functional requirements: ------------------------ - Rider gets estimated fare (by Start and Destination locations) - Rider requests a ride based on the estimate - Driver accepts/denies request + Navigates to pickup/drop-off Non-functional requirements: ---------------------------- - Low latency for rider-driver matching (<1min) - Consistency of rider-driver 1:1 matching - For one ride, only one driver will get request at a time - Highly available outside matching - Handle high-throughput (peak hours, popular events) Core entities: -------------- - Rider - Driver - Ride - Location API: --------------- - POST /getEstimate({from, to}) --> Partial<Ride> - POST /requestRide({ride_id}) --> Ride - POST /getRide({ride_id}) - gets the latest ride entity to display the ride details for both rider and driver apps - POST /updateLocation({lat, lon}) - both driver and rider can update their locations which may be used to display their on the map - drivers might update their location i.e. once per 5 seconds (it is needed for matching) - however this can be optimised on the client side, where we might say that location update is proportional to driver speed: for still drivers we update location rarely, for moving - more often - POST /acceptRide({ride_id, accept}) - ride is being accepted by the driver - POST /updateRide({ride_id, status}) 

Functional requirements: ------------------------ - Group chat (<100 users) - Send message - Receive message - Persist messages (user messages are available on multiple devices) Non-functional requirements: ---------------------------- - Low latency for message delivery (<500ms) - No lost messages (delivery guarantee) Core entities: -------------- - Users - Groups - Messages API: --------------- - createGroup({users: [], name: ''}) -> {group_id} - joinGroup({group_id}) -> "ACK"/ "ERR" - quitGroup({group_id}) -> "ACK"/ "ERR" - getMyGroups() -> {groups: []} - getGroupMessages({group_id, max_message_id, min_message_id}) -> {messages: []} WebSocket: - connect() - sendMessage({group_id, message, media: []}) -> "ACK"/ "ERR" - onGroupUpdated({group_id, participants}) -> "ACK" - onMessage({chat_id, user_id, message, attachments}) -> "ACK" 

Functional requirements: ------------------------ - Upload video - Watch video (read-heavy) - Comment on video Non-functional requirements: ---------------------------- - Reliable uploads (no corrupted/missing videos) - Read-heavy (read/write ratio ~100:1) - Availability > Consistency (1-2 min delay between upload and feed is ok) - Low latency (before video starts playing) Core entities: -------------- - User - Video Metadata - Video Chunks - Users - Comments API (RPC-like): --------------- - POST /initiateUpload {title, description, video, ...}} - server responds with S3 private upload URL - the actual video upload is going directly from client to S3 - GET /watch {video_id} - returns video chunks metadata - S3 (or CDN) URLs to manifest/playlist files - video streaming happens directly from CDNs - download by chunks (buffering, ~1Mb), no need to load all 

Functional requirements: ------------------------ - Upload file - Download file - Sync files across devices Non-functional requirements: ---------------------------- - Availability over consistency - Download the older version of the file instead of waiting for a new version to propagate is ok - low latency for uploads and downloads - support large files (50Gb) - resumable uploads - high data integrity (accurate syncs) Core entities: -------------- - File (row bytes) - File Metadata - Users API: --------------- - POST /uploadFile({metadata}) --> {upload_id, presigned_url} - POST /getFile({file_id}) --> {file{}, metadata} - POST /getChanges({since}) --> file_metadata[] 

Functional requirements: ------------------------ - Scrape the web starting from seed URLs - Extract and store text data Non-functional requirements: ---------------------------- - Fault-tolerant - Politeness (respect robots.txt, don't DDoS) - Scale to 1B pages - Efficient (crawl within a week) Core entities: -------------- - URLs - Text data - Domain metadata Interface: ---------- - input: seed URLs - output: text data Data Flow: ---------- 1. Take seed URLs from a frontier and the IP from DNS 2. Fetch HTML 3. Extract text 4. Store text in the database 5. Extract URLs from the page and add then to the frontier 6. Repeat steps 1-5 until all URLs are crawled