Downloaded 47 times
Uploaded byTrieu Nguyen
Reactive Data System in Practice
This document introduces the Reactive Data System (RDS) framework called RFX for solving fast data problems reactively. It discusses how RFX was developed to handle common issues like counting pageviews, unique users, and real-time marketing. RFX is an open source, full stack framework that uses various tools like Kafka, Spark, and Redis to process high volumes of event data in real-time for applications like analytics, advertising, and monitoring. The document provides an example architecture and topology for collecting tracking data, processing it through RFX components, and generating reports.
Reactive Data System in Practice
- 1. Reactive Data System inPractice Or how to solve Fast Data Problem and reacting to the World faster By Triều Nguyễn at http://trieu.xyz http://www.rfxlab.com (Reactive Data System Lab) λ(x)
- 2. 2008: Java Developer,develop Social Trading Network for a startup (Yopco) 2011: joined FPT Online, software engineer, worked in FPT ID project and VnExpress Mobile (Backend) 2012: backend engineer at Greengar (Scaling server and real- time services) 12/2012 to 05/2015: back to FPT Online, lead engineer at eClick Ad Platform 06/2015 to now: lead engineer at iTVad, a startup project at FPT for OTT and TV Advertising Solutions about me
- 3. 1. Just someconcepts 2. Ideas in practice 3. What is RFX framework ? 4. Why is RFX ? 5. How to solve problems with RFX ? a. Counting pageview of website b. Counting unique user of website c. Querying data more reactive and adaptive 6. Final ideas Contents of this talk
- 4. 1 - Justsome concepts
- 5. Fast Data vsBig Data
- 6. What is ReactiveData System ? is a software design approach that uses the principles of event driven architecture for the design and implementation of data intensive applications. Event Driven Architecture EDA is a framework that promotes production, detection, consumption and reaction to the events. Event based applications will run on RDS framework.
- 7. Why is ReactiveData System We want to ask bigger questions faster
- 8. 2 - Ideasin practice
- 9. Problem (User Storyin Practice) ● User does login with FB Connect at Ecommerce website ● User does a click on an item, then system should do the following tasks: 1. Find best items to recommend 2. Find a best item to send email marketing 3. If there are more than 100 users, that click on a item, run a marketing campaign on Facebook automatically.
- 10. An example fromE-Commerce
- 11. 3 - Whatis RFX framework ?
- 12. ● A frameworkfor big fast data problems ● A collection of Open Source Tools from Netty to Spark ● The mission of RFX 1. Build data product quickly with design patterns 2. React to critical events in real-time What is RFX or Reactive Function X ?
- 13. RFX is inspiredby the Actor Model Theory
- 14.
- 15. RFX is designedfor startup business I design RFX because I have to solve following problems both at FPT and at RFXLab.com ● Marketing Automation ● Product UX personalization ● Real-time Advertising ● System monitoring (frontend and backend) ● Email marketing ● Product Analytics ● Customer Analytics → So I build a full-stack framework (a collection of tools for both backend and frontend)
- 16.
- 17. Sub modules inRFX framework
- 18. 4 - Whyis RFX framework ?
- 19. Why Rfx ? ●Ideas since 2012 (from Actor model theory) ● R&D and Deployed in Production since 2013 ● Open Source: Apache License, Version 2.0 ● Full Stack: from Frontend to Backend ● Agile for Data Analytics ● Lightweight Lambda Architecture ● Really fast and near-real-time processing ● Tested with 1.000.000 events / second ● Simple development model for data engineer
- 20. Roadmap to thefuture
- 21. 5 - Howto solve problems with RFX ?
- 22. Problems User story inDigital Marketing: 1. Counting pageview of website 2. Counting unique user of website 3. Real-time marketing
- 23. Find rush hourin your system
- 24. Which device userreach our content
- 25. Solution Architecture Tracking (RFX-track) Event Queue (ApacheKafka) Event Processor (RFX-stream) RedisMongoDB Synch Data Jobs (RFX-job) Report (RFX-report) Event Actor (RFX-actor) Ad Server
- 27.
- 28. 6 - Finalideas
- 29. Summary ● Fast Datais cool ● Fast data: The next step after big data ● RFX is designed to solve common Fast Data problems in Digital Marketing and Advertising
- 30.
- 31. The end andthank you https://github.com/rfxlab http://rfxlab.com λ