Download to read offline
Event Driven Architectures with Apache Kafka
- 1. Event Driven Architectures with ApacheKafka MATT MASUDA MMASUDA@GMAIL.COM @MATTMASUDA
- 2. Agenda OVERVIEW OF WEB- BASED ARCHITECTURES EVENTDRIVEN OVERVIEW ADVANTAGES & DISADVANTAGES BRIEF KAFKA OVERVIEW EVENT DRIVEN WITH KAFKA DEMO
- 3.
- 4. Web service issues • Theservice API is a form of coupling • Service calls are synchronous • All downstream calls must complete before your operation can finish • Load balancing and service location
- 5. Event Driven Architectures • Leveragesasynchronous messaging platforms • Maintains the micro in microservice • Each service produces and/or consumes events • Events are the contracts between services • Asynchronous operation is tolerant of temporary service disruptions
- 6.
- 7.
- 8.
- 9. Hard things about event driven •Transactions are right out • Getting your result out means polling or notification through other means (Websockets, SSE, SignalR) • Events are your contracts => design is important • Messaging APIs can be less mature than web frameworks
- 10. Kafka Basics • Itis a distributed messaging platform. • It lets you publish and subscribe to streams of records. In this respect it is similar to a message queue or enterprise messaging system. • It lets you store streams of records in a fault-tolerant way. • It lets you process streams of records as they occur.
- 11. Kafka Concepts • Kafkais run as a cluster on one or more servers. • The Kafka cluster stores streams of records in categories called topics. • Each record consists of a key, a value, and a timestamp. • Producers publish streams of records to one or more topics. • Topics can be divided into partitions for scalability. • Consumers subscribe to one or more topics and process the streams of records produced to them. • Consumers have control over their position in a topic. • Consumers can be grouped for scalability.
- 12.
- 13.
- 14. Event driven with Kafka Solvesservice location Solves load balancing Pub/sub model makes it easy to add new services to existing event streams Durable storage – you can keep a record of system behavior and trends Read positioning means you can reprocess old events Requires Kafka
- 15. Resources Confluent blog: Journeyto Event Driven Saga pattern - http://microservices.io/patterns/data/saga.html Cloud Native Patterns (Manning) Microservices Patterns (Manning) Designing Data-Intensive Applications (O’Reilly) Apache Kafka web site - https://kafka.apache.org/intro
- 16.