Building Event Streaming Microservices with Spring Boot and Apache Kafka | Jan Svoboda, Confluent

Building Event Streaming Microservices with Spring Boot and Apache Kafka | Jan Svoboda, Confluent

• 1.
  Building Event Streaming Microserviceswith Spring Boot and Apache Kafka Jan Svoboda, Solution Engineer, May 2021
• 2.
  Jan Svoboda https://www.linkedin.com/in/svobodajan/ • Confluent,Solutions Engineer 2020 - now • Event Streaming, Event-Driven Microservices • Apache Kafka, Kafka Streams • Pivotal, Platform Architect, 2016 - 2020 • PaaS, Microservices • Cloud Foundry, Spring, Rabbit MQ • IBM, Solutions Architect, 2011 - 2016 • DevOps, Agile • Bluemix, Rational
• 3.
  Agenda 3 Motivation My journey tolearning Kafka Spring Microservices Architecture Why Spring + Kafka = ♥ Events Streaming Microservices Concepts Look Ma No Hands! Event Sourcing and Data Liberation Removing Database Dependency Microservices Decoupling Integration Pattern State Caching Storing State of a Microservice Demo See it to believe it...
• 4.
  My Motivation forKafka and Microservices Why should a microservices developer care about learning Kafka?
• 5.
  Building Microservices akaCloud Native Applications Modernizing Legacy Applications with New Technology Stack • 3-Layer Model-View-Controller Pattern has been very popular last 15 years • Organizations look for migration shortcuts (no budget=no fun) • New Toy Syndrome (just buy new technology and we are done…) 5 App PaaS D B UI UI App Service Reg. LB My session from Cloud Foundry Summit https://www.youtube.com/watch?v=9moTgBiEauE
• 6.
  What is yourMicroservices Journey? 6 James Watters (Pivotal SVP), Kafka Summit 2019 Keynote, https://youtu.be/9I3CDfHKfNY?t=582 ● Stage 1 -> Strangling the Monolith ○ DB based integration ● Stage 2-> Provide Independent Data Sources ○ REST based integration ● Stage 3 -> Build Event Streaming ○ Kafka based integration
• 7.
  Spring ♥Kafka Like PeanutButter and Jelly...
• 8.
  Spring? Spring Kafka?Spring Cloud Services? 8 ● Spring Kafka -> provides a core Kafka libraries ● Spring Cloud Stream + Kafka Binders -> provides an abstraction for building Kafka pipelines ● Spring Integration -> provides Kafka as an integration adapter (EIP) https://spring.io/projects/spring-cloud-stream https://spring.io/blog/2015/11/29/how-not-to- hate-spring-in-2016
• 9.
  (Re)Building Spring based Microserviceswith Kafka Look Ma, No Hands!
• 10.
  Demo Application -Architecture and Dataflow • Spring Boot (2 jars) • Apache Kafka (2 topics) • Spring Cloud Streams + Kafka Streams Binder (RocksDB as a state store) Jan:Google,http://google.com Jan:Google,http://google.com JanGoogle: Google, http://google.com JanGoogle,http://google.com https://github.com/griga23/bookmarksDemo
• 11.
  Event Streaming Microservices ArchitecturePatterns Getting hands dirty...
• 12.
  1. Event Sourcing- Removing Database Dependency My Data Liberation ● From Data sourcing to Event sourcing ● From synchronous Request/Reply to asynchronous Publish/Subscribe ● Store the latest key/values in Kafka Streams State Store ● Use Kafka Streams Interactive Queries to query the State Store ● Make data free for everyone to read https://docs.confluent.io/platform/current/streams/developer-guide/interactive-queries.html =>
• 13.
  2. CQRS -Microservices Decoupling Command Query Responsibility Segregation ● Pattern separates read and update operations for a data store. ● Implementing CQRS can maximize performance, scalability, and security. ● Allows a system to better evolve over time and prevents update commands from causing merge conflicts at the domain level. https://docs.microsoft.com/en- us/azure/architecture/patterns/cqrs
• 14.
  3. Scaling –Stateful Microservices • Multiple State Stores with different partitions • HTTP REST calls to provide State Stores inter communication Jan:Google,http://google.com Jan:Yahoo,http://yahoo.com JanGoogle:Google,http://google.com JanYahoo:Yahoo,http://yahoo.com JanGoogle:Google,http://google.co m JanYahoo:Yahoo,http://yahooo.co m https://docs.confluent.io/platform/current/streams/developer-guide/interactive-queries.html#querying-remote-state-stores-for-the-entire-app
• 15.
  4. Reverse Proxy– Routing to the Right State Store https://medium.com/bakdata/queryable-kafka-topics-with-kafka-streams-8d2cca9de33f • Spring Cloud Gateway for incoming HTTP traffic to redirect to proper State Store • State Store Host Awareness Jan:Google,http://google.com Jan:Google JanYahoo:Yahoo,http://yahoo.com
• 16.
  Demo https://github.com/griga23/bookmarksDemo
• 17.
  #CONFLUENTSKO2021 Thank you! jsvoboda@confliuent.io