structured-streaming

Here are 8 public repositories matching this topic...

aws-samples / iceberg-streaming-examples

This repo contains examples of high throughput ingestion using Apache Spark and Apache Iceberg. These examples cover IoT and CDC scenarios using best practices. The code can be deployed into any Spark compatible engine like Amazon EMR Serverless or AWS Glue. A fully local developer environment is also provided.

apache-spark structured-streaming apache-iceberg

Updated Jul 2, 2025
Java

Neuw84 / structured-streaming-avro-demo

Star

Spark 3.0.0 Structured Streaming Kafka Avro Demo

java spark spark-streaming structured-streaming

Updated Apr 21, 2023
Java

chenyyyang / spark-sql-custom-mq-dataSource

Star

Spark 3.1.x实现MQ Source的方法

mq structured-streaming spark-sql datasourcev2 spark3

Updated Feb 27, 2022
Java

Neuw84 / bds2k17

Star

Repository containing code for the Big Data Spain 2017 technical talk "Towards an Unified API for Spark and the IIoT" Edit

docker kafka spark cassandra nifi zeppelin structured-streaming

Updated Nov 15, 2017
Java

shaunmcglinchey / tweetpipe-spark-analyser

Star

This repository contains an alternative Apache Spark-based analysis tier for the TweetPipe streaming data pipeline

java streaming kafka apache-spark spring-boot lombok stream-processing apache-kafka tweet streaming-data structured-streaming

Updated Aug 23, 2020
Java

rezacsedu / Mining-Maximal-Frequent-Pattern-Spark

Star

Implementation of Static mining part of "Mining maximal frequent patterns in transactional databases and dynamic data streams: A spark-based approach" Information Sciences, Volume 432, March 2018, Pages 278-300

java data-mining spark data-stream frequent-pattern-mining structured-streaming maximal-frequent-pattern