A multi-module Spark-based data pipeline application built with Scala and Java.

• core - Core declarative data pipeline framework
• examples - Example implementations and sample jobs

# Build all modules sbt compile # Build specific module sbt core/compile sbt examples/compile

# Test all modules sbt test # Test specific module sbt core/test

# Run the simple example sbt examples/run

• Apache Spark 3.5.4
• Scala 2.12.19
• Java 17+

• core/ - Core framework with com.amazonaws.declarative.data.pipeline package
• examples/ - Sample jobs demonstrating the declarative data pipeline pattern
• Supports both Scala and Java components
• Includes unit tests for both languages