Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

A lightweight CLI tool for versioning data alongside source code and building data pipelines.

Beneath is a serverless real-time data platform ⚡️

ops0 is an AI-powered natural language DevOps CLI native to Claude AI with ansible, terraform, kubernetes, aws, azure and docker operations in a single cli. An open-source alternative to complex DevOps workflows, manual operations, etc. 🤖 ⚡ 👉 Natural Language DevOps Automation & Troubleshooting Tool

A high-performance, extremely flexible, and easily extensible universal workflow engine.

This open-source Terraform provider enables users to seamlessly integrate the Monte Carlo data reliabillity platform into their infrastructure as a code (IaC) workflows.

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

GlassFlow CLI to create and manage real-time data pipelines

A simple data processing pipeline supporting FIFO, fixed & dynamic worker pools, and broadcast stages.

Lightweight data streaming application that monitors SQL Server CDC-enabled tables for changes and streams events to various output destinations. Ideal for real-time analytics, event-driven architectures, and seamless integration with cloud-native workflow

Ministream is a small, stand-alone, real-time event messaging streaming server

Kubernetes-native data pipeline platform and orchestration

This project implements an ETL (Extract, Transform, Load) pipeline in Go for ingesting cryptocurrency market data from the CoinGecko API.

A set of plugins (mappers, sinks, etc.) for Numaflow pipelines

CLI Application holding a sentiment analysis data (Twitter tweets) pipeline with its own Web API to query results in the database. Written entirely in Go.

Data-pipelining in Go

Sigzag is an observability utility and backend service for datlin and is used to monitor, sign and log data pipeline transactions.

Playing with Apache Beam Tour: https://tour.beam.apache.org