DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Column-Oriented Databases: A Technical Overview

Column-Oriented Databases: A Technical Overview

Comments
6 min read
(Ⅱ) A Complete Guide to Core Data Warehouse Design Standards: From Layers, Types to Lifecycle

(Ⅱ) A Complete Guide to Core Data Warehouse Design Standards: From Layers, Types to Lifecycle

Comments
6 min read
ACID, Isolation Levels, and MVCC: Architecture and Execution in Relational Databases

ACID, Isolation Levels, and MVCC: Architecture and Execution in Relational Databases

2
Comments
10 min read
Why Data Partitioning Is Harder Than It Looks

Why Data Partitioning Is Harder Than It Looks

1
Comments
2 min read
How We Use OpenAI and Gemini Batch APIs to Qualify Thousands of Sales Leads

How We Use OpenAI and Gemini Batch APIs to Qualify Thousands of Sales Leads

1
Comments
7 min read
Mastering MLflow: Managing the Full ML Lifecycle

Mastering MLflow: Managing the Full ML Lifecycle

2
Comments
9 min read
Core Kafka Fundamentals for Data Engineering

Core Kafka Fundamentals for Data Engineering

1
Comments
14 min read
Why you need to learn Apache Airflow - right now

Why you need to learn Apache Airflow - right now

Comments
3 min read
🚀 How PySpark Helps Handle Terabytes of Data Easily

🚀 How PySpark Helps Handle Terabytes of Data Easily

Comments
2 min read
Apache Kafka Deep Dive: Concepts, Applications, and Production

Apache Kafka Deep Dive: Concepts, Applications, and Production

4
Comments
4 min read
🚀Git + Databricks: Why Both Are Essential for Modern Data Engineering

🚀Git + Databricks: Why Both Are Essential for Modern Data Engineering

3
Comments
2 min read
Scaling Databases with ClickHouse Sharding (Hands-On Simulation)

Scaling Databases with ClickHouse Sharding (Hands-On Simulation)

4
Comments
2 min read
(I) Principles of Data Model Architecture: Four Layers and Seven Stages

(I) Principles of Data Model Architecture: Four Layers and Seven Stages

5
Comments
7 min read
Composable Analytics with Agents: Leveraging Virtual Datasets and the Semantic Layer

Composable Analytics with Agents: Leveraging Virtual Datasets and the Semantic Layer

1
Comments
3 min read
Why Apache Airflow is the Cornerstone of Modern Data Engineering

Why Apache Airflow is the Cornerstone of Modern Data Engineering

4
Comments
5 min read
Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Comments 1
6 min read
🚀 The Future of Data Engineering: How AI and Automation are Changing the Game

🚀 The Future of Data Engineering: How AI and Automation are Changing the Game

Comments
2 min read
Apache Iceberg Dev List Digest August 25-29

Apache Iceberg Dev List Digest August 25-29

1
Comments
5 min read
The Blueprint of a Data Team: Roles, Responsibilities, and Specializations

The Blueprint of a Data Team: Roles, Responsibilities, and Specializations

2
Comments
10 min read
Wait, what? Ingestion into silver?

Wait, what? Ingestion into silver?

Comments
1 min read
Data Mesh: The Decentralized Revolution That Will Transform Your Data Architecture

Data Mesh: The Decentralized Revolution That Will Transform Your Data Architecture

Comments
4 min read
Event-Driven Architectures on AWS: Beyond Lambda

Event-Driven Architectures on AWS: Beyond Lambda

4
Comments
2 min read
🔄 ETL vs ELT: What’s the Difference and Why It Matters?

🔄 ETL vs ELT: What’s the Difference and Why It Matters?

Comments
2 min read
Two Years of Microsoft Fabric: Game Changer or Still Leveling Up? 🚀

Two Years of Microsoft Fabric: Game Changer or Still Leveling Up? 🚀

2
Comments
2 min read
What is the Modern Data Stack?

What is the Modern Data Stack?

1
Comments
3 min read
loading...