Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Unlock the full InfoQ experience

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources.

Don't have an InfoQ account?

Stay updated on topics and peers that matter to youReceive instant alerts on the latest insights and trends.
Quickly access free resources for continuous learningMinibooks, videos with transcripts, and training materials.
Save articles and read at anytimeBookmark articles to read whenever youre ready.

Logo - Back to homepage

News Articles Presentations Podcasts Guides

Topics

Development

Featured in Development

A Thirteen Billion-Year-Old Photograph

Kenneth Harris, a NASA veteran who worked on the James Webb Space Telescope, shares how hardware and software engineering combine to make the $13 billion observatory function. He explains how the technology mimics the human eye to process infrared light and reveals how software engineers use data packets to transform raw data into stunning, full-color images.

All in development

Architecture & Design

Featured in Architecture & Design

Building Distributed Event-Driven Architectures Across Multi-Cloud Boundaries

Teena Idnani explains how to architect and build resilient event-driven distributed systems in a multi-cloud reality. Using a fictional bank's migration journey, she shares practical, code-level solutions for overcoming major challenges: managing cross-cloud latency, ensuring event ordering and consistency, building resilience by design, and preventing duplicate events.

All in architecture-design

AI Infrastructure

Featured in AI, ML & Data Engineering

Deploy MultiModal RAG Systems with vLLM

Stephen Batifol discusses building and optimizing self-hosted, multimodal RAG systems. He breaks down vector search, nearest neighbor indexes (FLAT, IVF, HNSW), and the critical role of choosing the right embedding model. He then explains vLLM inference optimization (paged attention, quantization) and uses Mistral's Pixtral to detail multimodal large language model architecture.

All in ai-ml-data-eng

Culture & Methods

Featured in Culture & Methods

Exploring the Unintended Consequences of Automation in Software

This article lays out some of the common assumptions and misconceptions about automation and its role in software (and software incidents), what our research has found regarding how automation shows up in software incidents, and some ideas around how people can better design automated tools to help people better handle software incidents.

All in culture-methods

DevOps

Featured in DevOps

From Grassroots to Enterprise: Vanguard's Journey in SRE Transformation

Christina Yakomin shares Vanguard's SRE transformation: from quarterly testing of monoliths to a mature DevOps model with continuous delivery. She explains the SRE coaching hub, self-service tools, and advanced techniques like request-rate autoscaling. She details modern challenges, including region failure game days and testing AI-backed contact centers.

All in devops

Events

Helpful links

Choose your language

InfoQ Dev Summit Munich

Learn how senior devs at Mercedes-Benz, DKB & Zalando are solving critical dev challenges.

Last chance to register.

QCon San Francisco

Your team looks to you for what's next. Get the foresight to lead them through software's biggest shifts.

Early Bird ends Oct 14.

QCon AI New York

Trying to run AI at scale? Find the blueprints for enterprise AI from leaders who built them.

Early Bird ends Oct 14.

QCon London

Leadership asking for innovation? Get the evidence to bet on the right tech and lead with confidence.

Early Bird ends Oct 14.

InfoQ Homepage Spark SQL Content on InfoQ

Articles

RSS Feed

AI, ML & Data Engineering

Building Reproducible ML Systems with Apache Iceberg and SparkSQL: Open Source Foundations

Traditional data lakes are great for storing massive amounts of stuff, but they're terrible at the transactional guarantees and versioning that ML workloads desperately need. Apache Iceberg and SparkSQL bring database-like reliability to your data lake. Time travel, schema evolution, and ACID transactions help support reproducible machine learning experiments.

Anant Kumar
on Jul 31, 2025

Unlock the full InfoQ experience

Don't have an InfoQ account?

Topics

A Thirteen Billion-Year-Old Photograph

Building Distributed Event-Driven Architectures Across Multi-Cloud Boundaries

Deploy MultiModal RAG Systems with vLLM

Exploring the Unintended Consequences of Automation in Software

From Grassroots to Enterprise: Vanguard's Journey in SRE Transformation

Helpful links

Choose your language

Articles

Building Reproducible ML Systems with Apache Iceberg and SparkSQL: Open Source Foundations

A Thirteen Billion-Year-Old Photograph

Cursor 1.7 Adds Hooks for Agent Lifecycle Control

Myth Busters: is Rust a Slam Dunk?

11 Sessions Not to Miss at QCon San Francisco 2025

Building Distributed Event-Driven Architectures Across Multi-Cloud Boundaries

Uber Achieves 150M Reads per Second with CacheFront Improvements

Exploring the Unintended Consequences of Automation in Software

Using Brain Science to Communicate and Lead Technical Teams Effectively

Combining Continuous Delivery with Pair Programming: Lessons Learned

OpenAI Study Investigates the Causes of LLM Hallucinations and Potential Solutions

Claude Sonnet 4.5 Tops SWE-Bench Verified, Extends Coding Focus Beyond 30 Hours

PlanetScale Extends Database Platform to PostgreSQL

Azure Container Storage v2.0.0 Goes GA with Major Performance Boost

GitHub Adds Post-Quantum Secure SSH Key Exchange to Protect Git Data in Transit

From Grassroots to Enterprise: Vanguard's Journey in SRE Transformation

InfoQ Dev Summit Munich

QCon San Francisco

QCon AI New York

QCon London

InfoQ Software Architects' Newsletter

Articles