From Dev.to Community
Tech Articles
Curated developer articles, tutorials, and guides — auto-updated hourly
Sachin KamathJun 18, 2026 • 15 min read
Unlock Kafka Schemas with Karapace: A Hands-On Guide
TL;DR Kafka handles event streaming: producers write to topics, consumers read. JSON...
#dataengineering#apachekafkaschema#karapace#protobuf
1 1
nicolas.vbgh16h ago • 14 min read
Less Database, More Files
A few things I noticed. None of them alarming on their own. Our database doubles in size every...
#postgres#duckdb#database#dataengineering
1 0
Lê Đình Phú2d ago • 9 min read
How Uber Built Its Big Data System — From a Few TBs to 350 Petabytes with Sub-Hour Latency
A deep-dive into Uber's three architectural generations — from a Vertica warehouse to a Hadoop data ...
#bigdata#dataengineering#apachehudi#architecture
0 0
Muhammed Rasin O M4d ago • 6 min read
Synthetic Data for Data Engineering: How to test a Pipeline before the real data arrives
There is a quiet absurdity at the center of most data work, and once you notice it you cannot stop.....
#dataengineering#testing#python#databricks
0 0
Aditya Somani2d ago • 8 min read
Snowflake vs Databricks, BigQuery vs Redshift? The 2026 Guide to Right-Sizing Your Data Platform
TL;DR Big data platforms like Snowflake and BigQuery impose high pricing floors, like...
#analytics#architecture#database#dataengineering
0 0
Gabriel Henrique11h ago • 5 min read
Apache Iceberg in Production: Compaction, Catalogs, and the Pitfalls Nobody Warns You About
Apache Iceberg looked like the answer to everything when we first adopted it. Open format, ACID...
#dataengineering#data#python#tutorial
0 0
Marco Marques3d ago • 2 min read
I'm opening ContractForge — define data ingestion intent once, run it natively anywhere
Every data engineer who works across platforms knows this pain: You build a clean ingestion layer....
#dataengineering#datacontract#opensource#ai
1 0
Aditya Somani6d ago • 11 min read
An Engineer's Guide to DuckDB and Modern OLAP Databases
TL;DR Cloud warehouses are built for petabyte-scale enterprise needs, and for teams...
#analytics#database#dataengineering#serverless
0 0
Anatolii23h ago • 6 min read
Why I set `dynamic: false` on my Elasticsearch audit-log stream
Most Elasticsearch advice is about getting more out of it: better relevance, faster queries, smarter...
#architecture#backend#database#dataengineering
0 0
DataDriven1d ago • 7 min read
Your Data Engineering Take-Home Is Now 20 Hours of Free Work
Take-homes grew from 4 hours to 20. No pay, no feedback, AI banned with no rubric updates. The DE in...
#dataengineering#interview#career#beginners
0 0
Gabriel Henrique4d ago • 5 min read
Data Contracts in Production: Stop Trusting Your Upstream Sources
Your upstream data source changed a column type last night. Your pipeline ran at 2am, ingested...
#dataengineering#python#data#mlops
0 0
surajrkhonde3d ago • 20 min read
Phase 1: Document Ingestion - The Hidden Complexity Before Embeddings
The Complete Story: Why Most RAG Systems Fail Before They Start The Story...
#ai#dataengineering#llm#rag
0 0
Kanishga Subramani1d ago • 5 min read
Day 32: Configuring ClickHouse® Clusters with ClickHouse Keeper
As ClickHouse® deployments grow beyond a single server, ensuring high availability, scalability, and...
#clickhouse#analytics#database#dataengineering
3 2
Kanishga Subramani4d ago • 4 min read
Day 28 of 100 Days of ClickHouse® – Understanding TTL (Time To Live) for Automated Data Lifecycle Management
Introduction As data grows over time, storing every row forever becomes increasingly...
#clickhouse#database#dataengineering#analytics
3 0
Kanishga Subramani6d ago • 5 min read
Day 24 of 100 Days of ClickHouse: Working with the ClickHouse HTTP API
When most people think about interacting with a database, they imagine using a client library, a...
#clickhouse#devops#database#dataengineering
3 0
Kanishga Subramani3d ago • 4 min read
Day 29 of 100 Days of ClickHouse® - Understanding Distributed Tables
Introduction As datasets grow, a single ClickHouse® server may eventually become...
#analytics#clickhouse#database#dataengineering
2 0
Kanishga Subramani1d ago • 4 min read
Merges and Mutations in ClickHouse®
Introduction One of the reasons ClickHouse® delivers exceptional analytical performance is...
#clickhouse#devops#analyst#dataengineering
2 1
Kanishga Subramani1d ago • 4 min read
Day 33: Understanding ClickHouse® Query Execution Plans
Introduction When a query runs in ClickHouse®, the database does much more than simply...
#clickhouse#devops#analytics#dataengineering
2 0
Kanishga Subramani6d ago • 4 min read
Day 26 of 100 Days of ClickHouse: Understanding Distributed Tables
As your data grows, a single ClickHouse server may eventually reach its limits. Whether it's storage...
#clickhouse#devops#database#dataengineering
2 0
Kanishga Subramani5h ago • 4 min read
Day 36 – Best Practices for Choosing a Primary Key in ClickHouse®
Introduction When designing tables in ClickHouse®, one of the most important decisions...
#clickhouse#analytics#devops#dataengineering
1 0
Kanishga Subramani2h ago • 4 min read
Day 38: Optimizing Storage in ClickHouse® with Compression Codecs
Introduction One of the biggest reasons ClickHouse can analyze massive datasets so...
#clickhouse#devops#database#dataengineering
1 0
Kanishga Subramani22h ago • 5 min read
Day 34: Advanced ClickHouse® Aggregating Functions
Advanced ClickHouse® Aggregating Functions Introduction Aggregation is one of...
#devops#clickhouse#analytics#dataengineering
1 0
Kanishga Subramani2d ago • 5 min read
Day 31: Ingesting Data from Kafka into ClickHouse®
Introduction Real-time data ingestion is a fundamental requirement for modern analytics...
#clickhouse#analytics#database#dataengineering
2 0
Kanishga Subramani2d ago • 5 min read
Day 30 of 100 Days of ClickHouse® - Using ClickHouse® Arrays for Complex Data Structures
Introduction Modern analytical workloads often involve working with multi-valued and...
#analytics#clickhouse#database#dataengineering
2 0