Guides
Practical, engineer-written guides to running Apache Spark and Apache Iceberg faster and cheaper, from execution internals to hardware acceleration.
Featured guides
What Is Hardware Acceleration for Data Processing?
How purpose-built hardware speeds up analytics and data pipelines, and what to look for when evaluating it.
Iceberg Compaction Explained: Bin-Pack vs. Sort
Why Apache Iceberg tables need compaction, how bin-pack and sort strategies differ, and how to choose between them.
Spark Data Skew: How to Detect and Fix It
Why a few oversized partitions can slow an entire Spark job, how to spot skew in the Spark UI, and the techniques that fix it.
Browse by topic
All guides
Spark Data Skew: How to Detect and Fix It
Why a few oversized partitions can slow an entire Spark job, how to spot skew in the Spark UI, and the techniques that fix it.
Iceberg Compaction Explained: Bin-Pack vs. Sort
Why Apache Iceberg tables need compaction, how bin-pack and sort strategies differ, and how to choose between them.
What Is Hardware Acceleration for Data Processing?
How purpose-built hardware speeds up analytics and data pipelines, and what to look for when evaluating it.
Understanding the Spark Shuffle: A Practical Guide
What happens during a Spark shuffle, why it is often the most expensive part of a job, and the settings that make it cheaper.
How to Right-Size Spark Clusters on Amazon EMR
A step-by-step approach to sizing Spark executors and EMR clusters so jobs finish on time without overpaying.
Apache Iceberg Table Maintenance Checklist
The recurring maintenance tasks that keep Apache Iceberg tables fast and storage costs under control.