Archive - CanadianDataGuy’s No Fluff Newsletter

Optimizing Delta Lake Tables: Liquid Clustering vs. Partitioning with Z-Order

This guide will help you decide on your clustering methodology in Lakehouse

Oct 3 •

CanadianDataGuy

September 2024

What is Delta Lake? How does it work?

Dive into the inner workings of Delta Lake, from ACID transactions to time travel, and see how it combines the best of data warehouses and data lakes.

Sep 19 •

CanadianDataGuy

Build Your First Spark Streaming App with Stream-Batch Joins

Start your Spark Streaming journey with this beginner-friendly guide, featuring code for stream-batch joins and Delta table operations.

Sep 4 •

CanadianDataGuy

August 2024

I Spent 5 Hours Reading the Original White Paper on Spark and RDDs: Here’s What I Learned

Understanding the origins of Apache Spark and the power of in-memory data processing.

Aug 31 •

CanadianDataGuy

Upgrading Spark Stream with New Checkpoints

Step-by-step instructions on implementing new checkpoints in Spark Stream applications during major upgrades, complete with practical, executable code…

Aug 30 •

CanadianDataGuy

Boost Your Spark Streaming Skills with This Checklist

Dive into this step-by-step checklist for Spark Streaming, covering foundational tips and advanced techniques to enhance your data processing…

Aug 26 •

CanadianDataGuy

Kafka Explained: Key Concepts and Architecture Principles

Learn the fundamentals of Kafka, including its architecture, core components, and how it handles real-time data processing with high throughput.

Aug 24 •

CanadianDataGuy

May 2024

Top Strategies to Excel in 2024 Data Interviews

Discover how to prepare for data interviews with in-depth guides on SQL, Python, data modeling, and essential metrics for a successful data career.ccess

May 21 •

CanadianDataGuy

January 2024

Stream Anything on Databricks

Learn how to use Spark Streaming and Databricks Autoloader to efficiently process unsupported file formats, including a step-by-step guide and best…

Jan 4 •

CanadianDataGuy

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts