Data Platform Orchestration: Databricks and Airflow
TODO: excerpt.
Posts
Life stuff. What I'm up to, or thinking about.
Dead Letter Queues
TODO: excerpt.
Should our dataplatform run with a GUI? Or a Library?
TODO: excerpt.
LLM-assisted Data Platforms development
TODO: excerpt.
Logging and Monitoring
TODO: excerpt.
Microsoft Fabric Workspace Strategies
TODO: excerpt.
Schema Change Management Strategies
TODO: excerpt.
Setting up CDC against SQL Databases
TODO: excerpt.
Spark Fundamentals: Using Distributed Compute Correctly
TODO: excerpt.
Cosmos Flink streaming lab
Azure Cosmos DB's change feed piped through Kafka into a Flink SQL job doing windowed per-customer aggregation. The point wasn't just getting it running — it's a working-through of schema evolution under Schema Registry compatibility constraints, retention as a recovery-time budget, and why teams replay from a durable source instead of leaning on a DLQ.
Notion
Syncs a Goodreads library export into a Notion database — parses the CSV, builds an ISBN13 lookup (falling back to title+author when Goodreads leaves it blank), and upserts each book against the existing Notion pages, paginating through the database and backfilling missing ISBNs along the way.