Dustin Vannoy

@UCYdC0t9EFtyVAs0-cwqVCTw - 6.5K subscribers

This channel is a resource for you to learn about modern data technologies and practices from a Data Engineer perspective, from kickstart tutorials to tips to go to the next level. Dustin Vannoy is a data engineer and analytics solutions architect. His specialties are modern data pipelines, data lakes, and data warehouses. He loves to share knowledge with the data engineering and science community.

Home Videos Live Playlists

Databricks AI Dev Kit: Install for Copilot + VS Code

Databricks AI Dev Kit: Install for Copilot + VS Code Dustin Vannoy

1.6K views - 3 weeks ago

Databricks AI Dev Kit Demo - Install, DataGen, SDP, Dashboard

Databricks AI Dev Kit Demo - Install, DataGen, SDP, Dashboard Dustin Vannoy

2.3K views - 1 month ago

Introducing Databricks AI Dev Kit - Skills, MCP server, Builder App

Introducing Databricks AI Dev Kit - Skills, MCP server, Builder App Dustin Vannoy

6.2K views - 1 month ago

AI-Driven Development

AI-Driven Development Dustin Vannoy

1.6K views - 1 month ago

Claude Code: 5 Essentials for Data Engineering

Claude Code: 5 Essentials for Data Engineering Dustin Vannoy

13.2K views - 3 months ago

Databricks + Cursor IDE: Step-by-Step AI Coding Tutorial

Databricks + Cursor IDE: Step-by-Step AI Coding Tutorial Dustin Vannoy

10.1K views - 6 months ago

dbt + Databricks Overview: SQL-based ETL

dbt + Databricks Overview: SQL-based ETL Dustin Vannoy

14.4K views - 10 months ago

Databricks VS Code Extension: Serverless Compute!

Databricks VS Code Extension: Serverless Compute! Dustin Vannoy

2.1K views - 1 year ago

Unity Catalog OSS Spotlight

Unity Catalog OSS Spotlight Dustin Vannoy

1.1K views - 1 year ago

Developer Best Practices on Databricks: Git, Tests, and Automated Deployment

Developer Best Practices on Databricks: Git, Tests, and Automated Deployment Dustin Vannoy

15.9K views - 1 year ago

7 Best Practices for Development and CICD on Databricks

7 Best Practices for Development and CICD on Databricks Dustin Vannoy

4K views - 1 year ago

Databricks VS Code: Multiple Projects In VS Code Workspace

Databricks VS Code: Multiple Projects In VS Code Workspace Dustin Vannoy

1.7K views - 1 year ago

Databricks VS Code Extension v2: Upgrade steps

Databricks VS Code Extension v2: Upgrade steps Dustin Vannoy

732 views - 1 year ago

Databricks VS Code Extension v2: Setup and Feature Demo

Databricks VS Code Extension v2: Setup and Feature Demo Dustin Vannoy

20.7K views - 1 year ago

Databricks CI/CD: Azure DevOps Pipeline + DABs

Databricks CI/CD: Azure DevOps Pipeline + DABs Dustin Vannoy

30K views - 1 year ago

Databricks Asset Bundles: Advanced Examples

Databricks Asset Bundles: Advanced Examples Dustin Vannoy

17.6K views - 1 year ago

Introducing DBRX Open LLM - Data Engineering San Diego (May 2024)

Introducing DBRX Open LLM - Data Engineering San Diego (May 2024) Dustin Vannoy

408 views - 1 year ago

Monitoring Databricks with System Tables

Monitoring Databricks with System Tables Dustin Vannoy

7.6K views - 2 years ago

Databricks Monitoring with Log Analytics - Updated for DBR 11.3+

Databricks Monitoring with Log Analytics - Updated for DBR 11.3+ Dustin Vannoy

6.4K views - 2 years ago

Databricks CI/CD: Intro to Databricks Asset Bundles (DABs)

Databricks CI/CD: Intro to Databricks Asset Bundles (DABs) Dustin Vannoy

41.3K views - 2 years ago

Data + AI Summit 2023: Key Takeaways

Data + AI Summit 2023: Key Takeaways Dustin Vannoy

646 views - 2 years ago

PySpark Kickstart - Read and Write Data with Apache Spark

PySpark Kickstart - Read and Write Data with Apache Spark Dustin Vannoy

1.4K views - 2 years ago

Spark SQL Kickstart: Your first Spark SQL application

Spark SQL Kickstart: Your first Spark SQL application Dustin Vannoy

1.9K views - 2 years ago

PySpark Kickstart - Your first Apache Spark data pipeline

PySpark Kickstart - Your first Apache Spark data pipeline Dustin Vannoy

7K views - 2 years ago

Spark Environment - Azure Databricks Trial

Spark Environment - Azure Databricks Trial Dustin Vannoy

716 views - 3 years ago

Spark Environment - Databricks Community Edition

Spark Environment - Databricks Community Edition Dustin Vannoy

1.5K views - 3 years ago

Apache Spark DataKickstart - Introduction to Spark

Apache Spark DataKickstart - Introduction to Spark Dustin Vannoy

3K views - 3 years ago

Unity Catalog setup for Azure Databricks

Unity Catalog setup for Azure Databricks Dustin Vannoy

17.1K views - 3 years ago

Visual Studio Code Extension for Databricks

Visual Studio Code Extension for Databricks Dustin Vannoy

20.3K views - 3 years ago

Parallel Load in Spark Notebook - Questions Answered

Parallel Load in Spark Notebook - Questions Answered Dustin Vannoy

2.9K views - 3 years ago

Delta Change Feed and Delta Merge pipeline (extended demo)

Delta Change Feed and Delta Merge pipeline (extended demo) Dustin Vannoy

2.5K views - 3 years ago

Data Engineering SD: Rise of Immediate Intelligence - Apache Druid

Data Engineering SD: Rise of Immediate Intelligence - Apache Druid Dustin Vannoy

267 views - 3 years ago

Azure Synapse integration with Microsoft Purview data catalog

Azure Synapse integration with Microsoft Purview data catalog Dustin Vannoy

2.8K views - 3 years ago

Adi Polak - Chaos Engineering - Managing Stages in a Complex Data Flow - Data Engineering SD

Adi Polak - Chaos Engineering - Managing Stages in a Complex Data Flow - Data Engineering SD Dustin Vannoy

215 views - 3 years ago

Azure Synapse Spark Monitoring with Log Analytics

Azure Synapse Spark Monitoring with Log Analytics Dustin Vannoy

6.1K views - 3 years ago

Parallel table ingestion with a Spark Notebook (PySpark + Threading)

Parallel table ingestion with a Spark Notebook (PySpark + Threading) Dustin Vannoy

17.2K views - 3 years ago

SQL Server On Docker + deploy DB to Azure

SQL Server On Docker + deploy DB to Azure Dustin Vannoy

5.5K views - 3 years ago

Michael Kennedy - 10 tips for developers and data scientists - Data Engineering SD

Michael Kennedy - 10 tips for developers and data scientists - Data Engineering SD Dustin Vannoy

232 views - 3 years ago

Synapse Kickstart: Part 5 - Manage Hub

Synapse Kickstart: Part 5 - Manage Hub Dustin Vannoy

98 views - 4 years ago

Synapse Kickstart: Part 4 - Integrate and Monitor

Synapse Kickstart: Part 4 - Integrate and Monitor Dustin Vannoy

403 views - 4 years ago

Synapse Kickstart: Part 3 - Develop Hub (Spark/SQL Scripts)

Synapse Kickstart: Part 3 - Develop Hub (Spark/SQL Scripts) Dustin Vannoy

318 views - 4 years ago

Data Lifecycle Management with lakeFS - Data Engineering SD

Data Lifecycle Management with lakeFS - Data Engineering SD Dustin Vannoy

386 views - 4 years ago

Synapse Kickstart: Part 2 - Data Hub and Querying

Synapse Kickstart: Part 2 - Data Hub and Querying Dustin Vannoy

385 views - 4 years ago

Synapse Kickstart: Part 1 - Overview

Synapse Kickstart: Part 1 - Overview Dustin Vannoy

347 views - 4 years ago

Scheduling Synapse Spark Notebooks

Scheduling Synapse Spark Notebooks Dustin Vannoy

2K views - 4 years ago

Why is my Azure bill so high?

Why is my Azure bill so high? Dustin Vannoy

359 views - 4 years ago

Data Engineering SD: Change Data Capture

Data Engineering SD: Change Data Capture Dustin Vannoy

433 views - 4 years ago

Azure Synapse: Automated Tests with FluidTest

Azure Synapse: Automated Tests with FluidTest Dustin Vannoy

2.3K views - 4 years ago

Synapse Spark: Add External Python Libraries

Synapse Spark: Add External Python Libraries Dustin Vannoy

6.1K views - 4 years ago

Azure Synapse Spark with Kafka - Installing JAR files

Azure Synapse Spark with Kafka - Installing JAR files Dustin Vannoy

1.8K views - 4 years ago

Azure Stream Analytics with Event Hubs

Azure Stream Analytics with Event Hubs Dustin Vannoy

9K views - 4 years ago

Data Engineering SD: Intro to Great Expectations by Allen Sallinger

Data Engineering SD: Intro to Great Expectations by Allen Sallinger Dustin Vannoy

403 views - 4 years ago

Data Engineering SD: Stream Processing Frameworks

Data Engineering SD: Stream Processing Frameworks Dustin Vannoy

620 views - 4 years ago

Querying Azure Log Analytics (with KQL)

Querying Azure Log Analytics (with KQL) Dustin Vannoy

21.7K views - 4 years ago

Azure Databricks Monitoring with Log Analytics

Azure Databricks Monitoring with Log Analytics Dustin Vannoy

20.1K views - 4 years ago

Spark Monitoring: Streaming

Spark Monitoring: Streaming Dustin Vannoy

4.7K views - 4 years ago

Spark Monitoring: Basics

Spark Monitoring: Basics Dustin Vannoy

20.4K views - 4 years ago

KSQL Aggregations (with updates and deletes)

KSQL Aggregations (with updates and deletes) Dustin Vannoy

361 views - 4 years ago

Spark Streaming Join with Delta Lake Table (Slow Changing Data)

Spark Streaming Join with Delta Lake Table (Slow Changing Data) Dustin Vannoy

2.7K views - 4 years ago

Azure Synapse Spark Python

Azure Synapse Spark Python Dustin Vannoy

4.4K views - 5 years ago