Data Engineer Resources

Related resources for Data Engineer

Feature Engineering in Machine Learning: Techniques and Examples
Ananya Desai
1w
Using the COPY INTO Command to Load Data from Fabric Lakehouse to Fabric Warehouse
Abiola David
1w
Disabling Activities in Microsoft Fabric Data Pipelines: A Practical Guide for Data Engineers
Abiola David
May 04
How Genie Code is Transforming Data Workflows in Databricks
Abiola David
Apr 24
Why Are Iceberg Tables Becoming the Default for Data Lakehouses Over Delta Lake?
Riya Patel
Mar 30
Databricks Runtime 18.1 (Beta) — A Big Leap Forward for Data & AI Teams
Abiola David
Feb 26
Shift-Left Data Engineering: Improving Data Quality and Reliability Early
Tanuj
Feb 23
What Are the Most In-Demand IT Certifications?
Saurav Kumar
Feb 19
What Is the Role of Databricks in Big Data?
Aarav Patel
Feb 19
What Is Snowflake and Why Is It Popular in Data Engineering?
Aarav Patel
Feb 19
Extracting Data From PDFs in Power Query of Power BI
Abiola David
Feb 16
Migrate Bulk Tables from BigQuery to SQL Database using Fabric Data Pipeline
Abiola David
Feb 13
Connecting Lakehouses to Fabric Notebooks and Reading Delta Tables
Abiola David
Feb 10
Converting String Dates to Real Date Types in PySpark
Abiola David
Feb 02
Data Engineering Design Patterns
Deepika Sawant
Jan 19
Microsoft Fabric: Azure DevOps Git Integration using Service Principal
Abiola David
Jan 07
Directly Import a Notebook Into a Microsoft Fabric Workspace
Abiola David
Dec 06
How to Import a Notebook into Your Databricks Workspace (Step-by-Step Guide)
Abiola David
Dec 06
Power BI Modeling MCP Server in VS Code
Abiola David
Nov 19
Automated Databricks Continuous Integration and Continuous Deployment using Azure DevOps
Abiola David
Nov 19
Across Tenants Data Integration Using Copy Job
Abiola David
Nov 04
New Fabric Shortcuts Transformations
Abiola David
Oct 23
Automating SQL Query Generation Using LlamaIndex and Snowflake
Rohit Gupta
Oct 21
Automated Databricks Notebook CI/CD Using GitLab
Abiola David
Oct 01
Bridging Old and New: Why IBM WTX Still Matters
Lokesh Varman
Sep 15
Difference Between MLE, Data Scientist, and Data Engineer
Riya Patel
Sep 12
Microsoft Fabric Data Pipeline Masterclass
Abiola David
Aug 04
Microsoft Fabric Warehouse Integration in VS Code for Devs
Abiola David
Jul 25
Fabric Lakehouse to Snowflake: Seamless Data Integration Pipeline
Abiola David
Jul 25
Microsoft Fabric: The All-in-One Analytics Platform for the Data-Driven Era
Pranshu Singhal
Jul 23
Microsoft Fabric Data Engineer: A New Career Pathway
Abiola David
Jul 21
Microsoft Fabric Lakehouse And How To Engineer Data Into The Lakehouse
Abiola David
Jul 16
Understanding Azure Synapse Analytics
Pranshu Singhal
Jul 13
Creating Fabric F64 Capacity: A Guide for Data Engineers
Abiola David
Jul 08
How Medallion Architecture Transforms Your Data Strategy
Lokendra Singh
Jun 21
Understanding Sharding for Scalable Data Systems
Lokendra Singh
Jun 20
DBT Masterclass Leveraging Fabric Warehouse [End-to-End Data Engineering]
Abiola David
Jun 19
What is DBT (Data Build Tool)?
Lokendra Singh
Jun 19
Databricks Notebook Git Integration Merge and Revert with Power BI
Abiola David
Jun 13
Deep Clone vs Shallow Clone in Databricks
Lokendra Singh
Jun 13
Automated Azure Data Factory CI/CD using GitHub Actions
Abiola David
Jun 11
Understanding Working of Catalyst Optimizer in PySpark
Lokendra Singh
1y
Orchestrate Databricks Jobs using Fabric Data Pipeline [End-to-End Project]
Abiola David
1y
Append Multiple Data from Excel using Fabric Data Factory
Abiola David
1y
How to Seamlessly Integrate Microsoft Dataverse with Fabric Using Fabric Link
Tharunkumar Magudeeswaran
1y
Microsoft Fabric Solving Real Challenges for Data Experts
Naveen Kumar
1y
Automating SCD Type 4 in Azure SQL Database with Azure Data Factory
Tharunkumar Magudeeswaran
1y
Implementing SCD Type 4 in Azure SQL Database: A Step-by-Step Guide
Tharunkumar Magudeeswaran
1y
DP700 Exam Prep, Row-Level Security in Fabric Warehouse
Abiola David
1y
Arrow-Optimized Python UDFs in PySpark: Boosting Performance
Lokendra Singh
1y
Real-Time Streaming Data in Databricks [End to End Project]
Abiola David
1y
Databricks Unity Catalog: Mastering Metastore and Namespace
Wilson Mok
1y
Prepare and Execute Cross Warehouse Queries in Fabric using T-SQL Notebook
Abiola David
1y
Fabric Data Factory: On-Prems Folder to Fabric SQL Database
Abiola David
1y
Azure Databricks Activity in Fabric Data Pipeline + Power BI
Abiola David
1y
Azure Key Vault to Perform Data Integration in ADF
Abiola David
1y
Implement Databricks Lakehouse Federation (BigQuery)
Abiola David
1y
How to incrementally load data from Microsoft Fabric Lakehouse to Warehouse
Abiola David
1y
Azure Data Factory Lakehouse Connector to Read and Write Data
Abiola David
1y
Change Data Capture Automated Incremental Data Loading in Azure Data Factory
Abiola David
1y
Exploring the Seven Experiences in Microsoft Fabric
Naveen Kumar
1y
How to Create Lakehouse in Microsoft Fabric
Varun Setia
1y
Azure Data Factory PRESERVE HIERARCHY Copy Behaviour
Abiola David
1y
An Introduction to Microsoft Fabric Fundamentals
Naveen Kumar
1y
Read Multiple CSV Files from GitHub Repository to Databricks Notebook
Abiola David
1y
Mirror Azure Databricks Unity Catalog in Microsoft Fabric
Abiola David
1y
Data Engineering in Microsoft Fabric
CSharp TV
1y
Windowing Enhancements New T-SQL Enhancement in SQL Server
Naveen Kumar
1y
TRIM Function New T-SQL Enhancements in SQL Server
Naveen Kumar
1y
GENERATE_SERIES Function New T-SQL Enhancements in SQL Server
Naveen Kumar
1y
Read/Write From Fabric Lakehouse to Databricks Notebook using ABFSS Protocol
Abiola David
1y
STRING_SPLIT() Ordinal New T-SQL Enhancements in SQL Server
Naveen Kumar
1y
Write Data from Fabric Dataflow Gen2 to Azure SQL Database
Abiola David
1y
Implementing Azure Databricks AutoLoader (Incremental Data Loading)
Abiola David
1y
Read Multiple PARQUET Files from GCS to Azure SQL Database using Azure Data Factory
Abiola David
1y
Definitive Guide to Mirror Azure SQL Database in Fabric Warehouse
Abiola David
1y
Advanced ADF: Integrate Multiple Category of Files Leveraging FILTER and other Activities
Abiola David
1y
ADF: Provision Snowflake Database Table in Azure SQL DB and Create Power BI Report
Abiola David
1y
What is Databricks Delta Live Tables (DLT)
Lokendra Singh
1y
SQL Data Analysis using Azure Synapse Built in Serverless Pool
Abiola David
1y
Read Parquet Data to Fabric Warehouse using COPY INTO & INSERT INTO SQL Statements
Abiola David
1y
Azure Databricks: Create Storage Credentials and External Locations to ADLS Gen2
Abiola David
1y
Azure Data Factory: Append and Convert Multiple CSV Files To Parquet File
Abiola David
1y
Azure Data Factory: Clean and Transform Multiple Data using Dataflow
Abiola David
1y
Using Jupyter Notebooks for Data Analysis and Visualization
Dhanapal Chandran
1y
Automated Incremental Load from Snowflake Warehouse to Fabric Warehouse [End-to-End Project]
Abiola David
1y
Azure Data Factory: Implementation of Blog Event Trigger for Data Movement
Abiola David
2y
Maximizing Big Data Potential with ADLS and PySpark
Lokendra Singh
2y
Orchestrate BigQuery Data to Fabric Lakehouse using Data Pipeline
Abiola David
2y
Azure Data Factory to Copy Data From GCS to ADLS Gen 2
Abiola David
2y
Fabric Lakehouse Shortcut to Google Cloud Platform Storage
Abiola David
2y
Microsoft Fabric: A Comprehensive Overview
Pratik Somaiya
2y
Medallion Architecture: A Framework for Data Organization
Pratik Somaiya
2y
A Simple Guide to Creating an Azure Databricks Workspace
Pratik Somaiya
2y
Tips to Launch Your SQL Mission
Pratik Somaiya
2y
Power of APIs in Data Engineering
Harunraseed Basheer
2y
Optimizing Data Processes with Staging Areas in Business Intelligence
Amira Bedhiafi
2y
Metadata-Driven Architecture in Data Engineering
Harunraseed Basheer
2y
Power of Large Language Models with Hugging Face
Harunraseed Basheer
2y
How to enable CoPilot in Microdoft Fabric Admin Portal
Abiola David
2y

Topics

Feature Engineering in Machine Learning: Techniques and Examples

Using the COPY INTO Command to Load Data from Fabric Lakehouse to Fabric Warehouse

Disabling Activities in Microsoft Fabric Data Pipelines: A Practical Guide for Data Engineers

How Genie Code is Transforming Data Workflows in Databricks

Why Are Iceberg Tables Becoming the Default for Data Lakehouses Over Delta Lake?

Databricks Runtime 18.1 (Beta) — A Big Leap Forward for Data & AI Teams

Shift-Left Data Engineering: Improving Data Quality and Reliability Early

What Are the Most In-Demand IT Certifications?

What Is the Role of Databricks in Big Data?

What Is Snowflake and Why Is It Popular in Data Engineering?

Extracting Data From PDFs in Power Query of Power BI

Migrate Bulk Tables from BigQuery to SQL Database using Fabric Data Pipeline

Connecting Lakehouses to Fabric Notebooks and Reading Delta Tables

Converting String Dates to Real Date Types in PySpark

Data Engineering Design Patterns

Microsoft Fabric: Azure DevOps Git Integration using Service Principal

Directly Import a Notebook Into a Microsoft Fabric Workspace

How to Import a Notebook into Your Databricks Workspace (Step-by-Step Guide)

Power BI Modeling MCP Server in VS Code

Automated Databricks Continuous Integration and Continuous Deployment using Azure DevOps

Across Tenants Data Integration Using Copy Job

New Fabric Shortcuts Transformations

Automating SQL Query Generation Using LlamaIndex and Snowflake

Automated Databricks Notebook CI/CD Using GitLab

Bridging Old and New: Why IBM WTX Still Matters

Difference Between MLE, Data Scientist, and Data Engineer

Microsoft Fabric Data Pipeline Masterclass

Microsoft Fabric Warehouse Integration in VS Code for Devs

Fabric Lakehouse to Snowflake: Seamless Data Integration Pipeline

Microsoft Fabric: The All-in-One Analytics Platform for the Data-Driven Era

Microsoft Fabric Data Engineer: A New Career Pathway

Microsoft Fabric Lakehouse And How To Engineer Data Into The Lakehouse

Understanding Azure Synapse Analytics

Creating Fabric F64 Capacity: A Guide for Data Engineers

How Medallion Architecture Transforms Your Data Strategy

Understanding Sharding for Scalable Data Systems

DBT Masterclass Leveraging Fabric Warehouse [End-to-End Data Engineering]

What is DBT (Data Build Tool)?

Databricks Notebook Git Integration Merge and Revert with Power BI

Deep Clone vs Shallow Clone in Databricks

Automated Azure Data Factory CI/CD using GitHub Actions

Understanding Working of Catalyst Optimizer in PySpark

Orchestrate Databricks Jobs using Fabric Data Pipeline [End-to-End Project]

Append Multiple Data from Excel using Fabric Data Factory

How to Seamlessly Integrate Microsoft Dataverse with Fabric Using Fabric Link

Microsoft Fabric Solving Real Challenges for Data Experts

Automating SCD Type 4 in Azure SQL Database with Azure Data Factory

Implementing SCD Type 4 in Azure SQL Database: A Step-by-Step Guide

DP700 Exam Prep, Row-Level Security in Fabric Warehouse

Arrow-Optimized Python UDFs in PySpark: Boosting Performance

Real-Time Streaming Data in Databricks [End to End Project]

Databricks Unity Catalog: Mastering Metastore and Namespace

Prepare and Execute Cross Warehouse Queries in Fabric using T-SQL Notebook

Fabric Data Factory: On-Prems Folder to Fabric SQL Database

Azure Databricks Activity in Fabric Data Pipeline + Power BI

Azure Key Vault to Perform Data Integration in ADF

Implement Databricks Lakehouse Federation (BigQuery)

How to incrementally load data from Microsoft Fabric Lakehouse to Warehouse

Azure Data Factory Lakehouse Connector to Read and Write Data

Change Data Capture Automated Incremental Data Loading in Azure Data Factory

Exploring the Seven Experiences in Microsoft Fabric

How to Create Lakehouse in Microsoft Fabric

Azure Data Factory PRESERVE HIERARCHY Copy Behaviour

An Introduction to Microsoft Fabric Fundamentals

Read Multiple CSV Files from GitHub Repository to Databricks Notebook

Mirror Azure Databricks Unity Catalog in Microsoft Fabric

Data Engineering in Microsoft Fabric

Windowing Enhancements New T-SQL Enhancement in SQL Server

TRIM Function New T-SQL Enhancements in SQL Server

GENERATE_SERIES Function New T-SQL Enhancements in SQL Server

Read/Write From Fabric Lakehouse to Databricks Notebook using ABFSS Protocol

STRING_SPLIT() Ordinal New T-SQL Enhancements in SQL Server

Write Data from Fabric Dataflow Gen2 to Azure SQL Database

Implementing Azure Databricks AutoLoader (Incremental Data Loading)

Read Multiple PARQUET Files from GCS to Azure SQL Database using Azure Data Factory

Definitive Guide to Mirror Azure SQL Database in Fabric Warehouse

Advanced ADF: Integrate Multiple Category of Files Leveraging FILTER and other Activities

ADF: Provision Snowflake Database Table in Azure SQL DB and Create Power BI Report

What is Databricks Delta Live Tables (DLT)