Big Data

Big Data

Work with big data end to end. Learn ingestion, distributed storage, Spark, batch and streaming, data lakes vs warehouses, governance, quality, cost control, and security. Design pipelines that deliver timely, reliable analytics at scale.

Ask a question or start a discussion

Article Video EBook

Big Data

What Is Big Data and How Does It Differ From Traditional Databases?

Nidhi Sharma

1w

356

1

Big Data

How to Use Apache Kafka for Real-Time Data Streaming Applications

Saurav Kumar

3w

2.8k

0

Big Data

What Is Data Streaming Using Apache Kafka and How Does It Work?

Ananya Desai

Mar 18

1.7k

1

Big Data

What Is Data Streaming Using Apache Kafka and How Does It Work?

Nidhi Sharma

Mar 18

2.8k

1

Big Data

What Practices Help Design Efficient Data Pipelines for Streaming Data?

Nidhi Sharma

Mar 13

419

1

Big Data

What Practices Help Design Efficient Data Pipelines for Streaming Data?

Niharika Gupta

Mar 13

2.5k

1

Big Data

How can developers design efficient data storage strategies for large datasets?

Nidhi Sharma

Mar 11

2.4k

0

Big Data

When to Use Spark or a Data Warehouse in Data Science

Riya Patel

Sep 10

981

0

Big Data

Big Data Explained: Importance, Tools, Challenges & Future

Rishima

Aug 19

807

1

Big Data

Apache Spark Cluster Mode Deployment

Lokendra Singh

Jun 29

8.1k

4

Big Data

Catalyst Optimizer vs Tungsten Optimizer: Choosing the Right Spark Engine

Lokendra Singh

Jun 27

7.6k

3

Big Data

Parquet vs Delta Format: Choosing the Right Data Storage Solution

Lokendra Singh

Jun 25

7.4k

5

Big Data

Coalesce vs Repartition in Apache Spark

Lokendra Singh

Jun 24

12k

4

Big Data

How Medallion Architecture Transforms Your Data Strategy

Lokendra Singh

Jun 21

9.7k

5

Big Data

Understanding Sharding for Scalable Data Systems

Lokendra Singh

Jun 20

3.9k

4

Big Data

What is DBT (Data Build Tool)?

Lokendra Singh

Jun 19

7.1k

4

Big Data

Managed & External Tables in Unity Catalog

Lokendra Singh

Jun 17

9k

4

Big Data

Unity Catalog vs Hive Metastore

Lokendra Singh

Jun 16

6.9k

6

Big Data

Deep Clone vs Shallow Clone in Databricks

Lokendra Singh

Jun 13

4.4k

5

Big Data

On-Heap vs Off-Heap Memory Management in Databricks

Lokendra Singh

Jun 12

2.5k

5

Big Data

Understanding Working of Catalyst Optimizer in PySpark

Lokendra Singh

Jun 10

2.7k

5

Big Data

Azure Synapse Analytics Serverless and Dedicated SQL Pools

Pavan Nalluri

1y

2.6k

3

Big Data

Arrow-Optimized Python UDFs in PySpark: Boosting Performance

Lokendra Singh

1y

2.1k

3

Big Data

Data Maturity Assessment: Where Does Your Company Stand?

Sagar Lad

1y

2.3k

3

Big Data

Glimpse of Apache Flink

Rajkumar Jain

1y

1.9k

3