Step 1: Data SourcesStep 2: Integration and Data StorageStep 3: Data Models and AnalyticsStep 4: Visualization and Reporting.
Loading
Step 1: Data SourcesStep 2: Integration and Data StorageStep 3: Data Models and AnalyticsStep 4: Visualization and Reporting.
Data ingestion, the process of collecting or streaming information from various sources like log files, social media files, and SQL databases. It encounters three important challenges while ingestion Large table ingestion, Schema changes in the source and Change data capture
To Load or store the extracted data in HDFS or NoSQL by the HBase. It can be easily accessed and processed by applications.
This is an actual process of getting a bigdata solution. To analyze big data sets at terabyte or even petabyte-scale by MapReduce or Spark framework.