In this article, I will explain about creating a Linux-based Azure HDInsight cluster by using Azure portal.
Azure HDInsight is a fully managed spectrum with open source services in cloud which can be used to process massive amounts of data and get all the benefits of the broad open-source ecosystem with the global scale plus the highlights. We can use the open-source frameworks as Apache spark & hive, Kafka, etc.
So coming back to the point, HDInsight cluster is a distribution from the Cloud for Hadoop components. Hadoop refers to a type of cluster, which is mainly used in Map reducing and it is very easy to use. It requires less setup with required configurations, and it is very high in availability and reliability along with security, etc.
Prerequisites
- Azure portal login credentials.
Step 1
Log in to the Azure portal by using this
link
After logging into the Azure portal, we can see the dashboard. For creating a Linux-based cluster in HDInsight, go to Create a resource >> Analytics >> HDInsight.
Step 2
After selecting HDInsight, we need to provide some basic information for the mandatory fields, such as providing a unique name for the cluster.
After providing a unique name for the cluster, select cluster type for configuring the required settings. Now, select the cluster type as Hadoop from the cluster type dropdown and select the operating system as Linux and then "Select".
Further moving to the next step, we need to provide a cluster login password for the cluster login username.
Finally, we need to create a new resource group or we can use an existing one. Along with selecting the resource group, select the data location center of the cluster to be created and click Next.
Step 3
After we have configured the basic settings, we can move to another step for configuring storage settings. For that, select the primary storage type to Azure Storage from the dropdown and select Storage account >> Create a new storage account. Then, click Next.
Step 4
So far, we have configured both basic settings and storage settings. Next, we can see the summary of configuration is showing the configurations selected by us. Finally, click "Create". For creating a Linux-based cluster in HDInsight, the deployment may take up to 10 to 15 minutes.
Step 5
After successful deployment, we can see that the HDInsight has been created and the status is running. So, after that select the "Cluster dashboards" option.
It will be redirected to Ambari portal. Here, log into the portal by signing-in using cluster username and password.
After successful login, we can see the portal dashboard with running resources. Thus, we have created HDInsight in this article. In my next article, we will get hands-on knowledge of Apache Hive with Hadoop.