Azure Cognitive Services - Sentiment Analysis Tutorial

In this article, we’ll use various services provided by Azure such as Azure Cognitive Service, Azure Synapse Analytics, Azure Key Vault, Azure Spark Pool to perform Sentiment Analysis.  

Azure Cognitive Services 

Azure Cognitive Services is one of the services offered by Microsoft Azure that enables a wide range of organizations to build cognitive intelligence for applications with client library SDKs and REST APIs. Azure Cognitive Services allows developers to integrate cognitive features into applications with no prior knowledge of Machine Learning, Data Science, and Artificial Intelligence skillset. From Computer Vision to Natural Language Processing and Conversational AI, Azure Cognitive Services provides the support for a diverse prospect of applications. 

Sentiment Analysis  

Sentiment Analysis can be understood as the process of understanding the sentiment by extracting and identifying the underlying sentiment of subject information majorly from sources in form of text. This can be used across domains from Business Analytics, Brand Building, Monitoring the Sentiment in any field at any particular time and more. The Sentiment Analysis mainly uses text analysis, computational linguistics, and natural language processing. Opinions, Expressions and Emotions can be understood with the help of Sentiment Analysis from texts alone.  

Pre-requisites  

Before we start with this tutorial on Sentiment Analysis, we need to setup a few services in Azure. We would need an Azure Synapse Workspace, Text Analytics on Azure Cognitive Service (now called the Language Service), Azure Key Vault, Linked Services in Azure Synapse Analytics with Key vault and Azure Cognitive Service. Follow up the listed articles in order and make sure these services are well created before initiating the steps below for Sentiment Analysis.  

1. Azure Synapse Analytics – Create a Synapse Workspace 
2. Azure Synapse Analytics – Create Apache Spark Pool 
3. Azure Cognitive Services – Create Text Analytics Service for Natural Language Processing 
4. Azure Key Vault – Create Key Vault 
5. Azure Synapse Analytics – Link Services 

Once the above services are created and setup, we can now proceed with the steps below.  

Step 1 

Visit the Azure Portal. You’ll have the services running. Check out the Synapse Workspace you created and open it. Here mine is the ojashworkspace.  

Step 2 

In the Azure Synapse Workspace, you’ll have the Apache Pool running and the linked services connected with the Azure Key Vault, Azure Cognitive Service and Storage.  

Step 3 

Under Data in the Menu, Visit the Linked page. You’ll see your workspace under Data Lake Storage Gen2.  

Now, select the user and you can upload your data here to operate upon.  

Step 4 

Click on Upload and you’ll be taken to the upload files page. Here, we have the FabrikramComments.csv 

You can download the file from the github link.  

This data contains table with numerous columns on id, name, emailaddress, comment, spam core, and more. Here, we’ll use the sentences under Comment Column to function our Sentiment Analysis.  

Here, click on Raw. 

You'll be taken to another page. Now, save this as .csv file to your computer.  

Now, Click on Upload on Azure Synapse.  

We can see, the file has now been uploaded.  

Step 5 

Now, Right Click and explore as New Notebook > New Spark Table.  

You’ll be taken to a new notebook.  

Step 6 

Select the apachepool you created in Attach to. Now, you can add in the following query to save your table in the Apache Pool loading from the .csv file you just uploaded.  

%%pyspark
df = spark.read.load('abfss://[email protected]/data/FabrikamComments.csv', format='csv'
## If a header exists, uncomment the line below
, header=True
)
df.write.mode("overwrite").saveAsTable("default.YourTableName")

Step 7 

Next, refresh the Lake Database. You’ll see the new table you just created.  

Right Click it and explore as Machine Learning > Predict with a model. 

Step 8 

Now, here, we select the Sentiment Analysis under Pre-trained model.   

Click on Continue.  

Step 9 

Now, choose the linked service you created. Here, the Text Analytics – Language Service we created was named Cognitive Service 1. Next, under the Text Column to function upon the Sentiment Analysis, we select the Comment (string).  

Here, you need to realize, if you explore the data, you’ll see numerous Columns. We are operating the Sentiment Analysis on the Comment Section of the data.  

Step 10 

Next, Click on Open Notebook.  

Here, add in the following query. What this does is, it loads your data into the Spark DataFrame and operates Sentiment Analysis – specifically the Text Analytics and thus displays the result.  

Note: If in case, some errors pop up, you can easily scale your Apache Pool from the Scale Settings. I just went from 16GB Node size to 32 vCores, 256GB in seconds. But, we aware, it’ll cost you as per your usage increase in Node size.  Also make sure to select the Apache Pool too. 

Step 11 

Now, as you hit run, the job execution takes place. You can keep yourself updated with the job Process update from the Apache Spark Applications under Activities.  

Step 12 

Finally, we have the output in the table with the Sentiment Added as the analysis is done. Here, we can see the result of Sentiments from Positive, Negative and Neutral with various Confidence Scores for each comment sentences.  

Thus, we now have successfully performed Sentiment Analysis with our own uploaded data in Azure Synapse Analytics.  

Conclusion 

Hence, in this article, we learned about Sentiment Analysis and went through a step-by-step process to perform Sentiment Analysis in our own user uploaded data. Numerous other Azure Services came into play for us to attain this from the Azure Synapse Workspace, Azure Key Vault, Azure Cognitive Services, the Azure Data Lake Storage and Azure Apache Pool. Similarly, we can perform other machine learning analysis and predictions with different other offerings in Azure.