Getting Live Data From Twitter And Storing in Azure Blob Storage

In this section, I am going to talk about Logic Apps available in Azure. Here, I'll be pulling Live data from twitter and dumping the same in Azure Blob Storage. You can do tons of things with this data, means you can get this data and apply Machine Learning on the same and many more things. You can also setup a simple .NET app where you can project the twitter data analysis region wise.

By the way this is not the app talk. This is just to showcase how to setup Logic Apps upfront in Azure. As you can see in the following screenshot, I am in azure portal. This is Preview portal. Here, you can see that I already have different resource groups.

Apps upfront in Azure

resource group

summery

Then, I clicked on the browse button as shown below and searched for storage account and created the same with required stuffs. Here, I have chosen all the default settings.

default settings

standerd RAGRS

diagnostics

Recent

storage account

setting

Then, I setup the new container for the storage.

setup the new container

port

VHDS

After that, I have setup Blob Connector for Azure Blob firstly from the MarketPlace option with the container complete URI in there.

Marketplace

all resource

Azure Storage blob connector

package

Create azure storage

Search

deployeement

Now, I repeat the same steps, but this time I setup Twitter Connector. And, here I have also setup an app in Twitter API. Once, I created the Twitter API, I got consumer key and private key. The key I have mentioned below. And then after creating Twitter connector, I also copied the URL and pasted in twitter API in callback and site URL by editing the same.

Twitter API

package setting

twitter apps image

After completing above steps, I created Logic Apps as shown below in the screenshot.

Logic app

Create logic app

Here, I have selected the option, create from scratch. You can obviously select whatever template you want.

create from scratch

Twitter connector

twitter azure

Search tweets

Then, finally I have selected two more templates; Recurrence Template and Azure Blob Connector. Here, Recurrence Template is nothing but reflects the time-period. It is currently set for 1 hour which means, this job will run every hour and pull the data from Twitter. Second: Twitter command which I have set in the drop down.

/tweet_@{first(body('twitterconnector')).Tweeted_By}@{first(body('twitterconnector')).Tweeted_By}.txt. This will get Tweeted by data in the above format, then I have Azure Blob Connector, where this LIVE data is getting pushed.

resource

Azure Storage

Azure Storage blob

save app

Now, in the Logic app run section, when I clicked on the input link, it shows the query input. Here, you can see that I am querying #azure.

querying

And the following is the set of Live Data getting fetched from Twitter.

Data

Code

Thanks for joining me. I hope you enjoyed this post. We'll see more stuffs around the same in coming section. Till then, stay tuned and Happy Coding.