Nisal's Business Intelligence Blog

Learn, Grow and Share on Microsoft BI & Analytics Platform



Monday, October 16, 2017

Step-by-Step Twitter Sentiment Analysis Using Power BI Streaming Dataset, Microsoft Flow and Azure Text API

Sentiment Analysis is known as Opinion mining or emotion AI which is a branch of Natural Language Processing and text analytics where systematically identify, extract, quantify, and study affective states and subjective information. This kind a analysis widely apply to analyse the product or service reviews, voice of the customer, survey responses from online and social media feeds to analyze the attitude of the customer.

Basically from the sentiment analysis the output would be either Positive, Negative or Neutral. 
There are various algorithms and methods to do a sentiment analysis out there. In this post here I'm doing a sentiment analysis for iPhone 8 product by analyzing twitter feeds. Because, I wanted to know what others are thinking about the latest phone released by Apple. In order to do this task I'm using, 

Microsoft Azure cognitive services : Text Analytics (to run the sentiment analysis algorithms to get out the results) 
Microsoft Flow : to extract the twitter feeds and connect with Azure Text Analytics and insert data in to Power BI
Power BI Stream Analytics: To read the streaming data and visualize

Create a Power BI Streaming Data Set 

As the first step I need to create the Streaming data set in Power BI where I'm going to visualize the result at the end of this post. In order to do that I'm going to login to the Power BI service, and In my work space right top click the + Create and select Streaming data in the menu. 

Then select the API as the option to create the streaming data set. Here we are not using Azure streaming analytics or PubNub (which also a API for read stream data like data from IoT devices and sensors) to connect to the streaming data set. 

Then give a name for your streaming data set and add below fields in the row set. 

At the below switch the Historic data analysis On. This will enable your data set for both streaming and push data set. Read more on Power BI streaming data set creation in here 

Once you succeed with the streaming data set creation you will get a screen like below. 

You don't really to get the PushURL for this demo. I will show you how to configure this data set withing next couple of steps. 

One more additional step there to perform yet its important. You can Open your Power BI desktop and connect to the Power BI service and the stream data set you just now created. 

And select the Workspace and the dataset. In my case its 'Sentiment Analysis using Twitter'

I've created two calculated measures to the data set which are report level calculated measure.

One is for aggregate the Score. We can sum because its a Score which is vary between 0 and 1. (In the sentiment score we consider, 

If Score < 3 = Negative
3 < Score < 7 = Neutral
7 < Score = Positive

Other measure is for the Meaningful text value based on the score.

Because the score is a numerical value and end user might not understand the meaning of it at the first place.  Once you done the sketch of the report you can publish it to the service. We can do the additional seasoning to it once the data started to flow 😊

As I mentioned earlier you will need to create Azure Text Analysis cognitive service and invoke it to get the sentiment score. Lets create it now.

Create Azure Text Analysis Cognitive Service

Microsoft Azure Cognitive services toolkit contains bunch of tools which we can use to do the Machine learning and modern AI experiments. Image recognition, vision api, Natural Language processing few of them. Here we are going to create Text Analytics service. 

 Select the Text Analytics API and create.

Select the pricing model based on your requirement. For me I go with free tier. 

Fill the each field according to your requirement. Please, note that free tier only allows for 5000 transactions.

Check the "Pin to dashboard" and click the Create button in order to create. Once it created you will see it in the dashboard. You can walk through the menu bar if you want. 

You will need to Note down one of the API key to access this Text Analytics service from out side. 

We are done here with Text Analytics cognitive service configurations.

Extract Twitter Feeds, Detect Sentiment and Add Row Set to Power BI Streaming Dataset using Microsoft Flow

Now its time to login to site and create a flow by to extract twitter feeds, send those to to the Azure Text analytics service and the sentiment result add to the Power BI.

Go to Templates and Type "Twitter" and press enter to search twitter related templates.

Select the template 'Run sentiment analysis on tweets and push the results into Power BI'

Once you select the template it will create and go to the next step. In here you need to provide credentials to the twitter account where you scrap tweets, Azure Text Analytics API key which you are going to use to do the text mining and the power BI account credentials where you streaming data set resides. 

Click the Continue button at the below and let create the template flow for you. 

By default Twitter search as set to Power BI. I changed it to iphone 8 which is topic going to based for my sentiment analysis. 

As default Detect Sentiment flow also pre-configured to Tweet text. Only additional step you have to perform is to configure the Power BI streaming data set. 

Add your Power BI workspace as the Workspace and select the streaming dataset you created in the first step. Configure the other attributes as below by selecting from Dynamic content. 

Note: In this work flows you can add more than steps in the template. Even you can create your own flow from the scratch. 

Once you successfully completed the flow step will be like below screen.

Once you click the create button it will create the work flow and start to execute. 

Now everything is done and will go back to the Power BI service. You can see the report I created using the streaming data, which is coming with sentiment score analyzed by Azure cognitive service. I created a some extra report level calculated measures to get the count of Positive and Negative tweets.


This is just a Jump Start to Microsoft flow, Power BI streaming data and for Sentiment Analysis. I've not go deeper with any text mining algorithms and test the accuracy of the results. Text AI is some interesting area which we can apply these techniques directly in real world scenarios like evaluate the customer satisfaction. I hope you got some basic idea about what is Sentiment Analysis and how these tools can easily use to create your own Sentiment analysis solution without doing much coding 😊 

1 comment: