There are many projects all over the internet talking about sentiment analysis on Twitter. All of these projects are talking about creating a sense of tweets that have already been canceled. The data sets used by these projects are structured data sets that have been scrapped for a specific purpose.
But, what if you want to find the live sentiments for beginners (people who tweet are called tweeples) some celebrity or any event like, say, COVID at this moment?
In this article, we will see how to create a live Twitter sentiment analyzer using Tweepy, HuggingFace Transformers, and Streamlit. We will also publish this app with lit post and watch live emotion of any famous person.
- HuggingFace Transformers
- Application setup
- Building our app
Before we get our hands dirty, let’s, briefly, see some details about the different libraries we will be using in our application.
Tweepy is an open source Python package for accessing the Twitter API. With this package we can retrieve users’ tweets, retweets, status, followers, etc. We can also use this package to create Twitter bots that can post on our behalf. We can stream users’ tweets and check their live status. We can do a lot of things with Tweepy. You can check out the documentation for the full functionality of this Python package.
In our project, we will use this package to get live tweets based on a given search string.
hugging face is a company that intends to democratize artificial intelligence through open source. It contains all the natural language processing tools that can be used to analyze sentiment, generate text, and answer questions based on context. It has a hub of templates from which we can choose a template based on our app. Recently added a new feature called Spaces to host our apps built with Streamlit and Gradio.
Besides, it also provides a python library called converters. We will use this library in our project.
Streamlit is an open source Python package for building machine learning web applications in a very short time. Very easy to learn. It is best for machine learning models. We can also create visualization applications and dashboards with this tool. It also provides a way to publish our applications to its servers through Live Sharing. We can also publish our Streamlit apps to GCP, AWS, Heroku, or any platform we choose.
Twitter Live Sentiment Analyzer Setup
Our goal is to find live tweet feelings about a specific topic or person.
- We need to fetch tweets from Twitter based on our requirements.
- Look for the sentiments of these tweets.
- Show it to the user with the corresponding tweets.
So as a first step we get the tweets from Twitter using Tweepy and then we use huggingface adapters to find the feelings of those tweets and the user controls all this process through the interface we build with Streamlit to see the end result.
Let’s start building our app.
Building a Twitter Live Sentiment Analyzer
First, we install and import the necessary libraries. You can skip installing some libraries if you already have them installed on your system.
pip install tweepy
pip install transformers
pip install streamlit
pip install pandas
Now that we have installed all the necessary libraries, open your favorite VS or IDE code and create a .py file and import the installed libraries as below
import tweepy as tw import streamlit as st import pandas as pd from transformers import pipeline
We have imported all the necessary libraries
Now the first step we see in setting up the above app is to get tweets from Twitter. To do this, we have to access the Twitter API and use it to fetch the tweets. To access Twitter API, we need API Keys and API Key Secrets. To get it follow the steps below
- Create a Twitter account if you don’t have one
- Apply for a developer account on Twitter
- Once you have access to your developer account, start a project and get to instantiate the app
- This will generate your keys. Get “client key”, “client secret”, “access token” and “access token secret”
“Client Key” and “Client Secret” are nothing but “API Key” and “API Key Secret” respectively.
So if you want to create your own app, you should get your own values from Twitter.
A detailed explanation of getting these keys is explained in this youtube video. Please follow the steps in this video. If you do not understand the steps in this video or if you have any doubts about obtaining these values, comment below this article, I will reply to clarify your doubts.
Now that we have all the keys needed to access the Twitter API, we use the following Tweepy token to access it
consumer_key = 'type your API key here' consumer_secret="type your API key secret here" access_token = 'type your Access token here' access_token_secret="type your Access token secret here" auth = tw.OAuthHandler(consumer_key, consumer_secret) auth.set_access_token(access_token, access_token_secret) api = tw.API(auth, wait_on_rate_limit=True)
The above code establishes a connection between our app and Twitter.
Now that we’ve established a connection between our app and Twitter, let’s define an example sentiment analysis using the pipeline functionality for the face-hugging adapters we’ve already imported.
classifier = pipeline('sentiment-analysis')
Now that we have defined our example and established a connection between our app and Twitter, let’s give our app a title using Streamlit. You have used the name below, you can give your favorite name
st.title('Live Twitter Sentiment Analysis with Tweepy and HuggingFace Transformers') st.markdown('This app uses tweepy to get tweets from twitter based on the input name/phrase. It then processes the tweets through HuggingFace transformers pipeline function for sentiment analysis. The resulting sentiments and corresponding tweets are then put in a dataframe for display which is what you see as result.')
The above code gives a title and a simple description of what our app does.
using st.form(key=”enter name”):
search_words = st.text_input(‘Enter the name you want to know the feel of’)
number_of_tweets = st.number_input(‘Enter the number of recent tweets for which you want to know the sentiment (maximum of 50 tweets)’), 0,50,10)
Submit_button = st.form_submit_button (label = ‘submit’)
If the submit button:
tweets = tw.Cursor (api.search_tweets, q = search_words, lang = “en”). items (number_of_tweets)
tweet_list = [i.text for i in tweets]
p = [i for i in classifier(tweet_list)]
q =[p[i][‘label’] for i in range(len(p))]
df = pd.DataFrame(list(zip(tweet_list, q))), columns =[‘Latest ‘+str(number_of_tweets)+’ Tweets’+’ on ‘+search_words, ‘sentiment’])
Saint Writing (DF)
If __name __ == ‘__ main__’:
Explanation of above code
We need to take a name or phrase as input so we use st.text_input of Streamlit to get our desired name.
Similarly, we need to know for how many tweets we want to find sentiment and display them. Since this is a demo app that’s going to be deployed on a free hosting platform we have limited computing resources. So we limit the number of tweets to retrieve. In this case, it’s just 50.
we take number input using st.number_input to get the number of tweets.
Then we give a submit using st.form_submit_button.
If the user gives a search string and the number of tweets to be retrieved for sentiment analysis and press submit button, we need to get the tweets based on that input string and perform sentiment analysis on that, and then display them as a data frame.
The if code block does this job exactly.
Variable tweets stores the tweets that we fetched using the Tweepy library based on the user inputs. But the fetched tweets are in the form of Cursor objects. So we need to get the text of those tweets.
Variable tweet_list contains the tweets in text form.
The next variable p contains all the sentiment outputs using the classifier instance we defined previously.
Variable q stores the sentiment labels(positive or negative) of the tweets. We get them as shown in the above code.
In the next line, we create a data frame containing the extracted tweets and the corresponding sentiment labels.
Using the Streamlit command st.write(), we display the final result to the user.
Press ctrl+s to save the code in the .py file
So finally we built our app.
Testing the Live Twitter Sentiment Analyzer
Testing our app is very simple. Open the terminal and type the following code
streamlit run your_app_name.py
This will open the app in your browser in a separate window and you can start giving inputs to the app to get the live emoticons.
Post a live sentiment analyzer on Twitter
Now that we’ve built and tested our app, it’s time to publish our app. We will publish it using a Streamlit post. Streamlit offers the option to host three apps for free on its servers. To publish an app using Streamlit sharing, you must create a file named requirements. txt
Create a text file with the name requirements. txt And write the libraries that we used in our application
tweepy streamlit pandas transformers tensorflow
Write the above in your requirements.txt file. If you notice that we wrote TensorFlow very. But we didn’t use it in our app. This is because streamlining needs one of the deep learning libraries to run the transformer library. So we also give this library as a condition.
After creating this file, create a new GitHub repository and upload the .py file we created above and also requirements. txt a file.
Go to the live streaming site and create an account if you don’t already have one. You can see the New Application button, click on it and then give the name of the GitHub repo you just created and click on Publish.
Streamlit will take care of the rest and display your app. The end result should look like below
Finally, we have built and published our app. Now you can type the name of a celebrity or any topic name like COVID etc and also choose the number of tweets for which you want to find emoticons and click send button. You will see a table with n number of recent tweets and their sentiments.
You can find the above application that I created in this link – Streamlit. Will you try.
So we built a live Twitter sentiment analyzer in a simple way. These types of apps are useful for businesses to see how their brands feel on social media so that they can make informed decisions or at least address issues their customers face. If you have any doubts about the above app, please comment below so I can clarify. Thank you!!
- Picture 1: https://huggingface.co/
- Picture-2: https://streamlit.io/brand