Topics on social media change by the day. A hashtag that everyone is arguing about on Monday is gone by Thursday. That churn makes social media a hard thing to study by hand, and a natural thing to study with code. The article “Twitter Trends Analysis Using Python”, published on Towards Data Science, walks through one way to do it end to end: pulling tweets from the Twitter API, turning the raw text into a usable table, scoring how positive or negative each tweet is, and putting the results on a map.
The basic process
The post broke the workflow into three steps.
Collection of data. Tweets are fetched through the Twitter API. The important decisions here are filtering and sampling: which keywords or hashtags to search for, how many tweets to pull, and over what period. The choices made at this stage shape everything that follows, since a trend measured on ten thousand tweets from one afternoon is a different object from one measured over a month.
Processing of data. Raw tweets arrive as nested records with a lot of fields most analyses never use. The article stores the relevant fields, such as the text, timestamp, user, and any location information, in a data frame so they can be filtered, grouped, and counted like any other table. Text cleaning happens here as well: stripping links, mentions, and punctuation so that the words themselves can be compared.
Sentiment analysis. With clean text in hand, each tweet is scored for polarity, which is a number that says whether the language reads as positive or negative. Aggregating those scores over time or over regions shows whether the mood around a topic is shifting. The article also uses the location fields to map where tweets on a topic are coming from.
Why bother
The same pipeline serves a few different ends. In marketing or business it is a cheap way to learn how people are reacting to a product or campaign without running a survey. In research or journalism it gives a rough read of public opinion on an issue as it unfolds. And as a first project it touches most of the pieces a beginner needs to practise anyway: calling an API, cleaning text, working with data frames, and drawing a chart or map from the result.
The original article has the code for each step.
Source: Twitter Trends Analysis Using Python, Towards Data Science