A few months ago, I started to experiment with the Twitter Labs APIs. I wrote blog posts about each of the endpoints, what you can do with them and how to use them with C#.
After the blogs were written, I created a NuGet package to make it easy for myself and other developers to consume Twitter Labs API endpoints.
At the time of writing, the NuGet package has been downloaded almost 400 times and you can use it to process Twitter data in just a few lines of code.
I’ve housed this NuGet package in Azure and used it to build a social listening component and backend services that can search for, ingest, and process Twitter data. Information is then displayed in an ASP.NET Core web application called Social Opinion.
This has been a side project of mine for a while now. I recently shared a short thread about the background to this tool. You can see a more detailed timeline of the project in an earlier blog post of mine.
At the time of writing Social Opinion is in public preview and in this blog post share:
- an introduction the main features
- how to use Social Opinion
- what you can do with Social Opinion
- sample use cases for the tool
I also share details to help you onboard and get access to more data to help you surface actionable insights.
Key Features
One of the core pillars of Social Opinion is the concept of listening and analytics. You create a Campaign; supply one or more queries then tell Social Opinion how many Tweets you want to fetch.
Dashboards
Dashboards are then populated which contain signals and datasets related to your campaign configuration. The following information is currently viewable:
- Number of Processed Tweets
- Most Prolific User
- Tweets with Most Likes, Comments and Retweets received
- Most/least discussed Person, Place, Organisation and Product or Service
- Most/least discussed Hashtag, URL or Username
- Number of People, Places, Organisations or Products and services being discussed
- Number of Hashtags, URLS, Users, Domains, Entities or Annotations being discussed
- What is forming the public conversation (Processed Tweets)
- Who is forming the public conversation (Audience Intelligence)
Clicking on each View link will take you to further information for the selected Tile. For example, selecting the tile Hashtags we can see hashtags that are being discussed in this campaign:
You have the option of performing a search on this data and can also see how many times that hashtag has been mentioned. This is a common theme for all tiles in each of the dashboards.
Clicking on View Tweets takes you to the Tweets that were processed for the Campaign and displays the Processed Tweets dashboard.
Processed Tweets
This dashboard lets you what is forming the public conversation. It contains all Tweets that were processed by your Campaign. You can search, filter, and sort this data:
Clicking on the User links take you directly to the selected user on Twitter. Clicking on the Text links will take you to the Tweet. Clicking on the column headers Replies, Retweets, Likes and Quotes sorts the data by the selected header.
Audience Intelligence
This screen lets you see who is forming the public conversation. It contains every user that has sent Tweets that were processed by your Campaign. You can search, filter, and sort the data.
Clicking on the Username field will take you to the user account on Twitter. Clicking on the column headers Followers, Following, Tweets and Listed will sort the dataset by the selected header.
An Example
With an overview of the main features and datasets you can examine, here is an end to end example of Social Opinion in action. I’ve kept the implementation as simple as possible. The first thing you need to do is create a new Campaign. You can do this by clicking on “Add New” Campaign:
Here you provide a description for the Campaign, a query, and the number of Tweets you want to fetch. The query, despite being only one field, let’s you apply powerful filters with the Twitter API.
In the above example, I’m telling Social Opinion to search for Tweets that mention “msbotframework” but to “exclude all retweets” (-is:retweet). This makes is easier to find signal in the noise. I recommend taking a quick read through this guide to give you a full overview of the rules you can apply in this one text box.
Finally, we set the Campaign to active and click Save. In about 15-30 minutes, Social Opinion will begin to process data as per your configuration.
You can check the status by click on Campaigns->View from the sidebar navigation. Upon doing so you are taking the Campaign List screen.
Campaign List
The Campaign List screen lets you view all Campaigns that you have created. When you’re Campaign has ran successfully and processed data, the fields Tweets Processed, and Last Run Date will contain values.
In the above image we can see that 2,300 Tweets have been processed for our “msbotframework” Campaign. Clicking on the link Dashboard takes you to the Campaign Dashboard for the selected Campaign.
Campaign Dashboard
The Campaign Dashboard is split into 3 sections:
- Overview
- Highlights
- Details
The Overview section contains high level information about the Campaign:
Here we can see that 2,398 tweets have been processed and there are some key metrics around Tweets that have been processed. You can also access the Processed Tweets and Audience Intelligence dashboards.
The Highlights section contains key metrics related to the Campaign such as the most discussed Person, Place, Organisation or Product:
The Details section contains more refined information that has been identified by your campaign:
Entities
Clicking on the View links takes you to further details about the selected tile. In our example, we can see the most mentioned Entity are related to “Technology and computing”:
There is some mention of MacOS and coffee. Not surprising a lot of developers turn caffeine into code! This is useful as we now know that people talking about “msbotframework” are also using JavaScript, MacOs and frequently talking about coffee and other “generic drinks”.
A further look at page 2 of this dataset shows us that education, AI and infosec are other niche topics being discussed by users in relation to “msbotframework”:
Tip: This information can help you better understand your audience and help inform your advertising or marketing campaigns.
Hashtags
Selecting the Hashtags Tile from the campaign dashboard shows us the most mentioned hashtags. Unsurprisingly, “MSBotFramework” is the most popular:
There are some others such as “Coding”, “CodingTips” and “Programmer”.
Tip: This is useful information to know. If you want to create and share content with a target audience of “msbotframework” developers. You can widen your reach by factoring in these additional hashtags.
Other dashboards that contain similar functionality but for different data points include:
- People, Places, Organisations and Products Identified
- URLs, Users Mentioned, Domains and Annotations
Audience Intelligence
In the Audience Intelligence dashboard, we can see the users that are forming the conversation. In this example for our “msbotframework” Campaign, we’ve sorted by the number of Followers. We can see the user “micadam” has the most followers. My personal account has quite a way to catch up!
Tip: Use this information to help identify influencers, product, or brand advocates. You can then directly contact them to build relationships or discuss collaboration opportunities.
Processed Tweets
We can view the underlying Tweet content that is being used to arrive at all our insights in the Processed Tweets dashboard:
From the above image, we can see that KoderJS and paulom has recently been Tweeting about “msbotframework”. This screen makes it easy for you to instantly search for content on Twitter as per your campaign configuration.
Tip: Use this information to help you easily identify to who forming and engaging with the public discussion or hot topics. Directly contact users or sort on fields such as Replies or Likes to help you decide which threads to purposefully engage with.
Use Cases
I’ve touched on some of the use cases in the tips that are sprinkled throughout this blog post. Some other examples can include, but are not limited to:
- analysts and academics
- developers
- digital marketers
- researchers
- social media managers
Social Opinion puts your Twitter Analytics on autopilot and makes it easy for you to segment and generate audiences for marketing purposes. You can quickly identify what and who is forming the public conversation.
Detailed metrics let you drill down into the most or least popular content alongside actionable insights to help you make data drive decisions – all in a few mouse clicks.
Current State
The platform is online on a development URL and open for public use. Simply login with your existing Twitter account, create your campaign and go! You can find it here.
If you want to upgrade and process more data, then drop me a message on Twitter. I can supply you with a test credit card to upgrade your account.
Next Steps
Twitter are due to ship V2 of their API. This is one of the biggest releases in years. When this happens, I’ll be migrating Social Opinion from the Labs endpoints to their production counterparts. The functionality and data we’ve just seen will be the same.
Summary
In this blog post we’ve introduced the public preview version of Social Opinion. I am happy to hear about any feedback from the community and open to feature requests at this early stage.
Leave a Reply