Real-time updates using SignalR and the 2017 Twitter Innovation Challenge

Introduction

One of the main components I need to integrate with a prototype I’m building for the Twitter Innovation Challenge 2017 is SignalR.

In this post,

• I outline what you can do with SignalR
• how you can implement it
• why you might want to use it
• touch on how I’m using it

Why?

With SignalR, you can send updates from the web server to any clients that have a connection.  It’s ideal as I need to push Tweets to an ASP.NET dashboard in real-time whilst connected to the Twitter Streaming API.

There is a great post here by Jordi Corbilla that goes into the details of a SignalR end to end solution.  It features an ASP.NET website and a WinForms application.

In his demo, you can run several instances of the WinForms client and send messages to the web server via SignalR.  The messages get displayed in the ASP.NET website in real-time without the need for a page refresh.  Alternatively, you might want to use Signal to broadcast messages from the server to clients.

Main Components

The main components used in SignalR are:

• Connections
• Hubs

Connections

These are simple endpoints and will be familiar to you if you’ve ever used WCF.

Hubs

Think of a Hub as a direct link that allows you to call methods between the client/server and vice versa.

Almost like they were local methods.

The Hub is one of the most important components to initially setup. You need to write a class that inherits from Microsoft.AspNet.SignalR.Hub.  Have a Google around and you’ll find some other examples of the finer detail.

Example

You can have client side script like the following:

$("#turnOn").on("click", function () {
        myTwitterHub.server.startTwitterStream();
    });

With a respective server side method:

public async Task startTwitterStream ()
var task = TwitterStream.InvokeStream();

await task;

Laying the foundations

In a previous post, I explained how to plug a console application into the Twitter Steaming API using C# to extract thousands of tweets a second.  The following components are what’s needed for this MVP at a high level:

• Middleware to leverage the Twitter Steaming API and extract data
• SignalR Hub
• Apply machine learning and natural language processing Secret Sauce ™ to refine data and generate Twitter Audiences
• JavaScript to consume the server side methods via the SignalR Hub
• Web application to display refined data

I already have a good chunk of the middleware written from last years’ entry which helps massively so can focus on refining the ML and NLP elements.

Data Processing and The Hub

In its simplest form, the Hub needs to initiate the Streaming API on the web server.  When this happens, the web application will maintain an “always on” connection.

Custom business rules in the middleware will then be applied to Twitter data being extracted via the Streaming API.

Look and Feel

Whilst I’m not a graphic designer, Bootstrap makes it easy to put together quick mock-ups to help visualise how one of the dashboards might look:

Here we can see a few panels for configuration, real-time feeds (the SignalR element) and dynamic Tailored Audience creation.  Or something I’m calling Just-In-Time Audiences, (JiT Audiences).

Next Steps

This involves completing the SignalR code and integrating this with the website mock-up.

Watch this space!