How to identify sales leads using Microsoft Cognitive Services

Introduction

This is post six (I think!) in a mini series where I’ve been documenting the build process for a solution I’m submitting to the Twitter Innovation Challenge 2017 #Promote.

In this quick post, I leverage Microsoft Cognitive Services to further refine the machine learning capabilities of last years’ submission to help identify sales leads.

The process is:

data in -> cognitive service-> determine sales lead (y/n) -> add to sales pipeline / discard

Which API did I try and why?

You can see what APIs are available here.  The one I chose to use is called LUIS (Language Understanding Intelligent Service).  With LUIS, you can build an application and “train” LUIS to understand specific phrases and or keywords.  These utterances can then be assigned to an intent.  In this use case, the intents are:

• Sales lead
• No sales lead

This reminded me of my Masters a few years ago and Bayesian Theorem whereby I wrote software that performed sentiment analysis on datasets from social channels to determine positive/negative emotion.

MS Cognitive Services shields you from complex algorithms and with a few clicks you can build your own classifier. Here are the main steps.

Creating a test application

The first thing you must do is login into the LUIS Dashboard, you can do this with your Microsoft Account here.  When you’re in here, you can Create a New Application, you can see this here:

You fill out a few of the fields and click Create, your application then gets added to your My Apps grid.

Intents

I need to determine commercial intent, for example, does the text being processed suggest the person wishes to buy / try something or not?

To represent this, I created two Intents.

• Sales Lead
• No Sales Lead

You can see these here:

Now that each Intent has been defined, I had to then define utterances that represent each intent.  For example, “I was thinking of getting”, “I’m not going to”.

Adding Utterances to Intents

In the screen shot below, you can see the intent “no sales lead” and utterances that have been defined as representing said intent.  You simply key utterances in, hit return and they get added to the intent.  It’s worth mentioning that a predicted intent also gets set.  This is set in the form of a probability, e.g. avoid it = 0.91 (high probability of NOT being a sales lead.

No Sales Leads

Sales Leads

Ditto for the sales leads intent.  Again, you can see the utterances in the grid and their respective probabilities.

Results

When the Intents and Utterances have been defined, you can test the new application!  It must be trained first however.  Cognitive Services make this easy and it can be done by clicking the Train Application button.  It only took a few moments for this prototype, naturally for larger and more complex datasets with rules, entities etc. I imagine it might take longer.

Testing the application is also easy and can be done from the dashboard.  You can see this in the screenshots below.  You type in your freeform text then hit return.  LUIS will then attempt to identify the intent that’s being expressed!

Sales Lead

“I might get a new iPhone”

LUIS has correctly identified that the top scoring Intent is sales lead.

No Sales Lead

“avoid the iPhone”

In the screenshot below, you can see that with 0.72 probability, this text has correctly been defined as not being a potential sales lead.  Maybe this string suggests a brand/product issue though which again might be a further insight as to how a brand or product is being perceived online.

Publishing the LUIS Cognitive Service

With the application built and tested it can then be published, this is simple and done from the LUIS dashboard.  When published, the LUIS services is exposed as a REST endpoint.  I’ve been using the Bootstrap Key which only permits 10000 queries a month, it’s not sufficient for processing large volumes of data but as an MVP it’s OK for now.

Consuming the Published Service

Now that the service is published it can then be used in my submission for the Twitter Innovation Challenge.  To further test the endpoint, I fired up Postman and supplied the URL generated by the LUIS dashboard.

I supplied the query parameter “I might get a “, the LUIS service was then successfully invoked and returned JSON containing the top scoring Intent, i.e. “sales lead”.  Of course, this could be my .NET windows service, web application etc.

Twitter Data

With these building blocks in place, it doesn’t take much work to integrate the Twitter API to extract tweets and identify long tail leads in the form of Twitter usernames.

These can then be used to generate tailored audiences that can subsequently passed onto the Twitter Ads API and served creatives.

Closing Thoughts

Microsoft aren’t the only providers of such services, IBM’s Tone Analyzer attempts to derive the underlying tone in steams of text which opens up further opportunities.

Feeling sad?  Suggest a holiday? Expressing anger? Suggest meditation?  This raises some ethical points which I’m not going into!

Finally

Next steps are to integrate LUIS with existing middleware I’ve built and isolate sales leads based on user configurations.