I was recently granted access to the Azure Open AI service and had a look around and play with some of the features.
In this overview, I share what you can expect to find in Azure Open AI Studio.
Other things include:
-
- how you can use Azure Open AI Studio
- using the Playground to experiment with Prompt Engineering
- automatically generating C# code within Azure Open AI Studio
- using the Azure Open AI Client Library with C#
- creating an Azure Function that uses Azure Open AI to explain in English what a regular expression does
A video demo is available on YouTube that shows the function in action.
Full code is available on GitHub.
Prerequisites
Before using Azure Open AI Studio, you need to create an Azure Open AI resource in the Azure Portal.
You can request access to Azure Open AI using the following form.
Concepts in Azure Open AI Studio
The following concepts and components form Azure Open AI Studio:
Playground
Let’s you can experiment with models and the completions and chat endpoints without having to code :
Deployments
Set up a deployment to make API calls against a provided base model or a custom model. Finished deployments are available for use:
Models
Many models are available. Each provide different capabilities and attract a different price point. At the time of writing, there are over 20:
Custom Models
Train a custom model to your specific use case and data for better performance and more accurate results:
Out of the box and custom models can then be deployed to the playground and evaluated.
Testing and Training Datasets
Upload multiple datasets from your local file system, Azure Blob, or other shared web resources to further train and fine tune any custom models you want to build :
~
Playground Examples
The playground ships with a range of prompt examples out-of-the-box and currently includes the following:
- Summarize issue resolution from conversation
- Summarize key points from financial report (extractive)
- Summarize an article (abstractive)
- Generate product name ideas
- Generate an email
- Generate a product description (bullet points)Generate a listicle-style blog
- Generate a job description
- Generate a quiz
- Classify Text
- Classify and detect intent
- Cluster into undefined categories
- Analyze sentiment with aspects
- Extract entities from text
- Parse unstructured data
- Translate text
- Natural Language to SQL
- Natural language to Python
- Explain a SQL query
- Question answering
- Generate insights
- Chain of thought reasoning
- Chatbot
I found the prompts below to be the most interesting and I’ll explain why (examples are taken directly from the playground for you to get a feel for the them).
Summarize Issue Resolution from Conversation
This will parse a conversational transcript and output a concise set of datapoints that give you an at-glance view of the main parts of the conversation. Useful for aggregating and reporting specific data that you might be interested in.
Example Prompt:
Generate a summary of the below conversation in the following format: Customer problem: Outcome of the conversation: Action items for follow-up: Customer budget: Departure city: Destination city:
The Transcript:
Conversation: User: Hi there, I’m off between August 25 and September 11. I saved up 4000 for a nice trip. If I flew out from San Francisco, what are your suggestions for where I can go? Agent: For that budget you could travel to cities in the US, Mexico, Brazil, Italy or Japan. Any preferences? User: Excellent, I’ve always wanted to see Japan. What kind of hotel can I expect? Agent: Great, let me check what I have. First, can I just confirm with you that this is a trip for one adult? User: Yes it is Agent: Great, thank you, In that case I can offer you 15 days at HOTEL Sugoi, a 3 star hotel close to a Palace. You would be staying there between August 25th and September 7th. They offer free wifi and have an excellent guest rating of 8.49/10. The entire package costs 2024.25USD. Should I book this for you? User: That sounds really good actually. Please book me at Sugoi. Agent: I can do that for you! Can I help you with anything else today? User: No, thanks! Please just send me the itinerary to my email soon.
Response from Azure Open AI:
Customer problem: Looking for a 15 day trip from San Francisco to Japan. Outcome of the conversation: Customer booked a 15 day trip to Japan at HOTEL Sugoi, a 3 star hotel close to a Palace, for 2024.25USD. Action items for follow-up: Send customer the itinerary to their email. Customer budget: 4000USD Departure city: San Francisco Destination city: Japan
Use Cases
Use this to parse large volumes of data often found in helpdesks, conversational AI and chatbot transcripts.
Consider streams of audio that has been processed using speech-to-text, leverage this kind of og prompt and capability to summarise a podcast or chapters of an audio book.
~
Classify Text
Use this prompt to classify text into categories of your choice. In this example, a news headline is supplied and classified.
Example Prompt:
Classify the following news headline into 1 of the following categories: Business, Tech, Politics, Sport, Entertainment Headline 1: Donna Steffensen Is Cooking Up a New Kind of Perfection. The Internet’s most beloved cooking guru has a buzzy new book and a fresh new perspective Headline 2: Major Retailer Announces Plans to Close Over 100 Stores
Response from Azure Open AI:
Headline 1: Entertainment Headline 2: Business
Use Cases
This has many use cases. I built dashboard in my own SaaS last year that performed a similar task but used out of the box, and predefined categories created by the Twitter API (known has context annotations).
You can see this here:
Context annotations were static though.
Creating prompts to classify text capability gives you complete control over the categories you need to classify.
~
Classify and Detect Intent
This is one of my favourite types of prompts. In this example, the model will parse a body of text to determine the department and underlying intent being expressed within the order:
Example Prompt:
For the below text, provide two labels one each from the following categories: - Department: “Books”, “Home”, “Fashion”, “Electronics”, “Grocery”, “Others” - Order intent Subject: Request for Refund of Recent Book Purchase Dear [Business Name], I am writing to request a refund for the books I purchased from your store last week. Unfortunately, the books did not meet my expectations, and I would like to return them for a full refund. I have attached a copy of the purchase receipt to this email as proof of purchase. The books are in their original packaging and have not been used, so I hope that the refund process will be straightforward. Please let me know what steps I need to take to return the books and receive a refund. I look forward to hearing back from you soon. Thank you for your attention to this matter. Sincerely, [Your Name]
Response from Azure Open AI:
Department: Books Order Intent: Refund
Use Cases
What’s exciting about this type of prompt are the sheer number of applications you can use it in. Use this to surface the underlying intent being expressed in data then take the relevant action.
Use this type of insight to reroute conversations in chatbots based on a customer intent change, parse data in a business process or workflow.
Alternatively, use this in marketing or ad-tech to help you identify when people are expressing commercial intent and take the relevant action.
This was something I done in the past and you can see this in the below screen shot:
A natural language processing model was created and trained in LUIS to detect potential sales leads.
When commercial intent was detected, the person was contacted shortly via an automation integration.
~
Question Answering
Use this type of prompt to let you query an existing body of text using natural language in Q and A style and return a concise answer.
Prompt and Example:
A neutron star is the collapsed core of a massive supergiant star, which had a total mass of between 10 and 25 solar masses, possibly more if the star was especially metal-rich. Neutron stars are the smallest and densest stellar objects, excluding black holes and hypothetical white holes, quark stars, and strange stars. Neutron stars have a radius on the order of 10 kilometres (6.2 mi) and a mass of about 1.4 solar masses. They result from the supernova explosion of a massive star, combined with gravitational collapse, that compresses the core past white dwarf star density to that of atomic nuclei. Answer the following question from the text above. Q: How are neutron stars created?
Response from Azure Open AI:
A: Neutron stars are created from the supernova explosion of a massive star, combined with gravitational collapse, that compresses the core past white dwarf star density to that of atomic nuclei.
Use Cases
Use this to help you implement QnA datastores with local, or online documents within your existing IT portfolio. Consider, legal documentation, property leases, or corporate documentation.
I have a demo of Custom Question Answering on YouTube that shows how similar technology allows you to ingest real-estate / property leases and use natural language to get concise answers tenants may have about the lease.
Find out more about this here : How To Using Natural Language To Query Documents With Azure Cognitive Services Text Analytics Custom Question Answering – Jamie Maguire
~
Adjusting the Dials and Code Integration
You can fine tune model parameters directly from the playground. It can be used to generate code in Python and C#. You can also choose to create a JSON payload to send to an Azure Open AI REST endpoint or even create a CURL command:
An example of the autogenerated code :
// Note: the Azure OpenAI client library for .NET is in preview.
// Install the .NET library via NuGet: dotnet add package Azure.AI.OpenAI --version 1.0.0-beta.5
using Azure;
using Azure.AI.OpenAI;
OpenAIClient client = new OpenAIClient(
new Uri("https://payg-we-openai.openai.azure.com/"),
new AzureKeyCredential(Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY")));
// If streaming is selected
Response<StreamingCompletions> response = await client.GetCompletionsStreamingAsync(
"DavinciPrototype",
new CompletionsOptions()
{
Prompts = { "Classify the following news headline into 1 of the following categories: Business, Tech, Politics, Sport, Entertainment\n\nHeadline 1: Donna Steffensen Is Cooking Up a New Kind of Perfection. The Internet’s most beloved cooking guru has a buzzy new book and a fresh new perspective\nCategory: Entertainment\n\nHeadline 2: Major Retailer Announces Plans to Close Over 100 Stores\nCategory:" },
Temperature = (float)0,
MaxTokens = 60,
NucleusSamplingFactor = (float)1,
FrequencyPenalty = (float)0,
PresencePenalty = (float)0,
GenerationSampleCount = 1,
});
using StreamingCompletions streamingCompletions = response.Value;
// If streaming is not selected
Response<Completions> completionsResponse = await client.GetCompletionsAsync(
deploymentOrModelName: "DavinciPrototype",
new CompletionsOptions()
{
Prompts = { "Classify the following news headline into 1 of the following categories: Business, Tech, Politics, Sport, Entertainment\n\nHeadline 1: Donna Steffensen Is Cooking Up a New Kind of Perfection. The Internet’s most beloved cooking guru has a buzzy new book and a fresh new perspective\nCategory: Entertainment\n\nHeadline 2: Major Retailer Announces Plans to Close Over 100 Stores\nCategory:" },
Temperature = (float)0,
MaxTokens = 60,
NucleusSamplingFactor = (float)1,
FrequencyPenalty = (float)0,
PresencePenalty = (float)0,
GenerationSampleCount = 1,
});
Completions completions = completionsResponse.Value;
~
Demo: Using Azure.AI.OpenAI Client Library, Azure Functions, and Postman
A further example of how similar code can be used is shown here.
The method ExplainRegex accepts a regular expression as a parameter and makes a called to Azure Open AI using the Azure Open AI Client Library:
public static async Task<string> ExplainRegex(string regEx)
{
OpenAIClient client =
new OpenAIClient(new Uri("https://payg-we-openai.openai.azure.com/"), new AzureKeyCredential("azure-key"));
// If streaming is not selected
Response<Completions> completionsResponse = await client.GetCompletionsAsync(
deploymentOrModelName: "DavinciPrototype",
new CompletionsOptions()
{
Prompts = { "Explain what the below regular expression does : " + regEx },
Temperature = (float)0.7,
MaxTokens = 250,
NucleusSamplingFactor = (float)1,
FrequencyPenalty = (float)0,
PresencePenalty = (float)0,
GenerationSampleCount = 1,
});
Completions completions = completionsResponse.Value;
return completions.Choices.FirstOrDefault().Text;
}
This method is called within an using Azure Function using a GET request.
You can see it being tested for the following regex in Postman: \b[m]\w+
The following output is returned:
“"\n\nThis regular expression matches words that start with the letter \"m\" and have at least one other alphanumeric character after it. The \\b indicates a word boundary and the [m] indicates that the first letter should be \"m\". The \\w indicates that the following character should be an alphanumeric character."
The Azure Open AI Client Library makes it simple for you to consume Azure Open in a language of your choice.
Video Demo
Further Reading and Resources
The following links are resources will help you get s started with Azure Open AI and prompt creation:
- What is Azure OpenAI – What is Azure OpenAI Service? – Azure Cognitive Services | Microsoft Learn
- Apply for Access to Azure OpenAI Request Access to Azure OpenAI Service (microsoft.com)
- What is Prompt Engineering Azure OpenAI Service – Azure OpenAI | Microsoft Learn
- Azure Open AI Client Library Azure OpenAI Library for .NET
Code examples, more content, and repos on generative AI can be found here.
~
Summary
In this First Look piece, you’ve learned about Azure Open AI Studio.
You’ve seen the various components, capabilities, and how these can be used to help you build AI models.
You also saw how to use a model in code with C#.
Drop any questions or feedback below !
Leave a Reply