| | |

First Look: Integrating Semantic Kernel with OpenAI

Byjamie_maguire

Note: This post covers Semantic Kernel, which I’ve since moved on from. My new work is built on Microsoft Agent Framework. If you’re starting fresh, I’d recommend the Microsoft Agent Framework: The Complete Series instead, or see the section What Does It Mean For Your Current Semantic Kernel project in this blog.

What Is Semantic Kernel?

Semantic Kernel had been on my radar for a while.

Semantic Kernel is an open-source SDK designed to simplify the creation of agents that can interface with your existing code.

The SDK is extensible and supports models from OpenAI, Azure OpenAI, Hugging Face, and more.

Using the SDK means you don’t have to learn the inner workings of LLM provider APIs.

Semantic Kernel lets you build agents using C#, Python or Java.  Complex workflows and agentic AI orchestrations can be created using C#, Python or Java.

With the open-source SDK from Microsoft recently reaching V1, now is the time to dig into what this can do.

~

Orchestration Layer

Semantic Kernel sits at the centre of the agent stack.  It acts as the main orchestration layer for your agentic AI solution.

AI models, and plugins (your existing code) can be easily integrated to help you create innovative solutions and user experiences.

(image source Microsoft)

In the past, this involved manually stitching together multiple AI services and APIs such as LUIS, Bot Framework SDK or dialogs built using Bot Framework Composer.

Semantic Kernel, when integrated with LLMs, remove a lot of the manually plumbing and coding that was often required.

~

Who is Semantic Kernel For?

Semantic Kernel is aimed at developers who need to integrate AI models from OpenAI or Azure OpenAI etc. with existing code to create an end-to-end solution

~

Why Use Semantic Kernel?

In the last 24 months, we have seen an explosion in generative AI technology.  A simple and extensible framework is needed to help integrate these new models.  Using Semantic Kernel makes this process easier and quicker.

Semantic Kernel gives you a single place to leverage your existing code, cloud or locally running LLMs.

You could implement your bespoke solution or orchestration layer, naturally this would take longer, and you’ll have more control over the implementation.  The classic build V buy (except Semantic Kernel is open source!).

~

Key Concepts and Components in Semantic Kernel

Two of the main components that form the kernel are services and plugins.  Other key concepts include, plugins, planners, and personas.

Services

Services consist of both AI services and other services that are necessary to run your application (e.g., logging, telemetry, etc.).

Plugins

Plugins are code you have written that you want your AI agent to call or leverage within a prompt. For example, a plugin may call a 3rd party API to fetch information or load a record from a database. A plugin may also load previously saved memories from historical conversations.

(image source Microsoft)

Planners

To use a plugin, and to link them with other steps in a process a plan is required.

Planners are a special type of prompt that result in the generation of a plan to complete a specific task.

The simplest type of planner in a single prompt that helps the agent use function calling to complete a task.

Personas

You can influence how your agent interacts with users through a “persona”.  This is sometimes referenced as a meta prompt or system prompt.

The persona can modify the agent’s personality (e.g., friendly, sarcastic, helpful) and guide its responses in specific situations.

For instance, the persona can instruct the agent to ask for help if uncertain or to provide more detailed explanations to users.

ChatHistory

While models such as GPT from OpenAI lack the ability to retain chat history, Semantic Kernel offers a solution by providing Chat History functionality. This allows for answering questions based on previous context.

For example, users can request the chat application to share a story and then inquire about why it is humorous.

The response to the latter question will be connected to the initial story, showcasing the capability of chat history.

One useful way to setup the persona of your agent is to add a system prompt to the chat history before any interaction occurs.

You can see an example of this here:

ChatHistory chatMessages = new ChatHistory("
You are a friendly. You will complete required steps to the best of your knowledge. If the user doesn't provide enough information for you to complete a task however, you will ask clarifying questions until you have enough information to complete said task.");

~

An Example

With some of the basic theory explained, it’s time to look at a simple example. In this example, you will see the following:

  • Setup of a bare-bones kernel
  • Injecting the OpenAI chat completion service referencing a gpt-3.5-turbo model
  • Adding a meta prompt to chat history to affect the agent’s persona and reply style
  • Adding user prompts to chat history to let the current context reason over previous interactions

 

This example will be encapsulated within a .NET 8 console application and contain less that 100 lines of code.  The example does not use plugins (custom functions/code) or planners.

Open AI API Key and Model

First, we need to fetch the API key and model from the OpenAI developer portal.

These are used directly by Semantic Kernel when referencing the chat complete service:

Setting Up the Kernel

We setup the kernel, passing in the model id and api key and build it:

var builder = Kernel.CreateBuilder()
.AddOpenAIChatCompletion(modelId, apiKey);

Kernel kernel = builder.Build();

 

After setting up the kernel, we instantiate the chat history, passing in the system prompt:

// Create chat history

var history = new ChatHistory("You are a mischievous and jovial assistant. You will crack jokes when a user supplies a prompt.");

 

With the kernel settings defined and built. The ChatCompletionService is fetched from the Semantic Kernel service provider:

// Get chat completion service

var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

Initiating a Conversation

At this point, we have everything we need to start interacting with the agent.

A while loop is created to handle the conversation and let the user interact with the agent:

  // Start the conversation

  Console.Write("User > ");

  string? userInput;

  while ((userInput = Console.ReadLine()) != null)
  {
      // Add user input
      history.AddUserMessage(userInput);


      // Enable auto function calling
      OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()
      { 
      };


      // Get the response from the AI
      var result = 
      await chatCompletionService.GetChatMessageContentAsync(
          history,
          executionSettings: openAIPromptExecutionSettings,
          kernel: kernel);


      // Print the results
      Console.WriteLine("Assistant > " + result);


      // Add the message from the agent to the chat history
      history.AddMessage(result.Role, result.Content ?? string.Empty);

      // Get user input again
      Console.Write("User > ");
  }

 

For clarity, the entire code listing is as follows:

internal class Program
{
    private static string modelId = "gpt-3.5-turbo";
    private static string apiKey = "YOUR-API-KEY";


    static async Task Main(string[] args)
    {
        Console.WriteLine("Launching Semantic Kernel Sandpit");

        var builder = Kernel.CreateBuilder()
                .AddOpenAIChatCompletion(modelId, apiKey);


        Kernel kernel = builder.Build();
        // Create chat history
        var history = new ChatHistory("You are a mischevious and jovial assistant. You will crack jokes when a user supplies a prompt.");


        // Get chat completion service
        var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

        // Start the conversation
        Console.Write("User > ");
        string? userInput;
        
        while ((userInput = Console.ReadLine()) != null)
        {
            // Add user input
            history.AddUserMessage(userInput);

            // Enable auto function calling
            OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()

            {           
            };


            // Get the response from the AI
            var result = await chatCompletionService.GetChatMessageContentAsync(
                history,
                executionSettings: openAIPromptExecutionSettings,
                kernel: kernel);


            // Print the results
            Console.WriteLine("Assistant > " + result);

            // Add the message from the agent to the chat history
            history.AddMessage(result.Role, result.Content ?? string.Empty);

            // Get user input again
            Console.Write("User > ");
        }
    }
}

Interacting With the Agent

We can run the application and interact with the agent.

Asking the agent “how should I cook steak?” returns a cheeky response:

 

We can adjust the system prompt that gets added to the chat history to be more helpful rather than crack jokes:

// Create chat history
var history = new ChatHistory("You are helpful assistant.");

 

Running the application again, returns a more useful response:

And that’s it.

~

Video Demo

The following 6 minute YouTube video provides an overview of the blog and shows the agent in action:

~

Traditional Conversational AI v Semantic Kernel and LLMs

I have been designing and building conversational AI solutions in the Microsoft Cloud since 2016.

Historically, creating a simple agent could involve several components such as Bot Framework SDK or Bot Framework Composer for dialogue design, LUIS or CLU.

These would be stitched together to form an end-to-end solution. A large proportion of effort involving the design of dialogs and orchestration of how these interact with each other.

You can see one example of this here:

(traditional conversational AI design)

Some of my earlier blog posts discuss these concepts.

Semantic Kernel and LLM technology removes most of the need, effort, and overhead in terms of conversational design. Tasks such as creating dialogs to handle questions, training a language model to handle particular intents etc. are no longer required.

This reduces the amount of manual plumbing and services you need to integrate.

 

You still have the option of improving an LLM’s behaviour by implementing a pattern such as R.A.G (Retrieval Augmented Generation) to ground the experience with data unique to your use case.

Alternatively, can directly affect the orchestration of a LLM’s by using a technology such as Prompt Flow within Azure AI Studio.

More on these topics in future blog posts.

~

Summary

In this blog, we have seen how to setup Semantic Kernel with OpenAI and the ChatCompletion service.

We have also seen how to affect the agents persona by injecting a system prompt into the chat history.

The agent in this example and LLMs cannot interact with real-time data however.

In the next blog post we will see how to extend an AI agents behaviour by using plugins.

Stay tuned.

~

Further Information

You can learn more about Semantic Kernel and Open AI here:

 

Enjoy this blog post or have any questions?

Drop them below.

~

JOIN MY EXCLUSIVE EMAIL LIST
Get the latest content and code from the blog posts!
I respect your privacy. No spam. Ever.

Similar Posts

Leave a Reply