Semantic Kernel: Function Calling and Planners

Note: This post covers Semantic Kernel, which I’ve since moved on from. My new work is built on Microsoft Agent Framework. If you’re starting fresh, I’d recommend the Microsoft Agent Framework: The Complete Series instead, or see the section What Does It Mean For Your Current Semantic Kernel project in this blog.

In an earlier blog post, we saw how to implement Native Functions and Plugins using Semantic Kernel.

A Native Function was contained within the Plugin and was invoked directly.

This isn’t how you’d normally use a Native Function.  If you wanted to invoke code directly, you’d create a regular class with a deterministic method and execute it.

When augmenting applications with AI capabilities using Semantic Kernel and language model, the Planner component in Semantic Kernel decides which Native Function to use.

Under the hood, the Planner makes use of Function Calling.

In this blog post we will learn about Function Calling and Planners.

Specifically, the following will be covered:

• What is Function Calling
• How Does Function Calling Work
• What is a Planner
• How Planners Use Function Calling

 

A code walkthrough, examples, and video demo are included.  The video demo shows you how to create a customer service agent that can handle a variety of tasks.

~

What is Function Calling

Function Calling belongs to chat completions and lets a model call external tools, functions or APIs based on user input.

Doing this optimises the model output and increases the probability of generating more accurate responses.

This lets the AI perform specific and relevant tasks for a given use case. It might be fetching data, interacting with another API or database backend.  This increases the usefulness of AI models and helps you create a more interactive experience and useful solution.

Key components involved during function calling include:

• Chat Completions: Used to generate responses in a conversational manner based on user input.
• Function Definitions: These are predefined or dynamically provided specifications of the functions that a model can call. Each specification/function has a unique identifier, input parameters, and expected outputs.
• Execution Context: The context within which the function is called, including the current state of the conversation and any relevant user input.

 

An overview of how function calling works now follow.

~

How Does Function Calling Work

The following steps detail each of the required steps that are normally involved to implement function calling using a model such as those available from OpenAI.

I have tried to group these for clarity:

Input

1. User supplies a prompt that is sent to the AI model

Analyse

2. The model analyses the prompt to determine the user’s intent.

Action

3. The model extracts necessary parameters from the user prompt.
4. The model checks if fulfilling the user’s request requires an external function call.
5. If a function call is required, the AI model identifies the appropriate function to call.
6. The model calls the external function or API with the extracted parameters.

Reply

7. The external function or API processes the request and returns a response.
8. The model uses the function response to generate a user-friendly reply.
9. The model sends the final response back to the user.

 

A lot of steps.  Some of these are repeated in a loop until a satisfactory response is arrived at and a termination signal is sent to the model.

Learn more about Function Calling from OpenAI here.

~

What is a Planner

I’ve touched on Planners in earlier blog posts in the series but it’s worth a recap for completeness.

Once you have multiple Plugins, you need someway to orchestrate and run them.  This is where the Semantic Kernel Planner comes into play.

Whilst not a particular class or code you invoke, the Planner automatically identifies which Plugins and Native Functions to invoke to satisfy a user request.

The planning of which Plugins and Native Functions to run for a given user prompt are automatically handled at run-time by Semantic Kernel and the LLM.

So, remember those Function Calling steps (1-9) from earlier that I grouped?

You don’t need to be concerned with those.

This frees you up to implement core business logic rather than implementing orchestration code between your application and an AI model.

~

How Do Planners Use Function Calling

The following sequence diagram shows what happens when you make a request to a model using Semantic Kernel and Automatic Function Calling:

(above: sequence diagram where the model has detected a native function should be invoked)

The above steps include:

1. All available functions (and any parameters) in the kernel are serialized using JSON schema.
2. The serialized functions (and the current chat history) are sent to the model as part of the input.
3. The model processes the input and generates a response. The response can either be a chat message or a function call
4. If the response is a chat message, it is returned to the developer to print the response to the screen. If the response is a function call, however, Semantic Kernel extracts the function name and its parameters.
5. The extracted function name and parameters are used to invoke the native function in your kernel.  In this above sequence diagram, this is the Native Function MakeBooking.
6. The result of the function is then sent back to the model as part of the chat history.

 

Steps 2-6 are then repeated until the model sends a termination signal.

You don’t need to code any of the above steps.  This is done automatically for you by Semantic Kernel.

Activating automatic Function Calling in Semantic Kernel is done by setting properties on the PromptExecutionSettings object as part of the ChatCompletionService object instantiation:

OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new() 
{   
  ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions 
};

 

Adding this instructs to kernel to activate the automatic planning process.  Any plugins and associated Native Functions you have written are discoverable by the kernel for use.

~

An Example

With the theory explained, its time for an example.  In this example, we will continue with the customer service agent from earlier blog posts.

The agent handles basic customer service queries related to bookings. A customer and manager can interact with the agent.

There are 2 plugins in this example:

Plugin	Native Functions
BookingPlugin	1.      GetNextAvailableDate 2.      BookAppointment
Complaints Plugin	1.      FileComplaint 2.      FetchComplaints

 

Basic business logic:

• A customer can use all functions within the BookingPlugin.
• A customer can file a complaint.
• A manager can fetch all complaints.
• A customer cannot fetch all complaints

 

Definition for the BookingPlugin:

public sealed class BookingPlugin
{
  // a list of available dates the user can bookprivate List<DateTime>     _available = new List<DateTime>{


  // add dates for the next 9 days, ignore the time part
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 1,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 2,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 3,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 4,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 5,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 6,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 7,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 8,0,0,0),
new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day + 9,0,0,0),
};

// a list of booked dates
private List<DateTime> _booked = new List<DateTime>();

[KernelFunction, Description("Determines the next available date for a booking")]
public DateTime GetNextAvailableDate([Description("The date to start searching from")] DateTime startDate)
{
  // get the next available date
  return _available.FirstOrDefault(d => d > startDate);
}


[KernelFunction, Description("Allows the user to make a booking.")]
public string BookAppointment([Description("The date of the appointment")] DateTime date)
{
  // check if the date is available, ignore the time part
  date = date.Date;


if (_available.Contains(date))
{
  // remove the date from the available list
 _available.Remove(date);

  // add the date to the booked list
  _booked.Add(date);

  // return a confirmation message
  return $"Appointment booked for {date.ToShortDateString()}";
}
else
{
  // return an error message
  return "The selected date is not available";
}
}

 

Definition for the ComplaintPlugin:

public sealed class ComplaintPlugin
{
  // a list of complaints

  private List<string> _complaints = new List<string>();

  // lets the customer file a complaint

[KernelFunction, Description("Lets a customer file a complaint")]
public string FileComplaint([Description("The complaint message")] string message)
{
  // add the complaint to the list
  _complaints.Add(message);
  // return a confirmation message
  return "Complaint filed";
}


// lets the manager fetch all complaints
[KernelFunction, Description("Lets the hotel manager fetch all complaints")]
public string FetchComplaints()
{
  // return all complaints
  return string.Join(Environment.NewLine, _complaints);
}

}

 

The definition for the agent is as follows:

static async Task Main(string[] args)
{

  Console.WriteLine("Launching Semantic Kernel Sandpit");

  var builder = Kernel.CreateBuilder()
  .AddOpenAIChatCompletion(modelId, apiKey);

builder.Plugins.AddFromType<BookingPlugin>();
builder.Plugins.AddFromType<ComplaintPlugin>();

Kernel kernel = builder.Build();

// Enable planning / automatic function calling
OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()
{
  ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
};

// Create chat history and system prompt/
var history =
new ChatHistory("You are professional customer service agent that handles bookings." +

"If the person says 'I am a manager', the agent will let them access " +
"fetch all complaints in the ComplaintPlugin.");

// Get chat completion service
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

// Start the conversation
Console.Write("User > ");

string? userInput;
while ((userInput = Console.ReadLine()) != null)
{
  // Add user input
  history.AddUserMessage(userInput);
  // Get the response from the AI
  var result = await chatCompletionService.GetChatMessageContentAsync(
  history,

executionSettings: openAIPromptExecutionSettings,
kernel: kernel);

// Print the results
Console.WriteLine("Assistant > " + result);

// Add the message from the agent to the chat history
history.AddAssistantMessage(result.Content ?? string.Empty);

// Get user input again
Console.Write("User > ");

}
}

 

Note the following lines:

builder.Plugins.AddFromType<BookingPlugin>();
builder.Plugins.AddFromType<ComplaintPlugin>();

// Enable planning / automatic function calling
OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()
{
  ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
};

// Create chat history and system prompt/
var history =
new ChatHistory("You are professional customer service agent that handles bookings." +
"If the person says 'I am a manager', the agent will let them access " +
"fetch all complaints in the ComplaintPlugin.");

 

This makes each plugin available to Semantic Kernel and the LLM for use.

Setting ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions instructs the kernel to allow automatic function calling and planning.

We define a persona and system prompt to ensure the agent honours the business rules that we want.

This is added to the ChatHistory before the first user interaction.

~

Testing the Scenarios

We can run the application and simulate the following scenarios:

• Scenario 1: Customer asking for the next available date
• Scenario 2: Customer making a booking
• Scenario 3: Customer filing a complaint
• Scenario 4: Customer attempting to fetch all complaints
• Scenario 5: Customer making a booking and asking to file a complaint
• Scenario 6: Manager fetches all complaints

 

We can see how Semantic Kernel, the internal planner, and the OpenAI LLM decide which Native Function to use in each of the above scenarios.

Scenario 1: Customer asking for the next available date

Here, we can see the kernel and LLM has identified the native function that checks against the list of available dates and returned a date:

Scenario 2: Customer making a booking

When asking the agent to make a booking, it will place a booking using the relevant native function:

We can ask the agent to make a booking for next week, it can handle that too:

Scenario 3: Customer filing a complaint

We can tell the agent we want to file a complaint:

 

The agent asks what is wrong, the agent acknowledges the customer complaint:

Scenario 4: Customer attempting to fetch all complaints

Here we can see how the agent responds if a customer tries to fetch all complaints:

Scenario 5: Customer making a booking and asking to file a complaint

We can tell the agent we would like to make a booking, but then also make a complaint:

Now, this is important, in the above screenshot, we can see the kernel planner and LLM has discovered multiple plugins (Booking and Complaint).

The respective native functions to satisfy the request have also been identified.  Multi-turn questions and answers have been processed.  No orchestration code has been written to do this.

It has been handled automatically.

This is one of the most powerful features of Semantic Kernel and the LLM IMHO.

Scenario 6: Manager fetches all complaints

In this step, we act as the manager and ask to list all complaints.  Here we can see the customers complain from scenario 5:

 

In these scenarios, we’ve seen how Semantic Kernel and automatic Function Calling, when paired with an LLM can handle multi-turn conversations.

Native Functions handle the discrete tasks and orchestration code had to be written.  Powerful stuff.

~

Video Demo

In this 9 minute demo, you will see the customer service agent in action.  You will see it:

• handle a booking
• process multiple intents within a single prompt
• keep track of open threads in a conversation
• asking the customer to follow up on an open thread
• determine and check if the customer has access to management capabilities

 

No orchestration code is written!  Semantic Kernel and the LLM figure out what to run and when:

Summary

In this blog post, we’ve looked deeper into Function Calling.  We’ve seen what steps are typically involved when using function calling with the OpenAI Completions service.

We took a closer look at the Planner concept in Semantic Kernel.  We saw how the kernel can use the LLM to help decide which native functions to run in order to satisfy a user request.

We saw several examples of this in action.  We also seen how specifying system rules can be defined for the agent by adding logic using natural language to the ChatHistory.

In the next blog post, we will dive deeper into how the orchestration is being done by exploring logging.

Stay tuned.

~

Further Reading

You can learn more about Semantic Kernel, Planners, and Function Calling here:

• Semantic Kernel –   Semantic Kernel documentation
• Plugins and Native Functions – Provide native code to your agents
• Planners – What are Planners in Semantic Kernel
• Function Calling – Function calling – OpenAI API
• Tool Call Behaviour – ToolCallBehavior.AutoInvokeKernelFunctions

 

Enjoy what you’ve read, have questions about this content, or would like to see another topic? Drop me a note below.

 

You can schedule a call using my Calendly link to discuss consulting and development services.

~