Elasticsearch: Performance Tuning Search Results using Boosts

I recently had to find a way to optimise search results being returned from an API that connected to an Elastic Search instance.

For context, ingested content is highly technical in detail.

The underlying model that was mapped and indexed to represent the content contained around 10 fields.

The key fields were content title and content body.

This data us made available in a variety of ways to users with the primary mechanism being a Web API with a search endpoint and some filters.

~

The Problem

Due to the nature of the content, search terms can exist the content title and body.

The API was returning valid search results however they were polluted with content that contained brief mentions or links in the content body with the search terms.

Whilst second-tier content did contain some of the input search terms, the content  was not a first class citizen.

~

An Example

The title for ingested content may be how to implement a line chart using the SDK.

The content body might contain a code example of how to do this.

However, it’s common for content to contain links to different types of content.

For example, the footer in each page may contain shared links such as how to implement bar charts.

If an API request is sent with the search term bar chart, the response would also contain the line chart content”.

Clearly that is not helpful.

~

The Solution: Using a Boost

The solution is to optimise the search results by using a Boost.

So, what is Boost?

 

A boost is a technique that lets you increase the relevance score of certain fields in your query.

 

This lets you add more weight to a record in search results and prioritise certain fields within content based on their subjective importance.

 

You can use boost to fine tune search results and provide a better user experience.

 

In the earlier example, the content title was an accurate description of the content that was ingested.  The body was still valuable but it’s really the title that we are interested in.

 

Therefore, I was able to apply a boost to the title field when the query was ran, thereby ensuring that only content that was truly related to the initial search terms was returned.

~

Implementing Boosts using the NEST SDK and C#

I’ve been using the NEST SDK to integrate Elastic Search with the solution.  To implement a boost, you annotate the field(s) with a relevant score from 1 to 10.  1 being the lowest score and 10 being the highest score.

 

You can see an example of the core code here:

var mainQuery = new Nest.QueryStringQuery
      {
          Query = $"*{query.ToLower()}*",
          AnalyzeWildcard = true,
          Fields = new[] { "title^10", "body^1"} // Boost title over body
      };

 

In the above code, the content in the title is being prioritised over content found in the body.

 

A full example of this in action can be seen here:

var queryContainer = new QueryContainer();
var result = new SearchResult<Content>();

// Query for the text
queryContainer &= new Nest.QueryStringQuery
{
    Query = "*" + query.ToLower() + "*",
    AnalyzeWildcard = true,
    Fields = new[] { "title^10", "body^1"} // Boost title over body
};

queryContainer &= new Nest.BoolQuery
{
    Should =
    [
        queryContainer,
    ],
    MinimumShouldMatch = 1 // Ensures that at least one of the conditions must match,
};

// Execute search query
var responsedata = await _elasticClient.SearchAsync<Content>(s => s
    .Source()
    .Query(q => queryContainer)
    .Size(topRecords)); // Limit the number of results returned to topRecords


// Order the documents by date created in descending order
var documents = responsedata.Documents.ToList();
var totalDocuments = responsedata.Total; // Total number of documents matching the query

result = new SearchResult<Content>
{
    Content = documents,
    PagingMetadata = new PagingMetadata
    {
        TotalDocuments = (int)totalDocuments,
        TotalPages = 1, // Since we're returning all records, there's only one page
        CurrentPage = 1,
        PageSize = documents.Count // PageSize is equal to the number of documents returned
    }
};

return result;

 

 

In the above example, the code builds a search query and prioritises title over the body.

It combines the query into a bool query and ensures at least one condition is matched.

It then executes the search, retrieves the top records and returns some additional meta data to help with paging on the client side.

~

Summary

In this blog post, we have seen how to fine tune and optimise search results using boosts within Elasticsearch.  We saw how to adjust the order and relevance of search results using the NEST SDK.