Elastic Search Cheat Sheet

Ive been working with Elastic Search recently.

This post is more for my own refence when performing common tasks when creating code to integrate with Elastic Search using the Nest SDK.

You might find it useful too.

The following are covered:

• Creating an index
• Making a connection
• Adding a document to an Index
• Searching for documents
• Searching for documents using a parameterized field and value
• Updating a document
• Helpful REST API endpoints

~

Creating an Index

How to create an index for storing documents:

 public async Task<CreateIndexResponse> CreateIndexAsync(string indexName)
        {

            // check if index alread exists

            var exists = await _elasticClient.Indices.GetAsync(indexName);
              

            Console.WriteLine(exists.DebugInformation);

            if (exists.Indices.Count == 0)
            {

               var result = await _elasticClient.Indices.CreateAsync(indexDescription);

                Console.WriteLine(result.DebugInformation);

                return result;
            }
            else
            {
                Console.WriteLine(exists.DebugInformation);
          

            }
        }

~

Making a Connection

 

Uri _endpoint = new Uri({your-endpoint-url});

var pool = new SingleNodeConnectionPool(_esEndpoint);

var settings = new ConnectionSettings(pool)

// configure the client with authentication credentials and indices

.BasicAuthentication({username}, {password})

.DefaultIndex({your-index-name})

.DisableDirectStreaming()

.DefaultMappingFor<{your-given-model}>(i => i.IndexName({your-index-name})

.IdProperty(i => i.{your-given-model-id}))

.RequestTimeout(new TimeSpan(0, 0, 3, 0, 0));


_elasticClient = new ElasticClient(settings);

You can now use _elasticClient to invoke methods and interact with the index.

~

Adding a Document to an Index

How to add a document to an existing index:

public async Task<IndexResponse> AddDocument(string content)
{
        YourModel yourDocument = new YourModel
        {
            Uid = Guid.NewGuid(),
            Content = { your content~
        };

        var indexResponseAsync = await _elasticClient.IndexDocumentAsync(yourDocument);

        return indexResponseAsync;
}

~

Bulk Add Documents

In some cases, and in the interests of performance, I found myself performing bulk inserts.  Here is how these are performed:

public async Task<BulkResponse> BulkAddDocuments(List<YourModel> documents)
{
         
            // Perform bulk insert
            var bulkResponse = await _elasticClient.BulkAsync(b => b
                .Index<List<Models.YourModel>>()
                .IndexMany(documents)
            );

            if (bulkResponse.ApiCall.Success)
            {
                Console.WriteLine("Bulk insert successful!");
            }
            else
            {
                Console.WriteLine($"Bulk insert failed: {bulkResponse.DebugInformation}");
                foreach (var itemWithError in bulkResponse.ItemsWithErrors)
                {
                    if (itemWithError.Status != 201)
                    {
                        Console.WriteLine($"Error for item with index {itemWithError.Index}: {itemWithError.Error}");
                    }
                }
            }

            return bulkResponse;
}

 

Interestingly, you need to use bulkResponse.ApiCall.Success property to verify if the call was successful.

I read on GitHub that at the time of writing, there is no 1:1 mapping from the Elastic Seach Bulk Add REST API and the SDK.

~

Searching for Documents

How to search for documents using a given parameter:

public async Task<List<YourModel>> SearchDataAsync(string yourQuery)
{
        var responsedata = await _elasticClient.SearchAsync<YourModel>(s => s.Source()
                           .Query(q => q
                           .QueryString(qs => qs
                           .AnalyzeWildcard()
                           .Query("*" + yourQuery + "*")
                           .Fields(fs => fs
                               .Fields(f1 => f1.{yourfield})
                            )
                            )));

           var datasend = responsedata.Documents.ToList();
           return datasend;
}

~

Searching for Documents using a Parameterized Field and Value

Sometimes you prefer to programmatically set the field and properties you want to use in your search code.

Here is how to do that:

     string propertyName = field;
     string propertyValue = value;

     // Perform a term query to retrieve documents with the specified property value
     var searchResponse = _elasticClient.Search<Content>(s => s
                           .Query(q => q
                           .Match(m => m
                           .Field(propertyName)
                           .Query(propertyValue)))
                           .Size(maxRecordsToFetch)
                           );

     if (searchResponse.IsValid)
     {
           var documents = searchResponse.Documents;       
           return documents.ToList();
     }
     else
     {
           return null;
     }

~

Updating a Document

Updating a document can be done by using the following code:

YourDocument document = new Models.YourDocument 
            {
                AProperty = newValue,
            };

var updateResponse = await _elasticClient.UpdateAsync<YourDocument , object>(document.{yourdocid}, u => u
    .Doc(new
        {
            AProperty= document.AProperty,
        })
         .Index(YOUR_INDEX_NAME));
    return updateResponse;

~

Helpful REST API Endpoints

You can interact with your Elastic Search instance using REST API endpoints.  Here are some of the ones I found helpful:

 

Searching for documents:

POST http://localhost:9200/default-index/_search?size=100&pretty=true

 

Getting a count of all documents across all indexes:

POST http://localhost:9200/_all/_count

 

Deleting an index:

DEL http://localhost:9200/default-index

 

Modifying storage thresholds (when you get RED / threshold errors):

PUT http://localhost:9200/_cluster/settings

With the following Body

{
  "transient": {
    "cluster.routing.allocation.disk.watermark.low": "85%",
    "cluster.routing.allocation.disk.watermark.high": "95%",
    "cluster.routing.allocation.disk.watermark.flood_stage": "98%",
    "cluster.info.update.interval": "1m"
  }
}

~

Summary

That’s it for now. I might add more to this blog as I work more with Elastic Search and the NEST SDK.