I have Elastic Search Nest library code and need to mock the response i am getting from elastic search index.
var obj = service.Search<TestDocument>(new student().Query());
var Name= obj.Aggs.Terms("Name");
For Testing :
I am creating the Nest object after doing quick watch but facing issue -Aggregations - is a internal protected property and i am not able to set this value.
new Nest.KeyedBucket<object>
{
Key="XYZ school",
KeyAsString=null,
Aggregations=new Dictionary<string, IAggregationContainer>{}
}
Please suggest solution or any other approach i can use to mock elastic search nest object .
If you really want to stub the response from the client, you could do something like the following with Moq
var client = new Mock<IElasticClient>();
var searchResponse = new Mock<ISearchResponse<object>>();
var aggregations = new AggregateDictionary(new Dictionary<string, IAggregate> {
["Name"] = new BucketAggregate
{
Items = new List<KeyedBucket<object>>
{
new Nest.KeyedBucket<object>(new Dictionary<string, IAggregate>())
{
Key = "XYZ school",
KeyAsString = null,
DocCount = 5
}
}.AsReadOnly()
}
});
searchResponse.Setup(s => s.Aggregations).Returns(aggregations);
client.Setup(c => c.Search<object>(It.IsAny<Func<SearchDescriptor<object>, ISearchRequest>>()))
.Returns(searchResponse.Object);
var response = client.Object.Search<object>(s => s);
var terms = response.Aggregations.Terms("Name");
Another way would be to use the InMemoryConnection and return known JSON in response to a request..
For testing purposes however, it may be better to have an instance of Elasticsearch running, and perform integration tests against it. Take a look at Elastic.Xunit which provides an easy way to spin up an Elasticsearch cluster for testing purposes. This is used by the client in integration tests.
You can get Elastic.Xunit from the Appveyor feed.
Related
I am writing backend in C# for a website. I'd like to recreate index with little downtime.
After reading these two posts:
Nest Client c# 7.0 for elastic search removing Aliases
Recreate ElasticSearch Index with Nest 7.x
I come up with this:
var alias_exist = await _client.Indices.ExistsAsync(index_string_alias);
if (alias_exist.Exists)
{
var oldIndices = await _client.GetIndicesPointingToAliasAsync(index_string_alias);
var oldIndexName = oldIndices.First().ToString();
await _client.Indices.BulkAliasAsync(new BulkAliasRequest
{
Actions = new List<IAliasAction>
{
new AliasRemoveAction {Remove = new AliasRemoveOperation {Index = oldIndexName, Alias = index_string_alias}},
new AliasAddAction {Add = new AliasAddOperation {Index = index_string_unique, Alias = index_string_alias}}
}
});
} else
{
var putAliasResponse = await _client.Indices.PutAliasAsync(new PutAliasRequest(index_string_unique, index_string_alias));
}
}
I'd like to remove index_string_alias if exists and assign the alias to the newly created index_string_unique.
Also, I'd like to confirm that I can treat the alias as the index name in my other queries.
I am really new to Elastic Search and wonder how people figure out these things. I searched through the official documentation and found little information about the async functions in NEST. Where should I look for explanations for functions?
At first I had 1 index and my elasticclient was setup like below in my startup.cs
public static IServiceCollection AddElasticClient(this IServiceCollection services)
{
var elasticSettings = services.BuildServiceProvider().GetService<IOptions<ElasticSettings>>().Value;
var settings = new ConnectionSettings(new Uri(elasticSettings.Uri));
settings
.ThrowExceptions(elasticSettings.ThrowExceptions)
.PrettyJson(elasticSettings.PrettyJson)
.DefaultIndex(elasticSettings.Index)
.BasicAuthentication(elasticSettings.Username, elasticSettings.Password)
.DefaultMappingFor<CorrelationContext>(ms => ms.Ignore(p => p.DgpHeader));
var client = new ElasticClient(settings);
services.AddSingleton<IElasticClient>(client);
return services;
}
My writer looks like
public class ElasticWriter : IElasticWriter
{
private readonly IElasticClient _elasticClient;
public ElasticWriter(IElasticClient elasticClient)
{
_elasticClient = elasticClient ?? throw new ArgumentNullException(nameof(elasticClient));
}
public void Write(AuditElasticDoc doc)
{
var indexResponse = _elasticClient.IndexDocument(doc);
if (!indexResponse.IsValid)
{
throw indexResponse.OriginalException ?? new Exception("Invalid Elastic response when writing document.");
}
}
}
Now there is a new requirement by which they can provide the name of the index to write to.
All authentication data of the different indexes are provided through config settings, so I have everything available at startup.
The document type is always the same.
I found examples of specifying the index when querying but not when indexing.
Can I provide multiple indexes in my ElasticClient and specify the index when executing the IndexDocument?
Or do I need a separate client for each index?
If the latter, is there a way I can still use DI to inject the client in my writer or do I have to create one there at the spot?
Thx.
I'm using Nest 7.6.1
Instead of using IndexDocument, you can use IndexAsync method which will allow you to control additional request parameters
var indexResponse = await _elasticClient.IndexAsync(doc, descriptor => descriptor.Index("other"));
IndexDocument is a wrapper method, hiding the complexity of indexing documents from the clients. Have a look.
Request auth configuration
var indexResponse = await _elasticClient.IndexAsync(doc,
descriptor => descriptor
.Index("other")
.RequestConfiguration(rq => rq.BasicAuthentication("user", "pass")));
With nest I am doing a Terms aggregation .
I am also doing an inner TopHits aggregation .
My result give me all results infos in the response object except TopHits values which i can read thanks to TopHits() method.
I would like to have tophits values directly in result without using NEST TopHits() methode for reading into aggs. I would like to have all datas in info as we have in elastic search classic requests.
This is what i am actually doing :
My aggregation request:
var response = Client.Search<myclass>(s => s
.Type("type")
.Aggregations(a => a
.Terms("code_bucket", t => t
.Field("field_of_aggregation")
.Size(30)
.Order(TermsOrder.CountAscending)
.Aggregations(a2 => a2
.TopHits("code_bucket_top_hits", th => th.Size(20))
)
)));
I receive a result object in wich i can access all infos except TopHits.
if we examine results we can see TopHits values are stored in private field "_hits":
If I stringify result object , i can see the total number of TopHits, but I can't see the field _hits so i can see the documents:
JavaScriptSerializer js = new JavaScriptSerializer();
string json = js.Serialize(response);
json does not contains topHits result:
I can access to values but i need to use nest method TopHits():
var firstBucket= response.Aggs.Terms("code_bucket");
foreach (var bucket in firstBucket.Buckets)
{
var hits = bucket.TopHits("code_bucket_top_hits");
foreach (var hit in hits.Documents<myclass>())
{
var prop1= hit.prop1;
var prop2= hit.prop2;
}
}
}
But it would be really usefule if i could have all infos in one , like we have when we do elasticsearch request without nest
Do you know if there is a way?
NEST is a higher level abstraction over Elasticsearch that models each request and response with strong types, providing fluent and object initializer syntaxes to build requests, and methods to access portions of the response, without having to handle JSON serialization yourself.
Sometimes however, you might want to manage this yourself, which is what it sounds like you'd like to do. In these cases, Elasticsearch.Net can be used, which is a low level client for Elasticsearch and is unopinionated in how you model your requests and responses.
You can use the client in Elasticsearch.Net instead of NEST, however, the good news is NEST uses Elasticsearch.Net under the covers and also exposes the low level client through the .LowLevel property on IElasticClient. Why would you want to use the lowlevel client on NEST as opposed to just using Elasticsearch.Net directly? A major reason to do so is that you can take advantage of strong types for requests and responses when you need to and leverage NEST's usage of Json.NET for serialization, but bypass this and make calls with the low level client when you want/need to.
Here's an example
var client = new ElasticClient();
var searchRequest = new SearchRequest<Question>
{
Size = 0,
Aggregations = new TermsAggregation("top_tags")
{
Field = "tags",
Size = 30,
Order = new[] { TermsOrder.CountAscending },
Aggregations = new TopHitsAggregation("top_tag_hits")
{
Size = 20
}
}
};
var searchResponse = client.LowLevel.Search<JObject>("posts", "question", searchRequest);
// this will be of type JObject. Do something with it
searchResponse.Body
Here, I can use NEST's object initializer syntax to construct a request, but use the low level client to deserialize the response to a Json.NET JObject. You can deserialize to a T of your choosing by changing it in client.LowLevel.Search<T>(). You could for example use
var searchResponse = client.LowLevel.Search<string>("posts", "question", searchRequest);
to return a string, or
var searchResponse = client.LowLevel.Search<Stream>("posts", "question", searchRequest);
to return a stream, etc.
I am trying to replacing the documents on ES using NEST. I am seeing the following options are available.
Option #1:
var documents = new List<dynamic>();
`var blkOperations = documents.Select(doc => new BulkIndexOperation<T>`(doc)).Cast<IBulkOperation>().ToList();
var blkRequest = new BulkRequest()
{
Refresh = true,
Index = indexName,
Type = typeName,
Consistency = Consistency.One,
Operations = blkOperations
};
var response1 = _client.Raw.BulkAsync<T>(blkRequest);
Option #2:
var descriptor = new BulkDescriptor();
foreach (var eachDoc in document)
{
var doc = eachDoc;
descriptor.Index<T>(i => i
.Index(indexName)
.Type(typeName)
.Document(doc));
}
var response = await _client.Raw.BulkAsync<T>(descriptor);
So can anyone tell me which one is better or any other option to do bulk updates or deletes using NEST?
You are passing the bulk request to the ElasticsearchClient i.e. ElasticClient.Raw, when you should be passing it to ElasticClient.BulkAsync() or ElasticClient.Bulk() which can accept a bulk request type.
Using BulkRequest or BulkDescriptor are two different approaches that are offered by NEST for writing queries; the former uses an Object Initializer Syntax for building up a request object while the latter is used within the Fluent API to build a request using lambda expressions.
In your example, BulkDescriptor is used outside of the context of the fluent API, but both BulkRequest and BulkDescriptor implement IBulkRequest so can be passed to ElasticClient.Bulk(IBulkRequest).
As for which to use, in this case it doesn't matter so whichever you prefer.
I'm trying to use this library (which looks very nice) but I'm having difficulty understanding how to add extra params to my queries such as boost functions etc. How can this be done?
You can use the ExtraParams property of QueryOptions to add any parameter to the Solr querystring. Example:
ISolrOperations<Document> solr = ...
var results = solr.Query("myquery", new QueryOptions {
ExtraParams = new Dictionary<string, string> {
{"bf", "recip(rord(myfield),1,2,3)^1.5"}
}
});