Elasticsearch NEST Reuse ElasticClient for different index query - elasticsearch

How can I register ElasticClient as singleton in a .NET Core application but still able to specify the different index during the query?
For example:
In Startup.cs I register an elastic client object as singleton, by only mentioning the URL without specifying the index.
public void ConfigureServices(IServiceCollection services)
{
....
var connectionSettings = new ConnectionSettings(new Uri("http://localhost:9200"));
var client = new ElasticClient(connectionSettings);
services.AddSingleton<IElasticClient>(client);
....
}
Then when injecting ElasticClient singleton object above, I would like to use it for different indices in 2 different queries.
In the class below, I want to query from an index called "Apple"
public class GetAppleHandler
{
private readonly IElasticClient _elasticClient;
public GetAppleHandler(IElasticClient elasticClient)
{
_elasticClient = elasticClient;
}
public async Task<GetAppleResponse> Handle()
{
// I want to query (_elasticClient.SearchAsync<>) using an index called "Apple" here
}
}
From code below I want to query from an index called "Orange"
public class GetOrangeHandler
{
private readonly IElasticClient _elasticClient;
public GetOrangeHandler(IElasticClient elasticClient)
{
_elasticClient = elasticClient;
}
public async Task<GetOrangeResponse> Handle()
{
// I want to query (_elasticClient.SearchAsync<>) using an index called "Orange" here
}
}
How can I do this? If it's not possible, can you suggest other approach that will allow me to inject ElasticClient through .NET Core dependency injection and at the same time also allow me to query from 2 different indices of the same ES instance?

Just need to specify the index on the request
var defaultIndex = "person";
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool)
.DefaultIndex(defaultIndex)
.DefaultTypeName("_doc");
var client = new ElasticClient(settings);
var searchResponse = client.Search<Person>(s => s
.Index("foo_bar")
.Query(q => q
.Match(m => m
.Field("some_field")
.Query("match query")
)
)
);
Here the search request would be
POST http://localhost:9200/foo_bar/_doc/_search
{
"query": {
"match": {
"some_field": {
"query": "match query"
}
}
}
}
foo_bar index has been defined in the search request
_doc type has been inferred from the global rule on DefaultTypeName("_doc")

Related

Mock IDocumentQuery with ability to use query expressions

I need to be able to mock IDocumentQuery, to be able to test piece of code, that queries document collection and might use predicate to filter them:
IQueryable<T> documentQuery = client
.CreateDocumentQuery<T>(collectionUri, options);
if (predicate != null)
{
documentQuery = documentQuery.Where(predicate);
}
var list = documentQuery.AsDocumentQuery();
var documents = new List<T>();
while (list.HasMoreResults)
{
documents.AddRange(await list.ExecuteNextAsync<T>());
}
I've used answer from https://stackoverflow.com/a/49911733/212121 to write following method:
public static IDocumentClient Create<T>(params T[] collectionDocuments)
{
var query = Substitute.For<IFakeDocumentQuery<T>>();
var provider = Substitute.For<IQueryProvider>();
provider
.CreateQuery<T>(Arg.Any<Expression>())
.Returns(x => query);
query.Provider.Returns(provider);
query.ElementType.Returns(collectionDocuments.AsQueryable().ElementType);
query.Expression.Returns(collectionDocuments.AsQueryable().Expression);
query.GetEnumerator().Returns(collectionDocuments.AsQueryable().GetEnumerator());
query.ExecuteNextAsync<T>().Returns(x => new FeedResponse<T>(collectionDocuments));
query.HasMoreResults.Returns(true, false);
var client = Substitute.For<IDocumentClient>();
client
.CreateDocumentQuery<T>(Arg.Any<Uri>(), Arg.Any<FeedOptions>())
.Returns(query);
return client;
}
Which works fine as long as there's no filtering using IQueryable.Where.
My question:
Is there any way to capture predicate, that was used to create documentQuery and apply that predicate on collectionDocuments parameter?
Access the expression from the query provider so that it will be passed on to the backing collection to apply the desired filter.
Review the following
public static IDocumentClient Create<T>(params T[] collectionDocuments) {
var query = Substitute.For<IFakeDocumentQuery<T>>();
var queryable = collectionDocuments.AsQueryable();
var provider = Substitute.For<IQueryProvider>();
provider.CreateQuery<T>(Arg.Any<Expression>())
.Returns(x => {
var expression = x.Arg<Expression>();
if (expression != null) {
queryable = queryable.Provider.CreateQuery<T>(expression);
}
return query;
});
query.Provider.Returns(_ => provider);
query.ElementType.Returns(_ => queryable.ElementType);
query.Expression.Returns(_ => queryable.Expression);
query.GetEnumerator().Returns(_ => queryable.GetEnumerator());
query.ExecuteNextAsync<T>().Returns(x => new FeedResponse<T>(query));
query.HasMoreResults.Returns(true, true, false);
var client = Substitute.For<IDocumentClient>();
client
.CreateDocumentQuery<T>(Arg.Any<Uri>(), Arg.Any<FeedOptions>())
.Returns(query);
return client;
}
The important part is where the expression passed to the query is used to create another query on the backing data source (the array).
Using the following example subject under test for demonstration purposes.
public class SubjectUnderTest {
private readonly IDocumentClient client;
public SubjectUnderTest(IDocumentClient client) {
this.client = client;
}
public async Task<List<T>> Query<T>(Expression<Func<T, bool>> predicate = null) {
FeedOptions options = null; //for dummy purposes only
Uri collectionUri = null; //for dummy purposes only
IQueryable<T> documentQuery = client.CreateDocumentQuery<T>(collectionUri, options);
if (predicate != null) {
documentQuery = documentQuery.Where(predicate);
}
var list = documentQuery.AsDocumentQuery();
var documents = new List<T>();
while (list.HasMoreResults) {
documents.AddRange(await list.ExecuteNextAsync<T>());
}
return documents;
}
}
The following sample tests when an expression is passed to the query
[TestMethod]
public async Task Should_Filter_DocumentQuery() {
//Arrange
var dataSource = Enumerable.Range(0, 3)
.Select(_ => new Document() { Key = _ }).ToArray();
var client = Create(dataSource);
var subject = new SubjectUnderTest(client);
Expression<Func<Document, bool>> predicate = _ => _.Key == 1;
var expected = dataSource.Where(predicate.Compile());
//Act
var actual = await subject.Query<Document>(predicate);
//Assert
actual.Should().BeEquivalentTo(expected);
}
public class Document {
public int Key { get; set; }
}

Elasticsearch / NEST 6 - storing enums as string

Is it possible to store enums as string in NEST6?
I've tried this but it does not seem to work. Any suggestions?
var pool = new SingleNodeConnectionPool(new Uri(context.ConnectionString));
connectionSettings = new ConnectionSettings(pool, connection, SourceSerializer());
private static ConnectionSettings.SourceSerializerFactory SourceSerializer()
{
return (builtin, settings) => new JsonNetSerializer(builtin, settings,
() => new JsonSerializerSettings
{
Converters = new List<JsonConverter>
{
new StringEnumConverter()
}
});
}
Use the StringEnumAttribute attribute on the property. This signals to the internal serializer to serialize the enum as a string. In using this, you don't need to use the NEST.JsonNetSerializer package
If you'd like to set it for all enums, you can do so with
private static void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var connectionSettings = new ConnectionSettings(
pool,
(builtin, settings) => new JsonNetSerializer(builtin, settings,
contractJsonConverters: new JsonConverter[] { new StringEnumConverter() }));
var client = new ElasticClient(connectionSettings);
client.Index(new Product { Foo = Foo.Bar }, i => i.Index("examples"));
}
public class Product
{
public Foo Foo { get;set; }
}
public enum Foo
{
Bar
}
which yields a request like
POST http://localhost:9200/examples/product
{
"foo": "Bar"
}
I think the way that you're attempting to set converters should also work and is a bug that it doesn't. I'll open an issue to address.

In Elasticsearch Nest mapping one type to another

I am using the elasticsearch nest C# library to map documents. Everything goes smooth except a custom type, TypeX, I have which is treated as an object and stored as { }.
This custom type is a v1 uuid and is essentially treated as Guid. I would honestly like to cast it as a Guid for storage purposes since it implicitly casts back and forth. Thus elastic would see it as a Guid and not TypeX.
According to the attribute mapping section it seems I can change the type that way however I don't really want to expose Nest as a dependency for my type since it is used in numerous places.
Is it possible to setup this mapping from the connection or the Index to map TypeX to Guid and Guid to TypeX?
Nest: 6.0.1
ES: 6.2.2
System.Guid is mapped as a keyword data type with automapping with NEST. Given the following document
public class MyDocument
{
public Guid UserId { get; set; }
}
and the following mapping
var client = new ElasticClient();
var createIndexResponse = client.CreateIndex("foo", c => c
.Mappings(m => m
.Map<MyDocument>(mm => mm
.AutoMap()
)
)
);
will produce
{
"mappings": {
"mydocument": {
"properties": {
"userId": {
"type": "keyword"
}
}
}
}
}
Now, for your own type to be mapped as a keyword type, you can use fluent mapping if you don't want to attribute the POCO. Given the following POCO and custom Uuid type
public class Uuid
{
private string _value;
public Uuid(string value) => _value = value;
public override string ToString() => _value;
}
public class MyDocument
{
public Uuid UserId { get; set; }
}
these can be mapped with
var createIndexResponse = client.CreateIndex("foo", c => c
.Mappings(m => m
.Map<MyDocument>(mm => mm
.AutoMap()
.Properties(p => p
.Keyword(k => k
.Name(n => n.UserId)
)
)
)
)
);
which produces the same mapping as before. However, this is only half of the story, because we also need to control how Uuid is serialized and deserialized such that it is serialized as a JSON string and an instance can be constructed from a string. In NEST 6.x, we would need to use our own serializer for this, since the serializer used by NEST is internal.
The NEST.JsonSerializer nuget package contains a custom serializer that uses Json.NET, so you can write a JsonConverter to take care of serialization of the Uuid type
public class UuidConverter : JsonConverter
{
public override bool CanConvert(Type objectType) =>
typeof(Uuid).IsAssignableFrom(objectType);
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) =>
reader.TokenType == JsonToken.String
? new Uuid((string)reader.Value)
: null;
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
if (value != null)
writer.WriteValue(value.ToString());
else
writer.WriteNull();
}
}
private static void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
// configure NEST to use Json.NET for serialization of your documents
// and register a custom converter to handle Uuid type
var settings = new ConnectionSettings(pool, (builtin, s) =>
new JsonNetSerializer(builtin, s, contractJsonConverters:
new [] { new UuidConverter() }
))
.DefaultIndex("foo");
var client = new ElasticClient(settings);
var createIndexResponse = client.CreateIndex("foo", c => c
.Mappings(m => m
.Map<MyDocument>(mm => mm
.AutoMap()
.Properties(p => p
.Keyword(k => k
.Name(n => n.UserId)
)
)
)
)
);
var document = new MyDocument
{
UserId = new Uuid("123e4567-e89b-12d3-a456-426655440000")
};
var indexResponse = client.IndexDocument(document);
}
The document indexing request then serializes Uuid as a string
POST http://localhost:9200/foo/mydocument
{
"userId": "123e4567-e89b-12d3-a456-426655440000"
}
You can use fluent mapping instead of attribute mapping (next page in your attribute mapping link)

Specify default analyzer in NEST or Elasticsearch

How can I specify default analyzer in NEST? Or alternative in Elasticsearch? I want change standard analyzer to language analyzer!
If you are using automap in nest you can use an attribute like so
public class A
{
[Text(Analyzer = "NameOfTheAnalyzer")]
public string Prop1 { get; set; }
}
If you want the default mapping you can set it like so
var request = new CreateIndexRequest(indexName)
{
Mappings = new Mappings()
{
["_default_"] = new TypeMapping()
{
Properties = new Properties
{
["id"] = new KeywordProperty { Index = false },
["title"] = new TextProperty { Analyzer = "NameOfTheAnalyzer" }
}
}
}
};
var create = client.CreateIndex(request);

Make a new generic list from a list within a list in Linq with lambda expressions?

Okay so I have a POCO class that may contain another POCO class as an array. In some instances when I get the data I want to CREATE a list of the lists but not as a level down but all on the same level. I think I am missing something very simple so I thought I would ask here. I keep trying different syntax for Lambdas but the data is there, I can just never make it appear near the top. I would like the solution to be in lambdas if possible instead of doing the old school foreach. I was not sure if you can do this inline at all or if you have to declare a collection first and then add to it. Where I am at:
class Program
{
public class lowerlevel
{
public string ChildName;
}
public class upperlevel
{
public string ItemName;
public lowerlevel[] ChildNames;
}
static void Main(string[] args)
{
// Create a list of a POCO object that has lists in it as well.
List<upperlevel> items = new List<upperlevel>
{
// declaration of top level item
new upperlevel
{
ItemName = "FirstItem",
// declaration of children
ChildNames = new lowerlevel[]
{new lowerlevel {ChildName = "Part1"}, new lowerlevel {ChildName = "Part2"}},
},
// declaration of top level item
new upperlevel
{
ItemName = "SecondItem",
// declaration of children
ChildNames = new lowerlevel[] { new lowerlevel { ChildName = "Part3" } }
}
};
var stuff = items.Select(l1 => l1.ChildNames.ToList().Select(l2 =>
new lowerlevel
{
ChildName = l2.ChildName
}))
.ToList();
// Arghh! I just want to make a new list with lambdas that is NOT nested a level down! This is NOT what I want but it is valid.
stuff.ForEach(n => n.ToList().ForEach(n2 => n2.ChildName));
// I want this but it does not work as I am not doing the expression right
// stuff.Foreach(n => n.ChildName);
}
}
Try using a SelectMany() Rather than .Select()
var stuff = items.SelectMany...

Categories

Resources