I am looking for a code snippet of After functionality usage with NEST lib.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html#_after.
Thanks in advance for the code snippet
You can pass the CompositeKey from a previous composite aggregation as the .After() parameter for a new composite aggregation. For example
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool);
var client = new ElasticClient(settings);
var searchResponse = client.Search<object>(s => s
.From(0)
.AllIndices()
.AllTypes()
.Aggregations(a => a
.Composite("composite_agg", c => c
.Sources(so => so
.DateHistogram("date", dh => dh
.Field("timestamp")
.Interval("1d")
)
.Terms("product", t => t
.Field("product")
)
)
)
)
);
var compositeAgg = searchResponse.Aggregations.Composite("composite_agg");
searchResponse = client.Search<object>(s => s
.From(0)
.AllIndices()
.AllTypes()
.Aggregations(a => a
.Composite("composite_agg", c => c
.Sources(so => so
.DateHistogram("date", dh => dh
.Field("timestamp")
.Interval("1d")
)
.Terms("product", t => t
.Field("product")
)
)
.After(compositeAgg.AfterKey) // <-- pass the after key from previous agg response
)
)
);
Assuming you're using Elasticsearch 6.x (which you must be to be using Composite Aggregation), please update NEST client to latest (6.6.0 at this time), as it contains a bug fix for a CompositeKey with null values.
Related
I have a method to search in C# as follows:
public void Search(string data)
{
var searchResponse = client.Search<Products>(s => s
.From(0)
.Size(100)
.Query(q => q
.Match(m => m
.Field( f => f.ProductName)
.Query(data))));
int cnt = searchResponse.Documents.Count;
}
This returned 5 documents, which is valid.
But, I wanted to modify the above method as follows, so that I can pass the field to be searched dynamically.
public void Search(string data,string fieldName)
{
var searchResponse = client.Search<Products>(s => s
.From(0)
.Size(100)
.Query(q => q
.Match(m => m
.Field(fieldName)
.Query(data))));
int cnt = searchResponse.Documents.Count;
}
The above code does return any error, but return 0 documents.
Here I am passing ProductName to fieldName parameter. I know this will not work. I just want tell my intention here.
Field Infering
You can pass below to your query
var fieldString = new Field("fieldName");
var fieldString = new Field("fieldName");
var searchResponse = _elasticClient.Search<AllOpportunitySearchResult>(s => s
.From(0)
.Size(100)
.Query(q => q
.Match(m => m
.Field(fieldString)
.Query(data)
)
)
);
I want to use the UpdateByQuery method on the high level client but can't find any documentation for Nest. They have great documentation if I wanted to make a CURL request but nothing for NEST. https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html
If anyone has and example of them using it or can share documentation they have found that would be awesome!
Update By Query API is supported in NEST. Here's an example adapted from the integration tests. NEST Documentation for Index and Update APIs is planned :)
private static void Main()
{
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool)
.DefaultMappingFor<Test>(m => m
.IndexName("tests")
.TypeName("test")
);
var client = new ElasticClient(settings);
var index = IndexName.From<Test>();
if (client.IndexExists(index).Exists)
client.DeleteIndex(index);
client.CreateIndex(index, c => c
.Mappings(m => m
.Map<Test>(map => map
.Properties(props => props
.Text(s => s.Name(p => p.Text))
.Keyword(s => s.Name(p => p.Flag))
)
)
)
);
client.Bulk(b => b
.IndexMany(new[] {
new Test { Text = "words words", Flag = "bar" },
new Test { Text = "words words", Flag = "foo" }
})
.Refresh(Refresh.WaitFor)
);
client.Count<Test>(s => s
.Query(q => q
.Match(m => m
.Field(p => p.Flag)
.Query("foo")
)
)
);
client.UpdateByQuery<Test>(u => u
.Query(q => q
.Term(f => f.Flag, "bar")
)
.Script("ctx._source.flag = 'foo'")
.Conflicts(Conflicts.Proceed)
.Refresh(true)
);
client.Count<Test>(s => s
.Query(q => q
.Match(m => m
.Field(p => p.Flag)
.Query("foo")
)
)
);
}
public class Test
{
public string Text { get; set; }
public string Flag { get; set; }
}
Observe that the count from the first Count API call is 1, and on the second Count API call after the Update By Query API call, it's 2.
I have a multiSearch query as below. Basically I query product and category types. I would like to make this query optional without writing same code again.
Basically in some cases I want to query only product type, that means that it will not multisearch but a search query. How can I split this query into 2 search queries. Something like below I think.
return Client.MultiSearch(ms => ms
.Search<Product>("products", s => s
.Index(IndexName)
.Explain(explain)
.Query(q => q
.Bool(b => b
.Should(
sh => sh.MultiMatch(qs => qs
.Fields(d => d
.Field(Name + ".raw", NameBoost + 0.5)
.Field(Name, NameBoost)
.Type(TextQueryType.BestFields)
.Query(key))
))).From(startfrom).Size(size))
.Search<Category>("categories", s => s
.Index(IndexName)
.Explain(explain)
.Query(q => q.
Bool(b => b.
Should(sh => sh.
MultiMatch(m => m
.Fields(d => d
.Field(f => f.Name, NameBoost)
.Field(p => p.Name.Suffix("raw"), NameBoost + 0.5)).Type(TextQueryType.BestFields)
.Query(key)
)
))).From(startfrom).Size(size))
);
something like this below. I guess that it is called object initializer Syntax according to this article
Client.MultiSearch (SearchProductQuery && SearchCategoryQuery)
is it possible?
This fluent API multi search
client.MultiSearch(ms => ms
.Search<Product>("products", s => s
.Index(IndexName)
.Explain(explain)
.Query(q => q
.Bool(b => b
.Should(sh => sh
.MultiMatch(qs => qs
.Fields(d => d
.Field(Name + ".raw", NameBoost + 0.5)
.Field(Name, NameBoost)
)
.Type(TextQueryType.BestFields)
.Query(key)
)
)
)
)
.From(startfrom)
.Size(size)
)
.Search<Category>("categories", s => s
.Index(IndexName)
.Explain(explain)
.Query(q => q
.Bool(b => b
.Should(sh => sh
.MultiMatch(m => m
.Fields(d => d
.Field(f => f.Name, NameBoost)
.Field(p => p.Name.Suffix("raw"), NameBoost + 0.5)
)
.Type(TextQueryType.BestFields)
.Query(key)
)
)
)
)
.From(startfrom)
.Size(size)
)
);
would be this OIS API multi search
var multiSearch = new MultiSearchRequest
{
Operations = new Dictionary<string, ISearchRequest>
{
{ "products", new SearchRequest<Product>(IndexName)
{
Explain = true,
Query = new BoolQuery
{
Should = new QueryContainer[] {
new MultiMatchQuery
{
Fields =
((Fields)Field.Create(Name + ".raw", NameBoost + 0.5))
.And(Name, NameBoost),
Type = TextQueryType.BestFields,
Query = key
}
}
},
From = startfrom,
Size = size
}
},
{ "categories", new SearchRequest<Category>(IndexName)
{
Explain = true,
Query = new BoolQuery
{
Should = new QueryContainer[] {
new MultiMatchQuery
{
Fields =
((Fields)Infer.Field<Category>(f => f.Name, NameBoost))
.And<Category>(f => f.Name.Suffix("raw"), NameBoost + 0.5),
Type = TextQueryType.BestFields,
Query = key
}
}
},
From = startfrom,
Size = size
}
},
}
};
client.MultiSearch(multiSearch);
Take a look at the multi search integration tests for another example. I'll look at getting this added to the documentation.
I'm trying to calculate some stats from elasticsearch with a short time period but, eventhough I specified the interval, I'm still getting results from the whole data set.
This is my code
var minDate = new DateTime(1970, 1, 1);
var fromDate = DateTime.Parse("2014-11-27T11:00:00.000Z").AddTicks(-minDate.Ticks).AddHours(-2);
var toDate = DateTime.Parse("2014-11-27T11:15:00.000Z").AddTicks(-minDate.Ticks).AddHours(-2);
var results = client.Search<Dalsp>(s => s
.From(0)
.Size(100)
.FacetTermsStats(fts => fts
.FacetFilter(fdesc => fdesc
.Range(range =>
range.Greater(fromDate.Ticks / 10000).Lower(toDate.Ticks / 10000)
)
)
.KeyField(t => t.sp_name)
.ValueField(t => t.total_time)
)
);
What can be the problem? Thanks in advance.
Problem solved after using strings for timestamp, instead of long.
var results = client.Search<Dalsp>(s => s
.From(0)
.Size(100)
.Query(q =>
q.Range(range => range.OnField("#timestamp").LowerOrEquals(endDateStr).GreaterOrEquals(startDateStr))
)
.FacetTermsStats(fts => fts
.KeyField(t => t.sp_name)
.ValueField(t => t.total_time)
)
);
Im very very new to elasticsearch using the nest client, I am creating an index with a custom analyzer, however when testing using analyze it does not seem to use the custom analyzer. Mainly no edgengram tokens appear. Is there anything I am missing that would make my custom analyser the default for the index? When I check my mappings using elastichq they show my custom analyzer.
ConnectionSettings settings = new ConnectionSettings(new Uri("http://localhost:9200"), defaultIndex: "forum-app");
IndexSettings indsettings = new IndexSettings();
var an = new CustomAnalyzer();
an.CharFilter = new List<string>();
an.CharFilter.Add("html_strip");
an.Tokenizer = "edgeNGram";
an.Filter = new List<string>();
an.Filter.Add("standard");
an.Filter.Add("lowercase");
an.Filter.Add("stop");
indsettings.Analysis.Tokenizers.Add("edgeNGram", new Nest.EdgeNGramTokenizer
{
MaxGram = 15,
MinGram = 3
});
indsettings.Analysis.Analyzers.Add("forumanalyzer", an);
ElasticClient client = new ElasticClient(settings);
client.CreateIndex("forum-app", c => c
.NumberOfReplicas(0)
.NumberOfShards(1)
.AddMapping<Forum>(e => e.MapFromAttributes())
.Analysis(analysis => analysis
.Analyzers(a => a
.Add("forumanalyzer", an)
)));
//To index I just do this
client.Index(aForum);
You've added your custom analyzer to your index, but now you need to apply it your fields. You can do this on a field mapping level:
client.CreateIndex("forum-app", c => c
.NumberOfReplicas(0)
.NumberOfShards(1)
.AddMapping<Forum>(e => e
.MapFromAttributes()
.Properties(p => p
.String(s => s.Name(f => f.SomeProperty).Analyzer("formanalyzer")))
)
.Analysis(analysis => analysis
.Analyzers(a => a
.Add("forumanalyzer", an)
)
)
);
Or you can apply it to all fields by default by setting it as the default analyzer of your index:
client.CreateIndex("forum-app", c => c
.NumberOfReplicas(0)
.NumberOfShards(1)
.AddMapping<Forum>(e => e.MapFromAttributes())
.Analysis(analysis => analysis
.Analyzers(a => a
.Add("default", an)
)
)
);
More info here in regards to analyzer defaults.
Add a custom analyzer:
var indexSettings = new IndexSettings
{
NumberOfReplicas = 0, // If this is set to 1 or more, then the index becomes yellow.
NumberOfShards = 5
};
indexSettings.Analysis = new Analysis();
indexSettings.Analysis.Analyzers = new Analyzers();
indexSettings.Analysis.TokenFilters = new TokenFilters();
var customAnalyzer = new CustomAnalyzer
{
//CharFilter = new List<string> { "mapping " },
Tokenizer = "standard",
Filter = new List<string> { "lowercase", "asciifolding" }
};
indexSettings.Analysis.Analyzers.Add("customAnalyzerLowercaseSynonymAsciifolding", customAnalyzer);
And then when creating the index, you specify the analyzer:
var indexConfig = new IndexState
{
Settings = indexSettings
};
var createIndexResponse = elasticClient.CreateIndex(indexName, c => c
.InitializeUsing(indexConfig)
.Mappings(m => m
.Map<ElasticsearchModel>(mm => mm
.Properties(
p => p
.Text(t => t.Name(elasticsearchModel => elasticsearchModel.StringTest).Analyzer("customAnalyzerLowercaseSynonymAsciifolding"))
)
)
)
);
elasticClient.Refresh(indexName);
And then you query it with something like:
var response = elasticClient.Search<ElasticsearchModel>(s => s
.Index(indexName)
.Query(q => q
.SimpleQueryString(qs => qs
.Fields(fs => fs
.Field(f => f.StringTest, 4.00)
)
.Query(query)
)
)
);
var results = new List<ElasticsearchModel>();
results = response.Hits.Select(hit => hit.Source).ToList();