I have below code working fine in my java service.
Query searchQuery = new StringQuery(
"{\"bool\":{\"must\":[{\"match\":{\"id\":\"" + id + "\"}}]}}");
SearchHits<Instance> instanceSearchHits = elasticsearchOperations.search(searchQuery, Instance.class, IndexCoordinates.of("test"));
log.info("hits :: " + instanceSearchHits.getSearchHits().size());
Now, I want to save this query as a template in elastic and just pass params and search template from java service to elastic to execute the query.
Search Template added in Elastic
PUT _scripts/search-template-1
{
"script": {
"lang": "mustache",
"source": {
"query": {
"bool": {
"must": [
{
"term": {
"id": "{{id}}"
}
}
]
}
}
},
"params": {
"id": "id to search"
}
}
}
Call this template
GET test/_search/template
{
"id": "search-template-1",
"params": {
"id": "f52c2c62-e921-4410-847f-25ea0f3eeb40"
}
}
But unfortunately not able to find API reference for the same to call this search template from JAVA (spring-data-elasticsearch)
As mentioned by val this can be used to call the search template query from java
SearchTemplateRequest request = new SearchTemplateRequest();
request.setRequest(new SearchRequest("posts"));
request.setScriptType(ScriptType.STORED);
request.setScript("title_search");
Map<String, Object> params = new HashMap<>();
params.put("field", "title");
params.put("value", "elasticsearch");
params.put("size", 5);
request.setScriptParams(params);
SearchTemplateResponse response = client.searchTemplate(request, RequestOptions.DEFAULT);
SearchResponse searchResponse = response.getResponse();
SearchHits searchHits = searchResponse.getHits();
log.info("hits :: " + searchHits.getMaxScore());
searchHits.forEach(searchHit -> {
log.info("this is the response, " + searchHit.getSourceAsString());
});
This is currently not yet possible. There is an issue for this in Spring Data Elasticsearch.
Related
Currently I have a search template that I am trying to pass in a couple of parameters,
How can I construct my search terms using NEST to get the following result.
Template
PUT _scripts/company-index-template
{
"script": {
"lang": "mustache",
"source": "{\"query\": {\"bool\": {\"filter\":{{#toJson}}clauses{{/toJson}},\"must\": [{\"query_string\": {\"fields\": [\"companyTradingName^2\",\"companyName\",\"companyContactPerson\"],\"query\": \"{{query}}\"}}]}}}",
"params": {
"query": "",
"clauses": []
}
}
}
DSL query looks as follow
GET company-index/_search/template
{
"id": "company-index-template",
"params": {
"query": "sky*",
"clauses": [
{
"terms": {
"companyGroupId": [
1595
]
}
},
{
"terms": {
"companyId": [
158,
836,
1525,
2298,
2367,
3176,
3280
]
}
}
]
}
}
I would like to construct the above query in NEST but cant seem to find a good way to generate the clauses value.
This is what I have so far...
var responses = this.client.SearchTemplate<Company>(descriptor =>
descriptor
.Index(SearchConstants.CompanyIndex)
.Id("company-index-template")
.Params(objects => objects
.Add("query", queryBuilder.Query)
.Add("clauses", "*How do I contruct this JSON*");
UPDATE:
This is how I ended up doing it. I just created a dictionary with all my terms in it.
I do think there might be a beter why of doing it, but I cant find it.
new List<Dictionary<string, object>>
{
new() {{"terms", new Dictionary<string, object> {{"companyGroupId", companyGroupId}}}},
new() {{"terms", new Dictionary<string, object> {{"companyId", availableCompanies}}}}
}
And then I had to Serialize when I passed it to the Params method.
var response = this.client.SearchTemplate<Company>(descriptor =>
descriptor.Index(SearchConstants.CompanyIndex)
.Id("company-index-template")
.Params(objects => objects
.Add("query", "*" + query + "*")
.Add("clauses", JsonConvert.SerializeObject(filterClauses))));
I use spring data elastic search and have a this list in my elastic search.
{"appUserId": "id-test-app-user11", "apkId": 1, "event": "INSTALL", "date": "2020-06-01"}
...
{"appUserId": "id-test-app-user168", "apkId": 1, "event": "INSTALL", "date": "2020-12-06"}
I want to count by day the number of install of an apkId between a date range.
With this request, I can get all data bewteen my date range and an apkId provided in parameter
LocalDate today = LocalDate.now();
LocalDate beginningDate = today.minusDays(intervalle);
BoolQueryBuilder query = QueryBuilders.boolQuery();
query.must(QueryBuilders.rangeQuery("date")
.gte(convertToDateViaInstant(beginningDate))
.lte(convertToDateViaInstant(today)));
query.must(QueryBuilders.matchQuery("apkId", apkId));
query.must(QueryBuilders.matchQuery("event", Event.INSTALL));
return apkHistoryRepo.search(query);
But I don't know how to aggregate by date in order to have something like
{"2020-06-01": "500"}
...
{"2020-12-06": "10"}
Please how could I achieve this ?
Thanks in advance
You are looking for date histogram aggregation. Here is how you can use it in your query,
{
"query": {
"bool": {
"must": [
{
"term": {
"apkId": "1"
}
},
{
"term": {
"event": "INSTALL"
}
},
{
"range": {
"date": {
"gte": <start_date_here>,
"lte": <end_date_here>
}
}
}
]
}
},
"aggs": {
"per_day_count": {
"date_histogram": {
"field": "date",
"calendar_interval": "1d"
}
}
}
}
Final solution in Java
LocalDate today = LocalDate.now();
LocalDate beginningDate = today.minusDays(intervalle);
BoolQueryBuilder query = QueryBuilders.boolQuery();
query.must(QueryBuilders.rangeQuery("date")
.gte(beginningDate)
.lte(today));
query.must(QueryBuilders.matchQuery("apkId", apkId));
query.must(QueryBuilders.matchQuery("event", Event.INSTALL));
Iterable<ApkHistory> list = apkHistoryRepo.search(query);
AggregationBuilder aggregation = AggregationBuilders
.dateHistogram("nb_install_per_day")
.field("date")
.dateHistogramInterval(DateHistogramInterval.DAY);
SearchRequest searchRequest = new SearchRequest("apkhistory");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(query).aggregation(aggregation);
searchRequest.source(searchSourceBuilder);
try {
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
ParsedDateHistogram dateGroupBy = searchResponse.getAggregations().get("nb_install_per_day");
List<? extends Histogram.Bucket> bucketList = dateGroupBy.getBuckets();
for(Bucket b : bucketList) {
System.out.println(b.getKeyAsString() + " "+b.getDocCount());
}
System.out.println("test");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
I'm using NEST client for querying ES, but now I have a specific situation - I'm trying to proxy query to ES, but with specific query applied by default:
public IEnumerable<TDocument> Search<TDocument>(string indexName, string query, string sort, int page, int pageSize) where TDocument : class
{
var search = new SearchRequest(indexName)
{
From = page,
Size = pageSize,
Query = new RawQuery(query),
};
var response = this.client.Search<TDocument>(search);
return response.Documents;
}
Code above is just proxying query to ES, but what if I need to apply specific filter that should be always applied along with passed query?
So for example I'd want Active field to be true by default. How can I merge this raw query with some specific and always applied filter (without merging strings to formulate merged ES API call if possible).
Assuming that query is well formed JSON that corresponds to the query DSL, you could deserialize it into an instance of QueryContainer and combine it with other queries. For example
var client = new ElasticClient();
string query = #"{
""multi_match"": {
""query"": ""hello world"",
""fields"": [
""description^2.2"",
""myOtherField^0.3""
]
}
}";
QueryContainer queryContainer = null;
using (var stream = client.ConnectionSettings.MemoryStreamFactory.Create(Encoding.UTF8.GetBytes(query)))
{
queryContainer = client.RequestResponseSerializer.Deserialize<QueryContainer>(stream);
}
queryContainer = queryContainer && +new TermQuery
{
Field = "another_field",
Value = "term"
};
var searchResponse = client.Search<TDocument>(s => s.Query(q => queryContainer));
which will translate to the following query (assuming default index is _all)
POST http://localhost:9200/_all/_search?pretty=true&typed_keys=true
{
"query": {
"bool": {
"filter": [{
"term": {
"another_field": {
"value": "term"
}
}
}],
"must": [{
"multi_match": {
"fields": ["description^2.2", "myOtherField^0.3"],
"query": "hello world"
}
}]
}
}
}
It's possible build a query with aggregation (elasticsearch), using java-query-dsl?
ElasticSearch provides a client lib that helps you to build searches. You can find more about it here.
Here's an example of how you can do it:
// build the client
HttpHost host = new HttpHost("localhost", 9200, "http");
RestHighLevelClient client = new RestHighLevelClient(RestClient
.builder(new HttpHost[]{host}));
// build the search (set the conditions here)
BoolQueryBuilder boolQueryBuilder = boolQuery();
boolQueryBuilder.must(QueryBuilders.rangeQuery("age")
.from(25)
.to(40));
// build the aggregations (set the aggregations here)
TermsAggregationBuilder groupByGender = AggregationBuilders.terms("gender")
.field("gender")
.size(5);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(boolQueryBuilder);
sourceBuilder.aggregation(groupByGender);
// create and execute the search request
SearchRequest request = new SearchRequest()
.indices("customers")
.types("customer")
.allowPartialSearchResults(false)
.source(sourceBuilder)
.requestCache(true);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
which will produce something like:
GET customers/customer/_search
{
"query": {
"bool": {
"must": [
{
"range": {
"age": {
"gt": 25,
"lt": 40
}
}
}
]
}
},
"aggs": {
"gender": {
"terms": {
"field": "gender",
"size": 5
}
}
}
}
Trying to achieve following,
Retrieve articles that match given id and genre
Retrieve selected fields for matching records
I have tried this with Sense(chrome plugin),
POST /d3acampaign/article/_search
{
"fields": ["title","genre"] ,
"query": {
"filtered": {
"filter": {
"bool": {
"must": [{
"term": {
"id": "6"
}
},
{
"term": {
"genre": "metal"
}
}]
}
}
}
}
}
For C# code i am trying to build the query using following construct,
FilterContainer fc = null;
TermFilter title = new TermFilter()
{
Field = "id",
Value = "6",
};
TermFilter genre = new TermFilter()
{
Field = "genre",
Value = "metal",
};
fc = title & genre;
QueryContainer qc = new FilteredQuery() { Filter = fc };
var searchRequest = new SearchRequest
{
SearchType = Elasticsearch.Net.SearchType.QueryAndFetch,
Query = qc,
Indices = new IndexNameMarker[] {"journal"},
Types = new TypeNameMarker[] { "article" },
};
var r = client.SearchAsync<Article>(searchRequest);
var l = (List<Article>) r.Result.Documents;
I am able to run this query and get matching records, but i amnt sure how to specify selected fields to retrieve. Let me know what can be changed in C# code to specify necessary fields.
Thanks in advance.
Based on this answer you can modify your request object as follow:
var searchRequest = new SearchRequest
{
...
Fields = new List<PropertyPathMarker>
{
Property.Path<Article>(p => p.YourField)
}
};
or if you will decide to use source filtering:
var searchRequest = new SearchRequest
{
...
Source = new SourceFilter
{
Include = new []
{
Property.Path<Article>(p => p.YourField)
}
}
};
Hope it helps.