How can I create FiltersAggregation query using JEST AggregationBuilders or similar? I looked at FiltersAggregationIntegrationTest but query part is defined directly by JSON and I need something more like AggregationBuilders (as I'm using this for standard term aggregation for example)
Link to FiltersAggregationIntegrationTest:
https://github.com/searchbox-io/Jest/blob/master/jest/src/test/java/io/searchbox/core/search/aggregation/FiltersAggregationIntegrationTest.java
I've one possibility:
FilterAggregationBuilder testFilter = AggregationBuilders.filter("test");
testFilter.filter(FilterBuilders.typeFilter("typeName"));
new SearchSourceBuilder().aggregation(testFilter);
This is a filter by Type, but the FilterBuilders has a termFilter too.
Here is a solution that may work, I've cross posted this to the Jest github issue as well.
QueryBuilder filterTermsQuery = QueryBuilders.termsQuery("fieldName", "value1", "value2", "value3");
SearchSourceBuilder searchSourceBuilder = SearchSourceBuilder.searchSource()
.query(boolQueryBuilder)
.size(0)
.aggregation(
AggregationBuilders
.filter("filterAggName") // returns FilterAggregationBuilder
.filter(filterTermsQuery));
The gist is that you want to create a search source builder to use in the jest client, and supply that with and aggregation (which could also include sub-aggregations by chaining sub-aggregations on the AggregationBuilders method). Then define an aggregation of filter type available in AggregationBuilders. This returns a new builder of FilterAggregationBuilder where you can provide any QueryBuilder as the filter aggregation type. According to documentation, the .filter(termsQuery) call will cause only the documents that match the filter to fall into the bucket of this filter.
Hopefully this will resolve your issue unless I misunderstood your use case.
Related
If I'm using aggregations.bucket with a metric, how can I filter that to control the lookback period? Similarly can a filter be used in the same way for extended stats? Here's a code snippet that works (along with the kind of filter I'd like to use):
s = Search(using=client)
s.aggs.bucket('some_bucket_by_day', 'date_histogram', field='time_field', interval='day')
.metric('some_avg', 'avg', field='some_field')
Trying to filter somehow like this:
filter='range', **{'time_field': {'gte': 'now-10d'}}
Also if using extended_stats, could a filter work as well?
s.aggs.bucket('exchange_stats', 'extended_stats', field='some_field')
Thanks!
The filter can be applied at the query level, which will reduce the number of documents on which the aggregations need to be computed. Also extended_stats is a metric aggregation, not a bucket one. So you can do it like this:
// create search and filter the document by date
s = Search(using=client)
.filter('range', time_field={'gte': 'now-10d'})
// add some aggregations
s.aggs.bucket('some_bucket_by_day', 'date_histogram', field='time_field', interval='day')
.metric('some_avg', 'avg', field='some_field')
.metric('exchange_stats', 'extended_stats', field='some_field')
I need to filter over a few fields.
All is ok if I filter over one but adding the second one gives wrong results.
Fields are REQ_ID and CATEGORY
In Kibana, following will return expected results
REQ_ID: '1574778361496' and CATEGORY: 'WARNING'
In java, I tried with:
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
final SimpleQueryStringBuilder sqb = QueryBuilders.simpleQueryStringQuery("REQ_ID: '1574778361496' and CATEGORY: 'WARNING'");
searchSourceBuilder.query(sqb);
searchRequest.source(searchSourceBuilder);
This yields results equivalent to upper having or instead of and, logically.
I also tried:
final BoolQueryBuilder bool = QueryBuilders.boolQuery()
.filter(QueryBuilders.termQuery("REQ_ID", "1574778361496"))
.filter(QueryBuilders.termQuery("CATEGORY", "WARNING"))
;
searchSourceBuilder.query(bool);
This yields no results.
Not sure if this is a bug due to strange nature of high rest java client or am I reading the wrong manual?
client version: 7.4.2
elk version(sebp/elk:740): 7.4.0
Kibana use the Kibana Query Language which is a little different of the Lucene language.
You can switch to Lucene on the top right of Kibana.
But you need no use the correct Lucene syntax (both in kibana with lucene syntax and in your java code), with upper operators :
REQ_ID: 'id1' AND CATEGORY: 'WARNING'
^^^
And you must use a QueryStringQueryBuilder instead of a SimpleQueryStringBuilder :
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
final QueryStringQueryBuilder sqb = QueryBuilders.queryStringQuery("REQ_ID: 'id1' and CATEGORY: 'WARNING'");
searchSourceBuilder.query(sqb);
searchRequest.source(searchSourceBuilder);
You do not need the SimpleQueryString query for your use-case. Using a TermQuery is fine. I recommend to use the un-analyzed field in this case since you are only matching on an exact value. You could wrap this in a Bool clause within filter clauses. Example:
QueryBuilder query = QueryBuilders.boolQuery()
.filter(QueryBuilders.termQuery("REQ_ID.keyword", "1574778361496"))
.filter(QueryBuilders.termQuery("CATEGORY.keyword", "WARNING"));
this is query which i have done in sense
GET /_search?q=2016
it searches in whole db and get results for all entries which has "2016" in any field.
Giving q as query paramater run as query string query. So you have to use java api for Query String. You can use :
SearchResponse response = client.prepareSearch("index_name")
.setTypes("type1Name")
.setQuery(QueryBuilders.queryString("2016"))
.execute()
.actionGet();
I have a repository method that does a "starts with" (prefix) query on field userAccount.userName. When I search for string without space, it returns proper results. But when I search for strings with space in it, it throws an exception.
My repository method:
public List<EsUser> findByUserAccountUserNameStartingWith(String term);
Search String: Tom Cruise
Exception:
org.springframework.dao.InvalidDataAccessApiUsageException: Cannot
constructQuery '*"Tom Cruise"'. Use expression or multiple clauses
instead.
Queries against elasticsearch that use wildcards (e.g. *) must be a single token. By default, tokens are split by white space. "Tom Cruise" is two tokens.
If you need to include multiple tokens, consider implementing a custom Spring Data ES repository and use the following Elasticsearch API QueryBuilder. Something like this:
NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
QueryBuilder matchPhraseQuery = QueryBuilders.matchPhrasePrefixQuery("userName", "Tom Cruise");
QueryBuilder nestedQuery = QueryBuilders.nestedQuery("userAccount", matchPhraseQuery);
nativeSearchQueryBuilder.withQuery(nestedQuery);
NativeSearchQuery nativeSearchQuery = nativeSearchQueryBuilder.build();
//auto wire elastic search template
FacetedPage<EsUser> results = template.queryForPage(nativeSearchQuery, EsUser.class);
I have an existing ElasticSearch query that uses the Java API:
BoolQueryBuilder queryBuilder =
boolQuery().should(queryStringQuery(theUsersQueryString));
SearchResponse response = client.prepareSearch(...).setQuery(queryBuilder);
Now I want to add fuzziness to this, to allow minor misspellings to still return something to the user. My guess was that adding fuzziness parameters to the QueryBuilders object would be fruitful:
boolQuery().should(queryStringQuery(theUsersQueryString)
.fuzziness(Fuzziness.ONE)
.fuzzyMaxExpansions(4)
.fuzzyPrefixLength(2));
Unfortunately this doesn't seem to work and I have so far been unable to find good documentation for this. For example, I have the string John Deere in my database. If I use the query string deere I get a match, but not if I use query strings Deeree or Deeer.
My question is: how should I correctly fuzzify my query?
I opted to create a new query rather than modifying my existing one.
MultiMatchQueryBuilder fuzzyMmQueryBuilder = multiMatchQuery(
theUsersQueryString, "field1", "field2", ... , "fieldn").fuzziness("AUTO");
BoolQueryBuilder b = boolQuery().should(fuzzyMmQueryBuilder);
SearchRequestBuilder srb = client.prepareSearch(...).setQuery(b)...
SearchResponse res = srb.execute().actionGet();
This query exhibits fuzzy behaviour.