I have faceted queries working with elasticsearch 0.19.9. However I would like to return all facets that have a count > 0.
According to the documentation I should be able to:
{
"query" : {
"match_all" : { }
},
"facets" : {
"tag" : {
"terms" : {
"field" : "tag",
"all_terms" : true
}
}
}
}
As I understand, this should give me all facets even if count is 0.
However, this is still only returning the top 10 facets by count. Which is the default size. The only thing that seems to affect the number of returned facets is by actually setting "size" : N where N is the number of facets which will be returned.
I could set this to a really high number but that just seems to hack-ish.
Any ideas as to what I may be doing wrong?
You're not doing anything wrong. You figured it out correctly! There is an open issue on github to make the terms facet similar to the Terms Stats facet which allows you to set size=0 in order to get all the terms back. For now you just need to use an high value, which is a bit tricky, I agree. On the other hand be careful not to return too many entries!
{
"query" : {
"match_all" : { }
},
"facets" : {
"tag" : {
"terms" : {
"field" : "tag",
"size" : 2147483647,
"all_terms" : false
}
}
}
}
The only way to remove the "count: 0" is to put "all_terms" as false, and set your size number as high and as impossible as you can in your Elasticsearch instance (the example above is the largest signed value that an integer in PHP can have).
It may not be the best way, but this is the only known approach so far. Facet filter doesn't work with this at present (unless they updated and improved Elasticsearch to do it).
Related
On my eCommerce store I want to only include the first item in each group (grouped by item_id) in the final results. At the same time I don't want to lose my aggregations (little numbers next to attributes that indicate how many items with that attribute are found).
Here is a little example:
Suppose I make a search for items and only 25 show up. This is the result for the color aggregation that I currently get:
black (65)
green (32)
white (13)
And I want it to be:
black (14)
green (6)
white (5)
The numbers should amount to the total number the user actually sees on the page.
How could I achieve that with Elasticsearch? I have tried both Grouping (Top Hits) and Field Collapsing and both don't seem to fit my use case. Solr does it almost by default with its Grouping functionality.
It should be rather easy. When you are asking for aggregation you are simple sending request to the _search endpoint. Example:
POST /exams/_search
{
"aggs" : {
"avg_grade" : { "avg" : { "field" : "grade" } }
}
}
and in above example you will get aggregation for all the documents.
If you want to get aggregation for specific documents you just need to add specific query to the request body, like:
POST /exams/_search
{
"query": {
"bool" : {
"must" : {
"query_string" : {
"query" : "some query string here"
}
},
"filter" : {
"term" : { "user" : "kimchy" }
}
}
},
"aggs" : {
"avg_grade" : { "avg" : { "field" : "grade" } }
}
}
and you can send size and from parameters as well.
I am using terms aggregation on elasticsearch to get most used words in a index with 380607390 (380 millions) and i receive timeout on my application.
The aggregated field is a text with a simple analyzer( the field holds post content).
My question is:
The terms aggregation is the correct aggregation to do that? With a large content field?
{
"aggs" : {
"keywords" : {
"terms" : { "field" : "post_content" }
}
}
}
You can try this using min_doc_count. You would ofcourse not want to get those words which have been used just once or twice or thrice...
You can set min_doc_count as per your requirement. This would definitely
reduce the time.
{
"aggs" : {
"keywords" : {
"terms" : { "field" : "post_content",
"min_doc_count": 5 //----->Set it as per your need
}
}
}
}
I have two questions about the query seen on this capture:
How do I order by value in the sum_category field in the results?
I use respsize again in the query but it's not correct as you can see below.
Even if I make only an aggregration, why do all the documents come with the result? I mean, if I make a group by query in SQL it retrieves only grouped data, but Elasticsearch retrieves all documents as if I made a normal search query. How do I skip them?
Try this:
{
"query" : {
"match_all" : {}
},
"size" : 0,
"aggs" : {
"categories" : {
"terms" : {
"field" : "category",
"size" : 999999,
"order" : {
"sum_category" : "desc"
}
},
"aggs" : {
"sum_category" : {
"sum" : {
"field" : "respsize"
}
}
}
}
}
}
1). See the note in (2) for what your sort is doing. As for ordering the categories by the value of sum_category, see the order portion. There appears to be an old and closed issue related to that https://github.com/elastic/elasticsearch/issues/4643 but it worked fine for me with v1.5.2 of Elasticsearch.
2). Although you do not have that match_all query, I think that's probably what you are getting results for. And so the sort your specified is actually getting applied to those results. To not get these back, I just have size: 0 portion.
Do you want buckets for all the categories? I noticed you do not have size specified for the main aggregation. That's the size: 999999 portion.
Is there a way in Elasticsearch through filters, queries, aggregations etc to search for a list of document ids and have returned which ids did not hit?
With a small list it is easy enough to compare the results against the requested ids list but I'm dealing with lists of ids in the tens of thousands and it is not going to be performant to do that.
Do you mean, from https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-not-filter.html
"filtered" : {
"query" : {
"term" : { "name.first" : "shay" }
},
"filter" : {
"not" : {
"range" : {
"postDate" : {
"from" : "2010-03-01",
"to" : "2010-04-01"
}
}
}
}
}
Take a look at the guide at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
elasticsearch provides parameter to exclude certain facets from facets values like this.
"facets" : {
"tag" : {
"terms" : {
"field" : "tag",
"exclude" : ["term1", "term2"]
}
}
}
Is there any possibility to include certain facets?
I'm trying to get counts for facets that have been already selected by user along with global facets. E,g. you selected word science with count 20 (from autocomplete), i recompute facets to show other words that migh be selected, but the word science would not get to facet results since other words from global facets have count more than 400.
Is there any particular solution for this task?
Thanks for help
You can use scripting for that. The script will be run for each facet entry with the input variable term that contains the current value. The entry will be included or not on the final facet depending on the result of the script. If it returns false it will be excluded, otherwise it will be included.
"facets" : {
"tag" : {
"terms" : {
"field" : "tag",
"script" : "term == 'aaa' ? true : false"
}
}
}