I am aware of how using the facet feature of elasticsearch, we can get the aggregated value of values for a specified field/s based on search query result data.
I have an application where I am monitoring logs and using elasticsearch to search through the log entries. On UI front I have a paging mechanism in place and hence using async feature of the search to fetch 'n' entries at a time.
So my question is, if I modify my async search query to fetch the facet information for certain fields, will it give the aggregated value for the sub-set of result that is fetched as a result of an async query. or will it get the aggregated value for the entire search result (and not the sub-set which is returned to user).
Many thanks and regards,
Komal
Facets are returned for the entire search result. You can even set size to 0 in your request, which will result in not fetching any results and you will still get all facets.
Please refer here for detail documentation. You can give match all query to fetch facet on all documents {
"query" : {
"match_all" : { }
},
"facets" : {
"tag" : {
"terms" : {
"field" : "tag",
"size" : 10
}
}
}
}
Please post your code gist for more information.
Related
I need to query the entire index based on particular text value. I don't have field name to query for. Is it possible to search the documents based on particular text?
You can use query string.
You can specify multiple fields. If no field is specified it will search in entire document
{
"query": {
"query_string" : {
"query" : "text"
}
}
}
{
"query": {
"nested": {
"path": "product_vendors",
"query": {
"bool" :{
"must" : {
"bool" : {
"should" : [
{ "terms": {"product_vendors.manufacturer_style":["FSS235D-26","SG463-1128-5","SG463-2879-4"]}},
{ "terms": {"product_vendors.id":["71320"]}}
]
}
}
}
}
}
}
}
I have above elastic query, not able to understand this. Would anyone please explain what it means and what documents it will return?
Update : #christinabo , i tried your query , and results returned , but here some small issues , apart from the matched documents , two more additional documents are returning in those documents only vendor_id is matching , may i know why two extra unmatched documents are returning , do we need to some attribute or something to make sure strict search and return is allowed , can please suggest on this .
By observing the query, I can understand that there is a nested object in the data. I can imagine that it has this structure:
product_vendors: {
'id': 'the_id',
'manufacturer_style': 'some style'
}
In order to query a nested object, you need a nested query. This is why you have the nested keyword there. In a nested query, you need to specify the path (product_vendors) that leads to the embedded fields (id, manufacturer_style).
Then, the query defines a bool query with the must keyword, which means that the query which follows must appear in matching documents. In this case, what it must appear is another bool query, defined with the should keyword. This contains two terms sub-queries (one for manufacturer_style and one for id) and means that the matching documents should match one or two of them. Each sub-query queries the embedded field by specifying the whole route of the nested object, using the dot (i.e. product_vendors.manufacturer_style).
I would expect the query to return you the documents that match at least one of the terms queries, with the documents that match both to have higher score.
I hope that this explanation gives you an overall idea of this query.
More about bool queries from the documentation here.
I'm working on a tinder like app. In order to exclude profiles that user has swiped before, I use a "must_not" query like this:
must_not : [{"terms": { "swipedusers": ["userid1", "userid1", "userid1"…]}}]
I wonder what are the limits using this approach? is this a scalable approach that would also work when the swipedusers array contains 2000 user ids? If there is a better scalable approach to this I would be happy to know...
there is a better approach! and it called "terms lookup", is something like the traditional join that you could do on relational databases...
I could try to explain you here, but, all the information that you need is well documented on the official Elastic Search page:
https://www.elastic.co/guide/en/elasticsearch/reference/5.0/query-dsl-terms-query.html#query-dsl-terms-lookup
The final solution is having 2 indices, one for the registered users and another one to track swipes for each user.
Then, for each swipe, you should update the document containing current user swipes... Here you will need to add elements to an array, and this is another problem in ElasticSearch (big problem if you are using AWS managed ElasticSearch) that only can be solved using scripting...
More info at https://www.elastic.co/guide/en/elasticsearch/guide/current/partial-updates.html#_using_scripts_to_make_partial_updates
For your case, the query will result in something like:
GET /possible_matches/_search
{
"query" : {
"terms" : {
"user" : {
"index" : "swiped",
"type" : "users",
"id" : "current-user-id",
"path" : "swipedUserId"
}
}
}
}
Another thing that you should take in account is the replication configuration for the swipes index, since each node will perform "joins" with that index, is highly recommended to have a full copy of that index in each node. You could achieve this creating the index with the "auto_expand_replicas" with "0-all" value.
PUT /swipes
{
"settings": {
"auto_expand_replicas": "0-all"
}
}
I'm moving a sql server product catalog over to elasticsearch and want to preserve how the ui currently allows the user to navigate the options. I am using aggregates with post filter but cannot get the selected options siblings to show up in the aggregates.
An example of what I am trying to achieve is from the elastic docs.
GET /cars/transactions/_search
{
"size" : 0,
"query": {
"match": {
"make": "ford"
}
},
"post_filter": {
"term" : {
"color" : "green"
}
},
"aggs" : {
"all_colors": {
"terms" : { "field" : "color" }
}
}
}
So, the user has clicked on the green option and the returned documents show only green ford cars, but the aggregates list all of the colors available for ford with their counts, which can be added to a ui.
All of this is ok. But, there are many makes of car other than ford. If I added a 'makes' aggregate, then this query will only return ford in the aggregates list. If building the navigation ui dynamically from the returned results (as I am), then there would be no way to place all the other makes of car into the ui, unless I queried elasticsearch many times to build up my ui - which I don't want to do.
If I changed the query to a match-all and added the query to the post filter, then I would get the full list of car makes in the aggregation, but the counts would always be a global count from the match-all query and not reflective of the drill-down count.
Is it possible to do this with elasticsearch? I've gone through all the documents - several times, and tried many different query formats, but nothing has produced quite the right results so far.
Is it possible to do a suggestion completion on a type? I'm able to do it on an index.
POST /data/_suggest
{
"data" : {
"text" : "tr",
"completion" : {
"field" : "sattributes",
"size":50
}
}
}
when I do on a type:
POST /data/suggestion/_suggest
{
"data" : {
"text" : "tr",
"completion" : {
"field" : "sattributes",
"size":50
}
}
}
suggestion is the type.
I don't get any results. I need to do suggestion on two different types articles and books. Do I need to create separate indexes to make them work or is there a way in elasticsearch to accomplish this? In case if I have to search on my index data is there way to get 50 results for type article and 50 results for type book.
Any help is highly appreciated.
Lucene has no concept of types, so in Elasticsearch they are simply implemented as a hidden field called _type. When you search on a particular type, Elasticsearch adds a filter on that field.
The completion suggester doesn't use traditional search at all, which means that it can't apply a filter on the _type field. So you have a couple of options:
Use a different completion suggester field per type, eg suggestion_sattributes, othertype_sattributes
Index your data with the _type as a prefix, eg type1 actual words to suggest, then when you ask for suggestions, prepend type1 to the query
Use separate indices
In fact, option (2) above is being implemented at the moment as the new ContextSuggester which will allow you to do this (and more) automatically.