Is it possible to exclude certain field from result ? I'm using filtered query like this:
{
"size": 10,
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"text": {
"name": {
"query": "list",
"operator": "or",
"boost": 30
}
}
},
{
"text": {
"field2": {
"query": "list",
"operator": "or",
"boost": 0.2
}
}
},
{
"text": {
"field1": {
"query": "list",
"operator": "or",
"boost": 0.02
}
}
}
]
}
},
"filter": {
"and": [
{
"term": {
"_type": "product"
}
}
]
}
},
"filter": {
"partial_fields": {
"exclude": "field3"
}
}
},
"sort": [
{
"_score": "desc"
}
]
}
I've added filter partial_fields but it does not seem to have any effect. I'm using ES 0.9
Keep in mind that partial_fields support has been deprecated as of 1.0.0beta -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html#partial
I know you're on 0.9 but at some point you'll need to upgrade and this approach won't work. I'd suggest upgrading to a 1.x release and using source filtering instead:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html
partial_fields can return a partial representation of _source based on include and exclude patterns
So i guess you should specify a wildcard pattern for field name in exclude. If your field name is DATA then the exclude pattern should be DAT*..
Related
I have a query where I'm trying pull documents out of my index and sort them by a date. Additionally, if the document's ID matches a provided one then I boost that result.
When I run my query I'm noticing that some of the documents with a more recent sort date are not at the top of the results because Elasticsearch is giving them a different score than other documents. As a result my result order is incorrect. I don't see anything in my query that could be affecting the score. Anyone have any idea what's happening?
Here's the query I'm using:
{
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"match": {
"language.keyword": {
"query": "english",
"operator": "OR",
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"functions": [
{
"filter": {
"match": {
"id": {
"query": "ID1",
"operator": "OR",
"boost": 1
}
}
},
"weight": 10
}
],
"score_mode": "multiply",
"boost_mode": "multiply",
"boost": 1
}
},
"sort": [
{
"_score": {
"order": "desc"
}
},
{
"sortDate": {
"order": "desc"
}
}
]
}
I used the following filter and then searched for query string using Lucene to get the view that I was looking for.
{
"query": {
"match": {
"eventSource": {
"query": "ec2.amazonaws.com",
"type": "phrase"
}
}
}
}
I do not want to return event names those start with the word describe or get. Rest of the event names from ec2 event source should be returned.
!(eventName.keyword: Describe* OR eventName.keyword:
Get* )
The question is how to combine these 2 search requests into one?
I need to use that query from my application.
Update:
The Inspect menu of Kibana Discover tab generates this query. I am just trying to rewrite query_string part with usual match or match_phrase using boolean OR clause.
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "!(eventName.keyword: Describe* OR eventName.keyword: Get* )",
"analyze_wildcard": true
}
},
{
"match_phrase": {
"eventSource": {
"query": "ec2.amazonaws.com"
}
}
},
{
"range": {
"#timestamp": {
"format": "strict_date_optional_time",
"gte": "2020-07-09T08:39:15.947Z",
"lte": "2020-07-24T08:39:15.947Z"
}
}
}
],
"filter": [],
"should": [],
"must_not": []
}
}
You can easily use the boolean query's must_not clause to exclude the documents which you don't want in your search result and you can add as many as must_not as you want, it's fairly easy to do and can be done in a single query.
Please refer the example in the same link to get more info. Created sample in my local to show your the correct query, Please note instead of wildcard I am using the prefix query which is better and server your use-case.
Create index mapping
{
"mappings": {
"properties": {
"eventName": {
"type": "keyword"
}
}
}
}
Index sample doc
{
"eventName" : "Describe the events"
}
{
"eventName" : "the Describe events"
}
{
"eventName" : "Get the event"
}
{
"eventName" : "event Get"
}
Now search query to get only 2 and 3rd doc according to your req
{
"query": {
"bool": {
"must_not": [
{
"prefix": {
"eventName": "Desc"
}
},
{
"prefix": {
"eventName": "Get"
}
}
]
}
}
}
Search result
"hits": [
{
"_index": "ngramkey",
"_type": "_doc",
"_id": "2",
"_score": 0.0,
"_source": {
"eventName": "the Describe events"
}
},
{
"_index": "ngramkey",
"_type": "_doc",
"_id": "4",
"_score": 0.0,
"_source": {
"eventName": "event Get"
}
}
]
As suggested by the user "Opster Elasticsearch Ninja", I have merged must not boolean query like this...
{
"query": {
"bool": {
"must": [
{
"bool": {
"must_not": [
{
"prefix": {
"eventName.keyword": "Desc"
}
},
{
"prefix": {
"eventName.keyword": "Get"
}
}
]
}
},
{
"match_phrase": {
"eventSource": {
"query": "ec2.amazonaws.com"
}
}
},
{
"range": {
"#timestamp": {
"format": "strict_date_optional_time",
"gte": "2020-07-09T08:39:15.947Z",
"lte": "2020-07-24T08:39:15.947Z"
}
}
}
],
"filter": [],
"should": [],
"must_not": []
}
}
}
Here is my current elastic query:
{
"from": 0,
"size": 10,
"query": {
"function_score": {
"query": {
"bool": {
"must": [{
"multi_match": {
"query": "ocean",
"fields": [],
"fuzziness": "AUTO"
}}],
"must_not": [{
"exists": {
"field": "parentId"
}
}]
}
},
"functions" : [
{
"gauss": {
"createTime": {
"origin": "2020-07-09T23:50:00",
"scale": "365d",
"decay": 0.3
}
}
}
]
}
}
}
How do I properly add filters to this? I think maybe the fact that I'm using function_score makes this different? I would like to add a hard filter, for example, only show me results with uploadUser: 'Mr. Bean' ... but still keep the scoring in place for the results that pass this filter.
I tried using filter in various places, also using must but I either get no results or all the results.
I'm using Elastic Search 7. Thanks for your help
You can try this below search query:
Refer this ES official documentation to know more about Function score query
{
"from": 0,
"size": 10,
"query": {
"function_score": {
"query": {
"bool": {
"filter": {
"term": {
"uploadUser": "Mr. Bean"
}
},
"must": [
{
"multi_match": {
"query": "ocean",
"fields": [
],
"fuzziness": "AUTO"
}
}
],
"must_not": [
{
"exists": {
"field": "parentId"
}
}
]
}
},
"functions": [
{
"gauss": {
"createTime": {
"origin": "2020-07-09T23:50:00",
"scale": "365d",
"decay": 0.3
}
}
}
]
}
}
}
Hello elastic experts!
I am new to elasticsearch. I am trying to build a search query with multiple or matching. I am boosting the query for different matching conditions. But here I need a bit more information. I need to know which conditions contributed to the search result. Is there any way to know which match conditions were hit by the query string?
{
"query": {
"bool": {
"should": [
{
"term": {
"title.keyword": {
"value": "Ski trip",
"boost": 1
}
}
},
{
"match_phrase_prefix": {
"title": {
"query": "Ski trip",
"boost": 0.8
}
}
},
{
"match": {
"title": {
"query": "Ski trip",
"operator": "and",
"boost": 0.6
}
}
},
{
"match": {
"description": {
"query": "Ski trip",
"boost": 0.3
}
}
}
]
}
}
}
Is it possible to use facet script and facet filter in elasticsearch like this?
{
"facets": {
"judges": {
"terms": {
"field": "judges.untouched",
"size": 10,
"all_terms": false,
"script": { "script": "...", "params": { }}
},
"global_facets": false,
"facet_filter": {
"and": [
{
"query": {
"query_string": {
"query": "..... ",
"fields": [
"judges.analyzed"
],
"default_operator": "and",
"analyze_wildcard": true
}
}
}
]
}
}
}
}
Because when i run this query, elasticsearch raises error: Parse Failure [No facet type found for [and]]]; }.
Thanks
EDIT Incorrect answer. I'm leaving it because of context.
To clarify: and is an appropriate filter and should be accepted by facet_filter. Not sure what's up.
Untested, but from the docs: (http://www.elasticsearch.org/guide/reference/api/search/facets/)
All facets can be configured with an additional filter (explained in the Query DSL section)
So you need to put an appropriate query in facet_filter. And is NOT an appropriate filter (the error you receive could be clearer)
e.g:
"facet_filter" : {
"term" : { "user" : "kimchy"}
}
You'd probably want something like:
"facet_filter" : {
"query_string": {
"query": "..... ",
"fields": [
"judges.analyzed"
],
"default_operator": "and",
"analyze_wildcard": true
}
}
The syntax for an and filter is:
"facet_filter": {
"and": {
"filters": [
{
// filter definition
},
{
// another filter definition
}
]
}
}
But you're using only a single condition, so there's no need of an and filter.
You should just have:
"facet_filter": {
"query": {
"query_string": {
"query": "..."
}
}
}