malformed query, expected [END_OBJECT] but found [FIELD_NAME] - elasticsearch

The original query looks like this
{
"query": {
"bool": {
"must": [
{
"has_parent": {
"parent_type": "doc",
"query": {
"bool": {
"must": [
{
"terms": {
"id": [
713
]
}
},
{
"range": {
"created": {
"lte": "now/d"
}
}
},
{
"range": {
"expires": {
"gte": "now/d"
}
}
}
]
}
}
}
},
{
"term": {
"doc_type": "item"
}
},
{
"bool": {
"should": [
{
"term": {
"have_prices": true
}
},
{
"term": {
"is_folder": true
}
}
]
}
}
],
"must_not": {
"exists": {
"field": "folder"
}
}
}
},
"sort": [
{
"is_folder": {
"order": "desc"
}
},
{
"title_low.order": {
"order": "asc"
}
}
],
"size": 1000
}
And I got some result
"hits": {
"total": 19,
"max_score": null,
"hits": [
{
"_index": "prices",
"_type": "doc",
"_id": "item-6800004",
"_score": null,
"_routing": "1",
"_source": {
"id": 6800004,
"id_pricedoc": 713,
"title": "\"водка №1\" 1",
"title_low": "\"водка №1\" 1",
"supplier": {
"id": 7831,
"type": null
},
"supplier_nom": {
"id": 1375697,
"market_nom": {
"id": null
},
"codes": null,
"sup_code": "7a6713a5-73c1-3acb-9b62-9e38b2314dce",
"manufacturer": {
"id": null,
"title": null
}
},
"is_folder": false,
"folder": null,
"path": null,
"pricedoc_created": "2016-03-21",
"prices": [
{
"currency": "RUR",
"id_offer": 15735967,
"id_prcknd": 167,
"value": "391.50"
}
],
"have_prices": true,
"market": {
"title": null,
"uuid": null,
"folder": null,
"path": null
},
"_join_field_name": "doc_type",
"doc_type": {
"name": "item",
"parent": "doc-713"
}
},
"sort": [
0,
"\"водка №1\" 1"
]
}
Now I also would like get result where "id_prcknd": 167
Modified query looks like this
{
"query": {
"bool": {
"must": [
{
"has_parent": {
"parent_type": "doc",
"query": {
"bool": {
"must": [
{
"terms": {
"id": [
713
]
}
},
{
"range": {
"created": {
"lte": "now/d"
}
}
},
{
"range": {
"expires": {
"gte": "now/d"
}
}
}
]
}
}
}
},
{
"term": {
"doc_type": "item"
}
},
{
"bool": {
"should": [
{
"term": {
"have_prices": true
}
},
{
"term": {
"is_folder": true
}
}
]
}
}
],
"must_not": {
"exists": {
"field": "folder"
}
}
},
"nested": {
"path": "prices",
"query": {
"bool": {
"must": [
{
"match": {
"prices.id_prcknd": 167
}
}
]
}
}
},
"sort": [
{
"is_folder": {
"order": "desc"
}
},
{
"title_low.order": {
"order": "asc"
}
}
],
"size": 1000
}}
But I got an error Elasticsearch malformed query, expected [END_OBJECT] but found [FIELD_NAME]
Where am I wrong?
I wanna match objects where "id_prcknd": 167
The stackoverflow says the I post mostly the code, but it's because of large queries in elastic search.

What Elasticsearch was trying to say is that it did not expect to see other keys but bool in the dictionary (the value under "query").
In your example code you have something like this:
{
"query": {
"bool": {
"must": [
{...},
{...},
{...}
],
"must_not": {...},
"nested": {...}, // this should go under "must"
"sort": [...], // this should go on the same level as "query"
"size": 1000 // this should go on the same level as "query"
}
}
"bool" here refers to a bool query, and should be the only key in the dictionary.
What you should do is to move "nested" into its own dictionary and the fourth element of the must array (if I understood the logic of what you are trying to achieve correctly). Please note that also "sort" and "size" should be moved - this time, to the same level as "query".
The full query will look like this:
{
"query": {
"bool": {
"must": [
{
"has_parent": {
"parent_type": "doc",
"query": {
"bool": {
"must": [
{
"terms": {
"id": [
713
]
}
},
{
"range": {
"created": {
"lte": "now/d"
}
}
},
{
"range": {
"expires": {
"gte": "now/d"
}
}
}
]
}
}
}
},
{
"term": {
"doc_type": "item"
}
},
{
"bool": {
"should": [
{
"term": {
"have_prices": true
}
},
{
"term": {
"is_folder": true
}
}
]
}
},
{
"nested": {
"path": "prices",
"query": {
"bool": {
"must": [
{
"match": {
"prices.id_prcknd": 167
}
}
]
}
}
}
}
],
"must_not": {
"exists": {
"field": "folder"
}
}
}
},
"sort": [
{
"is_folder": {
"order": "desc"
}
},
{
"title_low.order": {
"order": "asc"
}
}
],
"size": 1000
}

Your json is incorrect: Error:Expecting closing } at end[Code 22, Structure 183]
Use a json validator (https://jsonformatter.curiousconcept.com/) for example.

Related

Custom sort after score calculation and the value of specific field

Need help to understand sorting in a query where _score is either 0 or 1 based on some conditions
but there is also an additional_filed type integer, sort should be in the order of _score * additional_field
sample index
PUT /product_t
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"placed" :{
"type": "integer"
},
"store" : {
"type": "nested"
}
}
}
}
sample document
PUT /product_t/_doc/10
{
"name": "awesome",
"count_sold": 199,
"stock": [
{
"id": 1,
"count": 10
},
{
"id": 2,
"count": 5
},
{
"id": 3,
"count": 0
}
]
}
Query Used :
POST /product_t/_search
{
"from": 0,
"size": 100,
"timeout": "300ms",
"query": {
"bool": {
"filter": [
{
"match": {
"name": {
"value": "awesome"
}
}
},
{
"nested": {
"path": "stock",
"query": {
"bool": {
"must": [
{
"match": {
"stock.id": 3
}
}
]
}
}
}
}
],
"should": [
{
"constant_score": {
"filter": {
"nested": {
"path": "stock",
"query": {
"bool": {
"must": [
{
"match": {
"stock.id": 3
}
},
{
"range": {
"stock.count": {
"gt": 0
}
}
}
]
}
}
}
},
"boost": 1
}
},
{
"constant_score": {
"filter": {
"nested": {
"path": "stock",
"query": {
"bool": {
"must": [
{
"match": {
"stock.id": 3
}
},
{
"range": {
"stock.count": {
"lte": 0
}
}
}
]
}
}
}
},
"boost": 0
}
}
]
}
}
}
count_sold is additional field
You can use script sort
"sort": {
"_script": {
"type": "number",
"script": {
"lang": "painless",
"source": "doc['count_sold'].value * _score"
},
"order": "asc"
}
}

What is wrong with my elasticsearch query ? Getting a expected end object error

I'm trying to do a elasticsearch query that does geolocation filter and does some matching on nested documents, but I'm getting this error whenever I add in the nested query.
"[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]"
{
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"query": {
"bool": {
"filter": {
"geo_distance": {
"distance": "10km",
"geolocation": [
-73.980090948125,
40.747844918436
]
}
},
"must": {
"multi_match": {
"query": "New York",
"fields": [
"name^2",
"city",
"state",
"zip"
],
"type": "best_fields"
}
}
},
"nested": {
"path": "amenities",
"query": {
"bool": {
"must": [
{
"match": {
"amenities.name": "Pool"
}
}
]
}
}
}
},
"aggs": {
"reviews": {
"nested": {
"path": "reviews"
},
"aggs": {
"avg_rating": {
"avg": {
"field": "reviews.rating"
}
}
}
}
}
}
You just has misplaced the nested query, try like this:
{
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"query": {
"bool": {
"filter": {
"geo_distance": {
"distance": "10km",
"geolocation": [
-73.980090948125,
40.747844918436
]
}
},
"must": [
{
"multi_match": {
"query": "New York",
"fields": [
"name^2",
"city",
"state",
"zip"
],
"type": "best_fields"
}
},
{
"nested": {
"path": "amenities",
"query": {
"match": {
"amenities.name": "Pool"
}
}
}
}
]
}
},
"aggs": {
"reviews": {
"nested": {
"path": "reviews"
},
"aggs": {
"avg_rating": {
"avg": {
"field": "reviews.rating"
}
}
}
}
}
}

Unsure how to structure ElasticSearch Range Query with nested properties

My object in ES looks like:
{
"_index": "myIndex",
"_type": "myType",
"_id": "75fd98d2-eca7-4a94-9dd8-1cc2c9b1fbbf",
"_version": 2,
"found": true,
"_source": {
"account_id": "100",
"import_ids": [
"4f4eef42-5493-464e-ac08-68a3a25a01fb"
],
"accept": "html",
"deleted_at": null,
"signup_form_ids": [
{
"timestamp": "2015-11-23T20:08:11.604000",
"signup_form_id": "1234"
}
],
"mailing_status": "active",
"group_ids": [
"0eddd2c0-ce70-4eb7-bcd8-9e41e41ac0b3"
],
"confirmed_opt_in_at": null,
"fields": [
{
"text_value": "My Company",
"name": "company"
},
{
"text_value": "Foo",
"name": "first-name"
},
{
"text_value": "Bar",
"name": "last_name"
},
{
"text_value": "555-555-5555",
"name": "phone"
}
],
"created_at": "2015-11-23T19:20:15.889000",
"last_modified_at": "2015-11-23T20:08:11.604000",
"bounce_count": 0,
"opted_out_at": null,
"archived_at": null,
"email": "example#example.com",
"opt_out_mailing_id": "None"
}
}
I am trying to run write a query that gives me all hits where the signup_form_ids.timestamp are lte now-7d/d. I'm looking at https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html#ranges-on-dates but unsure how to structure the query
This is what I have so far:
{
"query": {
"nested": {
"path": "signup_form_ids",
"bool": {
"must": [
{
"range": {
"timestamp" {
"lte": "now-7d/d"
}
}
}
]
}
},
"bool": {
"must": [
{
"bool": {
"must": []
}
},
{
"match": {
"account_id": "100"
}
},
{
"filtered": {
"filter": {
"missing": {
"field": "deleted_at"
}
}
}
}
]
}
},
"size": 500,
"from": 0
}
There are several things wrong here, and it's not entirely obvious which ones are artifacts of you adjusting your query to post here.
First, you're missing a colon after "timestamp" in your query. Also, you have an empty inner "bool". And your "range" query is inside a needless "bool". Also your "filtered" clause is redundant and you can just use the "filter" inside it.
But the main problems are that 1) your "nested" query needs to be inside your "bool" if you want all the conditions to apply, 2) your "nested" "range" filter needs to specify the full path to "timestamp" and 3) the "bool" inside your "nested" clause needs to be in a "filter".
So, making minimal adjustments to make the query work, the following query returns the document you posted (I changed the "lte" to "gte" so the document you posted would be returned, otherwise it doesn't match the query, yet):
POST /test_index/_search
{
"query": {
"bool": {
"must": [
{
"bool": {
"must": []
}
},
{
"match": {
"account_id": "100"
}
},
{
"filtered": {
"filter": {
"missing": {
"field": "deleted_at"
}
}
}
},
{
"nested": {
"path": "signup_form_ids",
"filter": {
"bool": {
"must": [
{
"range": {
"signup_form_ids.timestamp": {
"gte": "now-7d/d"
}
}
}
]
}
}
}
}
]
}
},
"size": 500,
"from": 0
}
If I clean it up to remove all the redundancies, I end up with:
POST /test_index/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"account_id": "100"
}
},
{
"missing": {
"field": "deleted_at"
}
},
{
"nested": {
"path": "signup_form_ids",
"filter": {
"range": {
"signup_form_ids.timestamp": {
"gte": "now-7d/d"
}
}
}
}
}
]
}
},
"size": 500,
"from": 0
}
Here is some code I used to play around with it:
http://sense.qbox.io/gist/ee96042c0505dfb07199b919d134b2a20c5a66fd

Nested bool should query not working with the minimum should match

I have the below query in which I want to get the search term "Schwarz" either in the name field or the message field. The language must be Austrian and the status type should be equal to the provided list. I'm getting the following exception and I can't figure out why:
QueryParsingException[[my_test_index] [_na] query malformed, no field after start_object]; }]",
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"term": {
"name": "Schwarz"
}
},
{
"term": {
"message": "Schwarz"
}
}
],
"minimum_should_match": 1
}
},
{
"terms": {
"status_type": [
"1",
"2",
"3",
"4",
"5",
"6",
"7"
]
}
},
{
"term": {
"language": "Austrian"
}
}
]
}
}
}
},
"sort": [
{
"total": {
"order": "desc"
}
}
]
}
Here is the query without the filter that still works:
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"match": {
"standard_analyzed_name": "Schwarz"
}
},
{
"match": {
"standard_analyzed_message": "Schwarz"
}
}
],
"must": [
{
"terms": {
"buzzsumo_status_type": [
"1",
"2",
"3",
"4",
"5",
"6",
"7"
]
}
},
{
"term": {
"language": "Austrian"
}
}
]
}
}
}
},
"sort": [
{
"total_interactions": {
"order": "desc"
}
}
]
}
In a filtered query you MUST have a filter part, which is not present here. I would suggest rewriting it like this, i.e. move the terms and term parts into the filter part:
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"term": {
"name": "Schwarz"
}
},
{
"term": {
"message": "Schwarz"
}
}
],
"minimum_should_match": 1
}
},
"filter": {
"bool": {
"must": [
{
"terms": {
"status_type": [
"1",
"2",
"3",
"4",
"5",
"6",
"7"
]
}
},
{
"term": {
"language": "Austrian"
}
}
]
}
}
}
},
"sort": [
{
"total": {
"order": "desc"
}
}
]
}
An alternative is to not use the filtered query and simply write it like this:
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"term": {
"name": "Schwarz"
}
},
{
"term": {
"message": "Schwarz"
}
}
],
"minimum_should_match": 1
}
},
{
"terms": {
"status_type": [
"1",
"2",
"3",
"4",
"5",
"6",
"7"
]
}
},
{
"term": {
"language": "Austrian"
}
}
]
}
},
"sort": [
{
"total": {
"order": "desc"
}
}
]
}

ElasticSearch: How to write query where string field is either null or empty?

I want to check for documents that have media_url == '' || media_url == null. I have a query:
{
"engagements": [
"blah"
],
"query": {
"from": 0,
"size": 2,
"sort": [
{
"bookmarked": {
"order": "desc"
}
},
{
"created_at": {
"order": "desc"
}
}
],
"facets": {},
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"car_id": "78778"
}
},
{
"range": {
"created_at": {
"gte": "2015-04-12T04:00:00.000Z",
"lte": "2015-05-13T03:59:59.999Z"
}
}
},
{
"term": {
"media_url": ""
}
}
],
"should": [
{
"term": {
"bookmarked": false
}
}
]
}
}
}
},
"aggregations": {
"word_frequencies": {
"terms": {
"field": "text",
"size": 150
}
}
},
"highlight": {
"fields": {
"text": {
"fragment_size": 1500
}
}
}
},
"api": "_search"
}
However, if I do what I do above, then records that are set to null wouldn't be returned. What should I do to return records with either '' or null as their media_url value?
Perhaps you can try using the "or" filter.
http://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-or-filter.html
{
"or": [
{
"term": {
"media_url": ""
}
},
{
"term": {
"media_url": null
}
}
]
}
Edit: Here's the full query (untested since I don't have an example document/index template)
{
"engagements": [
"blah"
],
"query": {
"from": 0,
"size": 2,
"sort": [
{
"bookmarked": {
"order": "desc"
}
},
{
"created_at": {
"order": "desc"
}
}
],
"facets": {},
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"car_id": "78778"
}
},
{
"range": {
"created_at": {
"gte": "2015-04-12T04:00:00.000Z",
"lte": "2015-05-13T03:59:59.999Z"
}
}
},
{
"or": [
{
"term": {
"media_url": ""
}
},
{
"term": {
"media_url": null
}
}
]
}
],
"should": [
{
"term": {
"bookmarked": false
}
}
]
}
}
}
},
"aggregations": {
"word_frequencies": {
"terms": {
"field": "text",
"size": 150
}
}
},
"highlight": {
"fields": {
"text": {
"fragment_size": 1500
}
}
}
},
"api": "_search"
}
You can use the missing filter to take care of null value or field itself is missing. You can combine the same with an empty string term to achieve what you want.
{
"or": [
{
"term": {
"media_url": ""
}
},
{
"missing": {
"field": "media_url"
}
}
]
}
Use the above instead of the single term query for "media_url" in the must clause of your Boolean filter.

Resources