I am new to Elasticsearch and working with a database. I would like to do a query like this:
curl -X GET http://localhost:9200/project/flat_order/_search?pretty=true -d'
{ query:{
bool:{
must:{range:{created_at:{gte:"2012-01-01 00:00:00",lte:"2012-02-01 00:00:00"}}}},
{should:[{term:{status:"canceled"}}],minimum_number_should_match:1}
}
}
}'
and I am getting an error.
"error" : "SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[YfeNrRnOTOqLOtQt65uPVw][project][1]: SearchParseException[[project][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [ { query:{ bool:{ must:{range:{created_at:{gte:\"2012-01-01 00:00:00\",lte:\"2012-02-01 00:00:00\"}}}}, {should:[{term:{status:\"canceled\"}}],minimum_number_should_match:1} } } }]]]; nested: QueryParsingException[[project] Failed to parse]; nested: JsonParseException[Unexpected character ('{' (code 123)): was expecting either valid name character (for unquoted name) or double-quote (for quoted) to start field name\n at [Source: [B#3742925a; line: 1, column: 102]]; }{[YfeNrRnOTOqLOtQt65uPVw][project][2]: SearchParseException[[project][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [ { query:{ bool:{ must:{range:{created_at:{gte:\"2012-01-01 00:00:00\",lte:\"2012-02-01 00:00:00\"}}}}, {should:[{term:{status:\"canceled\"}}],minimum_number_should_match:1} } } }]]]; nested: QueryParsingException[[project] Failed to parse]; nested: JsonParseException[Unexpected character ('{' (code 123)): was expecting either valid name character (for unquoted name) or double-quote (for quoted) to start field name\n at [Source: [B#3742925a; line: 1, column: 102]]; }]",
"status" : 500
}
What to do?
Thanks in advance.
This is the error I am getting without those braces(enclosing should)
"error" : "SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[YfeNrRnOTOqLOtQt65uPVw][project][0]: SearchParseException[[project][0]: query[created_at:[1325376000000 TO 1328054400999]],from[-1],size[-1]: Parse Failure [Failed to parse source [ { query:{ bool:{ must:{range:{created_at:{gte:\"2012-01-01 00:00:00\",lte:\"2012-02-01 00:00:00\"}}}}, should:[{term:{status:\"canceled\"}}],minimum_number_should_match:1 } } }]]]; nested: SearchParseException[[project][0]: query[created_at:[1325376000000 TO 1328054400999]],from[-1],size[-1]: Parse Failure [No parser for element [term]]]; }{[YfeNrRnOTOqLOtQt65uPVw][project][4]: SearchParseException[[project][4]: query[created_at:[1325376000000 TO 1328054400999]],from[-1],size[-1]: Parse Failure [Failed to parse source [ { query:{ bool:{ must:{range:{created_at:{gte:\"2012-01-01 00:00:00\",lte:\"2012-02-01 00:00:00\"}}}}, should:[{term:{status:\"canceled\"}}],minimum_number_should_match:1 } } }]]]; nested: SearchParseException[[project][4]: query[created_at:[1325376000000 TO 1328054400999]],from[-1],size[-1]: Parse Failure [No parser for element [term]]]; }]",
"status" : 500
}
Looking at the error detail, I believe the problem is that you've got an extra level of nesting for your should clause which you shouldn't have. Instead of:
must: { ... },
{ should: { ... },
minimum_number_should_match:1}
}
Try:
must: { ... },
should: { ... },
minimum_number_should_match:1}
EDIT: Your edited code looks like this is your query:
query:{ bool:{ must:{range:{created_at:{gte:\"2012-01-01 00:00:00\",
lte:\"2012-02-01 00:00:00\"}}}}, should:[{term:status:\"canceled\"}}],
minimum_number_should_match:1 }
which, formatted, would be:
query:{
bool:{
must:{
range:{created_at:{gte:\"2012-01-01 00:00:00\",lte:\"2012-02-01 00:00:00\"}}
}
},
should: [
{term:{status:\"canceled\"}}
],
minimum_number_should_match:1
}
... which has the should outside the bool part. In other words, you've got too many closing braces after your must clause. I believe it should be:
query:{
bool:{
must:{
range:{created_at:{gte:\"2012-01-01 00:00:00\",lte:\"2012-02-01 00:00:00\"}}
},
should: [
{term:{status:\"canceled\"}}
],
minimum_number_should_match:1
}
}
Related
I am using Elastic Search to implement an autosuggest field for an index called people:
The Mapping is as below for the field person_name_suggest -
person_name_suggest: {
type: "completion",
analyzer: "simple",
payloads: true,
preserve_separators: true,
preserve_position_increments: true,
max_input_length: 50,
context: {
office_scope: {
type: "category",
path: "office_scope",
default: [
"0"
]
}
}
},
The request that I need to Elastic Search is as follows:
{
"suggest":{
"suggestions":{
"text":"M","
completion":{
"field":"person_name_suggest",
"context":890,
"size":10
}
}
}
}
I get the following error -
"error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[OsbgjmewT569a-7ZoNCMtg][people_2016_10_29][0]: SearchParseException[[people_2016_10_29][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"suggest":{"suggestions":{"text":"M","completion":{"field":"person_name_suggest","context":890,"size":10}}}}]]]; nested: ElasticsearchIllegalArgumentException[suggester [completion] requires context to be setup]; }
From what I can see, I have the completion suggester set up right.
Could someone point me in the right direction?
The version of ElasticSearch being used is 1.6
There is a syntax error in query. You have to specify the name of context
Try this
{
"suggest":{
"suggestions":{
"text":"M","
completion":{
"field":"person_name_suggest",
"context":{"office_scope":890},
"size":10
}
}
}
}
I want to use script filter to get all match that the score value equals to maxScore value. but it is not work.
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "doc['match.score'].value == doc['match.maxScore'].value"
}
}
}
}
}
I have this error :
{
"error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[yk_PnKg9SOaf1d8acCsjvw][cobink_search][0]: SearchParseException[[cobink_search][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"filtered":{"filter":{"script":{"script":"doc['match.score'].value == doc['match.maxScore'].value"}}}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[0jYKWZlyQiiVPHlzmoMjkA][cobink_search][1]: RemoteTransportException[[Hammer Harrison][inet[/192.168.1.100:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[cobink_search][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"filtered":{"filter":{"script":{"script":"doc['match.score'].value == doc['match.maxScore'].value"}}}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[yk_PnKg9SOaf1d8acCsjvw][cobink_search][2]: SearchParseException[[cobink_search][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"filtered":{"filter":{"script":{"script":"doc['match.score'].value == doc['match.maxScore'].value"}}}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[yk_PnKg9SOaf1d8acCsjvw][cobink_search][3]: SearchParseException[[cobink_search][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"filtered":{"filter":{"script":{"script":"doc['match.score'].value == doc['match.maxScore'].value"}}}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[0jYKWZlyQiiVPHlzmoMjkA][cobink_search][4]: RemoteTransportException[[Hammer Harrison][inet[/192.168.1.100:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[cobink_search][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"filtered":{"filter":{"script":{"script":"doc['match.score'].value == doc['match.maxScore'].value"}}}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }]",
"status": 400
}
Thanks you guys :)
Dynamic scripting is disabled in your case. Add the following line in your elasticsearch.yml file to enable dynamic scripting. Then your query should work. But beware this can leave a security hole in your Elasticsearch cluster. Read more about it here and here.
script.disable_dynamic: false
In version 2.x and above you need to use
script.indexed: on
script.inline: on
in your elasticsearch.yml file, to enable the scripts.
I am new to the ELK stack and have it implemented with elasticsearch version 1.4.4, logstash version 1.4.2, and kibana version 4. I am able to pull a csv file into elasticsearch using logstash and have it display in kibana.
When displaying a date from the file, the values within the date are separated out as if the dash contained within is a separator (ex. value in field is 01-01-2015, when this is displayed in kibana (regardless of display type) there will be three field entries, 01, 01, and 2015). Kibana gives a message that this is due to it being an analyzed field.
Kibana 4 has a feature to use json directly from the dashboard builder, Visualization, to change this to a non-analyzed field so that the entire string will be used, rather than separating it.
I have tried multiple formats, but this is the one that seems it should work as kibana recognizes it as valid syntax:
{ "index" : "not_analyzed" }
but when attempting to apply the change, the dashboard does not change its structure and kibana generates the following exception:
Visualize: Request to Elasticsearch failed: {"error":"SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][0]: SearchParseException[[csvtest][0]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][0]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][1]: SearchParseException[[csvtest][1]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][1]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][2]: SearchParseException[[csvtest][2]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][2]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][3]: SearchParseException[[csvtest][3]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][3]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][4]: SearchParseException[[csvtest][4]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][4]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }]"} less
It can be seen within where the index: was changed to not_analyzed from analyzed; also the setting that has wildcard analyzed: true was also changed to false withing the advanced object configuration with the same result.
Try index Mapping and put the date field as non-analyzed.
For Example:
"<index name>": {
"mappings": {
"<Mapping type>": {
"properties": {
"City": {
"type": "string",
"index": "not_analyzed"
},
"Date": {
"type": "string",
"index": "not_analyzed"
}
}
}
I had a similar issue today with the following message:
Parse Failure [Unknown key for a VALUE_STRING in [logTime]: [offset].]]; }]
I was sending a date histogram aggregation request against Elasticsearch 1.4.5 with the following payload:
['logTime'].forEach(function (field) {
body.aggregations[field] = {
date_histogram: {
field: field,
interval: 'week',
time_zone: '+00:00',
offset: '15h',
min_doc_count: 0,
extended_bounds: {
min: 1440946800000,
max: 1441551599999
}
}
};
});
Note the use of offset parameter for the date_histogram. This parameter is introduced in Elasticsearch version 1.5.0 only. So, my 1.4.5 ES was complaining that this offset key was Unknown.
Replacing with post_offset as follows solved the problem though I had to adjust the value of the time_zone parameter as well. As a side note, post_offset is deprecated and replaced with offset since v1.5.
['logTime'].forEach(function (field) {
body.aggregations[field] = {
date_histogram: {
field: field,
interval: 'week',
time_zone: '+09:00',
post_offset: '-9h',
min_doc_count: 0,
extended_bounds: {
min: 1440946800000,
max: 1441551599999
}
}
};
});
I am using elasticsearch 1.1.
Normally in this version, percolator on nested documents should work.
Although, i am trying to do this but I get the following error:
failures: [
{
index: test
shard: 4
reason: BroadcastShardOperationFailedException[[test][4] ]; nested: PercolateException[failed to percolate]; nested: ElasticsearchIllegalArgumentException[Nothing to percolate];
}
]
I have the following percolator (sorry elasticsearch head removed me all the quotes):
{
_index: test
_type: .percolator
_id: 27
_version: 1
_score: 1
_source: {
query: {
filtered: {
query: {
match_all: { }
}
filter: {
nested: {
filter: {
term: {
city: london
}
}
path: location
}
}
}
}
}
}
And while trying to percolate this document I am getting the error:
{
...
"location": {
"date": "2014-05-05T15:07:58",
"namedplaces": {
"city": "london"
}
}
}
Any idea why it doesn't work ?
EDIT :
In elasticsearch log I got more precision about the error:
[2014-05-06 13:33:48,972][DEBUG][action.percolate ] [Tomazooma] [test][2], node[H42BBxajRs2w2NmllMnp7g], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.percolate.PercolateReque
st#7399452e]
org.elasticsearch.percolator.PercolateException: failed to percolate
at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:198)
at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:55)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$2.run(TransportBroadcastOperationAction.java:226)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: Nothing to percolate
at org.elasticsearch.percolator.PercolatorService.percolate(PercolatorService.java:187)
at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:194)
... 5 more
The documentation of ES is not really clear about it. But when you look at this page, you will see that when you are percolating you need to surround your indexed document by doc{}. It is indeed compulsory otherwise the exception that you've got will appears:
Try to do so on :
{
"doc":{
...
"location": {
"date": "2014-05-05T15:07:58",
"namedplaces": {
"city": "london"
}
}
}
}
I hope it will help ;-)
Another reason for the Nothing to percolate exception is not setting the Content-Length HTTP header.
Because the GET request has a body, it should also have a 'Content-Length' HTTP header, but not all APIs will set this for you; as I found out the hard way!
I have "organizations" index structure :
{
"organization" : {
"properties" : {
"id" : { "type" : "long" },
"name" : { "type" : "string" }
}
}
}
and another index "campaigns" :
{
"campaign" : {
"properties" : {
"id" : { "type" : "long" },
"name" : { "type" : "string" }
},
"_parent" : {
"type" : "organization"
}
}
}
when i search in campaigns that has specific organization id :
QueryBuilder qb = QueryBuilders.boolQuery()
.must(QueryBuilders.hasParentQuery(SearchEntityEnum.ORGANIZATION.getType(), QueryBuilders.queryString("*" + keyword.replaceAll(" ", "* *") + "*").field(SearchEntityFieldNames.CommonFieldNames.NAME)
.analyzeWildcard(true)));
SearchResponse response = client.prepareSearch(SearchEntityEnum.CAMPAIGN.getIndex())
.setTypes(SearchEntityEnum.CAMPAIGN.getType())
.setSearchType(SearchType.QUERY_AND_FETCH)
.setQuery(qb)
.execute().actionGet();
This error occur :
org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query_fetch], all shards failed; shardFailures {[-RCGH_JbTmeTGDSvWp9_wA][campaigns][2]: SearchParseException[[campaigns][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"bool":{"must":{"has_parent":{"query":{"query_string":{"query":"**","fields":["name"],"analyze_wildcard":true}},"parent_type":"organization"}}}}}]]]; nested: QueryParsingException[[campaigns] [has_parent] query configured 'parent_type' [organization] is not a valid type]; }{[-RCGH_JbTmeTGDSvWp9_wA][campaigns][1]: SearchParseException[[campaigns][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"bool":{"must":{"has_parent":{"query":{"query_string":{"query":"**","fields":["name"],"analyze_wildcard":true}},"parent_type":"organization"}}}}}]]]; nested: QueryParsingException[[campaigns] [has_parent] query configured 'parent_type' [organization] is not a valid type]; }{[-RCGH_JbTmeTGDSvWp9_wA][campaigns][4]: SearchParseException[[campaigns][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"bool":{"must":{"has_parent":{"query":{"query_string":{"query":"**","fields":["name"],"analyze_wildcard":true}},"parent_type":"organization"}}}}}]]]; nested: QueryParsingException[[campaigns] [has_parent] query configured 'parent_type' [organization] is not a valid type]; }{[-RCGH_JbTmeTGDSvWp9_wA][campaigns][0]: SearchParseException[[campaigns][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"bool":{"must":{"has_parent":{"query":{"query_string":{"query":"**","fields":["name"],"analyze_wildcard":true}},"parent_type":"organization"}}}}}]]]; nested: QueryParsingException[[campaigns] [has_parent] query configured 'parent_type' [organization] is not a valid type]; }{[-RCGH_JbTmeTGDSvWp9_wA][campaigns][3]: SearchParseException[[campaigns][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"bool":{"must":{"has_parent":{"query":{"query_string":{"query":"**","fields":["name"],"analyze_wildcard":true}},"parent_type":"organization"}}}}}]]]; nested: QueryParsingException[[campaigns] [has_parent] query configured 'parent_type' [organization] is not a valid type]; }
I had the same problem yesterday. I found out that the parent and child types both need to be in the same index in order for the parent to be visible to the child.
In this example, organization and campaign are different types in different indexes. You will need to create one index with both of these types defined within it. Everything else I believe is good here so let me know if you have any further issues.
I believe (might be wrong) that you should define campaigns as a type within organizations index. also you should set routing to true to make this works.