ElasticSearch sort exception - might be NEST related - elasticsearch

We recently upgraded from ES 0.23 to 0.90 and now my queries, eg:
{
"sort": {
"companyname.sort": {
"order": "asc"
}
}
}
cause an exception:
{
error: SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[Y8U7j2f-QvS75-6SfxlGuA][boss][0]:
QueryPhaseExecutionException[[boss][0]:
query[ConstantScore(cache(_type:SalesCompany))],from[0],size[10],sort[<custom:"companyname.sort":
org.elasticsearch.index.fielddata.fieldcomparator.BytesRefFieldComparatorSource#292f8639>]:
Query Failed [Failed to execute main query]]; nested: NullPointerException; }{[Y8U7j2f-QvS75-6SfxlGuA][boss][1]: QueryPhaseExecutionException[[boss][1]:
query[ConstantScore(cache(_type:SalesCompany))],from[0],size[10],sort[<custom:"companyname.sort":
org.elasticsearch.index.fielddata.fieldcomparator.BytesRefFieldComparatorSource#292f8639>]: Query Failed [Failed to execute main query]]; nested: NullPointerException; }]
status: 500
}
If I run the same query on our staging environment with an index that was created with 0.23 - but has ES 0.90 installed - it works fine. The mapping is:
companyname: {
type: multi_field
fields: {
companyname: {
type: string
store: true
analyzer: simple
}
sort: {
type: string
index: not_analyzed
store: true
omit_norms: true
index_options: docs
analyzer: simple
include_in_all: false
}
}
}
I'm using the latest version of NEST from a .Net C# app to create the index and query it.
I havent yet tried to recreate the index on the staging environment as I don't want to break that until I know how to fix it :)
Anyone got any ideas?
Thanks

I have recently switched to ES 0.90.7 and latest Nest. All is now well in the world.

Related

How to test analyzer-smartcn plugin for Elastic Search on local machine

I installed the smartcn plugin on my elastic search, restarted elasticsearch and tried to create an index with these settings:
PUT /test_chinese
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"default": {
"type": "smartcn"
}
}
}
}
}
}
However, when I run this in Marvel, I get this error back and I see a bunch of errors in Elastic search:
"error": "IndexCreationException[[test_chinese] failed to create
index]; nested: ElasticsearchIllegalArgumentException[failed to find
analyzer type [smartcn] or tokenizer for [default]]; nested:
NoClassSettingsException[Failed to load class setting [type] with
value [smartcn]]; nested:
ClassNotFoundException[org.elasticsearch.index.analysis.smartcn.SmartcnAnalyzerProvider];
", "status": 400
Any ideas what I might be missing?
I figured it out. I manually installed the plugins from the zip and it was causing issues... I reinstalled the right way but specific to 1.7 and it worked

ElasticsearchIllegalArgumentException[suggester [completion] requires context to be setup]

I am using Elastic Search to implement an autosuggest field for an index called people:
The Mapping is as below for the field person_name_suggest -
person_name_suggest: {
type: "completion",
analyzer: "simple",
payloads: true,
preserve_separators: true,
preserve_position_increments: true,
max_input_length: 50,
context: {
office_scope: {
type: "category",
path: "office_scope",
default: [
"0"
]
}
}
},
The request that I need to Elastic Search is as follows:
{
"suggest":{
"suggestions":{
"text":"M","
completion":{
"field":"person_name_suggest",
"context":890,
"size":10
}
}
}
}
I get the following error -
"error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[OsbgjmewT569a-7ZoNCMtg][people_2016_10_29][0]: SearchParseException[[people_2016_10_29][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"suggest":{"suggestions":{"text":"M","completion":{"field":"person_name_suggest","context":890,"size":10}}}}]]]; nested: ElasticsearchIllegalArgumentException[suggester [completion] requires context to be setup]; }
From what I can see, I have the completion suggester set up right.
Could someone point me in the right direction?
The version of ElasticSearch being used is 1.6
There is a syntax error in query. You have to specify the name of context
Try this
{
"suggest":{
"suggestions":{
"text":"M","
completion":{
"field":"person_name_suggest",
"context":{"office_scope":890},
"size":10
}
}
}
}

elasticsearch get statistics on analyzed field

i am trying to get statistics on analyzed string field.
i am trying to get AVG length of string field (in this example its title, and title sometimes can be empty/none).
tried:
GET book/_search
{
"facets" : {
"stat1" : {
"statistical" : {
"script" : "_source.title?.length()"
}
}
}
}
and i get an error:
Query Failed [Failed to execute main query]]; nested: NullPointerException; }]",
"status": 500
}
how can i accomplish that?
Any reason why you are using facets and not aggregations? Unless you use an Elasticsearch version that only supports facets, I recommend switching to aggregations. Facets are deprecated in 1.x and removed completely in 2.x.
And an aggregation like this one should work just fine:
GET /book/_search
{
"aggs": {
"stat1": {
"stats": {
"script": "_source.title?.length() ?: 0"
}
}
}
}

Getting error in Kibana 4 when using json input in Visualization while attempting to customize dashboard

I am new to the ELK stack and have it implemented with elasticsearch version 1.4.4, logstash version 1.4.2, and kibana version 4. I am able to pull a csv file into elasticsearch using logstash and have it display in kibana.
When displaying a date from the file, the values within the date are separated out as if the dash contained within is a separator (ex. value in field is 01-01-2015, when this is displayed in kibana (regardless of display type) there will be three field entries, 01, 01, and 2015). Kibana gives a message that this is due to it being an analyzed field.
Kibana 4 has a feature to use json directly from the dashboard builder, Visualization, to change this to a non-analyzed field so that the entire string will be used, rather than separating it.
I have tried multiple formats, but this is the one that seems it should work as kibana recognizes it as valid syntax:
{ "index" : "not_analyzed" }
but when attempting to apply the change, the dashboard does not change its structure and kibana generates the following exception:
Visualize: Request to Elasticsearch failed: {"error":"SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][0]: SearchParseException[[csvtest][0]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][0]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][1]: SearchParseException[[csvtest][1]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][1]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][2]: SearchParseException[[csvtest][2]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][2]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][3]: SearchParseException[[csvtest][3]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][3]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }{[ftpEMbcOTxu0Tdf0e8i-Ig][csvtest][4]: SearchParseException[[csvtest][4]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"#timestamp\":{\"gte\":1420092000000,\"lte\":1451627999999}}}],\"must_not\":[]}}}},\"size\":0,\"aggs\":{\"2\":{\"terms\":{\"field\":\"Conn Dt\",\"size\":100,\"order\":{\"1\":\"desc\"},\"index\":\"not_analyzed\"},\"aggs\":{\"1\":{\"cardinality\":{\"field\":\"Area Cd\"}}}}}}]]]; nested: SearchParseException[[csvtest][4]: query[ConstantScore(BooleanFilter(+cache(#timestamp:[1420092000000 TO 1451627999999])))],from[-1],size[0]: Parse Failure [Unknown key for a VALUE_STRING in [2]: [index].]]; }]"} less
It can be seen within where the index: was changed to not_analyzed from analyzed; also the setting that has wildcard analyzed: true was also changed to false withing the advanced object configuration with the same result.
Try index Mapping and put the date field as non-analyzed.
For Example:
"<index name>": {
"mappings": {
"<Mapping type>": {
"properties": {
"City": {
"type": "string",
"index": "not_analyzed"
},
"Date": {
"type": "string",
"index": "not_analyzed"
}
}
}
I had a similar issue today with the following message:
Parse Failure [Unknown key for a VALUE_STRING in [logTime]: [offset].]]; }]
I was sending a date histogram aggregation request against Elasticsearch 1.4.5 with the following payload:
['logTime'].forEach(function (field) {
body.aggregations[field] = {
date_histogram: {
field: field,
interval: 'week',
time_zone: '+00:00',
offset: '15h',
min_doc_count: 0,
extended_bounds: {
min: 1440946800000,
max: 1441551599999
}
}
};
});
Note the use of offset parameter for the date_histogram. This parameter is introduced in Elasticsearch version 1.5.0 only. So, my 1.4.5 ES was complaining that this offset key was Unknown.
Replacing with post_offset as follows solved the problem though I had to adjust the value of the time_zone parameter as well. As a side note, post_offset is deprecated and replaced with offset since v1.5.
['logTime'].forEach(function (field) {
body.aggregations[field] = {
date_histogram: {
field: field,
interval: 'week',
time_zone: '+09:00',
post_offset: '-9h',
min_doc_count: 0,
extended_bounds: {
min: 1440946800000,
max: 1441551599999
}
}
};
});

PercolateException[failed to percolate]; nested - Percolate API on nested document

I am using elasticsearch 1.1.
Normally in this version, percolator on nested documents should work.
Although, i am trying to do this but I get the following error:
failures: [
{
index: test
shard: 4
reason: BroadcastShardOperationFailedException[[test][4] ]; nested: PercolateException[failed to percolate]; nested: ElasticsearchIllegalArgumentException[Nothing to percolate];
}
]
I have the following percolator (sorry elasticsearch head removed me all the quotes):
{
_index: test
_type: .percolator
_id: 27
_version: 1
_score: 1
_source: {
query: {
filtered: {
query: {
match_all: { }
}
filter: {
nested: {
filter: {
term: {
city: london
}
}
path: location
}
}
}
}
}
}
And while trying to percolate this document I am getting the error:
{
...
"location": {
"date": "2014-05-05T15:07:58",
"namedplaces": {
"city": "london"
}
}
}
Any idea why it doesn't work ?
EDIT :
In elasticsearch log I got more precision about the error:
[2014-05-06 13:33:48,972][DEBUG][action.percolate ] [Tomazooma] [test][2], node[H42BBxajRs2w2NmllMnp7g], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.percolate.PercolateReque
st#7399452e]
org.elasticsearch.percolator.PercolateException: failed to percolate
at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:198)
at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:55)
at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$2.run(TransportBroadcastOperationAction.java:226)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: Nothing to percolate
at org.elasticsearch.percolator.PercolatorService.percolate(PercolatorService.java:187)
at org.elasticsearch.action.percolate.TransportPercolateAction.shardOperation(TransportPercolateAction.java:194)
... 5 more
The documentation of ES is not really clear about it. But when you look at this page, you will see that when you are percolating you need to surround your indexed document by doc{}. It is indeed compulsory otherwise the exception that you've got will appears:
Try to do so on :
{
"doc":{
...
"location": {
"date": "2014-05-05T15:07:58",
"namedplaces": {
"city": "london"
}
}
}
}
I hope it will help ;-)
Another reason for the Nothing to percolate exception is not setting the Content-Length HTTP header.
Because the GET request has a body, it should also have a 'Content-Length' HTTP header, but not all APIs will set this for you; as I found out the hard way!

Resources