I got the following errors when trying to search with posting highlighter:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "field 'author_name' was indexed without offsets, cannot highlight"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query_fetch",
"grouped": true,
"failed_shards": [
{
"shard": 1,
"index": "post",
"node": "abc",
"reason": {
"type": "illegal_argument_exception",
"reason": "field 'author_name' was indexed without offsets, cannot highlight"
}
}
],
"caused_by": {
"type": "illegal_argument_exception",
"reason": "field 'author_name' was indexed without offsets, cannot highlight"
}
},
"status": 400
}
And here's my mapping:
{
"post": {
"mappings": {
"page": {
"_routing": {
"required": true
},
"properties": {
"author_name": {
"type": "text",
"store": true,
"index_options": "offsets",
"fields": {
"keyword": {
"type": "keyword"
}
},
"analyzer": "the_analyzer",
"search_analyzer": "the_search_analyzer"
},
"editor": {
"properties": {
"author_name": {
"type": "keyword"
}
}
}
}
},
"blog_post": {
"_routing": {
"required": true
},
"properties": {
"author_name": {
"type": "text",
"store": true,
"index_options": "offsets",
"fields": {
"keyword": {
"type": "keyword"
}
},
"analyzer": "the_analyzer",
"search_analyzer": "the_search_analyzer"
},
"editor": {
"properties": {
"author_name": {
"type": "keyword"
}
}
}
}
},
"comments": {
"_routing": {
"required": true
},
"_parent": {
"type": "blog_post"
},
"properties": {
"author_name": {
"type": "text",
"store": true,
"index_options": "offsets",
"fields": {
"keyword": {
"type": "keyword"
}
},
"analyzer": "the_analyzer",
"search_analyzer": "the_search_analyzer"
}
}
}
}
}
}
And my query:
GET post/article/_search?routing=cat
{
"query": {
"bool": {
"filter": {
"term": {
"category": "cat"
}
},
"must": [
{
"query_string": {
"query": "bill",
"fields": ["author_name"]
}
}]
}
},
"highlight": {
"fields": {
"author_name": {}
}
}
}
Elasticsearch version: 5.1.1
Lucence version: 6.3.0
When I did _update_by_query it works for a while, before failing again (after more data added).
I did some Googling, and found this issue on Elasticsearch repo:
https://github.com/elastic/elasticsearch/issues/8558, cmiiw, basically said that I need to have the same mapping for the same field name, on the same index. But I already did that, but I didn't know if my editor object, that has author_name can cause that issue.
Lucence code that throws that error:
https://github.com/apache/lucene-solr/blob/e2521b2a8baabdaf43b92192588f51e042d21e97/lucene/highlighter/src/java/org/apache/lucene/search/uhighlight/FieldOffsetStrategy.java#L92
https://github.com/apache/lucene-solr/blob/e2521b2a8baabdaf43b92192588f51e042d21e97/lucene/highlighter/src/java/org/apache/lucene/search/uhighlight/FieldHighlighter.java#L162
Question: how do I fix this error? thanks
Related
I have a index with following mapping:
{
"error_message_1": {
"mappings": {
"dynamic": "strict",
"properties": {
"_class": {
"type": "keyword",
"index": false,
"doc_values": false
},
"message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
},
"ngrams": {
"type": "text",
"analyzer": "autocomplete"
}
},
"copy_to": [
"myownfield"
]
},
"errorName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
},
"ngrams": {
"type": "text",
"analyzer": "autocomplete"
}
},
"copy_to": [
"myownfield"
]
},
"myownfield": {
"type": "text"
}
}
}
}
}
I have document with following body:
{
"_index": "error_message_1",
"_source": {
"id": 1,
"message": "message",
"errorName": "errorName"
},
When I tried to execute search with body:
{
"fields": ["myownfield"]
}
I got following result:
"fields": {
"myownfield": [ "message, "errorName"].
Then I have array with search params: ["message", "errorName"]. How can I get this document by query?
I am following the following docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime-indexed.html
I have a field which I would like to not be scripted on runtime but rather on index-time, and according to above I can do that simply by putting the field and its script inside the mapping object as normal.
Here is a simplified version of the index I'm trying to create
{
"settings": {
"analysis": {
"analyzer": {
"case_insensitive_analyzer": {
"type": "custom",
"filter": ["lowercase"],
"tokenizer": "keyword"
}
}
}
},
"mappings": {
"properties": {
"id": {
"type": "text"
},
"events": {
"properties": {
"fields": {
"type": "text"
},
"id": {
"type": "text"
},
"event": {
"type": "text"
},
"time": {
"type": "date"
},
"user": {
"type": "text"
},
"state": {
"type": "integer"
}
}
},
"eventLast": {
"type": "date",
"on_script_error": "fail",
"script": {
"source": "def events = doc['events']; emit(events[events.length-1].time.value"
}
}
}
}
}
I'm getting this 400 error back:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "unknown parameter [script] on mapper [eventLast] of type [date]"
}
],
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [_doc]: unknown parameter [script] on mapper [eventLast] of type [date]",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "unknown parameter [script] on mapper [eventLast] of type [date]"
}
},
"status": 400
}
Essentially I'm trying to create a scripted indexed field that is calculated off the last event time in the events array of the document.
Thanks
Tldr;
As the error states, you can not define your script in here.
There is a specific way to create runtime fields in elasticsearch.
You need to put the definition at the root of the json in the runtime object.
Solution
{
"settings": {
"analysis": {
"analyzer": {
"case_insensitive_analyzer": {
"type": "custom",
"filter": ["lowercase"],
"tokenizer": "keyword"
}
}
}
},
"runtime": {
"eventLast": {
"type": "date",
"on_script_error": "fail",
"script": {
"source": "def events = doc['events']; emit(events[events.length-1].time.value"
}
}
},
"mappings": {
"properties": {
"id": {
"type": "text"
},
"events": {
"properties": {
"fields": {
"type": "text"
},
"id": {
"type": "text"
},
"event": {
"type": "text"
},
"time": {
"type": "date"
},
"user": {
"type": "text"
},
"state": {
"type": "integer"
}
}
}
}
}
}
I have an index mapping like this
"mappings": {
"properties": {
"filter": {
"type": "nested",
"properties": {
"Hersteller": {
"type": "nested",
"properties": {
"id": {
"type": "text",
"analyzer": "analyzerFilter",
"fielddata": true
},
"value": {
"type": "text",
"analyzer": "analyzerFilter",
"fielddata": true
}
}
},
"Modell": {
"type": "nested",
"properties": {
"id": {
"type": "text",
"analyzer": "analyzerFilter",
"fielddata": true
},
"value": {
"type": "text",
"analyzer": "analyzerFilter",
"fielddata": true
}
}
}
}
},
"id": {
"type": "text",
"analyzer": "analyzerFilter"
}
}
}
}
There are 2 nested layers filter.Modell. I need a query to get all unique filter.Modell.value where filter.Hersteller.value is equal some predefined value.
I am trying first without any condition
{
"size": 4,
"aggs": {
"distinct_filter": {
"nested": { "path": "filter" },
"aggs": {
"distinct_filter_modell": {
"nested": {
"path": "filter.Modell",
"aggs": {
"distinct_filter_modell_value": {
"terms": { "field": "filter.Modell.value" }
}
}
}
}
}
}
}
}
And I get issue like
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "Unexpected token START_OBJECT in [distinct_filter_modell].",
"line": 1,
"col": 144
}
],
"type": "parsing_exception",
"reason": "Unexpected token START_OBJECT in [distinct_filter_modell].",
"line": 1,
"col": 144
},
"status": 400
}
Thanks in advance
I am currently trying to update an index template on Elastic Search 6.7/6.8.
Templates are stored in the code and are applied each time my API starts.
There are no errors, the request returns 200.
For example, here is a template i am currently using:
{
"index_patterns": [ "*-ec2-reports" ],
"version": 11,
"mappings": {
"ec2-report": {
"properties": {
"account": {
"type": "keyword"
},
"reportDate": {
"type": "date"
},
"reportType": {
"type": "keyword"
},
"instance": {
"properties": {
"id": {
"type": "keyword"
},
"region": {
"type": "keyword"
},
"state": {
"type": "keyword"
},
"purchasing": {
"type": "keyword"
},
"keyPair": {
"type": "keyword"
},
"type": {
"type": "keyword"
},
"platform": {
"type": "keyword"
},
"tags": {
"type": "nested",
"properties": {
"key": {
"type": "keyword"
},
"value": {
"type": "keyword"
}
}
},
"costs": {
"type": "object"
},
"stats": {
"type": "object",
"properties": {
"cpu": {
"type": "object",
"properties": {
"average": {
"type": "double"
},
"peak": {
"type": "double"
}
}
},
"network": {
"type": "object",
"properties": {
"in": {
"type": "double"
},
"out": {
"type": "double"
}
}
},
"volumes": {
"type": "nested",
"properties": {
"id": {
"type": "keyword"
},
"read": {
"type": "double"
},
"write": {
"type": "double"
}
}
}
}
},
"recommendation": {
"type": "object",
"properties": {
"instancetype": {
"type": "keyword"
},
"reason": {
"type": "keyword"
},
"newgeneration": {
"type": "keyword"
}
}
}
}
}
},
"_all": {
"enabled": false
},
"numeric_detection": false,
"date_detection": false
}
}
}
I'd like to add a new keyword field under the properties object like this :
"exampleField": {
"type": "keyword"
}
but it seems the template is not applied to existing indexes.
When data is inserted into a specific index which use the template, it is stored like this:
"exampleField": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
because the template has not been updated beforehand.
I would expect it to be like:
"exampleField": {
"type": "keyword"
}
in the index and in the template.
Does someone have any idea on how to have this result?
Thank you, Alexandre.
I am totally new to elastic search. So please forgive me if this is a stupid Question and my Questions might have been answered somewhere else already but I couldn't find it.
I want to use Elastic Search as a search engine for PDF'S and docx's in my network. I used fscrawler to ingest the PDF's to elastic search. Since the documents I want to ingest are in several languages I wanted to use n-graming for stemming. To do so I wanted to update my mapping like this
PUT test/_mappings/_all
{
"mappings": {
"title": {
"properties": {
"title": {
"type": "text",
"fields": {
"de": {
"type": "string",
"analyzer": "german"
},
"en": {
"type": "string",
"analyzer": "english"
},
"general": {
"type": "string",
"analyzer": "trigrams"
}
}
}
}
}
}
}
And now I get this Errormessage
{ "error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [mappings : {title={properties={title={type=text,
fields={de={type=string, analyzer=german}, en={type=string,
analyzer=english}, general={type=string, analyzer=trigrams}}}}}}]"
}
],
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [mappings : {title={properties={title={type=text,
fields={de={type=string, analyzer=german}, en={type=string,
analyzer=english}, general={type=string, analyzer=trigrams}}}}}}]"
}, "status": 400 }
Do you have any idea how i can fix this? Or do you have an idea how I can ingest the files with the right mapping without using fscrawler?
those are my settings
{
"test": {
"settings": {
"index": {
"mapping": {
"total_fields": {
"limit": "2000"
}
},
"number_of_shards": "5",
"provided_name": "test",
"creation_date": "1542031632596",
"analysis": {
"filter": {
"trigrams_filter": {
"type": "ngram",
"min_gram": "3",
"max_gram": "3"
}
},
"analyzer": {
"fscrawler_path": {
"tokenizer": "fscrawler_path"
},
"trigrams": {
"filter": [
"lowercase",
"trigrams_filter"
],
"type": "custom",
"tokenizer": "standard"
}
},
"tokenizer": {
"fscrawler_path": {
"type": "path_hierarchy"
}
}
},
"number_of_replicas": "1",
"uuid": "7L3QE5_xRACECVbTFlFY-Q",
"version": {
"created": "6040399"
}
}
}
}
}
My mapping
{
"test": {
"mappings": {
"_doc": {
"dynamic_templates": [
{
"raw_as_text": {
"path_match": "meta.raw.*",
"mapping": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"type": "text"
}
}
}
],
"properties": {
"attachment": {
"type": "binary"
},
"attributes": {
"properties": {
"group": {
"type": "keyword"
},
"owner": {
"type": "keyword"
}
}
},
"content": {
"type": "text"
},
"file": {
"properties": {
"checksum": {
"type": "keyword"
},
"content_type": {
"type": "keyword"
},
"created": {
"type": "date",
"format": "dateOptionalTime"
},
"extension": {
"type": "keyword"
},
"filename": {
"type": "keyword",
"store": true
},
"filesize": {
"type": "long"
},
"indexed_chars": {
"type": "long"
},
"indexing_date": {
"type": "date",
"format": "dateOptionalTime"
},
"last_accessed": {
"type": "date",
"format": "dateOptionalTime"
},
"last_modified": {
"type": "date",
"format": "dateOptionalTime"
},
"url": {
"type": "keyword",
"index": false
}
}
},
"meta": {
"properties": {
"altitude": {
"type": "text"
},
"author": {
"type": "text"
},
"comments": {
"type": "text"
},
"contributor": {
"type": "text"
},
"coverage": {
"type": "text"
},
"created": {
"type": "date",
"format": "dateOptionalTime"
},
"creator_tool": {
"type": "keyword"
},
"date": {
"type": "date",
"format": "dateOptionalTime"
},
"description": {
"type": "text"
},
"format": {
"type": "text"
},
"identifier": {
"type": "text"
},
"keywords": {
"type": "text"
},
"language": {
"type": "keyword"
},
"latitude": {
"type": "text"
},
"longitude": {
"type": "text"
},
"metadata_date": {
"type": "date",
"format": "dateOptionalTime"
},
"modifier": {
"type": "text"
},
"print_date": {
"type": "date",
"format": "dateOptionalTime"
},
"publisher": {
"type": "text"
},
"rating": {
"type": "byte"
},
"relation": {
"type": "text"
},
"rights": {
"type": "text"
},
"source": {
"type": "text"
},
"title": {
"type": "text"
},
"type": {
"type": "text"
}
}
},
"path": {
"properties": {
"real": {
"type": "keyword",
"fields": {
"fulltext": {
"type": "text"
},
"tree": {
"type": "text",
"analyzer": "fscrawler_path",
"fielddata": true
}
}
},
"root": {
"type": "keyword"
},
"virtual": {
"type": "keyword",
"fields": {
"fulltext": {
"type": "text"
},
"tree": {
"type": "text",
"analyzer": "fscrawler_path",
"fielddata": true
}
}
}
}
}
}
}
}
}
}