I have the following mapping:
{
"properties": {
"type": {
"type": "keyword"
},
"body": {
"type": "text"
},
"id": {
"type": "keyword"
},
"date": {
"type": "date"
},
},
}
body field is going to be an email message, it's very long and I don't want to index it.
what is the proper way to exclude this field from indexing?
What I tried:
enabled: false - as I understand from the documentation, it's applied only to object type fields but in my case it's not really an object so I'm not sure whether I can use it.
index: false/'no' - this breaks the code at all and does not allow me to make a search. My query contains query itself and aggregations with filter. Filter contains range:
date: { gte: someDay.getTime(), lte: 'now' }
P.S. someDay is a certain day in my case.
The error I get after applying index: false in mapping to the body field is the following:
{
"error":
{
"root_cause":
[
{
"type": "number_format_exception",
"reason": "For input string: \"now\""
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards":
[
{
"shard": 0,
"index": "test",
"node": "eehPq21jQsmkotVOqQEMeA",
"reason":
{
"type": "number_format_exception",
"reason": "For input string: \"now\""
}
}
],
"caused_by":
{
"type": "number_format_exception",
"reason": "For input string: \"now\"",
"caused_by":
{
"type": "number_format_exception",
"reason": "For input string: \"now\""
}
}
},
"status": 400
}
I'm not sure how these cases are associated as the error is about date field while I'm adding index property to body field.
I'm using: "#opensearch-project/opensearch": "^1.0.2"
Please help me to understand:
how to exclude field from indexing.
why applying index: false to body field in mapping breaks the code an I get an error associated with date field.
You should just modify your mapping to this:
"body": {
"type": "text",
"index": false
}
And it should work
Related
I'm trying to partially update an existing document which is already in the index and is indexed well, meaning I can view it in Kibana which uses timestamp field to display the documents. I'm trying to update only the doc's name of id test_id
my request is:
POST shirkan_test/_update/test_id
{
"doc": {
"name": "shirkan"
},
"doc_as_upsert": true
}
Getting the following error:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse field [timestamp] of type [date] in document with id 'test_id'. Preview of field's value: '1.602505857664299E9'"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [timestamp] of type [date] in document with id 'test_id'. Preview of field's value: '1.602505857664299E9'",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "failed to parse date field [1.602505857664299E9] with format [epoch_second]",
"caused_by": {
"type": "date_time_parse_exception",
"reason": "Failed to parse with all enclosed parsers"
}
}
},
"status": 400
}
Much appreciate any help with this.
Thanks.
EDIT: adding index mapping
{
"mapping": {
"properties": {
"condition": {
"type": "text",
"index": false
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"rank": {
"type": "double"
},
"timestamp": {
"type": "date",
"format": "epoch_second"
}
}
}
}
EDIT 2: Changing timestamp format to strict_date_optional_time_nanos doesn't yield such an error and the upsert works well.
Looks like for now, the solution which worked for me to this problem is to change the timestamp field format from epoch_second to strict_date_optional_time_nanos. Other formats may work as well. I had to completely delete the index and recreate it since I came across the same error message when trying to re-index.
As mentioned in one of my comments, I filed a bug report here:
https://github.com/elastic/elasticsearch/issues/64050
I have this mapping
PUT /mytest
{
"mappings":{
"properties": {
"value": { "type": "object" }
}
}
}
When I insert this document
POST /mytest/_doc/4
{
"value": { "value": "test"}
}
I got the following error:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse field [value.value] of type [long] in document with id '4'. Preview of field's value: 'test'"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [value.value] of type [long] in document with id '4'. Preview of field's value: 'test'",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "For input string: \"test\""
}
},
"status": 400
}
I know the naming convention is bad, still, this is a valid JSON request, not sure why it doesn't allow it.
This error is telling you that you don't have a mapping for the property value within your value object property. The below example would property set the value.value property within your mytest index:
PUT mytest
{
"mappings": {
"properties": {
"value": {
"type": "object",
"properties": {
"value": {
"type": "text"
}
}
}
}
}
}
However, I don't think that's what your intention was. As a best practice, try following the Elastic Common Schema (ECS) for naming your index properties.
I have the following structure recorded in elastic:
PUT /movies
{
"mappings": {
"title": {
"properties": {
"title": {
"type": "string",
"fields": {
"de": {
"type": "string",
"analyzer": "german"
},
"en": {
"type": "string",
"analyzer": "english"
},
"fr": {
"type": "string",
"analyzer": "french"
},
"es": {
"type": "string",
"analyzer": "spanish"
}
}
}
}
}
}
}
But when I am trying to record values like this:
PUT movies/_doc/2
{
"title": "fox",
"field": "en"
}
I receive the following error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [movies] as the final mapping would have more than 1 type: [_doc, title]"
}
],
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [movies] as the final mapping would have more than 1 type: [_doc, title]"
},
"status": 400
}
Maybe I am doing something wrong since I am fairly new to elastic. My idea is to create one to one mapping and when I am searching for Fox in any of these languages to return results only in english since they are recorded in the DB.
Your mapping indicates a mapping type "title" but when you create the documents you use PUT movies/_doc/2 that indicates mapping type _doc which doesn't exist so ES will try to automatically create it, and in newer version of ES having multiple mapping types is forbidden.
You should just change it to: PUT movies/title/2
Using Wildcards in Field Names says field names can be specified with wildcards, for example:
{
"multi_match": {
"query": "Quick brown fox",
"fields": "*_title"
}
}
Now I'd like to query among all fields in the index, So I tried to use * to match all fields. But I got an error, for example:
{
"multi_match": {
"query": "nanjing jianye",
"fields": ["*"]
}
}
error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Invalid format: \"nanjing jianye\""
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "mycustomer",
"node": "YHKU1KllRJW-BiH9g5-McQ",
"reason": {
"type": "illegal_argument_exception",
"reason": "Invalid format: \"nanjing jianye\""
}
}
]
},
"status": 400
}
How to match the whole fields using wildcards instead of explicitly specifying the fields?
I have below mapping in my index:
{
"testIndex": {
"mappings": {
"type1": {
"properties": {
"text": {
"type": "string"
},
"time_views": {
"properties": {
"timestamp": {
"type": "long"
},
"views": {
"type": "integer"
}
}
}
}
}
}
}
}
"time_views" actually is an array, but inner attributes not array.
I want to sort my type1 records based on maximum value of "views" attribute of each type1 record. I read elasticsearch sort documentation, it's have solution for use cases that sorting is based on field (single or array) of single nested object. but what I want is different. I want pick maximum value of "views" for each document and sort the documents based on these values
I made this json query
{
"size": 10,
"query": {
"range": {
"timeStamp": {
"gte": 1468852617347,
"lte": 1468939017347
}
}
},
"from": 0,
"sort": [
{
"time_views.views": {
"mode": "max",
"nested_path": "time_views",
"order": "desc"
}
}
]
}
but I got this error
{
"error": {
"phase": "query",
"failed_shards": [
{
"node": "n4rxRCOuSBaGT5xZoa0bHQ",
"reason": {
"reason": "[nested] nested object under path [time_views] is not of nested type",
"col": 136,
"line": 1,
"index": "data",
"type": "query_parsing_exception"
},
"index": "data",
"shard": 0
}
],
"reason": "all shards failed",
"grouped": true,
"type": "search_phase_execution_exception",
"root_cause": [
{
"reason": "[nested] nested object under path [time_views] is not of nested type",
"col": 136,
"line": 1,
"index": "data",
"type": "query_parsing_exception"
}
]
},
"status": 400
}
as I mentioned above time_views is an array and I guess this error is because of that.
even I can't use sorting based on array field feature, because "time_views" is not a primitive type.
I think my last chance is write a custom sorting by scripting, but I don't know how.
please tell me my mistake if it's possible to achieve to what I'm want, otherwise give me a simple script sample.
tnx :)
The error message does a lot to explain what is wrong with the query. Actually, the problem is with the mapping. And I think you intended on using nested fields, since you are using nested queries.
You just need to make your time_views field as nested:
"mappings": {
"type1": {
"properties": {
"text": {
"type": "string"
},
"time_views": {
"type": "nested",
"properties": {
"timestamp": {
"type": "long"
},
"views": {
"type": "integer"
}
}
}
}
}
}