How to create a sub object in Elastic-search 7.x - elasticsearch

Earlier I was using 1.x version and was creating the sub objects mapping using below syntax.
"foo": {
"type": "integer",
"doc_values": true
},
"foo.bar": {
"type": "integer",
"doc_values": true
},
"foo.bar.baz": {
"type": "integer",
"doc_values": true
},
But now when I am using same mapping syntax in ES 7.x I am getting below error:-
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Can't merge a non object mapping [foo] with an object mapping [foo]"
}
],
"type": "illegal_argument_exception",
"reason": "Can't merge a non object mapping [foo] with an object mapping [foo]"
},
"status": 400
}
I am came accros this SO post Can’t merge a non object mapping with an object mapping error in machine learning(beta) module But, Note I am not updating the mapping, instead I am crating a new mapping still getting this error, please advise what to do?

An example of object type. Refer here for more info
"mappings": {
"properties": {
"user": {
"properties": {
"age": { "type": "integer" },
"name": {
"properties": {
"first": { "type": "text" },
"last": { "type": "text" }
}
}
}
}
}
In below name can be defined as object and further properties can be added using name.firstname etc. In your mapping foo is of type integer and then you are adding foo.bar so it is throwing error. foo must be of type object.
"properties" : {
"name" : {
"type" : "object"
},
"name.firstname":{
"type" : "text"
},
"name.lastname":{
"type" : "text"
}
}

Related

Update a string parameter in Elasticsearch _mapping

I have such a _mapping in Elasticsearch 6.8:
{
"grch38_test__wes__grch38__variants__20210222" : {
"mappings" : {
"variant" : {
"_meta" : {
"gencodeVersion" : "25",
"hail_version" : "0.2.20",
"genomeVersion" : "38",
"sampleType" : "WES",
"sourceFilePath" : "s3://my_folder/my_vcf.vcf"
},
...
My goal is to issue a query in Kibana to modify variant._meta.sourceFilePath. Following thread:
Elastic search mapping for nested json objects
I was able to come up with the query:
PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
"properties": {
"variant": {
"type": "nested",
"properties": {
"_meta": {
"type": "nested",
"properties": {
"type": "text",
"sourceFilePath": "s3://my_folder/my_vcf.vcf"
}
}
}
}
}
}
But its giving me an error:
elasticsearch mapping Expected map for property [fields] on field [name] but got a class java.lang.String
Full error message:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Expected map for property [fields] on field [type] but got a class java.lang.String"
}
],
"type": "mapper_parsing_exception",
"reason": "Expected map for property [fields] on field [type] but got a class java.lang.String"
},
"status": 400
}
I have also tried:
PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
"properties": {
"variant": {
"type": "nested",
"properties": {
"_meta": {
"type": "nested",
"properties": {
"sourceFilePath": {
"type": "text",
"value":"s3://my_folder/my_vcf.vcf"
}
}
}
}
}
}
}
But its telling me that value is unsupported:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Mapping definition for [sourceFilePath] has unsupported parameters: [value : s3://seqr-dp-data--prod/vcf/dev/grch38_test_contracted.vcf]"
}
],
"type": "mapper_parsing_exception",
"reason": "Mapping definition for [sourceFilePath] has unsupported parameters: [value : s3://seqr-dp-data--prod/vcf/dev/grch38_test_contracted.vcf]"
},
"status": 400
}
What am I doing wrong? How to modify the field?
_meta is a reserved field for storing application-specific metadata. It's not meant to be searchable and can be only retrieved through the GET Mapping API.
This means that if your _meta content was intended to be consistent with what the _meta field is designed for, you cannot apply any mappings to it. It's a "final" hashmap of concrete values and would need to be defined at the top level of your update-mapping payload:
PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
"_meta": {
"variant": { <-- shared index-level metadata
"gencodeVersion": "25",
"hail_version": "0.2.20",
"genomeVersion": "38",
"sampleType": "WES",
"sourceFilePath": "s3://my_folder/my_vcf.vcf"
}
},
"properties": {
"some_text_field": { <-- actual document properties
"type": "text"
}
}
}
If, on the other hand, your _meta field is an unfortunate naming coincidence, you can declare the mappings for it like so:
PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
"properties": {
"_meta": {
"properties": {
"variant": {
"properties": {
"gencodeVersion": {
"type": "text"
},
"genomeVersion": {
"type": "text"
},
"hail_version": {
"type": "text"
},
"sampleType": {
"type": "text"
},
"sourceFilePath": {
"type": "text"
}
}
}
}
}
}
}
and ingest documents of the form:
POST grch38_test__wes__grch38__variants__20210222/variant/_doc
{
"_meta": {
"variant": {
"gencodeVersion": "25",
"hail_version": "0.2.20",
"genomeVersion": "38",
"sampleType": "WES",
"sourceFilePath": "s3://my_folder/my_vcf.vcf"
}
}
}
But again, the _meta content would be document-specific, not index-wide!
And BTW, the nested mapping only makes sense if you're dealing with arrays of objects, not objects of objects.
But if you insist on wanting it, here's how you'd do it:
PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant?include_type_name
{
"properties": {
"_meta": {
"type": "nested", <---
"properties": {
"variant": {
"type": "nested", <---
"properties": {
"gencodeVersion": {
"type": "text"
},
"genomeVersion": {
"type": "text"
},
"hail_version": {
"type": "text"
},
"sampleType": {
"type": "text"
},
"sourceFilePath": {
"type": "text"
}
}
}
}
}
}
}

ElasticSearch action_request_validation_exception

I'm creating mapping for multiple type
here my query
PUT opl_consultation/_mapping
my json mapping file
{
"mappings": {
"article": {
"properties": {
"numero_noeud": { "type": "text" },
"intitule_fr": { "type": "text" },
"path_audio": { "type": "text" }
}
},
"hierarchie": {
"properties": {
"id_type_noeud_hie": { "type": "integer" },
"noeud_numero_hie": { "type": "text" },
"intitule_hie_fr": { "type": "text" }
}
},
"law_type": {
"properties": {
"id_type_loi": { "type": "integer" },
"Desc_law_type": { "type": "text" }
}
}
}
}
below the error a got
"type": "action_request_validation_exception",
"reason": "Validation Failed: 1: mapping type is missing;"
},
"status": 400
}
the version is Elasticsearch\6.4.2
In Elasticsearch 6.4.2 you cannot have more than one mapping type. See https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html
If you run your query instead as PUT opl_consultation with your mapping definition you will get the below error
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [opl_consultation] as the final mapping would have more than 1 type: [law_type, article, hierarchie]"
Instead, use a custom type field as described here

Set values in multifields elastic search

I have the following structure recorded in elastic:
PUT /movies
{
"mappings": {
"title": {
"properties": {
"title": {
"type": "string",
"fields": {
"de": {
"type": "string",
"analyzer": "german"
},
"en": {
"type": "string",
"analyzer": "english"
},
"fr": {
"type": "string",
"analyzer": "french"
},
"es": {
"type": "string",
"analyzer": "spanish"
}
}
}
}
}
}
}
But when I am trying to record values like this:
PUT movies/_doc/2
{
"title": "fox",
"field": "en"
}
I receive the following error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [movies] as the final mapping would have more than 1 type: [_doc, title]"
}
],
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [movies] as the final mapping would have more than 1 type: [_doc, title]"
},
"status": 400
}
Maybe I am doing something wrong since I am fairly new to elastic. My idea is to create one to one mapping and when I am searching for Fox in any of these languages to return results only in english since they are recorded in the DB.
Your mapping indicates a mapping type "title" but when you create the documents you use PUT movies/_doc/2 that indicates mapping type _doc which doesn't exist so ES will try to automatically create it, and in newer version of ES having multiple mapping types is forbidden.
You should just change it to: PUT movies/title/2

In Elasticsearch, can I set language-specific multi-fields?

Is it possible to use multi-fields to set and query multilingual fields?
Consider this mapping:
PUT multi_test
{
"mappings": {
"data": {
"_field_names": {
"enabled": false
},
"properties": {
"book_title": {
"type": "text",
"fields": {
"english": {
"type": "text",
"analyzer": "english"
},
"german": {
"type": "text",
"analyzer": "german"
},
"italian": {
"type": "text",
"analyzer": "italian"
}
}
}
}
}
}
}
I tried the following, but it doesn't work:
PUT multi_test/data/1
{
"book_title.english": "It's good",
"book_title.german": "Das gut"
}
The error seems to indicate I'm trying to add new fields:
{ "error": { "root_cause": [ { "type": "mapper_parsing_exception",
"reason": "Could not dynamically add mapping for field
[book_title.english]. Existing mapping for [book_title] must be of
type object but found [text]." } ], "type":
"mapper_parsing_exception", "reason": "Could not dynamically add
mapping for field [book_title.english]. Existing mapping for
[book_title] must be of type object but found [text]." }, "status":
400 }
What am I doing wrong here?
If my approach is unworkable, what is a better way to do this?
The problem is that you are using using fields for the field book_title.
Fields keyword is used when you want to keep same field and data in multiple ways i.e using different analyzers or some other setting changes but values should be same in all field names under fields.Here is the link describing what is keyword fields https://www.elastic.co/guide/en/elasticsearch/reference/2.4/multi-fields.html
In you use case the mapping should be like below
PUT multi_test
{
"mappings": {
"data": {
"_field_names": {
"enabled": false
},
"properties": {
"book_title": {
"properties": {
"english": {
"type": "text",
"analyzer": "english"
},
"german": {
"type": "text",
"analyzer": "german"
},
"italian": {
"type": "text",
"analyzer": "italian"
}
}
}
}
}
}
}
This will define book_title as object type and you can add multiple fields with different data under book_title

elasticsearch update mapping conflict exception

I have an index named "myproject-error-2016-08" which has only one type named "error".
When I hit :
GET myproject-error-2016-08/_mapping
It returns following result:
{
"myproject-error-2016-08": {
"mappings": {
"error": {
"properties": {
...
"responseCode": {
"type": "string"
},
...
}
}
}
}
}
I need to update the responseCode to not_analyzed,
hence I am using following following reuest :
PUT myproject-error-2016-08/_mapping/error
{
"properties": {
"responseCode": {
"type": "string",
"index": "not_analyzed"
}
}
}
And getting following exception :
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Mapper for [responseCode] conflicts with existing mapping in other types:\n[mapper [responseCode] has different [index] values, mapper [responseCode] has different [doc_values] values, cannot change from disabled to enabled, mapper [responseCode] has different [analyzer]]"
}
],
"type": "illegal_argument_exception",
"reason": "Mapper for [responseCode] conflicts with existing mapping in other types:\n[mapper [responseCode] has different [index] values, mapper [responseCode] has different [doc_values] values, cannot change from disabled to enabled, mapper [responseCode] has different [analyzer]]"
},
"status": 400
}
I have also tried following:
PUT myproject-error-2016-08/_mapping/error?update_all_types
{
...
}
But it returned the same response.
Elastic Search is :
$ ./elasticsearch -version
Version: 2.3.5, Build: 90f439f/2016-07-27T10:36:52Z, JVM: 1.8.0_91
You cannot change the type of a field once it's been created.
However, you can definitely create a not_analyzed sub-field like this:
PUT myproject-error-2016-08/_mapping/error
{
"properties": {
"responseCode": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
Then you'll need to re-index your data/logs in order to populate that sub-field and you'll be able to reference responseCode.raw in your queries.
UPDATE: Since ES5 not_analyzed string do not exist anymore and are now called keyword:
PUT myproject-error-2016-08/_mapping/error
{
"properties": {
"responseCode": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}
}
}

Resources