If I define an index template file template.json as follows:
{
"template": "types",
"mappings": {
"type1": {
"properties": {
"title": {
"type": "text"
}
}
},
"type2": {
"properties": {
"title": {
"type": "keyword"
}
}
}
}
}
and try post it:
curl -XPUT http://localhost:9200/_template/types -d#template.json
I get this response:
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "mapper [title] cannot be changed from type [keyword] to [text]"
}
],
"type" : "illegal_argument_exception",
"reason" : "mapper [title] cannot be changed from type [keyword] to [text]"
},
"status" : 400
}
I expect to be able to define multiple mappings within a template with different fields with different types.
What am I doing or assuming incorrectly?
I am using Elasticsearch 5.6.
If the fields have the same name, they need to be of the same field type, even if you are using version 5.6, which is the last one to allow the use of multiple mapping types in the same index.
If the field title in the type type1 is mapped as a text, the field title in the type type2 also needs to be mapped as a text.
Related
I have an EFK pipeline set up. Everyday a new index is created using the logstash-* prefix. Every time a new field is sent by Fluentd, the field is added to the index pattern logstash-*. I'm trying to create an index template that will disable indexing on a specific field when an index is created. I got this to work in ES 7.1 using the PUT below:
PUT _template/logstash-test
{
"index_patterns": ["logstash-*"],
"mappings": {
"dynamic_templates" : [
{
"params" : {
"path_match" : "params",
"mapping" : {
"enabled": false
}
}
}
]
}
}
However when I try this on Elasticsearch 6.8 I get the following error:
"type": "illegal_argument_exception",
"reason": "Malformed [mappings] section for type [dynamic_templates], should include an inner object describing the mapping"
It is a little different in Elasticsearch 6.X as it had mapping types, which is not used anymore.
Try something like this:
PUT _template/logstash-test
{
"index_patterns": ["logstash-*"],
"mappings": {
"_doc": {
"dynamic_templates" : [
{
"params" : {
"path_match" : "params",
"mapping" : {
"enabled": false
}
}
}
]
}
}
}
If your index has a different custom type and is not using the _doc type, you should use that in the mapping.
am new to elastic search.please help me out with this problem.
Am using elastic search version 7.13.2 .
I created an index with a custom analyzer and filter like this
PUT /analyzers_test
{
"settings": {
"analysis": {
"analyzer": {
"english_stop":{
"type":"standard",
"stopwords":"_english_"
},
"my_analyzer":{
"type":"custom",
"tokenizer":"standard",
"char_filter":["html_strip"
],
"filter":[
"lowercase",
"trim",
"my_stemmer"]
}
},
"filter": {
"my_stemmer":{
"type":"stemmer",
"name":"english"
}
}
}
}
}
Then i created a mapping for the document i will have and specified my custom analyzer i created earlier (there is no document in the index yet)
like so:
PUT /analyzers_test/_mapping
{
"properties": {
"description":{
"type": "text",
"analyzer": "my_analyzer"
},
"teaser":{
"type": "text"
}
}
}
When i try try to create a document like so
POST /analyzers_test/1
{
"description":"drinking",
"teaser":"drinking"
}
i get the following error
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "Mapper for [description] conflicts with existing mapper:\n\tCannot update parameter [analyzer] from [my_analyzer] to [default]"
}
],
"type" : "illegal_argument_exception",
"reason" : "Mapper for [description] conflicts with existing mapper:\n\tCannot update parameter [analyzer] from [my_analyzer] to [default]"
},
"status" : 400
}
Use index API to add document to your index. You are missing the _doc
"You cannot change the mapping (including the analyzer) of an existing field. What you need to do if you want to change the mapping of existing documents is reindex those documents to another index with the updated mapping." Abdon Pijpelink
Click the name for the original discussion
I have a mapping that I want to set in some indexes like below,
country_city_*
country_street_*
country_cars_*
But I don't want to set that mapping the that kinds of type like
county_person_*
country_city_errors
I have tried to use this below,
PUT /contry_*, -contry_person_*, -country_*_errors/_mapping
{
//some mappings in json
}
But I get this error
{
"error" : {
"root_cause" : [
{
"type" : "not_x_content_exception",
"reason" : "Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"
}
],
"type" : "not_x_content_exception",
"reason" : "Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"
},
"status" : 500
}
The compressor detection ... error is raised when you have disallowed whitespace in your request URI.
Multiple targets are indeed allowed but make sure you include the allow_no_indices option and of course properly structure your wildcards:
PUT /contry_*,*-contry_person_*,*-country_*_errors/_mapping?allow_no_indices=true
{
"properties": {
"someTextField": {
"type": "text"
}
}
}
BTW, the standard practice of creating mappings for indices that haven't been created yet is called index templates. In your particular case you could say:
PUT _index_template/country_template
{
"index_patterns": [
"contry_*",
"*-contry_person_*",
"*-country_*_errors" <-- any patterns you like but index names should NOT start with a hyphen
],
"template": {
"mappings": {
"properties": {
"someTextField": {
"type": "text"
},
"createdAt": {
"type": "date"
}
}
}
}
}
As you then ingest a new document into a new index on-the-fly:
POST contry_abc/_doc
{ "someTextField": "abc" }
the proper mapping will have been set:
GET contry_abc/_mapping
When putting an anaylzer into mapping using PUT /job/_mapping/doc/ but get conflicts.
But there isn't a anaylzer in mappings.
PUT /job/_mapping/doc/
{
"properties":{
"title": {
"type": "text",
"analyzer":"ik_smart",
"search_analyzer":"ik_smart"
}
}
}
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Mapper for [title] conflicts with existing mapping in other types:\n[mapper [title] has different [analyzer]]"
}
],
"type": "illegal_argument_exception",
"reason": "Mapper for [title] conflicts with existing mapping in other types:\n[mapper [title] has different [analyzer]]"
},
"status": 400
}
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
},
"fielddata": true
},
The output config is like this.
output {
elasticsearch {
hosts => ["<Elasticsearch Hosts>"]
user => "<user>"
password => "<password>"
index => "<table>"
document_id => "%{<MySQL_PRIMARY_KEY>}"
}
}
You cant update mapping in elasticsearch, you can add mapping but not update mapping. Elasticsearch use mapping at the indexation time, that s why you cant update mapping of an existing field. Analyzer is part of the mapping, in fact if you don't specify one es a default one, analyzer tell elastic how to index the documents.
create a new index with your new mappings (include analyzer)
reindex your documents from your existing index to the new one (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html)
Updating Mapping:
Once a document is indexed to an index i.e. the mapping is generated under a given type as like in our case Mapping of EmployeeCode, EmployeeName & isDevelopers' is generated under type "customtype", we cannot modify it afterwards. In case if we want to modify it, we need to delete the index first and then apply the modified mapping manually and then re-index the data. But If you want to add an a new property under a given type, then it is feasible. For example, our document attached our index "inkashyap-1002" under type "customtype" is as follows:
{
"inkashyap-1002": {
"mappings": {
"customtype": {
"properties": {
"EmployeeCode": {
"type": "long"
},
"isDeveloper": {
"type": "boolean"
},
"EmployeeName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
now let's add another property "Grade" :
curl -XPUT localhost:9200/inkashyap-1002(IndexName)/customtype(TypeName)/2 — d '{
"EmployeeName": "Vaibhav Kashyap",
"EmployeeCode": 13629,
"isDeveloper": true,
"Grade": 5
}'
Now hit the GET mapping API. In the results, you can see there is another field added called "Grade".
Common Error:
In the index "inkashyap-1002", so far we have indexed 2 documents. Both the documents had the same type for the field "EmployeeCode" and the type was "Long". Now let us try to index a document like below:
curl -XPUT localhost:9200/inkashyap-1002/customtype/3 -d '{
"EmployeeName": "Vaibhav Kashyap",
"EmployeeCode": "onethreesixtwonine",
"isDeveloper": true,
"Grade": 5
}'
Note that here the "EmployeeCode" is given in string type, which indicates that it is a string field. The response to the above request will be like below:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failedtoparse[
EmployeeCode
]"
}
],
"type": "mapper_parsing_exception",
"reason": "failedtoparse[
EmployeeCode
]",
"caused_by": {
"type": "number_format_exception",
"reason": "Forinputstring: \"onethreesixtwonine\""
}
},
"status": 400
}
In the above response, we can see the error "mapper_parsing_exception" on the field "EmployeeCode". This indicates that the expected field here was of another type and not string. In such cases re-index the document with the appropriate type
I tried to create an index with join datatype on new AWS Elasticsearch 6.0.
Following instruction on Elasticsearch 6.0 document: https://www.elastic.co/guide/en/elasticsearch/reference/6.0/parent-join.html
PUT my_index
{
"mappings": {
"doc": {
"properties": {
"my_join_field": {
"type": "join",
"relations": {
"question": "answer"
}
}
}
}
}
}
Then I received the following error:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "No handler for type [join] declared on field [my_join_field]"
}
],
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [doc]: No handler for type [join] declared on field [my_join_field]",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "No handler for type [join] declared on field [my_join_field]"
}
},
"status": 400
}
I am wondering if I'm missing something? Thanks!
I got the same problem. Root cause is a missing module on the ElasticSearch service packaged by AWS.
If you compare the configuration of a local installation and the one provided by AWS you will that this module is missing:
{
"name" : "parent-join",
"version" : "6.0.0",
"description" : "This module adds the support parent-child queries and aggregations",
"classname" : "org.elasticsearch.join.ParentJoinPlugin",
"has_native_controller" : false,
"requires_keystore" : false
}
Feature is now available in AWS ES 6.0