I am using Elasticsearch 6.8. And I'd like to save some meta data on my index. The index already existed. I followed this doc https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html#add-field-mapping
curl "http://localhost:9200/idx_1/_mapping"
{
"idx_1": {
"mappings": {
"1": {
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
In order to create _meta data, I need to create mapping type first.
And I run below code to create a _meta mapping type for version.
curl -X PUT -H 'Content-Type: application/json' "http://localhost:9200/idx_1/_mapping" -d '
{"_meta": { "version": {"type": "text"}}}'
I got below errors:
{
"error": {
"root_cause": [
{
"type": "action_request_validation_exception",
"reason": "Validation Failed: 1: mapping type is missing;"
}
],
"type": "action_request_validation_exception",
"reason": "Validation Failed: 1: mapping type is missing;"
},
"status": 400
}
It says mapping type is missing. I have specified the type for version as text. Why does it say missing type?
It turns out that I looked at the wrong document version. Based on the doc for Elasticsearch6, https://www.elastic.co/guide/en/elasticsearch/reference/6.3/mapping-meta-field.html, the correct request is:
curl -X PUT "http://localhost:9200/idx1/_mapping/_doc" -H 'Content-Type: application/json' -d '{"_meta": {"version": "1235kljsdlkf"}}'
Related
I am trying to follow this resource to update the mapping in an existing index but it is given an error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "mapper [NAME] cannot be changed from type [text] to [keyword]"
}
],
"type": "illegal_argument_exception",
"reason": "mapper [NAME] cannot be changed from type [text] to [keyword]"
},
"status": 400
}
Below is the request I am hitting:
curl -X PUT \
http://localhost:9200/company/_mapping \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-H 'postman-token: a8384316-7374-069c-05e5-5be4e0a8f6d8' \
-d '{
"dynamic": "strict",
"properties": {
"NAME": {
"type": "keyword"
},
"DOJ": {
"type": "date"
}
}
}'
I know I can re-create an index with the new mapping but why can't I update the existing one?
The comments in this thread are correct in that changing a text field to a keyword field would be considered a mapping breaking change (and raise the "cannot be changed from [type] to [type]" exception).
You can, however, extend the original text mapping to become a multi-field mapping containing a keyword:
curl -X PUT \
http://localhost:9200/company/_mapping \
-H 'content-type: application/json' \
-d '{
"dynamic": "strict",
"properties": {
"NAME": {
"type": "text",
"fields": {
"keyword": { <---
"type": "keyword"
}
}
},
"DOJ": {
"type": "date"
}
}
}'
Once this adjustment is through, you can easily pick up this new property through a blank Update by query call:
curl -X PUT \
http://localhost:9200/company/_update_by_query \
-H 'content-type: application/json'
Finally, you can target the NAME.keyword in your queries. The good thing is, you'll have retained the original text too.
You cannot change the mapping of an existing field, you need to create a new index with the correct mapping then reindex your data
Except for supported mapping parameters, you can’t change the mapping or field type of an existing field. Changing an existing field could invalidate data that’s already indexed.
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings
Can you specify the NAME? it happen to me because NAME included a point (e.g. "histological.group"
I'm trying to create an autocomplete index for my elasticsearch using the search_as_you_type datatype.
My first command I run is
curl --request PUT 'https://elasticsearch.company.me/autocomplete' \
'{
"mappings": {
"properties": {
"company_name": {
"type": "search_as_you_type"
},
"serviceTitle": {
"type": "search_as_you_type"
}
}
}
}'
which returns
{"acknowledged":true,"shards_acknowledged":true,"index":"autocomplete"}curl: (3) nested brace in URL position 18:
{
"mappings": {
"properties": etc.the rest of the json object I created}}
Then I reindex using
curl --silent --request POST 'http://elasticsearch.company.me/_reindex?pretty' --data-raw '{
"source": {
"index": "existing_index"
},
"dest": {
"index": "autocomplete"
}
}' | grep "total\|created\|failures"
I expect to see some "total":1000,"created":5etc but some kind of response from the terminal, but I get nothing. Also, when I check the mapping of my autocomplete index, by running curl -u thething 'https://elasticsearch.company.me/autocomplete/_mappings?pretty',
I get an empty mapping result:
{
"autocomplete" : {
"mappings" : { }
}
}
Is my error in the creation of my index or the reindexing? I'm expecting the autocomplete mappings to show the two fields I'm searching for, ie: "company_name" and "serviceTitle". Any ideas how to fix?
I am trying to load log records into elasticsearch (7.3.1) and showing the results in kibana. I am facing the fact that although records are loaded into elasticearch and a curl GET shows them, they are not visible in kibana.
Most of the time, this is because of the timestamp format. In my case, the proper timestamp format should be basic_date_time, but the index only has:
# curl -XGET 'localhost:9200/og/_mapping'
{"og":{"mappings":{"properties":{"#timestamp":{"type":"date"},"componentName":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}}}}%
I would like to add format 'basic_date_time' to the #timestamp properties, but each try I do is either not accepted by elasticsearch or does not change the index field.
I simply fail to get the right command to do the job.
For example, the simplest I could think of,
Z cr 23;curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/og/_mapping' -d'
{"mappings":{"properties":{"#timestamp":{"type":"date","format":"basic_date_time"}}}}
'
gives error
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters: [mappings : {properties={#timestamp={format=basic_date_time, type=date}}}]"}],"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters: [mappings : {properties={#timestamp={format=basic_date_time, type=date}}}]"},"status":400}%
and trying to do it via kibana with
PUT /og
{
"mappings": {
"properties": {
"#timestamp": { "type": "date", "format": "basic_date_time" }
}
}
}
gives
{
"error": {
"root_cause": [
{
"type": "resource_already_exists_exception",
"reason": "index [og/NIT2FoNfQpuPT3Povp97bg] already exists",
"index_uuid": "NIT2FoNfQpuPT3Povp97bg",
"index": "og"
}
],
"type": "resource_already_exists_exception",
"reason": "index [og/NIT2FoNfQpuPT3Povp97bg] already exists",
"index_uuid": "NIT2FoNfQpuPT3Povp97bg",
"index": "og"
},
"status": 400
}
I am not sure if I should even try this in kibana. But I would be very glad if I could find the right curl command to get the index changed.
Thanks for helping, Ruud
You can do it either via curl like this:
curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/og/_mapping' -d '{
"properties": {
"#timestamp": {
"type": "date",
"format": "basic_date_time"
}
}
}
'
Or in Kibana like this:
PUT /og/_mapping
{
"properties": {
"#timestamp": {
"type": "date",
"format": "basic_date_time"
}
}
}
Also worth noting is that once an index/mapping is created you can usually not modify it (very few exceptions). You can create a new index with the correct mapping and reindex your data into it.
When putting an anaylzer into mapping using PUT /job/_mapping/doc/ but get conflicts.
But there isn't a anaylzer in mappings.
PUT /job/_mapping/doc/
{
"properties":{
"title": {
"type": "text",
"analyzer":"ik_smart",
"search_analyzer":"ik_smart"
}
}
}
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Mapper for [title] conflicts with existing mapping in other types:\n[mapper [title] has different [analyzer]]"
}
],
"type": "illegal_argument_exception",
"reason": "Mapper for [title] conflicts with existing mapping in other types:\n[mapper [title] has different [analyzer]]"
},
"status": 400
}
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
},
"fielddata": true
},
The output config is like this.
output {
elasticsearch {
hosts => ["<Elasticsearch Hosts>"]
user => "<user>"
password => "<password>"
index => "<table>"
document_id => "%{<MySQL_PRIMARY_KEY>}"
}
}
You cant update mapping in elasticsearch, you can add mapping but not update mapping. Elasticsearch use mapping at the indexation time, that s why you cant update mapping of an existing field. Analyzer is part of the mapping, in fact if you don't specify one es a default one, analyzer tell elastic how to index the documents.
create a new index with your new mappings (include analyzer)
reindex your documents from your existing index to the new one (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html)
Updating Mapping:
Once a document is indexed to an index i.e. the mapping is generated under a given type as like in our case Mapping of EmployeeCode, EmployeeName & isDevelopers' is generated under type "customtype", we cannot modify it afterwards. In case if we want to modify it, we need to delete the index first and then apply the modified mapping manually and then re-index the data. But If you want to add an a new property under a given type, then it is feasible. For example, our document attached our index "inkashyap-1002" under type "customtype" is as follows:
{
"inkashyap-1002": {
"mappings": {
"customtype": {
"properties": {
"EmployeeCode": {
"type": "long"
},
"isDeveloper": {
"type": "boolean"
},
"EmployeeName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
now let's add another property "Grade" :
curl -XPUT localhost:9200/inkashyap-1002(IndexName)/customtype(TypeName)/2 — d '{
"EmployeeName": "Vaibhav Kashyap",
"EmployeeCode": 13629,
"isDeveloper": true,
"Grade": 5
}'
Now hit the GET mapping API. In the results, you can see there is another field added called "Grade".
Common Error:
In the index "inkashyap-1002", so far we have indexed 2 documents. Both the documents had the same type for the field "EmployeeCode" and the type was "Long". Now let us try to index a document like below:
curl -XPUT localhost:9200/inkashyap-1002/customtype/3 -d '{
"EmployeeName": "Vaibhav Kashyap",
"EmployeeCode": "onethreesixtwonine",
"isDeveloper": true,
"Grade": 5
}'
Note that here the "EmployeeCode" is given in string type, which indicates that it is a string field. The response to the above request will be like below:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failedtoparse[
EmployeeCode
]"
}
],
"type": "mapper_parsing_exception",
"reason": "failedtoparse[
EmployeeCode
]",
"caused_by": {
"type": "number_format_exception",
"reason": "Forinputstring: \"onethreesixtwonine\""
}
},
"status": 400
}
In the above response, we can see the error "mapper_parsing_exception" on the field "EmployeeCode". This indicates that the expected field here was of another type and not string. In such cases re-index the document with the appropriate type
I have a index in Elasticsearch, and want to update the type of a field named currentTimeStamp from long to date, so that Kibana can work on it. Following is my current output of _mapping (Other fields have been removed for brevity).
{
"myIndexname": {
"mappings": {
"myType": {
"properties": {
"currentTimeStamp": {
"type": "long"
}
}
}
}
}
}
When I try to run the following command for updating the type of the column to date type, I get the below mentioned error response. Any help on this is highly appreciated.
curl -X PUT myIndexname/_mapping/myType with the following payload
{
"myIndexname": {
"properties": {
"currentTimeStamp": {
"type": "date",
"format": "date_optional_time || epoch_millis"
}
}
}
}
Error response:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [optimizationframework : {properties={currentTimeStamp={type=date, format=date_optional_time || epoch_millis}}}]"
}
],
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [optimizationframework : {properties={currentTimeStamp={type=date, format=date_optional_time || epoch_millis}}}]"
},
"status": 400
}