change elasticsearch mapping - elasticsearch

I am trying to change the mapping using following code:
PUT /in_test/_mapping/keyword
{
"properties" : {
"term" : {
"type" : "text",
"index" : "not_analyzed"
}
}
}
But it is giving an error:
{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[tiebreaker-0000000000][172.17.0.24:19555][indices:admin/mapping/put]"
}
],
"type": "illegal_argument_exception",
"reason": "Could not convert [term.index] to boolean",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Failed to parse value [not_analyzed] as only [true] or [false] are allowed."
}
},
"status": 400
}
I also tried to recreate the index
by:
PUT /in_test
{
"mappings" : {
"keyword" : {
"properties" : {
"term" : {
"type" : "text",
"index" : "not_analyzed"
}
}
}
}
}
but I got:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [keyword]: Could not convert [term.index] to boolean"
}
],
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [keyword]: Could not convert [term.index] to boolean",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Could not convert [term.index] to boolean",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Failed to parse value [not_analyzed] as only [true] or [false] are allowed."
}
}
},
"status": 400
}
I also tried to change the _type to keywords but it is still not working.
Basically, I want to search for exact match of string and for that I am referring to this:
https://www.elastic.co/guide/en/elasticsearch/guide/current/_finding_exact_values.html#_term_query_with_text

That documentation page is from Elasticsearch version 2.X (See at the top of the page), and is no longer correct for modern versions of Elasticsearch.
The error you're getting is because "index" now only accepts true or false, and refers to whether or not the property is indexed at all - Since you're searching by this property, you want it to be true (the default).
Instead, try setting the type to "keyword" and it won't be tokenized. https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-keyword-analyzer.html#_definition_5
PUT /in_test
{
"mappings" : {
"keyword" : {
"properties" : {
"term" : {
"type" : "keyword"
}
}
}
}
}

Related

How to update field format in Opensearch/Elasticsearch?

I am trying to change the format of a string field in opensearch:
PUT my_index/_mapping
{
"mappings": {
"properties": {
"timestamp": {
"type": "date",
"format": "YYYY-MM-DD HH:mm:ss.SSS"
}
}
}
}
Response is
{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "Root mapping definition has unsupported parameters: [mappings : {properties={timestamp={format=YYYY-MM-DD HH:mm:ss.SSS, type=date}}}]"
}
],
"type" : "mapper_parsing_exception",
"reason" : "Root mapping definition has unsupported parameters: [mappings : {properties={timestamp={format=YYYY-MM-DD HH:mm:ss.SSS, type=date}}}]"
},
"status" : 400
}
I've spent days trying to figure this out, seems to me like Opensearch is just so unnecessarily complex.
You cannot change the type of an existing field once it's been created. You need to reindex your index with the wrong mapping into a new index with the right mapping.
First, create the new index:
PUT new_index
{
"mappings": {
"properties": {
"timestamp": {
"type": "date",
"format": "YYYY-MM-DD HH:mm:ss.SSS"
}
}
}
}
Then, reindex the old index into the new one
POST _reindex
{
"source": {
"index": "old_index"
},
"dest": {
"index": "new_index"
}
}

Error message - Unable to filter min_docs_count

EDIT:
Answer below
getting always following error when trying any aggregated query.
Tried googling and different aggregation constructs.
Elasticsearch API Hosted as "Logs Data Platform" by OVH.
Request
{
"aggs" : {
"servers" : {
"filter" : { "term": { "servertype": "1" } },
"aggs" : {
"avg_price" : { "avg" : { "field" : "serveramount" } }
}
}
}
}
Error response
{
"error": {
"root_cause": [
{
"type": "parse_exception",
"reason": "Unable to filter min_docs_count"
}
],
"type": "parse_exception",
"reason": "Unable to filter min_docs_count",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "[size] parameter cannot be negative, found [-1]"
}
},
"status": 400
}
Stupid me ... size=0 was missing in the query parameter.

Elastic Search GeoIp location not of type geo_point

I'm running ElasticSearch, Logstash and Kibana using Docker Compose based on the solution: https://github.com/deviantony/docker-elk.
I'm following this tutorial trying to add geoip information when processing my web logs: https://www.elastic.co/blog/geoip-in-the-elastic-stack.
In logstash I'm processing files from FileBeat and I've added geoip to my filter:
filter {
...
geoip {
source => "client_ip"
}
}
When I view the documents in Kibana they do contain additional information like geoip.country_name, geoip.city_name etc. but I expect the geoip.location field being of type geo_point in my index.
Here is an example of how some of the geoip fields are mapped:
Instead of geo_point I see location.lat and location.lon. Why are my location not of type geo_point? Do I need some kind of mapping etc.?
Both ingest-common, ingest-geoip, ingest-user-agent and x-pack are loaded when ElasticSearch starts up. I've refreshed the field list for my index in Kibana.
EDIT1:
Based on answer from #Val I'm trying to change the mapping of my index:
PUT iis-log-*/_mapping/log
{
"properties": {
"geoip": {
"dynamic": true,
"properties": {
"ip": {
"type": "ip"
},
"location": {
"type": "geo_point"
},
"latitude": {
"type": "half_float"
},
"longitude": {
"type": "half_float"
}
}
}
}
}
But that gives me this error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "mapper [geoip.ip] of different type, current_type [text], merged_type [ip]"
}
],
"type": "illegal_argument_exception",
"reason": "mapper [geoip.ip] of different type, current_type [text], merged_type [ip]"
},
"status": 400
}
In the article you referred to, they do explain that you need to put a specific mapping for the geo_point field in the "Mapping, for Maps" section.
If you're using the default index names (i.e. logstash-*) and the default mapping type (i.e. log), then the mapping is taken care of for you by Logstash. But if not, you need to install it yourself using:
PUT your_index
{
"mappings" : {
"_default_" : {
"_all" : {"enabled" : true, "norms" : false},
"dynamic_templates" : [ {
"message_field" : {
"path_match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text",
"norms" : false
}
}
}, {
"string_fields" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text", "norms" : false,
"fields" : {
"keyword" : { "type": "keyword", "ignore_above": 256 }
}
}
}
} ],
"properties" : {
"#timestamp": { "type": "date", "include_in_all": false },
"#version": { "type": "keyword", "include_in_all": false },
"geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "half_float" },
"longitude" : { "type" : "half_float" }
}
}
}
}
}
}
In the above mappings, you see the geoip.location field being treated as a geo_point.

Error adding index with mapping to elasticsearch

I am trying to define mapping for elasticsearch but getting the following error (using KIBANA sense in localhost):
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "analyzer [whitespace_analyzer] not found for field [country]"
}
],
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [package]: analyzer [whitespace_analyzer] not found for field [country]",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "analyzer [whitespace_analyzer] not found for field [country]"
}
},
"status": 400
}
Mapping given :
PUT /worldV
{
"mappings" : {
"package" : {
"properties" : {
"autosuggestionpackagedetail" : {
"type" : "string",
"index" : "not_analyzed"
},
"availability" : {
},
...... so on ....

Can not create Elasticsearch Index (logstash-2015.05.18)

I'm using Elasticsearch 2.4
Following the instruction from the Elasticsearch Kibana official documentation here, when I create the index logstash-2015.05.18, the error below were emitted.
# curl -XPUT http://10.15.0.70:9200/logstash-2015.05.18 -d '
{
"mappings": {
"log": {
"properties": {
"geo": {
"properties": {
"coordinates": {
"type": "geo_point"
}
}
}
}
}
}
}
';
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters: [“store” : true]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [“date”]: Root mapping definition has unsupported parameters: [“store” : true]","caused_by":{"type":"mapper_parsing_exception","reason":"Root mapping definition has unsupported parameters: [“store” : true]"}},"status":400}
Using the sense plugin of Kibana to create the index also gives me the same error
PUT logstash-2015.05.18
{
"mappings": {
"log": {
"properties": {
"geo": {
"properties": {
"coordinates": {
"type": "geo_point"
}
}
}
}
}
}
}
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [“store” : true]"
}
],
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [“date”]: Root mapping definition has unsupported parameters: [“store” : true]",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [“store” : true]"
}
},
"status": 400
}
Can someone tell me did I do something wrong when creating the index?
Had same trouble.
Removing elasticsearch data("/usr/local/var/elasticsearch", if you install it with Homebrew) fixed it for me.

Resources