I'm using olivere's elastic v.5 Go library (https://godoc.org/gopkg.in/olivere/elastic.v5) for my Elastic queries. If I have an elastic mapping like this:
"mappings": {
"boxes": {
"properties": {
"field1": {
"type": "string"
},
"field2": {
"type": "string"
},
"field3": {
"type": "date"
},
"field4": {
"type": "date"
}
}
}
And I want to get a list of all fields that have type 'date'.
I've looked into GetFieldMapping but that doesn't seem to have an option to filter the fields based on type.
I tried this:
elasticclient.GetFieldMapping().Index("someindex").Type("boxes").Field().Type("date").Pretty(true).Do(ctx)
That just gives me all the fields and their types. Is there a different syntax to do this? Thanks!
Related
I am using update_by_query to update document,and i have a nested field named "Myfield1".There is a string field named "id" in "Myfield1".In my case,the mapping something like this:
"mapping":{
"mytype":{
"properties": {
"Myfield1": {
"type": "nested",
"properties": {
"id": {
"type": "string"
},
"field2": {
"type": "long"
}
}
},
"Title": {
"type": "string"
}
}
}
}
The field "id" in nested is unique generated by uuid.Then i want to update field2 in the nested field Myfield by query with the following request.
{
"query": {
"nested":{
"path":"Myfield1",
"query":{"match":{"Myfield1.id":"uuid"}},
"inner_hits":{"from":0,"size":1}
}
},
"script":"for (field in ctx._source.Myfield1) {if(field.id=='uuid') {field.field2='new value'}}"
}
I know it work,but the efficiency of this script is too low.How can I write ”script“ to directly update the field2 in inner_hits.Or maybe there are other efficient ways.
Thanks!
I have an index in Elasticsearch with the following field mapping:
{
"version_data": {
"properties": {
"title": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"updated_at": {
"type": "date"
},
"updated_by": {
"type": "keyword"
}
}
}
}
I have already created some documents in it and now want to rename version_data field with _version_data.
Is there any way in the Elasticsearch to rename a field within the mapping and in documents?
The closest thing is the alias data type.
In your mapping you could link it from the old to the new name like this:
PUT test/_mapping
{
"properties": {
"_version_data": {
"type": "alias",
"path": "version_data"
}
}
}
BTW I would generally avoid leading underscored since those normally used for internal fields like _id.
I am implementing Completion Suggester in my application, and here goes my requirement:
I am using Elasticsearch 5.5.3 (which support multiple types). I have around 10 types in my Elasticsearch and each type has around 10 string fields. What I want to do is make a single search box, that would complete the phrase (of any fields of those 10 types) as user starts typing using completion suggester. What could be the best approach to it? Is using _all field a good idea?
Yes, that's perfectly doable using a "custom all field" field of type completion
First, create the index with all the types and make sure to copy each field in a custom field of type completion:
PUT my_index
{
"mappings": {
"type1": {
"properties": {
"field1": {
"type": "text",
"copy_to": "my_all"
},
"field2": {
"type": "text",
"copy_to": "my_all"
},
"my_all": {
"type": "completion"
}
}
},
"type1": {
"properties": {
"field1": {
"type": "text",
"copy_to": "my_all"
},
"field2": {
"type": "text",
"copy_to": "my_all"
},
"my_all": {
"type": "completion"
}
}
}
}
}
Then, you'd query the completion data like this (i.e. without specifying any mapping type and using the common my_all field):
POST my_index/_search
{
"suggest": {
"my-suggest": {
"prefix": "bla",
"completion": {
"field": "my_all"
}
}
}
}
I want to have an elasticsearch schema that has some pre defined fields including an object type field. I want to have all the fields inside that object type field to be string by default.
I have the following mappings and dynamic templates while creating the index.
PUT myindex
{
"mappings": {
"doc": {
"dynamic_templates": [
{
"default_string": {
"path_match": "myObj.*",
"match_mapping_type": "*",
"mapping": {
"type": "text"
}
}
}],
"properties": {
"dummy_field_name": { "type": "text" },
"timestamp": {
"type": "date",
"format": "epoch_second"
},
"myObj": {
"type": "object"
}
}
}
}
}
But when I submit a field inside the object with a numeric value, it is not mapping that field to string.
curl -XPOST "http://elastic-url:8080/myindex/test" -d
'{"dummy_field_name": "dymmy_value", "myObj":{ "filed_1": 123 ,
"field_2": "some value"}, "timestamp": 1522196333}'
"filed_1" is identified as a number field. But I want it to be stored as a string type.
Field types detected
Your mapping is defined for type "doc" but you are indexing to type "test", try matching them
we have an elastic search index with following mapping (showing only partial mapping relevant to this question)
"instFields": {
"properties": {
"_index": {
"type": "object"
},
"fieldValue": {
"fields": {
"raw": {
"index": "not_analyzed",
"type": "string"
}
},
"type": "string"
},
"sourceFieldId": {
"type": "integer"
}
},
"type": "nested"
}
as you can see fieldValue type is string: in original data in the database for that fieldValue column is stored in a JSON type column (in Postgresql). use case is such that when this data is stored fieldValue can be valid JsNumber, JsString,JsBoolean (any valid [JsValue][1] now question is that when storing this fieldValue in ES - it'll have to be a definite type - so we convert fieldValue to String while pushing data into ElasticSearch.
Following is a sample data from Elastic search
"instFields": [
{
"sourceFieldId": 1233,
"fieldValue": "Demo Logistics LLC"
},
{
"sourceFieldId": 1236,
"fieldValue": "169451"
}
]
this is where it gets interesting where now we want to run various metrics aggregations on fieldValue - for e.g. if sourceFieldId = 1236 then run [avg][3] on fieldValue - problem is fieldValue had to be stored as string in ES - due to originally fieldValue being JsValue type field in the application. what's the best way to create mapping in elastic search such that fieldValue can be stored with an appropriate type vs string type so various metrics aggregation can be run of fieldValue which are of type long (though encoded as string in ES)
One of the ways to achieve this is create different fields in elastic search with all possible type of JsValue (e.g. JsNumber, JsBoolean,JsString etc). now while indexing - application can derive proper type of JsValue field to find out whether it's JsString, JsNumber, JsBoolean etc.
on application side I can decode proper type of fieldValue being indexed
value match{
case JsString(s) =>
case JsNumber(n) =>
case JsBoolean(b)
}
now modify mapping in elastic search and add more fields - each with proper type - as shown below
"instFields": {
"properties": {
"_index": {
"type": "object"
},
"fieldBoolean": {
"type": "boolean"
},
"fieldDate": {
"fields": {
"raw": {
"format": "dateOptionalTime",
"type": "date"
}
},
"format": "dateOptionalTime",
"type": "date"
},
"fieldDouble": {
"fields": {
"raw": {
"type": "double"
}
},
"type": "double"
},
"fieldLong": {
"fields": {
"raw": {
"type": "long"
}
},
"type": "long"
},
"fieldString": {
"fields": {
"raw": {
"index": "not_analyzed",
"type": "string"
}
},
"type": "string"
},
"fieldValue": {
"fields": {
"raw": {
"index": "not_analyzed",
"type": "string"
}
},
"type": "string"
}
now at the time of indexing
value match{
case JsString(s) => //populate fieldString
case JsNumber(n) => //populate fieldDouble (there is also fieldLong)
case JsBoolean(b) //populate fieldBoolean
}
this way now boolean value is stored in fieldBoolean, number is stored in long etc. now running metrics aggregation becomes a normal business by going against fieldLong or fieldDouble field (depending on the query use case). notice fieldValue field is still there in ES mapping and index as before. Application will continue to convert value to string and store it in fieldValue as before - this way queries which don't care about types can only query fieldValue field in the index.
It sounds like you should have two separate fields, one for the case when the value is a string and one for when it is an instance of a number.
Depending on how you're indexing this data, it can be easy or hard. However, its a bit strange that you have a fields that could be a string or a number.
Regardless, elasticsearch is not going to be able to do both in a single field