elasticsearch index_name with multi_field - elasticsearch

I have 2 separate indexes, each containing a different type
I want to get combined records from both.
The problem is that one type has field 'email', the other has 'work_email'. However I want to treat them as the same thing for sorting purposes.
That is why I try to use index_name in one of the types.
Here are mappings:
Index1:
"mappings": {
"people": {
"properties": {
"work_email": {
"type": "string",
"index_name": "email",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
Index2:
"mappings": {
"companies": {
"properties": {
"email": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
I expect this to work:
GET /index1,index2/people,companies/_search?
{
"sort": [
{
"email.raw": {
"order": "asc"
}
}
]
}
But, I get an error that there is no such field in the 'people' type.
Am I doing something wrong, or is there a better way to achieve what I need?
Here you can find a recreation script that illustrates the problem: https://gist.github.com/pmishev/11375297

There is problem in the way you map the multi field..Check out the below mapping and try to index..You should get the results
"mappings": {
"people": {
"properties": {
"work_email":{
"type": "multi_field",
"fields": {
"work_email":{
"type": "string",
"index_name": "email"
},
"raw":{
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
We should specify the type to multi_field and under the fields we should specify the required fields....

I ended up adding a 'copy_to' property in my mapping:
"mappings": {
"people": {
"properties": {
"work_email": {
"type": "string",
"copy_to": "email",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
So now I can address both fields as email.
It's not ideal, as this means that the email field is actually indexed twice, but that was the only thing that worked.

Related

elasticsearch mapping exception when using dynamic templates

hi i am using elasticsearch to index some documents. but the documents will have some fileds like goal1Completion, goal2Completion....goal100Completion. so i was trying to do mapping with dynamic Templates. so i came up with following but it throws an error:
{
"mappings": {
"date": {
"properties": {
"sessions": {
"type": "long"
},
"viewId": {
"type": "string",
"index": "not_analyzed"
},
"webPropertyId": {
"type": "string",
"index": "not_analyzed"
},
"dynamic_templates": [
{
"goalCompletions": {
"match_pattern": "regex",
"match": "goal\\d+\\w+",
"mapping": {
"type": "long"
}
}
}
]
}
}
}
}
error:"reason": "Expected map for property [fields] on field [dynamic_templates] but got a class java.lang.String"
what could be thee problem here?
You need to pull dynamic_template from properties map.
{
"mappings": {
"date": {
"properties": {
"sessions": {
"type": "long"
},
"viewId": {
"type": "string",
"index": "not_analyzed"
},
"webPropertyId": {
"type": "string",
"index": "not_analyzed"
}
},
"dynamic_templates": [ <--- Pull this out of properties
{
"goalCompletions": {
"match_pattern": "regex",
"match": "goal\\d+\\w+",
"mapping": {
"type": "long"
}
}
}
]
}
}
}

In Elasticsearch, how to have an index with automatic mapping and all fields are not analyzed?

I tried to do:
PUT /index_name/
{ "index" : {
"analysis" : {
"analyzer" :"not_analyzed"
}
}
}
but I'm not sure it is the right way...
Edit:
I applied both methods from the answers. But I have no way to test it. If in fact it do
GET /index_name/_mapping
...
"metaData_requestHeaders_accept-language": {
"type": "string"
},
"metaData_requestHeaders_akamai-origin-hop": {
"type": "string"
},
"metaData_requestHeaders_alexatoolbar-alx_ns_ph": {
"type": "string"
},
"metaData_requestHeaders_authorization": {
"type": "string"
},
"metaData_requestHeaders_c": {
"type": "string"
},
"metaData_requestHeaders_cache-control": {
"type": "string"
},
"metaData_requestHeaders_ckiooe": {
"type": "string"
},
...
as you can see the automatic mapping does not show what analyzer is used. So I have no way to test that this is actually working. Any ideas?
Use Dynamic Templates like :
PUT my_index
{
"mappings": {
"my_type": {
"dynamic_templates": [
{
"strings": {
"match_mapping_type": "string",
"mapping": {
"type": "string",
"index": "not_analyzed"
}
}
}
]
}
}
}
After applying above template you should see something like this :
GET /my_index/_mapping
{
"my_index": {
"mappings": {
"my_type": {
"dynamic_templates": [
{
"strings": {
"mapping": {
"index": "not_analyzed",
"type": "string"
},
"match_mapping_type": "string"
}
}
],
"properties": {}
}
}
}
}
Above mapping indicate that all strings will be not_analyzed by default.
I guess what you're looking for is Elasticsearch Templates, that allow you to create mappings dynamically.
You want something like this:
PUT index_name
{
"mappings": {
"type_name": {
"dynamic_templates": [
{
"strings": {
"match_mapping_type": "string",
"mapping": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
]
}
}
}

Update ElasticSearch Mapping type without delete it

I have this mapping type on my Index.
{
"iotsens-summarizedmeasures": {
"mappings": {
"summarizedmeasure": {
"properties": {
"id": {
"type": "long"
},
"location": {
"type": "boolean"
},
"rawValue": {
"type": "string"
},
"sensorId": {
"type": "string"
},
"summaryTimeUnit": {
"type": "string"
},
"timestamp": {
"type": "date",
"format": "dateOptionalTime"
},
"value": {
"type": "string"
},
"variableName": {
"type": "string"
}
}
}
}
}
}
I want to update sensorId field to.
"sensorId": {
"type": "string",
"index": "not_analyzed"
}
Is there any way to update the index without delete and re-mapping it? I don't have to change type of field, I only set "index": "not_analyzed".
Thanks you.
What you can do is make a multi-field out of your existing sensorId field like this with a sub-field called raw which is not_analyzed:
curl -XPUT localhost:9200/iotsens-summarizedmeasures/_mapping/summarizedmeasure -d '{
"summarizedmeasure": {
"properties": {
"sensorId": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}'
However, you still have to re-index your data to make sure all sensorId.raw sub-fields get created.

MapperParsingException when creating mapping for elasticsearch index

Using sense, I'm trying to create a mapping for an index with three properties. When i try to create it i get the following response
{
"error": "MapperParsingException[Root type mapping not empty after parsing! Remaining fields: [mappings : {gram={properties={gram={type=string, fields={gram_bm25={type=string, similarity=BM25}, gram_lmd={type=string}}}, sentiment={type=string, index=not_analyzed}, word={type=string, index=not_analyzed}}}}]]",
"status": 400
}
This is what i have in the sense console
PUT /pos/_mapping/gram
{
"mappings": {
"gram": {
"properties": {
"gram": {
"type": "string",
"fields": {
"gram_bm25": {
"type": "string", "similarity": "BM25"
},
"gram_lmd": {
"type": "string"
}
}
},
"sentiment": {
"type": "string", "index": "not_analyzed"
},
"word": {
"type": "string", "index": "not_analyzed"
}
}
}
}
}
The name of the index is 'pos' and I call the type 'gram'.
I have created the index with the same name.
I have validated the json using http://jsonlint.com/
I tried using XPUT in the console and i got the 'aknowleged' response, but the mapping is still {} when i request it in sense.
this question does not solve my problem. I always use the same name everywhere.
Any suggestions?
Thanks!
You just have the API syntax wrong. You've combined two different methods, basically.
Either create your index, then apply a mapping:
DELETE /pos
PUT /pos
PUT /pos/gram/_mapping
{
"gram": {
"properties": {
"gram": {
"type": "string",
"fields": {
"gram_bm25": {
"type": "string",
"similarity": "BM25"
},
"gram_lmd": {
"type": "string"
}
}
},
"sentiment": {
"type": "string",
"index": "not_analyzed"
},
"word": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
Or do it all at once when you create the index:
DELETE /pos
PUT /pos
{
"mappings": {
"gram": {
"properties": {
"gram": {
"type": "string",
"fields": {
"gram_bm25": {
"type": "string",
"similarity": "BM25"
},
"gram_lmd": {
"type": "string"
}
}
},
"sentiment": {
"type": "string",
"index": "not_analyzed"
},
"word": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
Here's the code I used:
http://sense.qbox.io/gist/6d645cc069f5f0fcf14f497809f7f79aff7de161

ElasticSearch Update Multi-field Mapping

I have an existing mapping for a field, and I want to change it to a multi-field.
The existing mapping is
{
"my_index": {
"mappings": {
"my_type": {
"properties": {
"author": {
"type": "string"
},
"isbn": {
"type": "string",
"analyzer": "standard",
"fields": {
"ngram": {
"type": "string",
"search_analyzer": "keyword"
}
}
},
"title": {
"type": "string",
"analyzer": "english",
"fields": {
"std": {
"type": "string",
"analyzer": "standard"
}
}
}
}
}
}
}
}
Based on the documentation, I should be able to change "author" to a multi-field by executing the following
PUT /my_index
{
"mappings": {
"my_type": {
"properties": {
"author":
{
"type": "multi-field",
"fields": {
"ngram": {
"type": "string",
"indexanalyzer": "ngram_analyzer",
"search_analyzer": "keyword"
},
"name" : {
"type": "string"
}
}
}
}
}
}
}
But instead I get the following error:
{
"error": "IndexAlreadyExistsException[[my_index] already exists]",
"status": 400
}
Am I missing something really obvious?
Instead of PUT to /my_index do:
POST /my_index/_mapping
You won't be able to change the field type in an already existing index.
If you can't recreate your index you can make use of the copy to field to achieve a similar capability.
PUT /my_index
{
"mappings": {
"my_type": {
"properties": {
"author":
{
"type": "string",
"copy_to": ["author-name","author-ngram"]
}
"author-ngram": {
"type": "string",
"indexanalyzer": "ngram_analyzer",
"search_analyzer": "keyword"
},
"author-name" : {
"type": "string"
}
}
}
}
}
}
While I have not tried it in your particular example, it is indeed possible to update field mappings by first closing the index and then applying the mappings.
Example:
POST /my_index/_close
POST /my_index/_mapping
{
"my_field:{"new_mapping"}
}
POST /my_index/_open
I have tested it, by adding a "copy_to" mapping property to mapped field.
Based on https://gist.github.com/nicolashery/6317643.

Resources