How can I change the _index to an existing document in Elastic Search?
Example:
1) I create an index:
PUT /customer?pretty
2) I add a document:
POST /customer/_doc?pretty
{
"name": "John Doe"
}
3) I create another index:
PUT /customer2?pretty
How Do I move the document created in step 2 into the new _index customer2?
POST _reindex
{
"source": {
"index": "customer",
"type": "_doc",
"query": {
"term": {
"_id": "fMn2OmcBEGEHUvm1g7Mi"
}
}
},
"dest": {
"index": "customer2"
}
}
DELETE /customer2/_doc/fMn2OmcBEGEHUvm1g7Mi
where "fMn2OmcBEGEHUvm1g7Mi" is the id of the document.
There isn't a way to edit the meta fields in a document. The best way would be to reindex it into a new index and delete the older index.
POST _reindex
{
"source": {
"index": "customer"
},
"dest": {
"index": "customer2"
}
}
Related
Is it possible to set particular nested fields for reindexing?
According to docs https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-filter-source, selected fields are array.
POST _reindex
{
"source": {
"index": "twitter",
"_source": ["user", "_doc"]
},
"dest": {
"index": "new_twitter"
}
}
For example, we need reindex only nested fields of user like "name" and "birthdate":
How could it be done? We need something like this:
POST _reindex
{
"source": {
"index": "twitter",
"_source": { "user": ["name", "birthdate"], "_doc"]
},
"dest": {
"index": "new_twitter"
}
}
POST _reindex
{
"source": {
"index": "twitter",
"_source": [ "user.name", "user. birthdate", "_doc"]
},
"dest": {
"index": "twitter_new"
}
}
}
You need to use . to refer them.
I have an old index (elasticsearch index) has more than 20K objects, this index has fields
{
"title": "Test title",
"title_ar": "عنوان تجريبي",
"body": "<p>......</p>"
}
I want to _reindex them to convert all data to new mapping like this
{
"title_1": {
"en": "Test title",
"ar": "عنوان تجريبي"
},
"body": "<p>......</p>"
}
What is the best elasticsearch pipeline processor to make this conversion available in _reindex API?
I suggest to simply use the reindex API to do this:
POST _reindex
{
"source": {
"index": "old_index"
},
"dest": {
"index": "new_index"
},
"script": {
"source": "ctx._source.title = [ 'en' : ctx._source.title, 'ar': ctx._source.title_ar]",
"lang": "painless"
}
}
If in your old_index index you have this:
{
"title": "Test title",
"title_ar": "عنوان تجريبي",
"body": "<p>......</p>"
}
In your new index, you'll have this:
{
"title": {
"en": "Test title",
"ar": "عنوان تجريبي"
},
"body": "<p>......</p>"
}
I am able to create index and in discovery tab all the data is being populated shown properly.
{
"_index": "deal-expense",
"_type": "kafka-connect",
"_id": "deal-expense+0+63",
"_version": 2,
"_score": 1,
"_source": {
"EXPENSE_CODE": "NL**20",
"EXPENSE_CODE_DESCRIPTION": "DO NOT USE ****** ****** - ADAM",
"NO_OF_DEALS": 17
}
}
Data Visualization Requirement:
There might be multiple indexed documents for every EXPENSE_CODE
On Y Axis I need to display Max Of NO_OF_DEALS and On X Axis I need to display EXPENSE_CODE
Max Of NO_OF_DEALS are not being populated but EXPENSE_CODE is being populated.
My Configurations as shown below.
We need to create the elastic search index and map it to existing index mentioned in the question.
PUT /reindexed-deal-expense
{
"mappings": {
"doc": {
"properties": {
"geo": {
"properties": {
"EXPENSE_CODE": {
"type": "keyword"
},
"NO_OF_DEALS": {
"type": "integer"
}
}
}
}
}
}
}
and map it like this
POST _reindex
{
"source": {
"index": "deal-expense"
},
"dest": {
"index": "reindexed-deal-expense"
}
}
I'm attempting to add an un-analyzed version of an analyzed field, as a 'raw' multi-field, as per the ElasticSearch documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/2.4/multi-fields.html
This seems to be a common, well-supported pattern.
I've created the following index / field :
{
"person": {
"aliases": {},
"mappings": {
"employee": {
"properties": {
"userName": {
"type": "string",
"analyzer": "autocomplete",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
If I query the index directly, i.e. GET /person, I see the mapping as I've posted above, so I'm confident that there wasn't a syntax error, etc.
However, when we're pushing data into the index, a userName.raw field is not being created.
{
"_index": "person",
"_type": "employee",
"_id": "2",
"_version": 1,
"found": true,
"_source": {
"username": "Test Value"
}
}
Anyone see something I'm missing?
Thanks!
EDIT:
This was a novice mistake when creating my index.
PUT person
{
"person": {
"aliases": {},
"mappings": {
"employee": {
"properties": {
"email": {
Notice the person key is being PUT in the 'person' index. This was creating a nested person.
Correct syntax is to remove the extra "person"
PUT person
{
"aliases": {},
"mappings": {
"employee": {
"properties": {
"email": {
Please see Linoy.M.K's answer, as he is correct.
The 'raw' field will not appear when retrieving a record by ID. Its only useful as part of a query.
Adding multiple analyzers will not modify your source document means your source document will always have username only not username.raw
Added analyzers are useful when you do searching, means you can now search with username and username.raw to achieve different behavior like below.
GET /person/employee/_search
{
"query": {
"match": {
"username": "Te"
}
}
}
GET /person/employee/_search
{
"query": {
"match": {
"username.raw": "Test Value"
}
}
}
Having noticed my sort on an indexed string field doesn't work properly, I've discovered that it's sorting analyzed strings so "bags of words" and if I want it to work properly I have to sort on the non-analyzed string. My plan was to just change the string field to a multi-field, using information I found in those two articles:
https://www.elastic.co/blog/changing-mapping-with-zero-downtime (Upgrade to a multi-field part)
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html
Using Sense I've created this field mapping
PUT myindex/_mapping/type
{
"properties": {
"Title": {
"type": "string",
"fields": {
"Raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
And then I try to sort my search results using the newly made field. I've put all of the name variations I could think of after reading the articles:
POST myindex/_search
{
"_source" : ["Title","titlemap.Raw","titlemap.Title","titlemap.Title.Raw","Title.Title","Raw","Title.Raw"],
"size": 6,
"query": {
"multi_match": {
"query": "title",
"fields": ["Title^5"
],
"fuzziness": "auto",
"type": "best_fields"
}
},
"sort": {
"Title.Raw": "asc"
}
}
And that's what I get in response:
{
"_index": "myindex_2015_11_26_12_22_38",
"_type": "type",
"_id": "1205",
"_score": null,
"_source": {
"Title": "The title of the item"
},
"sort": [
null
]
}
Only the Title field's value is shown in the response and the sort criterium is null for every result.
Am I doing something wrong, or is there another way to do that?
The index name is not the same after re-indexing and thus the default mapping gets installed... that's probably why.
I suggest using an index template instead, so you don't have to care when to create the index and ES will do it for you. The idea is to create a template with the proper mapping you need and then ES will create every new index whenever it deems necessary, add the myindex alias and apply the proper mapping to it.
curl -XPUT localhost:9200/_template/myindex_template -d '{
"template": "myindex_*",
"settings": {
"number_of_shards": 1
},
"aliases": {
"myindex": {}
},
"mappings": {
"type": {
"properties": {
"Title": {
"type": "string",
"fields": {
"Raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}'
Then whenever you launch your re-indexing process a new index with a new name will be created BUT with the proper mapping and the proper alias.