I need to clone the content of a document in my elasticsearch index (in the same index) by using the kibana console. I need exactly the same fields in the _source of the document (of course, the copy will have another id). I tryed to:
GET the document
Create a new empty instance of document
Update the new document by
manually copying the properties of the result on (1):
POST /blog/post/VAv2FWoBKgnBpki61WiD/_update { "doc" : {
"content" : "..." ...
But the problem is the field contain veeeery long properties. And sometimes I got an error since the strings seem not to be scaped when I manually copy them from the Kibana interface.
I searched in the documentation but I can not find a query to duplicate a document, and it is a quite common think to do I think...
Any clue?
Make use of Reindex API. Here is what you can do.
Summary of steps:
Create a destination_index (dummy). Make sure the mapping is exact to that of source_index
Using Reindex API, reindex that particular document from source_index to desitnation_index. During this operation, update the _id (I've mentioned the script)
Reindex this document back from desitnation_index to source_index
Reindex Query
Step 1: Copy document from source_index to destination_index. (With the script)
POST _reindex
{
"source": {
"index": "source_index",
"query": {
"match": {
"_id": "1"
}
}
},
"dest": {
"index": "destination_index"
},
"script": {
"inline": "ctx._id=2",
"lang": "painless"
}
}
Note how I've added a script in the above query that would change the _id (_id is set as 2) of the document. Your destination_index would have all the fields with exact same values as that of source except for the _id field.
Step 2: Copy that document from destination_index to source_index
POST _reindex
{
"source": {
"index": "destination_index",
"query": {
"match": {
"_id": "2"
}
}
},
"dest": {
"index": "source_index"
}
}
Now search the source_index, it would have two documents with different _ids (_id=1 and _id=2) having exact same content.
Hope this helps!
Related
I am new to the elasticsearch and I am trying to get the first document from my index and update its list of object.
I tried several queries e.g:
POST localhost:9200/test-index/_update_by_query
{
"size":1,
"sort": [{"timestamp":"asc"}],
"script": {
"inline": "ctx._source.addresses.add(params.address)",
"params" : {
"address" :{
"street": "Yemen Road",
"number": 15,
"county": "Yemen"
}
}
}
}
But this updates all my documents.
What is the fastest way to do this?
Thank you in advance!
I got the following answer from the elasticsearch community:
Update by query does not support a size. You would need to run a query first using size and sorting, and then use the update API on that single document.
I have an existing elastic search index with the following document structure, without a routing_key
{
"_id",
"feild1"
"field2"
}
I need to migrate the data into a new index. The structure of the index remains the same with an added routing_key. The routing key needs to be updated to "field1_field2". Is there a simple Kibana script to migrate the data to the new index?
Combination of a simple painless and the reindex API of elastic search could be used to achieve this.
POST _reindex
{
"source": {
"index": "{old_index_name}",
"size": {batch_size}
},
"dest": {
"index": "{new_index_name}"
},
"script": {
"lang": "painless",
"inline": "if (ctx._source.participants.length > 0) {ctx._routing=ctx._source.field1+'-'+ctx._source.field2}"
}
}
I am upgrading from ElasticSearch 5.6 to 6.0 and I have standard logstash-* indexes. In those indexes I have multiple (doc) types "attachmentsDbStats" and "attachmentsFileStats" which have the same schema. The only difference is the value of _type and type. I have created a new index attachments-* where the type is "attachement" and I want to reindex documents of both types into the new index. Obviously b/c of the new single type restriction in 6.0, both need to have the same type. I have update all of the documents in my old index such that the "type" field has a value "attachment." When I run reindex I am not able to upload the documents due to the restriction on the single type. I have attempted to update the _type field in the old indexes but that is immutable. Any ideas how to reindex and convert the type during the conversion?
Something like this should get you started:
POST _reindex
{
"source": {
"index": "logstash-*"
},
"dest": {
"index": "attachments-*"
},
"script": {
"source": """
ctx._id = ctx._type + "-" + ctx._id;
ctx._source.type = ctx._type;
ctx._type = "attachement";
"""
}
}
Combining _type and _id into the new _id field, so it's definitely unique. Move the _type field to a custom type. And set the _type to "attachement" (note that the convention used by Elastic uses "doc" as the default type, but you can pick whatever you want as long as there is a single type).
Thank you for the tip. Fixing some of the spelling errors I had above this did the trick:
POST _reindex {
"source": {
"index": "logstash-*",
"query": {
"bool": {
"must": {
"term": {
"type": "attachments"
}
}
}
}
},
"dest": {
"index": "attachments.old"
},
"script": {
"source": "ctx._id = ctx._type + '-' + ctx._id; ctx._source.type = ctx._type; ctx._type = 'attachments';"
}
}
I've gone through the _reindex api documentation a few times, and can't figure out if it's possible or not. Once the document is copied from the source index to the destination index, is it possible to also remove the source document?
Here is the current _reindex api call body that I'm invoking:
{
"source": {
"index": "srcindex",
"type": "type",
"query": {
"range": {
"date": {
"from": <timestamp>
}
}
}
},
"dest": {
"index": "dstindex",
"type": "type"
}
}
Currently, It is not supported i.e copying then deleting immediately(effectively moving a document).
You can find good discussion happened on this topic here.
Eventually, you need to do _reindex then _delete_by_query to achieve your goal.
Hope this helps!
from documentation:
The most basic form of _reindex just copies documents from one index
to another. This will copy documents from the twitter index into the
new_twitter index
POST /_reindex
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter"
}
}
but i don't want a index with a new name... how preserve the index name (without alias)?