I am doing local development on my box and I have deleted some indices that were useful. I wasn't doing any snapshotting, is it possible to restore those indices if I ran the delete command about 5-6 hours ago and it acknowledged with a true.
No, it is not possible, if you deleted an index and do not have snapshots of it, you can't recover the data.
Related
i want to understand how snapshot works in elasticsearch
case1
snapshots are taken every day and snapshots older than 1 month are deleted
I have an index cities and for example there are 3 documents
{ barcelona, madrid, urumqi} and, for example, I deleted the barcelona document from the index, it turns out that if a month passes and the last snapshot in which this index was deleted, then I can no longer recover this document?
case2
I have an elasticsearch cluster and a fairly large number of indexes, the rotation is 3 months, if, for example, a couple of indexes change or all are deleted, then if I restore from a snapshot that was taken 3 months ago, will my cluster be fully restored 3 on months ago data? will snapshot process rewrite all data or not?
if you delete the snapshots that cover an index then you cannot recover any of the data in the index. so no, you cannot recover the document
a restore will restore the data from the time the snapshot is taken. which means yes, the full data from 3 months will be what you see
One of my index in elasticsearch has a shard which is unassigned therefore that data is not available and my cluster status is red ,
I want to restore this index i have a snapshot of the same , so i want to restore this index from the snapshot
Now i see 2 options of doing so
delete current index and then restore from backup
close current index and restore from backup
considering my index number of shards remain the same what is the advantage of closing the index and restoring rather than deleting it and then restoring it.
in your case it has no advantage
in other cases, you may want to close the index, restore to a previous state, delete the restore and then reopen the original, thereby saving a bit of extra work
I have 2TB of indices, trying to manually delete some indices removes them from Kibana, etc. I can delete it via curl or Kibana and it is acknowledged and removed. It is however not freeing up the space.
I went ahead and also removed the ILM from the index before deleting a few indices, still no luck.
Although I removed a whole index, also tried POST _forcemerge to no avail.
How can I recover space now that the indices are deleted?
For those who look at this later
Deleting a whole index should free up space instantly! Does not require _forcemerge, etc.
The issue here was the use of a ZFS file system which required a snapshot to be cleared to recover space.
I'm creating an index though logstash and pushing data to it from a MySQL database. But what I noticed in elasticsearch was once the whole data is uploaded, it starts deleting some of the docs. The total number of docs is 160729. Without the scheduler it works fine.
I inserted the cron scheduler in order to check whether new rows have been added to the table. Can that be the issue?
My logstash conf looks like this.
Where am I going wrong? Or is this behavior common?
Any help could be appreciated.
The docs.deleted number doesn't mean that your documents are being deleted, but simply that existing documents are being "updated" and the older version of the updated document is marked as deleted in the process.
Those documents marked as deleted will be eventually cleaned up as Lucene merges segments in the background.
From reading this article (Lucene's Handling of Deleted Documents), I understand that deleted documents in Elasticsearch are simply marked as deleted, such that they may remain on the disk for some time afterwards.
I was therefore wondering if there was a way to recover deleted documents in Elasticsearch?
Deleted documents and old document versions are totally removed by the segment merging process :(
This is the moment when those old deleted documents are purged from the filesystem. Deleted documents (or old versions of updated documents) are not copied over to the new bigger segment.
https://www.elastic.co/guide/en/elasticsearch/guide/current/merge-process.html