We are using Elasticsearch 5.6 hosted on AWS ElasticSearch service and using Rails searchkick gem (v2.5). We are planning to migrate to Elasticsearch 7 and searchkick v4 at some point to make use of the split index functionality.
I am checking whether I would need to rebuild the whole index, it isn't too hard to do for us, maybe over a day.
Over here: https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html
Here is a couple of conflicting pieces of information I see on this doc page:
"
Elasticsearch can usually be upgraded using a Rolling upgrade process so upgrading does not interrupt service. Rolling upgrades are supported:
• Between minor versions
• From 5.6 to 6.8
• From 6.8 to 7.3.2"
...
"Elasticsearch can read indices created in the previous major version. If you have indices created in 5.x or before, you must reindex or delete them before upgrading to 7.3.2. "
If one can do a rolling upgrade from 5.6 to 6.8, and from 6.8 to 7.3.2, and elasticsearch can read indices created in the previous major versions, then is the next statement you must reindex from 5.x to 7.3.2 true?
No.
I recommend this solution:
first you should build a new parallel 7.3 cluster. then "remote reindex" data from 5.6 to 7.3.
https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html
the other solution:
you can rolling upgrade to 6.5 then "reindex in place" the indices from 5.6 to 6.5.
in that moment you have two option:
rolling upgrade to 7.3 and use 6.5 indices (Lucene 7) and miss some new Lucene 8 feature.
rolling upgrade to 7.3 and again "reindex in place" the indices from 6.5 to 7.3 (Lucene 8). with means that you reindex your data two times.
Related
I have an elastic search cluster running 5.6. I plan to upgrade my cluster but i plan to do it by basically running a ES cluster 8.3 running in parallel and then moving data over to it.
The preferred way i think is to do snapshot and restore https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html
But I am confused about what exactly Snapshot version compatibility mean :
In the above does it mean that if i take a snapshot in elasticsearch 5.6, I cannot restore directly in version 8.3 ?? (which mean I have to first move to 6.x then to 7.x and finally to 8.x ) ??
The below index compatibility matrix, however says that a version in 5.x will work in 8.x ?
Am i missing something ? or can someone help me elaborate this?
So, the underlying problem is that data written in Lucene version N, can only be read with N+1. For Elasticsearch 5 to 8 the Lucene version was always 1 greater than the ES version (so 6 to 9).
That means, both for an upgrade or a restored snapshot: If your data was written with 5.x, you can only read / restore it with 6.x. For 7.x or 8.x you'll need to reindex the data. I would do a remote reindex straight from 5.x to 8.latest if possible: https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html
There are some small caveats but they will probably not apply to you:
This doesn't apply to source only snapshots, but those always need a reindex, so that's not going to add any benefit for you.
8.3 added a feature to still read snapshots from 5.0 on but it is slower, doesn't support all features, and it is a commercial feature (platinum license if I'm not mistaken).
Depending on what kind of data it is: If it's aging out (like logs or metrics), maybe you don't have to migrate it to the new cluster?
I want to upgrade my single node elasticsearch from version 5.30 to 7.2. What is the best possible ways of doing it?
Please follow the elasticsearch official document https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html for upgrading your elasticsearch version.
Some important points to take care from upgrading 5.x to 7.3:
Elasticsearch can read indices created in the previous major version. If you have indices created in 5.x or before, you must reindex or delete them before upgrading to 7.3.1. Elasticsearch nodes will fail to start if incompatible indices are present.
Take care of things mentioned in https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html#_preparing_to_upgrade
Let me know if you need additional information.
I want to upgrade elastic search version with updated mapping( i have renamed/added/deleted some field)
how can I migrate data , any reference apprecited
Thanks
There are multiple ways to achieve that. It depends on the amount of data the cluster holds, whether maintenance downtime is acceptable or not, and do you have backward compatibility issues with the fields names/types.
A simple (but a bit costly) way to upgrade would be to spin up a new cluster with version 6.0 and reindex the data from the old cluster to the new. another method can be to upgrade to latest 5.X version (5.6) and from there do a rolling restart. from https://www.elastic.co/guide/en/elastic-stack/6.0/upgrading-elastic-stack.html : "Rolling upgrades are supported when upgrading from Elasticsearch 5.6 to 6.0.1"
I am in need of a solution to upgrade/migrate elasticsearch from 1.7.6 to 7.0 with all the data and the reports. Is there any way to do it?
Thanks in advance.
You have to install elastic search 7.0 and reindex the data again.
You can't just migrate the old indexes of elastic search 1.7 to elastic search 7.0.
Elasticsearch can read indices created in the previous major version.
If you have indices created in 5.x or before, you must reindex or
delete them before upgrading to 7.1.1. Elasticsearch nodes will fail
to start if incompatible indices are present. Snapshots of 5.x or
earlier indices cannot be restored to a 7.x cluster even if they were
created by a 6.x cluster.
Refer elastic search documentation
I have Elasticsearch 1.5 installed on my machine, I am planning to migrate to Elasticsearch 5.6.1, how can I make it without losing my indexes.
From 1.x to 5.x you need to reindex to upgrade. You won't lose any of your data. Follow this link