We are using ElasticSearch 1.x on production for sometime now with millions of records.
We want to upgrade the version from 1.x to 6.x as:
There have been multiple updates by the company and the support for older versions is discontinued.
1.x does not support Kibana.
What's the best way to do it with explicit steps on data security?
Thanks!
I've recently did a migration from Elasticsearch 1.5 to 6.2.
Steps, that needs to be performed:
Update the mappings, there are a lot of changes that happened between those 2 versions (just as an example _all field is disable starting from 6.0). The official documentation should help you here.
After you updated the mappings you would need another cluster set up with desired version of Elasticsearch. Also update if needed your Logstash/Kibana.
Enable it to access your old cluster by adding your old cluster to the reindex.remote.whitelist in elasticsearch.yml, by doing: reindex.remote.whitelist: oldhost:9200
For each index that you need to migrate, you would need to manually create a new index in your new сluster with updated mappings from #1
Reindex from remote to pull documents from the old index into the new 6.x index
Full documentation regarding this one is available here - https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html
Related
I want to upgrade my single node elasticsearch from version 5.30 to 7.2. What is the best possible ways of doing it?
Please follow the elasticsearch official document https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html for upgrading your elasticsearch version.
Some important points to take care from upgrading 5.x to 7.3:
Elasticsearch can read indices created in the previous major version. If you have indices created in 5.x or before, you must reindex or delete them before upgrading to 7.3.1. Elasticsearch nodes will fail to start if incompatible indices are present.
Take care of things mentioned in https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html#_preparing_to_upgrade
Let me know if you need additional information.
I want to upgrade elastic search version with updated mapping( i have renamed/added/deleted some field)
how can I migrate data , any reference apprecited
Thanks
There are multiple ways to achieve that. It depends on the amount of data the cluster holds, whether maintenance downtime is acceptable or not, and do you have backward compatibility issues with the fields names/types.
A simple (but a bit costly) way to upgrade would be to spin up a new cluster with version 6.0 and reindex the data from the old cluster to the new. another method can be to upgrade to latest 5.X version (5.6) and from there do a rolling restart. from https://www.elastic.co/guide/en/elastic-stack/6.0/upgrading-elastic-stack.html : "Rolling upgrades are supported when upgrading from Elasticsearch 5.6 to 6.0.1"
I want to update all elasticsearch indices from elasticsearch 5.6.2 to 6.8 version.
I don't want to use X-PACK for migration.
I am not using any elastic products.
So, Is there any proper guideline of way for this migration?
I can do it manually like write new mappings without type for my existing indices and then dump documents from my current indices to new one.
But I've more than 150 indices and it will take time to create new mappings and migrate it manually.
Elastic is the company behind Elasticsearch, so by using that you are using an Elastic product ;-)
The only upgrade assistant I'm aware of is in the free Basic (X-Pack) version of Kibana. Otherwise you'll have to manually run through the features described in https://www.elastic.co/guide/en/kibana/6.0/xpack-upgrade-assistant.html.
I have Elasticsearch 1.5 installed on my machine, I am planning to migrate to Elasticsearch 5.6.1, how can I make it without losing my indexes.
From 1.x to 5.x you need to reindex to upgrade. You won't lose any of your data. Follow this link
What is the right way to upgrade elasticsearch from 1.7 to 5.2?
I've tried to upgrade it to 2.4 version first and done reindex procedure and then I installed elasticsearch v.5.2. When v.5.2 was started in logs saw this messages:
3) Error injecting constructor, java.lang.IllegalStateException: The
index [[logstash-2017.01.15/tk1-FBLGTQGl0vVW1xxPpg]] was created with
version [1.7.3] but the minimum compatible version is [2.0.0-beta1].
It should be re-indexed in Elasticsearch 2.x before upgrading to
5.2.2.
Looking at the official documentation, if you are running an Elasticsearch 1.x cluster, you have two options:
First upgrade to Elasticsearch 2.4.x, reindex the old indices, then upgrade to 5.x.
Create a new 5.x cluster and use reindex-from-remote to import indices directly from the 1.x cluster.
Reindex in place
The easiest way to reindex old (1.x) indices in place is to use the Elasticsearch Migration Plugin. You will need to upgrade to Elasticsearch 2.3.x or 2.4.x first.
The reindex utility provided in the migration plugin does the following:
Creates a new index with the Elasticsearch version appended to the old index name (e.g. my_index-2.4.1), copying the mappings and settings from the old index. Refresh is disabled on the new index and the number of replicas is set to 0 for efficient reindexing.
Sets the old index to read only to ensure that no data is written to the old index.
Reindexes all documents from the old index to the new index.
Resets the refresh_interval and number_of_replicas to the values used in the old index, and waits for the index to become green.
Adds any aliases that existed on the old index to the new index.
Deletes the old index.
Adds an alias to the new index with the old index name, e.g. alias my_index points to index my_index-2.4.1.
At the end of this process, you will have a new 2.x index which can be used by an Elasticsearch 5.x cluster.
Upgrading with reindex-from-remote
If you are running a 1.x cluster and would like to migrate directly to 5.x without first migrating to 2.x, you can do so using reindex-from-remote.
You will need to set up a 5.x cluster alongside your existing 1.x cluster. The 5.x cluster needs to have access to the REST API of the 1.x cluster.
For each 1.x index that you want to transfer to the 5.x cluster, you
will need to:
Create a new index in 5.x with the appropriate mappings and settings.
Set the refresh_interval to -1 and set number_of_replicas to 0 for faster reindexing.
Use reindex-from-remote to pull documents from the 1.x index into the new 5.x index. If you run the reindex job in the background (with wait_for_completion set to false), the reindex request will return a task_id which can be used to monitor progress of the reindex job in the task API: GET _tasks/TASK_ID.
Once reindex has completed, set the refresh_interval and number_of_replicas to the desired values (the defaults are 30s and 1 respectively).
Once the new index has finished replication, you can delete the old index.