Circumstances:
For 3 years there is an application, that uses tarantool (now it is 1.10.3), and once upon a time we've decided to move it to Kubernetes and replace old and ugly dockerfile, based on Jessie to the official image tarantool/tarantool:2.3.1. I don't know will it be all fine with the data.
In this case, I have two questions, and I will be really happy to read the correct answers from you:
Does tarantool 2.3.1-2-g92750c828 support recovering from snapshots, made by tarantool 1.10.3?
How can I surely load the snapshot data by the new version? It would be great to make it without restarting tarantool, because I have PVC like "emptyDir" that can't handle any data when pod restarts)
Yes, Tarantool 2.3 is compatible with Tarantool 1.10 in terms of binary protocol and snapshot format. If you can't simply run Tarantool 2.3 based on snapshot of 1.10, please file an issue - https://github.com/tarantool/tarantool/issues
I want to note several things: after upgrade to 2.3 it could be impossible to run it back on 1.10 (some system spaces/records incompatible will be created).
Sorry, I can't answer for this question for Kubernetes. But you can't simply stop and run again single instance. But upgrade without downtime is available for replication cluster - https://www.tarantool.io/en/doc/2.4/book/admin/upgrades/#upgrading-tarantool-in-a-replication-cluster
Related
I have an elastic search cluster running 5.6. I plan to upgrade my cluster but i plan to do it by basically running a ES cluster 8.3 running in parallel and then moving data over to it.
The preferred way i think is to do snapshot and restore https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html
But I am confused about what exactly Snapshot version compatibility mean :
In the above does it mean that if i take a snapshot in elasticsearch 5.6, I cannot restore directly in version 8.3 ?? (which mean I have to first move to 6.x then to 7.x and finally to 8.x ) ??
The below index compatibility matrix, however says that a version in 5.x will work in 8.x ?
Am i missing something ? or can someone help me elaborate this?
So, the underlying problem is that data written in Lucene version N, can only be read with N+1. For Elasticsearch 5 to 8 the Lucene version was always 1 greater than the ES version (so 6 to 9).
That means, both for an upgrade or a restored snapshot: If your data was written with 5.x, you can only read / restore it with 6.x. For 7.x or 8.x you'll need to reindex the data. I would do a remote reindex straight from 5.x to 8.latest if possible: https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html
There are some small caveats but they will probably not apply to you:
This doesn't apply to source only snapshots, but those always need a reindex, so that's not going to add any benefit for you.
8.3 added a feature to still read snapshots from 5.0 on but it is slower, doesn't support all features, and it is a commercial feature (platinum license if I'm not mistaken).
Depending on what kind of data it is: If it's aging out (like logs or metrics), maybe you don't have to migrate it to the new cluster?
I have a Spring Boot application that uses ElasticSearch 6.8 and I would like to migrate it to Elasticsearch 7.16 with least downtime. I can do rolling update but the problem with migration is that when I migrate my ES cluster from version 6 to 7, some features in my application fails because of breaking changes (for example total hit response change)
I also upgraded my ElasticSearch client to version 7 in a separate branch and I can deploy it as well but that client doesn't work with ES version 6. So I cannot first release the application and then do the ES migration. I thought about doing application deployment and ES migration at the same time with a few hours downtime but in case something goes wrong rollback may take too much time (We have >10TB data in PROD).
I still couldn't find a good solution to this problem. I'm thinking to migrate only ES data nodes to 7.16 version and keep master nodes in 6.8. Then do application deployment and migrate ElasticSearch master nodes together with a small downtime. Has anyone tried doing this? Would running data and master nodes of my ElasticSearch cluster in different versions (6.8 and 7.16) cause problem?
Any help / suggestion is much appreciated
The breaking change you mention can be alleviated by using the query string parameter rest_total_hits_as_int=true in your client code in order to keep getting total hit count as in version 6 (mentioned in the same link you shared).
Running master and data nodes with different versions is not supported and I would not venture into it. If you have a staging environment where you can test this upgrade procedure it's better.
Since 6.8 clients are compatible with 7.16 clusters, you can add that small bit to your 6.8 client code, then you should be able to upgrade your cluster to 7.16.
When your ES server is upgraded, you can upgrade your application code to use the 7.16 client and you'll be good.
As usual with upgrades, since you cannot revert them once started, you should test this on a test environment first.
I'm running an ELK stack and would like a strategy for automatically retiring logs older than a certain age... I have tried using elasticsearch-curator but it requires python 2.7 and I have python 2.6.6, and am anxious to upgrade python in case I break other packages.
Is there a similar product or does the elasticsearch api cater for such a requirement?
The older versions of curator will work with older versions of ElasticSearch ( I'm using curator version 3.1.0 with a 1.7.1 ES cluster.)
We started out using the Elasticsearch S3 archiving plugin but soon discovered certain limitations when wanting to restore data. We also experienced performance issues with the plugin, which tended to slow down the entire cluster. Since then, we have migrated to a new system in which we archive the data for us and our customers using our own code before indexing the data to Elasticsearch in a clear text format. This gives us all the flexibility we and our customers require.
You may be mistaking the dependency on a given version of the elasticsearch-py module for a version of Elasticsearch. Curator version 3.5.1 requires es-py 2.3, but works with any version of elasticsearch greater than 1.0.
We have a Elasticsearch cluster which is running on elasticsearch 1.4 and logstash 1.4 with 1 master and 4 data node, now I want to upgrade the version of elasticsearch to 1.7 and logstash to 1.5 without losing any data. So my plan is to create a new cluster with new nodes and restore the snapshot of the current cluster on that. Now my question is this the best way or upgrade the versions on the current cluster. I am bit of nervous because it a production logging stack working smoothly.I don't want to mess around with production cluster with testing
First of all, read documentation. As you said, you'd like to upgrade from 1.4 to 1.7, which means there's no significant version jump.
Documentation states that upgrading from 1.x version to another 1.x version you have to do a rolling upgrade. What's that? Quoting documentation:
A rolling upgrade allows the ES cluster to be upgraded one node at a
time, with no observable downtime for end users.
Which means you can shut node down one by one, upgrade its binaries and turn it back on. One node by one!.
Of course, always do a backup in case **** happens.
Basically I have followed the instructions here
But when I go to couchbase to setup the replication i get:
but I know it is there because
The machines can ping each other and no firewalls etc.....
I have found the following error in the log for elaticsearch every time i hit replicate in couchbase.
I am not sure what this means but....
Any pointers would be appreciated
Kindest Regards
Robin
Edit
Ps: lastest version of both couchbase and elasticsearch from today and running ubuntu server 12.04 LTS
I think that the CouchBase plugin has not been released yet for Elasticsearch 0.20.x but only for 0.19.x.
I think this is your issue here.
So try switching to a 0.19.9 ES version or wait for the next release of the CouchBase plugin.