We have 2 datacenters where we have deployed 2 elasticsearch clusters and they are configured to use CCR (Cross Cluster Replication) in order to sync each-other.
Unfortunatelly CCR requires a License which we are not going to renew. There for we were thinking to convert the existing archiecture with CCR in a Primary/Secondary:
Applications connect to the Primary ELK cluster
In case of Disaster Recovery (e.g. Datacenter 1 goes down) we configure the applications to use the secondary ELK cluster located in the DC 2
Once the DC 1 is up and running again, the ELK cluster on DC 1 become the new secondary
Is this a valid approach? Is it achievable?
any tutorial on how to configure the PRimary/Seconday clusters in ELK
or "creating a recurrent backup of ELK1 and restore it on ELK2" is the only solution?
We have solved this creating a primary/secondary Elasticsearch instance and we forwards the events with Logastash to both cluster at the same time
Related
I want to create a Failover cluster for MSMQ for two vm's in azure. I created two VM's in azure and have them domain joined. I can create the failover cluster with both nodes. However when i try to add a role for MSMQ i need an cluster shared disk. I tried to create a new managed disk in azure and attach it to the vm's but it still wasn't able to find the disk.
Also tried fileshare-sync, but still not working.
I found out i need iSCSI disk, there was this article https://learn.microsoft.com/en-us/azure/storsimple/storsimple-virtual-array-deploy3-iscsi-setup . But it is end of life next year.
So i am wondering if it is possible to setup a failover cluster for msmq on azure and if so how can i do it?
Kind regards,
You should be able to create a Cluster Shared Volume using Storage Spaces Direct across a cluster of Azure VMs. Here are instructions for a SQL failover cluster. I assume this should work for MSMQ, but I haven't set up MSMQ in over 10 years and I don't' know if requirements are different.
I would like to setup an IOT reporting database, with failover. The idea is to have a cluster with 2 nodes, one in a datacenter one at home.
If "home" looses internet connection, it continues to operate and upon
online status, "datacenter" would align to offline changes.
Now, I read the rethinkdb docs, that you need at least 3 nodes for a failover to function.
So the question is, is my scenario doable with just 2 nodes, if yes, how?
According to the docu https://www.rethinkdb.com/docs/start-a-server/
First, start RethinkDB on the first machine:
$ rethinkdb --bind all
Then start RethinkDB on the second machine:
$ rethinkdb --join IP_OF_FIRST_MACHINE:29015 --bind all
I am working on product in which we are using elasticsearch for search. Our production setup is in K8S (1.7.7) and we are able to scale it pretty well. Only thing I am not sure about is whether we should be hosting elasticsearch in k8s (it can go on dedicated host as well using label selector nodes) or it is advisable to host elasticsearch on VM than docker.
Our data set size is 2-3 GB and would go further. But this is the benchmark we can consider.
And elasticsearch cluster I am planning to have ti is - 3 master (with 2 as eligible master), one client node, and one data node. We can scale datanode and client node as data increases.
Is anyone did this before? thanks in advance.
IMO the best resource for Elasticsearch on Kubernetes is https://github.com/pires/kubernetes-elasticsearch-cluster
Note that while there are official Docker containers, no official solution for orchestration is being provided at the moment. This is currently covered by the community only.
3 master (with 2 as eligible master)
This doesn't make much sense. You'll want 3 master eligible nodes with the setting discovery.zen.minimum_master_nodes: 2 and one of the 3 nodes will be the actual master.
Just thinking about app arhitecture and whant to know is it possible at all to create local cluster for specific tables and connect it with cloud cluster?
And additional question - is it possible to choose where to create shard (on what machine) for particular table (to show cloud cluster that for this table i need shards in local cluster)?
As example, I whant to have table db.localTable be sharded in local cluster to reduce latency and increase performance due to run queries in local cluster and also have ability to run queries in cloud cluster when local cluster is not accessible. All data between clusters should be consistent.
Tnx in advance.
Actually, I've found the solution: to set specific servers for replicas and servers for shards you should use server-tags and perform changes using ReQL and tables setting. For details see - RethinkDB - Scaling, sharding and replication and RethinkDB - Architecture FAQ
It's a known fact that it is not possible to create a cluster in a single machine by changing ports. The workaround is to add virtual Ethernet devices to our machine and use these to configure the cluster.
I want to deploy a cluster of , let's say 6 nodes, on two ec2 instances. That means, 3 nodes on each machine. Is it possible? What should be the seed nodes address, if it's possible?
Is it a good idea for production?
You can use Datastax AMI on AWS. Datastax Enterprise is a suitable solution for production.
I am not sure about your cluster, because each node need its own config files and it is default. I have no idea how to change it.
There are simple instructions here. When you configure instances settings, you have to write advanced settings for cluster, like --clustername yourCluster --totalnodes 6 --version community etc. You also can install Cassandra manually by installing latest version java and cassandra.
You can build cluster by modifying /etc/cassandra/cassandra.yaml (Ubuntu 12.04) fields like cluster_name, seeds, listener_address, rpc_broadcast and token. Cluster_name have to be same for whole cluster. Seed is master node, which IP you should add for every node. I am confused about tokens