Does anyone know how to tell Elastic Search to stop node to node communications and then restart it..In my system I would like to tell it to stop until a certain condition then restart the communications ( synchronize data)
By node to node communications, do you mean data synchronization and shard relocations?
If yes, you can do it by setting cluster.routing.allocation.enable to none using cluster settings API.
If you don't mean data synchronization, you can achieve this by blocking the port 9300 (or which ever port ES is using for internal communication).
Please note that any node leaves the cluster will cause the elasticsearch to rebalance the shards and replica. The overall cluster loading increases when any node is lost since the cluster needs to fulfill the shard and replica settings by copying existing data to rest of nodes. Therefore, if the operation happens often, the considerable extra space will be consumed for additional shards and replicas.
If you fully understand the impact, you can try the shard allocation filtering. For example, exclude the host ip 10.0.0.1 from the cluster:
PUT _cluster/settings
{
"transient" : {
"cluster.routing.allocation.exclude._ip" : "10.0.0.1"
}
}
Other than ip, you can use node name or host name to exclude the node as well.
You can find the full documentation here: https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-filtering.html
Related
We have a small Elasticsearch cluster for 3 nodes: two in one datacenter and one in another for disaster recovery reasons. However, if the first two nodes fail simultaneously, the third one won't work either - it will just throw "master not discovered or elected yet".
I understand that this is intended - this is how Elasticsearch cluster should work. But is there some additional special configuration that I don't know to keep the third single node working, even if in the read-only mode?
nope, there's not. as you mentioned it's designed that way
you're probably not doing yourselves a lot of favours by running things across datacentres like that. network issues are not kind on Elasticsearch due to it's distributed nature
Elasticsearch runs in distributed mode by default. Nodes assume that there are or will be a part of the cluster, and during setup nodes try to automatically join the cluster.
If you want your Elasticsearch to be available for only node without the need to communicate with other Elasticsearch nodes. It works similar to a standalone server. To do this we can tell Elasticsearch to work in local only (disable network)
open your elasticsearch/config/elasticsearch.yml and set:
node.local: true
I'm researching Akka Cluster Sharding, after read Akka documents and some topic in internet, I thought that when cluster has an unreachable node, only communication to shards on this node are interrupted. It means that messages are told to shards on it will be buffered and senders will receive a timeout response and messages are told to shards on other available nodes will be send and receive normally.
But when I tried to implement a cluster with 4 nodes, and then stopped service of one node, checked via Akka HTTP Management and confirm it was set unreachable, then I tested sending messages from an actor in an available node to an actor in other available node and received Timeout response.
So, anyone can help me to confirm when cluster has unreachable node, entire sharding system will not work normally, or just unreachable node?
Thanks
From documentation:
Shard Coordinator State
The state of shard locations in the ShardCoordinator is persistent (durable) with Distributed Data or Persistence to survive failures. When a crashed or unreachable coordinator node has been removed (via down) from the cluster a new ShardCoordinator singleton actor will take over and the state is recovered. During such a failure period shards with known location are still available, while messages for new (unknown) shards are buffered until the new ShardCoordinator becomes available.
In this we have "During such a failure period shards with known location are still available", so when the Shard Coordinator is unreachable the whole cluster could not work properly.
For reference: Shard Coordinator State in doc.akka.io
Currently I connect to a ElasticSearch cluster as follows:
(esr/connect "localhost:9200")
But I am concerned about availability so plan to run an ElasticSearch cluster.
How do I modify my Elastisch code to connect to a cluster (so that if a node is unavailable I can fall back to another node)? Does it do this by default? The ElasticSearch java rest client seems to offer this functionality so does Elastisch?
You can have setup of cluster with multiple hosts, this can can be configured using elasticsearch.yaml configuration file like:
.....
.....
discovery.zen.ping.unicast.hosts: ['192.168.10.1:9300', '192.168.10.2:9300']
also elect one node as master and other as slave or data node
# Allow this node to be eligible as a master node (enabled by default):
#
node.master: true
#
# Allow this node to store data (enabled by default):
#
node.data: true
also you can explore more about the same by below links
about Zen discovery in clustered enviroment
Important configuration for elasticsearch
One of the benefits of using a service like elasticsearch is that it takes care of the availability part of the equation, in that ES itself will handle nodes going down. You do have to configure it intelligently, which is outside the scope of this question/answer.
The connect function here does not actually connect; it basically just creates a URI and options, and when you call a function like clojurewerkz.elastisch.rest.document/search, you give it the connection data, which is then used in an actual network operation.
So, you can call esr/connect as often as you like on as many URLs as you like, but you don't need to. I recommend reading elasticsearch's documentation to get familiar with the architecture, about nodes, clusters, indexes, shards, etc. -- and configure your elasticsearch cluster properly. But as far as the code itself goes, you are insulated from the architecture and need not worry about these details. This is true of elasticsearch's REST API, and thus the elastisch wrapper also provides this.
I have couple of Elasticsearch questions regarding client node:
Can I say: any nodes as long as they are opening HTTP port, I can treat them as "client" nodes, because we can do search/index through this node.
Actually we treat the node as client node when the cluster=false and data=false, if I set up 10 client nodes, do I need to route in my client side, I mean if I specify clientOne:9200 in my code as ES portal, then would clientOne forward other HTTP requests to other client nodes, otherwise, clientOne would be under very high pressure. i.e do they communicate with each other between client nodes?
When I specify client nodes in ES cluster, should I close other nodes' HTTP port? Because we can only query client nodes.
Do you think it's necessary to set up both data node and client node in the same machine, or just setup data node acts as client node as well, anyways it's in the same machine?
If the ES cluster would be heavily/frequently indexed while less searched, then I don't have to set up client node, because client node good for gathering data, right please?
For general search/index purpose should I use http port or tcp port, what's the difference in clients perspective please?
Yes, you can send queries via http to any node that has port 9200 open.
With node.data: false and node.master: false, you get a "client node". These are useful for offloading indexing and search traffic from your data nodes. If you have 10 of them, you would want to put a load balancer in front of them.
Closing the data node's http port (http.enabled: false) would keep them from serving client requests (probably good), though it would also prevent you from curl'ing them directly for stats, etc.
Client nodes are useful (see #2), so I wouldn't route traffic directly to your data nodes. Whether you run both a client and data node on the same piece of hardware would be dependent on the config of that machine (do you have sufficient RAM, etc).
Client node are also useful for indexing, because they know which data node should receive the data for storage. If you sent an indexing request to a random data node instead, the odds would be high that it would have to redirect that request to another node. That's a waste of time and resources, if you can create client nodes.
Having your clients join the cluster might give them access to more information about the cluster, but using http gives them a more generic "black box" interface. With http, you also don't have to keep your clients at the same version as your ES nodes.
Hope that helps.
I am new to elasticsearch. I have a cluster with 3 nodes on a same machine. To access each node I have separate url as the port changes(localhost:9200, localhost:9201, localhost:9202).
Now the question I have is that suppose my node 1(i.e. master node) dies then elasticsearch engine handle the situation very well and makes node 2 as master node but how does my application know that a node died and now I need to hit node 2 with port 9201?
Is there a way using which I always hit a single URL and internally it figures out which node to hit?
Thanks,
Pratz
The client search nodes with a discovery module. The name of the cluster in your clients configuration is important to get this working.
With a correct configuration (on client and cluster) you can bring a single node down without any (negative) effect on your client.
See the following links:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery.html
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html