My first config is like:
cluster.name: cluster
node.name: "node-1"
node.master: true
node.data: true
network.host: localhost
transport.tcp.port: 9302
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["localhost"]
And my second node's config is:
cluster.name: cluster
node.name: "node-2"
node.master: false
node.data: true
network.host: 172.28.29.49 (IP OF THE MASTER NODE)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["172.28.29.49:9302"]
I'm running this off two VM's.
The question is pretty basic, why aren't my two nodes connecting to each other.
IGNORE EVERYTHING BELLOW THIS IF YOU WANT. IT'S JUST SOME HISTORY:
The reason why I want to setup two nodes is because I lost heaps of data (https://stackoverflow.com/questions/35191543/elasticsearch-lost-data-after-power-cut).
My plan was to copy the already indexed indices into a new node, but that didn't work and just left me with a whole bunch of unassigned_shards. So I made a script to automatically relocate them, but I can't relocate them to the same node, which is why I need a new node. If someone knows how to relocate shards to the same node or know of another fix for my problem in my old question then that would be great
If you are running ES 2.x, then each node binds to localhost by default, unless specified otherwise.
In your case, the network configuration is not correct (both the bind host and the unicast hosts list). Your first node needs to bind to an IP address that your second host can see and you second host needs to bind to an IP address that your first host can see, otherwise both will be blind and not form any cluster.
So node-1 needs to be configured like this:
network.host: 172.28.29.49
discovery.zen.ping.unicast.hosts: ["IP.OF.NODE.2"]
And node-2 needs to be configured like this:
network.host: IP.OF.NODE.2
discovery.zen.ping.unicast.hosts: ["172.28.29.49:9302"] #Need port number here
Related
We have a long running single node ELK cluster running (master/data). I have decided to add additional data node. However Im getting the below error on the data node
30.X.XXX}{172.30.X.XXX:9300}{ml.enabled=true}], reason [RemoteTransportException[[master][172.30.X.XXX:9300][internal:discovery/zen/join]];
nested: IllegalStateException[failure when sending a validation request to node];
nested: RemoteTransportException[[data1][172.30.X.XXX:9300][internal:discovery/zen/join/validate]];
nested: IllegalArgumentException[Unknown NamedWriteable [org.elasticsearch.cluster.metadata.MetaData$Custom][licenses]]; ]
Below are the config files on master and new data node
Master Node:
cluster.name: my-application
node.name: master
node.master: true
node.data: true
path.data: /opt/elasticsearch
network.host: ["172.30.X.XX1","localhost"]
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["172.30.X.XX1"]
discovery.zen.minimum_master_nodes: 1
Data1 Node:
cluster.name: my-application
node.name: data1
node.master: false
node.data: true
path.data: /opt/elasticsearch
network.host: ["172.30.X.XX2","localhost"]
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["172.30.X.XX1"]
discovery.zen.minimum_master_nodes: 1
Tried pinging and checked telnet on 9200 and 9300 from master to data node and vice versa and it is working fine
I have tried deleting the data from /var/lib/elasticsearch/nodes/0 and restarted the data1, it didnt work
This happens if you try with a mix of xpack/commercial/non-open-source binaries of Elasticsearch and some nodes with the open-source binaries.
Unfortunately Elasticsearch tries to "trick" you into using their non-open-source version nowadays and this causes many unintended non-open-source installations.
A simple solution is to install the non-oss version everywhere, however you may not want to run the commercial version as you then need to adhere to the commercial license!
In order to convert to the open-source license on all nodes you can do the following:
You can set the following in /etc/elasticsearch/elasticsearch.yml and restart all nodes to disable some commercial features:
xpack.security.enabled: false
xpack.ml.enabled: false
Then you can change all nodes to the open-source binaries one by one in rolling fashion.
See also the following similar discussions:
https://discuss.elastic.co/t/elasticsearch-cluster-cant-join-new-node/126964
https://discuss.elastic.co/t/adding-a-new-node-on-a-different-subnet/125377/2- https://github.com/codelibs/elasticsearch-module/issues/3
https://discuss.elastic.co/t/transport-client-error-after-installing-x-pack-on-es-5-5-1/97021/4
https://discuss.elastic.co/t/bulk-indexing-with-x-pack-exception/92086/5
but it can't working after I setup shield
I added user to elastic by command
shield/esusers useradd es_admin -r admin
This is my master node config
cluster.name: vision
node.name: "node_master"
node.master: true
node.data: false
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["192.168.1.5"]
path.logs: /var/elastic/log
path.data: /var/elastic/data
This is my data node config
cluster.name: vision
node.name: "node_data"
node.master: false
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["192.168.1.5"]
path.logs: /var/elastic/log
path.data: /var/elastic/data
How can I connect data node to master node?
There is no extra work you need to do to join data and master node to form a cluster.It treats both type of nodes same.
Your hosts setting is mentioning only one host.
discovery.zen.ping.unicast.hosts: ["host1:port","host2:port"]
Each node will keep pinging the hosts listed above until both are initialized.Adding the local host is of no harm to array as ping wont fail but help in automated deployement of elasticsearch on multinode ecosystem.
since you are using shield make sure if you enabled ssl for node communicatioon then also specify the path to SSL keystore files.
I just setup a 3 node Elasticsearch cluster, with each node having common settings (pasted at the end of the post)
However, when I start my master node, and try to get the cluster status or even check if any one of the nodes is up, I get a 503 as the status code. Also, shutdowns (on any of the nodes) do not work.
Could someone please tell me what I'm doing wrong here? The log file on Node 1 says:
[ESNode1] observer: timeout notification from cluster service. timeout setting [30s], time since start [30s]
Here's snippets from the elasticsearch.yml config files:
Node 1
cluster.name: myCluster
node.name: ESNode1
node.master: true
node.data: true
discovery.zen.minimum_master_nodes: 2
discover.zen.ping.timeout: 20s #just for good measure
discovery.zen.ping.multicast.enabled: false
Node 2
cluster.name: myCluster
node.name: ESNode2
node.master: true
node.data: true
discovery.zen.minimum_master_nodes: 2
discover.zen.ping.timeout: 20s
discovery.zen.ping.multicast.enabled: false
Node 3
cluster.name: myCluster
node.name: ESNode3
node.master: false
node.data: true
discovery.zen.minimum_master_nodes: 2
discover.zen.ping.timeout: 20s
discovery.zen.ping.multicast.enabled: false
Thank you!
You configure that the minimum master nodes is 2. This means your cluster needs at least two master nodes. This is fine, however, together with the setting discovery.zen.ping.multicast.enabled: false this is hard to get working. This setting means you are not going to look for other nodes. So you should configure the nodes manually using the setting hosts.
You can find more information here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#unicast
An example for three nodes running on one machine:
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301","127.0.0.1:9302"]
Disabling multicast discovery means discovery pings will only be sent to specific addresses. The addresses/hosts are those specified in discovery.zen.ping.unicast.hosts.
Note that a single address can be specified. When a node joins it becomes aware of all nodes in the cluster, and can start communicating with them directly.
To clarify using Jettros' example:
discovery.zen.unicast.hosts:["127.0.0.1:9300"]
will cause the nodes bound to 9301 and 9302 to ping only 9300.
If 9301 joins first it 'already knows' all other nodes in the cluster (just 9300).
If 9302 subsequently joins it will become aware of 9301 and vice-versa.
If 9301 and 9302 cant join with 9300 the cluster will not be formed.
Assuming I have 5 machines I want to run an elasticsearch cluster on, and they are all connected to a shared drive. I put a single copy of elasticsearch onto that shared drive so all three can see it. Do I just start the elasticsearch on that shared drive on eall of my machines and the clustering would automatically work its magic? Or would I have to configure specific settings to get the elasticsearch to realize that its running on 5 machines? If so, what are the relevant settings? Should I worry about configuring for replicas or is it handled automatically?
its super easy.
You'll need each machine to have it's own copy of ElasticSearch (simply copy the one you have now) -- the reason is that each machine / node whatever is going to keep it's own files that are sharded accross the cluster.
The only thing you really need to do is edit the config file to include the name of the cluster.
If all machines have the same cluster name elasticsearch will do the rest automatically (as long as the machines are all on the same network)
Read here to get you started:
https://www.elastic.co/guide/en/elasticsearch/guide/current/deploy.html
When you create indexes (where the data goes) you define at that time how many replicas you want (they'll be distributed around the cluster)
It is usually handled automatically.
If autodiscovery doesn't work. Edit the elastic search config file, by enabling unicast discovery
Node 1:
cluster.name: mycluster
node.name: "node1"
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["node1.example.com"]
Node 2:
cluster.name: mycluster
node.name: "node2"
node.master: false
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["node1.example.com"]
and so on for node 3,4,5. Make node 1 master, and the rest only as data nodes.
Edit: Please note that by ES rule, if you have N nodes, then by convention, N/2+1 nodes should be masters for fail-over mechanisms They may or may not be data nodes, though.
Also, in case auto-discovery doesn't work, most probable reason is because the network doesn't allow it (and therefore disabled). If too many auto-discovery pings take place across multiple servers, the resources to manage those pings will prevent other services from running correctly.
For ex, think of a 10,000 node cluster and all 10,000 nodes doing the auto-pings.
Elastic Search 7 changed the configurations for cluster initialisation.
What is important to note is the ES instances communicate internally using the Transport layer(TCP) and not the HTTP protocol which is normally used to perform ops on the indices. Below is sample config for 2 machines cluster.
cluster.name: cluster-new
node.name: node-1
node.master: true
node.data: true
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.host: 102.123.322.211
transport.tcp.port: 9300
discovery.seed_hosts: [“102.123.322.211:9300”,"102.123.322.212:9300”]
cluster.initial_master_nodes:
- "node-1"
- "node-2”
Machine 2 config:-
cluster.name: cluster-new
node.name: node-2
node.master: true
node.data: true
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.host: 102.123.322.212
transport.tcp.port: 9300
discovery.seed_hosts: [“102.123.322.211:9300”,"102.123.322.212:9300”]
cluster.initial_master_nodes:
- "node-1"
- "node-2”
cluster.name: This has be same across all the machines that are going to be part of a cluster.
node.name : Identifier for the ES instance. Defaults to machine name if not given.
node.master: specifies whether this ES instance is going to be master or not
node.data: specifies whether this ES instance is going to be data node or not(hold data)
bootsrap.memory_lock: disable swapping.You can start the cluster without setting this flag. But its recommended to set the lock.More info: https://www.elastic.co/guide/en/elasticsearch/reference/master/setup-configuration-memory.html
network.host: 0.0.0.0 if you want to expose the ES instance over network. 0.0.0.0 is different from 127.0.0.1( aka localhost or loopback address).
It means all IPv4 addresses on the machine. If machine has multiple ip addresses with a server listening on 0.0.0.0, the client can reach the machine from any of the IPv4 addresses.
http.port: port on which this ES instance will listen to for HTTP requests
transport.host: The IPv4 address of the host(this will be used to communicate with other ES instances running on different machines). More info: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-transport.html
transport.tcp.port: 9300 (the port where the machine will accept the tcp connections)
discovery.seed_hosts: This was changed in recent versions. Initialise all the IPv4 addresses with TCP port(important) of ES instances that are going to be part of this cluster. This is going to be same across all ES instances that are part of this cluster.
cluster.initial_master_nodes: node names(node.name) of the ES machines that are going to participate in master election.(Quorum based decision making :- https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-quorums.html#modules-discovery-quorums)
I tried the steps that #KannarKK suggested on ES 2.0.2, however, I could not bring the cluster up and running. Evidently, I figured out something, as I had set tcp port number on Master, on the Slave configuration discovery.zen.ping.unicast.hosts needs Master's port number along with IP address ( tcp port number ) for discovery. So when I try following configuration it works for me.
Node 1
cluster.name: mycluster
node.name: "node1"
node.master: true
node.data: true
http.port : 9200
tcp.port : 9300
discovery.zen.ping.multicast.enabled: false
# I think unicast.host on master is redundant.
discovery.zen.ping.unicast.hosts: ["node1.example.com"]
Node 2
cluster.name: mycluster
node.name: "node2"
node.master: false
node.data: true
http.port : 9201
tcp.port : 9301
discovery.zen.ping.multicast.enabled: false
# The port number of Node 1
discovery.zen.ping.unicast.hosts: ["node1.example.com:9300"]
How can I have multiple nodes in my ElasticSearch? I'm using the following in elasticsearch.yml but only the last node starts, and the browser complains: The page at file://localhost/ says: undefined.
node.name: "No Data"
node.master: true
node.data: false
node.name: "Data One"
node.master: false
node.data: true
node.name: "Data Two"
node.master: false
node.data: true
I think the simplest way to do it is by specifying these parameters on the command line. To start three nodes you just need to run the following three commands in elasticsearch home directory:
$ bin/elasticsearch -Des.node.data=false -Des.node.master=true -Des.node.name=NoData
$ bin/elasticsearch -Des.node.data=true -Des.node.master=false -Des.node.name=DataOne
$ bin/elasticsearch -Des.node.data=true -Des.node.master=false -Des.node.name=DataTwo
Another solution is to create 3 different config files and start three nodes with -Des.config=path-to-config-file parameter.
First off, you should be trying to access elasticsearch using
[http://localhost:9200/][1], if you are using the default port bindings.
I would set up your master node to also be a data node, there is no reason not to. If you are trying to start 3 nodes on a single machine. But, starting 3 nodes all on the same machine doesn't make sense as anything other than an experiment. What are you trying to accomplish?
In windows for 6.x version, command attributes change to
elasticsearch -EsomeYamlPropety=someValue
First You need change an elasticsearch.yml properties to:
http.port: 9200-9299
transport.tcp.port: 9300-9399
node.max_local_storage_nodes: 2
Because You cant run nodes on single port, and when I try to use command with argument -Ehttp.port=9201 nodes where cant see each other and they create two different clusters with the same name.
Run the first node by a standard command:
.\bin\elasticsearch
Run the second node by command with attributes:
.\bin\elasticsearch -Enode.name=NodeTwo -Enode.master=false
for running 3 elasticsearch node on one machine, you should use these configs in elasticsearch.yml file of each node:
for master node :
cluster.name: mycluster
node.name: "node1"
node.master: true
node.data: true
network.host: 127.0.0.1
http.port: 9200-9299
transport.tcp.port: 9300-9399
discovery.zen.minimum_master_nodes: 2
for data nodes :
cluster.name: mycluster
node.name: "data-node-name"
node.master: false
node.data: true
network.host: 127.0.0.1
http.port: 9200-9299
transport.tcp.port: 9300-9399
discovery.zen.minimum_master_nodes: 2
and then u should run each node by :
cd path/to/elasticsearch/bin
path\bin>elasticsearch.bat