My cluster has a yellow health as it has only one single node, so the replicas remain unasigned simply because no other node is available to contain them.
I'am readed help in homepage:
I try add new node-data to my cluster, but it not appear when i check health's cluster.
This is config new node:
cluster.name: elasticsearch
node.name: node-data-1
node.master: false
node.data: true
node.ingest: false
node.ml: false
http.port: 9201
Old config, i don't edit. It's default.
Can someone explain me which files do I've to edit and what commands do I've to launch in order to create another node in my cluster? Do I've to run two ES instance? How can I do this?
Thanks in advance.
Make sure to remove the /data folder from each Elasticsearch install before executing the nodes. Also, in the nodes that won't be the master set the following property:
cluster.initial_master_nodes: ["machine_running_master"]
Related
First of all, I want to be clear that I looked at several guides and suggested similar questions before opening this post, but none of them worked for our case.
Here is our situation:
Last week, our Elasticsearch stopped adding new records as it reached the state where the disk was almost full. We changed the config to get some time and now is working as expected. However, we want to add a new server with elasticsearch and form a cluster as we don't want to resize the disk because we don't want to lose anything.
Here is the configuration for the main server /etc/elasticsearch/elasticsearch.yml
cluster.name: my-cluster
node.name: master-node
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.max_content_length: 100mb
discover.seed_hosts: ["ip_address_server_1", "ip_address_server_2"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true
As I said, we created another server with the same elasticsearch version (7.6.2)
cluster.name: my-cluster
node.name: another-node
node.data: true
node.master: false
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: ip_of_server
discover.seed_hosts: ["ip_address_server_1", "127.0.0.1"]
We tried restarting nodes at the same time, curls and everything but they can't see each other. At this point, I'm starting to think that the problem is the SSL configuration on the master, but I am not an elasticsearch expert, so I don't know what is exactly happening.
EDIT:
I took a look at the logs and here is what I found:
Node 2
[ico-elastic-node-2] master not discovered yet: have discovered [{node-2}{G5SfrEv0RxaxmYf8urIFtQ}{XpfNNouCQx2HmYfw2AvoQw}{ip_server_2}{ip_server_29300}{dil}{ml.machine_memory=12558602240, xpack.installed=true, ml.max_open_jobs=20}]; discovery will continue using [ip_server_1, 127.0.0.1:9300] from hosts providers and [] from last-known cluster state; node term 0, last-accepted version 0 in term 0
And the master:
[master-node] exception caught on transport layer [Netty4TcpChannel{localAddress=/ip:9300, remoteAddress=/ip:36660}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: No available authentication scheme
Currently master is yellow. What can we try next?
We have a long running single node ELK cluster running (master/data). I have decided to add additional data node. However Im getting the below error on the data node
30.X.XXX}{172.30.X.XXX:9300}{ml.enabled=true}], reason [RemoteTransportException[[master][172.30.X.XXX:9300][internal:discovery/zen/join]];
nested: IllegalStateException[failure when sending a validation request to node];
nested: RemoteTransportException[[data1][172.30.X.XXX:9300][internal:discovery/zen/join/validate]];
nested: IllegalArgumentException[Unknown NamedWriteable [org.elasticsearch.cluster.metadata.MetaData$Custom][licenses]]; ]
Below are the config files on master and new data node
Master Node:
cluster.name: my-application
node.name: master
node.master: true
node.data: true
path.data: /opt/elasticsearch
network.host: ["172.30.X.XX1","localhost"]
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["172.30.X.XX1"]
discovery.zen.minimum_master_nodes: 1
Data1 Node:
cluster.name: my-application
node.name: data1
node.master: false
node.data: true
path.data: /opt/elasticsearch
network.host: ["172.30.X.XX2","localhost"]
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["172.30.X.XX1"]
discovery.zen.minimum_master_nodes: 1
Tried pinging and checked telnet on 9200 and 9300 from master to data node and vice versa and it is working fine
I have tried deleting the data from /var/lib/elasticsearch/nodes/0 and restarted the data1, it didnt work
This happens if you try with a mix of xpack/commercial/non-open-source binaries of Elasticsearch and some nodes with the open-source binaries.
Unfortunately Elasticsearch tries to "trick" you into using their non-open-source version nowadays and this causes many unintended non-open-source installations.
A simple solution is to install the non-oss version everywhere, however you may not want to run the commercial version as you then need to adhere to the commercial license!
In order to convert to the open-source license on all nodes you can do the following:
You can set the following in /etc/elasticsearch/elasticsearch.yml and restart all nodes to disable some commercial features:
xpack.security.enabled: false
xpack.ml.enabled: false
Then you can change all nodes to the open-source binaries one by one in rolling fashion.
See also the following similar discussions:
https://discuss.elastic.co/t/elasticsearch-cluster-cant-join-new-node/126964
https://discuss.elastic.co/t/adding-a-new-node-on-a-different-subnet/125377/2- https://github.com/codelibs/elasticsearch-module/issues/3
https://discuss.elastic.co/t/transport-client-error-after-installing-x-pack-on-es-5-5-1/97021/4
https://discuss.elastic.co/t/bulk-indexing-with-x-pack-exception/92086/5
My first config is like:
cluster.name: cluster
node.name: "node-1"
node.master: true
node.data: true
network.host: localhost
transport.tcp.port: 9302
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["localhost"]
And my second node's config is:
cluster.name: cluster
node.name: "node-2"
node.master: false
node.data: true
network.host: 172.28.29.49 (IP OF THE MASTER NODE)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["172.28.29.49:9302"]
I'm running this off two VM's.
The question is pretty basic, why aren't my two nodes connecting to each other.
IGNORE EVERYTHING BELLOW THIS IF YOU WANT. IT'S JUST SOME HISTORY:
The reason why I want to setup two nodes is because I lost heaps of data (https://stackoverflow.com/questions/35191543/elasticsearch-lost-data-after-power-cut).
My plan was to copy the already indexed indices into a new node, but that didn't work and just left me with a whole bunch of unassigned_shards. So I made a script to automatically relocate them, but I can't relocate them to the same node, which is why I need a new node. If someone knows how to relocate shards to the same node or know of another fix for my problem in my old question then that would be great
If you are running ES 2.x, then each node binds to localhost by default, unless specified otherwise.
In your case, the network configuration is not correct (both the bind host and the unicast hosts list). Your first node needs to bind to an IP address that your second host can see and you second host needs to bind to an IP address that your first host can see, otherwise both will be blind and not form any cluster.
So node-1 needs to be configured like this:
network.host: 172.28.29.49
discovery.zen.ping.unicast.hosts: ["IP.OF.NODE.2"]
And node-2 needs to be configured like this:
network.host: IP.OF.NODE.2
discovery.zen.ping.unicast.hosts: ["172.28.29.49:9302"] #Need port number here
I have a server running elasticsearch and kibana. I have added a second node to form a cluster but only want that second node to replicate data from the master node.
Based on limited documentation on how to do this, I am running into issue on second with following error
[DEBUG][action.admin.indices.get ] [Match] no known master node, scheduling a retry
I am unable to determine the best configuration for both servers to achieve this but this is what I have done so far:
Master Node Config:
cluster.name: elasticsearch
node.master: true
path.data: /local00/elasticsearch/
path.work: /local00/el_temp/
network.host: 0.0.0.0
http.port: 9200
script.disable_dynamic: true
Node 2
cluster.name: elasticsearch
node.master: false
node.data: true
index.number_of_shards: 5
index.number_of_replicas: 1
path.data: /local00/elasticsearch/
path.work: /local00/el_temp/
network.host: 0.0.0.0
http.port: 9200
script.disable_dynamic: true
I am assuming I am missing additional config somewhere. Any help will be much appreciated.
Got it working with following changes answered here How to set up ES cluster?:
Node 1:
cluster.name: mycluster
node.name: "node1"
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["node1.example.com"]
Node 2:
cluster.name: mycluster
node.name: "node2"
node.master: false
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["node1.example.com"]
If you are trying to connect additional node to already existed ES cluster, make sure that this node also have all the same ES plugins as another nodes. If not - node cant be fully connected (other nodes can show it as connected but http-commands cant be launched on it) to ES cluster with error like:
[2016-07-21 11:56:59,564][DEBUG][action.admin.cluster.health] [dev-marvel1] no known master node, scheduling a retry
[2016-07-21 11:57:05,313][INFO ][rest.suppressed ] /_cluster/health Params: {pretty=true}
MasterNotDiscoveredException[waited for [30s]]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$4.onTimeout(TransportMasterNodeAction.java:154)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:239)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:574)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
In my case problem was with license plugin. After removing it everything became fine.
If you are searching for this because you are working on your local machine then the quickest solution is to free up nodes that are hung by individually killing all the processes. I searched for these processes using.
ps -alx | grep elastic
kill -9 {pid}
How can I have multiple nodes in my ElasticSearch? I'm using the following in elasticsearch.yml but only the last node starts, and the browser complains: The page at file://localhost/ says: undefined.
node.name: "No Data"
node.master: true
node.data: false
node.name: "Data One"
node.master: false
node.data: true
node.name: "Data Two"
node.master: false
node.data: true
I think the simplest way to do it is by specifying these parameters on the command line. To start three nodes you just need to run the following three commands in elasticsearch home directory:
$ bin/elasticsearch -Des.node.data=false -Des.node.master=true -Des.node.name=NoData
$ bin/elasticsearch -Des.node.data=true -Des.node.master=false -Des.node.name=DataOne
$ bin/elasticsearch -Des.node.data=true -Des.node.master=false -Des.node.name=DataTwo
Another solution is to create 3 different config files and start three nodes with -Des.config=path-to-config-file parameter.
First off, you should be trying to access elasticsearch using
[http://localhost:9200/][1], if you are using the default port bindings.
I would set up your master node to also be a data node, there is no reason not to. If you are trying to start 3 nodes on a single machine. But, starting 3 nodes all on the same machine doesn't make sense as anything other than an experiment. What are you trying to accomplish?
In windows for 6.x version, command attributes change to
elasticsearch -EsomeYamlPropety=someValue
First You need change an elasticsearch.yml properties to:
http.port: 9200-9299
transport.tcp.port: 9300-9399
node.max_local_storage_nodes: 2
Because You cant run nodes on single port, and when I try to use command with argument -Ehttp.port=9201 nodes where cant see each other and they create two different clusters with the same name.
Run the first node by a standard command:
.\bin\elasticsearch
Run the second node by command with attributes:
.\bin\elasticsearch -Enode.name=NodeTwo -Enode.master=false
for running 3 elasticsearch node on one machine, you should use these configs in elasticsearch.yml file of each node:
for master node :
cluster.name: mycluster
node.name: "node1"
node.master: true
node.data: true
network.host: 127.0.0.1
http.port: 9200-9299
transport.tcp.port: 9300-9399
discovery.zen.minimum_master_nodes: 2
for data nodes :
cluster.name: mycluster
node.name: "data-node-name"
node.master: false
node.data: true
network.host: 127.0.0.1
http.port: 9200-9299
transport.tcp.port: 9300-9399
discovery.zen.minimum_master_nodes: 2
and then u should run each node by :
cd path/to/elasticsearch/bin
path\bin>elasticsearch.bat