ELK cluster user password - elasticsearch

Question

Firstly I create a single node ELK and I use this conig in my elasticsearch.yml
# sed '/^#/d' /etc/elasticsearch/elasticsearch.yml
node.name: "elk01"
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
xpack.security.enabled: true
discovery.type: single-node
after I use this command and create auto built-in users
sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
and it's ok. Everything is working but I want elk-cluster. I create a new server and change config
elk01
# sed '/^#/d' /etc/elasticsearch/elasticsearch.yml
cluster.name: "elk-testcluster"
node.name: "elk01"
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["10.60.201.31", "10.60.201.32"]
cluster.initial_master_nodes: ["10.60.201.31"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
elk02
# sed '/^#/d' /etc/elasticsearch/elasticsearch.yml
cluster.name: "elk-testcluster"
node.name: "elk02"
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
discovery.seed_hosts: ["10.60.201.31", "10.60.201.32"]
cluster.initial_master_nodes: ["10.60.201.31"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
Right now when I use curl with username/password, I can getting a elk01 but not elk02
# curl -XGET "10.60.201.31:9200" -u elastic:passcreatedonelk01
{
"name" : "elk01",
"cluster_name" : "elk-testcluster",
"cluster_uuid" : "7513Zor7S3SHqVFzs0hEMQ",
"version" : {
"number" : "7.17.4",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "79878662c54c886ae89206c685d9f1051a9d6411",
"build_date" : "2022-05-18T18:04:20.964345128Z",
"build_snapshot" : false,
"lucene_version" : "8.11.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
# curl -XGET "10.60.201.32:9200" -u elastic:passcreatedonelk01
{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
When I create a new elasticsearch-setup-password on elk02, it's getting error.
sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
Failed to determine the health of the cluster running at http://10.60.201.32:9200
Unexpected response code [503] from calling GET http://10.60.201.32:9200/_cluster/health?pretty
Cause: master_not_discovered_exception
It is recommended that you resolve the issues with your cluster before running elasticsearch-setup-passwords.
It is very likely that the password changes will fail when run against an unhealthy cluster.
Do you want to continue with the password setup process [y/N]y
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
Unexpected response code [503] from calling PUT http://10.60.201.32:9200/_security/user/apm_system/_password?pretty
Cause: Cluster state has not been recovered yet, cannot write to the [null] index
Possible next steps:
* Try running this tool again.
* Try running with the --verbose parameter for additional messages.
* Check the elasticsearch logs for additional error details.
* Use the change password API manually.
ERROR: Failed to set password for user [apm_system].
When I make a cluster, is the use of a common password not provided? Or is it because I run an elasticsearch-setup-password before doing a cluster?

Answer 1

Once you enable ssl you need to add a certificate and key for each node for transport layer.
you can follow these instructions
https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html