Basic authentication in ELK Stack - elasticsearch

Question

I am trying to enable basic authentication in our ELK Stack. The below steps have tried to enable the authentication
1. docker run -d -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.1
2. docker exec -it stoic_darwin /bin/bash
3. In side the container executed # bin/elasticsearch-certutil ca
4. No password entered and exited from the container.
5. Copied the generated file to host system - docker cp stoic_darwin:/usr/share/elasticsearch/elastic-stack-ca.p12 .
6. Updated the docker-compose file as below.
version: '3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.10.1
container_name: elasticsearch
environment:
- node.name=elasticsearch
- discovery.seed_hosts=elasticsearch
- cluster.initial_master_nodes=elasticsearch
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.security.enabled=true
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.keystore.type=PKCS12
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.keystore.path=elastic-stack-ca.p12
- xpack.security.transport.ssl.truststore.path=elastic-stack-ca.p12
- xpack.security.transport.ssl.truststore.type=PKCS12
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./elastic-stack-ca.p12:/usr/share/elasticsearch/config/elastic-stack-ca.p12
- esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
kibana:
image: docker.elastic.co/kibana/kibana:7.10.1
container_name: kibana
environment:
ELASTICSEARCH_URL: "http://elasticsearch:9200"
ELASTICSEARCH_USERNAME: "kibana"
ELASTICSEARCH_PASSWORD: "kibana"
ports:
- 5601:5601
depends_on:
- elasticsearch
volumes:
esdata1:
driver: local
7. # docker-compose up -d elasticsearch
8. But it fails with below errors.
elasticsearch | "at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:524) ~[?:?]",
elasticsearch | ElasticsearchSecurityException[failed to load SSL configuration [xpack.security.transport.ssl]]; nested: ElasticsearchException[failed to initialize SSL TrustManager - not permitted to read truststore file [/usr/share/elasticsearch/config/elastic-stack-ca.p12]]; nested: AccessDeniedException[/usr/share/elasticsearch/config/elastic-stack-ca.p12];
elasticsearch | Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/config/elastic-stack-ca.p12
elasticsearch | at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
elasticsearch | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
elasticsearch | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
elasticsearch | at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
elasticsearch | "at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:142) ~[?:?]",
elasticsearch | "at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:455) ~[?:?]",
elasticsearch | "at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:288) ~[?:?]",
elasticsearch | "at org.elasticsearch.node.Node.lambda$new$15(Node.java:553) ~[elasticsearch-7.10.1.jar:7.10.1]",
elasticsearch | "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]",
elasticsearch | "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]",
elasticsearch | "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]",
elasticsearch | at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
I believe to enable the basic authendiation with ELK requires the SSL certificate to connect with single/multiple clusters. So how can i resolve this error?
And also is there way to generate the cerfiticatePerformed at step-3 and to setup the build-in users password using bin/elasticsearch-setup-passwords interactive command in docker-compose itself?
(Or) if there any simple way to enable authentication through docker compose ll be helpful. Pls help me with the steps. thanks in advance.
Above error was due to permission have modified the permission and fixed it. Now my logstash contentiously throwing below erorrs.
logstash | [2021-01-27T06:27:42,365][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/_xpack'"}
logstash | [2021-01-27T06:27:42,738][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
logstash | [2021-01-27T06:28:12,358][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/_xpack'"}
logstash | [2021-01-27T06:28:12,753][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
logstash | [2021-01-27T06:28:42,366][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/_xpack'"}
logstash | [2021-01-27T06:28:42,766][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
In my logstash conf file output section already have provided authentication credentials.
output {
elasticsearch {
action => "index"
hosts => "http://elasticsearch:9200"
index => "project-info"
user => "elastic"
password => "password"