I am deploying EFK stack using elastic repo's helm charts. Elasticsearch pods are running into continuous errors.
**kubectl logs <pod-name> output**
java.lang.IllegalArgumentException: unknown setting [node.ml] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
elasticsearch.yml:
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
**Roles enabled in Values.yaml:**
roles:
master: "true"
ingest: "true"
data: "true"
remote_cluster_client: "true"
ml: "true"
Related
helllo everyone i have elk deployed on k8s cluster using helm charts "7.17.1".
i'm trying to set up security for elasticsearch, i added these lines in the elasticsearch yaml file
esConfig:
elasticsearch.yml: |
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
and i didn't know how to create a certificate since i can't access the pod to create it there.
any solution would be appreciated since i've been stuck for 2 weeks.
I am launching Elasticsearch cluster in K8S and below is the spec file. It failed to launch the pod with below error. I am trying to disable authentication and want to connect to the cluster without any credentials. But it stops me doing that. It says the configuration is internal use. What is the correct way for me to set this settings?
Warning ReconciliationError 84s elasticsearch-controller Failed to apply spec change: adjust resources: adjust discovery config: Operation cannot be fulfilled on elasticsearches.elasticsearch.k8s.elastic.co "datasource": the object has been modified; please apply your changes to the latest version and try again
Normal AssociationStatusChange 1s (x16 over 86s) es-monitoring-association-controller Association status changed from [] to []
Warning Validation 1s (x20 over 84s) elasticsearch-controller [spec.nodeSets[0].config.xpack.security.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[0].config.xpack.security.http.ssl.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported, spec.nodeSets[0].config.xpack.security.transport.ssl.enabled: Forbidden: Configuration setting is reserved for internal use. User-configured use is unsupported]
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: datasource
spec:
version: 7.14.0
nodeSets:
- name: node
count: 2
config:
node.store.allow_mmap: false
xpack.security.http.ssl.enabled: false
xpack.security.transport.ssl.enabled: false
xpack.security.enabled: false
volumeClaimTemplates:
- metadata:
name: elasticsearch-data
spec:
accessModes:
- ReadWriteOnce
storageClassName: ebs-sc
resources:
requests:
storage: 1024Gi
You can try this:
https://discuss.elastic.co/t/cannot-disable-tls-and-security-in-eks/222335/2
I have tested and its working fine for me without any issues:
cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: quickstart
spec:
version: 7.15.0
nodeSets:
- name: default
count: 1
config:
node.master: true
node.data: true
node.ingest: true
node.store.allow_mmap: false
xpack.security.authc:
anonymous:
username: anonymous
roles: superuser
authz_exception: false
EOF
To Disable basic authentication:
https://www.elastic.co/guide/en/elasticsearch/reference/7.14/anonymous-access.html
To disable SSL self signed certificate:
https://www.elastic.co/guide/en/cloud-on-k8s/0.9/k8s-accessing-elastic-services.html#k8s-disable-tls
im new on ELK stack,i am lead to Kibana Alert config but i got stuck at Kibana and Elastic search CA step when follow this link: https://www.elastic.co/guide/en/kibana/7.x/configuring-tls.html#configuring-tls-kib-es
elasticsearch.yml
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: transport_key.p12
xpack.security.transport.ssl.truststore.path: transport_key.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: http.p12
kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["https://localhost:9200"]
elasticsearch.username: "kibana"
elasticsearch.password: "kibana_password"
kibana.index: ".kibana"
xpack.security.encryptionKey: "32 random letters"
csp.strict: true
xpack.encryptedSavedObjects.encryptionKey: "32 random letters"
server.ssl.enabled: true
server.ssl.certificate: "/path/to/kibana-server.crt"
server.ssl.key: "/path/to/kibana-server.key"
elasticsearch.ssl.certificateAuthorities: [ "path/to/config/elasticsearch-ca.pem" ]
when i started kibana by./bin/kibana promt show me:
enter image description here
I run on: ubuntu 18.04.4 LTS, and ELL run install dicrectly. Please tell me what i was wrong.
We just installed Elasticsearch 7.x. We want to use the x-pack security module. We already automated everything via Ansible but we have a problem creating/setting the built in users with password:
ElsticSearch how to:
Run on system: /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive or auto.
Then you are asked for a password for each user in ElasticSearch. Are there any possibilities to automate this? Like some autoanswer question in Ansible or anything else?
Thanks
You can try to use interactive mode and ansible expect module: https://docs.ansible.com/ansible/latest/modules/expect_module.html
hosts: all
name: "Elasticsearch with SSL/TLS enabled"
roles:
-
role: elastic.elasticsearch
vars:
es_api_port: 9200
es_config:
action.destructive_requires_name: true
bootstrap.memory_lock: true
cluster.name: lab
discovery.seed_hosts: "0.0.0.0:9300"
discovery.type: single-node
http.port: 9200
indices.query.bool.max_clause_count: 8192
network.host: "0.0.0.0"
node.data: true
node.master: true
node.ml: false
node.name: lab1
reindex.remote.whitelist: "*:*"
search.max_buckets: 250000
transport.port: 9300
xpack.ilm.enabled: true
xpack.ml.enabled: false
xpack.monitoring.collection.enabled: true
xpack.monitoring.collection.interval: 30s
xpack.monitoring.enabled: true
xpack.security.audit.enabled: false
#xpack.security.enabled: true
xpack.sql.enabled: true
xpack.watcher.enabled: false
es_api_basic_auth_username: "elastic"
es_api_basic_auth_password: "changeme"
es_data_dirs:
- /opt/elasticsearch/data
es_heap_size: 2g
es_plugins:
-
plugin: ingest-attachment
es_validate_certs: false
es_version: "7.17.0"
es_users:
native:
elastic:
password: helloakash1234
kibana_system:
password: hellokibana1234
logstash_system:
password: hellologs1234
This works fine for me!!
es_users:
native:
elastic:
password: helloakash1234
With the above mentioned code the username will be "elastic" and the password will be "helloakash1234"
If you use the auto mode, then random passwords are generated and written to the console that you can maybe read.
Another solution is to call the Change password API in order to change user passwords after the fact.
I'm trying to generate wildcard certificate using the latest traefik helm chart version 1.35.0 this is the acme configurations but I can find in the logs that traefik request a certificate for each sub domain and no wildcard certificate seems to get requested
acme:
enabled: true
staging: false
email: info#mydomain.com
logging: true
challengeType: "dns-01"
dnsProvider:
name: cloudflare
cloudflare:
CLOUDFLARE_EMAIL: "<EMAIL>"
CLOUDFLARE_API_KEY: "<KEY>"
persistence:
enabled: true
domains:
enabled: true
domainList:
- main: "*.mydomain.com"
- sans:
- "mydomain.com"
it should be domainsList instead of domainList