I have installed elasticsearch on kubernetes using the helm chart
. I need to enable xpack security for that I need to create a certificate using elasticsearch-certutil. I can't find it's location on kubernetes.
the error I am getting now
2019-06-25T10:20:56.882057213Z "Caused by:
org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory",
I 2019-06-25T10:20:56.882063036Z "atorg.elasticsearch.xpack.core.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:61) ~[?:?]",
I 2019-06-25T10:20:56.882068596Z "at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:382) ~[?:?]",
I 2019-06-25T10:20:56.882074256Z "at java.util.HashMap.computeIfAbsent(HashMap.java:1133) ~[?:?]",
I 2019-06-25T10:20:56.882079897Z "at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$2(SSLService.java:426) ~[?:?]",
I 2019-06-25T10:20:56.882085280Z "at java.util.HashMap.forEach(HashMap.java:1333) ~[?:?]",
I 2019-06-25T10:20:56.882120138Z "at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:423) ~[?:?]",
I 2019-06-25T10:20:56.882136977Z "at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:119) ~[?:?]",
I 2019-06-25T10:20:56.882143717Z "at org.elasticsearch.xpack.core.XPackPlugin.<init>(XPackPlugin.java:144) ~[?:?]",
I 2019-06-25T10:20:56.882149641Z "at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]",
I 2019-06-25T10:20:56.882155163Z "at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]",
I 2019-06-25T10:20:56.882168785Z "at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]",
I 2019-06-25T10:20:56.882175111Z "at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]",
I 2019-06-25T10:20:56.882181018Z "at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]",
I 2019-06-25T10:20:56.882228253Z "at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882234700Z "at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882240443Z "at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882246040Z "at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882251927Z "at org.elasticsearch.node.Node.<init>(Node.java:308) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882257697Z "at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882263355Z "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882271710Z "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882318705Z "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.1.jar:7.1.1]",
I 2019-06-25T10:20:56.882344091Z "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.1.jar:7.1.1]",
this is my configuration esConfig:
elasticsearch.yml: |
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/ca.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsea/config/certs/ca.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/ca.p12
xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/ca.p12
I used kubectl exec -it elasticsearch-master-0 -- /bin/bash to enter the container and generate certificate in the exact path /usr/share/elasticsearch/certs
Because you are using helm charts you need to set all those things inside your template.
First you should make yourself familiar with https://github.com/helm/charts/tree/master/stable/elasticsearch
Second you will need to check what is wrong with one of your pods elasticsearch-master-2 because it's CrashLoopBackOff.
You can do that using kubectl describe pods elasticsearch-master-2 and check events at the bottom.
As for Xpack and Enabling elasticsearch interal monitoring:
Requires version 6.3+ and standard non oss repository defined. Starting with 6.3 Xpack is partially free and enabled by default. You need to set a new config to enable the collection of these internal metrics. (https://www.elastic.co/guide/en/elasticsearch/reference/6.3/monitoring-settings.html)
To do this through this helm chart override with the three following changes:
image.repository: docker.elastic.co/elasticsearch/elasticsearch
cluster.xpackEnable: true
cluster.env.XPACK_MONITORING_ENABLED: true
Note: to see these changes you will need to update your kibana repo to image.repository: docker.elastic.co/kibana/kibana instead of the oss version
You should also read Get a Shell to a Running Container, which explains you can connect to a pod.
In Your example using kubectl exec -it elasticsearch-master-0 -- /bin/bash
Related
I've had a 3 node setup in GKE. And one of my pod creation is in CrashLoopBackOff state and it is not recovering. The log suggests the below java.lang.IllegalArgumentException. But the other 2 pods they have no such issue. They are up and running. I'm completely unsure of the issue, can someone help me?
Is the issue, a by-product of install-plugins in the YML file?
If yes, why didn't the same problem occur with other pods? Can you please help me with it?
Exception:
"type": "server", "timestamp": "2022-08-29T19:52:29,743Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "dev", "node.name": "dev-es-data-hot-1", "message": "uncaught exception in thread [main]",
"stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown secure setting [dev-es-snapshot-backup-feeb83405c27.json] please check that any required plugins are installed, or check the breaking changes documentation for removed settings",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) ~[elasticsearch-cli-7.16.3.jar:7.16.3]",
"at org.elasticsearch.cli.Command.main(Command.java:77) ~[elasticsearch-cli-7.16.3.jar:7.16.3]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-7.16.3.jar:7.16.3]",
"Caused by: java.lang.IllegalArgumentException: unknown secure setting [dev-es-snapshot-backup-feeb83405c27.json] please check that any required plugins are installed, or check the breaking changes documentation for removed settings",
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:561) ~[elasticsearch-7.16.3.jar:7.16.3]",
uncaught exception in thread [main]
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:507) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:477) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:447) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:137) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.node.Node.<init>(Node.java:500) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.3.jar:7.16.3]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.3.jar:7.16.3]",
"... 6 more"] }
Here is my YAML config:
- name: data-hot-ingest
count: 3
config:
node.roles: ["data_hot", "ingest", "data_content"]
node.attr.data: hot
node.store.allow_mmap: false
xpack.security.authc:
anonymous:
username: anon
roles: monitoring_user
podTemplate:
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: In
values:
- hot
initContainers:
- name: install-plugins
command:
- sh
- -c
- |
bin/elasticsearch-plugin install --batch repository-gcs
- name: set-virtual-mem
command:
- sysctl
- -w
- vm.max_map_count=262144
containers:
- name: elasticsearch
resources:
requests:
memory: "64Gi"
cpu: "30000m"
limits:
memory: "65Gi"
cpu: "30000m"
env:
- name: ES_JAVA_OPTS
value: -Xms32g -Xmx32g
readinessProbe:
httpGet:
scheme: HTTPS
port: 8080
volumeClaimTemplates:
- metadata:
name: elasticsearch-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 350Gi
storageClassName: gold
EDIT:
We have this secure setting configured, which is linked to a secret in our
secureSettings:
- secretName: credentials
[ANSWERING MY OWN QUESTION]
Trying to resolve the below exception:
java.lang.IllegalArgumentException: unknown secure setting [dev-es-snapshot-backup-feeb83405c27.json]
I tried comparing the yaml config of the pods, and I found that the pods running successfully do not have a secure setting. But the pod that was crash looping, had the secure setting under elastic-internal-secure-settings
- name: elastic-internal-secure-settings
secret:
defaultMode: 420
optional: false
secretName: dev-es-secure-settings
And in the operator yaml, I found this:
secureSettings:
- secretName: credentials
Just to confirm the behaviour, I upscaled the statefulset, and found the new pod also crash looping with the same error. So someone had tried the secure setting last month, and it crash looped the pod, and didn't reset it back to normal. Once I removed the secure-setting from the operator yaml, the pods started running without any issue.
I am trying to start elastic search with private ip address but it does not get started it shows some errors in error log which i have shared below.
elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#Also Tried with Private IP Address network.host: 52.50.122.93
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["52.50.122.93", "127.0.0.1", "[::1]"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
elasticsearch.log
[2019-05-21T17:22:28,068][ERROR][o.e.b.Bootstrap ] [WIN-CQKBIA6F350] Exception
java.lang.IllegalStateException: failed to obtain node locks, tried [[C:\ELKStack\elasticsearch-7.1.0\data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:297) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.node.Node.<init>(Node.java:272) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) [elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-7.1.0.jar:7.1.0]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) [elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.1.0.jar:7.1.0]
[2019-05-21T17:22:28,085][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [WIN-CQKBIA6F350] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[C:\ELKStack\elasticsearch-7.1.0\data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[C:\ELKStack\elasticsearch-7.1.0\data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:297) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.node.Node.<init>(Node.java:272) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]
... 6 more
you need to set one of these values
[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
ex discovery.seed_hosts:
- 192.168.1.10:9300
- 192.168.1.11
- seeds.mydomain.com
The error is clear in the log file
the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
You have to set the cluster.initial_master_node or discovery.seed_hosts setting
Also don't forget to set the node.name and the cluster.name, you can also start ES and set the master node with this command-line:
bin/elasticsearch -Ecluster.initial_master_nodes=master-a,master-b,master-c
https://www.elastic.co/guide/en/elasticsearch/reference/master/important-settings.html
https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-discovery-bootstrap-cluster.html
https://www.elastic.co/guide/en/elasticsearch/reference/master/discovery-settings.html
I really wana start elasticsearch for connect opencart.
I have set network.host: 0.0.0.0 (also try to my local IP), http.port: 9200, and give executive authority to my new account. I use centOs6, Java 1.8, and don't use root account.
I want to use elasticsearch on my server, but I have some issue.
my error log is that
[2019-03-21T18:12:27,970][INFO ][o.e.p.PluginsService ] [gxhuK_x] loaded module [x-pack-watcher]
[2019-03-21T18:12:27,971][INFO ][o.e.p.PluginsService ] [gxhuK_x] no plugins loaded
[2019-03-21T18:12:32,061][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [gxhuK_x] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/etc/system-release];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.2.jar:6.6.2]
Caused by: org.elasticsearch.ElasticsearchException: failed to bind service
at org.elasticsearch.node.Node.<init>(Node.java:626) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /etc/system-release
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:?]
at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_131]
at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_131]
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0_131]
at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_131]
at java.nio.file.Files.newBufferedReader(Files.java:2784) ~[?:1.8.0_131]
at java.nio.file.Files.readAllLines(Files.java:3202) ~[?:1.8.0_131]
at java.nio.file.Files.readAllLines(Files.java:3242) ~[?:1.8.0_131]
at org.elasticsearch.monitor.os.OsProbe.readOsRelease(OsProbe.java:591) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.monitor.os.OsProbe.getPrettyName(OsProbe.java:546) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.monitor.os.OsProbe.osInfo(OsProbe.java:533) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.monitor.os.OsService.<init>(OsService.java:48) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.monitor.MonitorService.<init>(MonitorService.java:47) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.node.Node.<init>(Node.java:397) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.node.Node.<init>(Node.java:265) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.2.jar:6.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.2.jar:6.6.2]
... 6 more
I have installed by tar.gz.
and I changed something of config/elasticsearch.yml
I didn't change anything else
someone know this error? please help me.
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/data/elasticsearch
#
# Path to log files:
#
path.logs: /var/logs/elasticsearch
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
bootstrap.system_call_filter: false
You need to make sure that the directory on the host machine is owned by 1000:1000. Run the following command.
sudo chown 1000:1000 <directory you wish to mount>
Source
Please check the following:
Did you create /var/data/elasticsearch and /var/logs/elasticsearch directories in your filesystem?
Did you set the directories above to be writable with your user permission?
Update:
Read the source code of elasticsearch in Github. Link Here
It seemed that there some problem when the code is trying to access /etc/system-release. You might to take a look at that file. Also It mentioned that it is fallback for older Red Hat-like OS. So you might want to use another OS if possible.
In my variant trouble was in a permissions of data directory. Data directory was not default and changed to /elk-data/
To resolve the issue I make 'chmod -R elasticsearch:wheel /elk-data'
Trying to configure GCE Discovery.
Have this in elasticsearch.yml
cloud:
gce:
project_id: ..........
zone: us-west1-a
discovery:
type: gce
Look at the log messages:
loaded plugin [discovery-gce]
but also failes with Unknown discovery type [gce]
.
[2018-01-16T09:11:25,247][INFO ][o.e.p.PluginsService ] [elasticsearch-cluster-2-node-2] loaded plugin [discovery-gce]
[2018-01-16T09:11:27,019][ERROR][o.e.b.Bootstrap ] Exception
java.lang.IllegalArgumentException: Unknown discovery type [gce]
at org.elasticsearch.discovery.DiscoveryModule.<init>(DiscoveryModule.java:110) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.node.Node.<init>(Node.java:441) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.node.Node.<init>(Node.java:245) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:322) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:130) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:121) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:69) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.0.0.jar:6.0.0]
What the heck is going on here? Any recommendation?
Apparently the EL maintaining team did a BWC change
In 5.1 we introduced the discovery.zen.hosts_provider setting to better reflect it. This PR removes BWC code in those plugins (GCE included) as it is not needed anymore.
- .put("discovery.type", "gce")
+ .put("discovery.zen.hosts_provider", "gce")
so the correct elasticsearch.yml configuration should reference
discovery:
zen.hosts_provider: gce
I'm trying to compose the ELK architecture by using docker compose. The following is the compose file:
version: '2'
services:
elasticsearch_assets:
image: elasticsearch
volumes:
- ./elasticsearch/config:/usr/share/elasticsearch/config
- ./elasticsearch/data:/usr/share/elasticsearch/data
- ./elasticsearch/logs:/usr/share/elasticsearch/logs
command: /bin/true
elasticsearch:
image: elasticsearch
volumes_from:
- elasticsearch_assets:rw
depends_on:
- elasticsearch_assets
And here folliwing a screenshot of the project structure:
When I run docker-compose up I get the following error:
Starting elkdocker_elasticsearch_assets_1
Starting elkdocker_elasticsearch_1
Attaching to elkdocker_elasticsearch_assets_1, elkdocker_elasticsearch_1
elkdocker_elasticsearch_assets_1 exited with code 0
elasticsearch_1 | [2016-03-22 01:28:59,939][WARN ][bootstrap ] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade
elasticsearch_1 | Exception in thread "main" java.lang.IllegalStateException: Unable to access 'path.scripts' (/usr/share/elasticsearch/config/scripts)
elasticsearch_1 | Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/config/scripts
elasticsearch_1 | at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
elasticsearch_1 | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
elasticsearch_1 | at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
elasticsearch_1 | at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
elasticsearch_1 | at java.nio.file.Files.createDirectory(Files.java:674)
elasticsearch_1 | at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
elasticsearch_1 | at java.nio.file.Files.createDirectories(Files.java:767)
elasticsearch_1 | at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:337)
elasticsearch_1 | at org.elasticsearch.bootstrap.Security.addPath(Security.java:314)
elasticsearch_1 | at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:248)
elasticsearch_1 | at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:212)
elasticsearch_1 | at org.elasticsearch.bootstrap.Security.configure(Security.java:118)
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.setupSecurity(Bootstrap.java:196)
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:167)
elasticsearch_1 | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:285)
elasticsearch_1 | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
elasticsearch_1 | Refer to the log for complete error details.
elkdocker_elasticsearch_1 exited with code 1
Do you have any idea why?
regarding this error
Unable to access 'path.scripts' error
just create a sub folder called scripts under the config folder
it will fix the error
./elasticsearch/config ==> mkdir ./elasticsearch/config/scripts
Change the docker compose to something like this
-v /elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
instead of pointing to empty directory