elasticsearch-head can't connection to elasticsearch with http://localhost:9200 - elasticsearch

elasticsearch version : elasticsearch-5.2.0
elasticsearch-head version : latest version
Start service successfully:
#### node1 ####
deMacBook-Pro:bin yyq$ ./elasticsearch
[2017-02-15T13:08:57,174][INFO ][o.e.n.Node ] [yml-es-node-1] initializing ...
[2017-02-15T13:08:57,303][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-1] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [21.4gb], net total_space [232.6gb], spins? [unknown], types [hfs]
[2017-02-15T13:08:57,304][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-1] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-02-15T13:08:57,355][INFO ][o.e.n.Node ] [yml-es-node-1] node name [yml-es-node-1], node ID [hqVAcDOdRu-LXuAgUEX9oQ]
[2017-02-15T13:08:57,358][INFO ][o.e.n.Node ] [yml-es-node-1] version[5.2.0], pid[93922], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Mac OS X/10.11.6/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_25/25.25-b02]
[2017-02-15T13:08:58,802][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [aggs-matrix-stats]
[2017-02-15T13:08:58,803][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [ingest-common]
[2017-02-15T13:08:58,803][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-expression]
[2017-02-15T13:08:58,803][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-groovy]
[2017-02-15T13:08:58,803][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-mustache]
[2017-02-15T13:08:58,803][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-painless]
[2017-02-15T13:08:58,803][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [percolator]
[2017-02-15T13:08:58,804][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [reindex]
[2017-02-15T13:08:58,804][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [transport-netty3]
[2017-02-15T13:08:58,804][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [transport-netty4]
[2017-02-15T13:08:58,805][INFO ][o.e.p.PluginsService ] [yml-es-node-1] no plugins loaded
[2017-02-15T13:09:02,610][INFO ][o.e.n.Node ] [yml-es-node-1] initialized
[2017-02-15T13:09:02,610][INFO ][o.e.n.Node ] [yml-es-node-1] starting ...
[2017-02-15T13:09:02,953][INFO ][o.e.t.TransportService ] [yml-es-node-1] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}
[2017-02-15T13:09:06,125][INFO ][o.e.c.s.ClusterService ] [yml-es-node-1] detected_master {yml-es-node-2}{FWhiCkI7TV6hJOlmT_fgQg}{URIaxsK2T0aP7jRRe6TDfg}{127.0.0.1}{127.0.0.1:9301}, added {{yml-es-node-2}{FWhiCkI7TV6hJOlmT_fgQg}{URIaxsK2T0aP7jRRe6TDfg}{127.0.0.1}{127.0.0.1:9301},}, reason: zen-disco-receive(from master [master {yml-es-node-2}{FWhiCkI7TV6hJOlmT_fgQg}{URIaxsK2T0aP7jRRe6TDfg}{127.0.0.1}{127.0.0.1:9301} committed version [385]])
[2017-02-15T13:09:06,175][INFO ][o.e.h.HttpServer ] [yml-es-node-1] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}
[2017-02-15T13:09:06,175][INFO ][o.e.n.Node ] [yml-es-node-1] started
#### node2 ####
deMacBook-Pro:bin yyq$ ./elasticsearch
[2017-02-15T10:31:03,936][INFO ][o.e.n.Node ] [yml-es-node-2] initializing ...
[2017-02-15T10:31:04,087][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-2] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [19.3gb], net total_space [232.6gb], spins? [unknown], types [hfs]
[2017-02-15T10:31:04,087][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-2] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-02-15T10:31:04,104][INFO ][o.e.n.Node ] [yml-es-node-2] node name [yml-es-node-2], node ID [FWhiCkI7TV6hJOlmT_fgQg]
[2017-02-15T10:31:04,112][INFO ][o.e.n.Node ] [yml-es-node-2] version[5.2.0], pid[85158], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Mac OS X/10.11.6/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_25/25.25-b02]
[2017-02-15T10:31:05,830][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [aggs-matrix-stats]
[2017-02-15T10:31:05,830][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [ingest-common]
[2017-02-15T10:31:05,830][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [lang-expression]
[2017-02-15T10:31:05,830][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [lang-groovy]
[2017-02-15T10:31:05,830][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [lang-mustache]
[2017-02-15T10:31:05,830][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [lang-painless]
[2017-02-15T10:31:05,831][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [percolator]
[2017-02-15T10:31:05,831][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [reindex]
[2017-02-15T10:31:05,831][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [transport-netty3]
[2017-02-15T10:31:05,831][INFO ][o.e.p.PluginsService ] [yml-es-node-2] loaded module [transport-netty4]
[2017-02-15T10:31:05,832][INFO ][o.e.p.PluginsService ] [yml-es-node-2] no plugins loaded
[2017-02-15T10:31:09,871][INFO ][o.e.n.Node ] [yml-es-node-2] initialized
[2017-02-15T10:31:09,871][INFO ][o.e.n.Node ] [yml-es-node-2] starting ...
[2017-02-15T10:31:10,171][INFO ][o.e.t.TransportService ] [yml-es-node-2] publish_address {127.0.0.1:9301}, bound_addresses {[fe80::1]:9301}, {[::1]:9301}, {127.0.0.1:9301}
[2017-02-15T10:31:13,327][INFO ][o.e.c.s.ClusterService ] [yml-es-node-2] detected_master {yml-es-node-1}{hqVAcDOdRu-LXuAgUEX9oQ}{VrY6MUvhQNetUmbzcRnKMg}{127.0.0.1}{127.0.0.1:9300}, added {{yml-es-node-1}{hqVAcDOdRu-LXuAgUEX9oQ}{VrY6MUvhQNetUmbzcRnKMg}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-receive(from master [master {yml-es-node-1}{hqVAcDOdRu-LXuAgUEX9oQ}{VrY6MUvhQNetUmbzcRnKMg}{127.0.0.1}{127.0.0.1:9300} committed version [223]])
[2017-02-15T10:31:13,388][INFO ][o.e.h.HttpServer ] [yml-es-node-2] publish_address {127.0.0.1:9201}, bound_addresses {[fe80::1]:9201}, {[::1]:9201}, {127.0.0.1:9201}
[2017-02-15T10:31:13,388][INFO ][o.e.n.Node ] [yml-es-node-2] started
deMacBook-Pro:elasticsearch-head-master yyq$ grunt server
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on http://localhost:9100
When I opened the url:http://localhost:9100 and clicked [connect] button, but I'm unable to connect to elasticsearch.
Could anyone help me?
This is the screenshot

In your elasticsearch.yml configuration, you probably have the following setting:
http.port: 9201
This is also visible in your logs:
[...] publish_address {127.0.0.1:9201}, bound_addresses {[fe80::1]:9201}, {[::1]:9201}, {127.0.0.1:9201}
^
|
your HTTP port
So in your elasticsearch-head interface, you need to connect to localhost:9201 not localhost:9200.
UPDATE
You also need to add http.cors.enabled: true in your elasticsearch.yml configuration and restart your ES server

As mentioned in the previous post, to be able to run the head plugin to check the health status of the ElastiSearch cluster, you need to enable the CORS's environmental variables. One way is to mount the config file for ES as described here. Alternatively, you can define the environmental variables directly in your docker-compose as documented in this GitHub issue. I am adding the code below for completeness, which worked for me:
version: '3'
services:
# search engine
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.0.1
container_name: elasticsearch
environment:
- discovery.type=single-node
- http.port=9200
- http.cors.enabled=true
- http.cors.allow-origin=http://localhost:1358,http://127.0.0.1:1358
- http.cors.allow-headers=X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
- http.cors.allow-credentials=true
- bootstrap.memory_lock=true
- 'ES_JAVA_OPTS=-Xms512m -Xmx512m'
ports:
- '9200:9200'
- '9300:9300'
# elasticsearch browser
dejavu:
image: appbaseio/dejavu:3.2.3
container_name: dejavu
ports:
- '1358:1358'
links:
- elasticsearch
Note that using - http.cors.allow-origin=* works and it could be handy for certain development setups.

Related

Issue with limits with elasticsearch

I have installed elasticsearch on my server having this ip address W.X.Y.Z
I am able to run
./elasticsearch-6.3.2/bin/elasticsearch
The elasticsearch service is accessible on localhost (remote server) but I can't access it from my locale machine with the browser at this address http://W.X.Y.Z:9200/
Then I tried to change the elasticsearch-6.3.2/config/elasticsearch.yml file by adding the following lines (please see the attached screenshot)
network.host: 0.0.0.0
http.port: 9200
And at this time, I'm not able to run
./elasticsearch-6.3.2/bin/elasticsearch
I have updated /etc/security/limits.conf file by adding
elasticsearch - nofile 65536
elasticsearch - nproc 4096
this command ulimit -a still showing me this screen
And always, I'm not able to run ./elasticsearch command, I'm getting this result
[2018-11-22T09:50:06,714][INFO ][o.e.n.Node ] [] initializing ...
[2018-11-22T09:50:06,866][INFO ][o.e.e.NodeEnvironment ] [b1qDVeW] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [14.2gb], net total_space [17.2gb], types [rootfs]
[2018-11-22T09:50:06,871][INFO ][o.e.e.NodeEnvironment ] [b1qDVeW] heap size [1015.6mb], compressed ordinary object pointers [true]
[2018-11-22T09:50:06,873][INFO ][o.e.n.Node ] [b1qDVeW] node name derived from node ID [b1qDVeW5RRWfC58mHtWaKQ]; set [node.name] to override
[2018-11-22T09:50:06,873][INFO ][o.e.n.Node ] [b1qDVeW] version[6.3.2], pid[15624], build[default/tar/053779d/2018-07-20T05:20:23.451332Z], OS[Linux/3.16.0-5-amd64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_191/25.191-b12]
[2018-11-22T09:50:06,875][INFO ][o.e.n.Node ] [b1qDVeW] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.Z9qppca6, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/developer/elasticsearch-6.3.2, -Des.path.conf=/home/developer/elasticsearch-6.3.2/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2018-11-22T09:50:12,367][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [aggs-matrix-stats]
[2018-11-22T09:50:12,373][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [analysis-common]
[2018-11-22T09:50:12,373][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [ingest-common]
[2018-11-22T09:50:12,374][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [lang-expression]
[2018-11-22T09:50:12,374][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [lang-mustache]
[2018-11-22T09:50:12,374][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [lang-painless]
[2018-11-22T09:50:12,375][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [mapper-extras]
[2018-11-22T09:50:12,375][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [parent-join]
[2018-11-22T09:50:12,375][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [percolator]
[2018-11-22T09:50:12,376][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [rank-eval]
[2018-11-22T09:50:12,376][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [reindex]
[2018-11-22T09:50:12,376][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [repository-url]
[2018-11-22T09:50:12,376][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [transport-netty4]
[2018-11-22T09:50:12,376][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [tribe]
[2018-11-22T09:50:12,377][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-core]
[2018-11-22T09:50:12,377][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-deprecation]
[2018-11-22T09:50:12,377][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-graph]
[2018-11-22T09:50:12,378][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-logstash]
[2018-11-22T09:50:12,378][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-ml]
[2018-11-22T09:50:12,378][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-monitoring]
[2018-11-22T09:50:12,378][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-rollup]
[2018-11-22T09:50:12,379][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-security]
[2018-11-22T09:50:12,379][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-sql]
[2018-11-22T09:50:12,379][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-upgrade]
[2018-11-22T09:50:12,380][INFO ][o.e.p.PluginsService ] [b1qDVeW] loaded module [x-pack-watcher]
[2018-11-22T09:50:12,380][INFO ][o.e.p.PluginsService ] [b1qDVeW] no plugins loaded
[2018-11-22T09:50:21,545][INFO ][o.e.x.s.a.s.FileRolesStore] [b1qDVeW] parsed [0] roles from file [/home/developer/elasticsearch-6.3.2/config/roles.yml]
[2018-11-22T09:50:22,679][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/15672] [Main.cc#109] controller (64 bit): Version 6.3.2 (Build 903094f295d249) Copyright (c) 2018 Elasticsearch BV
[2018-11-22T09:50:23,497][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-11-22T09:50:24,036][INFO ][o.e.d.DiscoveryModule ] [b1qDVeW] using discovery type [zen]
[2018-11-22T09:50:25,981][INFO ][o.e.n.Node ] [b1qDVeW] initialized
[2018-11-22T09:50:25,982][INFO ][o.e.n.Node ] [b1qDVeW] starting ...
[2018-11-22T09:50:26,351][INFO ][o.e.t.TransportService ] [b1qDVeW] publish_address {94.177.240.139:9300}, bound_addresses {[::]:9300}
[2018-11-22T09:50:26,407][INFO ][o.e.b.BootstrapChecks ] [b1qDVeW] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: max number of threads [3903] for user [developer] is too low, increase to at least [4096]
[2018-11-22T09:50:26,445][INFO ][o.e.n.Node ] [b1qDVeW] stopping ...
[2018-11-22T09:50:26,483][INFO ][o.e.n.Node ] [b1qDVeW] stopped
[2018-11-22T09:50:26,484][INFO ][o.e.n.Node ] [b1qDVeW] closing ...
[2018-11-22T09:50:26,511][INFO ][o.e.n.Node ] [b1qDVeW] closed
[2018-11-22T09:50:26,521][INFO ][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new native processes can be started
Your problem is not related to network address, but to limits.
Have a look at what returns the linux shell command :
ulimit -a
Look at values max user processes and virtual memory, there are too low for running Elasticsearch. Temporary raise them with the ulimit command :
> ulimit -u 4096
> ulimit -v 262144
or permanently modify them like this :
sysctl -w vm.max_map_count=262144
(as explained here : https://www.elastic.co/guide/en/elasticsearch/reference/master/vm-max-map-count.html)
and edit /etc/security/limits.conf
elasticsearch - nofile 65536
elasticsearch - nproc 4096
(as explained here : https://www.elastic.co/guide/en/elasticsearch/reference/master/setting-system-settings.html)

how to change kernel parameters of a pod in Openshift?

Trying to run Elastic Search 6.2.4 on Openshift but it is not running and the container exits with the code 137.
[2018-06-01T14:24:58,148][INFO ][o.e.p.PluginsService ] [jge060C]
loaded module [ingest-common]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [lang-expression]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [lang-mustache]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [lang-painless]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [mapper-extras]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [parent-join]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [percolator]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [rank-eval]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [reindex]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [repository-url]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [transport-netty4]
[2018-06-01T14:24:58,149][INFO ][o.e.p.PluginsService ] [jge060C] loaded module [tribe]
[2018-06-01T14:24:58,150][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [ingest-geoip]
[2018-06-01T14:24:58,150][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [ingest-user-agent]
[2018-06-01T14:24:58,150][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-core]
[2018-06-01T14:24:58,150][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-deprecation]
[2018-06-01T14:24:58,151][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-graph]
[2018-06-01T14:24:58,151][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-logstash]
[2018-06-01T14:24:58,151][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-ml]
[2018-06-01T14:24:58,151][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-monitoring]
[2018-06-01T14:24:58,151][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-security]
[2018-06-01T14:24:58,151][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-upgrade]
[2018-06-01T14:24:58,151][INFO ][o.e.p.PluginsService ] [jge060C] loaded plugin [x-pack-watcher]
[2018-06-01T14:25:01,592][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/131] [Main.cc#128] controller (64 bit): Version 6.2.4 (Build 524e7fe231abc1) Copyright (c) 2018 Elasticsearch BV
[2018-06-01T14:25:03,271][INFO ][o.e.d.DiscoveryModule ] [jge060C] using discovery type [zen]
[2018-06-01T14:25:04,305][INFO ][o.e.n.Node ] initialized
[2018-06-01T14:25:04,305][INFO ][o.e.n.Node ] [jge060C] starting ...
[2018-06-01T14:25:04,497][INFO ][o.e.t.TransportService ] [jge060C] publish_address {10.131.3.134:9300}, bound_addresses {[::]:9300}
[2018-06-01T14:25:04,520][INFO ][o.e.b.BootstrapChecks ] [jge060C] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2018-06-01T14:25:04,531][INFO ][o.e.n.Node ] [jge060C] stopping ...
[2018-06-01T14:25:04,623][INFO ][o.e.n.Node ] [jge060C] stopped
[2018-06-01T14:25:04,624][INFO ][o.e.n.Node ] [jge060C] closing ...
[2018-06-01T14:25:04,634][INFO ][o.e.n.Node ] [jge060C] closed
As you can see from the logs, the vm max heap size has to be increased. As it turns out to be a kernel parameter, how to change that for the pod that is running ES?
Kernel command line parameters can't be changed per pod, but vm.max_map_count is parameter you can change via sysctl.
See these two similar SO question for a solution:
How to pass `sysctl` flags to docker from k8s?
Enabling net.ipv4.ip_forward for a container
There is also a more general explanation in the official Kubernetes documentation on sysctl
You cannot change kernel parameters for individual pods; they have to be fixed in the underlying operating system.

Sebp/elk not running on docker with error

hello all engineering on the world
Im 1 problem to after install ELK stack on docker
my infrastructure:
1- Windows 10 New Creators Update
2- my laptop model: N552VX (8Gig RAM)
3- docker toolbox without prerequisites = sample java machine and more ...
and install sebp/elk on terminal docker with command : $ docker pull sebp/elk
installation successfully and not running --- > An error was pointed out that a number of errors
SEE FULL ERROR == http://textuploader.com/d99r1
Plz check error !
bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-05-12T08:47:16,687][INFO ][o.e.n.Node ] [Zn8_vUw] stopping ...
[2017-05-12T08:47:16,721][INFO ][o.e.n.Node ] [Zn8_vUw] stopped
[2017-05-12T08:47:16,721][INFO ][o.e.n.Node ] [Zn8_vUw] closing ...
[2017-05-12T08:47:16,749][INFO ][o.e.n.Node ] [Zn8_vUw] closed
[2017-05-12T09:36:39,466][INFO ][o.e.n.Node ] [] initializing ...
[2017-05-12T09:36:39,746][INFO ][o.e.e.NodeEnvironment ] [Zn8_vUw] using [1] data paths, mounts [[/var/lib/elasticsearch (c/Users)]], net usable_space [87.9gb], net total_space [118.6gb], spins? [possibly], types [vboxsf]
[2017-05-12T09:36:39,750][INFO ][o.e.e.NodeEnvironment ] [Zn8_vUw] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-05-12T09:36:39,760][INFO ][o.e.n.Node ] node name [Zn8_vUw] derived from node ID [Zn8_vUwXTyGl4E35KOKo5Q]; set [node.name] to override
[2017-05-12T09:36:39,761][INFO ][o.e.n.Node ] version[5.4.0], pid[71], build[780f8c4/2017-04-28T17:43:27.229Z], OS[Linux/4.4.66-boot2docker/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_121/25.121-b13]
[2017-05-12T09:36:42,884][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [aggs-matrix-stats]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [ingest-common]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [lang-expression]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [lang-groovy]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [lang-mustache]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [lang-painless]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [percolator]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [reindex]
[2017-05-12T09:36:42,885][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [transport-netty3]
[2017-05-12T09:36:42,886][INFO ][o.e.p.PluginsService ] [Zn8_vUw] loaded module [transport-netty4]
[2017-05-12T09:36:42,887][INFO ][o.e.p.PluginsService ] [Zn8_vUw] no plugins loaded
[2017-05-12T09:36:50,337][INFO ][o.e.d.DiscoveryModule ] [Zn8_vUw] using discovery type [zen]
[2017-05-12T09:36:52,802][INFO ][o.e.n.Node ] initialized
[2017-05-12T09:36:52,803][INFO ][o.e.n.Node ] [Zn8_vUw] starting ...
[2017-05-12T09:36:53,754][INFO ][o.e.t.TransportService ] [Zn8_vUw] publish_address {172.17.0.2:9300}, bound_addresses {0.0.0.0:9300}
[2017-05-12T09:36:53,792][INFO ][o.e.b.BootstrapChecks ] [Zn8_vUw] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-05-12T09:36:53,816][ERROR][o.e.b.Bootstrap ] [Zn8_vUw] node validation exception
bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-05-12T09:36:53,840][INFO ][o.e.n.Node ] [Zn8_vUw] stopping ...
[2017-05-12T09:36:53,932][INFO ][o.e.n.Node ] [Zn8_vUw] stopped
[2017-05-12T09:36:53,932][INFO ][o.e.n.Node ] [Zn8_vUw] closing ...
[2017-05-12T09:36:53,998][INFO ][o.e.n.Node ] [Zn8_vUw] closed
and installing with Guide : http://elk-docker.readthedocs.io/
Will guide me so that I can have a elk stack ?
Try using docker compose
docker-compose.yml
elk:
image: sebp/elk
ports:
- "5601:5601"
- "9200:9200"
- "5044:5044"
environment:
- ELASTICSEARCH_USER=elastic
- ELASTICSEARCH_PASSWORD=s0m3C0mpl3xP#ssw0rd!
- LOGSTASH_USER=elastic
- LOGSTASH_PASSWORD=s0m3C0mpl3xP#ssw0rd!
- KIBANA_USER=kibana
- KIBANA_PASSWORD=s0m3C0mpl3xP#ssw0rd!
$ docker-compose up -d elk
Source : https://medium.com/#gchandra/easy-way-setup-elk-on-docker-10938f3fcf96
As error showing that max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
so you need to increase memory by running below command
vm.max_map_count = 262144
and run it again

How to access remotely to elasticsearch 5?

I want to access remotely to my elasticsearch version 5.
I tried to add this line in the elasticseach.yml file as I did with elassticsearch version 4 (which worked):
network.bind_host: 0.0.0.0
However, it doesn't work and elasticsearch displays an error and does not work.
[sagemcom#localhost bin]$ ./elasticsearch
[2016-11-07T12:17:58,432][INFO ][o.e.n.Node ] [] initializing ...
[2016-11-07T12:17:58,564][INFO ][o.e.e.NodeEnvironment ] [NqVMQ-B] using [1] data paths, mounts [[/home (/dev/mapper/centos-home)]], net usable_space [13.5gb], net total_space [23gb], spins? [possibly], types [xfs]
[2016-11-07T12:17:58,565][INFO ][o.e.e.NodeEnvironment ] [NqVMQ-B] heap size [1.9gb], compressed ordinary object pointers [true]
[2016-11-07T12:17:58,574][INFO ][o.e.n.Node ] [NqVMQ-B] node name [NqVMQ-B] derived from node ID; set [node.name] to override
[2016-11-07T12:17:58,577][INFO ][o.e.n.Node ] [NqVMQ-B] version[5.0.0], pid[3400], build[253032b/2016-10-26T04:37:51.531Z], OS[Linux/3.10.0-123.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_73/25.73-b02]
[2016-11-07T12:17:59,776][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [aggs-matrix-stats]
[2016-11-07T12:17:59,776][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [ingest-common]
[2016-11-07T12:17:59,776][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [lang-expression]
[2016-11-07T12:17:59,777][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [lang-groovy]
[2016-11-07T12:17:59,777][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [lang-mustache]
[2016-11-07T12:17:59,777][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [lang-painless]
[2016-11-07T12:17:59,778][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [percolator]
[2016-11-07T12:17:59,779][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [reindex]
[2016-11-07T12:17:59,779][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [transport-netty3]
[2016-11-07T12:17:59,780][INFO ][o.e.p.PluginsService ] [NqVMQ-B] loaded module [transport-netty4]
[2016-11-07T12:17:59,781][INFO ][o.e.p.PluginsService ] [NqVMQ-B] no plugins loaded
[2016-11-07T12:18:03,138][INFO ][o.e.n.Node ] [NqVMQ-B] initialized
[2016-11-07T12:18:03,139][INFO ][o.e.n.Node ] [NqVMQ-B] starting ...
[2016-11-07T12:18:03,349][INFO ][o.e.t.TransportService ] [NqVMQ-B] publish_address {127.0.0.1:9300}, bound_addresses {[::]:9300}
[2016-11-07T12:18:03,355][INFO ][o.e.b.BootstrapCheck ] [NqVMQ-B] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
[2016-11-07T12:18:03,364][INFO ][o.e.n.Node ] [NqVMQ-B] stopping ...
[2016-11-07T12:18:03,402][INFO ][o.e.n.Node ] [NqVMQ-B] stopped
[2016-11-07T12:18:03,402][INFO ][o.e.n.Node ] [NqVMQ-B] closing ...
[2016-11-07T12:18:03,417][INFO ][o.e.n.Node ] [NqVMQ-B] closed
If you look at the error error says
"ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
"
Solve this issue as
Development Mode: If you are running elastic search in development mode we don't need to set the max_map_count. Just run as follows
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:5.6.4
Production Mode: : I f you run in production you need to set the property as below.
Linux
The vm.max_map_count setting should be set permanently in /etc/sysctl.conf:
grep vm.max_map_count /etc/sysctl.conf
vm.max_map_count=262144
OR
sysctl -w vm.max_map_count=262144
Then Accessing it from remote is different case.

Fail when start a new container with elasticsearch 5.0

I run:
$ docker run --rm -ti -p 9200:9200 -p 9300:9300 elasticsearch
But I can't start elasticsearch 5.0, here the log:
Unable to find image 'elasticsearch:latest' locally
latest: Pulling from library/elasticsearch
43c265008fae: Already exists
af36d2c7a148: Already exists
2b7b4d10e1c1: Already exists
9f7579daddb2: Already exists
a985511f2468: Already exists
6c2b485fcc3e: Already exists
00b73c83b440: Already exists
a38975861823: Pull complete
b9d0b4aaf934: Pull complete
f5f25bb3de04: Pull complete
93426e1e8953: Pull complete
590c2ab9b639: Pull complete
31bf48850601: Pull complete
21aad7ce70fa: Pull complete
Digest: sha256:29205bca045c8d083f777dfc453f4f1ff3d2c08ea4f529f88795166c58e5607e
Status: Downloaded newer image for elasticsearch:latest
[2016-10-31T20:56:22,023][INFO ][o.e.n.Node ] [] initializing ...
[2016-10-31T20:56:22,129][INFO ][o.e.e.NodeEnvironment ] [N-vpIo0] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda8)]], net usable_space [48.7gb], net total_space [98.3gb], spins? [possibly], types [ext4]
[2016-10-31T20:56:22,130][INFO ][o.e.e.NodeEnvironment ] [N-vpIo0] heap size [1.9gb], compressed ordinary object pointers [true]
[2016-10-31T20:56:22,130][INFO ][o.e.n.Node ] [N-vpIo0] node name [N-vpIo0] derived from node ID; set [node.name] to override
[2016-10-31T20:56:22,132][INFO ][o.e.n.Node ] [N-vpIo0] version[5.0.0], pid[1], build[253032b/2016-10-26T05:11:34.737Z], OS[Linux/3.16.0-4-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_102/25.102-b14]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [aggs-matrix-stats]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [ingest-common]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [lang-expression]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [lang-groovy]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [lang-mustache]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [lang-painless]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [percolator]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [reindex]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [transport-netty3]
[2016-10-31T20:56:22,785][INFO ][o.e.p.PluginsService ] [N-vpIo0] loaded module [transport-netty4]
[2016-10-31T20:56:22,786][INFO ][o.e.p.PluginsService ] [N-vpIo0] no plugins loaded
[2016-10-31T20:56:22,875][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2016-10-31T20:56:24,241][INFO ][o.e.n.Node ] [N-vpIo0] initialized
[2016-10-31T20:56:24,241][INFO ][o.e.n.Node ] [N-vpIo0] starting ...
[2016-10-31T20:56:24,380][INFO ][o.e.t.TransportService ] [N-vpIo0] publish_address {172.17.0.4:9300}, bound_addresses {[::]:9300}
[2016-10-31T20:56:24,389][INFO ][o.e.b.BootstrapCheck ] [N-vpIo0] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
[2016-10-31T20:56:24,400][INFO ][o.e.n.Node ] [N-vpIo0] stopping ...
[2016-10-31T20:56:24,426][INFO ][o.e.n.Node ] [N-vpIo0] stopped
[2016-10-31T20:56:24,426][INFO ][o.e.n.Node ] [N-vpIo0] closing ...
[2016-10-31T20:56:24,433][INFO ][o.e.n.Node ] [N-vpIo0] closed
Please, How can I solve this issue?
As said at the Docker image documentation in Host Setup section:
Note: since 5.0, Elasticsearch only listens on localhost by default,
so this image sets network.host to 0.0.0.0 (given that localhost is
not terribly useful in the Docker context).
As a result, Elasticsearch is more strict about the bootstrap checks
that it performs, especially when checking the value of
vm.max_map_count which is not namespaced and thus must be set to an
acceptable value on the host (as opposed to simply using --sysctl on
docker run).
So you just have to execute this on the host machine:
sysctl -w vm.max_map_count=262144
And you are good to go with the 5.0.0 version
To set this value permanently, update the vm.max_map_count setting in /etc/sysctl.conf. To verify after rebooting, run sysctl vm.max_map_count.
More information here
This is a problem related with virtual memory settings, the following command should works:
sysctl -w vm.max_map_count=262144
More details can be found here https://www.elastic.co/guide/en/elasticsearch/reference/5.0/vm-max-map-count.html

Resources