No response from elasticsearch thru sense and browser - windows

I've installed ElasticSearch 5.3 on my Windows 10 machine recently and after following the installation process, ES does not respond when I access http://localhost:9200 and from Chrome's Sense.
However, when I use curl 'http://localhost:9200' on the command line, it sends this:
{
"name" : "test",
"cluster_name" : "my-application",
"cluster_uuid" : "9-wvh6UXTHSKmkdpblqyyA",
"version" : {
"number" : "5.3.0",
"build_hash" : "3adb13b",
"build_date" : "2017-03-23T03:31:50.652Z",
"build_snapshot" : false,
"lucene_version" : "6.4.1"
},
"tagline" : "You Know, for Search"
}
which indicates that the ES is running/configured correctly(?).
I also configured network.host: to 0.0.0.0/127.0.0.1/:: of elasticsearch.yml still no effect.
Here is the latest log info:
[2017-04-07T18:21:06,481][INFO ][o.e.n.Node ] [test] initializing ...
[2017-04-07T18:21:06,633][INFO ][o.e.e.NodeEnvironment ] [test] using [1] data paths, mounts [[OSDisk (C:)]], net usable_space [129.2gb], net total_space [232.3gb], spins? [unknown], types [NTFS]
[2017-04-07T18:21:06,634][INFO ][o.e.e.NodeEnvironment ] [test] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-04-07T18:21:06,681][INFO ][o.e.n.Node ] [test] node name [test], node ID [Ab5g3zN0S7qNOuP_asF-iQ]
[2017-04-07T18:21:06,682][INFO ][o.e.n.Node ] [test] version[5.3.0], pid[312], build[3adb13b/2017-03-23T03:31:50.652Z], OS[Windows 10/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_121/25.121-b13]
[2017-04-07T18:21:08,288][INFO ][o.e.p.PluginsService ] [test] loaded module [aggs-matrix-stats]
[2017-04-07T18:21:08,289][INFO ][o.e.p.PluginsService ] [test] loaded module [ingest-common]
[2017-04-07T18:21:08,291][INFO ][o.e.p.PluginsService ] [test] loaded module [lang-expression]
[2017-04-07T18:21:08,291][INFO ][o.e.p.PluginsService ] [test] loaded module [lang-groovy]
[2017-04-07T18:21:08,292][INFO ][o.e.p.PluginsService ] [test] loaded module [lang-mustache]
[2017-04-07T18:21:08,294][INFO ][o.e.p.PluginsService ] [test] loaded module [lang-painless]
[2017-04-07T18:21:08,295][INFO ][o.e.p.PluginsService ] [test] loaded module [percolator]
[2017-04-07T18:21:08,296][INFO ][o.e.p.PluginsService ] [test] loaded module [reindex]
[2017-04-07T18:21:08,296][INFO ][o.e.p.PluginsService ] [test] loaded module [transport-netty3]
[2017-04-07T18:21:08,298][INFO ][o.e.p.PluginsService ] [test] loaded module [transport-netty4]
[2017-04-07T18:21:08,299][INFO ][o.e.p.PluginsService ] [test] loaded plugin [ltr-query]
[2017-04-07T18:21:11,330][INFO ][o.e.n.Node ] [test] initialized
[2017-04-07T18:21:11,330][INFO ][o.e.n.Node ] [test] starting ...
[2017-04-07T18:21:11,612][INFO ][o.e.t.TransportService ] [test] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2017-04-07T18:21:14,657][INFO ][o.e.c.s.ClusterService ] [test] new_master {test}{Ab5g3zN0S7qNOuP_asF-iQ}{XYHTUZ8AQN6kYrEPLChFNg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-04-07T18:21:14,826][INFO ][o.e.h.n.Netty4HttpServerTransport] [test] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2017-04-07T18:21:14,826][INFO ][o.e.n.Node ] [test] started
[2017-04-07T18:21:14,982][INFO ][o.e.g.GatewayService ] [test] recovered [1] indices into cluster_state
[2017-04-07T18:21:15,510][INFO ][o.e.c.r.a.AllocationService] [test] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[tmdb][0]] ...]).

Got it! Referring from Network Settings Documentation, I tried to run it using this script: elasticsearch.bat -E network.host=_local_, and it worked!
I also changed the configuration file(./config/elasticsearch.yml). Just uncomment the line and change it to network.host: _local_. After that, I simply ran elasticsearch.bat directly.
OR
to put it simply just comment the network.host line.

Related

Elasticsearch node gets disconnected from cluster

I'm running Elasticsearch version 7.10.2 with 8 nodes of cluster (3 master, 3 data, and 2 client node).
It was running smoothly till now , now the nodes began to disconnect from the cluster one node at a day , when I checked the node Elasticsearch was stopped. I have noticed that node get disconnected randomly
last node that was disconnected was data-node-3 at 12:10 , I want to know the main reason why Elasticsearch getting stopped also please suggest on how to resolve this problem.
Note:-we only restart master-1(Monday) , master-2(Tuesday) and master-3(Wednesday) using cronjob we have not set any cronjob for data and client nodes.
cluster information:-
Name RAM Heap Cpu
client 1 4gb 2gb 2
client 2 4gb 2gb 2
data 1 8gb 4gb 2
data 2 8gb 4gb 2
data 3 8gb 4gb 2
master 1 2gb 1gb 2
master 2 2gb 1gb 2
master 3 2gb 1gb 2
I am attaching log here of datanode-3
[2022-08-21T12:10:00,193][INFO ][o.e.n.Node ] [data-node-3] version[7.10.2], pid[534], build[default/deb/747e1cc71def077253878a59143c1f785afa92b9/2021-01-13T00:42:12.435326Z], OS[Linux/5.15.0-1016-gcp/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2022-08-21T12:10:00,381][INFO ][o.e.n.Node ] [data-node-3] JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]
[2022-08-21T12:10:00,382][INFO ][o.e.n.Node ] [data-node-3] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms4g, -Xmx4g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-5474094125273943758, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=2147483648, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=deb, -Des.bundled_jdk=true]
[2022-08-21T12:10:18,654][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [aggs-matrix-stats]
[2022-08-21T12:10:18,656][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [analysis-common]
[2022-08-21T12:10:18,656][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [constant-keyword]
[2022-08-21T12:10:18,660][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [flattened]
[2022-08-21T12:10:18,661][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [frozen-indices]
[2022-08-21T12:10:18,663][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [ingest-common]
[2022-08-21T12:10:18,663][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [ingest-geoip]
[2022-08-21T12:10:18,672][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [ingest-user-agent]
[2022-08-21T12:10:18,673][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [kibana]
[2022-08-21T12:10:18,674][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [lang-expression]
[2022-08-21T12:10:18,674][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [lang-mustache]
[2022-08-21T12:10:18,675][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [lang-painless]
[2022-08-21T12:10:18,675][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [mapper-extras]
[2022-08-21T12:10:18,677][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [mapper-version]
[2022-08-21T12:10:18,679][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [parent-join]
[2022-08-21T12:10:18,680][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [percolator]
[2022-08-21T12:10:18,681][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [rank-eval]
[2022-08-21T12:10:18,682][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [reindex]
[2022-08-21T12:10:18,682][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [repositories-metering-api]
[2022-08-21T12:10:18,684][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [repository-url]
[2022-08-21T12:10:18,685][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [search-business-rules]
[2022-08-21T12:10:18,685][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [searchable-snapshots]
[2022-08-21T12:10:18,697][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [spatial]
[2022-08-21T12:10:18,698][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [systemd]
[2022-08-21T12:10:18,698][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [transform]
[2022-08-21T12:10:18,699][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [transport-netty4]
[2022-08-21T12:10:18,701][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [unsigned-long]
[2022-08-21T12:10:18,702][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [vectors]
[2022-08-21T12:10:18,702][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [wildcard]
[2022-08-21T12:10:18,704][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-analytics]
[2022-08-21T12:10:18,709][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-async]
[2022-08-21T12:10:18,710][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-async-search]
[2022-08-21T12:10:18,710][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-autoscaling]
[2022-08-21T12:10:18,711][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-ccr]
[2022-08-21T12:10:18,712][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-core]
[2022-08-21T12:10:18,713][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-data-streams]
[2022-08-21T12:10:18,714][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-deprecation]
[2022-08-21T12:10:18,714][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-enrich]
[2022-08-21T12:10:18,717][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-eql]
[2022-08-21T12:10:18,719][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-graph]
[2022-08-21T12:10:18,722][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-identity-provider]
[2022-08-21T12:10:18,723][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-ilm]
[2022-08-21T12:10:18,724][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-logstash]
[2022-08-21T12:10:18,724][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-ml]
[2022-08-21T12:10:18,725][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-monitoring]
[2022-08-21T12:10:18,732][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-ql]
[2022-08-21T12:10:18,742][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-rollup]
[2022-08-21T12:10:18,743][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-security]
[2022-08-21T12:10:18,743][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-sql]
[2022-08-21T12:10:18,744][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-stack]
[2022-08-21T12:10:18,744][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-voting-only-node]
[2022-08-21T12:10:18,745][INFO ][o.e.p.PluginsService ] [data-node-3] loaded module [x-pack-watcher]
[2022-08-21T12:10:18,746][INFO ][o.e.p.PluginsService ] [data-node-3] no plugins loaded
[2022-08-21T12:10:19,097][INFO ][o.e.e.NodeEnvironment ] [data-node-3] using [1] data paths, mounts [[/ (/dev/root)]], net usable_space [77.9gb], net total_space [96.7gb], types [ext4]
[2022-08-21T12:10:19,098][INFO ][o.e.e.NodeEnvironment ] [data-node-3] heap size [4gb], compressed ordinary object pointers [true]
[2022-08-21T12:10:22,654][INFO ][o.e.n.Node ] [data-node-3] node name [data-node-3], node ID [KG1DQKuASBq2GpwYCz4pkQ], cluster name [la-prod7-cluster], roles [transform, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold]
[2022-08-21T12:10:37,839][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [data-node-3] [controller/974] [Main.cc#114] controller (64 bit): Version 7.10.2 (Build 40a3af639d4698) Copyright (c) 2021 Elasticsearch BV
[2022-08-21T12:10:38,910][INFO ][o.e.x.s.a.s.FileRolesStore] [data-node-3] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2022-08-21T12:10:40,701][INFO ][o.e.t.NettyAllocator ] [data-node-3] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=512kb, suggested_max_allocation_size=512kb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=2mb}]
[2022-08-21T12:10:40,845][INFO ][o.e.d.DiscoveryModule ] [data-node-3] using discovery type [zen] and seed hosts providers [settings]
[2022-08-21T12:10:41,685][WARN ][o.e.g.DanglingIndicesState] [data-node-3] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-08-21T12:10:42,542][INFO ][o.e.n.Node ] [data-node-3] initialized
[2022-08-21T12:10:42,543][INFO ][o.e.n.Node ] [data-node-3] starting ...
[2022-08-21T12:10:43,019][INFO ][o.e.t.TransportService ] [data-node-3] publish_address {10.115.0.141:9300}, bound_addresses {10.115.0.141:9300}
[2022-08-21T12:10:44,459][INFO ][o.e.b.BootstrapChecks ] [data-node-3] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-08-21T12:10:44,464][INFO ][o.e.c.c.Coordinator ] [data-node-3] cluster UUID [uSrx3heuSVypgStFinWJrw]
[2022-08-21T12:10:48,038][INFO ][o.e.c.s.ClusterApplierService] [data-node-3] master node changed {previous [], current [{master-node-1}{Z3sr97BDRFmgi2Mu_6NxXA}{Ge6zQsnLShmFzLCiSE2i2g}{10.115.0.21}{10.115.0.21:9300}{m}{xpack.installed=true, transform.node=false}]}, added {{master-node-2}{t4Pbwly1S0q084vosUjmNw}{m6Rwe3xLSquVPA2Oj0bFxQ}{10.115.0.49}{10.115.0.49:9300}{m}{xpack.installed=true, transform.node=false},{data-node-1}{aesOUIaRTA-4xHPDSZ-ZyA}{x1xAw9K8RGuGCzahKeE3EA}{10.115.0.95}{10.115.0.95:9300}{cdhlrstw}{ml.machine_memory=8340889600, ml.max_open_jobs=20, xpack.installed=true, transform.node=true},{master-node-3}{ka0XuT6zQMOLL_CTqa3ysA}{AmKed0FWR8CnZoNE5JsUAw}{10.115.0.114}{10.115.0.114:9300}{m}{xpack.installed=true, transform.node=false},{master-node-1}{Z3sr97BDRFmgi2Mu_6NxXA}{Ge6zQsnLShmFzLCiSE2i2g}{10.115.0.21}{10.115.0.21:9300}{m}{xpack.installed=true, transform.node=false},{client-node-2}{UOSU11VmS_GByWchh0jV8w}{X2uXIsMsQPu_34SaOKk5uw}{10.115.0.81}{10.115.0.81:9300}{ilr}{ml.machine_memory=4126511104, ml.max_open_jobs=20, xpack.installed=true, transform.node=false},{data-node-2}{qEU0XgvoSCmgqfBLZIEOLg}{zfVDj9HWT-OlVnhfyv6VSg}{10.115.0.112}{10.115.0.112:9300}{cdhlrstw}{ml.machine_memory=8332742656, ml.max_open_jobs=20, xpack.installed=true, transform.node=true},{client-node-1}{FvYReNu1SF-SX7aT5d3GLQ}{iKJIdSP_RAey-Q9N-OkH2Q}{10.115.0.80}{10.115.0.80:9300}{ilr}{ml.machine_memory=4126502912, ml.max_open_jobs=20, xpack.installed=true, transform.node=false}}, term: 168, version: 15421, reason: ApplyCommitRequest{term=168, version=15421, sourceNode={master-node-1}{Z3sr97BDRFmgi2Mu_6NxXA}{Ge6zQsnLShmFzLCiSE2i2g}{10.115.0.21}{10.115.0.21:9300}{m}{xpack.installed=true, transform.node=false}}
[2022-08-21T12:10:48,072][INFO ][o.e.n.Node ] [data-node-3] stopping ...
[2022-08-21T12:10:48,077][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [data-node-3] [controller/974] [Main.cc#154] ML controller exiting
[2022-08-21T12:10:48,078][INFO ][o.e.x.m.p.NativeController] [data-node-3] Native controller process has stopped - no new native processes can be started
[2022-08-21T12:10:48,080][INFO ][o.e.x.w.WatcherService ] [data-node-3] stopping watch service, reason [shutdown initiated]
[2022-08-21T12:10:48,081][INFO ][o.e.x.w.WatcherLifeCycleService] [data-node-3] watcher has stopped and shutdown
[2022-08-21T12:10:48,511][INFO ][o.e.h.AbstractHttpServerTransport] [data-node-3] publish_address {10.115.0.141:9200}, bound_addresses {10.115.0.141:9200}
[2022-08-21T12:10:48,511][INFO ][o.e.n.Node ] [data-node-3] started
[2022-08-21T12:10:49,584][INFO ][o.e.c.s.ClusterSettings ] [data-node-3] updating [xpack.monitoring.collection.enabled] from [false] to [true]
[2022-08-21T12:10:50,534][INFO ][o.e.x.s.a.TokenService ] [data-node-3] refresh keys
[2022-08-21T12:10:50,880][INFO ][o.e.x.s.a.TokenService ] [data-node-3] refreshed keys
[2022-08-21T12:10:51,977][INFO ][o.e.l.LicenseService ] [data-node-3] license [4d78bbbf-fb01-4d13-8178-86cc235014f1] mode [basic] - valid
[2022-08-21T12:10:51,980][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [data-node-3] Active license is now [BASIC]; Security is enabled
[2022-08-21T12:10:51,995][INFO ][o.e.c.c.Coordinator ] [data-node-3] master node [{master-node-1}{Z3sr97BDRFmgi2Mu_6NxXA}{Ge6zQsnLShmFzLCiSE2i2g}{10.115.0.21}{10.115.0.21:9300}{m}{xpack.installed=true, transform.node=false}] failed, restarting discovery
org.elasticsearch.transport.NodeDisconnectedException: [master-node-1][10.115.0.21:9300][disconnected] disconnected
[2022-08-21T12:10:52,001][INFO ][o.e.c.c.JoinHelper ] [data-node-3] failed to join {master-node-1}{Z3sr97BDRFmgi2Mu_6NxXA}{Ge6zQsnLShmFzLCiSE2i2g}{10.115.0.21}{10.115.0.21:9300}{m}{xpack.installed=true, transform.node=false} with JoinRequest{sourceNode={data-node-3}{KG1DQKuASBq2GpwYCz4pkQ}{KmQunMToQcS5Stxb5IT7MA}{10.115.0.141}{10.115.0.141:9300}{cdhlrstw}{ml.machine_memory=8332750848, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, minimumTerm=168, optionalJoin=Optional.empty}
org.elasticsearch.transport.NodeDisconnectedException: [master-node-1][10.115.0.21:9300][internal:cluster/coordination/join] disconnected
[2022-08-21T12:10:52,002][INFO ][o.e.c.c.JoinHelper ] [data-node-3] failed to join {master-node-1}{Z3sr97BDRFmgi2Mu_6NxXA}{Ge6zQsnLShmFzLCiSE2i2g}{10.115.0.21}{10.115.0.21:9300}{m}{xpack.installed=true, transform.node=false} with JoinRequest{sourceNode={data-node-3}{KG1DQKuASBq2GpwYCz4pkQ}{KmQunMToQcS5Stxb5IT7MA}{10.115.0.141}{10.115.0.141:9300}{cdhlrstw}{ml.machine_memory=8332750848, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, minimumTerm=168, optionalJoin=Optional.empty}
org.elasticsearch.transport.NodeDisconnectedException: [master-node-1][10.115.0.21:9300][internal:cluster/coordination/join] disconnected
[2022-08-21T12:10:52,073][INFO ][o.e.c.c.Coordinator ] [data-node-3] master node [{master-node-1}{Z3sr97BDRFmgi2Mu_6NxXA}{Ge6zQsnLShmFzLCiSE2i2g}{10.115.0.21}{10.115.0.21:9300}{m}{xpack.installed=true, transform.node=false}] failed, restarting discovery
org.elasticsearch.transport.ConnectTransportException: [master-node-1][10.115.0.21:9300] disconnected during check
at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler$1.handleException(LeaderChecker.java:279) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:605) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.cluster.coordination.LeaderChecker$CheckScheduler.handleWakeUp(LeaderChecker.java:248) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.cluster.coordination.LeaderChecker.updateLeader(LeaderChecker.java:158) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.cluster.coordination.Coordinator.becomeFollower(Coordinator.java:635) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.cluster.coordination.Coordinator.handlePublishRequest(Coordinator.java:342) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.cluster.coordination.PublicationTransportHandler.acceptState(PublicationTransportHandler.java:230) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.cluster.coordination.PublicationTransportHandler.handleIncomingPublishRequest(PublicationTransportHandler.java:210) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.cluster.coordination.PublicationTransportHandler.lambda$new$0(PublicationTransportHandler.java:102) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:257) ~[?:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:224) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.lambda$messageReceived$0(SecurityServerTransportInterceptor.java:306) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeSystemUser(AuthorizationService.java:401) ~[?:?]
at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:205) ~[?:?]
at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.lambda$inbound$1(ServerTransportFilter.java:129) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:323) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$6(AuthenticationService.java:384) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:395) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:320) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:261) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:173) ~[?:?]
at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:120) ~[?:?]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:313) ~[?:?]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:72) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:305) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:743) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.10.2.jar:7.10.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: org.elasticsearch.transport.NodeNotConnectedException: [master-node-1][10.115.0.21:9300] Node not connected
at org.elasticsearch.transport.ClusterConnectionManager.getConnection(ClusterConnectionManager.java:189) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:682) ~[elasticsearch-7.10.2.jar:7.10.2]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:602) ~[elasticsearch-7.10.2.jar:7.10.2]
... 31 more
[2022-08-21T12:10:52,100][INFO ][o.e.n.Node ] [data-node-3] stopped
[2022-08-21T12:10:52,101][INFO ][o.e.n.Node ] [data-node-3] closing ...
[2022-08-21T12:10:52,161][INFO ][o.e.n.Node ] [data-node-3] closed
[2022-08-21T13:25:16,191][INFO ][o.e.n.Node ] [data-node-3] version[7.10.2], pid[1587], build[default/deb/747e1cc71def077253878a59143c1f785afa92b9/2021-01-13T00:42:12.435326Z], OS[Linux/5.15.0-1016-gcp/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]

Elasticsearch is not starting after re-installation

I recently noted that Elasticsearch was not starting after a reboot. Eventualy I deleted it and re-installed from scratch. However I'm still having the same problem. When I try to start it, systemctl shows the following status:
$ sudo systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; disabled; vendor preset: enabled)
Active: failed (Result: timeout) since Wed 2020-09-16 16:22:32 EDT; 3min 23s ago
Docs: https://www.elastic.co
Process: 1716 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=143)
Main PID: 1716 (code=exited, status=143)
Sep 16 16:20:59 enk-network-sniffer systemd[1]: Starting Elasticsearch...
Sep 16 16:22:29 enk-network-sniffer systemd[1]: elasticsearch.service: start operation timed out. Terminating.
Sep 16 16:22:32 enk-network-sniffer systemd[1]: elasticsearch.service: Failed with result 'timeout'.
Sep 16 16:22:32 enk-network-sniffer systemd[1]: Failed to start Elasticsearch.
Looking at
/var/log/elasticsearch/elasticsearch.log
shows the following logs:
[2020-09-16T16:42:33,268][INFO ][o.e.e.NodeEnvironment ] [enk-network-sniffer] using [1] data paths, mounts [[/ (/dev/sda5)]], net usable_space [23.4gb], net total_space [38.6gb], types [ext4]
[2020-09-16T16:42:33,301][INFO ][o.e.e.NodeEnvironment ] [enk-network-sniffer] heap size [1gb], compressed ordinary object pointers [true]
[2020-09-16T16:42:33,717][INFO ][o.e.n.Node ] [enk-network-sniffer] node name [enk-network-sniffer], node ID [RljBYx0ZQx6E87E__MV3gQ], cluster name [elasticsearch]
[2020-09-16T16:42:33,720][INFO ][o.e.n.Node ] [enk-network-sniffer] version[7.7.1], pid[1690], build[default/deb/ad56dce891c901a492bb1ee393f12dfff473a423/2020-05-28T16:30:01.040088Z], OS[Linux/5.4.0-47-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/14.0.1/14.0.1+7]
[2020-09-16T16:42:33,721][INFO ][o.e.n.Node ] [enk-network-sniffer] JVM home [/usr/share/elasticsearch/jdk]
[2020-09-16T16:42:33,722][INFO ][o.e.n.Node ] [enk-network-sniffer] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-8868144046098870714, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=deb, -Des.bundled_jdk=true]
[2020-09-16T16:42:54,521][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [aggs-matrix-stats]
[2020-09-16T16:42:54,522][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [analysis-common]
[2020-09-16T16:42:54,531][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [constant-keyword]
[2020-09-16T16:42:54,531][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [flattened]
[2020-09-16T16:42:54,532][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [frozen-indices]
[2020-09-16T16:42:54,535][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [ingest-common]
[2020-09-16T16:42:54,535][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [ingest-geoip]
[2020-09-16T16:42:54,536][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [ingest-user-agent]
[2020-09-16T16:42:54,538][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [lang-expression]
[2020-09-16T16:42:54,539][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [lang-mustache]
[2020-09-16T16:42:54,543][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [lang-painless]
[2020-09-16T16:42:54,543][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [mapper-extras]
[2020-09-16T16:42:54,547][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [parent-join]
[2020-09-16T16:42:54,548][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [percolator]
[2020-09-16T16:42:54,551][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [rank-eval]
[2020-09-16T16:42:54,552][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [reindex]
[2020-09-16T16:42:54,552][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [repository-url]
[2020-09-16T16:42:54,555][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [search-business-rules]
[2020-09-16T16:42:54,556][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [spatial]
[2020-09-16T16:42:54,556][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [systemd]
[2020-09-16T16:42:54,561][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [tasks]
[2020-09-16T16:42:54,562][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [transform]
[2020-09-16T16:42:54,562][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [transport-netty4]
[2020-09-16T16:42:54,565][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [vectors]
[2020-09-16T16:42:54,567][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-analytics]
[2020-09-16T16:42:54,571][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-async-search]
[2020-09-16T16:42:54,574][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-autoscaling]
[2020-09-16T16:42:54,574][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ccr]
[2020-09-16T16:42:54,577][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-core]
[2020-09-16T16:42:54,580][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-deprecation]
[2020-09-16T16:42:54,581][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-enrich]
[2020-09-16T16:42:54,582][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-eql]
[2020-09-16T16:42:54,583][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-graph]
[2020-09-16T16:42:54,586][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-identity-provider]
[2020-09-16T16:42:54,588][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ilm]
[2020-09-16T16:42:54,592][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-logstash]
[2020-09-16T16:42:54,593][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ml]
[2020-09-16T16:42:54,594][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-monitoring]
[2020-09-16T16:42:54,602][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ql]
[2020-09-16T16:42:54,602][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-rollup]
[2020-09-16T16:42:54,602][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-security]
[2020-09-16T16:42:54,612][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-sql]
[2020-09-16T16:42:54,612][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-voting-only-node]
[2020-09-16T16:42:54,633][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-watcher]
[2020-09-16T16:42:54,634][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] no plugins loaded
I'm not being able to find any error or warning in these files. Any idea on what might be wrong or where else I can look at to find more clues?
UPDATE:
Here is the /etc/elasticsearch/elasticsearch.yml file:
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
discovery.type: single-node
cluster.initial_master_nodes: []
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
After increasing the heap size, I obtained the following logs:
[2020-09-17T11:47:21,107][INFO ][o.e.e.NodeEnvironment ] [enk-network-sniffer] using [1] data paths, mounts [[/ (/dev/sda5)]], net usable_space [174.4gb], net total_space [196.1gb], types [ext4]
[2020-09-17T11:47:21,129][INFO ][o.e.e.NodeEnvironment ] [enk-network-sniffer] heap size [4gb], compressed ordinary object pointers [true]
[2020-09-17T11:47:23,118][INFO ][o.e.n.Node ] [enk-network-sniffer] node name [enk-network-sniffer], node ID [RljBYx0ZQx6E87E__MV3gQ], cluster name [elasticsearch]
[2020-09-17T11:47:23,128][INFO ][o.e.n.Node ] [enk-network-sniffer] version[7.7.1], pid[1966], build[default/deb/ad56dce891c901a492bb1ee393f12dfff473a423/2020-05-28T16:30:01.040088Z], OS[Linux/5.4.0-47-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/14.0.1/14.0.1+7]
[2020-09-17T11:47:23,129][INFO ][o.e.n.Node ] [enk-network-sniffer] JVM home [/usr/share/elasticsearch/jdk]
[2020-09-17T11:47:23,129][INFO ][o.e.n.Node ] [enk-network-sniffer] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms4g, -Xmx4g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-11788675126322072724, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=2147483648, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=deb, -Des.bundled_jdk=true]
[2020-09-17T11:47:54,845][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [aggs-matrix-stats]
[2020-09-17T11:47:54,845][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [analysis-common]
[2020-09-17T11:47:54,849][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [constant-keyword]
[2020-09-17T11:47:54,850][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [flattened]
[2020-09-17T11:47:54,850][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [frozen-indices]
[2020-09-17T11:47:54,850][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [ingest-common]
[2020-09-17T11:47:54,850][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [ingest-geoip]
[2020-09-17T11:47:54,851][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [ingest-user-agent]
[2020-09-17T11:47:54,852][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [lang-expression]
[2020-09-17T11:47:54,852][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [lang-mustache]
[2020-09-17T11:47:54,853][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [lang-painless]
[2020-09-17T11:47:54,853][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [mapper-extras]
[2020-09-17T11:47:54,854][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [parent-join]
[2020-09-17T11:47:54,854][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [percolator]
[2020-09-17T11:47:54,855][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [rank-eval]
[2020-09-17T11:47:54,855][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [reindex]
[2020-09-17T11:47:54,856][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [repository-url]
[2020-09-17T11:47:54,856][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [search-business-rules]
[2020-09-17T11:47:54,856][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [spatial]
[2020-09-17T11:47:54,856][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [systemd]
[2020-09-17T11:47:54,857][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [tasks]
[2020-09-17T11:47:54,857][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [transform]
[2020-09-17T11:47:54,858][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [transport-netty4]
[2020-09-17T11:47:54,858][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [vectors]
[2020-09-17T11:47:54,859][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-analytics]
[2020-09-17T11:47:54,860][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-async-search]
[2020-09-17T11:47:54,860][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-autoscaling]
[2020-09-17T11:47:54,860][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ccr]
[2020-09-17T11:47:54,861][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-core]
[2020-09-17T11:47:54,861][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-deprecation]
[2020-09-17T11:47:54,862][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-enrich]
[2020-09-17T11:47:54,862][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-eql]
[2020-09-17T11:47:54,862][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-graph]
[2020-09-17T11:47:54,863][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-identity-provider]
[2020-09-17T11:47:54,863][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ilm]
[2020-09-17T11:47:54,864][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-logstash]
[2020-09-17T11:47:54,864][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ml]
[2020-09-17T11:47:54,865][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-monitoring]
[2020-09-17T11:47:54,867][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-ql]
[2020-09-17T11:47:54,868][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-rollup]
[2020-09-17T11:47:54,868][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-security]
[2020-09-17T11:47:54,868][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-sql]
[2020-09-17T11:47:54,869][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-voting-only-node]
[2020-09-17T11:47:54,869][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] loaded module [x-pack-watcher]
[2020-09-17T11:47:54,870][INFO ][o.e.p.PluginsService ] [enk-network-sniffer] no plugins loaded
ES is still failing to start.
Thanks.
As there is no error message, its difficult to pinpoint to the exact cause, but could you provide your elasticsearch.yml and I also see you have defined 1gb of Heap space, could you try increasing it?
Only interesting this in the logs are:
Active: failed (Result: timeout) since Wed 2020-09-16 16:22:32 EDT;
3min 23s ago
Docs: https://www.elastic.co
Process: 1716 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p
${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=143) Main
PID: 1716 (code=exited, status=143)
You can notice the timeout and 143 status and you can read more about it in
https://serverfault.com/questions/695849/services-remain-in-failed-state-after-stopped-with-systemctl and see if you can figure out the error logs or more info to debug the issue.

Wagtail Elasticsearch failed on tags

I created a customised Document model in wagtail admin by
class CustomizedDocument(Document):
...
And i have updated settings.WAGTAILDOCS_DOCUMENT_MODEL.
However, i realised my search on tags fails. I suspect there is something to do with elasticsearch. But i m really new to that.
Here is the tracestack message from elasticsearch
[INFO ][o.e.n.Node ] initialized
[INFO ][o.e.n.Node ] [oceLPbj] starting ...
[INFO ][o.e.t.TransportService ] [oceLPbj] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}
[INFO ][o.e.c.s.ClusterService ] [oceLPbj] new_master {oceLPbj}{oceLPbjSQ7ib2pTjx9gpPg}{WDDcwdlISnu-EW8mjQcOxQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[INFO ][o.e.h.n.Netty4HttpServerTransport] [oceLPbj] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}
[INFO ][o.e.n.Node ] [oceLPbj] started
[INFO ][o.e.g.GatewayService ] [oceLPbj] recovered [4] indices into cluster_state
[INFO ][o.e.c.r.a.AllocationService] [oceLPbj] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[wagtail__wagtaildocs_document][1], [wagtail__wagtaildocs_document][3]] ...]).
[INFO ][o.e.c.m.MetaDataCreateIndexService] [oceLPbj] [wagtail__wagtaildocs_document_awv9f81] creating index, cause [api], templates [], shards [5]/[1], mappings []
[INFO ][o.e.c.m.MetaDataMappingService] [oceLPbj] [wagtail__wagtaildocs_document_awv9f81/CzKReMnsQ9qziLnOQQ5K3g] create_mapping [wagtaildocs_abstractdocument_wagtaildocs_document_distributor_portal_customizeddocument]
[INFO ][o.e.c.m.MetaDataMappingService] [oceLPbj] [wagtail__wagtaildocs_document_awv9f81/CzKReMnsQ9qziLnOQQ5K3g] create_mapping [wagtaildocs_abstractdocument_wagtaildocs_document]
Could anyone help me find out what's going on here? Any help is appreciated. Let me know what else information i should provide.

Run Elasticsearch 5.2 on 2 machines but in the same cluster

i want to run Elasticsearch on two servers in the same cluster.
The Problem is : I can't connect the 2 servers on elasticsearch. I mean i can't get the 2 nodes on the same cluster.
can somebody tell me if my Configuration in the elasticsearch.yml is right and
Sever 1:
cluster.name: MyData
node.name: Node_1
network.host: '192.160.122.4'
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.160.122.4","192.160.122.3"]
Server2:
cluster.name: MyData
node.name: Node_2
network.host: '192.160.122.3'
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.160.122.4","192.160.122.3"]
what i need to change it ?
Thanks
to the Logs:
Log of server 1:
[2017-03-21T10:38:18,859][INFO ][o.e.n.Node ] [Node_1]
initialized
[2017-03-21T10:38:18,906][INFO ][o.e.n.Node ] [Node_1]
starting ...
[2017-03-21T10:38:19,764][INFO ][o.e.t.TransportService ] [Node_1]
publish_address {192.160.122.4:9300}, bound_addresses {192.160.122.4:9300}
[2017-03-21T10:38:19,764][INFO ][o.e.b.BootstrapChecks ] [Node_1] bound
or publishing to a non-loopback or non-link-local address, enforcing
bootstrap checks
[2017-03-21T10:38:22,899][INFO ][o.e.c.s.ClusterService ] [Node_1]
new_master {Master-Node_1}{4ZEftg6TRCOJqE0kEv-Mrg}{MYrWUFABQ5OOT0US73j13w}
{192.160.122.4}{192.160.122.4:9300}, reason: zen-disco-elected-as-master
([0]
nodes joined)
[2017-03-21T10:38:22,950][INFO ][o.e.h.HttpServer ] [Node_1]
publish_address {192.160.122.4:9200}, bound_addresses {192.160.122.4:9200}
[2017-03-21T10:38:22,997][INFO ][o.e.n.Node ] [Node_1] started
[2017-03-21T10:38:23,948][INFO ][o.e.g.GatewayService ] [Node_1]
recovered [5] indices into cluster_state
[2017-03-21T10:38:32,357][INFO ][o.e.c.r.a.AllocationService] [Node_1]
Cluster health status changed from [RED] to [YELLOW] (reason: [shards
started [[node_1][4]] ...]).
Log of Server2:
[2017-03-21T11:55:57,277][INFO ][o.e.n.Node ] [Node_2] initialized
[2017-03-21T11:55:57,293][INFO ][o.e.n.Node ] [Node_2] starting ...
[2017-03-21T11:56:01,099][INFO ][o.e.t.TransportService ] [Node_2] publish_address {192.160.122.3:9300}, bound_addresses {192.160.122.3:9300}
[2017-03-21T11:56:01,115][INFO ][o.e.b.BootstrapChecks ] [Node_2] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-03-21T11:56:01,115][ERROR][o.e.b.Bootstrap ] [Node_2] node validation exception
bootstrap checks failed
JVM is using the client VM [Java HotSpot(TM) Client VM] but should be using a server VM for the best performance
[2017-03-21T11:56:01,146][INFO ][o.e.n.Node ] [Node_2] stopping ...
[2017-03-21T11:56:01,193][INFO ][o.e.n.Node ] [Node_2] stopped
[2017-03-21T11:56:01,193][INFO ][o.e.n.Node ] [Node_2] closing ...
[2017-03-21T11:56:01,240][INFO ][o.e.n.Node ] [Node_2] closed

org.elasticsearch.action.NoShardAvailableActionException: null

Before x-pack plugin installed,My ElasticSearch is health;
After x-pack plugin installed,some security error occured,
so,I'm setting
xpack.security.enabled: false
in my elasticsearch.yml and kibana.yml, and then restarting elasticsearch successed.
But sometime the following errors occured.
deMacBook-Pro:bin yyq$ ./elasticsearch
[2017-02-16T11:44:15,349][INFO ][o.e.n.Node ] [yml-es-node-1] initializing ...
[2017-02-16T11:44:15,542][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-1] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [9.1gb], net total_space [232.6gb], spins? [unknown], types [hfs]
[2017-02-16T11:44:15,543][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-1] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-02-16T11:44:15,665][INFO ][o.e.n.Node ] [yml-es-node-1] node name [yml-es-node-1], node ID [hqVAcDOdRu-LXuAgUEX9oQ]
[2017-02-16T11:44:15,670][INFO ][o.e.n.Node ] [yml-es-node-1] version[5.2.0], pid[18949], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Mac OS X/10.11.6/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_25/25.25-b02]
[2017-02-16T11:44:21,007][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [aggs-matrix-stats]
[2017-02-16T11:44:21,008][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [ingest-common]
[2017-02-16T11:44:21,008][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-expression]
[2017-02-16T11:44:21,008][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-groovy]
[2017-02-16T11:44:21,008][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-mustache]
[2017-02-16T11:44:21,008][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-painless]
[2017-02-16T11:44:21,008][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [percolator]
[2017-02-16T11:44:21,009][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [reindex]
[2017-02-16T11:44:21,009][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [transport-netty3]
[2017-02-16T11:44:21,009][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [transport-netty4]
[2017-02-16T11:44:21,010][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded plugin [x-pack]
[2017-02-16T11:44:28,308][INFO ][o.e.n.Node ] [yml-es-node-1] initialized
[2017-02-16T11:44:28,309][INFO ][o.e.n.Node ] [yml-es-node-1] starting ...
[2017-02-16T11:44:28,688][INFO ][o.e.t.TransportService ] [yml-es-node-1] publish_address {127.0.0.1:9301}, bound_addresses {[fe80::1]:9301}, {[::1]:9301}, {127.0.0.1:9301}
[2017-02-16T11:44:31,856][INFO ][o.e.c.s.ClusterService ] [yml-es-node-1] detected_master {yml-es-node-2}{5flQkMYSR-ab1dgNA6coog}{fdX0iBHhSzG3Mrri_5ooNA}{127.0.0.1}{127.0.0.1:9300}, added {{yml-es-node-2}{5flQkMYSR-ab1dgNA6coog}{fdX0iBHhSzG3Mrri_5ooNA}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-receive(from master [master {yml-es-node-2}{5flQkMYSR-ab1dgNA6coog}{fdX0iBHhSzG3Mrri_5ooNA}{127.0.0.1}{127.0.0.1:9300} committed version [1]])
[2017-02-16T11:44:31,881][INFO ][o.e.h.HttpServer ] [yml-es-node-1] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}
[2017-02-16T11:44:31,882][INFO ][o.e.n.Node ] [yml-es-node-1] started
[2017-02-16T11:44:32,853][INFO ][o.e.l.LicenseService ] [yml-es-node-1] license [854880ad-c73b-4bdc-8813-918c125d442b] mode [trial] - valid
[2017-02-16T11:47:32,980][INFO ][o.e.m.j.JvmGcMonitorService] [yml-es-node-1] [gc][184] overhead, spent [261ms] collecting in the last [1s]
[2017-02-16T11:48:37,797][DEBUG][o.e.a.s.TransportSearchAction] [yml-es-node-1] All shards failed for phase: [query]
org.elasticsearch.action.NoShardAvailableActionException: null
at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:115) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:154) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:53) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:173) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:145) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:87) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.search.TransportMultiSearchAction.executeSearch(TransportMultiSearchAction.java:113) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.search.TransportMultiSearchAction.doExecute(TransportMultiSearchAction.java:88) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.search.TransportMultiSearchAction.doExecute(TransportMultiSearchAction.java:41) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:149) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:173) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:145) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:87) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:75) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:64) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.client.support.AbstractClient.multiSearch(AbstractClient.java:560) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.rest.action.search.RestMultiSearchAction.lambda$prepareRequest$0(RestMultiSearchAction.java:78) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.rest.action.search.RestMultiSearchAction$$Lambda$1847/1830071645.accept(Unknown Source) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:82) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:162) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.http.HttpServer.dispatchRequest(HttpServer.java:115) [elasticsearch-5.2.0.jar:5.2.0]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:515) [transport-netty4-5.2.0.jar:5.2.0]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:70) [transport-netty4-5.2.0.jar:5.2.0]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:66) [transport-netty4-5.2.0.jar:5.2.0]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:76) [transport-netty4-5.2.0.jar:5.2.0]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:527) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:481) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.7.Final.jar:4.1.7.Final]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_25]
anyone can help me?
Delete data directory and restarting elasticsearch,kibana,elasticsearch-head,show some error messages.
deMacBook-Pro:bin yyq$ ./elasticsearch
[2017-02-17T10:56:37,391][INFO ][o.e.n.Node ] [yml-es-node-1] initializing ...
[2017-02-17T10:56:37,875][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-1] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [4.6gb], net total_space [232.6gb], spins? [unknown], types [hfs]
[2017-02-17T10:56:37,875][INFO ][o.e.e.NodeEnvironment ] [yml-es-node-1] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-02-17T10:56:37,879][INFO ][o.e.n.Node ] [yml-es-node-1] node name [yml-es-node-1], node ID [nIE4QoyTQ9yh8Nz_vidq5A]
[2017-02-17T10:56:37,888][INFO ][o.e.n.Node ] [yml-es-node-1] version[5.2.0], pid[45986], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Mac OS X/10.11.6/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_25/25.25-b02]
[2017-02-17T10:56:42,151][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [aggs-matrix-stats]
[2017-02-17T10:56:42,160][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [ingest-common]
[2017-02-17T10:56:42,161][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-expression]
[2017-02-17T10:56:42,161][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-groovy]
[2017-02-17T10:56:42,162][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-mustache]
[2017-02-17T10:56:42,162][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [lang-painless]
[2017-02-17T10:56:42,162][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [percolator]
[2017-02-17T10:56:42,164][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [reindex]
[2017-02-17T10:56:42,165][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [transport-netty3]
[2017-02-17T10:56:42,166][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded module [transport-netty4]
[2017-02-17T10:56:42,167][INFO ][o.e.p.PluginsService ] [yml-es-node-1] loaded plugin [x-pack]
[2017-02-17T10:56:49,011][INFO ][o.e.n.Node ] [yml-es-node-1] initialized
[2017-02-17T10:56:49,012][INFO ][o.e.n.Node ] [yml-es-node-1] starting ...
[2017-02-17T10:56:49,531][INFO ][o.e.t.TransportService ] [yml-es-node-1] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}
[2017-02-17T10:56:55,760][INFO ][o.e.c.s.ClusterService ] [yml-es-node-1] detected_master {yml-es-node-2}{0got1rmqQX-HcD9Wp3uEUA}{2C-_m1pkQv-YKzAPs3v5JA}{127.0.0.1}{127.0.0.1:9301}, added {{yml-es-node-2}{0got1rmqQX-HcD9Wp3uEUA}{2C-_m1pkQv-YKzAPs3v5JA}{127.0.0.1}{127.0.0.1:9301},}, reason: zen-disco-receive(from master [master {yml-es-node-2}{0got1rmqQX-HcD9Wp3uEUA}{2C-_m1pkQv-YKzAPs3v5JA}{127.0.0.1}{127.0.0.1:9301} committed version [1]])
[2017-02-17T10:56:55,797][INFO ][o.e.h.HttpServer ] [yml-es-node-1] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}
[2017-02-17T10:56:55,797][INFO ][o.e.n.Node ] [yml-es-node-1] started
[2017-02-17T10:56:56,701][INFO ][o.e.l.LicenseService ] [yml-es-node-1] license [1e32f145-313f-4129-b3f2-666bbf279cc5] mode [trial] - valid
------------------------------------------
deMacBook-Pro:bin yyq$ ./kibana
log [02:57:14.219] [info][status][plugin:kibana#5.2.0] Status changed from uninitialized to green - Ready
log [02:57:14.302] [info][status][plugin:elasticsearch#5.2.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [02:57:14.327] [info][status][plugin:xpack_main#5.2.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [02:57:14.545] [info][status][plugin:graph#5.2.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [02:57:14.554] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml
log [02:57:14.558] [info][status][plugin:reporting#5.2.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [02:57:14.594] [info][status][plugin:searchprofiler#5.2.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [02:57:14.603] [info][status][plugin:tilemap#5.2.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [02:57:14.613] [info][status][plugin:console#5.2.0] Status changed from uninitialized to green - Ready
log [02:57:14.853] [info][status][plugin:timelion#5.2.0] Status changed from uninitialized to green - Ready
log [02:57:14.859] [info][listening] Server running at http://localhost:5601
log [02:57:14.860] [info][status][ui settings] Status changed from uninitialized to yellow - Elasticsearch plugin is yellow
log [02:57:19.563] [info][status][plugin:xpack_main#5.2.0] Status changed from yellow to yellow - No existing Kibana index found
log [02:57:19.565] [info][status][plugin:graph#5.2.0] Status changed from yellow to yellow - No existing Kibana index found
log [02:57:19.565] [info][status][plugin:reporting#5.2.0] Status changed from yellow to yellow - No existing Kibana index found
log [02:57:19.567] [info][status][plugin:searchprofiler#5.2.0] Status changed from yellow to yellow - No existing Kibana index found
log [02:57:19.568] [info][status][plugin:tilemap#5.2.0] Status changed from yellow to yellow - No existing Kibana index found
log [02:57:19.568] [info][status][plugin:elasticsearch#5.2.0] Status changed from yellow to yellow - No existing Kibana index found
log [02:57:49.583] [error][status][plugin:xpack_main#5.2.0] Status changed from yellow to red - Unable to create Kibana index ".kibana"
log [02:57:49.584] [error][status][plugin:graph#5.2.0] Status changed from yellow to red - Unable to create Kibana index ".kibana"
log [02:57:49.584] [error][status][plugin:reporting#5.2.0] Status changed from yellow to red - Unable to create Kibana index ".kibana"
log [02:57:49.587] [error][status][plugin:searchprofiler#5.2.0] Status changed from yellow to red - Unable to create Kibana index ".kibana"
log [02:57:49.590] [error][status][plugin:tilemap#5.2.0] Status changed from yellow to red - Unable to create Kibana index ".kibana"
log [02:57:49.593] [error][status][plugin:elasticsearch#5.2.0] Status changed from yellow to red - Unable to create Kibana index ".kibana"
log [02:57:49.594] [error][status][ui settings] Status changed from yellow to red - Elasticsearch plugin is red
log [02:57:52.120] [error][status][plugin:xpack_main#5.2.0] Status changed from red to red - Elasticsearch is still initializing the kibana index.
log [02:57:52.120] [error][status][plugin:graph#5.2.0] Status changed from red to red - Elasticsearch is still initializing the kibana index.
log [02:57:52.121] [error][status][plugin:reporting#5.2.0] Status changed from red to red - Elasticsearch is still initializing the kibana index.
log [02:57:52.122] [error][status][plugin:searchprofiler#5.2.0] Status changed from red to red - Elasticsearch is still initializing the kibana index.
log [02:57:52.123] [error][status][plugin:tilemap#5.2.0] Status changed from red to red - Elasticsearch is still initializing the kibana index.
log [02:57:52.124] [error][status][plugin:elasticsearch#5.2.0] Status changed from red to red - Elasticsearch is still initializing the kibana index.
log [03:00:45.436] [warning][process] UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): Error: reply interface called twice
at emitPendingUnhandledRejections (internal/process/promises.js:57:27)
at runMicrotasksCallback (internal/process/next_tick.js:61:9)
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)
error [03:00:45.465] TypeError: Cannot read property 'toJSON' of undefined
at withXpackInfo (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/plugins/x-pack/plugins/xpack_main/server/lib/replace_injected_vars.js:5:32)
at replaceInjectedVars$ (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/plugins/x-pack/plugins/xpack_main/server/lib/replace_injected_vars.js:10:12)
at tryCatch (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:61:40)
at GeneratorFunctionPrototype.invoke [as _invoke] (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:328:22)
at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:94:21)
at invoke (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:136:37)
at callInvokeWithMethodAndArg (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:172:16)
at previousPromise (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:194:19)
at AsyncIterator.enqueue (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:193:13)
at AsyncIterator.prototype.(anonymous function) [as next] (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:94:21)
at Object.runtime.async (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/regenerator/runtime.js:215:14)
at replaceInjectedVars (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/plugins/x-pack/plugins/xpack_main/server/lib/replace_injected_vars.js:3:22)
at /Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/src/ui/index.js:67:22
at undefined.next (native)
at step (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/src/ui/index.js:9:273)
at /Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/src/ui/index.js:9:443
at /Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/src/ui/index.js:9:99
at tryCatcher (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/bluebird/js/main/util.js:26:23)
at ReductionPromiseArray._promiseFulfilled (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/bluebird/js/main/reduce.js:109:18)
at ReductionPromiseArray.init (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/bluebird/js/main/promise_array.js:92:18)
at ReductionPromiseArray.init (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/bluebird/js/main/reduce.js:42:10)
at Async._drainQueue (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/bluebird/js/main/async.js:128:12)
at Async._drainQueues (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/bluebird/js/main/async.js:133:10)
at Immediate.Async.drainQueues (/Users/yyq/Documents/es/kibana-5.2.0-darwin-x86_64/node_modules/bluebird/js/main/async.js:15:14)
-------------------------------------
deMacBook-Pro:elasticsearch-head-master yyq$ grunt server
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on http://localhost:9100

Resources