ElasticsearchSecurityException: invalid configuration for xpack.security.transport.ssl - elasticsearch

I'm getting the following error when trying to spin up an elasticsearch cluster in EC2.
org.elasticsearch.ElasticsearchSecurityException: invalid configuration for xpack.security.transport.ssl - [xpack.security.transport.ssl.enabled] is not set, but the following settings have been configured in elasticsearch.yml : [xpack.security.transport.ssl.keystore.secure_password,xpack.security.transport.ssl.truststore.secure_password]
This is the script I'm using to bootstrap the cluster
"#!/bin/bash",
"sudo yum update -y",
"sudo yum install java-1.8.0 -y",
"sudo rpm -i https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-${var.elastic_version}-x86_64.rpm",
"sudo systemctl daemon-reload",
"sudo systemctl enable elasticsearch.service",
"sudo chmod -R 777 /etc/elasticsearch",
"sudo sed -i 's#-Xms1g#-Xms${aws_instance.elastic_datanodes[count.index].root_block_device[0].volume_size / 2}g#g' /etc/elasticsearch/jvm.options",
"sudo sed -i 's#-Xmx1g#-Xmx${aws_instance.elastic_datanodes[count.index].root_block_device[0].volume_size / 2}g#g' /etc/elasticsearch/jvm.options",
# "sudo sed -i 's/#network.host: 192.168.0.1/network.host: 0.0.0.0/g' /etc/elasticsearch/elasticsearch.yml",
"sudo rm /etc/elasticsearch/elasticsearch.yml",
"sudo cp elasticsearch.yml /etc/elasticsearch/elasticsearch.yml",
"sudo systemctl start elasticsearch.service"
This is my elasticsearch yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: ${cluster_name}
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ${node_name}
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: ${node}
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts:
%{ for seed_host in seed_hosts ~}
- "${seed_host}"
%{ endfor ~}
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes:
%{ for master_node in initial_master_nodes ~}
- "${master_node}"
%{ endfor ~}
node.roles: [data, master]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
Why is this happening? I can probably get around this by disabling all security settings, but I'd rather not do that for obvious reasons.

Tldr;
I believe you need to set
xpack.security.transport.ssl.enabled: true
In your elasticsearch.yml file.

Related

How to solve elasticsearch No alive nodes found in your cluster?

When I research some text my service response No alive nodes found in your cluster. Also service show "Type: Elasticsearch\Common\Exceptions\NoNodesAvailableException"
my elasticsearch.yml file is below:
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
How can I solve this problem? I use php and client builder. Actually it was worked before but my when disc is full I got an error.(Watermark) So, I decrease disc size but now. Cluster not found.

Elasticsearch not running when I try to edit the elasticsearch.yml file

elasticsearch.bat file opens and closes instantly when I try to add something to the elasticsearch.yml file. -- Version 7.4
I've tried adding the following lines in elasticsearch.yml file:
action.auto_create_index=false
index.mapper_dynamic=false
It works fine when everything is commented with '#' though. This is the configuration file for elasticsearch:--
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
and I'm trying to enter my lines at the last here (without the #)
action.auto_create_index=false
index.mapper_dynamic=false
I got the exception like this:
Caused by: java.lang.IllegalArgumentException: the [action.auto_create_index] setting value [false] is too restrictive. disable [action.auto_create_index] or set it to [.watches,.triggered_watches,.watcher-history-*]
it means that some Watcher need to create index when ES start, so you can set it in config file.
action.auto_create_index: ".watches,.triggered_watches,.watcher-history-*"
About index.mapper_dynamic, it's a index level config, now ES only allow set it by using PUT index/_settings

Can't access Port 9200 Remotely

Have just started using elastic search 5.4.0, first i have installed it on Mac(local) and then in server.
In Mac when i am running in browser as 'http://localhost:9200' it is showing me output :
{
"name" : "Node_1",
"cluster_name" : "bhujang_bipl",
"cluster_uuid" : "N75mGOiSQCufpBl_2B7jSg",
"version" : {
"number" : "5.4.0",
"build_hash" : "780f8c4",
"build_date" : "2017-04-28T17:43:27.229Z",
"build_snapshot" : false,
"lucene_version" : "6.5.0"
},
"tagline" : "You Know, for Search"
}
which means i am connected to 9200 which is elasticsaerch.
But when i am running it on server by http://192.168.1.126:9200 , then it is not getting connected and shows me an output as :page not found !!
I disabled firewall on both the server and client as solutions I got from internet but couldn't get it working.
My elasticsaerch.yml file is
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#network.host:192.168.1.126
#
# Set a custom port for HTTP:
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
#The default list of hosts is ["127.0.0.1", "[::1]","103.240.195.45","103.240.195.45"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 3
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
Can anyone help me with what can be the best possible mistake from my side ?
From the link
https://github.com/elastic/elasticsearch/issues/19987
add this two line in the yml file
transport.host: localhost
network.host: 192.168.1.126
It doesn't work for me remotely, it's hosted on Azure windows 2019 server. I have opened port ranges 9200-9300
transport.host: localhost
transport.tcp.port: 9300
http.port: 9200
#network.host: 0.0.0.0
network.host: 192.168.1.126
http.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin : "*"
http.cors.allow-methods : OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers : X-Requested-With,X-Auth-Token,content-Type,content-Length

Two nodes not joining Elasticsearch cluster

I have an elasticsearch cluster in google cloud consisting of 3 nodes in which one is a master node. Now i don't know why my two nodes are not joining cluster only one node is in cluster right now. I used below command to start/stop/restart the other two nodes
sudo service elasticsearch start
when i ran above command for the two nodes there were no error shown but now when i try to use below command to retrieve my indexed data i got error on both of the nodes.
curl 'http://localhost:9200/merchants/_search?pretty=true' -d '{
"query" : {
"match_all" : { }
}
}'
**curl: (7) Failed to connect to localhost port 9200: Connection refused**
my elasticsearch.yml file is given below
script.inline: on
script.indexed: on
script.engine.groovy.inline.aggs: on
script.groovy.sandbox.enabled: false
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elasticsearch-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: "elasticsearch-1-1-vm"
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# bootstrap.mlockall: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["10.128.0.5", "10.128.0.6", "10.128.0.4"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true
Can anyone please tell me what can be the best way to restart the elasticsearch cluster, i think my sudo elasticsearch start command is not working properly otherwise i must not have gotten connection refused error. Please if anyone know anything about ES cluster please help me i am stuck in this problem from past one week and havn't gotten anything useful till know.

Elasticsearch 2.0 can see data in Indices from browser tab

I have Elasticsearch 2.0 running on windows 7 local machine, connected to Mysql DB waiting on new entries and all works fine. I am using a cluster with 3 nodes. Cluster Health is green.
Elasticsearch
when I click on the browser tab and then on the Indices I want to look at it seem to be empty even though it says there are 2 documents?
when I open the Any Request tab and do a search for the Indices "audit-2015.11.20" it returns the data? have I to configure something else?
Elasticsearch location on windows is C:\elasticsearch-2.0.0
I start each Elasticsearch individually from cmd like so: C:\elasticsearch-2.0.0\elasticsearch-2.0.0-Master\bin>elasticsearch changing the name for each.
Am I forgetting to do something. My configurations file for elasticsearch-2.0.0-Master is as follows.
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: AppTransport
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# bootstrap.mlockall: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind adress to a specific IP (IPv4 or IPv6):
#
# network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Elasticsearch nodes will find each other via unicast, by default.
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1", "127.0.0.1:9201", "127.0.0.1:9202"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true
on the other Elasticsearch yaml configuration file I have just changed the Node node.name the Network http.port numbers and Discovery discovery.zen.ping.unicast.hosts.
so node 2 is http.port: 9201 and discovery is discovery.zen.ping.unicast.hosts: ["127.0.0.1", "127.0.0.1:9201", "127.0.0.1:9202"] and node 3 is http.port: 9202 and discovery is discovery.zen.ping.unicast.hosts: ["127.0.0.1", "127.0.0.1:9201", "127.0.0.1:9202"].
Thanks in advance
G
There is an open issue for elasticsearch-head. It does not fully work with the current 2.x versions for now.
One of the issues is the following: https://github.com/mobz/elasticsearch-head/issues/217

Resources