I'm running Elasticsearch on a non-production RHEL6 server. I only have a regular user account with no root access. I'm in a very locked-down corporate environment so getting root will be time-consuming and I need a work-around.
When I start the process I get these errors:
max file descriptors [8192] for elasticsearch process is too low, increase to at least [65536]
max number of threads [1024] for user [salimfadhley] is too low, increase to at least [2048]
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
Is there a way to make ElasticSearch ignore this error condition and just start up.
I'm fully aware that ignoring errors is normally considered unwise, however on this occasion I just need to get ES up and running so that I can devote my attention to other aspects of this project: Getting the system limits raised will take more time than I currently have available.
#ThomasDecaux is correct that you can technically disable bootstrap. But you can only disable bootstrap checks sometimes. Here is when you can disable bootstrap checks:
configured your elasticsearch.yml with discovery.type: single-node
or your N nodes are all using zen discovery on localhost
If you need to run N nodes on N machines as one cluster, then no you can not disable bootstrap checks.
When you configure your elasticsearch.yml to use an external interface and you don't have discovery.type: single-node then bootstrap checks can not be disabled. I tried.
I had a machine failing the bootstrap tests but I didn't have sudo permission to fix it. I tried to disable checks by passing the -Des.enforce.bootstrap.checks=false to the Java JVM options but the bootstrap checks were still enabled.
Here is a github issue from 2018 where the developers say you can't disable bootstrap checks https://github.com/elastic/elasticsearch/issues/31933 :
" There is no command line option for disabling bootstrap checks. The
es.enforce.bootstrap.checks option is used to enable them when they
are disabled due to Elasticsearch not detecting that it is being used
in production (single node, only reachable through localhost or using
single-node discovery). "
Yes you can!
(I found) this is very dirty, but if you configure discovery.type as single-node, no bootstrap check will run.
Yes, that means you cannot test a cluster in your laptop.
See https://github.com/elastic/elasticsearch/issues/21655
Related
Using Elasticsearch sink connector to insert data to ES 7.2 instance hosted on VM.
Getting this : Elastic search max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
Is it possible to ignore bootstrap checks ?
How can I increase the virtual memory for Elasticsearch in the docker container ?
Bootstrap checks inspect a variety of Elasticsearch and system settings. If youre in development mode, any bootstrap checks that fail appear as warnings in the Elasticsearch log. If youre in production mode failed bootstrap will cause Elasticsearch to refuse to start.
Elasticsearch work mode is configured implicit. As soon as you configure a network setting like network.host,Elasticsearch assumes that you are moving to production and will upgrade the above warnings to exceptions.
Regarding your specific case you need to increase it on the host machine not the docker by running this command:
sudo sysctl -w vm.max_map_count=262144 . and then restart your docker-containers.
BTW not recommended but if you are running a single node you can skip the bootstrap checks by not binding transport to an external interface or by binding transport to an external interface and setting the discovery type to single-node.
As suggested by https://crate.io/docs/crate/guide/en/latest/deployment/cloud/aws/ec2-setup.html. I've been trying to setup a CrateDb (3.0.2) cluster. However setting network.host: _site_ prevents the node from starting. Setting everything back to default settings of course starts the instance but this is bound to localhost.
All I'm trying to do is to evaluate and set up CrateDB cluster. Sadly the error received isn't useful as it seems to complain about heap size (on a fresh Windows install with 8GB ram and empty database). Does anyone know how to set up a production instance ?
are you installing it on amazon web services or on windows? the documentation above is for AWS. here's the correct documentation: https://crate.io/docs/crate/guide/en/latest/scaling/multi-node-setup.html
I've started using Liferay v7, and am getting a lot of the following log messages:
17:14:12,265 WARN [elasticsearch[Mirage][management][T#1]][decider:157] [Mirage] high disk watermark [90%] exceeded on [fph02E6ISIWnZ5cxWw_mow][Mirage][/Users/randy/FasterPayments/src/eclipse/com.rps.portal/com.rps.portal.backoffice/bundles/data/elasticsearch/indices/LiferayElasticsearchCluster/nodes/0] free: 46gb[9.9%], shards will be relocated away from this node
To be honest, I'd rather not spend time learning about ElasticSearch right now, is it possible to simply disable ElasticSearch within Liferay 7 dev environment? Or other action to remove these log messages?
Go to Control Panel / Configuration / System Settings / Foundation / Elasticsearch.
Under "Additional Configurations" enter
cluster.routing.allocation.disk.threshold_enabled: True
cluster.routing.allocation.disk.watermark.low: 30gb
cluster.routing.allocation.disk.watermark.high: 20gb
or whatever are appropriate values for your system (there is value in being warned that the disk is almost full).
Save & Restart (the values seem not to be picked up at runtime).
Liferay needs an index/search engine, say ElasticSearch or SOLR. It defaults to ElasticSearch in DXP. It makes no sense disabling it.
The warnings tell you you've reached your configured disk shared allocation. You can change this settings in your elasticSearch.yml (cluster.routing.allocation.disk.watermark.high).
If your logs annoy you, you can change your logging settings. Not sure If it's still valid in DXP, but have a look at https://dev.liferay.com/es/discover/deployment/-/knowledge_base/6-2/liferays-logging-system.
I have two different machines running elastic search server instances. They automatically create a cluster and changes made on one instance reflect on other instance on different machine. I have changed the cluster.name property in elasticsearch.yml file in config folder and the issue is resolved. I wanted to know if i can start elastic search server instance in non-cluster mode ?
You can't start the es server in non-cluster mode.
But if you want the two servers to run independently (in its own cluster), there are 2 options that I can think of:
Disable multicast and don't set the hosts for them in unicast
Change the cluster.name to make them have different names
The easiest is to set node.local: true
This prevents elasticsearch from trying to connect to other nodes.
Using a custom name is also a good idea in any case just to prevent unintended exchange of data. Use something else for production, testing, and development.
Can somebody provide some instructions on how to configure ElasticSearch for replication. I am running ES in windows and understand that if I run the bat files multiple times on the same server, a separate instance of ES is started and they all connect to each other.
I will be moving to a production environment soon and will have a three node set up, each node being on a different server. Can someone point me at some documentation which gives me a bit more control of the replication set up.
Have a look at the discovery documentation. It works out-of-the-box with multicast discovery, even though you could have problems with firewalls etc., but I would recommend against it in production. I would rather use unicast and configure the host names of the nodes belonging to the cluster in the elasticsearch.yml. That way you make sure nobody is going to join the production cluster from his own machine.
One other thing I would do is configure a proper cluster name specific for every environment.
Replication is set to each index in Elasticsearch, not set to the server or node. That is, each index can have different number of replication setting. The number of replica setting is 1 by default.
The number of replication is not related or restricted to number of node set up. If the number of replication is greater than the number of data nodes, only the index health becomes yellow since some replications are not allocated, anything still works fine.
You can reference to the document for further information: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html