Elasticsearch 1.7.2 on CentOS, 8GB RAM
We see that ES_HEAP_SIZE should be increased to 4g.
The only place this seems declared in the ES environment is in /etc/init.d/elasticsearch
We set it to 4g in this init file, restarted ES, but the jvm "heap_max_in_bytes" (as returned from /_nodes/stats ) did not move from the default 1g value.
Where and how can we get control of ES_HEAP_SIZE ?
(I should add: The similar looking threads here on SO are either dated [e.g. apply to earlier versions of ES and do not apply to 1.7.x] or are for other platforms [win, osx], or are do not work [have tried them, and you can see many of the responses are tagged 'this is a hack don't do it'])
(I should further note that the ES docs document this element, and suggest what to set it to, but do not instruct how or where.)
Note: Below is for Elasticsearch 1.7.x. For 5.3 and higher, it is different.
Per a comment that is rather buried on How to change Elasticsearch max memory size
On CentOS /etc/sysconfig/elasticsearch is the appropriate place to make these changes.
This has been tested and verified on my CentOS 7 environment. Strongly expect it to also fly on CentOS 6.
Related
I have an elastic search cluster running 5.6. I plan to upgrade my cluster but i plan to do it by basically running a ES cluster 8.3 running in parallel and then moving data over to it.
The preferred way i think is to do snapshot and restore https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html
But I am confused about what exactly Snapshot version compatibility mean :
In the above does it mean that if i take a snapshot in elasticsearch 5.6, I cannot restore directly in version 8.3 ?? (which mean I have to first move to 6.x then to 7.x and finally to 8.x ) ??
The below index compatibility matrix, however says that a version in 5.x will work in 8.x ?
Am i missing something ? or can someone help me elaborate this?
So, the underlying problem is that data written in Lucene version N, can only be read with N+1. For Elasticsearch 5 to 8 the Lucene version was always 1 greater than the ES version (so 6 to 9).
That means, both for an upgrade or a restored snapshot: If your data was written with 5.x, you can only read / restore it with 6.x. For 7.x or 8.x you'll need to reindex the data. I would do a remote reindex straight from 5.x to 8.latest if possible: https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html
There are some small caveats but they will probably not apply to you:
This doesn't apply to source only snapshots, but those always need a reindex, so that's not going to add any benefit for you.
8.3 added a feature to still read snapshots from 5.0 on but it is slower, doesn't support all features, and it is a commercial feature (platinum license if I'm not mistaken).
Depending on what kind of data it is: If it's aging out (like logs or metrics), maybe you don't have to migrate it to the new cluster?
We have a Linux machine with 56GB of hard disk, but querying the max available space of ES shows only 28GB. We would like the ES to use more of the available space on the machine, but could not find the configuration for this. Thanks.
Elasticsearch uses all the space that is available on your machine. However if this is a single machine in your cluster, then no copies of data will be created (as it does only make sense in a distributed environment to be failsafe friendly). So if you dont have more data, then no more is used.
It might make sense to talk about your setup and provide more information.
I've started using Liferay v7, and am getting a lot of the following log messages:
17:14:12,265 WARN [elasticsearch[Mirage][management][T#1]][decider:157] [Mirage] high disk watermark [90%] exceeded on [fph02E6ISIWnZ5cxWw_mow][Mirage][/Users/randy/FasterPayments/src/eclipse/com.rps.portal/com.rps.portal.backoffice/bundles/data/elasticsearch/indices/LiferayElasticsearchCluster/nodes/0] free: 46gb[9.9%], shards will be relocated away from this node
To be honest, I'd rather not spend time learning about ElasticSearch right now, is it possible to simply disable ElasticSearch within Liferay 7 dev environment? Or other action to remove these log messages?
Go to Control Panel / Configuration / System Settings / Foundation / Elasticsearch.
Under "Additional Configurations" enter
cluster.routing.allocation.disk.threshold_enabled: True
cluster.routing.allocation.disk.watermark.low: 30gb
cluster.routing.allocation.disk.watermark.high: 20gb
or whatever are appropriate values for your system (there is value in being warned that the disk is almost full).
Save & Restart (the values seem not to be picked up at runtime).
Liferay needs an index/search engine, say ElasticSearch or SOLR. It defaults to ElasticSearch in DXP. It makes no sense disabling it.
The warnings tell you you've reached your configured disk shared allocation. You can change this settings in your elasticSearch.yml (cluster.routing.allocation.disk.watermark.high).
If your logs annoy you, you can change your logging settings. Not sure If it's still valid in DXP, but have a look at https://dev.liferay.com/es/discover/deployment/-/knowledge_base/6-2/liferays-logging-system.
Im running ElasticSearch 1.3.2 on Centos 6.5 with the terminal like so:
bin/elasticsearch -console
My server has 16GB of RAM. How do I give 8GB of it to ES?
This post may have the answer but I just couldnt piece it together. Further the docs have only confused me more...
Its good practice to should keep 50% memory for Elasticsearch Heap Size. As you mentioned you don't have service wrapper. In this case you can assign heap size while starting elasticsearch services.
bin/elasticsearch ES_HEAP_SIZE=8000m
How to make elasticsearch apply new configuration?
I changed one string in file ~ES_HOME/config/elasticsearch.yml:
# Disable HTTP completely:
#
http.enabled: false
Then tried to reload elasticsearch:
elasticsearch reload
Then tried to restart elasticsearch:
elasticsearch restart
Then checked and see that http requests are still acceptable to elastic search.
So my settings are not applied.
My os is os X. ElasticSearch version is 1.2.0
Strangely or not so, the supposed way to do it is just to stop the service, and start it again :)
I.E. get its pid (running ps axww | grep elastic), and then kill ESpid ; just be sure to use the TERM signal, to give it a chance to close properly.
Some *nix elasticsearch distros have control scripts wrappers for start/stop , but I don't think OS X does.
And on a side note, you have probably found the Cluster Update Settings API, and though it provides quite a few options, regretfully it can't be used to change that particular setting.
HTH
P.S. And yep, in Windows setup the services.msc is the way to do it, but doubt this is helpful for you :)
When you have installed the current version (7.4 at the time of writing) of Elasticsearch on macOS with Homebrew, you can run:
brew services restart elastic/tap/elasticsearch-full
This will restart Elasticsearch and reload te configuration.
If ElasticSearch is installed on Windows setup then you have to restart ElasticSearch windows service.
Thanks.