I'm attempting to test elastic replication and install multiple elastic instances on the same host.
I've created an additional elastic search configuration file and set the following config property:
http.port: 9500
The other elastic search configuration file contains the default value :
http.port: 9200
I attempt to start elastic using :
./bin/elasticsearch -Ees.config=./config/elasticsearch.yml
but receive error :
uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [es.config] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
How to utilize 2 elasticsearch instances on the same host ?
Is there an alternative to the es.config parameter ?
ES_PATH_CONF=/path/to/my/config ./bin/elasticsearch
This is the way to do it according to the documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html (depending on the version you are using, it might differ).
I would recommend using a docker setup for this endeavour as described here (official elasticsearch documentation) https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
As #Alkis Kalogeris already stated, I would also recommend using docker/docker-compose. You would just need to expose a different port to your localhost.
Related
Looking at the logs in one of the filebeat pods i can see this:
2021-01-04T10:10:52.754Z DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:129 add_cloud_metadata: fetchMetadata ran for 2.351101ms
2021-01-04T10:10:52.754Z INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:93 add_cloud_metadata: hosting provider type detected as openstack, metadata={"ava
ilability_zone":"us-east-1c","instance":{"id":"i-08f536567bd9945df","name":"ip-10-101-2-178.ec2.internal"},"machine":{"type":"m5.2xlarge"},"provider":"openstack"}
2021-01-04T10:10:52.755Z DEBUG [processors] processors/processor.go:120 Generated new processors: add_cloud_metadata={"availability_zone":"us-east-1c","instance":{"id":"i-08f5
36567bd9945df","name":"ip-10-101-2-178.ec2.internal"},"machine":{"type":"m5.2xlarge"},"provider":"openstack"}, add_docker_metadata=[match_fields=[] match_pids=[process.pid, process.ppid]]
2021-01-04T10:10:52.755Z INFO instance/beat.go:392 filebeat stopped.
2021-01-04T10:10:52.755Z ERROR instance/beat.go:956 Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (pat
h.data).
Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).
as you can see the filebeat stopped with an error :
data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).
After searching the problem in github/forum i found this :
https://discuss.elastic.co/t/data-path-already-locked-by-another-beat/219852/4
Which looks like my problem,
Im using the default filebeat-kubernetes.yaml , and there is no information in ELK / Filebeats docs on how to add unique paths in the filebeat-kubernetes.yaml
where do i add them and how do i make them unique?
Thanks
I had the same problem. It means that your data path (/var/lib/filebeats) are locked by another filebeat instance. So execute sudo systemctl stop filebeat (in my case) to ensure that you don't have running filebeat
and then run filebeat with sudo filebeat -e which prints logs in console
I also tried link, that you shared, but it didn't help me. Here another solutions, may be it would help you: https://discuss.elastic.co/t/data-path-already-locked-by-another-beat/219852/2
In addition to #Anton's answer, In one of the scenarios, I had a lock file in the data path. This could be /var/lib/filebeat/filebeat.lock depending on the configuration. Delete the file and run sudo filebeat -e
If you want to run Elastic stack as a service, the solution is just to restart all of the stack in this order:
Elasticsearch
Kibana
Logstash
Filebeat(s)
which is already suggested in this link.
I have found some answers to this question on StackOverflow but those are a bit old. I need to configure multiple instances with the latest version (7.9.3) of elasticsearch and couldn't find any answers which are compatible with that.
Step 01
Set environment variable on windows
ES_HOME = ../elasticsearch
ES_PATH_CONF = ../elastcisearch/config
Step 02
created elasticsearch_1.yml and elastcsearch_2.yml config files
Steo 03
run the elasticsearch.bat with the config file
bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.1.yml
then the following error will be thrown
ERROR: D is not a recognized option
Note - I know that the above approach is a bit old and how to do the above steps with the current (7.9) version
No need to give -D option and don't create two config file in a single ES installation folder.
Instead, just change the elasticsearch.config in two ES installation folder and run them on separate ports.
ES-1 elasticsearch.config
http.port: 9200
ES-2 elasticsearch.config
http.port: 9500
And after this, simply start both the node, one by one from using bin/elasticsearch command.
I tried two hosters. Massivegrid and Unispace.
On both I created a new environment.
Then choose docker install. Choose elasticsearch.
After it finished, I took the url ipv4 address and pasted on the browser with port as 9200 and nothing popped up.
So I went into terminal and netstat told me there was no elastic service running. In fact nothing related to elastic was installed. No /etc/elasticsearch.
I used this tutorial https://docs.jelastic.com/elasticsearch
I failed on "Connection via Public IP" part.
Wondering what am I missing.
Edit: I created another new enviorment and installed Elasticsearch 6.8.1 image and this works as per tutorial. The newer range from 7.0 onwards is all blank inside. No Java, no elasticsearch etc.
In order to fix the problem with version 7.2 the next steps can be taken:
Add the following string in the /usr/local/bin/docker-entrypoint.sh file:
ulimit -n 65536
And in the elasticsearch.yml file (/usr/share/elasticsearch/config/elasticsearch.yml) please add:
cluster.initial master_nodes: node-1
After that restart the container.
Correct setting for elasticsearch.yml should look like the following:
cluster.initial_master_nodes: node-1
Hello I am starting work with kibana and elasticsearch. I am being able to run elasticsearch at port 9200 but kibana is not running at port 5601. The following two images are given for clarification
Kibana is not running and showing the page is not available
Kibana doesn't support space in the folder name. Your folder name is
GA Works
Remove the space between those two words kibana will then run without errors and you will be able to access at
http://localhost:5601
You can rename the folder with
GA_Works
Have you
a) Set the elasticsearch_url to point at your Elasticsearch instance in file kibana/config.yml?
b) Run ./bin/kibana (or bin\kibana.bat on windows) (after setting the above config)
?
If you tried all of the above and still it doesn't work make sure that the kibana process is running first. I found that /etc/init.d/kibana4_init doesn't start the process. If that is the case then try: opt/kibana/bin/kibana.
I also made kibana user:group owner of the folder/files.
I am facing problems with Elasticsearch.
I am unable to get the results. I checked in log files i got the following error:
ERROR:
2014-10-30 08:52:46,971][DEBUG][action.search.type ] [Lianda] [135] Failed to execute fetch phase
[Error: Runtime.getRuntime().exec("cd").getInputStream(): Cannot run program "cd": java.io.IOException: error=2, No such file or directory]
[Near : {... w InputStreamReader(Runtime.getRuntime().exec("cd" ....}]
Below are the version I am using:
elastic search version: 0.90.5
java version: 1.6.0_33 64 bit
plugin installed: phonetic
The strange thing is that, whenever I am getting this error, I restart the elastic search server and its works.
So I think something is getting overloaded.
Based on seeing the Runtime.getRuntime().exec() call, it could be related to a dynamic scripting vulnerability in the defaults of elastic search prior to version 1.2. See this document on scripting security.
If that is the source of your problem, you can put in a fix in your current version (or upgrade to a newer version). From the link above:
If you are running an Elasticsearch node prior to the 1.2.x release,
you can make this change on your system by putting the following
setting into elasticsearch.yml:
script.disable_dynamic: true
Then restart each node in your cluster. Dynamic scripting will now be
disabled. If you are running Elasticsearch 1.2.x or later, dynamic
scripting is already disabled by default.