I have found some answers to this question on StackOverflow but those are a bit old. I need to configure multiple instances with the latest version (7.9.3) of elasticsearch and couldn't find any answers which are compatible with that.
Step 01
Set environment variable on windows
ES_HOME = ../elasticsearch
ES_PATH_CONF = ../elastcisearch/config
Step 02
created elasticsearch_1.yml and elastcsearch_2.yml config files
Steo 03
run the elasticsearch.bat with the config file
bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.1.yml
then the following error will be thrown
ERROR: D is not a recognized option
Note - I know that the above approach is a bit old and how to do the above steps with the current (7.9) version
No need to give -D option and don't create two config file in a single ES installation folder.
Instead, just change the elasticsearch.config in two ES installation folder and run them on separate ports.
ES-1 elasticsearch.config
http.port: 9200
ES-2 elasticsearch.config
http.port: 9500
And after this, simply start both the node, one by one from using bin/elasticsearch command.
Related
I tried two hosters. Massivegrid and Unispace.
On both I created a new environment.
Then choose docker install. Choose elasticsearch.
After it finished, I took the url ipv4 address and pasted on the browser with port as 9200 and nothing popped up.
So I went into terminal and netstat told me there was no elastic service running. In fact nothing related to elastic was installed. No /etc/elasticsearch.
I used this tutorial https://docs.jelastic.com/elasticsearch
I failed on "Connection via Public IP" part.
Wondering what am I missing.
Edit: I created another new enviorment and installed Elasticsearch 6.8.1 image and this works as per tutorial. The newer range from 7.0 onwards is all blank inside. No Java, no elasticsearch etc.
In order to fix the problem with version 7.2 the next steps can be taken:
Add the following string in the /usr/local/bin/docker-entrypoint.sh file:
ulimit -n 65536
And in the elasticsearch.yml file (/usr/share/elasticsearch/config/elasticsearch.yml) please add:
cluster.initial master_nodes: node-1
After that restart the container.
Correct setting for elasticsearch.yml should look like the following:
cluster.initial_master_nodes: node-1
ElasticSearch 6.2.2 on Linux Ubuntu 16.04.3 VM in Azure. It had been up and running fine and then after I rebooted the machine a few days ago I could not get the ElasticSearch service to start at all. Issue was shared and solved here: (ElasticSearch Fails to Start on Ubuntu 16.04.3 - status=1 Failure) by increasing the heap size in the jvm.options file.
Now I have the ElasticSearch service running but I cannot ping it at all. I have tried to ping it from both inside the VM (as localhost:9200) and from outside, (similar to how I make calls to our other ES boxes, and do so successfully) but I'm told Could Not Get Any Reponse (Postman syntax).
The part that is making this impossible to diagnose is nothing is getting written to the ElasticSearch logs! The last time anything was written to any log at /var/log/elasticsearch was before I rebooted the machine a couple days ago.
I have checked the settings in elasticsearch.yml and all seems to be in-line with the elasticsearch.yml that's on a different box of ours in a different location which runs another ElasticSearch instance of ours without any issue.
EDIT: per request - the elasticsearch.yml file from the box that is NOT working correctly is here: http://s000.tinyupload.com/index.php?file_id=72318548245343478927 For comparison purposes, the elasticsearch.yml file from the box that IS working correctly is here: http://s000.tinyupload.com/index.php?file_id=20127693354114612595 Please note that the one that IS working correctly has 3 nodes whereas the one that is not working has only one node, so there will be some slight differences between the yml files because of this.
Check if path.logs: /var/log/elasticsearch is defined in elasticsearch.yml. Add this line if not present.
Check whether the user has permission to write into /var/log/elasticsearch. Change the permission of the files. sudo chmod 777 /var/log/elasticsearch/* and sudo chmod 777 /var/log/elasticsearch
Open /etc/init.d/elasticsearch and check whether ES_PATH_CONF is defined as ES_PATH_CONF="/etc/elasticsearch"
You may try commenting the following lines on log4j2.properties under /etc/elasticsearch.
logger.xpack_security_audit_logfile.name = org.elasticsearch.xpack.security.audit.logfile.LoggingAuditTrail
logger.xpack_security_audit_logfile.level = info
logger.xpack_security_audit_logfile.appenderRef.audit_rolling.ref = audit_rolling
logger.xpack_security_audit_logfile.additivity = false
Use netstat -nultp | grep 9200 and check whether the port is being listened to.
The issue was with the line in the ElasticSearch.yml file which showed as
"10.5.11.6""
That extra quotation mark at the end is what was causing the entire problem.
For anyone that this can benefit, the ElasticSearch.yml file is extremely sensitive when it comes to space, punctuation and case: even an extra space somewhere can cause the entire service to crash. Be very diligent with your edits to elasticsearch.yml.
There are ways to debug:
1. Check if you have ES service running on that particular host via `ps -ef | grep elastic`
2. Look on which port es is listening (or not) ? via netstat
3. it might be a case that your es is running and but is binding not to localhost but to the instance IP . You should be getting the hint on the elasticsearch.yaml
4. Make sure your /usr/share/elasticsearch/elasticsearch.yaml is the file that is being picked up and not the default at /etc/elasticsearch.yaml
5. Configure logging in elasticsearch.yaml to the location
Hope this helps?
Maybe I am thick, but I can't seem to find a way to pass ES a config file path from the command line. I have been searching and reading for 45 mins now (including several posts on Stack Overflow), and none of the proposed solutions works.
Here are the ones I tried:
elasticsearch -Des.config=/path/to/my/elasticsearch.yml
==> ERROR: D is not a recognized option
elasticsearch -Ees.config=/path/to/my/elasticsearch.yml
==> org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [es.config] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
elasticsearch -Econfig=/path/to/my/config.yml
==> org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [config] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
elasticsearch -Epath.conf=/path/to/config/dir/with/elasticsearch.yml
==> No exception, but the program terminates without any output whatsoever (no error message). Since I didn't specify the -d option, I am assuming that it's not running as a daemon and that therefore, the ES server is not running by the end of that.
Can anyone pull me out of the mud here?
Thx.
I too struggled with the same issue and tried the same sort of commands as you did. The problem here is caused due to the version of elastic search.
If your version is above 5.0.0 and as per this none of the above commands will work. Also it looks like they have limited the types of parameters that can be passed from the command line.
The easiest way is to just cd to the directory you installed elasticsearch and then just ./bin/elasticsearch (Make sure you don't execute it as root, it doesn't allow you to run as root.)
The issue here is that after every new version of ES, some older functionality gets removed/updated which is frustrating. I'm currently working with Elasticsearch v6.4.0 and as for now this works.
I am following these link:http://programming-tips.in/kafka-set-up-apache-kafka-on-windows/ and https://dzone.com/articles/running-apache-kafka-on-windows-os for Apache Kafka setup on a local machine.
After successfully running zookeeper for which my screen looks like:
My server.properties file looks like:
log.dirs="C:\Program Files\kafka_2.11-0.11.0.0\kafka-logs"
listeners=PLAINTEXT://:9092
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
My zookeeper.properties file looks like:
dataDir="C:\Program Files\kafka_2.11-0.11.0.0\zookeeper-data"
clientPort=2181
But, when I am trying to run the following command:
C:\Program Files\kafka_2.11-0.11.0.0>\bin\windows\kafka-server-start.bat \config\server.properties
Although, I have give all the paths correct which I have verified many times, I get this error:
The system cannot find the path specified.
What could be the possible solution?
Kafka on Windows currently doesn't support spaces in the install path. Try moving your Kafka folder so the path does not contain spaces. This issue is reported in a JIRA, and should be fixed in one of the upcoming releases.
The problem seems to be with leading '\' in your path, the following should work
C:\Program Files\kafka_2.11-0.11.0.0>bin\windows\kafka-server-start.bat config\server.properties
I'm trying to start the Redis cluster servers by turning on 6 servers from ports 7000 to 7005, each with a redis.conf in their own directories on my macOS Sierra. I can start the first server fine (either of the 6) and here's that output and the info in the cli: Here's an example of one of these commands I run, using redis 3.2.1
redis-server /private/etc/redis-3.2.1/src/7002/redis.conf
but starting another would give this error:
11245:M 06 Mar 22:45:22.536 * Increased maximum number of open files to 10032 (it was originally set to 7168).
11245:M 06 Mar 22:45:22.537 # Sorry, the cluster configuration file nodes.conf is already used by a different Redis Cluster node. Please make sure that different nodes use different cluster configuration files.
Following the docs, I have each redis.conf configured to this with their corresponding port numbers
port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
This used to work for me. I don't know for certain if it's related, but since then I have built these files into Docker images and containers. However, as far as I can tell I have deleted them, and also this file: /Users/MyUserAccount/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux
I also just deleted all the directories and recreated them in a different directory, but still it does not work. What can I do to get these ports available for Redis Cluster again?
UPDATE:
Also, my nodes.conf file is not being recreated in any of the port folders, and all of them only has the redis.conf file. Before when it worked, there was a nodes.conf file generated with 2 other files (I think a dump file and one other one).
Looks like a nodes.conf is being generated from wherever I call redis-server from, and I am able to start the servers if I cd into the different directories. This seems kind of inconvenient since before I just had a script that called from a single location the redis.conf at their absolute paths. But at least I have some solution.