I'm trying to fire up an instance of elasticsearch and then an instance of kibana (which needs to wait until ES is up) using a script. I can't just do ./bin/elasticseach && ./bin/kibana or something similar to that because the first script runs in the foreground which means the second command wont run. What's the best way I can do this while ensuring kibana only starts when ES is up and running?
If you have no way to tell when ES is up, I can only suggest:
./bin/elasticseach & sleep 10 && ./bin/kibana
Where you have to guesstimate in how much time it will be ready
Assuming ./bin/elasticsearch blocks the command line until it is 'up', you can just use a ';' between the commands to run them one after the other.
./bin/elasticseach; ./bin/kibana
But since elasticsearch just blocks the command line until stopped, you could do something else. You can run it like a daemon, so it doesn't block the command line. (Here is the documentation about starting and stopping es)
./bin/elasticseach -d -p PID; ./bin/kibana; kill `cat PID`
Related
I've downloaded and set up elasticsearch on an EC2 instance that I use to run Jenkins. I'd like to use Jenkins to run some unit tests that use the local elasticsearch.
My problem is that I haven't found a way on how to start the elasticsearch locally and run the tests after, since the script doesn't proceed after starting ES, because the job is not killed or anything.
I can do this by starting ES manually through SSH and then building a project with only the unit tests. However, I'd like to automate the ES launching.
Any suggestions on how I could achieve this? I've tried now using single "Execute shell" block and two "Execute shell" blocks.
It is happening because you starting elasticsearch command in blocking way. It means command will wait until elasticsearch server is shutdown. Jenkins just keep waiting.
You can use following command
./elasticsearch 2>&1 >/dev/null &
or
nohup ./elasticsearch 2>&1 >/dev/null &
it will run command in non-blocking way.
You can also add small delay to allow elasticsearch server start
nohup ./elasticsearch 2>&1 >/dev/null &; sleep 5
Im running my logstash config file in Ubuntu using the following command.
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
Its working, However I recently realized that every time I run this command it starts another instance. Now I think there are six instances running. Because each new record i create shows as six in elasticsearch.
How can I stop all these other instances and is there any way to check how many are running?
Thanks
You can use the pkill command and specify the name of the process(es) you want to kill
pkill logstash
Or the killall command works as well the same way
killall logstash
As Val states, pkill should work to resolve what you are facing.
To avoid this in future why don't you create a small service file so which uses a PID file so you can't have multiple instances running? Here is what I did:
http://www.logstashbook.com/code/3/logstash-central.init
Ive got a script that takes a quite a long time to run, as it has to handle many thousands of files. I want to make this script as fool proof as possible. To this end, I want to check if the user ran the script using nohup and '&'. E.x.
me#myHost:/home/me/bin $ nohup doAlotOfStuff.sh &. I want to make 100% sure the script was run with nohup and '&', because its a very painful recovery process if the script dies in the middle for whatever reason.
How can I check those two key paramaters inside the script itself? and if they are missing, how can I stop the script before it gets any farther, and complain to the user that they ran the script wrong? Better yet, is there way I can force the script to run in nohup &?
Edit: the server enviornment is AIX 7.1
The ps utility can get the process state. The process state code will contain the character + when running in foreground. Absence of + means code is running in background.
However, it will be hard to tell whether the background script was invoked using nohup. It's also almost impossible to rely on the presence of nohup.out as output can be redirected by user elsewhere at will.
There are 2 ways to accomplish what you want to do. Either bail out and warn the user or automatically restart the script in background.
#!/bin/bash
local mypid=$$
if [[ $(ps -o stat= -p $mypid) =~ "+" ]]; then
echo Running in foreground.
exec nohup $0 "$#" &
exit
fi
# the rest of the script
...
In this code, if the process has a state code +, it will print a warning then restart the process in background. If the process was started in the background, it will just proceed to the rest of the code.
If you prefer to bailout and just warn the user, you can remove the exec line. Note that the exit is not needed after exec. I left it there just in case you choose to remove the exec line.
One good way to find if a script is logging to nohup, is to first check that the nohup.out exists, and then to echo to it and ensure that you can read it there. For example:
echo "complextag"
if ( $(cat nohup.out | grep "complextag" ) != "complextag" );then
# various commands complaining to the user, then exiting
fi
This works because if the script's stdout is going to nohup.out, where they should be going (or whatever out file you specified), then when you echo that phrase, it should be appended to the file nohup.out. If it doesn't appear there, then the script was nut run using nohup and you can scold them, perhaps by using a wall command on a temporary broadcast file. (if you want me to elaborate on that I can).
As for being run in the background, if it's not running you should know by checking nohup.
I'm creating a pallet crate for elasticsearch. I was stuck on the service not starting however after looking at the logs it seems that it's not really anything to do with pallet. I am using the elasticsearch apt package for 1.0 which includes an init script. If I run sudo service elasticsearch start then ES starts with no problems. If pallet does this for me then it records standard out as having started it successfully
start elasticsearch
* Starting Elasticsearch Server
...done.
However it is not started.
sudo service elasticsearch status
* elasticsearch is not running
I messed around with the init script and I found if I added sleep 1 after starting the daemon then it works correctly with pallet.
start-stop-daemon --start -b --user "$ES_USER" -c "$ES_USER" --pidfile "$PID_FILE" --exec $DAEMON -- $DAEMON_OPTS
#this sleep will allow it to work
#sleep 1
log_end_msg $?
I don't understand what is going on?
I've seen issues like this before, too. It generally comes down to expecting something to have finished before the script finishes, which may not always happen with services since they fork off background tasks that may still get killed when the ssh connection is terminated.
For these kinds of things you should use Pallet's built in code for running things under supervision. This also has the advantage of making it very easy to switch from plain init.d to runit or daemontools later, which is especially useful for Elasticsearch because it's a JVM process and nearly any JVM will eventually crash if you let it run long enough.
I have a program that runs as a daemon, using the C command fork(). It creates a new instance that runs in the background. The main instance exists after that.
What would be the best option to check if the service is running? I'm considering:
Create a file with the process id of the program and check if it's running with a script.
Use ps | grep to find the program in the running proccess list.
Thanks.
I think it will be better to manage your process with supervisord, or other process control system.
Create a cron job that runs every few minutes (or whatever you're comfortable with) and does something like this:
/path/to/is_script_stopped.sh && /path/to/script.sh
Write is_script_stopped.sh using any of the methods that you suggest. If your script is stopped cron will evaluate your script, if not, it won't.
To the question, you gave in the headline:
This simple endless loop will restart yourProgram as soon as it fails:
#!/bin/bash
for ((;;))
do
yourProgram
done
If your program depends on a resource, which might fail, it would be wise to insert a short pause, to avoid, that it will catch all system resources when failing million times per second:
#!/bin/bash
for ((;;))
do
yourProgram
sleep 1
done
To the question from the body of your post:
What would be the best option to check if the service is running?
If your ps has a -C option (like the Linux ps) you would prefer that over a ps ax | grep combination.
ps -C yourProgram