Puppet init script doesn't create the pid file? - bash

CentOS release 5.4 (Final)
puppet-server-2.7.19-1.el5 is installed from the puppetlabs repo.
puppetmaster is started successfull, but it doesn't create the pid file. It is the reason for [ FAILED ] message when stopping:
/etc/init.d/puppetmaster stop
Stopping puppetmaster: [FAILED]
The init script: http://fpaste.org/nsfI/
The /etc/rc.d/init.d/functions library: http://fpaste.org/ox5Q/
And this is what I get when running in the debug mode: http://fpaste.org/DkoS/
I know the way to echo the pid to a file manually after starting, but why doesn't daemon function's --pidfile work?
daemon $PUPPETMASTER $PUPPETMASTER_OPTS --masterport=${PUPPETMASTER_PORTS[$i]} --pidfile=/var/run/puppet/puppetmaster.${PUPPETMASTER_PORTS[$i]}.pid
Sure, Puppet master is running as puppet user:
ps -ef | grep [p]uppet
puppet 23418 1 0 18:13 ? 00:00:00 /usr/bin/ruby /usr/sbin/puppetmasterd
and the owner of /var/run/puppet/ folder is puppet:
# ls -ld /var/run/puppet/
drwxr-xr-x 2 puppet puppet 4096 Sep 17 18:46 /var/run/puppet/

It is up to the controlled program (in this case puppetmasterd), not the daemon() function, to create the pidfile; daemon() relies on this.
Confirm where puppetmasterd creates its pidfile (it could be /var/run/puppet.pid, /var/lib/puppet/run/master.pid, etc.) To find out, inspect the contents of puppetmasterd (if a script), or kill puppetmasterd then strace -f puppetmasterd 2>&1 | grep '\.pid'.
Modify the value of pidfile in your /etc/init.d/puppetmaster accordingly.

So, it looks to me like there could be a couple of possibilities here:
If you are trying to use the --pidfile option of the daemon command I believe you have a syntax problem.
Red Hat's daemon command has the following (unhelpful) signature: Usage: daemon [+/-nicelevel] {program}. What isn't altogether clear is that anything that you include after the program location is treated as an option passed to the program, not to the daemon function call.
So, in your case you are passing the --pidfile argument to $PUPPETMASTER itself as opposed to daemon(). You could remedy this by using the following: daemon --pidfile=/var/run/puppet/puppetmaster.${PUPPETMASTER_PORTS[$i]}.pid $PUPPETMASTER $PUPPETMASTER_OPTS --masterport=${PUPPETMASTER_PORTS[$i]}
The second option here is that $PUPPETMASTER (or rather, the program behind it) might daemonize itself, and if so, could be responsible for creating its own .pid file. The process management tool Circus works in this way. It's probably an option in a configuration file or for the program representing by $PUPPETMASTER.
I'm not a Puppet user, so I won't be able to help you with the specifics here. But I would look into the Puppet labs documentation to find out more about this option.
It is important to note if $PUPPETMASTER is daemonizing itself, then the --pidfile argument being passed to daemon() will have no effect.
Good hunting!

Related

Bash script runs but no output on main commands and not executed

I'm setting a cron job that is a bash script containing the below:
#!/bin/bash
NUM_CONTAINERS=$(docker ps -q | wc -l)
if [ $NUM_CONTAINERS -lt 40 ]
then
echo "Time: $(date). Restart containers."
cd /opt
pwd
sudo docker kill $(docker ps -q)
docker-compose up -d
echo "Completed."
else
echo Nothing to do
fi
The output is appended to a log file:
>> cron.log
However the output in the cron file only shows:
Time: Sun Aug 15 10:50:01 UTC 2021. Restart containers.
/opt
Completed.
Both command do not seem to execute as I don't see any change in my containers either.
These 2 non working commands work well in a standalone .sh script without condition though.
What am I doing wrong?
User running the cron has sudo privileges, and we can see the second echo printing.
Lots of times, things that work outside of cron don't work within cron because the environment is not set up in the same way.
You should generally capture standard output and error, to see if something going wrong.
For example, use >> cron.log 2>&1 in your crontab file, this will capture both.
There's at least the possibility that docker is not in your path or, even if it is, the docker commands are not working for some other reason (that you're not seeing since you only capture standard output).
Capturing standard error should help out with that, if it is indeed the issue.
As an aside, I tend to use full path names inside cron scripts, or set up very limited environments at the start to ensure everything works correctly (once I've established why it's not working correctly).

Starting service as non root user

Can someone help us understand how to properly start our programs service as the services user (marty for example).
We're using init.d to start our process (java application), but when the system(s) boot (Ubuntu and Debian) because the service script is run as root, we're having problems where the application is starting as root too and the PID file is being created by root which is messing things up.
We tried using sudo, but this is not a great solution as we dont want the sudo process running too with our application as a child process plus we need this to work on other systems that may not have sudo. Please help.
In the init script, you can check the $UID of the calling user.
If it is root, you can run the service with "runuser". If it is marty - run it directly, if it is another user - exit with error for example.
Here's some example bash (untested):
start() {
if [ $UID -eq 0 ]; then
runuser -s /bin/bash marty -c "$DAEMON start $DAEMONOPTS"
elif [ "$USER" = "marty" ]; then
$DAEMON start $DAEMONOPTS
else
print "Please run me with root or marty."
exit 2
fi
}
Same for stop and any other functions as required.
Feel free to modify the runuser command as necessary, maybe you won't need the shell for example.
Use start-stop-daemon which accepts a user name and an executable as parameters.

how do I get etcd values into my systemd service on coreOS?

I have two services A and B.
A sets a value in etcd as it's being started, say the public IP address which it gets from an environment file:
ExecStartPost=/usr/bin/etcdctl set /A_ADDR $COREOS_PUBLIC_IPV4
B needs that value as it starts up, as well as its own IP address. So something like this would be nice:
ExecStart=/usr/bin/docker run -e MY_ADDR=$COREOS_PUBLIC_IPV4 -e A_ADDR=$ETCD_A_ADDR mikedewar/B
but that's obviously not possible as etcd variables don't present as systemd environment variables like that. Instead I can do some sort of /usr/bin/bash -c 'run stuff' in my ExecStart but it's awkward especially as I need systemd to expand $COREOS_PUBLIC_IPV4 and my new bash shell to expand $(etcdctl get /A_ADDR). It also reeks of code smell and makes me think I'm missing something important.
Can someone tell me the "right" way of getting values from etcd into my ExecStart declaration?
-- update
So I'm up and running with
ExecStart=/usr/bin/bash -c 'source /etc/environment && /usr/bin/docker run -e A_ADDR=$(/usr/bin/etcdctl get /A_ADDR) -e MY_ADDR=$COREOS_PUBLIC_IPV4 mikedewar/B'
but it's pretty ugly. Still can't believe I'm not missing something..
I've was struggling with the same thing until recently. After reading much of the documentation of CoreOS and systemd, here is a slightly 'cleaner' version of what you're doing:
[Service]
EnvironmentFile=/etc/environment
ExecStart=/bin/sh -c '/usr/bin/docker run -e A_ADDR=$(/usr/bin/etcdctl get /A_ADDR) -e MY_ADDR=$COREOS_PUBLIC_IPV4 mikedewar/B'
Additionally, I have adopted a pattern where my services depend on a systemd 'oneshot' service that will compute some value and write it in to /etc/environment. This allows you to keep more complex shell scripting out of the main service unit and place it into it's own oneshot service unit.
Here are the docs for EnvironmentFile: http://www.freedesktop.org/software/systemd/man/systemd.exec.html#EnvironmentFile=
Finally, a quick gotchya: you must use a shell invocation if you use any variable in your ExecStart/Stop commands. systemd does no shell invocation when executing the command you provide, so variables will not be expanded.
I am currently using such a workaround:
I've created scripts which extracts data from particular etcd directory
#! /bin/sh
for entry in `etcdctl ls /my_dir --recursive` ; do
echo ' -e '`grep -o '[^/]*$' <<< ${entry}`=`etcdctl get ${entry}`
done
its output looks following:
-e DATABASE_URL=postgres://m:m#mi.cf.us-.rds.amazonaws.com:5432/m
-e WEB_CONCURRENCY=4
So then eventually I can in my init file place that in such way
/bin/sh -c '/usr/bin/docker run -p 9000:9000 $(/home/core/envs.sh) me/myapp -D FOREGROUND'
It's not the most elegant way, and I'd love to know how to improve it, but placing that for loop as a one-liner requires lots of escaping.
Can you container read directly from etcd as it starts, over the docker0 bridge IP, instead of passing in the values? This will also allow you to do more complex logic on the response, parse JSON if you are storing it as the etcd value, etc.

at command is not executed

I am trying to make the at bash command to run, but it is not working for me. I can add a job to the queue, but it is not run when the time is up. What am I doing wrong?
hpek#melda:~$ cat /usr/lib/cron/at.deny
hpek#melda:~$ atq
hpek#melda:~$ at now +2 minutes
echo TTTEEEST
job 12 at Sun May 6 02:09:00 2012
hpek#melda:~$ date
Sun May 6 02:10:24 CEST 2012
hpek#melda:~$ atq
12 Sun May 6 02:09:00 2012
hpek#melda:~$
UPD2021.08.06 atd was removed in MacOS 11.5.2 (or earlier)
$ sudo find / -name atd -print 2>/dev/null
$
It is working fine. It's just that commands running with at don't write their output to the terminal that you called it from.
Try:
at now +2 minutes
echo TTTEEEST > new_file_test
You'll see the file appear in two minutes.
Take a look at /var/at/jobs and see if your at jobs are listed there. (It may be a different directory based upon OS).
By default, at isn't enabled on most systems. In order for at jobs to actually get executed, the atrun command must execute.
This command is executed either through launchd or through the cron depending upon the system.
The exact mechanisms are different from system to system, so you'll have to read all the various manpages on at, atrun, etc. to verify if at is really enabled on your system, and whether you have permissions to run at jobs. There's normally both an ant allow and an ant deny file on your system, so you need to check both. You must be both in the allowed file, and also not in the deny file.
On top of that, you have to make sure that at is even enabled on your system (due to security concerns, it is usually disabled).
Check your mail:
An at - or batch - command invoked from a su(1) shell will
retain the current userid. The user will be mailed standard
error and standard output from his commands, if any. Mail
will be sent using the command /usr/sbin/sendmail. If at is
executed from a su(1) shell, the owner of the login shell
will receive the mail.
I've found this here
First make sure that the at daemon is running using a command like this:
# ps -ef | grep atd
root 8231 1 0 18:10 ? 00:00:00 /usr/sbin/atd
If you don’t see atd running start it with this command:
# /etc/init.d/atd start
To analyze at(d) execution problems it may help to
Check /etc/at.allow and /etc/at.deny for existance and contents (see man at.allow)
Check owner and access rights of the atd spool directory
(/var/spool/cron/atjobs, daemon:daemon, 770 on Ubuntu)
Check if atd daemon is running (systemctrl -a | grep atd on Ubuntu)
tail -f /var/log/syslog (at execution time)
journalctl -b | grep atd (found that here, contains further hints)
The "Permission denied" error may also be caused by limited number of logins defined (for the at running user) in /etc/security/limits.conf.
I too faced same issue with echo when used along with at command. But if you try to redirect the output using > or >> then it works perfectly. Also try some other system commands like reboot, ls > some_text_file etc to make sure at command is working. Else try
sudo systemctl enable --now atd to enable at daemon
Conclution First:
atrun or atd daemon is not running, causing this problem.
Guideline to solve your case:
check at mannul by man at first.
In my case, I(macos 10.13.6) found following info:
Note that at is implemented through the launchd(8) daemon periodically invoking atrun(8), which is disabled by default. See atrun(8) for information about enabling atrun.
check atrun mannul by man atrun, I found :
The atrun utility runs commands queued by at(1). It is invoked periodically by launchd(8) as specified in the com.apple.atrun.plist property list. By default the property list contains the
Disabled key set to true, so atrun is never invoked.
Execute the following command as root to enable atrun:
launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist

Can Cron Jobs Use Gnome-Open?

I am running Ubuntu 11.10 (Unity interface) and I created a Bash script that uses 'gnome-open' to open a series of web pages I use every morning. When I manually execute the script in the Terminal, the bash script works just fine. Here's a sample of the script (it's all the same so I've shortened it):
#!/bin/bash
gnome-open 'https://docs.google.com';
gnome-open 'https://mail.google.com';
Since it seemed to be working well, I added a job to my crontab (mine, not root's) to execute every weekday at a specific time.
Here's the crontab entry:
30 10 * * 1,2,3,4,5 ~/bin/webcheck.sh
The problem is this error gets returned for every single 'gnome-open' command in the bash script:
GConf-WARNING **: Client failed to connect to the D-BUS daemon:
Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
GConf Error: No D-BUS daemon running
Error: no display specified
I did some searching to try and figure this out. The first thing I tried was relaunching the daemon using SIGHUP:
killall -s SIGHUP gconfd-2
That didn't work so I tried launching the dbus-daemon using this code from the manpage for dbus-launch:
## test for an existing bus daemon, just to be safe
if test -z "$DBUS_SESSION_BUS_ADDRESS" ; then
## if not found, launch a new one
eval `dbus-launch --sh-syntax --exit-with-session`
echo "D-Bus per-session daemon address is: $DBUS_SESSION_BUS_ADDRESS"
fi
But that didn't do anything.
I tried adding simply 'dbus-launch' at the top of my bash script and that didn't work either.
I also tried editing the crontab to include the path to Bash, because I saw that suggestion on another thread but that didn't work.
Any ideas on how I can get this up and running?
Here is how the problem was solved. It turns out the issue was primarily caused by Bash not having access to an X window session (or at least that's how I understood it). So my problem was solved by editing my crontab like so:
30 10 * * 1,2,3,4,5 export DISPLAY=:0 && ~/bin/webcheck.sh
The "export DISPLAY=:0" statement told cron which display to use. I found the answer on this archived Ubuntu forum after searching for "no display specified" or something like that:
http://ubuntuforums.org/archive/index.php/t-105250.html
So now, whenever I'm logged in, exactly at 10:30 my system will automatically launch a series of webpages that I need to look at every day. Saves me having to go through the arduous process of typing in my three-letter alias every time :)
Glad you asked!
It depends on when it is run.
If the Gnome GDM Greeter is live, you can use the DBUS session from the logon dialog, if you will. You can, e.g., use this to send notifications to the logon screen, if no-one is logged in:
function do_notification
{
for pid in $(pgrep gnome-session); do
unset COOKIE
COOKIE="$(grep -z DBUS_SESSION_BUS_ADDRESS /proc/$pid/environ|cut -d= -f2-)"
GNUSER="$(ps --no-heading -o uname $pid)"
echo "Notifying user $GNUSER (gnome-session $pid) with '$#'"
sudo -u "$GNUSER" DBUS_SESSION_BUS_ADDRESS="$COOKIE" /usr/bin/notify-send -c "From CRON:" "$#"
done
unset COOKIE
}
As you can see the above code simply runs the same command (notify-send) on all available gnome-sessions, when called like:
do_notification "I wanted to let you guys know"
You can probably pick this apart and put it to use for your own purposes.

Resources