Background
I have multiple (30+) custom services/timers which have the same structure (below in Contents section), but different timings. They have to be executed only when they are scheduled and cannot be run manually at random times. The application behind these jobs is often updated and during the update these jobs must be disabled/stopped and re-enabled/started-per-schedule after the update.
Question
I've created new systemd timer/service files and placed them under /etc/systemd/system/. How do I schedule this to start for first time based on OnCalendar without manually starting the timer or rebooting the server?
Contents
I have a service file:
# /etc/systemd/system/dummy.service
[Unit]
Description=dummy Service
Wants=dummy.timer
[Service]
Type=oneshot
User=root
ExecStart=/bin/ping -c 30 8.8.8.8
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
And a corresponding timer:
# /etc/systemd/system/dummy.timer
[Unit]
Description=dummy Timer
Requires=dummy.service
[Timer]
Unit=dummy.service
OnCalendar=*-*-* *:00:00
AccuracySec=1ms
RandomizedDelaySec=0
[Install]
WantedBy=timers.target
They are placed in /etc/systemd/system/ and then I run the following commands to enable the timer:
systemctl daemon-reload
systemctl enable dummy.timer
And if I run systemctl list-timers --all 'dummy*', the output is as per below, and the timer never executes..
NEXT LEFT LAST PASSED UNIT ACTIVATES
n/a n/a n/a n/a dummy.timer dummy.service
The only way I can make it start running on its schedule is either by rebooting the server (which is not an option) or by manually starting the timer with systemctl start dummy.timer, which is also not what I need.
I've been searching for solution for quite some time now and all I could find was either start manually or reboot. Having found anything SystemD man pages...
The workaround I have in place is using systemd-analyze calendar to read the next run time, date to convert it to %Y%m%d%H%M format and then schedule it with linux at command. This is far from elegant and has issues with jobs that run once a week or once a month, and the server is rebooted after application update and before scheduled run time.
Any ideas?
Thank you!
Ah, found the answer just after commenting.
The timer is implictly bound to the service of the same name. The Requires= line in the timer definition causes it to run the service. If you remove this line and then enable/start the timer it will only trigger on its schedule. Not at boot, and not when the start is performed, i.e.
systemctl enable --now dummy.timer
Related
I have an opensplice publisher on Ubuntu 20.04 that is started via systemd.
If the publisher starts via systemd then the data is not pubished, but also no errors are reported or present in the opensplice log files.
The publisher works if I run it from a command line or if I stop and restart the service.
The QoS are the same for the publisher and subscriber.
The publisher and subscriber applications are running on different machines.
There are no other participants on the network. All the machines are rebooted and the order of reboot does not change the observed behaviour.
The systemd service is:
[Unit]
Description=Publisher Process
Documentation=
After=network.target
StartLimitIntervalSec=0
[Service]
Type=simple
WorkingDirectory=/opt/publisher/bin
ExecStart=/opt/publisher/bin/publisher.sh
Restart=always
RestartSec=2
[Install]
WantedBy=multi-user.target
The publisher.sh is:
#!/bin/bash
cd /opt/publisher/bin
source bashrc_local
# We just keep running the application (in case of a crash)
while true; do
./publisher
sleep 15
done
I have a work around that feels a little bit naff.
#!/bin/bash
cd /opt/publisher/bin
source bashrc_local
timeout 30 ./remote_processor
killall remote_processor
# We just keep running the application (in case of a crash)
while true; do
./publisher
sleep 15
done
Any ideas on how I can remove my work around?
Edit 16 Sept 22
The issue appears to be systemd start order and dependencies as I have run into the same issue with a program publishing data via UDP which is not using DDS.
Changing the dependencies so the services are started just before the user login does not help.
check your environment variables as systemd will not run with the same environment as your bash console
in particular have you set the OSPL_URI variable to point at the config?
if using the commercial version, OSPL_HOME and ADLINK_LICENSE will also need to be set
Does the PATH variable include your OSPL shared libraries?
These are all setup by running the $OSPL_HOME\release.com script in your bash session
I tend to manually add the required ones to the service file
e.g.
Environment=OSPL_URI=file:///opt/ospl.xml
I have the following script:
whie true
do
#code
sleep 60
done
I then wanted to create a service to start the machine and launch this script as service:
created my.service at /etc/systemd/system/my.service
[Unit]
Description=my Script
[Service]
Type=forking
ExecStart=/bin/script.sh
[Install]
WantedBy=multi-user.target
problem occurs when i systemctl start my.service
it goes to while true loop and hang in there, how can i run this service and make it run in the background ?
According to systemd specification at link. Type=forking is not exactly correct kind of start-up in your case
If set to forking, it is expected that the process configured with
ExecStart= will call fork() as part of its start-up. The parent
process is expected to exit when start-up is complete and all
communication channels are set up. The child continues to run as the
main service process, and the service manager will consider the unit
started when the parent process exits. This is the behavior of
traditional UNIX services. If this setting is used, it is recommended
to also use the PIDFile= option, so that systemd can reliably identify
the main process of the service. systemd will proceed with starting
follow-up units as soon as the parent process exits.
The Type=simple can be correct one. You can try with it
If set to simple (the default if ExecStart= is specified but neither
Type= nor BusName= are), the service manager will consider the unit
started immediately after the main service process has been forked
off. It is expected that the process configured with ExecStart= is the
main process of the service. In this mode, if the process offers
functionality to other processes on the system, its communication
channels should be installed before the service is started up (e.g.
sockets set up by systemd, via socket activation), as the service
manager will immediately proceed starting follow-up units, right after
creating the main service process, and before executing the service's
binary. Note that this means systemctl start command lines for simple
services will report success even if the service's binary cannot be
invoked successfully (for example because the selected User= doesn't
exist, or the service binary is missing).
I would like systemd to manage the tup monitor, so I wrote a service unit:
[Unit]
Description=Monitor source files for changes
[Service]
Type=simple
ExecStart=/usr/bin/tup monitor -f
ExecStop=/usr/bin/tup stop
WorkingDirectory=/some/dir
StandardOutput=journal+console
StandardError=journal+console
[Install]
WantedBy=multi-user.target
Whereas starting the monitor manually works, trying to do it with systemd fails with the following error:
setpgid: Operation not permitted
tup error: Unable to set process group for tup's subprocesses.
The error originates in this file and seems to be due to systemd not giving the process the capabilities it needs, so I went further:
[Unit]
Description=Monitor source files for changes
[Service]
Type=simple
ExecStart=/usr/bin/tup monitor -f
ExecStop=/usr/bin/tup stop
CapabilityBoundingSet=CAP_SETUID CAP_SETGID
AmbientCapabilities=CAP_SETUID CAP_SETGID
WorkingDirectory=/some/dir
StandardOutput=journal+console
StandardError=journal+console
[Install]
WantedBy=multi-user.target
Still, it doesn't work.
How can I tell systemd to keep the permissions tup requires?
systemd does not restrict any permissions on services by default, so I don’t think that’s the problem. The setpgid(2) manpage mentions three possible conditions for EPERM, though:
An attempt was made to move a process into a process group in a different session, or to change the process group ID of one of the children of the calling process and the child was in a different session, or to change the process group ID of a session leader (setpgid(), setpgrp()).
I think the third one is the most likely cause of the error: tup probably expects to be run from a shell, in which case the shell would be the session leader and tup tries to establish its own process group, but systemd places each service in its own session as part of setting up a clean execution environment for the service (see also daemon(7)), so that step fails.
Unless you can somehow disable that step in tup (it doesn’t need to fork, either, or do a number of other things which systemd makes unnecessary), I think the only workaround would be an ugly hack:
ExecStart=/bin/sh -c '/usr/bin/tup monitor -f'
With this, the session leader will be the shell, not tup, so the setpgid should work again.
Try this
[Unit]
Description=Monitor source files for changes
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/tup monitor -f
ExecStop=/usr/bin/tup stop
CapabilityBoundingSet=CAP_SETUID CAP_SETGID
AmbientCapabilities=CAP_SETUID CAP_SETGID
WorkingDirectory=/some/dir
StandardOutput=journal+console
StandardError=journal+console
[Install]
WantedBy=multi-user.target
I've added After=network.target, this will make sure TUP service starts after the network is fully active and avoid the service failing due to network issues
Is there some way to trigger an event (e.g. running a script to push some logs to S3) when an EC2 instance is stopped/terminated?
I have looked into triggering the script using a service in /usr/lib/systemd/system but I haven't had any luck with that yet. I have heard that networking capabilities on the instance can be shutdown before a service is triggered and if true, that could be why the script is not executing correctly.
So the answer is not really AWS specific, but it is working for me now (tested on EC2 instance stopping and terminating).
I've created a system.d service file:
/usr/lib/systemd/system/my_shutdown.service
[Unit]
Description=my_shutdown Service
Before=shutdown.target reboot.target halt.target
Requires=network-online.target network.target
[Service]
KillMode=none
ExecStart=/bin/true
ExecStop=/path/to/my_script.sh
RemainAfterExit=yes
Type=oneshot
[Install]
WantedBy=multi-user.target
Added this service to multi-user.target:
systemctl enable my_shutdown.service
Alternatively you can manually create the symlink:
ln -s /usr/lib/systemd/system/my_shutdown.service /etc/systemd/system/multi-user.target.wants/my_shutdown.service
Started the service and tested by stopping/terminating the instance.
systemctl start my_shutdown.service
My understanding:
Description: a description of our service.
Before: we want our service to stop before these targets are started.
Requires: our service requires that network capabilities are available. These targets must not be stopped before our service starts/stops.
KillMode: none; do not kill our process.
ExecStart: /bin/true; a command that does nothing but returns a success. Run when are service is started.
ExecStop: the script to run. Run when are service is being stopped.
RemainAfterExit: consider our service active even when all its processes exited.
Type: oneshot; it is expected that the process has to exit before systemd starts follow-up units.
WantedBy: the target we want to add our service to.
References:
https://www.freedesktop.org/software/systemd/man/systemd.service.html
https://www.freedesktop.org/software/systemd/man/systemd.kill.html#
https://www.freedesktop.org/software/systemd/man/systemd.special.html
https://www.freedesktop.org/software/systemd/man/systemd.target.html
You can trigger events, such as pushing logs to S3 on specific events, with CloudWatch... Learn more here: https://aws.amazon.com/cloudwatch/
Firstly this is a double post with serverfault, Please advise what is better site for this type of question.
This is my first time with upstart. I can not get the following script to launch when waiting for the network to initiate. It will work if using standard
start on runlevel [2345]
stop on runlevel [!2345]
I have also tried various combinations of
start on filesystem and net-device-up IFACE=eth0
start on filesystem and net-device-up IFACE!=lo
Any suggestions would be grateful
description "test for on network start"
start on (started network-interface
or started network-manager
or started networking)
stop on (stopping network-interface
or stopping network-manager
or stopping networking)
script
touch /home/ec2-user/myFile.txt
end script
the event that signifies that the network is completely configured is static-network-up. You will most likely also need the file system event for any service that uses /home, /var, and /usr.