I want to enable core dump generation for a systemd service which gives a SEGV status and exits.
scd.service: main process exited, code=killed, status=11/SEGV
Unit scd.service entered failed state.
scd.service failed.
my bashrc file already has ulimit -c unlimited.
after running systemd-coredumpctl the output is
No coredumps found.
Note : I am able to generate core dump for normal programs executed via terminal but unable to do so for a systemd service
You need to set LimitCORE=infinity in your service file (scd.service), not your bashrc.
More info in https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Process%20Properties
Related
I have an opensplice publisher on Ubuntu 20.04 that is started via systemd.
If the publisher starts via systemd then the data is not pubished, but also no errors are reported or present in the opensplice log files.
The publisher works if I run it from a command line or if I stop and restart the service.
The QoS are the same for the publisher and subscriber.
The publisher and subscriber applications are running on different machines.
There are no other participants on the network. All the machines are rebooted and the order of reboot does not change the observed behaviour.
The systemd service is:
[Unit]
Description=Publisher Process
Documentation=
After=network.target
StartLimitIntervalSec=0
[Service]
Type=simple
WorkingDirectory=/opt/publisher/bin
ExecStart=/opt/publisher/bin/publisher.sh
Restart=always
RestartSec=2
[Install]
WantedBy=multi-user.target
The publisher.sh is:
#!/bin/bash
cd /opt/publisher/bin
source bashrc_local
# We just keep running the application (in case of a crash)
while true; do
./publisher
sleep 15
done
I have a work around that feels a little bit naff.
#!/bin/bash
cd /opt/publisher/bin
source bashrc_local
timeout 30 ./remote_processor
killall remote_processor
# We just keep running the application (in case of a crash)
while true; do
./publisher
sleep 15
done
Any ideas on how I can remove my work around?
Edit 16 Sept 22
The issue appears to be systemd start order and dependencies as I have run into the same issue with a program publishing data via UDP which is not using DDS.
Changing the dependencies so the services are started just before the user login does not help.
check your environment variables as systemd will not run with the same environment as your bash console
in particular have you set the OSPL_URI variable to point at the config?
if using the commercial version, OSPL_HOME and ADLINK_LICENSE will also need to be set
Does the PATH variable include your OSPL shared libraries?
These are all setup by running the $OSPL_HOME\release.com script in your bash session
I tend to manually add the required ones to the service file
e.g.
Environment=OSPL_URI=file:///opt/ospl.xml
I have the following script:
whie true
do
#code
sleep 60
done
I then wanted to create a service to start the machine and launch this script as service:
created my.service at /etc/systemd/system/my.service
[Unit]
Description=my Script
[Service]
Type=forking
ExecStart=/bin/script.sh
[Install]
WantedBy=multi-user.target
problem occurs when i systemctl start my.service
it goes to while true loop and hang in there, how can i run this service and make it run in the background ?
According to systemd specification at link. Type=forking is not exactly correct kind of start-up in your case
If set to forking, it is expected that the process configured with
ExecStart= will call fork() as part of its start-up. The parent
process is expected to exit when start-up is complete and all
communication channels are set up. The child continues to run as the
main service process, and the service manager will consider the unit
started when the parent process exits. This is the behavior of
traditional UNIX services. If this setting is used, it is recommended
to also use the PIDFile= option, so that systemd can reliably identify
the main process of the service. systemd will proceed with starting
follow-up units as soon as the parent process exits.
The Type=simple can be correct one. You can try with it
If set to simple (the default if ExecStart= is specified but neither
Type= nor BusName= are), the service manager will consider the unit
started immediately after the main service process has been forked
off. It is expected that the process configured with ExecStart= is the
main process of the service. In this mode, if the process offers
functionality to other processes on the system, its communication
channels should be installed before the service is started up (e.g.
sockets set up by systemd, via socket activation), as the service
manager will immediately proceed starting follow-up units, right after
creating the main service process, and before executing the service's
binary. Note that this means systemctl start command lines for simple
services will report success even if the service's binary cannot be
invoked successfully (for example because the selected User= doesn't
exist, or the service binary is missing).
I'm using Ubuntu 18.04.4 LTS which uses systemd, but the squid package packaged with this version of Ubuntu is configured to start via init.d. It starts and runs via systemctl start squid.service if I start it manually after the system has booted.
However, I'm using a ZFS mount point ("/media") to store the cache data, and during the boot process squid is starting before this mount point is active. Consequently I'm getting the error "Failed to verify one of the swap directories". Full output of systemctl status squid is here
I'd like to tell systemd to wait until after media.mount has completed in the most minimally invasive way possible (e.g. without modifying the /etc/init.d/squid file that is maintained by the package). To that end I created the /etc/systemd/system/squid.service.d/override.conf file like so:
% cat /etc/systemd/system/squid.service.d/override.conf
[Unit]
Wants=network.target network-online.target nss-lookup.target media.mount
After=network.target network-online.target nss-lookup.target media.mount
[Install]
WantedBy=multi-user.target
But squid is still starting too early.
Is what I want to do possible? Or do I have to bite the bullet and define a native /etc/systemd/system/squid.service file and remove the /etc/init.d/squid init script?
I would like systemd to manage the tup monitor, so I wrote a service unit:
[Unit]
Description=Monitor source files for changes
[Service]
Type=simple
ExecStart=/usr/bin/tup monitor -f
ExecStop=/usr/bin/tup stop
WorkingDirectory=/some/dir
StandardOutput=journal+console
StandardError=journal+console
[Install]
WantedBy=multi-user.target
Whereas starting the monitor manually works, trying to do it with systemd fails with the following error:
setpgid: Operation not permitted
tup error: Unable to set process group for tup's subprocesses.
The error originates in this file and seems to be due to systemd not giving the process the capabilities it needs, so I went further:
[Unit]
Description=Monitor source files for changes
[Service]
Type=simple
ExecStart=/usr/bin/tup monitor -f
ExecStop=/usr/bin/tup stop
CapabilityBoundingSet=CAP_SETUID CAP_SETGID
AmbientCapabilities=CAP_SETUID CAP_SETGID
WorkingDirectory=/some/dir
StandardOutput=journal+console
StandardError=journal+console
[Install]
WantedBy=multi-user.target
Still, it doesn't work.
How can I tell systemd to keep the permissions tup requires?
systemd does not restrict any permissions on services by default, so I don’t think that’s the problem. The setpgid(2) manpage mentions three possible conditions for EPERM, though:
An attempt was made to move a process into a process group in a different session, or to change the process group ID of one of the children of the calling process and the child was in a different session, or to change the process group ID of a session leader (setpgid(), setpgrp()).
I think the third one is the most likely cause of the error: tup probably expects to be run from a shell, in which case the shell would be the session leader and tup tries to establish its own process group, but systemd places each service in its own session as part of setting up a clean execution environment for the service (see also daemon(7)), so that step fails.
Unless you can somehow disable that step in tup (it doesn’t need to fork, either, or do a number of other things which systemd makes unnecessary), I think the only workaround would be an ugly hack:
ExecStart=/bin/sh -c '/usr/bin/tup monitor -f'
With this, the session leader will be the shell, not tup, so the setpgid should work again.
Try this
[Unit]
Description=Monitor source files for changes
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/tup monitor -f
ExecStop=/usr/bin/tup stop
CapabilityBoundingSet=CAP_SETUID CAP_SETGID
AmbientCapabilities=CAP_SETUID CAP_SETGID
WorkingDirectory=/some/dir
StandardOutput=journal+console
StandardError=journal+console
[Install]
WantedBy=multi-user.target
I've added After=network.target, this will make sure TUP service starts after the network is fully active and avoid the service failing due to network issues
Hi is someone aware about stopping and starting CDH(cloudera distribution Hadoop) Services with script we are doing this for production servers. For an instance if servers are restarted then before reboot all the Hadoop services stops gracefully and on startup the start.
I have a 8 Node Hadoop cluster on RHEL with cloudera 5.4.7 installed on it.
Till now i have identified few ways to do that one is here on link it says i have to use chkconfig to register the service on OS for eg as below:
sudo chkconfig hadoop-hdfs-namenode on
But when i am doing that i am getting error as
error reading information on service hadoop-hdfs-namenode: No such file or directory
which clearly states that it is unable to find the file i have specifed.
Then i searched for file and it is located in
/opt/cloudera/parcels/CDH-5.4.7-1.cdh5.4.7.p0.3/etc/rc.d/init.d/hadoop-hdfs-namenode
/opt/cloudera/parcels/CDH-5.4.7-1.cdh5.4.7.p0.3/etc/default/hadoop-hdfs-namenode
the i tried executing the same commands from the folder itself where files are located but the same error. The permissions are fine on file and tried ./ as well but same error.
I am also able to list all the process which are currently running by
sudo jps
14035 -- process information unavailable
10615 -- process information unavailable
15323 -- process information unavailable
5486 -- process information unavailable
2001 -- process information unavailable
46991 -- process information unavailable
42667 -- process information unavailable
33732 Jps
2698 -- process information unavailable
2727 -- process information unavailable
7901 -- process information unavailable
42624 -- process information unavailable
As one can see process names are not coming but these are hadoop process so to stop the process i can kill all of them but this is not the way to gracefully stop hadoop managed by cloudera. Please let me know if anyone is aware of anything which can help me moving forward.
Thanks to cloudera they provide a way to boot services on system startup. Below is the way to do that:
Click on the service
Go to the configuration
Search for Automatically Restart Process
Check the Check-Box.
It will restart the services on bootup.
you can do this by executing curl command form shell script. For example to start solr service you can use
curl -u admin:admin -X POST http://ipaddress:7180/api/v4/clusters//services/solr1/commands/start -H 'Content-type:aplication/json; charset=utf-8';
For More details on the visit
http://cloudera.github.io/cm_api/apidocs/v10/index.html