I have a problem with my power failure monitoring scheme / automatic shutdown of Windows 10 computer.
On my network I have a Linux box that has a UPS unit connected via USB and maintained with software which makes the computer shut down when power failure occurs.
On another computer, also powered from the same UPS unit running headless Windows 10 server, I have setup a simple bash script (cygwin) running from Task Scheduler.
It is supposed to shutdown this computer once the aforementioned Linux box stops responding to ping on the network (because it shuts down when power failure is detected).
The UPS unit is setup to shut the power down after 10 minutes from power failure, giving plenty of time for the Windows machine to detect lack of response from Linux box and executing shutdown.
Here is the monitoring script code:
#!/bin/bash
upshost="<linux-box-ip>"
echo "UPS Host: $upshost"
while true
do
date
ping -c3 $upshost >/dev/null
if [[ $? -ne 0 ]]
then
echo "WARNING: Ping failed, trying again in 30 seconds ..."
sleep 30
ping -c3 $upshost >/dev/null
[[ $? -eq 0 ]] || break
fi
echo "Host $upshost responded to ping."
sleep 30
done
date
echo "ERROR: Host $upshost didn't respond to ping."
echo "System will shutdown in 1 minute."
shutdown -h +1 "System will shutdown in 1 minute."
NOTE: is the actual IP address of my Linux box which is always the same (reserved in the router setup).
The task scheduler properties:
Runs from SYSTEM account, whether user is logged on or not, with highest privileges.
Triggers: at system startup
Actions: Start a program
program/script -> c:\cygwin64\bin\bash
arguments -> -l -c "/cygdrive/c/bin/pwrwdog.sh > /cygdrive/c/tmp/pwrwdog.trc"
runs in -> c:\cygwin64\bin
Start if any network connection is available.
I see in traces and in task scheduler history that script properly monitors the network response from Linux box, breaks out of the monitoring loop when host stops responding and invokes shutdown command. The problem is - the shutdown doesn't actually happen:
Host <linux-box-ip> responded to ping.
Sun, Jan 3, 2021 7:23:48 AM
Host <linux-box-ip> responded to ping.
Sun, Jan 3, 2021 7:24:20 AM
WARNING: Ping failed, trying again in 30 seconds ...
Sun, Jan 3, 2021 7:25:16 AM
ERROR: Host <linux-box-ip> didn't respond to ping.
System will shutdown in 1 minute.
System will shutdown in 1 minute.
What do I miss?
Related
When I upload a big amount of files through FTP for some reason my WiFi connection stops working. The connection doesn't show any problem and for solving this it's enough disable and re-enable the wifi. How to restart automatically it using AppleScript in the Script Editor?
My solution's this script that every second checks the connection to google and if there is a timeout close and reopen the en0 connection.
log "Internet checker: let's go"
repeat while true
try
do shell script "nc -w 4 -z 8.8.8.8 53"
# do shell script "nc -w 4 -z apple.com 80"
log "External TCP call test works"
on error
log "Restarting internet..."
do shell script "networksetup -setairportpower Wi-Fi off"
delay 2
do shell script "networksetup -setairportpower Wi-Fi on"
log "Done."
delay 8
end try
delay 2 # wait a moment before the next internet checking
end repeat
Update 24 Aug 022: This should be my 'definitive edition' of the script: privileges are no more requested. It used to be pretty slow to discover the lack of internet connection despite the -w timeout, so I try to ask directly to a "standard de facto" IP (a DNS server).
For more information about nc command you can consult this link: https://www.computerhope.com/unix/nc.htm
I have a simple example of a service unit and bash script on Red Hat Enterprise Linux 7 using Type=notify that I am trying to get working.
When the service unit is configured to start the script as root, things work as expected. When adding User=testuser it fails. While the script initially starts (as seen on process list) the systemctl service never receives the notify message indicating ready so it hangs and eventually times out.
[Unit]
Description=My Test
[Service]
Type=notify
User=testuser
ExecStart=/home/iatf/test.sh
[Install]
WantedBy=multi-user.target
Test.sh (owned by testuser with execute permission)
#!/bin/bash
systemd-notify --status="Starting..."
sleep 5
systemd-notify --ready --status="Started"
while [ 1 ] ; do
systemd-notify --status="Processing..."
sleep 3
systemd-notify --status="Waiting..."
sleep 3
done
When run as root systemctl status test displays the correct status and status messages as sent from my test.sh bash script. When User=testuser the service hangs and then timesout and journalctl -xe reports:
Jul 15 13:37:25 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7193.
Jul 15 13:37:28 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7290.
Jul 15 13:37:31 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7388.
Jul 15 13:37:34 tstcs03.ingdev systemd[1]: Cannot find unit for notify message of PID 7480.
I am not sure what those PIDs are as they do not appear on ps -ef list
This appears to be known limitation in the notify service type
From a pull request to the systemd man pages
Due to current limitations of the Linux kernel and the systemd, this
command requires CAP_SYS_ADMIN privileges to work
reliably. I.e. it's useful only in shell scripts running as a root
user.
I've attempted some hacky workarounds with sudo and friends but they won't work as systemd - generally failing with
No status data could be sent: $NOTIFY_SOCKET was not set
This refers to the socket that systemd-notify is trying to send data to - its defined in the service environment but I could not get it reliably exposed to a sudo environment
You could also try using a Python workaround described here
python -c "import systemd.daemon, time; systemd.daemon.notify('READY=1'); time.sleep(5)"
Its basically just a sleep which is not reliable and the whole point of using notify is reliable services.
In my case - I just refactored to use root as the user - with the actual service as a child under the main service with the desired user
sudo -u USERACCOUNT_LOGGED notify-send "hello"
I have a NAS with a usb plugged UPS on it.
I have installed nut client (only the client) on my raspberry to monitor the UPS.
I must plug the UPS to the NAS because it's the only device which recognizes my UPS (chinese generic, unfortunately no choice in my country).
No way to make it directly work plugged to the raspberry.
As I've installed only nut client into my raspberry, /etc/nut was empty. I've created 3 files:
nut.conf
MODE=netclient
upsmon.conf
MONITOR login#nas_ip 1 admin password slave
SHUTDOWNCMD "/sbin/shutdown -h now"
NOTIFYFLAG ONBATT SYSLOG+WALL+EXEC
NOTIFYFLAG ONLINE SYSLOG+WALL+EXEC
NOTIFYCMD "/etc/nut/notifycmd"
the notifycmd script:
#!/bin/bash
#
# NUT NOTIFYCMD script
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin
trap "exit 0" SIGTERM
if [ "$NOTIFYTYPE" = "ONLINE" ]
then
echo $0: power restored | wall
# curl command to send me a sms
# Cause all instances of this script to exit.
killall -s SIGTERM `basename $0`
fi
if [ "$NOTIFYTYPE" = "ONBATT" ]
then
echo $0: 40 minutes till system powers down... | wall
#curl command to send me a sms
# Loop with one second interval to allow SIGTERM reception.
let "n = 2400"
while [ $n -ne 0 ]
do
sleep 1
let "n--"
done
echo $0: commencing shutdown | wall
sleep 10
upsmon -c fsd
fi
My UPS is powerful enough to wait 40 minutes before shuting down the nas and the raspberry remotely.
I receive a sms when power loss and power restored.
But there is a special case: if the raspberry lost connection with the ups when it is on battery, the raspberry shut down almost immediately... I think this is a protection behavior, but i'd like to insert a curl command which sends a sms before shutdown, like "power loss and connexion failure with ups"
(i know the command itself, this is not the problem)
I know I must add something like "DEADTIME" or "NOTIFYFLAG NOCOMM" or "COMMBAD" in upsmon.conf, but not sure what to choose...
And what do i have to add in my "notifycmd" file ?
Thanks
You may raise the value of DEADTIME, if you feel that this would be good for your Raspberry Pi. From a recent upsmon.conf:
# upsmon requires a UPS to provide status information every few seconds
# (see POLLFREQ and POLLFREQALERT) to keep things updated. If the status
# fetch fails, the UPS is marked stale. If it stays stale for more than
# DEADTIME seconds, the UPS is marked dead.
#
# A dead UPS that was last known to be on battery is assumed to have gone
# to a low battery condition. This may force a shutdown if it is providing
# a critical amount of power to your system.
If you know for sure that the UPS has enough battery for your Raspberry Pi (even when you can't talk with it) you may rise DEADTIME to something like 120 (2 minutes) or even 300 (5 minutes). Just keep in mind that the value should be a multiple of POLLFREQ and POLLFREQALERT.
After this, upsmon will have some more time to run your NOTIFYCMD.
Now, just add a case for COMMBAD:
COMMBAD is triggered as long as the communication is lost
COMMOK is triggered as long as the communication is recovered
NOCOMM is triggered as long as the communication has not been recovered for some time
Also, consider using the NOTIFYCMD just for notifying something, not doing the dirty job (e.g. shutting down the system).
I've written a scrip that works fine to start and stop a server.
#!/bin/bash
PID_FILE='/var/run/rserve.pid'
start() {
touch $PID_FILE
eval "/usr/bin/R CMD Rserve"
PID=$(ps aux | grep Rserve | grep -v grep | awk '{print $2}')
echo "Starting Rserve with PID $PID"
echo $PID > $PID_FILE
}
stop () {
pkill Rserve
rm $PID_FILE
echo "Stopping Rserve"
}
case $1 in
start)
start
;;
stop)
stop
;;
*)
echo "usage: rserve {start|stop}" ;;
esac
exit 0
If I start it by running
rserve start
and then start monit it will correctly capture the PID and the server:
The Monit daemon 5.3.2 uptime: 0m
Remote Host 'localhost'
status Online with all services
monitoring status Monitored
port response time 0.000s to localhost:6311 [DEFAULT via TCP]
data collected Mon, 13 May 2013 20:03:50
System 'system_gauss'
status Running
monitoring status Monitored
load average [0.37] [0.29] [0.25]
cpu 0.0%us 0.2%sy 0.0%wa
memory usage 524044 kB [25.6%]
swap usage 4848 kB [0.1%]
data collected Mon, 13 May 2013 20:03:50
If I stop it, it will properly kill the process and unmonitor it. However if I start it again, it won't start the server again:
ps ax | grep Rserve | grep -vc grep
1
monit stop localhost
ps ax | grep Rserve | grep -vc grep
0
monit start localhost
[UTC May 13 20:07:24] info : 'localhost' start on user request
[UTC May 13 20:07:24] info : monit daemon at 4370 awakened
[UTC May 13 20:07:24] info : Awakened by User defined signal 1
[UTC May 13 20:07:24] info : 'localhost' start: /usr/bin/rserve
[UTC May 13 20:07:24] info : 'localhost' start action done
[UTC May 13 20:07:34] error : 'localhost' failed, cannot open a connection to INET[localhost:6311] via TCP
Here is the monitrc:
check host localhost with address 127.0.0.1
start = "/usr/bin/rserve start"
stop = "/usr/bin/rserve stop"
if failed host localhost port 6311 type tcp with timeout 15 seconds for 5 cycles
then restart
I had problem start or stop process via shell too.
One solution might be add "/bin/bash" in the config like this:
start program = "/bin/bash /urs/bin/rserv start"
stop program = "/bin/bash /urs/bin/rserv stop"
It worked for me.
monit is a silent killer. It does not tell you anything. Here are things I would check which monit won't help you identify
Check permissions of all the files you are reading / writing. If you are redirecting output to a file, make sure that file is writable by uid and gid you are using to execute the program
Again check exec permission on the program you are trying to run
Specify full path to any program you are trying to execute ( not strictly necessary, but you don't have to worry about path not being set if you always specify full path )
Make sure you can run the program outside of monit without any error before trying to investigate why monit is not starting.
If the Monit log is displaying
failed to start (exit status -1) -- no output
Then it may be that you're trying to run a script without any of the Bash infrastructure. You can run such a command by wrapping it in /bin/bash -c, like so:
check process my-process
matching "my-process-name"
start program = "/bin/bash -c '/etc/init.d/my-init-script'"
When monit starts it checks for its own pidfile and checks if the process with
matching PID is running already - if it does, then it just wakes up this
process.
in your case, check if this pid is being used by some other process:
ps -ef |grep 4370
if yes, then you need to remove the below file(usually under /run directory) and start monit again:
monit.pid
For me, the issue was that the stop command was not being run, even though I specifically specified "then restart" on the configuration.
The solution was just to change:
start program = "/etc/init.d/.... restart"
I'm downloading huge sums of data at night and my PC in running through a UPS. Is there any way I can detect a power failure and command my PC to shut down automatically? Because I work at night, and there's no one to switch off the PC, it would be really helpful if anyone could help. Is it possible?
Thanks.
IMPORTANT: The scripts presented below will cause the system to shutdown whenever the network is down, so use them at your own risk!
A not so elegant way of doing what you want, if you have a network host (e.g. your router) that responds to ICMP echo requests and is not powered by the UPS (or at least the networking equipment is not powered by the UPS), would be to ping that host every few seconds and if it's down then shutdown the PC:
#!/bin/bash
while :
do
ping -c 1 -w 5 192.168.0.1 &> /dev/null
if [ $? -gt 0 ]; then
shutdown -hP now
break
fi
sleep 10s
done
You will have to change 192.168.0.1 to the IP address of the network host you want to ping.
You will also have to make the script executable with chmod +x <script_name> and place a call to it in /etc/rc.local (do not forget to append a & to make it run in the background) which will run the script as root on boot.
For completeness' sake, if the PC was running Windows XP one could use the following batch file:
:loop
ping -n 1 -w 5000 192.168.0.1
if not %ERRORLEVEL% == 0 (
shutdown -s
goto end
)
sleep 10
goto loop
:end
Note that the Windows batch file requires the sleep command which can be installed as part of the Windows Server 2003 Resource Kit Tools package (available as a free download from Microsoft's site)
If it's an APC and it has a data port you can use PowerChute. It's a java-based GUI (which could be a problem if this is your server) that does exactly what you are requesting.