unbound variable when run a crontab task in mac os x - macos

when I run script in mac os x like this:
*/1 * * * * /Users/dolphin/Library/"Mobile Documents"/com~apple~CloudDocs/Document/source/dolphin/dolphin-scripts/bash/cron/latex_compile_alive_monitor.sh >> /Users/dolphin/shell.log
the output is:
From dolphin#dolphins-MacBook-Pro.local Tue Jan 21 20:47:01 2020
Return-Path: <dolphin#dolphins-MacBook-Pro.local>
X-Original-To: dolphin
Delivered-To: dolphin#dolphins-MacBook-Pro.local
Received: by dolphins-MacBook-Pro.local (Postfix, from userid 501)
id 7430417C4C19; Tue, 21 Jan 2020 20:47:00 +0800 (CST)
From: dolphin#dolphins-MacBook-Pro.local (Cron Daemon)
To: dolphin#dolphins-MacBook-Pro.local
Subject: Cron <dolphin#dolphins-MacBook-Pro> /Users/dolphin/Library/"Mobile Documents"/com~apple~CloudDocs/Document/source/dolphin/dolphin-scripts/bash/cron/latex_compile_alive_monitor.sh >> /Users/dolphin/shell.log
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=dolphin>
X-Cron-Env: <USER=dolphin>
X-Cron-Env: <HOME=/Users/dolphin>
Message-Id: <20200121124701.7430417C4C19#dolphins-MacBook-Pro.local>
Date: Tue, 21 Jan 2020 20:47:00 +0800 (CST)
+ BOOK_PATH='/Users/dolphin/Library/Mobile Documents/com~apple~CloudDocs/Document/source/dolphin/summary/'
+ COMMAND='/Library/TeX/texbin/latexmk -pdfxe -pvc -xelatex -interaction=nonstopmode '
+ PROCESS_NAME_KEYWORDS_MAP=(["dolphin-book-2020.tex"]="${COMMAND} ./dolphin-book-2020/dolphin-book-2020.tex" ["the-book-of-mine.tex"]="${COMMAND} ./the-books-of-mine/the-book-of-mine.tex" ["kubelet-learn.tex"]="${COMMAND} ./kubelet-learn/kubelet-learn.tex")
/Users/dolphin/Library/Mobile Documents/com~apple~CloudDocs/Document/source/dolphin/dolphin-scripts/bash/cron/latex_compile_alive_monitor.sh: line 15: dolphin: unbound variable
which variable was unboud? COMMAND? I am already defined in script.this is my script:
#!/usr/bin/env bash
# 当使用未初始化的变量时,程序自动退出
set -u
# 当任何一行命令执行失败时,自动退出脚本
set -e
# 在运行结果之前,先输出执行的那一行命令
set -x
BOOK_PATH="/Users/dolphin/Library/Mobile Documents/com~apple~CloudDocs/Document/source/dolphin/summary/"
COMMAND="/Library/TeX/texbin/latexmk -pdfxe -pvc -xelatex -interaction=nonstopmode "
declare -A PROCESS_NAME_KEYWORDS_MAP=(
["dolphin-book-2020.tex"]="${COMMAND} ./dolphin-book-2020/dolphin-book-2020.tex"
["the-book-of-mine.tex"]="${COMMAND} ./the-books-of-mine/the-book-of-mine.tex"
["kubelet-learn.tex"]="${COMMAND} ./kubelet-learn/kubelet-learn.tex"
)
cd "${BOOK_PATH}"
for key in ${!PROCESS_NAME_KEYWORDS_MAP[#]}
do
PID_COUNT=`ps -ef | grep "${key}" | grep -v "grep" | wc -l`
if [[ ${PID_COUNT} -lt 1 ]]; then
nohup `${PROCESS_NAME_KEYWORDS_MAP[${key}]}` &
else
echo "process already exists..."
fi
done

maybe remove the space in the end of COMMAND

Related

Count string occurences in file

I am trying to count the occurence of a particular line in a file using shell script. The content of the file is:
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
The shell script code is:
file="vsftpd1.log"
count=0
while read str; do
echo $str
if [[ $str == *"230 Login successfull."* ]];
then
count=$((count+1))
fi
done < $file
echo $count
The output of the last line should be four. I have noticed that if condition is executing only one time. But it should run four times. Please help me to find mistake in the code, if any.
Thankxx in advance!
You could simplify that with grep (match occurences) and its -c (count occurences).
#!/bin/sh
file=vsftpd1.log
grep -c '230 Login successfull.' "$file"
If you need to capture the number into a variable:
count=$(grep -c '230 Login successfull.' "$file")
echo $count
That entire (rather inefficient) snippet of code can be replaced with the much simpler:
count=$(grep -c '230 Login successfull.' vsftpd1.log)
echo ${count}
As more explanation, grep -c returns the line count of the lines that match your search pattern. There is no need for a bash-based loop which reads one line at a time and checks the pattern - this is likely to be much faster in the grep executable.
The following transcript shows this in action:
pax:~$ grep '230 Login successfull.' qq.in
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
Mon Dec 23 06:21:00 2019 [pid 3294] [ftpuser] FTP response: Client "127.0.0.1", "230 Login successfull."
pax:~$ count=$(grep -c '230 Login successfull.' qq.in) ; echo ${count}
4

Linux CRON: unexpected EOF while looking for matching ``' [duplicate]

This question already has an answer here:
/bin/sh: 1: Syntax error: EOF in backquote substitution
(1 answer)
Closed 4 years ago.
I am running this command on cron:
0 4 1 * * myfunc `date +%Y-%m`; anotherfunc
on my mail file /var/spool/mail/myuser, I get this message:
From root#myserver.localdomain Thu Nov 1 04:00:01 2018
Return-Path: <root#myserver.localdomain>
X-Original-To: myuser
Delivered-To: daniele#myserver.localdomain
Received: by myserver.localdomain (Postfix, from userid 500)
id 5F3A081CF9; Thu, 1 Nov 2018 04:00:01 +0100 (CET)
From: root#myserver.localdomain (Cron Daemon)
To: myuser#myserver.localdomain
Subject: Cron <myuser#myserver> myfunc `date +
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/myuser>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=myuser>
X-Cron-Env: <USER=myuser>
Message-Id: <20181101030001.5F3A081CF9#myserver.localdomain>
Date: Thu, 1 Nov 2018 04:00:01 +0100 (CET)
/bin/sh: -c: line 0: unexpected EOF while looking for matching ``'
/bin/sh: -c: line 1: syntax error: unexpected end of file
what am I doing wrong?
I found the reason: it's because % should be escaped with \
the correct version would be:
0 4 1 * * myfunc `date +\%Y-\%m`; anotherfunc
https://unix.stackexchange.com/questions/29578/how-can-i-execute-date-inside-of-a-cron-tab-job

crontab script fail: end of file unexpected (expecting ")") when call $(date)

I want to add
0 5 1 * * goaccess -f /var/log/nginx/access.log -a > /home/xan/reports/report-week-$(date +%Y.%m.%d).html
but crontab always complains about that:
Subject: Cron <root#deimos> goaccess -f /var/log/nginx/access.log -a > /home/xan/reports/report-week-$(date +
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
Message-Id: <E1bIogT-0001FX-9n#deimos>
Date: Fri, 01 Jul 2016 05:00:01 +0200
/bin/sh: 1: Syntax error: end of file unexpected (expecting ")")
What the proper syntax to do that?
% has special meaning in a crontab (it represents a newline), so you need to escape it to specify a literal percent sign.
0 5 1 * * goaccess ... > /home/xan/reports/report-week-$(date +\%Y.\%m.\%d).html

my job disappears without trace

I am running a weather forecast model and I want to automate the modek runs via crontab. When I submit my job interactive through:
qsub -I -l nodes=8:ppn=8 -l walltime=2:00:00
The job runs ok.
When I submit through crontab, the job disappears without any trace, no error file any where, no output file, except mail which reads as:
From: root#master.cluster (Cron Daemon)
To: test#master.cluster
Subject: Cron <test#master> PATH=/opt/torque/bin:/usr/bin:/bin:. qsub /home/test
/WRF/SCRIPTS/wrf_00_run.sh
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/test>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=test>
X-Cron-Env: <USER=test>
Message-Id: <20141220080001.D0F6B5C08ED#master.cluster>
Date: Sat, 20 Dec 2014 10:00:01 +0200 (CAT)
152.master.cluster
Here is my crontab entry
50 09 * * * PATH=/opt/torque/bin:/usr/bin:/bin:. qsub /home/test/WRF/SCRIPTS/wrf_submit_00.sh
and wrf_submit_00.sh has two dependent jobs to run consecutively
#!/bin/bash
cd WRF/WRFV3/run
FIRST =`qsub wrf_initialise.sh`
echo $FIRST
SECOND = `qsub -w depend=afterok:$FIRST wrf_00_run.sh`
echo $SECOND
exit 0
and the first job is
#!/bin/bash
#PBS -l nodes=8:ppn=8
cd WRF/WRFV3/run
echo -n "this script is running on: "
hostname -f
date
echo ""
echo "### PBS_NODEFILE (${PBS_NODEFILE}) ###"
cat ${PBS_NODEFILE}
echo ""
mpirun ./real.exe
exit 0
When I submit the two jobs separately through crontab I get the same disappearance.
Please help!, am stuck. Google search of a similar problem gives me no clue.

Cronjob missing content from mail body, manually works fine

Edit: Solution found via Barmar's answer. Added full smartctl command path and it works via crontab now.
I have the below script:
#!/bin/bash
#set -x
EMAIL="admin#domain.co.uk"
FILE="/root/scripts/hddreport.txt"
HOST=`hostname`
HDD01="/dev/sda"
P=`ping -c 1 $HOST | sed '1 ! d' | awk '{print $3}'`
cd /root/scripts/
echo -en "HDD health check on the server hosting" $HOST $P > $FILE
echo -e "\n" >> $FILE
smartctl -H $HDD01 >> $FILE
# The above commands do correctly write the content to $FILE (proved by removing the rm command at the bottom and doing cat on the file after)
smartctl -H $HDD01
echo "\nEmailed you the health of the Hard Drive $HDD01\n"
mailx -s "HDD health check complete on `date`" $EMAIL < $FILE
rm $FILE
which runs fine by doing bash /root/scripts/diskhealth.sh as it shows this in my mailbox:
HDD health check on the server hosting domain.co.uk (0.0.0.0)
smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
SMART Health Status: OK
But when I let it run via crontab using any of the following syntax:
X 20 * * * /bin/bash /root/scripts/diskhealth.sh
X 20 * * * /bin/sh /root/scripts/diskhealth.sh
X 20 * * * /root/scripts/diskhealth.sh
it puts everything but the smartctl disk check:
HDD health check on the server hosting domain.co.uk (0.0.0.0)
Here's what it shows if I add extra echo lines:
This is a test
HDD health check on the server hosting domain.co.uk (0.0.0.0)
Amended script for "This is a test" below:
#!/bin/bash
#set -x
EMAIL="admin#domain.co.uk"
FILE="/root/scripts/hddreport.txt"
HOST=`hostname`
HDD01="/dev/sda"
P=`ping -c 1 $HOST | sed '1 ! d' | awk '{print $3}'`
cd /root/scripts/
echo "This is a test" > $FILE
echo -en "HDD health check on the server hosting" $HOST $P >> $FILE
echo -e "\n" >> $FILE
smartctl -H $HDD01 >> $FILE
smartctl -H $HDD01
echo "\nEmailed you the health of the Hard Drive $HDD01\n"
mailx -s "HDD health check complete on `date`" $EMAIL < $FILE
rm $FILE
Here is the /var/log/syslog output from cron:
Jun 6 20:25:01 hostname /USR/SBIN/CRON[1018112]: (root) CMD (bash /root/scripts/diskhealth.sh)
Jun 6 20:25:01 hostname postfix/pickup[1016576]: 5740356613F: uid=0 from=<root>
Jun 6 20:25:01 hostname postfix/cleanup[1018125]: 5740356613F: message-id=<20130606192501.5740356613F#hostname>
Jun 6 20:25:01 hostname postfix/qmgr[292015]: 5740356613F: from=<root#hostname>, size=465, nrcpt=1 (queue active)
Jun 6 20:25:01 hostname postfix/pickup[1016576]: 631F156613E: uid=0 from=<root>
Jun 6 20:25:01 hostname postfix/cleanup[1018125]: 631F156613E: message-id=<20130606192501.631F156613E#hostname>
Jun 6 20:25:01 hostname postfix/qmgr[292015]: 631F156613E: from=<root#hostname>, size=759, nrcpt=1 (queue active)
Jun 6 20:25:01 hostname pvemailforward[1018132]: forward mail to <root#localhost.localdomain>
Jun 6 20:25:01 hostname postfix/pickup[1016576]: B597B566148: uid=65534 from=<nobody>
Jun 6 20:25:01 hostname postfix/cleanup[1018125]: B597B566148: message-id=<20130606192501.631F156613E#hostname>
Jun 6 20:25:01 hostname postfix/local[1018131]: 631F156613E: to=<root#hostname>, orig_to=<root>, relay=local, delay=0.39, delays=0.16/0/0/0.23, dsn=2.0.0, status=sent (delivered to command: /usr/bin/pvemailforward)
Jun 6 20:25:01 hostname postfix/qmgr[292015]: 631F156613E: removed
Jun 6 20:25:01 hostname postfix/qmgr[292015]: B597B566148: from=<nobody#hostname>, size=963, nrcpt=1 (queue active)
Jun 6 20:25:01 hostname postfix/smtp[1018135]: B597B566148: to=<root#localhost.localdomain>, relay=none, delay=0.16, delays=0.12/0/0.04/0, dsn=5.4.4, status=bounced (Host or domain name not found. Name service error for name=localhost.localdomain type=A: Host not found)
Jun 6 20:25:01 hostname postfix/qmgr[292015]: B597B566148: removed
Jun 6 20:25:01 hostname postfix/cleanup[1018125]: D6570566147: message-id=<20130606192501.D6570566147#hostname>
Jun 6 20:25:01 hostname postfix/smtp[1018130]: 5740356613F: to=<admin#domain.co.uk>, relay=ASPMX.L.GOOGLE.COM[173.194.67.27]:25, delay=0.68, delays=0.12/0/0.19/0.36, dsn=2.0.0, status=sent (250 2.0.0 OK 1370546701 iy4si8635735wic.1 - gsmtp)
Jun 6 20:25:01 ds9453 postfix/qmgr[292015]: 5740356613F: removed
The email is received, just missing the smartctl output.
Cron jobs don't run your .profile. So if smartctl is in a directory you add to $PATH in your profile, it won't be found when you run via cron. Try using the full pathname to the command.

Resources