I'm trying to update the crontab from a GNU Make file.
The idea is like this: I look through the existing cron table and filter out all entries marked as mine (via the comment) and save that to a temporary file. Then I add my jobs to that temporary file and make it a new cron table. That way the make file can be run several times without harming other people's jobs.
This is the relevant part of the make file:
crontab.tmp: $(CRON_FILES)
#echo -n "Generating new cron table combining existing one with a new one ..."
if $$(crontab -l); then crontab -l | grep -v "MAX-CRON-JOB"; fi > $#
#cat $(CRON_FILES) | awk '{print $$0, " ## MAX-CRON-JOB"}' >> $#
#echo "OK"
.PHONY: cronjobs
cronjobs: crontab.tmp
#echo -n "Installing cron commands... "
#crontab $<
#echo "OK"
The troubling part is this line:
if $$(crontab -l); then crontab -l | grep -v "MAX-CRON-JOB"; fi > $#
When the cron table is empty it somehow breaks the make, while the respective generated bash command:
if $(crontab -l); then crontab -l | grep -v "MAX-CRON-JOB"; fi > crontab.tmp
Works OK from the command line.
Here is an error from the make (nothing particularly informative, if you ask me...):
Generating new cron table combining existing one with a new one ...if $(crontab -l); then crontab -l | grep -v "MAX-CRON-JOB"; fi > crontab.tmp
make: *** [crontab.tmp] Error 1
What am I missing here?
Why are you 'escaping' crontab? As far as I can tell,
if crontab -l; then crontab -l | grep -v "MAX-CRON-JOB"; fi > $#
should work fine.
Why use a conditional at all? If the cron table is empty, so be it:
crontab.tmp: $(CRON_FILES)
#echo -n "Generating new cron table..."
#crontab -l | grep -v "MAX-CRON-JOB" > $#
#cat $(CRON_FILES) | awk '{print $$0, " ## MAX-CRON-JOB"}' >> $#
#echo "OK"
Try changing this line to include a test (square brackets):
if [ $$(crontab -l) ]; then crontab -l | grep -v "MAX-CRON-JOB"; fi > $#
because, at least for me, this doesn't work at a bash prompt without it:
if $(crontab -l); then crontab -l | grep -v "MAX-CRON-JOB"; fi > $#
yields:
-bash: 0: command not found
Related
I have a bash/zsh command with multiple pipes | that fails when using set -o pipefail. For simplicity assume the command is
set -o pipefail; echo "123456" | head -c2 | grep 5 | cat
How do I quickly find out which command is the first to fail and why? I know I can check the exit code, but that doesn't show which part of the pipeline failed first.
Is there something simpler than the rather verbose check of building up the pipeline one by one and checking for the first failing exit code?
Edit: I removed the contrived code example I made up as it confused people about my purpose of asking. The actual command that prompted this question was:
zstdcat metadata.tsv.zst | \
tsv-summarize -H --group-by Nextclade_pango --count | \
tsv-filter -H --ge 'count:2' | \
tsv-select -H -f1 >open_lineages.txt
In bash, use echo "${PIPESTATUS[#]}" right after the command to get the exit status for each component in a space separated list:
#!/bin/bash
$ set -o pipefail; echo "123456" | head -c2 | grep 5 | cat
$ echo ${PIPESTATUS[#]}
0 0 1 0
Beware zsh users, you need to use the lower case pipestatus instead:
#!/bin/zsh
$ set -o pipefail; echo "123456" | head -c2 | grep 5 | cat
$ echo $pipestatus
0 0 1 0
In fish you can also simply use echo $pipestatus for the same output.
${PIPESTATUS[#]} right after is the answer you were looking for. However, I want to advise on the first example. It's a good habit to anticipate error, so instead of testing after you should have check the path prior everything.
if [ -d "/nonexistent_directory" ]; then
# here pipe shouldn't fail to grep
# ...unless there's something wrong with "foo"
# ...but "grep" may be a failure if the pattern isn't found
if ! ls -1 "/nonexistent_directory" | grep 'foo' ; then
echo "The command 'grep foo' failed."
# else
# echo "The pipeline succeeded."
fi
else
echo "The command 'ls /nonexistent_directory' failed."
fi
Whenever possible, avoid greping ls output in script, that' fragile...
For a half-finished script that already uses the output of a program I also need the name and the parameters of the program that was used to pipe to my script.
So I run it like this:
yay something | ./myscript
Now I need to store "yay something" into a variable.
There is a way to to get previous runned commands or the current one by using set -o history -o histexpand and echo !! or echo $0 but that doesn't include what I wrote right before the pipe.
Maybe you would suggest to pass the name of the program and it's parameter to my script as parameters and then run it there but I don't want this (pass a command as an argument to bash script).
UPDATED SOLUTION (old below):
#!/bin/bash -i
#get processes
processes=$(> >(ps -f))
echo beginning:
echo "$processes"
#filter bin/bash -i
pac=$(echo "$processes" | sed '1,/bin\/bash -i/!d')
pac=$(echo "$pac" | tail -2 | head -1)
#kill
delete=$(echo $pac | grep -oP "(?<=$USER\s)\w+")
pac=$(echo "$pac" | grep -o -P '(?<=00:00:00).*(?=)')
echo "$delete"
kill -9 "$delete"
#print
echo " "
echo end:
echo "${pac:1}"
Note: When you use echo, man or cat then $pac will be empty.
OLD Text:
Thanks to Charles for his enormous effort and his link that finally led me to processes=$(> >(ps -f)).
Here a working example. You can e.g. use it with vi test | ./testprocesses (or nano or package helpers like yay or trizen but it won't work with echo, man nor with cat):
#!/bin/bash -i
#get processes
processes=$(> >(ps -f))
echo beginning:
echo $processes
#filter
pac=$(echo $processes | grep -o -P '(?<=CM).*(?=testprocesses)' | grep -o -P '(?<=D).*(?=testprocesses)' | grep -o -P "(?<=00:00:00).*(?=$USER)")
#kill
delete=$(echo $pac | grep -oP "(?<=$USER\s)\w+")
pac=$(echo $pac | grep -o -P '(?<=00:00:00).*(?=)')
kill -9 $delete
#print
echo " "
echo end:
echo $pac
The kill part is necessary to kill the vi instance else it will still be running and eventually interfer with future executions of the script.
I'm currently monitoring a log file and my ultimate goal is to write a script that uses tail -n0 -f and execute a certain command once grep finds a correspondence. My current code:
tail -n 0 -f $logfile | grep -q $pattern && echo $warning > $anotherlogfile
This works but only once, since grep -q stops when it finds a match. The script must keep searching and running the command, so I can update a status log and run another script to automatically fix the problem. Can you give me a hint?
Thanks
use a while loop
tail -n 0 -f "$logfile" | while read LINE; do
echo "$LINE" | grep -q "$pattern" && echo "$warning" > "$anotherlogfile"
done
awk will let us continue to process lines and take actions when a pattern is found. Something like:
tail -n0 -f "$logfile" | awk -v pattern="$pattern" '$0 ~ pattern {print "WARN" >> "anotherLogFile"}'
If you need to pass in the warning message and path to anotherLogFile you can use more -v flags to awk. Also, you could have awk take the action you want instead. It can run commands via the system() function where you pass the shell command to run
I'm trying to clean up this script I have and this piece of code is annoying me because I know it can be more DRY:
if grep --version | grep "GNU" > /dev/null ;
then
grep -P -r -l "\x0d" $dir | grep "${fileRegex}"
else
grep -r -l "\x0d" $dir | grep "{$fileRegex}"
fi
My thoughts are to somehow conditionally set a string variable to either "grep -P" or "egrep" and then in a single line do something like:
$(cmdString) -r -l "\x0d" $dir | grep "${fileRegex}"
Or something like that but it doesn't work.
Are you worried about a host which has GNU grep but not egrep? Do such hosts exist?
If not why not just always use egrep? (Though -P and egrep are not the same thing.)
That being said you don't use strings for this (see BashFAQ#50).
You use arrays: grepcmd=(egrep) or grepcmd=(grep -P) and then "${grepcmd[#]}" ....
You can also avoid needing perl mode entirely if you use $'\r' or similar (assuming your shell understands that quoting method).
You can do this:
if grep --version | grep "GNU" > /dev/null
then
cmdString=(grep -P)
else
cmdString=(egrep)
fi
"${cmdString[#]}" -r -l "\x0d" "$dir" | grep "{$fileRegex}"
#Etan Reisner's suggestion worked well. For those that are interested in the final code (this case is for tabs, not windows line endings but it is similar):
fileRegex=${1:-".*\.java"}
if grep --version | grep "GNU" > /dev/null ;
then
cmdString=(grep -P)
else
cmdString=(grep)
fi
arr=$("${cmdString[#]}" -r -l "\x09" . | grep "${fileRegex}")
if [ -n "$dryRun" ]; then
for i in $arr; do echo "$i"; done
else
for i in $arr; do expand -t 7 "$i" > /tmp/e && mv /tmp/e "$i"; done
fi
I have following code in my build script:
if [ -z "$1" ]; then
make -j10 $1 2>&1 | tee log.txt && notify-send -u critical -t 7 "BUILD DONE"
else
make -j10 $1 2>&1 | tee log.txt | grep -i --color "Error" && notify-send -u critical -t 7 "BUILD DONE"
fi
I tried to optimize it to:
local GREP=""
[[ ! -z "$1" ]] && GREP="| grep -i --color Error" && echo "Grepping for ERRORS"
make -j10 $1 2>&1 | tee log.txt "$GREP" && notify-send -u critical -t 7 "BUILD DONE"
But error thrown in make line if $1 isn't empty. I just can't figure out how to pass command with grep pipe through the variable.
Like others have already pointed out, you cannot, in general, expect a command in a variable to work. This is a FAQ.
What you can do is execute commands conditionally. Like this, for example:
( make -j10 $1 2>&1 && notify-send -u critical -t 7 "BUILD DONE" ) |
tee log.txt |
if [ -z "$1" ]; then
grep -i --color "Error"
else
cat
fi
This has the additional unexpected benefit that the notify-send is actually conditioned on the exit code of make (which is probably what you intended) rather than tee (which I would expect to succeed unless you run out of disk or something).
(Or if you want the notification regardless of the success status, change && to just ; -- I think this probably makes more sense.)
This is one of those rare Useful Uses of cat (although I still feel the urge to try to get rid of it!)
You can't put pipes in command variables:
$ foo='| cat'
$ echo bar $foo
bar | cat
The linked article explains how to do such things very well.
As mentioned in #l0b0's answer, the | will not be interpreted as you are hoping.
If you wanted to cut down on repetition, you could do something like this:
if [ $(make -j10 "$1" 2>&1 > log.txt) ]; then
[ "$1" ] && grep -i --color "error" log.txt
notify-send -u critical -t 7 "BUILD DONE"
fi
The inside of the test is common to both branches. Instead of using tee so that the output can be piped, you can just indirect the output to log.txt. If "$1" isn't empty, grep for any errors in log.txt. Either way, do the notify-send.