I have a bash script that keeps taking screenshots, as fast as possible, it normally takes 1-2ms per run.
I also want to have a separate process, running constantly on another thread, that will take said screenshot files and do something with them (crop, read, whatever).
Sometimes I can run both side by side just fine, but sometimes process B is trying to read the file that process A is still writing to.
How to approach this?
I imagine something like a cache/copy/buffer will help, but I'm not sure.
I remembered about file locks and they are exactly what I need for this.
While taking the screenshots I will use UNIX's flock (with an exclusive lock) and while reading the files I will use Ruby's File#flock with a shared lock.
This should do it!
I made a couple simple files that do stuff before and after a control volley - in this case, just report which file is at which point in the loop, so I literally just symlinked it to a second name. I called them a & b.
$: cat a
for a in 1 2 3
do echo "$0 $a before: $(date)"
kill -CONT -1 # NOTE: sends the signal to ALL procs in group
kill -STOP $$
echo "$0 $a AFTER: $(date)"
done
kill -CONT -1
b is the same file, just another name.
run both. Use a wait to clean up the prompt at the end.
$: ./a;./b;wait
./a 1 before: Mon, Aug 10, 2020 12:56:41 PM
[1]+ Stopped ./a
./b 1 before: Mon, Aug 10, 2020 12:56:41 PM
[2]+ Stopped ./b
./a 1 AFTER: Mon, Aug 10, 2020 12:56:41 PM
./a 2 before: Mon, Aug 10, 2020 12:56:41 PM
[1]+ Stopped ./a
./b 1 AFTER: Mon, Aug 10, 2020 12:56:41 PM
./b 2 before: Mon, Aug 10, 2020 12:56:41 PM
[2]+ Stopped ./b
./a 2 AFTER: Mon, Aug 10, 2020 12:56:41 PM
./a 3 before: Mon, Aug 10, 2020 12:56:41 PM
[1]+ Stopped ./a
./b 2 AFTER: Mon, Aug 10, 2020 12:56:42 PM
./b 3 before: Mon, Aug 10, 2020 12:56:42 PM
[2]+ Stopped ./b
./a 3 AFTER: Mon, Aug 10, 2020 12:56:42 PM
[1]- Done ./a
./b 3 AFTER: Mon, Aug 10, 2020 12:56:42 PM
[2]+ Done ./b
This throws a lot of trash to stderr, but you can redirect that.
$: { ./a;./b;wait; } >log 2>err
Maybe it will offer some inspiration.
The easiest way to solve this is to write the file in a different folder on the same drive, or with a different extension, so that you can easily exclude it from the selection (for example, instead of images/foo.jpg, write to images/foo.jpg.temp, and list with *.jpg), then use atomic move when it is finished (mv images/foo.jpg.temp images/foo.jpg).
Related
I have 2 commands which I am executing over shell command1 and command2. command1 takes long to complete (~2 minutes). So I can put it running in background using & but after that I want to execute command2 automatically. Can I do this on shell command line?
Try:
( command1; command2 ) &
Edit: Larger PoC:
demo.sh:
#!/bin/bash
echo Starting at $(date)
( echo Starting background process at $(date); sleep 5; echo Ending background process at $(date) ) &
echo Last command in script at $(date)
Running demo.sh:
$ ./demo.sh
Starting at Thu Mar 1 09:11:04 MST 2018
Starting background process at Thu Mar 1 09:11:04 MST 2018
Last command in script at Thu Mar 1 09:11:04 MST 2018
$ Ending background process at Thu Mar 1 09:11:09 MST 2018
Note that the script ended after "Last command in script", but the background process did its "Ending background process" echo 5 seconds later. All of the commands in the (...)& structure are run serially, but are all collectively forked to the background.
You can do so by putting both the commends in a shell script and do like below:-
command1 &
wait
command2 & #if you want to run second command also in background
I am using /bin/script to capture the output of a command and preserve the colors and formatting. Using subshell and assigning to variable does not always work well. E.g.,:
foo="$( ls --color 2>&1 )"
I can use /bin/script to capture stdout:
$ script -qc "echo foo" >& /dev/null && cat typescript
Script started on Mon 06 Nov 2017 10:40:40 PM PST
foo
I can use /bin/script to capture stderr (ls of non-existent directory goes to stderr):
$ script -qc "ls vxzcvcxvc" >& /dev/null && cat typescript
Script started on Mon 06 Nov 2017 10:38:17 PM PST
ls: cannot access vxzcvcxvc: No such file or directory
My problem arises when the script run inside /bin/script mucks with file descriptors.
I am not able to use /bin/script to capture redirected stderr:
$ script -qc "ls vxzcvcxvc 2>&1" >& /dev/null && cat typescript
Script started on Mon 06 Nov 2017 10:47:13 PM PST
I have tried other ways as well:
$ script -qc "echo foo1 && >&2 echo foo2 && echo foo3" >& /dev/null && cat typescript
Script started on Mon 06 Nov 2017 10:46:09 PM PST
foo1
foo3
I assume /bin/script is doing its own file descriptor magic (redirecting output to file), so I am left wondering what to do if the script I am calling does its own redirection.
Tangential question: The primary culprit is a logging line that does
printf "${1}" 1>&2
in order to print logging to stderr. Is there a way to output to stderr without mucking with file descriptors (assuming this is the reason /bin/script fails to pick it up)?
I would like to run a program conditionally in the background, if the condition is met, we run it as a daemon, otherwise we wait for the result:
if [ -z "${CONDITION}" ]; then
echo "background" && npm install -g yarn &
else
echo "foreground" && npm install -g yarn
fi
is there a shorthand way to do this? (I assume in the else block, that given the syntax that this process will not run in the background). I guess we could also conditionally add "wait" instead of an "&" character.
I think your wait idea is good. I can't think of a better way.
npm install -g yarn &
[ -n "$CONDITION" ] && { echo "waiting for install"; wait; }
Not sure that you want the echo stuff or not, but if it's time consuming, you might want an indication.
HTH
Update: Simple script to show how it works
C=""
date
sleep 5s &
[ -n "$C" ] && { echo "waiting"; wait; }
date
If C="", you get an output like so:
Thu Dec 8 11:42:54 CET 2016
Thu Dec 8 11:42:54 CET 2016
(and of course, sleep 5s is still running while your script is finished)
If you set C="something", you get:
Thu Dec 8 11:42:42 CET 2016
waiting
Thu Dec 8 11:42:47 CET 2016
this is my first question (in that forum) so please be patient ... ;-)
To the problem:
I'm trying to run a binary on an raspi that crashes by chance once in a few hours. As the binary gives its output usually to stdout, I'm trying to use it with screen and pipe its output to a file. Doing so, I've written a small wrapper script which is called by cron every five minutes. My idea was, that, if the output file don't changes over a certain period, than the process is killed and restarted.
Here 's my /etc/crontab:
*/5 * * * * pi bash /home/pi/myscript.sh >/dev/null 2>/dev/null
Here 's the myscript:
#!/bin/bash
# Input file
FILE=/home/pi/output.txt
# How many seconds before file is deemed "older"
OLDTIME=300
# Get current and file times
CURTIME=$(date +%s)
FILETIME=$(stat $FILE -c %Y)
TIMEDIFF=$(expr $CURTIME - $FILETIME)
# Check if file older
if [ $TIMEDIFF -gt $OLDTIME ]; then
#echo "File is older, do stuff here"
bash /home/pi/check_myscript_is_running.sh
fi
Here 's the script that checks:
#!/bin/bash
case "$(pidof processname | wc -w)" in
0) echo "Restarting process: $(date)" >> ~/output.txt
screen -dm /home/pi/binary -l output.txt &
;;
1) # all ok
;;
*) echo "Removed double process: $(date)" >> ~output.txt
kill $(pidof process | awk '{print $1}')
;;
esac
But obviously the last script doesn't start the process anew and I'm getting mails from the cron:
From pi#raspberrypi Fri Jul 01 16:42:25 2016
Return-path: <pi#raspberrypi>
Envelope-to: pi#raspberrypi
Delivery-date: Fri, 01 Jul 2016 16:42:25 +0200
Received: from pi by raspberrypi with local (Exim 4.84_2)
(envelope-from <pi#raspberrypi>)
id 1bIzeD-00007c-0A
for pi#raspberrypi; Fri, 01 Jul 2016 16:42:25 +0200
From: root#raspberrypi (Cron Daemon)
To: pi#raspberrypi
Subject: Cron <pi#raspberrypi> pi /home/pi/startprocess.sh
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/pi>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=pi>
Message-Id: <E1bIzeD-00007c-0A#raspberrypi>
Date: Fri, 01 Jul 2016 16:42:25 +0200
/bin/sh: 1: pi: not found
I don't have a script startprocess.sh and I thought that with the output pipe the mails would be suppressed ...
But the main question is: Why is the script that should restart the process if the output file has'nt changed for five minutes not running ?
Cheers and regards,
JD.
If I understand right: In some environments the cron PATH is not set, so calling up pi in your crontab leads to the error mail you get.
Try to fully qualify your script call with an absolute path, e.g. like this:
*/5 * * * * /path/to/script/pi /usr/bin/bash /home/pi/myscript.sh >/dev/null 2>/dev/null
I will like to write (using bash) something like
while no_user_key_pressed
{
do_something....
}
There are a few options using C++, Java, ncurses and others o/s specific. I want a simple bash portable function.
^c interrupt should be used to kill the remaining code. Imagine something like: 'Press any key to stop test'
You can use a small timeout on read -t.
The drawback is that the user must press < RETURN >, not "any key".
For example:
while ! read -t 0.01
do
echo -en "$(date)\r"
done
echo "User pressed: $REPLY"
Tested on bash 3.2 (OS X)
The ! is because read returns a failure (false) if the timeout expires.
You can trap Ctrl-c in a way that does not kill the remaining code:
$ cat test.sh
#!/usr/bin/env bash
trap 'break' INT
while true
do
date
sleep 1
done
echo done
$ ./test.sh
Tue 28 Jun 12:01:22 UTC 2016
Tue 28 Jun 12:01:23 UTC 2016
Tue 28 Jun 12:01:24 UTC 2016
^Cdone