I'd like to source a script (it sets up variables in my shell):
source foo.sh args
But under flock so that only one instance operates at a time (it does a lot of disk access which I'd like to ensure is serialized).
$ source flock lockfile foo.sh args
-bash: source: /usr/bin/flock: cannot execute binary file
and
$ flock lockfile source foo.sh args
flock: source: Success
don't work.
Is there some simple syntax for this I'm missing? Let's assume I can't edit foo.sh to put the locking commands inside it.
You can't source a script directly via flock because it is an external command and source is a shell builtin. You actually have two problems because of this:
flock doesn't know any command called source because it's built into bash
Even if flock could run it, the changes would not affect the state of the calling shell as you'd want with source because it's happening in a child process.
And, passing flock to source won't work because source expects a script. To do what you want, you need to lock by fd. Here's an example
#!/bin/bash
exec 9>lockfile
flock 9
echo whoopie
sleep 5
flock -u 9
Run two instances of this script in the same directory at the same time and you will see one wait for the other.
Related
I'm not familiar with the exec command. A bash tutorial about how to lock files throws this:
exec 200>lockfile
flock 200
...
flock -u 200
I got that it is creating a file named lockfile and assigning it an FD of 200. Then the second command locks that file. When done working with the file, the last command unlocks it.
Doing so, any other concurrent instance of the same script will stay at that second line until the first instance unlocks the file. Cool.
Now, what I don't understand is what is exec doing.
Directly from the bash command line, both options seem to work:
exec 200>lockfile
200>lockfile
But when the second option is used in the script, a "Bad file descriptor error" is raised.
Why is exec needed to avoid the error?
--- edit ---
After some more "serious research", I've found an answer here. The exec command makes the FD stay for the entire script or current shell.
So doing:
200>lockfile flock 200
Would work. But later flock -u 200 would raise a "Bad FD error".
The manual seems to mention shell replacement with given command. What does that has to do with file descriptors?
This is explained in the second sentence:
exec: exec [-cl] [-a name] file [redirection ...]
Exec FILE, replacing this shell with the specified program.
If FILE is not specified, the redirections take effect in this
shell. [...]
Essentially, doing exec 42> foo.txt from inside myscript.sh opens foo.txt for writing on FD 42 in the current process.
This is similar to running ./myscript.sh 42> foo.txt from a shell in the first place, or using open and dup2 in a C program.
Bash Script:
#!/bin/bash
...
exec {LOCK} > foo.out
flock -x ${LOCK}
...
I understand that exec without an argument simply redirects all the output of the current shell to the foo.out file. Questions:
What does the first argument to exec {LOCK} mean, given that it seems to have a special significance because it is in curly braces (but not ${...}).
What is the value of ${LOCK} and where did it come from (I don't think that I defined this variable)?
This is not valid or useful bash. It will just result in two different error messages.
Instead, the intended code was this:
#!/bin/bash
...
exec {LOCK}> foo.out
flock -x ${LOCK}
...
It uses:
{name}> to open for writing and assign fd number to name
exec to apply the redirection to the current, keeping the fd open for the duration of the shell
flock to lock the assigned fd, which it will inherit from the current shell
So effectively, it creates a mutex based on the file foo.out, ensuring that only one instance is allowed to run things after the flock at a time. Any other instances will wait until the previous one is done.
Here is what I finally figured out:
exec {LOCK}> foo.out
changes stdout of the current shell to the file foo.out. The fd for the open file is set the variable ${LOCK}. Setting fd to the {LOCK} variable is a feature of bash.
flock -x ${LOCK}
is simply locking using the file descriptor.
bash scripting noob here. I've found this article: https://www.shellhacks.com/print-usage-exit-if-arguments-not-provided/ that suggests putting
[ $# -eq 0 ] && { echo "Usage: $0 argument"; exit 1; }
at the top of a script to ensure arguments are passed. Seems sensible.
However, when I do that and test that that line does indeed work (by running the script without supplying any arguments: . myscript.sh) then the script does indeed exit but so does the bash session that I was calling the script from. This is very irritating.
Clearly I'm doing something wrong but I don't know what. Can anyone put me straight?
. myscript.sh is a synonym for source myscript.sh, which runs the script in the current shell (rather than as a separate process). So exit terminates your current shell. (return, on the other hand, wouldn't; it has special behaviour for sourced scripts.)
Use ./myscript.sh to run it "the normal way" instead. If that gives you a permission error, make it executable first, using chmod a+x myscript.sh. To inform the kernel that your script should be run with bash (rather than /bin/sh), add the following as the very first line in the script:
#!/usr/bin/env bash
You can also use bash myscript.sh if you can't make it executable, but this is slightly more error-prone (somebody might do sh myscript.sh instead).
Question seems not clear if you're sourcing script source script_name or . script_name it's interpreted in current bash process, if you're running a function it's the same it's running in same process, otherwise, calling a script, caller bash forks a new bash process and waits until it terminates (so running exit doesn't exit caller process), but when running exit builtin in in current bash it exits current process.
I have a bunch of wrapper shell scripts which manipulate command line arguments and do some stuff before invoking another binary at the end. Is there any reason to not always exec the binary at the end? It seems like this would be simpler and more efficient, but I never see it done.
If you check /usr/bin, you will likely find many many shell scripts that end with an exec command. Just as an example, here is /usr/bin/ps2pdf (debian):
#!/bin/sh
# Convert PostScript to PDF.
# Currently, we produce PDF 1.4 by default, but this is not guaranteed
# not to change in the future.
version=14
ps2pdf="`dirname \"$0\"`/ps2pdf$version"
if test ! -x "$ps2pdf"; then
____ps2pdf="ps2pdf$version"
fi
exec "$ps2pdf" "$#"
exec is used because it eliminates the need for keeping the shell process active after it is no longer needed.
My /usr/bin directory has over 150 shell scripts that use exec. So, the use of exec is common.
A reason not to use exec would be if there was some processing to be done after the binary finished executing.
I disagree with your assessment that this is not a common practice. That said, it's not always the right thing.
The most common scenario where I end a script with the execution of another command, but can't reasonably use exec, is if I need a cleanup hook to be run after the command at the end finishes. For instance:
#!/bin/sh
# create a temporary directory
tempdir=$(mktemp -t -d myprog.XXXXXX)
cleanup() { rm -rf "$tempdir"; }
trap cleanup 0
# use that temporary directory for our program
exec myprog --workdir="$tempdir" "$#"
...won't actually clean up tempdir after execution! Changing that exec myprog to merely myprog has some disadvantages -- continued memory usage from the shell, an extra process-table entry, signals being potentially delivered to the shell rather than to the program that it's executing -- but it also ensures that the shell is still around on myprog's exit to run any traps required.
My default shell is bash. I have set some environment variables in my .bashrc file.
I installed a program which use .cshrc file. It contains the path to several cshell scripts.
When I run the following commands in the shell windows it works perfectly :
exec csh
source .cshrc
exec bash
I have tried to put these commands in bash script, unfortunately it didn't work.
is there another way to write a script in order to get the same result as running commands from a shell windows.
I hope my question is now clear
Many thanks for any help
WARNING : don't put the following script in your .bashrc, it will reload bash and so reload .bashrc again and again (stopable with C-c anyway)
Use preferable this script in your kit/CDS stuff startup script. (cadence presumably)
WARNING 2 : if anything in your file2source fails, the whole 'trick' stops.
Call this script : cshWrapper.csh
#! /bin/csh
# to launch using
# exec cshWrapper.csh file2source.sh
source $1
exec $SHELL -i
and launch it using
exec ./cshWrapper.csh file2source.sh
it will : launch csh, source your file and came back to the same parrent bash shell
Example :
$> ps
PID TTY TIME CMD
7065 pts/0 00:00:02 bash
$>exec ./cshWrapper.csh toggle.csh
file sourced
1
$> echo $$
7065
where in my case i use the file toggle.csh
#! /bin/csh
# source ./toggle.csh
if ! $?TOGGLE then
setenv TOGGLE 0
endif
if ($?TOGGLE) then
echo 'file sourced'
if ($TOGGLE == 0) then
setenv TOGGLE 1
else
setenv TOGGLE 0
endif
endif
echo $TOGGLE
Hope it helps
New proposal, since I faced another problem with exec.
exec kills whatever remains in the script, except if you force a fork by using a pipe after it `exec script |cat'. In such case if you have environment variable in the script, they are not spread back to the script itself, which is not what we want. The only solution I found is to use 3 files (let's call them for the example : main.bash that call first.cshrc and second.sh).
#! /bin/bash
#_main.bash_
exec /bin/csh -c "source /path_to_file/cshrc; exec /bin/bash -i -c /path_to_file/second.sh"
# after exec nothing remains (like Attila the Hun)
# the rest of the script is in 'second.sh'
With that manner, i can launch in a single script call, an old cshrc design kit, and still process some bash command after, and finally launch the main program in bash (let say virtuoso)