Lets take for example next function:
version 1 - with variables:
backup () {
for arname in `arname_f`
do
slapcat -b "$setnames" -l "$bkdir"/"$ardate"_"$arname".ldif || exit 1
done
}
and run it just with code:
backup;
version 2 - with positional arguments:
backup () {
for arname in `arname_f`
do
slapcat -b "$1" -l "$2"/"$3"_"$4".ldif || exit 1
done
}
and let's run with such code:
backup $setnames $bkdir $ardate $arname;
Is there any difference in this slants?
The question has nothing to do with Bash as such.
The #1 is the example of "Spaghetti" coding style (global variables) hated by most professionals and simply sane people. It will eventually cause a major problem when someone somewhere changes the parameter and the function starts misbehaving, and you won't have a clue of who/what has changed what where.
The #2 is close to how I would do it. Though, of cause, there may well be a valid reason to prefer #1, it depends.
Related
I am trying to write a simple Bash completion script for a program that runs its arguments as a command. A good example of this is kind of program is the prime-run script provided by the nvidia-prime package:
#!/bin/bash
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia "$#"
This script sets a few environment variables, which instructs the prime driver to use the Nvidia dGPU on a hybrid system. The first argument is treated as the command, and all trailing arguments are passed through. So for example you can run prime-run code . and VSCode will start in the current directory using the dGPU.
Therefore from a completion-script POV, what we want is to basically try to complete as if the prime-run token isn't there (hence "transparent proxy"-like behaviour). To give a rather contrived example:
> prime-run journalc<TAB>
(completes journalctl)
> prime-run journalctl --us<TAB>
(completes --user)
However I am finding this surprisingly difficult in Bash (not that I know how in other shells). So the question is simple: is it possible and if so how?
Ideas I've (hopelessly) had
The simple complete -A command prime-run: the first argument gets completed as a command as expected (let's call it foo), but the following arguments are also completed as commands rather than as arguments to foo
Use some combination of compgen and complete -p to invoke the completion function of foo, but AFAIK the completion function for all foo is locally defined and thus uncallable
TL;DR
bash-completion provides a function named _command_offset (permalink), which is exactly what I need.
# A meta-command completion function for commands like sudo(8), which need to
# first complete on a command, then complete according to that command's own
# completion definition.
Keep reading if you are interested in how I got here.
So I was daydreaming the other day, when it hit me - doesn't sudo basically have the exact same behaviour I want? So the task became simple - reverse engineer the completion script for sudo. Source available here: permalink.
Turns out, most of the code has to do with completing the various options, so it's safe to simply throw most of it out:
L 8-11, 50-52: Related to sudo's edit mode. Safe to ditch.
L 19-24, 27-39, 43-49: These complete sudo's options. Safe to ditch.
So we're left with this:
_sudo()
{
local cur prev words cword split
_init_completion -s || return
for ((i = 1; i <= cword; i++)); do
if [[ ${words[i]} != -* ]]; then
local PATH=$PATH:/sbin:/usr/sbin:/usr/local/sbin
local root_command=${words[i]}
_command_offset $i
return
fi
done
$split && return
} &&
complete -F _sudo sudo sudoedit
The for and if block are there to deal with sudo's options that precede the "guest command". Safe to ditch (after replacing all $i with 1).
The variable $split is only referenced in _init_completion (permalink), and it seems to be used for handling different argument styles (--foo=bar v.s. --foo bar). Same with the -s flag. Irrelevant.
Appending to $PATH and setting $root_command have to do with privilege escalation. Only relevant to sudo.
So after the dust has cleared, by process of elimination, I ended up with this simple chunk of code:
_my-script()
{
local cur prev words cword
_init_completion || return
_command_offset 1
} && complete -F _my-script my-script
Declaring these four local variables and calling _init_completion is standard for all completion scipts, so really it's as simple as one command. Of course someone had to write the massively-complex _command_offset function so lucky me I guess?
Anyways, thank you for reading the story of me messing around and hopefully this will be helpful to some other person in the future.
I see this pattern every once in a while, especially in questions about Bash prompt customization.
alias f='_ () { useful code; }; _'
I can see no reason at all to create an alias here. The obvious refactoring
f () { useful code; }
which avoids declaring an alias altogether, and simply defines the function once and for all, seems simpler, more understandable, less brittle, and more efficient. (In case it's not obvious, the alias ends up redeclaring the function every time you invoke the alias.)
For example, Make a Bash alias that takes a parameter? has several answers which exhibit this technique. bash script to run lftp with ftp path is a question which has code like this in a question about the actual functionality inside the function, and the OP doesn't explain why even though I prodded gently.
Is this just plainly an antipattern, or is there an actual reason to do this? Under what circumstances would this design make sense?
This is not about aliases with a space after them, or about code obfuscation (the examples I have found are generally entirely readable, apart from this mystifying technique).
Here are my 2 cents on this and it represents my personal opinion as well as understanding on the topic.
Using aliases with functions is to some extent a personal preference of developers. I will add some differences between the two approaches, which may also account for personal preferences of using aliases vs functions
There are times when most of the things I want to do are possible with aliases itself but only a few require to take a parameter. So instead of mixing aliases with functions, I use an alias with the function itself
Example:
alias kgps='kubectl get pods --all-namespaces | grep '
This works great and I can search my kubernetes pods. Now for deleting these pods, I need to pass the same parameter but in between the command, so I use an alias with a function inside
alias kdp="_(){ kubectl get pods --all-namespaces | grep \$1 | awk '{print \$2}' | xargs kubectl delete pod; }; _"
So most of my shortcut commands are possible to execute through aliases and only few which needs such things I use aliases with functions.
Aliases vs Functions
Now there are few differences between aliases and functions which I would like to highlight
Aliases can override system commands much more easily compared to functions
If I need to override ls, I can do that much easier with alias
alias ls='ls -altrh'
While a function equivalent of the same would be like below
ls() { command ls -altrh "$#";}
ls() { /bin/ls -altrh "$#";}
Aliases intention is mostly for shortcuts
Aliases are majorly used to create shortcut commands while functions are used for a lot of things, complex combinations of commands, auto-completion, bash prompts
Aliases are easier to manage
Run alias command you get a list of currently active aliases
$ alias
....
vs='vagrant ssh'
vu='vagrant up'
vus='vu && vs'
....
To get the list of functions we need to use declare -f or another similar command
$ declare -f | wc -l
8226
$ alias | wc -l
217
Now if I post a partial output of declare -f I get
$ declare -f
...
vi_mode_prompt_info () {
return 1
}
virtualenv_prompt_info () {
return 1
}
work_in_progress () {
if $(git log -n 1 2>/dev/null | grep -q -c "\-\-wip\-\-")
then
echo "WIP!!"
fi
}
zle-line-finish () {
echoti rmkx
}
zle-line-init () {
echoti smkx
}
zsh_stats () {
fc -l 1 | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n20
}
As you can see there are lots of functions which are used but are not relevant to me. While the alias command gives me a very concise output and I can easily see what all is there. In my case, 100% of them are shortcut commands
Escaping aliases and functions syntax is different for system commands
To escape a defined alias you need to prefix it with \ while for functions you need to either use command <originalcommand> or absolute path of the command /bin/originalcommand
Aliases have higher priority over function
Look at the below example
alias ls='echo alias && ls'
$ ls() { /bin/ls -al }
alias
$ ls
alias
total 23173440
drwxrwxr-x+ 255 tarunlalwani staff 8160 Jul 30 22:39 .
drwxr-xr-x+ 113 tarunlalwani staff 3616 Jul 30 23:12 ..
...
As you can see when we run the ls command, first the alias is used and then the next ls is calling the function.
This becomes also a way of wrapping an exiting function with the same name and re-using the original function inside as well, which can only be done using alias and promotes the format in the question
I found this answer too [U&L] In Bash, when to alias, when to script, and when to write a function? which explains the benefit of defining a function in an alias.
The benefit of doing so over declaring a function is that your alias
cannot be simply overwritten by source-ing (or using .) a script which
happens to declare a same-named function.
I found an Ask Ubuntu question about a related topic where one of the answers alleges that this is a misunderstanding of a different design principle: give the function a long and descriptive name, and create a shorter alias for convenience.
This still offers no insight into why you would have the alias redeclare the function every time.
You can use the alias for turning on and off a function that you don't want to change.
Suppose you have have code that calls the function _. You can switch the implementation of the function for another one with
alias f='_ () { echo "useful code"; }; _'
alias g='_ () { echo "Other useful code"; }; _'
alias h='_ () { echo "Special code"; }; _'
And now you can call
f
_
g
_
h
_
f
#DavidC.Rankin commented correctly, that it looked terrible.
I agree.
I thought of some way to use it. You might use it for testing software, something like
alias ok='commitTransaction () { echo "commited"; return 0; }'
alias nok='commitTransaction () { echo "unknown error"; return 1; }'
alias locked='commitTransaction () { echo "locked"; return 2; }'
alias slow='commitTransaction () { sleep 20; echo "commited"; return 0; }'
And now the tester can run his testcases:
ok
# And now start ok test
nok
# And now start nok test
Still hacking, why not make a better teststub?
Is this just plainly an antipattern (sic)...
I think its prevalence may just be cargo cult programming. Aliases are easy to understand, so users often learn them first. As users' skill and needs increase, they discover aliases lack flexible argument processing. So they do a web search, like "shell alias parameter passing", and find posts suggesting the pattern:
alias foo='_() { echo $2 $3 $1; }; _'
Lo and behold, it works. Users are happy, and they move on.
But because the _() sequence looks a lot like a shell incantation (2>&1, >>, etc.), users never think that _() is just compact syntax for function _ and never go to the next step of learning functions. With this alias pattern, they get all the benefit of functions and don't have to learn "new" syntax. Most probably never even notice the nasty side effect: overwriting any prior functions named _.
I searched through Usenet from 1981 to 1991, but I didn't find any direct evidence of this theory, however.
... or is there an actual reason to do this? Under what circumstances would this design make sense?
In the five days I drafted this answer, every reason I conjured came back to an argument against it. The selected answer - that aliases can't be masked in subshells - is a solid reason, though I've never thought to be that paranoid: I don't go around sourc'ing code I've not fully vetted.
I'm trying to find a way to emulate the behavior of set -e in a function, but only within the scope of that function.
Basically, I want a function where if any simple command would trigger set -e it returns 1 up one level. The goal is to isolate sets of risky jobs into functions so that I can gracefully handle them.
If you want any failing command to return 1, you can achieve that by following each command with || return 1.
For instance:
false || return 1 # This will always return 1
I am a big fan of never letting any command fail without explicit handling. For my scripts, I am using an exception handling technique where I return errors in a way that is not return codes, and trap all errors (with bash traps). Any command with a non-zero return code automatically means an improperly handled situation or bug, and I prefer my scripts to fail as soon as such a situation occurs.
Caution: I highly advise against using this technique. If you run the function in a subshell environment, you almost get the behavior you desire. Consider:
#!/bin/bash
foo() ( # Use parens to get a sub-shell
set -e # Does not impact the main script
echo This is executed
false
echo This should *not* be executed
)
foo # Function call fails, returns 1
echo return: $?
# BUT: this is a good reason to avoid this technique
if foo; then # Set -e is invalid in the function
echo Foo returned 0!!
else
echo fail
fi
false # Demonstrates that set -e is not set for the script
echo ok
Seems like you are looking for "nested exceptions" somewhat like what Java gives. For your requirement of scoping it, how about doing a set -e at the beginning of the function and making sure to run set +e before returning from it?
Another idea, which is not efficient or convenient, is to call your function in a subshell:
# some code
(set -e; my_function)
if [[ $? -ne 0 ]]; then
# the function didn't succeed...
fi
# more code
In any case, please be aware that set -e is not the greatest way to handle errors in a shell script. There are way too many issues making it quite unreliable. See these related posts:
What does set -e mean in a bash script?
Error handling in Bash
The approach I take for large scripts that need to exist for a long time in a production environment is:
create a library of functions to do all the standard stuff
the library will have a wrapper around each standard action (say, mv, cp, mkdir, ln, rm, etc.) that would validate the arguments carefully and also handle exceptions
upon exception, the wrapper exits with a clear error message
the exit itself could be a library function, somewhat like this:
--
# library of common functions
trap '_error_handler' ERR
trap '_exit_handler' EXIT
trap '_int_handler' SIGINT
_error_handler() {
# appropriate code
}
# other handlers go here...
#
exit_if_error() {
error_code=${1:-0}
error_message=${2:-"Uknown error"}
[[ $error_code == 0 ]] && return 0 # it is all good
# this can be enhanced to print out the "stack trace"
>&2 printf "%s\n" $error_message
# out of here
my_exit $error_code
}
my_exit() {
exit_code=${1:-0}
_global_graceful_exit=1 # this can be checked by the "EXIT" trap handler
exit $exit_code
}
# simple wrapper for cp
my_cp() {
# add code to check arguments more effectively
cp $1 $2
exit_if_error $? "cp of '$1' to '$2' failed"
}
# main code
source /path/to/library.sh
...
my_cp file1 file2
# clutter-free code
This, along with effective use of trap to take action on ERR and EXIT events, would be a good way to write reliable shell scripts.
Doing more research, I found a solution I rather like in Google's Shell Style Guide. There are some seriously interesting suggestions here, but I think I'm going to go with this for readability:
if ! mv "${file_list}" "${dest_dir}/" ; then
echo "Unable to move ${file_list} to ${dest_dir}" >&2
exit "${E_BAD_MOVE}"
fi
I come across set -e a some time ago and I admit I love it.
Now, after some time I'm back to write some bash scripting.
My question is if there are some best practices when to use set -e and when not to use it (.e.g. in small/big scripts etc.) or should I rather use a pattern like cmd || exit 1 to track errors?
Yes, you should always use it. People make fun of Visual Basic all the time, saying it's not a real programming language, partly because of its “On Error Resume Next” statement. Yet that is the default in shell! set -e should have been the default. The potential for disaster is just too high.
In places where it's ok for a command to fail, you can use || true or its shortened form ||:, e.g.
grep Warning build.log ||:
In fact you should go a step further, and have
set -eu
set -o pipefail
at the top of every bash script.
-u makes it an error to reference a non-existent environment variable such as ${HSOTNAME}, at the cost of requiring some gymnastics with checking ${#} before you reference ${1}, ${2}, and so on.
pipefail makes things like misspeled-command | sed -e 's/^WARNING: //' raise errors.
If your script code checks for errors carefully and properly where necessary, and handles them in an appropriate manner, then you probably don't ever need or want to use set -e.
On the other hand if your script is a simple sequential list of commands to be run one after another, and if you want the script to terminate if any one of those fail, then sticking set -e at the top would be exactly what you would want to do to keep your script simple and uncluttered. A perfect example of this would be if you're creating a script to compile a set of sources and you want the compile to stop after the first file with errors is encountered.
More complex scripts can combine these methods since you can use set +e to turn its effect back off again and go back to explicit error checking.
Note that although set -e is supposed to cause the shell to exit IFF any untested command fails, it is wise to turn it off again when your code is doing its own error handling as there can easily be weird cases where a command will return a non-zero exit status that you're not expecting, and possibly even such cases that you might not catch in testing, and where sudden fatal termination of your script would leave something in a bad state. So, don't use set -e, or leave it turned on after using it briefly, unless you really know that you want it.
Note also that you can still define an error handler with trap ERR to do something on an error condition when set -e is in effect, as that will still be run before the shell exits.
You love it!?
For my self, I prefer in a wide, having in my .bashrc a line like this:
trap '/usr/games/fortune /usr/share/games/fortunes/bofh-excuses' ERR
( on debian: apt-get install fortunes-bofh-excuses :-)
But it's only my preference ;-)
More seriously
lastErr() {
local RC=$?
history 1 |
sed '
s/^ *[0-9]\+ *\(\(["'\'']\)\([^\2]*\)\2\|\([^"'\'' ]*\)\) */cmd: \"\3\4\", args: \"/;
s/$/", rc: '"$RC/"
}
trap "lastErr" ERR
Gna
bash: Gna : command not found
cmd: "Gna", args: "", rc: 127
Gna gna
cmd: "Gna", args: "gna", rc: 127
"Gna gna" foo
cmd: "Gna gna", args: "foo", rc: 127
Well, from there, you could:
trap "lastErr >>/tmp/myerrors" ERR
"Gna gna" foo
cat /tmp/myerrors
cmd: "Gna gna", args: "foo", rc: 1
Or better:
lastErr() {
local RC=$?
history 1 |
sed '
s/^ *[0-9]\+ *\(\(["'\'']\)\([^\2]*\)\2\|\([^"'\'' ]*\)\) */cmd: \"\3\4\", args: \"/;
s/$/", rc: '"$RC/
s/^/$(date +"%a %d %b %T ")/"
}
"Gna gna" foo
cat /tmp/myerrors
cmd: "Gna gna", args: "foo", rc: 1
Tue 20 Nov 18:29:18 cmd: "Gna gna", args: "foo", rc: 127
... You could even add other informations like $$, $PPID, $PWD or maybe your..
When this option is on, if a simple command fails for any of the reasons listed in Consequences of Shell Errors or returns an exit status value >0, and is not part of the compound list following a while, until, or if keyword, and is not a part of an AND or OR list, and is not a pipeline preceded by the ! reserved word, then the shell shall immediately
exit.
I get a file descriptor leak when running the following code:
function get_fd_count() {
local fds
cd /proc/$$/fd; fds=( * ) # avoid a StackOverflow source colorizer bug
echo "${#fds[#]}"
}
function fd_leak_func() {
while : ; do
echo ">> Current FDs: $(get_fd_count)"
read retval new_state < <(set +e; new_state=$(echo foo); retval=$?; printf "%d %s\n" $retval $new_state)
done
}
fd_leak_func
Tested on both 3.2.25 and 4.0.28.
This only happens when the loop is happening within a function; every time we return to top-level context, the extra file descriptors are closed.
Is this intended behavior? More to the point, are workarounds available?
Followup: After reporting to the bash-bug mailing list, this was confirmed as a bug. Chet indicated that a fix will be included in the next release (as of 4/17/2010).
Here's a simplified example:
$ fd_leaker() { while :; do read a < <(pwd); c=(/proc/$$/fd/*); c=${#c[#]}; echo $c; done; }
$ fd_leaker
This one is not fixed by using /bin/true but it's mostly fixed by using (exit 0) But I get "bash: echo: write error: Interrupted system call" errors using that "fix" or if I use /bin/pwd instead of the builtin pwd.
It also seems to be specific to read. I tried grep . < <(pwd) > /dev/null and it worked properly. When I tried while read a; do :; done < <( pwd)
The extra file descriptors in the form of:
lr-x------ 1 user user 64 2010-04-15 19:26 39 -> pipe:[8357879]
I really don't think the runaway creation of them is intended, after all there's nothing recursive going on. I really don't see how adding something in the loop should fix things.
Putting a /bin/true at the end of the loop fixes it, but I don't know why or how, or why it happens in the first place.
It seems to be fixed in bash-4.2. I tested with bash-4.2.28 particularly