Generate warning when multiple executables in path when executing command in bash - bash

Say I have:
>which -a foo
/bin/foo
/usr/bin/foo
I want something like:
>foo
Warning: multiple foo in PATH
... foo gets executed ...
This functionality would have saved me really really lots of time today.
I should have guessed this is happening earlier but the problem was unclear
to me at the beggining and I started to dig in quite opposite direction.

Well, you can do it, but it is not so easy as you may think.
First, you need to create a function that will check
all directories in the PATH, and look there for the command you try to run.
Then you need to bind this function to the DEBUG trap of your current shell.
I wrote a small script that does that:
$ cat /tmp/1.sh
check_command()
{
n=0
DIRS=""
for i in $(echo $PATH| tr : " ")
do
if [ -x "$i/$1" ]
then
n=$[n+1]
DIRS="$DIRS $i"
fi
done
if [ "$n" -gt 1 ]
then
echo "Warning: there are multiple commands in different PATH directories: "$DIRS
fi
}
preexec () {
check_command $1
}
preexec_invoke_exec () {
[ -n "$COMP_LINE" ] && return # do nothing if completing
local this_command=`history 1 | sed -e "s/^[ ]*[0-9]*[ ]*//g"`;
preexec "$this_command"
}
trap 'preexec_invoke_exec' DEBUG
Example of usage:
$ . /tmp/1.sh
$ sudo cp /bin/date /usr/bin/test_it
$ sudo cp /bin/date /bin/test_it
$ test_it
Warning: there are multiple commands in different PATH directories: /usr/bin /bin
Wed Jul 11 15:14:43 CEST 2012
$

It is possible, though a bit trick to generalise. See my answer to https://unix.stackexchange.com/q/42579/20437 for the history and PROMPT_COMMAND magic. Your checkSanity function would look something like this:
checkSanity() {
cmd="${1%% *}" # strip everything but the first word
candidates=$(which -a $cmd | wc -l)
if (( candidates > 1 )); then
echo "Warning: multiple $cmd in PATH"
fi
}
But that will print the warning after the command finishes, not at the start. Use the DEBUG trap instead to get the wanted result:
trap 'checkSanity "$BASH_COMMAND"' DEBUG

Related

bash script: how to "exit" from sourced script, and allow to work non sourced?

I have a script that I'd like people to source, but optionally so. So they can run it with or without sourcing it, it's up to them.
e.g. The following should both work:
$ . test.sh
$ test.sh
The problem is, test.sh contains exit statements if correct args aren't passed in. If someone sources the script, then the exit commands exit the terminal!
I've done a bit of research and see from this StackOverflow post that I could detect if it's being sourced, and do something different, but what would that something different be?
The normal way to exit from a sourced script is simply to return (optionally adding the desired exit code) outside of any function. Assuming that when run as a command we have the -e flag set, this will also exit from a shell program:
#!/bin/sh -eu
if [ $# = 0 ]
then
echo "Usage $0 <argument>" >&2
return 1
fi
If we're running without -e, we might be able to return || exit instead.
There may be better ways to do this, but here's a sample script showing how I got this to work:
bparks#home
$ set | grep TESTVAR
bparks#home
$ ./test.sh
Outputs some useful information to the console. Please pass one arg.
bparks#home
$ set | grep TESTVAR
bparks#home
$ . ./test.sh
Outputs some useful information to the console. Please pass one arg.
bparks#home
$ set | grep TESTVAR
bparks#home
$ ./test.sh asdf
export TESTVAR=me
bparks#home
$ set | grep TESTVAR
bparks#home
$ . ./test.sh asdf
bparks#home
$ set | grep TESTVAR
TESTVAR=me
bparks#home
$
test.sh
#!/usr/bin/env bash
# store if we're sourced or not in a variable
(return 0 2>/dev/null) && SOURCED=1 || SOURCED=0
exitIfNotSourced(){
[[ "$SOURCED" != "0" ]] || exit;
}
showHelp(){
IT=$(cat <<EOF
Outputs some useful information to the console. Please pass one arg.
EOF
)
echo "$IT"
}
# Show help if no args supplied - works if sourced or not sourced
if [ -z "$1" ]
then
showHelp
exitIfNotSourced;
return;
fi
# your main script follows
# this sample shows exporting a variable if sourced,
# and outputting this to stdout if not sourced
if [ "$SOURCED" == "1" ]
then
export TESTVAR=me
else
echo "export TESTVAR=me"
fi
Checkout this answer for better description and porper solution.
And here is how it is used in docker-entrypoint.sh in official Mysql image:
# check to see if this file is being run or sourced from another script
_is_sourced() {
# https://unix.stackexchange.com/a/215279
[ "${#FUNCNAME[#]}" -ge 2 ] \
&& [ "${FUNCNAME[0]}" = '_is_sourced' ] \
&& [ "${FUNCNAME[1]}" = 'source' ]
}

Eval error Argument list too long when expandind a command

I got some code from here that works pretty well until I get "Argument list too long"
I am NOT a developer and pretty old too :) so if it is not much to ask please explain.
Is there a way the expand DIRCMD like eval does and pass each of the commands one at the time so eval does not break?
for (( ifl=0;ifl<$((NUMFIRSTLEVELDIRS));ifl++ )) { FLDIR="$(get_rand_dirname)"
FLCHILDREN="";
for (( ird=0;ird<$((DIRDEPTH-1));ird++ )) {
DIRCHILDREN=""; MOREDC=0;
for ((idc=0; idc<$((MINDIRCHILDREN+RANDOM%MAXDIRCHILDREN)); idc++)) {
CDIR="$(get_rand_dirname)" ;
# make sure comma is last, so brace expansion works even for 1 element? that can mess with expansion math, though
if [ "$DIRCHILDREN" == "" ]; then
DIRCHILDREN="\"$CDIR\"" ;
else
DIRCHILDREN="$DIRCHILDREN,\"$CDIR\"" ;
MOREDC=1 ;
fi
}
if [ "$MOREDC" == "1" ] ; then
if [ "$FLCHILDREN" == "" ]; then
FLCHILDREN="{$DIRCHILDREN}" ;
else
FLCHILDREN="$FLCHILDREN/{$DIRCHILDREN}" ;
fi
else
if [ "$FLCHILDREN" == "" ]; then
FLCHILDREN="$DIRCHILDREN" ;
else
FLCHILDREN="$FLCHILDREN/$DIRCHILDREN" ;
fi
fi
}
cd $OUTDIR
DIRCMD="mkdir -p $OUTDIR/\"$FLDIR\"/$FLCHILDREN"
eval "$DIRCMD"
echo "$DIRCMD"
}
I tried echo $DIRCMD but do not get the expanded list of commands
'echo mkdir -p /mnt/nvme-test/rndpath/"r8oF"/{"rc","XXKR","p0H"}/{"5Dw0K","oT","rV","coU","uo"}/{"3m5m","uEdA","w4SJp","49"}'
I had trouble following the code, but if I understood it correctly, you dynamically generate a mkdir -p command with a brace expansion:
'mkdir -p /mnt/nvme-test/rndpath/"r8oF"/{"rc","XXKR","p0H"}/{"5Dw0K","oT","rV","coU","uo"}/{"3m5m","uEdA","w4SJp","49"}'
Which then fails when you eval it due to your OS' maximum argument limit.
To get around that, you can instead generate a printf .. command since this is a Bash builtin and not subject to the argument limit, and feed its output to xargs:
dircmd='printf "%s\0" /mnt/nvme-test/rndpath/"r8oF"/{"rc","XXKR","p0H"}/{"5Dw0K","oT","rV","coU","uo"}/{"3m5m","uEdA","w4SJp","49"}'
eval "$dircmd" | xargs -0 mkdir -p
If your xargs doesn't support -0, you can instead use printf "%s\n" and xargs mkdir -p, though it won't behave as well if your generated names contain spaces and such.
If this is for benchmarking, you may additionally be interested to know that you can now use xargs -0 -n 1000 -P 8 mkdir -p to run 8 mkdirs in parallel, each creating 1000 dirs at a time.

bash: script to identify specific alias causing a bug

[Arch Linux v5.0.7 with GNU bash 5.0.3]
Some .bashrc aliases seem to conflict with a bash shell-scripts provided by pyenv and pyenv-virtualenvwrapper.I tracked down the problem running the script, using set -x and with all aliases enabled, and saw finally that the script exits gracefully with exit code is 0 only when aliases are disabled with unalias -a. So this has to do with aliases... but which one ?
To try to automate that, I wrote the shell-script below:
It un-aliases one alias at a time, reading iteratively from the complete list of aliases,
It tests the conflicting shell script test.sh against that leave-one-out alias configuration, and prints something in case an error is detected,
It undoes the previous un-aliasing,
It goes on to un-aliasing the next alias.
But the two built-ins alias and unalias do not fare well in the script cac.sh below:
#! /usr/bin/bash
[ -e aliases.txt ] && rm -f aliases.txt
alias | sed 's/alias //' | cut -d "=" -f1 > aliases.txt
printf "File aliases.txt created with %d lines.\n" \
"$(wc -l < <(\cat aliases.txt))"
IFS=" "
n=0
while read -r line || [ -n "$line" ]; do
n=$((n+1))
aliasedAs=$( alias "$line" | sed 's/alias //' )
printf "Line %2d: %s\n" "$n" "$aliasedAs"
unalias "$line"
[ -z $(eval "$*" 1> /dev/null) ] \ # check output to stderr only
&& printf "********** Look up: %s\n" "$line"
eval "${aliasedAs}"
done < <(tail aliases.txt) # use tail + proc substitution for testing only
Use the script like so: $ cac.sh test.sh [optional arguments to test.sh] Any test.sh will do. It just needs to return some non-empty string to stderr.
The first anomaly is that the file aliases.txt is empty as if the alias builtin was not accessible from within the script. If I start the script from its 3rd line, using an already populated aliases.txt file, the script fails at the second line within the while block, again as if alias could not be called from within the script. Any suggestions appreciated.
Note: The one liner below works in console:
$ n=0;while read -r line || [ -n "$line" ]; do n=$((n+1)); printf "alias %d : %s\n" "$n" "$(alias "$line" | sed 's/alias //')"; done < aliases.txt
I would generally advise against implementing this as an external script at all -- it makes much more sense as a function that can be evaluated directly in your interactive shell (which is, after all, where all the potentially-involved aliases are defined).
print_result() {
local prior_retval=$? label=$1
if (( prior_retval == 0 )); then
printf '%-30s - %s\n' "$label" WORKS >&2
else
printf '%-30s - %s\n' "$label" BROKEN >&2
fi
}
test_without_each_alias() {
[ "$#" = 1 ] || { echo "Usage: test_without_each_alias 'code here'" >&2; return 1; }
local alias
(eval "$1"); print_result "Unchanged aliases"
for alias in "${!BASH_ALIASES[#]}"; do
(unalias "$alias" && eval "$1"); print_result "Without $alias"
done
}
Consider the following:
rm_in_home_only() { [[ $1 = /home/* ]] || return 1; rm -- "$#"; }
alias rm=rm_in_home_only # alias actually causing our bug
alias red_herring=true # another alias that's harmless
test_without_each_alias 'touch /tmp/foobar; rm /tmp/foobar; [[ ! -e /tmp/foobar ]]'
...which emits something like:
Unchanged aliases - BROKEN
Without rm - WORKS
Without red_herring - BROKEN
Note that if the code you pass executes a function, you'll want to be sure that the function is defined inside the eval'd code; since aliases are parser behavior, they take place when functions are defined, not when functions are run.
#Kamil_Cuk, #Benjamin_W and #cdarke all pointed to the fact that a noninteractive shell (as that spawned from a bash script) does not have access to aliases.
#CharlesDuffy pointed to probable word splitting and glob expansion resulting in something that could be invalid test syntax in the original [ -z $(eval "$*" 1> /dev/null) ] block above, or worse yet in the possibility of $(eval "$*" 1> /dev/null) being parsed as a glob resulting in unpredictable script behavior. Block corrected to: [ -z "$(eval "$*" 1> /dev/null)" ].
Making the shell spawned by cac.sh interactive, with #! /usr/bin/bash -i. make the two built-ins alias and unalias returned non-null result when invoked, and BASH_ALIASES[#] became accessible from within the script.
#! /usr/bin/bash -i
[ -e aliases.txt ] && rm -f aliases.txt
alias | sed 's/alias //' | cut -d "=" -f1 > aliases.txt
printf "File aliases.txt created with %d lines.\n" \
"$(wc -l < <(\cat aliases.txt))"
IFS=" "
while read -r line || [ -n "$line" ]; do
aliasedAs=$( alias "$line" | sed 's/alias //' )
unalias "$line"
[ -z "$(eval "$*" 2>&1 1>/dev/null)" ] \ # check output to stderr only
&& printf "********** Look up: %s\n" "$line"
eval "${aliasedAs}"
done < aliases.txt
Warning: testing test.sh resorts to the eval built-in. Arbitrary code can be executed on your system if test.sh and optional arguments do not come from a trusted source.

OS X Bash, 'watch' command

I'm looking for the best way to duplicate the Linux 'watch' command on Mac OS X. I'd like to run a command every few seconds to pattern match on the contents of an output file using 'tail' and 'sed'.
What's my best option on a Mac, and can it be done without downloading software?
With Homebrew installed:
brew install watch
You can emulate the basic functionality with the shell loop:
while :; do clear; your_command; sleep 2; done
That will loop forever, clear the screen, run your command, and wait two seconds - the basic watch your_command implementation.
You can take this a step further and create a watch.sh script that can accept your_command and sleep_duration as parameters:
#!/bin/bash
# usage: watch.sh <your_command> <sleep_duration>
while :;
do
clear
date
$1
sleep $2
done
Use MacPorts:
$ sudo port install watch
The shells above will do the trick, and you could even convert them to an alias (you may need to wrap in a function to handle parameters):
alias myWatch='_() { while :; do clear; $2; sleep $1; done }; _'
Examples:
myWatch 1 ls ## Self-explanatory
myWatch 5 "ls -lF $HOME" ## Every 5 seconds, list out home directory; double-quotes around command to keep its arguments together
Alternately, Homebrew can install the watch from http://procps.sourceforge.net/:
brew install watch
It may be that "watch" is not what you want. You probably want to ask for help in solving your problem, not in implementing your solution! :)
If your real goal is to trigger actions based on what's seen from the tail command, then you can do that as part of the tail itself. Instead of running "periodically", which is what watch does, you can run your code on demand.
#!/bin/sh
tail -F /var/log/somelogfile | while read line; do
if echo "$line" | grep -q '[Ss]ome.regex'; then
# do your stuff
fi
done
Note that tail -F will continue to follow a log file even if it gets rotated by newsyslog or logrotate. You want to use this instead of the lower-case tail -f. Check man tail for details.
That said, if you really do want to run a command periodically, the other answers provided can be turned into a short shell script:
#!/bin/sh
if [ -z "$2" ]; then
echo "Usage: $0 SECONDS COMMAND" >&2
exit 1
fi
SECONDS=$1
shift 1
while sleep $SECONDS; do
clear
$*
done
I am going with the answer from here:
bash -c 'while [ 0 ]; do <your command>; sleep 5; done'
But you're really better off installing watch as this isn't very clean...
If watch doesn't want to install via
brew install watch
There is another similar/copy version that installed and worked perfectly for me
brew install visionmedia-watch
https://github.com/tj/watch
Or, in your ~/.bashrc file:
function watch {
while :; do clear; date; echo; $#; sleep 2; done
}
To prevent flickering when your main command takes perceivable time to complete, you can capture the output and only clear screen when it's done.
function watch {while :; do a=$($#); clear; echo "$(date)\n\n$a"; sleep 1; done}
Then use it by:
watch istats
Try this:
#!/bin/bash
# usage: watch [-n integer] COMMAND
case $# in
0)
echo "Usage $0 [-n int] COMMAND"
;;
*)
sleep=2;
;;
esac
if [ "$1" == "-n" ]; then
sleep=$2
shift; shift
fi
while :;
do
clear;
echo "$(date) every ${sleep}s $#"; echo
$#;
sleep $sleep;
done
Here's a slightly changed version of this answer that:
checks for valid args
shows a date and duration title at the top
moves the "duration" argument to be the 1st argument, so complex commands can be easily passed as the remaining arguments.
To use it:
Save this to ~/bin/watch
execute chmod 700 ~/bin/watch in a terminal to make it executable.
try it by running watch 1 echo "hi there"
~/bin/watch
#!/bin/bash
function show_help()
{
echo ""
echo "usage: watch [sleep duration in seconds] [command]"
echo ""
echo "e.g. To cat a file every second, run the following"
echo ""
echo " watch 1 cat /tmp/it.txt"
exit;
}
function show_help_if_required()
{
if [ "$1" == "help" ]
then
show_help
fi
if [ -z "$1" ]
then
show_help
fi
}
function require_numeric_value()
{
REG_EX='^[0-9]+$'
if ! [[ $1 =~ $REG_EX ]] ; then
show_help
fi
}
show_help_if_required $1
require_numeric_value $1
DURATION=$1
shift
while :; do
clear
echo "Updating every $DURATION seconds. Last updated $(date)"
bash -c "$*"
sleep $DURATION
done
Use the Nix package manager!
Install Nix, and then do nix-env -iA nixpkgs.watch and it should be available for use after the completing the install instructions (including sourcing . "$HOME/.nix-profile/etc/profile.d/nix.sh" in your shell).
The watch command that's available on Linux does not exist on macOS. If you don't want to use brew you can add this bash function to your shell profile.
# execute commands at a specified interval of seconds
function watch.command {
# USAGE: watch.commands [seconds] [commands...]
# EXAMPLE: watch.command 5 date
# EXAMPLE: watch.command 5 date echo 'ls -l' echo 'ps | grep "kubectl\\\|node\\\|npm\\\|puma"'
# EXAMPLE: watch.command 5 'date; echo; ls -l; echo; ps | grep "kubectl\\\|node\\\|npm\\\|puma"' echo date 'echo; ls -1'
local cmds=()
for arg in "${#:2}"; do
echo $arg | sed 's/; /;/g' | tr \; \\n | while read cmd; do
cmds+=($cmd)
done
done
while true; do
clear
for cmd in $cmds; do
eval $cmd
done
sleep $1
done
}
https://gist.github.com/Gerst20051/99c1cf570a2d0d59f09339a806732fd3

Checking in bash and csh if a command is builtin

How can I check in bash and csh if commands are builtin? Is there a method compatible with most shells?
You can try using which in csh or type in bash. If something is a built-in command, it will say so; otherwise, you get the location of the command in your PATH.
In csh:
# which echo
echo: shell built-in command.
# which parted
/sbin/parted
In bash:
# type echo
echo is a shell builtin
# type parted
parted is /sbin/parted
type might also show something like this:
# type clear
clear is hashed (/usr/bin/clear)
...which means that it's not a built-in, but that bash has stored its location in a hashtable to speed up access to it; (a little bit) more in this post on Unix & Linux.
In bash, you can use the type command with the -t option. Full details can be found in the bash-builtins man page but the relevant bit is:
type -t name
If the -t option is used, type prints a string which is one of alias, keyword, function, builtin, or file if name is an alias, shell reserved word, function, builtin, or disk file, respectively. If the name is not found, then nothing is printed, and an exit status of false is returned.
Hence you can use a check such as:
if [[ "$(type -t read)" == "builtin" ]] ; then echo read ; fi
if [[ "$(type -t cd)" == "builtin" ]] ; then echo cd ; fi
if [[ "$(type -t ls)" == "builtin" ]] ; then echo ls ; fi
which would result in the output:
read
cd
For bash, use type command
For csh, you can use:
which command-name
If it's built-in, it will tell so.
Not sure if it works the same for bash.
We careful with aliases, though. There may be options for that.
The other answers here are close, but they all fail if there is an alias or function with the same name as the command you're checking.
Here's my solution:
In tcsh
Use the where command, which gives all occurrences of the command name, including whether it's a built-in. Then grep to see if one of the lines says that it's a built-in.
alias isbuiltin 'test \!:1 != "builtin" && where \!:1 | egrep "built-?in" > /dev/null || echo \!:1" is not a built-in"'
In bash/zsh
Use type -a, which gives all occurrences of the command name, including whether it's a built-in. Then grep to see if one of the lines says that it's a built-in.
isbuiltin() {
if [[ $# -ne 1 ]]; then
echo "Usage: $0 command"
return 1
fi
cmd=$1
if ! type -a $cmd 2> /dev/null | egrep '\<built-?in\>' > /dev/null
then
printf "$cmd is not a built-in\n" >&2
return 1
fi
return 0
}
In ksh88/ksh93
Open a sub-shell so that you can remove any aliases or command names of the same name. Then in the subshell, use whence -v. There's also some extra archaic syntax in this solution to support ksh88.
isbuiltin() {
if [[ $# -ne 1 ]]; then
echo "Usage: $0 command"
return 1
fi
cmd=$1
if (
#Open a subshell so that aliases and functions can be safely removed,
# allowing `whence -v` to see the built-in command if there is one.
unalias "$cmd";
if [[ "$cmd" != '.' ]] && typeset -f | egrep "^(function *$cmd|$cmd\(\))" > /dev/null 2>&1
then
#Remove the function iff it exists.
#Since `unset` is a special built-in, the subshell dies if it fails
unset -f "$cmd";
fi
PATH='/no';
#NOTE: we can't use `whence -a` because it's not supported in older versions of ksh
whence -v "$cmd" 2>&1
) 2> /dev/null | grep -v 'not found' | grep 'builtin' > /dev/null 2>&1
then
#No-op
:
else
printf "$cmd is not a built-in\n" >&2
return 1
fi
}
Using the Solution
Once you applied the aforementioned solution in the shell of your choice, you can use it like this...
At the command line:
$ isbuiltin command
If the command is a built-in, it prints nothing; otherwise, it prints a message to stderr.
Or you can use it like this in a script:
if isbuiltin $cmd 2> /dev/null
then
echo "$cmd is a built-in"
else
echo "$cmd is NOT a built-in"
fi

Resources