How can I check in bash and csh if commands are builtin? Is there a method compatible with most shells?
You can try using which in csh or type in bash. If something is a built-in command, it will say so; otherwise, you get the location of the command in your PATH.
In csh:
# which echo
echo: shell built-in command.
# which parted
/sbin/parted
In bash:
# type echo
echo is a shell builtin
# type parted
parted is /sbin/parted
type might also show something like this:
# type clear
clear is hashed (/usr/bin/clear)
...which means that it's not a built-in, but that bash has stored its location in a hashtable to speed up access to it; (a little bit) more in this post on Unix & Linux.
In bash, you can use the type command with the -t option. Full details can be found in the bash-builtins man page but the relevant bit is:
type -t name
If the -t option is used, type prints a string which is one of alias, keyword, function, builtin, or file if name is an alias, shell reserved word, function, builtin, or disk file, respectively. If the name is not found, then nothing is printed, and an exit status of false is returned.
Hence you can use a check such as:
if [[ "$(type -t read)" == "builtin" ]] ; then echo read ; fi
if [[ "$(type -t cd)" == "builtin" ]] ; then echo cd ; fi
if [[ "$(type -t ls)" == "builtin" ]] ; then echo ls ; fi
which would result in the output:
read
cd
For bash, use type command
For csh, you can use:
which command-name
If it's built-in, it will tell so.
Not sure if it works the same for bash.
We careful with aliases, though. There may be options for that.
The other answers here are close, but they all fail if there is an alias or function with the same name as the command you're checking.
Here's my solution:
In tcsh
Use the where command, which gives all occurrences of the command name, including whether it's a built-in. Then grep to see if one of the lines says that it's a built-in.
alias isbuiltin 'test \!:1 != "builtin" && where \!:1 | egrep "built-?in" > /dev/null || echo \!:1" is not a built-in"'
In bash/zsh
Use type -a, which gives all occurrences of the command name, including whether it's a built-in. Then grep to see if one of the lines says that it's a built-in.
isbuiltin() {
if [[ $# -ne 1 ]]; then
echo "Usage: $0 command"
return 1
fi
cmd=$1
if ! type -a $cmd 2> /dev/null | egrep '\<built-?in\>' > /dev/null
then
printf "$cmd is not a built-in\n" >&2
return 1
fi
return 0
}
In ksh88/ksh93
Open a sub-shell so that you can remove any aliases or command names of the same name. Then in the subshell, use whence -v. There's also some extra archaic syntax in this solution to support ksh88.
isbuiltin() {
if [[ $# -ne 1 ]]; then
echo "Usage: $0 command"
return 1
fi
cmd=$1
if (
#Open a subshell so that aliases and functions can be safely removed,
# allowing `whence -v` to see the built-in command if there is one.
unalias "$cmd";
if [[ "$cmd" != '.' ]] && typeset -f | egrep "^(function *$cmd|$cmd\(\))" > /dev/null 2>&1
then
#Remove the function iff it exists.
#Since `unset` is a special built-in, the subshell dies if it fails
unset -f "$cmd";
fi
PATH='/no';
#NOTE: we can't use `whence -a` because it's not supported in older versions of ksh
whence -v "$cmd" 2>&1
) 2> /dev/null | grep -v 'not found' | grep 'builtin' > /dev/null 2>&1
then
#No-op
:
else
printf "$cmd is not a built-in\n" >&2
return 1
fi
}
Using the Solution
Once you applied the aforementioned solution in the shell of your choice, you can use it like this...
At the command line:
$ isbuiltin command
If the command is a built-in, it prints nothing; otherwise, it prints a message to stderr.
Or you can use it like this in a script:
if isbuiltin $cmd 2> /dev/null
then
echo "$cmd is a built-in"
else
echo "$cmd is NOT a built-in"
fi
Related
I want to create a utility function for bash to remove duplicate lines. I am using function
function remove_empty_lines() {
if ! command -v awk &> /dev/null
then
echo '[x] ERR: "awk" command not found'
return
fi
if [[ -z "$1" ]]
then
echo "usage: remove_empty_lines <file-name> [--replace]"
echo
echo "Arguments:"
echo -e "\t--replace\t (Optional) If not passed, the result will be redirected to stdout"
return
fi
if [[ ! -f "$1" ]]
then
echo "[x] ERR: \"$1\" file not found"
return
fi
echo $0
local CMD="awk '!seen[$0]++' $1"
if [[ "$2" = '--reload' ]]
then
CMD+=" > $1"
fi
echo $CMD
}
If I am running the main awk command directly, it is working. But when i execute the same $CMD in the function, I am getting this error
$ remove_empty_lines app.js
/bin/bash
awk '!x[/bin/bash]++' app.js
The original code is broken in several ways:
When used with --reload, it would truncate the output file's contents before awk could ever read those contents (see How can I use a file in a command and redirect output to the same file without truncating it?)
It didn't ever actually run the command, and for the reasons described in BashFAQ #50, storing a shell command in a string is inherently buggy (one can work around some of those issues with eval; BashFAQ #48 describes why doing so introduces security bugs).
It wrote error messages (and other "diagnostic content") to stdout instead of stderr; this means that if your function's output was redirected to a file, you could never see its errors -- they'd end up jumbled into the output.
Error cases were handled with a return even in cases where $? would be zero; this means that return itself would return a zero/successful/truthy status, not revealing to the caller that any error had taken place.
Presumably the reason you were storing your output in CMD was to be able to perform a redirection conditionally, but that can be done other ways: Below, we always create a file descriptor out_fd, but point it to either stdout (when called without --reload), or to a temporary file (if called with --reload); if-and-only-if awk succeeds, we then move the temporary file over the output file, thus replacing it as an atomic operation.
remove_empty_lines() {
local out_fd rc=0 tempfile=
command -v awk &>/dev/null || { echo '[x] ERR: "awk" command not found' >&2; return 1; }
if [[ -z "$1" ]]; then
printf '%b\n' >&2 \
'usage: remove_empty_lines <file-name> [--replace]' \
'' \
'Arguments:' \
'\t--replace\t(Optional) If not passed, the result will be redirected to stdout'
return 1
fi
[[ -f "$1" ]] || { echo "[x] ERR: \"$1\" file not found" >&2; return 1; }
if [ "$2" = --reload ]; then
tempfile=$(mktemp -t "$1.XXXXXX") || return
exec {out_fd}>"$tempfile" || { rc=$?; rm -f "$tempfile"; return "$rc"; }
else
exec {out_fd}>&1
fi
awk '!seen[$0]++' <"$1" >&$out_fd || { rc=$?; rm -f "$tempfile"; return "$rc"; }
exec {out_fd}>&- # close our file descriptor
if [[ $tempfile ]]; then
mv -- "$tempfile" "$1" || return
fi
}
First off the output from your function call is not an error but rather the output of two echo commands (echo $0 and echo $CMD).
And as Charles Duffy has pointed out ... at no point is the function actually running the $CMD.
As for the inclusion of /bin/bash in your function's echo output ... the main problem is the reference to $0; by definition $0 is the name of the running process, which in the case of a function is the shell under which the function is being called. Consider the following when run from a bash command prompt:
$ echo $0
-bash
As you can see from your output this generates /bin/bash in your environment. See this and this for more details.
On a related note, the reference to $0 within double quotes causes the $0 to be evaluated, so this:
local CMD="awk '!seen[$0]++' $1"
becomes
local CMD="awk '!seen[/bin/bash]++' app.js"
I'm thinking what you want is something like:
echo $1 # the name of the file to be processed
local CMD="awk '!seen[\$0]++' $1" # escape the '$' in '$0'
becomes
local CMD="awk '!seen[$0]++' app.js"
That should fix the issues shown in your function's output; as for the other issues ... you're getting a good bit of feedback in the various comments ...
I want to write some wrappers around the sha1sum function in bash. From the manpage:
SHA1SUM(1) User Commands SHA1SUM(1)
NAME
sha1sum - compute and check SHA1 message digest
SYNOPSIS
sha1sum [OPTION]... [FILE]...
DESCRIPTION
Print or check SHA1 (160-bit) checksums.
With no FILE, or when FILE is -, read standard input.
How can I set up my wrapper so that it works in the same way? I.e.:
my_wrapper(){
# some code here
}
that could work both as:
my_wrapper PATH_TO_FILE
and
echo -n "blabla" | my_wrapper
I think this is somehow related to Redirect standard input dynamically in a bash script but not sure how to make it 'nicely'.
Edit 1
I program in a quite defensive way, so I use in my whole script:
# exit if a command fails
set -o errexit
# make sure to show the error code of the first failing command
set -o pipefail
# do not overwrite files too easily
set -o noclobber
# exit if try to use undefined variable
set -o nounset
Anything that works with that?
You can use this simple wrapper:
args=("$#") # save arguments into an array
set -o noclobber nounset pipefail errexit
set -- "${args[#]}" # set positional arguments from array
my_wrapper() {
[[ -f $1 ]] && SHA1SUM "$1" || SHA1SUM
}
my_wrapper "$#"
Note that you can use:
my_wrapper PATH_TO_FILE
or:
echo -n "blabla" | my_wrapper
This code works for me, put it in a file named wrapper
#!/bin/bash
my_wrapper(){
if [[ -z "$1" ]];then
read PARAM
else
PARAM="$1"
fi
echo "PARAM:$PARAM"
}
Load the function in your environment
. ./wrapper
Test the function with input pipe
root#51ce582167d0:~# echo hello | my_wrapper
PARAM:hello
Test the function with parameter
root#51ce582167d0:~# my_wrapper bybye
PARAM:bybye
Ok, so the answers posted here are fine often, but in my case with defensive programming options:
# exit if a command fails
set -o errexit
# exit if try to use undefined variable
set -o nounset
things do not work as well. So I am now using something in this kind:
digest_function(){
# argument is either filename or read from std input
# similar to the sha*sum functions
if [[ "$#" = "1" ]]
then
# this needs to be a file that exists
if [ ! -f $1 ]
then
echo "File not found! Aborting..."
exit 1
else
local ARGTYPE="Filename"
local PARAM="$1"
fi
else
local ARGTYPE="StdInput"
local PARAM=$(cat)
fi
if [[ "${ARGTYPE}" = "Filename" ]]
then
local DIGEST=$(sha1sum ${PARAM})
else
local DIGEST=$(echo -n ${PARAM} | sha1sum)
fi
}
[Arch Linux v5.0.7 with GNU bash 5.0.3]
Some .bashrc aliases seem to conflict with a bash shell-scripts provided by pyenv and pyenv-virtualenvwrapper.I tracked down the problem running the script, using set -x and with all aliases enabled, and saw finally that the script exits gracefully with exit code is 0 only when aliases are disabled with unalias -a. So this has to do with aliases... but which one ?
To try to automate that, I wrote the shell-script below:
It un-aliases one alias at a time, reading iteratively from the complete list of aliases,
It tests the conflicting shell script test.sh against that leave-one-out alias configuration, and prints something in case an error is detected,
It undoes the previous un-aliasing,
It goes on to un-aliasing the next alias.
But the two built-ins alias and unalias do not fare well in the script cac.sh below:
#! /usr/bin/bash
[ -e aliases.txt ] && rm -f aliases.txt
alias | sed 's/alias //' | cut -d "=" -f1 > aliases.txt
printf "File aliases.txt created with %d lines.\n" \
"$(wc -l < <(\cat aliases.txt))"
IFS=" "
n=0
while read -r line || [ -n "$line" ]; do
n=$((n+1))
aliasedAs=$( alias "$line" | sed 's/alias //' )
printf "Line %2d: %s\n" "$n" "$aliasedAs"
unalias "$line"
[ -z $(eval "$*" 1> /dev/null) ] \ # check output to stderr only
&& printf "********** Look up: %s\n" "$line"
eval "${aliasedAs}"
done < <(tail aliases.txt) # use tail + proc substitution for testing only
Use the script like so: $ cac.sh test.sh [optional arguments to test.sh] Any test.sh will do. It just needs to return some non-empty string to stderr.
The first anomaly is that the file aliases.txt is empty as if the alias builtin was not accessible from within the script. If I start the script from its 3rd line, using an already populated aliases.txt file, the script fails at the second line within the while block, again as if alias could not be called from within the script. Any suggestions appreciated.
Note: The one liner below works in console:
$ n=0;while read -r line || [ -n "$line" ]; do n=$((n+1)); printf "alias %d : %s\n" "$n" "$(alias "$line" | sed 's/alias //')"; done < aliases.txt
I would generally advise against implementing this as an external script at all -- it makes much more sense as a function that can be evaluated directly in your interactive shell (which is, after all, where all the potentially-involved aliases are defined).
print_result() {
local prior_retval=$? label=$1
if (( prior_retval == 0 )); then
printf '%-30s - %s\n' "$label" WORKS >&2
else
printf '%-30s - %s\n' "$label" BROKEN >&2
fi
}
test_without_each_alias() {
[ "$#" = 1 ] || { echo "Usage: test_without_each_alias 'code here'" >&2; return 1; }
local alias
(eval "$1"); print_result "Unchanged aliases"
for alias in "${!BASH_ALIASES[#]}"; do
(unalias "$alias" && eval "$1"); print_result "Without $alias"
done
}
Consider the following:
rm_in_home_only() { [[ $1 = /home/* ]] || return 1; rm -- "$#"; }
alias rm=rm_in_home_only # alias actually causing our bug
alias red_herring=true # another alias that's harmless
test_without_each_alias 'touch /tmp/foobar; rm /tmp/foobar; [[ ! -e /tmp/foobar ]]'
...which emits something like:
Unchanged aliases - BROKEN
Without rm - WORKS
Without red_herring - BROKEN
Note that if the code you pass executes a function, you'll want to be sure that the function is defined inside the eval'd code; since aliases are parser behavior, they take place when functions are defined, not when functions are run.
#Kamil_Cuk, #Benjamin_W and #cdarke all pointed to the fact that a noninteractive shell (as that spawned from a bash script) does not have access to aliases.
#CharlesDuffy pointed to probable word splitting and glob expansion resulting in something that could be invalid test syntax in the original [ -z $(eval "$*" 1> /dev/null) ] block above, or worse yet in the possibility of $(eval "$*" 1> /dev/null) being parsed as a glob resulting in unpredictable script behavior. Block corrected to: [ -z "$(eval "$*" 1> /dev/null)" ].
Making the shell spawned by cac.sh interactive, with #! /usr/bin/bash -i. make the two built-ins alias and unalias returned non-null result when invoked, and BASH_ALIASES[#] became accessible from within the script.
#! /usr/bin/bash -i
[ -e aliases.txt ] && rm -f aliases.txt
alias | sed 's/alias //' | cut -d "=" -f1 > aliases.txt
printf "File aliases.txt created with %d lines.\n" \
"$(wc -l < <(\cat aliases.txt))"
IFS=" "
while read -r line || [ -n "$line" ]; do
aliasedAs=$( alias "$line" | sed 's/alias //' )
unalias "$line"
[ -z "$(eval "$*" 2>&1 1>/dev/null)" ] \ # check output to stderr only
&& printf "********** Look up: %s\n" "$line"
eval "${aliasedAs}"
done < aliases.txt
Warning: testing test.sh resorts to the eval built-in. Arbitrary code can be executed on your system if test.sh and optional arguments do not come from a trusted source.
How do I check if file exists in bash?
When I try to do it like this:
FILE1="${#:$OPTIND:1}"
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
elif
<more code follows>
I always get following output:
requested file doesn't exist
The program is used like this:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
Any ideas please?
I will be glad for any help.
P.S. I wish I could show the entire file without the risk of being fired from school for having a duplicate. If there is a private method of communication I will happily oblige.
My mistake. Fas forcing a binary file into a wrong place. Thanks for everyone's help.
Little trick to debugging problems like this. Add these lines to the top of your script:
export PS4="\$LINENO: "
set -xv
The set -xv will print out each line before it is executed, and then the line once the shell interpolates variables, etc. The $PS4 is the prompt used by set -xv. This will print the line number of the shell script as it executes. You'll be able to follow what is going on and where you may have problems.
Here's an example of a test script:
#! /bin/bash
export PS4="\$LINENO: "
set -xv
FILE1="${#:$OPTIND:1}" # Line 6
if [ ! -e "$FILE1" ] # Line 7
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1" # Line 12
fi
And here's what I get when I run it:
$ ./test.sh .profile
FILE1="${#:$OPTIND:1}"
6: FILE1=.profile
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1"
fi
7: [ ! -e .profile ]
12: echo 'Found File .profile'
Found File .profile
Here, I can see that I set $FILE1 to .profile, and that my script understood that ${#:$OPTIND:1}. The best thing about this is that it works on all shells down to the original Bourne shell. That means if you aren't running Bash as you think you might be, you'll see where your script is failing, and maybe fix the issue.
I suspect you might not be running your script in Bash. Did you put #! /bin/bash on the top?
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
You may want to use getopts to parse your parameters:
#! /bin/bash
USAGE=" Usage:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
"
while getopts gpr:d: option
do
case $option in
g) g_opt=1;;
p) p_opt=1;;
r) rfunction_id="$OPTARG";;
d) dfunction_id="$OPTARG";;
[?])
echo "Invalid Usage" 1>&2
echo "$USAGE" 1>&2
exit 2
;;
esac
done
if [[ -n $rfunction_id && -n $dfunction_id ]]
then
echo "Invalid Usage: You can't specify both -r and -d" 1>&2
echo "$USAGE" >2&
exit 2
fi
shift $(($OPTIND - 1))
[[ -n $g_opt ]] && echo "-g was set"
[[ -n $p_opt ]] && echo "-p was set"
[[ -n $rfunction_id ]] && echo "-r was set to $rfunction_id"
[[ -n $dfunction_id ]] && echo "-d was set to $dfunction_id"
[[ -n $1 ]] && echo "File is $1"
To (recap) and add to #DavidW.'s excellent answer:
Check the shebang line (first line) of your script to ensure that it's executed by bash: is it #!/bin/bash or #!/usr/bin/env bash?
Inspect your script file for hidden control characters (such as \r) that can result in unexpected behavior; run cat -v scriptFile | fgrep ^ - it should produce NO output; if the file does contain \r chars., they would show as ^M.
To remove the \r instances (more accurately, to convert Windows-style \r\n newline sequences to Unix \n-only sequences), you can use dos2unix file to convert in place; if you don't have this utility, you can use sed 's/'$'\r''$//' file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file); to remove all \r instances (even if not followed by \n), use tr -d '\r' < file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file).
In addition to #DavidW.'s great debugging technique, you can add the following to visually inspect all arguments passed to your script:
i=0; for a; do echo "\$$((i+=1))=[$a]"; done
(The purpose of enclosing the value in [...] (for example), is to see the exact boundaries of the values.)
This will yield something like:
$1=[-g]
$2=[input.txt]
...
Note, though, that nothing at all is printed if no arguments were passed.
Try to print FILE1 to see if it has the value you want, if it is not the problem, here is a simple script (site below):
#!/bin/bash
file="${#:$OPTIND:1}"
if [ -f "$file" ]
then
echo "$file found."
else
echo "$file not found."
fi
http://www.cyberciti.biz/faq/unix-linux-test-existence-of-file-in-bash/
Instead of plucking an item out of "$#" in a tricky way, why don't you shift off the args you've processed with getopts:
while getopts ...
done
shift $(( OPTIND - 1 ))
FILE1=$1
The bash builtin type works well to figure out what a given command will do and how it is defined, but you cannot directly extract the file path in case the command ultimately resolves to a file. For example, to only perform a $PATH lookup, you can use which:
$ ls true
ls: cannot access true: No such file or directory
$ ls `which true`
/bin/true
Assume I had an alias:
alias notfalse=true
Then I couldn't just ask which, but I could ask type:
$ type notfalse
notfalse is aliased to `true'
But what I want is to have it resolve the alias by looking into $PATH (no, the various flags to type do not seem to work).
$ ls `somebuiltin notfalse`
/bin/true
Ignore that there is a true builtin (which I'm shadowing with the alias), this is just an example.
type -P is close to what you want. From the man page:
The -P flag forces a PATH search for each NAME, even if it is an alias,
builtin, or function, and returns the name of the disk file that would
be executed.
For example, I have ls aliased to ls -G, and type -P ls returns /bin/ls.
However, this fails in the case of built-ins that shadow executables (like the built-in true shadows /bin/true). I'm not sure there is a way around that. It also fails for choroba's example of csort.
You may parse the output of alias notfalse and then use type -P (as already suggested by chepner).
getaliaswithpath() {
alias_str="$(alias "$1")"
if [[ -n "$alias_str" ]]; then
alias_str="${alias_str#*\'}"
alias_str="${alias_str%%\'*}"
type -P "$alias_str"
else
type -P "$1"
fi
}
alias notfalse=true
getaliaswithpath notfalse
Improved version of getaliaswithpath() (2013-09-13):
getaliaswithpath() {
declare IFS alias_str exitflag substr
exitflag=0
alias_str="$(builtin alias "$1")"
if [[ -n "$alias_str" ]]; then
alias_str="${alias_str#*\'}"
alias_str="${alias_str%%\'*}"
# stop at first successful PATH lookup (before first | symbol)
IFS=" "
for substr in $alias_str; do
[[ $exitflag -eq 1 ]] && return 1
[[ "${substr:0:1}" == '-' ]] && continue # skip cmd line options
[[ "${substr//=/}" != "$substr" ]] && continue # skip substr containing = symbol
[[ "${substr}" == '|' ]] && { echo 'no cmd binary found' 1>&2; return 1; } # stop at first | symbol
[[ "${substr:0:1}" == '|' ]] && { echo 'no cmd binary found' 1>&2; return 1; } # stop if substr begins with |
if [[ "${substr: -1}" == '|' ]]; then # if substr ends with | ...
exitflag=1
substr="${substr%?}"
elif [[ "${substr//|/}" != "$substr" ]]; then # if substr contains | symbol ...
substr="${substr%%|*}" # ... extract first part up to first |
fi
#echo builtin type -P "$substr"
builtin type -P "$substr" && return 0
done
else
builtin type -P "$1"
fi
}
(
set -f # disable globbing
# test cases
alias usort='LC_ALL=C sort -u | cat -n'
alias usort='LC_ALL=C sort -u| cat -n'
alias usort='LC_ALL=C sort -u |cat -n'
alias usort='LC_ALL=C sort -u|cat -n'
alias usort='LC_ALL=C sort|cat -n'
getaliaswithpath usort
)
You can try the following:
ls `alias | /usr/bin/which --read-alias notfalse | tail -1`
By the way, I have my own aliased which which is this:
alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'