Listing defined functions in Bash - bash

I'm trying to write some code in bash which uses introspection to select the appropriate function to call.
Determining the candidates requires knowing which functions are defined. It's easy to list defined variables in bash using only parameter expansion:
$ prefix_foo="one"
$ prefix_bar="two"
$ echo "${!prefix_*}"
prefix_bar prefix_foo
However, doing this for functions appears to require filtering the output of set -- a much more haphazard approach.
Is there a Right Way?

How about compgen:
compgen -A function # compgen is a shell builtin

$ declare -F
declare -f ::
declare -f _get_longopts
declare -f _longopts_func
declare -f _onexit
...
So, Jed Daniel's alias,
declare -F | cut -d" " -f3
cuts on a space and echos the 3rd field:
$ declare -F | cut -d" " -f3
::
_get_longopts
_longopts_func
_onexit

I have an entry in my .bashrc that says:
alias list='declare -F |cut -d" " -f3'
Which allows me to type list and get a list of functions. When I added it, I probably understood what was happening, but I can't remember to save my life at the moment.
Good luck,
--jed

zsh only (not what was asked for, but all the more generic questions have been closed as a duplicate of this):
typeset -f +
From man zshbuiltins:
-f The names refer to functions rather than parameters.
+ If `+' appears by itself in a separate word as the last
option, then the names of all parameters (functions with -f)
are printed, but the values (function bodies) are not.
Example:
martin#martin ~ % cat test.zsh
#!/bin/zsh
foobar()
{
echo foobar
}
barfoo()
{
echo barfoo
}
typeset -f +
Output:
martin#martin ~ % ./test.zsh
barfoo
foobar

Use the declare builtin to list currently defined functions:
declare -F

This has no issues with IFS nor globbing:
readarray -t funcs < <(declare -F)
printf '%s\n' "${funcs[#]##* }"
Of course, that needs bash 4.0.
For bash since 2.04 use (a little trickier but equivalent):
IFS=$'\n' read -d '' -a funcs < <(declare -F)
If you need that the exit code of this option is zero, use this:
IFS=$'\n' read -d '' -a funcs < <( declare -F && printf '\0' )
It will exit unsuccesful (not 0) if either declare or read fail. (Thanks to #CharlesDuffy)

One (ugly) approach is to grep through the output of set:
set \
| egrep '^[^[:space:]]+ [(][)][[:space:]]*$' \
| sed -r -e 's/ [(][)][[:space:]]*$//'
Better approaches would be welcome.

Pure Bash:
saveIFS="$IFS"
IFS=$'\n'
funcs=($(declare -F)) # create an array
IFS="$saveIFS"
funcs=(${funcs[#]##* }) # keep only what's after the last space
Then, run at the Bash prompt as an example displaying bash-completion functions:
$ for i in ${funcs[#]}; do echo "$i"; done
__ack_filedir
__gvfs_multiple_uris
_a2dismod
. . .
$ echo ${funcs[42]}
_command

This collects a list of function names matching any of a list of patterns:
functions=$(for c in $patterns; do compgen -A function | grep "^$c\$")
The grep limits the output to only exact matches for the patterns.
Check out the bash command type as a better alternative to the following. Thanks to Charles Duffy for the clue.
The following uses that to answer the title question for humans rather than shell scripts: it adds a list of function names matching the given patterns, to the regular which list of shell scripts, to answer, "What code runs when I type a command?"
which() {
for c in "$#"; do
compgen -A function |grep "^$c\$" | while read line; do
echo "shell function $line" 1>&2
done
/usr/bin/which "$c"
done
}
So,
(xkcd)Sandy$ which deactivate
shell function deactivate
(xkcd)Sandy$ which ls
/bin/ls
(xkcd)Sandy$ which .\*run_hook
shell function virtualenvwrapper_run_hook
This is arguably a violation of the Unix "do one thing" philosophy, but I've more than once been desperate because which wasn't finding a command that some package was supposed to contain, me forgetting about shell functions, so I've put this in my .profile.

#!/bin/bash
# list-defined-functions.sh
# Lists functions defined in this script.
#
# Using `compgen -A function`,
# We can save the list of functions defined before running out script,
# the compare that to a new list at the end,
# resulting in the list of newly added functions.
#
# Usage:
# bash list-defined-functions.sh # Run in new shell with no predefined functions
# list-defined-functions.sh # Run in current shell with plenty of predefined functions
#
# Example predefined function
foo() { echo 'y'; }
# Retain original function list
# If this script is run a second time, keep the list from last time
[[ $original_function_list ]] || original_function_list=$(compgen -A function)
# Create some new functions...
myfunc() { echo "myfunc is the best func"; }
function another_func() { echo "another_func is better"; }
function superfunction { echo "hey another way to define functions"; }
# ...
# function goo() { echo ok; }
[[ $new_function_list ]] || new_function_list=$(comm -13 \
<(echo $original_function_list) \
<(compgen -A function))
echo "Original functions were:"
echo "$original_function_list"
echo
echo "New Functions defined in this script:"
echo "$new_function_list"

Related

Running du against the home directory of a user whose name came from a variable [duplicate]

I have a variable in my bash script whose value is something like this:
~/a/b/c
Note that it is unexpanded tilde. When I do ls -lt on this variable (call it $VAR), I get no such directory. I want to let bash interpret/expand this variable without executing it. In other words, I want bash to run eval but not run the evaluated command. Is this possible in bash?
How did I manage to pass this into my script without expansion? I passed the argument in surrounding it with double quotes.
Try this command to see what I mean:
ls -lt "~"
This is exactly the situation I am in. I want the tilde to be expanded. In other words, what should I replace magic with to make these two commands identical:
ls -lt ~/abc/def/ghi
and
ls -lt $(magic "~/abc/def/ghi")
Note that ~/abc/def/ghi may or may not exist.
If the variable var is input by the user, eval should not be used to expand the tilde using
eval var=$var # Do not use this!
The reason is: the user could by accident (or by purpose) type for example var="$(rm -rf $HOME/)" with possible disastrous consequences.
A better (and safer) way is to use Bash parameter expansion:
var="${var/#\~/$HOME}"
Due to the nature of StackOverflow, I can't just make this answer unaccepted, but in the intervening 5 years since I posted this there have been far better answers than my admittedly rudimentary and pretty bad answer (I was young, don't kill me).
The other solutions in this thread are safer and better solutions. Preferably, I'd go with either of these two:
Charle's Duffy's solution
Håkon Hægland's solution
Original answer for historic purposes (but please don't use this)
If I'm not mistaken, "~" will not be expanded by a bash script in that manner because it is treated as a literal string "~". You can force expansion via eval like this.
#!/bin/bash
homedir=~
eval homedir=$homedir
echo $homedir # prints home path
Alternatively, just use ${HOME} if you want the user's home directory.
Plagarizing myself from a prior answer, to do this robustly without the security risks associated with eval:
expandPath() {
local path
local -a pathElements resultPathElements
IFS=':' read -r -a pathElements <<<"$1"
: "${pathElements[#]}"
for path in "${pathElements[#]}"; do
: "$path"
case $path in
"~+"/*)
path=$PWD/${path#"~+/"}
;;
"~-"/*)
path=$OLDPWD/${path#"~-/"}
;;
"~"/*)
path=$HOME/${path#"~/"}
;;
"~"*)
username=${path%%/*}
username=${username#"~"}
IFS=: read -r _ _ _ _ _ homedir _ < <(getent passwd "$username")
if [[ $path = */* ]]; then
path=${homedir}/${path#*/}
else
path=$homedir
fi
;;
esac
resultPathElements+=( "$path" )
done
local result
printf -v result '%s:' "${resultPathElements[#]}"
printf '%s\n' "${result%:}"
}
...used as...
path=$(expandPath '~/hello')
Alternately, a simpler approach that uses eval carefully:
expandPath() {
case $1 in
~[+-]*)
local content content_q
printf -v content_q '%q' "${1:2}"
eval "content=${1:0:2}${content_q}"
printf '%s\n' "$content"
;;
~*)
local content content_q
printf -v content_q '%q' "${1:1}"
eval "content=~${content_q}"
printf '%s\n' "$content"
;;
*)
printf '%s\n' "$1"
;;
esac
}
How about this:
path=`realpath "$1"`
Or:
path=`readlink -f "$1"`
A safe way to use eval is "$(printf "~/%q" "$dangerous_path")". Note that is bash specific.
#!/bin/bash
relativepath=a/b/c
eval homedir="$(printf "~/%q" "$relativepath")"
echo $homedir # prints home path
See this question for details
Also, note that under zsh this would be as as simple as echo ${~dangerous_path}
Here is a ridiculous solution:
$ echo "echo $var" | bash
An explanation of what this command does:
create a new instance of bash, by... calling bash;
take the string "echo $var" and substitute $var with the value of the variable (thus after the substitution the string will contain the tilde);
take the string produced by step 2 and send it to the instance of bash created in step one, which we do here by calling echo and piping its output with the | character.
Basically the current bash instance we're running takes our place as the user of another bash instance and types in the command "echo ~..." for us.
Expanding (no pun intended) on birryree's and halloleo's answers: The general approach is to use eval, but it comes with some important caveats, namely spaces and output redirection (>) in the variable. The following seems to work for me:
mypath="$1"
if [ -e "`eval echo ${mypath//>}`" ]; then
echo "FOUND $mypath"
else
echo "$mypath NOT FOUND"
fi
Try it with each of the following arguments:
'~'
'~/existing_file'
'~/existing file with spaces'
'~/nonexistant_file'
'~/nonexistant file with spaces'
'~/string containing > redirection'
'~/string containing > redirection > again and >> again'
Explanation
The ${mypath//>} strips out > characters which could clobber a file during the eval.
The eval echo ... is what does the actual tilde expansion
The double-quotes around the -e argument are for support of filenames with spaces.
Perhaps there's a more elegant solution, but this is what I was able to come up with.
why not delve straight into getting the user's home directory with getent?
$ getent passwd mike | cut -d: -f6
/users/mike
I believe this is what you're looking for
magic() { # returns unexpanded tilde express on invalid user
local _safe_path; printf -v _safe_path "%q" "$1"
eval "ln -sf ${_safe_path#\\} /tmp/realpath.$$"
readlink /tmp/realpath.$$
rm -f /tmp/realpath.$$
}
Example usage:
$ magic ~nobody/would/look/here
/var/empty/would/look/here
$ magic ~invalid/this/will/not/expand
~invalid/this/will/not/expand
Here is the POSIX function equivalent of Håkon Hægland's Bash answer
expand_tilde() {
tilde_less="${1#\~/}"
[ "$1" != "$tilde_less" ] && tilde_less="$HOME/$tilde_less"
printf '%s' "$tilde_less"
}
2017-12-10 edit: add '%s' per #CharlesDuffy in the comments.
Here's my solution:
#!/bin/bash
expandTilde()
{
local tilde_re='^(~[A-Za-z0-9_.-]*)(.*)'
local path="$*"
local pathSuffix=
if [[ $path =~ $tilde_re ]]
then
# only use eval on the ~username portion !
path=$(eval echo ${BASH_REMATCH[1]})
pathSuffix=${BASH_REMATCH[2]}
fi
echo "${path}${pathSuffix}"
}
result=$(expandTilde "$1")
echo "Result = $result"
Simplest: replace 'magic' with 'eval echo'.
$ eval echo "~"
/whatever/the/f/the/home/directory/is
Problem: You're going to run into issues with other variables because eval is evil. For instance:
$ # home is /Users/Hacker$(s)
$ s="echo SCARY COMMAND"
$ eval echo $(eval echo "~")
/Users/HackerSCARY COMMAND
Note that the issue of the injection doesn't happen on the first expansion. So if you were to simply replace magic with eval echo, you should be okay. But if you do echo $(eval echo ~), that would be susceptible to injection.
Similarly, if you do eval echo ~ instead of eval echo "~", that would count as twice expanded and therefore injection would be possible right away.
For anyone's reference, a function to mimic python's os.path.expanduser() behavior (no eval usage):
# _expand_homedir_tilde ~/.vim
/root/.vim
# _expand_homedir_tilde ~myuser/.vim
/home/myuser/.vim
# _expand_homedir_tilde ~nonexistent/.vim
~nonexistent/.vim
# _expand_homedir_tilde /full/path
/full/path
And the function:
function _expand_homedir_tilde {
(
set -e
set -u
p="$1"
if [[ "$p" =~ ^~ ]]; then
u=`echo "$p" | sed 's|^~\([a-z0-9_-]*\)/.*|\1|'`
if [ -z "$u" ]; then
u=`whoami`
fi
h=$(set -o pipefail; getent passwd "$u" | cut -d: -f6) || exit 1
p=`echo "$p" | sed "s|^~[a-z0-9_-]*/|${h}/|"`
fi
echo $p
) || echo $1
}
Just to extend birryree's answer for paths with spaces: You cannot use the eval command as is because it seperates evaluation by spaces. One solution is to replace spaces temporarily for the eval command:
mypath="~/a/b/c/Something With Spaces"
expandedpath=${mypath// /_spc_} # replace spaces
eval expandedpath=${expandedpath} # put spaces back
expandedpath=${expandedpath//_spc_/ }
echo "$expandedpath" # prints e.g. /Users/fred/a/b/c/Something With Spaces"
ls -lt "$expandedpath" # outputs dir content
This example relies of course on the assumption that mypath never contains the char sequence "_spc_".
You might find this easier to do in python.
(1) From the unix command line:
python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' ~/fred
Results in:
/Users/someone/fred
(2) Within a bash script as a one-off - save this as test.sh:
#!/usr/bin/env bash
thepath=$(python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' $1)
echo $thepath
Running bash ./test.sh results in:
/Users/someone/fred
(3) As a utility - save this as expanduser somewhere on your path, with execute permissions:
#!/usr/bin/env python
import sys
import os
print os.path.expanduser(sys.argv[1])
This could then be used on the command line:
expanduser ~/fred
Or in a script:
#!/usr/bin/env bash
thepath=$(expanduser $1)
echo $thepath
Just use eval correctly: with validation.
case $1${1%%/*} in
([!~]*|"$1"?*[!-+_.[:alnum:]]*|"") ! :;;
(*/*) set "${1%%/*}" "${1#*/}" ;;
(*) set "$1"
esac&& eval "printf '%s\n' $1${2+/\"\$2\"}"
I have done this with variable parameter substitution after reading in the path using read -e (among others). So the user can tab-complete the path, and if the user enters a ~ path it gets sorted.
read -rep "Enter a path: " -i "${testpath}" testpath
testpath="${testpath/#~/${HOME}}"
ls -al "${testpath}"
The added benefit is that if there is no tilde nothing happens to the variable, and if there is a tilde but not in the first position it is also ignored.
(I include the -i for read since I use this in a loop so the user can fix the path if there is a problem.)
for some reason when the string is already quoted only perl saves the day
#val="${val/#\~/$HOME}" # for some reason does not work !!
val=$(echo $val|perl -ne 's|~|'$HOME'|g;print')
I think that
thepath=( ~/abc/def/ghi )
is easier than all the other solutions... or I am missing something? It works even if the path does not really exists.

Giving relative address as an input to read in bash scripts [duplicate]

I have a variable in my bash script whose value is something like this:
~/a/b/c
Note that it is unexpanded tilde. When I do ls -lt on this variable (call it $VAR), I get no such directory. I want to let bash interpret/expand this variable without executing it. In other words, I want bash to run eval but not run the evaluated command. Is this possible in bash?
How did I manage to pass this into my script without expansion? I passed the argument in surrounding it with double quotes.
Try this command to see what I mean:
ls -lt "~"
This is exactly the situation I am in. I want the tilde to be expanded. In other words, what should I replace magic with to make these two commands identical:
ls -lt ~/abc/def/ghi
and
ls -lt $(magic "~/abc/def/ghi")
Note that ~/abc/def/ghi may or may not exist.
If the variable var is input by the user, eval should not be used to expand the tilde using
eval var=$var # Do not use this!
The reason is: the user could by accident (or by purpose) type for example var="$(rm -rf $HOME/)" with possible disastrous consequences.
A better (and safer) way is to use Bash parameter expansion:
var="${var/#\~/$HOME}"
Due to the nature of StackOverflow, I can't just make this answer unaccepted, but in the intervening 5 years since I posted this there have been far better answers than my admittedly rudimentary and pretty bad answer (I was young, don't kill me).
The other solutions in this thread are safer and better solutions. Preferably, I'd go with either of these two:
Charle's Duffy's solution
Håkon Hægland's solution
Original answer for historic purposes (but please don't use this)
If I'm not mistaken, "~" will not be expanded by a bash script in that manner because it is treated as a literal string "~". You can force expansion via eval like this.
#!/bin/bash
homedir=~
eval homedir=$homedir
echo $homedir # prints home path
Alternatively, just use ${HOME} if you want the user's home directory.
Plagarizing myself from a prior answer, to do this robustly without the security risks associated with eval:
expandPath() {
local path
local -a pathElements resultPathElements
IFS=':' read -r -a pathElements <<<"$1"
: "${pathElements[#]}"
for path in "${pathElements[#]}"; do
: "$path"
case $path in
"~+"/*)
path=$PWD/${path#"~+/"}
;;
"~-"/*)
path=$OLDPWD/${path#"~-/"}
;;
"~"/*)
path=$HOME/${path#"~/"}
;;
"~"*)
username=${path%%/*}
username=${username#"~"}
IFS=: read -r _ _ _ _ _ homedir _ < <(getent passwd "$username")
if [[ $path = */* ]]; then
path=${homedir}/${path#*/}
else
path=$homedir
fi
;;
esac
resultPathElements+=( "$path" )
done
local result
printf -v result '%s:' "${resultPathElements[#]}"
printf '%s\n' "${result%:}"
}
...used as...
path=$(expandPath '~/hello')
Alternately, a simpler approach that uses eval carefully:
expandPath() {
case $1 in
~[+-]*)
local content content_q
printf -v content_q '%q' "${1:2}"
eval "content=${1:0:2}${content_q}"
printf '%s\n' "$content"
;;
~*)
local content content_q
printf -v content_q '%q' "${1:1}"
eval "content=~${content_q}"
printf '%s\n' "$content"
;;
*)
printf '%s\n' "$1"
;;
esac
}
How about this:
path=`realpath "$1"`
Or:
path=`readlink -f "$1"`
A safe way to use eval is "$(printf "~/%q" "$dangerous_path")". Note that is bash specific.
#!/bin/bash
relativepath=a/b/c
eval homedir="$(printf "~/%q" "$relativepath")"
echo $homedir # prints home path
See this question for details
Also, note that under zsh this would be as as simple as echo ${~dangerous_path}
Here is a ridiculous solution:
$ echo "echo $var" | bash
An explanation of what this command does:
create a new instance of bash, by... calling bash;
take the string "echo $var" and substitute $var with the value of the variable (thus after the substitution the string will contain the tilde);
take the string produced by step 2 and send it to the instance of bash created in step one, which we do here by calling echo and piping its output with the | character.
Basically the current bash instance we're running takes our place as the user of another bash instance and types in the command "echo ~..." for us.
Expanding (no pun intended) on birryree's and halloleo's answers: The general approach is to use eval, but it comes with some important caveats, namely spaces and output redirection (>) in the variable. The following seems to work for me:
mypath="$1"
if [ -e "`eval echo ${mypath//>}`" ]; then
echo "FOUND $mypath"
else
echo "$mypath NOT FOUND"
fi
Try it with each of the following arguments:
'~'
'~/existing_file'
'~/existing file with spaces'
'~/nonexistant_file'
'~/nonexistant file with spaces'
'~/string containing > redirection'
'~/string containing > redirection > again and >> again'
Explanation
The ${mypath//>} strips out > characters which could clobber a file during the eval.
The eval echo ... is what does the actual tilde expansion
The double-quotes around the -e argument are for support of filenames with spaces.
Perhaps there's a more elegant solution, but this is what I was able to come up with.
why not delve straight into getting the user's home directory with getent?
$ getent passwd mike | cut -d: -f6
/users/mike
I believe this is what you're looking for
magic() { # returns unexpanded tilde express on invalid user
local _safe_path; printf -v _safe_path "%q" "$1"
eval "ln -sf ${_safe_path#\\} /tmp/realpath.$$"
readlink /tmp/realpath.$$
rm -f /tmp/realpath.$$
}
Example usage:
$ magic ~nobody/would/look/here
/var/empty/would/look/here
$ magic ~invalid/this/will/not/expand
~invalid/this/will/not/expand
Here is the POSIX function equivalent of Håkon Hægland's Bash answer
expand_tilde() {
tilde_less="${1#\~/}"
[ "$1" != "$tilde_less" ] && tilde_less="$HOME/$tilde_less"
printf '%s' "$tilde_less"
}
2017-12-10 edit: add '%s' per #CharlesDuffy in the comments.
Here's my solution:
#!/bin/bash
expandTilde()
{
local tilde_re='^(~[A-Za-z0-9_.-]*)(.*)'
local path="$*"
local pathSuffix=
if [[ $path =~ $tilde_re ]]
then
# only use eval on the ~username portion !
path=$(eval echo ${BASH_REMATCH[1]})
pathSuffix=${BASH_REMATCH[2]}
fi
echo "${path}${pathSuffix}"
}
result=$(expandTilde "$1")
echo "Result = $result"
Simplest: replace 'magic' with 'eval echo'.
$ eval echo "~"
/whatever/the/f/the/home/directory/is
Problem: You're going to run into issues with other variables because eval is evil. For instance:
$ # home is /Users/Hacker$(s)
$ s="echo SCARY COMMAND"
$ eval echo $(eval echo "~")
/Users/HackerSCARY COMMAND
Note that the issue of the injection doesn't happen on the first expansion. So if you were to simply replace magic with eval echo, you should be okay. But if you do echo $(eval echo ~), that would be susceptible to injection.
Similarly, if you do eval echo ~ instead of eval echo "~", that would count as twice expanded and therefore injection would be possible right away.
For anyone's reference, a function to mimic python's os.path.expanduser() behavior (no eval usage):
# _expand_homedir_tilde ~/.vim
/root/.vim
# _expand_homedir_tilde ~myuser/.vim
/home/myuser/.vim
# _expand_homedir_tilde ~nonexistent/.vim
~nonexistent/.vim
# _expand_homedir_tilde /full/path
/full/path
And the function:
function _expand_homedir_tilde {
(
set -e
set -u
p="$1"
if [[ "$p" =~ ^~ ]]; then
u=`echo "$p" | sed 's|^~\([a-z0-9_-]*\)/.*|\1|'`
if [ -z "$u" ]; then
u=`whoami`
fi
h=$(set -o pipefail; getent passwd "$u" | cut -d: -f6) || exit 1
p=`echo "$p" | sed "s|^~[a-z0-9_-]*/|${h}/|"`
fi
echo $p
) || echo $1
}
Just to extend birryree's answer for paths with spaces: You cannot use the eval command as is because it seperates evaluation by spaces. One solution is to replace spaces temporarily for the eval command:
mypath="~/a/b/c/Something With Spaces"
expandedpath=${mypath// /_spc_} # replace spaces
eval expandedpath=${expandedpath} # put spaces back
expandedpath=${expandedpath//_spc_/ }
echo "$expandedpath" # prints e.g. /Users/fred/a/b/c/Something With Spaces"
ls -lt "$expandedpath" # outputs dir content
This example relies of course on the assumption that mypath never contains the char sequence "_spc_".
You might find this easier to do in python.
(1) From the unix command line:
python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' ~/fred
Results in:
/Users/someone/fred
(2) Within a bash script as a one-off - save this as test.sh:
#!/usr/bin/env bash
thepath=$(python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' $1)
echo $thepath
Running bash ./test.sh results in:
/Users/someone/fred
(3) As a utility - save this as expanduser somewhere on your path, with execute permissions:
#!/usr/bin/env python
import sys
import os
print os.path.expanduser(sys.argv[1])
This could then be used on the command line:
expanduser ~/fred
Or in a script:
#!/usr/bin/env bash
thepath=$(expanduser $1)
echo $thepath
Just use eval correctly: with validation.
case $1${1%%/*} in
([!~]*|"$1"?*[!-+_.[:alnum:]]*|"") ! :;;
(*/*) set "${1%%/*}" "${1#*/}" ;;
(*) set "$1"
esac&& eval "printf '%s\n' $1${2+/\"\$2\"}"
I have done this with variable parameter substitution after reading in the path using read -e (among others). So the user can tab-complete the path, and if the user enters a ~ path it gets sorted.
read -rep "Enter a path: " -i "${testpath}" testpath
testpath="${testpath/#~/${HOME}}"
ls -al "${testpath}"
The added benefit is that if there is no tilde nothing happens to the variable, and if there is a tilde but not in the first position it is also ignored.
(I include the -i for read since I use this in a loop so the user can fix the path if there is a problem.)
for some reason when the string is already quoted only perl saves the day
#val="${val/#\~/$HOME}" # for some reason does not work !!
val=$(echo $val|perl -ne 's|~|'$HOME'|g;print')
I think that
thepath=( ~/abc/def/ghi )
is easier than all the other solutions... or I am missing something? It works even if the path does not really exists.

Indirect parameter substitution in shell script

I'm having a problem with a shell script (POSIX shell under HP-UX, FWIW). I have a function called print_arg into which I'm passing the name of a parameter as $1. Given the name of the parameter, I then want to print the name and the value of that parameter. However, I keep getting an error. Here's an example of what I'm trying to do:
#!/usr/bin/sh
function print_arg
{
# $1 holds the name of the argument to be shown
arg=$1
# The following line errors off with
# ./test_print.sh[9]: argval=${"$arg"}: The specified substitution is not valid for this command.
argval=${"$arg"}
if [[ $argval != '' ]] ; then
printf "ftp_func: $arg='$argval'\n"
fi
}
COMMAND="XYZ"
print_arg "COMMAND"
I've tried re-writing the offending line every way I can think of. I've consulted the local oracles. I've checked the online "BASH Scripting Guide". And I sharpened up the ol' wavy-bladed knife and scrubbed the altar until it gleamed, but then I discovered that our local supply of virgins has been cut down to, like, nothin'. Drat!
Any advice regarding how to get the value of a parameter whose name is passed into a function as a parameter will be received appreciatively.
You could use eval, though using direct indirection as suggested by SiegeX is probably nicer if you can use bash.
#!/bin/sh
foo=bar
print_arg () {
arg=$1
eval argval=\"\$$arg\"
echo "$argval"
}
print_arg foo
In bash (but not in other sh implementations), indirection is done by: ${!arg}
Input
foo=bar
bar=baz
echo $foo
echo ${!foo}
Output
bar
baz
This worked surprisingly well:
#!/bin/sh
foo=bar
print_arg () {
local line name value
set | \
while read line; do
name=${line%=*} value=${line#*=\'}
if [ "$name" = "$1" ]; then
echo ${value%\'}
fi
done
}
print_arg foo
It has all the POSIX clunkiness, in Bash would be much sorter, but then again, you won't need it because you have ${!}. This -in case it proves solid- would have the advantage of using only builtins and no eval. If I were to construct this function using an external command, it would have to be sed. Would obviate the need for the read loop and the substitutions. Mind that asking for indirections in POSIX without eval, has to be paid with clunkiness! So don't beat me!
Even though the answer's already accepted, here's another method for those who need to preserve newlines and special characters like Escape ( \033 ): Storing the variable in base64.
You need: bc, wc, echo, tail, tr, uuencode, uudecode
Example
#!/bin/sh
#====== Definition =======#
varA="a
b
c"
# uuencode the variable
varB="`echo "$varA" | uuencode -m -`"
# Skip the first line of the uuencode output.
varB="`NUM=\`(echo "$varB"|wc -l|tr -d "\n"; echo -1)|bc \`; echo "$varB" | tail -n $NUM)`"
#====== Access =======#
namevar1=varB
namevar2=varA
echo simple eval:
eval "echo \$$namevar2"
echo simple echo:
echo $varB
echo precise echo:
echo "$varB"
echo echo of base64
eval "echo \$$namevar1"
echo echo of base64 - with updated newlines
eval "echo \$$namevar1 | tr ' ' '\n'"
echo echo of un-based, using sh instead of eval (but could be made with eval, too)
export $namevar1
sh -c "(echo 'begin-base64 644 -'; echo \$$namevar1 | tr ' ' '\n' )|uudecode"
Result
simple eval:
a b c
simple echo:
YQpiCmMK ====
precise echo:
YQpiCmMK
====
echo of base64
YQpiCmMK ====
echo of base64 - with updated newlines
YQpiCmMK
====
echo of un-based, using sh instead of eval (but could be made with eval, too)
a
b
c
Alternative
You also could use the set command and parse it's output; with that, you don't need to treat the variable in a special way before it's accessed.
A safer solution with eval:
v=1
valid_var_name='[[:alpha:]_][[:alnum:]_]*$'
print_arg() {
local arg=$1
if ! expr "$arg" : "$valid_var_name" >/dev/null; then
echo "$0: invalid variable name ($arg)" >&2
exit 1
fi
local argval
eval argval=\$$arg
echo "$argval"
}
print_arg v
print_arg 'v; echo test'
Inspired by the following answer.

How to manually expand a special variable (ex: ~ tilde) in bash

I have a variable in my bash script whose value is something like this:
~/a/b/c
Note that it is unexpanded tilde. When I do ls -lt on this variable (call it $VAR), I get no such directory. I want to let bash interpret/expand this variable without executing it. In other words, I want bash to run eval but not run the evaluated command. Is this possible in bash?
How did I manage to pass this into my script without expansion? I passed the argument in surrounding it with double quotes.
Try this command to see what I mean:
ls -lt "~"
This is exactly the situation I am in. I want the tilde to be expanded. In other words, what should I replace magic with to make these two commands identical:
ls -lt ~/abc/def/ghi
and
ls -lt $(magic "~/abc/def/ghi")
Note that ~/abc/def/ghi may or may not exist.
If the variable var is input by the user, eval should not be used to expand the tilde using
eval var=$var # Do not use this!
The reason is: the user could by accident (or by purpose) type for example var="$(rm -rf $HOME/)" with possible disastrous consequences.
A better (and safer) way is to use Bash parameter expansion:
var="${var/#\~/$HOME}"
Due to the nature of StackOverflow, I can't just make this answer unaccepted, but in the intervening 5 years since I posted this there have been far better answers than my admittedly rudimentary and pretty bad answer (I was young, don't kill me).
The other solutions in this thread are safer and better solutions. Preferably, I'd go with either of these two:
Charle's Duffy's solution
Håkon Hægland's solution
Original answer for historic purposes (but please don't use this)
If I'm not mistaken, "~" will not be expanded by a bash script in that manner because it is treated as a literal string "~". You can force expansion via eval like this.
#!/bin/bash
homedir=~
eval homedir=$homedir
echo $homedir # prints home path
Alternatively, just use ${HOME} if you want the user's home directory.
Plagarizing myself from a prior answer, to do this robustly without the security risks associated with eval:
expandPath() {
local path
local -a pathElements resultPathElements
IFS=':' read -r -a pathElements <<<"$1"
: "${pathElements[#]}"
for path in "${pathElements[#]}"; do
: "$path"
case $path in
"~+"/*)
path=$PWD/${path#"~+/"}
;;
"~-"/*)
path=$OLDPWD/${path#"~-/"}
;;
"~"/*)
path=$HOME/${path#"~/"}
;;
"~"*)
username=${path%%/*}
username=${username#"~"}
IFS=: read -r _ _ _ _ _ homedir _ < <(getent passwd "$username")
if [[ $path = */* ]]; then
path=${homedir}/${path#*/}
else
path=$homedir
fi
;;
esac
resultPathElements+=( "$path" )
done
local result
printf -v result '%s:' "${resultPathElements[#]}"
printf '%s\n' "${result%:}"
}
...used as...
path=$(expandPath '~/hello')
Alternately, a simpler approach that uses eval carefully:
expandPath() {
case $1 in
~[+-]*)
local content content_q
printf -v content_q '%q' "${1:2}"
eval "content=${1:0:2}${content_q}"
printf '%s\n' "$content"
;;
~*)
local content content_q
printf -v content_q '%q' "${1:1}"
eval "content=~${content_q}"
printf '%s\n' "$content"
;;
*)
printf '%s\n' "$1"
;;
esac
}
How about this:
path=`realpath "$1"`
Or:
path=`readlink -f "$1"`
A safe way to use eval is "$(printf "~/%q" "$dangerous_path")". Note that is bash specific.
#!/bin/bash
relativepath=a/b/c
eval homedir="$(printf "~/%q" "$relativepath")"
echo $homedir # prints home path
See this question for details
Also, note that under zsh this would be as as simple as echo ${~dangerous_path}
Here is a ridiculous solution:
$ echo "echo $var" | bash
An explanation of what this command does:
create a new instance of bash, by... calling bash;
take the string "echo $var" and substitute $var with the value of the variable (thus after the substitution the string will contain the tilde);
take the string produced by step 2 and send it to the instance of bash created in step one, which we do here by calling echo and piping its output with the | character.
Basically the current bash instance we're running takes our place as the user of another bash instance and types in the command "echo ~..." for us.
Expanding (no pun intended) on birryree's and halloleo's answers: The general approach is to use eval, but it comes with some important caveats, namely spaces and output redirection (>) in the variable. The following seems to work for me:
mypath="$1"
if [ -e "`eval echo ${mypath//>}`" ]; then
echo "FOUND $mypath"
else
echo "$mypath NOT FOUND"
fi
Try it with each of the following arguments:
'~'
'~/existing_file'
'~/existing file with spaces'
'~/nonexistant_file'
'~/nonexistant file with spaces'
'~/string containing > redirection'
'~/string containing > redirection > again and >> again'
Explanation
The ${mypath//>} strips out > characters which could clobber a file during the eval.
The eval echo ... is what does the actual tilde expansion
The double-quotes around the -e argument are for support of filenames with spaces.
Perhaps there's a more elegant solution, but this is what I was able to come up with.
why not delve straight into getting the user's home directory with getent?
$ getent passwd mike | cut -d: -f6
/users/mike
I believe this is what you're looking for
magic() { # returns unexpanded tilde express on invalid user
local _safe_path; printf -v _safe_path "%q" "$1"
eval "ln -sf ${_safe_path#\\} /tmp/realpath.$$"
readlink /tmp/realpath.$$
rm -f /tmp/realpath.$$
}
Example usage:
$ magic ~nobody/would/look/here
/var/empty/would/look/here
$ magic ~invalid/this/will/not/expand
~invalid/this/will/not/expand
Here is the POSIX function equivalent of Håkon Hægland's Bash answer
expand_tilde() {
tilde_less="${1#\~/}"
[ "$1" != "$tilde_less" ] && tilde_less="$HOME/$tilde_less"
printf '%s' "$tilde_less"
}
2017-12-10 edit: add '%s' per #CharlesDuffy in the comments.
Here's my solution:
#!/bin/bash
expandTilde()
{
local tilde_re='^(~[A-Za-z0-9_.-]*)(.*)'
local path="$*"
local pathSuffix=
if [[ $path =~ $tilde_re ]]
then
# only use eval on the ~username portion !
path=$(eval echo ${BASH_REMATCH[1]})
pathSuffix=${BASH_REMATCH[2]}
fi
echo "${path}${pathSuffix}"
}
result=$(expandTilde "$1")
echo "Result = $result"
Simplest: replace 'magic' with 'eval echo'.
$ eval echo "~"
/whatever/the/f/the/home/directory/is
Problem: You're going to run into issues with other variables because eval is evil. For instance:
$ # home is /Users/Hacker$(s)
$ s="echo SCARY COMMAND"
$ eval echo $(eval echo "~")
/Users/HackerSCARY COMMAND
Note that the issue of the injection doesn't happen on the first expansion. So if you were to simply replace magic with eval echo, you should be okay. But if you do echo $(eval echo ~), that would be susceptible to injection.
Similarly, if you do eval echo ~ instead of eval echo "~", that would count as twice expanded and therefore injection would be possible right away.
For anyone's reference, a function to mimic python's os.path.expanduser() behavior (no eval usage):
# _expand_homedir_tilde ~/.vim
/root/.vim
# _expand_homedir_tilde ~myuser/.vim
/home/myuser/.vim
# _expand_homedir_tilde ~nonexistent/.vim
~nonexistent/.vim
# _expand_homedir_tilde /full/path
/full/path
And the function:
function _expand_homedir_tilde {
(
set -e
set -u
p="$1"
if [[ "$p" =~ ^~ ]]; then
u=`echo "$p" | sed 's|^~\([a-z0-9_-]*\)/.*|\1|'`
if [ -z "$u" ]; then
u=`whoami`
fi
h=$(set -o pipefail; getent passwd "$u" | cut -d: -f6) || exit 1
p=`echo "$p" | sed "s|^~[a-z0-9_-]*/|${h}/|"`
fi
echo $p
) || echo $1
}
Just to extend birryree's answer for paths with spaces: You cannot use the eval command as is because it seperates evaluation by spaces. One solution is to replace spaces temporarily for the eval command:
mypath="~/a/b/c/Something With Spaces"
expandedpath=${mypath// /_spc_} # replace spaces
eval expandedpath=${expandedpath} # put spaces back
expandedpath=${expandedpath//_spc_/ }
echo "$expandedpath" # prints e.g. /Users/fred/a/b/c/Something With Spaces"
ls -lt "$expandedpath" # outputs dir content
This example relies of course on the assumption that mypath never contains the char sequence "_spc_".
You might find this easier to do in python.
(1) From the unix command line:
python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' ~/fred
Results in:
/Users/someone/fred
(2) Within a bash script as a one-off - save this as test.sh:
#!/usr/bin/env bash
thepath=$(python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' $1)
echo $thepath
Running bash ./test.sh results in:
/Users/someone/fred
(3) As a utility - save this as expanduser somewhere on your path, with execute permissions:
#!/usr/bin/env python
import sys
import os
print os.path.expanduser(sys.argv[1])
This could then be used on the command line:
expanduser ~/fred
Or in a script:
#!/usr/bin/env bash
thepath=$(expanduser $1)
echo $thepath
Just use eval correctly: with validation.
case $1${1%%/*} in
([!~]*|"$1"?*[!-+_.[:alnum:]]*|"") ! :;;
(*/*) set "${1%%/*}" "${1#*/}" ;;
(*) set "$1"
esac&& eval "printf '%s\n' $1${2+/\"\$2\"}"
I have done this with variable parameter substitution after reading in the path using read -e (among others). So the user can tab-complete the path, and if the user enters a ~ path it gets sorted.
read -rep "Enter a path: " -i "${testpath}" testpath
testpath="${testpath/#~/${HOME}}"
ls -al "${testpath}"
The added benefit is that if there is no tilde nothing happens to the variable, and if there is a tilde but not in the first position it is also ignored.
(I include the -i for read since I use this in a loop so the user can fix the path if there is a problem.)
for some reason when the string is already quoted only perl saves the day
#val="${val/#\~/$HOME}" # for some reason does not work !!
val=$(echo $val|perl -ne 's|~|'$HOME'|g;print')
I think that
thepath=( ~/abc/def/ghi )
is easier than all the other solutions... or I am missing something? It works even if the path does not really exists.

lambda functions in bash

Is there a way to implement/use lambda functions in bash? I'm thinking of something like:
$ someCommand | xargs -L1 (lambda function)
I don't know of a way to do this, however you may be able to accomplish what you're trying to do using:
somecommand | while read -r; do echo "Something with $REPLY"; done
This will also be faster, as you won't be creating a new process for each line of text.
[EDIT 2009-07-09]
I've made two changes:
Incorporated litb's suggestion of using -r to disable backslash processing -- this means that backslashes in the input will be passed through unchanged.
Instead of supplying a variable name (such as X) as a parameter to read, we let read assign to its default variable, REPLY. This has the pleasant side-effect of preserving leading and trailing spaces, which are stripped otherwise (even though internal spaces are preserved).
From my observations, together these changes preserve everything except literal NUL (ASCII 0) characters on each input line.
[EDIT 26/7/2016]
According to commenter Evi1M4chine, setting $IFS to the empty string before running read X (e.g., with the command IFS='' read X) should also preserve spaces at the beginning and end when storing the result into $X, meaning you aren't forced to use $REPLY.
if you want true functions, and not just pipes or while loops (e.g. if you want to pass them around, as if they were data) I’d just not do lambdas, and define dummy functions with a recurring dummy name, to use right away, and throw away afterwards. Like so:
# An example map function, to use in the example below.
map() { local f="$1"; shift; for i in "$#"; do "$f" "$i"; done; }
# Lambda function [λ], passed to the map function.
λ(){ echo "Lambda sees $1"; }; map λ *
Like in proper functional languages, there’s no need to pass parameters, as you can wrap them in a closure:
# Let’s say you have a function with three parameters
# that you want to use as a lambda:
# (As in: Partial function application.)
trio(){ echo "$1 Lambda sees $3 $2"; }
# And there are two values that you want to use to parametrize a
# function that shall be your lambda.
pre="<<<"
post=">>>"
# Then you’d just wrap them in a closure, and be done with it:
λ(){ trio "$pre" "$post" "$#"; }; map λ *
I’d argue that it’s even shorter than all other solutions presented here.
What about this?
somecommand | xargs -d"\n" -I{} echo "the argument is: {}"
(assumes each argument is a line, otherwise change delimiter)
#!/bin/bash
function customFunction() {
eval $1
}
command='echo Hello World; echo Welcome;'
customFunction "$command"
GL
Source
if you want only xargs (due parallel -P N option for example), and only bash as function code, then bash -c can be used as parameter for xargs.
seq 1 10 | tr '\n' '\0' | xargs -0 -n 1 bash -c 'echo any bash code $0'
tr and -0 option are used here to disable any xargs parameters substitutions.
Yes. One can pass around a string variable representing a command call, and then execute the command with eval.
Example:
command='echo howdy'
eval "$command"
The eval trick has been already mentioned but here's my extended example of bash closures:
#!/usr/bin/env bash
set -e
function multiplyBy() {
X="$1"
cat <<-EOF
Y="\$1"
echo "$X * \$Y = \$(( $X * \$Y ))"
EOF
}
function callFunc() {
CODE="$1"
shift
eval "$CODE"
}
MULT_BY_2=`multiplyBy 2`
MULT_BY_4=`multiplyBy 4`
callFunc "$MULT_BY_2" 10
callFunc "$MULT_BY_4" 10
PS I've just came up with this for a completely different purpose and was just searching google to see if sb is using that. I actually needed to evaluate a reusable function in the context (shell) of main script.

Resources