BASH - how echo works inside EOF tags - bash

I would like to execute the followings:
PASSWORD="mypassword"
RUNCOMMAND=$(cat <<EOF
echo $PASSWORD | sudo -S sudo echo "this is it babe"
EOF
)
But instead of this is it babe, I get the following result:
mypassword | sudo -S sudo echo "this is it babe"
I tried with cat <<\EOF, cat <<'EOF' still no luck.
Any ideas?

You are confusing a heredoc with a pipeline.
heredoc with variable expansion:
cat <<EOF
some text, possibly with variables: ${HOME} / $(whoami)
EOF
some text, possibly with variables: /home/attie / attie
heredoc without variable expansion:
cat <<"EOF"
some text, possibly with variables: ${HOME} / $(whoami)
EOF
some text, possibly with variables: ${HOME} / $(whoami)
pipeline with variable expansion (note the quotes, "):
echo "some text, possibly with variables: ${HOME} / $(whoami)" | cat
some text, possibly with variables: /home/attie / attie
pipeline without variable expansion (note the quotes, '):
echo 'some text, possibly with variables: ${HOME} / $(whoami)' | cat
some text, possibly with variables: ${HOME} / $(whoami)
${...} expands an environment variable
$(...) runs a command, and substitutes its stdout
It also looks like you're trying to have your password entered into sudo - this won't work, as sudo will repoen the terminal to acquire your password, before passing it's stdin to the final application.

You are starting from a false premise, that eval $RUNCOMMAND is something you should do. It is not; variables are for data, functions are for code.
run_command () {
docker_run_options=(
--restart=always
--name "${USER_NAME}_$(date +%Y%m%d-%H%M%S)"
-d
-e "VIRTUAL_HOST=$USER_VIRTUAL_HOST"
-e "VIRTUAL_PORT=$USER_VIRTUAL_PORT"
-e "PORT=$USER_VIRTUAL_PORT"
-p "$USER_VIRTUAL_PORT:$USER_VIRTUAL_PORT"
)
echo "$1" | sudo -S sudo docker run "${docker_run_options[#]}" "$USER_IMAGE"
}
fun_run_command () {
run_command "PASSWORD"
}

The final solution is rather simple:
PASSWORD="mypassword"
RUNCOMMAND=$(cat <<EOF
echo $PASSWORD | sudo -S sudo echo "this is it babe"
EOF
)
And execute it via eval:
eval $RUNCOMMAND
Sorry for stealing your times with this obvious problem guys:)
The usecase for the above is to echo a given command before really executing it.
Like this:
fun_run_command(){
# execute the final command
echo `eval $RUNCOMMAND`
}
fun_echo_command(){
# echo the command which will be launched (fun_run_command())
echo ${RUNCOMMAND//$PASSWORD/PASSWORD}
}
RUNCOMMAND=$(cat <<EOF
echo $PASSWORD | sudo -S sudo docker run --restart=always \
--name ${USER_NAME}_`date +%Y%m%d-%H%M%S` \
-d \
-e "VIRTUAL_HOST=$USER_VIRTUAL_HOST" \
-e "VIRTUAL_PORT=$USER_VIRTUAL_PORT" \
-e "PORT=$USER_VIRTUAL_PORT" \
-p $USER_VIRTUAL_PORT:$USER_VIRTUAL_PORT \
$USER_IMAGE
EOF
)
As you can see the command what I launch is quite long,
so it is always make sense to doublecheck what is executed by the script.
Having copy&paste the same command to multiple function is prone to error.

Related

How to get a bash variable from inside postgre's?

I'm kind of new in bash script and postgresql.
I saw in another question a way to run a bash script as psql user here.
I tried making a bash function as follow,
postgres_create_db(){
sudo su postgres <<- EOF
if psql -lqt | cut -d \| -f 1 | grep -qw nokia_aaa_poc_db; then
psql -c '\dt'
else
psql -c 'CREATE DATABASE nokia_AAA_poc_db;'
fi
EOF
exit
}
where this function will be called further in code, but I wonder if I can add a RETURN to the function that's actualy returning a varible that was first declared inside postgres bash (in between the EOF's). Like bellow:
postgres_create_db(){
sudo su postgres <<- EOF
if psql -lqt | cut -d \| -f 1 | grep -qw nokia_aaa_poc_db; then
psql -c '\dt'
exists=1 #where thats a variable that I want to access outside the postgres bash.
else
psql -c 'CREATE DATABASE nokia_AAA_poc_db;'
fi
EOF
exit
return exists
}
but it gives an error on shellcheck
return exists
^-- SC2152: Can only return 0-255. Other data should be written to stdout.
Functions in bash can only return values from 0 to 255 where 0 is success. Reference: Return value in a Bash function
So you can echo the variable like this instead:
#!/usr/bin/env bash
postgres_test() {
psql -c '\dt' &> /dev/null
declare exists=1
echo $exists
}
printf "%s\n" "$(postgres_test)"
This prints "1".
You'll also notice that I redirected the output of the Postgres command to /dev/null. This is because it would be combined in the function's output otherwise.
You might wish to redirect that output to a file instead.

Why can't pass the variable's value into file in /etc directory?

I want to pass the value of the $ip variable into the file /etc/test.json with bash.
ip="xxxx"
sudo bash -c 'cat > /etc/test.json <<EOF
{
"server":"$ip",
}
EOF'
I expect the content of /etc/test.json to be
{
"server":"xxxx",
}
However the real content in /etc/test.json is:
{
"server":"",
}
But if I replace the target directory /etc/ with /tmp
ip="xxxx"
cat > /tmp/test.json <<EOF
{
"server":"$ip",
}
EOF
the value of the $ip variable gets passed into /tmp/test.json:
$ cat /tmp/test.json
{
"server":"xxxx",
}
In Kamil Cuk's example, the subprocess is cat > /etc/test.json which contains no variable.
sudo sh -c 'cat > /etc/test.json' << EOF
{
"server":"$ip",
}
EOF
It does not export the $ip variable at all.
Now let's make an analysis for the following:
ip="xxxx"
sudo bash -c "cat > /etc/test.json <<EOF
{
"server":\""$ip"\",
}
EOF"
The different parts in
"cat > /etc/test.json <<EOF
{
"server":\""$ip"\",
}
EOF"
will concatenate into a long string and as a command .Why can the $ip variable inherit the value from its father process here?
There are two reasons for this behavior:
Per default, variables are no passed to the environment of subsequently executed commands.
The variable is not expanded in the current context, because your command is wrapped in single quotes.
Exporting the variable
Place an export statement before the variable, see man 1 bash
The supplied names are marked for automatic export to the environment of subsequently executed commands.
And as noted by Léa Gris you also need to tell sudo to preserve the environment with the -E or --preserve-environment flag.
export ip="xxxx"
sudo -E bash -c 'cat > /etc/test.json <<EOF
{
"server":"$ip",
}
EOF'
Expand the variable in the current context:
This is the reason your second command works, you do not have any quotes around the here document in this example.
But if I replace the target directory /etc/ with /tmp [...] the value of the $ip variable gets passed into /tmp/test.json
You can change your original snippet by replacing the single quotes with double quotes and escaping the quotes around your ip:
ip="xxxx"
sudo bash -c "cat > /etc/test.json <<EOF
{
"server":\""$ip"\",
}
EOF"
Edit: For your additional questions:
In Kamil Cuk's example, the subprocess is cat > /etc/test.json which contains no variable.
sudo sh -c 'cat > /etc/test.json' << EOF
{
"server":"$ip",
}
EOF
It does not export the $ip variable at all.
Correct and you did not wrap the here document in single quotes. Therefore $ip is substituted in the current context and the string passed to subprocesses standard input is
{
"server":"xxxx",
}
So in this example the subprocess does not need to know the $ip variable.
Simple example
$ x=1
$ sudo -E sh -c 'echo $x'
[sudo] Password for kalehmann:
This echos nothing because
'echo $x' is wrapped in single quotes. $x is therefore not substituted in the current context
$x is not exported. Therefore the subprocess does not know its value.
$ export y=2
$ sudo -E sh -c 'echo $y'
[sudo] Password for kalehmann:
2
This echos 2 because
'echo $y' is wrapped in single quotes. $x is therefore not substituted in the current context
$y is exported. Therefore the subprocess does know its value.
$ z=3
$ sudo -E sh -c "echo $z"
[sudo] Password for kalehmann:
3
This echos 3 because
"echo $z" is wrapped in double quotes. $z is therefore substituted in the current context
There little need to do the here document inside the subshell. Just do it outside.
sudo tee /etc/test.json <<EOF
{
"server":"$ip",
}
EOF
or
sudo sh -c 'cat > /etc/test.json' << EOF
{
"server":"$ip",
}
EOF
Generally, it is not safe to build a fragment of JSON using string interpolation, because it requires you to ensure the variables are properly encoded. Let a tool like jq to that for you.
Pass the output of jq to tee, and use sudo to run tee to ensure that the only thing you do as root is open the file with the correct permissions.
ip="xxxx"
jq --arg x "$ip" '{server: $x}' | sudo tee /etc/test.json > /dev/.null

How to make runuser correctly forward all command line arguments, instead of trying to interpret them?

I got this simple script:
#!/bin/bash
SOURCE_USER=$USER
DESTINE_USER=$1
id -u $SOURCE_USER > /dev/null 2>&1
if [ "$?" == "1" ] || [ -z $SOURCE_USER ]
then
printf "Error: Invalid source user '$SOURCE_USER'\\n"
exit 1
fi
if [ -z $DESTINE_USER ]
then
printf "Error: Invalid destine user '$DESTINE_USER'\\n"
exit 1
fi
SOURCE_GROUPS=$(id -Gn ${SOURCE_USER} | sed "s/${SOURCE_USER} //g" | sed "s/ ${SOURCE_USER}//g" | sed "s/ /,/g")
SOURCE_SHELL=$(awk -F : -v name=${SOURCE_USER} '(name == $1) { print $7 }' /etc/passwd)
id -u $DESTINE_USER > /dev/null 2>&1
if [ "$?" == "1" ]
then
printf "Creating destine user %s\\n" "$DESTINE_USER"
useradd --groups ${SOURCE_GROUPS} --shell ${SOURCE_SHELL} --create-home ${DESTINE_USER}
passwd ${DESTINE_USER}
xhost '+si:localuser:$DESTINE_USER'
sudo usermod -G "$SOURCE_USER" "$DESTINE_USER"
else
printf "Updating destine user '%s' with groups '%s' and shell '%s'\\n" "$DESTINE_USER" "$SOURCE_GROUPS" "$SOURCE_SHELL"
sudo usermod -a -G "$SOURCE_GROUPS" "$DESTINE_USER"
sudo chsh -s "$SOURCE_SHELL" "$SOURCE_USER"
fi
sudo runuser sublime_vanilla -c "${#:2}"
I run it like this:
$ bash run_as_user.sh sublime_vanilla /usr/bin/subl -n "./New Empty File"
But when I run it, I got this error:
runuser: invalid option -- 'n'
Try 'runuser --help' for more information.
But if I replace sudo runuser sublime_vanilla -c "${#:2}" with sudo runuser sublime_vanilla -c "\"$2\" \"$3\" \"$4\" \"$5\" \"$6\" \"$7\" \"$8\" \"${#:9}\""
Then, Sublime Text correctly opens the file "./New Empty File" in a new window.
How to make runuser correctly understand all argument with a variable number of command line arguments, i.e., without hard coding "\"$2\" \"$3\" \"$4\" ..."?
This is slightly different from your last question because you have to make the expansion of the arguments into a single string for the -c option.
The bash printf formatter %q is your friend here:
cmd=$( printf '%q ' "${#:2}" )
sudo runuser sublime_vanilla -c "$cmd"
On the other hand, a quick perusal through the runuser man page suggests:
sudo runuser -u sublime_vanilla "${#:2}"
Another thought: sudo runuser -u sublime_vanilla -- "${#:2}" with the double hyphens to indicate the end of the runuser options.

Adding spaces to stdout

Is it possible to add spaces to the left of every output to stdout (and stderr if possible) when I run commands in a bash shell script?
I'd like to do something like:
#!/bin/bash
echo Installing: Something
echo " => installing prerequisite1"
## INSERT MAGICAL LEFT SPACES COMMAND HERE ##
apt-get install -q -y prerequisite
## ANOTHER MAGICAL CANCELLING LEFT SPACES COMMAND HERE ##
echo " => installing prerequisite2"
# ... the padding again ...
wget http://abc.com/lostzilla.tar.gz
tar vzxf lostzilla.tar.gz
cd lostzilla-1.01
./configure
make && make install
# ... end of padding ...
echo Done.
Any idea?
EDIT: Added quotes to the echo command, otherwise they won't be padded.
Yes, you can quote them for simple things:
echo ' => installing prerequisite1'
and pipe the output through sed for complex things:
tar vzxf lostzilla.tar.gz 2>&1 | sed 's/^/ /'
The 2>&1 puts stdout and stderr onto the stdout stream and the sed replaces every start-of-line marker with three spaces.
How well this will work on something like wget which does all sorts of cursor manipulations I'm not sure.
Example shown here:
pax> ls -1 p*
phase1.py
phase1.sh
phase2.py
phase2.sh
primes.c
primes.exe
primes.sh
primes.stat
pax> ls -1 p* | sed 's/^/ /'
phase1.py
phase1.sh
phase2.py
phase2.sh
primes.c
primes.exe
primes.sh
primes.stat
One trick I've used in the past is to ensure that the scripts themselves take care of the indentation:
#!/bin/bash
if [[ "${DONT_EVER_SET_THIS_VAR}" = "" ]] ; then
export DONT_EVER_SET_THIS_VAR=except_for_here
$0 | sed 's/^/ /'
exit
fi
ls -1 p*
This will re-run the script with indentation through sed if it's not already doing so. That way, you don't have to worry about changing all your output statements. A bit of a hack, I know, but I tend to just do what's necessary for quick-and-dirty shell scripts.
If you want to turn spacing on and off, use the following awk script:
#!/usr/bin/gawk -f
/^#SPACEON/ { spaces=1; }
/^#SPACEOFF/ { spaces=0; }
!/^#SPACE/ {
if(spaces) {
print " " $0;
} else {
print $0;
}
}
Note that there are slight problems with your bash scipt. Notably, the use of => in your echo statements will output the character = to the file "installing".
#!/bin/bash
echo Installing: Something
echo '=> installing prerequisite1'
echo '#SPACEON'
echo You would see apt-get install -q -y prerequisite
echo '#SPACEOFF'
echo '=> installing prerequisite2'
echo '#SPACEON'
echo You would see wget http://abc.com/lostzilla.tar.gz
echo You would see tar vzxf lostzilla.tar.gz
echo You would see cd lostzilla-1.01
echo You would see ./configure
echo You would see make \&\& make install
echo '#SPACEOFF'
echo Done.
Combining the two gives me:
$ ./do-stuff | ./magic-spacing
Installing: Something
=> installing prerequisite1
You would see apt-get install -q -y prerequisite
=> installing prerequisite2
You would see wget http://abc.com/lostzilla.tar.gz
You would see tar vzxf lostzilla.tar.gz
You would see cd lostzilla-1.01
You would see ./configure
You would see make && make install
Done.
Where do-stuff is your bash script and magic-spacing is my awk script above.
Depending on how the command writes to stdout, you can just indent with a simple awk script:
$ echo -e 'hello\nworld' | awk '{print " ",$0}'
hello
world
Quite un-magical you can use printf to do the following:
# space padding for single string
printf "%-4s%s\n" "" "=> installing prerequisite1"
# space padding for single command output
# use of subshell leaves original IFS intact
( IFS=$'\n'; printf " %s\n" $(command ls -ld * 2>&1) )
# note: output to stderr is unbuffered
( IFS=$'\n'; printf " %s\n" $(command ls -ld * 1>&2) )
It's also possible to group commands by enclosing them in curly braces and space-padd their output like so:
{
cmd1 1>&2
cmd2 1>&2
cmd3 1>&2
} 2>&1 | sed 's/.*/ &/'
It's possible to redirect stdout to stderr script/shell-wide using exec ...
(
exec 1>&2
command ls -ld *
) 2>&1 | sed 's/^/ /'
Use python pyp (The Pyed Piper):
ls -ld | pyp "' '+p"

have to determine all users home directories - tilde scripting problem

Assume someuser has a home directory /home/someuser
NAME=someuser
In bash - what expression to I use combining tilde (~) and $NAME to return the users home directory?
HOMEDIRECTORY=~someuser
echo $HOMEDIRECTORY
/home/someuser
NAME=someuser
echo ~$NAME
~someuser
any suggestions?
Safer:
eval HOMEDIRECTORY="$(printf "~%q" "$NAME")"
Here the %q option to printf quotes and escapes dangerous characters.
If $NAME is joe, you'd get something like /home/joe. For root, you might get /root. For "abc;rm something" you'd get "~abc;rm something" instead of having something removed.
If you have access to getent:
getent passwd "$NAME" | cut -d: -f 6
Tilde ( ~ ) it's the same as $HOME so, not all the user will have as root to home the same directory.
But if you insist in using the tilde this do the work:
echo ~/../$NAME
See:
$ pwd
/home/oreyes
$ export NAME=john
$ export DIRECTORYNAME=~/../$NAME
$ cd $DIRECTORYNAME
$ pwd
/home/john
Interesting difference between bash and csh, where ~$VARNAME actually does what you'd expect!
This is ugly, but it seems to work in bash:
homedir=`eval "echo ~$USERNAME"`
Now $homedir holds the home directory associated with $USERNAME.
BEST METHOD
Required: nothing
(n.b., this is the same technique as getent without requiring getent)
home() { # returns empty string on invalid user
grep "^$1:" /etc/passwd | cut -d ':' -f 6
}
# grep "^$user:" /etc/passwd | cut -d ':' -f 6
/var/lib/memcached
NICE METHOD FOR ROOT LINUX
Required: Linux, root (or sudo)
home() { # returns errorlevel 1 on invalid user
su "$1" -s '/bin/sh' -c 'echo $HOME'
}
# su memcached -s '/bin/sh' -c 'echo $HOME'
/var/lib/memcached
SOLUTION FOR COMPLETE EXPANSION
magic() { # returns unexpanded tilde express on invalid user
local _safe_path; printf -v _safe_path "%q" "$1"
eval "ln -sf $_safe_path /tmp/realpath.$$"
readlink /tmp/realpath.$$
rm -f /tmp/realpath.$$
}
Example usage:
$ magic ~nobody/would/look/here
/var/empty/would/look/here
$ magic ~invalid/this/will/not/expand
~invalid/this/will/not/expand
METHOD FOR HARNESSING CSH
This is a BASH script, it just calls csh.
Required: csh
home() { # return errorlevel 1 on invalid user
export user=$1; csh -c "echo ~$user"
}
$ export user=root; csh -c "echo ~$user"
/var/root
$ export user=nodfsv; csh -c "echo ~$user"
Unknown user: nodfsv.
METHOD OF DESPERATION
Required: finger (deprecated)
home() {
finger -m "$1" |
grep "^Directory:" |
sed -e 's/^Directory: //' -e 's/ .*//'
}
# finger -m "haldaemon" |
> grep "^Directory:" |
> sed -e 's/^Directory: //' -e 's/ .*//'
/home/haldaemon
You can combined the grep operation into sed, but since this method is sucky, I wouldn't bother.
one alternative way
awk -F":" '{print "user: "$1", Home directory is: "$6}' /etc/passwd

Resources