Unbound variable - bash

this is a bug I have found nothing about after a relentless search
I'm trying to run a bootstrap file in an EC2 instance, part of an EMR cluster v6.4.0. As the bootstrap action takes longer than 5 minutes, we execute it as a subprocess of the form
#!/bin/bash
var="var"
cat << EOF > ~/bootstrap.sh
intra="intra"
echo $var
echo $intra
EOF
/bin/bash ~/bootstrap.sh
exit 0
But the var "intra" is never set, and the bootstrap action returns the error line n: intra: unbound variable
If you execute that script the "intra" var is not printed.
Why can't I assign variables in a subprocess? Thank you!

When using that type of heredoc (<<WORD), you must escape literal $ characters using \$. Same goes for the backtick character (`):
#!/bin/bash
var="var"
cat << EOF > ~/bootstrap.sh
intra="intra"
echo $var
echo \$intra
EOF
/bin/bash ~/bootstrap.sh
exit 0
Another way of generating an equivalent bootstrap script is to use the literal heredoc form <<'WORD':
#!/bin/bash
var="var"
# This line will be inserted as-is without variable and subshell expansion:
cat << 'EOF1' > ~/bootstrap.sh
intra="intra"
EOF1
# We will allow this ONE line to expand.
cat << EOF2 >> ~/bootstrap.sh
echo $var
EOF2
# Back to literal insertions, no escaping necessary.
cat << 'EOF3' >> ~/bootstrap.sh
echo $intra
EOF3
/bin/bash ~/bootstrap.sh
exit 0
Inspecting the contents of ~/bootstrap.sh is a good place to start debugging.

Related

How to echo literal output in bash

I would like to "copy-paste" the following lines in a script into my bashrc:
# VIM,TMUX stuff
VIM="$(which vim)"
For example:
echo "# VIM,TMUX stuff" >> ~/.bash_profile
echo 'VIM="$(which vim)"' >> ~/.bash_profile
However, it seems to 'escape' some of the items. How would I do the equivalent of a literal 'copy-paste' as I'm trying to do above?
Use heredoc in bash:
cat <<-'EOF' > ~/.bash_profile
# VIM,TMUX stuff
VIM="$(which vim)"
EOF

Read file line by line in sourced bash script within heredoc

I have two scripts. Script A includes script B and calls a function in script B.
The setup looks like this:
Test file - ~/file.txt
one==1.0.0
two==2.0.0
three==3.0.0
four==4.0.0
Script A - ~/script_a.sh
#!/bin/bash
source script_b.sh
func_one
Script B - ~/script_b.sh
#!/bin/bash
# Note: don't forget to change the spaces to tabs else heredoc won't work
my_user=$USER
func_two() {
# Here, I need run everything in the heredoc as user $my_user
sudo su - $my_user -s /bin/bash <<- EOF
while read -r line || [[ -n "$line" ]];
do
# **This is the problem line**
# I can confirm that all the lines are being
# read but echo displays nothing
echo "$line"
# The line below will be printed 4 times as there are 4 lines in the file of interest
echo "Test"
done < "/home/$my_user/file.txt"
EOF
}
func_one() {
func_two
}
To run
cd ~
bash script_a.sh
Question: Why is the line echo "$line" not producing any output?
The problem is that bash is substituting $line with its value (nothing) before it gets passed to su. Escaping the dollar sign should fix it. So $line should be changed to \$line in both places in script_b.sh.

ssh command cannot return the result executed on remote machine

when I use ssh to execute remote file in shell script, so that I could get the result executed on remote machine, however, I met a problem which the user defined variables could not be recognized.
I created a shell script run.sh like this
#!/bin/bash
ssh jenkins#10.122.214.55 << EOF
./test.sh && ret=1 || ret=0
echo "before"
echo ${ret}
echo "after"
echo ${HOME}
exit ${ret}
EOF
the content of test.sh which is called by run.sh :
#!/bin/bash
echo "lala"
exit 1
when I call ./run.sh
it print like this
lala
before
after
/home/jenkins
Why did not it echo ${ret}? After ./run.sh is called, echo $? is 0 which is unexpected, I thought it should echo 1 here.
Thanks!
Because the variables in the heredoc are being expanded by the local script before being sent to the standard input of ssh.
In other words, your heredoc behaves similarly to
#!/bin/bash
string="echo \"before\"
echo ${ret}
echo \"after\"
echo ${HOME}
exit ${ret}
./test.sh && ret=1 || ret=0"
echo "$string" | ssh jenkins#10.122.214.55
which makes it more obvious that the string has the variables interpolated.
There are a couple ways around this: either you can escape the $ symbols (e.g. echo \${ret}) so that the intended string is passed through, or you can use a verbatim heredoc
#!/bin/bash
ssh jenkins#10.122.214.55 <<'EOF'
./test.sh && ret=1 || ret=0
echo "before"
echo ${ret}
echo "after"
echo ${HOME}
exit ${ret}
EOF
By single-quoting the delimiter, we ensure that no expansion takes place inside the heredoc.

bash return value in pipe to bash

We have a script, with a return code. For example
#!/bin/bash
exit 42
which is works fine:
$ ./script ; echo $?
42
but if i go:
$ bash << EOF
./script ; echo $?
EOF
0
so it prints 0, while one would expect it to still print 42
Your $? is being expanded before executing the script. If you don't want your variables to expand in the heredoc (not a pipe) put single quotes around the name:
bash <<'EOF'
./script; echo $?
EOF
That wil prevent $? from being expanded while passing the string to the new bash command. It will, instead, be evaluated in the string which is what you seem to be going for.

Bash script: syntax error: unexpected end of file [duplicate]

This question already has answers here:
here-document gives 'unexpected end of file' error
(10 answers)
Closed 6 years ago.
I have the following file and I have chmod a+x on the file.
When I try to run it, I get a line 75: syntax error: unexpected end of file. What is the error with my script? What do I need to do to fix it?
#!/bin/sh
# log directory for ascp transfer
logdirectory=/tmp/log20079
# log for this script
baselog=$logdirectory/sent4files.log
#directory of where the files to be transferred are located
basedirectory=/tmp/test20079/
#remote host data
REMOTE_DIR=/Upload
REMOTE_HOST=xxx
REMOTE_USER=xxx
# extensions of file
FEATURE_EXT=feature
KEYART_EXT=keyart
TRAILER_EXT=trailer
METADATA_EXT=metadata
# files to be excluded, must match exclude_file_suffix
COMPLETE_EXT=complete
SENT_EXT=sent
# file to send on completion
FILE_ON_COMPLETE="$basedirectory"delivery.complete
if [ "$TYPE" == "File" ]; then
if [ "$STARTSTOP" == "Stop" ]; then
if [ "$STATE" == "success" ]; then
filename=$(basename "$FILE")
extension="${filename##*.}"
# check the extension here, and create files ending in .sent as a flag
case "$extension" in
"$FEATURE_EXT")
cat > "$basedirectory$FEATURE_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Feature sent" >> $baselog
;;
"$KEYART_EXT")
cat > "$basedirectory$KEYART_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Keyart sent" >> $baselog
;;
"$TRAILER_EXT")
cat > "$basedirectory$TRAILER_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Trailer sent" >> $baselog
;;
"$METADATA_EXT")
cat > "$basedirectory$METADATA_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Metadata sent" >> $baselog
;;
esac
# check that all four files exists
if [ -e "$basedirectory$FEATURE_EXT.$SENT_EXT" ] && [ -e "$basedirectory$KEYART_EXT.$SENT_EXT" ] && [ -e "$basedirectory$TRAILER_EXT.$SENT_EXT" ] && [ -e "$basedirectory$METADATA_EXT.$SENT_EXT" ]; then
echo "All files sent" >> $baselog
echo>$FILE_ON_COMPLETE
# $FILE_ON_COMPLETE "$REMOTE_USER#$REMOTE_HOST:$REMOTE_DIR"
rm "$basedirectory$FEATURE_EXT.$SENT_EXT"
rm "$basedirectory$KEYART_EXT.$SENT_EXT"
rm "$basedirectory$TRAILER_EXT.$SENT_EXT"
rm "$basedirectory$METADATA_EXT.$SENT_EXT"
rm $FILE_ON_COMPLETE
fi
fi
fi
fi
Heredocs are tricky beasts to get right. If you use 'EOF' that's exactly what the closing line needs to be, with no whitespace at the front like you have.
Alternatively, you can use the <<- variant which strips off all leading tab characters from the lines in the heredoc and the closing line as per the following transcript (where <tab> is the TAB character):
pax> cat <<-'eof'
...> 1
...< 2
...> <tab>eof
...> 4
...> eof
1
2
<tab>eof
4
pax> cat <<-'eof'
...> 1
...> 2
...> <tab>eof
1
2
Using the <<- variant allows for neater files, thoug it's no good if you want to preserve leading tabs of course. From the bash manpage:
If the redirection operator is <<-, then all leading tab characters are stripped from input lines and the line containing delimiter. This allows here-documents within shell scripts to be indented in a natural fashion.
Of course, if you're just wanting to use those files as flag files, there's a better way than cat with a heredoc. Just use:
touch "$basedirectory$FEATURE_EXT.$SENT_EXT"
This will create the file if it doesn't exist and update the modification time if it does, just like the cat but without messing about with heredocs. It won't empty the file but, if you need that for some reason:
rm -f "$basedirectory$FEATURE_EXT.$SENT_EXT"
touch "$basedirectory$FEATURE_EXT.$SENT_EXT"
will do the trick.
However, since the heredoc does actually output a single empty line (one \n character), you can opt for:
echo >"$basedirectory$FEATURE_EXT.$SENT_EXT"
instead.
The end of your heredocs EOF can not have any whitespace in front of them. Remove those spaces.

Resources