I would like to "copy-paste" the following lines in a script into my bashrc:
# VIM,TMUX stuff
VIM="$(which vim)"
For example:
echo "# VIM,TMUX stuff" >> ~/.bash_profile
echo 'VIM="$(which vim)"' >> ~/.bash_profile
However, it seems to 'escape' some of the items. How would I do the equivalent of a literal 'copy-paste' as I'm trying to do above?
Use heredoc in bash:
cat <<-'EOF' > ~/.bash_profile
# VIM,TMUX stuff
VIM="$(which vim)"
EOF
Related
this is a bug I have found nothing about after a relentless search
I'm trying to run a bootstrap file in an EC2 instance, part of an EMR cluster v6.4.0. As the bootstrap action takes longer than 5 minutes, we execute it as a subprocess of the form
#!/bin/bash
var="var"
cat << EOF > ~/bootstrap.sh
intra="intra"
echo $var
echo $intra
EOF
/bin/bash ~/bootstrap.sh
exit 0
But the var "intra" is never set, and the bootstrap action returns the error line n: intra: unbound variable
If you execute that script the "intra" var is not printed.
Why can't I assign variables in a subprocess? Thank you!
When using that type of heredoc (<<WORD), you must escape literal $ characters using \$. Same goes for the backtick character (`):
#!/bin/bash
var="var"
cat << EOF > ~/bootstrap.sh
intra="intra"
echo $var
echo \$intra
EOF
/bin/bash ~/bootstrap.sh
exit 0
Another way of generating an equivalent bootstrap script is to use the literal heredoc form <<'WORD':
#!/bin/bash
var="var"
# This line will be inserted as-is without variable and subshell expansion:
cat << 'EOF1' > ~/bootstrap.sh
intra="intra"
EOF1
# We will allow this ONE line to expand.
cat << EOF2 >> ~/bootstrap.sh
echo $var
EOF2
# Back to literal insertions, no escaping necessary.
cat << 'EOF3' >> ~/bootstrap.sh
echo $intra
EOF3
/bin/bash ~/bootstrap.sh
exit 0
Inspecting the contents of ~/bootstrap.sh is a good place to start debugging.
I have two scripts. Script A includes script B and calls a function in script B.
The setup looks like this:
Test file - ~/file.txt
one==1.0.0
two==2.0.0
three==3.0.0
four==4.0.0
Script A - ~/script_a.sh
#!/bin/bash
source script_b.sh
func_one
Script B - ~/script_b.sh
#!/bin/bash
# Note: don't forget to change the spaces to tabs else heredoc won't work
my_user=$USER
func_two() {
# Here, I need run everything in the heredoc as user $my_user
sudo su - $my_user -s /bin/bash <<- EOF
while read -r line || [[ -n "$line" ]];
do
# **This is the problem line**
# I can confirm that all the lines are being
# read but echo displays nothing
echo "$line"
# The line below will be printed 4 times as there are 4 lines in the file of interest
echo "Test"
done < "/home/$my_user/file.txt"
EOF
}
func_one() {
func_two
}
To run
cd ~
bash script_a.sh
Question: Why is the line echo "$line" not producing any output?
The problem is that bash is substituting $line with its value (nothing) before it gets passed to su. Escaping the dollar sign should fix it. So $line should be changed to \$line in both places in script_b.sh.
I am trying to add echo statements in /etc/profile but they execute.
Example: This is what I wrote
echo CURRENTYEAR="`/bin/date +%y`" >> /etc/profile
echo CURRENTWEEK="`/bin/date +%V` " >> /etc/profile
echo VERSION="$CURRENTYEAR"."$CURRENTWEEK" >> /etc/profile
echo export VERSION >> /etc/profile
echo export SVN_HOME="https://example.com/svn/road" >> /etc/profile
echo SVN_BRANCH="$SVN_HOME/branches/qa_weekly/$VERSION" >> /etc/profile
echo export SVN_TAG="https://example.cpm/svn/road/tags" >> /etc/profile
echo export SVN_TRUNK="https://example.com/svn/empire/trunk" >> /etc/profile
export PATH=$PATH:/road >> /etc/profile
export SVN_BRANCH
Result: what I get
CURRENTYEAR=15
CURRENTWEEK=33
VERSION=.
export VERSION
export SVN_HOME=https://example.com/svn/road
SVN_BRANCH=/branches/qa_weekly/
export SVN_TAG=https://example.com/svn/road/tags
export SVN_TRUNK=https://example.com/svn/road/trunk
I want it as below in /etc/profile
CURRENTYEAR=`/bin/date +%y`
CURRENTWEEK=`/bin/date +%V`
VERSION=$CURRENTYEAR"."$CURRENTWEEK
export VERSION
SVN_BRANCH=$SVN_HOME/branches/qa_weekly/$VERSION
SVN_TAG=$SVN_HOME/tags
SVN_TRUNK=$SVN_HOME/trunk
Thanks in Advance.
Put your strings in single quotes:
echo 'CURRENTYEAR="`/bin/date +%y`"' >> /etc/profile
#....^.............................^
http://www.gnu.org/software/bash/manual/bashref.html#Single-Quotes
Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
Rather than multiple echo statements, use cat with a quoted here document:
cat >> /etc/profile <<'EOF'
CURRENTYEAR=`/bin/date +%y`
CURRENTWEEK=`/bin/date +%V`
VERSION="$CURRENTYEAR.$CURRENTWEEK"
export VERSION
export SVN_HOME="https://example.com/svn/road"
SVN_BRANCH="$SVN_HOME/branches/qa_weekly/$VERSION"
export SVN_TAG="https://example.cpm/svn/road/tags"
export SVN_TRUNK="https://example.com/svn/empire/trunk"
export PATH=$PATH:/road
export SVN_BRANCH
EOF
Quoting EOF in the first line prevents any substitutions from occurring in the body of the here document.
I want to create a multiline file with Make, having exact content:
#!/bin/bash
if [ "$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$( cat <<EOF
-Dmapred.job.tracker=$JT
EOF
)
${HADOOP_HOME}/bin/hadoop $1 $HADOOP_CONFIG_VARS ${*:2} 2>&1 | grep -v SLF4J
How can I tell make to output a file with this exact content somewhere?
I tried this:
define SCRIPT_CONTENT
#!/bin/bash
if [ "$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$( cat <<EOF
-Dmapred.job.tracker=$JT
EOF
)
${HADOOP_HOME}/bin/hadoop $1 $HADOOP_CONFIG_VARS ${*:2} 2>&1 | grep -v SLF4J
endef
export SCRIPT_CONTENT
bin/script:
#echo "$$SCRIPT_CONTENT" > bin/script
This paricular solution 1) wipes $ and first char after $-es and 2) is ugly because the definition should happen outside of the particular target where it's needed :(
I also tried this:
bin/script:
#echo '
#!/bin/bash
if [ "$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$( cat <<EOF
-Dmapred.job.tracker=$JT
EOF
)
${HADOOP_HOME}/bin/hadoop $1 $HADOOP_CONFIG_VARS ${*:2} 2>&1 | grep -v SLF4J
' > bin/script
This returns error when in make, works outside of make...
Any suggestion is very welcome!
Make wants any $ characters that should be reproduced literally to be escaped by inserting another $ in front of them.
More broadly, though, it seems like you're trying to use Make as a shell-script replacement. The more idomatic way to do this would be to put that content in a source file that you can copy to the destination, or to put it in a script that will write it into a specified destination. The Makefile then just has to invoke the copy command or the script.
With the help from this magnificent answer, I cooked up the following.
# From https://stackoverflow.com/a/8316519/874188
define \n
endef
define SCRIPT_CONTENT
#!/bin/bash
if [ "$$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$$( cat <<EOF
-Dmapred.job.tracker=$$JT
EOF
)
$${HADOOP_HOME}/bin/hadoop $$1 $$HADOOP_CONFIG_VARS $${*:2} 2>&1 | grep -v SLF4J
endef
bin/script:
echo '$(subst $(\n),\n,$(SCRIPT_CONTENT))' >$#
When testing, I found that I needed to have a semicolon at the end of the echo line when it didn't have any redirection. I can speculate that there is a built-in echo which gets invoked when there are no shell metacharacters in the command line?
Also, notice that the definition cannot contain any single quotes, and that all dollar signs have to be doubled. Maybe one or the other of these restrictions could be removed; I was unsuccessful, but then I didn't spend too much time or effort.
You cannot do this in make. Beyond what Novelocrat says regarding $, there's the fact that make is line-oriented and does not have any ability to generate a command that contains a newline character. All newlines that appear unescaped (without a backslash before them) are parsed by make as ending that recipe line, and each recipe line is sent to a different invocation of the shell. If you want the entire command to be sent as a single string to the same shell, then you must escape the newlines.
However, make will remove all backslash/newline pairs before it runs the command.
The only possible way to do this completely within make is to generate the file one line at a time, like this:
bin/script:
#echo '#!/bin/bash' > $#
#echo 'if [ "$$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi' >> $#
#echo 'export CONFIG_VARS=$$( cat <<EOF' >> $#
#echo ' -Dmapred.job.tracker=$$JT' >> $#
#echo 'EOF' >> $#
#echo ')' >> $#
#echo '$${HADOOP_HOME}/bin/hadoop $$1 $$HADOOP_CONFIG_VARS $${*:2} 2>&1 | grep -v SLF4J' >> $#
As Novelocrat says, the typical way this is done is to have the script file as a separate file and copy it where you want it, rather than generating it.
This question already has answers here:
here-document gives 'unexpected end of file' error
(10 answers)
Closed 6 years ago.
I have the following file and I have chmod a+x on the file.
When I try to run it, I get a line 75: syntax error: unexpected end of file. What is the error with my script? What do I need to do to fix it?
#!/bin/sh
# log directory for ascp transfer
logdirectory=/tmp/log20079
# log for this script
baselog=$logdirectory/sent4files.log
#directory of where the files to be transferred are located
basedirectory=/tmp/test20079/
#remote host data
REMOTE_DIR=/Upload
REMOTE_HOST=xxx
REMOTE_USER=xxx
# extensions of file
FEATURE_EXT=feature
KEYART_EXT=keyart
TRAILER_EXT=trailer
METADATA_EXT=metadata
# files to be excluded, must match exclude_file_suffix
COMPLETE_EXT=complete
SENT_EXT=sent
# file to send on completion
FILE_ON_COMPLETE="$basedirectory"delivery.complete
if [ "$TYPE" == "File" ]; then
if [ "$STARTSTOP" == "Stop" ]; then
if [ "$STATE" == "success" ]; then
filename=$(basename "$FILE")
extension="${filename##*.}"
# check the extension here, and create files ending in .sent as a flag
case "$extension" in
"$FEATURE_EXT")
cat > "$basedirectory$FEATURE_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Feature sent" >> $baselog
;;
"$KEYART_EXT")
cat > "$basedirectory$KEYART_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Keyart sent" >> $baselog
;;
"$TRAILER_EXT")
cat > "$basedirectory$TRAILER_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Trailer sent" >> $baselog
;;
"$METADATA_EXT")
cat > "$basedirectory$METADATA_EXT.$SENT_EXT" << 'EOF'
EOF
echo "Metadata sent" >> $baselog
;;
esac
# check that all four files exists
if [ -e "$basedirectory$FEATURE_EXT.$SENT_EXT" ] && [ -e "$basedirectory$KEYART_EXT.$SENT_EXT" ] && [ -e "$basedirectory$TRAILER_EXT.$SENT_EXT" ] && [ -e "$basedirectory$METADATA_EXT.$SENT_EXT" ]; then
echo "All files sent" >> $baselog
echo>$FILE_ON_COMPLETE
# $FILE_ON_COMPLETE "$REMOTE_USER#$REMOTE_HOST:$REMOTE_DIR"
rm "$basedirectory$FEATURE_EXT.$SENT_EXT"
rm "$basedirectory$KEYART_EXT.$SENT_EXT"
rm "$basedirectory$TRAILER_EXT.$SENT_EXT"
rm "$basedirectory$METADATA_EXT.$SENT_EXT"
rm $FILE_ON_COMPLETE
fi
fi
fi
fi
Heredocs are tricky beasts to get right. If you use 'EOF' that's exactly what the closing line needs to be, with no whitespace at the front like you have.
Alternatively, you can use the <<- variant which strips off all leading tab characters from the lines in the heredoc and the closing line as per the following transcript (where <tab> is the TAB character):
pax> cat <<-'eof'
...> 1
...< 2
...> <tab>eof
...> 4
...> eof
1
2
<tab>eof
4
pax> cat <<-'eof'
...> 1
...> 2
...> <tab>eof
1
2
Using the <<- variant allows for neater files, thoug it's no good if you want to preserve leading tabs of course. From the bash manpage:
If the redirection operator is <<-, then all leading tab characters are stripped from input lines and the line containing delimiter. This allows here-documents within shell scripts to be indented in a natural fashion.
Of course, if you're just wanting to use those files as flag files, there's a better way than cat with a heredoc. Just use:
touch "$basedirectory$FEATURE_EXT.$SENT_EXT"
This will create the file if it doesn't exist and update the modification time if it does, just like the cat but without messing about with heredocs. It won't empty the file but, if you need that for some reason:
rm -f "$basedirectory$FEATURE_EXT.$SENT_EXT"
touch "$basedirectory$FEATURE_EXT.$SENT_EXT"
will do the trick.
However, since the heredoc does actually output a single empty line (one \n character), you can opt for:
echo >"$basedirectory$FEATURE_EXT.$SENT_EXT"
instead.
The end of your heredocs EOF can not have any whitespace in front of them. Remove those spaces.