cat with here document not writing quoted strings to file - macos

I'm writing a shell script that automatically writes a bash script using cat. However, when I run the script, the produced bash script does not include data found in quotation marks. Here's my script:
cat <<EOS > script.bash
scriptDir=`dirname "$0"`
scriptName=`basename "$0"`
singleLine=0
function process ()
{
for file
do
if [ "${file}" = "-singleLine" ]
then
singleLine=1
else
if [ "${file}" = "-" ]
then
processStdIn
else
processFile "${file}"
fi
fi
done
}
function processFile ()
{
file="$1"
if [ -f "${file}" ]
then
printf "<table border=\"1\">"
if [ ${singleLine} -eq 0 ]
then
echo
fi
cat "${file}" | sed 's|,|</td><td>|g' |
while read line
do
printf "<tr><td>${line}</td></tr>"
if [ ${singleLine} -eq 0 ]
then
echo
fi
done
printf "</table>"
echo
else
echo "${file} does not exist!" >&2
fi
}
function processStdIn ()
{
tempFile=`GetTempPathName.ksh "${scriptName}"`
while read line
do
echo "${line}" >> "${tempFile}"
done
processFile "${tempFile}"
rm "${tempFile}"
}
function usage ()
{
echo " Usage: ${scriptName} [ -singleLine ] \"file 1\" [ . . . \"file N\" ]"
echo
echo " You may substitute '-' in place of a file to read from STDIN."
echo
echo " Use the -singleLine flag if you want to generate the HTML on"
echo " a single line. This flag may show up anywhere on the command-"
echo " line. But only data specified after this flag will be"
echo " generated in this manner."
}
if [ $# -gt 0 ]
then
process "$#"
else
usage
fi
EOS
Here's script.bash:
scriptDir=.
scriptName=test.sh
singleLine=0
function process ()
{
for file
do
if [ "" = "-singleLine" ]
then
singleLine=1
else
if [ "" = "-" ]
then
processStdIn
else
processFile ""
fi
fi
done
}
function processFile ()
{
file=""
if [ -f "" ]
then
printf "<table border=\"1\">"
if [ -eq 0 ]
then
echo
fi
cat "" | sed 's|,|</td><td>|g' |
while read line
do
printf "<tr><td></td></tr>"
if [ -eq 0 ]
then
echo
fi
done
printf "</table>"
echo
else
echo " does not exist!" >&2
fi
}
function processStdIn ()
{
tempFile=
while read line
do
echo "" >> ""
done
processFile ""
rm ""
}
function usage ()
{
echo " Usage: [ -singleLine ] \"file 1\" [ . . . \"file N\" ]"
echo
echo " You may substitute '-' in place of a file to read from STDIN."
echo
echo " Use the -singleLine flag if you want to generate the HTML on"
echo " a single line. This flag may show up anywhere on the command-"
echo " line. But only data specified after this flag will be"
echo " generated in this manner."
}
if [ 0 -gt 0 ]
then
process ""
else
usage
fi
How can I get cat to include the quoted material?

Replace:
cat <<EOS > script.bash
With:
cat <<'EOS' > script.bash
This prevents shell expansion of what is in the here document.
Documentation
From man bash:
The format of here-documents is:
<<[-]word
here-document
delimiter
No parameter expansion, command substitution, arithmetic expansion, or pathname expansion is performed on word. If any
characters in word are quoted, the delimiter is the result of quote removal on word, and the lines in the here-document
are not expanded. If word is unquoted, all lines of the here-document are subjected to parameter expansion, command sub‐
stitution, and arithmetic expansion. In the latter case, the character sequence \<newline> is ignored, and \ must be used
to quote the characters \, $, and `.
The most important part of that is:
If word is unquoted, all lines of the here-document are subjected to
parameter expansion, command substitution, and arithmetic expansion.
To stop all that expansion, just quote word which in your case is EOS.

Related

Bash script - Using spaces as string

I'm trying to build a really simple TODO list with a bash script. It should allow user to add and remove a task and also see the entire list.
I've done it with the following script. But I've issues allowing a given task to take a whitespace as a string. For instance, if I'm adding a task with the command: ./programme_stack.sh add 1 start projet n1, it will only add a task with "start".
I've read a couple of things online, I know, I should double quote the variables but after trying this, it doesn't work. I must be missing something on the road.
Here is my script:
#!/bin/bash
TACHES=$HOME/.todo_list
# functions
function remove() {
res_remove=$(sed -n "$1p" $TACHES)
sed -i "$1d" $TACHES
}
function list() {
nl $TACHES
}
function add() {
if [ ""$(($(wc -l $TACHES | cut -d " " -f 1) + 1))"" == "$1" ]
then
echo "- $2" >> $TACHES
else
sed -i "$1i - $2" $TACHES
fi
echo "Task \"$2\" has been add to the index $1"
}
function isNumber() {
re='^[0-9]+$'
if ! [[ $# =~ $re ]] ; then
res_isNumber=true
else
res_isNumber=false
fi
}
# application
case $1 in
list)
list
;;
done)
shift
isNumber $#
if ! [[ "$res_isNumber" = false ]] ; then
echo "done must be followed by an index number"
else
nb_taches=$(wc -l $TACHES | cut -d " " -f 1)
if [ "$1" -ge 1 ] && [ "$1" -le $nb_taches ]; then
remove $1
echo "Well done! Task $i ($res_remove) is completed"
else
echo "this task doesn't exists"
fi
fi
;;
add)
shift
isNumber $1
if ! [[ "$res_isNumber" = false ]] ; then
echo "add must be followed by an index number"
else
index_max=$(($(wc -l $TACHES | cut -d " " -f 1) + 1))
if [ "$1" -ge 1 ] && [ "$1" -le $index_max ]; then
add $1 $2
else
echo "Idex must be between 1 and $index_max"
fi
fi
;;
*)
echo "./programme_stack.sh (list|add|done) [args]"
;;
esac
Can you guys see what I'm missing?
Many thanks!!
To get the script to support embedded spaces, 2 changes are needed
1) Accept embedded space - either
1A) Pass in the task name in quote script add nnn "say hello", OR
1B) Concatenate all the input parameters into single string.
2) Quote the task name to prevent it from being broken into individual words
In the code, implement 1B and 2
add)
...
if [ "$1" -ge 1 ] && [ "$1" -le $index_max ]; then
num=$1
shift
# Combine all remaining arguments
todo="$#"
add "$num" "$todo"
...

Iterating through a folder that's passed in as a paramter to a Bash script

I'm trying to iterate over a folder, running a grep on each file, and putting them into separate files, tagged with a .res extension. Here's what I have so far....
#!/bin/bash
directory=$(pwd)
searchterms="searchterms.txt"
extension=".end"
usage() {
echo "usage: fmat [[[-f file ] [-d directory ] [-e ext]] | [-h]]"
echo " file - text file containing a return-delimited list of materials"
echo " directory - directory to process"
echo " ext - file extension of files to process"
echo ""
}
while [ "$1" != "" ]; do
case $1 in
-d | --directory ) shift
directory=$1
;;
-f | --file ) shift
searchterms=$1
;;
-e | --extension ) shift
extension=$1
;;
-h | --help ) usage
exit
;;
* ) usage
exit 1
esac
shift
done
if [ ! -d "$directory" ]; then
echo "Sorry, the directory '$directory' does not exist"
exit 1
fi
if [ ! -f "$searchterms" ]; then
echo "Sorry, the searchterms file '$searchterms' does not exist"
exit 1
fi
echo "Searching '$directory' ..."
for file in "${directory}/*"; do
printf "File: %s\n" ${file}
[ -e "$file" ] || continue
printf "%s\n" ${file}
if [ ${file: -3} == ${extension} ]; then
printf "%s will be processed\n" ${file}
#
# lots of processing here
#
fi
done
I know that it's down to my poor understanding of of globbing... but I can't get the test on the extension to work.
Essentially, I want to be able to specify a source directory, a file with search terms, and an extension to search for.
NOW, I realise there may be quicker ways to do this, e.g.
grep -f searchterms.txt *.end > allchanges.end.res
but I may have other processing I need to do to the files, and I want to save them into separate files: so bing.end, bong.end, would be grep'ed into bing.end.res, bong.end.res .
Please let me know, just how stupid I'm being ;-)
Just for completeness sake, here's the last part, working, thanks to #chepner and #Gordon Davisson :
echo "Searching '$directory' ..."
for file in "${directory}"/*; do
[ -e "$file" ] || continue
# show which files will be processed
if [[ $file = *.${extension#.} ]]; then
printf "Processing %s \n" "$file"
head -n 1 "${file}" > "${file}.res"
grep -f $searchterms "${file}" >> "${file}.res"
fi
done
You just need to leave the * out of the quotes, so that it isn't treated as a literal *:
for file in "${directory}"/*; do
Unlike most languages, the quotes don't define a string (as everything in bash is already a string: it's the only data type). They simply escape each character inside the quotes. "foo" is exactly the same as \f\o\o, which (because escaping most characters doesn't really have any effect) is the same as foo. Quoted or not, all characters not separated by word-splitting characters are part of the same word.
http://shellcheck.net will catch this, although not with the most useful error message. (It will also catch the other parameter expansions that you did not quote but should.)

Deleting a line that contains a string

I'm trying to delete a line that contains a string pass through an argument, but I can't get it to work. I'm on OSX 10.9.
sed -i '' '/$2/d' /etc/hosts
Shouldn't that work? It just keeps the file as is. Nothing changes. My command is sudo hosts remove junior.dev.
Here is my shell script:
#!/bin/sh
let $# || { echo No arguments supplied. Example: hosts add 192.168.2.2 mysite.dev; exit 1; }
if [ $1 = "add" ]; then
if [ -z "$2" ] || [ -z "$3" ]; then
echo "You must supply an IP address and a host name."
exit 1;
else
echo "$2\t$3" >> /etc/hosts
echo "Done."
fi
fi
if [ $1 = "remove" ]; then
if [ -z "$2" ]; then
echo "You must supply a host name."
exit 1;
else
sed -i '' '/$2/d' /etc/hosts
echo "Done."
fi
fi
Use double-quotes ":
$ echo "$foo"
> bar
$ echo '$foo'
> $foo
When using double-quotes ", the variables are expanded, when using single-quotes ', they are not expanded.

bash script gives directory listing as well when tokenizing

Here in the code below, line is a line of strings returned as the output of a command.
When I run the script, it gives me all the tokens of the string, but appends a listing of the directory to it as well. I really can't figure out what I am doing wrong.
for word in $line
do
inner_count=$((inner_count + 1))
echo $word
done
Here is the entire piece of code:
while read -r line
do
if [ "$count" = "2" ];
then
inner_count=0
#parse each line
#if [ "$debug" = "1" ] ; then printf "%s\n" "$line" > /dev/kmsg ; fi
for word in $line
do
if [ "$inner_count" = "0" ]; then tmp1="$word" ; fi
if [ "$inner_count" = "4" ]; then temp2="$word" ;fi
inner_count=$((inner_count + 1))
done
fi
count=$((count + 1))
done < <(batctl tg)
The most likely issue that I can think of that could produce this would be that there is a * in $line, and the shell is expanding that (globbing). You can disable globbing with set -f.
Try:
set -f # disable globbing
for word in $line
do
inner_count=$((inner_count + 1))
echo "$word"
done
set +f # re-enable globbing

Why are the bash -n and -z test operators not inverses for $#

function wtf() {
echo "\$*='$*'"
echo "\$#='$#'"
echo "\$#='"$#"'"
echo "\$#='""$#""'"
if [ -n "$*" ]; then echo " [ -n \$* ]"; else echo "![ -n \$* ]"; fi
if [ -z "$*" ]; then echo " [ -z \$* ]"; else echo "![ -z \$* ]"; fi
if [ -n "$#" ]; then echo " [ -n \$# ]"; else echo "![ -n \$# ]"; fi
if [ -z "$#" ]; then echo " [ -z \$# ]"; else echo "![ -z \$# ]"; fi
}
wtf
produces
$*=''
$#=''
$#=''
$#=''
![ -n $* ]
[ -z $* ]
[ -n $# ]
[ -z $# ]
though it seems to me that [-n $#] should be false because 7.3 Other Comparison Operators indicates that [ -n "$X" ] should be the inverse of [ -z "$X" ] for all $X.
-z
string is null, that is, has zero length
String='' # Zero-length ("null") string variable.
if [ -z "$String" ]
then
echo "\$String is null."
else
echo "\$String is NOT null."
fi # $String is null.
-n
string is not null.
The -n test requires that the string be quoted within the test brackets. Using an unquoted string with ! -z, or even just the unquoted string alone within test brackets (see Example 7-6) normally works, however, this is an unsafe practice. Always quote a tested string. [1]
I know $# is special but I did not know it was special enough to violate boolean negation. What is happening here?
$ bash -version | head -1
GNU bash, version 4.2.42(2)-release (i386-apple-darwin12.2.0)
The actual numeric exit codes are all 1 or 0 as per
$ [ -n "$#" ]; echo "$?"
0
When $# is empty, "$#" doesn't expand to an empty string; it is removed altogether. So your test is not
[ -n "" ]
but rather
[ -n ]
Now -n isn't an operator, but just a non-empty string, which always tests as true.
"$#" doesn't do what you expect. It's not a different form of "$*", it expands to the quoted list of arguments passed to the current script.
If there are no arguments, it expands to nothing. If there are two arguments a and b c, then it expands to "a" "b c" (i.e. it preserves whitespace in arguments) while "$*" expands to "a b c" and $* would expand to a b c (three words).

Resources