I am trying to escape spaces in a path name /Volumes/NO NAME. I need to access files present within this directory via a bash script. I have used sed and printf commands and neither seems to have worked. The code I have so far:
while read line
do
if [[ -d ${line} ]]; then
echo "${line} is a directory. Skipping this.";
elif [[ -f ${line} ]]; then
spacedOutLine=`echo ${line} | sed -e 's/ /\\ /g'`;
#spacedOutLine=$(printf %q "$line");
echo "line = ${line} and spacedOutLine = ${spacedOutLine}";
else
echo "Some other type of file. Unrecognized.";
fi
done < ${2}
Neither of this seems to have worked. For an input like: /Volumes/NO NAME/hello.txt, the output is:
/Volumes/NO: No such file or directory
NAME/hello.txt: No such file or directory
I am on a Mac and using bash via the terminal. I have also gone through a lot of other posts on SO about this which have not helped me.
Use double quotes while calling the ${line} variable.
"${line}"
Related
I'm reading some file paths and names from a text file and trying to test if file exists. I'm not sure what I'm doing wrong but first echo returns filepath and file name whilst the echo inside the if statement doesn't. Any ideas?
#!/bin/bash
while read line; do
echo $line
if [ -f "$line" ]; then
echo "found: $line"
fi
done < /mbackup/temp/images.txt
The only change is adding the -r option to read. That option is documented as:
Backslash does not act as an escape character. The backslash is considered to be part of the line. In particular, a backslash-newline pair may not then be used as a line continuation.
This helps prevent special characters in file names from interfering with your script.
I test this with files containing special characters and it works as you expected.
#!/bin/bash
while read -r line; do
echo $line
if [ -f "$line" ]; then
echo "found: $line"
fi
done < /mbackup/temp/images.txt
I'm creating a bash script to read a file in line by line, that is formatted later to be organised by name and then date. I cannot see why this code isn't working at this time though no errors show up even though I have tried with the input and output filename variables on their own, with a directory finder and export command.
export inputfilename="employees.txt"
export outputfilename="branch.txt"
directoryinput=$(find -name $inputfilename)
directoryoutput=$(find -name $outputfilename)
n=1
if [[ -f "$directoryinput" ]]; then
while read line; do
echo "$line"
n=$((n+1))
done < "$directoryoutput"
else
echo "Input file does not exist. Please create a employees.txt file"
fi
All help is very much appreciated, thank you!
NOTE: As people noticed, I forgot to add in the $ sign on the data transfer to file, but it was just in copying my code, I do have the $ sign in my actual application and still no result
Reading in File line by line w/ Bash
The best and idiomatic way to read file line by line is to do:
while IFS= read -r line; do
// parse line
printf "%s" "$line"
done < "file"
More on this topic can be found on bashfaq
However don't read files in bash line by line. You can (ok, almost) always not read a stream line by line in bash. Reading a file line by line in bash is extremely slow and shouldn't be done. For simple cases all the unix tools with the help of xargs or parallel can be used, for more complicated awk and datamesh are used.
done < "directoryoutput"
The code is not working, because you are passing to your while read loop as input to standard input the content of a file named directoryoutput. As such a file does not exists, your script fails.
directoryoutput=$(find -name $outputfilename)
One can simply append the variable value with newline appended to a read while loop using a HERE-string construction:
done <<< "$directoryoutput"
directoryinput=$(find -name $inputfilename)
if [[ -f "$directoryinput" ]]
This is ok as long as you have only one file named $inputfilename in your directory. Also it makes no sense to find a file and then check for it's existance. In case of more files, find return a newline separated list of names. However a small check if [ "$(printf "$directoryinput" | wc -l)" -eq 1 ] or using find -name $inputfilename | head -n1 I think would be better.
while read line;
do
echo "$line"
n=$((n+1))
done < "directoryoutput"
The intention is pretty clear here. This is just:
n=$(<directoryoutput wc -l)
cat "directoryoutput"
Except that while read line removed trailing and leading newlines and is IFS dependent.
Also always remember to quote your variables unless you have a reason not to.
Have a look at shellcheck which can find most common mistakes in scripts.
I would do it more like this:
inputfilename="employees.txt"
outputfilename="branch.txt"
directoryinput=$(find . -name "$inputfilename")
directoryinput_cnt=$(printf "%s\n" "$directoryinput" | wc -l)
if [ "$directoryinput_cnt" -eq 0 ]; then
echo "Input file does not exist. Please create a '$inputfilename' file" >&2
exit 1
elif [ "$directoryinput_cnt" -gt 1 ]; then
echo "Multiple file named '$inputfilename' exists in the current path" >&2
exit 1
fi
directoryoutput=$(find . -name "$outputfilename")
directoryoutput_cnt=$(printf "%s\n" "$directoryoutput" | wc -l)
if [ "$directoryoutput_cnt" -eq 0 ]; then
echo "Input file does not exist. Please create a '$outputfilename' file" >&2
exit 1
elif [ "$directoryoutput_cnt" -gt 1 ]; then
echo "Multiple file named '$outputfilename' exists in the current path" >&2
exit 1
fi
cat "$directoryoutput"
n=$(<"$directoryoutput" wc -l)
This part of my script is comparing each line of a file to find a preset string. If the string does NOT exist as a line in the file, it should append it to the end of the file.
STRING=foobar
cat "$FILE" | while read LINE
do
if [ "$STRING" == "$LINE" ]; then
export ISLINEINFILE="yes"
fi
done
if [ ! "$ISLINEINFILE" == yes ]; then
echo "$LINE" >> "$FILE"
fi
However, it appears as if both $LINE and $ISLINEINFILE are both cleared upon finishing the do loop. How can I avoid this?
Using shell
If we want to make just the minimal change to your code to get it working, all we need to do is switch the input redirection:
string=foobar
while read line
do
if [ "$string" == "$line" ]; then
islineinfile="yes"
fi
done <"$file"
if [ ! "$islineinfile" == yes ]; then
echo "$string" >> "$file"
fi
In the above, we changed cat "$file" | while do ...done to while do...done<"$file". With this one change, the while loop is no longer in a subshell and, consequently, shell variables created in the loop live on after the loop completes.
Using sed
I believe that the whole of your script can be replaced with:
sed -i.bak '/^foobar$/H; ${x;s/././;x;t; s/$/\nfoobar/}' file*
The above adds line foobar to the end of each file that doesn't already have a line that matches ^foobar$.
The above shows file* as the final argument to sed. This will apply the change to all files matching the glob. You could list specific files individually if you prefer.
The above was tested on GNU sed (linux). Minor modifications may be needed for BSD/OSX sed.
Using GNU awk (gawk)
awk -i inplace -v s="foobar" '$0==s{f=1} {print} ENDFILE{if (f==0) print s; f=0}' file*
Like the sed command, this can tackle multiple files all in one command.
Why does my variable set in a do loop disappear?
It disappears because it is set in a shell pipeline component. Most shells run each part of a pipeline in a subshell. By Unix design, variables set in a subshell cannot affect their parent or any already running other shell.
How can I avoid this?
There are several ways:
The simplest is to use a shell that doesn't run the last component of a pipeline in a subshell. This is ksh default behavior, e.g. use that shebang:
#!/bin/ksh
This behavior can also be bash one when the lastpipe option is set:
shopt -s lastpipe
You might use the variable in the same subshell that set it. Note that your original script indentation is wrong and might lead to the incorrect assumption that the if block is inside the pipeline, which isn't the case. Enclosing the whole block with parentheses will rectify that and would be the minimal change (two extra characters) to make it working:
STRING=foobar
cat "$FILE" | ( while read LINE
do
if [ "$STRING" == "$LINE" ]; then
export ISLINEINFILE="yes"
fi
done
if [ ! "$ISLINEINFILE" == yes ]; then
echo "$LINE" >> "$FILE"
fi
)
The variable would still be lost after that block though.
You might simply avoid the pipeline, which is straigthforward in your case, the cat being unnecessary:
STRING=foobar
while read LINE
do
if [ "$STRING" == "$LINE" ]; then
export ISLINEINFILE="yes"
fi
done < "$FILE"
if [ ! "$ISLINEINFILE" == yes ]; then
echo "$LINE" >> "$FILE"
fi
You might use another argorithmic approach, like using sed or gawk as suggested by John1024.
See also https://unix.stackexchange.com/a/144137/2594 for standard compliance details.
The output is blank fr the below script. What is it missing? I am trying to grep a string
#!/bin/ksh
file=$abc_def_APP_13.4.5.2
if grep -q abc_def_APP $file; then
echo "File Found"
else
echo "File not Found"
fi
In bash, use the <<< redirection from a string (a 'Here string'):
if grep -q abc_def_APP <<< $file
In other shells, you may need to use:
if echo $file | grep -q abc_def_APP
I put my then on the next line; if you want your then on the same line, then add ; then after what I wrote.
Note that this assignment:
file=$abc_def_APP_13.4.5.2
is pretty odd; it takes the value of an environment variable ${abc_def_APP_13} and adds .4.5.2 to the end (it must be an env var since we can see the start of the script). You probably intended to write:
file=abc_def_APP_13.4.5.2
In general, you should enclose references to variables holding file names in double quotes to avoid problems with spaces etc in the file names. It is not critical here, but good practices are good practices:
if grep -q abc_def_APP <<< "$file"
if echo "$file" | grep -q abc_def_APP
Yuck! Use the shell's string matching
if [[ "$file" == *abc_def_APP* ]]; then ...
The command:
value=${value%?}
will remove the last character from a variable.
Is there any logical reason why it would not work from within a script?
In my script it has no effect whatsoever.
if [[ $line =~ "What I want" ]]
then
if [[ $CURRENT -eq 3 ]]
then
echo "line is " $line
value=`echo "$line" | awk '{print $4}'`
echo "value = "$value
value=${value%?}
echo "value = $value "
break
fi
fi
I cant post the whole script, but this is the piece I refer to. The loop is being entered properly, but the 2 echo $value lines return the same thing.
Edit - this question still stands. The code works fine line bu line in a terminal, but all together in a script it fails.
Echo adds an extra line character to $value in this line:
value=`echo "$line" | awk '{print $4}'`
And afaik that extra char is removed with %?, so it seems it does not change anything at all.
Try echo -n instead, which does not add \n to the string:
value=`echo -n "$line" | awk '{print $4}'`
Since you have provided only the relevant part in the code and not the whole file, I'm going to assume that the first line of your file reads `#!/bin/sh'. This is your problem. What you are trying to do (parameter expansion) is specific to bash, so unless /bin/sh points to bash via a symlink, then you are running the script in a shell which does not understand bash parameter expansion.
To see what /bin/sh really is you can do: ls -l /bin/sh. And to remedy the situation, force the script to run in bash by changing the `shebang' at the top to read `#!/bin/bash'