Why is this loop breaking after the first iteration? - bash

Question

Trying to loop through all files in a directory, check them for the existence of a string, and add it if it doesn’ exist. This is what I have:
#!/bin/bash
FILES=*
for f in $FILES
do
echo "Processing $f file..."
if grep -Fxq '<?xml version="1.0" encoding="UTF-8"?>' $f
then
continue
else
echo '<?xml version="1.0" encoding="UTF-8"?>' | cat - $f > temp && mv temp $f
fi
done
… but the script stops after the first loop. Any ideas why?

Answer 1

A simpler solution would be to use the inplace edit option -i of the sed tool like below
sed -i '1{/^<?xml version="1.0" encoding="UTF-8"?>/!{
s/^/<?xml version="1.0" encoding="UTF-8"?>\n/}}' /path/to/files/*
What are we doing above
The inplace option -i with sed makes any change to the file written to the file.
By 1{} we are processing just the first line of the file
The /^<?xml version="1.0" encoding="UTF-8"?>/! part checks if the string is NOT(note the ! at the end) present in the beginning of the line.
If the above condition is not true we substitute the beginning of the line (^) with <?xml version="1.0" encoding="UTF-8"?>\n using
s/^/<?xml version="1.0" encoding="UTF-8"?>\n/
The rest is closing the curly brackets in the correct order :)
That said, in your original script I see variables like FILES. It is discouraged to use uppercase variables as user variables as they are reserved as environment variables and might lead to conflict. So use files instead.
Again doing
file=*
has the implication of [ word splitting ] and produce undesired results if you have non standard files that contain spaces or even new lines. What you could do is
files=( * ) # This put the files in an array
for file in "${files[#]}" # Double quoting the array prevents word splitting
do
# Do something with "$file" but why bother when you've a one-liner with sed? ;-)
done
Note: For sed manual visit [ here ]

Answer 2

I want to clear some things up about word splitting and filename expansion I saw here in the comments.
When using variable assignment, quoting Bash Reference Manual, only the following expansions are done: tilde expansion, parameter expansion, command substitution, arithmetic expansion. This means that there really is just an asterisk in your variable $files as there is no filename expansion taking place. So at this point you don't need to worry about newlines, spaces etc. because there are no actual files in your variable. You can see this with declare -p files.
This is the reason you don't have to quote when assigning to a variable.
var=$othervariable
is the same as:
var="$othervariable"
Now, when you use your variable $files in the for loop for f in $files (note that you cannot quote $files here because the filename expansion wouldn't take place) that variable gets expanded and undergoes word splitting. But the actual value is JUST the asterisk and word splitting won't do anything with the result! Quoting the manual again:
After word splitting, unless the -f option has been set (see The Set
Builtin), Bash scans each word for the characters ‘*’, ‘?’, and ‘[’.
If one of these characters appears, then the word is regarded as a
pattern, and replaced with an alphabetically sorted list of filenames
matching the pattern (see Pattern Matching).
Meaning of this is that filename expansion is done after variable expansion and word splitting. So the files expanded by the filename expansion won't be split by IFS! Therefore, the following code works just fine:
#!/usr/bin/env bash
files=*
for f in $files; do
echo "<<${f}>>"
done
and correctly outputs:
<<file with many spaces>>
<<filewith* weird characters[abc]>>
<<normalfile>>
A shorter version of this is obviously to use for f in * instead of the variable $files. You also definitely want to quote any usage of $f in your loop as that expansion really does undergo the word splitting.
This being said, your loop should function properly.