Bash removes underscores from my filenames? - bash

Question

I'm trying to move files from one S3 bucket to another and put them into a folder structure by date. Put simply all the files at the moment are going into one folder and that folder has over 500,000 files inside of and I now need to sort through all these files and put them into folders by month.
The file names are similar to:
"This_is_a_file_20150403.xml"
So I loop through all the files within an S3 bucket, tokensize and get the date. I create a yearmonth variable ignoring the day and move them into another S3 bucket. But the filename changes to:
"This is a file 20150403.xml"
So when I try to move it, AWS can't find the file. Why has bash removed the underscores from the filename? I tried temporarily storing the filename in tempFilename but it still had the underscores removed.
The code I have at the moment is:
#!/bin/bash
count=0
for filename in $(aws s3 ls s3://stagingbucket)
do
echo $filename
tempFilename=$filename
(IFS='_'; for word in $filename;
do
echo $filename
if [ "$count" -eq 2 ]; then
yearmonth=${word:0:6}
echo $tempFilename
aws s3 cp s3://stagingbucket/$filename s3://archivebucket/$yearmonth/
fi
count=$((count + 1))
done)
done
Any ideas?

Answer 1

Let's look at what your code actually does.
echo $foo
String-splits $foo, breaking it into pieces based on characters in IFS
Evaluates each piece as a glob expression
Passes each result of those glob expressions to echo as a separate argument
echo then prints those arguments with spaces between them.
Instead, use:
echo "$foo"
...to keep your string together -- and, likewise, quote all your other expansions as well.
For much the same reasons (unintended glob expressions), for word in $filename is evil; don't do that.
IFS=_ read -a words <<<"$filename"
for word in "${words[#]}"; do
echo "Processing $word"
done