Bash - Concatenate files in a directory ordered by date - bash

Question

i need some help with a simple script i m writing. The script takes as input a directory that contains files in the likes of :
FILENAME20160220.TXT
FILENAME20160221.TXT
FILENAME20160222.TXT
...
The script needs to have the directory as input, concatenate them into a new file called :
FILENAME.20160220_20160222.TXT
The above filenames need to have the "Earliest"_"Latest" date it finds. The script i ve written so far is this, but it doesnt produce the necessary output. Can someone help me tinker with it?
declare FILELISTING="FILELISTING.TXT"
declare SOURCEFOLDER="/Cat_test/cat_test/"
declare TEMPFOLDER="/Cat_Test/cat_test/temp/"
# Create temporary folder
cd $SOURCEFOLDER
mkdir $TEMPFOLDER
chk_abnd $?
# Move files into temporary folder
mv *.TXT $SOURCEFOLDER $TEMPFOLDER
chk_abnd $?
# Change directory to temporary folder
cd $TEMPFOLDER
chk_abnd $?
# Iterate through files in temp folder and create temporary listing files
for FILE in $TEMPFOLDER
do
echo $FILE >> $FILELISTING
done
# Iterate through the lines of FILELISTING and store dates into array for sorting
while read lines
do
array[$i] = "${$line:x:y}"
(( i++ ))
done <$FILELISTING
# Sort dates in array
for ((i = 0; i < $n ; i++ ))
do
for ((j = $i; j < $n; j++ ))
do
if [ $array[$i] -gt $array[$j] ]
then
t=${array[i]}
array[$i]=${array[$j]}
array[$j]=$t
fi
done
done
# Get first and last date of array and construct output filename
OT_FILE=FILENAME.${array[1]}_${array[-1]}.txt
# Sort files in folder
# Cat files into one
cat *.ACCT > "$OT_FILE.temp"
chk_abnd $?
# Remove Hex 1A
# tr '\x1A' '' < "$OT_FILE.temp" > $OT_FILE
# Cleanup - Remove File Listing
rm $FILE_LISTING
chk_abnd $?
rm $OT_FILE.temp
chk_abnd $?

Answer 1

Assuming that the base list of your files can be identified using FILENAME*.TXT which is nice and simple, ls can be used to generate an ordered list which will by default be ordered ascending alphabetically and thus (because of the date format you've chosen) in date ascending order.
You can get the earliest and lateest dates as follows:
$ earliest=$( ls -1 FILENAME*.TXT | head -1 | cut -c9-16 )
$ echo $earliest
20160220
$ latest=$( ls -1 FILENAME*.TXT | tail -1 | cut -c9-16 )
$ echo $latest
20160222
Therefore your file name can be produced using:
filename="FILENAME.${earliest}_${latest}.TXT"
And the concatenation should be as simple as:
cat $( ls -1 FILENAME*.TXT ) > ${filename}
though if you are writing to the same directory, you may wish to direct the output first to a temporary name that doesn't meet this pattern and then rename it. Perhaps something like:
earliest=$( ls -1 FILENAME*.TXT | head -1 | cut -c9-16 )
latest=$( ls -1 FILENAME*.TXT | tail -1 | cut -c9-16 )
filename="FILENAME.${earliest}_${latest}.TXT"
cat $( ls -1 FILENAME*.TXT ) > temp_${filename}
mv temp_${filename} ${filename}

Answer 2

Here are some hints, cat does most of the work.
If your filenames have fixed size date fields, as in your example, lexical sorting is enough.
ls -1 FILENAME* > allfiles
aggname=$(cat allfiles | sed -rn '1s/([^0-9]*)/\1./p;$s/[^0-9]*//p' |
paste -sd-)
cat allfiles | xargs cat > $aggname
you can combine the last two steps into one, but more readable this way.
don't reinvent the wheel.