bash: Check if a file in sequence is missing - bash

Question

I have a bunch of files in a directory whose names contain numbers.
/mnt/exp-data/6/instrument/caen2018/stage0/S0Test_26060_500ns_CW_0ns_CBT_0ns_DEBT.root
/mnt/exp-data/6/instrument/caen2018/stage0/S0Run_26061_500ns_CW_0ns_CBT_0ns_DEBT.root
/mnt/exp-data/6/instrument/caen2018/stage0/S0Test_26063_500ns_CW_0ns_CBT_0ns_DEBT.root
/mnt/exp-data/6/instrument/caen2018/stage0/S0Run_26065_500ns_CW_0ns_CBT_0ns_DEBT.root
What I'd like to do is find which files are missing and then do something with those. In the above case the files which contains numbers 26062, 26064 are missing.
So far I'm doing the following
#___________________________________________________________________________________________________________________________
#-3-Find the missing runs
REPLAYED_RUNS_DIR=/ /mnt/exp-data/6/instrument/caen2018/stage0
echo "..........Looking for non replayed runs in the range $smallest_run-$biggest_run"
for (( i=$smallest_run; i<=$biggest_run; ++i));do
filename="$REPLAYED_DATA_DIR/*$i*.root"
#echo $filename
if [ ! -f $filename ]; then
echo "Run $i does not exist."
./produce_file $i
fi
done
This snippet manages to find files that are missing, however I have a few issues:
In some cases I get the following error for files that do exist and I have no idea why.
./check_missing.sh: line 53: [: /mnt/exp-data/6/instrument/caen2018/stage0/S0Run_25829_500ns_CW_0ns_CBT_0ns_DEBT.root: binary operator expected
If I uncomment echo filename the I get as an output the full name and directory of the files, as if I was doing ls instead of echo`. Is this to be expected?
Is there a better way (i,e, faster, more efficient) to do what I'm trying to do?

Answer 1

Here is a script to do this in a more massive way.
#!/bin/bash
d="path/to/directory"
start=$1
end=$2
join -v1 <(
seq "$start" "$end"
) <(
find "$d" -type f -printf "%f\0" |
awk -F"/" -v RS="\0" '{split($NF,a,"_"); print a[2]}' | sort
) | xargs -r -n1 echo ./produce_file
join -v1 file1 file2 will output all lines of file1 not in file2. In place of those two files, using process substitution, we put the sequence to be tested, and the filenames by find, filtered by awk to get the number in them and finally sorted, because join wants sorted inputs.
Finally you can pipe the result to your script produce_file. -r stands for --no-run-if-empty which is a GNU extension to avoid one execution with empty input if the previous command gave no arguments.
Remove echo after testing. If your script can process multiple number arguments, remove -n1 also, to process all of them at once.
Testing:
> mkdir -p path/to/directory
touch path/to/directory/S0Test_26060_0ns_CBT_0ns_DEBT.root
touch path/to/directory/S0Run_26061_500ns.root
touch path/to/directory/S0Test_26063_500ns_CW.root
touch path/to/directory/S0Test_26065_500ns.root
touch path/to/directory/S0Test_30000_500ns.root
> bash test.sh 26060 26065
./produce_file 26062
./produce_file 26064

Answer 2

I found another way to do it using find and [ -z "$filename"] to check if find returned an empty entry.
for (( i=$start; i<=$endn; ++i ));do
filename=`find "$DIR" -type f -name "*$i*"`
if [ -z "$filename" ]; then
echo "File $i does not exist."
fi
done