Are delete events considered close_write in inotifywait? - bash

Question

I have a simple inotifywait script that watches for FTP file uploads to be closed and then moving them to a aws s3. It seems to be working except that in the inotify logs, it indicates that the file was not found ( although the file was indeed uploaded to s3 ).
The s3 move command moves the file to the cloud and deletes it locally. Could this be because inotifywait detects deleting the file as a close_write event ?
Why is inotify seems to be executing the commands twice ?
TARGET=/home/*/ftp/files
inotifywait -m -r -e close_write $TARGET |
while read directory action file
do
if [[ "$file" =~ .*mp4$ ]]
then
echo COPY PATH IS "$directory$file"
aws s3 mv "$directory$file" s3://bucket
fi
done
example logs:
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
COPY PATH IS /home/user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4
COPY PATH IS /home/user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4
COPY PATH IS /home/user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4
move: ../user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4 to s3://bucket/user-cam-1_00_20220516114055.mp4
upload: ../user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4 to s3://bucket/user-cam-1_00_20220516114055.mp4
move failed: ../user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4 to s3://bucket/user-cam-1_00_20220516114055.mp4 [Errno 2] No such file or directory: '/home/user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4'
rm: cannot remove '/home/user/ftp/files/2022/05/16/user-cam-1_00_20220516114055.mp4': No such file or directory

Answer 1

Cleaned-up your script and added some safety with quotes and check for already processed file in case the filesystem triggers duplicate events for same file.
#!/usr/bin/env bash
# Prevents expanding pattern without matches
shopt -s nullglob
# Expands pattern into an array
target=(/home/*/ftp/files/)
# Creates temporary directory and cleanup trap
declare -- tmpdir=
if tmpdir=$(mktemp -d); then
trap 'rm -fr -- "$tmpdir"' EXIT INT
else
# or exit error if it fails
exit 1
fi
# In case no target matches, exit error
[ "${#target[#]}" -gt 0 ] || exit 1
s3move() {
local -- p=$1
local -- tmp="$tmpdir/$p"
printf 'Copy path is: %s\n' "$p"
# Moves the file to temporary dir
# so it is away from inotify watch dir ASAP
mv -- "$p" "$tmp"
# Then perform the slow remote copy to s3 bucket
# Remove the echo onces it is ok
echo aws s3 mv "$p" s3://bucket
# File has been copied to s3, tmp file no longer needed
rm -f -- "$tmp"
}
while read -r -d '' p; do
# Skip if file does not exist, as it has already been moved away
# case of a duplicate event for already processed file
[ -e "$p" ] || continue
s3move "$p"
done < <(
# Good practice to spell long option names in a script
# --format will print null-delimited full file path
inotifywait \
--monitor \
--recursive \
--event close_write \
--includei '.*\.mp4$' \
--format '%w%f%0' \
"${target[#]}" 2>/dev/null
)