Removing spec characters with sed and mv but keeping / for destination, using full path - bash

Question

I am trying to remove special characters from specific files in files.txt. I need the mv command to use the full path to write the corrected file to the same location. The source and destination directories both contain spaces.
files.txt
/home/user/scratch/test2 2/capital:lets?.log
/home/user/scratch/test2 2/:31apples.tif
/home/user/scratch/test2 2/??testdoc1.txt
script.sh
#!/bin/bash
set -x
while IFS="" read -r p || [ -n "$p" ]
do
printf '%s\n' "$p"
mv "$p" $(echo "$p" | sed -e 's#[^A-Za-z0-9._-/]#_#g')
done < /home/user/scratch/files.txt
Here is the error that I get:
+ IFS=
+ read -r p
+ printf '%s\n' '/home/user/scratch/test2 2/??testdoc1.txt'
/home/user/scratch/test2 2/??testdoc1.txt
++ sed -e 's#[^A-Za-z0-9._-/]#_#g'
sed: -e expression #1, char 22: Invalid range end
++ echo '/home/user/scratch/test2 2/??testdoc1.txt'
+ mv '/home/user/scratch/test2 2/??testdoc1.txt'
mv: missing destination file operand after '/home/user/scratch/test2 2/??testdoc1.txt'
If I remove the / from sed -e 's#[^A-Za-z0-9._-]#_#g' command it will try to write the file like this:
++ sed -e 's#[^A-Za-z0-9._-]#_#g'
++ echo '/home/user/scratch/test2 2/??testdoc1.txt'
+ mv '/home/user/scratch/test2 2/??testdoc1.txt' _home_user_scratch_test2_2___testdoc1.txt
I have tried changing the delimiter in sed to something other than a / but the issue persists. If I try using mv "$p" "$(echo "$p" | sed -e 's|/[^/]*/\{0,1\}$||;s|^$|/|')" mv errors with this is the same file.
Am I approaching this problem wrong? This feels like it should have been an easier task.
EDIT:
The solution below gives me an issue with the file itself:
' echo '/mnt/data/bucket/Desktop/For_the_New_Director/Part Number Assignment/__Prod_Development/.Memeo 40'\'' flat w:boat plane.xls.plist
/mnt/data/bucket//Desktop/For_the_New_Director/Part Number Assignment/__Prod_Development/.Memeo 40' flat w:boat plane.xls.plist
+ dir='/mnt/data/bucket/Desktop/For_the_New_Director/Part Number Assignment/__Prod_Development'
= */* ]]/data/bucket/Desktop/For_the_New_Director/Part Number Assignment/__Prod_Development/.Memeo 40' flat w:boat plane.xls.plist
' file='.Memeo 40'\'' flat w:boat plane.xls.plist
+ echo .Memeo '40'\''' flat w:boat $'plane.xls.plist\r'
.Memeo 40' flat w:boat plane.xls.plist
+ echo /mnt/data/bucket/Desktop/For_the_New_Director/Part Number Assignment/__Prod_Development
/mnt/data/bucket/Desktop/For_the_New_Director/Part Number Assignment/__Prod_Development
The actual filename is: .Memeo 40' flat w:boat plane.xls.plist
Why is it changing the filename when trying to do the move?

Answer 1

There are two problems in your substitution:
In the character class description [^A-Za-z0-9._-/], the last part
_-/ is interpreted as a range of characters between _ and /,
which is invalid. To avoid this, you need to escape the hyphen character
with a backslash, or put the hyphen at the beginning or the end of the
character class.
The directory name test2 2 includes the special character and
the sed command converts the directory name into test2_2,
which does not exist. Assuming you want to change the filenames only
keeping the directory names as is, we need to process the directory names
and filenames separately.
Then would you please try the following:
set -x
while IFS= read -r p || [ -n "$p" ]; do
echo "$p"
dir=${p%/*} # extract directory name
[[ $p = */* ]] || dir="." # in case $p does not contain "/"
file=${p##*/} # extract filename
mv -- "$p" "$dir/${file//[^-A-Za-z0-9._]/_}"
done < /home/user/scratch/files.txt