Multi-variable string not being returned via SFTP get command - shell

Question

I'm working on a ksh script to retrieve a file every hour via sftp that will be put on a job scheduler to be run hourly. The script must navigate to a folder with yesterday's date (remote/path/yyyymmdd/). The filename also has yesterday's date and a timestamp (filename_yyyymmdd_hhmmss.dat). Since the job will be scheduled, my script has to include the previous hour - ex. if the job runs at 11:02, the file to retrieve would be filename_yyyymmdd_10mmss.dat. The minutes and seconds will always be the same - ex 4949. There will be multiple files in the remote directory and I only want to retrieve the latest one so that there are not multiple input files being processed by our jobs. The remote directory will also have other files being created regularly, so I can't retrieve just the last modified files.
I have variables to return yesterday's date and the previous hour, but the sftp command isn't returning the full filename and isn't retrieving the file. I've tried concatenating the variables, using brackets & quotes & parenthesis, assigning multiple variables to a single variable, and exporting the variables.
vdate=$(TZ=bb24 date '+%Y%m%d')
vhour=$(date '+%H')
prevhour=$((vhour - 1))
sftp user#host << EOF
lcd /my/dir/
cd /remote/path/$vdate/
get filename_$vdate_$prevhour*.dat
bye
EOF
exit
When running the script, the file cannot be found and the full filename isn't
returned:
File "/remote/path/20190411/filename_20190411" not found.
instead of
File "/remote/path/20190411/filename_20190411_10*.dat" not found.
Every combination of variables that I try returns the same not found - ending after filename_$vdate.
I've tried some other combinations but always get the same not found:
newvar=${vdate}_${prevhour}
get filename_$newvar*.dat
and
newvar=${vdate}\\_${prevhour}
get filename_$newvar*.dat
File "/remote/path/20190411/filename_20190411" not found.

Answer 1

You have a problem in your script at prevhour=$((vhour - 1))
this way a text 02 after you make subtraction, it will be 1 and not 01 and it will match to undesired files, or even none as 00 - 1 is -1
[edvin]$ vdate=$(TZ=bb24 date '+%Y%m%d')
[edvin]$ vhour=$(date '+%H')
[edvin]$ prevhour=$((vhour - 1))
[edvin]$ echo $vhour
03
[edvin]$ echo $prevhour
2
[edvin]$ prevhour=$(date -d '1 hour ago' '+%H')
[edvin]$ echo $prevhour
02
date's -d option not avaliable on some system.
I believe in that in your attempt the shell is considered the * as part of the variable prevhour as you did not put it into {} that separate variables from sorrunding text.
This is my working solution based by your attempt:
#!/bin/ksh
r_host='server2'
r_user='edvin'
l_dir='./content'
r_dir='./Test_folder'
# this still not cover the case of midnight
# it put 00 to 23 but day have to be yesterday as well
##vdate=$(TZ=bb24 date '+%Y%m%d')
##vhour=$(date '+%H') # not used
##prevhour=$(date -d '1 hour ago' '+%H')
# vtime = YYYYmmdd_HH -1 H
vtime=$(TZ=bb24 date -d '1 hour ago' '+%Y%m%d_%H')
sftp ${r_user}#${r_host} << EOF
lcd ${l_dir}
cd ${r_dir}
get filename_${vtime}*.dat
bye
EOF
exit
Output:
[edvin]$ ./script.ksh
Connected to server2.
sftp> lcd ./content
sftp> cd ./Test_folder
sftp> get filename_20190415_02*.dat
Fetching /home/edvin/Test_folder/filename_20190415_020000.dat to filename_20190415_020000.dat
Fetching /home/edvin/Test_folder/filename_20190415_020100.dat to filename_20190415_020100.dat
Fetching /home/edvin/Test_folder/filename_20190415_020200.dat to filename_20190415_020200.dat
Fetching /home/edvin/Test_folder/filename_20190415_020300.dat to filename_20190415_020300.dat
Fetching /home/edvin/Test_folder/filename_20190415_020400.dat to filename_20190415_020400.dat
Fetching /home/edvin/Test_folder/filename_20190415_020500.dat to filename_20190415_020500.dat
Fetching /home/edvin/Test_folder/filename_20190415_020600.dat to filename_20190415_020600.dat
Fetching /home/edvin/Test_folder/filename_20190415_020700.dat to filename_20190415_020700.dat
Fetching /home/edvin/Test_folder/filename_20190415_020800.dat to filename_20190415_020800.dat
Fetching /home/edvin/Test_folder/filename_20190415_020900.dat to filename_20190415_020900.dat
Fetching /home/edvin/Test_folder/filename_20190415_021000.dat to filename_20190415_021000.dat
sftp> bye
There is many thing can go wrong still in this solution,
like if remote directory not exist, not accessible, script will still go on with the rest of the command, same for the local directory and for the files as well. The connection also can run various problems you might want to handle. You like to schedule it so might a solution needed to avoid script spawn over and over again if one already run.
scp would be more preferred way to do this, as you use password less authentication.
If scp is not an option for some reason, with expect this can be handled quite well.