How to identify/get the file by its timestamp in a batch file? - windows

I have a list of csv files with date and time appended like "Account_data_yyyymmdd.csv" which are added daily along with its timestamp to source dir .I have to identify latest file ie.'Account_data_2020_08_05.csv' and set the value in variable . so i can pass it as argument
Files in source dir
Account_data_2020_08_05.csv
Account_data_2020_08_04.csv
Account_data_2020_08_03.csv
I have to find the recently placed file based on its timestamp & pass it as input for calling another batch process. Highlighted text is the argument to batch file.How to find latest file based on its timestamp and pass it as argument for
echo "start"
call process.bat "C:\CSVDataLod" AccntDataloadprocess ***"dataAccess.name=C:\SourceDir\ Account_data_%year%_%month%_%date%.csv"***

That's surprisingly easy. Use dir with the /on switch to sort by name (see dir /? for that switch and the others I used, if you are not familiar with them) and put a for /f loop around to capture the output. The following code sets the variable %last% to each line of the output, keeping the last one only:
for /f "delims=" %%a in ('dir /a-d /on /b Account_data_*.csv') do set "last=%%a"
echo %last%

The easiest and fastest method to get name of CSV file with newest date in file name is using command DIR with option /O-N to get the CSV file names output ordered by name in reverse order. The file name with newest name is output first by DIR in this case. The output of DIR has to be captured and processed with FOR. The FOR loop is exited after running the other batch file with first file name output by DIR.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FileFound="
set "FileNamePattern=Account_data_20??_??_??.csv"
if /I "%~x1" == ".csv" set "FileNamePattern=%~nx1"
for /F "delims=" %%I in ('dir "C:\SourceDir\%FileNamePattern%" /A-D /B /O-N 2^>nul') do (
echo Processing file %%I ...
call process.bat "C:\CSVDataLod" AccntDataloadprocess "dataAccess.name=C:\SourceDir\%%I"
if /I not "%~1" == "/A" goto EndBatch
set "FileFound=1"
)
if not defined FileFound echo There is no file "%FileNamePattern%" in directory "C:\SourceDir".
:EndBatch
endlocal
I recommend to open a command prompt and run
dir "C:\SourceDir\Account_data_20??_??_??.csv" /A-D /B /O-N
Then you know which lines are processed by FOR. Next run
dir "C:\SourceDir\Account_data_20??_??_??.csv" /A-D /B
dir "C:\SourceDir\Account_data_20??_??_??.csv" /A-D /B /ON
to see how DIR outputs the CSV file names without specifying a specific order resulting in printing the file names as returned by the file system and explicitly ordered by name in alphabetical order instead of reversed alphabetical order.
The file system NTFS returns a list of file names matched by a wildcard pattern in local specific alphabetic order while FAT file systems like FAT16, FAT32, exFAT return the file names not ordered at all. In real all file systems return the file names in order as stored in the table of the file system. The file systems use just different methods on how to add a file name to table of the file system. The FAT file systems append a new file name always at end of the table of a directory while NTFS inserts a new file name in table of a directory using a local specific alphabetic sort algorithm.
Read the Microsoft documentation about Using command redirection operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background with %ComSpec% /c and the command line within ' appended as additional arguments.
Edit:
The batch file can be run with /a or /A as argument to process all CSV files matching the wildcard pattern from newest to oldest instead of just the newest. The batch file can be also run with name of a .csv file in source directory to process this specific CSV file instead of the newest CSV file.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
dir /?
echo /?
endlocal /?
for /?
goto /?
setlocal /?

Related

Windows cmd cd into newest directory

I have a deployment directory that contains subdirectories, one for each deployment. I'm trying to write a batch script that, among other things, performs a cd into the newest one of these directories.
I know how to do this in bash (has already been ansered here as well), but I don't know how to accomplish the same thing in Windows cmd. Can anyone help me?
In a batch file following lines can be used to changed to the subdirectory with newest modification date:
#echo off
for /F "eol=| delims=" %%I in ('dir * /AD /B /O-D 2^>nul') do cd "%%I" & goto DoneCD
echo No subdirectory found in: "%CD%"
:DoneCD
The command FOR with option /F starts a new command process with %ComSpec% /c and the command line specified between ' as further arguments in background. So executed by FOR is with usual Windows installation path:
C:\Windows\System32\cmd.exe /c dir * /AD /B /O-D 2>nul
DIR executed by background command process searches with the specified arguments
in current directory
for directories because of option /AD (attribute directory)
matching the wildcard pattern * (all)
and outputs
in bare format because of option /B just the directory names without path never enclosed in "
ordered reverse by last modification date because of option /O-D and not using option /TC (creation date) or /TA (last access date) which means first the newest modified directory and last the oldest modified directory.
The output by DIR is written to handle STDOUT of the started background command process.
2>nul redirects the error message output by DIR on not finding any directory in current directory from handle STDERR to device NUL to suppress this error message.
Read the Microsoft article about Using Command Redirection Operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background.
FOR captures everything written by DIR to handle STDOUT of started command process and processes this output line by line after started cmd.exe terminated itself.
FOR ignores empty lines which do not occur here because of DIR outputs the list of directory names without empty lines because of using /B.
FOR would split up by default a line into substrings (tokens) using normal space and horizontal tab character as delimiters. After this substring splitting is done FOR would by default check if the first substring starts with default end of line character ; in which case the line would be ignored like an empty line. Otherwise FOR would assign first space/tab delimited string to the specified loop variable I and would execute the command line with CD and GOTO.
A directory name could be for example  ;Test Folder, i.e. a directory name starting with a space and a semicolon and containing one more space. Such a directory name would be split up to ;Test (without space at beginning) and Folder and next ignored by FOR because of ;Test starts with a semicolon.
For that reason the end of line character is redefined from default semicolon to a vertical bar with eol=| which is a character no file or folder name can contain according to Microsoft documentation about Naming Files, Paths, and Namespaces. And line splitting behavior is disabled with delims= at end of options argument string after for /F which defines an empty list of delimiters. So the directory name as output by DIR is assigned to loop variable I without any modification even on being a very unusual name for a directory.
FOR executes command CD which changes current directory to the last modified subdirectory of the current directory and next command GOTO is executed to continue the processing of the batch file on the line below the label line :DoneCD. So the FOR loop execution is broken already after processing first directory name with command GOTO.
It is of course possible to use other commands after the FOR command line and the label line :DoneCD than just the ECHO line reporting that no subdirectory was found in current directory as shown by referencing dynamic environment variable CD like a command line to exit batch processing on this unusual use case or error condition case.
This FOR command line with the command GOTO to exit FOR loop after CD cannot be used in a Windows command prompt window. A solution for Windows command prompt window would be:
set "DoneCD=" & (#for /F "eol=| delims=" %I in ('dir * /AD /B /O-D 2^>nul') do #if not defined DoneCD cd "%I" & set "DoneCD=1") & set "DoneCD="
In a batch file this single line with multiple commands would be written as
#set "DoneCD=" & (#for /F "eol=| delims=" %%I in ('dir * /AD /B /O-D 2^>nul') do #if not defined DoneCD cd "%%I" & set "DoneCD=1") & set "DoneCD="
or better readable in its multi-line version with an additional echo as
#echo off
set "DoneCD="
for /F "eol=| delims=" %%I in ('dir * /AD /B /O-D 2^>nul') do (
if not defined DoneCD (
cd "%%I"
set "DoneCD=1"
)
)
if not defined DoneCD echo No subdirectory found in: "%CD%"
set "DoneCD="
First the environment variable DoneCD is deleted if it is defined by chance.
Next FOR runs cmd.exe with DIR as described above and processes the first output directory with newest modification date. The IF condition is true on newest directory as the environment variable was definitely undefined before execution of FOR. So command CD is executed to change the current directory to newest subdirectory. Then the environment variable DoneCD is defined with value 1. Any other value would be also possible like on using set "DoneCD=%%I". Important here is that for the other subdirectories output by DIR the environment variable DoneCD is now defined and so the IF condition is always false. So there is no attempt made to change in current subdirectory of initial current directory into a subdirectory not existing here or existing by chance also in the subdirectory.
Finally the environment variable DoneCD is deleted again if defined at all during execution of FOR.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
cd /?
dir /?
echo /?
for /?
goto /?
if /?
set /? ... explaining on last help page dynamic environment variable CD.

Backup script - find file by pattern, read it's full name and pass to script

I'm trying to use 7-Zip for backup purposes.
I have already wrote script for full backup:
#echo off
set source="c:\Source"
set destination="C:\Dest"
set dd=%DATE:~0,2%
set mm=%DATE:~3,2%
set yyyy=%DATE:~6,4%
set curdate=%dd%-%mm%-%yyyy%
"C:\Program Files\7-Zip\7z.exe" a -tzip -ssw -mx6 -r0 %destination%\Full_%curdate%.zip %source%
The new script intended for incremental backup is started after the full backup is made. But I don't really get how to make my second script to read files from directory and look for the file staring like full_xx_xx_xxxx.zip and assign its filename to a variable and then pass it to the script for incremental backup.
I tried script below, but it's not working:
#echo off
set source="c:\Source"
set destination="c:\Dest"
set exten="Full_*.zip"
set passwd="NAS"
set dd=%DATE:~0,2%
set mm=%DATE:~3,2%
set yyyy=%DATE:~6,4%
set curdate=%dd%-%mm%-%yyyy%
for %%a in %exten do echo %%a
"C:\Program Files\7-Zip\7z.exe" u -tzip -ssw -r0 %destination%\%%a.zip -u- -up0q3x2z0!"%destination%\diff_%date%.zip" %source%
There are multiple mistakes in both scripts.
I recommend reading first How to set environment variables with spaces? and Why is no string output with 'echo %var%' after using 'set var = text' on command line?. The syntax set variable="value in quotes" is often not good because it assigns the string "value in quotes" with the double quotes and all trailing spaces/tabs which might exist also in batch file to the environment variable with name variable. This syntax is problematic on concatenating the string value of the environment variable with other strings as done in posted code with %destination% because of the " being now somewhere in middle of the final argument string instead of enclosing the entire argument string. Better is the syntax set "variable=value without or with spaces" with " left to variable name because of the double quotes are interpreted now as argument string separators and perhaps existing spaces/tabs on line after second " are ignored by Windows command processor.
The usage of dynamic environment variable DATE makes it possible to quickly get current locale date in a format usable for file/folder names. But it must be taken into account that the date format of value of DATE depends on region/country/locale set for the user account which is used on running the batch file. I suppose that echo %DATE% results in an output of a date in format DD.MM.YYYY and so the command lines using DATE are correct for you with your user account according to the configured country.
The FOR command line is completely wrong and results in an exit of batch file execution with an error message output by cmd.exe interpreting the batch file line by line. This error output can be seen on running the batch file from within a command prompt window instead of double clicking on the batch file. See debugging a batch file for details on how to debug a batch file to find syntax errors like this reported by Windows command processor during execution of a batch file.
So I suggest for the first batch file:
#echo off
set "Source=C:\Source"
set "Destination=C:\Dest"
set "CurrentDate=%DATE:~6,4%-%DATE:~3,2%-%DATE:~0,2%"
"%ProgramFiles%\7-Zip\7z.exe" a -tzip -ssw -mx6 -r0 "%Destination%\Full_%CurrentDate%.zip" "%Source%"
The current locale date is assigned to the environment variable CurrentDate in format YYYY-MM-DD instead of DD-MM-YYYY. The date format YYYY-MM-DD is the international date format according to ISO 8601. It has one big advantage in comparison to all locale date formats in file names: The file names with date in format YYYY-MM-DD sorted alphabetically as usual are at the same time sorted chronological. That makes it much easier for people and scripts finding a specific file in a list of file names with date in file name.
I am not really sure what you want to do with the second batch file. So I can only suppose what you want to do and suggest for the second batch file:
#echo off
set "Source=C:\Source"
set "Destination=C:\Dest"
set "CurrentDate=%DATE:~6,4%-%DATE:~3,2%-%DATE:~0,2%"
set "NamePattern=Full_*.zip"
for /F "skip=1 eol=| delims=" %%I in ('dir "%Destination%\%NamePattern%" /A-D /B /O-N 2^>nul') do (
"%ProgramFiles%\7-Zip\7z.exe" u -tzip -ssw -r0 "%Destination%\%%I" -u- -up0q3x2z0!"%Destination%\Diff_%CurrentDate%.zip" "%Source%"
goto Done
)
:Done
The FOR loop runs command DIR with using a separate command process started in background to get the list of Full_*.zip file names in destination directory sorted reverse by name which means the full backup ZIP file created today before with first batch file is at top on using date format YYYY-MM-DD and the previously created ZIP file from yesterday (or whenever the last but one full ZIP file was created) is output as second line.
FOR skips the first line with ZIP file name with current date and runs 7-Zip with previously created ZIP file (yesterday) to create the difference ZIP file. Then the FOR loop is exited without processing all other full ZIP files with a jump to the label below the FOR loop.
Both batch files together:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "Source=C:\Source"
set "Destination=C:\Dest"
set "CurrentDate=%DATE:~6,4%-%DATE:~3,2%-%DATE:~0,2%"
set "NamePattern=Full_*.zip"
rem Create full ZIP backup.
"%ProgramFiles%\7-Zip\7z.exe" a -tzip -ssw -mx6 -r0 "%Destination%\Full_%CurrentDate%.zip" "%Source%"
rem Create difference ZIP backup with files added/changed in source directory
rem in comparison to the files compressed into last but on full ZIP backup.
for /F "skip=1 eol=| delims=" %%I in ('dir "%Destination%\%NamePattern%" /A-D /B /O-N 2^>nul') do (
"%ProgramFiles%\7-Zip\7z.exe" u -tzip -ssw -r0 "%Destination%\%%I" -u- -up0q3x2z0!"%Destination%\Diff_%CurrentDate%.zip" "%Source%"
goto Done
)
:Done
endlocal
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
dir /?
echo /?
endlocal /?
for /?
rem /?
set /?
setlocal /?

Execute all commands in current folder sequentially

Assuming I have a bunch of sqlcmd commands in .cmd files, order alphabetically e.g.:
01.setup.cmd
02.version1.cmd
03.version2.cmd
04.version3.cmd
how could one sequentially execute these in correct order with another .cmd file?
On windows:
for /F "tokens=*" %a in ('dir /b *.cmd') do call "%a"
This just loops over the result of dir /b *.cmd calling each in turn.
explanation from the docs:
FOR /F processing of a text file consists of reading the file, one
line of text at a time and then breaking the line up into individual
items of data called 'tokens'. The DO command is then executed with
the parameter(s) set to the token(s) found.
So my command says:
"tokens=*" don't give me individual tokens, give me the whole line as one hit
%a - name the line variable %a (note: it'll needs to be escaped as %%a if you're putting it in a batch file
('dir /b *.cmd') This is the input that it'll loop over. A bare directory listing for all .cmd files
then what I want it to do. Call the command %a.
If I didn't add the tokens bit it would work fine until you find a space in the file names.

batch script to generate a list with file names newer than a saved date timestamp

I have a landing directory where we receive 10-15 files everyday. I then run a batch script to create a list with file names that have newly landed and then informatica reads the list of file names to process the new source files.
The problem here is, if a file is edited on same day after informatica loads the file. My batch script does not identify the updated file because the file has same date and name.
Is there a way to compare files based on timestamp and generate a file list? Any help will be greatly appreciated. Thanks!
The current batch script code:
rem this batch script is used to list all the files hosted newly to a xyz landing directory
set LandingDir=\\path\to\landing\directory\*.csv
set DateFile=%sys_var%\script\DateFile.txt
set LastRunDateFile=%sys_var%\scripts\LastRunDateFile.txt
set Temp_File_List=%sys_var%\scripts\Temp_File_List.txt
set File_List=%sys_var%\SrcFiles\File_List.txt
set /P _PreviousDate=<%DateFile%
type %DateFile% > %LastRunDateFile%
xcopy "%LandingDir%" /l /s /d:%_PreviousDate% .>%Temp_File_List%
type %Temp_File_List% | findstr /v File(s)>%File_List%
echo %date:~4,2%-%date:~7,2%-%date:~10,4% >%DateFile%
On Windows there is the archive attribute always set on a file automatically if a file is modified in any way.
Using the archive file attribute makes the task much easier than storing last modification files times of all files processed and comparing last modification file times on next run.
All needed to be done is removing archive attribute on file being already processed, i.e. added to the file list.
Example:
#echo off
setlocal
set "FilePattern=*.csv"
set "sys_var=C:\Temp\Test"
set "File_List=%sys_var%\SrcFiles\File_List.txt"
set "LandingDir=\\server\share\path\to\landing\directory"
if exist "%File_List%" del "%File_List%"
for /F "delims=" %%I in ('dir "%LandingDir%\%FilePattern%" /AA-D /B 2^>nul') do (
echo %LandingDir%\%%I>>"%File_List%"
%SystemRoot%\System32\attrib.exe -a "%LandingDir%\%%I"
)
if not exist "%File_List%" echo No new file!
endlocal
The command DIR returns because of /AA-D just files (not directories) with archive attribute set in bare format because of /B.
So output by DIR and processed by FOR are just the names of the files with archive attribute set without path and always without surrounding double quotes even if the file name contains a space or another special character.
The file names would be returned by DIR with full path on using additionally DIR option /S for listing recursively all files in specified directory and in all subdirectories matching the file pattern (and having archive attribute set).
Each file name is written into the file list file and then the archive attribute is removed from the file to ignore this file automatically on next run of the batch file except the archive attribute is set again because the file was modified in the meantime.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
attrib /?
del /?
dir /?
echo /?
endlocal /?
for /?
set /?
setlocal /?
See also the Microsoft article Using command redirection operators for an explanation of >> and 2>nul with escaping > with ^ to be interpreted on execution of DIR instead of FOR.

Batch file for loop executes on one machine only

I have written the following .bat file, and it runs perfectly on my Windows 2000 machine, but will not run on my Windows 7 or Windows XP machines. Basically it just loops through the current directory and runs a checksum program which returns the checksum. The output of the program is saved to a text file and then formatted to remove the checksum of the output file.
#Echo Off
for /r %%f in (*.txt) do crc32sum.exe %%f >> all_checksums.txt
ren all_checksums.txt old.txt
findstr /v /e /c:"all_checksums.txt" old.txt > all_checksums.txt
del old.txt
When I run this file on my Win2k PC with a bunch of text files and the crc32sum.exe in a folder, it outputs the file. On other machines it outputs a blank file. I turned Echo on and kept only the for loop line and found that the output from executing the crc32sum.exe is nothing. If you manually run the crc32sum.exe file it outputs the checksum no problem.
Any ideas as to how to fix this?
EDIT: Here is a link to the software: http://www.di-mgt.com.au/src/digsum-1.0.1.zip
EDIT2: New development, it seems that the file works if the path of the folder has no spaces in it i.e. C:\temp or C:\inetpub\ftproot or C:\users\admin\Desktop\temp. Does anyone know how I can make this work with paths that have spaces? %%~f doesnt work it says unexpected.
Try this modified batch code which worked on Windows XP SP3 x86:
#echo off
goto CheckOutput
rem Command DEL does not terminate with an exit code greater 0
rem if the deletion of a file failed. Therefore the output to
rem stderr must be evaluated to find out if deletion was
rem successful or (for a single file) the file existence is
rem checked once again. For details read on Stack Overflow
rem the answer http://stackoverflow.com/a/33403497/3074564
rem The deletion of the file was successful if file created
rem from output message has size 0 and therefore the temp
rem file can be deleted and calculation of the CRC32 sums
rem can be started.
:DeleteOutput
del /F "all_checksums.txt" >nul 2>"%TEMP%\DelErrorMessage.tmp"
for %%E in ("%TEMP%\DelErrorMessage.tmp") do set "FileSize=%%~zE"
if "%FileSize%" == "0" (
set "FileSize="
del "%TEMP%\DelErrorMessage.tmp"
goto CalcCRC32
)
set "FileSize="
echo %~nx0: Failed to delete file %CD%\all_checksums.txt
echo.
type "%TEMP%\DelErrorMessage.tmp"
del "%TEMP%\DelErrorMessage.tmp"
echo.
echo Is this file opened in an application?
echo.
set "Retry=N"
set /P "Retry=Retry (N/Y)? "
if /I "%Retry%" == "Y" (
set "Retry="
cls
goto CheckOutput
)
set "Retry="
goto :EOF
:CheckOutput
if exist "all_checksums.txt" goto DeleteOutput
:CalcCRC32
for /R %%F in (*.txt) do (
if /I not "%%F" == "%CD%\all_checksums.txt" (
crc32sum.exe "%%F" >>"all_checksums.txt"
)
)
The output file in current directory is deleted if already existing from a previous run. Extra code is added to verify if deletion was successful and informing the user about a failed deletion with giving the user the possibility to retry after closing the file in an application if that is the reason why deletion failed.
The FOR command searches because of option /R recursive in current directory and all its subdirectories for files with extension txt. The name of each found file with full path always without double quotes is hold in loop variable F for any text file found in current directory or any subdirectory.
The CRC32 sum is calculated by 32-bit console application crc32sum in current directory for all text files found with the exception of the output file all_checksums.txt in current directory. The output of this small application is redirected into file all_checksums.txt with appending the single output line to this file.
It is necessary to enclose the file name with path in double quotes because even with no *.txt file containing a space character or one of the special characters &()[]{}^=;!'+,`~ in its name, the path of the file could contain a space or one of those characters.
For the files
C:\Temp\test 1.txt
C:\Temp\test 2.txt
C:\Temp\test_3.txt
C:\Temp\TEST\123-9.txt
C:\Temp\TEST\abc.txt
C:\Temp\TEST\hello.txt
C:\Temp\TEST\hellon.txt
C:\Temp\Test x\test4.txt
C:\Temp\Test x\test5.txt
the file C:\Temp\all_checksums.txt contains after batch execution:
f44271ac *test 1.txt
624cbdf9 *test 2.txt
7ce469eb *test_3.txt
cbf43926 *123-9.txt
352441c2 *abc.txt
0d4a1185 *hello.txt
38e6c41a *hellon.txt
1b4289fa *test4.txt
f44271ac *test5.txt
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
cls /?
del /?
echo /?
for /?
goto /?
if /?
rem /?
set /?
type /?
One of the help pages output on running for /? informs about %~I, %~fI, %~dI, %~pI, %~nI, %~xI, %~sI, %~aI, %~tI, %~zI.
Using in a batch file f (in lower case) as loop variable and referencing it with %%~f is a syntax error as command processor expects next the loop variable. %%~ff would be right, but could be different to %%~fI (name of a file/folder with full path and extension without quotes) in comparison to %%~I (string without surrounding quotes).
It is not advisable to use (those) small letters as loop variable. It is better to use upper case letters or character # as loop variable. The loop variable and also those modifiers are case sensitive while nearly everything else in a batch file is case insensitive.

Resources