I have a directory with files and a ControlFile.txt that contains the a list of SHA256 sums for various files. I'm trying to come up with a Batch process to loop through the files in the directory, calculate the SHA256 value of each file, and then compare whether or not the calculated SHA256 exists or not in the ControlFile.txt and branch accordingly.
I've attempted to produce a working script with the following but I believe I'm missing some key elements:
for /R . %%f in (*.*) do (
find /c "(certutil -hashfile "%%f" SHA256 | findstr /V "hash")" ControlFile.txt > NUL
if %errorlevel% equ 1 goto notfound
echo "%%f" found
goto done
:notfound
echo "%%f" notfound
goto done
:done)
I believe I may need to set a variable for the given SHA256 value and use that within the loop to produce the comparative function I'm trying to achieve, but my knowledge with batch files and cmd is limited. Any code advice would be greatly appreciated.
#ECHO OFF
SETLOCAL
rem The following settings for the source directory and filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "filename1=%sourcedir%\q74148620.txt"
FOR /f "delims=" %%b IN ('dir /s /b /a-d "u:\j*" ') DO (
FOR /f %%y IN ('certutil -hashfile "%%b" SHA256 ^| find /V ":"') do (
findstr /x "%%y" "%filename1%" > NUL
IF ERRORLEVEL 1 (
ECHO "%%b" NOT found
) ELSE (
ECHO "%%b" found
)
)
)
GOTO :EOF
I used a filemask of j* for testing - change to suit.
Simply run the certutil routine on each file in turn, filter out any lines that contain :, leaving the SHA256 data. Locate that value as /x an exact mach to a line in the SHA256 values file. If a match is found, errorlevel is set to 0, non-0 otherwise, then switch on errorlevel.
=== minor revision to not include subdirectories ===
#ECHO OFF
SETLOCAL
rem The following settings for the source directory and filename are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "filename1=%sourcedir%\q74148620.txt"
PUSHD "%sourcedir%"
FOR /f "delims=" %%b IN ('dir /b /a-d') DO (
FOR /f %%y IN ('certutil -hashfile "%%b" SHA256 ^| find /V ":"') do (
findstr /x "%%y" "%filename1%" > NUL
IF ERRORLEVEL 1 (
ECHO "%%b" NOT found
) ELSE (
ECHO "%%b" found
)
)
)
POPD
GOTO :EOF
The dir options /b shows the names only (not size, date, etc.), /s would scan subdirectories and also generate the filenames with their full path, and /a-d suppresses directory names. u:\j* is the start location for the listing; drive u:, all files that begin j (that was for testing).
The pushd command makes the nominated directory current, so the revised dir command will scan only that directory for all filenames but no directorynames (since no starting directory and filemask is provided).
The popd command returns to the original directory.
Related
I have this filename which gets dropped into a directory on a windows share, where it needs to be located into the right location on the users' local machines, depending on if the filename has one of the keywords in it or not.
For instance:
z:\mailbox\in\Very_but#very-very!long#filenameWITH-keyword+in^it
where z: is accessible from a set of win-7 or win-10 machines. And on these machines, there are these two directories:
c:\incoming\special
c:\incoming\regular
if the filename has one of the keywords embedded in it, it needs to be copied into the c:\incoming\special folder, if not it needs to go into c:\incoming\regular folder.
my keywords are in a file, say, c:\keywords.txt
at any given time (checked every 5 minutes) there may be no files, or only one file in the z:\mailbox\in directory.
so, this is what I come up for, which is not working: (the batch file assumes there is a file to process, I haven't figured what to do, if there isn't one, yet)
dir /b z:\mailbox\in > tmp.out
set /p file=<tmp.out
del tmp.out
set keyword_found_flag=0
for /F "tokens=*" %%keyword in (c:\keywords.txt) do (
echo %file% | find /i "%%keyword"
if errorlevel=0 set keyword_found_flag=1
)
The errorlevel is always zero regardless if the keyword is found or not
IF keyword_found_flag=1 (
copy z:\mailbox\in\%file% c:\incoming\special
) ELSE (
copy z:\mailbox\in\%file% c:\incoming\regular
)
I am not sure what to do here. Any help is appreciated
Why not use operator (&&) to set your flag if execution is match?
echo %file% | find /i "%%keyword" >nul && set keyword_found_flag=1
You also don't need create/delete tmp.out file for your process, use double for to for the file name in z:\mailbox\in and, and use another for to check file contains in c:\keywords.txt with file names in z:\mailbox\in...
#echo off
set "keyword_found_flag=" && for /f tokens^=* %%i in ('dir /on /b /a:-d "z:\mailbox\in\*.*"
')do for /F tokens^=* %%K in ('type c:\keywords.txt')do echo="%%~i"| find /i "%%~K" && (
set "keyword_found_flag=1") || (set "keyword_found_flag=0")
Or...
#echo off
set "keyword_found_flag=" && for /f tokens^=* %%i in ('dir /on /b /a: -d "z:\mailbox\in\*.*"
')do for /F tokens^=* %%K in ('type c:\keywords.txt')do echo="%%~i"|find /i "%%~K" >nul && (
set "keyword_found_flag=1" && copy /v "%%~dpnxi" "c:\incoming\special\" ) || (
set "keyword_found_flag=0" && copy /v "%%~dpnxi" "c:\incoming\regular\" )
I want to create a .bat script to copy only one random file from each folder (also subfolders, so recursively) whilst also keeping the folder structure. I've tried the following code which comes close to what I want but doesn't copy the folder structure and one file per folder.
#ECHO OFF
SETLOCAL EnableExtensions EnableDelayedExpansion
SET Destination=H:\Temp
SET FileFilter=.ape
SET SubDirectories=/S
SET Source=%~dp1
SET FileList1Name=FileList1.%RANDOM%.txt
SET FileList1="%TEMP%\%FileList1Name%"
SET FileList2="%TEMP%\FileList2.%RANDOM%.txt"
ECHO Source: %Source%
IF /I {%SubDirectories%}=={/S} ECHO + Sub-Directories
IF NOT {"%FileFilter%"}=={""} ECHO File Filter: %FileFilter%
ECHO.
ECHO Destination: %Destination%
ECHO.
ECHO.
ECHO Building file list...
CD /D "%Source%"
DIR %FileFilter% /A:-D-H-S /B %SubDirectories% > %FileList1%
FOR /F "tokens=1,2,3 delims=:" %%A IN ('FIND /C ":" %FileList1%') DO SET TotalFiles=%%C
SET TotalFiles=%TotalFiles:~1%
ECHO The source has %TotalFiles% total files.
ECHO Enter the number of random files to copy to the destination.
SET /P FilesToCopy=
ECHO.
IF /I %TotalFiles% LSS %FilesToCopy% SET %FilesToCopy%=%TotalFiles%
SET Destination="%Destination%"
IF NOT EXIST %Destination% MKDIR %Destination%
SET ProgressTitle=Copying Random Files...
FOR /L %%A IN (1,1,%FilesToCopy%) DO (
TITLE %ProgressTitle% %%A / %FilesToCopy%
REM Pick a random file.
SET /A RandomLine=!RANDOM! %% !TotalFiles!
REM Go to the random file's line.
SET Line=0
FOR /F "usebackq tokens=*" %%F IN (%FileList1%) DO (
IF !Line!==!RandomLine! (
REM Found the line. Copy the file to the destination.
XCOPY /V /Y "%%F" %Destination%
) ELSE (
REM Not the random file, build the new list without this file included.
ECHO %%F>> %FileList2%
)
SET /A Line=!Line! + 1
)
SET /A TotalFiles=!TotalFiles! - 1
REM Update the master file list with the new list without the last file.
DEL /F /Q %FileList1%
RENAME %FileList2% %FileList1Name%
)
IF EXIST %FileList1% DEL /F /Q %FileList1%
IF EXIST %FileList2% DEL /F /Q %FileList2%
ENDLOCAL
The destination should be set in the .bat code like the code above. Can anybody please help me with this? Thanks in advance!
Copying a directory tree structure (folders only) is trivial with XCOPY.
Selecting a random file from a given folder is not too difficult. First you need the count of files, using DIR /B to list them and FIND /C to count them. Then use the modulo operator to select a random number in the range. Finally use DIR /B to list them again, FINDSTR /N to number them, and another FINDSTR to select the Nth file.
Perhaps the trickiest bit is dealing with relative paths. FOR /R can walk a directory tree, but it provides a full absolute path, which is great for the source, but doesn't do any good when trying to specify the destination.
There are a few things you could do. You can get the string length of the root source path, and then use substring operations to derive the relative path. See How do you get the string length in a batch file? for methods to compute string length.
Another option is to use FORFILES to walk the source tree and get relative paths directly, but it is extremely slow.
But perhaps the simplest solution is to map unused drive letters to the root of your source and destination folders. This enables you to use the absolute paths directly (after removing the drive letter). This is the option I chose. The only negative aspect of this solution is you must know two unused drive letters for your system, so the script cannot be simply copied from one system to another. I suppose you could programatically
discover unused drive letters, but I didn't bother.
Note: It is critical that the source tree does not contain the destination
#echo off
setlocal
:: Define source and destination
set "source=c:\mySource"
set "destination=c:\test2\myDestination"
:: Replicate empty directory structure
xcopy /s /t /e /i "%source%" "%destination%"
:: Map unused drive letters to source and destination. Change letters as needed
subst y: "%source%"
subst z: "%destination%"
:: Walk the source tree, calling :processFolder for each directory.
for /r y:\ %%D in (.) do call :processFolder "%%~fD"
:: Cleanup and exit
subst y: /d
subst z: /d
exit /b
:processFolder
:: Count the files
for /f %%N in ('dir /a-d /b %1 2^>nul^|find /c /v ""') do set "cnt=%%N"
:: Nothing to do if folder is empty
if %cnt% equ 0 exit /b
:: Select a random number within the range
set /a N=%random% %% cnt + 1
:: copy the Nth file
for /f "delims=: tokens=2" %%F in (
'dir /a-d /b %1^|findstr /n .^|findstr "^%N%:"'
) do copy "%%D\%%F" "z:%%~pnxD" >nul
exit /b
EDIT
I fixed an obscure bug in the above code. The original COPY line read as follows:
copy "%%~1\%%F" "z:%%~pnx1" >nul
That version fails if any of the folders within the source tree contain %D or %F in their name. This type of problem always exists within a FOR loop if you expand a variable with %var% or expand a :subroutine parameter with %1.
The problem is easily fixed by using %%D instead of %1. It is counter-intuitive, but FOR variables are global in scope as long as any FOR loop is currently active. The %%D is inaccessible throughout most of the :processFolder routine, but it is available within the FOR loops.
The "natural" way to process a directory tree is via a recursive subroutine; this method minimize the problems inherent to this process. As I said at this post: "You may write a recursive algorithm in Batch that gives you exact control of what you do in every nested subdirectory". I taken the code at this answer, that duplicate a tree, and slightly modified it in order to solve this problem.
#echo off
setlocal
set "Destination=H:\Temp"
set "FileFilter=*.ape"
rem Enter to source folder and process it
cd /D "%~dp1"
call :processFolder
goto :EOF
:processFolder
setlocal EnableDelayedExpansion
rem For each folder in this level
for /D %%a in (*) do (
rem Enter into it, process it and go back to original
cd "%%a"
set "Destination=%Destination%\%%a"
if not exist "!Destination!" md "!Destination!"
rem Get the files in this folder and copy a random one
set "n=0"
for %%b in (%FileFilter%) do (
set /A n+=1
set "file[!n!]=%%b"
)
if !n! gtr 0 (
set /A "rnd=!random! %% n + 1"
for %%i in (!rnd!) do copy "!file[%%i]!" "!Destination!"
)
call :processFolder
cd ..
)
exit /B
Here is anther approach using xcopy /L to walk through all files in the source directory, which does not actually copy anything due to /L but returns paths relative to the source directory. For explanation of the code see all the remarks:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Define source and destination directories here:
set "SOURCE=%dp~1"
set "DESTIN=H:\Temp"
rem Change to source directory:
cd /D "%SOURCE%"
rem Reset index number:
set /A "INDEX=0"
rem Walk through output of `xcopy /L`, which returns
rem all files in source directory as relative paths;
rem `find` filters out the summary line; `echo` appends one more line
rem with invalid path, just to process the last item as well:
for /F "delims=" %%F in ('
2^> nul xcopy /L /S /I /Y "." "%TEMP%" ^
^| find ".\" ^
^& echo^(C:\^^^|\^^^|
') do (
rem Store path to parent directory of current item:
set "CURRPATH=%%~dpF"
setlocal EnableDelayedExpansion
if !INDEX! EQU 0 (
rem First item, so build empty directory tree:
xcopy /T /E /Y "." "%DESTIN%"
endlocal
rem Set index and first array element, holding
rem all files present in the current directory:
set /A "INDEX=1"
set "ITEMS_1=%%F"
) else if "!CURRPATH!"=="!PREVPATH!" (
rem Previous parent directory equals current one,
rem so increment index and store current file:
set /A "INDEX+=1"
for %%I in (!INDEX!) do (
endlocal
set /A "INDEX=%%I"
set "ITEMS_%%I=%%F"
)
) else (
rem Current parent directory is not the previous one,
rem so generate random number from 1 to recent index
rem to select a file in the previous parent directory,
rem perform copying task, then reset index and store
rem the parent directory of the current (next) item:
set /A "INDEX=!RANDOM!%%!INDEX!+1"
for %%I in (!INDEX!) do (
xcopy /Y "!ITEMS_%%I!" "%DESTIN%\!ITEMS_%%I!"
endlocal
set /A "INDEX=1"
set "ITEMS_1=%%F"
)
)
rem Store path to parent directory of previous item:
set "PREVPATH=%%~dpF"
)
endlocal
exit /B
For this approach the destination directory can also be located within the source directory tree.
I have a Directory with a deep Directory->Sub-directory tree structure. I need to write a batch file to copy all the numbered files (files with names as digits and no alphabetic characters) from all the sub-directories.
For example, a sub-directory might contain the following files:
WR10091.txt
AX10091.htm
10091.txt
AX10091.xml
10091.xml
I need to copy 10091.txt and 10091.xml to another location. I can copy files like AX10091.xml and AX10091.htm by specifying AX*.*. But I cannot figure out how to copy just numbered files with no alphabetic characters. There are thousands of directories and the directory structure does not have any pattern (the depth of a tree branch can vary considerably).
Any help will be appreciated.
#echo off
setlocal enableextensions disabledelayedexpansion
set "source=%cd%"
set "target=x:\target\folder"
for /r "%source%" %%a in (*) do (
(for /f "delims=0123456789" %%b in ("%%~na") do (
break
)) || echo copy "%%~fa" "%target%"
)
In this code the for %%a will iterate over all the files under the indicated folder. For each of them, the for /f %%b will try to tokenize the file name (%%~na) using numbers as delimiters. If the file name only contains numbers, there will be nothing to process (only delimiters) and the inner for raises errorlevel. This is checked with conditional execution (the code after the || is executed if the previous command fails) and if errorlevel was raised the copy operation is echoed to console.
If the output is correct, remove the echo to perform the copy.
note: the break in the inner for loop is included just to have a command that does nothing when files with non numeric names are found.
#echo off
for /f "tokens=* delims=" %%a in ('dir /b /s /a:-d "*"') do (
echo %%~na|findstr /e /b /r "[1-9]*" >nul 2>nul && (
copy %%~fa c:\somewhere\
)
)
should be executed in the same directory as the files.
for /f "delims=" %%a in ('dir /b/s/a-d ^| findstr /reic:"\\[0-9][0-9]*\..*" /c:"\\[0-9][0-9]*"') do copy "%%~a" "targetDir"
This might not work with XP and/or Vista, but this can be fixed if needed (see What are the undocumented features and limitations of the Windows FINDSTR command).
After successfully removing a bunch of Google Drive Folder duplicates, some files retain a "filename(2)"name.
Is there a way to batch rename every file so the parenthesis and the number inside the parenthesis is gone?
That includes folders and sub-folders.
Try like this :
Create a file test.bat with the code below in it and replace the path to test in the var $path
#echo off
set $path="C:\Users\CN Micros\Desktop\PROGRAMMATION\test"
for /f "tokens=1-3 delims=^(^)" %%a in ('dir /b/a-d %$path%') do (
if exist %$path%\"%%a(%%b)%%c" echo ren %$path%\"%%a(%%b)%%c" "%%a%%c"
)
pause
Then run it in the CMD or by double clicking.
If the output is ok for you remove the echo
The program create 3 tokens : %%a = what's before the (), %%b What's inside the () and %%c what's after the ().
Then we arrange this 3 tokens to rename the files without the ().
If you have some file who have the same final name ie : "file(1)name", "file(2)name" --> "filename"
It will work only with the first one. If you have this case you have to add a counter at the end of file to be sure that they will be renamed.
This will create renfiles.bat.txt for you to examine in Notepad and then rename to .bat and execute if you are happy with it.
#echo off
dir /b /a-d *(*).* |find /i /v "%~nx0" |find /i /v "repl.bat" |repl "(.*)\(.*\)(\..*)" "ren \q$&\q \q$1$2\q" xa >"renfiles.bat.txt"
This uses a helper batch file called repl.bat - download from: https://www.dropbox.com/s/qidqwztmetbvklt/repl.bat
Place repl.bat in the same folder as the batch file or in a folder that is on the path.
Edit: This version will recurse through subdirectories:
#echo off
dir /b /s /a-d *(*).* |find /i /v "%~nx0" |find /i /v "repl.bat" |repl ".*\\(.*)\(.*\)(\..*)" "ren \q$&\q \q$1$2\q" xa >"renfiles.bat.txt"
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
FOR /f "delims=" %%a IN (
'dir /b /s /a-d "%sourcedir%\*" '
) DO (
SET "name=%%~na"
SETLOCAL ENABLEDELAYEDEXPANSION
SET "newname=!name:)=!"
SET "newname=!newname:(=!"
IF "!name!" neq "!newname!" (
IF EXIST "%%~dpa!newname!%%~xa" (ECHO cannot RENAME %%a
) ELSE (ECHO(REN "%%a" "!newname!%%~xa")
)
endlocal
)
GOTO :EOF
You'd need to set your required directory into sourcedir. I used u:\sourcedir which suits my testing.
The required REN commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(REN to REN to actually rename the files.
I am attempting to use the batch file below to move files from one folder to another. The batch commands will create sub folders within the destination folder based on the create dates against each file in the source folder.
The problem is that the source folder contains sub folders and the batch commands cannot recurse into sub folders.
Please advise how to modify the batch file to allow recurse into sub folders on the source folder.
Thanks
Rialet
echo %1 "-" %2
If [%1]==[] ECHO "Source Directory parameter required"&GOTO :EOF
If [%2]==[] ECHO "Target Directory parameter required"&GOTO :EOF
SET TrimQuote=%2
for /f "useback tokens=*" %%T in ('%2') do set TrimQuote=%%~T
REM echo %TrimQuote%
::loop through files only
For /F "TOKENS=1 DELIMS=%_TabSpace%" %%B In ('dir %1 /a-d /B /OD') DO (
REM echo "%%B - " %%B
For /F "TOKENS=1 DELIMS=%_TabSpace%" %%D In ('dir %1\"%%B" /a-d /OD ^| findstr /B [0-9][0-9]/[0-9]') DO (
REM echo "%%D - " %%D
for /F "tokens=1,2,3,4 delims=/ " %%b in ("%%D") do (
REM echo "b = " %2\%%c%%a\%%b
REM echo %2\%%d%%c\%%b
if NOT exist %2\%%d%%c\%%b md %2\%%d%%c\%%b
move %1\"%%B" %2\%%d%%c\%%b\
)
)
)
Try using
For /F "TOKENS=1 DELIMS=%_TabSpace%" %%B In ('dir %1 /a-d /S /B /OD') DO (
the /s will cause recursion. The downside is that the output of the dir command is then d:\path\file.ext - which may not marry well with your "TOKENS=1 DELIMS=%_TabSpace%". You'd probably need to use "delims=" (ie, no delimiters, hence entire line in token1).
You can then retrieve the various parts of the full-filename as %%~dB, %%~pB, %%~nB and %%~xB (the drive, path, naem and extension - and you can combine these parts if you wish by using %%~nxB for name+extension, for instance.
Supplemental info - batch commented.
#ECHO OFF
SETLOCAL
:: The above two lines are a traditional batch introduction.
:: The first turns `ECHO`ing of the command to the console OFF
:: The second makes all changes to the environment 'local'
:: which means that any variable changes made during the batch
:: will be undone at the end, restoring the original environment.
:: Note that the official remarks/comments method is
REM This is a remark
:: But the double-colon method is commonly used as :: is less intrusive
:: Echo the two parameters given to the batch (%1 and %2)
echo %1 "-" %2
:: The original parameter-present detection is weak. This is a better method
SET target=%~1
If not defined target ECHO "Source Directory parameter required"&GOTO :EOF
SET target=%~2
If not defined target ECHO "Target Directory parameter required"&GOTO :EOF
:: Note `trimquote` (batch is largely case-insensitive) is a meaningless name
:: New name TARGET is better. Setting to %~2 removes enclosing quotes from
:: string assigned to variable.
::loop through files only
:: `"delims="` means there are no delimiters, so the entire line is assigned to
:: the variable `%%B` (FOR loop variablenames ("metavariables") ARE case-sensitive!)
:: The line being assigned comes from the output of the `DIR` command
:: which is filenames only (/a-d) in subdirectories (/s) in basic form (/b)
:: (ie name only, no dates, sizes, headers or summary) and in order of date (/od)
For /F "DELIMS=" %%B In ('dir "%~1" /a-d /S /B /OD') DO (
REM echo "%%B - " %%B
REM within a FOR loop, better to use REM remarks than :: remarks (version-dependent)
REM I believe the intention of the original here was to pick up the filedate
REM It wouldn't work since FINDSTR is looking for lines that begin (/B) with
REM 2 digits, a slash and one digit, but the date format about to be processed...
REM For /F "TOKENS=1 DELIMS=%_TabSpace%" %%D In ('dir %1\"%%B" /a-d /OD ^| findstr /B [0-9][0-9]/[0-9]') DO (
REM echo "%%D - " %%D
REM Process date - 4 elements separated by space or /. Pick the last three
REM so implictly format is DAYNAME xx/yy/zz BUT the elements would be applied
REM to %%b, %%c, %%d, %%e
REM for /F "tokens=1,2,3,4 delims=/ " %%b in ("%%D") do (
for /F "tokens=1,2,3,4 delims=/ " %%a in ("%%~tB") do (
REM echo "b = " %2\%%c%%a\%%b
REM echo %2\%%d%%c\%%b
REM Make a new directory. 2>nul suppresses error message if already exists
md "%TARGET%\%%d%%c\%%b" 2>nul
move "%%B" "%TARGET%\%%d%%c\%%b\"
)
)
Bit of a nightmare really - No idea of what format date you are using, nor what format target directory structure you want. This should "flatten" the structure, so any file filename.ext would be placed in %target%\xx\yy\zz regardless of where in your source structure the file originally resides. There is also no protection about multiple instances of the same filename.ext with the same DATE but in different subdirectories in the source. Need a lot more clarification of the entire scenario to be more certain. Really just commenting and changing the existing (presumed-working but evidently-faulty) batch...
You can also use the XCOPY function to copy a parent and child folders.