I'm trying to compare filenames from two folders and delete if duplicate found from ParentFolder.
I'll explain bit more as follows,
Files I'm trying to process through batch script
ParentFolder
300103666600005_20221215T075949_0090002504.xml
300103666600003_20221215T075948_00900025054.xml
300103666600003_20221215T075977_0090002506223.xml
300103666600004_20221215T095950_009000250702.xml
And then the content modified files will be copied to a child folder(TargetFolder1) by adding _signed suffix at the end of the file name as shown below,
300103666600005_20221215T075949_0090002504_signed.xml
300103666600003_20221215T075948_00900025054_signed.xml
300103666600003_20221215T075977_0090002506223_signed.xml
300103666600004_20221215T095950_009000250702_signed.xml
What I was trying to do...
First copy the XML files which are not exists in the child folders.
Get the file name from ParentFolder without extension & assume it to a variable called match1
Get the file name from child folder without extension by omitting _signed & assume it to a variable called match2
Check whether match1 = match2. If so, delete that matching file from Source folder only. (Which means we already processed & signed this file and we no longer need unprocessed file in the ParentFolder)
Following is the code I used to achieve this,
#echo off
setlocal EnableExtensions EnableDelayedExpansion
set "ParentFolder=D:\Test\QGenV1"
set "TargetFolder1=D:\Test\QGenV1\ESIGN"
set "TargetFolder2=D:\Test\QGenV1\ECOMP"
if not exist "%SourceFolder1%\" echo ERROR: Missing source folder 1: "%SourceFolder1%"& goto ErrorExit
if not exist "%TargetFolder%\" echo ERROR: Missing source folder 2: "%SourceFolder2%"& goto ErrorExit
md "%TargetFolder1%" 2>nul
md "%TargetFolder2%" 2>nul
if exist "%TargetFolder1%\" goto Process
if exist "%TargetFolder2%\" goto Process
echo ERROR: Failed to create target folder: "%TargetFolder1%"
echo ERROR: Failed to create target folder: "%TargetFolder2%"
:ErrorExit
echo/
pause
exit /B 1
:Process
rem Copy all files being unique in ParentFolder to TargetFolder1.
for %%I in ("%ParentFolder%\*.xml") do (
if not exist "%TargetFolder2%\%%~nxI" COPY "%%I" "%TargetFolder2%\"
if not exist "%TargetFolder1%\%%~nxI" COPY "%%I" "%TargetFolder1%\"
)
rem Delete existing files
for /f "tokens=*" %%l in ("%ParentFolder%\*.xml") do (
set filename=%%~nl
echo From Parent !match1!
for /f "tokens=*" %%A in ("%TargetFolder1%\*.xml") do (
set filename2=%%~nA
echo From Target !match2!
if "!match2!" == "!match1!" DEL /F "%ParentFolder%\!match1!.xml"
)
)
pause
endlocal
What Happened & what went wrong?
Copying only the XML file from Parent folder to Child folders
(TargetFolder1 & TargetFolder2) - Success
Extracting Filename without extension - Success
Extracting Filename without _signed suffix from TargetFolder1 -
Success
Comparing & deleting the file from ParentFolder - Partially
successful. And that is the problem.
What do I mean by 'Partial success is that the deleting process only works until the first occurrence of a file delete happens. Iterating through remaining files to compare & delete process doesn't continue if the second comparison is unmatched.
Any help would be great...
Related
I have a series of photos in a folder with name format similar to this:
BA-ML-6256_Gocchup1.jpg
BA-ML-6256_Gocchup2.jpg
BA-ML-17302_Gocchup1.jpg
BA-ML-17302_Gocchup2.jpg
I want to create new folders like below that contain the files:
BA-ML-6256
BA-ML-17302
I tried using this script:
#echo off
for %%i in (*) do (
if not "%%~ni" == "organize" (
md "%%~ni" && move "%%~i" "%%~ni"
)
)
but it created these 4 folders instead:
BA-ML-6256_Gocchup1
BA-ML-6256_Gocchup2
BA-ML-17302_Gocchup1
BA-ML-17302_Gocchup2
Please help me create a batch script that would make this work.
#ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
FOR /f "tokens=1*delims=_" %%b IN ('dir /b /a-d "%sourcedir%\*_*.jpg"') DO (
ECHO MD "%sourcedir%\%%b"
ECHO MOVE "%sourcedir%\%%b_%%c" "%sourcedir%\%%b"
)
GOTO :eof
Always verify against a test directory before applying to real data.
The required MD commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO MD to MD to actually create the directories. Append 2>nul to suppress error messages (eg. when the directory already exists)
The required MOVE commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO MOVE to MOVE to actually move the files. Append >nul to suppress report messages (eg. 1 file moved)
Naturally, you could remove the %sourcedir%\ throughout if you want to process the current directory.
The dir command lists the filenames only in the directory, and then only those that match *_*.jpg, that is anything1_anything2.jpg
The for /f assigns anything1 to %%b and anything2.jpg to %%c, using the _ as a delimiter (see for /? from the prompt, or thousands of items on SO for examples/documentation)
Then make the directory and move the file.
Simples!
You need to parse the file names to derive the target folder name.
Sample Batch File
The following batch file should do as you desire:
#echo off
for %%i in (*.jpg) do (
for /f "delims=_" %%j in (%%i) do (
md %%j
move "%%~i" "%%j"
)
)
)
This splits the file name at the "_" character to derive the target folder
Example Output
I started with this folder layout:
F:\projects\sx\batch>tree /f
Folder PATH listing
Volume serial number is 9C33-6BBD
F:.
create-test-data.cmd
md-and-move.cmd
BA-ML-6256_Gocchup1.jpg
BA-ML-6256_Gocchup2.jpg
BA-ML-17302_Gocchup1.jpg
BA-ML-17302_Gocchup2.jpg
No subfolders exist
Running the batch file gives this output:
F:\projects\sx\batch>md-and-move.cmd
1 file(s) moved.
A subdirectory or file BA-ML-6256 already exists.
1 file(s) moved.
1 file(s) moved.
A subdirectory or file BA-ML-17302 already exists.
1 file(s) moved.
and results in this folder layout:
F:\projects\sx\batch>tree /f
Folder PATH listing
Volume serial number is 9C33-6BBD
F:.
│ create-test-data.cmd
│ md-and-move.cmd
│
├───BA-ML-6256
│ BA-ML-6256_Gocchup1.jpg
│ BA-ML-6256_Gocchup2.jpg
│
└───BA-ML-17302
BA-ML-17302_Gocchup1.jpg
BA-ML-17302_Gocchup2.jpg
I am trying to move files to a particular folder
(destination folder), but I do not want to overwrite any
files. Instead, if the file already exist in that particular
folder (destination folder), then move that file in another
folder (overflow folder), but keep folder structure.
I tried with xcopy and robocopy, but It appears that it
can't be done like that.
This is the script I use to move files but not overwrite.
robocopy "C:\DummySourcePath\" "C:\DummyDestantionPath\" /E /XC /XN /XO
Now it looks like your goal is to take a folder or directory and take all contents from this folder (including from sub-directory's) and copy it to one main folder. Any duplicate files you want rather to be sent to an overflow rather then to be overwritten.
If you don't want to overwright any files, an overflow folder will work. However what if there are two duplicate files trying to be dumped into the overflow directory? To get around this, we can simply name {File} to {File}(1), {File}(2), exc. This part of the script was taken from michael_heath on the post - batch script to copy files listed in a txt document and keep duplicates.
Essentially we are using an FOR /R statement along with an IF statement to check if the target directory contains the file or not. If it does, the ELSE will move it to overflow with further anti-overwright protections.
#ECHO OFF
#setlocal enableextensions enabledelayedexpansion
rem | Configure directories
set "source=C:\Source-Directory"
set "target=C:\Target-Directory"
set "overflow=Overflow-Directory"
rem | Scan target directory for a duplicate file name.
rem | If a duplicate was found, run a function to copy it to a overflow directory.
rem | If it already exists in the overflow directory rename it to {Name}(1), {Name}(2), exc.
rem | The overflow {Name}(1) protection was originally scripted by: michael_heath
FOR /R "%source%" %%i in (*.*) do (
If not exist "%target%\%%~nxi" (copy "%%i" "%target%") ELSE (call :index "%%~i" "%overflow%\%%~nxi" "1"))
rem | Run finished code here or end script with "goto :eof"
goto :eof
:index source, overflow, count
setlocal
set /a "cnt=%~3"
if exist "%overflow%\%~n2(%cnt%)%~x2" (
call :index "%~1" "%~2" "%cnt%+1"
) else copy "%~1" "%overflow%\%~n2(%cnt%)%~x2"
#echo off
setlocal
set "source=%cd%\source"
set "target=%cd%\target"
set "overflow=%cd%\overflow"
for /r "%source%" %%A in (*.*) do call :copyfile "%%~A" "move" "report"
2>nul rd "%source%"
exit /b
:copyfile source [, operation [, report]]
setlocal
set "curpath=%~1"
set "operation=%~2"
set "report=%~3"
if defined report echo "%curpath%"
call set "destpath=%%curpath:%source%=%target%%%"
if exist "%destpath%" call set "destpath=%%curpath:%source%=%overflow%%%"
if exist "%destpath%" (
if defined report echo exist in "%destpath%"
exit /b 1
)
if "%operation%" == "copy" (
if defined report echo copy to "%destpath%"
echo f|>nul xcopy "%curpath%" "%destpath%"
) else (
for %%A in ("%destpath%") do (
if not exist "%%~dpA" md "%%~dpA" || (
if defined report echo md failed with "%%~dpA"
exit /b 1
)
)
if defined report (
echo move to "%destpath%"
move "%curpath%" "%destpath%"
) else >nul move "%curpath%" "%destpath%"
for %%A in ("%curpath%") do 2>nul rd "%%~dpA"
)
exit /b 0
move on same partition is a move in the
master file table only.
move to different partition is an actual
copy and delete on successful copy.
copy always does copy.
I implemented both operations and can be set by the
2nd optional argument to the called label of :copyfile
by passing "move" or "copy".
"move" is default if argument is not "copy".
The 3rd optional argument to the label :copyfile
is to output a progress report.
This argument if defined, will echo information
(like paths etc.) about the operation.
The 1st argument to the label :copyfile is the
path to the file to be copied or moved.
The for /r loop recurses the source directory and
calls :copyfile with the path of each file found.
:copyfile will set curpath to the source file and
will set destpath to the path to target, which is
the source path substituted with target path.
If destpath exist, then destpath is set to the path
to overflow, which is the source path substituted
with overflow path. If still destpath exist, then
the label is exited.
The move operation uses rd to remove empty folders
from source.
The copy operation uses xcopy as it makes the
destination folder structure so that md is not used.
The echo f piped to xcopy is to answer the "file
or folder?" prompt.
At end of script, the source folder will be removed
if empty.
Set the source, target and overflow variables
at the top of the script to the actual paths.
View set /? about variable substitution used in
the script. It is used to replace the source
directory path with the another directory path,
to create the destination path.
I have a folder that contains files; each document should have .pdf and .xml format. I need to write a BAT file to run from a scheduled task to verify that both documents exist for each.
My logic is:
loop through files in the folder
strip each file to its name without extension
check that same name files exist for both .xml and pdf.
if not mark a flag variable as problem
when done, if the flag variable is marked, send an Email notification
I know how to use blat to sending email, but I'm having trouble to execute the loop. I found a way to get path and file name without extension but can't merge them.
I've used batch files a few time, before but I'm far from an expert. What am I missing?
Here's the code I have so far:
set "FolderPath=E:\TestBat\Test\"
echo %FolderPath%
for %%f in (%FolderPath%*) do (
set /p val=<%%f
For %%A in ("%%f") do (
Set Folder=%%~dpA
Set Name=%%~nxA
)
echo Folder is: %Folder%
echo Name is: %Name%
if NOT EXIST %FolderPath%%name%.xml
set flag=MISSING
if NOT EXIST %FolderPath%%name%.pdf
set flag=MISSING
)
echo %Flag%
pause
There is no need for fancy code for a task such as this:
#Echo Off
Set "FolderPath=E:\TestBat\Test"
If /I Not "%CD%"=="%FolderPath%" PushD "%FolderPath%" 2>Nul||Exit/B
Set "flag="
For %%A In (*.pdf *.xml) Do (
If /I "%%~xA"==".pdf" (If Not Exist "%%~nA.xml" Set "flag=MISSING")
If /I "%%~xA"==".xml" (If Not Exist "%%~nA.pdf" Set "flag=MISSING")
)
If Defined flag Echo=%flag%
Timeout -1
Something like this :
set "FolderPath=E:\TestBat\Test\"
pushd "%FolderPath%"
for %%a in (*.xml) do (
if exist "%%~na.pdf"(
echo ok
) else (
rem do what you want here
echo Missing
)
)
popd
Is this what you want?
#echo off
setlocal enabledelayedexpansion
set "FolderPath=E:\TestBat\Test\"
echo !FolderPath!
for /f "usebackq delims=" %%f in (`dir !FolderPath! /B`) do (
set /p val=<%%f
For %%A in ("%%f") do (
Set Folder=%%~dpA
Set name=%%~nxA
)
echo Folder is: !Folder!
echo Name is: !name!
if NOT EXIST !FolderPath!!name!.xml set flag=MISSING
if NOT EXIST !FolderPath!!name!.pdf set flag=MISSING
)
echo Flag: !flag!
pause
endlocal
You should reformat your code and keep in mind that the grama for batch file is critical. BTW, if you are trying to update the existing batch variable and read it later, you should enable localdelayedexpansion and use ! instead of %.
Keep it simple:
#echo off
pushd "E:\TestBat\Test" || exit /B 1
for %%F in ("*.pdf") do if not exist "%%~nF.xml" echo %%~nxF
for %%F in ("*.xml") do if not exist "%%~nF.pdf" echo %%~nxF
popd
This returns all files that appear orphaned, that is, where the file with the same name but the other extension (.pdf, .xml) is missing. To implement a variable FLAG to indicate there are missing files, simply append & set "FLAG=missing" to each for line and ensure FLAG is empty initially. Then you can check it later by simply using if defined FLAG.
Note: This does not cover the e-mail notification issue. Since I do not know the BLAT tool you mentioned, I have no clue how you want to transfer the listed files to it (command line arguments, temporary file, or STDIN stream?).
In case there is a huge number of files in the target directory, another approach might be better in terms of performance, provided that the number of file system accesses is reduced drastically (note that the above script accesses the file system within the for loop body by if exist, hence for every iterated file individually). So here is an attempt relying on a temporary file and the findstr command:
#echo off
pushd "E:\TestBat\Test" || exit /B 1
rem // Return all orphaned `.pdf` files:
call :SUB "*.pdf" "*.xml"
rem // Return all orphaned `.xml` files:
call :SUB "*.xml" "*.pdf"
popd
exit /B
:SUB val_pattern_orphaned val_pattern_missing
set "LIST=%TEMP%\%~n0_%RANDOM%.tmp"
> "%LIST%" (
rem // Retrieve list of files with one extension:
for %%F in ("%~2") do (
rem /* Replace the extension by the other one,
rem then write the list to a temporary file;
rem this constitutes a list of expected files: */
echo(%%~nF%~x1
)
)
rem /* Search actual list of files with the other extension
rem for occurrences of the list of expected files and
rem return each item that does not match: */
dir /B /A:-D "%~1" | findstr /L /I /X /V /G:"%LIST%"
rem // Clean up the temporary file:
del "%LIST%"
exit /B
To understand how it works, let us concentrate on the first sub-routine call call :SUB "*.pdf" "*.xml" using an example; let us assume the target directory contains the following files:
AlOnE.xml
ExtrA.pdf
sAmplE.pdf
sAmplE.xml
So in the for loop a list of .xml files is gathered:
AlOnE.xml
sAmplE.xml
This is written to a temporary file but with the extensions .xml replaced by .pdf:
AlOnE.pdf
sAmplE.pdf
The next step is to generate a list of actually existing .pdf files:
ExtrA.pdf
sAmplE.pdf
This is piped into a findstr command line, that searches this list for search strings that are gathered from the temporary file, returning non-matching lines only. In other words, findstr returns only those lines of the input list that do not occur in the temporary file:
ExtrA.pdf
To finally get also orphaned .xml files, the second sub-routine call is needed.
Since this script uses a temporary file containing a file list which is processed once by findstr to find any orphaned files per extension, the overall number of file system access operations is lower. The weakest part however is the for loop (containing string concatenation operations).
I want to accomplish the following: copy File A from Directory A to Directory B but File A already exist in Directory B and i don't want to overwrite it, simply put the new file in Directory B with a (1) or something so i'll know the difference. i can't manually name the file in the script as i plan to make it a batch file and run it every few hours therefore resulting in the same file existing with different names. IE. File A, File A(1), File A(2) so on and so on. please help. Thank you
Use Xcopy command with parameter /D
C:\> xcopy D:\source_dest E:\target_dest /E /D
/E parameter make that it copy files with subfolders
/D parameter make that it copy files without overwrite
if want more help inform me .if it works vote up
A cyclic copy refers to the situation where the source directory
contains the destination directory. Since the destination directory
is part of the source, the copying process eventually starts copying
the destination directory to a deeper portion of the destination.
This process will continue until, of course, the finite disk space
eventually runs out.
To avoid this condition but to achieve the objective, one can
specify a temporary destination which is on another volume (e.g.,
D:\temp\) and later copy the temporary destination to the final
destination, and delete the temporary directory at the end.
just check file exist before run xcopy command then run copy command
if NOT EXIST c:\directory B\File A ( xcopy c:\Directory A\File A c:\Directory B )
The following is made for folders but you can modify it for files.
Makes a backup of existed folder directory (and all contents in it) with new increase number in folder name [like TEST(1)... TEST(2)... folder].
The first part :Start will set the %PathFolder% variable with path to folder name (%userprofile%\Desktop\TEST) and then will search if exist. If NOT exists, will create it (xcopy), else, If exist, will directing the %PathFolder% variable to :Search label (for-)loop section for further handling...
Tip: Can be used %1 variable to set PathFolder set "PathFolder=%1" so it will work when you Drag-n-Drop a folder on this current saved batch file.
The second part :Search will search in the %PathFolder% variable (%userprofile%\Desktop\TEST) and will make a COPY of "TEST" folder (and all contents in it) with an increase number added in parenthesis at the end of folder name [like TEST(1)]. If already exist TEST(1) then will copy TEST folder as TEST(2) ... or TEST(3) ... and so on.
::Make a backup of existed folder directory (and all contents in it) with new increase number in folder name.
#echo off
setlocal EnableDelayedExpansion
set "IncrNum=1"
:Start
::The following will set the "%PathFolder%" variable with path to folder name (%userprofile%\Desktop\TEST) and then will search if exist. If NOT exists, will create it (xcopy), else, If exist, will directing the "%PathFolder%" variable to :Search label (for-)loop section for further handling...
::Tip: Can be used %1 instead of %userprofile%\Desktop\TEST in [set "PathFolder=..."] line so it will work when you Drag-n-Drop a folder on this current saved batch file.
set "PathFolder=%userprofile%\Desktop\TEST"
if NOT exist "%PathFolder%" (
xcopy %PathFolder% %PathFolder% /i /y
exit /b
) else (
goto :Search
)
exit /b
:Search
::The following will search in the "%PathFolder%" variable (%userprofile%\Desktop\TEST) and will make a COPY of "TEST" folder (and all contents in it) with an increase number added in parenthesis at the end of folder name [like TEST(1)]. If alredy exist TEST(1) then will copy TEST folder as TEST(2) ... or TEST(3) ... and so on.
for /f "tokens=*" %%G in ('dir /b /s /a:d "%PathFolder%*"') do (
if exist %%G^(%IncrNum%^) (
echo At "%%~dpG" a folder "%%~nG(%IncrNum%)" alredy existed.
set /a IncrNum+=1
goto :Search
) else (
echo.
echo.
echo The "%%~nG" folder and all contents in it
echo will be copied now as "%%G(%IncrNum%)".
echo.
pause
xcopy %%G %%G^(%IncrNum%^) /i /y
exit /b
)
)
:EndBatch
set "PathFolder="
pause
exit
I want to parse a directory and its subdirectories and copy all files in a target directory ignoring the original folder structure. All files shall be copied directly in the target directory.
I found a nice solution here for the Windows command line:
for /r FolderA %f in (*) do #copy "%f" target
https://stackoverflow.com/a/1502183/772434
That in general works fine, but I have duplicate files in the source directory which have the same name. There are too many files, so I can not handle those exceptions by hand.
How can I handle those files in the script automatically?
Options:
overwrite files during copying
keep copies and rename file e. g. by adding "__123" a number at the end of the file name
compare files (MDS5 sum or similar) and create copy only if files are different.
I would prefer option 3 (compare file content and keep numbered copies only if files are really different), but I'm also interested in solutions for options 1 and 2 (more pragmatic).
Here's an approach. i haven't tested it, so i hope it works as expected.
There is a global variable called CURRENT_DUPLICATE_SUFFIX. every time a file with the same name is found, but with different content, this value gets incremented and appended to the resulting file name
#echo off
SET SOURCE_DIR=C:\temp\LogAnalyzer
SET TARGET_DIR=C:\temp\test\output
SET CURRENT_DUPLICATE_SUFFIX=1
for /r "%SOURCE_DIR%" %%f in (*) do CALL :SUB1 "%%f"
GOTO END
:SUB1
SET PURE_FILE_NAME=%~nx1
SET TARGET_FILE_NAME=%TARGET_DIR%\%PURE_FILE_NAME%
IF NOT EXIST "%TARGET_FILE_NAME%" (
COPY %1 "%TARGET_DIR%"
GOTO :EOF
)
REM if we are here, the target file exists.
echo n |comp %1 "%TARGET_FILE_NAME%" >NUL 2>NUL
REM in case the files have the same content
IF %ERRORLEVEL%==0 GOTO :EOF
SET TARGET_FILE_NAME=%TARGET_FILE_NAME%_%CURRENT_DUPLICATE_SUFFIX%
SET /A CURRENT_DUPLICATE_SUFFIX=%CURRENT_DUPLICATE_SUFFIX%+1
COPY %1 "%TARGET_FILE_NAME%
GOTO :EOF
:END