I am looking to automatically clean directories that contain original photos and smaller resolutions of a single photo.
I have the following structure for file names
original_image.jpg
original_image-1024x768.jpg
original_image-800x600.jpg
original_image-640x480.jpg
Is there a way, using a windows script (cmd, not PowerShell) to look through files in a directory, and delete any files that has the same name followed by a dash, a group of digits, and x, another group of digits, then the samme extension as the original file?
OOh - not that easy! Be careful!
#ECHO OFF
SETLOCAL enabledelayedexpansion
rem The following setting for the source directory is a name
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
FOR /f "delims=" %%b IN (
'dir /b /a-d "%sourcedir%\*" ^|findstr /i /v /r ".*-[0-9]*x[0-9]*.*" '
) DO (
SET "filter=%%~nb-[0-9]*x[0-9]*\%%~xb"
SET "filter=!filter: =\ !"
FOR /f "delims=" %%e IN (
'dir /b /a-d "%sourcedir%\%%~nb*%%~xb" ^|findstr /i /r "!filter!" '
) DO ECHO DEL "%%e"
)
GOTO :EOF
Always verify against a test directory before applying to real data.
The required DEL commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO DEL to DEL to actually delete the files.
The outer loop (%%b) processes a 'dir' list of the directory /a-d without directorynames and /b in basic form (names only - no headers. footers or details.)
The list is passed to a findstr command to filter the filename pattern required. The pipe | must be escaped by a caret^ to tell cmd that the pipe belongs to the single-quoted command to be executed, not to the for.
The findstr filter is /i case-insensitive /r using a regular expression. The /v option outputs those lines that do not match the filter. The regular expression is .* any number of any character, - -literal dash [0-9]* any number of numeric characters x literal "x" [0-9]* any numerics again and .* any characters.
The delims= causes the filenames to be delivered literally to %%b by setting no delimiters and hence just one token. See for /? from the prompt or endless examples on SO for documentation.
Next step is to set up the filter for the next findstr. This is %%~nb the name part of the filename in %%b and %%~xb the extension part (including the dot). The \ escapes the dot contributed by %%~xb, making it a literal dot instead of a single-character-match.
The next step replaces each "space" with "<kbd>space" See set /? from the prompt or endless examples on SO for documentation.
Finally, execute another dir but this time, look for files matching the pattern of the filename and the extension of %%b, separated by anything and filtering using the string established in filter.
delayedexpansion is required since filter is being changed within a code block (parenthesised sequence of lines) - so !var! retrieves the current value of the variable where %var% is the original vale (when the block was encountered).
Why the extra complexity?
Suppose the file list includes
original_image.jpg
original_image-1024x768.jpg
original_image-800x600.jpg
"original image.jpg"
"original image-1024x768.jpg"
"original image-800x600.jpg"
Then because a Space in a findstr causes an or of the strings before and after the space, so whereas the %%e dir selects on those files matching "original image*.jpg", this includes "original image.jpg". The regex constructed would be "original image-[0-9]*x[0-9]*\.jpg" which matches "original" and therefore "original image.jpg" will be selected for deletion.
Related
I'm trying to copy specific files from C: to "X: for example". The files are named with the same format.
A1234_ZZabc123_DT1_F1.tst
A4567_ZZdef4567_DT2_F2.tst
A8901_ZZghi1289_DT1.tst
A2345_ZZjfkbu12_to_modify.tst
A6789_ZZlmny568_F1_to_modify.tst
A1234_ZZabc478_DT1.txt
I want to copy only the .tst files, and with the same format as the first 3 Axxxx_ZZyyyxxx_DTx_Fx.tst where x=number and y=letter.
After ZZ, it might be 4 letters and 3 numbers, or 5 letters and 4 numbers, like a "namecode".
Example: ZZkusha122 or ZZkus1551.
I need to copy the folders along with the files too.
I'm new to coding and really need some help.
I need to find and copy all those files along 10k+ files together
You claim that the first 3 of your example filenames fit the pattern you describe. I believe that only two do.
#ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
FOR /f "delims=" %%e IN (
'dir /b /a-d "%sourcedir%\A*.tst"^|findstr /X /I /R "A[0-9][0-9][0-9][0-9]_ZZ[a-z][a-z][a-z]_DT[0-9]_F[0-9].tst" '
) DO ECHO COPY "%sourcedir%\%%e" X:
)
GOTO :EOF
Always verify against a test directory before applying to real data.
The required COPY commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO COPY to COPY to actually copy the files. Append >nul to suppress report messages (eg. 1 file copied)
Simply execute a dir command that reports only filenames matching the *.tst mask. Filter this list with findstr which /X exactly matches the regular expression provided. findstr only has a limited implementation of regular expressions. The /I forces a case-insensitive match. If you want case-sensitive, remove the /I and change each [a-z] to [a-zA-Z] (leave as-is if you want lower-case only in these positions.)
See findstr /? from the prompt for more documentation, or search for examples on SO.
---- revision to cater for multiple filemasks and subdirectories ---
#ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "maskfile=%sourcedir%\q74442552.txt"
FOR /f "tokens=1*delims=" %%e IN (
'dir /b/s /a-d "%sourcedir%\*.tst"^|findstr /E /I /R /g:"%maskfile%" '
) DO ECHO COPY "%%e" X:
)
GOTO :EOF
Changes:
Establish a file, name irrelevant, as maskfile
The dir command requires the /s switch to scan subdirectories
The filemask for the dir command loses the initial A
The findstr command replaces the /X switch with /E
The findstr command loses the regex expression. These are transferred to a file and the file is nominated by the /g: switch.
The copy command loses the source-directory as the directory will be included in %%e
The file "q74442552.txt" contains lines that are of the form
A[0-9][0-9][0-9][0-9]_ZZ[a-z][a-z][a-z]_DT[0-9]_F[0-9].tst
A[0-9][0-9][0-9][0-9]_ZZ[a-z][a-z][a-z]_to.*.tst
This time, %%e acquires the full pathname of the files found. Since the filemask ends .tst, the only filenames to pass the dir filter will be those that end .tst.
The /e switch tells findstr to match string that End with the regex strings in the file specified as /g:.
The strings in the file must comply with Microsoft's partial regex implementation, one to a line.
In summary, findstr uses as regex
Any character,literally
[set] any character of a set of characters
[^set] any character not in a set of characters
. any character
.* any number of any character
prefix any of the special characters with\ to use it literally
a set may include a range by using low-high
So - you then need to brew-your own using the examples I've supplied. The second line matches Axxxx_ZZyyy_to{anything}.tst for instance.
--- Minor revision to deal with maintaining destination-tree -----
(see notes to final revision for why this doesn't quite work)
#ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "maskfile=%sourcedir%\q74442552.txt"
SET "destdir=u:\your results"
FOR /f "tokens=1*delims=" %%e IN (
'dir /b/s /a-d "%sourcedir%\*.tst"^|findstr /E /I /R /g:"%maskfile%" '
) DO ECHO "%%~nxe"&XCOPY /Y /D /S "%sourcedir%\%%~nxe" "%destdir%\">nul
)
GOTO :EOF
This version adds the destination root directory as destdir.
The dir ... findstr... works as before to list the filenames to copy.
The prior version used echo copy to report the proposed copy operation, but the destination was always the same directory.
The replacement XCOPY line maintains the directory structure at the destination.
Note : the XCOPY is "live". The files will be copied to the destination if run as-is. Always verify against a test directory before applying to real data.
To "defuse" the XCOPY, add the /L switch and remove the >nul. This will cause XCOPY to report the source name that would be copied instead of copying it. (The >nul suppresses the report)
The /D only copies source files that eitherr do not exist in the destination of have a later datestamp in the source.
The action is to xcopy each filename found (%%~nxe) from the source directory tree to the destination. Therefore, any file xyz.tst found anywhere in the source tree will be xcopyd to the destination tree. The /D means that once xyz.tst is encountered on the source tree, it will be skipped should it be encountered again.
--- Final (I hope) revision ---
#ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=U:\Users\tocil\Desktop\aoi"
SET "maskfile=%sourcedir%\q74442552.txt"
SET "destdir=u:\your results"
FOR /f "tokens=1*delims=" %%e IN (
'dir /b/s /a-d "%sourcedir%\*.tst"^|findstr /E /I /R /g:"%maskfile%" '
) DO (
rem drive and path to 'dirname' - has terminal "\"
SET "dirname=%%~dpe"
rem remove the sourcedir from dirname
FOR %%y IN ("%sourcedir%") DO CALL SET "dirname=%%dirname:%%~y=%%"
rem copy or xcopy the file to the destination.
FOR /f "tokens=2delims==" %%y IN ('set dirname') DO XCOPY /Y "%%e" "%destdir%%%y">nul
)
)
GOTO :EOF
Always verify against a test directory before applying to real data.
Note to self: Only if the filemask provided to XCOPY is ambiguous (ie. contains ? or *) will XCOPY obey the /s switch unless the target file exists in the starting source directory.
hence
xcopy /s sourcedir\myfile destdir
will copy myfile from the entire tree ONLY if sourcedir\myfile exists.
xcopy /s sourcedir\myf?le destdir
will copy myfile from the entire tree regardless. Unfortunately it will also copy myfale and myfule as well.
Hence, the new approach.
First, perform a dir /b /s to get all of the filenames and filter as before. This is being assigned to %%e.
Take the drive and path only of the file and assign to dirname.
The next step is a little complex. First, set the value of %%y to the name of the source directory. Next, use a parser trick to remove that name from dirname. The mechanics are: Parse the %%dirname:%%~y=%% (because the call causes the set to be executed in a sub-shell) whuch does the normal left-to-right evaluation. %% is an escaped-%, so is replaced by %; %%y is an active metavariable so is replaced by (the name of the source directory) and the ~ causes the quotes to be stripped from that name. The resultant command executed is thus SET "dirname=%dirname:nameofsourcedirectory=%"
So now we can construct a copy-class instruction. dirname now contains the relative directory for the destination, which we can extract from the environment by parsing a set listing (Could also be done with delayed expansion) where %%y gets set to the relative directory and has both a leading and trailing backslash, so the destination directory is simply "%destdir%%%y". XCOPY then knows to create that directory if necessary (%%y has a trailing backslash) and we know the source filename is in %%e.
You could also use a copy to do the same thing, but you'd need to create the destination directory first. Another advantage of XCOPY is that you can also specify the /d switch to not copy files that have an earlier date over files that have a later date.
As said in Q-title, I am trying to find a particular directory called Local State, but it could be spelled by some Apps as LocalState or Local State, anyone of which is surely present in every Apps' folder inside %USERPROFILE%, which I am trying to list out.
Now for that I had to write two lines, one for finding LocalState which works well, and it's as given:
pushd "%USERPROFILE%"
for /d /r %%h in (LocalState) do if exist "%%h" echo "%%h"
popd
But with the almost same line when I try to find Local State folder it doesn't show the paths as expected, as it adds extra quotes around the searched folder. See this:
pushd "%USERPROFILE%"
for /d /r %%h in ("Local State") do if exist "%%h" echo "%%h"
popd
gives this, which is weird, as any action can't be taken on this extra quoted path:
....
....
"C:\Users\<Username>\AppData\Local\BraveSoftware\Brave-Browser\User Data\"Local State""
"C:\Users\<Username>\AppData\Local\Google\Chrome\User Data\"Local State""
....
....
Now I am wondering is it possible with one line only I am able to search folder name like LocalState or Local State in the specified folder with batch script ? Something like this?
for /d /r %%h in ("Local? State") do if exist "%%h" echo "%%h"
And it would show paths in regular proper quoted format like:?
....
....
"C:\Users\<Username>\AppData\Local\BraveSoftware\Brave-Browser\User Data\Local State"
"C:\Users\<Username>\AppData\Local\Google\Chrome\User Data\Local State"
....
....
Or if that's not at all possible, then how can I find folder names with spaces and echo those paths in proper quoted format with no extra, unrequired quotes ?
Why do the FOR command lines not work as expected?
The strings LocalState and "Local State" are not interpreted by for as folder name to search for because of neither containing * nor ?. The command FOR searches only for non-hidden files or with option /D for non-hidden folders on specifying a wildcard pattern.
There was tried:
for /d /r %%h in (LocalState) do if exist "%%h" echo "%%h"
for /d /r %%h in ("Local State") do if exist "%%h" echo "%%h"
The command lines above result in searching recursively for directories (including hidden ones) and assign to the loop variable h each found directory with full path not enclosed in " concatenated with the specified string LocalState or "Local State".
Example: The current directory is C:\Temp with following directory structure:
C:\Temp
Development & Test(!)
Folder 2
The IF condition is executed with following strings assigned to loop variable h:
C:\Temp\LocalState
C:\Temp\Development & Test(!)\LocalState
C:\Temp\Folder 2\LocalState
C:\Temp\"Local State"
C:\Temp\Development & Test(!)\"Local State"
C:\Temp\Folder 2\"Local State"
The directory names 4 to 6 are problematic on as they contain themselves two double quotes resulting in executing the IF conditions with not correct specified names for file system entries – directory or file or reparse point – that makes no difference for IF in this case with no backslash at end.
Somebody might think this behavior of FOR does not make sense, but that behavior is useful in some use cases, for example on creation of a file with a specific name in each folder of a directory tree.
The problem here is that there cannot be added * at beginning or at end, i.e. use *LocalState or "Local State*" because of that can result in false positives. FOR would really search now for non-hidden directories of which name ends with LocalState or starts with Local State.
So the usage of the following command line would not be good:
for /d /r %%h in (*LocalState "Local State*") do echo "%%h"
What are possible solutions?
A very fast possible solution is:
for /F "delims=" %%h in ('dir "%USERPROFILE%\LocalState" "%USERPROFILE%\Local State" /AD /B /S 2^>nul') do echo "%%h"
There is started in background one more cmd.exe with option /c and the specified command line within ' appended as additional arguments.
DIR searches first
for just directories because of option /AD
with the name LocalState or the name Local State
in the specified directory %USERPROFILE% and
all its subdirectories because of option /S and
outputs just the fully qualified directory name because of the options /B (bare format) and /S.
DIR is so smart to search in each directory for both directories names. So the entire directory tree is searched by DIR only once for both directory names at the same time.
The started cmd.exe closes itself once DIR finished.
The cmd.exe instance processing the batch file captures all fully qualified folder names output by DIR and FOR processes them now line by line.
The FOR option delims= defines an empty list of delimiters to turn off the default line splitting behavior on normal spaces and horizontal tabs. That is required because of each folder name should be assigned completely one after the other to the loop variable h for further processing and not just the part up to first space character in a full folder name.
Other solutions are:
for /F "delims=" %%h in ('dir "%USERPROFILE%\Local*State" /AD /B /S 2^>nul ^| %SystemRoot%\System32\findstr.exe /E /I /L /C:LocalState /C:"Local State"') do echo "%%h"
for /F "delims=" %%h in ('dir "%USERPROFILE%\Local*State" /AD /B /S 2^>nul ^| %SystemRoot%\System32\findstr.exe /E /I /R /C:"Local *State"') do echo "%%h"
DIR searches in both cases for directories of which name starts with Local and ends with State (case-insensitive) recursively in specified folder %USERPROFILE%.
There is used FINDSTR on the first command line to filter out all false positive found directories of which fully qualified directory name does not end with the case-insensitive and literally interpreted string LocalState or Local State like Local & State.
There is used FINDSTR on the second command line to filter out all false positive found directories of which fully qualified directory name is at end not matched by the case-insensitive interpreted regular expression Local *State which matches LocalState and Local State and also Local State (two spaces) because of * is interpreted here as preceding character (the space) zero or more times. Please notice the difference. In a wildcard pattern * means any character zero or more times, but not here in the regular expression search string interpreted by FINDSTR where it means preceding character zero or more times.
The two solutions searching with DIR for the directories with a wildcard pattern and using FINDSTR to filter out false positive found directories are a bit slower than the solution using just DIR with the two directory names to search for.
In all provided solutions could be modified the DIR option /AD to /AD-L to ignore junctions and symbolic directory links (reparse points) and find just real directories.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
dir /?
echo /?
findstr /?
for /?
Read the Microsoft documentation about Using command redirection operators for an explanation of 2>nul and |. The redirection operators > and | must be escaped with caret character ^ on the FOR command lines to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded command line with using a separate command process started in background.
This will work in a batch-file run under cmd on windows.
FOR /F "delims=" %%A IN ('powershell -NoLogo -NoProfile -Command ^
"(Get-ChildItem -Recurse -Directory -Filter 'Local*State').FullName"') DO (ECHO Directory name is %%~A)
Requires PowerShell 5.1 or later. Find your version with the command
$PSVersionTable.PSVersion.ToString() or (Get-Host).Version.ToString()
I have a file, such as -
foofile_1.ext
A script should read the numerical part of the file, and then rename the file with the next integer, i.e., after execution, the file name should be
foofile_2.ext
I can do it with a C++ / c application or even in bash but not sure how to write a batch script to perform this rename. The filename before the _ isn't going to change, and _ will aaways appear in the same position within the filename.
I can strip the filename to _, but recognizing the numerical is an implementation I am not familiar with. Once I recognize the numerical, I can increment it and rename the file.
Something to consider is that renaming foofile_1.ext to foofile_2.ext will fail should foofile_2.ext already exist. One way to get around it is to rename in descending numerical order, I posted an answer like that before on SO.
I am however not going to post the same answer here, nor link that answer. I will however show one other method:
#echo off
setlocal enabledelayedexpansion
for /f "tokens=1,*delims=_" %%i in ('dir /b /a-d "*_*.ext"') do (
echo %%~nj | findstr /R /V /C:"[A-Z]">nul && (
set /a numeric=%%~nj+1
ren "%%~i_%%~j" "%%~i_-hld-!numeric!%%~xj"
)
)
for /f "delims=" %%f in ('dir /b /a-d "*_-hld-*.ext"') do (
set "name=%%~f"
ren "%%~f" !name:-hld-=!
)
Considering that your input is as you said and does not contain earlier _'s anywhere. This will just take each file with the *_*.ext format. We split by the _ into two tokened metavaiables (%%i and %%j) We take the numeric value and increment by one, then rejoin %%i which is pre _. This however is where the problem comes when the file you are trying to rename to exists, so for that, first we test if %%~nj does not have Alphabetical characters only, using findstr (not doing special characters in this free code) Secondly we give a temporary addition to the name to prevent name clashing.
Once we are done, we simply do a rename on all the files containing the _-hld- temp inclusion.
This question already has an answer here:
At which point does `for` or `for /R` enumerate the directory (tree)?
(1 answer)
Closed 3 years ago.
I can add a prefix to a series of text files using:
:: rename files
for %%a in (*.txt) do (
ren "%%a" "Seekret file %%a"
:: ECHO %%a Seekret file %%a
)
which will turn
a.txt
b.txt
c.txt
into
Seekret file a.txt
Seekret file b.txt
Seekret file c.txt
However, the above code seems to rename the first file twice with the prefix. I end up with
Seekret file Seekret file a.txt
and I have no idea why. Any ideas?
Use
for /f "delims=" %%a in ('dir /b /a-d *.txt') do (
What is happening is that the version you are using sees the renamed-file as a new file. The dir version builds a list of the filenames and then executes the for on each line, so the list is already built and static and cmd isn't trying to operate on a moving target.
Also - use rem, not :: within a code-block (parenthesised sequence of instructions) as this form of comment is in fact a broken label and labels are not allowed in a code block.
Yes, this can happen, especially on FAT32 and exFAT drives because of these file systems do not return the list of directory entries matched by a wildcard pattern to calling executable in an alphabetic order. for processes the directory entries matching *.txt one after the other and the command ren results in changing the directory entries, i.e. the file names list is modified while iterating over it.
The solution is using:
for /F "eol=| delims=" %%I in ('dir *.txt /A-D /B 2^>nul') do ren "%%I" "Seekret file %%I"
FOR runs in this case in background %ComSpec% /c with the command line specified between ' which means with Windows installed into directory C:\Windows:
C:\Windows\System32\cmd.exe /C dir *.txt /A-D /B 2>nul
So one more command process is started in background which executes DIR which
searches in current directory
just for files because of option /A-D (attribute not directory)
including files with hidden attribute set (use /A-D-H to exclude hidden files)
matching the wildcard pattern *.txt
and outputs in bare format just the file names because of option /B.
An error message output by DIR to handle STDERR in case of not finding any directory entry matching these criteria is suppressed by redirecting it to device NUL.
Read the Microsoft article about Using Command Redirection Operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background.
The file names without path are output by DIR to handle STDOUT of background command process. This output is captured by FOR respectively the command process executing the batch file.
After started command process terminated itself, FOR processes the captured list of file names. All changes done on directory during the loop iterations do not matter anymore for that reason. The file names list does not change anymore.
The options eol=| delims= are needed to get the complete file names assigned one after the other to loop variable I even on starting with ; or containing a space character. eol=| redefines default end of line character ; to a vertical bar which no file name can contain. delims= defines an empty list of delimiters to disable default line splitting behavior on normal spaces and horizontal tabs.
Note: :: is an invalid label and not a comment. Labels inside a command block are not allowed and usually result in undefined behavior on execution of the command block. Use command REM (remark) for a comment.
Even better would be:
for /F "eol=| delims=" %%I in ('dir *.txt /A-D /B 2^>nul ^| %SystemRoot%\System32\findstr.exe /B /I /L /V /C:"Seekret file "') do ren "%%I" "Seekret file %%I"
FINDSTR is used here to output from list of file names output by DIR and redirected to STDIN of FINDSTR all file names which
do not because of /V (inverted result)
begin because of option /B
case-insensitive because of option /I
with the literally interpreted because of option /L (redundant to /C:)
string Seekret file .
Option /C: is needed to specify the search string containing two spaces as using just "Seekret file" would result in searching literally and case-insensitive for either Seekret OR file at begin of a line. In a search string specified with just "..." each space is interpreted by FINDSTR as an OR expression like | in a Perl regular expression string.
A search string specified with /C: is interpreted implicitly as literal string, but with using /R (instead of /L) it would be possible to get this string interpreted as regular expression string on which a space is interpreted as space and not as OR expression. It is possible to specify multiple search strings using multiple times /C:.
My recommendation on using FINDSTR: Use always either /L or /R to make it clear for FINDSTR and for every reader of the command line how FINDSTR should interpret the search string(s) specified with "..." or with /C:"...".
I guess I'll throw my hat in too, since I'm not really a fan of looping through dir output and no one else is currently accounting for this script already having been run:
#echo off
set "dir=C:\Your\Root\Directory"
set "pfx=Seekret file "
setlocal enabledelayedexpansion
for /r "%dir%" %%A in (*.txt) do (
set "txt=%%~nA"
if not "!txt:~0,13!"=="%pfx%" ren "%%A" "%pfx%%%~nxA"
)
pause
for /r will loop recursively through all .txt files, set each one as parameter %%A (per iteration), set a variable txt as parameter %%A reduced to just its name (%%~nA), and then it compares the first 13 characters of the text file to your example prefix (which is 13 characters long when you include the space: Seekret file) - if they match the loop does nothing; if they do not match, the loop will rename %%A to include the prefix at the beginning. If you don't want it to be recursive, you can use for %%A in ("%dir%"\*.txt) do ( instead. Other than that, you'll just change !txt:~0,13! depending on what your prefix is or how many letters into a filename you want to check. You also don't have to set your directory and prefix variables, I just prefer to do so because it makes the block look cleaner - and it's easier to go back and change one value as opposed to every place that value occurs in a script.
Reference: for /r, ren, variable substrings
I have a batch file which copies some local files up to a google storage area using the gsutil tool. The gsutil tool produces a nice log file showing the details of the files that were uploaded and if it was OK or not.
Source,Destination,Start,End,Md5,UploadId,Source Size,Bytes Transferred,Result,Description
file://C:\TEMP\file_1.xlsx,gs://app1/backups/file_1.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.804000Z,CPHHZfdlt6AePAPz6JO2KQ==,,18753,18753,OK,
file://C:\TEMP\file_2.xlsx,gs://app1/backups/file_2.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.813000Z,aTKCOQSPVwDycM9+NGO28Q==,,18753,18753,OK,
What I would like to do is to
check the status result in column 8 (OK or FAIL)
If the status is OK then move the source file to another folder (so that it is not uploaded again).
The problem is that the source filename is appended with "file://" which I can't seem to remove, example
file://C:\TEMP\file_1.xlsx
needs to be changed into this
C:\TEMP\file_1.xlsx
I am using a for /f loop and I am not sure if the manipulation of the variables %%A is different within a for /f loop.
#echo off
rem copy the gsutil log file into a temp file and remove the header row using the 'more' command.
more +1 raw_results.log > .\upload_results.log
rem get the source file name (column 1) and the upload result (OK) from column 8
for /f "tokens=1,8 delims=," %%A in (.\upload_results.log) do (
echo The source file is %%A , the upload status was %%B
set line=%%A
set line=!line:file://:=! >> output2.txt echo !line!
echo !line!
)
The output is like this.
The source file is file://C:\TEMP\file_1.xlsx , the upload status was OK
The source file is file://C:\TEMP\file_2.xlsx , the upload status was OK
I'm expecting it to dump the altered values out into a new file but it is not producing anything at the moment.
Normally I would extract from a specific character to the end of the string with something like this but it doesn't work with my For/f loop.
%var:~7%
Any pointers or a different way of doing it greatly appreciated.
Since the part to remove seems fixed it is easier to use substrings.
Also using for /f "skip=1" evades he neccessity of the external command more +1 and another intermediate file.
#echo off & setlocal EnableDelayedExpansion
type NUL>output2.txt
for /f "skip=1 eol=| tokens=1,8 delims=," %%A in (.\upload_results.log) do (
echo The source file is %%A , the upload status was %%B
set "line=%%A"
set "line=!line:~7!"
echo(!line!>>output2.txt
echo(!line!
)
File names and paths can contain also one or more exclamation marks. The line set line=%%A is parsed by Windows command processor a second time before execution with enabled delayed expansion. See How does the Windows Command Interpreter (CMD.EXE) parse scripts? Every ! inside the string assigned to loop variable A is on this line interpreted as begin or end of a delayed expanded environment variable reference. So the string of loop variable A is assigned to environment variable line with an unwanted modification if file path/name contains one or more exclamation marks.
For that reason it is best to avoid usage of delayed expansion. The fastest solution is for this task using a second FOR to get file:// removed from string assigned to loop variable A.
#echo off
del output2.txt 2>nul
for /F "skip=1 tokens=1,8 delims=," %%A in (upload_results.log) do (
echo The source file is %%A , the upload status was %%B.
for /F "tokens=1* delims=/" %%C in ("%%~A") do echo %%D>>output2.txt
)
Even faster would be without the first echo command line inside the loop:
#echo off
(for /F "skip=1 delims=," %%A in (upload_results.log) do (
for /F "tokens=1* delims=/" %%B in ("%%~A") do echo %%C
))>output2.txt
The second solution can be written also as single command line:
#(for /F "skip=1 delims=," %%A in (upload_results.log) do #for /F "tokens=1* delims=/" %%B in ("%%~A") do #echo %%C)>output2.txt
All solutions do following:
The outer FOR processes ANSI (fixed one byte per character) or UTF-8 (one to four bytes per character) encoded text file upload_results.log line by line with skipping the first line and ignoring always empty lines and lines starting with a semicolon which do not occur here.
The line is split up on every occurrence of one or more commas into substrings (tokens) with assigning first comma delimited string to specified loop variable A. The first solution additionally assigns eighth comma delimited string to next loop variable B according to ASCII table.
The inner FOR processes the string assigned to loop variable A with using / as string delimiter to get assigned to specified loop variable file: and to next loop variable according to ASCII table the rest of the string after first sequence of forward slashes which is the full qualified file name.
The full qualified file name is output with command echo and appended either directly to file output2.txt (first solution) or first to a memory buffer which is finally at once written into file output2.txt overwriting a perhaps already existing file with that file name in current directory.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
del /?
echo /?
for /?
See also the Microsoft article about Using command redirection operators for an explanation of the redirections >, >> and 2>nul