I am looking for a list of attributes and what they do in a for loop inside the command prompt.
Specifically I have a .bat file that copies a file from the root of the C:\ drive and pastes it inside all folders found in a pre-specified directory (i.e. C:\Users\John\Test Directory).
This is the command:
#echo off
for /D %%a in (C:\Users\John\Test Directory\*.*) do xcopy /y /d C:\test_file.txt "%%a\"
The .bat does exactly what I need it to do, but I do not understand what the "%%a" does in the command. I see similar commands that use %%g, %%f, etc, but nothing that defines why those were chosen or what they specifically do. Are those attributes arbitrary or do they have a defined function? I seemingly can't find any information on the attributes so any insight is appreciated!
Arbitrary. You can use any letter, upper or lower, and even symbols.
for %%# in... do command %%#
would work just as well. But when working with multiple tokens per iteration, it's better to use the alphabet. Here's an example why:
for /f "usebackq tokens=1* delims==" %%I in ("textfile.txt") do (
set "config[%%~I]=%%~J"
)
This is because %%I contains the text matched prior to the first equal sign, and %%J contains everything after the first equal sign. This answer shows that example in context.
The answer to your question is hinted in the for command's documentation. help for in the cmd console for full details. Specifically:
Some examples might help:
FOR /F "eol=; tokens=2,3* delims=, " %i in (myfile.txt) do #echo %i %j %k
would parse each line in myfile.txt, ignoring lines that begin with
a semicolon, passing the 2nd and 3rd token from each line to the for
body, with tokens delimited by commas and/or spaces. Notice the for
body statements reference %i to get the 2nd token, %j to get the
3rd token, and %k to get all remaining tokens after the 3rd.
This page explains further:
FOR Parameters
The first parameter has to be defined using a single character, for example the letter G.
FOR %%G IN ...
In each iteration of a FOR loop, the IN ( ....) clause is evaluated and %%G set to a different value
If this clause results in a single value then %%G is set equal to that value and the command is performed.
If the clause results in a multiple values then extra parameters are implicitly defined to hold each. These are automatically assigned in alphabetical order %%H %%I %%J ...(implicit parameter definition)
If the parameter refers to a file, then enhanced variable reference can be used to extract the filename/path/date/size.
You can of course pick any letter of the alphabet other than %%G.
%%G is a good choice because it does not conflict with any of the pathname format letters (a, d, f, n, p, s, t, x) and provides the longest run of non-conflicting letters for use as implicit parameters.
G > H > I > J > K > L > M
Format letters are case sensitive, so using a capital letter is also a good way to avoid conflicts %%A rather than %%a.
Just in the interest of thoroughness, it should be pointed out that:
when using for in a cmd console, use single percents.
when using for in a bat script, use double percents.
Use a tilde when retrieving the iterative variable to strip surrounding quotation marks from the value. (e.g. "Hello world!" becomes Hello world!). It's convenient to use this to force a desired format. "%%~G" would always be quoted, whether the captured value was quoted or not. You should always do this when capturing file names with a for loop.
Tilde notation also allows expanding paths, retrieving file size, and other conversions. See the last couple of pages of help for in a cmd console for more details.
Related
From this example right here.
https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/for#additional-references
for %f in (*.doc *.txt) do type %f
In the preceding example, each file that has the .doc or .txt extension in the current directory is substituted for the %f variable until the contents of every file are displayed. To use this command in a batch file, replace every occurrence of %f with %%f. Otherwise, the variable is ignored and an error message is displayed.
Are these different from variables in this example?
https://www.tutorialspoint.com/batch_script/batch_script_variables.htm
set message=Hello World
echo %message%
What are these called? How do I manipulate them?
in batch scripting slang it is called for loop token and the example above will work only in the command prompt. For a batch file you'll need double %.
These tokens changes their values on each iteration of the for loop. Example (this can be executed in the command prompt):
for %a in (1 2) do #echo %a
this will have two iterations - in the first the value if the %%a token it will be 1 and on the seconds 2.
You can use for loops to read files (with /f switch and no quotes) ,to iterate files (with no switch) or directories (with /d switch), iterate through strings (again without switch but using wild cards in strings is not possible) to read files (with /f and no quotes), or process a string (again with /f).
you can also split the value of each iteration with "delims" option and then you'll need more consecutive letters :
for /f "tokens=1,2 delims=-" %a in ("1-2-3") do #echo %a:%b
this will split the string in the quotes by - and will take the first and the second part accessible by the %a and %b tokens.
More on for loops:
https://ss64.com/nt/for.html
https://ss64.com/nt/for2.html
https://ss64.com/nt/for_r.html
https://ss64.com/nt/for_d.html
https://ss64.com/nt/for_f.html
https://ss64.com/nt/for_cmd.html
What I'm trying to do is take a text file with a bunch of strings to search for, each on its own line, and search for each one of these strings in a file (check.txt). I want the output to be a text file with a list of all the strings that COULDN'T be found.
I've tried a few things so far.
for /F "tokens=*" %%A in search.txt do (
#echo on
FINDSTR %%A check.txt
IF ERRORLEVEL 1 echo %%A FAIL > fail_match.txt
)
Another attempt I made (this one was just to tell me if the whole list was good or not) was
#echo on
FINDSTR /g:search.txt check.txt > a_match.txt
IF ERRORLEVEL 1 echo bad > a_match.txt
I realize that these are incredibly basic, and I'm sure there's some easy answer that I just don't understand. I'm not a programmer; I just want to make my job a lot easier (and faster).
To clarify, my list of things to search for is in search.txt, my list of things to check them against is check.txt. Check.txt is a json file, so it's all one enormous line. I don't know if that will make a difference or not. I want a list of all lines in search.txt that are not in check.txt.
Your search scheme seems naive on two fronts:
1) JSON is not guaranteed to be a single line. A valid JASON may have any amount of whitespace, including newlines. This could cause problems if your search string logically matches across multiple lines.
2) What about substring matches? Suppose one search string is bat, and your JSON contains bath. I doubt you would want to consider that a match.
It is possible that neither of the above concerns are a problem for your case. Assuming they aren't, then there may be a fairly simple solution using FINDSTR.
You were close on your first try, except
A) - Your FOR /F IN() clause is missing parentheses
B) - You want to force each search string to be interpreted as a string literal, possibly with spaces. That requires the /C: option.
C) - You assume leading spaces are not significant in your search string ("tokens=*" strips leading spaces)
D) - You assume no search lines begin with semicolon. (The default EOF character is semicolon, and FOR /F skips all lines that begin with the EOF character)
E) - Quotes and backslashes must be escaped within a search string: \" -> \\\\\", \ -> \\, " -> \". See What are the undocumented features and limitations of the Windows FINDSTR command? for more information.
Points C) and D) may be fixed by disabling EOF and DELIMS using the following odd syntax:
for delims^=^ eof^= %%A in ...
Point E) can be addressed by defining a variable and adding escape sequences via search and replace. But this requires delayed expansion, but delayed expansion will corrupt FOR /F variables upon expansion if they contain !. So delayed expansion must be strategically toggled on and off within the loop.
Instead of using IF ERRORLEVEN n, you can use conditional command concatenation || to take action if the previous command failed.
You don't need to see the output of the FINDSTR command, so that can be redirected to NUL.
You can improve performance by redirecting just once, outside the loop.
#echo off
setlocal disableDelayedExpansion
>fail_match.txt (
for /f delims^=^ eol^= %%A in (search.txt) do (
set "search=%%A"
setlocal enableDelayedExpansion
set "search2=!search:\"=\\"!"
set "search2=!search2:\=\\!"
set "search2=!search2:"=\"!"
findstr /c:"!search2!" check.txt >nul || echo !search!
endlocal
)
)
If none of your search strings begin with ;, and no search string contains " or \, then the solution can be as simple as:
#echo off
setlocal disableDelayedExpansion
>fail_match.txt (
for /f "delims=" %%A in (search.txt) do findstr /c:"%%A" check.txt >nul || echo %%A
)
if I read your question right (output all lines of check.txt that are not in search.txt), this single line should do:
findstr /v /x /g:search.txt check.txt > nomatch.txt
the code is:
setlocal EnableDelayedExpansion
FOR /f "usebackq tokens=*" %%X in (`dir /a-d /s /b "!search_path!" 2^>^&1`) DO #(
set file_path=%%X
rem do other stuff
)
Delayed expansion is on because the source path might have special characters like backticks percentages exclamation and ^ escape sing. All these characters are allowed in windows paths and I don't know if and where they will be present.
The problem arise what to do with double percent parameter %%X, how to pass it to another variable without expansion. If DE is on the exclamation sings will be treated as variables with and that would result with a range of weird errors. The same thing is if I disable DE - the same situation, but this time with percentages.
Any idea how to make these lines safe for every possible allowed path that can be found in windows system with no matter how weird characters ?
The problem boils to how to safe pass data from double percent for parameter into normal %variable% so the data can safe passed through delayed expansion from that moment.
I would try to adapt FOR /R to your needs, which will solve some of your escape efforts. You can check the format/match of the file listing in your loop vs. in the dir.
FOR /R will traverse your directory tree (which you're doing anyway) and return the files that match the pattern you give.
Quick example to list all files of type TXT in a directory and it's sub-directories goes like this:
UPDATED:
This prints the contents of two files in my directory that have exclamation points in them:
#echo off
for /r %%i in (ex*!*.txt) do (
type %%~i
)
Note the absense of delayed variable expansion. Add'l variable references are found at the bottom of the for /? listing.
What does this bat code do?
for /f %%i in ('dir /b Client\Javascript\*_min.js') do (
set n=%%~ni
set t=!n:~0,-4!
cp Client\Javascript\%%i build\Client\Javascript\!t!.js
)
What does %%~ni,~n:~0,-4!,%%i,!t! mean?
Keep in mind that in batch files, you need to escape percentage signs unless you're referring to arguments given to the batch file. Once you remove those, you get
for /f %i in ('dir /b Client\Javascript\*_min.js') do (
set n=%~ni
set t=!n:~0,-4!
cp Client\Javascript\%i build\Client\Javascript\!t!.js
)
%i is the declaration of a variable used to place the current file for has found. %~ni extracts the filename portion of %i. !n:~0,-4! uses delayed expansion to remove the last four characters from %n% (set in the previous line) !t! is simply delayed expansion of the %t% variable set in the previous line.
Delayed expansion is used because otherwise, the variables will be substituted as soon as the line is encountered, and future iterations will not re-expand the variable.
for /f %%i in ('dir /b Client\Javascript\*_min.js') do (
Iterate over every file in the Client\Javascript folder that match "*_min.js". Thedircommand andfor /f` are totally unneeded here, though and only complicate things, especially when file names contain spaces, commas and the like. A more robust and simpler alternative would be
for %%i in (Client\Javascript\*_min.js) do (
But that's just beside the point. People tend to write unelegant batch files sometimes, ignoring the pitfalls and common errors. That's just one example of that.
set n=%%~ni
Creates a variable n, containing the file name (without any directory information or extension) of the file currently processed. We remember that the for statement iterates over every file it finds. With this line starts what it does with those files.
set t=!n:~0,-4!
Creates a second variable, t, containing everything but the last four characters of the file name. This essentially strips away the "_min"
cp Client\Javascript\%%i build\Client\Javascript\!t!.js
Finally, this copies the original file to the directory build\Client\Javascript with the new name, just constructed. So a file like Client\Javascript\foo_min.js will be copied to Client\Javascript\foo.js. The !t! here is just a delayed-evaluated environment variable. More on that below. Here it should suffice that it just inserts the contents of said variable at that point in the line.
Again, bad practice here that will break in numerous interesting ways:
cp is not a command on Windows so this batch will assume cygwin, GNUWin32 or similar things installed. I tend to avoid having too many unneeded dependencies and stick to what Windows provides; in this case the copy command. Two bytes won't kill anyone here, I think.
No quotes are around either argument. Leads to interesting results when spaces start appearing in the file name. Not good, either.
As for why delayed expansion was used (! instead of % surrounding the variables: The for command consists of everything in the block delimited by parentheses here as well. The entire block is parsed at once and normal variable expansion takes place when a line/command is parsed. That would mean that every variable in the block would be evaluated before the loop even runs, leaving just the following:
for /f %%i in ('dir /b Client\Javascript\*_min.js') do (
set n=%%~ni
set t=
cp Client\Javascript\%%i build\Client\Javascript\.js
)
which is certainly not what you want in this case.
Delayed expansion is always needed when creating and using variables in a loop such as this. A workaround not needing delayed expansion would be to offload the loop interior into a subroutine:
for /f %%i in ('dir /b Client\Javascript\*_min.js') do call :process "%%i"
goto :eof
:process
set n=%~n1
set t=%n:0,-4%
copy "Client\Javascript\%~1" "build\Client\Javascript\%t%.js"
goto :eof
Since the subroutine is not a single "block" (something delimited by parentheses) it will be parsed line by line as usual. Therefore it's safe to use normal expansion instead of delayed expansion here.
A complete help for the FOR command can be found on the Microsoft TechNet site. See here for more information on delayed expansion :
// Pseudo code
for each file named *_min.js in the specified directory
n is set to the file name (*_min)
t is set to the file name, excluding the last 4 characters (*)
the file is copied and renamed t.js to the specified directory
%~ni expands to just the filename part of i.
!n:~0,-4! expands to all but the last four characters of n.
In general, help for at the command prompt will give an overview of the multitude of ways for can expand variables these days.
Over the past few years I've really found Stackoverflow to be very helpful, and decided to create an account - this is my first post.
Example situation - I have a fair few of these images, of different subjects -
AAA_BBB_randomDigits_front.jpg
AAA_BBB_randomDigits_left.jpg
AAA_BBB_randomDigits_right.jpg
ZZZ_EEE_randomDigits_front.jpg
ZZZ_EEE_randomDigits_left.jpg
ZZZ_EEE_randomDigits_right.jpg
I would like them to all be grouped up in folders as -
AAA_BBB_randomDigits\(contains left, front and right)
ZZZ_EEE_randomDigits\(contains left, front and right)
The code I currently have works -
#echo off
for /f "tokens=1-3 delims=_" %%a in ('dir /b /a-d *_*_*_*.*') do (
md "%%a_%%b_%%c" 2>nul
move "%%a_%%b_%%c*" "%%a_%%b_%%c"
)
pause
However, I would love it if someone could explain to me -
What's %%a?
What's dir /b /a-d and why do I need it?
Is it neccessary to have #echo off and pause?
Thanks guys, I really appreciate it.
For documentation, see commandname /? from the prompt.
dir /b /a-d filemask performs a directory listing /b specifies filenames only - no size, date, header or footer. The /a-d excludes directorynames.
You need it to provide the names to the for /f command.
for /f reads the "filename" in parentheses (it can be a real filename or a single-quoted command (like dir) or a double-quoted literal string) and assigns values to the metavariable (in this case, %%a) according to the specified options (the part in quotes directly following the /f).
The delims option specifies which set of characters is used for parsing the line of data arriving from the "file" specified. The line is then interpreted as a series of tokens, separated by delimiter-sequences. By default, delims is Space and Tab. It's common to turn delims off entirely using "...delims=" in which case, there is but one token (the entire line). Note that any characters between delims= and " are equally-ranking and case-sensitive - it is a set of delimiters which replaces Space and Tab, not a delimiter-string.
The tokens option specifies which tokens are selected, by number, starting at 1. The special token * means "the remainder of the line following the highest-number token specified (including any delimiter characters)". By default, tokens=1.
%%a is a metavariable. It is the variable that holds the first token number selected from the tokens= list. Each selected token number is assigned to the next metavariable name in alphabetical sequence, hence in your example, since you have tokens=1-3 then %%a is the first token, %%b the second and %%c the third. Metavariables are always one letter (some other characters are sometimes used - but numerics are definitely not allowed) and the name is case-sensitive (normally, batch is case-insensitive). %%a, %%A and %a% are all different variables. %a% and %A% are the same variable.
A metavariable is only valid within the for loop where it was created. When the for loop ends, the variable disappears.
#echo off simply turns off the command-echoing that batch would otherwise produce (show the command on the console, then execute it). It's used to reduce clutter on the display. When debugging a section of code, it's normal to set echo to on (echo on) and then off again (echo off) to show precisely what instructions are being executed. The # means "don't report this command to the console"
The pause simply waits until a response is received from the keyboard. It's used to allow the display to be read rather than simply continuing directly to the next instruction. It's often used in debugging and also to allow the result of a batch to be held for the user if the batch is executed by using point-click-and-giggle.