I'm working on a batch script that is suppose to prompt the user for a list of projects, and then process each of those projects in turn. My thought was that this could be done with a for loop, but it's not working. For some reason it's treating the entire string entered by the user (CanalyzerIF CanoeIF CometIF) as a single token.
echo Enter the names of the projects, deliniating each with a space:
set /P PROJECT_LIST=
echo.
echo DEBUG: PROJECT_LIST is %PROJECT_LIST%
echo These are the projects you specified:
for /F "tokens=*" %%i in ("%PROJECT_LIST%") do (
echo %%i
)
My script output looks like this...
DEBUG: PROJECT_LIST is CanalyzerIF CanoeIF CometIF
These are the projects you specified:
CanalyzerIF CanoeIF CometIF
...when what I expect/want to see is this:
DEBUG: PROJECT_LIST is CanalyzerIF CanoeIF CometIF
These are the projects you specified:
CanalyzerIF
CanoeIF
CometIF
It doesn't seem to matter if I use percent signs (%) or exclamation marks (!) to wrap PROJECT_LIST. Anyone know how to fix this?
try with (plain FOR can be used for itration):
echo Enter the names of the projects, deliniating each with a space:
set /P "PROJECT_LIST="
echo.
echo DEBUG: PROJECT_LIST is %PROJECT_LIST%
echo These are the projects you specified:
for %%i in (%PROJECT_LIST%) do (
echo %%i
)
In addition to #npocmaka's perfect solution of the problem using for without /F, I want to provide a solution with the /F option, mainly for demonstration purpose.
First let's take a look at for /F without option string, so using default options, which are like "tokens=1 delims= _" (_ stands for a tab here). This means to take the first space- or tab-separated token and assign it to a given variable like %%i (that is, the first project in your list) and to ignore the rest.
To get multiple items, you need to specify exactly which ones you want to extract. For instance, to get tokens 2,3,4 & 6, state "tokens=2-4,6".
The first of these tokens is assigned to the given variable %%i, the other ones to %%j, %%k, %%l (which constitute implicitly defined variables).
The special token * means to pass all remaining tokens to a single variable, or, in other words, to treat them as a single token (for example, "tokens=1,*" passes the first token to %%i and all the rest to %%j).
Understanding all this leads us to the main problem using for /F: you need to know how many tokens are available. But for this application, we don't know that.
The following uses option string "tokens=1,*" to extract the first token and all the rest; there is a kind of while-loop wrapped around (composed by if and goto) that is intended to reprocess all the rest until nothing is available any more (herein I explicitly defined the space to be the only acceptable delimiter):
echo Enter the names of the projects, deliniating each with a space:
set /P PROJECT_LIST=
echo.
echo DEBUG: PROJECT_LIST is %PROJECT_LIST%
echo These are the projects you specified:
set PROJECT_TEMP=%PROJECT_LIST%
:LOOP
if not defined PROJECT_TEMP goto :NEXT
for /F "tokens=1,* delims= " %%i in ("%PROJECT_TEMP%") do (
echo. %%i
set PROJECT_TEMP=%%j
)
goto :LOOP
:NEXT
So the main problem in the original code is the option string "tokens=*", which defines to pass all tokens to the variable %%i.
Type for /? for more details on all this.
Related
Just to be thorough, I'll state here my whole project and what I'm aiming at.
I intend to adapt a shell script to work in Windows cmd, as this is intended for people who are not going to have some sophisticate language available.
for g in $(curl -Ls https://api.chess.com/pub/player/hikaru/games/archives | jq -rc ".archives[]") ; do curl -Ls "$g" | jq -rc ".games[].pgn" ; done >> games.pgn
For some reason, Chess.com's API doesn't have a very important feature that Lichess' does, to export all games of a single player, so what I can do manually is to use https://api.chess.com/pub/player/hikaru/games/archives to export all available monthly archives and then hit the API for each one of them. (hikaru inside this will be a set variable, it's the nickname of the desired player to export).
The result for this command is something like
{"archives":["https://api.chess.com/pub/player/hikaru/games/2015/11","https://api.chess.com/pub/player/hikaru/games/2015/12","https://api.chess.com/pub/player/hikaru/games/2016/02","https://api.chess.com/pub/player/hikaru/games/2016/03","https://api.chess.com/pub/player/hikaru/games/2016/04","https://api.chess.com/pub/player/hikaru/games/2016/05"]}
to which I only have to append /pgn to get the desired result.
Obviously, cmd doesn't have jq available, so this involves "parsing" the string inside a batch file.
I figured if I just could replace every occurrence of " with a linebreak and echo the results, I could then use find (or findstr) to easily get a list of lines that only would need to be prefaced with curl and appended with /pgn to get my final result.
The big question is: how do I replace " with a linebreak in cmd? I found a few answers, but none of them seems to work with a special character, part of the problem is that I also didn't understand these answers enough to try and adapt them.
A second way of perhaps achieving the same result would be replacing [, ] and , with line breaks, but then I would also have to worry with deleting the final " to append /pgn, so if I'm able to do the former, it would be cleaner.
in batch/cmd, a for loop is used to process a list (separated by default delimiters like space, tab, comma). So just replace [ and ] with a space or comma, and you have a nice list to split. Finally, use find to filter the output to the relevant parts and you're done:
#Echo off
setlocal
set "string={"archives":["https://api.chess.com/pub/player/hikaru/games/2015/11","https://api.chess.com/pub/player/hikaru/games/2015/12","https://api.chess.com/pub/player/hikaru/games/2016/02","https://api.chess.com/pub/player/hikaru/games/2016/03","https://api.chess.com/pub/player/hikaru/games/2016/04","https://api.chess.com/pub/player/hikaru/games/2016/05"]}"
set "string=%string:[= %"
set "string=%string:]= %"
for %%a in (%string%) do echo %%~a|find "/"
Output:
https://api.chess.com/pub/player/hikaru/games/2015/11
https://api.chess.com/pub/player/hikaru/games/2015/12
https://api.chess.com/pub/player/hikaru/games/2016/02
https://api.chess.com/pub/player/hikaru/games/2016/03
https://api.chess.com/pub/player/hikaru/games/2016/04
https://api.chess.com/pub/player/hikaru/games/2016/05
(in case you wonder: the tilde in echo %%~a removes surrounding quotes)
Stephan's answer gave me the directions I needed to research more and build my own solution. This is not the final script to my project, but it does solve every problem presented in my original question:
#echo off
setLocal enabledelayedexpansion
for /f "delims=" %%a in (input.txt) do (
for %%b in (%%a) do (
set string=%%b
set "string=!string:[=,!"
set "string=!string:]=,!"
echo !string!>>replaced.txt
)
)
for /f "delims=" %%c in (replaced.txt) do (
for %%d in (%%c) do (
echo %%~d>>echo.txt
)
)
for /f %%e in (echo.txt) do echo curl %%~e/pgn|find ".">>list.txt
I basically run 3 sets of loops, the first one loads my input (this could not be done via set because there's a size limit, using a nested loop works around that) and replaces [ and ] for commas.
The second loop sorts again the output. This is done basically to trim unwanted characters from the first and last line.
The last loop generates a list of curl commands that will later be executed into a PGN file (which is a chess file).
This ends the scope of the question, but since my project wasn't that complex, I'll present it's final version, which improves on Compo's answer, in case someone else stumbles upon this question:
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Chess.com and Lichess API Scraper ::
:: Author: fabiorzfreitas ::
:: Extract all games from a player from Chess.com and Lichess ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This tool uses Chess.com and Lichess APIs to extract all games from a given player. ::
#echo off
setLocal enabledelayedexpansion
echo.
echo.
echo.
echo All input must be lowcase!
echo.
echo You can skip the input bellow by pressing Enter
echo.
echo.
echo.
set /p lichess="Input Lichess nickname and press Enter: "
set /p chess="Input Chess.com nickname and press Enter: "
echo.
:Lichess
if not defined lichess goto :Chess
curl https://lichess.org/api/games/user/%lichess% >> Games.pgn
:Chess
if not defined chess goto :End
(for /f "usebackq tokens=2 delims=[]" %%g in (`curl https://api.chess.com/pub/player/%chess%/games/archives`) do (
for %%h In (%%g) do curl "%%~h/pgn" >> Games.pgn
)
)
:End
exit
Based upon your own answer, it seems as if you could remove at least one of those steps by using the brackets [ and ], as delimiters.
You could also nest a for loop within another instead of having individual ones and writing to files.
Here it is as a single line batch-file:
#(For /F "UseBackQ Tokens=2 Delims=[]" %%G In ("input.txt") Do #For %%H In (%%G) Do #Echo curl.exe "%%~H/pgn") 1>"list.txt"
To do it directly in cmd:
(For /F "UseBackQ Tokens=2 Delims=[]" %G In ("input.txt") Do #For %H In (%G) Do #Echo curl.exe "%~H/pgn") 1>"list.txt"
I have a script that needs to extract a YouTube URL from a text file.
Here's what I have in the text file (output.txt):
---------- NUMBER11.TXT
<link itemprop="url" href="http://www.youtube.com/channel/UCnxGkOGNMqQEUMvroOWps6Q">
Note the text file has a line of empty space to start, which is annoying, and the URL is on line 3. Something that doesn't show up in the formatting for this site is the 11 spaces before the actual href starting as well. I'd like to separate it from the mass of other junk.
I've tried something like this:
set /p long= < output.txt
echo %long%
set short1=%long:^<link itemprop^="url" href^="=%
echo %short1% > o1.txt
I thought this would remove the selected text from the file, but I think this is a little over my head.
I'm getting the output.txt from firstly a curl of a youtube video page, and secondly from a find command here:
find "href=""http://www.youtube.com/channel/" %vd% > output.txt
Maybe I'm making this more complicated than it is?
Using batch-files to access files with special characters, like redirect, it can cause some problems, so it is not recommended, but I felt like posting an answer anyway, so given you exact example, here is one way. If your example is not as per your post, which I highly expect it to be, then this probably would not work.
#echo off
setlocal enabledelayedexpansion
for /f "usebackq delims=" %%i in ("output.txt") do for %%a in (%%i) do (
set "var=%%~a"
set "var=!var:>=!"
set "var=!var:"=!"
if "!var:~0,4!" == "http" echo !var!
)
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "filename1=%sourcedir%\q64572433.txt"
set "url="
FOR /f "tokens=4,5delims=>= " %%a IN (%filename1%) DO if "%%~a"=="href" set "url=%%~b"
echo URL=%url%
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances. The listing uses a setting that suits my system.
I used a file named q64572433.txt containing your data for my testing.
The for command tokenises each line of the file, using =, > and space as delimiters (the 3 characters between delims= and ")
On the line of interest, token 4 would be href and token 5 the url - and this is the only line where href is the fourth token. When that is detected, assign the 5th token (in %%b) to the variable, removing the quotes with ~ for good measure.
I would suggest you parse the results directly from your curl command instead of outputting them to a text file, and then using find against that output.
However, instead of using find.exe, I would suggest you use the following method using findstr.exe instead, to get the URL assigned to any line containing href= followed by "http: or "https and subsequently followed by youtube.com.
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
For /F Tokens^=*EOL^= %%G In (
'%__APPDIR__%findstr.exe /IR "href=\"http[s:].*youtube\.com" "output.txt"'
) Do (Set "Line=%%G" & SetLocal EnableDelayedExpansion
For /F Tokens^=2Delims^=^" %%H In ("!Line:*href=!") Do EndLocal & Echo %%H)
Pause
If you want the output stored as a variable, instead of Echoing it, change Echo %%H to Set "URL=%%H". You could then use %URL%, (or "%URL%" if you need it doublequoted), elsewhere in your script.
Preface
While writing a separate piece of code, I encountered a problem with question marks in for loops. As shown below, the question mark is not accessed in the for loop.
Batch file:
#echo off
for %%x in (the, quick, ?, brown, fox) do (
echo %%x
)
Output:
the
quick
brown
fox
This also does not work in the CMD (using %x instead of %%x), or when using "", [], ^, \, % or other common methods of character escaping.
Using a counter variable to determine the number of times the code within the parentheses was accessed only results in a total count of 4, meaning it is clearly not a problem with the echo command.
Question
Why doesn't a question mark work in a standard for loop, and how would I go about fixing it?
It's because ? will be expanded into a list of filenames one character long. The "naked" for is using that list as a list of filenames.
If you run the following commands, you'll see this in action:
c:\> echo xx >a
c:\> echo xx >b
c:\> for %i in (1, ?) do echo %x
1
a
b
If you look at Rob van der Woude's excellent scripting pages, you'll see that the for page has options for processing command output, numbers and files - it's not really suited for arbitrary strings.
One way to get around that is to provide your own for-like command as shown in the following example:
#echo off
setlocal enableextensions enabledelayedexpansion
rem Call the callback function for each argument.
set escapee=/
call :doFor :processEach 1 2 ? 4 5
echo.Escapee was %escapee%
rem Execute simple command for each argument.
call :doFor echo 6 7 ? 9 10
endlocal
goto :eof
:processEach
set escapee=%escapee%%1/
goto :eof
:doFor
setlocal
rem Get action.
set cbAction=%1
shift
:dfloop
rem Process each argument with callback or command.
if not "%1" == "" (
call %cbAction% %1
shift
goto :dfloop
)
endlocal&&set escapee=%escapee%
goto :eof
This provides a single functions which can handle both callbacks and simple commands. For more complex commands, provide a callback function and it will get called with each argument in turn. The callback function can be arbitrarily complex but keep in mind that, because it's operating within a setlocal, changes to environment variables cannot escape back to the caller.
As a way around this, it allows one variable, escapee, to escape the scope - you could also add more if needed.
For simple commands (like echo) where you just need the argument placed at the end, you do the same thing. It doesn't need a callback function but it's restricted to very simple scenarios.
Also keep in mind that, although this seems like a lot of code, the vast majority of it only needs to exist in one place. To use it, you simply need a one-liner like the sample:
call :doFor echo my hovercraft is full of eels
Also keep in mind that there may be other characters that do not fare well, even with this scheme. It solves the ? issue but others may still cause problems. I suspect that this would be an ideal opportunity to add PowerShell to your CV, for example, a command that's almost bash-like in it's elegance and zen-ness:
PShell> foreach ($item in #("1", "?", "3", "4")) { echo $item }
1
?
3
4
You could switch to FOR /F.
But FOR /F is used to process multiple lines to split them into tokens.
In your case you don't need multiple tokens, you need one loop per item.
That can be done by splitting the items with linefeeds.
I'm using # as item delimiter, but you are free to use any other character
#echo off
setlocal EnableDelayedExpansion
(set \n=^
%=EMPTY=%
)
set "itemList=the#quick#?#brown#fox"
for %%L in ("!\n!") DO (
FOR /F "delims=" %%x in ("!itemList:#=%%~L!") DO echo - %%x -
)
Output:
- the -
- quick -
- ? -
- brown -
- fox -
I've been coding with batch many years, and I'm suprised to realize this issue until now!
I found another way to deal with this problem. May be somebody prefers it, like me.
In my particularly case, I'm using the FOR LOOP to get some named arguments of the current function. This is what I did:
:SomeFunct
rem Replace ?
set "args=%*"
set "args=%args:?=`%"
rem Iterate args
for %%p in (%args%) do (
for /f "tokens=1,* delims=: " %%a in ("%%~p") do (
rem Get and store values
if /i "%%~a" equ "/a" set "argA=%%~b"
if /i "%%~a" equ "/b" set "argB=%%~b"
if /i "%%~a" equ "/c" set "argC=%%~b"
)
)
rem Restore ?
if defined argA set "argA=%argA:`=?%"
if defined argB set "argB=%argB:`=?%"
if defined argC set "argC=%argC:`=?%"
rem I use the args
rem ...
rem Return
goto:eof
I call the function like this:
rem Calling example
call:SomeFunct "/a:Is there" "/b:a question mark" "/c in the arguments?"
I feel like I'm going around in circles with FOR loop options.
I'm trying to take a string (output of a command) and split it on commas, then use each value to SET, e.g.
String: USER=Andy,IP=1.2.3.4,HOSTNAME=foobar,PORT=1234
So I want to split on comma and then literally use that variable in SET. I don't know ahead of time how many many variables there will be.
I've tried things like:
FOR %%L IN (%MYSTRING%) DO ECHO %%L
but that splits on the equals sign too so I end up with
USER
Andy
IP
1.2.3.4
etc
I just want to be able to do the following so I can SET USER=Andy etc, something like:
FOR %%L IN (%MYSTRING%) DO SET %%L
What option or flags am I missing?
The default delimiters for elements in plain FOR command (no /F option) are spaces, tab, commas, semicolons and equal signs, and there is no way to modify that, so you may use FOR /F command to solve this problem this way:
#echo off
set MYSTRING=USER=Andy,IP=1.2.3.4,HOSTNAME=foobar,PORT=1234
:nextVar
for /F "tokens=1* delims=," %%a in ("%MYSTRING%") do (
set %%a
set MYSTRING=%%b
)
if defined MYSTRING goto nextVar
echo USER=%USER%, IP=%IP%, HOSTNAME=%HOSTNAME%, PORT=%PORT%
Another way to solve this problem is first taking the variable name and then executing the assignment for each pair of values in a regular FOR command:
setlocal EnableDelayedExpansion
set MYSTRING=USER=Andy,IP=1.2.3.4,HOSTNAME=foobar,PORT=1234
set varName=
for %%a in (%MYSTRING%) do (
if not defined varName (
set varName=%%a
) else (
set !varName!=%%a
set varName=
)
)
echo USER=%USER%, IP=%IP%, HOSTNAME=%HOSTNAME%, PORT=%PORT%
EDIT 2023/01/20: New method added
I know this is a very old question. However, I can't resist the temptation to post a new very interesting method to solve this old problem:
#echo off
set MYSTRING=USER=Andy,IP=1.2.3.4,HOSTNAME=foobar,PORT=1234
set "%MYSTRING:,=" & set "%"
echo USER=%USER%, IP=%IP%, HOSTNAME=%HOSTNAME%, PORT=%PORT%
If you want to know where the magic is, remove the #echo off line, execute the program and carefully review the screen...
In case your input is something like HOSTNAME:PORT and you need to split into separate variables then you can use this
#echo off
set SERVER_HOST_PORT=10.0.2.15:8080
set SERVER_HOST_PORT=%SERVER_HOST_PORT::=,%
for /F "tokens=1* delims=," %%a in ("%SERVER_HOST_PORT%") do (
set SERVER_HOST=%%a
set SERVER_PORT=%%b
)
echo SERVER_HOST=%SERVER_HOST%
echo SERVER_PORT=%SERVER_PORT%
Over the past few years I've really found Stackoverflow to be very helpful, and decided to create an account - this is my first post.
Example situation - I have a fair few of these images, of different subjects -
AAA_BBB_randomDigits_front.jpg
AAA_BBB_randomDigits_left.jpg
AAA_BBB_randomDigits_right.jpg
ZZZ_EEE_randomDigits_front.jpg
ZZZ_EEE_randomDigits_left.jpg
ZZZ_EEE_randomDigits_right.jpg
I would like them to all be grouped up in folders as -
AAA_BBB_randomDigits\(contains left, front and right)
ZZZ_EEE_randomDigits\(contains left, front and right)
The code I currently have works -
#echo off
for /f "tokens=1-3 delims=_" %%a in ('dir /b /a-d *_*_*_*.*') do (
md "%%a_%%b_%%c" 2>nul
move "%%a_%%b_%%c*" "%%a_%%b_%%c"
)
pause
However, I would love it if someone could explain to me -
What's %%a?
What's dir /b /a-d and why do I need it?
Is it neccessary to have #echo off and pause?
Thanks guys, I really appreciate it.
For documentation, see commandname /? from the prompt.
dir /b /a-d filemask performs a directory listing /b specifies filenames only - no size, date, header or footer. The /a-d excludes directorynames.
You need it to provide the names to the for /f command.
for /f reads the "filename" in parentheses (it can be a real filename or a single-quoted command (like dir) or a double-quoted literal string) and assigns values to the metavariable (in this case, %%a) according to the specified options (the part in quotes directly following the /f).
The delims option specifies which set of characters is used for parsing the line of data arriving from the "file" specified. The line is then interpreted as a series of tokens, separated by delimiter-sequences. By default, delims is Space and Tab. It's common to turn delims off entirely using "...delims=" in which case, there is but one token (the entire line). Note that any characters between delims= and " are equally-ranking and case-sensitive - it is a set of delimiters which replaces Space and Tab, not a delimiter-string.
The tokens option specifies which tokens are selected, by number, starting at 1. The special token * means "the remainder of the line following the highest-number token specified (including any delimiter characters)". By default, tokens=1.
%%a is a metavariable. It is the variable that holds the first token number selected from the tokens= list. Each selected token number is assigned to the next metavariable name in alphabetical sequence, hence in your example, since you have tokens=1-3 then %%a is the first token, %%b the second and %%c the third. Metavariables are always one letter (some other characters are sometimes used - but numerics are definitely not allowed) and the name is case-sensitive (normally, batch is case-insensitive). %%a, %%A and %a% are all different variables. %a% and %A% are the same variable.
A metavariable is only valid within the for loop where it was created. When the for loop ends, the variable disappears.
#echo off simply turns off the command-echoing that batch would otherwise produce (show the command on the console, then execute it). It's used to reduce clutter on the display. When debugging a section of code, it's normal to set echo to on (echo on) and then off again (echo off) to show precisely what instructions are being executed. The # means "don't report this command to the console"
The pause simply waits until a response is received from the keyboard. It's used to allow the display to be read rather than simply continuing directly to the next instruction. It's often used in debugging and also to allow the result of a batch to be held for the user if the batch is executed by using point-click-and-giggle.