String validation with multiple levels required using CMD command - validation

I have three requirements I need to meet when validating a password. I have figure out with the help of others how to verify that the password is at least seven characters long and the user name is not part of the password.
My last requirement is to check to see if a string contains characters from three of the following four groups:
English uppercase characters (A through Z)
English lowercase characters (a through z)
Base 10 digits (0 through 9)
Non-alphabetic characters (for example, !, $, #, %)
For example (Hou$e or House1) would pass but (House, house or hou$e) would fail
The call to ":checkRequirement3" is where I would like to make this finial check. The password is valid if all three requirements are meet.
#echo off
setlocal
set /p userName=Username:
set /p userPassword=Password:
call :strlen result userPassword
call :checkRequirement1
call :checkRequirement2
call :checkRequirement3
ECHO Finished
Pause
:strlen <resultVar> <stringVar>
REM THIS DETERMINES THE LENGTH OF THE PASSWORD
(
setlocal EnableDelayedExpansion
set "s=!%~2!#"
set "len=0"
for %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if "!s:~%%P,1!" NEQ "" (
set /a "len+=%%P"
set "s=!s:~%%P!"
)
)
)
(
endlocal
set "%~1=%len%"
exit /b
)
:checkRequirement1
REM THIS CHECKS IF PASSWORD IS AT LEAST 7 CHARACTERS LONG
(
if %result% GEQ 7 (
exit /b
)else (
GOTO passwordFail
)
:checkRequirement2
REM THIS CHECKS IF THE USER NAME IS INCLUDED IN THE PASSWORD
setlocal enabledelayedexpansion
set replacedUsername=!userPassword:%userName%=!
if not !replacedUsername!==%userPassword% (
GOTO passwordFail
)else (
exit /b
)
:checkRequirement3
REM THIS CHECKS IF THE PASSWORD CONTAINS CHARACTERS FROM 3 OF THE FOLLOWING GROUPS
REM English uppercase characters (A through Z)
REM English lowercase characters (a through z)
REM Base 10 digits (0 through 9)
REM Non-alphabetic characters (for example, !, $, #, %)
ECHO This Requirement not finished
Pause
exit /b
:passwordFail
ECHO Password Failed Requirement
PAUSE
exit /b
)

#ECHO OFF
SETLOCAL
FOR %%t IN (Hou$e House1 House, house hou$e hou%%%%se hou^^se) DO CALL :test "%%t"
GOTO :EOF
:test
SET /a count=0
>"q43120516.txt" ECHO %~1
TYPE "q43120516.txt"
FOR %%s IN ("[ABCDEFGHIJKLMNOPQRSTUVWXYZ]"
"[abcdefghijklmnopqrstuvwxyz]"
"[!##&$%%^]"
"[0123456789]") DO FINDSTR /r %%s "q43120516.txt">nul&IF NOT ERRORLEVEL 1 SET /a count+=1
ECHO found %count% groups IN %~1
DEL "q43120516.txt"
GOTO :EOF
naturally, the name of the temporary file "q43120516.txt" is irrelevant.
A few little things to note here:
Certain characters, like % and ^ which have a special meaning to cmd need to be doubled - sometimes quadrupled.
Yes, I'm aware that in theory you could use echo %~1|. Try it.
Yes, I'm aware that in theory you could use [A-Z]. Try it.

This will correctly verify that the password is using 3 out of the 4 groups.
setlocal enabledelayedexpansion
SET /P userPassword=UserPassword:
SET /P userPassword=UserPassword=%userPassword:&=^&%
SET /a count=0
echo %userPassword:&=^&% | findstr /R /C:"[ABCDEFGHIJKLMNOPQRSTUVWXYZ]">null&if not errorlevel 1 SET /a count+=1
echo %userPassword:&=^&% | findstr /R /C:"[abcdefghijklmnopqrstuvwxyz]">null&if not errorlevel 1 SET /a count+=1
echo %userPassword:&=^&% | findstr /R /C:"[!##&$%%^]">null&if not errorlevel 1 SET /a count+=1
echo %userPassword:&=^&% | findstr /R /C:"[0123456789]">null&if not errorlevel 1 SET /a count+=1
If %count% GEQ 3 (
echo Password Meets requirement
exit /b
) else (
goto passwordFail
)

Related

Return a value from a called batch file label

I have two labels in my batch file. The initial label MAIN shall stay in control, so it Calls the second label, which ends with exit /b.
My script's Main label Calls the other, passing it arguments, which will be used to search strings wothin a text file.
When returning to the Calling label, it slways receives an empty return string.
I think this has something to do with the variable expansion in a loop. Who knows?
Here is the Script:
#echo off
SETLOCAL ENABLEDELAYEDEXPANSION
:MAIN
call :getReturnValue "1234 0815 4321 12815" "readBackVal"
if !errorlevel! equ 0 (
echo readback=!readBackVal!
echo readback=%readBackVal%
)
pause
exit /b 0
REM Function, which checks if the give return value is in a specific textfile (line for line check)
:getReturnValue
set "myExpectedValueList=%~1"
set "retval=%~2"
set "file=textexample.txt"
for %%i in (%myExpectedValueList%) do (
for /f "tokens=*" %%a in (%file%) do (
echo %%a|findstr /r "^.*%%i$"
)
if !errorlevel! equ 0 (
(endlocal
set /a "%retval%=%%i")
)
exit /b 0
)
)
exit /b 1
Here is the sample textfile textexample.txt:
Setup returns with errorcode=0815
Here is the answer i looked for:
Hi, first i want to inform that i made some changes due to the Answer of
#OJBakker. This changes are listed at the bottom of the script.
The problem was to return a value from a called function/label to the calling function/label. The stich here is, that the magic
is done in the (endlocal...) section of the called function/label -> means the return of the variable.
Before the endlocal command is executed, the compiler replaces the variables in this section by their values and afterwards executes the command´s from left to right. Means following:
First, the compiler sees following:
(endlocal
if "%retval%" neq "" (call set /a %retval%=%%i)
)
Second, the compiler replaces the variables by their values:
(endlocal
if "readBackVal" neq "" (set /a "readBackVal"=1815)
)
Third: This command is executed
(endlocal
if "readBackVal" neq "" (set /a "readBackVal"=1815)
)
Now here is my complete script (i also fixed some other problems with it which i commented at the bottom of the script
#echo off
SETLOCAL ENABLEDELAYEDEXPANSION
:MAIN
setlocal
call :getReturnValue "1234 1815 4321 12815" "readBackVal"
if "!errorlevel!" equ "0" (
echo readback=!readBackVal!
)
pause
exit /b 0
REM Function, which checks if the give return value is in a specific textfile (line for line check)
:getReturnValue
setlocal
set "myExpectedValueList=%~1"
set "retval=%~2"
set "file=textexample.txt"
for %%i in (%myExpectedValueList%) do (
for /f "tokens=*" %%a in (%file%) do (
echo %%a|findstr /r "^.*%%i$" >NUL
)
if "!errorlevel!" equ "0" (
(endlocal
if "%retval%" neq "" (set /a %retval%=%%i)
)
exit /b 0
)
)
exit /b 1
REM Changes to initial posting:
REM Added "setlocal" keyword to the function "getReturnValue"
REM Corrected an invalid paranthesis in the (endlocal...) section
REM Changed the file "textexample.txt" -> 0815 to 1815 to remove leading zero (findstr. Problem),
REM Added check, if parameter "retval" has been passed to the called function e.g. is not empty
REM FINAL -> applied double variable expansion (call set /a ...) to return the value proper
REM to the :MAIN function.
#echo off
SETLOCAL ENABLEDELAYEDEXPANSION
:MAIN
call :getReturnValue "1234 0815 4321 12815" "readBackVal"
if %errorlevel% equ 0 (echo readback=%readBackVal%)
pause
endlocal
exit /b 0
REM Function, which checks if the give return value is in a specific textfile (line for line check)
:getReturnValue
set "myExpectedValueList=%~1"
set "retval=%~2"
set "file=textexample.txt"
for %%i in (%myExpectedValueList%) do (
for /f "tokens=*" %%a in (%file%) do (
echo %%a| >con 2>&1 findstr /r "^.*%%i$"
if !errorlevel! equ 0 (
set /a "%retval%=%%i"
exit /b 0
)
)
)
exit /b 1
rem changes:
rem endlocal moved to main.
rem check for errorlevel moved to within the commandblock of the inner for-loop.
rem 'exit /b 0' moved to within the if. This exit line stopped the for after the first item.
rem redirection added to findstr command. Now the output shows the remaining problem.
rem Invalid number. Numeric constants are either decimal (17), hexadecimal (0x11), or octal (021).
rem Findstr really does not like the value 0815, especially the starting zero.
rem I am not sure how to change the regexp so findstr won't barf at the leading zero.
rem Maybe someone else can solve this remaining problem.

Confusion on "for /l in %%i ..." always renders "0"

I'm a little confused on why in this code, %rem% always comes back as 0 (even when tested with prime numbers). Can someone please help me? Thanks :D
:PRIME
cls
echo What number would you like to check?
set /p num=
set num2=%num%-1
for /l %%i in (2 1 %num2%) do (
set /a rem=%num% %% %%i
)
if %rem% equ 0 goto NOT_PRIME
goto YES_PRIME
:YES_PRIME
echo %num% is a prime number.
goto AGAIN_PRIME
:NOT_PRIME
echo %num% is not a prime number.
goto AGAIN_PRIME
:AGAIN_PRIME
echo Would you like to check another number? (y/n)
set /p ans=
if '%ans%'=='y' goto PRIME
if '%ans%'=='n' goto START
This is only a portion of the code. The problem is that every number that I test, I get "%num% is not a prime number."
There are two problems in this section:
set num2=%num%-1
for /l %%i in (2 1 %num2%) do (
set /a rem=%num% %% %%i
)
if %rem% equ 0 goto NOT_PRIME
First, you need to use set /a to do calculations on a variable, so it should be:
set /a num2=%num%-1
Second, your for loop runs through all your calculations correctly, but your if line ends up checking only the results of the very last calculation. You need to enable delayed expansion and then include the if statement inside the for loop, like this:
for /l %%i in (2 1 %num2%) do (
set /a rem=%num% %% %%i
if !rem! equ 0 goto NOT_PRIME
)

Preserve spaces with <Nul set /p

I'm trying to create a typewriter effect but SET /P is stripping the spaces since I'm on win7. Is there another way or a trick I'm missing?
Here is the code I've come up with so far:
#echo off
setlocal enabledelayedexpansion
set text=This is a test
call :strlen len text
for /l %%b in (0,1,%len%) do (
set /a T=!random! %% 3
ping -n !T! 127.0.0.1>nul
<nul set /p=!text:~%%b,1!
)
exit /b
:strlen <resultVar> <stringVar>
setlocal EnableDelayedExpansion
set "s=!%~2!#" & set "len=0"
for %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if "!s:~%%P,1!" NEQ "" ( set /a "len+=%%P" & set "s=!s:~%%P!" ))
endlocal &set "%~1=%len%"
exit /b
If output is strictly for the screen, and never redirected to a file, then there is a simple solution using a backspace character. Prefix each SET /P output with some printable character, followed by a backspace, followed by your desired string.
#echo off
setlocal enabledelayedexpansion
:: Define BS to contain backspace character
for /f %%A in ('"prompt $H&for %%B in (1) do rem"') do set "BS=%%A"
set text=This is a test
call :strlen len text
for /l %%b in (0,1,%len%) do (
set /a T=!random! %% 3
ping -n !T! 127.0.0.1>nul
<nul set /p=.!BS!!text:~%%b,1!
)
exit /b
:strlen <resultVar> <stringVar>
setlocal EnableDelayedExpansion
set "s=!%~2!#" & set "len=0"
for %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if "!s:~%%P,1!" NEQ "" ( set /a "len+=%%P" & set "s=!s:~%%P!" ))
endlocal &set "%~1=%len%"
exit /b
The problem with the above is the dot and backspace characters will appear in any output that is redirected to a file. Another possible problem is the technique fails if the cursor is at the last character position on the line because a linefeed will automatically be issued after the dot, and then the backspace cannot back up to the prior line.
If you must redirect output to a file, or if you want a robust solution that works in all cases, then jeb has a very clever solution at Output text without linefeed, even with leading space or =.

LastIndexOf in Windows batch

I need to implement a function in a Windows batch script to get the LastIndexOf a character into a given string.
For example: Given the following string, I need to get the last index of character '/':
/name1/name2/name3
^
So I need to get the value:
12
Joey's solution works, but the character to find is hard coded, and it is relatively slow.
Here is a parametized function that is fast and can find any character (except nul) within the string. I pass the name of variables containing the string and the character instead of string literals so that the function easily supports all characters.
#echo off
setlocal
set "test=/name1/name2/name3"
set "char=/"
::1st test simply prints the result
call :lastIndexOf test char
::2nd test stores the result in a variable
call :lastIndexOf test char rtn
echo rtn=%rtn%
exit /b
:lastIndexOf strVar charVar [rtnVar]
setlocal enableDelayedExpansion
:: Get the string values
set "lastIndexOf.char=!%~2!"
set "str=!%~1!"
set "chr=!lastIndexOf.char:~0,1!"
:: Determine the length of str - adapted from function found at:
:: http://www.dostips.com/DtCodeCmdLib.php#Function.strLen
set "str2=.!str!"
set "len=0"
for /L %%A in (12,-1,0) do (
set /a "len|=1<<%%A"
for %%B in (!len!) do if "!str2:~%%B,1!"=="" set /a "len&=~1<<%%A"
)
:: Find the last occurrance of chr in str
for /l %%N in (%len% -1 0) do if "!str:~%%N,1!" equ "!chr!" (
set rtn=%%N
goto :break
)
set rtn=-1
:break - Return the result if 3rd arg specified, else print the result
( endlocal
if "%~3" neq "" (set %~3=%rtn%) else echo %rtn%
)
exit /b
It wouldn't take much modification to create a more generic :indexOf function that takes an additional argument specifying which occurance to find. A negative number could specify to search in reverse. So 1 could be the 1st, 2 the 2nd, -1 the last, -2 penultimate, etc.
(Note: I'm assuming Windows batch files because, frankly, I have only seen a single question asking for an actual DOS batch file here so far. Most people simply misattribute “DOS” to anything that has a window of gray-on-black monospaced text without knowing what they're actually talking of.)
Just loop through it, updating the index as you go:
#echo off
setlocal enabledelayedexpansion
set S=/name1/name2/name3
set I=0
set L=-1
:l
if "!S:~%I%,1!"=="" goto ld
if "!S:~%I%,1!"=="/" set L=%I%
set /a I+=1
goto l
:ld
echo %L%
I know this question is a bit old now, but I needed a function that could find the location of a substring (of any length) within a string, and adapted dbenham's solution for my purposes. This function also works with individual characters within a string, as asked for in the original question, and can search for specific instances (as suggested by dbenham).
To use this function, the actual strings must be passed. Dbenham does note that this supports fewer characters than passing the actual variables, but I find that this variant is more reuseable (especially with pipes).
The third argument takes the instance that should be found, with negative numbers specifying to search from the end. The index returned is the offset from the start of the string to the first character in the substring.
#ECHO off
SET search_string=sub
CALL :strIndex "The testing subjects subjects to testing." "%search_string%" -2
ECHO %ERRORLEVEL%
PAUSE
EXIT
:strIndex string substring [instance]
REM Using adaptation of strLen function found at http://www.dostips.com/DtCodeCmdLib.php#Function.strLen
SETLOCAL ENABLEDELAYEDEXPANSION
SETLOCAL ENABLEEXTENSIONS
IF "%~2" EQU "" SET Index=-1 & GOTO strIndex_end
IF "%~3" EQU "" (SET Instance=1) ELSE (SET Instance=%~3)
SET Index=-1
SET String=%~1
SET "str=A%~1"
SET "String_Length=0"
FOR /L %%A IN (12,-1,0) DO (
SET /a "String_Length|=1<<%%A"
FOR %%B IN (!String_Length!) DO IF "!str:~%%B,1!"=="" SET /a "String_Length&=~1<<%%A"
)
SET "sub=A%~2"
SET "Substring_Length=0"
FOR /L %%A IN (12,-1,0) DO (
SET /a "Substring_Length|=1<<%%A"
FOR %%B IN (!Substring_Length!) DO IF "!sub:~%%B,1!"=="" SET /a "Substring_Length&=~1<<%%A"
)
IF %Substring_Length% GTR %String_Length% GOTO strIndex_end
SET /A Searches=%String_Length%-%Substring_Length%
IF %Instance% GTR 0 (
FOR /L %%n IN (0,1,%Searches%) DO (
CALL SET StringSegment=%%String:~%%n,!Substring_Length!%%
IF "%~2" EQU "!StringSegment!" SET /A Instance-=1
IF !Instance! EQU 0 SET Index=%%n & GOTO strIndex_end
)) ELSE (
FOR /L %%n IN (%Searches%,-1,0) DO (
CALL SET StringSegment=%%String:~%%n,!Substring_Length!%%
IF "%~2" EQU "!StringSegment!" SET /A Instance+=1
IF !Instance! EQU 0 SET Index=%%n & GOTO strIndex_end
))
:strIndex_end
EXIT /B %Index%

Windows batch script to parse CSV file and output a text file

I've seen a response on another page (Help in writing a batch script to parse CSV file and output a text file) - brilliant code BTW:
#ECHO OFF
IF "%~1"=="" GOTO :EOF
SET "filename=%~1"
SET fcount=0
SET linenum=0
FOR /F "usebackq tokens=1-10 delims=," %%a IN ("%filename%") DO ^
CALL :process "%%a" "%%b" "%%c" "%%d" "%%e" "%%f" "%%g" "%%h" "%%i" "%%j"
GOTO :EOF
:trim
SET "tmp=%~1"
:trimlead
IF NOT "%tmp:~0,1%"==" " GOTO :EOF
SET "tmp=%tmp:~1%"
GOTO trimlead
:process
SET /A linenum+=1
IF "%linenum%"=="1" GOTO picknames
SET ind=0
:display
IF "%fcount%"=="%ind%" (ECHO.&GOTO :EOF)
SET /A ind+=1
CALL :trim %1
SETLOCAL ENABLEDELAYEDEXPANSION
ECHO !f%ind%!!tmp!
ENDLOCAL
SHIFT
GOTO display
:picknames
IF %1=="" GOTO :EOF
CALL :trim %1
SET /a fcount+=1
SET "f%fcount%=%tmp%"
SHIFT
GOTO picknames
It works brilliantly for an example csv file I made in the format:
Header,Name,Place
one,two,three
four,five,six
However the actual file I want to change comprises of 64 fields - so I altered the tokens=1-10 to tokens=1-64 and increased the %%a etc right up to 64 variables (the last being called %%BL for example). Now, however, when I run the batch on my 'big' csv file (with the 64 tokens) nothing happens. No errors (good) but no output! (bad). If anyone can help that would be fantastic... am soooo close to getting the whole app working if I can just nail this last bit! Or if anyone has some example code that will do similar for an indefinite number of tokens... Ultimately I want to make a string which will be something like:
field7,field12,field15,field18
Important update - I don't think Windows batch is a good option for your needs because a single FOR /F cannot parse more than 31 tokens. See the bottom of the Addendum below for an explanation.
However, it is possible to do what you want with batch. This ugly code will give you access to all 64 tokens.
for /f "usebackq tokens=1-29* delims=," %%A in ("%filename%") do (
for /f "tokens=1-26* delims=," %%a in ("%%^") do (
for /f "tokens=1-9 delims=," %%1 in ("%%{") do (
rem Tokens 1-26 are in variables %%A - %%Z
rem Token 27 is in %%[
rem Token 28 is in %%\
rem Token 29 is in %%]
rem Tokens 30-55 are in %%a - %%z
rem Tokens 56-64 are in %%1 - %%9
)
)
)
The addendum provides important info on how the above works.
If you only need a few of the tokens spread out amongst the 64 on the line, then the solution is marginally easier in that you might be able to avoid using crazy characters as FOR variables. But there is still careful bookkeeping to be done.
For example, the following will give you access to tokens 5, 27, 46 and 64
for /f "usebackq tokens=5,27,30* delims=," %%A in ("%filename%") do (
for /f "tokens=16,30* delims=," %%E in ("%%D") do (
for /f "tokens=4 delims=," %%H in ("%%G") do (
rem Token 5 is in %%A
rem Token 27 is in %%B
rem Token 46 is in %%E
rem Token 64 is in %%H
)
)
)
April 2016 Update - Based on investigative work by DosTips users Aacini, penpen, and aGerman, I have developed a relatively easy method to simultaneously access thousands of tokens using FOR /F. The work is part of this DosTips thread. The actual code can be found in these 3 posts:
Work with a fixed number of columns
Work with varying numbers of columns
Dynamically choose which tokens to expand within the DO clause
Original Answer
FOR variables are limited to a single character, so your %%BL strategy can't work. The variables are case sensitive. According to Microsoft you are limited to capturing 26 tokens within one FOR statement, but it is possible to get more if you use more than just alpha. Its a pain because you need an ASCII table to figure out which characters go where. FOR does not allow just any character however, and the maximum number of tokens that a single FOR /F can assign is 31 +1. Any attempt to parse and assign more than 31 will quietly fail, as you have discovered.
Thankfully, I don't think you need that many tokens. You simply specify which tokens you want with the TOKENS option.
for /f "usebackq tokens=7,12,15,18 delims=," %%A in ("%filename%") do echo %%A,%%B,%%C,%%D
will give you your 7th, 12th, 15th and 18th tokens.
Addendum
April 2016 Update A couple weeks ago I learned that the following rules (written 6 years ago) are code page dependent. The data below has been verified for code pages 437 and 850. More importantly, the FOR variable sequence of extended ASCII characters 128-254 does not match the byte code value, and varies tremendously by code page. It turns out the FOR /F variable mapping is based on the underlying UTF-(16?) code point. So the extended ASCII characters are of limited use when used with FOR /F. See the thread at http://www.dostips.com/forum/viewtopic.php?f=3&t=7703 for more information.
I performed some tests, and can report the following (updated in response to jeb's comment):
Most characters can be used as a FOR variable, including extended ASCII 128-254. But some characters cannot be used to define a variable in the first part of a FOR statement, but can be used in the DO clause. A few can't be used for either. Some have no restrictions, but require special syntax.
The following is a summary of characters that have restrictions or require special syntax. Note that text within angle brackets like <space> represents a single character.
Dec Hex Character Define Access
0 0x00 <nul> No No
09 0x09 <tab> No %%^<tab> or "%%<tab>"
10 0x0A <LF> No %%^<CR><LF><CR><LF> or %%^<LF><LF>
11 0x0B <VT> No %%<VT>
12 0x0C <FF> No %%<FF>
13 0x0D <CR> No No
26 0x1A <SUB> %%%VAR% %%%VAR% (%VAR% must be defined as <SUB>)
32 0x20 <space> No %%^<space> or "%%<space>"
34 0x22 " %%^" %%" or %%^"
36 0x24 $ %%$ %%$ works, but %%~$ does not
37 0x25 % %%%% %%~%%
38 0x26 & %%^& %%^& or "%%&"
41 0x29 ) %%^) %%^) or "%%)"
44 0x2C , No %%^, or "%%,"
59 0x3B ; No %%^; or "%%;"
60 0x3C < %%^< %%^< or "%%<"
61 0x3D = No %%^= or "%%="
62 0x3E > %%^> %%^> or "%%>"
94 0x5E ^ %%^^ %%^^ or "%%^"
124 0x7C | %%^| %%^| or "%%|"
126 0x7E ~ %%~ %%~~ (%%~ may crash CMD.EXE if at end of line)
255 0xFF <NB space> No No
Special characters like ^ < > | & must be either escaped or quoted. For example, the following works:
for /f %%^< in ("OK") do echo "%%<" %%^<
Some characters cannot be used to define a FOR variable. For example, the following gives a syntax error:
for /f %%^= in ("No can do") do echo anything
But %%= can be implicitly defined by using the TOKENS option, and the value accessed in the DO clause like so:
for /f "tokens=1-3" %%^< in ("A B C") do echo %%^< %%^= %%^>
The % is odd - You can define a FOR variable using %%%%. But The value cannot be accessed unless you use the ~ modifier. This means enclosing quotes cannot be preserved.
for /f "usebackq tokens=1,2" %%%% in ('"A"') do echo %%%% %%~%%
The above yields %% A
The ~ is a potentially dangerous FOR variable. If you attempt to access the variable using %%~ at the end of a line, you can get unpredictable results, and may even crash CMD.EXE! The only reliable way to access it without restrictions is to use %%~~, which of course strips any enclosing quotes.
for /f %%~ in ("A") do echo This can crash because its the end of line: %%~
for /f %%~ in ("A") do echo But this (%%~) should be safe
for /f %%~ in ("A") do echo This works even at end of line: %%~~
The <SUB> (0x1A) character is special because <SUB> literals embedded within batch scripts are read as linefeeds (<LF>). In order to use <SUB> as a FOR variable, the value must be somehow stored within an environment variable, and then %%%VAR% will work for both definition and access.
As already stated, a single FOR /F can parse and assign a maximum of 31 tokens. For example:
#echo off
setlocal enableDelayedExpansion
set "str="
for /l %%n in (1 1 35) do set "str=!str! %%n"
for /f "tokens=1-31" %%A in ("!str!") do echo A=%%A _=%%_
The above yields A=1 _=31 Note - tokens 2-30 work just fine, I just wanted a small example
Any attempt to parse and assign more than 31 tokens will silently fail without setting ERRORLEVEL.
#echo off
setlocal enableDelayedExpansion
set "str="
for /l %%n in (1 1 35) do set "str=!str! %%n"
for /f "tokens=1-32" %%A in ("!str!") do echo this example fails entirely
You can parse and assign up to 31 tokens and assign the remainder to another token as follows:
#echo off
setlocal enableDelayedExpansion
set "str="
for /l %%0 in (1 1 35) do set "str=!str! %%n"
for /f "tokens=1-31*" %%# in ("!str!") do echo #=%%A ^^=%%^^ _=%%_
The above yields #=1 ^=31 _=32 33 34 35
And now for the really bad news. A single FOR /F can never parse more than 31 tokens, as I learned when I looked at Number of tokens limit in a FOR command in a Windows batch script
#echo off
setlocal enableDelayedExpansion
set "str="
for /l %%n in (1 1 35) do set "str=!str! %%n"
for /f "tokens=1,31,32" %%A in ("!str!") do echo A=%%A B=%%B C=%%C
The very unfortunate output is A=1 B=31 C=%C
My answer is comprised of two parts. The first one is a new answer I posted in help-in-writing-a-batch-script-to-parse-csv-file-and-output-a-text-file question that have not any limit in the number of fields.
The second part is a modification to that answer that allows to select which fields will be extracted from the csv file by additional parameters placed after the file name. The modified code is in UPPERCASE LETTERS.
#echo off
setlocal EnableDelayedExpansion
rem Create heading array:
set /P headingRow=< %1
set i=0
for %%h in (%headingRow%) do (
set /A i+=1
set heading[!i!]=%%~h
)
REM SAVE FILE NAME AND CREATE TARGET ELEMENTS ARRAY:
SET FILENAME=%1
IF "%2" == "" (FOR /L %%J IN (1,1,%i%) DO SET TARGET[%%J]=%%J) & GOTO CONTINUE
SET J=0
:NEXTTARGET
SHIFT
IF "%1" == "" GOTO CONTINUE
SET /A J+=1
SET TARGET[%J%]=%1
GOTO NEXTTARGET
:CONTINUE
rem Process the file:
call :ProcessFile < %FILENAME%
exit /B
:ProcessFile
set /P line=
:nextLine
set line=:EOF
set /P line=
if "!line!" == ":EOF" goto :EOF
set i=0
SET J=1
for %%e in (%line%) do (
set /A i+=1
FOR %%J IN (!J!) DO SET TARGET=!TARGET[%%J]!
IF !i! == !TARGET! (
for %%i in (!i!) do echo !heading[%%i]!%%~e
SET /A J+=1
)
)
goto nextLine
exit /B
For example:
EXTRACTCSVFIELDS THEFILE.CSV 7 12 15 18
EDIT A simpler method
Below is a new version that is both simpler and easier to understand because it use a list of target elements instead of an array:
#echo off
setlocal EnableDelayedExpansion
rem Create heading array:
set /P headingRow=< %1
set i=0
for %%h in (%headingRow%) do (
set /A i+=1
set heading[!i!]=%%~h
)
REM CREATE TARGET ELEMENTS LIST:
IF "%2" == "" (
SET TARGETLIST=
FOR /L %%J IN (1,1,%i%) DO SET TARGETLIST=!TARGETLIST! %%J
) ELSE (
SET TARGETLIST=%*
SET TARGETLIST=!TARGETLIST:* =!
)
rem Process the file:
call :ProcessFile < %1
exit /B
:ProcessFile
set /P line=
:nextLine
set line=:EOF
set /P line=
if "!line!" == ":EOF" goto :EOF
set i=0
for %%e in (%line%) do (
set /A i+=1
for %%i IN (!i!) DO (
IF "!TARGETLIST:%%i=!" NEQ "!TARGETLIST!" (
echo !heading[%%i]!%%~e
)
)
)
goto nextLine
exit /B
Also, this version does not require the desired fields be given in order.
EDIT
Oops! The for parameters stuff distracted my attention, so I was not aware of your last request:
"Ultimately I want to make a string which will be something like:
field7,field12,field15,field18"
Just modify the last part of the program to do that:
:ProcessFile
set /P line=
:nextLine
set line=:EOF
set /P line=
if "!line!" == ":EOF" goto :EOF
set i=0
set resultString=
for %%e in (%line%) do (
set /A i+=1
for %%i IN (!i!) DO (
IF "!TARGETLIST:%%i=!" NEQ "!TARGETLIST!" (
set resultString=!resultString!%%~e,
)
)
)
set resultString=%resultString:~0,-1%
echo Process here the "%resultString%"
goto nextLine
exit /B
You may also remove the creation of the heading array, because you want NOT the headings! ;)
Using %%# and %%` (not documented here) as start variables the max you can get is 71:
#echo off
for /f "tokens=1-31* delims=," %%# in ("%filename%") do (
echo:
echo 1=%%#
echo 2=%%A
echo 3=%%B
echo 4=%%C
echo 5=%%D
echo 6=%%E
echo 7=%%F
echo 8=%%G
echo 9=%%H
echo 10=%%I
echo 11=%%J
echo 12=%%K
echo 13=%%L
echo 14=%%M
echo 15=%%N
echo 16=%%O
echo 17=%%P
echo 18=%%Q
echo 19=%%R
echo 20=%%S
echo 21=%%T
echo 22=%%U
echo 23=%%V
echo 24=%%W
echo 25=%%X
echo 26=%%Y
echo 27=%%Z
echo 28=%%[
echo 29=%%\
echo 30=%%]
echo 31=%%^^
for /F "tokens=1-30* delims=," %%` in ("%%_") do (
echo 32=%%`
echo 33=%%a
echo 34=%%b
echo 35=%%c
echo 36=%%d
echo 37=%%e
echo 38=%%f
echo 39=%%g
echo 40=%%h
echo 41=%%i
echo 42=%%j
echo 43=%%k
echo 44=%%l
echo 45=%%m
echo 46=%%n
echo 47=%%o
echo 48=%%p
echo 49=%%q
echo 50=%%r
echo 51=%%s
echo 52=%%t
echo 53=%%u
echo 54=%%v
echo 55=%%w
echo 56=%%x
echo 57=%%y
echo 58=%%z
echo 59=%%{
echo 60=%%^|
echo 61=%%}
for /F "tokens=1-9* delims=," %%0 in ("%%~") do (
echo 62=%%0
echo 63=%%1
echo 64=%%2
echo 65=%%3
echo 66=%%4
echo 67=%%5
echo 68=%%6
echo 69=%%7
echo 70=%%8
echo 71=%%9
)
)
)
When I read this problem again and the solution proposed in the most-voted answer, I thought that a much simpler way to make good use of a series of nested FOR /F commands could be developed. I started to write such a method, that would allowed to use 127 additional tokens placing they in the ASCII 128-254 characters range. However, when my program was completed I discovered that the ASCII characters in the "natural" 128..254 order could not be used for this purpose...
Then, a group of people were interested in this problem and they made a series of discoveries and developments that culminated in a method that allows to use many tokens (more than 43,000!) in a series of nested FOR /F commands. You may read a detailed description of the research and development involved in this discovery at this DosTips topic.
Finally, I used the new method to modify my program, so it now allows the processing of up to 4094 simultaneous tokens (from a text file with long lines), but in a simple way. My application consists in a Batch file, called MakeForTokens.bat, that you may run with the number of desired tokens in the parameter. For example:
MakeForTokens.bat 64
The program generates a Batch file, called ForTokens.bat, that contain all the code necessary to manage such an amount of simultaneous tokens, including examples of how to process a file. In this way, the users just needs to insert their own file names and desired tokens in order to get a working program.
In this particular case, this would be the final ForTokens.bat file that solve the problem as stated in this question, after most descriptive comments were deleted:
#echo off & setlocal EnableDelayedExpansion & set "$numTokens=65"
Rem/For Step 1: Define the series of auxiliary variables that will be used as FOR tokens.
call :DefineForTokens
Rem/For Step 2: Define an auxiliary variable that will contain the desired tokens when it is %expanded%.
call :ExpandTokensString "tokens=7,12,15,18"
Rem/For Step 3: Define the variable with the "delims" value that will be used in the nested FOR's.
set "delims=delims=,"
Rem/For Step 4: Create the macro that contain the nested FOR's.
call :CreateNestedFors
Rem/For Step 5: This is the main FOR /F command that process the file.
for /F "usebackq tokens=1-31* %delims%" %%%$1% in ("filename.txt") do %NestedFors% (
Rem/For Step 6: Process the tokens.
Rem/For To just show they, use the "tokens" variable defined above:
echo %tokens%
Rem/For You may also process individual tokens via another FOR /F command:
for /F "tokens=1-%tokens.len%" %%a in ("%tokens%") do (
echo Field #7: %%a
echo Field #12: %%b
echo Field #15: %%c
echo Field #18: %%d
)
)
goto :EOF
Support subroutines. You must not modify any code below this line.
:DefineForTokens
for /F "tokens=2 delims=:." %%p in ('chcp') do set /A "_cp=%%p, _pages=($numTokens/256+1)*2"
set "_hex= 0 1 2 3 4 5 6 7 8 9 A B C D E F"
call set "_pages=%%_hex:~0,%_pages%%%"
if %$numTokens% gtr 2048 echo Creating FOR tokens variables, please wait . . .
(
echo FF FE
for %%P in (%_pages%) do for %%A in (%_hex%) do for %%B in (%_hex%) do echo %%A%%B 3%%P 0D 00 0A 00
) > "%temp%\forTokens.hex.txt"
certutil.exe -decodehex -f "%temp%\forTokens.hex.txt" "%temp%\forTokens.utf-16le.bom.txt" >NUL
chcp 65001 >NUL
type "%temp%\forTokens.utf-16le.bom.txt" > "%temp%\forTokens.utf8.txt"
(for /L %%N in (0,1,%$numTokens%) do set /P "$%%N=") < "%temp%\forTokens.utf8.txt"
chcp %_cp% >NUL
del "%temp%\forTokens.*.txt"
for %%v in (_cp _hex _pages) do set "%%v="
exit /B
:CreateNestedFors
setlocal EnableDelayedExpansion
set /A "numTokens=$numTokens-1, mod=numTokens%%31, i=numTokens/31, lim=31"
if %mod% equ 0 set "mod=31"
set "NestedFors="
for /L %%i in (32,31,%numTokens%) do (
if !i! equ 1 set "lim=!mod!"
set "NestedFors=!NestedFors! for /F "tokens=1-!lim!* %delims%" %%!$%%i! in ("%%!$%%i!") do"
set /A "i-=1"
)
for /F "delims=" %%a in ("!NestedFors!") do endlocal & set "NestedFors=%%a"
exit /B
:ExpandTokensString variable=tokens definitions ...
setlocal EnableDelayedExpansion
set "var=" & set "tokens=" & set "len=0"
if "%~2" equ "" (set "params=%~1") else set "params=%*"
for %%a in (!params!) do (
if not defined var (
set "var=%%a"
) else for /F "tokens=1-3 delims=-+" %%i in ("%%a") do (
if "%%j" equ "" (
if %%i lss %$numTokens% set "tokens=!tokens! %%!$%%i!" & set /A len+=1
) else (
if "%%k" equ "" (set "k=1") else set "k=%%k"
if %%i leq %%j (
for /L %%n in (%%i,!k!,%%j) do if %%n lss %$numTokens% set "tokens=!tokens! %%!$%%n!" & set /A len+=1
) else (
for /L %%n in (%%i,-!k!,%%j) do if %%n lss %$numTokens% set "tokens=!tokens! %%!$%%n!" & set /A len+=1
)
)
)
)
endlocal & set "%var%=%tokens%" & set "%var%.len=%len%"
exit /B
You may download the MakeForTokens.bat application from this site.

Resources