Enumerating the delims of a FOR /F statement - for-loop

I have a FOR /F statement which I want it to enumerate the delims the command is parsing. For example:
FOR /F "delims=," %%A IN ("This,is,a,comma,delimited,sentence") DO (
some command to enumerate delims
)
:OUT
I want it to account for each delimited item in that sentence. In this case it would output 6
EDIT: I know the long way would be to do a check on each one.. but I'm trying to avoid that method:
IF %%A == [] SET enum=0 && GOTO:OUT
IF %%B == [] SET enum=1 && GOTO:OUT
etc.

There is NO way to directly enumerate items in a FOR /F "delims=..." command. The usual way to do that is via a loop that count one item, eliminate it from the sentence and repeat while there was a counted item.
However, depending on the specific delimiter and the rest of characters in the sentence, you may use a FOR command (with no /F option) that will REPEAT its code with each item separated BY THE STANDARD BATCH DELIMITERS, that are comma, semicolon and equal-sign, besides spaces. In your particular example:
SET ENUM=0
FOR %%A IN (This,is,a,comma,delimited,sentence) DO SET /A ENUM+=1
directly count the number of comma-separated items. If the delimiter is a character other than comma, semicolon or equal-sign, a possible solution is a three steps method:
1- Replace spaces, comma, semicolon and equal-sign for another know character(s).
2- Replace the delimiter for any Batch standard delimiter (space, comma, etc).
3- Use a simple FOR to directly enumerate the items.

There IS a way to directly enumerate items in a FOR /F "delims=..." command.
You only need to insert some newlines into the string.
setlocal EnableDelayedExpansion
set LF=^
rem ** Two empty lines are required
set count=0
FOR /F "tokens=* delims=" %%a in ("item1!LF!item2!LF!item3") DO (
set /a count+=1
echo !count!: "%%a"
)
echo(
set "CSV=This,is,a,comma,delimited,sentence"
echo Or with comma separeted text: !CSV!
for %%L in ("!LF!") do set "CSV=!CSV:,=%%~L!"
set count=0
FOR /F "tokens=* delims=" %%a in ("!CSV!") DO (
set /a count+=1
echo !count!: "%%a"
)

Aacini's suggestion to use a simple FOR instead of FOR /F is a good solution, unless the string might contain wildcard characters * or ?. The ? character could be protected by search and replace, but there is no efficient way to replace * in batch.
If you run into the wildcard problem then you can revert to using FOR /F in a loop to parse one word at a time. Most people use GOTO to accomplish the loop because you have no way of knowing how many words you will find. But GOTO is relatively slow. You can achieve a significant performance boost by using an outer FOR loop (not FOR /L) with an arbitrarily large number of items. Within the body you can exit the loop with a GOTO whenever there are no more words. If the loop falls through without exhausting the words, you can use a GOTO to restart the loop. An outer loop with 100 items will only perform 1 GOTO per 100 parsed words.
#echo off
setlocal enableDelayedExpansion
set "str=This,is,a,comma,delimited,sentence"
set "L10=1 2 3 4 5 6 7 8 9 0"
:loop - The outer FOR loop can handle 100 words before a single GOTO is needed
for %%. in (%L10% %L10% %L10% %L10% %L10% %L10% %L10% %L10% %L10% %L10%) do (
for /f "tokens=1* delims=," %%A in ("!str!") do (
echo %%A
if "%%B" == "" goto :break
set "str=%%B"
)
)
goto :loop
:break
As with all FOR loops, you must worry about corruption of ! and ^ if delayed expansion is enabled when the %%A variable is expanded.
FOR /L should not be used for the outer loop because FOR /L always finishes counting all iterations, even if you use GOTO within the body.

Related

Windows Batch: Turning DelayedExpansion on/off inside a loop and preserving the value of variables while doing so? [duplicate]

This question already has answers here:
Make an environment variable survive ENDLOCAL
(8 answers)
Closed 3 years ago.
I'm trying to use a FOR loop to read the lines in a text file, but I also need to keep track of some variables and evaluate them. The easiest way to do that is by enabling DelyaedExpansion. Actually, it seems to be the ONLY way as everything else I've tried in relation to variables fails miserably if I don't use it. Unfortunately, this means that if any of the lines of text in the file contain exclamation points, they will be stripped out.
I thought I had found a solution by reading a line of text and putting it into a variable, THEN enabling DelayedExpansion, doing the variable operations, and finally using ENDLOCAL & SET VARIABLE=%VARIABLE% to preserve the value. Unfortunately that doesn't seem to work if the ENDLOCAL statement is inside a loop.
For example;
echo off
for /F "delims=" %%F in (test.txt) do (
set Line=%%F
setlocal enabledelayedexpansion
set /a Count=Count+1
echo !Count! - !Line!
endlocal & set Count=%Count%
)
echo Total: %Count%
Each time the loop repeats, the value of "Count" is reset to zero.
If I move the SETLOCAL command before the FOR command, it will strip any "!" from the text, which is unacceptable.
Please note: The example above is only a small part of a much larger script that does many things with the variables inside the loop. I have boiled the problem down to the bare minimum to make it easy to understand. I need to preserve "!" in text read from a file while also being able to perform multiple variable operations within each loop.
So I either need a way to read text from a file, one line at a time, with DeleyedExpansion enabled AND preserve any "!" in the text, or preserve the value of variables that are defined within the SETLOCAL/ENDLOCAL commands within a loop.
With Help from dbenham and his answer here, There is a Solution that exists for this Scenario.
The key, as Dave has Shown, is in Setting the variables PRIOR to using SetlocalEnableDelayedExpansion so that ! is preserved.
#echo off
Set "count=0"
For /F "delims=" %%F in (test.txt) do (
Call :LineParse "%%~F"
)
REM The Below Loop demonstrates Preservation of the Values
Setlocal EnableDelayedExpansion
For /L %%a in (1,1,!count!) DO (
ECHO(!line[%%a]!
)
Endlocal
pause
exit
:LineParse
Set /a count+=1
Set "Line[%count%]=%~1"
Setlocal EnableDelayedExpansion
ECHO(!Line[%count%]!
ECHO(Total: !count!
(
ENDLOCAL
)
GOTO :EOF
There are still a few characters that will not be parsed as desired with this Method, noted in test.txt:
test.txt
Safe Characters: ! > * & ` ' . ] [ ~ # # : , ; ~ } { ) ( / \ ? > < = - _ + $ |
problem Characters: ^ "" %%
problem examples:
line disappears " from single doublequote
but not "" from escaped doublequote
%% will not display unless escaped. % unescaped Percent Symbols will attempt to expand %
caret doubles ^ ^^ ^^^
Don't need to complicate...
Just replace:
echo/ to set /p
setlocal enabledelayedexpansion to cmd /v /c
#echo off
for /F "delims=" %%F in ('type test.txt')do set /a "Count+=1+0" && (
(echo/ & cmd /v/s/c "set/p "'=!Count! - %%F"<nul")>>".\newfile.txt")
cmd /v /c echo/ Total: !Count! && call set "Count="<nul && goto :EOF

Delete first few lines from a text file with Windows batch scripting

Preferably a one-liner, how could I delete a range of lines at the beginning from a large (3MB+) text file in a timely fashion (few seconds max). I've seen solutions using for /f along with findstr, but the for loop made it extremely slow, and the tool more cannot handle larger files without hanging.
#echo off &setlocal
set "testing.txt=%~1"
(for /f "delims=" %%i in ('findstr /n "^" "testing.txt"') do (
set "line=%%i"
for /f "delims=:" %%a in ("%%i") do set "row=%%a"
setlocal enabledelayedexpansion
set "line=!line:*:=!"
if !row! gtr 100 echo(!line!
endlocal
))>output.txt
Here is an attempt. It is incredibly slow. Any recommendations would be appreciated.
This is the fastest way to eliminate the first lines in a large file. Is written as one-liner, as you requested:
#echo off
< testing.txt ( (for /L %%i in (1,1,100) do set /P "=") & findstr "^" ) > output.txt
However, be aware that this method can only manage lines up to 1023 characters long because it uses set /P command to read and discard not desired lines...
For a description of this method, see this answer.
As far as I got you want to skip the first 100 lines of your huge text file and want to return the rest.
Well, when I look to your code the first thing I see is you have two for /F loops nested, which might slow things down.
The inner loop just splits off a preceding line number that is separated by a colon from the rest.
For this purpose you could (mis-)use set /A, which is capable of converting a string into a numeric value, when you use its implicit variable expansion (hence no % or !); this process stops when the first non-numeric character is encountered, which is the : in our situation. So just replace the inner for /F loop with:
set /A "row=line"
This will for sure speed things up a bit. However, regard that this limits the input text file to 2^31 - 1 lines. Note, that the number of characters/bytes per line is still limited to about 8190.
By the way, you do not have to do set "line=!line:*:=!" as a separate step, just remove this and replace echo(!line! with echo(!line:*:=!.
If you do not need to preserve empty lines, the whole approach is as simple as this:
#echo off
for /F usebackq^ skip^=100^ delims^=^ eol^= %%i in ("testing.txt") do echo(%%i
This does not limit the file size, but the line lengths must not exceed 8191 characters/bytes.

How to sort lines of a text file containing version numbers in format major.minor.build.revision numerical?

I have a txt file with values like this:
3.6.4.2
3.6.5.1
3.6.5.10
3.6.5.11
3.6.5.12
3.6.5.13
3.6.5.2
3.6.7.1
3.6.7.10
3.6.7.11
3.6.7.2
3.6.7.3
I need to write a batch script and return a sorted output. The problem is with last column, numbers .10 and .11 should go after .3 and so. I need the "latest version" to be on the bottom, which in this case is 3.6.7.11
In Linux I used "sort -t"." -k1n,1 -k2n,2 -k3n,3 -k4n,4" but I can't get it working with batch script.
Also I am not allowed to use Cygwin or PowerShell for some reasons.
In my batch code I am so far trying only various versions of this but nothing is working for me:
sort /+n versions.txt
The output used in this question is simply
sort versions.txt
It looks like that command sort is doing it correctly until I don't have 2 digits number used.
This is a common problem in Batch files. All sorting methods use a string comparison, where "10" comes before "2", so it is necessary to insert left zeros in the numbers less than 10. The Batch file below do that, but instead of generate a new file with the fixed numbers, it uses they to create an array that will be automatically sorted. After that, the array elements are shown in its natural (sorted) order.
EDIT: I modified the code in order to manage two digits numbers in the four parts.
#echo off
setlocal EnableDelayedExpansion
for /F "tokens=1-4 delims=." %%a in (input.txt) do (
rem Patch the four numbers as a two digits ones
set /A "a=100+%%a, b=100+%%b, c=100+%%c, d=100+%%d"
rem Store line in the proper array element
set "line[!a:~1!!b:~1!!c:~1!!d:~1!]=%%a.%%b.%%c.%%d"
)
rem Show array elements
for /F "tokens=2 delims==" %%a in ('set line[') do echo %%a
Output:
3.6.4.2
3.6.5.1
3.6.5.2
3.6.5.10
3.6.5.11
3.6.5.12
3.6.5.13
3.6.7.1
3.6.7.2
3.6.7.3
3.6.7.10
3.6.7.11
Based on your example this will work. If you should somehow end up with examples like 3.6.5.02 and 3.6.5.2, then this code will not work.
#echo off
setlocal EnableDelayedExpansion
for /F "tokens=1-4 delims=. " %%G in (FILE.TXT) do (
set N=0%%J
set SORT[%%G%%H%%I!N:~-2!]=%%G.%%H.%%I.%%J
)
for /F "tokens=2 delims==" %%N in ('set SORT[') do echo %%N
pause
Here is my solution working with 2 temporary files which works also if one of the other 3 version numbers becomes ever greater than 9.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
set "VersionsFile=versions.txt"
rem Delete all temporary files perhaps existing from a previous
rem run if user of batch file has broken last batch processing.
if exist "%TEMP%\%~n0_?.tmp" del "%TEMP%\%~n0_?.tmp"
rem Insert a leading 0 before each number in version string if the
rem number is smaller than 10. And insert additionally a period at
rem start of each line. The new lines are written to a temporary file.
for /F "useback tokens=1-4 delims=." %%A in ("%VersionsFile%") do (
if %%A LSS 10 ( set "Line=.0%%A." ) else ( set "Line=.%%A." )
if %%B LSS 10 ( set "Line=!Line!0%%B." ) else ( set "Line=!Line!%%B." )
if %%C LSS 10 ( set "Line=!Line!0%%C." ) else ( set "Line=!Line!%%C." )
if %%D LSS 10 ( set "Line=!Line!0%%D" ) else ( set "Line=!Line!%%D" )
echo !Line!>>"%TEMP%\%~n0_1.tmp"
)
rem Sort this temporary file with output written to one more temporary file.
rem The output could be also printed to __stdout__ and directly processed.
%SystemRoot%\System32\sort.exe "%TEMP%\%~n0_1.tmp" /O "%TEMP%\%~n0_2.tmp"
rem Delete the versions file before creating new with sorted lines.
del "%VersionsFile%"
rem Read sorted lines, remove all leading zeros after a period and also
rem the period inserted at start of each line making it easier to remove
rem all leading zeros. The lines are written back to the versions file.
for /F "useback delims=" %%L in ("%TEMP%\%~n0_2.tmp") do (
set "Line=%%L"
set "Line=!Line:.0=.!"
set "Line=!Line:~1!"
echo !Line!>>"%VersionsFile%"
)
rem Finally delete the two temporary files used by this batch file.
del "%TEMP%\%~n0_?.tmp" >nul
endlocal
The first temporary file with unsorted lines contains for input example:
.03.06.04.02
.03.06.05.01
.03.06.05.10
.03.06.05.11
.03.06.05.12
.03.06.05.13
.03.06.05.02
.03.06.07.01
.03.06.07.10
.03.06.07.11
.03.06.07.02
.03.06.07.03
The second temporary file with the sorted lines contains for input example:
.03.06.04.02
.03.06.05.01
.03.06.05.02
.03.06.05.10
.03.06.05.11
.03.06.05.12
.03.06.05.13
.03.06.07.01
.03.06.07.02
.03.06.07.03
.03.06.07.10
.03.06.07.11
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /? ... explains %~n0 (name of batch file without path and file extension)
del /?
echo /?
endlocal /?
for /?
if /?
rem /?
set /?
setlocal /?
sort /?
Easiest solution would be to invoke PowerShell and treat the version numbers as actual System.Version objects. That way the Major, Minor, Build, and Revision segments will be treated as integers and sorted accordingly. You can call this from a batch script:
powershell "(gc textfile.txt | %%{[version]$_} | sort) -split ' '"
That's it. Easy one-liner. If doing it from the cmd prompt, replace the double %% with a single %. Here's a breakdown of the command:
Get the following as a string:
Get the contents of textfile.txt
For each line, cast the data as a System.Version object.
Sort as versions
The string will be a single line separated by spaces. Split on the spaces.
Output is as follows:
3.6.4.2
3.6.5.1
3.6.5.2
3.6.5.10
3.6.5.11
3.6.5.12
3.6.5.13
3.6.7.1
3.6.7.2
3.6.7.3
3.6.7.10
3.6.7.11
Partial credit should go to this question and accepted answer.
In pure batch scripting, you could use the following code snippet:
#echo off
setlocal EnableExtensions EnableDelayedExpansion
> "versions.tmp" (
for /F "usebackq tokens=1,2,3,4 delims=." %%I in ("versions.txt") do (
set "ITEM1=000%%I" & set "ITEM2=000%%J" & set "ITEM3=000%%K" & set "ITEM4=000%%L"
echo !ITEM1:~-4!.!ITEM2:~-4!.!ITEM3:~-4!.!ITEM4:~-4!^|%%I.%%J.%%K.%%L
)
)
< "versions.tmp" (
for /F "tokens=2 delims=|" %%S in ('sort') do (
echo %%S
)
)
del /Q "versions.tmp"
endlocal
exit /B
This creates a temporary file, which contains the original line, prefixed with padded version numbers and a separtor |. Padded numbers means that each component is padded with leading zeros to four digits. Here is an example based on youe sample data:
0003.0006.0004.0002|3.6.4.2
0003.0006.0005.0001|3.6.5.1
0003.0006.0005.0010|3.6.5.10
0003.0006.0005.0011|3.6.5.11
0003.0006.0005.0012|3.6.5.12
0003.0006.0005.0013|3.6.5.13
0003.0006.0005.0002|3.6.5.2
0003.0006.0007.0001|3.6.7.1
0003.0006.0007.0010|3.6.7.10
0003.0006.0007.0011|3.6.7.11
0003.0006.0007.0002|3.6.7.2
0003.0006.0007.0003|3.6.7.3
This temporary file is then passed over to sort which does a purely alphabetic sorting. Since the numbers are padded, the sort order equals the true alphanumeric order. Here is the sorting result using the above example:
0003.0006.0004.0002|3.6.4.2
0003.0006.0005.0001|3.6.5.1
0003.0006.0005.0002|3.6.5.2
0003.0006.0005.0010|3.6.5.10
0003.0006.0005.0011|3.6.5.11
0003.0006.0005.0012|3.6.5.12
0003.0006.0005.0013|3.6.5.13
0003.0006.0007.0001|3.6.7.1
0003.0006.0007.0002|3.6.7.2
0003.0006.0007.0003|3.6.7.3
0003.0006.0007.0010|3.6.7.10
0003.0006.0007.0011|3.6.7.11
Finally, if you want to return the latest version number only, echo %%S by set "LVER=%%S" and place echo !LVER! after the closing ) of the second for /F loop.
Update:
Here is a solution that does not produce any temporary files, but uses a pipe | instead. Since a pipe creates new cmd instances for both left and right sides, and due to the fact that the (console) outputs are built in tiny bits and that there are multiple arithmetic operations done, it is rather slow:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
(
for /F "usebackq tokens=1,2,3,4 delims=." %%I in ("versions.txt") do #(
set /A "10000+%%I" & echo( ^| set /P "=."
set /A "10000+%%J" & echo( ^| set /P "=."
set /A "10000+%%K" & echo( ^| set /P "=."
set /A "10000+%%L" & echo(
)
) | (
for /F "tokens=1,2,3,4 delims=." %%S in ('sort') do #(
set /A "%%S-10000" & echo( ^| set /P "=."
set /A "%%T-10000" & echo( ^| set /P "=."
set /A "%%U-10000" & echo( ^| set /P "=."
set /A "%%V-10000" & echo(
)
)
endlocal
exit /B
Left Side of the Pipe:
Instead of the substring expansion syntax like in the above approach using a temporary file, I add 10000 to every component of the version numbers (similar to Aacini's answer) in order to avoid delayed expansion, because this is not enabled in either new cmd instance. To output the resulting values, I make use of the fact that either of the for /F loops are running in cmd context rather than in batch context, where set /A outputs the result to STDOUT. set /A does not terminate its output with a line-break, so I use set /P to append a . after each but the last item, which in turn does not append a line-break. For the last item I append a line-break using a blank echo.
Right Side of the Pipe:
The sorting is again accomplished by the sort command, whose output is parsed by for /F. Here the previously added value 10000 is subtracted from each component to retrieve the original numbers. For outputting the result to the console, the same technique is used as for the other side of the pipe.
Piped Data:
The data passed over by the pipe looks like this (relying on the example of the question once again):
10003.10006.10004.10002
10003.10006.10005.10001
10003.10006.10005.10010
10003.10006.10005.10011
10003.10006.10005.10012
10003.10006.10005.10013
10003.10006.10005.10002
10003.10006.10007.10001
10003.10006.10007.10010
10003.10006.10007.10011
10003.10006.10007.10002
10003.10006.10007.10003

Batch File - Insert Line into file

I'm trying to insert a line into a file using the following code (from Write batch variable into specific line in a text file)
#echo off
setlocal enableextensions enabledelayedexpansion
set inputfile=variables.txt
set tempfile=%random%-%random%.tmp
copy /y nul %tempfile%
set line=0
for /f "delims=" %%l in (%inputfile%) do (
set /a line+=1
if !line!==4 (
echo WORDS YOU REPLACE IT WITH>>%tempfile%
) else (
echo %%l>>%tempfile%
)
)
del %inputfile%
ren %tempfile% %inputfile%
endlocal
My problem is the file has comment lines (which start with semicolons) which need to be kept
; directory during network startup. This statement must indicate a local disc
; drive on your PC and not a network disc drive.
LOCALDRIVE=C:\TEMP;
; PANELISATION PART/NET NAMING CONVENTION
; When jobs are panelised, parts/nets are renamed for each panel step by
When I run the batch file, it ignores the semicolon lines, So I only get:
LOCALDRIVE=C:\TEMP;
What do I need to do to keep the semicolon lines?
The EOL option determines what lines are to be ignored. The default value is a semicolon. If you know a character that can never appear in the first position of a line, then you can simply set EOL to that character. For example, if you know a line can't start with |, then you could use
for /f "eol=| delims=" %%l in (%inputfile%) do ...
There is an awkward syntax that disables EOL completely, and also disables DELIMS:
for /f delims^=^ eol^= %%l in (%inputfil%) do ...
Note that FOR /F always discards empty lines, so either of the above would result in:
; directory during network startup. This statement must indicate a local disc
; drive on your PC and not a network disc drive.
LOCALDRIVE=C:\TEMP;
; PANELISATION PART/NET NAMING CONVENTION
; When jobs are panelised, parts/nets are renamed for each panel step by
A trick is used if you want to preserve empty lines. Use FIND or FINDSTR to insert the line number before each line, and then use expansion find/replace to remove the line number. Now you know the line never begins with ;, so you can ignore the EOL option.
for /f "delims=" %%L in ('findstr /n "^" "%inputfile%"') do (
set "ln=%%L"
set "ln=!ln:*:=!"
REM You now have the original line, do whatever needs to be done here
)
But all of the above have a potential problem in that you have delayed expansion enabled when you expand the FOR variable, which means that any content containing ! will be corrupted. To solve this you must toggle delayed expansion on and off within the loop:
setlocal disableDelayedExpansion
...
for /f "delims=" %%L in (findstr /n "^" "%inputfile%") do (
set "ln=%%L"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
REM You now have the original line with ! preserved, do whatever needs done here
endlocal
)
Also, when ECHOing an empty line, it will print out ECHO is off unless you do something like
echo(!ln!
It takes time to open and position the write cursor to the end every time you use >> within the loop. It is faster to enclose the entire operation in one set of parentheses and redirect once. Also, you can replace the DEL and REN with a single MOVE command.
Here is a final robust script:
#echo off
setlocal disableDelayedExpansion
set "inputfile=variables.txt"
set line=0
>"%inputfile%.new" (
for /f "delims=" %%L in (findstr /n "^" "%inputfile%") do (
set "txt=%%L"
set /a line+=1
setlocal enableDelayedExpansion
set "txt=!txt:*:=!"
if !line! equ 4 (
echo New line content here
) else (
echo(!txt!
)
endlocal
)
)
move /y "%inputfile%.new" "%inputfile%" >nul
endlocal
That is an awful lot of work for such a simple task, and it requires a lot of arcane knowledge.
There is a much quicker hack that works as long as
your first 4 lines do not exceed 1021 bytes
none of your first 3 lines have trailing control characters that need to be preserved
the remaining lines do not have <tab> characters that must be preserved (MORE converts <tab> into a string of spaces.
#echo off
setlocal enableDelayedExpansion
set "inputfile=variables.txt"
>"%inputfile%.new" (
<"%inputfile%" (
for /l %%N in (1 1 3) do (
set "ln="
set /p "ln="
echo(!ln!
)
)
echo New line content here
more +4 "%inputfile%"
)
move /y "%inputfile%.new" "%inputfile%"
That is still a lot of work and arcane knowledge.
I would use my JREPL.BAT utility
Batch is really a terrible tool for text processing. That is why I developed JREPL.BAT to manipulate text using regular expressions. It is a hybrid JScript/batch script that runs natively on any Windows machine from XP onward. It is extremely versatile, robust, and fast.
A minimal amount of code is required to solve your problem with JREPL. Your problem doesn't really require the regular expression capabilities.
jrepl "^" "" /jendln "if (ln==4) $txt='New content here'" /f "variables.txt" /o -
If used within a batch script, then you must use call jrepl ... because JREPL.BAT is also a batch script.
By default, the FOR command treats ; as the end-of-line character, so all those lines that start with ; are being ignored.
Add eol= to your FOR command, like this:
for /f "eol= delims=" %%l in (%inputfile%) do (
It looks like you're echoing just the line delimiter, not the whole line:
echo %%l>>%tempfile%
I'm rusty on ms-dos scripts, so I can't give you more than that.

Spurious spaces in output of for loop

I'm trying to use a batch file to convert a file containing sql code into a single environment variable for use with the MSSQL utility bcp.
For example, if InFile.sql contains
-- This is a simple statement
SELECT *
FROM table
The output of ECHO %query% should be
SELECT * FROM people
The code below works for me most of the time
SETLOCAL=ENABLEDELAYEDEXPANSION
:: Replace VarOld with VarNew
FOR /f "delims=" %%a IN ('TYPE InFile.sql') DO ( SET line=%%a & ECHO !line:table=people! >> TmpFile1 )
:: Remove comment lines starting with '-' and remove newline characters
(FOR /f "eol=- delims=" %%a in (TmpFile1) DO SET/p=%%a ) <nul >TmpFile2
:: Create variable 'Query'
FOR /f "delims=" %%a IN ('TYPE TmpFile2') DO SET query=%%a
however, the first FOR loop adds 3 space characters at the end of each line and the second FOR loop adds another space character so the result is
SELECT * FROM people
I could cope with the additional spaces (although the purist in me wasn't happy!) until I had to use it with a long SQL query and multiple replacement steps - every line in the file was having 12 space characters added. The additional spaces are enough to make the resulting query around 8300 characters long - too much for Windows' 8196 character limit for a batch file line.
Can anybody see how I can remove these spurious spaces?
Using tokens=* in a for loop should trim whitespace as you're capturing a line of infile.sql. Here's a proof of concept, echoing %query% contained within quotation marks to illustrate the trimming:
#echo off
setlocal enabledelayedexpansion
set query=
if "%~1"=="" goto usage
if not exist "%~1" goto usage
for /f "usebackq eol=- tokens=*" %%I in ("%~f1") do (
set "sub=%%I"
set query=!query! !sub:table=people!
)
:: strip the leading space from %query%
echo "%query:~1%"
goto :EOF
:usage
echo Usage: %~nx0 sqlfile
Example output:
C:\Users\me\Desktop>type infile.sql
-- This is a simple statement
SELECT *
FROM table
C:\Users\me\Desktop>test.bat infile.sql
"SELECT * FROM people"
The fundamental issue is that trailing spaces ARE significant in SET statements and ECHO statements before the redirectors.
In your code, you need to remove the spaces after %%a and people! in the first FOR Thus:
FOR /f "delims=" %%a IN ('TYPE InFile.sql') DO (SET line=%%a&ECHO !line:table=people!>> TmpFile1)
The next problem is a little more subtle. In
(FOR /f "eol=- delims=" %%a in (TmpFile1) DO SET/p=%%a ) <nul >TmpFile2
the space following /p=%%a is REQUIRED because it provides the separator between the text taken from the lines when building TmpFile2 - and that leads to a superfluous trailing space. Try replacing that space with a Q for instance - just for testing.
Hence, you need to delete the final space from QUERY after it's been constructed in your final FOR
SET query=%query:~0,-1%

Resources