Is it possible to use a piped stdin stream inside a batch file?
I want to be able to redirect the output of one command into my batch file process.bat list so:
C:\>someOtherProgram.exe | process.bat
My first attempt looked like:
echo OFF
setlocal
:again
set /p inputLine=""
echo.%inputLine%
if not (%inputLine%)==() goto again
endlocal
:End
When I test it with type testFile.txt | process.bat it prints out the first line repeatedly.
Is there another way?
set /p doesn't work with pipes, it takes one (randomly) line from the input.
But you can use more inside of an for-loop.
#echo off
setlocal
for /F "tokens=*" %%a in ('more') do (
echo #%%a
)
But this fails with lines beginning with a semicolon (as the FOR-LOOP-standard of eol is ;).
And it can't read empty lines.
But with findstr you can solve this too, it prefix each line with the linenumber, so you never get empty lines.
And then the prefix is removed to the first colon.
#echo off
setlocal DisableDelayedExpansion
for /F "tokens=*" %%a in ('findstr /n "^"') do (
set "line=%%a"
setlocal EnableDelayedExpansion
set "line=!line:*:=!"
echo(!line!
endlocal
)
Alternatively, on some environments (like WinRE) that don't include findstr, an alternative with find.exe might suffice. find will accept a null search string "", and allows search inversion. This would allow something like this:
#echo off
setlocal DisableDelayedExpansion
for /F "tokens=*" %%a in ('find /v ""') do (
...
The set "line=!line:*:=!" syntax is:
set requires one parameter that is a=b.
If a contains a space or something, you'll have to use the quotation marks around this parameter. Here I don't see any
!line:*:=!
For this syntax, you can type 'set /?' to see the official description on using variables.
!var! is like %var%, to get the value. But !var! means delayed expansion.
line var name
the first : variable modification mark.
**:= **:=(empty), replace the string in the variable's value matches "*:"(virtually from the string start to first : occurence) with (empty), i.e. delete the substring from start to first colon.
FOR /F "tokens=1* delims=]" %%A IN ('FIND /N /V ""') DO (
> CON ECHO.%%B
>> %File% ECHO.%%B
)
Source here: http://www.robvanderwoude.com/unixports.php#TEE
Alternatively, on some environments (like WinRE) that don't include findstr, an alternative with find.exe might suffice. find will accept a null search string "", and allows search inversion. This would allow something like this:
#echo off
setlocal DisableDelayedExpansion
for /F "tokens=*" %%a in ('find /v ""') do (
set "line=%%a"
echo(!line!
)
Related
I have one .txt file with text and some empty space lines. While doing some operations in .txt file by using for loop, found originally contained empty space lines are gone.
Example :
I have a temp.txt file with below input :
#
Require
#
#Rel task
#
HOST_ID=
ACTIVATE_ID
#
#
After doing some operations using FOR Loop I am getting result in same .txt file like below :
#
Require
#
#Rel task
#
HOST_ID=56767\Ether
ACTIVATE_ID
#
#
So, I don't want to ignore those empty space lines.
I tried to use usebacq keyword in for loop, but not working.
I tried each and possible way to do this, but failed everytime.
Here, is my code :
#echo off
setlocal enableextensions disabledelayedexpansion
set rem=56767\Ether
set "textFile=temp.txt"
for /f "delims=" %%i in ('type "%textFile%" ^& break ^> "%textFile%" ') do (
set "line=%%i"
setlocal enabledelayedexpansion
if !line!==HOSTID= (
set "foo=!line:HOSTID=HOSTID=%rem%!"
set "line=!foo:~0,-1!"
)
>>"%textFile%" echo(!line!
endlocal
)
:EXIT
pause
In above code, I am doing to append some extra string in front of provided string in same .txt file.
Here's an alternative method, the idea being that you don't need delayed expansion, and, (for safety), you're not trying to read from and write to the same file.
What it does is to first determine if your required string exists in the file, if it doesn't the script closes, if it does it sets a variable to its line number. Then the file is parsed outputting the contents of each line untouched except for the line matching the saved variable, where it appends %rem% to it.
When the script ends your original file should exist along side it with a .bak extension.
#Echo Off
SetLocal EnableExtensions
Set "rem=56767\Ether"
Set "textFile=temp.txt"
Set "lineNum="
For /F Delims^=: %%G In ('^""%__AppDir__%findstr.exe" /INR "\<HOST_ID=\>" "%textFile%" 2^>NUL^"')Do Set "lineNum=%%G"
If Not Defined lineNum GoTo :EOF
For %%G In ("%textFile%")Do Set "_=%%~nG.bak" & Copy /Y "%%G" "%%~nG.bak">NUL
(For /F "Tokens=1*Delims=:" %%G In ('^""%__AppDir__%findstr.exe" /N "^" "%_%"^"')Do If %%G NEq %lineNum% (Echo=%%H)Else Echo %%H%rem%)>"%textFile%"
Pause
I am not sure if your file has a space before HOST_ID= like it currently does, but both with/without space has been added in this example:
#echo off
setlocal enableextensions disabledelayedexpansion
set "rem=56767\Ether"
set "textFile=temp.txt"
for /f "tokens=1,* delims=[]" %%i in ('type "%textFile%" ^| find /v /n "" ^& break^>"%textFile%"') do (
set "line=%%j"
setlocal enabledelayedexpansion
if "!line!" == "HOST_ID=" set "line=!line!%rem%"
if "!line!" == " HOST_ID=" set "line=!line!%rem%"
echo(!line!
endlocal
)
:EXIT
pause
Delimiters are [] because we add line numbers which you do not see as [n] before each entry by using find /V /N "". This way we retain the empty lines.
This DOS batch script is stripping out the blank lines and not showing the blank lines in the file even though I am using the TYPE.exe command to convert the file to make sure the file is ASCII so that the FIND command is compatible with the file. Can anyone tell me how to make this script include blank lines?
#ECHO off
FOR /F "USEBACKQ tokens=*" %%A IN (`TYPE.exe "build.properties" ^| FIND.exe /V ""`) DO (
ECHO --%%A--
)
pause
That is the designed behavior of FOR /F - it never returns blank lines. The work around is to use FIND or FINDSTR to prefix the line with the line number. If you can guarantee no lines start with the line number delimiter, then you simply set the appropriate delimiter and keep tokens 1* but use only the 2nd token.
::preserve blank lines using FIND, assume no line starts with ]
::long lines are truncated
for /f "tokens=1* delims=]" %%A in ('type "file.txt" ^| find /n /v ""') do echo %%B
::preserve blank lines using FINDSTR, assume no line starts with :
::long lines > 8191 bytes are lost
for /f "tokens=1* delims=:" %%A in ('type "file.txt" ^| findstr /n "^"') do echo %%B
::FINDSTR variant that preserves long lines
type "file.txt" > "file.txt.tmp"
for /f "tokens=1* delims=:" %%A in ('findstr /n "^" "file.txt.tmp"') do echo %%B
del "file.txt.tmp"
I prefer FINDSTR - it is more reliable. For example, FIND can truncate long lines - FINDSTR does not as long as it reads directly from a file. FINDSTR does drop long lines when reading from stdin via pipe or redirection.
If the file may contain lines that start with the delimiter, then you need to preserve the entire line with the line number prefix, and then use search and replace to remove the line prefix. You probably want delayed expansion off when transferring the %%A to an environment variable, otherwise any ! will be corrupted. But later within the loop you need delayed expansion to do the search and replace.
::preserve blank lines using FIND, even if a line may start with ]
::long lines are truncated
for /f "delims=" %%A in ('type "file.txt" ^| find /n /v ""') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*]=!"
echo(!ln!
endlocal
)
::preserve blank lines using FINDSTR, even if a line may start with :
::long lines >8191 bytes are truncated
for /f "delims=*" %%A in ('type "file.txt" ^| findstr /n "^"') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
echo(!ln!
endlocal
)
::FINDSTR variant that preserves long lines
type "file.txt" >"file.txt.tmp"
for /f "delims=*" %%A in ('findstr /n "^" "file.txt.tmp"') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
echo(!ln!
endlocal
)
del "file.txt.tmp"
If you don't need to worry about converting the file to ASCII, then it is more efficient to drop the pipe and let FIND or FINDSTR open the file specified as an argument, or via redirection.
There is another work around that completely bypasses FOR /F during the read process. It looks odd, but it is more efficient. There are no restrictions with using delayed expansion, but unfortunately it has other limitations.
1) lines must be terminated by <CR><LF> (this will not be a problem if you do the TYPE file conversion)
2) lines must be <= 1021 bytes long (disregarding the <CR><LF>)
3) any trailing control characters are stripped from each line.
4) it must read from a file - you can't use a pipe. So in your case you will need to use a temp file to do your to ASCII conversion.
setlocal enableDelayedExpansion
type "file.txt">"file.txt.tmp"
for /f %%N in ('find /c /v "" ^<"file.txt.tmp"') do set cnt=%%N
<"file.txt.tmp" (
for /l %%N in (1 1 %cnt%) do(
set "ln="
set /p "ln="
echo(!ln!
)
)
del "file.txt.tmp"
I wrote a very simple program that may serve as replacement for FIND and FINDSTR commands when they are used for this purpose. My program is called PIPE.COM and it just insert a blank space in empty lines, so all the lines may be directly processed by FOR command with no further adjustments (as long as the inserted space don't cares). Here it is:
#ECHO off
if not exist pipe.com call :DefinePipe
FOR /F "USEBACKQ delims=" %%A IN (`pipe ^< "build.properties"`) DO (
ECHO(--%%A--
)
pause
goto :EOF
:DefinePipe
setlocal DisableDelayedExpansion
set pipe=´)€ì!Í!ŠÐŠà€Ä!€ü.t2€ü+u!:æu8²A€ê!´#€ì!Í!².€ê!´#€ì!Í!²+€ê!´#€ì!Í!Šò€Æ!´,€ì!Í!"Àu°´LÍ!ëÒ
setlocal EnableDelayedExpansion
echo !pipe!>pipe.com
exit /B
EDIT: Addendum as answer to new comment
The code at :DefinePipe subroutine create a 88 bytes program called pipe.com, that basically do a process equivalent to this pseudo-Batch code:
set "space= "
set line=
:nextChar
rem Read just ONE character
set /PC char=
if %char% neq %NewLine% (
rem Join new char to current line
set line=%line%%char%
) else (
rem End of line detected
if defined line (
rem Show current line
echo %line%
set line=
) else (
rem Empty line: change it by one space
echo %space%
)
)
goto nextChar
This way, empty lines in the input file are changed by lines with one space, so FOR /F command not longer omit they. This works "as long as the inserted space don't cares" as I said in my answer.
Note that the pipe.com program does not work in 64-bits Windows versions.
Antonio
Output lines including blank lines
Here's a method I developed for my own use.
Save the code as a batch file say, SHOWALL.BAT and pass the source file as a command line parameter.
Output can be redirected or piped.
#echo off
for /f "tokens=1,* delims=]" %%a in ('find /n /v "" ^< "%~1"') do echo.%%ba
exit /b
EXAMPLES:
showall source.txt
showall source.txt >destination.txt
showall source.txt | FIND "string"
An oddity is the inclusion of the '^<' (redirection) as opposed to just doing the following:
for /f "tokens=1,* delims=]" %%a in ('find /n /v "" "%~1"') do echo.%%ba
By omitting the redirection, a leading blank line is output.
Thanks to dbenham, this works, although it is slightly different than his suggestion:
::preserve blank lines using FIND, no limitations
for /f "USEBACKQ delims=" %%A in (`type "file.properties" ^| find /V /N ""`) do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*]=!"
echo(!ln!
endlocal
)
As mentioned in this answer to the above question, it doesn't seem that lines are skipped by default using for /f in (at least) Windows XP (Community - Please update this answer by testing the below batch commands on your version & service pack of Windows).
EDIT: Per Jeb's comment below, it seems that the ping command, in at least Windows XP, is
causing for /f to produce <CR>'s instead of blank lines (If someone knows specifically why, would
appreciate it if they could update this answer or comment).
As a workaround, it seems that the second default delimited token (<space> / %%b in the example)
returns as blank, which worked for my situation of eliminating the blank lines by way of an "parent"
if conditional on the second token at the start of the for /f, like this:
for /f "tokens=1,2*" %%a in ('ping -n 1 google.com') do (
if not "x%%b"=="x" (
{do things with non-blank lines}
)
)
Using the below code:
#echo off
systeminfo | findstr /b /c:"OS Name" /c:"OS Version"
echo.&echo.
ping -n 1 google.com
echo.&echo.
for /f %%a in ('ping -n 1 google.com') do ( echo "%%a" )
echo.&echo.&echo --------------&echo.&echo.
find /?
echo.&echo.
for /f %%a in ('find /?') do ( echo "%%a" )
echo.&echo.
pause
.... the following is what I see on Windows XP, Windows 7 and Windows 2008, being the only three versions & service packs of Windows I have ready access to:
So I'm building a messaging program in batch (I know, it's newbish) and the program takes user input, puts it in my .txt file log.txt, and types it on the screen. I want the output to look like this...
Title
----------------------
contents
of
the
file
here
----------------------
User input here>>
This may seem simple, but the file will be constantly updated by users and I want the program to only display a range of lines to keep that message area stays the same size. I found a simple program to display specific lines, but I can't make them move down one line each time log.txt is changed. Here it is:
#setlocal enableextensions enabledelayedexpansion
#echo off
set lines=1
set curr=1
for /f "delims=" %%a in ('type bob.txt') do (
for %%b in (!lines!) do (
if !curr!==%%b echo %%a
)
set /a "curr = curr + 1"
)
endlocal
(By the way, this program is called lines.bat. I just call it in cmd to test it.)
To return a defined number of lines starting from a certain line number, you can do the following:
#echo off
setlocal EnableExtensions
rem define the (path to the) text file here:
set "TEXT_FILE=log.txt"
rem define the line number here:
set /A "LINE_NUMBER=1"
rem define the number of lines here:
set /A "LINE_COUNT=5"
set /A "LINE_LIMIT=LINE_NUMBER+LINE_COUNT-1"
for /F delims^=^ eol^= %%L in ('findstr /N /R "^" "%TEXT_FILE%"') do (
setlocal DisableDelayedExpansion
set "LINE=%%L"
setlocal EnableDelayedExpansion
for /F "tokens=1 delims=:" %%N in ("!LINE!") do set "LNUM=%%N"
set "LINE=!LINE:*:=!"
if !LNUM! GEQ !LINE_NUMBER! (
if !LNUM! LEQ !LINE_LIMIT! (
echo !LINE!
)
)
endlocal
endlocal
)
endlocal
The findstr command with the /R "^" search pattern returns all lines. The findstr switch /N lets every line precede with a line number (starting from 1) and a colon. The : is used to split the line in two parts: the first part representing the line number is checked whether it is in the range to be returned; the second part is the original line of text which is simply output in case. Even empty lines are taken into account.
You might ask why not simply using the above mentioned : as a delims delimiter option for for /F, but this would cause problems with lines of text starting with :.
The toggling of delayed expansion is necessary to avoid trouble with special characters like !, for instance.
To return the last defined number of lines, the following approach can be used:
#echo off
setlocal EnableExtensions
rem define the (path to the) text file here:
set "TEXT_FILE=log.txt"
rem define the number of lines here:
set /A "LINE_COUNT=5"
for /F "tokens=1 delims=:" %%L in ('findstr /N /R "^" "%TEXT_FILE%"') do (
set /A "LINE_SKIP=%%L"
)
set /A "LINE_SKIP-=LINE_COUNT"
if %LINE_SKIP% GTR 0 (
set "LINE_SKIP=skip^=%LINE_SKIP%^ "
) else (
set "LINE_SKIP="
)
for /F %LINE_SKIP%delims^=^ eol^= %%L in ('findstr /N /R "^" "%TEXT_FILE%"') do (
setlocal DisableDelayedExpansion
set "LINE=%%L"
setlocal EnableDelayedExpansion
set "LINE=!LINE:*:=!"
echo !LINE!
endlocal
endlocal
)
endlocal
Again, the findstr /N /R "^" command is used. But here, we have an additional for /F loop first, which merely counts the number of lines in the text file, extracting the line number preceded by findstr. The second for /F loop is quite similar to the above approach, but a dynamic skip option is introduced, so that the loop starts iterating through the last lines only; the rest is almost the same as above, except that the conditions concerning the current line number have been removed.
I know I could do the counting of lines also by using find /C /V "" rather than looping through the findstr /N /R "^" output, but if there are one or more empty lines at the end of the file, find returns a number one less as the findstr method, so I went for findstr consistently.
Also here, delayed expansion is toggled to avoid trouble with the ! character.
I'm searching how to keep empty lines when I browse a file with a for loop.
for /f "tokens=1* delims=[" %%i in ('type "test1.txt" ^| find /v /n ""') do (
SET tmp=%%i
echo !tmp! >> test2.txt
)
Actually it works for everybody, but as far as I'm concerned it does not work.
For instance if test1.txt content is:
Hello I come from France
I live in Paris
I'm sorry I don't know English, could we speak French please?
If it doesn't bother you
Thank you
Result in test2.txt will be:
[1
[2
[3
[4
[5
[6
[7
If I put off the "1" near the star "*", result is:
[1]Hello I come from France
[2]I live in Paris
[3]
[4]I'm sorry I don't know English, could we speak French please?
[5]If it doesn't bother you
[6]
[7]Thank you
Desired output is:
Hello I come from France
I live in Paris
I'm sorry I don't know English, could we speak French please?
If it doesn't bother you
Thank you
Can you please help me to solve this trouble?
This could be done as
#echo off
setlocal enableextensions disabledelayedexpansion
for /f "tokens=1,* delims=]" %%a in ('
find /n /v "" ^< "file1.txt"
') do (
>> "file2.txt" echo(%%b
)
The output from the inner find command is like
[123]texttexttext
The code uses the closing bracket as delimiter, so the tokens (we are requesting two tokens: 1,* or 1*) are
[123 texttexttext
^ ^
1 2
%%a %%b
But, as repeated delimiters are handled as only one delimiter, if one line begins with a closing bracket it will be removed. This can be prevented
as
#echo off
setlocal enableextensions disabledelayedexpansion
for /f "tokens=1,* delims=0123456789" %%a in ('
find /n /v "" ^< "file1.txt"
') do (
set "line=%%b"
setlocal enabledelayedexpansion
>>"file2.txt" echo(!line:~1!
endlocal
)
Here the numbers are used as delimiters and the line is tokenized as
[ ]texttexttext
^ ^
%%a %%b
Then the value of the second token is stored inside a variable, with delayed expansion disabled to avoid problems with exclamations inside the data (that will be handled/replaced by the parser if delayed expansion is active)
Once the data is inside the variable, delayed expansion is activated (something needed as we want to retrieve the contents from a variable changed inside a block of code) to output the line from the second position (first character in string is 0) to remove the closing bracket. Once done, delayed expansion is disabled again.
edited as the OP has to incorporate it to a larger/complex script, this code should face the most usual problems
#echo off
rem For this test we will have delayed expansion from the start
setlocal enableextensions enabledelayedexpansion
rem External code block that will make delayed expansion necessary
if 1==1 (
rem Variables changed inside block
set "input_file=file1.txt"
set "output_file=file2.txt"
rem Grab a reference to the content of the file variables
for %%i in ("!input_file!") do for %%o in ("!output_file!") do (
rem Prepare the environment for file work
setlocal disabledelayedexpansion
rem Prepare output file
type nul > "%%~fo"
rem Process input file and write to output file
for /f "tokens=1,* delims=0123456789" %%a in ('
find /n /v "" ^< "%%~fi"
') do (
set "line=%%b"
setlocal enabledelayedexpansion
>>"%%~fo" echo(!line:~1!
endlocal
)
rem Restore the previous environment
endlocal
)
)
Here is a slightly different variant using the findstr command rather than find and doing the redirection to the output file file2.txt only once rather than per for /F loop iteration:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
>> "file2.txt" (
for /F "delims=" %%a in ('findstr /N "^" "file1.txt"') do (
set "line=%%a"
setlocal EnableDelayedExpansion
echo(!line:*:=!
endlocal
)
)
endlocal
The findstr command precedes every line by a line number and a colon, like this:
1:Hello I come from France
The sub-string substitution portion !line:*:=! replaces everything up to the first colon (due to *) by nothing, thus removing this.
Replace the >> operator by > in case you want to overwrite an already existing file rather than to append to it.
Send your loop command to con: device:
In command line:
for /F tokens^=^* %F in ('type File.txt ^>con:')do #echo/%F
In your bat file:
for /F tokens^=^* %%F in ('type File.txt ^>con:')do echo;%%F
I have a .CSV that I am trying to sort through to create another file from the data, but when I run it through, it skips blank entries. For example, if a line is
value,value,value,,,value
and I try to get the 4th column, it would spit out 6th. Presumably because it is the next valid value. I don't want it to skip the blank entry as it can mess up the tables I'm trying to make. Anyone know how to resolve this? (Any tips are welcome as I suck at batch scripts)
Here is my script:
FOR /F "tokens=1,2,3,4,5,6,7,8,9,10,11,12,13,14 delims=," %%a in (file.csv) DO (
echo %%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n
)
pause
This is the standard behaviour of the FOR/F loop, consecutive delims only used as one delimiter.
But you can use a workaround with a second FOR/F.
Prefix each column with another character, split the line at the delim and remove the prefix.
setlocal EnableDelayedExpansion
FOR /F "delims=" %%L in (test.bat) DO (
set "line=%%L,,,,,,,,"
set "line=#!line:,=,#!"
FOR /F "tokens=1,2,3,4 delims=," %%a in ("!line!") DO (
set "param1=%%a"
set "param2=%%b"
set "param3=%%c"
set "param4=%%d"
set "param1=!param1:~1!"
set "param2=!param2:~1!"
set "param3=!param3:~1!"
set "param4=!param4:~1!"
echo !param1! !param2! !param3! !param4!
)
)
As jeb already mentiones in his answer, for /F treats consecutive delimiters as one. To avoid that, you could replace each delimiter , by "," and enclose each full line by "", so each field appears as being enclosed within "", which can easily be removed finally by the ~ modifier of the for /F variable; so there is no need to do any more string manipulations (like sub-string expansion) later on:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
for /F "usebackq delims=" %%# in ("file.csv") do (
set "LINE=%%#"
setlocal EnableDelayedExpansion
for /F "tokens=1-4 delims=," %%A in (^""!LINE:,="^,"!"^") do (
endlocal
echo Field 1: %%~A
echo Field 2: %%~B
echo Field 3: %%~C
echo Field 4: %%~D
setlocal EnableDelayedExpansion
)
endlocal
)
endlocal
This might not work properly if the data contain , characters by themselves (remember that , is not considered as delimiter in case a field is enclosed within "" in the original CSV file).
The toggling of delayed expansion is done to not lose any exclamation marks in the data.