What does "2^>^&1" mean in batch script? - windows

I'm just learning batch script. I'm reviewing the getJavaVersion.bat script on GitHub. I understood what the 2^>^&1 expression in the following line of code is used for. But I couldn't understand how this syntax (2^>^&1) is used. Can you help me with this?
for /f tokens^=2-5^ delims^=.-_^" %%j in ('java -fullversion 2^>^&1') do set "jver=%%j%%k%%l%%m"
The commands below show the generated values:
for /f "delims=" %%i in ('java -fullversion') do set output=%%i
echo %output%
:: OUTPUT >> java full version "18.0.1.1+2-6"
for /f tokens^=2-5^ delims^=.-_^" %%j in ('java -fullversion 2^>^&1') do set "jver=%%j%%k%%l%%m"
echo %jver%
:: OUTPUT >> 18011+2

The command java -version or java -fullversion returns the output at the STDERR stream (handle 2) rather than at the STDOUT stream (handle 1). for /F, together with a command behind the in keyword, captures and parses the command output at the STDOUT stream. To capture the STDERR stream you need to redirect it by 2>&1, meaning that handle 2 (STDERR) is redirected where handle 1 points to (STDOUT). To ensure that the redirection is not applied to the for /F command itself, you need to properly escape it (^) in order for the special characters > and & to lose their particular meaning until the whole for /F command line was processed. The command to be captured eventually contains the redirection expression in an unescaped manner.

Related

Why is no error output redirected from STDERR to file by my batch file?

I have a simple health check script for checking status of windows share drives listed below.
I am getting the desired output when the command works. However, when there is an error, I do not get output in the output file. I have looked at STDOUT and STDERR as a possible solution, but I am not able get this to work yet.
Can anyone please help me with this code.
#echo off
echo START > results.txt
for /f %%s in (server_list.txt) do (
echo %%s >> results.txt
for /f "delims=" %%n in ('net use \\%%s') do (
echo %%s - %%n >> results.txt 2> err.txt
)
)
The output file looks like this.
START
server1
server1 - The command completed successfully.
server2
server2 - The command completed successfully.
However, there is nothing in the err.txt file. I have made sure to put some incorrect entries in file server_list.txt to get a few errors for testing. The error output is displayed in the command line window instead of being printed to the err.txt file.
First, read the Microsoft article about Using Command Redirection Operators.
Second, the command FOR executes the command line net use \\%%s in a new command process using cmd.exe in background which means with no visible console window. The output of this background command process written to STDOUT is captured by FOR and processed next line by line. The error output written to STDERR is redirected by FOR to error output of currently running process which is the reason why the error output can be seen in opened console window of command process executing the batch file.
The command line echo %%s - %%n >> results.txt 2> err.txt outputs the output of background command process executing net use captured by FOR and assigned to loop variable n to STDOUT of current command process which is redirected to file results.txt.
Additionally the error output by command ECHO is redirected to file err.txt if ECHO would output an error message. But ECHO does never output an error message. That is the reason why err.txt does not contain any error message output by command NET.
I suggest to capture with FOR standard and error output of net use and evaluate the output in body of FOR loop.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
del err.txt 2>nul
echo START > results.txt
for /F %%I in (server_list.txt) do (
for /f "delims=" %%J in ('%SystemRoot%\System32\net.exe use "\\%%I" 2^>^&1') do (
set "NetOutput=%%J"
if not "!NetOutput:successfully=!" == "!NetOutput!" (
echo %%I - %%J>>results.txt
) else (
echo %%I - %%J>>err.txt
)
)
)
endlocal
When the line output by command NET from any output contains the word successfully the line from server_list.txt and the success message is written to file results.txt. Otherwise the most likely error output is written also with line from server_list.txt to file err.txt.

Why does this error message appear /before/ the command?

My script:
echo hello
FOR /F %%G IN temp\clist.txt DO type %%G
The output:
S:\TLIB admin\Sets' choicelist orphans>echo hello
hello
temp\clist.txt was unexpected at this time.
S:\TLIB admin\Sets' choicelist orphans>FOR /F %G IN temp\clist.txt DO type %G
S:\TLIB admin\Sets' choicelist orphans>
This assumes that the temp folder is located in the current working directory.
#echo off
echo hello
FOR /F "usebackq delims=" %%G IN ("temp\clist.txt") DO echo %%G
Or you could use this if you just want to type the file.
type "temp\clist.txt"
Why the error is shown before the command? Because the problem raises not at the command execution, but at command parse.
When the parser reads the line and tries to convert it to a valid internal representation of what the line is intended to do, it sees the for command but the line can not be interpreted as a valid command (as stated in other answers/comments), so parser outputs the error description to stderr (by default the console), and, if echo is on, the literal that can not be interpreted is sent to stdout, but not executed, as it has been discarded by the parser.

How to suppress specific lines in Windows cmd output?

On Windows, I have written a simple bat script that calls another tool. However, this tool outputs a few specific debug lines when using certain options (seems to be a bug in the original code, which I can not/don't want to modify).
Is there a way to suppress lines output by a command that matches a certain pattern (like starting with DEBUG: )?
(FYIO: the tool is latexdiff.exe, and there are some print STDERR "DEBUG:... lines in the perl source code that are not conditional to the debug variable and printed everytime the --flatten option is used. I don't want to suppress stderr completely either.)
You could try the following:
latexdiff.exe 2>&1| findstr /v /b "DEBUG:"
The /v option basically turns the pattern around, i.e. findstr.exe will let everything through, not matching the pattern. The /b option simply says that the pattern should occur at the beginning of a line.
The 2>&1 redirects STDERR to STDOUT and is required because, as you said, the lines are written to STDERR, not STDOUT. Note that as a "side effect" all output now is written to STDOUT.
Update if there is other output on STDOUT that you need to have, you could do something like this
latexdiff.exe old.tex new.tex > diff.tex 2> latexdiff.stderr
type latexdiff.stderr | findstr /v /b "DEBUG:"
That is, redirect STDOUT to your diff file, redirect STDERR to some file. Afterwards, you just type the file to see error messages.
You might want to put that into a batch file of it's own, like so:
#echo off
setlocal
REM determine a suitable temporary filename
set error_file=%TEMP%\latexdiff.%RANDOM%.stderr
REM run actual diff and save its exit code for later
latexdiff.exe "%~1" "%~2" > "%~3" 2> "%error_file%"
set error_level=%ERRORLEVEL%
REM dump error messages
type "%error_file%" | findstr /v /b "DEBUG:"
REM remove temporary error file and exit with latexdiff's exit code.
del /q "%error_file%"
exit /b %error_level%
You can then call it like: latexdiff_wrapper.cmd old.tex new.tex diff.tex. Appart from using temporary files, the downside is, that error messages will not appear while processing, but at the very end. If that is not an issue, it shouldn't because the diff should be fast, you might find that solution useful.

How do I echo and send console output to a file in a bat script?

I have a batch script that executes a task and sends the output to a text file. Is there a way to have the output show on the console window as well?
For Example:
c:\Windows>dir > windows-dir.txt
Is there a way to have the output of dir display in the console window as well as put it into the text file?
No, you can't with pure redirection.
But with some tricks (like tee.bat) you can.
I try to explain the redirection a bit.
You redirect one of the ten streams with > file or < file
It is unimportant, if the redirection is before or after the command,
so these two lines are nearly the same.
dir > file.txt
> file.txt dir
The redirection in this example is only a shortcut for 1>, this means the stream 1 (STDOUT) will be redirected.
So you can redirect any stream with prepending the number like 2> err.txt and it is also allowed to redirect multiple streams in one line.
dir 1> files.txt 2> err.txt 3> nothing.txt
In this example the "standard output" will go into files.txt, all errors will be in err.txt and the stream3 will go into nothing.txt (DIR doesn't use the stream 3).
Stream0 is STDIN
Stream1 is STDOUT
Stream2 is STDERR
Stream3-9 are not used
But what happens if you try to redirect the same stream multiple times?
dir > files.txt > two.txt
"There can be only one", and it is always the last one!
So it is equal to dir > two.txt
Ok, there is one extra possibility, redirecting a stream to another stream.
dir 1>files.txt 2>&1
2>&1 redirects stream2 to stream1 and 1>files.txt redirects all to files.txt.
The order is important here!
dir ... 1>nul 2>&1
dir ... 2>&1 1>nul
are different. The first one redirects all (STDOUT and STDERR) to NUL,
but the second line redirects the STDOUT to NUL and STDERR to the "empty" STDOUT.
As one conclusion, it is obvious why the examples of Otávio Décio and andynormancx can't work.
command > file >&1
dir > file.txt >&2
Both try to redirect stream1 two times, but "There can be only one", and it's always the last one.
So you get
command 1>&1
dir 1>&2
And in the first sample redirecting of stream1 to stream1 is not allowed (and not very useful).
Just use the Windows version of the UNIX tee command (found from http://unxutils.sourceforge.net) in this way:
mycommand > tee outpu_file.txt
If you also need the STDERR output, then use the following.
The 2>&1 combines the STDERR output into STDOUT (the primary stream).
mycommand 2>&1 | tee output_file.txt
If you don't need the output in real time (i.e. as the program is writing it) you could add
type windows-dir.txt
after that line.
The solution that worked for me was: dir > a.txt | type a.txt.
Yes, there is a way to show a single command output on the console (screen) and in a file. Using your example, use...
#ECHO OFF
FOR /F "tokens=*" %%I IN ('DIR') DO ECHO %%I & ECHO %%I>>windows-dir.txt
Detailed explanation:
The FOR command parses the output of a command or text into a variable, which can be referenced multiple times.
For a command, such as DIR /B, enclose in single quotes as shown in example below. Replace the DIR /B text with your desired command.
FOR /F "tokens=*" %%I IN ('DIR /B') DO ECHO %%I & ECHO %%I>>FILE.TXT
For displaying text, enclose text in double quotes as shown in example below.
FOR /F "tokens=*" %%I IN ("Find this text on console (screen) and in file") DO ECHO %%I & ECHO %%I>>FILE.TXT
... And with line wrapping...
FOR /F "tokens=*" %%I IN ("Find this text on console (screen) and in file") DO (
ECHO %%I & ECHO %%I>>FILE.TXT
)
If you have times when you want the output only on console (screen), and other times sent only to file, and other times sent to both, specify the "DO" clause of the FOR loop using a variable, as shown below with %TOECHOWHERE%.
#ECHO OFF
FOR %%I IN (TRUE FALSE) DO (
FOR %%J IN (TRUE FALSE) DO (
SET TOSCREEN=%%I & SET TOFILE=%%J & CALL :Runit)
)
GOTO :Finish
:Runit
REM Both TOSCREEN and TOFILE get assigned a trailing space in the FOR loops
REM above when the FOR loops are evaluating the first item in the list,
REM "TRUE". So, the first value of TOSCREEN is "TRUE " (with a trailing
REM space), the second value is "FALSE" (no trailing or leading space).
REM Adding the ": =" text after "TOSCREEN" tells the command processor to
REM remove all spaces from the value in the "TOSCREEN" variable.
IF "%TOSCREEN: =%"=="TRUE" (
IF "%TOFILE: =%"=="TRUE" (
SET TEXT=On screen, and in "FILE.TXT"
SET TOECHOWHERE="ECHO %%I & ECHO %%I>>FILE.TXT"
) ELSE (
SET TEXT=On screen, not in "FILE.TXT"
SET TOECHOWHERE="ECHO %%I"
)
) ELSE (
IF "%TOFILE: =%"=="TRUE" (
SET TEXT=Not on screen, but in "FILE.TXT"
SET TOECHOWHERE="ECHO %%I>>FILE.txt"
) ELSE (
SET TEXT=Not on screen, nor in "FILE.TXT"
SET TOECHOWHERE="ECHO %%I>NUL"
)
)
FOR /F "tokens=*" %%I IN ("%TEXT%") DO %TOECHOWHERE:~1,-1%
GOTO :eof
:Finish
ECHO Finished [this text to console (screen) only].
PAUSE
My option was this:
Create a subroutine that takes in the message and automates the process of sending it to both console and log file.
setlocal
set logfile=logfile.log
call :screenandlog "%DATE% %TIME% This message goes to the screen and to the log"
goto :eof
:screenandlog
set message=%~1
echo %message% & echo %message% >> %logfile%
exit /b
If you add a variable to the message, be sure to remove the quotes in it before sending it to the subroutine or it can screw your batch.
Of course this only works for echoing.
command > file >&1
If you want to append instead of replace the output file, you may want to use
dir 1>> files.txt 2>> err.txt
or
dir 1>> files.txt 2>>&1
I made a simple C# console which can handle real-time output to both cmd screen and log
class Tee
{
static int Main(string[] args)
{
try
{
string logFilePath = Path.GetFullPath(args[0]);
using (StreamWriter writer = new StreamWriter(logFilePath, true))
{
for (int value; (value = Console.In.Read()) != -1;)
{
var word = Char.ConvertFromUtf32(value);
Console.Write(word);
writer.Write(word);
}
}
}
catch (Exception)
{
return 1;
}
return 0;
}
}
The batch file usage is the same as how you use Unix tee
foo | tee xxx.log
And here is the repository which includes the Tee.exe in case you don't have tool to compile
https://github.com/iamshiao/Tee
I like atn's answer, but it was not as trivial for me to download as wintee, which is also open source and only gives the tee functionality (useful if you just want tee and not the entire set of unix utilities). I learned about this from davor's answer to Displaying Windows command prompt output and redirecting it to a file, where you also find reference to the unix utilities.
The solution provided by "Tomas R" works perfect for the OP's question and it natively available.
Try:
chkdsk c: > output.txt | type output.txt
The output is of this command involves a completion percentage that gets serially output to the file hence it will look a bit messy (i.e. the text will be appended as it progress). This does not happen to the bits that gets output to STDOUT (the screen). It is how it would be if you just do the same command without redirection.
I think you want something along the lines of this:
echo Your Msg> YourTxtFile.txt
Or if you want a new line:
echo Your Msg>> YourTxtFile.txt
These commands are great for logs.
Note: This will sometimes glitch and replace the whole text file on my computer.
Another Note: If the file does not exist, it will create the file.

Hidden features of Windows batch files

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
What are some of the lesser know, but important and useful features of Windows batch files?
Guidelines:
One feature per answer
Give both a short description of the feature and an example, not just a link to documentation
Limit answers to native funtionality, i.e., does not require additional software, like the Windows Resource Kit
Clarification: We refer here to scripts that are processed by cmd.exe, which is the default on WinNT variants.
(See also: Windows batch files: .bat vs .cmd?)
Line continuation:
call C:\WINDOWS\system32\ntbackup.exe ^
backup ^
/V:yes ^
/R:no ^
/RS:no ^
/HC:off ^
/M normal ^
/L:s ^
#daily.bks ^
/F daily.bkf
PUSHD path
Takes you to the directory specified by path.
POPD
Takes you back to the directory you "pushed" from.
Not sure how useful this would be in a batch file, but it's a very convenient command to use in the command prompt:
C:\some_directory> start .
This will open up Windows Explorer in the "some_directory" folder.
I have found this a great time-saver.
I have always found it difficult to read comments that are marked by a keyword on each line:
REM blah blah blah
Easier to read:
:: blah blah blah
Variable substrings:
> set str=0123456789
> echo %str:~0,5%
01234
> echo %str:~-5,5%
56789
> echo %str:~3,-3%
3456
The FOR command! While I hate writing batch files, I'm thankful for it.
FOR /F "eol=; tokens=2,3* delims=, " %i in (myfile.txt) do #echo %i %j %k
would parse each line in myfile.txt, ignoring lines that begin with a semicolon, passing the 2nd and 3rd token from each line to the for body, with tokens delimited by commas and/or spaces.
Notice the for body statements reference %i to get the 2nd token, %j to get the 3rd token, and %k to get all remaining tokens after the 3rd.
You can also use this to iterate over directories, directory contents, etc...
Rather than litter a script with REM or :: lines, I do the following at the top of each script:
#echo OFF
goto :START
Description of the script.
Usage:
myscript -parm1|parm2 > result.txt
:START
Note how you can use the pipe and redirection characters without escaping them.
The path (with drive) where the script is : ~dp0
set BAT_HOME=%~dp0
echo %BAT_HOME%
cd %BAT_HOME%
The %~dp0 piece was mentioned already, but there is actually more to it:
the character(s) after the ~ define the information that is extracted.
No letter result in the return of the patch file name
d - returns the drive letter
p - returns the path
s - returns the short path
x - returns the file extension
So if you execute the script test.bat below from the c:\Temp\long dir name\ folder,
#echo off
echo %0
echo %~d0
echo %~p0
echo %~dp0
echo %~x0
echo %~s0
echo %~sp0
you get the following output
test
c:
\Temp\long dir name\
c:\Temp\long dir name\
.bat
c:\Temp\LONGDI~1\test.bat
\Temp\LONGDI~1\
And if a parameter is passed into your script as in
test c:\temp\mysrc\test.cpp
the same manipulations can be done with the %1 variable.
But the result of the expansion of %0 depends on the location!
At the "top level" of the batch it expands to the current batch filename.
In a function (call), it expands to the function name.
#echo off
echo %0
call :test
goto :eof
:test
echo %0
echo %~0
echo %~n0
The output is (the batchfile is started with myBatch.bat )
myBatch.bat
:test
:test
myBatch
By using CALL, EXIT /B, SETLOCAL & ENDLOCAL you can implement subroutines with local variables.
example:
#echo off
set x=xxxxx
call :sub 10
echo %x%
exit /b
:sub
setlocal
set /a x=%1 + 1
echo %x%
endlocal
exit /b
This will print
11
xxxxx
even though :sub modifies x.
Sneaky trick to wait N seconds (not part of cmd.exe but isn't extra software since it comes with Windows), see the ping line. You need N+1 pings since the first ping goes out without a delay.
echo %time%
call :waitfor 5
echo %time%
goto :eof
:waitfor
setlocal
set /a "t = %1 + 1"
>nul ping 127.0.0.1 -n %t%
endlocal
goto :eof
Escaping the "plumbing":
echo ^| ^< ^> ^& ^\ ^^
Being able to run commands and process the output (like backticks of '$()' in bash).
for /f %i in ('dir /on /b *.jpg') do echo --^> %i
If there are spaces in filenames, use this:
for /f "tokens=*" %i in ('dir /on /b *.jpg') do echo --^> %i
Creating an empty file:
> copy nul filename.ext
To hide all output from a command redirect to >nul 2>&1.
For example, the some command line programs display output even if you redirect to >nul. But, if you redirect the output like the line below, all the output will be suppressed.
PSKILL NOTEPAD >nul 2>&1
EDIT: See Ignoring the output of a command for an explanation of how this works.
PAUSE
Stops execution and displays the following prompt:
Press any key to continue . . .
Useful if you want to run a batch by double-clicking it in Windows Explorer and want to actually see the output rather than just a flash of the command window.
The equivalent of the bash (and other shells)
echo -n Hello # or
echo Hello\\c
which outputs "Hello" without a trailing newline. A cmd hack to do this:
<nul set /p any-variable-name=Hello
set /p is a way to prompt the user for input. It emits the given string and then waits, (on the same line, i.e., no CRLF), for the user to type a response.
<nul simply pipes an empty response to the set /p command, so the net result is the emitted prompt string. (The variable used remains unchanged due to the empty reponse.)
Problems are: It's not possible to output a leading equal sign, and on Vista leading whitespace characters are removed, but not on XP.
Search and replace when setting environment variables:
> #set fname=%date:/=%
...removes the "/" from a date for use in timestamped file names.
and substrings too...
> #set dayofweek=%fname:~0,3%
Integer arithmetic:
> SET /A result=10/3 + 1
4
Command separators:
cls & dir
copy a b && echo Success
copy a b || echo Failure
At the 2nd line, the command after && only runs if the first command is successful.
At the 3rd line, the command after || only runs if the first command failed.
Output a blank line:
echo.
You can chain if statements to get an effect like a short-circuiting boolean `and'.
if foo if bar baz
To quickly convert an Unicode text file (16bit/char) to a ASCII DOS file (8bit/char).
C:\> type unicodeencoded.txt > dosencoded.txt
as a bonus, if possible, characters are correctly mapped.
if block structure:
if "%VS90COMNTOOLS%"=="" (
echo: Visual Studio 2008 is not installed
exit /b
)
Delayed expansion of variables (with substrings thrown in for good measure):
#echo off
setlocal enableextensions enabledelayedexpansion
set full=/u01/users/pax
:loop1
if not "!full:~-1!" == "/" (
set full2=!full:~-1!!full2!
set full=!full:~,-1!
goto :loop1
)
echo !full!
endlocal
Doesn't provide much functionality, but you can use the title command for a couple of uses, like providing status on a long script in the task bar, or just to enhance user feedback.
#title Searching for ...
:: processing search
#title preparing search results
:: data processing
Don't have an editor handy and need to create a batch file?
copy con test.bat
Just type away the commands, press enter for a new line.
Press Ctrl-Z and Enter to close the file.
example of string subtraction on date and time to get file named "YYYY-MM-DD HH:MM:SS.txt"
echo test > "%date:~0,4%-%date:~5,2%-%date:~8,2% %time:~0,2%_%time:~3,2%_%time:~6,2%.txt"
I use color to indicate if my script end up successfully, failed, or need some input by changing color of text and background. It really helps when you have some machine in reach of your view but quite far away
color XY
where X and Y is hex value from 0 to F, where X - background, Y - text, when X = Y color will not change.
color Z
changes text color to 'Z' and sets black background, 'color 0' won't work
for names of colors call
color ?
Total control over output with spacing and escape characters.:
echo. ^<resourceDir^>/%basedir%/resources^</resourceDir^>
TheSoftwareJedi already mentioned the for command, but I'm going to mention it again as it is very powerful.
The following outputs the current date in the format YYYYMMDD, I use this when generating directories for backups.
for /f "tokens=2-4 delims=/- " %a in ('DATE/T') do echo %c%b%a

Resources