I have a batch file which copies some local files up to a google storage area using the gsutil tool. The gsutil tool produces a nice log file showing the details of the files that were uploaded and if it was OK or not.
Source,Destination,Start,End,Md5,UploadId,Source Size,Bytes Transferred,Result,Description
file://C:\TEMP\file_1.xlsx,gs://app1/backups/file_1.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.804000Z,CPHHZfdlt6AePAPz6JO2KQ==,,18753,18753,OK,
file://C:\TEMP\file_2.xlsx,gs://app1/backups/file_2.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.813000Z,aTKCOQSPVwDycM9+NGO28Q==,,18753,18753,OK,
What I would like to do is to
check the status result in column 8 (OK or FAIL)
If the status is OK then move the source file to another folder (so that it is not uploaded again).
The problem is that the source filename is appended with "file://" which I can't seem to remove, example
file://C:\TEMP\file_1.xlsx
needs to be changed into this
C:\TEMP\file_1.xlsx
I am using a for /f loop and I am not sure if the manipulation of the variables %%A is different within a for /f loop.
#echo off
rem copy the gsutil log file into a temp file and remove the header row using the 'more' command.
more +1 raw_results.log > .\upload_results.log
rem get the source file name (column 1) and the upload result (OK) from column 8
for /f "tokens=1,8 delims=," %%A in (.\upload_results.log) do (
echo The source file is %%A , the upload status was %%B
set line=%%A
set line=!line:file://:=! >> output2.txt echo !line!
echo !line!
)
The output is like this.
The source file is file://C:\TEMP\file_1.xlsx , the upload status was OK
The source file is file://C:\TEMP\file_2.xlsx , the upload status was OK
I'm expecting it to dump the altered values out into a new file but it is not producing anything at the moment.
Normally I would extract from a specific character to the end of the string with something like this but it doesn't work with my For/f loop.
%var:~7%
Any pointers or a different way of doing it greatly appreciated.
Since the part to remove seems fixed it is easier to use substrings.
Also using for /f "skip=1" evades he neccessity of the external command more +1 and another intermediate file.
#echo off & setlocal EnableDelayedExpansion
type NUL>output2.txt
for /f "skip=1 eol=| tokens=1,8 delims=," %%A in (.\upload_results.log) do (
echo The source file is %%A , the upload status was %%B
set "line=%%A"
set "line=!line:~7!"
echo(!line!>>output2.txt
echo(!line!
)
File names and paths can contain also one or more exclamation marks. The line set line=%%A is parsed by Windows command processor a second time before execution with enabled delayed expansion. See How does the Windows Command Interpreter (CMD.EXE) parse scripts? Every ! inside the string assigned to loop variable A is on this line interpreted as begin or end of a delayed expanded environment variable reference. So the string of loop variable A is assigned to environment variable line with an unwanted modification if file path/name contains one or more exclamation marks.
For that reason it is best to avoid usage of delayed expansion. The fastest solution is for this task using a second FOR to get file:// removed from string assigned to loop variable A.
#echo off
del output2.txt 2>nul
for /F "skip=1 tokens=1,8 delims=," %%A in (upload_results.log) do (
echo The source file is %%A , the upload status was %%B.
for /F "tokens=1* delims=/" %%C in ("%%~A") do echo %%D>>output2.txt
)
Even faster would be without the first echo command line inside the loop:
#echo off
(for /F "skip=1 delims=," %%A in (upload_results.log) do (
for /F "tokens=1* delims=/" %%B in ("%%~A") do echo %%C
))>output2.txt
The second solution can be written also as single command line:
#(for /F "skip=1 delims=," %%A in (upload_results.log) do #for /F "tokens=1* delims=/" %%B in ("%%~A") do #echo %%C)>output2.txt
All solutions do following:
The outer FOR processes ANSI (fixed one byte per character) or UTF-8 (one to four bytes per character) encoded text file upload_results.log line by line with skipping the first line and ignoring always empty lines and lines starting with a semicolon which do not occur here.
The line is split up on every occurrence of one or more commas into substrings (tokens) with assigning first comma delimited string to specified loop variable A. The first solution additionally assigns eighth comma delimited string to next loop variable B according to ASCII table.
The inner FOR processes the string assigned to loop variable A with using / as string delimiter to get assigned to specified loop variable file: and to next loop variable according to ASCII table the rest of the string after first sequence of forward slashes which is the full qualified file name.
The full qualified file name is output with command echo and appended either directly to file output2.txt (first solution) or first to a memory buffer which is finally at once written into file output2.txt overwriting a perhaps already existing file with that file name in current directory.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
del /?
echo /?
for /?
See also the Microsoft article about Using command redirection operators for an explanation of the redirections >, >> and 2>nul
CMD Q: I want to remove the extension of a filename.
It is actually a complete path, like C:/Me/My/Path/filename.xxxx
i know that the extension has 4 chars, like shown in example above.
How can i get rid of the extension?
Thanks.
In terminal:
set file=C:/Me/My/Path/filename.1234
for /F "tokens=*" %A IN ("%file%") DO #echo variable ^%file^%: %~dpnA
In batch file:
#echo off
set file=C:/Me/My/Path/filename.1234
echo If called with path as batch parameter: %~dpn1
for /F "tokens=*" %%A IN ("%file%") DO echo variable %%file%%: %%~dpnA
This does the trick:
#echo off
set fullpath= C:/Me/My/Path/filename.xxxx
set withoutext=%fullpath:~0,-5%
echo %withoutext%
The result is:
c:\>test
C:/Me/My/Path/filename
In this example script the last 5 characters are removed from the variable %fullpath%.
The syntax is explained here: http://www.computerhope.com/forum/index.php?topic=78901.0
In cases where the length of the extension is unknown this would not work.
ren C:/Me/My/Path/filename.xxxx filename.
this should do it (the "." at the end is important)
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
What are some of the lesser know, but important and useful features of Windows batch files?
Guidelines:
One feature per answer
Give both a short description of the feature and an example, not just a link to documentation
Limit answers to native funtionality, i.e., does not require additional software, like the Windows Resource Kit
Clarification: We refer here to scripts that are processed by cmd.exe, which is the default on WinNT variants.
(See also: Windows batch files: .bat vs .cmd?)
Line continuation:
call C:\WINDOWS\system32\ntbackup.exe ^
backup ^
/V:yes ^
/R:no ^
/RS:no ^
/HC:off ^
/M normal ^
/L:s ^
#daily.bks ^
/F daily.bkf
PUSHD path
Takes you to the directory specified by path.
POPD
Takes you back to the directory you "pushed" from.
Not sure how useful this would be in a batch file, but it's a very convenient command to use in the command prompt:
C:\some_directory> start .
This will open up Windows Explorer in the "some_directory" folder.
I have found this a great time-saver.
I have always found it difficult to read comments that are marked by a keyword on each line:
REM blah blah blah
Easier to read:
:: blah blah blah
Variable substrings:
> set str=0123456789
> echo %str:~0,5%
01234
> echo %str:~-5,5%
56789
> echo %str:~3,-3%
3456
The FOR command! While I hate writing batch files, I'm thankful for it.
FOR /F "eol=; tokens=2,3* delims=, " %i in (myfile.txt) do #echo %i %j %k
would parse each line in myfile.txt, ignoring lines that begin with a semicolon, passing the 2nd and 3rd token from each line to the for body, with tokens delimited by commas and/or spaces.
Notice the for body statements reference %i to get the 2nd token, %j to get the 3rd token, and %k to get all remaining tokens after the 3rd.
You can also use this to iterate over directories, directory contents, etc...
Rather than litter a script with REM or :: lines, I do the following at the top of each script:
#echo OFF
goto :START
Description of the script.
Usage:
myscript -parm1|parm2 > result.txt
:START
Note how you can use the pipe and redirection characters without escaping them.
The path (with drive) where the script is : ~dp0
set BAT_HOME=%~dp0
echo %BAT_HOME%
cd %BAT_HOME%
The %~dp0 piece was mentioned already, but there is actually more to it:
the character(s) after the ~ define the information that is extracted.
No letter result in the return of the patch file name
d - returns the drive letter
p - returns the path
s - returns the short path
x - returns the file extension
So if you execute the script test.bat below from the c:\Temp\long dir name\ folder,
#echo off
echo %0
echo %~d0
echo %~p0
echo %~dp0
echo %~x0
echo %~s0
echo %~sp0
you get the following output
test
c:
\Temp\long dir name\
c:\Temp\long dir name\
.bat
c:\Temp\LONGDI~1\test.bat
\Temp\LONGDI~1\
And if a parameter is passed into your script as in
test c:\temp\mysrc\test.cpp
the same manipulations can be done with the %1 variable.
But the result of the expansion of %0 depends on the location!
At the "top level" of the batch it expands to the current batch filename.
In a function (call), it expands to the function name.
#echo off
echo %0
call :test
goto :eof
:test
echo %0
echo %~0
echo %~n0
The output is (the batchfile is started with myBatch.bat )
myBatch.bat
:test
:test
myBatch
By using CALL, EXIT /B, SETLOCAL & ENDLOCAL you can implement subroutines with local variables.
example:
#echo off
set x=xxxxx
call :sub 10
echo %x%
exit /b
:sub
setlocal
set /a x=%1 + 1
echo %x%
endlocal
exit /b
This will print
11
xxxxx
even though :sub modifies x.
Sneaky trick to wait N seconds (not part of cmd.exe but isn't extra software since it comes with Windows), see the ping line. You need N+1 pings since the first ping goes out without a delay.
echo %time%
call :waitfor 5
echo %time%
goto :eof
:waitfor
setlocal
set /a "t = %1 + 1"
>nul ping 127.0.0.1 -n %t%
endlocal
goto :eof
Escaping the "plumbing":
echo ^| ^< ^> ^& ^\ ^^
Being able to run commands and process the output (like backticks of '$()' in bash).
for /f %i in ('dir /on /b *.jpg') do echo --^> %i
If there are spaces in filenames, use this:
for /f "tokens=*" %i in ('dir /on /b *.jpg') do echo --^> %i
Creating an empty file:
> copy nul filename.ext
To hide all output from a command redirect to >nul 2>&1.
For example, the some command line programs display output even if you redirect to >nul. But, if you redirect the output like the line below, all the output will be suppressed.
PSKILL NOTEPAD >nul 2>&1
EDIT: See Ignoring the output of a command for an explanation of how this works.
PAUSE
Stops execution and displays the following prompt:
Press any key to continue . . .
Useful if you want to run a batch by double-clicking it in Windows Explorer and want to actually see the output rather than just a flash of the command window.
The equivalent of the bash (and other shells)
echo -n Hello # or
echo Hello\\c
which outputs "Hello" without a trailing newline. A cmd hack to do this:
<nul set /p any-variable-name=Hello
set /p is a way to prompt the user for input. It emits the given string and then waits, (on the same line, i.e., no CRLF), for the user to type a response.
<nul simply pipes an empty response to the set /p command, so the net result is the emitted prompt string. (The variable used remains unchanged due to the empty reponse.)
Problems are: It's not possible to output a leading equal sign, and on Vista leading whitespace characters are removed, but not on XP.
Search and replace when setting environment variables:
> #set fname=%date:/=%
...removes the "/" from a date for use in timestamped file names.
and substrings too...
> #set dayofweek=%fname:~0,3%
Integer arithmetic:
> SET /A result=10/3 + 1
4
Command separators:
cls & dir
copy a b && echo Success
copy a b || echo Failure
At the 2nd line, the command after && only runs if the first command is successful.
At the 3rd line, the command after || only runs if the first command failed.
Output a blank line:
echo.
You can chain if statements to get an effect like a short-circuiting boolean `and'.
if foo if bar baz
To quickly convert an Unicode text file (16bit/char) to a ASCII DOS file (8bit/char).
C:\> type unicodeencoded.txt > dosencoded.txt
as a bonus, if possible, characters are correctly mapped.
if block structure:
if "%VS90COMNTOOLS%"=="" (
echo: Visual Studio 2008 is not installed
exit /b
)
Delayed expansion of variables (with substrings thrown in for good measure):
#echo off
setlocal enableextensions enabledelayedexpansion
set full=/u01/users/pax
:loop1
if not "!full:~-1!" == "/" (
set full2=!full:~-1!!full2!
set full=!full:~,-1!
goto :loop1
)
echo !full!
endlocal
Doesn't provide much functionality, but you can use the title command for a couple of uses, like providing status on a long script in the task bar, or just to enhance user feedback.
#title Searching for ...
:: processing search
#title preparing search results
:: data processing
Don't have an editor handy and need to create a batch file?
copy con test.bat
Just type away the commands, press enter for a new line.
Press Ctrl-Z and Enter to close the file.
example of string subtraction on date and time to get file named "YYYY-MM-DD HH:MM:SS.txt"
echo test > "%date:~0,4%-%date:~5,2%-%date:~8,2% %time:~0,2%_%time:~3,2%_%time:~6,2%.txt"
I use color to indicate if my script end up successfully, failed, or need some input by changing color of text and background. It really helps when you have some machine in reach of your view but quite far away
color XY
where X and Y is hex value from 0 to F, where X - background, Y - text, when X = Y color will not change.
color Z
changes text color to 'Z' and sets black background, 'color 0' won't work
for names of colors call
color ?
Total control over output with spacing and escape characters.:
echo. ^<resourceDir^>/%basedir%/resources^</resourceDir^>
TheSoftwareJedi already mentioned the for command, but I'm going to mention it again as it is very powerful.
The following outputs the current date in the format YYYYMMDD, I use this when generating directories for backups.
for /f "tokens=2-4 delims=/- " %a in ('DATE/T') do echo %c%b%a