Batch script extract contents between two strings - windows

I am trying to write this Batch script to extract the two parameters from an XML file that looks like this:
<?xml version="1.0" encoding="utf-8" ?>
<!--<var name="SqlConnection" value="data source=SERVERNAME;initialcatalog=DB_NAME;user id=JackDaniels;password=G235X" />-->
<var name="SqlConnection" value="data source=SERVERNAME;initial catalog=DB_Name;Integrated Security=SSPI" />
My objective is to extract SERVERNAME and DB_NAME from the line that is not commented out.
So far my code looks like this:
#echo off
setlocal enableextensions disabledelayedexpansion
set "connectionString="
set result=""
set "INPUT_FILE=DBConnection.config"
FOR /F "tokens=* delims=<var eol=!" %%x in (%INPUT_FILE%) DO (
ECHO %%x
)
PAUSE
I'm just not sure how to get everything right after "data source=" and "initial catalog=". Is there an easy way to do this?

The adequate way to extract this data is not via a Batch file, but with the methods suggested in a comment. However, the Batch file below perform this extraction in a relatively simple way:
#echo off
setlocal EnableDelayedExpansion
rem From the line that have "<var" followed by "value"...
for /F "delims=" %%a in ('findstr "\<var.*value" input.txt') do (
rem ... process the parts separated by space or equal sign, excepting if enclosed in quotes...
for %%b in (%%a) do (
rem ... and store the part after "value" variable
if "!var!" equ "value" set "value=%%~b"
set "var=%%~b"
)
)
rem Separate "value" string at semicolons and assign each part via SET command
for %%a in ("%value:;=" "%") do set %%a
rem Show results:
echo data source=%data source%
echo initial catalog=%initial catalog%
echo Integrated Security=%Integrated Security%
Output example:
data source=SERVERNAME
initial catalog=DB_Name
Integrated Security=SSPI
Of course, if the data format described in the code changes, the program will fail...

#ECHO Off
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "filename1=%sourcedir%\q42420941.txt"
FOR %%v IN (initial catalog data source initial_catalog data_source) DO SET "%%v="
FOR /f "delims=<>" %%z IN ('findstr /B /L /C:"\<var name" "%filename1%"') DO (
FOR %%y IN (%%z) DO (
FOR %%a IN (%%~y) DO (
SET "alive=Y"
FOR %%m IN (initial catalog data source) DO IF /i "%%a"=="%%m" SET "alive="&set %%a=Y
IF DEFINED alive (
IF DEFINED initial IF DEFINED catalog SET "initial_catalog=%%a"
IF DEFINED data IF DEFINED source SET "data_source=%%a"
)
IF DEFINED catalog IF NOT DEFINED initial SET alive=y
IF DEFINED source IF NOT DEFINED data SET alive=y
IF DEFINED alive FOR %%v IN (initial catalog data source) DO set "%%v="
)
)
)
ECHO %initial_catalog% and %data_source%
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances.
I used a file named q42420941.txt containing your data for my testing.
first, clear out the variable-names.
next, parse each line that passes the findstr which is looking for lines that /L literally /B begin with the /c: character-sequence "< escaped by \) and take the first token delimited by > or <.
This neatly strips the awkward <> from the string and assigns the contents of the selected line to %%z
Next, use a simple for to present each token in the line to %%y.
Then with the quotes stripped off of %%y assign each token to %%a.
Search for a match against the keywords, and set the variable of the same name if found. Clear alive if a keyword is found.
If the string in %%a is not one of the keywords, then check to see whether initial and catalog are both set. If so, this is the required string, so assign it.
if catalog is found but not initial then mark as alive
If alive is still set, then we can clear the flags and wait for the start of another sequence.
OK - it has its faults. It won't detect initial_catalog/data_source if either is one of the four keywords (unlikely) and it assumes that the wanted data is the token following the two keywords - the = becomes a separator in the for list.

Since many of you suggested that Batch is not an adequate way of doing this, I decided to play around with PowerShell, and was able to accomplish what I wanted with just a few lines, and some Regex!
$configPath = $PSScriptRoot + "DBConnection.config"
[xml]$XmlDocument = Get-Content -Path $configPath
$dataSource = $XmlDocument.var.Attributes[1].Value.ToString() # Extract the uncommented line, and the second attribute "value"
$serverName = ($dataSource -split 'data source=([^;]*);')[1]
$db_name = ($dataSource -split 'initial catalog=([^;]*);')[1]
$user_id = ($dataSource -split 'id=([^;]*);')[1]
$user_pass = ($dataSource -split 'password=([^;]*)')[1]

Related

BIG Query command using BAT file

echo Give yearmonth "yyyyMM"
setlocal enabledelayedexpansion
SET /p yearmonth=
SET ClientName[0]=abc
SET ClientName[1]=def
SET i = 0
:myLoop
if defined ClientName[%i%] (
call bq query --use_legacy_sql=false "CREATE EXTERNAL TABLE `test.!ClientName[%%i]!.%yearmonth%` OPTIONS (format = 'CSV',skip_leading_rows = 1 uris = ['gs://test/!ClientName[%%i]!/AWS/%yearmonth%/Metrics/data/*.csv'])"
set /a "i+=1"
GOTO :myLoop
)
Hi, I am trying to create a batch so that i can run Multiple BIG QUERY at once.
Above i tried to write a batch script putting command in a loop .
I am trying to create a table by using yearmonth as user input and then create array to create a table with different client name .
But I am unable to print if i =0 ClientName[i] = abc in a call query i am using !ClientName[%%i]! to print but its not working.
Call query inside loop is not running in GCP console, when i executed the bat file .
Can you please help me resolve this
It is bad practice to set variables as standalone alphabetical characters like i. One reason is exactly as you have experienced, you have confused for metavariable %%i with a set variable %i%.
You are expanding in the loop, but have not enabledelayedexpansion so there are 2 ways, which we will get to in a second.
setting variables should not have spaces before or after = excluding the likes of set /a
So, Method 1, without delayedexpansion (note how the variables are used with double %% in the loop with the call command).
#echo off
echo Give yearmonth "yyyyMM"
SET /p yearmonth=
SET ClientName[0]=abc
SET ClientName[1]=def
SET num=0
:myLoop
if defined ClientName[%num%] (
call bq query --use_legacy_sql=false "CREATE EXTERNAL TABLE `test.%%ClientName[%num%]%%.%yearmonth%` OPTIONS (format = 'CSV',skip_leading_rows = 1 uris = ['gs://test/%%ClientName[%num%]%%/AWS/%yearmonth%/Metrics/data/*.csv'])"
set /a num+=1
GOTO :myLoop
)
Method 2: (better method using delayedexpansion)
#echo off
setlocal enabledelayedexpansion
echo Give yearmonth "yyyyMM"
SET /p yearmonth=
SET ClientName[0]=abc
SET ClientName[1]=def
SET num=0
:myLoop
if defined ClientName[%num%] (
call bq query --use_legacy_sql=false "CREATE EXTERNAL TABLE `test.!ClientName[%num%]!.%yearmonth%` OPTIONS (format = 'CSV',skip_leading_rows = 1 uris = ['gs://test/!ClientName[%num%]!/AWS/%yearmonth%/Metrics/data/*.csv'])"
set /a num+=1
GOTO :myLoop
)
I would go for a more intuitive looping mechanism:
For /F "Tokens=1,* Delims==" %%G In ('"(Set ClientName[) 2>NUL"') Do Call "%ProgramFiles%\Google\Cloud SDK\google-cloud-sdk\bin\bq.cmd" query --use_legacy_sql=false "CREATE EXTERNAL TABLE `test.%%G.%yearmonth%` OPTIONS (format = 'CSV',skip_leading_rows = 1 uris = ['gs://test/%%G/AWS/%yearmonth%/Metrics/data/*.csv'])"
It does not require delayed expansion, or incrementing of variables, and does nothing if no variable named ClientName[n] is defined.
It would therefore mean you could have your variables listed but at some point comment one or more out:
Set "ClientName[0]=abc"
Set "ClientName[1]=def"
Rem Set "ClientName[2]=ghi"
Set "ClientName[3]=jkl"
When the loop ran, unlike in the incrementing mechanism, there would be no unnecessary parsing of a parenthesized If Defined %ClientName[2]% … code block.
However, there are several things in your question which open up the possibility of potential issues. The biggest being that you are seemingly expecting user input, but at the same time trusting that the input is provided, and is formatted according to your expectations. You must always, when writing scripts requiring interaction, assume that the end user cannot or will not always follow the instructions.
Here therefore is a more robustly designed script to perform your intended query:
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
Rem Define a variable named bq pointing to your bigquery batch file.
Set "bq=%ProgramFiles%\Google\Cloud SDK\google-cloud-sdk\bin\bq.cmd"
Rem If the bigquery batch file does not exist, terminate the script.
If Not Exist "%bq%" Exit /B
Rem Ensure that there are not existing ClientName[n] variables defined.
For /F "Delims=" %%G In ('"(Set ClientName[) 2>NUL"') Do Set "%%G="
Rem Define your ClientName[n] variable list.
Set "ClientName[0]=abc"
Set "ClientName[1]=def"
:GetYearMonth
Rem Ensure that no existing variable named YearMonth is defined.
Set "YearMonth="
Rem Request user input of date saved to the variable named YearMonth.
Set /P "YearMonth=Please provide the date in the format yyyyMM>"
Rem If no input was received, ask again.
If Not Defined YearMonth GoTo GetYearMonth
Rem Remove any doublequotes from the YearMonth variable value.
Set "YearMonth=%YearMonth:"=%"
Rem If YearMonth variable value is not within allowed range, ask again.
Rem This example uses 197001 - 202212
(Set YearMonth) 2>NUL | %SystemRoot%\System32\findstr.exe /R^
/C:"^YearMonth=19[789][0123456789]0[123456789]$"^
/C:"^YearMonth=19[789][0123456789]1[012]$"^
/C:"^YearMonth=20[01][0123456789]0[123456789]$"^
/C:"^YearMonth=20[01][0123456789]1[012]$"^
/C:"^YearMonth=202[012]0[123456789]$"^
/C:"^YearMonth=202[012]1[012]$"^
1>NUL || GoTo GetYearMonth
Rem As input information is valid loop your command.
For /F "Tokens=1,* Delims==" %%G In ('"(Set ClientName[) 2>NUL"'
) Do Call "%bq%" query --use_legacy_sql=false "CREATE EXTERNAL TABLE `test.%%G.%YearMonth%` OPTIONS (format = 'CSV',skip_leading_rows = 1 uris = ['gs://test/%%G/AWS/%YearMonth%/Metrics/data/*.csv'])"
EndLocal

How to iterate through items in an .ini file with a batch file?

I am currently trying to loop through every item from a .ini file and to work with the values later on. But I can't figure out how. My config.ini file looks like this:
[items]
item_1=XXXXX
item_2=XXXXX
item_3=XXXXX
item_4=XXXXX
[SomeSection]
......
I found a way to iterate and echo every item from the config.ini file, like so:
#echo off
for /F %%i in (config.ini) do (
echo %%i
)
My problem is that I want to work with specific values. So I have to check the categorie and the keys from the config.ini file. I tried using this, but I ran into errors:
#echo off
for /F %%i in (config.ini) do (
SET item = %%i
if %item%==[items] (
rem do something here with the key and values now
)
)
As I already mentioned, I am not able to save the values to another variable, which leads to my problem that I can't work with them.
A quite simple approach was to determine the line number of the target section header in advance, then skip such as many lines when reading the configuration file, stopping as soon as there occurs another string enclosed within brackets:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_CONFIG=%~dp0config.ini" & rem // (path to configuration file)
set "_SECT=items" & rem // (section name without brackets)
rem // Clean up variables whose names begin with `$`:
for /F "delims==" %%V in ('2^> nul set "$"') do set "%%V="
rem // Gather number of line containing the given section (ignoring case):
for /F "delims=:" %%N in ('findstr /N /I /X /C:"[%_SECT%]" "%_CONFIG%"') do set "SKIP=%%N"
rem // Read configuration file, skipping everything up to the section header:
for /F "usebackq skip=%SKIP% delims=" %%I in ("%_CONFIG%") do (
rem // Leave loop as soon as another section header is reached:
for /F "tokens=1* delims=[]" %%K in ("%%I") do if "[%%K]%%L"=="%%I" goto :NEXT
rem // Do something with the key/value pair, like echoing it:
echo(%%I
rem // Assign a variable named of `$` + key and assign the value:
set "$%%I"
)
:NEXT
rem // Return assigned variables:
set "$"
endlocal
exit /B
This script would assign the following variables, based on your sample configuration file:
$item_1=XXXXX
$item_2=XXXXX
$item_3=XXXXX
$item_4=XXXXX

Windows Batch - Find word in one string matching word in another string and capture output

While this may seem easy to some, I've struggled for hours on it.
I have a file:
MYFOLDER,JobE,JobD_ENDED_OK,
MYFOLDER,JobD,JobC_ENDED_OK,JobD_ENDED_OK
MYFOLDER,JobD,JobB_ENDED_OK,
MYFOLDER,JobC,JobA_ENDED_OK,JobC_ENDED_OK
MYFOLDER,JobB,JobA_ENDED_OK,JobB_ENDED_OK
MYFOLDER,JobA,,JobA_ENDED_OK
I need to loop through and find where token 4 in one line matches token 3 in another line and then echo a statement to a file. I am looking for an output file that shows this:
MYFOLDER_JobA_MYFOLDER_JobB matches JobA_ENDED_OK
MYFOLDER_JobA_MYFOLDER_JobC matches JobA_ENDED_OK
MYFOLDER_JobB_MYFOLDER_JobD matches JobB_ENDED_OK
MYFOLDER_JobC_MYFOLDER_JobD matches JobC_ENDED_OK
MYFOLDER_JobD_MYFOLDER_JobE matches JobD_ENDED_OK
I know it's a FOR loop with a DO, I am just not getting the rest of it.
Any assistance is greatly appreciated.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
SET "filename1=%sourcedir%\q46510665.txt"
SET "outfile=%destdir%\outfile.txt"
(
FOR /f "usebackqdelims=" %%h IN ("%filename1%") DO (
SET "col4line=%%h"
SET "col4line=!col4line:,=|,|!"
FOR /f "tokens=1-4delims=," %%a IN ("!col4line!") DO IF "%%d" neq "|" (
FOR /f "usebackqdelims=" %%H IN ("%filename1%") DO (
SET "col3line=%%H"
SET "col3line=!col3line:,=|,|!"
FOR /f "tokens=1-4delims=," %%A IN ("!col3line!") DO (
IF "%%d|"=="%%C" (
SET "reportline=%%a_%%b_%%A_%%B matches %%C"
ECHO !reportline:^|=!
)
)
)
)
)
)>"%outfile%"
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
I used a file named q46510665.txt containing your data for my testing.
Produces the file defined as %outfile%
For each line in the file, set a variable col4line to the entire line via %%h, then replace each , with |,| so that successive , will be separated. Tokenise on , and ignore any line which has simply | as its 4th token (ie last-column-empty).
Repeat the process for every line in the file this time through %%H into col3line (note case differential to use different actual metavariables) and if the third column matches the fourth column+| from the outer loop, assemble the report line from the tokens and output, removing the |s.

How to extract the numbers after a character in windows batch files

Hi I do need to extract the last part of a string after the last dot
Example:
1.2.37 ==> I need the 37
1.2.567 ==> I need the 567
as you can see the number of characters after the dot is not fixed so expressions like
base=%fullver:~0,-2%
Can't be used. How can I achieve this?
#echo off
setlocal enableextensions disabledelayedexpansion
set "fullver=1.2.456"
for %%a in ("%fullver:.=\%") do set "base=%%~na"
echo %base%
The trick is to replace the dots with backslashes, process the string as a path and retrieve the name of the last element in it.
Alternatively, if all the elements need to be retrieved, instead of a for, a for /f is used to tokenize the variable using the dots as separators
#echo off
setlocal enableextensions disabledelayedexpansion
set "fullver=1.2.456"
for /f "tokens=1-3 delims=." %%a in ("%fullver%") do (
set "major=%%a"
set "minor=%%b"
set "build=%%c"
)
echo [%major%] [%minor%] [%build%]
I found the following question which actually tokenizes the string.
How to split a string in a Windows batch file?
May be you can try using this to delimit it with "." and take the last value stored in the string variable. Not sure if there is a simple way, but this works.
Here is an edited Version to fit your Needs:
#echo off
setlocal ENABLEDELAYEDEXPANSION
REM Set a string with an arbitrary number of substrings separated by semi colons
set teststring=1.2.5.234
for /f "tokens=1 delims=." %%a IN ("!teststring!") DO set firststring=%%a
echo !firststring!
REM Do something with each substring
:stringLOOP
REM Stop when the string is empty
if "!teststring!" EQU "" goto END
for /f "delims=." %%a in ("!teststring!") do set substring=%%a
REM Now strip off the leading substring
:striploop
set stripchar=!teststring:~0,1!
set teststring=!teststring:~1!
if "!teststring!" EQU "" goto stringloop
if "!stripchar!" NEQ "." goto striploop
goto stringloop
:END
echo !substring!
endlocal
I prefer MC ND's answer if you are looking for only the last node, or if you know how many nodes there are.
Here is a method to capture all nodes if the total number of nodes is unknown:
#echo off
setlocal enableDelayedExpansion
set "fullver=1.2.456"
:: Parse each node and store in an "array"
set cnt=0
for %%A in (%fullver:.= %) do (
set /a cnt+=1
set "node.!cnt!=%%A"
)
:: Show the results
for /l %%N in (1 1 %cnt%) do echo node.%%N = !node.%%N!
Another solution! This one gets the first and last parts of the string:
#echo off
setlocal
set "testString=1.2.5.234"
set "first="
for %%a in ("%testString:.=" "%") do (
if not defined first set "first=%%~a"
set "last=%%~a"
)
echo First: %first%
echo Last: %last%
As a bonus, this method correctly process special Batch characters that may appear in the string, excepting wild-cards.
You can use the below command to achieve what you want.
base=%fullver:~~4,3%
4 implies 4th digit i.e., 5 and 3 implies 3 digits from 4.
The output will be
567

Windows Batch to read file and parse lines into tokens and variables

I've made a good deal of headway by searching this site and learning the ridiculous language that is Windows batch scripting, but I'm now at a point where I'm stuck. I have a text file with a variable number of lines, each of which looks something like:
AA8315,"United States",N777AN,"American Airlines",AAL98,B772,"Boeing 777-223",AAL,"2013-06-11 23:30:47.923","2013-06-12 00:01:14.459"
My batch file:
set THEDATE=2013-06-12
set THEDATABASE=c:\Kinetic\BaseStation\Basestation.sqb
set THECSVFILE=c:\Flights.csv
set THEOUTPUTFILE=c:\FlightsNew.csv
set THISLINE=""
if exist %THECSVFILE% del %THECSVFILE%
if exist %THEOUTPUTFILE% del %THEOUTPUTFILE%
:: allow time for the csv file to be deleted
timeout /t 2 /nobreak
c:\sqlite3.exe -header -csv %THEDATABASE% "select Aircraft.ModeS, Aircraft.ModeSCountry as Country, Aircraft.Registration as Reg, Aircraft.RegisteredOwners as Owner, Flights.Callsign, Aircraft.ICAOTypeCode as Type, Aircraft.Type as Model, Aircraft.OperatorFlagCode as 'Op Flag', Flights.StartTime as 'First Seen', Flights.EndTime as 'Last Seen' from Aircraft INNER JOIN Flights ON (Aircraft.AircraftID=Flights.AircraftID) where Flights.EndTime like '%THEDATE% %%' order by Flights.EndTime DESC;" >> %THECSVFILE%
::allow time for the csv to be written to file
timeout /t 5 /nobreak
::read %THECSVFILE% and loop through each line
for /F "usebackq tokens=* delims=" %%A in (%THECSVFILE%) do (
set the_line=%%A
call :process_line
)
:process_line
for /F "usebackq tokens=1,2,3,4,5,6,7,8,9,10 delims=[,]" %%1 in (%the_line%) do (
set hexcode=%%1
set country=%%2
set reg=%%3
set owner=%%4
set callsign=%%5
set planetype=%%6
set model=%%7
set opflag=%%8
set firstseen=%%9
set lastseen=%%10
set THISLINE=%hexcode%,%country%,%reg%,%owner%,%callsign%,%planetype%,%model%,%opflag%,%firstseen%,%lastseen%
echo %THISLINE% > %THEOUTPUTFILE%
)
(I'm assigning the tokens to variables because I will be doing additional validation and formatting of them later. I need to get this part working first!)
When executed, the script does indeed loop through each line of the file, however it does not seem to be assigning %%1 to the variable hexcode.
The output of the executed command looks like this:
C:\>for /F "usebackq tokens=1,2,3,4,5,6,7,8,9,10 delims=[,]" %1 in (AA8315 "United States" N777AN "American Airlines" AAL98 B772 "Boeing 777-223" AAL "2013-06-11 23:30:47.923" "2013-06-12 00:01:14.459") do (
set hexcode=%1
set country=%2
set reg=%3
set owner=%4
set callsign=%5
set planetype=%6
set model=%7
set opflag=%8
set firstseen=%9
set lastseen=%10
set THISLINE=,"United States" ,N807FD ,"Fedex Express" ,FDX1378 ,,"Airbus A310-324" ,FDX ,"2013-06-12 22:56:54.639" ,"2013-06-12 23:05:31.822"
echo "" 1>c:\FlightsNew.csv
)
The system cannot find the file AA8315.
Any help is greatly appreciated!
this works here:
for /f "tokens=1-10delims=," %%a in ("AA8315,"United States",N777AN,"American Airlines",AAL98,B772,"Boeing 777-223",AAL,"2013-06-11 23:30:47.923","2013-06-12 00:01:14.459"") do (
set hexcode=%%a
set country=%%b
set reg=%%c
set owner=%%d
set callsegn=%%e
set planefype=%%f
set model=%%g
set opflag=%%h
set firstseen=%%i
set lastseen=%%j
set THISLINE=%%a,%%b,%%c,%%d,%%e,%%f,%%g,%%h,%%i,%%j
)
>"c:\FlightsNew.csv" echo %THISLINE%
I'm not sure, why you need the tokens.
I have always had problems with comma separated values in a for loop. Here's what I did to make your code work.
Test.txt
AA8315,"United States",N777AN,"American Airlines",AAL98,B772,"Boeing 777-223",AAL,"2013-06-11 23:30:47.923","2013-06-12 00:01:14.459"
BatchFile.bat
set THECSVFILE=test.txt
::read %THECSVFILE% and loop through each line
for /F "usebackq tokens=* delims=" %%A in (%THECSVFILE%) do (
set the_line=%%A
call :process_line
)
goto TheEnd
:process_line
for /F "usebackq tokens=1,2,3,4,5,6,7,8,9,10 delims=~" %%1 in ('%the_line:,=~%') do (
set hexcode=%%1
set country=%%2
set reg=%%3
set owner=%%4
set callsign=%%5
set planetype=%%6
set model=%%7
set opflag=%%8
set firstseen=%%9
set lastseen=%%10
set THISLINE=%hexcode%,%country%,%reg%,%owner%,%callsign%,%planetype%,%model%,%opflag%,%firstseen%,%lastseen%
echo %THISLINE% > %THEOUTPUTFILE%
)
:TheEnd
Notice the :process_line for loop. I had to add single quotes around the %the_line% so it didn't try to interpret the string as a filename. Then I replaced all commas with the ~ character, and used the ~ character as the delimiter. It may not work precisely with all your data (if it contains single quotes or the ~ character), but it does work with this one record and gets you moving in the right direction again.
You can only use letters fro the metavariable (%%1 in your code) - but the lower-case and upper-case letters are distinct.
Yes, you can use some other characters, but the contiguous blocks avaliable for "tokens=1-10" (which is an easier version of 1,2,3...) are a..z and A..Z
%0..%9 are reserved for the parameters to the batch or batch-procedure.

Resources