Windows Batch Scripting: Parse CSV file and extract data - windows
I am new to windows batch scripting so pardon my ignorance. I have a CSV file that looks like this:
ColumnA,ColumnB,ColumnC
01/02/2015,ABC,111
01/03/2015,DEF,222
01/03/2015,HHH,333
01/05/2015,XYZ,767
The number of rows in this file will vary but the columns will remain the same. I need to extract the date column A, row 1 and date from column A, last row.
In this case I need to extract 01/02/2015 and 01/05/2015. Next I want to store both these dates in separate variable.
How can I achieve this? I have no idea where to begin. The only thing I have is this:
for /F "tokens=1 delims=," %%s in (IntFlow.csv) do #echo %%s
This one line of code cuts the first column and echo's it. I don't know what do next. Please advice.
Use a for /f loop to parse the csv file line-by-line. Use skip=1 to skip the header row. Only set the first variable if it isn't already defined. Then just set the last variable on every loop iteration. It's bound to be correct sooner or later, right? :)
#echo off
setlocal
set "first="
set "last="
for /f "skip=1 usebackq delims=," %%I in ("test.csv") do (
if not defined first set "first=%%~I"
set "last=%%~I"
)
echo %first%
echo %last%
tokens=1 is already the default behavior. usebackq allows you to enclose the filename in quotes, so it should work if your csv filename includes a space or some other unsavory character in the name.
I see in your comment you've decided to use PowerShell instead. Here's a PowerShell script that'll let you populate variables with the dates.
$csv = ipcsv test.csv
$first = ($csv | select -first 1 | %{ $_.ColumnA })
$last = ($csv | select -last 1 | %{ $_.ColumnA })
$first
$last
Related
Remove Commas in CSV without adding extra columns
I have this script intended to remove "extra commas" within a string in a CSV. #echo off & setlocal EnableDelayedExpansion echo Removing commas from FILE CSV file... type nul > "H:\CONVERT_CSV.csv" for /F "delims=" %%i in ('type "H:\FILE.csv"') do ( set row=%%i set row=!row:"=! echo.!row!>> "H:\CONVERT_CSV.csv" ) But my script, instead of removing the comma, it adds an extra field within that record. I know the problem sits in the set row!row"=! line, but I can not figure out what. Please help Example From this: Column A,Column B Now,All,2 Then,After To this: Column A,Column B Now,All 2 Then,After EDIT: EXTRA INFO: the csv has: 123 ,the time, "Gone, For good", 2023 Then it should become: 123 ,the time, "Gone For good", 2023
batch to isulate last item in a text file with muliple item
for kindness I have a myTextFile.txt that contain something like this: "C0001.MXF" "C0002.MXF" "C0003.MXF" ... "C0020.MXF" I would like do a batch that echo only the last item, in this case C0020.mxf Possibly I need to put the result in 2 variables: one called var1 = "C0020.MXF" and another call var2 = C0020.MXF (the same of var1 without the "") How can I do? thanks
for /f %%a in (mytextfile.txt) do set var=%%~a echo %var% derived from any of hundreds of similar solutions. Since you claim this doesn't work, then the only conclusion I can draw is that the file is a single line, not a series of lines as proposed. for /f "delims=" %%a in (mytextfile.txt) do for %%b in (%%a) do set var=%%~b echo %var%
Batch script extract contents between two strings
I am trying to write this Batch script to extract the two parameters from an XML file that looks like this: <?xml version="1.0" encoding="utf-8" ?> <!--<var name="SqlConnection" value="data source=SERVERNAME;initialcatalog=DB_NAME;user id=JackDaniels;password=G235X" />--> <var name="SqlConnection" value="data source=SERVERNAME;initial catalog=DB_Name;Integrated Security=SSPI" /> My objective is to extract SERVERNAME and DB_NAME from the line that is not commented out. So far my code looks like this: #echo off setlocal enableextensions disabledelayedexpansion set "connectionString=" set result="" set "INPUT_FILE=DBConnection.config" FOR /F "tokens=* delims=<var eol=!" %%x in (%INPUT_FILE%) DO ( ECHO %%x ) PAUSE I'm just not sure how to get everything right after "data source=" and "initial catalog=". Is there an easy way to do this?
The adequate way to extract this data is not via a Batch file, but with the methods suggested in a comment. However, the Batch file below perform this extraction in a relatively simple way: #echo off setlocal EnableDelayedExpansion rem From the line that have "<var" followed by "value"... for /F "delims=" %%a in ('findstr "\<var.*value" input.txt') do ( rem ... process the parts separated by space or equal sign, excepting if enclosed in quotes... for %%b in (%%a) do ( rem ... and store the part after "value" variable if "!var!" equ "value" set "value=%%~b" set "var=%%~b" ) ) rem Separate "value" string at semicolons and assign each part via SET command for %%a in ("%value:;=" "%") do set %%a rem Show results: echo data source=%data source% echo initial catalog=%initial catalog% echo Integrated Security=%Integrated Security% Output example: data source=SERVERNAME initial catalog=DB_Name Integrated Security=SSPI Of course, if the data format described in the code changes, the program will fail...
#ECHO Off SETLOCAL SET "sourcedir=U:\sourcedir" SET "filename1=%sourcedir%\q42420941.txt" FOR %%v IN (initial catalog data source initial_catalog data_source) DO SET "%%v=" FOR /f "delims=<>" %%z IN ('findstr /B /L /C:"\<var name" "%filename1%"') DO ( FOR %%y IN (%%z) DO ( FOR %%a IN (%%~y) DO ( SET "alive=Y" FOR %%m IN (initial catalog data source) DO IF /i "%%a"=="%%m" SET "alive="&set %%a=Y IF DEFINED alive ( IF DEFINED initial IF DEFINED catalog SET "initial_catalog=%%a" IF DEFINED data IF DEFINED source SET "data_source=%%a" ) IF DEFINED catalog IF NOT DEFINED initial SET alive=y IF DEFINED source IF NOT DEFINED data SET alive=y IF DEFINED alive FOR %%v IN (initial catalog data source) DO set "%%v=" ) ) ) ECHO %initial_catalog% and %data_source% GOTO :EOF You would need to change the setting of sourcedir to suit your circumstances. I used a file named q42420941.txt containing your data for my testing. first, clear out the variable-names. next, parse each line that passes the findstr which is looking for lines that /L literally /B begin with the /c: character-sequence "< escaped by \) and take the first token delimited by > or <. This neatly strips the awkward <> from the string and assigns the contents of the selected line to %%z Next, use a simple for to present each token in the line to %%y. Then with the quotes stripped off of %%y assign each token to %%a. Search for a match against the keywords, and set the variable of the same name if found. Clear alive if a keyword is found. If the string in %%a is not one of the keywords, then check to see whether initial and catalog are both set. If so, this is the required string, so assign it. if catalog is found but not initial then mark as alive If alive is still set, then we can clear the flags and wait for the start of another sequence. OK - it has its faults. It won't detect initial_catalog/data_source if either is one of the four keywords (unlikely) and it assumes that the wanted data is the token following the two keywords - the = becomes a separator in the for list.
Since many of you suggested that Batch is not an adequate way of doing this, I decided to play around with PowerShell, and was able to accomplish what I wanted with just a few lines, and some Regex! $configPath = $PSScriptRoot + "DBConnection.config" [xml]$XmlDocument = Get-Content -Path $configPath $dataSource = $XmlDocument.var.Attributes[1].Value.ToString() # Extract the uncommented line, and the second attribute "value" $serverName = ($dataSource -split 'data source=([^;]*);')[1] $db_name = ($dataSource -split 'initial catalog=([^;]*);')[1] $user_id = ($dataSource -split 'id=([^;]*);')[1] $user_pass = ($dataSource -split 'password=([^;]*)')[1]
Batch script issue for getting dynamic column of csv file?
This is script which i am using for export the data from csv and write into a .trn extension file. The code execute for certain column of csv file data right, i mean to say if the column no indexing A to Z, code execute right,but when after the Z column means for AA,AB ect the data gets wrong. Code you can see here: Batch Script #echo off set "line1=^%%a,cf,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,,,,,,,,,,,,," set "line2=^%%a,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,qbconid,,,^%%b,,,,,,,,," ....................................................................................... ....................................................................................... set "line26=^%%a,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,txtstate,,,^%%z,,,,,,,,," set "line27=^%%a,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,pmtper,,,^%%aa,,,,,,,,," set "line28=^%%a,#,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,minfee,^%%ab,,,,,,,,," (for /F "tokens=1-28 skip=1 delims=," %%a in ('type "NewPortfolios.csv"') do ( echo %line1% echo %line2% ............ ............ echo %line26% echo %line27% echo %line28% )) >"file.trn" And the csv file data is: A B C AA AB $portcode $qbconId $name ....... $pmtper #minfee asingh12 123456789 Ajay Singh....... ajay 123 Output is: asingh12,cf,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,,,,,,,,,,,,, asingh12,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,qbconid,,,123456789,,,,,,,,, ................................................................................... asingh12,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,pmtper,,,asingh12a,,,,,,,,, asingh12,#,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,minfee,asingh12b,,,,,,,,, In the above 2 lines you can see that the column value is coming wrong. As you can see CSV data the last 2 column value should be ajay and 123 but it is coming as asingh12a and asingh12b.These are the values of a and b.It should come from AA and AB. Can someone please help me in this.? **EDITED HERE** Hi Aacini - Hope you are doing well. I am facing again an issues with .csv file convert in to .txt file. Now this time getting a data from .csv with Quotes(") that's why the batch script is not running properly. Please find the sample data from .csv in to .txt format- "_portcode","_qbconId","_name","_address","_city","_state","_zip","_bmeth","_ffee","_brak1","_rate0","_brak2","_rate1","_brak3","_rate2","_brak4","_rate3","_brak5","_rate4","_rate5","_bre","_custact","_custody","_qbmgrid","_refby","_txstate","_pmtper","_minfee" "schorira","001c000000WtrL1AAJ",,"9232 Collegeview Cir","Bloomington","MN","55437","T",,"1000000.0","1.25",,,,,,,,,"1.0","Robert M. Schofield IRA Rollover",,"Schwab","JTB","Patrick Stephens","MN","Quarterly","0". Previous time in .csv data there is no Quotes. Please help me out. If you want more clarification from my side please let me know. I appreciate your help in advance. Hi Acini, Till now we are waiting for your reply. Please help me out.
EDIT: I fixed a couple details, it should correctly run now. EDIT #2: Subtle bug fixed, and an example added #echo off set "letter=abcdefghijklmnopqrstuvwxyz" set "line1=!a!,cf,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,,,,,,,,,,,,," set "line2=!a!,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,qbconid,,,!b!,,,,,,,,," set "line26=!a!,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,txtstate,,,!z!,,,,,,,,," set "line27=!a!,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,pmtper,,,!aa!,,,,,,,,," set "line28=!a!,#,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,minfee,!ab!,,,,,,,,," setlocal EnableDelayedExpansion (for /F "usebackq skip=1 delims=" %%a in ("NewPortfolios.csv") do ( set i1=0 set "line=%%a" for %%b in ("!line:,=" "!") do for /F "tokens=1,2" %%i in ("!i1! !i2!") do ( if %%i lss 26 ( set var=!letter:~%%i,1! ) else ( set var=a!letter:~%%j,1! ) set "!var!=%%~b! set /A i1+=1, i2=i1-26 ) echo %line1% echo %line2% echo %line26% echo %line27% echo %line28% )) > file.trn Previous Batch program can output up to a maximum of 52 columns (a..z aa..az), but it is very easy to increment this range. With this input data: A, B, C, D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z, AA, AB $portcode,$qbconId,$name, D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z,$pmtper,#minfee asingh12,123456789,Ajay Singh,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z,ajay,123 the output is: asingh12,cf,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,,,,,,,,,,,,, asingh12,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,qbconid,,,123456789,,,,,,,,, asingh12,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,txtstate,,,Z,,,,,,,,, asingh12,$,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,pmtper,,,ajay,,,,,,,,, asingh12,#,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,y,minfee,123,,,,,,,,,
Sort files by Timestamp with the Windows DIR command
I want to sort following files with DIR by their Timestamp, which is defined in the filename - oldest first: Timestamp = YYYYDDMM NAME_20121410.dat NAME_20121509.dat NAME_20121609.dat The result should look like this: NAME_20121509.dat NAME_20121609.dat NAME_20121410.dat How do i achieve that with an one-liner?
As the date format is stupid, you need to split first the filenames and sort the rearranged names. Not a one liner, but it works (inside a batch file). #echo off ( for %%A in (*_*.dat) do #( call set "name=%%A" call set "nameDate=%%name:*_=%%" call set "Year=%%nameDate:~0,4%%" call set "Day=%%nameDate:~4,2%%" call set "Month=%%nameDate:~6,2%%" call echo %%Year%%%%Month%%%%Day%% %%name%% ) ) | sort | ( for /F "tokens=1,* delims= " %%A in ('more') DO #echo %%B )
if you can use PowerShell, try this: PS II> ls *.dat | sort {$_ -replace '(\d{2})(\d{2})\.*','$2$1'}