Selected columns without duplicates using vbs - vbscript
It’s really appreciated if someone would have help me on this using vbs script.
I have text file with 4 gb data (below sample data--at end it is) and I need only these three columns data out of my attached textfile
"20150901162037","20150901162037","LDAP_Prod/xyrep, tett"
Without duplicate data since I have many of them are similar ldap ids are there.
so i need to take only with unique of ids with three columns,Please can you help me how to achive using vbs script.
Finally I need my output will be following way with out dulicates:
"20150901162037","20150901162037","LDAP_Prod/xyrep, tett"
"20150901162037","20150901162037","LDAP_Prod/ttin_tess"
.
…
my sample data is here:
"100aEB","20150901162037","20150901162037","LDAP_Prod/xyrep, tett","N","11937.91666666667","0","Test_Plan","360610","BAU","Version1","Budget","LDDRR","31029","21240","36542_31029","2016","Jan","6050210000",""
"100aEB","20150901162037","20150901162037","LDAP_Prod/ xyrep, tett ","N","11937.91666666667","0","SS_Plan","360610","BAU","Version1","Budget","LDGR","31029","21240","36542_31029","2016","Feb","6050210000",""
"100aEB","20150901162037","20150901162037","LDAP_Prod/ttin_tess","N","11937.91666666667","0","LLL_Plan","360610","BAU","Version1","Budget","LDGR","31029","21240","36542_31029","2016","Aug","6050210000",""
"100aEB","20150901162037","20150901162037","LDAP_Prod/ xyrep, tett ","N","11937.91666666667","0","SS_Plan","360610","BAU","Version1","Budget","LDGR","31029","21240","36542_31029","2016","Feb","6050210000",""
"100aEB","20150901162037","20150901162037","LDAP_Prod/ttin_tess","N","11937.91666666667","0","LLL_Plan","360610","BAU","Version1","Budget","LDGR","31029","21240","36542_31029","2016","Aug","6050210000",""
"100aEB","20150901162037","20150901162037","LDAP_Prod/ttin_tess","N","11937.91666666667","0","LLL_Plan","360610","BAU","Version1","Budget","LDGR","31029","21240","36542_31029","2016","Aug","6050210000",""
"100aEB","20150901162037","20150901162037","LDAP_Prod/ttenetess","N","11937.91666666667","0","LLL_Plan","360610","BAU","Version1","Budget","LDGR","31029","21240","36542_31029","2016","Aug","6050210000",""
"100aEB","20150901162037","20150901162037","LDAP_Prod/ttin_tess","N","11937.91666666667","0","LLL_Plan","360610","BAU","Version1","Budget","LDGR","31029","21240","36542_31029","2016","Aug","6050210000",""
Describe your input file using a schema.ini file:
[33851749.txt]
Format=CSVDelimited
ColNameHeader=False
DecimalSymbol=.
Col1=AA Text
Col2=BB Text
Col3=CC Text
Col4=DD Text
(minimalistic/need to know version to get it 'to work'; you surely can do better)
Open a ADODB Connection to your input file, select your 3 columns 'distinct'ly, and write the records quoted to StdOut (or a file):
Option Explicit
Const adClipString = 2
Dim oFS : Set oFS = CreateObject("Scripting.FileSystemObject")
Dim oDB : Set oDB = CreateObject("ADODB.Connection")
Dim sCS : sCS = Join(Array( _
"Provider=Microsoft.Jet.OLEDB.4.0" _
, "Data Source=" & oFS.GetAbsolutePathName("..\data") _
, "Extended Properties=""text""" _
), ";")
oDB.Open sCS
Dim sSQL : sSQL = "Select Distinct BB, CC, DD from [33851749.txt]"
Dim oRS : Set oRS = oDb.Execute(sSQL)
Do Until oRS.EOF
WScript.Echo """" & oRS.GetString(adClipString, 1, """,""", """", """""")
Loop
oDB.Close
That should give you:
cscript 33851749.vbs
"20150901162037","20150901162037","LDAP_Prod/ xyrep, tett"
"20150901162037","20150901162037","LDAP_Prod/ttenetess"
"20150901162037","20150901162037","LDAP_Prod/ttin_tess"
"20150901162037","20150901162037","LDAP_Prod/xyrep, tett"
Related
OLEDB connection avoid auto convert on text to int
when run a simple vbs code to read a csv.
Example CSV:
AB,CD
XYZABDER,TLK431
..
with following code
Set objConnection = CreateObject("ADODB.Connection")
Set objRecordset = CreateObject("ADODB.Recordset")
strPathtoTextFile = "C:\xyz\"
objConnection.Open "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=" & strPathtoTextFile & ";" & _
"Extended Properties=""HDR=YES;FMT=Delimited"""
objRecordset.Open "SELECT * FROM myfile.csv", objConnection
Do Until objRecordset.EOF
MsgBox "AB: " & objRecordset.Fields.Item("AB")
MsgBox "CD: " & objRecordset.Fields.Item("CD")
objRecordset.MoveNext
Loop
it alsways converts "TLK431" to 431. As soon as i change the "TLK" to "TLB" or whatever its read as string "TLB431" for "TL" or "TLK" its converting to int 431.
Cant understand the reason for this auto conversion.
For now my workaround is a schema file. But i doesnt like it and want to ask for a better solution and the reason for this conversion.
Thx
This may be an encoding issue. If the CSV file contains Unicode characters, ensure that both the CSV and VBS files are saved as Unicode (UTF-16), otherwise, save both files as ANSI. Do not use UTF-8.
Use VBScript to VLOOKUP values, or evaluate formula?
For processing orders we're using VBScripts to import them into accounting software. There are several suppliers, each with their own file format, mostly CSV and XML. The first step is to extract all the order lines (custom function per supplier), do some additional processing and then write it to the database, which is the same for all suppliers.
One new supplier uses Excel files with all the order lines in one sheet, except for the corresponding VAT percentage value which are available in another sheet. The VAT percentage per item can be looked up using the itemcode from the order sheet.
The company only has LibreOffice Calc and I understand you could do something like this in macro. However, it is a fully automated process and every other file is already handled by VBScript so I'd rather not make an exception or handle just this one order type manually (opening Calc and running the macro). So it has to be VBS and LibreOffice in this case.
Here is the VBScript code I have so far:
Option Explicit
' variables
Dim oSM, oDesk
Dim sFilename
Dim oDoc
Dim oSheet
Dim iLine
Dim sCode, iCount, sDesc, fCost, Perc
Set oSM = WScript.CreateObject("com.sun.star.ServiceManager")
Set oDesk = oSM.createInstance("com.sun.star.frame.Desktop")
sFilename = "file:///C:/orders/import/supplier_orderlist_08-01-2019.xls"
set oDoc = oDesk.loadComponentFromURL( sFilename, "_blank", 0, Array() )
set oSheet = oDoc.getSheets().getByName("Orderlist")
For iLine = 11 to 12 ' testing first 2 lines
sCode = oSheet.getCellByPosition(1, iLine).getString()
iCount = oSheet.getCellByPosition(2, iLine).getString()
sDesc = oSheet.getCellByPosition(5, iLine).getString()
fCost = oSheet.getCellByPosition(8, iLine).getString()
'lookup doesn't work
Perc = Macro_VLOOKUP(sCode, oDoc)
WScript.Echo sCode & " - " & iCount & "x - " & sDesc & " => " & fCost & ", " & Perc & "%"
Next 'iLine
WScript.Quit 1
Function Macro_VLOOKUP(SearchValue, oDocGlob)
Dim oSheetLook, CellRange
Dim Column, Mode, svc, arg, Value
Set oSheetLook = oDocGlob.getSheets().getByName("Itemlisttotal")
Set CellRange = oSheetLook.getCellRangeByName("A1:B10000")
Column = 1
Mode = 0
svc = createUnoService("com.sun.star.sheet.FunctionAccess") '<- error: variable not defined
arg = Array(SearchValue, CellRange, Column, Mode)
Value = svc.callFunction("VLOOKUP", arg)
Macro_VLOOKUP = Value
End Function
It gives an error on the line with createUnoService:
Variable not defined 'createUnoService'
which is probably a LibreOffice Basic function and needs to be translated to the VBScript equivalent. There isn't much documentation or examples on this, so I can only guess, but Set svc = WScript.CreateObject("com.sun.star.sheet.FunctionAccess") also doesn't work and gives a "class name not found" error.
Is it possible to do a VLOOKUP (or something similar) from VBScript in LibreOffice Calc?
Or is there a way to evaluate a cell formula from a string at runtime?
VBScript/SQL Formatting Issue
Okay so the script that I have written is almost fully functional but I would like to add two things to it. What it currently does is you type in a CallID number that is associated with other data on a database in SQL Server. When you type in the number into the msgbox it retrieves all the other columns that are associated with that particular number and outputs it to the command prompt and also outputs it to a text file on the hard drive. This is good, but the formatting is horrible. How would I go about improving the formatting of the file so it is more reading. Currently it is just a line with space seperating each piece of data. Also what I would like to be able to do is also have the name of each Column under each piece of data that is associated with that column. Any help would be appreciated. Here is my code with sensitive information omitted:
Dim strQuery
strQuery = InputBox("Enter CallID Please:")
Dim sServer
Dim sLogin
Dim sPwd
Dim sDb
Dim oCn
Dim oRs
Dim strFullQuery
Dim strfield
Const ForReading = 1
sServer = ""
sLogin = ""
sPwd = ""
sDb = ""
Set oCn = CreateObject( "ADODB.Connection" ) ' set oCn to create an object called ADODB.Connection
Set oRs = CreateObject( "ADODB.Recordset" ) ' set oRs to create an object called ADODB.Recordset
oCn.ConnectionString = "PROVIDER=SQLOLEDB" & _
";SERVER=" & sServer & _
";UID=" & sLogin & _
";PWD=" & sPwd & _
";DATABASE=" & sDb & " "
oCn.ConnectionTimeout=600
oCn.open 'Open the connection to the server
strFullQuery = "select * from dbo.CallLog where CallID=" + strQuery 'this is the SQL statement that runs a query on the DB
oRs.Open strFullQuery,oCn 'This opens the record set and has two parameters, strFullQuery and oCn
If oRs.EOF Then 'If open record set is at the end of file then...
wscript.echo "There are no records to retrieve; Check that you have the correct record number." 'echo that there is no records to retrieve.
End if
'if there are records then loop through the fields
oRs.MoveFirst 'move to the first object in the record set and set it as the current record
Do Until oRs.EOF ' Do while not open record set is not end of file
Set objFileSystem = WScript.CreateObject("Scripting.FileSystemObject") 'Set objFileSystem to create object Scripting.FileSystemObject
Set objOutPutFile = objFileSystem.CreateTextFile("c:\test.txt", True) 'Set objOutPutFile to create object objFileSystem.CreateTextFile
objOutPutFile.WriteLine strColumnNames
strfield = oRs.GetString
if strfield <> "" then 'If strfield doesn't equal "" then
WScript.echo strfield
objOutPutFile.WriteLine strfield &"|"
'objFileSystem.close
objOutPutFile.close
end If
'oRs.MoveNext 'Move to the next object in the record set
Loop
oCn.Close
You can add space to make fixed-widths. Let's say you know that every field will be 20 characters or less: objOutPutFile.WriteLine strfield & String(20-Len(strfield)," ") & "|"
VBscript comparing md5 "If then"
I am trying to compare some Md5's of some files to see if they are the same after I have copied them to another drive, and if they are the same then to delete the original files. I am lost on the syntax because I have never done something like this before.
I am using a free utility vmd5.exe (command line) to get the md5's. I am just not sure how to tell vbscript that if the outputs are the same then to go ahead and delete the files. This is just a snippet of the part where I am trying to compare the two md5's but this is what I have so far:
Dim md5Command, md5Command2
md5Command = "C:\Program Files\vmd5.exe " vmd5 & " " & C:\Testscripts\
md5Command2 = "C:\Program Files\vmd5.exe " vmd5 & " " & E:\CopyTestFolder\
If md5Command = md5Command2 then
objFSO.DeleteFolder("C:\ScriptOutput") 'Can either delete entire archived folder, or just .zip files in folder
objFSO.DeleteFile("C:\Testscripts\Archive*.evtx") 'This will be path where archived logs are located once finished
Else
End If
This syntax is wrong for sure. But from what I know about IF then's I need it to look something like this. I am just not sure if I need to be telling the script to run the commands in the "set" part or how I can tell the script to grab the md5 output from the command line and compare it to another.
After the Else statement I would like it to either end the script or just output a text file with the different md5's but I am not to that point yet and haven't really decided on anything.
If there is a better way to do something like this I would also be up to do that, this is about all I was coming up with this morning though.
EDIT: I thought of something that might be possible. If I tell the command line to output the contents of the output to a text file then I could compare two different text files and if the contents match then it could proceed with the rest of the script. I don't know how to get this to work but the logic seems to make sense.
Here is what the output of the text file that it creates looks like:
Vallen VMD5 R2009.1215
Filename MD5 sum
------------------------------------------------------------
[C:\ScriptOutput\]
Testzip.zip d5db2ff8c372a12c145170fb7340e682
To tackle your task, you have to solve some sub problems:
String concatenation in VBScript: "... exe " vmd5 & " " - to splice in the content of the variable vmd5, you need the concatenation operator on both sides - but is that what you want to do? " " & C:\Testscripts\ - to append the literal 'C:\Testscripts\', you need to (double) quote the literal - but then you could combine all the components into one string literal.
In VBScript " are used as string literal delimiters; they don't work like backticks in more powerful scripting languages. md5Command must hold the command you want to execute; to get the result of that command is a different kettle of fish.
To shell out/execute a command, you'll have to use the .Exec or the .Run method of the WScript.Shell object and collect the output.
Depending on the (output of) the tool you use, you won't be able to compare the results via the = operator - e.g. the pathes or the order of the files/cheksums may differ. So you'll need a strategy for parsing the captured output.
Which sub problem do you want to deal with first?
As your comments prove, getting the syntax right should be the starting point. This:
Dim aDirs : aDirs = Array("..\data\one", "..\data\two")
' Join an array of the components (no more problems
' with (forgetting (to concat)) separators)
' Use qq() function to lessen the noise
Dim sCmdT : sCmdT = Join(Array( _
"fciv" _
, "-add" _
, qq("§DIR§") _
), " ")
Dim nDir, sDir, sCmd
For nDir = 0 To UBound(aDirs)
sDir = aDirs(nDir)
' Use replace on a template to avoid repetition
sCmd = Replace(sCmdT, "§DIR§", sDir)
WScript.Echo "sCmd: |" & sCmd & "|"
Next
output:
sCmd: |fciv -add "..\data\one"|
sCmd: |fciv -add "..\data\two"|
illustrates 3 methods to make it easier/less errorprone to 'build' (shell) commands or SQL statements. (Implementation of the qq() function is left as exercise).
As I don't have the vmd5 utility, I'll use fciv in the further examples.
The next (version of the) script:
Dim aDirs : aDirs = Array("..\data\one", "..\data\two")
Dim sCmdT : sCmdT = Join(Array( _
"fciv" _
, "-add" _
, qq("§DIR§") _
), " ")
Dim oWSH : Set oWSH = CreateObject("WScript.Shell")
Dim nDir, sDir, sCmd, oExec, sRes
For nDir = 0 To UBound(aDirs)
sDir = aDirs(nDir)
sCmd = Replace(sCmdT, "§DIR§", sDir)
Set oExec = oWSH.Exec(sCmd)
sRes = oExec.Stdout.ReadAll()
WScript.Echo sRes
Next
output:
CmpMd500 - compare md5 checksums
==============================================================
//
// File Checksum Integrity Verifier version 2.05.
//
09fea378b96141413f5f09444573f0f3 ..\data\one\version.txt
4945c1ffd9ceb14c83e003091c6e8455 ..\data\one\README.md
4c4c34f7b6f0863056615d2cbcdf6912 ..\data\one\History.txt
//
// File Checksum Integrity Verifier version 2.05.
//
09fea378b96141413f5f09444573f0f3 ..\data\two\version.txt
4945c1ffd9ceb14c83e003091c6e8455 ..\data\two\README.md
4c4c34f7b6f0863056615d2cbcdf6912 ..\data\two\History.txt
==============================================================
demonstrates the absolute minimum of code to execute a command and capture the output - a production version must add a lot of code for error handling. At the same time, it shows sample output to discuss how to parse/compare the checksums. Can you post sample output of your vmd5 utility?
Whether you get the output of the md5 utility directly (my above sample) or from a file, you'll need a Regular Expression to parse the string into data that can be processed further. A simple script to work with files like you published in your question:
Dim reMd5File : Set reMd5File = New RegExp
reMd5File.Global = True
reMd5File.Multiline = True
reMd5File.Pattern = "^(\S+)\s+(\w{32})"
Dim sDir : sDir = "..\data\three"
Dim oFile
For Each oFile In goFS.GetFolder(sDir).Files
Dim sAll : sAll = oFile.OpenAsTextStream(ForReading).ReadAll()
WScript.Echo sAll
Dim oMTS : Set oMTS = reMd5File.Execute(sAll)
Dim oMT
For Each oMT In oMTS
WScript.Echo "** parsed:", qq(oMT.Submatches(1)), qq(oMT.Submatches(0))
Next
Next
output:
CmpMd501 - compare md5 checksums
==================================================================
Vallen VMD5 R2009.1215
Filename MD5 sum
------------------------------------------------------------
[C:\ScriptOutput\]
Testzip.zip d5db2ff8c372a12c145170fb7340e682
version.txt 09fea378b96141413f5f09444573f0f3
README.md 4945c1ffd9ceb14c83e003091c6e8455
History.txt 4c4c34f7b6f0863056615d2cbcdf6912
** parsed: "d5db2ff8c372a12c145170fb7340e682" "Testzip.zip"
** parsed: "09fea378b96141413f5f09444573f0f3" "version.txt"
** parsed: "4945c1ffd9ceb14c83e003091c6e8455" "README.md"
** parsed: "4c4c34f7b6f0863056615d2cbcdf6912" "History.txt"
********************
Vallen VMD5 R2009.1215
Filename MD5 sum
------------------------------------------------------------
[C:\ScriptOutput\]
Testzip.zip d5db2ff8c372a12c145170fb7340e682
** parsed: "d5db2ff8c372a12c145170fb7340e682" "Testzip.zip"
********************
==================================================================
xpl.vbs: Erfolgreich beendet. (0) [0.14844 secs]
After working thru that code, you'll have no problem with this script, that adds
'storing of the results in dictionaries' to my 'read .Exec output' version:
Dim aDirs : aDirs = Array("..\data\one", "..\data\two")
Dim sCmdT : sCmdT = Join(Array( _
"fciv" _
, "-add" _
, qq("§DIR§") _
), " ")
Dim oWSH : Set oWSH = CreateObject("WScript.Shell")
ReDim aRes(UBound(aDirs))
Dim reMd5File : Set reMd5File = New RegExp
reMd5File.Global = True
reMd5File.Multiline = True
reMd5File.Pattern = "^(\w{32})\s(.+?)\s+$"
Dim nDir, sDir, sCmd, oExec, sRes, oMTS, oMT
For nDir = 0 To UBound(aDirs)
sDir = aDirs(nDir)
sCmd = Replace(sCmdT, "§DIR§", sDir)
Set oExec = oWSH.Exec(sCmd)
sRes = oExec.Stdout.ReadAll()
Set aRes(nDir) = CreateObject("Scripting.Dictionary")
Set oMTS = reMd5File.Execute(sRes)
For Each oMT in oMTS
aRes(nDir)(goFS.GetBaseName(oMT.SubMatches(1))) = oMT.SubMatches(0)
Next
Next
Dim sFile
For nDir = 0 To UBound(aDirs)
For Each sFile In aRes(nDir).Keys
WScript.Echo aRes(nDir)(sFile), sFile
Next
WScript.Echo
Next
output:
===========================================
09fea378b96141413f5f09444573f0f3 version
4945c1ffd9ceb14c83e003091c6e8455 README
0252535193507019a0eb97328d28dd80 robic
4c4c34f7b6f0863056615d2cbcdf6912 History
09fea378b96141413f5f09444573f0f3 version
4945c1ffd9ceb14c83e003091c6e8455 README
4c4c34f7b6f0863056615d2cbcdf6912 History
c46264f8101b6c1609c77b4c674bd327 Rakefile
===========================================
The next - and last, I hope - step would be to do the comparisons (Are files missing from one folder?, Do the checksums for the 'same' file differ?). Any ideas from your side?
Lookup function in VB6
I have a table in an excel file called " Employee_States_File".This table contains two columns Name and States. Both columns are filled with data. The province table contains abbreviation of the States such as " NE, WA" and so. I have another excel file called " States_File" which contains a table that has two columns: Abbreviation and FullStateName. This table is considered to be a lookup table to look for the full state names based on the abbreviation. Now, I want to write a code in VB6 so that when the user click a button, All the abbreviation names in the table of the excel file " Employee_States" are changes into the full state names based on the lookup table of the excel sheet " States_File". does it make sense? Please help,
If you are allowed to convert them to csv files (comma delimited text files) you can use the Jet Database engine and do normal SQL joins. I use this to setup the connection. Public Function OpenTextConnection(ByVal FileName As String) As Connection Dim FSO As FileSystemObject Dim DBFolder As String Dim TS As TextStream Set FSO = New FileSystemObject DBFolder = FSO.GetParentFolderName(FileName) If FSO.FileExists(FSO.BuildPath(DBFolder, "Schema.ini")) Then FSO.DeleteFile (FSO.BuildPath(DBFolder, "Schema.ini")) End If Set TS = FSO.CreateTextFile(FSO.BuildPath(DBFolder, "Schema.ini")) TS.WriteLine "[" & FSO.GetFileName(FileName) & "]" TS.WriteLine "Format=CSVDelimited" TS.WriteLine "ColNameHeader = True" TS.WriteLine "MaxScanRows = 0" TS.Close Set OpenTextConnection = New Connection If FSO.FolderExists(DBFolder) Then OpenTextConnection.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & DBFolder & ";Extended Properties=""text;HDR=Yes;FMT=Delimited;"";" Else MsgBox DBFolder & " Does not exists.", vbExclamation End If End Function Each file is a table in the connection and you can do SQL joins. Note the example is just a simply open up a table. You can use any valid SQL Syntax. Dim DB1 As Connection Dim TB As Recordset Dim FSO As FileSystemObject Dim tImport As New DBImportList Set FSO = New FileSystemObject Set tImport = New DBImportList If FSO.FileExists(FileName) Then Set DB1 = OpenTextConnection(FileName) Set TB = New Recordset TB.Open "SELECT * FROM [" & FSO.GetFile(FileName).Name & "]", DB1, adOpenKeyset, adLockOptimistic, adCmdText End If