Arrays to filter, copy and past - quick macro method - performance

I want to make an Excel workbook that I have much quicker.
I have a big product database with the product names, quantities, delivery number and delivery date (ProductDB). I put in another sheet the products that I have sold (product names and quantity sold) and want to filter and copy those that are corresponding from the database so I can calculate in the second step the remaining quantity and past the remaining quantity to the database.
Everything is working well and the calculation is good. The only thing is, the Advancedfilter xlfiltercopy option is too slow if I have to input 5000 lines of product names.
I have heard that arrays are much faster. How could I do that? The current way I do it is like this:
Sub UseFilter()
Application.ScreenUpdating = False
Sheet1.Range("G1:Z100000").Cells.Delete
Dim lastrow As Long, c As Range
Dim myrange As Range
Dim rngCell As Range
Dim wksSheet As Worksheet
Dim wksSheetDB As Worksheet
lastrow = Sheet1.Cells(Rows.Count, "A").End(xlUp).Row
Sheet1.Columns("G").NumberFormat = "0"
Filter product codes from the database according to sold product codes:
Set myrange = Range("A1:A" & lastrow)
For Each c In myrange
If Len(c.Value) <> 0 Then
ThisWorkbook.Worksheets(Worksheets.Count).Columns("A:D").AdvancedFilter xlFilterCopy, _
Sheet1.Range("A1:A" & lastrow), Sheet1.Range("G1"), False
End If
Next
Sort the filtered list first by product code, then by the delivery number:
Dim lngRowMax As Long
Dim wsf As WorksheetFunction
With Sheet1
lastrow = Cells(Rows.Count, 8).End(xlUp).Row
Range("G1:J" & lastrow).Sort Key1:=Range("G1:G" & lastrow), _
Order1:=xlAscending, Key2:=Range("I1:I" & lastrow), _
Order2:=xlAscending, Header:=xlYes, DataOption1:=xlSortTextAsNumbers
Set wsf = Application.WorksheetFunction
lngRowMax = .UsedRange.Rows.Count
End With
I'm only interested in filtering and copying of the corresponding product information (name (A), quantity (B), delivery nr (C) and date (D)). Does anyone know how I can do that?
Thank you very much in advance. I'm really looking forward for a solution that improves the pace of the file. Currently it is unbelievably slow.

i had the same problem with advanced filter being so slow. you might want to consider using dictionary. for my 2 spreadsheets i wanted to compare i made 2 dictionaries and compared the values and it was so amazingly fast. dictionaries are really easy and a simple google search you can find a ton of tutorials and examples. good luck.

There is a possible solution with dictionaries, but I have only one small issue. I will explain after the code:
'Count num rows in the database
NumRowsDB = ThisWorkbook.Worksheets(Worksheets.Count).Range("A2", Range("A2").End(xlDown)).Rows.Count
' --------------------- SAVE DATABASE DATA -----------------------
'Dictionary for all DB data
Set dbDict = CreateObject("Scripting.Dictionary")
Set dbRange = Range("A2:A" & (NumRowsDB + 1))
For Each SKU In dbRange
If Len(SKU.Value) <> 0 Then
' check if the SKU allready exists, if not create a new array list for that dictionary entry
' a list is necessary because there can be multiple entries in the db range with the same SKU
If Not dbDict.Exists(CStr(SKU.Value)) Then
Set prodList = CreateObject("System.Collections.ArrayList")
dbDict.Add CStr(SKU.Value), prodList
End If
' for this specific product code, save the range where the product information is saved in the dictionary
rangeStr = "A" & SKU.Row & ":D" & SKU.Row
dbDict(CStr(SKU.Value)).Add (rangeStr)
End If
Next
' ----------- READ SALE/Reverse/Consumption INFO ------------------
NumRowsSale = Range("A2", Range("A2").End(xlDown)).Rows.Count
Set saleRange = Range("A2:A" & (NumRowsSale + 1))
Dim unionRange As Range
For Each sale In saleRange
' check if the SKU for the sale exists in db
If Len(sale.Value) <> 0 And dbDict.Exists(CStr(sale.Value)) Then
For Each dbRange In dbDict(CStr(sale.Value))
If unionRange Is Nothing Then
Set unionRange = Range(dbRange)
Else
Set unionRange = Union(unionRange, Range(dbRange))
End If
Next
End If
Next
unionRange.Copy Destination:=Range("G2") 'copy all received ranges to G2
Set dbDict = Nothing
The line "NumRowsDB = ThisWorkbook.Worksheets(Worksheets.Count).Range("A2", Range("A2").End(xlDown)).Rows.Count" is not working. I have to refer to another sheet (the last sheet which is the current database) to get the data. What is the problem that I cannot refer to another sheet in the same workbook?
Thank you for your suggestions.

Related

VBA to hide SlicerItems

I have a slicer that contains items from a pivot table. My users wants the ability to hide certain slicer items even though these items contains data. Is there a way to hide sliceritems via VBA code?
Why not just filter them out? If this is an OLAP PivotTable, see Pivot Table filter out only 1 option
If this is a "Traditional" PivotTable, then you can use a variation of the code I posted at Pivotfields multiple filter
Here's some amended code:
Sub FilterSlicer_Inverse()
Dim slr As Slicer
Dim sc As SlicerCache
Dim si As SlicerItem
Dim i As Long
Dim vItem As Variant
Dim vSelection As Variant
Dim pt As PivotTable
Set sc = ActiveWorkbook.SlicerCaches("Slicer_test")
vSelection = Array("1", "2") <= List the items you want to hide in here
For Each pt In sc.PivotTables
pt.ManualUpdate = True 'Stops PivotTable from refreshing after each PivotItem is changed
Next pt
With sc
.ClearAllFilters
On Error Resume Next 'In case one of the items isn't found
For Each vItem In vSelection
.SlicerItems(vItem).Selected = False
Next vItem
On Error GoTo 0
End With
For Each pt In sc.PivotTables
pt.ManualUpdate = False
Next pt
End Sub
Note that it doesn't matter if you change the PivotItems or the SlicerItems...you get the same result.

Excel VBA - Stops Selecting Visible Rows on Filter after many interations

My code takes a file name from TextBox1, opens that file and indexes through all the unique values in column B. It then takes a second file, file name in TextBox2, and filters it based on the current index from the first file. It then takes the filtered results and copies them to a sheet in the new workbook. A new sheet is then generated in the new workbook to paste the next filtered result.
My issue is that I have many rows of data and for some reason the filtered data is not be selected after many iterations. My program selects all filtered data when it starts, but at some point it just begins selecting the headers instead of all the visible cells. Let me know if I am missing something or if there is a quick workaround. Thank you.
Sub NewFileGenerate()
Dim I As Integer
Dim N As Integer
Dim X As Integer
Dim IndexedCell As String
X = 1
Windows(TextBox1.Value).Activate
'Need to count only populated cells
I = Sheets(1).Columns(2).Cells.SpecialCells(xlCellTypeConstants).Count
Set Newbook = Workbooks.Add
For N = 2 To I - 1
Application.CutCopyMode = True
Windows(TextBox1.Value).Activate
IndexedCell = Cells(N, 2).Value
Windows(TextBox2.Value).Activate
With Sheets(1)
With .Range("A1", "Z" & I - 1)
.AutoFilter Field:=5, Criteria1:=IndexedContract
.SpecialCells(xlCellTypeVisible).Copy
End With
End With
Newbook.Activate
ActiveSheet.Paste Destination:=Sheets(X).Range("A1:Z1")
Cells.Select
Selection.Font.Size = 10
Cells.EntireColumn.AutoFit
Cells.Select
X = X + 1
If X > 3 Then
Worksheets.Add(After:=Worksheets(Worksheets.Count)).Name = "Sheet" & X
End If
Application.CutCopyMode = False
Next N
End Sub
I'm guessing that your source worksheet (textbox2.value) has more rows than your index worksheet (textbox1.value). You set I equal to the number of rows in your index worksheet, then you tell the autofilter to only use that number of rows. You need to change the line "With .Range("A1", "Z" & I - 1)" so it picks up all of the rows in your source worksheet.

VLOOKUP using batch scripting

I am new to scripting and perform lot of activity as an analyst using excel sheets.
I have two files with list of items in it.
File1 contains 1 column
File2 contains 2 columns.
I want to check if the list present in column1 of file2 is same as in column1 of file2. If yes then it should print column1File1, column1File2 and coulmn2File2 in file3 else it should print "NA", column1File2, column2File2 in file3.
Please help, It will simplify my work a lot.
I made this program a while ago, although it will iterate through sheets in 1 workbook, and compare cell by cell, it may set you in the right direction. It would take a cell in 1 "master" sheet and then iterate through each sheet to find that in a particular column. After it found it the counter would increment, then it would take the next cell in the master sheet and so on. you could alter to use multiple books and take whatever cells you want and compare them.
Sub Open_Excel()
'Use worksheetNum as sheet to read/write data
Set currentWorkSheet = objExcel.ActiveWorkbook.Worksheets(worksheetNum)
'How many rows are used in the current worksheet
usedRowsCount = currentWorkSheet.UsedRange.Rows.Count
'Use current worksheet cells for values
Set Cells = currentWorksheet.Cells
'Loop through each row in the worksheet
For curRow = startRow to (usedRowsCount)
'Get computer name to ping to
strEmailAddressSource = Cells(curRow,colEmailAddressSource).Value
strServerSource = Cells(curRow,colHostserverSource).Value
strLocationSource = Cells(curRow,colLocationSource).Value
'make the values unique
strconcatenation = strServerSource & strLocationSource
Call Comparison()
Next
End Sub
'********************************************************************************************
'**** Comparison
'********************************************************************************************
'Comparison test
Sub Comparison()
'choose the worksheets to go through
For worksheetCounter = 6 to 9 'workSheetCount
Set currentWorkSheetComparison = objExcel.ActiveWorkbook.Worksheets(worksheetCounter)
usedRowsCountNew = currentWorkSheetComparison.UsedRange.Rows.Count
'First row to start the comparison from
For rowCompare = 2 to (usedRowsCountNew)
strEmailLot = currentWorkSheetComparison.Cells(rowCompare,colEmailAddressLot).Value
comp1 = StrComp(strEmailAddressSource,strEmailLot,0)
comp2 = StrComp(strconcatenation,reportConcat,0)
'check if the values match
If ((comp1 = 0) AND (comp2 = 0)) THEN
countvalue = countvalue + 1
End If
Next
Next
End Sub

How to copy rows of from one sheet to another sheet using vbscript

Suppose I have Sheet(1) in an excel. Now i do also have 2500 rows which has data for the columns from A to BO.Now I want the data to copy from these sheet to another sheet of the same Excel file for 2500 rows but not the whole the columns,rather i need only columns from A to AA data to copy to the new sheet.
So how to frame it using VBscript?
Please help me.
How to copy rows of from one sheet to another sheet using vbscript
To copy data from one sheet to another you can use the Copy en PasteSpecial commands. To do this with a .vbs script do the following:
' Create Excel object
Set objExcel = CreateObject("Excel.Application")
' Open the workbook
Set objWorkbook = objExcel.Workbooks.Open _
("C:\myworkbook.xlsx")
' Set to True or False, whatever you like
objExcel.Visible = True
' Select the range on Sheet1 you want to copy
objWorkbook.Worksheets("Sheet1").Range("A1:AA25").Copy
' Paste it on Sheet2, starting at A1
objWorkbook.Worksheets("Sheet2").Range("A1").PasteSpecial
' Activate Sheet2 so you can see it actually pasted the data
objWorkbook.Worksheets("Sheet2").Activate
If you want to do this in Excel with a VBS macro you can also call the copy and paste methods. Only your workbook object will be something like ActiveWorkbook
This code is Working fine. Just Copy and paste it.
Dim CopyFrom As Object
Dim CopyTo As Object
Dim CopyThis As Object
Dim xl As Object
xl = CreateObject("Excel.Application")
xl.Visible = False
CopyFrom = xl.Workbooks.Open("E:\EXCEL\From.xls")
CopyTo = xl.Workbooks.Open("E:\EXCEL\To.xls")
For i = 0 To 1
''To use a password: Workbooks.Open Filename:="Filename", Password:="Password"
If i = 0 Then
CopyThis = CopyFrom.Sheets(1)
CopyThis.Copy(After:=CopyTo.Sheets(CopyTo.Sheets.Count))
CopyTo.Sheets(3).Name = "Sheet3"
Else
CopyThis = CopyFrom.Sheets(2)
CopyThis.Copy(After:=CopyTo.Sheets(CopyTo.Sheets.Count))
CopyTo.Sheets(4).Name = "Sheet4"
End If
Next
CopyTo.Sheets(1).Activate()
CopyTo.Save()
'CopyTo.SaveAs("E:\EXCEL\Check.xls")
xl.Quit()
Sub buildMissingSheet(strMissingSheet) 'Just passing the missing sheet name in
' Master Sheet code
' Working on creating the "Master Sheet" at this time...May need to seperate the the code a little.
Dim GetRows1 As Worksheet
Dim GetRows2 As Worksheet
Dim PutRows As Worksheet
Dim sglRowNum As Single, i%
If strMissingSheet = strMASTERSHEET Then ' Create the strMASTERSHEET
Set GetRows1 = Sheets(strRAWDATA) ' These two sheets could be missing but will code around that later.
Set GetRows2 = Sheets(strDATAWITH) ' The two sheets I am getting rows from
' Just creating a new worksheet here assuming it is missing
Worksheets.Add(After:=Worksheets(5)).Name = strMissingSheet
Set PutRows = Sheets(strMissingSheet) ' Missing sheet must be created before declaring.
PutRows.Select 'Select the sheet being built.
With Cells(1, 1)
.Value = strRAWDATA 'Not copying rows here but left it in this example anyway
.AddComment
.Comment.Visible = False
.Select
.Comment.Text Text:= _
Chr(10) & "Name of sheet including header and the last 32 entries at the time this sheet was updated."
End With
'Here is where we copy the whole row from one sheet to the other.
GetRows1.Rows(1).Copy PutRows.Rows(2) 'Copy header row from existing sheet to "Master Sheet" for instance.
GetRows1.Select
sglRowNum = ReturnLastRow(ActiveSheet.Cells) 'return last row with data on active sheet
' I wanted the last few rows of data "32 rows" so found the end of the sheet this code can be found on the internet in several places including this site.
'Now the code you may have been looking for move 32 row of data from one sheet to another.
For i = 1 To 32 'Start at row 3 on the Put sheet after sheet name and header.
GetRows1.Rows(sglRowNum - (32 - i)).Copy PutRows.Rows(i + 2)
Next i
end sub

VBA : Querying Access with Excel on server

I'm working with Excel project wich helps to calculate the price of any peace of furniture. The first task is to pick all the materials from the database.
This is the code:
Sub Material_search()
Dim cnt As New ADODB.connection
Dim rst As New ADODB.Recordset
Dim rcArray As Variant
Dim sSQL As String
Dim db_path As String, db_conn As String
Dim item As String
item = Replace(TextBox1.Text, " ", "%") ' Search word
sSQL = "Select Data, NomNr, Preke, Matas, Kaina, Tiek from VazPirkPrekes " & _
"Where VazPirkPrekes.PirkVazID IN (SELECT VazPirkimo.PirkVazID FROM VazPirkimo Where VazPirkimo.Sandelys like '%ALIAVOS')" & _
" and Year(VazPirkPrekes.Data)>=2011 and Preke Like '%" + item + "%' and Kaina > 0" & _
" Order by Preke, Data Desc"
db_path = Sheets("TMP").Range("B6").value
db_conn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & db_path & ";"
cnt.Open db_conn
rst.Open sSQL, cnt, adOpenForwardOnly, adLockReadOnly
ListBox1.Clear
If Not rst.EOF Then
rcArray = (rst.GetRows)
rcArray = WorksheetFunction.Transpose(rcArray)
Dim a As Variant
With ListBox1
.ColumnCount = 6
.list = rcArray
.ListIndex = -1
End With
End If
rst.Close: Set rst = Nothing
cnt.Close: Set cnt = Nothing
Label4.Caption = UBound(ListBox1.list) + 1
End Sub
recently I came up with some trouble while querying Access mdb file. The problem is when database file is on local disk, the search works very fast, but when i put database file on server, the search takes 10 times longer, which is not acceptable.
Is there any optimisation for this code ? or is it a server problem
Thanks in advance
That query requires Access' database engine retrieve all 190K rows from both tables. It's not surprising it is slow, and the slowness is compounded when the db engine must retrieve 2 * 190K rows across the network.
If TextBox1.Text contains "foo", this is the statement you're asking the db engine to run:
Select Data, NomNr, Preke, Matas, Kaina, Tiek
from VazPirkPrekes
Where
VazPirkPrekes.PirkVazID IN (
SELECT VazPirkimo.PirkVazID
FROM VazPirkimo
Where VazPirkimo.Sandelys like '%ALIAVOS')
and Year(VazPirkPrekes.Data)>=2011
and Preke Like '%foo%'
and Kaina > 0
Order by Preke, Data Desc
The engine must retrieve all 190K rows from the VazPirkimo table before it can determine which of them include Sandelys values which end with "ALIAVOS". If your selection criterion was for values which start with "ALIAVOS", the engine could use an index on Sandelys to limit the number of rows it must retrieve from VazPirkimo. However, since that approach is probably not an option for you, consider adding a numeric field, Sandelys_group, to VazPirkimo and create an index on Sandelys_group. Give all rows where Sandelys ends with "ALIAVOS" the same Sandelys_group number (1). Then your "IN ()" condition could be this:
SELECT VazPirkimo.PirkVazID
FROM VazPirkimo
Where VazPirkimo.Sandelys_group = 1
The index on Sandelys_group will allow the db engine to retrieve only the matching rows, which will hopefully be a small subset of the 190K rows in the table.
There are other changes you can make to speed up your query. Look at this criterion from your WHERE clause:
Year(VazPirkPrekes.Data)>=2011
That forces the db engine to retrieve all 190K rows from VazPirkPrekes before it can determine which of them are from 2011. With an index on Data, this should be much faster:
VazPirkPrekes.Data >= #2011-01-01# AND VazPirkPrekes.Data < #2012-01-01#
This WHERE criterion will be faster with an index on Kaina:
Kaina > 0
Your ORDER BY begs for indexes on Preke and Data.
Order by Preke, Data Desc
Any or all of those changes could help speed up the query, though I don't know by how much. The killer is this WHERE criterion:
Preke Like '%foo%'
The issue here is similar to the problem with the "Sandelys like" comparison. Since this asks for the rows where Preke contains "foo", rather than starts with "foo", the db engine can't take advantage of an index on Preke to retrieve only the matching rows. It must retrieve all 190K VazPirkPrekes rows to figure out which match. Unless you can use a different criterion for this one, you will be limited as to how much you can speed up the query.
Thanks for the optimization tips, but as I said the problem occurs only when I put data base file on server. And there is not much help from optimization. But I thought about other idea.
The search of empty blank "" returns about 40k records (these records covers everything I need) . So I'm going to put all these records on a distinct sheet on workbook_activate event and later do the query only in that sheet.
Sub Database_upload()
Application.DisplayAlerts = False
On Error Resume Next
Sheets("DATA_BASE").Delete
On Error GoTo 0
Application.DisplayAlerts = False
Sheets.Add
ActiveSheet.name = "DATA_BASE"
Sheets("DATA_BASE").Visible = False: Sheets("DARBALAUKIS").activate
Dim cnt As New ADODB.connection
Dim rcArray As Variant
Dim sSQL As String
Dim db_path As String, db_conn As String
Dim item As String
Dim qQt As QueryTable
item = "" 'search for empty blanks
sSQL = "Select Data, NomNr, Preke, Matas, Kaina, Tiek from VazPirkPrekes " & _
"Where VazPirkPrekes.PirkVazID IN (SELECT VazPirkimo.PirkVazID FROM VazPirkimo Where VazPirkimo.Sandelys like '%ALIAVOS')" & _
" and Year(VazPirkPrekes.Data)>=2011 and Preke Like '%" + item + "%' and Kaina > 0" & _
" Order by Preke, Data Desc"
db_path = Sheets("TMP").Range("B6").value
db_conn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & db_path & ";"
db_conn = "ODBC;DSN=MS Access 97 Database;"
db_conn = db_conn & "DBQ=" & db_path
Set qQt = Sheets("Sheet1").QueryTables.Add(connection:=db_conn, Destination:=Sheets("Sheet1").Range("A1"), Sql:=sSQL)
qQt.Refresh BackgroundQuery:=False
End Sub
Results:
Program takes longer on startup, but the search time is acceptable - for me the problem is solved :)

Resources