Loading data in to a combo box is slow - performance

I have a VB6 application with a search screen. On the search, I have 9 combo boxes. Some of the combo boxes only have a couple items, but some have a couple hundred items. It takes a long time (couple of seconds) to populate the data.
Each combo box is configured the same: Sorted = False, Style = 2 - Dropdown List
3 of the combo boxes have less that 20 items.
1 has 130 items.
4 have approximately 250 items
1 has almost 700 items.
I fill all nine combo boxes with similar code.
While Not RS.EOF
cmbX.List(i) = RS("Description")
cmbX.ItemData(i) = RS("Id")
i = i + 1
RS.MoveNext
Wend
I tried setting Visible = False but it had no effect on performance.
Is there another way to fill the combo box that will perform better than my existing method?

Here is something you can try. According to this post you can shave about 60% off your overhead by using a Windows API function to populate the combo box, instead of the usual AddItem method:
Private Const CB_ERR As Long = -1
Private Const CB_ADDSTRING As Long = &H143
Private Const CB_RESETCONTENT As Long = &H14B
Private Const CB_SETITEMDATA As Long = &H151
Private Declare Function SendMessage Lib "user32" Alias "SendMessageA" (ByVal _
hwnd As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long
Public Sub AddItem(cmb As ComboBox, Text As Variant, Optional ItemData As Long)
Dim l As Long
Dim s As String
If VarType(Text) = vbString Then
s = Text
Else
s = Trim$(Str$(Text))
End If
l = SendMessage(cmb.hwnd, CB_ADDSTRING, 0&, ByVal s)
If l <> CB_ERR Then
SendMessage cmb.hwnd, CB_SETITEMDATA, l, ByVal ItemData
End If
End Sub
Public Sub Clear(cmb As ComboBox)
SendMessage cmb.hwnd, CB_RESETCONTENT, 0, 0&
End Sub
You might be able to shave a little more off by omitting the function call, and just calling the API function directly.

You really need to rethink your design. No user is going to want to choose between 700 items in a combo box. It'll make your app seem cluttered, if you don't correct it.
A picture always comes to my mind when I hear a situation like this:

Some suggestions:
Use With RS.
Use the Recordset object's RecordCount property rather than test for EOF on every iteration (if RecordCount = -1 then you should alter the cursor type, cursor location, etc to ensure RecordCount is supported).
Use a For..Next loop rather than maintain you own iterator variable.
Use the bang operator (!).
For example:
With RS
Debug.Assert .RecordCount >= 0
Dim counter As Long
For counter = 0 To .RecordCount - 1
cmbX.List(counter) = !Description
cmbX.ItemData(counter) = !Id
.MoveNext
Next
End With
Perhaps something else to consider is setting the combo's Sorted property to False and if sorting is required then either use the Recordset's Sort property or do the sorting at source (e.g. using an ORDER BY clause in SQL code).

You can try telling the combobox not to repaint itself while you add the new items. You can do this with WM_SETREDRAW. EDIT - apparently this didn't help, probably because the combo box is being hidden while it's filled, which probably gives you all the same benefits.
Private Declare Function SendMessage Lib "user32" Alias "SendMessageA"( _
ByVal hwnd As Long, ByVal wMsg As Long, _
ByVal wParam As Long, lParam As Any) As Long
Private Const WM_SETREDRAW = &HB
Call SendMessage(cmbX.hWnd, WM_SETREDRAW, 0, 0&)
'DO STUFF'
Call SendMessage(cmbX.hwnd, WM_SETREDRAW, 1, 0&)
Warning: lots of otherwise excellent VB6 websites tell you to use LockWindowUpdate instead. Do not do this, or you will get bugs. Also the disfavour of Raymond Chen!

Why are you prepopulating the list and then repopulating List(i) and ItemData(i)? Instead you should do the following
While Not RS.EOF
cmbX.AddItem RS("Description")
cmbX.ItemData(cmbX.NewIndex) = RS("Id")
RS.MoveNext
Wend
You will see zero performance difference between Robert Harvey's answer and the code above. For added speed, apply MarkJ's answer.
Another problem could be the cursor you are using to bring the data back from the database. Since you are loading every single item from the recordset, there is very little reason to have a server-side cursor. So you might want to specify Client side cursor when retrieving the recordset.

That does seem like a long time. How are you opening your recordsets? Are you using firehose (readonly, forward-only recordset) cursors? If you aren't, you might get some performance improvement from that. Make sure your SQL statements are ONLY returning the data they need for the combo boxes (i.e. DO NOT use a SELECT *).
If your SQL statements contain WHERE clauses or JOINS, make sure you have indexes on the appropriate fields.
If you are using ACCESS as a backend you will get an immediate speed improvement by upsizing to SQL Server Express.

'' Populate COMBOBOX
I had to wait 20 seconds to my program populate the city combo box with 80,000 records
I tried several methods but all they were worse
this was my original code
Me.txt_City.DataSource = tblCities
Me.txt_City.ValueMember = "city"
Me.txt_City.DisplayMember = "city"
and guess what, I just moved the first line of code to the end
and then the population took only 5 seconds
Me.txt_City.ValueMember = "city"
Me.txt_City.DisplayMember = "city"
Me.txt_City.DataSource = tblCities
try it

Related

Not sure how user defined type records/elements are populated

In this code I don't understand how the records IngLower and IngUpper of the LargeInt type are populated. For instance, if I add udtFreeBytesAvail to the watch list and set a breakpoint at the 16th line the records of udtFreeBytesAvail become populated, I know it's because of the parameters of the API function GetDiskFreeSpaceEx, but I don't understand why both records become populated, and not just one for example. And also why both records have different values. I appreciate your help, and sorry if my problem is not well described as I'm new to programming.
Public Type LargeInt
lngLower As Long
lngUpper As Long
End Type
Public Declare Function GetDiskFreeSpaceEx Lib "kernel32.dll" Alias "GetDiskFreeSpaceExA" (ByVal lpDirectoryName As String, lpFreeBytesAvailableToCaller As LargeInt, lpTotalNumberOfBytes As LargeInt, lpTotalNumberofFreeBytes As LargeInt) As Long
Public Function FreeDiskSpace(ByVal sDriveLetter As String) As Double
Dim udtFreeBytesAvail As LargeInt, udtTtlBytes As LargeInt
Dim udtTTlFree As LargeInt
Dim dblFreeSpace As Double
If GetDiskFreeSpaceEx(sDriveLetter, udtFreeBytesAvail, udtTtlBytes, udtTTlFree) Then
If udtFreeBytesAvail.lngLower < 0 Then
dblFreeSpace = udtFreeBytesAvail.lngUpper * 2 ^ 32 + udtFreeBytesAvail.lngLower + 4294967296#
Else
dblFreeSpace = udtFreeBytesAvail.lngUpper * 2 ^ 32 + udtFreeBytesAvail.lngLower
End If
End If
FreeDiskSpace = dblFreeSpace
End Function
Other already explained it, but as you still seem to struggle to understand why it works, let me try and clarify this:
The UDT "keeps" its members (the 2 Long integers in this case) in a consecutive memory location, the 1st byte of lngUpper directly follows the last byte of lngLower, occupying in total 8 bytes (=64 bits) consecutive memory. The API GetDiskFreeSpaceEx writes 64 consecutive bits at the start of the variables provided to it as lpFreeBytesAvailableToCaller etc. Hence it "fills out" both 32 bit Long variables of the UDT.
When you pass a parameter by reference (ByRef - note that it's the implicit default) and then assign it in that procedure...
Public Sub DoThing(ByRef value As Long)
value = 42
End Sub
Then to invoke that procedure you need to give it an argument - if you just give it a literal...
DoThing 10
...then nothing happens, the variable pointer just falls out of scope. But give it a local variable...
Dim local As Long
DoThing local
Debug.Print local
And that should output 42.
This is exactly what's going on here, except instead of a local variable, it's a UDT member - T.S. put it succinctly:
GetDiskFreeSpaceEx populates it – T.S. 8 mins ago
Public Declare Function GetDiskFreeSpaceEx Lib "kernel32.dll" Alias "GetDiskFreeSpaceExA" ( _
ByVal lpDirectoryName As String, _
lpFreeBytesAvailableToCaller As LargeInt, _
lpTotalNumberOfBytes As LargeInt, _
lpTotalNumberofFreeBytes As LargeInt) As Long
The function itself returns a Long integer, but the real output is the ByRef parameters it takes - notice the inputs are passed by value (ByVal).
I don't understand why both records become populated, and not just one for example
Assuming "records" is about the UDT members lngLower and lngUpper, it's really just what a UDT does... You need to play with them a little, see how they work. Try making your own procedure that takes a UDT:
Public Sub DoThing(ByRef value As LargeInt)
value.lngLower = 1
value.lngUpper = 2
End Sub
The API function is doing pretty much exactly that - there are two values because a LargeInt is two values.

Inserting results of Hyperion Smart View VBA Function into list box and/or worksheet range

I've seen answers to questions similar to this one, but I haven't been able to find anything that addresses this exact situation.
Goal: Populate a list box and/or a worksheet with the output of a HypQueryMembers function. For example, I would like to get a list of the descendants of account 10100 without having to perform an ad hoc query and zoom in. I know how to get the return code, e.g. 0 if successful, but I want the actual output. I found some code that populated a list box, but I haven't been able to get it to work for me. I receive the error "Could not set the List property. Invalid property array index." My code follows:
Sub TestQueryMbrs()
Dim X As Integer
Dim arrAccounts
X = HypQueryMembers(Empty, "10100", HYP_DESCENDANTS, Empty, Empty, Empty, Empty, arrAccounts)
If X <> 0 Then
MsgBox "Unable to populate members." & vbCr & vbCr & "Error: " & X, vbCritical + vbOKOnly
Else
UserForm2.ListBox1.List = arrAccounts
UserForm2.Show
End If
End Sub
Any idea what I'm doing wrong? Also, I would like to accomplish the same thing, but populate a worksheet rather than a list box. But one step at a time!
Thanks!
Is 10100 the genuine name of the field? I suspect by you calling it account that 10100 should be replaced by Account instead as that parameter seems to be limited to the field name rather than an individual member. However, I have yet to determine how to get descendents of a particular account/cost centre so your way may be the correct way to do this.
I'd suggest trying that change to just "Account" though and then try adding as variant to Dim arrAccounts and then deleting Dim arrAccounts altogether if that doesn't work.
You may also not be able to populate the listbox before Userform2 is shown? You could wrap application.screenupdating =FALSE then TRUE around the Userform2 changes so the user doesn't see the list box being populated if there is a lag.
I had the same issue today and ran across this post - I realize it's years old...
Oracle's documentation on this function is a bit confusing... at least the example they used at the bottom of the page: https://docs.oracle.com/cd/E72988_01/DGSMV/ch12s15.html
For example, you need to enter the integer value for HYP_DESCENDANTS, which is 2 instead of the text shown in your code.
The following is a good snippet of code to verify your array:
Declare PtrSafe Function HypQueryMembers Lib "HsAddin" (ByVal vtSheetName As Variant, ByVal vtMemberName As Variant, ByVal vtPredicate As Variant, ByVal vtOption As Variant, ByVal vtDimensionName As Variant, ByVal vtInput1 As Variant, ByVal vtInput2 As Variant, ByRef vtMemberArray As Variant) As Long
Sub Example_HypQueryMembers()
sts = HypQueryMembers("INSERT WORKSHEET NAME HERE", "INSERT SMARTVIEW MEMBER HERE", 1, Empty, Empty, Empty, Empty, vArray)
If IsArray(vArray) Then
cbItems = UBound(vArray) + 1
MsgBox ("Number of elements = " + Str(cbItems))
For i = 0 To UBound(vArray)
MsgBox ("Member = " + vArray(i))
Next
Else
MsgBox ("Return Value = " + Str(vArray))
End If
End Sub

Variable to refer to Control array?

I know that it isn't possible to generate a control array at runtime.
However, is it possible to have several control arrays that I sub in at different points, referred to via a variable? IE
dim CArray() as controls
CArray = Labels
Foo(CArray)
CArray = OtherLabels
Foo(CArray)
My attempts to do this so far lead to a compile-time error, as it won't allow the control array actually on the form to be referred to without using a method or element. It gives the error:
Arugment not optional
You could use an object:
Private Sub Form_Load()
Dim CArray As Object
Set CArray = Labels
Foo CArray
End Sub
Sub Foo(CtrlArray As Object)
Dim i as Long
For i = 0 To CtrlArray.ubound
CtrlArray(i).Caption = Rnd
Next
End Sub

Optimize performance of Removing Hidden Rows in VBA

I am using the following code to remove hidden/filtered lines after applying autofilters to a big sheet in VBA (big means roughly 30,000 rows):
Sub RemoveHiddenRows()
Dim oRow As Range, rng As Range
Dim myRows As Range
With Sheets("Sheet3")
Set myRows = Intersect(.Range("A:A").EntireRow, .UsedRange)
If myRows Is Nothing Then Exit Sub
End With
For Each oRow In myRows.Columns(1).Cells
If oRow.EntireRow.Hidden Then
If rng Is Nothing Then
Set rng = oRow
Else
Set rng = Union(rng, oRow)
End If
End If
Next
If Not rng Is Nothing Then rng.EntireRow.Delete
End Sub
The code comes from here: Delete Hidden/Invisible Rows after Autofilter Excel VBA
Moreover I read this thread: Speeding Up Code that Removes Hidden Rows on a Sheet
The situation: I have applied 5 different filters to a table consisting of 12 columns, therefore a lot of rows are filtered out (hidden) after the process. When I try to delete those, the code above takes a very long time. In my case I don't know if Excel was still working, so I had to force an exit. That leads to the following question:
Is there any other way than looping through all the hidden rows and deleting them?
An idea which came to my mind was to copy only the remaining unfiltered (that is non-hidden) content to a new sheet and afterwards delete the old sheet, which contains the full information. If so, how can that be done?
I don't think you need to involve another worksheet. Simply copy the rows below the existing Range.CurrentRegion property and then remove the filter and delete the original data.
Sub RemoveHiddenRows()
With Sheets("Sheet10")
With .Cells(1, 1).CurrentRegion
With .Offset(1, 0).Resize(.Rows.Count - 1, .Columns.Count)
If CBool(Application.Subtotal(103, .Columns(1))) Then
.Cells.Copy Destination:=.Cells(.Rows.Count + 1, 1)
End If
.AutoFilter
.Cells(1, 1).Resize(.Rows.Count, 1).EntireRow.Delete
End With
End With
End With
End Sub
You may also receive some good, focused help on this subject by posting on Code Review (Excel).
You can improve performance significantly with a function like this:
Option Explicit
Public Sub deleteHiddenRows(ByRef ws As Worksheet)
Dim rngData As Range, rngVisible As Range, rngHidden As Range
With ws
Set rngData = .UsedRange
With rngData
Set rngVisible = .SpecialCells(xlCellTypeVisible)
Set rngHidden = .Columns(1)
End With
End With
If Not (rngVisible Is Nothing) Then
ws.AutoFilterMode = False
' invert hidden / visible
rngHidden.Rows.Hidden = False
rngVisible.Rows.Hidden = True
' delete hidden and show visible
rngData.SpecialCells(xlCellTypeVisible).Delete
rngVisible.Rows.Hidden = False
End If
End Sub
I tested it on a file with 2 filters applied to it
The function was adapted from the code in this suggestion

Excel VBA: Writing an array to cells is very slow

I am working with VBA in Excel to retrieve some information from the Reuters 3000 Database. The data I retrieve comes as a bidimensional array consisting of one column holding dates and other column holding numeric values.
After I retrieve the information, a process that takes no more than 2 seconds, I want to write this data to a worksheet. In the worksheet I have a column with dates and several other columns with numeric values, each column containing values of a same category. I iterate over the rows of the array to get the date and numeric value and I keep those in a variable, then I search for the date on the date column of the worksheet and after I've found the date I write the value. Here is my code:
Private Sub writeRetrievedData(retrievedData As Variant, dateColumnRange As String, columnOffset As Integer)
Dim element As Long: Dim startElement As Long: Dim endElement As Long
Dim instrumentDate As Variant: Dim instrumentValue As Variant
Dim c As Variant: Dim dateCellAddress As Variant
Application.ScreenUpdating = False
Sheets("Data").Activate
startElement = LBound(retrievedData, 1): endElement = UBound(retrievedData, 1)
Application.DisplayStatusBar = True
Application.StatusBar = "Busy writing data to worksheet"
For element = startElement To endElement
instrumentDate = retrievedData(element, 1): instrumentValue = retrievedData(element, 2)
Range(dateColumnRange).Select
Set c = Selection.Find(What:=instrumentDate, After:=ActiveCell, LookIn:=xlValues, _
LookAt:=xlWhole, SearchOrder:=xlByColumns, SearchDirection:=xlNext, _
MatchCase:=False, SearchFormat:=False)
If Not c Is Nothing Then
c.offset(0, columnOffset).Value = instrumentValue
End If
Next element
Application.DisplayStatusBar = False
End Sub
My problem is that this process is very slow, even if I have only 5 rows in the array it takes about 15 seconds to complete the task. As I want to repeat this process several times (once per each set of data I retrieve from the database), I would like to decrease the execution time as much as possible.
As you can see, I am disabling the update of the screen, which is one of the most recurrent actions to improve performance. Does anybody have a suggestion on how I can further decrease the execution time?
PS. I know the data retrieval process does not take much because I already tested that part (displaying values on a MsgBox as soon as the data has been retrieved)
Thanks in advanced.
This is what I did to improve the performance:
Avoid selecting the cell when the value is going to be written. This was a suggestion of Tim Williams.
I set the property Application.Calculation to xlCalculationManual
Instead of using the Find() function to search for the date, I loaded all the dates from the worksheet into an array and iterate over this array to get the row number. This turns out to be faster than the Find() function.
Private Function loadDateArray() As Variant
Dim Date_Arr() As Variant
Sheets("Data").Activate
Date_Arr = Range(Cells(3, 106), Cells(3, 106).End(xlDown))
loadDateArray = Date_Arr
End Function
Private Function getDateRow(dateArray As Variant, dateToLook As Variant)
Dim i As Double: Dim dateRow As Double
For i = LBound(dateArray, 1) To UBound(dateArray, 1)
If dateArray(i, 1) = dateToLook Then
dateRow = i
Exit For
End If
Next i
getDateRow = dateRow
End Function
Thank you all for your help!
By not selecting the sheet, you can add a bit more speed. Instead of
Sheets("Data").Activate
Date_Arr = Range(Cells(3, 106), Cells(3, 106).End(xlDown))
loadDateArray = Date_Arr
Try
With Sheets("Data")
Date_Arr = .Range(Cells(3, 106), Cells(3, 106).End(xlDown))
End With

Resources