VBA : Querying Access with Excel on server - performance

Question

I'm working with Excel project wich helps to calculate the price of any peace of furniture. The first task is to pick all the materials from the database.
This is the code:
Sub Material_search()
Dim cnt As New ADODB.connection
Dim rst As New ADODB.Recordset
Dim rcArray As Variant
Dim sSQL As String
Dim db_path As String, db_conn As String
Dim item As String
item = Replace(TextBox1.Text, " ", "%") ' Search word
sSQL = "Select Data, NomNr, Preke, Matas, Kaina, Tiek from VazPirkPrekes " & _
"Where VazPirkPrekes.PirkVazID IN (SELECT VazPirkimo.PirkVazID FROM VazPirkimo Where VazPirkimo.Sandelys like '%ALIAVOS')" & _
" and Year(VazPirkPrekes.Data)>=2011 and Preke Like '%" + item + "%' and Kaina > 0" & _
" Order by Preke, Data Desc"
db_path = Sheets("TMP").Range("B6").value
db_conn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & db_path & ";"
cnt.Open db_conn
rst.Open sSQL, cnt, adOpenForwardOnly, adLockReadOnly
ListBox1.Clear
If Not rst.EOF Then
rcArray = (rst.GetRows)
rcArray = WorksheetFunction.Transpose(rcArray)
Dim a As Variant
With ListBox1
.ColumnCount = 6
.list = rcArray
.ListIndex = -1
End With
End If
rst.Close: Set rst = Nothing
cnt.Close: Set cnt = Nothing
Label4.Caption = UBound(ListBox1.list) + 1
End Sub
recently I came up with some trouble while querying Access mdb file. The problem is when database file is on local disk, the search works very fast, but when i put database file on server, the search takes 10 times longer, which is not acceptable.
Is there any optimisation for this code ? or is it a server problem
Thanks in advance

Answer 1

That query requires Access' database engine retrieve all 190K rows from both tables. It's not surprising it is slow, and the slowness is compounded when the db engine must retrieve 2 * 190K rows across the network.
If TextBox1.Text contains "foo", this is the statement you're asking the db engine to run:
Select Data, NomNr, Preke, Matas, Kaina, Tiek
from VazPirkPrekes
Where
VazPirkPrekes.PirkVazID IN (
SELECT VazPirkimo.PirkVazID
FROM VazPirkimo
Where VazPirkimo.Sandelys like '%ALIAVOS')
and Year(VazPirkPrekes.Data)>=2011
and Preke Like '%foo%'
and Kaina > 0
Order by Preke, Data Desc
The engine must retrieve all 190K rows from the VazPirkimo table before it can determine which of them include Sandelys values which end with "ALIAVOS". If your selection criterion was for values which start with "ALIAVOS", the engine could use an index on Sandelys to limit the number of rows it must retrieve from VazPirkimo. However, since that approach is probably not an option for you, consider adding a numeric field, Sandelys_group, to VazPirkimo and create an index on Sandelys_group. Give all rows where Sandelys ends with "ALIAVOS" the same Sandelys_group number (1). Then your "IN ()" condition could be this:
SELECT VazPirkimo.PirkVazID
FROM VazPirkimo
Where VazPirkimo.Sandelys_group = 1
The index on Sandelys_group will allow the db engine to retrieve only the matching rows, which will hopefully be a small subset of the 190K rows in the table.
There are other changes you can make to speed up your query. Look at this criterion from your WHERE clause:
Year(VazPirkPrekes.Data)>=2011
That forces the db engine to retrieve all 190K rows from VazPirkPrekes before it can determine which of them are from 2011. With an index on Data, this should be much faster:
VazPirkPrekes.Data >= #2011-01-01# AND VazPirkPrekes.Data < #2012-01-01#
This WHERE criterion will be faster with an index on Kaina:
Kaina > 0
Your ORDER BY begs for indexes on Preke and Data.
Order by Preke, Data Desc
Any or all of those changes could help speed up the query, though I don't know by how much. The killer is this WHERE criterion:
Preke Like '%foo%'
The issue here is similar to the problem with the "Sandelys like" comparison. Since this asks for the rows where Preke contains "foo", rather than starts with "foo", the db engine can't take advantage of an index on Preke to retrieve only the matching rows. It must retrieve all 190K VazPirkPrekes rows to figure out which match. Unless you can use a different criterion for this one, you will be limited as to how much you can speed up the query.

Answer 2

Thanks for the optimization tips, but as I said the problem occurs only when I put data base file on server. And there is not much help from optimization. But I thought about other idea.
The search of empty blank "" returns about 40k records (these records covers everything I need) . So I'm going to put all these records on a distinct sheet on workbook_activate event and later do the query only in that sheet.
Sub Database_upload()
Application.DisplayAlerts = False
On Error Resume Next
Sheets("DATA_BASE").Delete
On Error GoTo 0
Application.DisplayAlerts = False
Sheets.Add
ActiveSheet.name = "DATA_BASE"
Sheets("DATA_BASE").Visible = False: Sheets("DARBALAUKIS").activate
Dim cnt As New ADODB.connection
Dim rcArray As Variant
Dim sSQL As String
Dim db_path As String, db_conn As String
Dim item As String
Dim qQt As QueryTable
item = "" 'search for empty blanks
sSQL = "Select Data, NomNr, Preke, Matas, Kaina, Tiek from VazPirkPrekes " & _
"Where VazPirkPrekes.PirkVazID IN (SELECT VazPirkimo.PirkVazID FROM VazPirkimo Where VazPirkimo.Sandelys like '%ALIAVOS')" & _
" and Year(VazPirkPrekes.Data)>=2011 and Preke Like '%" + item + "%' and Kaina > 0" & _
" Order by Preke, Data Desc"
db_path = Sheets("TMP").Range("B6").value
db_conn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & db_path & ";"
db_conn = "ODBC;DSN=MS Access 97 Database;"
db_conn = db_conn & "DBQ=" & db_path
Set qQt = Sheets("Sheet1").QueryTables.Add(connection:=db_conn, Destination:=Sheets("Sheet1").Range("A1"), Sql:=sSQL)
qQt.Refresh BackgroundQuery:=False
End Sub
Results:
Program takes longer on startup, but the search time is acceptable - for me the problem is solved :)