Faster way to calculate distance between two locations (ZIP codes) - performance

I am writing a VBA script that finds ZIP Codes inside of a specified radius. I have an Access Database with multiple records in a table. Each record has a Name, Address, and Zip Code field on the table. The VBA code on access prompts the user for a zip code and search radius then calculates the distance between the user input zip code and the zip code for each record. Once each distance is calculated the record is displayed to the form as long as it falls within the radius input field.
The code that I have written works but the execution time takes too long (around 30 secs for 2000ish records). How can I decrease the time it takes for this VBA code to run? Here is the code I have written:
Private Sub Command65_Click()
Dim StartTime As Double
Dim SecondsElapsed As Double
Dim i, j As Integer
Dim db As Database
Dim rs As Recordset
Dim ZIP, r As Double
Dim arr(33144, 3) As Double
Dim lat1, long1, lat2, long2, theta As Double
Dim Distance As Integer
Dim deg2rad, rad2deg As Double
Const PI As Double = 3.14159265359
'Dim Variables
StartTime = Timer
deg2rad = PI / 180
rad2deg = 180 / PI
r = Text1.Value
ZIP = Text2.Value
'Get radius and prompted zip code from form
Set db = CurrentDb
Set rs = db.OpenRecordset("US Zip Codes")
'Open the Table named "US Zip Codes"
For i = 0 To 33143
arr(i, 0) = rs.Fields("ZIP")
arr(i, 1) = rs.Fields("LAT")
arr(i, 2) = rs.Fields("LNG")
rs.MoveNext
Next i
'Loop through each Zip Code record and store the Zip Code, Lattitude Point, and Longitude Point to an array
For i = 0 To 33143
If ZIP = arr(i, 0) Then
lat1 = arr(i, 1) * deg2rad
long1 = arr(i, 2) * deg2rad
End If
Next i
'Loop through the zip code array to get Zip Code's corresponding LAT and LONG
Set rs = db.OpenRecordset("Clinics")
'Open the Table named "Clinics"
For j = 0 To 2094
If rs("Clinic ZIP") = ZIP Then
Distance = 0
'If Zip Code 1 and Zip Code 2 are equal to each other, Distance = 0
ElseIf rs("Clinic ZIP") <> "" Then
zip2 = rs("Clinic ZIP")
For i = 0 To 33143
If zip2 = arr(i, 0) Then
lat2 = arr(i, 1) * deg2rad
long2 = arr(i, 2) * deg2rad
End If
Next i
'Loop through the zip code array to get the second Zip Code's corresponding LAT and LONG
theta = long1 - long2
Distance = ArcCOS(Sin(lat1) * Sin(lat2) + Cos(lat1) * Cos(lat2) * Cos(theta)) * rad2deg * 60 * 1.1515
'Calculate Distance between the two zip codes
Else
Distance = 999
'Set Arbitrary Value if the zip code field is empty
End If
rs.Edit
rs.Fields("Distance") = Distance
rs.Update
rs.MoveNext
Next j
Me.Filter = "Distance<=" & r
Me.FilterOn = True
'Filter the table with calculated distance by prompted radius
Forms("Zip Search").Requery
rs.Close
Set rs = Nothing
db.Close
SecondsElapsed = Round(Timer - StartTime, 2)
MsgBox "This code ran successfully in " & SecondsElapsed & " seconds", vbInformation
End Sub

I just did a test with a table of 1,976 restaurant locations:
ID lon lat poi_name
-- --------- -------- ---------------------------------------------
1 -114.063 51.0466 Palomino Smokehouse: Calgary, AB
2 -114.055 51.0494 Bookers BBQ Grill and Crab Shack: Calgary, AB
3 -86.97871 34.58037 Big Bob Gibson's Original: Decatur, AL
4 -87.01763 34.56587 Big Bob Gibson's #2: Decatur, AL
5 -86.364 32.26995 DJ's Old Post Office: Hope Hull, AL
...
Using the GreatCircleDistance function available from ...
http://www.cpearson.com/excel/LatLong.aspx
... I ran the following query to calculate the distance from a given point
PARAMETERS prmLon IEEEDouble, prmLat IEEEDouble;
SELECT BBQ2.ID, BBQ2.lon, BBQ2.lat, BBQ2.poi_name,
GreatCircleDistance([prmLat],[prmLon],[lat],[lon],True,False) AS km
FROM BBQ2;
and the results came back in less than a second.
Then to return results within a certain number of kilometers from a given point I used
PARAMETERS prmLon IEEEDouble, prmLat IEEEDouble, prmWithinKm IEEEDouble;
SELECT * FROM
(
SELECT BBQ2.ID, BBQ2.lon, BBQ2.lat, BBQ2.poi_name,
GreatCircleDistance([prmLat],[prmLon],[lat],[lon],True,False) AS km
FROM BBQ2
)
WHERE km <= [prmWithinKm];
and again, the results came back in less than a second.

Applying comments to revise code, consider following which assumes ArcCos() is a public custom UDF. Also, two query objects are referenced as data sources. One is a query of the ZipCodes table which calculates the lat and long values. The other is a query that joins Clinics table to the above query.
Private Sub Command65_Click()
Dim StartTime As Double
Dim lat1 As Double, long1 As Double
Dim Distance As Integer
'Dim Variables
StartTime = Timer
lat1 = DLookup("lat", "qryLatLongZip", "Zip='" & Me.Text2 & "'")
long1 = DLookup("long", "qryLatLongZip", "Zip='" & Me.Text2 & "'")
CurrentDb.Execute "UPDATE qryClinicsLatLongZip SET Distance = ArcCos(Sin(" & lat1 & ") * Sin(lat) + Cos(" & lat1 & ") * Cos(lat) * Cos(" & long1 & "-long)) * (180 / 3.14159265359) * 60 * 1.1515"
Me.Filter = "Distance<=" & Me.Text1
Me.FilterOn = True
'Filter the form with calculated distance by prompted radius
MsgBox "This code ran successfully in " & Round(Timer - StartTime, 2) & " seconds", vbInformation
End Sub
However, in a multi-user database, users will conflict with each other writing Distance to shared table. If there are multiple users then db would have to be split and a temp table (table is permanent, records are temporary) in frontend for writing records to and that would be report RecordSource. A solution avoiding record edit/saving would be most desirable and now I see an answer has been provided doing exactly that.

Related

how to create automatic number based on string, date in vb6

i try to make auto number based on format string, date and counter like USR20180815001, USR20180815002, when the day/date change it reset the counter to USR20180806001, USR20180806001, and so on. here is my code so far....
Sub AutomaticNumber()
Call Connection
Set Rs_User = New ADODB.Recordset
Rs_User.Open "SELECT * FROM TBL_USER WHERE id_user IN (SELECT
MAX(id_user)FROM TBL_USER)ORDER BY id_user DESC", Conn
Rs_User.Requery
Dim x As String * 15
Dim count As Long
With Rs_User
If .EOF Then
x = "USR" + Format(Date, "yyyymmdd") + "001"
NewNumber = x
Else
If Left(!id_user, 8) <> Format(Date, "yyyymmdd") Then
x = "USR" + Format(Date, "yyyymmdd") + "001"
Else
Count = Right(!id_user, 3) + 1
x = "USR" + Format(Date, "yyyymmdd") + Right("00" &
Count, 2)
End If
End If
NewNumber = x
End With
End Sub
this code can result USR20180805001, but when i try to add another record to databases, it can not add since the code failed to counter/increase the last 3 digit on the right. hence show error duplicate entry. thank you for the attention.

Excel VBA Performance - 1 million rows - Delete rows containing a value, in less than 1 min

I am trying to find a way to filter large data and remove rows in a worksheet, in less than one minute
The goal:
Find all records containing specific text in column 1, and delete the entire row
Keep all cell formatting (colors, font, borders, column widths) and formulas as they are
.
Test Data:
:
.
How the code works:
It starts by turning all Excel features Off
If the workbook is not empty and the text value to be removed exists in column 1
Copies the used range of column 1 to an array
Iterates over every value in array backwards
When it finds a match:
Appends the cell address to a tmp string in the format "A11,A275,A3900,..."
If the tmp variable length is close to 255 characters
Deletes rows using .Range("A11,A275,A3900,...").EntireRow.Delete Shift:=xlUp
Resets tmp to empty and moves on to the next set of rows
At the end, it turns all Excel features back On
.
The main issue is the Delete operation, and total duration time should be under one minute. Any code-based solution is acceptable as long as it performs under 1 minute.
This narrows the scope to very few acceptable answers. The answers already provided are also very short and easy to implement. One performs the operation in about 30 seconds, so there is at least one answer that provides an acceptable solution, and other may find it useful as well
.
My main initial function:
Sub DeleteRowsWithValuesStrings()
Const MAX_SZ As Byte = 240
Dim i As Long, j As Long, t As Double, ws As Worksheet
Dim memArr As Variant, max As Long, tmp As String
Set ws = Worksheets(1)
max = GetMaxCell(ws.UsedRange).Row
FastWB True: t = Timer
With ws
If max > 1 Then
If IndexOfValInRowOrCol("Test String", , ws.UsedRange) > 0 Then
memArr = .Range(.Cells(1, 1), .Cells(max, 1)).Value2
For i = max To 1 Step -1
If memArr(i, 1) = "Test String" Then
tmp = tmp & "A" & i & ","
If Len(tmp) > MAX_SZ Then
.Range(Left(tmp, Len(tmp) - 1)).EntireRow.Delete Shift:=xlUp
tmp = vbNullString
End If
End If
Next
If Len(tmp) > 0 Then
.Range(Left(tmp, Len(tmp) - 1)).EntireRow.Delete Shift:=xlUp
End If
.Calculate
End If
End If
End With
FastWB False: InputBox "Duration: ", "Duration", Timer - t
End Sub
Helper functions (turn Excel features off and on):
Public Sub FastWB(Optional ByVal opt As Boolean = True)
With Application
.Calculation = IIf(opt, xlCalculationManual, xlCalculationAutomatic)
.DisplayAlerts = Not opt
.DisplayStatusBar = Not opt
.EnableAnimations = Not opt
.EnableEvents = Not opt
.ScreenUpdating = Not opt
End With
FastWS , opt
End Sub
Public Sub FastWS(Optional ByVal ws As Worksheet = Nothing, _
Optional ByVal opt As Boolean = True)
If ws Is Nothing Then
For Each ws In Application.ActiveWorkbook.Sheets
EnableWS ws, opt
Next
Else
EnableWS ws, opt
End If
End Sub
Private Sub EnableWS(ByVal ws As Worksheet, ByVal opt As Boolean)
With ws
.DisplayPageBreaks = False
.EnableCalculation = Not opt
.EnableFormatConditionsCalculation = Not opt
.EnablePivotTable = Not opt
End With
End Sub
Finds last cell with data (thanks #ZygD - now I tested it in several scenarios):
Public Function GetMaxCell(Optional ByRef rng As Range = Nothing) As Range
'Returns the last cell containing a value, or A1 if Worksheet is empty
Const NONEMPTY As String = "*"
Dim lRow As Range, lCol As Range
If rng Is Nothing Then Set rng = Application.ActiveWorkbook.ActiveSheet.UsedRange
If WorksheetFunction.CountA(rng) = 0 Then
Set GetMaxCell = rng.Parent.Cells(1, 1)
Else
With rng
Set lRow = .Cells.Find(What:=NONEMPTY, LookIn:=xlFormulas, _
After:=.Cells(1, 1), _
SearchDirection:=xlPrevious, _
SearchOrder:=xlByRows)
If Not lRow Is Nothing Then
Set lCol = .Cells.Find(What:=NONEMPTY, LookIn:=xlFormulas, _
After:=.Cells(1, 1), _
SearchDirection:=xlPrevious, _
SearchOrder:=xlByColumns)
Set GetMaxCell = .Parent.Cells(lRow.Row, lCol.Column)
End If
End With
End If
End Function
Returns the index of a match in the array, or 0 if a match is not found:
Public Function IndexOfValInRowOrCol( _
ByVal searchVal As String, _
Optional ByRef ws As Worksheet = Nothing, _
Optional ByRef rng As Range = Nothing, _
Optional ByRef vertical As Boolean = True, _
Optional ByRef rowOrColNum As Long = 1 _
) As Long
'Returns position in Row or Column, or 0 if no matches found
Dim usedRng As Range, result As Variant, searchRow As Long, searchCol As Long
result = CVErr(9999) '- generate custom error
Set usedRng = GetUsedRng(ws, rng)
If Not usedRng Is Nothing Then
If rowOrColNum < 1 Then rowOrColNum = 1
With Application
If vertical Then
result = .Match(searchVal, rng.Columns(rowOrColNum), 0)
Else
result = .Match(searchVal, rng.Rows(rowOrColNum), 0)
End If
End With
End If
If IsError(result) Then IndexOfValInRowOrCol = 0 Else IndexOfValInRowOrCol = result
End Function
.
Update:
Tested 6 solutions (3 tests each): Excel Hero's solution is the fastest so far (removes formulas)
.
Here are the results, fastest to the slowest:
.
Test 1. Total of 100,000 records, 10,000 to be deleted:
1. ExcelHero() - 1.5 seconds
2. DeleteRowsWithValuesNewSheet() - 2.4 seconds
3. DeleteRowsWithValuesStrings() - 2.45 minutes
4. DeleteRowsWithValuesArray() - 2.45 minutes
5. QuickAndEasy() - 3.25 minutes
6. DeleteRowsWithValuesUnion() - Stopped after 5 minutes
.
Test 2. Total of 1 million records, 100,000 to be deleted:
1. ExcelHero() - 16 seconds (average)
2. DeleteRowsWithValuesNewSheet() - 33 seconds (average)
3. DeleteRowsWithValuesStrings() - 4 hrs 38 min (16701.375 sec)
4. DeleteRowsWithValuesArray() - 4 hrs 37 min (16626.3051757813 sec)
5. QuickAndEasy() - 5 hrs 40 min (20434.2104492188 sec)
6. DeleteRowsWithValuesUnion() - N/A
.
Notes:
ExcelHero method: easy to implement, reliable, extremely fast, but removes formulas
NewSheet method: easy to implement, reliable, and meets the target
Strings method: more effort to implement, reliable, but doesn't meet requirement
Array method: similar to Strings, but ReDims an array (faster version of Union)
QuickAndEasy: easy to implement (short, reliable and elegant), but doesn't meet requirement
Range Union: implementation complexity similar to 2 and 3, but too slow
I also made the test data more realistic by introducing unusual values:
empty cells, ranges, rows, and columns
special characters, like =[`~!##$%^&*()_-+{}[]\|;:'",.<>/?, separate and multiple combinations
blank spaces, tabs, empty formulas, border, font, and other cell formatting
large and small numbers with decimals (=12.9999999999999 + 0.00000000000000001)
hyperlinks, conditional formatting rules
empty formatting inside and outside data ranges
anything else that might cause data issues
I'm providing the first answer as a reference
Others may find it useful, if there are no other options available
Fastest way to achieve the result is not to use the Delete operation
Out of 1 million records it removes 100,000 rows in an average of 33 seconds
.
Sub DeleteRowsWithValuesNewSheet() '100K records 10K to delete
'Test 1: 2.40234375 sec
'Test 2: 2.41796875 sec
'Test 3: 2.40234375 sec
'1M records 100K to delete
'Test 1: 32.9140625 sec
'Test 2: 33.1484375 sec
'Test 3: 32.90625 sec
Dim oldWs As Worksheet, newWs As Worksheet, rowHeights() As Long
Dim wsName As String, t As Double, oldUsedRng As Range
FastWB True: t = Timer
Set oldWs = Worksheets(1)
wsName = oldWs.Name
Set oldUsedRng = oldWs.Range("A1", GetMaxCell(oldWs.UsedRange))
If oldUsedRng.Rows.Count > 1 Then 'If sheet is not empty
Set newWs = Sheets.Add(After:=oldWs) 'Add new sheet
With oldUsedRng
.AutoFilter Field:=1, Criteria1:="<>Test String"
.Copy 'Copy visible data
End With
With newWs.Cells
.PasteSpecial xlPasteColumnWidths
.PasteSpecial xlPasteAll 'Paste data on new sheet
.Cells(1, 1).Select 'Deselect paste area
.Cells(1, 1).Copy 'Clear Clipboard
End With
oldWs.Delete 'Delete old sheet
newWs.Name = wsName
End If
FastWB False: InputBox "Duration: ", "Duration", Timer - t
End Sub
.
At high level:
It creates a new worksheet, and keeps a reference to the initial sheet
AutoFilters column 1 on the searched text: .AutoFilter Field:=1, Criteria1:="<>Test String"
Copies all (visible) data from initial sheet
Pastes column widths, formats, and data to the new sheet
Deletes initial sheet
Renames the new sheet to the old sheet name
It uses the same helper functions posted in the question
The 99% of the duration is used by the AutoFilter
.
There are a couple limitations I found so far, the first can be addressed:
If there are any hidden rows on the initial sheet, it unhides them
A separate function is needed to hide them back
Depending on implementation, it might significantly increase duration
VBA related:
It changes the Code Name of the sheet; other VBA referring to Sheet1 will be broken (if any)
It deletes all VBA code associated with the initial sheet (if any)
.
A few notes about using large files like this:
The binary format (.xlsb) reduce file size dramatically (from 137 Mb to 43 Mb)
Unmanaged Conditional Formatting rules can cause exponential performance issues
The same for Comments, and Data validation
Reading file or data from network is much slower than working with a locall file
A significant gain in speed can be achieved if the source data do not contain formulas, or if the scenario would allow (or want) the formulas to be converted into hard values during the conditional row deletions.
With the above as a caveat, my solution uses the AdvancedFilter of the range object. It's about twice as fast as DeleteRowsWithValuesNewSheet().
Public Sub ExcelHero()
Dim t#, crit As Range, data As Range, ws As Worksheet
Dim r&, fc As Range, lc As Range, fr1 As Range, fr2 As Range
FastWB True
t = Timer
Set fc = ActiveSheet.UsedRange.Item(1)
Set lc = GetMaxCell
Set data = ActiveSheet.Range(fc, lc)
Set ws = Sheets.Add
With data
Set fr1 = data.Worksheet.Range(fc, fc.Offset(, lc.Column))
Set fr2 = ws.Range(ws.Cells(fc.Row, fc.Column), ws.Cells(fc.Row, lc.Column))
With fr2
fr1.Copy
.PasteSpecial xlPasteColumnWidths: .PasteSpecial xlPasteAll
.Item(1).Select
End With
Set crit = .Resize(2, 1).Offset(, lc.Column + 1)
crit = [{"Column 1";"<>Test String"}]
.AdvancedFilter xlFilterCopy, crit, fr2
.Worksheet.Delete
End With
FastWB False
r = ws.UsedRange.Rows.Count
Debug.Print "Rows: " & r & ", Duration: " & Timer - t & " seconds"
End Sub
On my elderly Dell Inspiron 1564 (Win 7 Office 2007) this:
Sub QuickAndEasy()
Dim rng As Range
Set rng = Range("AA2:AA1000001")
Range("AB1") = Now
Application.ScreenUpdating = False
With rng
.Formula = "=If(A2=""Test String"",0/0,A2)"
.Cells.SpecialCells(xlCellTypeFormulas, xlErrors).EntireRow.Delete
.Clear
End With
Application.ScreenUpdating = True
Range("AC1") = Now
End Sub
took about 10 seconds to run. I am assuming that column AA is available.
EDIT#1:
Please note that this code does not set Calculation to Manual. Performance will improve if the Calculation mode is set to Manual after the "helper" column is allowed to calculate.
I know I'm incredibly late with my answer here however future visitors may find it very useful.
Please Note: My approach requires an index column for the rows to end up in the original order, however if you do not mind the rows being in a different order then an index column isn't needed and the additional line of code can be removed.
My approach: My approach was to simply select all the rows in the selected range (column), sort them in ascending order using Range.Sort and then collecting the first and last index of "Test String" within the selected range (column). I then create a range from the first and last indices and use Range.EntrieRow.Delete to remove all the rows which contain "Test String".
Pros:
- It is blazing fast.
- It doesn't remove formatting, formulas, charts, pictures or anything like the method which copies to a new sheet.
Cons:
- A decent size of code to implement however it is all straight-forward.
Test Range Generation Sub:
Sub DevelopTest()
Dim index As Long
FastWB True
ActiveSheet.UsedRange.Clear
For index = 1 To 1000000 '1 million test
ActiveSheet.Cells(index, 1).Value = index
If (index Mod 10) = 0 Then
ActiveSheet.Cells(index, 2).Value = "Test String"
Else
ActiveSheet.Cells(index, 2).Value = "Blah Blah Blah"
End If
Next index
Application.StatusBar = ""
FastWB False
End Sub
Filter And Delete Rows Sub:
Sub DeleteRowFast()
Dim curWorksheet As Worksheet 'Current worksheet vairable
Dim rangeSelection As Range 'Selected range
Dim startBadVals As Long 'Start of the unwanted values
Dim endBadVals As Long 'End of the unwanted values
Dim strtTime As Double 'Timer variable
Dim lastRow As Long 'Last Row variable
Dim lastColumn As Long 'Last column variable
Dim indexCell As Range 'Index range start
Dim sortRange As Range 'The range which the sort is applied to
Dim currRow As Range 'Current Row index for the for loop
Dim cell As Range 'Current cell for use in the for loop
On Error GoTo Err
Set rangeSelection = Application.InputBox("Select the (N=) range to be checked", "Get Range", Type:=8) 'Get the desired range from the user
Err.Clear
M1 = MsgBox("This is recommended for large files (50,000 or more entries)", vbYesNo, "Enable Fast Workbook?") 'Prompt the user with an option to enable Fast Workbook, roughly 150% performace gains... Recommended for incredibly large files
Select Case M1
Case vbYes
FastWB True 'Enable fast workbook
Case vbNo
FastWB False 'Disable fast workbook
End Select
strtTime = Timer 'Begin the timer
Set curWorksheet = ActiveSheet
lastRow = CLng(rangeSelection.SpecialCells(xlCellTypeLastCell).Row)
lastColumn = curWorksheet.Cells(1, 16384).End(xlToLeft).Column
Set indexCell = curWorksheet.Cells(1, 1)
On Error Resume Next
If rangeSelection.Rows.Count > 1 Then 'Check if there is anything to do
lastVisRow = rangeSelection.Rows.Count
Set sortRange = curWorksheet.Range(indexCell, curWorksheet.Cells(curWorksheet.Rows(lastRow).Row, 16384).End(xlToLeft)) 'Set the sort range
sortRange.Sort Key1:=rangeSelection.Cells(1, 1), Order1:=xlAscending, Header:=xlNo 'Sort by values, lowest to highest
startBadVals = rangeSelection.Find(What:="Test String", LookAt:=xlWhole, MatchCase:=False).Row
endBadVals = rangeSelection.Find(What:="Test String", LookAt:=xlWhole, SearchDirection:=xlPrevious, MatchCase:=False).Row
curWorksheet.Range(curWorksheet.Rows(startBadVals), curWorksheet.Rows(endBadVals)).EntireRow.Delete 'Delete uneeded rows, deleteing in continuous range blocks is quick than seperated or individual deletions.
sortRange.Sort Key1:=indexCell, Order1:=xlAscending, Header:=xlNo 'Sort by index instead of values, lowest to highest
End If
Application.StatusBar = "" 'Reset the status bar
FastWB False 'Disable fast workbook
MsgBox CStr(Round(Timer - strtTime, 2)) & "s" 'Display duration of task
Err:
Exit Sub
End Sub
THIS CODE USES FastWB, FastWS AND EnableWS BY Paul Bica!
Times at 100K entries (10k to be removed, FastWB True):
1. 0.2 seconds.
2. 0.2 seconds.
3. 0.21 seconds.
Avg. 0.2 seconds.
Times at 1 million entries (100k to be removed, FastWB True):
1. 2.3 seconds.
2. 2.32 seconds.
3. 2.3 seconds.
Avg. 2.31 seconds.
Running on: Windows 10, iMac i3 11,2 (From 2010)
EDIT
This code was originally designed with the purpose of filtering out numeric values outside of a numeric range and has been adapted to filter out "Test String" so some of the code may be redundant.
Your use of arrays in calculating the used range and row count may effect the performance. Here's another approach which in testing proves efficient across 1m+ rows of data - between 25-30 seconds. It doesn't use filters so will delete rows even if hidden. Deleting a whole row won't effect formatting or column widths of the other remaining rows.
First, check if the ActiveSheet has "Test String". Since you're only interested in Column 1 I used this:
TCount = Application.WorksheetFunction.CountIf(sht.Columns(1), "Test String")
If TCount > 0 Then
Instead of using your GetMaxCell() function I simply used Cells.SpecialCells(xlCellTypeLastCell).Row to get the last row:
EndRow = sht.Cells.SpecialCells(xlCellTypeLastCell).Row
Then loop through the rows of data:
While r <= EndRow
To test if the cell in Column 1 is equal to "Test String":
If sht.Cells(r, 1).Text) = "Test String" Then
To delete the row:
Rows(r).Delete Shift:=xlUp
Putting it all together full code below. I've set ActiveSheet to a variable Sht and added turned of ScreenUpdating to improve efficiency. Since it's a lot of data I make sure to clear variables at the end.
Sub RowDeleter()
Dim sht As Worksheet
Dim r As Long
Dim EndRow As Long
Dim TCount As Long
Dim s As Date
Dim e As Date
Application.ScreenUpdating = True
r = 2 'Initialise row number
s = Now 'Start Time
Set sht = ActiveSheet
EndRow = sht.Cells.SpecialCells(xlCellTypeLastCell).Row
'Check if "Test String" is found in Column 1
TCount = Application.WorksheetFunction.CountIf(sht.Columns(1), "Test String")
If TCount > 0 Then
'loop through to the End row
While r <= EndRow
If InStr(sht.Cells(r, 1).Text, "Test String") > 0 Then
sht.Rows(r).Delete Shift:=xlUp
r = r - 1
End If
r = r + 1
Wend
End If
e = Now 'End Time
D = (Hour(e) * 360 + Minute(e) * 60 + Second(e)) - (Hour(s) * 360 + Minute(s) * 60 + Second(s))
Application.ScreenUpdating = True
DurationTime = TimeSerial(0, 0, D)
MsgBox Format(DurationTime, "hh:mm:ss")
End Sub

VBScript Split Large excel file into smaller File after every 50,000 rows

I have been asked to split a very large excel file 1,000,000+ rows into smaller excel
files after a certain number of rows that the user decides via an inputBox, but before this is to happen I have to ask the user if they would like to replace specfic columns with "#####" using another inputBox once the info for the columns has been stored to a variable userCensor, then I would like to take the number that was entered for the row split, store it as userSplit and split the file at the interval specified in userSplit.
This is what I have so far and I am currently experienceing a major brain fart and don't know where to go from here:
Set app = CreateObject("Excel.Application")
Set fso = CreateObject("Scripting.FileSystemObject")
For Each f In fso.GetFolder("Y:\BLAHBLAHBLAH").Files
If LCase(fso.GetExtensionName(f)) = "xls" Then
Set wb = app.Workbooks.Open(f.Path)
set sh = wb.Sheets("Sheet 1") row = 1
lastRow = sh.UsedRange.Rows.Count
lastColumn = sh.UsedRange.Columns.Count
strRow = lastRow
userSplit = InputBox("Enter when you want to split between 1 - " + strRow)
strColumn = lastColumn
userCensor = InputBox("Enter Columns to censor (Format example: 'A:A' deletes column A) Between 1 - " + strColumn)
If userCensor.IsNumeric Then Columns(userCensor).Select
Selection.Replace("######")
For r = row to LastRow If lastColumn > 1 Then
Else
It isn't much to go off but any help would be much appreciated!
Thanks again!
You could try something like this for dividing the content into smaller parts:
firstRow = ws.UsedRange.Rows(1).Row
lastRow = ws.UsedRange.Rows(ws.UsedRange.Rows.Count).Row
userSplit = CLng(InputBox("Enter when you want to split between 1 - " _
& lastRow-firstRow+1))
n = 0
For srcRow = firstrow To lastrow
dstRow = (srcRow - firstRow) Mod userSplit + 1
If dstRow = 1 Then
n = (srcRow - firstRow) \ userSplit
If n > 0 Then
wb2.SaveAs "C:\path\to\out" & n & ".xls"
wb2.Close
End If
Set wb2 = xl.Workbooks.Add
End If
ws1.Cells(srcRow, 1).EntireRow.Copy
wb2.Sheets(1).Cells(dstRow, 1).PasteSpecial xlAll
Next
wb2.SaveAs "C:\path\to\out" & (lastRow - firstRow) \ userSplit & ".xls"
wb2.Close
As for deleting columns, wouldn't it be easier to actually delete the columns instead of replacing their content with something else?

display a series of images in Excel 2010 using VBA

As part of a larger project I need to display a series of images within the one macro. When I run this it displays the final image after the last msgbox, with the others layered underneath:
Sub Macro4()
Dim x As Integer
Dim Pic As Object
Dim picname As String
For x = 1 To 7
picname = ThisWorkbook.Path & "/" & "pic" & x & ".png"
ActiveSheet.Pictures.Insert(picname).Select
MsgBox (x)
Next x
End Sub
The Msgbox command is there to slow the process down so that I can see, or in this case not see, the pictures change.
The images are called pic1.png, pic2.png etc
How do I get the separate images to show during the macro?
RE-EDIT:
So here is the picture function and the larger function which plays a randomised piece of Musique Concret.
Private Declare Function PlaySound Lib "winmm.dll" _
Alias "PlaySoundA" (ByVal lpszName As String, _
ByVal hModule As Long, ByVal dwFlags As Long) As Long
showpic gives an image relative to the pitch of the sound being played.
Function showpic(value)
Dim v As Integer
v = value
picname = ThisWorkbook.Path & "/" & "pic" & v & ".png"
ActiveSheet.Shapes.AddPicture (picname), True, True, a1, a1, 170, 170
DoEvents
End Function
play runs a series of specific sound files generated by a randomised process choosing instrument and pitch and creating the requisite filename. The "piece" runs for 'notes' seconds and is triggered by a separate macro that changes the value in a given 'cell' to match the 'condition'.
Function play(Cell, condition, notes)
Dim WAVFile As String
Dim x As Integer
Dim y As Integer
Dim z As Integer
Dim c As String
Dim d As String
Dim currentcell As String
Dim pitchcell As String
Const SND_SYNC = &H0
Const SND_ASYNC = &H1
Const SND_FILENAME = &H20000
On Error GoTo ErrHandler
For x = 1 To notes
If Evaluate(Cell.value & condition) Then
y = x + 65
z = x + 39
c = Chr(y)
d = Chr(z)
If y > 90 Then c = "A" & d
currentcell = c & 6
pitchcell = c & 3
showpic (Sheets("Sheet1").Range(pitchcell).value)
WAVFile = ThisWorkbook.Path & "/" & Sheets("Sheet1").Range(currentcell).Text & ".wav"
Call PlaySound(WAVFile, 0&, SND_SYNC)
End If
Next x
ErrHandler:
play = False
Exit Function
End Function
I have three problems:
It doesn't show the first image file
It performs a 'calculate' or F9 at the beginning so re-randomises the piece and doesn't play the displayed sequence; I presume this is caused by the first DoEvents.
It now plays twice! However, the second time through it does show all the image files.
Edit:
Since my initial answer didn't work for you, how about achieving your results a completely different way. This method using Application.OnTime to re-run your sub and insert the next picture.
Sub NextPicture()
Static x As Integer
Dim pic As Object
Dim picname As String
'Reset x to 0 because I assumed you want to rotate through the pictures.
'If you want it to stop replace with If x = 7 then exit sub
If x = 7 Then x = 0
x = x + 1
picname = ThisWorkbook.Path & "/" & "pic" & x & ".png"
ActiveSheet.Pictures.Insert(picname).Select
'Using 00:00:05 = 5 seconds, change the amount to speed up or slow down the picture changes
Application.OnTime Now + TimeValue("00:00:05"), "NextPicture"
End Sub
Original Answer
I could not get your code to work, but I could if I used Shapes.AddPicture, which is actually more robust because you get/have to specify the location and size.
Syntax:
expression.AddPicture(Filename, LinkToFile, SaveWithDocument, Left, Top, Width, Height)
Example:
ActiveSheet.Shapes.AddPicture(picname, True, True, 100, 100, 70, 70)

Random selection from list

I have a list of items in an Excel worksheet, A1-B115. At the moment I can enter 10 variables which retrieves the correct data from the list.
Code now:
C1=1 - run through A1-A115 and check for the value to be between 1000-2000; if so, copy the B value somewhere.
C2=1 - run through A1-A115 and check for the value to be between 2001-3000; if so, copy the B value somewhere.
....
What I would like to do is that I can enter a value (example: 25 or 30) and that my macro randomly selects the right amount of values.
Code I would like to do: C1: 30 -> randomly selects 30 values from B1-B115
This will do the trick.
Sub PickRandomItemsFromList()
Const nItemsToPick As Long = 10
Const nItemsTotal As Long = 115
Dim rngList As Range
Dim varRandomItems() As Variant
Dim i As Long
Set rngList = Range("B1").Resize(nItemsTotal, 1)
ReDim varRandomItems(1 To nItemsToPick)
For i = 1 To nItemsToPick
varRandomItems(i) = rngList.Cells(Int(nItemsTotal * Rnd + 1), 1)
Next i
' varRandomItems now contains nItemsToPick random items from range rngList.
End Sub
As discussed in the comments, this will allow individual items to be picked more than once within the nItemsToPick picked, if for example number 63 happens to be randomly picked twice. If you don't want this to happen, then an additional loop will have to be added to check whether the item about to be picked is already in the list, for example like so:
Sub PickRandomItemsFromList()
Const nItemsToPick As Long = 10
Const nItemsTotal As Long = 115
Dim rngList As Range
Dim idx() As Long
Dim varRandomItems() As Variant
Dim i As Long
Dim j As Long
Dim booIndexIsUnique As Boolean
Set rngList = Range("B1").Resize(nItemsTotal, 1)
ReDim idx(1 To nItemsToPick)
ReDim varRandomItems(1 To nItemsToPick)
For i = 1 To nItemsToPick
Do
booIndexIsUnique = True ' Innoncent until proven guilty
idx(i) = Int(nItemsTotal * Rnd + 1)
For j = 1 To i - 1
If idx(i) = idx(j) Then
' It's already there.
booIndexIsUnique = False
Exit For
End If
Next j
If booIndexIsUnique = True Then
Exit Do
End If
Loop
varRandomItems(i) = rngList.Cells(idx(i), 1)
Next i
' varRandomItems now contains nItemsToPick unique random
' items from range rngList.
End Sub
Note that this will loop forever if nItemsToPick > nItemsTotal !
I would use a collection to make sure you don't get any duplicates.
Function cItemsToPick(NrOfItems As Long, NrToPick As Long) As Collection
Dim cItemsTotal As New Collection
Dim K As Long
Dim I As Long
Set cItemsToPick = New Collection
If NrToPick > NrOfItems Then Exit Function
For I = 1 To NrOfItems
cItemsTotal.Add I
Next I
For I = 1 To NrToPick
K = Int(cItemsTotal.Count * Rnd + 1)
cItemsToPick.Add cItemsTotal(K)
cItemsTotal.Remove (K)
Next I
Set cItemsTotal = Nothing
End Function
You can test this function with the following code:
Sub test()
Dim c As New Collection
Dim I As Long
Set c = cItemsToPick(240, 10)
For I = 1 To c.Count
Debug.Print c(I)
Next I
End Sub

Resources