Faster iteration - performance

Question

I have this code, which is part of a function that returns a list of SQL rows based on a time range.
The query itself (1st line of code) is quite fast. But the foreach loop that extract the relevant data takes a while to complete.
I have around 350.000 lines to iterate, and despite it's has to take a while, I was wondering if there is any change I could make in order to make it faster.
$SqlDocmasterTableResuls = $this.SqlConnection.GetSqlData("SELECT DOCNUM, DOCLOC FROM MHGROUP.DOCMASTER WHERE ENTRYWHEN between '" + $this.FromDate + "' and '" + $this.ToDate + "'")
[System.Collections.ArrayList]$ListOfDocuments = [System.Collections.ArrayList]::New()
if ($SqlDocmasterTableResuls.Rows.Count)
{
foreach ($Row in $SqlDocmasterTableResuls.Rows)
{
$DocProperties = #{
"DOCNUM" = $Row.DOCNUM
"SOURCE" = $Row.DOCLOC
"DESTINATION" = $Row.DOCLOC -replace ([regex]::Escape($this.iManSourceFileServerName + ":" + $this.iManSourceFileServerPath.ROOTPATH)),
([regex]::Escape($this.iManDestinationFileServerName + ":" + $this.iManDestinationFileServerPath.ROOTPATH))
}
$DocObj = New-Object -TypeName PSObject -Property $DocProperties
$ListOfDocuments.Add($DocObj)
}
return $ListOfDocuments

Answer 1

Avoid appending to an array in a loop. The best way to capture loop data in a variable is to simply collect the loop output in a variable:
$ListOfDocuments = foreach ($Row in $SqlDocmasterTableResuls.Rows) {
New-Object -Type PSObject -Property #{
"DOCNUM" = $Row.DOCNUM
"SOURCE" = $Row.DOCLOC
"DESTINATION" = $Row.DOCLOC -replace ...
}
}
You don't need the surrounding if conditional, because if the table doesn't have any rows the loop should skip right over it, leaving you with an empty result.
Since you want to return the list anyway, you don't even need to collect the loop output in a variable. Just leave the output as it is and it will get returned anyway.
Also avoid repeating operations in a loop when their result doesn't change. Calculate the escaped source and destination paths once before the loop:
$srcPath = [regex]::Escape($this.iManSourceFileServerName + ':' + $this.iManSourceFileServerPath.ROOTPATH)
$dstPath = [regex]::Escape($this.iManDestinationFileServerName + ':' + $this.iManDestinationFileServerPath.ROOTPATH)
and use the variables $srcPath and $dstPath inside the loop.
Something like this should do:
$SqlDocmasterTableResuls = $this.SqlConnection.GetSqlData("SELECT ...")
$srcPath = [regex]::Escape($this.iManSourceFileServerName + ':' + $this.iManSourceFileServerPath.ROOTPATH)
$dstPath = [regex]::Escape($this.iManDestinationFileServerName + ':' + $this.iManDestinationFileServerPath.ROOTPATH)
foreach ($Row in $SqlDocmasterTableResuls.Rows) {
New-Object -Type PSObject -Property #{
'DOCNUM' = $Row.DOCNUM
'SOURCE' = $Row.DOCLOC
'DESTINATION' = $Row.DOCLOC -replace $srcPath, $dstPath
}
}
return

Answer 2

[edit - per Ansgar Wiechers, the PSCO accelerator is only available with ps3+.]
one other thing that may help is to replace New-Object with [PSCustomObject]. that is usually somewhat faster to use. something like this ...
$DocObj = [PSCustomObject]$DocProperties
another way to use that type accelerator is to do what Ansgar Wiechers did in his code sample, but use the accelerator instead of the cmdlet. like this ...
[PSCustomObject]#{
'DOCNUM' = $Row.DOCNUM
'SOURCE' = $Row.DOCLOC
'DESTINATION' = $Row.DOCLOC -replace $srcPath, $dstPath
}
hope that helps,
lee