Where-Object or Compare-Object which would be better? - performance

Question

This may be a philosophical question but I would like to know how the following 2 items differ, from a speed and efficiency perspective. In PowerShell I have 2 objects that look like this:
$ObjectA = #()
1..10 | foreach-object{
$obj = New-Object System.Object
$obj | Add-Member -Type NoteProperty -Name index -Value $_
$ObjectA += $obj
}
$ObjectB = #()
5..15 | foreach-0bject{
$obj = New-Object System.Object
$obj | Add-Member -Type NoteProperty -Name index -Value $_
$ObjectB += $obj
}
Now, I want to get the objects that exist in both. I can do it 1 of 2 ways.
Solution 1:
$ObjectA | foreach-object{
$ind = $_
$matching = $ObjectB | where {$_ -eq $ind}
if (![string]::IsNullOrEmpty($matching)){
##do stuff with the match
}
}
Solution 2:
$matches = Compare-Object $ObjectA $ObjectB -Property index | where {$_.SideIndicator -eq '=='} -PassThru
$matches | foreach-object {
##do stuff with the matches.
}
My question is, when my array of objects gets very large (30K+) which one is going to be a better solution from a performance perspective? I don't know how the Compare-Object cmdlet works internally so I really don't know. Or does it not matter?
Thanks in advance.

Answer 1

As #Knows Not Much has pointed out, Compare-Object usually offers better performance than iterating the collection and comparing objects yourself. But the other answer fails to use the -ExcludeDifferent parameter and instead iterates over the Compare-Object output. This means doing many useless string comparisons for the SideIndicator property. For optimal performance, and simpler code, just use -IncludeEqual and -ExcludeDifferent:
$ObjectA = #()
1..10000 | %{
$obj = New-Object System.Object
$obj | Add-Member -Type NoteProperty -Name index -Value $_
$ObjectA += $obj
}
$ObjectB = #()
1000..7000 | %{
$obj = New-Object System.Object
$obj | Add-Member -Type NoteProperty -Name index -Value $_
$ObjectB += $obj
}
# Iterating over the result of Compare-Object takes 2.6 seconds.
Measure-Command { $matches_where_eq = Compare-Object $ObjectA $ObjectB -Property index -IncludeEqual | where {$_.SideIndicator -eq '=='} ; echo $matches_where_eq.count }
# Using -IncludeEqual and -ExcludeDifferent takes 2.1 seconds (80% of previous).
Measure-Command { $matches_ed_ie = Compare-Object $ObjectA $ObjectB -Property index -ExcludeDifferent -IncludeEqual; echo $matches_ed_ie.Count }

Answer 2

Even if you take a dataset of size 10000 you can easily see that compare object is way way faster.
I modified your code to make it work on powershell 3.0
cls
$ObjectA = #()
1..10000 | %{
$obj = New-Object System.Object
$obj | Add-Member -Type NoteProperty -Name index -Value $_
$ObjectA += $obj
}
$ObjectB = #()
1000..7000 | %{
$obj = New-Object System.Object
$obj | Add-Member -Type NoteProperty -Name index -Value $_
$ObjectB += $obj
}
Measure-Command {
$count = 0
$matches = Compare-Object $ObjectA $ObjectB -Property index -IncludeEqual | where {$_.SideIndicator -eq '=='}
}
echo $matches.length
echo $matches.Count
Measure-Command {
$count = 0
$ObjectA | %{
$ind = $_
$matching = $ObjectB | where {$_.Index -eq $ind.Index}
if (![string]::IsNullOrEmpty($matching)){
$count = $count + 1
}
}
echo $count
}
The compare-object returns in less than 5 seconds .... but the other approach just gets stuck forever.

Answer 3

This will build a regex search out of one array and then perform a regex match against the other array.
Solution 3:
[regex]$RegMatch = '(' + (($ObjectA |foreach {[regex]::escape($_)}) –join "|") + ')'
$ObjectB -match $RegMatch
Might want to throw some logic at that to build the regex out of the smaller set of data and then run the larger set against it to speed things up, but I'm pretty sure this would be fastest.