Text File Processing using powershell - Performance issue - performance

Question

I am using powershell script below to read and process one 17 MB text file. The input file contains around 200 000 rows and 12 columns. Currently the script takes almost 1 hour to process the input file. How to optimize the processing time?
Script:
$fields = Get-Content Temp.txt
$results = #()
foreach($i in $fields)
{
$field = $i -split '\t' -replace '^\s*|\s*$'
$field1 = $field[0]
$field2 = $field[1]
$field3 = $field[2]
$field4 = $field[3]
$field5 = $field[4]
$field6 = $field[5]
$field7 = $field[6]
$field8 = $field[7]
$field9 = $field[8]
$field10 = $field[9]
$field11 = $field[10]
$field12 = $field[11]
if ($field1 -eq "4803" -and $field[2].substring(0,2) -eq "60")
{
$field2 = "5000000"
}
else
{
$field2 = $field[1]
}
$details = #{
Column1 = $field1
Column2 = $field2
Column3 = $field3
Column4 = $field4
Column5 = $field5
Column6 = $field6
Column7 = $field7
Column8 = $field8
Column9 = $field9
Column10 = $field10
Column11 = $field11
Column12 = $field12
}
$results += New-Object PSObject -Property $details
}
$results | ForEach-Object { '{0} {1} ... {11}' -f $_.Column1,$_. Column1,... $_.Column12 } | Set-Content -path Temp.txt
[Environment]::Exit(0)

Answer 1

Unless I'm missing something here the goal is to take in tab delimited data, modify one field based on another, and then outputting as CSV data, correct? If so this one-liner should execute MUCH faster.
Import-Csv test.txt -Header #(1..12) -Delimiter `t | % {if(($($_.2) -eq "4803") -and($($_.3).substring(0,2) -eq "60")){$_.2 = "5000000"};$_} | export-csv test2.csv -NoTypeInformation
It avoids all the weird string parsing and gets around the biggest problem which is
$results += New-Object PSObject -Property $details
That line is copying your entire array into a new array for each line of your script, which is problematic for performance. the rest of the changes just make things slightly faster.

Answer 2

If this were me, I would start to think about not using Get-Content if your files are going to get much bigger. Memory consumption will start to become an issue, and using Get-Content won't scale well if your files get really big because you pull everything into memory. And remember it will be more memory than the size of the file, because it has to represent things as objects (which is still smaller than an XML DOM, but regardless, it takes memory).
So first of all, you could loop through the input file using a stream reader, I have an example here: https://stackoverflow.com/a/32337282/380016
You can also write your output file using the stream writer, instead of concatting a big object like you are, only too loop through it and write it to a file at the end.
In the while loop of my example, you can still split the string as you are, if you want, do your manipulations, and then write it out. No need to accumulate it and wait to do it all at the end.
This approach should be faster and should use hardly any memory.