powershell: delete files previously used as argument inside script - windows

Question

I need to periodically scan a folder for new fontfiles to install and delete them afterwards using a powershell script.
During processing I want to skip already installed files and to achieve that I need to resolve the "real" fontname of the provided file.
I figured out everything and it seems to work everything but the file deletion.
The deletion did work until I added the font name resolution using this GlythTypeInterface Object. It seems like the invoked object does "file lock" the fontfile resulting in an UnauthorizedAccessException.
Thats why I tried some garbage collection stuff I found but I can't make it work.
My code so far:
Add-Type -AssemblyName PresentationCore
$FONTS = 0x14
$Path="C:\_fonts_to_install"
$FontItem = Get-Item -Path $Path
$FontList = Get-ChildItem -Path "$FontItem\*" -Include ('*.fon','*.otf','*.ttc','*.ttf')
$objShell = New-Object -ComObject Shell.Application
$objFolder = $objShell.Namespace($FONTS)
$Fontdir = dir $Path
$username = $env:UserName
foreach($File in $FontList) {
$try = $true
$installedFonts = #(Get-ChildItem C:\Users\$username\AppData\Local\Microsoft\Windows\Fonts | Where-Object {$_.PSIsContainer -eq $false} | Select-Object basename)
$fontObject = New-Object -TypeName Windows.Media.GlyphTypeface -ArgumentList $File.fullname
$fontName = $fontObject.Win32FamilyNames.Values
Write-Host $fontName
$fontObject = $null
Remove-Variable fontObject
foreach($font in $installedFonts)
{
if ($font -match $fontName)
{
$try = $false
}
}
if ($try)
{
$objFolder.CopyHere($File.fullname)
}
Write-Host $File
Remove-Item $File -Force -Verbose
}