Azure Custom Script Extension continually restarting - windows

Question

Having a bit of trouble with the Azure Custom script extension on a windows VM. I have created a script that installs a particular business application. In order to accomplish this, the script calls several other scripts and a reboot is required in between two scripts.
The script is designed such that it can "pick up where it left off" by checking whether certain files exist or if registry keys exist. The reboot portion of this goes fine, however, I'm noticing that the script handler sets several registry keys to keep track of its state in
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure\ScriptHandler
I see a registry value called "MreSeqNumStarted", which I presume is how the script handler keeps track of what script it's executing. There is only one script fired by the extension currently, and this value is set to 0.
What I'm seeing, is that, when the machine is rebooted after the script has completed, the script fires again. This occurs with every reboot of the virtual machine. I believe the script extension is not marking the script as "completed", and I'm somewhat at a loss as to why. The basic (sanitized) flow of the script is below:
trap {$_.Exception.Message; Get-PSCallstack;continue}
if(Test-Path C:\ScriptLog.txt)
{
Add-Content "C:\ScriptLog.txt" "`nScript Previously Run, Prior log exists."
} else {
Write-Host "Begin Script" | Out-File "C:\ScriptLog.txt"
}
#$storagePath = [uri]::EscapeDataString($storagePath)
#Create Directory Structure and download scripts/files
if(Test-Path C:\PreparePlatform\) {
Add-Content "C:\ScriptLog.txt" "`nFiles Already Downloaded... Continuing"
} else {
New-Item -ItemType Directory -Force -Path C:\SoftwareName\PreparePlatform\
(New-Object Net.WebClient).DownloadFile('SAS-URL-GOES-HERE','C:\SoftwareName\PreparePlatform\PreparePlatform.zip');(new-object -com shell.application).namespace('C:\SoftwareName\PreparePlatform').CopyHere((new-object -com shell.application).namespace('C:\SoftwareName\PreparePlatform\PreparePlatform.zip').Items(),16)
(New-Object Net.WebClient).DownloadFile('SAS-URL-GOES-HERE','C:\SoftwareName\SOMEZIP.zip');(new-object -com shell.application).namespace('C:\SoftwareName').CopyHere((new-object -com shell.application).namespace('C:\SoftwareName\SOMEZIP.zip').Items(),16)
Add-Content "C:\ScriptLog.txt" "`nExtracted Install Media"
}
if(Test-Path C:\SoftwareName\preparedone.txt) {
Add-Content "C:\ScriptLog.txt" "`nPreparePlatform already completed... Continuing"
} else {
Invoke-Expression "Set-Location c:\SoftwareName\PreparePlatform\; C:\SoftwareName\PreparePlatform\Prepare-WindowsPlatform.ps1"
CreateSQLUser -identifierValue $value -saPassword $sqlSaPassword -tenantPass $TenantSqlPassword
CreateSQLLogin -identifierValue $value -saPassword $SqlSaPassword -tenantPass $TenantSqlPassword
Write-Host "PreparePlatform Complete" | Out-File "C:\SoftwareName\preparedone.txt"
}
if(Test-Path C:\SoftwareName\UpdatePlatform.txt) {
Add-Content "C:\ScriptLog.txt" "`nUpdatePlatform already completed... Continuing"
} else {
Add-Content "C:\ScriptLog.txt" "`nBegin Update-Platform"
Invoke-Expression "Set-Location C:\SoftwareName\PreparePlatform\; C:\SoftwareName\PreparePlatform\Update-WindowsPlatform.ps1"
Write-Host "UpdatePlatform Complete" | Out-File "C:\SoftwareName\UpdatePlatform.txt"
}
if(Test-Path C:\SoftwareName\Rebooted.txt) {
Add-Content "C:\ScriptLog.txt" "`nReboot already completed... Continuing"
} else {
Write-Host "Rebooting" | Out-File "C:\SoftwareName\Rebooted.txt"
Restart-Computer -Force
Restart-Computer -Force
Start-Sleep 3600 #Make sure nothing else executes
}
# TODO: Make more robust for potentially more nodes... need to delay 1 hour for each node
If($env:computername -eq 'pvm-SoftwareName-1') { Write-Output "Second Node, waiting 1 hour"; Add-Content "C:\ScriptLog.txt" "`nNot First Node, waiting 1 hour" ;Start-Sleep -Seconds 3600 }
Add-Content "C:\ScriptLog.txt" "`nPrepare SoftwareName Installers"
Set-Location C:\SoftwareName\Script\; C:\SoftwareName\Script\Prepare-SoftwareNameInstallers.ps1 -neededParameters $values
Add-Content "C:\ScriptLog.txt" "`nPrepare SoftwareName Installers complete"
try
{
if(-not (Get-Service -Name 'SoftwareName server')) {
C:\SoftwareName\Script\Install-SoftwareNameBP.ps1 -neededParameters $values
} else {
Add-Content "C:\ScriptLog.txt" "`n ALREADY INSTALLED!"
}
}
catch
{
$_.Exception.Message;Get-PSCallStack
if (-not (Get-Service -Name 'SoftwareName Server').Status -eq 'Running')
{throw "Service unavailable"}
}
Write-Output "Install Finished."
try
{
if(-not ((Invoke-WebRequest -Uri https://IISSiteURL/).StatusCode -eq '200'))
C:\SoftwareName\Script\Install-SoftwareNameSF.ps1 -neededParameters $values
}
catch
{
$_.Exception.Message;Get-PSCallStack
if(-not (Test-Path -Path "HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{AGUID}") )
{throw "Smartforms Install failed"}
}
try
{
#Begin
C:\SoftwareName\Script\Install-SoftwareNameSP.ps1 -neededParameters $values
}
catch
{
$_.Exception.Message;Get-PSCallStack
if(-not (Test-Path -Path "HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{AGUID}") )
{throw "Sharepoint Install failed"}
}
try
{
#[bool]$skipDeploy
#if($env:computername -eq 'pvm-SoftwareName-0') { $skipDeploy = $false } else { $skipDeploy = $true }
#Begin
C:\SoftwareName\Script\Install-SoftwareNameMG.ps1 -neededParameters $values
catch
{
$_.Exception.Message;Get-PSCallStack
if(-not (Test-Path -Path "HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{AGUID}") )
{throw "Management Install failed"}
}
try
{
C:\SoftwareName\Script\Install-Finalize.ps1 -neededParameters $values
}
catch
{
$_.Exception.Message;Get-PSCallStack
if(-not (Test-Path -Path "HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{AGUID}") )
{throw "Install Finalize Failed"}
}
#Kill the script
Add-Content "C:\ScriptLog.txt" "`nScript Completed"
Exit
Below is the expected output of C:\ScriptLog.txt
Begin Scheduled Job Regstration
Extracted Install Media
Begin PreparePlatform
Completed PreparePlatform
Begin Update-Platform
Script Previously Run, Prior log exists.
Files Already Downloaded... Continuing
PreparePlatform already completed... Continuing
UpdatePlatform already completed... Continuing
Reboot already completed... Continuing
Prepare SOFTWARENAME Installers
Prepare SOFTWARENAME Installers complete
C:\SOFTWARENAME\Script\Install-K2BP.ps1 -parameterValues $values
C:\SOFTWARENAME\Script\Install-K2SF.ps1 -parameterValues $values
C:\SOFTWARENAME\Script\Install-K2SP.ps1 -parameterValues $values
C:\SOFTWARENAME\Script\Install-K2MG.ps1 -parameterValues $values
C:\SOFTWARENAME\Script\Install-Finalize.ps1 -parameterValues $values
Script Completed
After a reboot, the same log file:
Begin Scheduled Job Regstration
Extracted Install Media
Begin PreparePlatform
Completed PreparePlatform
Begin Update-Platform
Script Previously Run, Prior log exists.
Files Already Downloaded... Continuing
PreparePlatform already completed... Continuing
UpdatePlatform already completed... Continuing
Reboot already completed... Continuing
Prepare SOFTWARENAME Installers
Prepare SOFTWARENAME Installers complete
C:\SOFTWARENAME\Script\Install-SOFTWARENAMEBP.ps1 -neededParameters $values
C:\SOFTWARENAME\Script\Install-SOFTWARENAMESF.ps1 -neededParameters $values
C:\SOFTWARENAME\Script\Install-SOFTWARENAMESP.ps1 -neededParameters $values
C:\SOFTWARENAME\Script\Install-SOFTWARENAMEMG.ps1 -neededParameters $values
C:\SOFTWARENAME\Script\Install-Finalize.ps1 -neededParameters $values
Script Completed
Script Previously Run, Prior log exists.
Files Already Downloaded... Continuing
PreparePlatform already completed... Continuing
UpdatePlatform already completed... Continuing
Reboot already completed... Continuing
Prepare SOFTWARENAME Installers
Prepare SOFTWARENAME Installers complete
C:\SOFTWARENAME\Script\Install-SOFTWARENAMEBP.ps1 -neededParameters $values...
Edit1: I've commented out all calls to external scripts. The script at this point only writes output to a file effectivley and traps any exceptions in doing so. The script continues to re-fire with every VM reboot.