"The process cannot access the file 'Default.rd.xml' because it is being used by another process." on AppVeyor CI - visual-studio

Question

I am doing some work on cordova-windows (https://github.com/apache/cordova-windows), which is using AppVeyor for testing on Windows. One of the things that was missing, were tests with Visual Studio 2017 (only VS2015 was used to test before). So I added those and it works like a charm - mostly.
Unfortunately we now have a very strange test failure:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/cordova-windows/build/1.0.458
Started
Creating Cordova Windows Project:
Path: testcreate 応用
Namespace: com.test.app
Name: 応用
Windows project created with cordova-windows#5.1.0-dev
Building project: C:\projects\cordova-windows\testcreate 応用\CordovaApp.Windows10.jsproj
Configuration : release
Platform : x64
Patching 10 in prebuild event...
Injected base.js reference to the www/index.html
Removing /( *)(<script\s+(?:type="text\/javascript"\s+)?src="\/\/Microsoft.WinJS.2.0\/js\/base.js">\s*<\/script>)(\s*)/ from www/index.html
Removing /( *)(<script\s+(?:type="text\/javascript"\s+)?src="\/\/Microsoft.Phone.WinJS.2.1\/js\/base.js">\s*<\/script>)(\s*)/ from www/index.html
CordovaApp.Windows10 -> C:\projects\cordova-windows\testcreate ??\build\windows\release\x64\win10\CordovaApp.Windows10_1.0.0.0_x64.appx
CordovaApp.Windows10 -> C:\projects\cordova-windows\testcreate ??\build\windows\release\x64\win10\Upload\CordovaApp.Windows10_1.0.0.0_x64.appx
CordovaApp.Windows10 -> C:\projects\cordova-windows\testcreate ??\AppPackages\CordovaApp.Windows10_1.0.0.0_Test\CordovaApp.Windows10_1.0.0.0_x64.appxbundle
CordovaApp.Windows10 -> C:\projects\cordova-windows\testcreate ??\AppPackages\CordovaApp.Windows10_1.0.0.0\CordovaApp.Windows10_1.0.0.0_x64.appxbundle
CordovaApp.Windows10 -> C:\projects\cordova-windows\testcreate ??\AppPackages\CordovaApp.Windows10_1.0.0.0_x64_bundle.appxupload
Your package has been successfully created.
Building project: C:\projects\cordova-windows\testcreate 応用\CordovaApp.Windows10.jsproj
Configuration : release
Platform : x86
Patching 10 in prebuild event...
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\AppxPackage\Microsoft.AppXPackage.Targets(2975,5): error MSB3231: Unable to remove directory "build\windows\bld\PackageUploadLayout\". The process cannot access the file 'Default.rd.xml' because it is being used by another process. [C:\projects\cordova-windows\testcreate ??\CordovaApp.Windows10.jsproj]
C:\projects\cordova-windows\testcreate 応用\cordova\node_modules\q\q.js:155
throw e;
^
Error: C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe: Command failed with exit code 1
at ChildProcess.whenDone (C:\projects\cordova-windows\testcreate 応用\cordova\node_modules\cordova-common\src\superspawn.js:169:23)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:920:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:230:5)
The relevant part is this:
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\AppxPackage\Microsoft.AppXPackage.Targets(2975,5): error MSB3231: Unable to remove directory "build\windows\bld\PackageUploadLayout\". The process cannot access the file 'Default.rd.xml' because it is being used by another process. [C:\projects\cordova-windows\testcreate ??\CordovaApp.Windows10.jsproj]
This is not happening when I run the tests locally, meaning I can in no way reproduce what is going on here.
Any idea what process could block this Default.rd.xml file or build\windows\bld\PackageUploadLayout\?
How can I find out in a CI environment like AppVeyor?
Update:
Investigated a bit more by adding /clp:Verbosity=normal to the msbuild calls that are executed.
It runs a similar msbuild call 3 times (once per architecture). The first one succeeds, but during the second one the failure occurs. Makes sense as in the first iteration the folder doesn't exist yet, which it confirms with this output: Directory "build\windows\bld\PackageUploadLayout\" doesn't exist. Skipping..
The last output before the failure then is Removing directory "build\windows\bld\PackageUploadLayout\". which matches what we get in the error message. No indicator why the removing doesn't work though.
Super strange: With /clp:Verbosity=detailed added to the msbuild call, the build actually succeeds! My guess: Because the output takes time, whatever is having a lock on the folder or file releases it.

Answer 1

This is not exactly an answer, but rather investigation report. It is just not enough room comments to describe it. Here is what I did:
forked your repo
created AppVeyor project
added RDP
connected with RDP
installed procmon
added filter to monitor objects which path ends with bld\PackageUploadLayout\Properties\Default.rd.xml
run npm test manually from CMD
When the same error happened I see this:
I am not sure I understand why msbuild failed this way. I opened SHARING VIOLATION event, switched to process and see this:
What catches my eye is that msbuild version is 14. Should be 15 on Visual Studio 2017 image. We have version 14 installed for specific scenarios but default one is 15 (run where msbuild and you will see C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe). I am not sure where this path is set in your scripts, but I feel if you make it run correct msbuild version this may help.