Ad-Hoc codesigning for device succeeds in Studio, Fails in Jenkins - visual-studio

Question

I have a Xamarin Forms application that supports Android and iOS. I've generated Jenkins builds to compile them. All of the Android builds work. The iOS Debug build compiles fine. The Ad-Hoc build, however, fails to build completely for an iPhone target. It appears to be failing during codesigning. It works if I target the iPhoneSimulator, but if I target iPhone device it fails.
Tool /usr/bin/codesign execution started with arguments: -v --force --sign 81088F8E194139DC4C6CE640716944E41FB0709F --entitlements "/Users/Shared/Jenkins/.jenkins/workspace/{project path}/obj/iPhone/Ad-Hoc/Entitlements.xcent" --deep "/Users/Shared/Jenkins/.jenkins/workspace/{project path}/bin/iPhone/Ad-Hoc/AppName.app"
bin/iPhone/Ad-Hoc/AppName.app : error : /Users/Shared/Jenkins/.jenkins/workspace/{project path}/bin/iPhone/Ad-Hoc/AppName.app: unknown error -1=ffffffffffffffff [/Users/Shared/Jenkins/.jenkins/workspace/{project path}/iDriverMobile.iOS.csproj]
If I open up the Solution in Visual Studio, right in the Jenkins workspace folder so it's using the exact same files, then compilations works fine, which is really frustrating.
Looking at differences between the two outputs, it seems that the working build (from Studio) has AOT output for all of the assemblies that looks like this:
Mono Ahead of Time compiler - compiling assembly /Users/Shared/Jenkins/.jenkins/workspace/{project path}/obj/iPhone/Ad-Hoc/mtouch-cache/32/Build/OpenNETCF.Google.Analytics.dll
The failing build has none of those. Instead, it has a couple lines that look like this:
MTOUCH : warning MT0095: Aot files could not be copied to the destination directory /Users/Shared/Jenkins/.jenkins/workspace/{project path}/obj/iPhone/Ad-Hoc/mtouch-cache/64/Build/Msym/Msym/tmp: Could not start process. [/Users/Shared/Jenkins/.jenkins/workspace/{project path}/AppName.csproj]
The worst part of all of this is that these builds did work, but then I restarted the Mac Mini that Jenkins is running on and things went downhill. I can't figure out what the difference is between what Studio is doing and the command line call to msbuild. They both point to the same binaries.
Additional Information
This still fails with the latest updates as of today (5/24/17). This is the environment:
Mac OS X 10.12.5
List item
XCode 8.3.2
Xamarin.iOS 10.10.0.36
Visual Studio 2017 Community for Mac 7.0.1 (build 24)
Mono 5.0.1.1
What doesn't fix it:
Creating a new Jenkins build
Changing the Jenkins workspace path
Opening up permissions (777) to the entire Jenkins folder
Enabling LLVM
Disabling all linking
Completely uninstalling and re-installing Jenkins
Using xbuild instead of msbuild
Swearing a lot
My middle finger

Answer 1

Try to delete the derived data folder in DerivedData of your app. It looks like YourAPP_ dasfdsfsdafdsasfdsaf, according to this from Apple Developer Forum.
The DerivedData data folder is located at ~/Library/Developer/Xcode/DerivedData/
If this does not work, all the symptoms point to a signing certificate (also called, signing identity) issue.
It seems like when it was compiled from command line, /usr/bin/codesign can not access signing identity 81088F8E194139DC4C6CE640716944E41FB0709F. It could be many different reasons, unfortunately:
keychain was locked
codesign is not allowed to access the signing
identity.
multiple identities exist in keychain and wrong signing
identity was selected
Wrong provision
profile was matched for Ad Hoc build.
Try to add following code snippets before running msbuild, assuming your signing identity is in keychain ~/Library/Keychains/login.keychain:
security unlock-keychain -p <password> ~/Library/Keychains/login.keychain
security set-keychain-settings -l -u -t 3600 ~/Library/Keychains/login.keychain
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k -p <password> ~/Library/Keychains/login.keychain
It is not a good idea to have keychain password stored in the build script, you can follow this guide to hide them.