What I'm trying to do
I'm trying to run several tests written in the XCTest framework without code signing, for the purpose of not having to deal with the hassle of importing code signing identities on CI.
What I've tried
I've removed the development team and set the signing certificate to "Sign to run locally" for both the app and test target. Building with
xcodebuild build-for-testing -scheme <my scheme>
works on both CI and locally, but attempting to run the tests with
xcodebuild test-without-building -scheme <my scheme>
fails after around 10s with the error <name> (83559) encountered an error (Failed to load the test bundle.. <test binary> not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc? a few lines down sheds more light about the issue. Seems like macOS didn't like the unsigned test bundle and refused to load it.
Running the test command like so:
xcodebuild test-without-building -xctestrun /Users/vinkwok/Library/Developer/Xcode/DerivedData/ChessFrontend-carjpisbuprsabeulpsycoxjqjea/Build/Products/ChessFrontend_macosx13.0-arm64.xctestrun -destination "platform=macOS,id=<id>,arch=arm64"
throws the following error:
Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: The test runner exited with code 1 before checking in. ...)
It looks like the same test bundle error is occurring, except this time it's not printed out because the test runner simply failed to start.
I've also attempted to sign the test bundle with my development cert with codesign, but that just caused xcodebuild test-without-building to complain that the bundle was missing a development team.
Could anyone point me in the right direction about how to run tests on CI environments without the hassle of dealing with signing and code signing identities? Answers would be greatly appreciated!
Related
When running my XCUITests on Azure Devops and I use 2 simulators, I get the following error:
Unable to boot the Simulator. launchd failed to respond. (Underlying Error: Failed to start launchd_sim: could not bind to session, launchd_sim may have crashed or quit responding)))
When I use one simulator it works. Is there a way to reset the simulator are delete deviated data? Any other idea what could help is highly appreciated.
EDIT: Here are the 2 simulators I run:
destinations: |
platform=iOS Simulator,name=iPhone 8,OS=15.4
platform=iOS Simulator,name=iPhone 14,OS=16.1
I first ran them maxParallel: 4. Now I reduced to maxParallel: 1 with the result that it sometimes works and sometimes not. Really strange.
In the Xcode project I use a testplan where "Execute in parallel (if possible)" is not checked.
Hi I have an macOS Application which needs to be configured for CI/CD purpose on the Jenkins.
I need to run the test cases before making the application available for the distribution or before creating a package file.
I am using the below mentioned command
xcodebuild -workspace xxx.xcworkspace -sdk macosx11.3 -scheme xxx -destination "platform=OS X,arch=x86_64" clean test
when I execute this command in the Jenkins it throws me an error
ERROR: The test runner encountered an error (Failed to establish communication with the test runner. If you believe this error represents a bug, please attach the result bundle at /Users/ec2-user/Library/Developer/Xcode/DerivedData/xxx-gkolnyonjdnyixfknefsjvbsyzia/Logs/Test/Run-xxx-2021.05.24_11-58-07-+0000.xcresult. (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service on pid 0 named com.apple.testmanagerd.control was invalidated.))
I need to make use of macOS itself to run as a simulator. As Jenkins runs as a Daemon on the ec-2 instance.
NOTE: NOT USING XCODE PLUGIN TO EXECUTE MY COMMANDS
Please help!!!
My MacOS system was a slave and I was running my scripts through the Linux server.
macOS was connected using the SSH earlier then this authentication method was required to be changed to access the UI of the Mac
Test cases running now
Based on observation, macOS Big Sur, Monterey, and probably subsequent versions no longer allow tests to be run through a SSH session. By changing the way Jenkins controller connects to the slave, you can prevent the command from being run through SSH.
The first way, Jenkins built-in, to do this and easiest is to use the JNLP method for agent communication. This will involve running a Jenkins agent on the slave machine that communicates with the Jenkins controller. In the end, the agent on the machine is running the test command rather than through a SSH session.
The second way is to build your own agent. It could be as simple as a file change trigger leading to a script executing. This is only necessary if for some reason the Jenkins slave is not able to connect to the Jenkins controller due to some networking constraints.
Getting this error when running unit tests on Xcode 10 / Swift 4.2 on Travis CI with xcode10 image. Tests running properly from command line on local.
xcodebuild Error Domain=NSPOSIXErrorDomain Code=53 "Software caused connection abort" UserInfo={NSLocalizedDescription=Error returned in reply: Connection interrupted}
xcodebuild Error Domain=IDETestOperationsObserverErrorDomain Code=4 "Error returned in reply: Connection interrupted" UserInfo={NSLocalizedDescription=Error returned in reply: Connection interrupted, NSLocalizedRecoverySuggestion=If you believe this error represents a bug, please attach the result bundle at /Users/travis/Library/Developer/Xcode/DerivedData/...}
Testing failed:
xctest encountered an error (Error returned in reply: Connection interrupted)
** TEST EXECUTE FAILED **
Don't recall having seen this "connection interrupted" error before.
I was experiencing this as well on some Xcode 10 and 10.1 Beta agents as well. After our team turned off parallel testing it seemed to work correctly again.
I'm not sure if turning off parallel testing is a good solution for you, but it worked for us. I can't find any documented bugs yet that say this is an issue.
So after trying different things, including turning off tests parallelization, as Brian suggested before, we figured out this seems to be related to having both multiple destinations and parallelization.
There is a new -disable-concurrent-destination-testing flag that can be passed to xcodebuild, to force multiple destinations to be run sequentially.
By using this flag, we can enable both test parallelization and randomization while testing multiple destinations, all in a single command.
xcodebuild -scheme <YourScheme> -destination <DestinationA> -destination <DestinationB> -quiet -disable-concurrent-destination-testing test-without-building
This seems to be working well both locally and on CI servers.
My integration succeeded but archive fails every-time with following warning mentioned below -:
**Warning: Build Service Error:** Creating archive failed: xcodebuild exited with status 65.
**Warning: Build Service Error:** Skipping exporting from archive because creating the archive failed.
I had the same problem. In my case the code signing was set to automatic. When I change it to an distribution certificate it worked.
I was having this issue after updating to the Server beta. I was able to fix it by adding my team to the Xcode server settings.
Its a problem of certificate. Try to check if you have the certificate on your machine corresponding to the build you want to make.
[Not a duplicate of similar questions as explained further down]
I'm getting a code signing error when running XCodeBuild from within Jenkins but it will build ok from the command line or from within Xcode.
Several people have had this problem in the past and the common theme with them is that Jenkins gets run at launch time as the daemon user and thus tries to access the system keychain.
The solutions that people have applied are copying credentials to the system keychain or running a command to set which keychain to use.
However in my case if I look at launchd in the Activity Manager the user is showing up as the user I am logged onto the machine as, therefore Jenkins should be running as this user and not as the daemon user.
I tried setting which keychain to use by adding this command to the Jenkins script before running XCodeBuild
security list-keychains -s /Users/[user]/Library/Keychains/login.keychain
But that did not solve the problem.
The error I am getting is:
Code Sign error: The identity 'iPhone Developer: NNNNN (9TYX5WAM63)' doesn't match any valid, non-expired certificate/private key pair in your keychains"
So I tried moving the credentials to the system keychain but now get this error in Jenkins, but its still fine from the command line:
Code Sign error: Provisioning profile 'F152C66E-B99A-47F6-B262-376CE4403D71' can't be found
Also when I move the credentials to the system keychain I am no longer able to build from within XCOde - I get the same error as the top error message above.
I've also tried editing the org.jenkins-ci.plist file to set the user as who I'm logged onto the machine as, in accordance with this, but that too had no effect.
Missing certificates and keys in the keychain while using Jenkins/Hudson as Continuous Integration for iOS and Mac development
Any ideas what I could try next?
I feel for you as the last time we messed with this it was quite the challenge. We did this when we set up a CI server for a project. If I remember correctly (broadly) we followed these steps:
1) Set Jenkins up as a developer on the Apple Dev account
2) Created a cert and dev provisioning profile specifically for Jenkins
3) Setup a Jenkins user on the CI Server and delete any previous certs/prov profiles from xcode for that user
4) Add the Jenkins cert to the AD-Hoc Dist Profile (If you are trying to distribute an .ipa to HockeyApp you will need to to this for an archive build.)
5) Go into the xcode project file and delete any existing references to Provisioning Profiles
6) Download and Install the certs for this user and the appropriate Prov Profiles.
7) Before the build unlock the Jenkins user keychain. Do this only if the Jenkins user is not the user running the xcode build.
Verify that Xcode shows the Prov Profile as valid in the organiser when logged in as the Jenkins user.
I know my answer is somewhat vague and my intent is to be helpful. This type of error is usually caused by one of three things. Xcode cant find the profile, Xcode cant find the cert or Xcode found more then one profile (supposed to produce a different error but doesn't always) and has a mismatch of some kind. Usually redoing the Prov Profiles is the least painful solution.
Best of luck man. You will solve it it's just a huge headache!
Probably not the answer you are looking for, but i gave up on the XCodeBuild plugin for a number of reasons, and running my build through "Execute Shell" step.
You said your command line build works, so you already know the commands that are required. Just put that into the shell.
xcodebuild -verbose -alltargets -configuration Debug clean build CODE_SIGN_IDENTITY="${CODE_SIGN_IDENTITY}" PROVISIONING_PROFILE=${PROVISIONING_PROFILE}
&&
/usr/bin/xcrun -sdk iphoneos PackageApplication -v "${WORKSPACE}/client_trunk/build/Debug-iphoneos/${Application}.app" -o "${WORKSPACE}/client_trunk/build/Debug-iphoneos/${Application}-Debug-${shortVer}.${revVer}.ipa" --sign "${CODE_SIGN_IDENTITY}" --embed "/Users/[youruser]/Library/MobileDevice/Provisioning Profiles/${PROVISIONING_PROFILE}.mobileprovision"
Above, ${CODE_SIGN_IDENTITY} is the one that looks like iPhone Developer: blah
And ${PROVISIONING_PROFILE} is the hex number for the profile like F152C66E-B99A-47F6-B262-376CE4403D71
Follow instruction on the Xcode Plugin home page :
If this prompt is not showing on the build machine, you can force it to appear by running the codesign command that failed from a terminal on the build machine: /usr/bin/codesign --force --sign "iPhone Distribution: .....
Execute failed signing command from the terminal as a Jenkins user and select "Always allow"
Using Jenkins I got this error:
/Users/Shared/Jenkins/Home/jobs/ExampleTabbed-Integration/workspace/build/Debug-iphoneos/ExamplesTabbed.app: User interaction is not allowed.
Command /usr/bin/codesign failed with exit code 1
I fixed it by doing the following:
Add Build Step before the XCode (jenkins plugin) step
Add the following as the execute command:
security unlock-keychain -p "passwordhere" ${HOME}/Library/Keychains/login.keychain
In the XCode (via plugin) configuration I'm using the following:
Unlock Keychaing ? (unchecked)
Keychain path: ${HOME}/Library/Keychains/login.keychain
Keychain password: (empty)
It might be possible to remove the pre-step and use the actual xcode config to do it, but it's working as described with no problems.
Note this doesn't take into account using a more secure solution yet, but this was my solution to the xcode signing issue. I'm running on OSX with the Jenkins installer and running as a launchctl command, used by the default Jenkins installer application. If that helps.
Subprocesses of Java/Jenkins will need to have unlocked access to the Keychain.
Let's say your build / code-sign / release process is working fine. The only blocker is when you try running the exact same process in Jenkins.
Chances are that the keychain is locked, not for the user, but for Jenkins sub-processes.
The security command to unlock the keychain will need to be run as a Jenkins job, just once. Even if the same logged in user has already unlocked the keychain in the terminal.
So setup a simple freestyle job in Jenkins and add the following shell command as a build phase.
security unlock-keychain -p $PASSWORD /Users/$MACOS_USER/Library/Keychains/login.keychain-db
Provide $PASSWORD as an input parameter for the Jenkins job.
$MACOS_USER the macos username of the Mac keychain owner.
Once you've run this job choose "Always Allow" from the keychain prompt. Now, try your regular build/release job & it should work. Assuming you have all the required certificates in the login keychain.
NEVER HARDCODE PASSWORDS INTO CONFIG/JOBS/CODE etc.