Run macOS Test Cases on the Jenkins Pipeline - macos

Hi I have an macOS Application which needs to be configured for CI/CD purpose on the Jenkins.
I need to run the test cases before making the application available for the distribution or before creating a package file.
I am using the below mentioned command
xcodebuild -workspace xxx.xcworkspace -sdk macosx11.3 -scheme xxx -destination "platform=OS X,arch=x86_64" clean test
when I execute this command in the Jenkins it throws me an error
ERROR: The test runner encountered an error (Failed to establish communication with the test runner. If you believe this error represents a bug, please attach the result bundle at /Users/ec2-user/Library/Developer/Xcode/DerivedData/xxx-gkolnyonjdnyixfknefsjvbsyzia/Logs/Test/Run-xxx-2021.05.24_11-58-07-+0000.xcresult. (Underlying Error: Couldn’t communicate with a helper application. Try your operation again. If that fails, quit and relaunch the application and try again. The connection to service on pid 0 named com.apple.testmanagerd.control was invalidated.))
I need to make use of macOS itself to run as a simulator. As Jenkins runs as a Daemon on the ec-2 instance.
NOTE: NOT USING XCODE PLUGIN TO EXECUTE MY COMMANDS
Please help!!!

My MacOS system was a slave and I was running my scripts through the Linux server.
macOS was connected using the SSH earlier then this authentication method was required to be changed to access the UI of the Mac
Test cases running now

Based on observation, macOS Big Sur, Monterey, and probably subsequent versions no longer allow tests to be run through a SSH session. By changing the way Jenkins controller connects to the slave, you can prevent the command from being run through SSH.
The first way, Jenkins built-in, to do this and easiest is to use the JNLP method for agent communication. This will involve running a Jenkins agent on the slave machine that communicates with the Jenkins controller. In the end, the agent on the machine is running the test command rather than through a SSH session.
The second way is to build your own agent. It could be as simple as a file change trigger leading to a script executing. This is only necessary if for some reason the Jenkins slave is not able to connect to the Jenkins controller due to some networking constraints.

Related

Running GUI Testing on bamboo agent as service

There is a way to run CI of GUI testing in bamboo agent as service on windows server?
I don't think it is possible, just run the bamboo agent as a proccess.
I am using puppeteer as libray to run the GUI testing on electron app.
any type of another solution will help :)
If you can achieve running the build via PowerShell or cmd prompt, you may do so with a script task.
For this, you have to make the agent run as a local user as explained in this page. This way, you have the same environment as the local user and the commands to execute your electron app will work successfully.

drone.io how to run command on local server

I have drone installed on a Mac mini as a CI/CD for Android and iOS apps. To build for iOS I need to run commands on the server itself. I know there is a ssh plugin, but is there another way to run commands on the host from a drone pipeline step?
I would suggest taking a look at the exec runner, sounds like exactly what you would need.
The Exec runner executes build pipelines directly on the host machine without isolation, using the default shell. This runner is not suitable for un-trusted workloads for security reasons.
https://exec-runner.docs.drone.io/

Is there a way to open an interactive Command Prompt from a service?

When I try to execute "vstest.console.exe" file in Jenkins CI (service mode), it responses
Error: Could not start test run for unit tests for Windows Store app:
Unit tests for Windows Store apps cannot be run from a service or non
interactive process. Please run unit tests from an interactive
process..
However it will be able to run successful if I execute the same command from a Command prompt.
So I want to ask if there is a way to open Command Prompt from a service such as Jenkins CI.
I have tried with "start ExecuteVSTest.bat /I /K /REALTIME" but the result is still the same.
I couldn't find a way to open Command Prompt from a service. However I am able to resolve the problem between Jenkins CI and "vstest.console.exe".
We just need to deploy Jenkins from the WAR file via Tomcat server instead of installing Jenkins service native package.
Since the Tomcat is running in the console mode so Jenkins CI won't have the same trouble when executing "vstest.console.exe" as when it run in service mode.

Issue with IOS Instruments running from command line on build machine

I've successfully run IOS UI Automation Instruments from command line on my local Mac. When it starts to test, the simulator will automatically pop up and then do the stuff.
However, when I SSH to a build server (with OSX and xcode, will use Jenkins eventually) and try to do the same thing, it doesn't work well.
Building process is fine, but as long as I run instruments, the terminal will then hang there without doing anything. I figure it has something to do with iphone simulator GUI but since it's CLI, I don't know what's going on...
Do you guys have any idea? Thanks!!
.
After executing the command from Jenkins through SSH; in most cases you would see a pop up window as in the screen shot. You will have to get past this screen to execute your tests. This post addresses how to get past that issue Stop "developer tools access needs to take control of another process for debugging to continue" alert
I've had the same issue. instruments -w "$DEVICE" were ignoring parameters.
Same with
DEVICE_TARGET='iPhone 6 (8.1 Simulator)' cucumber --format json -o test-reports/cucumber-8.1-iphone.json
It were ignoring DEVICE_TARGET parameter.
Solved via running Jenkins slave NOT as a SSH slave but as the Slave agent via java web start
http://i.stack.imgur.com/lNlPA.png
As i understand this issue is caused by OS X launchagent that managing sshd and performs scope of restrictions for remote users.
IMPORTANT! Slave agent should be started from OS X desktop, not via ssh! I've connected via VNC and launched slave java web agent on the OS X UI terminal.

Jenkins : Selenium GUI tests are not visible on Windows

When I run my selenium test (mvn test) from jenkins (windows) I see only the console output. I don't see the real browsers getting opened . How can I configure jenkins so that I can see the browsers running the test?
I had the same problem, i got the solution after many attempts.
This solution works ONLY on windows XP
If you are using jenkins as a windows service you need to do the following :
1) In windows service select the service of jenkins
2) Open properties window of the service -> Logon-> enable the checkbox "Allow service to interact with desktop"
After then you should reboot the service jenkins
Hope this help you :)
UPDATE:
Actually, I'm working on a an automation tool using Selenium on Windows 10, I've installed Jenkins ver. 2.207 as windows application (EXE file), it's running as windows service and ALL drivers (Chrome, FireFox, IE) are visible during test executions WITHOUT performing a mere configuration on the System or Jenkins
I got the solution. I ran jenkins from command prompt as "java -jar jenkins.war" instead of the windows installer version. Now I can see my browser based tests being executed.
If you are already doing what #Sachin suggests in a comment (i.e. looking at the machine where Jenkins actually runs) and still do not see the browsers, then your problem may be the following:
If you run Jenkins as a service in the background it won't open apps in the foreground. You may either try to run it not as a service in the foreground, or run it as a Local System account and check Allow the service to interact with desktop option. In the latter case you may get into permission problems, though.
Update: To make sure this answer is understood properly by others: Jenkins Windows 'native' installation is not really native. It's a wrapper around Java that runs it as a service.
To interact with desktop GUI, you should launch slave agent via JNLP:
https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-LaunchslaveagentviaJavaWebStart
After adding the node in Jenkins (configured as Java Web Start launch), just make a startup batch script on the node machine:
java -jar slave.jar -jnlpUrl http://{Your Jenkins Server}:8080/computer/{Your Jenkins Node}/slave-agent.jnlp
(slave.jar can be downloaded from http://{Your Jenkins Server}:8080/jnlpJars/slave.jar)
See more answers here:
How to run GUI tests on a jenkins windows slave without remote desktop connection?
In the case of Windows 7 you should not install jenkins as windows application (because in this recent version, Microsoft decided to give services their own hidden desktop even you enable the functionality "interact with desktop" in jenkins service), you may have to deploy it from a war file as follows:
1) Download jenkins.war from Jenkins official site
2) Deploy it by the command prompt : java -jar {directoryOfJenkinsFile}/jenkins.war
3) Now you can access jenkins administration on http:// localhost:8080
Hope that helps you !
this is an issue for Jenkins. on Windows it is possible to access logon user's session (screen) under system account. to make the UI testing visible, Jenkins needs to bypass UAC (user access
control) at background. this solution works for me with my own service running as system account.
I also faced the same issue earlier in my local machine (Windows 10).
My test was running perfectly from the NetBeans but when I moved to Jenkins it was only running in console mode. I was unable to view the UI.
So for that, you just need to make your local machine as a Jenkins slave by creating a new slave node in your Jenkins and select that node to execute the Jenkins job.
If jenkins installed by windows installer it is showing only Console out put only. To see browsers download jenkins.war file and run java -jar jenkins.war from command line.
Go through this site:
http://learnseleniumtesting.com/jenkins-and-continuous-test-execution/
If you have the following situation,
You are able to login to the remote machine
You don't see the Jenkins agent window
This slave machine is accessed by many users then try the following,
then try the following suggestion.
Login to slave machine
Go to Task manager
Users
Logout all the users
Then login again.
This worked for me.

Resources