I am running UI tests in a single Xcode bot. The bots are run on a separate computer which is only used for this task and nothing else. The tests start running fine at first, but after several successful runs they start all failing with this error.
UI Testing Failure - App state is still not terminated
After the first test fails with this error - all tests after it fail as well. The only cure I found is to restart the server computer. How can I prevent this error?
OS X 10.11 (15A263e), Xcode 7.0 (7A192o), OS X Server 5.0.1 (15S2235)
Related
We are running Automation scripts for Mobile devices with Android, and IOS.
We are trying to run two feature files for two Android devices(One Plus and Samsung S10).
Initially scripts are ran on both the devices but when the first Feature file execution is stopped then automatically the execution process on Second device also stopped.
Appium version - 1.16.0
We tried changing the order of the devices also like Running Feature file of Samsung s10 first and One Plus later and changed by initiating the execution of feature file of OnePlus device first and Samsung S10 next.
In both the cases the same happened when execution of first initiated is ended the other process is stopped suddenly without any error
We are retrieving the capabilities of the devices from database.
Running these scripts on Mac machine
I suspect that switching devices in the middle of a run is probably not a good idea
So to start I'd suggest creating a project folder for each device you are trying to test. Then run cucumber in one folder to test the first device and then a second folder to test the second.
Once you have that working then you can think about how to run both things at the same time, without having multiple project folders on your machine. There are lots of ways to do this
setup a dedicated CI machine
use a CI service like CircleCI
...
The key idea here is to parallize using multiple Cucumber instances rather than a single cucumber instance.
We're trying to run our tests automatically on a test machine where all the non-UI tests run perfectly well. We have scripts that run the tests on the command line and collect results for analysis.
The test machine (a Mac mini) runs in a headless manner.
For UI tests, however, the test fails to launch the app with an error of the kind,
t = 83.86s Assertion Failure: <TestCaseName>: Application '<XCUIApplicationImpl: 0x60400026e300 com.<app-identifier> at <app-name>.app>' failed to activate (current state: Running Background)
It seems that it's due to the fact that there's no GUI session around when the machine is running headless (macOS calls such a session an Aqua session). We've tried to find a way around this, but haven't made any real progress.
If anyone has any ideas, we'd appreciate.
I believe I was able to get around this by disabling the "Require Password" setting in System Preferences > Security & Privacy and not allowing the machine to sleep or start a screen saver. There's no screen attached to the Mac Mini so that's fine.
It seems really bizarre that Visual Studio for Mac (the recently-renamed Xamarin Studio) doesn't support automated unit testing for their mobile device (Android/iOS) applications. Actually, I guess it isn't that bizarre, they REALLY want you to pay for their cloud testing service.
How can I run my own automated unit tests, on my own machine? Here's what I mean by that - I want to create an application that:
Contains C# files containing unit tests that require NO user interface
Can be invoked from my Jenkins or Hudson automated build server
Can run on either a physical iDevice attached to the build server, or the iPad simulator
Runs all unit tests, reports their pass/fail status back to the build server, and then exits, all without any human intervention.
It seems like I could do this on the simulator using
xcrun simctl spawn
and then either
Have my test app writes its output to its storage and figure out where on my Mac's hard disk the simulator storage lives, or
Have my test app send its output to the simulator's pasteboard and grab it using
xcrun simctl pbpaste
But these both require that I find a way to write a C# app that runs on iOS, runs my tests, stores the output where the "real world" can find it, and then shuts down.
And I have to figure out how to copy all of the necessary EXEs and DLLs over to the simulated iPad? Or have Visual Studio package it all into an actual Mac app?
To run unit tests on an actual device you can use NUnit.Xamarin. This is a library that creates a Xamarin.Forms app that you can run on the real device, and this app will look for unit tests and run them:
https://github.com/nunit/nunit.xamarin
You have to run this manually as an app and look at the results. If you want to automate that part then you can create a Xamarin UITest automated test to launch your app, let the tests run then download the results.
https://developer.xamarin.com/guides/testcloud/uitest/
This is something we do using Jenkins as part of our standard mobile app pull requests. Jenkins runs a script that runs the UITest unit tests to launch the unit test app, wait for a result then grab the output and either pass or fail the UI test depending on the result of the screen output.
We just look for 'Test passed' or 'Test Failed', and if the test fails we manually run them to look for what went wrong, but you could expand on the UI test to download the results.
I am trying to run unit tests for Mac application in xcode 5.0 .
The unit tests run fine if I have the sandbox disabled but as soon as I enabled the sandbox the tests are unable to run, failing with error loading bundle (IDEBundleInjection.c: Error loading bundle).
I've seen a few other reports about this error and looks like the general assumption is that it won't work. The suggested workaround is to create a new configuration in xcode with sandboxing disabled and running the tests with that configuration.
This works, of course, but it seems like a poor workaround since it will be running the tests in a different environment than the real application normally would and potentially missing on a whole category of bugs resulting from different permissions, directories etc.
I can hardly believe that Apple intended developers to disable sandboxing for testing, seems very short sighted. Has anyone figured out a better solution for running unit tests for an application that is intended to run in sandbox mode?
I work on automated tests for DirectX based applications that run on various automated servers. The servers are setup to run as a service, which means when the tests execute they are executed under session 0.
However, the tests started failing after installing the windows platform update kb2670838. When initializing the application, a DXGI_ERROR_NOT_CURRENTLY_AVAILABLE error occurs after a call to CreateSwapChain.
I have found that some API calls do not work under session 0. (See http://msdn.microsoft.com/en-us/library/windows/desktop/hh404562(v=vs.85).aspx).
But last week this worked. I posted this in various Microsoft forums, but it seems no one else is having this problem. So my question is has anyone else experienced this? And how did you work around it?