Xcodebuild: [MT] IDETestOperationsObserverDebug - xcode

I try to run my test on a iphonesimulator with Xcodebuild but I get always the same errors.
My command line:
xcodebuild clean test -workspace App.xcworkspace -scheme "UI Tests" -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 6,OS=9.2'
The stacktrace:
xcodebuild[31826:741511] [MT] IDETestOperationsObserverDebug: (796137A0-CB6A-490D-BF89-767DF586BE26) Beginning test session 796137A0-CB6A-490D-BF89-767DF586BE26 with Xcode 7C68 on target <DVTiPhoneSimulator: 0x7fef34dce060> {
SimDevice: SimDevice : iPhone 6 (ED6BBC90-CCD7-4AFE-ABBD-AFAE28B7EEC7) : state={ Booted } deviceType={ SimDeviceType : com.apple.CoreSimulator.SimDeviceType.iPhone-6 } runtime={ SimRuntime : 9.2 (13C75) - com.apple.CoreSimulator.SimRuntime.iOS-9-2 }
} (9.2 (13C75))
2016-02-22 13:07:10.489 XCTRunner[32908:746212] Running tests...
2016-02-22 13:07:10.490 XCTRunner[32908:746212] Looking for test bundles in /Users/me/Library/Developer/CoreSimulator/Devices/ED6BBC90-CCD7-4AFE-ABBD-AFAE28B7EEC7/data/Containers/Bundle/Application/2D77EC0A-85FA-4C8B-BC62-27334E6278C5/UITests-Runner.app/PlugIns
2016-02-22 13:07:10.491 XCTRunner[32908:746212] Found test bundle at /Users/me/Library/Developer/CoreSimulator/Devices/ED6BBC90-CCD7-4AFE-ABBD-AFAE28B7EEC7/data/Containers/Bundle/Application/2D77EC0A-85FA-4C8B-BC62-27334E6278C5/UITests-Runner.app/PlugIns/UITests.xctest
2016-02-22 13:07:10.492 XCTRunner[32908:746212] Looking for configurations in /Users/me/Library/Developer/CoreSimulator/Devices/ED6BBC90-CCD7-4AFE-ABBD-AFAE28B7EEC7/data/Containers/Bundle/Application/2D77EC0A-85FA-4C8B-BC62-27334E6278C5/UITests-Runner.app/PlugIns/UITests.xctest
2016-02-22 13:07:10.494 XCTRunner[32908:746212] Found configuration <XCTestConfiguration: 0x7fd750623340>
testBundleURL:file:///Users/me/Library/Developer/Xcode/DerivedData/App-axxxxxxxxxxx/Build/Products/Debug-iphonesimulator/UITests-Runner.app/PlugIns/UITests.xctest/
productModuleName:UITests
testsToSkip:UITests/test_App()
testsToRun:(null)
reportResultsToIDE:YES
sessionIdentifier:<__NSConcreteUUID 0x7fd750626d10> 796137A0-CB6A-490D-BF89-767DF586BE26
pathToXcodeReportingSocket:(null)
disablePerformanceMetrics:no
treatMissingBaselinesAsFailures:no
baselineFileURL:(null)
targetApplicationPath:/Users/me/Library/Developer/Xcode/DerivedData/App-axxxxxxxxxxxxxx/Build/Products/Debug-iphonesimulator/App.app
targetApplicationBundleID:de.xxxxx
reportActivities:YES
testsMustRunOnMainThread:YES
13:07:10.629 XCTRunner[32908:746223] _XCT_testBundleReadyWithProtocolVersion:minimumVersion: reply received
13:07:10.635 XCTRunner[32908:746221] _IDE_startExecutingTestPlanWithProtocolVersion:16
2016-02-22 13:07:10.992 XCTRunner[32908:746212] Continuing to run tests in the background with task ID 1
Test Suite 'Selected tests' started at 2016-02-22 13:07:11.557
Test Suite 'UITests.xctest' started at 2016-02-22 13:07:11.558
Test Suite 'UITests.xctest' passed at 2016-02-22 13:07:11.558.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
** TEST FAILED **
When I call the test direct on Xcode, I have any problems.
Any ideas?
Thank for helping.
Cheers.

Related

Finished requesting crash reports. Continuing with testing - Jenkins pipeline Xcode Test cases failing

Destination simulator is: platform=iOS Simulator,name=iPhone 6s Plus,OS=12.4
Excluded paths from coverage are: .*Tests.*
Creating directory sonar-reports/
Running tests
+ xcodebuild test -workspace Project.xcworkspace -scheme ProjectTests -configuration Debug -enableCodeCoverage YES -destination 'platform=iOS Simulator,name=iPhone 6s Plus,OS=12.4' -destination-timeout 60
2019-10-17 13:25:05.905 xcodebuild[5811:47314] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/XXUSERXX/Library/Developer/Xcode/DerivedData/Project-gcyoajxbeybhskdjmpfudvsghpth/Logs/Test/Test-ProjectTests-2019.10.17_13-20-39-+0530.xcresult/1_Test/Diagnostics/ProjectTests-3CAB9DB7-C238-45F1-984F-C3C061536B14/ProjectTests-674B5C61-8938-4243-BEE2-66691C2F4489/Session-ProjectTests-2019-10-17_132505-kBU2rX.log
2019-10-17 13:25:05.907 xcodebuild[5811:47306] [MT] IDETestOperationsObserverDebug: (340B3860-B575-4819-A2D6-C529266893C8) Beginning test session ProjectTests-340B3860-B575-4819-A2D6-C529266893C8 at 2019-10-17 13:25:05.906 with Xcode 10G8 on target <DVTiPhoneSimulator: 0x7f8b11877220> {
SimDevice: iPhone 6s Plus (CE03A4E0-9AD3-4BB4-8146-AE5C1288ABC3, iOS 12.4, Shutdown)
} (12.4 (16G73))
2019-10-17 13:25:28.196 xcodebuild[5811:47306] [MT] IDETestOperationsObserverDebug: (340B3860-B575-4819-A2D6-C529266893C8)
Finished requesting crash reports. Continuing with testing
I've been facing above error message since last week, I'm using Jenkins pipeline to build Xcode and then run the test cases for Sonar report with Xcode 12.3 MAC OS Catalina.
I even tried to run in different machines but getting different error messages, fastlane is not installed in the current system also not included in the script file.
After waiting about 30 - 40 minutes I've got the following screen which shows that the Test cases are failed.
2019-10-17 13:25:28.196 xcodebuild[5811:47306] [MT] IDETestOperationsObserverDebug: (340B3860-B575-4819-A2D6-C529266893C8) Finished requesting crash reports. Continuing with testing.
2019-10-17 13:43:38.360 xcodebuild[5811:47306] [MT] IDETestOperationsObserverDebug: 1112.476 elapsed -- Testing started completed.
2019-10-17 13:43:38.361 xcodebuild[5811:47306] [MT] IDETestOperationsObserverDebug: 0.001 sec, +0.001 sec -- start
2019-10-17 13:43:38.361 xcodebuild[5811:47306] [MT] IDETestOperationsObserverDebug: 1112.476 sec, +1112.474 sec -- end
Failing tests:
ProjectTests:
*******ModelTests.testCreate****Details()
*******ModelTests.testCreate****Details()
Test session results and logs:
/Users/XXUSERXX/Library/Developer/Xcode/DerivedData/Project-gcyoajxbeybhskdjmpfudvsghpth/Logs/Test/Test-ProjectTests-2019.10.17_13-20-39-+0530.xcresult
** TEST FAILED **
+ returnValue=65
+ set +x
ERROR - Command 'xcodebuild test -workspace Project.xcworkspace -scheme ProjectTests -configuration Debug -enableCodeCoverage YES -destination platform=iOS Simulator,name=iPhone 6s Plus,OS=12.4 -destination-timeout 60' failed with error code: 65
Test cases are working locally when ran in the current system, but failing when running via Jenkins pipeline for Sonar Testing.
Thanks

Xcpretty not working when running xcode ui tests in parallel

I've started to run my ui tests in parallel, in order to improve performance. However, when using xcpretty, I don't get now which tests have passed nor what made the tests fail (I only get which tests have failed). Is there any way to solve this? Or an alternative to xcpretty that works with the outputs in the parallel tests. I want to have a nice terminal output, as when using sequential testing.
This is my script:
xcodebuild \
-workspace './code/ios/myApp/myApp.workspace' \
-scheme 'myApp' \
-destination 'platform=iOS Simulator,name=iPhone 6' \
test | xcpretty -c
This is the output I got when running tests sequentially (And the output I want to continue having when running them in parallel):
Selected tests
[15:35:45]: ▸ Test Suite UITests.xctest started
[15:35:45]: ▸ RegisterTest
[15:36:48]: ▸ ✗ testRegisterBrazil, failed - Couldn't find:
"homeBottomBar_myAccountButton" Button
[15:42:50]: ▸ ✓ testRegisterUSA (61.241 seconds)
[15:42:50]: ▸ Executed 4 tests, with 1 failures (1 unexpected) in 425.314 (425.319) seconds
This is the output I get now:
Failing tests:
UITests:
RegisterTest.testRegisterBrazil()
** TEST FAILED **

Visual Studio Team Services - Run Functional Tests build task aborting after attempting test run

When I attempt to use the Run Functional Tests task after successfully executing the Visual Studio Test Agent Deployment task, the test run is aborting after a period of time with no actionable error reported.
2017-09-18T18:30:56.0416875Z ##[section]Starting: Run Functional Tests
2017-09-18T18:30:56.0436877Z ==============================================================================
2017-09-18T18:30:56.0436877Z Task : Run Functional Tests
2017-09-18T18:30:56.0436877Z Description : Run Coded UI/Selenium/Functional tests on a set of machines (using Test Agent)
2017-09-18T18:30:56.0446878Z Version : 1.0.53
2017-09-18T18:30:56.0446878Z Author : Microsoft Corporation
2017-09-18T18:30:56.0446878Z Help : [More Information] (https://go.microsoft.com/fwlink/?LinkId=624389)
2017-09-18T18:30:56.0446878Z ===========================================================================
2017-09-18T18:30:56.0486882Z Preparing task execution handler.
2017-09-18T18:30:56.2957129Z Executing the powershell script: C:\agent_work_tasks\RunVisualStudioTestsusingTestAgent_d353d6a2-e361-4a8f-8d8c-123bebb71028\1.0.53\RunDistributedTests.ps1
2017-09-18T18:30:58.2319065Z DistributedTests: Environment WinRm Protocol HTTP.
2017-09-18T18:30:58.2419075Z DistributedTests: Creating run for selected test assemblies with following parameters
2017-09-18T18:30:58.2419075Z DistributedTests: SourceFilter: ***test*.dll TestCaseFilter:
2017-09-18T18:30:58.2419075Z DistributedTests: Run title: TestRun_TEST - Agile-ASP.NET (PREVIEW)-CI_20170918.17
2017-09-18T18:30:58.2429076Z DistributedTests: is automated: True
2017-09-18T18:30:58.2429076Z DistributedTests: build location: E:\Websites\BuildTest\
2017-09-18T18:30:58.2429076Z DistributedTests: build id: 147
2017-09-18T18:30:58.2429076Z DistributedTests: test configuration mapping:
2017-09-18T18:30:59.1269960Z DistributedTests: Test Run with Id 4463 Queued
2017-09-18T18:30:59.3450178Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:31:09.6130445Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:31:19.8980729Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:31:30.1380968Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:31:40.4581287Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:31:50.6271455Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:32:00.8831710Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:32:11.0691895Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:32:21.3022127Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:32:31.6102434Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:32:41.8402663Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:32:52.4743296Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:33:02.7353556Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:33:13.0463866Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:33:23.4314250Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:33:33.6404458Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:33:44.0194836Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:33:54.6215437Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:34:04.9425757Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:34:15.1385952Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:34:26.2367049Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:34:36.1997011Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:34:46.4097220Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:34:56.6257435Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:35:06.9417750Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:35:17.7388546Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:35:27.4378244Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:35:37.6538459Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:35:47.8708675Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:35:58.1668970Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:36:08.3655493Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:36:18.5925493Z DistributedTests: Test run '4463' is in 'InProgress' state.
2017-09-18T18:36:28.8865493Z DistributedTests: Test run '4463' is in 'Aborted' state.
2017-09-18T18:36:38.8945493Z ##[warning]DistributedTests: Test run is aborted. Logging details of the run logs.
2017-09-18T18:36:39.1035493Z ##[warning]DistributedTests: New test run created.
2017-09-18T18:36:39.1045493Z Test Run queued for Project Collection Build Service
2017-09-18T18:36:39.1045493Z
2017-09-18T18:36:39.1055493Z ##[warning]DistributedTests: Test discovery started.
2017-09-18T18:36:39.1065493Z ##[warning]DistributedTests: Test Run Discovery Aborted . Test run id : 4463
2017-09-18T18:36:39.1085493Z ##[warning]DistributedTests: UnExpected error occurred during test execution. Try again.
2017-09-18T18:36:39.1095493Z ##[warning]DistributedTests: Error : NUnit VS Adapter 2.1.1.0 discovering test is finished
2017-09-18T18:36:39.1095493Z
2017-09-18T18:36:39.1105493Z ##[warning]DistributedTests: Test run aborted. Test run id: 4463
2017-09-18T18:36:39.1195493Z ##[error]System.Exception: The test run was aborted, failing the task.
2017-09-18T18:36:39.1715493Z ##[error]PowerShell script completed with 1 errors.
2017-09-18T18:36:39.1715493Z ##[section]Finishing: Run Functional Tests
These are the logs from deploying the Test Agent, where I can also see no issues, but I am including them for reference:
2017-09-18T18:24:59.2450114Z ##[section]Starting: Visual Studio Test Agent Deployment
2017-09-18T18:24:59.2470116Z ==============================================================================
2017-09-18T18:24:59.2470116Z Task : Visual Studio Test Agent Deployment
2017-09-18T18:24:59.2470116Z Description : Deploy and configure Test Agent to run tests on a set of machines
2017-09-18T18:24:59.2470116Z Version : 2.1.20
2017-09-18T18:24:59.2470116Z Author : Microsoft Corporation
2017-09-18T18:24:59.2470116Z Help : More Information
2017-09-18T18:24:59.2480117Z ==============================================================================
2017-09-18T18:25:01.7892658Z ****************************************************************
2017-09-18T18:25:01.7892658Z Task Input Information
2017-09-18T18:25:01.7902659Z ----------------------------------------------------------------
2017-09-18T18:25:01.7932662Z winRmProtocal = Http
2017-09-18T18:25:01.7932662Z testCertificate = true
2017-09-18T18:25:01.7952664Z runAsProcess = true
2017-09-18T18:25:01.7962665Z logonAutomatically = true
2017-09-18T18:25:01.7962665Z disableScreenSaver = true
2017-09-18T18:25:01.7972666Z isDataCollectionOnly = false
2017-09-18T18:25:01.7982667Z testPlatform = 15.0
2017-09-18T18:25:01.7992668Z agentLocation = \TestAgent\TestAgent.zip
2017-09-18T18:25:01.8002669Z updateTestAgent = false
2017-09-18T18:25:01.8072676Z ****************************************************************
2017-09-18T18:25:02.4813350Z Starting to deploy and configure test agent on test machines.
2017-09-18T18:25:02.4813350Z Inputs provided are:
2017-09-18T18:25:02.4813350Z Use HTTP Port: True
2017-09-18T18:25:02.4813350Z Disable Screen saver: True
2017-09-18T18:25:02.4823351Z Login Automatically: True
2017-09-18T18:25:02.4823351Z Run as process: True
2017-09-18T18:25:02.4823351Z Agent location: \TestAgent\TestAgent.zip
2017-09-18T18:25:02.4823351Z Update test agent: False
2017-09-18T18:25:02.4823351Z Run test agent as DataCollection only : False
2017-09-18T18:25:02.5973466Z Task 'VerifyTestAgentInstalled' on machine is being run
2017-09-18T18:25:07.8168685Z Deployment started on target machine...
2017-09-18T18:25:07.8168685Z
2017-09-18T18:25:07.8168685Z Query for Testplatfrom version: 15.0
2017-09-18T18:25:07.8168685Z
2017-09-18T18:25:07.8168685Z Testing container: 'REGISTRY::HKEY_CLASSES_ROOT\CLSID{177F0C4A-1CD3-4DE7-A32C-71DBBB9FA36D}'
2017-09-18T18:25:07.8168685Z
2017-09-18T18:25:07.8168685Z Exists.
2017-09-18T18:25:07.8178686Z
2017-09-18T18:25:07.8178686Z Getting Visual Studio setup instances.
2017-09-18T18:25:07.8178686Z
2017-09-18T18:25:07.8178686Z Found 1 instances.
2017-09-18T18:25:07.8178686Z
2017-09-18T18:25:07.8178686Z
2017-09-18T18:25:07.8178686Z
2017-09-18T18:25:07.8178686Z Description : Supports running automated tests and load tests remotely
2017-09-18T18:25:07.8188687Z DisplayName : Visual Studio Test Agent 2017
2017-09-18T18:25:07.8188687Z Id : c7ded16e
2017-09-18T18:25:07.8188687Z InstallDate : System.Runtime.InteropServices.ComTypes.FILETIME
2017-09-18T18:25:07.8188687Z Name : VisualStudio/15.3.4+26730.15
2017-09-18T18:25:07.8188687Z Path : C:\TestAgent2017
2017-09-18T18:25:07.8188687Z Version : 15.3.26730.15
2017-09-18T18:25:07.8198688Z VersionString : 15.3.26730.15
2017-09-18T18:25:07.8198688Z
2017-09-18T18:25:07.8198688Z
2017-09-18T18:25:07.8198688Z
2017-09-18T18:25:07.8198688Z
2017-09-18T18:25:07.8198688Z
2017-09-18T18:25:07.8198688Z Test Agent already exists
2017-09-18T18:25:07.8198688Z
2017-09-18T18:25:07.8198688Z Version: 15.3.26730.15
2017-09-18T18:25:07.8208689Z
2017-09-18T18:25:07.8208689Z DTAAgentExecutionService is not present on the machine
2017-09-18T18:25:07.8208689Z
2017-09-18T18:25:07.8208689Z DTAExecutionHost is not running
2017-09-18T18:25:07.8208689Z
2017-09-18T18:25:07.8208689Z
2017-09-18T18:25:07.8208689Z RemoteDeployerSource Verbose: 2 : [RemoteDeployer][18:Sep:17:15:25:5:1565; 7084, 4] Created public key for communication
2017-09-18T18:25:07.8218690Z RemoteDeployerSource Information: 12 : [RemoteDeployer][18:Sep:17:15:25:5:1565; 7084, 4] Deployment service - VisualStudioRemoteDeployerc612dfc5-c424-4fda-ae9a-cd2f906bce94 is running...
2017-09-18T18:25:07.8228691Z RemoteDeployerSource Verbose: 4 : [RemoteDeployer][18:Sep:17:15:25:6:1861; 7084, 6] Input file found
2017-09-18T18:25:07.8228691Z RemoteDeployerSource Verbose: 5 : [RemoteDeployer][18:Sep:17:15:25:6:1861; 7084, 6] Reading input parameters
2017-09-18T18:25:07.8238692Z RemoteDeployerSource Verbose: 6 : [RemoteDeployer][18:Sep:17:15:25:6:2017; 7084, 6] Finished reading input parameters
2017-09-18T18:25:07.8238692Z
2017-09-18T18:25:07.8238692Z Task 'VerifyTestAgentInstalled' on machine completed.
2017-09-18T18:25:08.8289697Z Task 'CopyFilesToTestAgent' on machine is being run
2017-09-18T18:25:09.7900658Z
2017-09-18T18:25:09.7900658Z \ was deleted successfully.
2017-09-18T18:25:09.7900658Z
2017-09-18T18:25:09.7900658Z Copying recursively from C:\agent_work_tasks\DeployVisualStudioTestAgent_52a38a6a-1517-41d7-96cc-73ee0c60d2b6\2.1.20\TestExecution.zip to C:\TestAgent on machine succeeded.
2017-09-18T18:25:09.7910659Z
2017-09-18T18:25:09.7910659Z \ was deleted successfully.
2017-09-18T18:25:09.7910659Z
2017-09-18T18:25:09.7910659Z
2017-09-18T18:25:09.7910659Z Task 'CopyFilesToTestAgent' on machine completed.
2017-09-18T18:25:09.7950663Z Task 'SetupTestMachineForUiTests' on machine is being run
2017-09-18T18:26:09.7980660Z Task 'SetupTestMachineForUiTests' on machine is taking time. Please Wait
2017-09-18T18:27:09.7860642Z Task 'SetupTestMachineForUiTests' on machine is taking time. Please Wait
2017-09-18T18:28:09.7890639Z Task 'SetupTestMachineForUiTests' on machine is taking time. Please Wait
2017-09-18T18:29:09.7920636Z Task 'SetupTestMachineForUiTests' on machine is taking time. Please Wait
2017-09-18T18:29:59.6180457Z Deployment started on target machine...
2017-09-18T18:29:59.6200459Z
2017-09-18T18:29:59.6200459Z Test User
2017-09-18T18:29:59.6200459Z
2017-09-18T18:29:59.6210460Z Test UserDomain
2017-09-18T18:29:59.6230462Z
2017-09-18T18:29:59.6240463Z Executing command : powercfg.exe /Change monitor-timeout-ac 0 ; powercfg.exe /Change monitor-timeout-dc 0
2017-09-18T18:29:59.6260465Z
2017-09-18T18:29:59.6260465Z Error :
2017-09-18T18:29:59.6270466Z
2017-09-18T18:29:59.6280467Z Output :
2017-09-18T18:29:59.6280467Z
2017-09-18T18:29:59.6280467Z Updating machine reboot count to : 1
2017-09-18T18:29:59.6280467Z
2017-09-18T18:29:59.6290468Z Currently test user is not logged in. Rebooting machine.
2017-09-18T18:29:59.6290468Z
2017-09-18T18:29:59.6290468Z Enabling auto logon
2017-09-18T18:29:59.6290468Z
2017-09-18T18:29:59.6290468Z
2017-09-18T18:29:59.6310470Z Deployment started on target machine...
2017-09-18T18:29:59.6310470Z
2017-09-18T18:29:59.6310470Z Test User
2017-09-18T18:29:59.6310470Z
2017-09-18T18:29:59.6310470Z Test UserDomain
2017-09-18T18:29:59.6320471Z
2017-09-18T18:29:59.6320471Z Executing command : powercfg.exe /Change monitor-timeout-ac 0 ; powercfg.exe /Change monitor-timeout-dc 0
2017-09-18T18:29:59.6320471Z
2017-09-18T18:29:59.6320471Z Error :
2017-09-18T18:29:59.6330472Z
2017-09-18T18:29:59.6330472Z Output :
2017-09-18T18:29:59.6330472Z
2017-09-18T18:29:59.6330472Z Configuration for UI testing is completed
2017-09-18T18:29:59.6330472Z
2017-09-18T18:29:59.6330472Z
2017-09-18T18:29:59.6340473Z 5:11:9740; 428, 4] Created public key for communication
2017-09-18T18:29:59.6340473Z RemoteDeployerSource Information: 12 : [RemoteDeployer][18:Sep:17:15:25:11:9896; 428, 4] Deployment service - VisualStudioRemoteDeployerd61e768d-1f6c-4de3-912b-ba961a2bb337 is running...
2017-09-18T18:29:59.6350474Z RemoteDeployerSource Verbose: 4 : [RemoteDeployer][18:Sep:17:15:25:13:0036; 428, 6] Input file found
2017-09-18T18:29:59.6360475Z RemoteDeployerSource Verbose: 5 : [RemoteDeployer][18:Sep:17:15:25:13:0036; 428, 6] Reading input parameters
2017-09-18T18:29:59.6360475Z RemoteDeployerSource Verbose: 6 : [RemoteDeployer][18:Sep:17:15:25:13:0192; 428, 6] Finished reading input parameters
2017-09-18T18:29:59.6370476Z
2017-09-18T18:29:59.6370476Z Task 'SetupTestMachineForUiTests' on machine completed.
2017-09-18T18:29:59.6390478Z Task 'TestAgentConfiguration' on machine is being run
2017-09-18T18:30:44.9905825Z Deployment started on target machine...
2017-09-18T18:30:44.9905825Z
2017-09-18T18:30:44.9915826Z Test User
2017-09-18T18:30:44.9915826Z
2017-09-18T18:30:44.9915826Z Test UserDomain
2017-09-18T18:30:44.9915826Z
2017-09-18T18:30:44.9925827Z Testing container: 'REGISTRY::HKEY_CLASSES_ROOT\CLSID{177F0C4A-1CD3-4DE7-A32C-71DBBB9FA36D}'
2017-09-18T18:30:44.9925827Z
2017-09-18T18:30:44.9925827Z Exists.
2017-09-18T18:30:44.9935828Z
2017-09-18T18:30:44.9935828Z Getting Visual Studio setup instances.
2017-09-18T18:30:44.9935828Z
2017-09-18T18:30:44.9935828Z Found 1 instances.
2017-09-18T18:30:44.9945829Z
2017-09-18T18:30:44.9945829Z
2017-09-18T18:30:44.9945829Z
2017-09-18T18:30:44.9945829Z Description : Supports running automated tests and load tests remotely
2017-09-18T18:30:44.9955830Z DisplayName : Visual Studio Test Agent 2017
2017-09-18T18:30:44.9955830Z Id : c7ded16e
2017-09-18T18:30:44.9965831Z InstallDate : System.Runtime.InteropServices.ComTypes.FILETIME
2017-09-18T18:30:44.9965831Z Name : VisualStudio/15.3.4+26730.15
2017-09-18T18:30:44.9975832Z Path : C:\TestAgent2017
2017-09-18T18:30:44.9975832Z Version : 15.3.26730.15
2017-09-18T18:30:44.9985833Z VersionString : 15.3.26730.15
2017-09-18T18:30:44.9985833Z
2017-09-18T18:30:44.9985833Z
2017-09-18T18:30:44.9985833Z
2017-09-18T18:30:44.9985833Z
2017-09-18T18:30:44.9985833Z
2017-09-18T18:30:44.9995834Z VS path C:\TestAgent2017\Common7\IDE; Test window path C:\TestAgent2017\Common7\IDE\CommonExtensions\Microsoft\TestWindow
2017-09-18T18:30:44.9995834Z
2017-09-18T18:30:45.0025837Z Path: C:\TestAgent\TfsAssemblies\Newtonsoft.Json.dll
2017-09-18T18:30:45.0025837Z
2017-09-18T18:30:45.0035838Z Loading module from path 'C:\TestAgent\TfsAssemblies\Newtonsoft.Json.dll'.
2017-09-18T18:30:45.0035838Z
2017-09-18T18:30:45.0045839Z Loading module from path 'C:\TestAgent\TfsAssemblies\System.Net.Http.Formatting.dll'.
2017-09-18T18:30:45.0045839Z
2017-09-18T18:30:45.0075842Z Loading module from path 'C:\TestAgent\TfsAssemblies\Microsoft.TeamFoundation.DistributedTask.Task.TestExecution.dll'.
2017-09-18T18:30:45.0085843Z
2017-09-18T18:30:45.0095844Z Loading module from path 'C:\TestAgent\TfsAssemblies\Microsoft.TeamFoundation.Common.dll'.
2017-09-18T18:30:45.0105845Z
2017-09-18T18:30:45.0105845Z Loading module from path 'C:\TestAgent\TfsAssemblies\Microsoft.TeamFoundation.Test.WebApi.dll'.
2017-09-18T18:30:45.0115846Z
2017-09-18T18:30:45.0135848Z Loading module from path 'C:\TestAgent\TfsAssemblies\Microsoft.VisualStudio.Services.Common.dll'.
2017-09-18T18:30:45.0135848Z
2017-09-18T18:30:45.0145849Z Loading module from path 'C:\TestAgent\TfsAssemblies\Microsoft.VisualStudio.Services.WebApi.dll'.
2017-09-18T18:30:45.0145849Z
2017-09-18T18:30:45.0155850Z Loading module from path 'C:\TestAgent\PrivateAssemblies\MS.VS.TestService.Client.Utility.dll'.
2017-09-18T18:30:45.0165851Z
2017-09-18T18:30:45.0165851Z ****************************************************************
2017-09-18T18:30:45.0165851Z
2017-09-18T18:30:45.0165851Z Configure Test Agent
2017-09-18T18:30:45.0165851Z
2017-09-18T18:30:45.0175852Z ----------------------------------------------------------------
2017-09-18T18:30:45.0175852Z
2017-09-18T18:30:45.0175852Z SetupPath : (C:\TestAgent)
2017-09-18T18:30:45.0175852Z
2017-09-18T18:30:45.0175852Z
2017-09-18T18:30:45.0185853Z AsServiceOrProcess : (Process)
2017-09-18T18:30:45.0185853Z
2017-09-18T18:30:45.0185853Z EnvironmentUrl : (dta://env/39a888b4-a200-42b4-8866-eaedb1da410a/_apis//201709181525023083)
2017-09-18T18:30:45.0185853Z
2017-09-18T18:30:45.0185853Z
2017-09-18T18:30:45.0185853Z Capabilities : ()
2017-09-18T18:30:45.0185853Z
2017-09-18T18:30:45.0195854Z TestAgentVersion : (15.0)
2017-09-18T18:30:45.0195854Z
2017-09-18T18:30:45.0195854Z ****************************************************************
2017-09-18T18:30:45.0195854Z
2017-09-18T18:30:45.0195854Z Register the Agent with Id: 14
2017-09-18T18:30:45.0195854Z
2017-09-18T18:30:45.0195854Z Starting scheduled task on Windows 7.
2017-09-18T18:30:45.0195854Z
2017-09-18T18:30:45.0195854Z
2017-09-18T18:30:45.0205855Z RemoteDeployerSource Verbose: 2 : [RemoteDeployer][18:Sep:17:15:30:3:6202; 3168, 4] Created public key for communication
2017-09-18T18:30:45.0205855Z RemoteDeployerSource Information: 12 : [RemoteDeployer][18:Sep:17:15:30:3:6202; 3168, 4] Deployment service - VisualStudioRemoteDeployerc5e37723-02f4-42db-93b4-2eead65fe9cd is running...
2017-09-18T18:30:45.0205855Z RemoteDeployerSource Verbose: 4 : [RemoteDeployer][18:Sep:17:15:30:4:6489; 3168, 6] Input file found
2017-09-18T18:30:45.0215856Z RemoteDeployerSource Verbose: 5 : [RemoteDeployer][18:Sep:17:15:30:4:6489; 3168, 6] Reading input parameters
2017-09-18T18:30:45.0215856Z RemoteDeployerSource Verbose: 6 : [RemoteDeployer][18:Sep:17:15:30:4:6801; 3168, 6] Finished reading input parameters
2017-09-18T18:30:45.0215856Z
2017-09-18T18:30:45.0215856Z Task 'TestAgentConfiguration' on machine completed.
2017-09-18T18:30:45.0775912Z ##[section]Finishing: Visual Studio Test Agent Deployment
Notes: The tests I am attempting to run are NUnit tests, and I have installed the NUnit package and the NUnit 3 Test Adapter package to the project that is being built
Based on the log, the reason is “Test run directory 'E:\Websites\BuildTest\' does not contain any assemblies that match the specified criteria: 'NewTestingApp\UnitTestProject1\bin\Debug\UnitTestProject1.dll'
Based on windows machine copy file, there isn’t NewTestingApp folder, so log on to test machine and check the actual path of test assembly.
As a workaround to this issue I have used the Batch Script task in VSTS to execute a batch script that executes NUnit Console (available here). The batch script to achieve this simply looks like:
"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" {Path to your project}\YourProjectName.csproj
In order to track the results of testing in VSTS, I simply added the Publish Test Results task and pointed it to the TestResults.xml file generated by NUnit Console.
Although different than the Run Functional Tests task that uses a test agent, this method fulfills all of my testing needs (with fewer headaches).

How do you configure the XCode Jenkins plugin to run tests?

The instructions for running tests with the Jenkins XCode plugin say to set the test target (which I've done), the SDK (which I've done) and the configuration (which I tried with nothing, Debug, and Test).
However I keep getting "...is not configured for running".
How do I actually get it to run tests?
This is the output:
+ xcodebuild -workspace /Users/MyDir/.Jenkins/jobs/MyTests/workspace/folder/MyWorkspace.xcworkspace -scheme MyTestScheme clean
xcodebuild: error: Failed to build workspace MyWorkspace with scheme MyTestScheme.
Reason: Scheme "MyTestScheme" is not configured for running.
If within XCode for MyTestScheme if I choose Product/Run then I get the same error message, but if I choose Product/Test then the test code executes successfully. The output from a sucessful run in Xcode is:
2013-08-28 11:10:25.828 otest[65917:303] Unknown Device Type. Using UIUserInterfaceIdiomPhone based on screen size
Test Suite 'Multiple Selected Tests' started at 2013-08-28 18:10:26 +0000
Test Suite '/Users/MyDir/Library/Developer/Xcode/DerivedData/MyWorkspace-ctngidolzdhijvbymvghygtoaiiw/Build/Products/Debug-iphonesimulator/MyTestScheme.octest(Tests)' started at 2013-08-28 18:10:26 +0000
Test Suite 'MyTests' started at 2013-08-28 18:10:26 +0000
Test Case '-[MyTests test1]' started.
2013-08-28 11:10:26.029 otest[65917:303] MDN: (null)
Test Case '-[MyTests testA1]' passed (0.346 seconds).
For me the problem was using the test schema. The debug schema should be used instead.
To configure the Xcode plugin for unit testing you need to write "test" in the "Custom xcodebuild arguments" field inside the "Advanced Xcode build options".
XCode plugin maintainer here. I don't know the answer, but I would love to help you out.
Have you tried fiddling with the destination argument ?
E.g. -destination 'OS=8.0,name=iPhone'
or -destination 'platform=iOS Simulator,OS=8.0,name=iPhone 6s'
(adjust depending on your needs)
If that doesn't work, please try to copy the output generated by the run of the tests from XCode itself.

Jenkins running xcode unit tests not working

I'm trying to configure CI with Jenkins for Xcode. I'm getting the following error in the console. The tests are running and I get a test output with JUnit. However it still says that it cannot runscripts.
Test Suite 'All tests' started at 2013-05-29 23:44:13 +0000
Test Suite '/Users/originate/.jenkins/jobs/Test/workspace/build/Debug-iphonesimulator/HelloWorldTests.octest(Tests)' started at 2013-05-29 23:44:13 +0000
Test Suite 'HelloWorldTests' started at 2013-05-29 23:44:13 +0000
Test Case '-[HelloWorldTests testExample]' started.
Test Case '-[HelloWorldTests testExample]' passed (0.000 seconds).
Test Suite 'HelloWorldTests' finished at 2013-05-29 23:44:13 +0000.
Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite '/Users/originate/.jenkins/jobs/Test/workspace/build/Debug-iphonesimulator/HelloWorldTests.octest(Tests)' finished at 2013-05-29 23:44:13 +0000.
Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'All tests' finished at 2013-05-29 23:44:13 +0000.
Executed 1 test, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
Command /bin/sh failed with exit code 1
** BUILD FAILED **
The following build commands failed:
PhaseScriptExecution "Run Script" build/HelloWorld.build/Debug-iphonesimulator/HelloWorldTests.build/Script-09E70126174EF7C800DFCE82.sh
(1 failure)
Build step 'Xcode' marked build as failure
Recording test results
Build step 'Upload to Testflight' marked build as failure
Finished: FAILURE
EDIT
The following error is given from running unit tests on command line in addition to the above error:
Session could not be started: Error Domain=DTiPhoneSimulatorErrorDomain Code=2 "Simulator session timed out." UserInfo=0x7fb9d35003a0 {NSLocalizedDescription=Simulator session timed out.}
The script for the unit tests that I am running is:
#!/usr/bin/env ruby
if ENV['SL_RUN_UNIT_TESTS'] then
launcher_path = "ios-sim"
test_bundle_path= File.join(ENV['BUILT_PRODUCTS_DIR'], "#{ENV['PRODUCT_NAME']}.#{ENV['WRAPPER_EXTENSION']}")
environment = {
'DYLD_INSERT_LIBRARIES' => "/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection",
'XCInjectBundle' => test_bundle_path,
'XCInjectBundleInto' => ENV["TEST_HOST"]
}
environment_args = environment.collect { |key, value| "--setenv #{key}=\"#{value}\""}.join(" ")
app_test_host = File.dirname(ENV["TEST_HOST"])
passed = system("#{launcher_path} launch \"#{app_test_host}\" #{environment_args} --args -SenTest All #{test_bundle_path}")
exit(1) if !passed
exit 0
else
puts "SL_RUN_UNIT_TESTS not set - Did not run unit tests!"
exit 0
end
Please help me out.
Thanks!

Resources