I am running a TFS nightly build that for the last few days has not been able to complete all its tests. It fails after several hours with a "Test run is aborted" message. Previous to this the tests always ran successfully, and no major changes(or even minor) have been made to the system that runs these tests.
Information:
Two MStest runs in the build(unit tests)
Timeout is set to 20 hours
Runs for approx. 15 hours before failure
Tests are set to continue on failure
When I look in the TFS log for the latest run it lists the following(2017-04-11T06:42:47.5500707Z):
[warning]DistributedTests: Test run is aborted. Logging details of the run logs.
[warning]DistributedTests: New test run created.
[warning]Test Run queued for Project Collection Build Service
[warning]DistributedTests: Test discovery started.
[warning]DistributedTests: Test Run Discovery Completed . Test run id: 533
[warning]DistributedTests: 290 test cases discovered.
[warning]DistributedTests: Test execution started. Test run id : 533
[warning]DistributedTests: Test run timed out. Test run id : 533
[warning]DistributedTests: Test run aborted. Test run id: 533
[error]The test run was aborted, failing the task.
When I look at the run log(worker_20170410-234426-utc_864.log) I see:
06:42:47.659516 BaseLogger.LogConsoleMessage(scope.JobId =
7ced7f31-e360-47f3-b334-ef20faeaf000, message = ##[error]The test run
was aborted, failing the task.) 06:42:47.659516
Microsoft.TeamFoundation.DistributedTask.Agent.Common.AgentExecutionTerminationException:
PowerShell script completed with errors. at
Microsoft.TeamFoundation.DistributedTask.Handlers.PowerShellHandler.Execute(ITaskContext
context, CancellationToken cancellationToken) at
Microsoft.TeamFoundation.DistributedTask.Worker.JobRunner.RunTask(ITaskContext
context, TaskWrapper task, CancellationTokenSource tokenSource)
In the test log, I don't see any errors in the VS, just a warning about not able to connect(I see these often):
W, 2060, 5, 2017/04/10, 16:26:03.595, XXXTESTING\QTController.exe,
Test of LoadTestResultConnectString failed: A network-related or
instance-specific error occurred while establishing a connection to
SQL Server. The server was not found or was not accessible. Verify
that the instance name is correct and that SQL Server is configured to
allow remote connections. (provider: SQL Network Interfaces, error: 26
- Error Locating Server/Instance Specified)
I also see an error thrown in the Application Event log at the same time:
The description for Event ID 0 from source Application cannot be
found. Either the component that raises this event is not installed on
your local computer or the installation is corrupted. You can install
or repair the component on the local computer.
If the event originated on another computer, the display information
had to be saved with the event.
The following information was included with the event:
Error Handler Exception: System.ServiceModel.CommunicationException:
There was an error reading from the pipe: The pipe has been ended.
(109, 0x6d). ---> System.IO.IOException: The read operation failed,
see inner exception. ---> System.ServiceModel.CommunicationException:
There was an error reading from the pipe: The pipe has been ended.
(109, 0x6d). ---> System.IO.PipeException: There was an error reading
from the pipe: The pipe has been ended. (109, 0x6d).....
the message resource is present but the message is not found in the
string/message table
The issue is that I really don't know how to interpret these messages, each log just says "test run was aborted, failing the task", I'm not even certain the powershell issue is what caused it. I'm also not sure that the error thrown in the application log is related, though it was thrown at exactly the same time that the run failed.
It's also difficult to research this issue, when you really don't know what's causing the test agent to fail. There are posts related to VS, and to the TFS Test Agent, but these don't strike me as related issues, and of course there is this somewhat unhelpful post about the Powershell message.
Has anyone seen this sort of issue before? I don't think anything on my build server has changed over the last few days(maybe updates...), what do you think would cause an issue like this to occur?
If you look at the failed build(containing tests) after it is aborted in the "Build" section of TFS, its says it was "Aborted", that's it... If you look at results of the build(containing tests) in the "Test" section of TFS it specified that the test run "Exceeded Timeout".
Apparently MSTest was running up against the default value of this little gem. I think it defaults to 8 hours when not specified, but I'm not too sure about this. Anyways I set the following setting in my "Default.testsettings" file:
<?xml version="1.0" encoding="utf-8"?>
<TestSettings name="TestSettings1">
<Execution>
<Timeouts runTimeout="200000000" />
</Execution>
</TestSettings>
Seems to resolve the issue. Tests runs successfully and no longer time out.
Related
When i run msdeploy from teamcity i get ERROR_CONNECTION_TERMINATED. But if i run it from cmd on the same server (where teamcity is running) it goes through. ERROR_CONNECTION_TERMINATED occours only if site is up to date, if it has to do alot of stuff then there is no error.
I'm not running fiddler that could be causing this.
Here is command:
msdeploy -verb:sync -source:contentPath="%teamcity.build.checkoutDir%\publish\web" -dest:contentPath="%DeployPath%\web",ComputerName="%MSDeploy.ComputerName%",Username="%MSDeploy.Username%",Password="%MSDeploy.Password%",AuthType="Basic" -useCheckSum -skip:Directory="\\web\\uploads$" -skip:File="\\app_offline.htm$" -allowUntrusted
Output:
[11:32:15][Step 9/9] PowerShell Executable:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
[11:32:15][Step 9/9] Working directory:
C:\TeamCity\buildAgent\work\81d93b891723185
[11:32:15][Step 9/9] Command:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
[11:32:15][Step 9/9] PowerShell arguments: -NonInteractive,
-ExecutionPolicy, ByPass, -File, C:\TeamCity\buildAgent\temp\buildTmp\powershell4517896340075893898.ps1
[11:32:16][Step 9/9] Info: Using ID
'a6102aa0-ae6b-43b5-9f15-5a6c0977232f' for connections to the remote
server.
[11:32:17][Step 9/9] Info: Object filePath
(C:\inetpub\dev.proplatform\web\app_offline.htm) skipped due to skip
directive 'CommandLineSkipDirective 2'.
[11:32:27][Step 9/9] Info: Object filePath
(C:\inetpub\dev.proplatform\web\ePeroWs\app_offline.htm) skipped due
to skip directive 'CommandLineSkipDirective 2'.
[11:32:27][Step 9/9] Info: Object filePath
(C:\inetpub\dev.proplatform\web\ipsWebApi\app_offline.htm) skipped due
to skip directive 'CommandLineSkipDirective 2'.
[11:32:37][Step 9/9] Info: Object dirPath
(C:\inetpub\dev.proplatform\web\uploads) skipped due to skip directive
'CommandLineSkipDirective 1'.
[11:32:37][Step 9/9] Info: Using ID
'98d4de3d-2ed9-4a43-b156-ebf69bac99dc' for connections to the remote
server.
[11:33:13][Step 9/9] Error Code: ERROR_CONNECTION_TERMINATED
[11:33:13][Step 9/9] More Information: Web Deploy experienced a
connection problem with the server and had to terminate the
connection. Contact your server administrator if the problem
persists. Learn more at:
https://go.microsoft.com/fwlink/?LinkId=221672#ERROR_CONNECTION_TERMINATED.
[11:33:13][Step 9/9] Error: Unexpected end of file has occurred. The
following elements are not closed: results. Line 1, position 550.
[11:33:13][Step 9/9] Error count: 1.
I tried running command in cmd and powershell on teamcity. I got error on TC, but if i run command directlly in cmd or powershell on the same server i got no error.
I found what the problem was. It was Windows real time protection which was cousing this problem. So i added process msdeploy.exe as an exception and now it works fine.
I hope this helps somebody in the future. I spent days trying to figure it out what the problem was.
I have a blocking issue installing openwhisk with docker
I typed make quick-start right after a git pull of the project incubator-openwhisk-devtools. My OS is Fedora 29, docker version is 18.09.0, docker-compose version is 1.22.0. JDk 8 Oracle.
I get the following error:
[...]
adding the function to whisk ...
ok: created action hello
invoking the function ...
error: Unable to invoke action 'hello': The server is currently unavailable (because it is overloaded or down for maintenance). (code ciOZDS8VySDyVuETF14n8QqB9wifUboT)
[...]
[ERROR] [#tid_sid_unknown] [Invoker] failed to ping the controller: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for health-0: 30069 ms has passed since batch creation plus linger time
[ERROR] [#tid_sid_unknown] [KafkaProducerConnector] sending message on topic 'health' failed: Expiring 1 record(s) for health-0: 30009 ms has passed since batch creation plus linger time
Please note that controller-local-logs.log is never created.
If I issue a touch controller-local-logs.log in the right directory the log file is always empty after I try to issue make quick-start again.
http://localhost:8888/ping gives me the right answer: pong.
http://localhost:9222 is not reacheable.
Where am I wrong?
Thank you in advance
I am using MSBuild. I have Java 8 installed.
I am running the following commands:
SonarQube.Scanner.MSBuild.exe begin /k:"ABC" /d:sonar.host.url="http://localhost:9000" /d:sonar.login="8b839xxxxxxxxxxxxxxxxxxxxxxx6b00125bf92" /d:sonar.verbose=true
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\msbuild.exe" /t:rebuild
SonarQube.Scanner.MSBuild.exe end /d:sonar.login="8b839xxxxxxxxxxxxxxxxxxxxxxx6b00125bf92"
The last step fails:
ERROR: Error during SonarQube Scanner execution
ERROR: The 'report' parameter is missing
ERROR:
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.
The SonarQube Scanner did not complete successfully
12:53:21.909 Creating a summary markdown file...
12:53:21.918 Post-processing failed. Exit code: 1
The MSBuild version is greater than 14.
Java 8 is properly installed. Documentation indicates that Java 8 is adequate.
Any idea on what could be wrong?
Where do I add the -X switch? I tried on all 3 statements
Update :I installed Java SDK 9. Still same issue.
Update :With verbose logging and using /n naming parameter:
INFO: Analysis report generated in 992ms, dir size=4 MB
INFO: Analysis reports compressed in 549ms, zip size=1 MB
INFO: Analysis report generated in C:\ABC\.sonarqube\out\.sonar\scanner-report
DEBUG: Upload report
DEBUG: POST 400 http://localhost:9000/api/ce/submit?projectKey=ABC | time=1023ms
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 54.833s
INFO: Final Memory: 51M/170M
INFO: ------------------------------------------------------------------------
DEBUG: Execution getVersion
DEBUG: Execution stop
ERROR: Error during SonarQube Scanner execution
ERROR: The 'report' parameter is missing
ERROR:
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.
Process returned exit code 1
The SonarQube Scanner did not complete successfully
Creating a summary markdown file...
Post-processing failed. Exit code: 1
I've struggled the same problem with SonarQube and I've finally found a solution:
You need to restart sonar service after using evaluation token.
Please note this isn't the answer, however I feel this feedback is valuable to getting this question answered.
I can reproduce this issue in POSTMan with a POST request to:
http://localhost:9000/api/ce/submit?projectKey=myProjectKey
This returns
{
"errors": [
{
"msg": "The 'report' parameter is missing"
}
]
}
You can get a similar error by removing the projectKey query parameter. I tried adding a report query parameter and received the same error:
http://localhost:9000/api/ce/submit?projectKey=brian3016&report=report
Given this, I feel there is a problem with their code. It should have included a report parameter when creating the POST request, but it failed to do so.
Verbose output seems to have changed from using the -X switch to /d:sonar.verbose=true. E.G.
SonarScanner.MSBuild.exe begin /k:"myProjectKey" /d:sonar.host.url="http://localhost:9000" /d:sonar.login="myLogin" /d:sonar.verbose=true
Note the verbose logging didn't give me any valuable insight.
(Also note that the documentation currently says to use SonarQube.Scanner.MSBuild.exe, but the verbose logger told me to switch to SonarScanner.MSBuild.exe)
SO...how we we report this issue to someone that can fix it? Their documentation says to go to Stackoverflow. So here we are.
I thought it may have been an issue with a project. So I created a new project with nothing other than the startup template Console Application. Same error.
In my case SonarQube 7.9.1 (deployed with Helm to Kubernetes cluster) was missing temp directory /opt/sonarqube/temp/tc/work/Tomcat/localhost/ROOT after Helm rollback. No idea what happened to it.
Logfile /opt/sonarqube/logs/web.log inside SonarQube pod had this error:
2021.02.02 06:57:03 WARN web[AXdZ6l6MParQCncJACv3][o.s.s.w.ServletRequest] Can't read file part for parameter report
java.io.IOException: The temporary upload location [/opt/sonarqube/temp/tc/work/Tomcat/localhost/ROOT] is not valid
The fix was to exec into pod and create the missing directory. Would like to know the reason though...
The issue is with the sonar service starting up.
First try to stop the SonarStart.bat by using Ctrl+c, and then try to open localhost:9000 ( or whichever port you configured sonar server).
If it is still opening then go to task manager and search for wrapper.exe service and stop the service. If no service is found then go to:
Task manager>Details> and stop all java.exe process.
Note: If you running many Java applications, right-click the java.exe and choose goto service, and stop only those java.exe that belongs to AppX deployment.services
Now start sonarstart.bat as administrator..
today i face the same error when using jenkins to scanner the code.
get the error when POST /api/ce/submit and get 400 code by add the sonar.verbose=true
i use the below step to check reason
first to restart the sonarqube => failed
check the report file size by using "du -sh" get 108m and DB server support 1G => failed
login the sonar-qube server and check the access.log, web.log and another log, finally find the error reason " Processing of multipart/form-data request failed. No space left on device", so i check the server by command "df -h", some devices are used 100% => so i remove some no-using file and fix it!!!
check if you have enough memory
ex: free -m
In my case I had to upgrade memory.
We have set up continuous environment at xcode9 at dedicated mac.Xcode-server is set up on that machine and bot has been created to build project and execute tests.
Build required cocoapods , hence pre-integration script is also used.
Now, when bot starts UI tests, the app installs at simulator , but doesn't launch and eventually consoles log following error :
Test Target 'xx' encountered an error (Early unexpected exit , operation never finished bootsrapping - no restart will be attempted)
On investigation of 'Raw build logs' we found Main Thread Checker error shows up with firebase warning.
Main Thread Checker: UI API called on a background thread: -[UIApplication applicationState] PID: 34330, TID: 355611, Thread name: (none), Queue name: NSOperationQueue 0x600000239dc0 (QOS: UNSPECIFIED), QoS: 0 Backtrace:
Had mitigated this issue by unchecking Edit Scheme->Test/Debug->Main Thread Checker under diagnostic section , while running tests locally.
Does same error showing up at continuous environment means that we have to pass the unchecking of Main Thread Checker as an argument ? Or is there something we are missing ?
Kindly help.
I'm trying to build a simple cluster based on Windows XP. I compiled OpenMPI-1.4.2 successfully, and tools like mpicc and ompi_info work too, but I can't get my mpirun working properly. The only output I can see is
Z:\>orterun --hostfile z:\hosts.txt -np 2 hostname
[host0:04728] Failed to initialize COM library. Error code = -2147417850
[host0:04728] [[8946,0],0] ORTE_ERROR_LOG: Error in file ..\..\openmpi-1.4.2
\orte\mca\ess\hnp\ess_hnp_module.c at line 218
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
orte_plm_init failed
--> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
[host0:04728] [[8946,0],0] ORTE_ERROR_LOG: Error in file ..\..\openmpi-1.4.2
\orte\runtime\orte_init.c at line 132
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
orte_ess_set_name failed
--> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
[host0:04728] [[8946,0],0] ORTE_ERROR_LOG: Error in file ..\..\..\..\openmpi
-1.4.2\orte\tools\orterun\orterun.c at line 543
Where z:\hosts.txt appears as follows:
host0
host1
Z: is a shared network drive available to both host0 and host1.
What my problem is and how do I fix it?
Upd:
Ok, this problem seems to be fixed. It seems to me that WideCap driver and/or software components causes this error to appear. A "clean" machine runs local task successfully. Anyway, I still cannot run a task within at least 2 machines, I'm getting following message:
Z:\>mpirun --hostfile z:\hosts.txt -np 2 hostname
connecting to host1
username:MAIN\cluster
password:********
Save Credential?(Y/N) y
[host0:04728] This feature hasn't been implemented yet.
[host0:04728] Could not connect to namespace cimv2 on node host1. Error code =-2147217400
--------------------------------------------------------------------------
mpirun was unable to start the specified application as it encountered an error.
More information may be available above.
--------------------------------------------------------------------------
I googled a little and did all the things as described here: http://www.open-mpi.org/community/lists/users/2010/03/12355.php but I'm still getting the same error. Can anyone help me?
Upd2:
Error code -2147217400 might be WMI error WBEM_E_INVALID_PARAMETER (0x80041008) which occures when one of the parameters passed to the WMI call is not correct. Does this mean that the problem is in OpenMPI source code itself? Or maybe it's because of wrong/outdated wincred.h and credui.lib I used while building OpenMPI from the source code?