We have a springboot service which has the capability to recover itself after database restart. But all of a sudden we noticed "recoverer is already running, abandoning this recovery request" in the logs and healthcheck of service is failed. We had to get the service restarted in both our datacenters.
Has anybody faced the similar issue?
==Edit
Below are the configurations:
spring.jta.log-dir=target/transaction-logs
spring.jta.bitronix.datasource.className=bitronix.tm.resource.jdbc.lrc.LrcXADataSource
spring.jta.bitronix.datasource.driverProperties.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.jta.bitronix.datasource.driverProperties.url=
spring.jta.bitronix.datasource.driverProperties.user=
spring.jta.bitronix.datasource.driverProperties.password=
spring.jta.bitronix.datasource.test-query=select 1
spring.jta.bitronix.datasource.max-pool-size=100
spring.jta.bitronix.datasource.prepared-statement-cache-size=100
Related
In workflow Monitor I tried to connect with Integration Service. But it is showing communicating with Integration service, upto 180 seconds. After that it was disconnected. Other components are connecting without any issues only monitor this issue is happening. Please advise on this and provide solutions for this issue.
Thanks
Viswa S
Unable To save identifier IN COUCHBASE, identifier: identifiertail::123::459, , Details: com.couchbase.client.core.error.AmbiguousTimeoutException:
UpsertRequest, Reason: TIMEOUT
"scope":"_default","type":"kv"},"timeoutMs":30000,"timings":{"encodingMicros":102,"totalMicros":36987410}}
at com.couchbase.client.core.msg.BaseRequest.cancel(BaseRequest.java:163)
at com.couchbase.client.core.Timer.lambda$register$2(Timer.java:157)
at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672)
at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747)
at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472)
at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Got above error while inserting records from app in logs. But when I checked the file, it was inserted and errors stopped coming after restart of Pods.
What could be reason that got fixed with the Pod restart?
Few things to add to this:
Couchbase cluster was healthy.
Network connectivity was good.
spring boot version: 4.1
Couchbase: 6.0.3
connect timeout: 60s
Thanks
Ritz
I have a IBM WebSphere 8.5.5.13 ND on Windows 2016 standard edition with JDK 1.7 enabled. I see that, node agent and server1 (application server) are getting stopped everyday but the deployment manager is still up and running (i.e. admin console can be accessed). So, I have to start nodeagent and the associated server manually everyday. Investigation done so far
Checked if the windows servers are getting restarted everyday? No they are not
Checked nodeagent start and stop server logs but there are no entries to see, some command was issued for stopping
Checked application server profile (server1) logs but nothing is there.
FYI, I don't have clustering done on WAS but it is planned for the future.
I don't no where else I can look for the reason the node agent and server1 is getting stopped everyday.
okay, this is what I found out. In my case I have
Dmgr01 - registered under windows service
Node agent - not registered under windows service
application server - no need or never register application server if you have deployment manager
Since my node agent was not registered under windows service, whenever I log-off or my session is killed due to in-activity, the default behavior is that, all running processes (jave.exe) associated with WebSphere will be crashed and there will not be any trace of it. This is why, I was unable to find the any logs.
I registered my node agent as windows service and everything worked.
I am testing a deployment of MQ 9.0.1 using Docker, and trying to administer this instance using the Web console. However I am receiving a warning when attaching widgets stating the "Queue manager unavailable".
Warning messages
In the console I can see my Queues are running and have created local queues and can run commands against them.
Also if I try to modify the Queue security records I get another error:
Failed to get authority recordsWMQWB0003E: The request could not be completed due to an internal web application server error.
Any thoughts or suggestions around this issue?
There is no definitive fix for MQ9.0.1 but upgrading to use MQ9.0.2 will resolve issues with hosting MQ Console on linux.
We have started to explore Spring Cloud Consul and we see that if the Consul agent is down the app crashes when it started... is it the expected behavior? we have expected that the app will wait to the agent to be up and retry at least several times or at least that this behavior will be configurable...
Also if the agent was up at start it successfully registers the service in the catalog but if at some point the agent went down even for several seconds the app will fail to talk the agent and will not retry to talk with the agent again... This cause a scenario that the app is up but it is no longer talking to the agent again we would expect a retry...
It is an open issue that we are tracking.