Spring Boot Micro Service Not Defined in Registry When JMS Server Not Reachable - spring-boot

Question

I have a strange issue that took me several days to narrow down. Basically, I have a Jhipster project based on Spring boot Version 2.1.10.RELEASE, which contains 4 microservices. We are interested here in 2 of them: Gateway and Corehub.
In the gateway, I have an angular app that performs a POST to /services/corehub/api/someendpoint which used to be working and that is failing now with different error messages, but the one I have more regularly is
{
"type": "https://www.jhipster.tech/problem/problem-with-message",
"title": "Method Not Allowed",
"status": 405,
"detail": "Request method 'POST' not supported",
"path": "/services/ambientcorehub/api/trips",
"message": "error.http.405"
}
I ended up looking at the traces of the Registry that keeps track of the microservices for internal communication and I found out that when this error occurs, I cannot find the corehub in the traces anymore. So it looks like the corehub micro service is not registered.
An other GIT branch of this service does not have this problem, so I performed a diff between these two branches and I removed the changes until I could narrow down the problem.
So, in the corehub, I have a JMS listener based on this mq-jms-spring implementation. The maven dependency is as follows:
<dependency>
<groupId>com.ibm.mq</groupId>
<artifactId>mq-jms-spring-boot-starter</artifactId>
<version>2.2.7</version>
</dependency>
I commented out my JmsListener class and its associated JmsContext (to get access to a topic), and kept only the configuration properties defining access to the server, along with the port, channel, topic name, etc.
If I comment the above maven dependency, my service works again.
If I keep the maven dependency, with the configuration only, my corehub microservice is not registered in the Registry and becomes not accessible anymore from the gateway and thus the Angular UI.
What is important to note, is that I have currently some network issue which prevents me from accessing the JMS Server.
So I believe the exception that is raised by this IBM library because the JMS server is not reachable, breaks the registration of the microservice towards the spring boot registry.
Here are the traces that come over and over in the corehub console:
2020-07-22 08:21:45.316 WARN 7964 --- [nfoReplicator-0] o.s.boot.actuate.jms.JmsHealthIndicator : JMS health check failed
com.ibm.msg.client.jms.DetailedIllegalStateException: JMSWMQ0018: Failed to connect to queue manager '' with connection mode 'Client' and host name '172.31.14.1(9010)'.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:489)
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:215)
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:448)
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8475)
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createProviderConnection(WMQConnectionFactory.java:7815)
at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl._createConnection(JmsConnectionFactoryImpl.java:303)
at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:236)
at com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6005)
at com.ibm.mq.jms.MQConnectionFactory.createConnection(MQConnectionFactory.java:6030)
at org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:409)
at org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:349)
at org.springframework.jms.connection.SingleConnectionFactory.getConnection(SingleConnectionFactory.java:327)
at org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:242)
at org.springframework.boot.actuate.jms.JmsHealthIndicator.doHealthCheck(JmsHealthIndicator.java:52)
at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:82)
at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:95)
at org.springframework.cloud.netflix.eureka.EurekaHealthCheckHandler.getHealthStatus(EurekaHealthCheckHandler.java:110)
at org.springframework.cloud.netflix.eureka.EurekaHealthCheckHandler.getStatus(EurekaHealthCheckHandler.java:106)
at com.netflix.discovery.DiscoveryClient.refreshInstanceInfo(DiscoveryClient.java:1406)
at com.netflix.discovery.InstanceInfoReplicator.run(InstanceInfoReplicator.java:117)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:203)
... 24 common frames omitted
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;AMQ9204: Connection to host '172.31.14.1(9010)' rejected. [1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2538;AMQ9204: Connection to host '/172.31.14.1:9010' rejected. [1=java.net.ConnectException[Connection timed out: connect],3=/172.31.14.1:9010,4=TCP,5=Socket.connect]],3=172.31.14.1(9010),5=RemoteTCPConnection.bindAndConnectSocket]
at com.ibm.mq.jmqi.remote.api.RemoteFAP$Connector.jmqiConnect(RemoteFAP.java:13558)
at com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1426)
at com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1385)
at com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.jmqiConnect(InterceptedJmqiImpl.java:377)
at com.ibm.mq.ese.jmqi.ESEJMQI.jmqiConnect(ESEJMQI.java:562)
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:381)
... 23 common frames omitted
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;AMQ9204: Connection to host '/172.31.14.1:9010' rejected. [1=java.net.ConnectException[Connection timed out: connect],3=/172.31.14.1:9010,4=TCP,5=Socket.connect]
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.bindAndConnectSocket(RemoteTCPConnection.java:901)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1381)
at com.ibm.mq.jmqi.remote.impl.RemoteConnection.connect(RemoteConnection.java:976)
at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getNewConnection(RemoteConnectionSpecification.java:553)
at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSessionFromNewConnection(RemoteConnectionSpecification.java:233)
at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSession(RemoteConnectionSpecification.java:141)
at com.ibm.mq.jmqi.remote.impl.RemoteConnectionPool.getSession(RemoteConnectionPool.java:127)
at com.ibm.mq.jmqi.remote.api.RemoteFAP$Connector.jmqiConnect(RemoteFAP.java:13302)
... 28 common frames omitted
Caused by: java.net.ConnectException: Connection timed out: connect
at java.base/java.net.PlainSocketImpl.connect0(Native Method)
at java.base/java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:101)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/java.net.Socket.connect(Socket.java:558)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$4.run(RemoteTCPConnection.java:1022)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$4.run(RemoteTCPConnection.java:1014)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.connectSocket(RemoteTCPConnection.java:1014)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.bindAndConnectSocket(RemoteTCPConnection.java:805)
... 35 common frames omitted
Here are some version numbers:
jhipster-dependencies.version: 3.0.7
Spring boot version: 2.1.10.RELEASE
ibmmq-jms-spring version(s) that are affected by this issue: Version 2.2.7
Java version (including vendor and platform): AdoptOpenJDK\jdk-11.0.6.10-hotspot
A small code sample that demonstrates the issue.
Here is my configuration in application.yml:
spring:
jms:
# Used for JMS Message reception.
isPubSubDomain: false
application:
oag:
# This can be a queue or a topic (if subdomain is defined)
# In case of a topic, sub domain must be set to public.
queueName: "BRIDGE.XXX.TO.YYY.TST"
isTopic: false
ibm:
mq:
queueManager:
channel: XXX_GWT11.BT1
connName: 172.31.14.1(9010)
user: xxxx
password:
Would it be possible to catch this exception to avoid breaking regular Spring Boot registration mechanism?
I cannot afford having my cluster down because I cannot access the JMS server.
Beside this, I opened this message on IBM MQ side here and a person suggested me to stop the JMS health indicator. So I set the following property to no avail:
management:
endpoint:
jms:
# Prevent Unreachable JMS Server from unregistering corehub from the registry, leading to unreachable microservice from the Gateway
enabled: false
Corresponding documentation is here
Any help would be greatly appreciated.
Thank you
Christophe

Answer 1

If you do not want your application to be considered unhealthy when JMS is down, disabling the JMS health indicator is what I would recommend. It hasn't worked for you as you have used management.endpoint.jms.enabled. The correct property to use is management.health.jms.enabled:
management:
health:
jms:
enabled: false