Database failure shuts down ActiveMQ windows service using JDBC persistence - windows

I have an ActiveMQ broker running as a Windows service. Its using jdbcPersistenceAdapter with Oracle data source and Oracle's Universal Connection Pooling (UCP).
When the database is down (due to network problems or scheduled maintenance), the ActiveMQ windows service shuts down completely. This, of course makes the broker unavailable even after the database is restored.
I have tried connection validation in UCP, DBCP with connection validation and even MySQL data source without any success. The service shuts down within 30 seconds of database failure (I believe this is because the default cleanupInterval is 30 seconds).
Is there a way to prevent the windows service from shutting down and make it wait for database availability?
Any help is greatly appreciated.
Here is my current configuration from activemq.xml:
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#oracle-ds"/>
</persistenceAdapter>
<bean id="oracle-ds" class="oracle.ucp.jdbc.PoolDataSourceFactory"
factory-method="getPoolDataSource" p:URL="jdbc:oracle:thin:#localhost:1521:amq"
p:connectionFactoryClassName="oracle.jdbc.pool.OracleDataSource"
p:validateConnectionOnBorrow="true" p:user="appuser" p:password="userspassword" />

generally, you should use a JDBC master/slave to support failover from another broker when the database becomes unavailable...
see http://activemq.apache.org/jdbc-master-slave.html
that said, there is a known issue with JDBC master/slave failover that was fixed in 5.6.0...
see https://issues.apache.org/jira/browse/AMQ-1958

Related

Topic subscriber connection in Tibco

I have a process that starts with a topic subscriber.
What happens with a topic subscriber in Tibco if the EMS server shuts down?
I guess it will reconnect. But how many times or for how long it will try to reconnect?
The behaviour depends on various settings, set either on the client or server.
If you for example use a Java client you can set the behaviour through the connection factory:
TibjmsConnectionFactory.setReconnAttemptCount(10);
TibjmsConnectionFactory.setReconnAttemptDelay(1000);
TibjmsConnectionFactory.setReconnAttemptTimeout(1000);
You also can configure this behaviors on the server by using JNDI connections. Those can be defined through the factories.conf file.
BusinessWorks 5 does not reconnect by default, but honors all JNDI propagated settings.
It will try to connect indefinitely.

weblogic11g datasource not releasing the connections to the pool

We have developed a Java application with the below specifications.
Frameworks: spring,hibernate
Database: oracle
Server: weblogic 11g
Problem here is when we use Weblogic Data Source the connections are not releasing to the pool which is causing Database server RAM consumption completely after some transactions. But when we use basic Data source in the application the connections are releasing immediately after each transaction and there was no RAM consumption at Database server end. How can I use container based data source?
This will depend on the parameters which you have selected while configuring data source for weblogic. Please check the following
1) Do not enable "Pinned to Thread" properties
2) Enable "Inactive connection time out" to appropriate value. It may be possible that you application has leaked connection

Spring boot/Amazon PostgreSQL RDS connection pool issue

I am troubleshooting an issue with a Spring Boot app connecting to a PostgreSQL database. The app runs normally, but under fairly moderate load it will begin to log errors like this:
java.sql.SQLException: Timeout after 30000ms of waiting for a connection.
This is running on an Amazon EC2 instance connecting to a PostgreSQL RDS. The app is configured like the following:
spring.datasource.url=jdbc:postgresql://[rds_path]:5432/[db name]
spring.datasource.username=[username]
spring.datasource.password=[password]
spring.datasource.max-active=100
In the AWS console, I see 60 connections active to the database, but that is across several Spring Boot apps (not all this app). When I query the database for current activity using pg_stat_activity, I see all but one or 2 connections in an idle state. It would seem the Spring Boot app is not using all available connections? Or is somehow leaking connections? I'm trying to interpret how pg_stat_activity would show so many idle connections and the app still getting connection pool time outs.
Figured it out. Spring is using the Hikari database connection pooling (didn't realize that until after more closely inspecting the stack trace). Hikari configuration parameters have different names, to set the pool size you use maximum-pool-size. Updated that and problem solved.

Stale connection in IBM MQ when connecting with WAS and Spring-JMS

We are using Websphere Applicaion Server with Spring JMS, Around 25 applications connect to a IBM MQ. Off-late we are seeing lots of stale connection on the MQ channel to which all these applications connect.
By stale connections I mean the connection are not being used for many days and the application keeps creating new connections. We are not able to identify which application creates these connections that are not being used but they all use the same framework code
WAS version = 8.5.5
Spring = 4.1.2
The Spring jms:listener-container has the following configuration
connection-factory = org.springframework.jms.connection.DelegatingConnectionFactory
acknowledge=auto
concurrency=2-10
Any pointers on any configurations that can be done on QueueConnectionFactory (JMS Resource) on WAS or on the spring side would be helpful.
I know I have not given much information, but the problem is that there are no errors / exceptions, the application creates these connections to the MQ channel and all connections gets cleared when the server is restarted
Adding one more question
We use org.springframework.jms.connection.DelegatingConnectionFactory , for replying back, Does it make sense for us to close the session once we have send back the message?
Thanks in Advance
Charlie

Spring JMS: Creating multiple connection to a queue

To process a large number of messages coming to a queue i need guarantee of at least one jms connection to be there at any time. I am using spring and spring allows to have multiple sessions on a single connection only. In case one and only connection fails, application will come to standstill till spring reconnects to the JMS bridge.
So how can i create more than one connection to a queue in Spring, also how can i do connection pooling here.
The answer to this depends on whether you are using Spring inside a J2EE container(jboss etc.) or in a standalone application.
Standalone - you'll find pooling connections to be a problem. Springs SingleConnectionFactory can be setup to renew the connection on an exception garaunteeing that at some point a connection will come online and start processing the queue again, but you'll still have the problem of waiting for that single connection to renew, plus depending on what messaging implementation your dealing with and how it does load balancing you may find yourself stuck with a connection to a single node in a cluster.
If you are running in a container you can rely on the containers connection factory which will be much more robust. JBoss Messaging in the container for instance will failover seamlessly to other nodes and handles pooling under the covers, but if your working in the container its usually easier to bail on JMS template which kind of sucks and use whatever that container provides.

Resources