servicemix, oracle datasource, connection pooling, max connections - jdbc

I'm trying to configure connection pooling in servcemix 4.4.1. I've created a oracle datasource object and set the properties like user, password,url. I deployed the following xml as osgi service in my smx.
Here's my datasource configuration:
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource">
<property name="URL" value="jdbc:oracle:thin:#connection:url"/>
<property name="user" value="****"/>
<property name="password" value="*****"/>
</bean>
<service interface="javax.sql.DataSource" ref="dataSource">
<service-properties>
<entry key="osgi.jndi.service.name" value="oracleds"/>
</service-properties>
</service>
</blueprint>
The datasource object is created succesfully and I'm able to retrieve the object using jndi lookup in my bundle.
Now I want to apply connection pooling for which I need to declare "maximum connections". Please suggest me how can I declare max no. of connections in the above xml
Thanks in advance.

Here is an example of my blueprint.xml:
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
xsi:schemaLocation=
"http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd"
default-activation="eager">
<cm:property-placeholder id="server.placeholder" persistent-id="test.datasource">
<cm:default-properties>
<cm:property name="driverClassName" value="oracle.jdbc.pool.OracleDataSource" />
<cm:property name="validationQuery" value="SELECT 1 FROM DUAL" />
<cm:property name="defaultReadOnly" value="false" />
<cm:property name="defaultAutoCommit" value="true" />
<cm:property name="maxActive" value="100" />
<cm:property name="whenExhaustedAction" value="2" />
<cm:property name="maxWait" value="-1" />
<cm:property name="maxIdle" value="8" />
<cm:property name="minIdle" value="1" />
<cm:property name="testOnBorrow" value="true" />
<cm:property name="testOnReturn" value="true" />
<cm:property name="timeBetweenEvictionRunsMillis" value="-1" />
<cm:property name="numTestsPerEvictionRun" value="3" />
<cm:property name="minEvictableIdleTimeMillis" value="1800000" />
<cm:property name="testWhileIdle" value="false" />
<cm:property name="softMinEvictableIdleTimeMillis" value="-1" />
<cm:property name="lifo" value="true" />
</cm:default-properties>
</cm:property-placeholder>
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value= "${driverClassName}" />
<property name="url" value="jdbc:oracle:thin:#${db.host}:${db.port}:${db.instance}" />
<property name="username" value="${db.user}" />
<property name="password" value="${db.password}" />
<property name="maxIdle" value="1" />
</bean>
<bean id="connectionFactory" class="org.apache.commons.dbcp.DataSourceConnectionFactory">
<argument ref="dataSource" />
</bean>
<bean id="connectionPool" class="org.apache.commons.pool.impl.GenericObjectPool" >
<argument><null/></argument>
<argument value="${maxActive}" />
<argument value="${whenExhaustedAction}" />
<argument value="${maxWait}" />
<argument value="${maxIdle}" />
<argument value="${minIdle}" />
<argument value="${testOnBorrow}" />
<argument value="${testOnReturn}" />
<argument value="${timeBetweenEvictionRunsMillis}" />
<argument value="${numTestsPerEvictionRun}" />
<argument value="${minEvictableIdleTimeMillis}" />
<argument value="${testWhileIdle}" />
<argument value="${softMinEvictableIdleTimeMillis}" />
<argument value="${lifo}" />
</bean>
<bean id="pooledConnectionFactory" class="org.apache.commons.dbcp.PoolableConnectionFactory" >
<argument ref="connectionFactory" />
<argument ref="connectionPool" />
<argument><null/></argument>
<argument value="${validationQuery}" />
<argument value="${defaultReadOnly}" />
<argument value="${defaultAutoCommit}" />
</bean>
<bean id="poolingDataSource" class="org.apache.commons.dbcp.PoolingDataSource" depends-on="pooledConnectionFactory">
<argument ref="connectionPool" />
</bean>
<service interface="javax.sql.DataSource" ref="poolingDataSource"> <!--ref="oracleDataSource"-->
<service-properties>
<entry key="osgi.jndi.service.name" value="jdbc/testdb"/>
<entry key="datasource.name" value="jdbc/testdb" />
</service-properties>
</service>
</blueprint>
This datasource work fine.
I think the attribute "maxActive" close to what you need.

You can choose between two ways:
use oracle.jdbc.pool.OracleConnectionPoolDataSource as datasource class
use commons-dbcp https://gist.github.com/splatch/2d631ec09f9be0730b1e (this is fork of somebody else gist)

Related

camel-sql consumer DB connection issue

I am using Apache Camel with Spring boot. I am using my spring boot app as jar without any server.
In app I am having a Camel-SQL consumer route with delay of 1000. This consumer then further calls other route which use splitter and parallel processing.
Through Jprofiler I have observed that my DB connection are not getting closed which are opened through consumer.
I also have dbcp2 settings in application.properties and using Oracle DB. I am observing this connection issue for the first time. Did anyone also faced the same issue at anytime? Or do I need to add any other configuration.
Any help will be appreciated.
Here is one of my Oracle configurations:
<cm:property-placeholder id="server.placeholder" persistent-id="name.of.company.datasource">
<cm:default-properties>
<cm:property name="db.host" value="host"/>
<cm:property name="db.port" value="1521"/>
<cm:property name="db.instance" value="dbname"/>
<cm:property name="db.user" value="user"/>
<cm:property name="db.password" value="password"/>
<cm:property name="driverClassName" value="oracle.jdbc.pool.OracleDataSource" />
<cm:property name="validationQuery" value="SELECT 1 FROM DUAL" />
<cm:property name="defaultReadOnly" value="false" />
<cm:property name="defaultAutoCommit" value="true" />
<cm:property name="maxActive" value="100" />
<cm:property name="whenExhaustedAction" value="2" />
<cm:property name="maxWait" value="-1" />
<cm:property name="maxIdle" value="8" />
<cm:property name="minIdle" value="1" />
<cm:property name="testOnBorrow" value="true" />
<cm:property name="testOnReturn" value="true" />
<cm:property name="timeBetweenEvictionRunsMillis" value="-1" />
<cm:property name="numTestsPerEvictionRun" value="3" />
<cm:property name="minEvictableIdleTimeMillis" value="1800000" />
<cm:property name="testWhileIdle" value="false" />
<cm:property name="softMinEvictableIdleTimeMillis" value="-1" />
<cm:property name="lifo" value="true" />
</cm:default-properties>
</cm:property-placeholder>
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value= "${driverClassName}" />
<property name="url" value="jdbc:oracle:thin:#${db.host}:${db.port}:${db.instance}" />
<property name="username" value="${db.user}" />
<property name="password" value="${db.password}" />
<property name="maxIdle" value="1" />
</bean>
<bean id="connectionFactory" class="org.apache.commons.dbcp.DataSourceConnectionFactory">
<argument ref="dataSource" />
</bean>
<bean id="connectionPool" class="org.apache.commons.pool.impl.GenericObjectPool" >
<argument><null/></argument>
<argument value="${maxActive}" />
<argument value="${whenExhaustedAction}" />
<argument value="${maxWait}" />
<argument value="${maxIdle}" />
<argument value="${minIdle}" />
<argument value="${testOnBorrow}" />
<argument value="${testOnReturn}" />
<argument value="${timeBetweenEvictionRunsMillis}" />
<argument value="${numTestsPerEvictionRun}" />
<argument value="${minEvictableIdleTimeMillis}" />
<argument value="${testWhileIdle}" />
<argument value="${softMinEvictableIdleTimeMillis}" />
<argument value="${lifo}" />
</bean>
<bean id="pooledConnectionFactory" class="org.apache.commons.dbcp.PoolableConnectionFactory" >
<argument ref="connectionFactory" />
<argument ref="connectionPool" />
<argument><null/></argument>
<argument value="${validationQuery}" />
<argument value="${defaultReadOnly}" />
<argument value="${defaultAutoCommit}" />
</bean>
<bean id="poolingDataSource" class="org.apache.commons.dbcp.PoolingDataSource" depends-on="pooledConnectionFactory">
<argument ref="connectionPool" />
</bean>
<service interface="javax.sql.DataSource" ref="poolingDataSource">
<service-properties>
<entry key="osgi.jndi.service.name" value="jdbc/oracle/db"/>
<entry key="datasource.name" value="jdbc/oracle/db" />
</service-properties>
</service>
Maybe it will help you.
I think DB connection just doesn't have time to go back to the pool between queries.
You need to configure the connection timeout and similar parameters (idle params, minEvictableIdleTimeMillis, timeBetweenEvictionRunsMillis, numTestsPerEvictionRun).

Spring-Batch MongoDB : Skip functionality is not working properly in MongoItemReader scenario

Skip exception class functionality is not working properly when MongoItemreader is being used.
Issues:
1. Spring batch is stuck at a particular data row (in my case 15). It calls MessageContextReadConverter and onReadError for same row number 15 -- 3 times.
2. skiplistener is never invoked.
Scenario:
Suppose 20 rows are fetched from MongoDB and at row number 15 configurable exception ".ConversionFailedException" is thrown from a custom class derived from Converter, (MessageContextReadConverter implements Converter)
. MessageContextReadConverter is set during mongoTemplate creation,.
Now, exception thrown at row number 15 is received in custom readlistener (public void onReadError(Exception ex)).
Configuration XML:
<context:property-placeholder location="classpath:application.properties" />
<context:component-scan base-package="com.XXX.YYY.batch.kernel" />
<context:component-scan base-package="com.XXX.YYY.batch.dao" />
<context:annotation-config />
<!-- Enable Annotation based Declarative Transaction Management -->
<tx:annotation-driven proxy-target-class="true" transaction-manager="transactionManager" />
<!-- Creating TransactionManager Bean, since JDBC we are creating of type DataSourceTransactionManager -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<batch:job id="txnLogJob" restartable="false">
<batch:step id="txnload">
<tasklet allow-start-if-complete="true" transaction-manager="transactionManager">
<chunk reader="txnLogReader" processor="txnLogProcessor"
writer="txnLogItemWriter" commit-interval="10" skip-limit="3">
<skippable-exception-classes>
<include class="org.springframework.core.convert.ConversionFailedException" />
</skippable-exception-classes>
</chunk>
<listeners>
<listener ref="stepListener" />
<listener ref="chunklistener" />
<listener ref="readlistener" />
<listener ref="skiplistener" />
</listeners>
</tasklet>
</batch:step>
<batch:listeners>
<batch:listener ref="completionListener" />
</batch:listeners>
</batch:job>
<bean id="txnLogReader" class="org.springframework.batch.item.data.MongoItemReader"
scope="step">
<property name="template" ref="mongoTemplate" />
<property name="query" value=" { }" />
<property name="pageSize" value="50" />
<property name="sort">
<map>
<entry key="audit_info.created_on"
value="#{T(org.springframework.data.domain.Sort.Direction).ASC}" />
</map>
</property>
<property name="collection" value="txnlog" />
<property name="targetType" value="com.XXX.YYY.kernel.msg.MessageContext" />
</bean>
<bean id="completionListener"
class="com.XXX.YYY.batch.listeners.JobCompletionNotificationListener" />
<bean id="stepListener"
class="com.XXX.YYY.batch.listeners.StepExecutionListener" />
<bean id="chunklistener"
class="com.XXX.YYY.batch.listeners.ChunkExecutionListener" />
<bean id="readlistener"
class="com.XXX.YYY.batch.listeners.ReadExecutionListener" />
<bean id="skiplistener"
class="com.XXX.YYY.batch.listeners.SkipExecutionListener" />
<bean id="jobParametersDAOImpl" class="com.XXX.YYY.batch.dao.JobParametersDAOImpl" />
<bean id="batchLoader" class="com.XXX.YYY.batch.kernel.BatchLoader" />
<bean id="batchjobParameter" class="com.XXX.YYY.batch.dao.Batch_Job_Parameters" />
<!-- <bean id="reportWriter" class="org.springframework.batch.item.data.MongoItemWriter">
<property name="template" ref="mongoTemplate" /> <property name="collection"
value="txnlog" /> name of the collection to write </bean> -->
<bean id="txnLogItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"
scope="step">
<property name="shouldDeleteIfExists" value="true" />
<property name="resource" value="file:target/test-outputs/output.txt" />
<property name="lineAggregator">
<bean
class="org.springframework.batch.item.file.transform.PassThroughLineAggregator" />
</property>
</bean>
<bean id="txnLogProcessor"
class="com.XXX.YYY.batch.processor.MessageContextItemProcessor" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="databaseType" value="MYSQL" />
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="dataSource" class="com.XXX.YYY.common.DataSource"
destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="connectionProperties" value="${jdbc.connectionProperties}" />
<property name="initialSize" value="${jdbc.initialSize}" />
<property name="maxTotal" value="${jdbc.maxTotal}" />
<property name="maxIdle" value="${jdbc.maxIdle}" />
<property name="minIdle" value="${jdbc.minIdle}" />
<property name="maxWaitMillis" value="${jdbc.maxWaitMillis}" />
<property name="testOnBorrow" value="${jdbc.testOnBorrow}" />
<property name="testWhileIdle" value="${jdbc.testWhileIdle}" />
<property name="testOnReturn" value="${jdbc.testOnReturn}" />
<property name="validationQuery" value="${jdbc.validationQuery}" />
</bean>

how to configure jms queue inside blueprint.xml

how to configure jms queue(using activemq inside karaf) inside blueprint.xml which present inside karaf deploy folder..
below is my code which shows config for jms connection inside blueprint.xml..
<bean id="activemqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616" />
<property name="userName" value="karaf" />
<property name="password" value="karaf" />
</bean>
<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
</bean>
<bean id="resourceManager" class="org.apache.activemq.pool.ActiveMQResourceManager" init-method="recoverResource">
<property name="transactionManager" ref="transactionManager" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
<property name="resourceName" value="activemq.localhost" />
</bean>
<reference id="transactionManager" interface="javax.transaction.TransactionManager" />
<service ref="activemqConnectionFactory" interface="javax.jms.ConnectionFactory">
<service-properties>
<entry key="name" value="connectionFactory" />
<entry key="osgi.jndi.service.name" value="jms/connectionFactory" />
</service-properties>
</service>
You don't configure a queue in OSGi, like you did in J2EE. You just use the connection factory in your plain Java or to configure the camel-jms component.

Spring Web Flow + Thymeleaf

I want to use Spring Web Flow along with Thymeleaf, which was previously configured with Spring MVC. It works, however I get "org.springframework.webflow.definition.registry.NoSuchFlowDefinitionException" on every page. I want to use SWF on certain pages and leave the rest as it was - handled by controllers without flow. Is it possible?
Here's my view config:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:oxm="http://www.springframework.org/schema/oxm"
xmlns:webflow="http://www.springframework.org/schema/webflow-config"
xsi:schemaLocation="http://www.springframework.org/schema/oxm
http://www.springframework.org/schema/oxm/spring-oxm.xsd
http://www.springframework.org/schema/webflow-config
http://www.springframework.org/schema/webflow-config/spring-webflow-config-2.3.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="templateResolver"
class="org.thymeleaf.templateresolver.ServletContextTemplateResolver">
<property name="prefix" value="/WEB-INF/thymeleaf/" />
<property name="suffix" value=".html" />
<property name="templateMode" value="HTML5" />
<!-- Template cache is set to false (default is true). -->
<property name="cacheable" value="false" />
</bean>
<bean id="templateEngine" class="org.thymeleaf.spring4.SpringTemplateEngine">
<property name="templateResolver" ref="templateResolver" />
<property name="additionalDialects">
<set>
<bean class="org.thymeleaf.extras.springsecurity3.dialect.SpringSecurityDialect" />
</set>
</property>
</bean>
<webflow:flow-executor id="flowExecutor"></webflow:flow-executor>
<bean name="/*"
class="org.springframework.webflow.mvc.servlet.FlowController">
<property name="flowExecutor" ref="flowExecutor" />
</bean>
<webflow:flow-registry id="flowRegistry"
flow-builder-services="flowBuilderServices">
<webflow:flow-location-pattern value="/**/*-flow.xml" />
</webflow:flow-registry>
<webflow:flow-builder-services id="flowBuilderServices"
view-factory-creator="mvcViewFactoryCreator" />
<bean
class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
<property name="contentNegotiationManager" ref="cnManager" />
<property name="viewResolvers">
<list>
<bean class="org.springframework.web.servlet.view.BeanNameViewResolver">
<property name="order" value="1" />
</bean>
<bean id="thymeleafViewResolver" class="org.thymeleaf.spring4.view.AjaxThymeleafViewResolver">
<property name="viewClass"
value="org.thymeleaf.spring4.view.FlowAjaxThymeleafView" />
<property name="templateEngine" ref="templateEngine" />
<property name="order" value="2" />
</bean>
</list>
</property>
</bean>
<bean id="mvcViewFactoryCreator"
class="org.springframework.webflow.mvc.builder.MvcViewFactoryCreator">
<property name="viewResolvers" ref="thymeleafViewResolver" />
</bean>
<bean id="thymeleafViewResolver" class="org.thymeleaf.spring4.view.AjaxThymeleafViewResolver">
<property name="viewClass"
value="org.thymeleaf.spring4.view.FlowAjaxThymeleafView" />
<property name="templateEngine" ref="templateEngine" />
</bean>
<!-- Simple strategy: only path extension is taken into account -->
<bean id="cnManager"
class="org.springframework.web.accept.ContentNegotiationManagerFactoryBean">
<property name="favorPathExtension" value="true" />
<property name="ignoreAcceptHeader" value="true" />
<property name="defaultContentType" value="text/html" />
<property name="mediaTypes">
<map>
<entry key="html" value="text/html" />
<entry key="xml" value="application/xml" />
<entry key="atom" value="application/atom+xml" />
</map>
</property>
</bean>

MaxOpenPreparedStatements limit reached

While starting the application once the scheduler starts running , I get the following error
Caused by: java.sql.SQLException: MaxOpenPreparedStatements limit reached
Below is the properties
<entry key="jdbc.preparedStatementCacheSize" value="500" />
<entry key="jdbc.initialSize" value="5" />
<entry key="jdbc.maxActive" value="100" />
<entry key="jdbc.maxIdle" value="${jdbc.initialSize}" />
<entry key="jdbc.minIdle" value="0" />
<entry key="jdbc.validationQuery" value="select database()" />
<entry key="jdbc.maxWait" value="16000"/>
The bean datasource
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="url" value="${jdbc.url}"/>
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="poolPreparedStatements" value="true" />
<property name="maxOpenPreparedStatements" alue="${jdbc.preparedStatementCacheSize}"/>
<property name="initialSize" value="${jdbc.initialSize}" />
<property name="maxActive" value="${jdbc.maxActive}" />
<property name="maxIdle" value="${jdbc.maxIdle}" />
<property name="minIdle" value="${jdbc.minIdle}" />
<property name="validationQuery" value="${jdbc.validationQuery}" />
</bean>
Instead of org.apache.commons.dbcp.BasicDataSource if I use
org.springframework.jdbc.datasource.DriverManagerDataSource then I wont get this exception.
I am using Spring 2.5. Please help me in resolving this exception.

Resources