Apache Camel: Call route again or dynamic routes - jms

I'm using JBoss Fuse 6.1.0 with Camel 2.10.0.redhat-60024.
The list of routes are already known. For example: #start, #step1, #step2, #step3, #finish.
But I don't know sequence. Or sometimes some route may not be needed. I will know that only at #router route (please, see below code).
For example: #start, #step2, #step1, #step3, #finish. or #start, #step1, #step3, #finish. or etc.
But Camel 2.10.0 doesn't have such thing as dynamicRouter. I decided to do something like that:
<?xml version="1.0" encoding="UTF-8"?>
...
<camelContext id="blueprintContext" trace="true"
xmlns="http://camel.apache.org/schema/blueprint">
<route id="start">
<from uri="..." />
<to uri="vm:router" />
</route>
<route id="router">
<from uri="vm:router" />
<bean ref="stepGenerator" method="nextStep" />
<choice>
<when>
<simple>${header.step} contains 'step1'</simple>
<to uri="vm:step1" />
</when>
<when>
<simple>${header.step} contains 'step2'</simple>
<to uri="vm:step2" />
</when>
<when>
<simple>${header.step} contains 'step3'</simple>
<to uri="vm:step3" />
</when>
<when>
<simple>${header.step} contains 'finish'</simple>
<to uri="vm:finish" />
</when>
</choice>
</route>
<route id="step1">
<from uri="vm:step1" />
<log message="Step 1 started." />
<!-- Some logic -->
<to uri="vm:router" />
</route>
<route id="step2">
<from uri="vm:step2" />
<log message="Step 2 started." />
<!-- Some logic -->
<to uri="vm:router" />
</route>
<route id="step3">
<from uri="vm:step3" />
<log message="Step 3 started." />
<!-- Some logic -->
<to uri="vm:router" />
</route>
<route id="finish">
<from uri="vm:finish" />
<log message="Finished!" />
</route>
</camelContext>
Imagine that we have next sequence: #start, #step1, #step2, #step3, #finish. If you to try to run this will be stopped at #start -> #router-> #step1.
In #step1 <to uri="vm:router" /> not working. If you call route twice it will not work. Why?
How can I solve this situation in Camel 2.10.0?

According to the camel website the dynamic Router pattern was introduced in
Version 2.5. I am sure that will work for you...

Related

How to pass message from activemq queue to cxf client rest

I am creating a text message and put it in the activemq queue and i display it in the log. Now i need to pass this message to the cxf rs client to use it in the parameter. I am using blueprint to define the camel routes and cxf client.
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
default-activation="eager" xmlns:xsi="http://www.w3.org/2001/XMLSchema- instance"
xmlns:jaxws="http://cxf.apache.org/blueprint/jaxws" xmlns:cxf="http://camel.apache.org/schema/blueprint/cxf"
xmlns:sec="http://cxf.apache.org/configuration/security" xmlns:jaxrs="http://cxf.apache.org/blueprint/jaxrs"
xmlns:http="http://cxf.apache.org/transports/http/configuration"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0
http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint/cxf
http://camel.apache.org/schema/blueprint/cxf/camel-cxf.xsd
http://cxf.apache.org/blueprint/jaxrs
http://cxf.apache.org/schemas/blueprint/jaxrs.xsd
http://cxf.apache.org/configuration/security
http://cxf.apache.org/schemas/configuration/security.xsd
http://cxf.apache.org/transports/http/configuration
http://cxf.apache.org/schemas/configuration/http-conf.xsd">
<!-- Beans -->
<bean id="myTransform" class="cxfcamel.MyTransform"/>
<bean id="serviceBean" class="cxfcamel.GreetingService" />
<bean id="rsprocessor" class="cxfcamel.RSProcessor"/>
<!-- Web Services -->
<jaxrs:server id="customerService" address="http://localhost:7171 /customers">
<jaxrs:serviceBeans>
<ref component-id="serviceBean" />
</jaxrs:serviceBeans>
</jaxrs:server>
<cxf:rsClient id="rsClient"
address="http://localhost:7171/customers/entry-point/register/nosJ"
serviceClass="cxfcamel.GreetingService">
</cxf:rsClient>
<!-- Camel Routes -->
<camelContext id="camel"
xmlns="http://camel.apache.org/schema/blueprint">
<route>
<from uri="timer://projectTimer?repeatCount=1" />
<bean ref="myTransform" method="transform" />
<to uri="activemq:queue:LOG.ME" />
</route>
<route>
<from uri="activemq:queue:LOG.ME" />
<to uri="log:ExampleActiveMQRouterBlueprint" />
</route>
<route>
<from uri="activemq:queue:LOG.ME" />
<setHeader headerName="Content-Type">
<constant>application/json</constant>
</setHeader>
<setHeader headerName="CamelHttpMethod">
<constant>PUT</constant>
</setHeader>
<to uri="cxfrs:bean:rsClient" />
</route>
</camelContext>
Can any one help me please?
Thanks
Both routes listen to activemq:queue:LOG.ME . A queue in ActiveMQ will consume the message and any other queue will not receive the message. You need to do 1 of 2 things:
Turn your queue into a topic so that both routes will receive the message. Topic vs Queue
Arrange your routes so only one route is listening to activemq:queue:LOG.ME.
There are two ways to accomplish this:
Turn your cxfrs:bean:rsClient invocation into a cxfrs:http://localhost:7171/customers/entry-point/register/nosJ and append the parameters at the end.
The documentation on this isn't very clear, but you might be able to use setHeader:
<setHeader headerName="CamelCxfRsQueryMap">
expression which returns a map
</setHeader>
This expression can be <bean>, a <groovy> embedded expression, etc.

Need help setting variable in Camel

I am trying to set a variable inside of my camel code so I can call this variable when I log my route execution. This variable needs to be set from an xpath statement.
Below is the code that is not working and I suspect that I need to set a variable that is equal to my xpath statement that is found in the log message but I don't know how to do that.
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:camel="http://camel.apache.org/schema/blueprint"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://localhost:61616"/>
<property name="userName" value="user"/>
<property name="password" value="password"/>
</bean>
<camelContext id="blueprintContext" trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<route id="Test_Message_Content_Route">
<from uri="activemq:queue:FirstQueue?username=user&password=password&concurrentConsumers=1&maxConcurrentConsumers=5"/>
<choice>
<when>
<xpath>//destination[text()='TEST']</xpath>
<log message="Test route invoked"/>
<split>
<xpath resultType="java.lang.String">//message_payload/text()</xpath>
<log message="Routed $xpath{//id/text()} to TEST QUEUE"/>
<to uri="activemq:queue:TestQueue?username=user&password=password"/>
</split>
</when>
<when>
<xpath>//destination[text()='DEV']</xpath>
<log message="Dev route invoked"/>
<split>
<xpath resultType="java.lang.String">//message_payload/text()</xpath>
<log message="Routed $xpath{//id/text()} to DEV QUEUE"/>
<to uri="activemq:queue:DevQueue?username=user&password=password"/>
</split>
</when>
<otherwise>
<log message="Sending message to DL Queue"/>
<to uri="activemq:queue:DLQueue?username=user&password=password"/>
</otherwise>
</choice>
</route>
</camelContext>
</blueprint>
I now have this working by using setHeader in my camel context like the following:
<setHeader headerName="id"><xpath>//id/text()</xpath></setHeader>
<log message="Routed ${header.id} to Test queue"/>

Apache Camel + spring remoting + jax-ws + multiple parameters

I have following camel route:
<route id="myRoute">
<from uri="cxf:bean:TestEndpoint />
<process ref="TestProcessor" />
<to uri="bean:TestWS?method=doSomething" />
</route>
where TestWS:
<bean id="TestWS" class="org.springframework.remoting.jaxws.JaxWsPortProxyFactoryBean">
<property name="serviceInterface" value="org.example.TestWS />
<property name="wsdlDocumentUrl" value="http://localhost:8080/TestWSImplService/TestWSImpl?wsdl" />
<property name="namespaceUri" value="http://org.example" />
<property name="serviceName" value="TestWSImplService" />
<property name="portName" value="TestWSImplPort" />
</bean>
and TestWS:
#WebService(targetNamespace = "http://org.example")
public interface TestWS {
public String doSomething(Object param1, Object param2);
}
Could you tell me what TestProcessor should return in order to proper call TestWS with multiple parameters?
Thanks
You need to add multiParameterArray=true option to the endpoint to the bean [1] URI.
<to uri="bean:TestWS?method=doSomething&multiParameterArray=true" />
Then your processor should set body as an array of parameters you want to pass to the service:
exchange.getIn().setBody(new Object[]{"param1", "param2"});
[1] https://camel.apache.org/bean.html

WSO2- Error Using Clone Mediator to send message to an endpoint

I am working on WSO2 proxy service that invokes a lot of other DSS services. My requirement is to send the response of one of the DSS service on to a jms queue and at the same time continue with the exsiting flow in the proxy service i.e. continue invoking and processing the other DSS services.
I need to get the response of Simple Product Service (sequence key -if4_simpleProduct) and send it back on to the queue, but after that continue with logic that is placed in the proxy service and sequence. I tried using CLONE Mediator for the same. But my prxy service flow stops after coming in to the CLONE Meditaor flow. Below is the snippet for proxy service and sequence.
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="SPE_Payload"
transports="jms" startOnLoad="true" trace="enable">
<description />
<target>
<inSequence>
<transaction action="new" />
<xslt key="StepToCommon" />
<log level="full" />
<enrich>
<source type="body" clone="true" />
<target type="property" property="MSG_PAYLOAD" />
</enrich>
<sequence key="if4_simpleProduct" />
<filter xpath="boolean (//product/simpleProduct/altPrdCodes/pn_apc_id/text())">
<enrich>
<source clone="true" xpath="//product/simpleProduct/altPrdCodes" />
<target type="property" property="ALT_PRDS" />
</enrich>
<sequence key="if4_alternateProducts" />
</filter>
....
.....
<transaction action="commit" />
</inSequence>
</target>
<parameter name="transport.jms.ContentType">application/xml</parameter>
<parameter name="transport.jms.Destination">test</parameter>
</proxy>
Given below is the code of for sequence while calling Simple Product Service
<sequence xmlns="http://ws.apache.org/ns/synapse" name="if4_simpleProduct"
onError="myErrorHandlerSeq">
<payloadFactory>
<format>
<p:O_SimpleProduct xmlns:p="http://ws.wso2.org/dataservice">
<p:PC_STEP_SKU_ID>$1</p:PC_STEP_SKU_ID>
</p:O_SimpleProduct>
</format>
<args>
<arg xmlns:m="http://wso2.org/services/product" evaluator="xml"
expression="//product/simpleProduct/step_id/text()" />
</args>
</payloadFactory>
<header name="Action" value="urn:O_SimpleProduct" />
<callout serviceURL="local://localhost/services/productEnrichment"
action="urn:O_SimpleProduct" useServerConfig="true">
<source xmlns:ns="http://org.apache.synapse/xsd" xpath="$body/child::*[fn:position()=1]" />
<target xmlns:ns="http://org.apache.synapse/xsd" xpath="$body/child::*[fn:position()=1]" />
</callout>
<clone sequential="false">
<property name="RESPONSE" value="true" />
<header name="To" action="remove" />
<target>
<endpoint>
<address
uri="jms:/step.IF04Output?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue" />
</endpoint>
</target>
<target>
<filter xmlns:ns="http://org.apache.synapse/xsd" xmlns:ds="http://wso2.org/services/product"
source="//ds:result/ds:set/ds:PC_STATUS/text()" regex="0">
<then>
<log level="custom">
<property name="STATUS"
value="************DONE INSERTING SIMPLE ATTRIBUTES***************" />
</log>
</then>
<else>
<log level="full" category="ERROR">
<property name="STATUS"
value="************ERROR OCCURED WHILE INSERTING SIMPLE ATTRIBUTES. ROLLBACKING THE TRANSACTION***************" />
<property name="PC_RP_ID" expression="get-property('GEN_RP_ID')" />
</log>
<transaction action="rollback" />
<drop />
</else>
</filter>
<property xmlns:ds="http://wso2.org/services/product" name="GEN_RP_ID"
expression="//ds:result/ds:set/ds:PC_RP_ID/text()" />
<enrich>
<source type="property" clone="true" property="MSG_PAYLOAD" />
<target type="body" />
</enrich>
</target>
</clone>
</sequence>
I am struggling with the issue since last two days. tried sing Clone meditior in several places. but it does not work. Could you please provide your suggestions soon as I have a demo to go tomorrow and I need to sort it out :(
Cheers,
Aanchal
I think this is the part you are struggling;
<callout serviceURL="local://localhost/services/productEnrichment"
action="urn:O_SimpleProduct" useServerConfig="true">
<source xmlns:ns="http://org.apache.synapse/xsd" xpath="$body/child::*[fn:position()=1]" />
<target xmlns:ns="http://org.apache.synapse/xsd" xpath="$body/child::*[fn:position()=1]" />
</callout>
<clone sequential="false">
<property name="RESPONSE" value="true" />
<header name="To" action="remove" />
<target>
<endpoint>
<address
uri="jms:/step.IF04Output?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue" />
</endpoint>
</target>
Since your requirement is get response form one of the DS service and send that to queue..I think it fails because you are removing "To" header..Use callout mediator inside clone mediator's one of the "Target" configuration after sending to JMS queue drop the message there..In the other target, define your rest of the logic..

DBCP Idle Connections not being reused in Camel Route

I am pretty sure the idle connections are not being re-used or I am leaking connections. I have a simple route that start from a file consumer. The file consumer consumes text files. After picking up the file I check a table to ensure that this is not a duplicate file.
I then convert the message body from a file to string. I then split the file up and run the individual pieces through a route depending on what type of record it is. Each one of these routes eventually inserts this record into staging table on a server running on MySQL.
Below is a simplified version of the route.
<bean id="myPool" class="java.util.concurrent.Executors" factory-method="newFixedThreadPool">
<argument index="0" value="8"/>
</bean>
<camelContext trace="false" handleFault="true" errorHandlerRef="redeliveryErrorHandler" id="FileETLProcess" xmlns="http://camel.apache.org/schema/blueprint">
<errorHandler type="DefaultErrorHandler" useOriginalMessage="true" id="redeliveryErrorHandler">
<redeliveryPolicy maximumRedeliveries="3" redeliveryDelay="25" backOffMultiplier="2" useExponentialBackOff="false" retryAttemptedLogLevel="INFO"/>
</errorHandler>
<onException useOriginalMessage="true">
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<log message="STARTING ERROR GENERAL HANDLER"/>
<bean ref="GeneralError"/>
<to uri="smtp://mailsrv?to=x.x#yadda.com&from=Error#Camel.com.au&subject=GENERAL ERROR: A File Could Not Be Imported&contentType=text/html"/>
<to uri="file:d:/Inbox/.badFile?fileName=${file:onlyname.noext}_GENERALERROR_${date:now:yyyyMMddHHmmss}.${file:name.ext}"/>
</onException>
<route id="ExtractFileRoute">
<from uri="file:d:/Inbox?delay=10000&move=.donebackup/${date:now:yyyyMMdd}/${file:onlyname.noext}_DONE_${date:now:yyyyMMddHHmmss}.${file:name.ext}&readLock=changed&include=.*.dl&maxMessagesPerPoll=0&sortBy=${file:length}"/>
<bean ref="FileCheck"/>
<choice>
<when>
<simple>${header.ACCEPTEDFILE} == 'YES'</simple>
<log message="File Import Route Started At:${date:now:yyyy-MM-dd HH:mm:ss}"/>
<convertBodyTo type="java.lang.String"/>
<log message="Converted File To String:${date:now:yyyy-MM-dd HH:mm:ss} handing data to File To DB route."/>
<split parallelProcessing="true" executorServiceRef="myPool" streaming="true" shareUnitOfWork="true">
<tokenize token="\n"></tokenize>
<setHeader headerName="SPLITFINISHED">
<simple>${property.CamelSplitComplete}</simple>
</setHeader>
<setHeader headerName="SPLITNUMBER">
<simple>${property.CamelSplitIndex}</simple>
</setHeader>
<bean ref="EnrichHeader"/>
<choice>
<when>
<simple>${header.RECORDTYPE} == 'HEADER'</simple>
<doTry>
<unmarshal ref="bindyHeader"/>
<bean ref="HeaderPersist"/>
<choice>
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
</choice>
<doCatch>
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<bean ref="RecordErrorReport"/>
<choice>
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
</choice>
</doCatch>
</doTry>
</when>
<when>
<simple>${header.RECORDTYPE} == 'A'</simple>
<doTry>
<unmarshal ref="bindyAccount"/>
<bean ref="AccountPersist"/>
<choice>
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
</choice>
<doCatch>
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<bean ref="RecordErrorReport"/>
<choice>
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
</choice>
</doCatch>
</doTry>
</when>
<when>
<simple>${header.RECORDTYPE} == 'C'</simple>
<doTry>
<unmarshal ref="bindyComaker"/>
<bean ref="CoMakerPersist"/>
<choice>
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
</choice>
<doCatch>
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<bean ref="RecordErrorReport"/>
<choice>
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
</choice>
</doCatch>
</doTry>
</when>
Some other beans here........
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
<otherwise>
<to uri="smtp://ims-mail?to=ops#ipanic&from=Error#yadda.com&subject=URGENT:UNKOWN RECORD TYPE FOUND IN FILE"/>
<choice>
<when>
<simple>${property.CamelSplitComplete} == true</simple>
<to uri="direct:auxrecordsmove"/>
</when>
</choice>
</otherwise>
</choice>
</split>
</when>
<otherwise>
<to uri="file:d:/RMSInbox/.badFile?fileName=${file:onlyname.noext}_POSSIBLE_DUPLICATE_ERROR_${date:now:yyyyMMddHHmmss}.${file:name.ext}"/>
<bean ref="FileErrorReport"/>
<to uri="smtp://ims-mail?to=ops#panic.com&from=Error#yadda.com&subject=ERROR: A File Could Not Be Imported&contentType=text/html"/>
</otherwise>
</choice>
</route>
So each message on this route eventually hits a bean which will insert it into a database. So I added DBCP to the dependencies and then declare it in my osgi xml blueprint as follows:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://Canttouchthis:3306/ETLDB"/>
<property name="username" value="ETLUser"/>
<property name="password" value="password"/>
<property name="initialSize" value="2"/>
<property name="maxActive" value="16"/>
<property name="maxIdle" value="16"/>
<property name="minIdle" value="2"/>
<property name="timeBetweenEvictionRunsMillis" value="180000"/>
<property name="minEvictableIdleTimeMillis" value="180000"/>
<property name="testOnBorrow" value="true"/>
<property name="testWhileIdle" value="true"/>
<property name="testOnReturn" value="true"/>
<property name="validationQuery" value="SELECT 1"/>
<property name="maxWait" value="10000"/>
<property name="removeAbandoned" value="true"/>
<property name="logAbandoned" value="false"/>
<property name="removeAbandonedTimeout" value="300"/>
</bean>
I also declare my beans that will do the processing like this:
<bean id="AccountPersist" class="com.foo.NewAccount.AccountInformationToDatabase">
<property name="dataSource" ref="dataSource"/>
</bean>
Now when the split on the file has been finished I want to ensure that the records match up. Basically the file has account records and some auxiliary information. So I check in the routes when the split is finished and then once the file is completely in the staging tables I run some additional sanity checks in MySQL.
This second route looks something like this:
<route trace="false" id="MoveMatchingAuxRecordsFromStage">
<from uri="direct:auxrecordsmove"/>
<log message="File Import Route Ended At:${date:now:yyyy-MM-dd HH:mm:ss}"/>
<log message="ETL Route Start AT: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<log message="Moving Matching Comaker records at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<bean ref="CoMakerETL"/>
<log message="Matching Comaker records move finised at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<log message="Moving Matching Credit History records at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<bean ref="CreditHistoryETL"/>
<log message="Matching Credit History records move finised at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<log message="Moving Matching Extra Detail records at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<bean ref="ExtraDetailInformationETL"/>
<log message="Matching Extra Detail records move finised at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<log message="Moving Legal Information records at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<bean ref="LegalInformationETL"/>
<log message="Matching Legal Information records move finised at: ${date:now:yyyy-MM-dd HH:mm:ss}"/>
<log message="ETL Route Finished At ${date:now:yyyy-MM-dd HH:mm:ss}"/>
</route>
So in my testing everything went fine I could import a file quiet effectively like this. My problems started when I placed more than 5 files in the folder. Basically I watch MySQL grow the connection pool to the maximum size and then not re-use the connections.
So we hit 16 concurrent connections they go to sleep after the a couple of files has been loaded then somewhere at the 4,5,6 file all of a sudden I get the following error:
Cannot get a connection, pool error Timeout waiting for idle object
Or as it appears in the log
[ pool-3-thread-35] oveMatchingAuxRecordsFromStage INFO Matching Extra Detail records move finised at: 2013-07-26 17:41:59
[ pool-3-thread-35] oveMatchingAuxRecordsFromStage INFO Moving Legal Information records at: 2013-07-26 17:41:59
[ pool-3-thread-35] DefaultErrorHandler INFO Failed delivery for (MessageId: ID-IMS-WS2013-001-52799-1374824474993-0-2693 on ExchangeId: ID-IMS-WS2013- 001-52799-1374824474993-0-3230). On delivery attempt: 0 caught: java.lang.Exception: Cannot get a connection, pool error Timeout waiting for idle object
[thread #0 - file://d:/RMSInbox] ExtractRMSNewAccountFileRoute INFO STARTING ERROR GENERAL HANDLER
The MySQL max connections have been pushed to a massive 512. I have tried various pool sizes, threading options etc.
As a matter of interest all of my JDBC code follows this structure. It is not complicated SQL just insert statements....
public class RecordParserErrorReporter {
private static final String SQL_INSERT="INSERT INTO `ETL`.`ETLLog` "+
" ( "+
" `ETL_log_text`, "+
" `filename`, "+
" `record_number`, "+
" `error_message`) "+
" VALUES "+
" ( "+
" ?, "+
" ?, "+
" ?, "+
" ? "+
" ); ";
private BasicDataSource dataSource;
public BasicDataSource getDataSource() {
return dataSource;
}
public void setDataSource(BasicDataSource dataSource) {
this.dataSource = dataSource;
}
public void HandleError
(
#Body String msgBody
, #Headers Map hdr
, Exchange exch
)
{
Connection conn = null;
PreparedStatement stmt=null;
try
{
Exception e = exch.getProperty(Exchange.EXCEPTION_CAUGHT,Exception.class);
conn= dataSource.getConnection();
stmt =conn.prepareStatement(SQL_INSERT);
stmt.setString(1, msgBody);
stmt.setString(2, (String)hdr.get("CamelFileName"));
stmt.setInt(3, (Integer)hdr.get("SPLITNUMBER"));
stmt.setString(4, e.getMessage());
stmt.executeUpdate();
}
catch (Exception e)
{
System.out.println(e.getMessage());
}
finally
{
try
{
if (stmt!=null)
{
stmt.close();
}
if (conn!=null)
{
conn.close();
conn= null;
}
}
catch(SQLException e)
{
System.out.println(e.getMessage());
}
}
}
}
How do I track down why my connections are not being reused? Or if I am leaking connections how do I track that down?
Update:
I set the file consumer to consume 1 file on each poll with a second delay between the polls. I can see it creating a new connection pool for each time the route starts and then not reusing the previously created pool. Appears that for every file I queue a new connection pool is created. Not exactly what I want. Is this correct.
I have included a screen shot of how this looks see below:
Update 2:
This not running under spring DM but a OSGI blue print. I am pretty convinced that the datasource is getting instantiated more than once.
Update 3:
Well ok so the datasource is not getting instantiated more than once. The idle connections are simply not used. I did find something interesting though so I suspect this might be related to what I am see more info in the link here: http://fusesource.com/forums/thread.jspa?threadID=4659&tstart=15
Judging by the amount of view this question is getting I am pretty much stuck out there.
Found the problem. This was a real picnic error or id10t mistake on my side.
In my ETL beans I had the following lines of code
try
{
conn= dataSource.getConnection();
stmt =conn.prepareStatement(SQL_ETL_INSERT);
stmt.setString(1, (String)hdr.get("CamelFileName"));
stmt.executeUpdate();
conn= dataSource.getConnection();
stmt =conn.prepareStatement(SQL_ETL_DELETE);
stmt.setString(1, (String)hdr.get("CamelFileName"));
stmt.executeUpdate();
}
catch (Exception e)
{
throw new Exception(e.getMessage());
}
finally
{
try
{
if (stmt!=null)
{
stmt.close();
stmt= null;
}
if (conn!=null)
{
conn.close();
conn= null;
}
}
catch(SQLException e)
{
throw new Exception(e.getMessage());
}
}
Notice I run the conn= dataSource.getConnection(); twice! Then I release only one of the connections. Copy and paste and late nights coding does not mix.

Resources