CamelJdbcColumnNames header missed - spring

I try to use the jdbc component from camel. I found the documentation here: http://camel.apache.org/jdbc.html.
It works well as the result is available from the database but there is no header in the queued answer called CamelJdbcColumnNames as mentioned in the documentation.All i can see is CamelJdbcRowCount. My camel version is 2.15.1.Do i have to turn a switch to enable this?
Here is an extract of my spring-config.xml:
<bean id="ds" class="org.springframework.jdbc.datasource.SimpleDriverDataSource">
<property name="driverClass" value="oracle.jdbc.driver.OracleDriver"/>
<property name="url" value="jdbc:oracle:thin:#myhost:1521:mydbsid"/>
<property name="username" value="myuser"/>
<property name="password" value="mypass"/>
</bean>
<route id="route db">
<from uri="file://data/inbox" />
<to uri="jdbc:ds" />
<to uri="jms:sqlret" />
</route>
EDIT:
To exclude the jms i added a Processor. With this i want to debug the message header. This is a new extract of my spring-config.xml:
<bean id="jdbccheck" class="mypackage.JdbcCheck"></bean>
<route id="route db">
<from uri="file://data/inbox" />
<to uri="jdbc:ds" />
<process ref="jdbccheck"/>
<to uri="jms:sqlret" />
</route>
The Processor code:
public class JdbcCheck implements Processor {
private static final Logger LOG = Logger.getLogger(JdbcCheck.class.getName());
#Override
public void process(Exchange exchange) throws Exception {
LOG.info(exchange.getIn().getHeaders().toString());
}
}
The log message:
{breadcrumbId=ID-chris-HP-50597-1429955241877-0-1, CamelFileAbsolute=false, CamelFileAbsolutePath=C:\daten\chris\source\netbeans\GbLuna\data\inbox2\in.sql, CamelFileContentType=null, CamelFileLastModified=1429953640254, CamelFileLength=36, CamelFileName=in.sql, CamelFileNameConsumed=in.sql, CamelFileNameOnly=in.sql, CamelFileParent=data\inbox2, CamelFilePath=data\inbox2\in.sql, CamelFileRelativePath=in.sql, CamelJdbcRowCount=837}
The last var-/value pair is CamelJdbcRowCount=837 which seems to me that it works somehow. But for further processing i want to deal with the column names. So: how to get CamelJdbcColumnNames?

Ah okay got it now, its because you send the data to a JMS endpoint. And JMS specification only support a number of data types for JMS headers/properties. And that is usually String, numbers and primitive types.
You can read more about this on the Camel JMS documentation page, and from the JMS spec/javadoc.
The column names header is stores as a header of Java collection type and that is not supported by JMS.
http://camel.apache.org/jms
If you enable the Camel tracer you should be able to see the header before the message is routed to the JMS endpoint: http://camel.apache.org/tracer

Related

Spring SAML: SAML message intended destination endpoint did not match recipient endpoint

I am getting 'Caused by: org.opensaml.xml.security.SecurityException: SAML message intended destination endpoint did not match recipient endpoint' exception while SSO between my app SP and client IdP.
Server log show the difference in schemas, see below:
Checking SAML message intended destination endpoint against receiver endpoint
2019-03-05 15:02:44.599 DEBUG [204 default task-41][BaseSAMLMessageDecoder] Intended message destination endpoint: https://my.app.com/app-gateway/saml/SSO
2019-03-05 15:02:44.599 DEBUG [204 default task-41][BaseSAMLMessageDecoder] Actual message receiver endpoint: http://my.app.com/app-gateway/saml/SSO
2019-03-05 15:02:44.600 ERROR [204 default task-41][BaseSAMLMessageDecoder] SAML message intended destination endpoint 'https://my.app.com/app-gateway/saml/SSO' did not match the recipient endpoint 'http://my.app.com/app-gateway/saml/SSO'
My application is running on STG on 2 instances with the LB in front, therefore I use SAMLContextProviderLB context provider instead of SAMLContextProviderImpl:
<bean id="contextProvider" class="org.springframework.security.saml.context.SAMLContextProviderLB">
<property name="scheme" value="https"/>
<property name="serverName" value="my.app.com"/>
<property name="serverPort" value="443"/>
<property name="includeServerPortInRequestURL" value="false"/>
<property name="contextPath" value="/app-gateway"/>
</bean>
<bean id="metadataGeneratorFilter" class="org.springframework.security.saml.metadata.MetadataGeneratorFilter">
<constructor-arg>
<bean class="org.springframework.security.saml.metadata.MetadataGenerator">
<property name="entityBaseURL" value="https://my.app.com/app-gateway1"/>
<property name="entityId" value="${cas.sso.entityId}"/>
<property name="includeDiscoveryExtension" value="false"/>
<property name="extendedMetadata" ref="extendedMetadata"/>
<property name="keyManager" ref="keyManager"/>
</bean>
</constructor-arg>
</bean>
In the source code of getActualReceiverEndpointURI the receiver endpoint URL is being taken from request httpRequest obj. Thus, I am trying to understand at which step that wrong URL http://my.app.com/app-gateway/saml/SSO was set to it. Can anyone explain me it?
protected String getActualReceiverEndpointURI(SAMLMessageContext messageContext) throws MessageDecodingException {
InTransport inTransport = messageContext.getInboundMessageTransport();
if (! (inTransport instanceof HttpServletRequestAdapter)) {
log.error("Message context InTransport instance was an unsupported type: {}",
inTransport.getClass().getName());
throw new MessageDecodingException("Message context InTransport instance was an unsupported type");
}
HttpServletRequest httpRequest = ((HttpServletRequestAdapter)inTransport).getWrappedRequest();
StringBuffer urlBuilder = httpRequest.getRequestURL();
return urlBuilder.toString();
}
You might want to check the following page :
https://developer.jboss.org/thread/240113
I had a similar issue, even with X-Forwarded-Proto properly set on the LB, the request was still interpreted in http only.
The backend must be aware of the header.
add proxy-address-forwarding="true" on the http listener and two filter-ref
<http-listener name="default" socket-binding="http" proxy-address-forwarding="true"/>
<filter-ref name="server-header"/>
<filter-ref name="x-powered-by-header"/>
Hope this help,
For Apache Tomcat server, which is running behind AWS Application load balancer, need to enable the RemoteIPValue so that based on the x-forwarded-proto header, Tomcat will overwrite scheme(https) & port(443) accordingly.
In server.xml
<Valve className="org.apache.catalina.valves.RemoteIpValve" protocolHeader="X-Forwarded-Proto" internalProxies="10\.\d+\.\d+\.\d+|192\.168\.\d+\.\d+|169\.254\.\d+\.\d+|127\.\d+\.\d+\.\d+|172\.(1[6-9]|2[0-9]|3[0-1])\.\d+\.\d+" />

Spring amqp not publishing message to the queue but to Exchange

I am trying to test & benchmark spring-amqp for RabbitMQ with multiple queues so I was creating rabbit template for each queue and using it to send message. The message sent is successful and I can see a message published in the exchange but I don't see anything in the queue. I am guessing it's very minor setting but can't figure it out.
This is my applicationContext.xml
<bean id="banchmarkConnectionFactory" class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
<constructor-arg ref="benchmarkAmqpHost"/>
<property name="username" ref="benchmarkAmqpUser"/>
<property name="password" ref="benchmarkAmqpPass"/>
<property name="virtualHost" ref="benchmarkAmqpVHost"/>
<property name="channelCacheSize" value="10"/>
</bean>
<rabbit:template id="benchmarkAmqpTemplate"
connection-factory="banchmarkConnectionFactory"
exchange="my_exchange"
queue="BenchmarkQueue"
routing-key="BenchmarkQueue" />
<rabbit:admin connection-factory="banchmarkConnectionFactory"/>
<rabbit:queue name="BenchmarkQueue" auto-delete="true" durable="false" auto-declare="true"/>
This is my code which uses the benchmarkAmqpTemplate to publish to the queue.
public class publishMessage {
#Autowired
private RabbitTemplate benchmarkAmqpTemplate;
protected void publish(String payload) {
benchmarkAmqpTemplate.setQueue("BenchmarkQueue");
benchmarkAmqpTemplate.convertAndSend("my_exchange", "BenchmarkQueue", payload);
}
}
When I used the HelloWorld example it did publish a message in the queue so was wondering if I am doing something wrong.
UPDATE
I was able to solve this by adding direct-exchange tag in my context xml. My full xml looks like this:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:rabbit="http://www.springframework.org/schema/rabbit"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsdhttp://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit.xsd">
<bean id="banchmarkConnectionFactory" class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
<constructor-arg ref="benchmarkAmqpHost"/>
<property name="username" ref="benchmarkAmqpUser"/>
<property name="password" ref="benchmarkAmqpPass"/>
<property name="virtualHost" ref="benchmarkAmqpVHost"/>
<property name="channelCacheSize" value="10"/>
</bean>
<rabbit:template id="benchmarkAmqpTemplate"
connection-factory="banchmarkConnectionFactory"
exchange="my_exchange"
queue="BenchmarkQueue"
routing-key="BenchmarkQueue" />
<rabbit:admin connection-factory="banchmarkConnectionFactory"/>
<rabbit:queue name="BenchmarkQueue" auto-delete="true" durable="false" auto-declare="true"/>
<rabbit:direct-exchange name="my_exchange">
<rabbit:bindings>
<rabbit:binding queue="BenchmarkQueue" key="BenchmarkQueue" />
</rabbit:bindings>
</rabbit:direct-exchange>
</beans>
Sorry, but it looks like you misunderstood AMQP protocol a bit.
The message is published to the Exchange with the proper routingKey.
The publisher (RabbitTemplate) doesn't need to know about queues at all.
The queue is a part of receiver, subscriber to the queue.
There is one more feature in between - binding. The queue is bound to the Exchange under the appropriate routingKey. One queue can be bound to several exchanges with different routing keys. By default all queues are bound to the default exchange ("") with routingKeys equal to their names.
Please, refer for more info to the RabbitMQ site.

Spring Integration, jms Inbound gateway for WMQ; Unable to consume messages

I have recently started exploring Spring Integration as that is one of the option we want to evaluate for our project.
The issue i am facing is below.
I have created a JMS inbound gateway to listen to WMQ queue and i am expecting the inboudnd gateway (using DML) should pick up the messages as and when they are available on queue(event - driven).
But some how the example isn't working. It fails to pick messages from the queue. However i can see (using a tool) that there are consumers created on the queue.
Help here is really appreaciated.
Code snippet below.
<bean id="mqFactory" class="com.ibm.mq.jms.MQConnectionFactory">
<property name="hostName" value="${mq.hostName}"/>
<property name="port" value="${mq.port}"/>
<property name="queueManager" value="${mq.queueManager}"/>
<property name="channel" value="${mq.channel}"/>
<property name="transportType" value="${mq.transportType}"/>
<property name="SSLCipherSuite" value="${mq.SSLCipherSuite}"/>
</bean>
<bean id="inCachingConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="mqFactory" />
<property name="sessionCacheSize" value="5" />
</bean>
<bean id="requestQueue-mq" class="com.ibm.mq.jms.MQQueue">
<constructor-arg value="${mq.example.queue}"/>
</bean>
<bean id="demoBean" class="com.jpmchase.example.spring.DemoBean">
</bean>
<jms:inbound-gateway id="wMQ_in_gateway" concurrent-consumers="2" max-concurrent-consumers="5" connection-factory="inCachingConnectionFactory" request-destination="requestQueue-mq"
request-channel="demoChannel" />
<integration:channel id="demoChannel">
</integration:channel>
<integration:service-activator input-channel="demoChannel" ref="demoBean"/>
Below is the service-activator java code.
enter #MessageEndpoint
public class DemoBean {
#ServiceActivator
public String upperCase(String input) {
System.out.println("inside the service activator " + input);
return "JMS response: " + input.toUpperCase();
}
here

Spring integration MQTT publish & subscribe to multiple topics

I am trying to build an application which subscribes to multiple mqtt topics, get the information, process it and form xmls and upon processing trigger an event so that these can be sent to some cloud server and the successful response from there to be sent back to the mqtt channel.
<int-mqtt:message-driven-channel-adapter
id="mqttAdapter" client-id="${clientId}" url="${brokerUrl}" topics="${topics}"
channel="startCase" auto-startup="true" />
<int:channel id="startCase" />
<int:service-activator id="startCaseService"
input-channel="startCase" ref="msgPollingService" method="pollMessages" />
<bean id="mqttTaskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="10" />
</bean>
<bean id="msgPollingService" class="com.xxxx.xxx.mqttclient.mqtt.MsgPollingService">
<property name="taskExecutor" ref="mqttTaskExecutor" />
<property name="vendorId" value="${vendorId}" />
</bean>
My question is how do I publish this to multiple channels, i.e. if I have an option to publish X message to Y topic. At present I have the below:
<int:channel id="outbound" />
<int-mqtt:outbound-channel-adapter
id="mqtt-publish" client-id="kj" client-factory="clientFactory"
auto-startup="true" url="${brokerUrl}" default-qos="0"
default-retained="true" default-topic="${responseTopic}" channel="outbound" />
<bean id="eventListner" class="com.xxxx.xxxx.mqttclient.event.EventListener">
<property name="sccUrl" value="${url}" />
<property name="restTemplate" ref="restTemplate" />
<property name="channel" ref="outbound" />
</bean>
I can publish this like:
channel.send(MessageBuilder.withPayload("customResponse").build());
Can I do something like:
channel.send(Message<?>, topic)
Your configuration looks good. However the MessageChannel is an abstraction for loosely-coupling and gets deal only with Message.
So, you request a-la channel.send(Message<?>, topic) isn't correct for Messaging concepts.
However we have a trick for you. From AbstractMqttMessageHandler:
String topic = (String) message.getHeaders().get(MqttHeaders.TOPIC);
.....
this.publish(topic == null ? this.defaultTopic : topic, mqttMessage, message);
So, what you need from your code is this:
channel.send(MessageBuilder.withPayload("customResponse").setHeader(MqttHeaders.TOPIC, topic).build());
In other words you should send a Message with mqtt_topic header to achieve a dynamic publication from <int-mqtt:outbound-channel-adapter>.
From other side we don't recommend to use MessageChannels directly from the application. The <gateway> with service interface is for such a case for end-application. Where that topic can be one of service method argument marked as #Header(MqttHeaders.TOPIC)

ActiveMQ, Camel, Spring, simple route not working

I am having some trouble with a simple camel route that should be grabbing a message from an ActiveMQ topic that I have and then printing out the contents of the messages to the console through the use of log.
Right now all that it is is the camel-context.xml, and a java class that is producing the topic in ActiveMQ and adding a simple string message to the queue. I am using the ActiveMQ interface to check to see if the topic is being created, and it is, but my message is not being added to the topic nor is it being routed through the camel route. Running main I can get the output of my sys out to the console, and I see that 1 message is "enqueued" and 1 message is "dequeued" in the activemq interface. I just do not get any output to the console from the "log message" in my route.
Any help or tips would be greatly appreciated since I am new to all 3 of these technologies, and I just want to get this simple "Hello World" working.
Thank you! The two files are found below:
After further testing I think that it just has something to do with the way that I am trying to log the contents of the message, because I know that it is picking up my camel route because I added a second topic and told the camel route to route the messages to it like the following:
to uri="activemq:topic:My.SecondTestTopic"
and I am able to see if being redirected to that queue in the activeMQ interface.
TestMessageProducer.java
package com.backend;
import javax.jms.*;
import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
public class TestMessageProducer {
private static String url = ActiveMQConnection.DEFAULT_BROKER_URL;
public static void main(String[] args) throws JMSException {
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
Topic topic = session.createTopic("My.TestTopic");
MessageProducer producer = session.createProducer(topic);
TextMessage message = session.createTextMessage();
message.setText("THIS IS A TEST TEXT MESSAGE BEING SENT TO THE TOPIC AND HOPEFULLY BEING PICKED UP BY THE" +
"CAMEL ROUTE");
producer.send(message);
System.out.println("Sent message '" + message.getText() + "'");
connection.close();
}
}
Camel-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<spring:beans xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://camel.apache.org/schema/spring"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:alch="http://service.alchemy.kobie.com/"
xsi:schemaLocation="http://www.springframework.org/schema/beans classpath:META-INF/spring/spring-beans.xsd
http://camel.apache.org/schema/spring classpath:META-INF/spring/camel-spring.xsd">
<!-- load properties -->
<spring:bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<spring:property name="locations" value="file:backend.properties" />
</spring:bean>
<spring:bean id="properties"
class="org.apache.camel.component.properties.PropertiesComponent">
<spring:property name="location" value="file:backend.properties" />
</spring:bean>
<spring:bean id="jmsConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory">
<spring:property name="brokerURL" value="tcp://0.0.0.0:61616?useLocalHost=true" />
</spring:bean>
<spring:bean id="pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory">
<spring:property name="maxConnections" value="8" />
<spring:property name="maximumActive" value="500" />
<spring:property name="connectionFactory" ref="jmsConnectionFactory" />
</spring:bean>
<spring:bean id="jmsConfig"
class="org.apache.camel.component.jms.JmsConfiguration">
<spring:property name="connectionFactory" ref="pooledConnectionFactory"/>
<spring:property name="transacted" value="false"/>
<spring:property name="concurrentConsumers" value="1"/>
</spring:bean>
<spring:bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<spring:property name="configuration" ref="jmsConfig"/>
</spring:bean>
<!-- camel configuration -->
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="activemq:topic:My.TestTopic"/>
<log message="Output of message from Queue: ${in.body}"/>
<to uri="activemq:topic:My.SecondTestTopic" />
</route>
And you have started the Camel application first. Eg as you send non persistent messages to a topic. And if there is no active subscribers when sending, then the messages is not received by anybody. You may want to use persistent durable topic instead.
I suspect you are creating two seperate instances of an ActiveMQ broker. Can you update your TestMessageProducer to use the URL tcp://localhost:61616 ? Also, can you use jconsole to check the topic activity in the activemq instance on both VMs ?
=== UPDATE ===
I missed the bit about you verifying that the 2nd topic did receive the message, so your route is working.... Must be the logger. If you have the camel source code in your IDE, you could turn the debugger on and place a break point on
org.apache.camel.component.log.LogProducer
.process(Exchange exchange, AsyncCallback callback)
to see what happens and if it is called. Which logging package do you have configured ?

Resources