I'm trying to read the jms header in apache-camel route. The following is the route in which I'm reading body & header.
String endPointTopic = "activemq:topic:geoLoc";
String endPointTopicOut = endPointTopic + "_outbox";
from(endPointTopic)
.log("Message from Topic is ${body} & header is ${header.Action}")
.to(endPointTopicOut);
Basically the And from the logs I'm able to see the following, which means I'm able to read the body but not the id in header.
Message from Topic is GeoLocationInfoDTO{id=2, geoLocationUUId='null',
geoLocationName='null', geoLocationDesc='null',
geoLocationPolygon='null', geoLocationCenterLatitude='null',
geoLocationCenterLongitude='null'} & header is
And the following is the code in which I'm publishing the message to activeMQ through jms template.
private MessageHeaders getMessageHeaders(HttpMethod action) {
log.debug("DomainPublisher : getMessageHeaders");
Map <String, Object> headerMap = new HashMap<>();
headerMap.put("Action", action);
return new MessageHeaders(headerMap);
}
public void publish(BaseDTO dto, HttpMethod action) {
log.debug("DomainPublisher : type is : {} : ", dto.getClass().getName());
getJmsMessagingTemplate().convertAndSend(topicMap.get(dto.getClass().getName()), dto, getMessageHeaders(action));
}
Note: I also tried to log the header id like ${header.id} instead of ${header.Action} but nothing is getting printed.
And I also wanted to know all the headers that are available to the jms message.
You can log exchange with all headers and properties as shown in this example:
.to("log:like-to-see-all?level=INFO&showAll=true&multiline=true")
http://camel.apache.org/log.html
More information about JMS headers can be found here: http://camel.apache.org/jms.html
List of possible headers:
JMSCorrelationID - The JMS correlation ID.
JMSDeliveryMode - The JMS delivery mode.
JMSDestination - The JMS destination.
JMSExpiration - The JMS expiration.
JMSMessageID - The JMS unique message ID.
JMSPriority - The JMS priority (with 0 as the lowest priority and 9 as the highest).
JMSRedelivered - Is the JMS message redelivered.
JMSReplyTo - The JMS reply-to destination.
JMSTimestamp - The JMS timestamp.
JMSType - The JMS type.
JMSXGroupID - The JMS group ID.
As per the Claus Ibsen comment looks like JMS headers only allow certain types as headers and camel will drop invalid headers. And it looks like HttpMethod (type Enum) is been dropped by Camel. All I have to do in my code is convert the Enum as String while constructing the header.
headerMap.put("Action", action);
to
headerMap.put("Action", action.toString());
The JMS headers can be viewed from the karaf client console by running the following command:
activemq:browse --amqurl tcp://localhost:61616 --msgsel JMSMessaageID='1' -Vheader TEST.FOO
Note: The above are all example values, change according to your config.
Related
I am building a Springboot application using Spring Kafka where I am getting messages from a topic. I have to modify those messages and then produce them to another topic. I don't want to consume any other message till I have processed my current one. How can I achieve this?
#KafkaListener(
topics = "${event.topic.name}",
groupId = "${event.topic.group.id}",
containerFactory = "eventKafkaListenerContainerFactory"
)
public void consume(Event event) {
logger.info(String.format("Event created(from consumer)-> %s", event));
}
"event" is a json object which I am receiving as a message.
See https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html#consumerconfigs_max.poll.records:
max.poll.records
The maximum number of records returned in a single call to poll().
Type: int
Default: 500
With Spring Boot you can configure it as this property:
spring.kafka.consumer.maxPollRecords
So, you set it to 1 and no more records are going to be polled from this consumer until you return from your #KafkaListener method.
Please find the use case we need to implement.
First, we need to invoke a Kafka producer a message as a rest service, they will process and give back the response in another topic.
For us, It is a request-reply topic we need to reply back for the same request the response, using replykafka template is working fine, but we can set co-relation id in the header.
As a topic message metadata there are sending in attributes, is there any way to map the co-relation id with request topic message and reply topic message.
Explain to you better.
One microservice expects the payload as given below with correlationId in payload.
{
"operationDate": "2020-09-16T11:58:25",
"correlationId": "-5544538377183901824042719876882142227",
"birthDate": "2013-12-12",
"firstNameEn": "boby",
"firstNameAr": "الشيخ",
}
The microservice will process the payload and will give a response in another topic as.
{
"correlationId": -5544538377183901824042719876882142227,
"consumerId": null,
"userid": 123456,
"statusCode": "SUCCESS",
"errors": null
}
Now as this we need to implement using spring ReplyingKafkaTemplate.
As ReplyingKafkaTemplate will work with correlationId in the header only
Assuming you mean you want to include the topic(s) in the correlation id, see
/**
* Set a function to be called to establish a unique correlation key for each request
* record.
* #param correlationStrategy the function.
* #since 2.3
*/
public void setCorrelationIdStrategy(Function<ProducerRecord<K, V>, CorrelationKey> correlationStrategy) {
You can create your own correlation id, based on the ProducerRecord (which has the topic()).
You just need to make sure it is unique. If you manually set the KafkaHeaders.REPLY_TOPIC, it will be visible to the strategy.
EDIT
With the correlation id in the payload, use setCorrelationIdStrategy to extract the correlationId from the payload and add a RecordInterceptor to do the same on the reply side.
Thanks for the hint.
I have done as overriding the payload with Kafka header correlationId.
#Override
protected ListenableFuture<SendResult> doSend(ProducerRecord producerRecord) {
if(producerRecord.value()!=null){
// i have appeneded the header correlationId in th payload
}
return super.doSend(producerRecord);
}
And in Replay onMessage ,i have populated the response payload correlationId to the header.
#Override
public void onMessage(List<ConsumerRecord<K, R>> data) {
data.forEach(
krConsumerRecord -> //update each record header
);
super.onMessage(data);
}
In this way was successfully integrated request-response semantics with correlationId in the request and response payload.
I am using the following: https://github.com/alanxz/SimpleAmqpClient
I am trying to send headers along with my message but the headers are not being sent; what am I doing wrong?
Here is how my code looks like. I have a configuration object with some basic configuration values.
auto channel = AmqpClient::Channel::Create("localhost", 5672, configuration.UserName, configuration.Password, configuration.VirtualHost, 131072);
channel->DeclareQueue(configuration.QueueName, false, true, false, true);
auto messageBody = "simple json string message nothing fancy"
auto message = AmqpClient::BasicMessage::Create(messageBody);
message->DeliveryMode(AmqpClient::BasicMessage::delivery_mode_t::dm_nonpersistent);
message->ContentType("application/json");
message->Type("XYZRequest");
message->AppId("a guid");
auto headerTable = message->HeaderTable();
headersTable.insert(std::pair<string, string>("Key-1", "value-1"));
headersTable.insert(std::pair<string, string>("Key-2", "value-2"));
channel->BasicPublish(std::string(), configuration.ScoreQueueName, message);
This sends the message to the queue and I can see all the details (AppID, Type, Message Body, etc.) on the RabbitMq management portal except the headers.
What am I missing? Is it some configuration or what is it?
I would appreciate if some one can give me a link to a basic tutorial on how to send headers.
I am stuck. Please help.
message->HeaderTable() does not return a reference to the headers, it returns a copy of it.
To set the headers you must construct the headersTable first, then use message->HeadersTable(headersTable).
Table headersTable;
headersTable.insert(std::pair<string, string>("Key-1", "value-1"));
headersTable.insert(std::pair<string, string>("Key-2", "value-2"));
message->HeadersTable(headersTable);
How to consume specific header messages from queue. I am using camel activemq.
routebuilder:
.....
from("activemq:Q1").
.setHeader("myHeader",xpath(...))
.to("activemq:Q2")
.....
and I trying to consume those messages which has the specific header in another class something like.
....
ConsumerTemplate consumerTemplate = camelContext.createConsumerTemplate();
Exchange exchange = consumerTemplate.receive("activemq:Q2",10000);
String body = exchange.getIn().getBody(String.class);
String customvalue = exchange.getIn().getHeader("myHeader", String.class);
.....
How can I get only those messages which has myHeader=123.?
You can use JMS message selectors. In the Camel consumer endpoint you can use the selector option: http://camel.apache.org/jms
Something a long the lines of
Exchange exchange = consumerTemplate.receive("activemq:Q2?selector=myHeader",10000);
Though I can't remember if the name of the header is enough or you would need to do
Exchange exchange = consumerTemplate.receive("activemq:Q2?selector=myHeader %3D '*'",10000);
Where %3D is = encoded.
everyone. I have an HTTP API for posting messages in a RabbitMQ broker and I need to implement the request-response pattern in order to receive the responses from the server. So I am something like a bridge between the clients and the server. I push the messages to the broker with specific routing-key and there is a Consumer for that messages, which is publishing back massages as response and my API must consume the response for every request. So the diagram is something like this:
So what I do is the following- For every HTTP session I create a temporary responseQueue(which is bound to the default exchange, with routing key the name of that queue), after that I set the replyTo header of the message to be the name of the response queue(where I will wait for the response) and also set the template replyQueue to that queue. Here is my code:
public void sendMessage(AbstractEvent objectToSend, final String routingKey) {
final Queue responseQueue = rabbitAdmin.declareQueue();
byte[] messageAsBytes = null;
try {
messageAsBytes = new ObjectMapper().writeValueAsBytes(objectToSend);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
MessageProperties properties = new MessageProperties();
properties.setHeader("ContentType", MessageBodyFormat.JSON);
properties.setReplyTo(responseQueue.getName());
requestTemplate.setReplyQueue(responseQueue);
Message message = new Message(messageAsBytes, properties);
Message receivedMessage = (Message)requestTemplate.convertSendAndReceive(routingKey, message);
}
So what is the problem: The message is sent, after that it is consumed by the Consumer and its response is correctly sent to the right queue, but for some reason it is not taken back in the convertSendAndReceived method and after the set timeout my receivedMessage is null. So I tried to do several things- I started to inspect the spring code(by the way it's a real nightmare to do that) and saw that is I don't declare the response queue it creates a temporal for me, and the replyTo header is set to the name of the queue(the same what I do). The result was the same- the receivedMessage is still null. After that I decided to use another template which uses the default exchange, because the responseQueue is bound to that exchange:
requestTemplate.send(routingKey, message);
Message receivedMessage = receivingTemplate.receive(responseQueue.getName());
The result was the same- the responseMessage is still null.
The versions of the amqp and rabbit are respectively 1.2.1 and 1.2.0. So I am sure that I miss something, but I don't know what is it, so if someone can help me I would be extremely grateful.
1> It's strange that RabbitTemplate uses doSendAndReceiveWithFixed if you provide the requestTemplate.setReplyQueue(responseQueue). Looks like it is false in your explanation.
2> To make it worked with fixed ReplyQueue you should configure a reply ListenerContainer:
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory);
container.setQueues(responseQueue);
container.setMessageListener(requestTemplate);
3> But the most important part here is around correlation. The RabbitTemplate.sendAndReceive populates correlationId message property, but the consumer side has to get deal with it, too: it's not enough just to send reply to the responseQueue, the reply message should has the same correlationId property. See here: how to send response from consumer to producer to the particular request using Spring AMQP?
BTW there is no reason to populate the Message manually: You can just simply support Jackson2JsonMessageConverter to the RabbitTemplate and it will convert your objectToSend to the JSON bytes automatically with appropriate headers.