I have 2 issues regarding Spring AWS-SQS sdk, (Or maybe I am doing it wrong).
First is that I used CLI earlier and I managed to get the message Id of the sent message example:
aws sqs send-message --queue-url https://sqs.us-west-2.amazonaws.com/testqueue --message-body hooray
'{
"MD5OfMessageBody": "d3101ad",
"MessageId": "jdhj-933"
}
Now I tried with spring-cloud-starter-aws-messaging and I setup a Queue Messaging template like this
private final QueueMessagingTemplate queueMessagingTemplate;
public SqsQueueService(#Qualifier("amazonSQSAsync") final AmazonSQSAsync amazonSQS) {
this.queueMessagingTemplate = new QueueMessagingTemplate(amazonSQS);
}
public void sendMessage(String queueName, String queueMessage) {
Map<String, Object> headers = new HashMap<>();
queueMessagingTemplate.convertAndSend(queueName, queueMessage, headers);
}
I can seem to get the message Id of the sent message using queueMessagingTemplate.convertAndSend(queueName, queueMessage, headers);
I need the messageId to fulfil some business logic.
The second issue is my listener can receive messages however the messageID is null as well;
#Async
#SqsListener(value = "${notification.sqs-queue-url}", deletionPolicy = SqsMessageDeletionPolicy.NEVER)
public void listen(Acknowledgment acknowledgment, String message, String messageId) {
//messageId is equal to message here. which is wrong for me
}
The message is always equal to messageId, which is confusing, Any advice on where I maybe going wrong?
I changed the listner method signature to
#Async
#SqsListener(value = "${queue-url}", deletionPolicy = SqsMessageDeletionPolicy.NEVER)
public void listen(Acknowledgment acknowledgment, String message, #Headers MessageHeaders headers) throws ExecutionException, InterruptedException {
String messageId = (String) headers.get("MessageId");
acknowledgment.acknowledge().get();
}
Then extracted the messageId from the headers map
Related
I already tried disabling the Default Error handling of Spring boot w/c throws
{
"timestamp": 1575346220347,
"status": 500,
"error": "Internal Server Error",
"exception": "org.springframework.web.client.HttpClientErrorException",
"message": "401 Unauthorized",
"path": "/auth/login" }
By adding the ff. Config.
#SpringBootApplication(exclude = ErrorMvcAutoConfiguration.class)
and
spring.autoconfigure.exclude: org.springframework.boot.autoconfigure.web.servlet.error.ErrorMvcAutoConfiguration
But I'm getting a bunch of HTML formatted Response instead of the JSON response it should be getting from the server.
You can Use Controller Advice to make a global exception handler. Inside the ControllerAdvice class, you can use #ExceptionHandler annotation to handle exceptions. Here is a good article about ControllerAdvice. https://medium.com/#jovannypcg/understanding-springs-controlleradvice-cd96a364033f
I was not able to disable SpringBoots automatic handling of Error responses however I was able to get the proper JSON Error Response by wrapping my Rest Template request in a try catch and using a library in the rest template as it turns out there is a bug in Rest Template that wouldn't allow me to retrieve the Response body.
From
private final RestTemplate restTemplate = new RestTemplate();
To
private final RestTemplate restTemplate = new RestTemplate(new HttpComponentsClientHttpRequestFactory());
Try-Catch Wrapping
ResponseEntity resp = null;
try{
resp = restTemplate.postForEntity(hostUrl+loginUrl, request,Object.class);
}catch(HttpClientErrorException e) {
ErrorDto result = new ObjectMapper().readValue(e.getResponseBodyAsString(), ErrorDto.class);
return new ResponseEntity<>(result, e.getStatusCode());
}
ErrorDto.java
#JsonIgnoreProperties(ignoreUnknown = true)
public class ErrorDto {
#JsonProperty("Message")
private String message;
#JsonProperty("Reason")
private String reason;
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
public String getReason() {
return reason;
}
public void setReason(String reason) {
this.reason = reason;
}
}
I'm sending a message to Kafka using the ReplyingKafkaTemplate and it's sending the message with a kafka_correlationId. However, when it hits my #KafkaListener method and forwards it to a reply topic, the headers are lost.
How do I preserve the kafka headers?
Here's my method signature:
#KafkaListener(topics = "input")
#SendTo("reply")
public List<CustomOutput> consume(List<CustomInput> inputs) {
... /* some processing */
return outputs;
}
I've created a ProducerInterceptor so I can see what headers are being sent from the ReplyingKafkaTemplate, as well as from the #SendTo annotation. From that, another strange thing is that the ReplyingKafkaTemplate is not adding the documented kafka_replyTopic header to the message.
Here's how the ReplyingKafkaTemplate is configured:
#Bean
public KafkaMessageListenerContainer<Object, Object> replyContainer(ConsumerFactory<Object, Object> cf) {
ContainerProperties containerProperties = new ContainerProperties(requestReplyTopic);
return new KafkaMessageListenerContainer<>(cf, containerProperties);
}
#Bean
public ReplyingKafkaTemplate<Object, Object, Object> replyingKafkaTemplate(ProducerFactory<Object, Object> pf, KafkaMessageListenerContainer<Object, Object> container) {
return new ReplyingKafkaTemplate<>(pf, container);
}
I'm not sure if this is relevant, but I've added Spring Cloud Sleuth as a dependency as well, and the span/trace headers are there when I'm sending messages, but new ones are generated when a message is forwarded.
Arbitrary headers from the request message are not copied to the reply message by default, only the kafka_correlationId.
Starting with version 2.2, you can configure a ReplyHeadersConfigurer which is called to determine which header(s) should be copied.
See the documentation.
Starting with version 2.2, you can add a ReplyHeadersConfigurer to the listener container factory. This is consulted to determine which headers you want to set in the reply message.
EDIT
BTW, in 2.2 the RKT sets up the replyTo automatically if there is no header.
With 2.1.x, it can be done, but it's a bit involved and you have to do some of the work yourself. The key is to receive and reply a Message<?>...
#KafkaListener(id = "so55622224", topics = "so55622224")
#SendTo("dummy.we.use.the.header.instead")
public Message<?> listen(Message<String> in) {
System.out.println(in);
Headers nativeHeaders = in.getHeaders().get(KafkaHeaders.NATIVE_HEADERS, Headers.class);
byte[] replyTo = nativeHeaders.lastHeader(KafkaHeaders.REPLY_TOPIC).value();
byte[] correlation = nativeHeaders.lastHeader(KafkaHeaders.CORRELATION_ID).value();
return MessageBuilder.withPayload(in.getPayload().toUpperCase())
.setHeader("myHeader", nativeHeaders.lastHeader("myHeader").value())
.setHeader(KafkaHeaders.CORRELATION_ID, correlation)
.setHeader(KafkaHeaders.TOPIC, replyTo)
.build();
}
// This is used to send the reply - needs a header mapper
#Bean
public KafkaTemplate<?, ?> kafkaTemplate(ProducerFactory<Object, Object> kafkaProducerFactory) {
KafkaTemplate<Object, Object> kafkaTemplate = new KafkaTemplate<>(kafkaProducerFactory);
MessagingMessageConverter messageConverter = new MessagingMessageConverter();
messageConverter.setHeaderMapper(new SimpleKafkaHeaderMapper("*")); // map all byte[] headers
kafkaTemplate.setMessageConverter(messageConverter);
return kafkaTemplate;
}
#Bean
public ApplicationRunner runner(ReplyingKafkaTemplate<String, String, String> template) {
return args -> {
Headers headers = new RecordHeaders();
headers.add(new RecordHeader("myHeader", "myHeaderValue".getBytes()));
headers.add(new RecordHeader(KafkaHeaders.REPLY_TOPIC, "so55622224.replies".getBytes())); // automatic in 2.2
ProducerRecord<String, String> record = new ProducerRecord<>("so55622224", null, null, "foo", headers);
RequestReplyFuture<String, String, String> future = template.sendAndReceive(record);
ConsumerRecord<String, String> reply = future.get();
System.out.println("Reply: " + reply.value() + " myHeader="
+ new String(reply.headers().lastHeader("myHeader").value()));
};
}
I am building a simple REST api which connects a web server to a back end service, which performs a simple check and sends a response.
So client (over HTTP) -> to Web Server (over ACTIVEMQ/CAMEL)-> to Checking-Service, and back again.
The endpoint for the GET request is "/{id}". I'm trying to make this send a message through queue:ws-out to queue:cs-in and map it all the way back again to the original GET request.
The Checking-Service (cs) code is fine, it simply changes a value in the CheckMessage object to true using jmslistener.
I've searched the web thoroughly for examples, but can't get anything to work. The closest one I found was the following.
This is what I have so far on the Web Server (ws).
RestController
import ...
#RestController
public class RESTController extends Exception{
#Autowired
CamelContext camelContext;
#Autowired
JmsTemplate jmsTemplate;
#GetMapping("/{id}")
public String testCamel(#PathVariable String id) {
//Object used to send out
CheckMessage outMsg = new CheckMessage(id);
//Object used to receive response
CheckMessage inMsg = new CheckMessage(id);
//Sending the message out (working)
jmsTemplate.convertAndSend("ws-out", outMsg);
//Returning the response to the client (need correlation to the out message"
return jmsTemplate.receiveSelectedAndConvert("ws-in", ??);
}
}
Listener on ws
#Service
public class WSListener {
//For receiving the response from Checking-Service
#JmsListener(destination = "ws-in")
public void receiveMessage(CheckMessage response) {
}
}
Thanks!
your receive messages from "ws-in" with 2 consumers jmsTemplate.receiveSelectedAndConvert and WSListener !! message from a queue is consumed by one of the 2.
you send messages to "ws-out" and consume from "ws-in" ?? last queue
is empty and not receive any message, you have to send messages to
it
you need a valid selector to retrieve the message with receiveSelectedAndConvert based on JMSCorrelationID as the example you mntioned or the id received from the rest request but you need to add this id to the message headers like below
this.jmsTemplate.convertAndSend("ws-out", id, new MessageCreator() {
#Override
public Message createMessage(Session session) throws JMSException {
TextMessage tm = session.createTextMessage(new CheckMessage(id));
tm.setJMSCorrelationID(id);
return tm;
}
});
return jmsTemplate.receiveSelectedAndConvert("ws-in", "JMSCorrelationID='" + id+ "'");
forward messages from "ws-out" to "ws-in"
#Service
public class WSListener {
//For receiving the response from Checking-Service
#JmsListener(destination = "ws-out")
public void receiveMessage(CheckMessage response) {
jmsTemplate.convertAndSend("ws-in", response);
}
}
How can I retrieve JMS Message headers after sending the message but without consuming the message ?
this is my message sending code
jmsTemplate.convertAndSend(que, text, message -> {
LOGGER.info("setting JMS Message header values");
message.setStringProperty(RequestContext.HEADER_ID, id);
// LOGGER.info(message.getJMSMessageId()); -- this gives a null value here
return message;
});
the message headers get generated only after the message is posted to the queue so iis there a simple way to retrieve JMS message headers when using MessagePostProcessor?
I've referred the links - here and here but not much of help :(
You can't get the JmsMessageID header until the message is actually sent; the post processor just allows you to modify the converted message JUST BEFORE it is sent.
However, your second link should work ok, since it saves off a reference to the message which you can access later.
EDIT
Confirmed:
#SpringBootApplication
public class So48001045Application {
public static void main(String[] args) {
SpringApplication.run(So48001045Application.class, args).close();
}
#Bean
public ApplicationRunner runner(JmsTemplate template) {
return args -> {
final AtomicReference<Message> msg = new AtomicReference<>();
template.convertAndSend("foo", "bar", m -> {
msg.set(m);
return m;
});
System.out.println(msg.get().getJMSMessageID());
};
}
}
and
ID:host.local-61612-1514496441253-4:1:1:1:1
We have the following Camel route in our application:
from(webServiceUri).routeId("webServiceRoute")
.unmarshal(jaxb)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
final Message in = exchange.getIn();
final DataRequest body = in.getBody(DataRequest.class);
final DataRequest.Items items = body.getItems();
itemValidator.validate(items.getItem());
getContext().createProducerTemplate().sendBody(importUri, body);
DataResponse response = new DataResponse();
response.setReturnCode(ReturnCode.SUCCESS);
in.setBody(response);
}
})
.marshal(jaxb);
We want the "webServiceRoute" to return the response user as soon as the processor has validated the data and forwarded the message to the "importUri". But right now it seems like the response is not returned to the caller until the "importUri" exchange is completed. So my question is what is the "correct" way to asynchronously forward the received request to another queue? There will not be any reply from the "importUri" exchange (i.e. it should be InOnly).
You can replace .sendBody(importUri, body) by .asyncSendBody(importUri, body).
Nevertheless I find your route looks strange to me, why do you use a processor to forward your message. I would write something like:
DataResponse successResponse = new DataResponse();
response.setReturnCode(ReturnCode.SUCCESS);
from(webServiceUri).routeId("webServiceRoute")
.unmarshal(jaxb)
.bean(WebServiceRouteHelper.class,"validate")
.to(importUri)
.setBody(constant(sucessResponse))
.marshal(jaxb);
class WebServiceRouteHelper {
public DataRequest validate(DataRequest dataRequest) throws Exception {
final DataRequest.Items items = body.getItems();
itemValidator.validate(items.getItem());
return dataRequest;
}
}