Jackson2JsonMessageConverter not work for rabbitmq - spring

I use following code for message converter:
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, Queue queue,
MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(queue.getName());
container.setMessageListener(listenerAdapter);
container.setMessageConverter(new Jackson2JsonMessageConverter());
return container;
}
My listener is declared:
public void receiveMessage(List<Map<String, Object>> message) {
try {
System.out.println("Received <" + new String(message, "UTF-8") + ">");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
}
But it always try to gives follow error:
Failed to invoke target method 'receiveMessage' with argument type = [class [B], value = [{[B#40c2d9c5}]","at org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter.invokeListenerMethod(MessageListenerAdapter.java:408)
It seems it tries to invoke byte[] as argument instead of convert json string to List>.

The converter requires a content_type message property that contains the token json - e.g. application/json. You should see a WARN log if you are using at least version 1.6.1.
log.warn("Could not convert incoming message with content-type ["
+ contentType + "], 'json' keyword missing.");
If you can't change the producer to set the content type properly, you can subclass the converter...
#Override
public Object fromMessage(Message message, Object conversionHint) throws MessageConversionException {
message.getMessageProperties().setContentType("application/json");
return super.fromMessage(message, conversionHint);
}

Related

RabbitMQ Spring "Cannot determine target ConnectionFactory for lookup key" when using Java lambda parallelStream

We have a Spring Java application using RabbitMQ, and here is the scenario:
There is a consumer receiving messages from a queue and sending them to another one. We are using "SimpleRabbitListenerContainerFactory" as the container factory, but when sending the messages to the other queue inside a "parallelStream" we've got an IllegalStateException "Cannot determine target ConnectionFactory for lookup key" Exception
When we remove the "parallelStream" it works flawlessly.
public void sendMessage(final StagingMessage stagingMessage, final Long timestamp, final String country) {
final List<TransformedMessage> messages = processMessageList(stagingMessage);
messages.parallelStream().forEach(message -> {
final TransformedMessage transformedMessage = buildMessage(timestamp, ApiConstants.POST_METHOD, country);
myMessageSender.sendQueue(country, transformedMessage);
});
}
Connectio Facotory, where the lookup key is set:
#Configuration
#EnableRabbit
public class RabbitBaseConfig {
#Autowired
private QueueProperties queueProperties;
#Bean
#Primary
public ConnectionFactory connectionFactory(final ConnectionFactory connectionFactoryA, final ConnectionFactory connectionFactoryB) {
final SimpleRoutingConnectionFactory simpleRoutingConnectionFactory = new SimpleRoutingConnectionFactory();
final Map<Object, ConnectionFactory> map = new HashMap<>();
for (final String queue : queueProperties.getAQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryA);
}
for (final String queue : queueProperties.getBQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryB);
}
simpleRoutingConnectionFactory.setTargetConnectionFactories(map);
return simpleRoutingConnectionFactory;
}
#Bean
public Jackson2JsonMessageConverter jackson2JsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
Welcome to stack overflow!
You should always show the pertinent code and configuration beans when asking questions like this.
I assume you are using the RoutingConnectionFactory.
It uses a ThreadLocal to store the lookup key so the send has to happen on the same thread that set the key.
You generally should never go asynchronous in a listener anyway; you risk message loss. To increase concurrency, use the concurrency properties on the container.
EDIT
One technique would be to convey the lookup key in a message header:
#Bean
public RabbitTemplate template(ConnectionFactory rcf) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(rcf);
Expression expression = new SpelExpressionParser().parseExpression("messageProperties.headers['cfSelector']");
rabbitTemplate.setSendConnectionFactorySelectorExpression(expression);
return rabbitTemplate;
}
#RabbitListener(queues = "foo")
public void listen1(String in) {
IntStream.range(0, 10)
.parallel()
.mapToObj(i -> in + i)
.forEach(val -> {
this.template.convertAndSend("bar", val.toUpperCase(), msg -> {
msg.getMessageProperties().setHeader("cfSelector", "[bar]");
return msg;
});
});
}

Spring AMQP, CorrelationId and GZipPostProcessor: UnsupportedEncodingException

I have a Project with Spring AMQP (1.7.12.RELEASE).
If I put a value for the correlationId field (etMessageProperties (). SetCorrelationId) and I use GZipPostProcessor, the following error always occurs:
"org.springframework.amqp.AmqpUnsupportedEncodingException: java.io.UnsupportedEncodingException: gzip"
To solve it, it seems that it works using the following code:
DefaultMessagePropertiesConverter messageConverter = new DefaultMessagePropertiesConverter();
messageConverter.setCorrelationIdAsString(DefaultMessagePropertiesConverter.CorrelationIdPolicy.STRING);
template.setMessagePropertiesConverter(messageConverter);
but I do not know what implications it will have to use it in real with clients that do not use Spring AMQP (I establish this field if the message that has reached me has it).
I enclose a complete example of code:
#Configuration
public class SimpleProducerGZIP
{
static final String queueName = "spring-boot";
#Bean
public CachingConnectionFactory connectionFactory() {
com.rabbitmq.client.ConnectionFactory factory = new com.rabbitmq.client.ConnectionFactory();
factory.setHost("localhost");
factory.setAutomaticRecoveryEnabled(false);
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(factory);
return connectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin() {
RabbitAdmin rabbitAdmin = new RabbitAdmin(connectionFactory());
rabbitAdmin.setAutoStartup(true);
return rabbitAdmin ;
}
#Bean
Queue queue() {
Queue qr = new Queue(queueName, false);
qr.setAdminsThatShouldDeclare(amqpAdmin());
return qr;
}
#Bean
public RabbitTemplate rabbitTemplate()
{
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setEncoding("gzip");
template.setBeforePublishPostProcessors(new GZipPostProcessor());
// TODO :
DefaultMessagePropertiesConverter messageConverter = new DefaultMessagePropertiesConverter();
messageConverter.setCorrelationIdAsString(DefaultMessagePropertiesConverter.CorrelationIdPolicy.STRING);
template.setMessagePropertiesConverter(messageConverter);
return template;
}
public static void main(String[] args)
{
#SuppressWarnings("resource")
ApplicationContext context = new AnnotationConfigApplicationContext(SimpleProducerGZIP.class);
RabbitTemplate _rabbitTemplate = context.getBean(RabbitTemplate.class);
int contador = 0;
try {
while(true)
{
contador = contador + 1;
int _nContador = contador;
System.out.println("\nInicio envio : " + _nContador);
Object _o = new String(("New Message : " + contador));
try
{
_rabbitTemplate.convertAndSend(queueName, _o,
new MessagePostProcessor() {
#SuppressWarnings("deprecation")
#Override
public Message postProcessMessage(Message msg) throws AmqpException {
if(_nContador%2 == 0) {
System.out.println("\t--- msg.getMessageProperties().setCorrelationId ");
msg.getMessageProperties().setCorrelationId("NewCorrelation".getBytes(StandardCharsets.UTF_8));
}
return msg;
}
}
);
System.out.println("\tOK");
}catch (Exception e) {
System.err.println("\t\tError en envio : " + contador + " - " + e.getMessage());
}
System.out.println("Fin envio : " + contador);
Thread.sleep(500);
}
}catch (Exception e) {
System.err.println("Exception : " + e.getMessage());
}
}
}
The question is, if I change the configuration of the rabbitTemplate so that the error does not happen, can it have implications for clients that use Spring AMQP or other alternatives?
--- EDIT (28/03/2019)
This is the complete stack trace with the code:
org.springframework.amqp.AmqpUnsupportedEncodingException: java.io.UnsupportedEncodingException: gzip
at org.springframework.amqp.rabbit.support.DefaultMessagePropertiesConverter.fromMessageProperties(DefaultMessagePropertiesConverter.java:211)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doSend(RabbitTemplate.java:1531)
at org.springframework.amqp.rabbit.core.RabbitTemplate$3.doInRabbit(RabbitTemplate.java:716)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1455)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1411)
at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:712)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertAndSend(RabbitTemplate.java:813)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertAndSend(RabbitTemplate.java:791)
at es.jab.example.SimpleProducerGZIP.main(SimpleProducerGZIP.java:79)
Caused by: java.io.UnsupportedEncodingException: gzip
at java.lang.StringCoding.decode(Unknown Source)
at java.lang.String.<init>(Unknown Source)
at java.lang.String.<init>(Unknown Source)
at org.springframework.amqp.rabbit.support.DefaultMessagePropertiesConverter.fromMessageProperties(DefaultMessagePropertiesConverter.java:208)
... 8 more
I'd be interested to see the complete stack trace for more information about the problem.
This code was part of a transition from a byte[] correlation Id to a String. This was needed to avoid a byte[]/String/byte[] conversion.
When the policy is String, you should use the correlationIdString property instead of correlationId. Otherwise, the correlationId won't be mapped in outbound messages (we don't look at correlationId in that case). For inbound messages it controls which property is populated.
In 2.0 and later, correlationId is now a String instead of a byte[] so this setting is no longer needed.
EDIT
Now I see the stack trace, this...
template.setEncoding("gzip");
...is wrong.
/**
* The encoding to use when inter-converting between byte arrays and Strings in message properties.
*
* #param encoding the encoding to set
*/
public void setEncoding(String encoding) {
this.encoding = encoding;
}
There is no such Charset as gzip. This property has nothing to do with the message content, it is simply used when converting byte[] to/from String. It is UTF-8 by default.

JMS Message Persistence on ActiveMQ

I need to ensure redelivery of JMS messages when the consumer fails
The way the producer is set up now - DefaultJmsListenerContainerFactory and Session.AUTO_ACKNOWLEDGE
I'm trying to build a jar and try in here to save the message into the server, once the app is able to consume, the producer in the jar will produce the message to the app.
Is that a good approach to do so?! any other way/recommendation to improve this?
public void handleMessagePersistence(final Object bean) {
ObjectMapper mapper = new ObjectMapper();
final String beanJson = mapper.writeValueAsString(bean); // I might need to convert to xml instead
// parameterize location of persistence folder
writeToDriver(beanJson);
try {
Producer.produceMessage(beanJson, beanJson, null, null, null);
} catch (final Exception e) {
LOG.error("Error producing message ");
}
}
here what I have to writ out the meesage:
private void writeToDriver(String beanJson) {
File filename = new File(JMS_LOCATION +
LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS")) + ".xml");
try (final FileWriter fileOut = new FileWriter(filename)) {
try (final BufferedWriter out = new BufferedWriter(fileOut)) {
out.write(beanJson);
out.flush();
}
} catch (Exception e) {
LOG.error("Unable to write out : " + beanJson, e);
}
}

#Transactional rabbit listener cause infinite loop

Hi I have problem with my Rabbit listener which cause infinite loop on exception (requeue message). My configuration looks:
#Bean(name = "defContainer")
public RabbitListenerContainerFactory containerFactory(ConnectionFactory connectionFactory, PlatformTransactionManager transactionManager){
SimpleRabbitListenerContainerFactory containerFactory = new SimpleRabbitListenerContainerFactory();
containerFactory.setConnectionFactory(connectionFactory);
containerFactory.setConcurrentConsumers(5);
containerFactory.setAcknowledgeMode(AcknowledgeMode.AUTO);
containerFactory.setTransactionManager(transactionManager);
containerFactory.setMessageConverter(messageConverterAmqp());
containerFactory.setDefaultRequeueRejected(false);
return new TxRabbitListenerContainerFactory(containerFactory);
}
where transactionManager is JpaTransactionManager for transaction on postgre db.
TxRabbitListenerContainerFactory is my factory which set setAlwaysRequeueWithTxManagerRollback to false:
public class TxRabbitListenerContainerFactory implements RabbitListenerContainerFactory {
private SimpleRabbitListenerContainerFactory factory;
public TxRabbitListenerContainerFactory(SimpleRabbitListenerContainerFactory factory) {
this.factory = factory;
}
#Override
public MessageListenerContainer createListenerContainer(RabbitListenerEndpoint endpoint) {
SimpleMessageListenerContainer container = factory.createListenerContainer(endpoint);
container.setAlwaysRequeueWithTxManagerRollback(false);
return container;
}
}
Now I have listner like:
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = "topic.two", durable = "true"),
exchange = #Exchange(value = "topic.def", type = "topic", durable = "true"),
key = "letter.*"
), errorHandler = "rabErrorHandler", containerFactory = "defContainer")
#Transactional
public Motorcycle topicLetters(Motorcycle motorcycle) throws Exception{
motorcycle.setId(UUID.randomUUID().toString());
Testing testing = new Testing();
testingRepository.save(testing);
throwEx();
return motorcycle;
}
where method throwEx(); throw unchecked exception.
Data from DB are properly rollbacked (not commited), but message are constantly requeued, see it in listener:
#Bean
public RabbitListenerErrorHandler rabErrorHandler(){
return new RabbitListenerErrorHandler() {
#Override
public Object handleError(Message message, org.springframework.messaging.Message<?> message1, ListenerExecutionFailedException e) throws Exception {
System.out.println("FFFFFFFFFFF");
return null;
}
};
}
How to prevvent infinite loope, and why is it happend ?
EDIT:
Logs: pasted logs
Set defaultRequeueRejected to false on the container factory.
To programmatically decide when to requeue or nor, leave that at true and throw an AmqpRejectAndDontRequeueException when you don't want it requeued.
EDIT
There's something not adding up...
protected void prepareHolderForRollback(RabbitResourceHolder resourceHolder, RuntimeException exception) {
if (resourceHolder != null) {
resourceHolder.setRequeueOnRollback(isAlwaysRequeueWithTxManagerRollback() ||
RabbitUtils.shouldRequeue(isDefaultRequeueRejected(), exception, logger));
}
}
If both booleans are false, we don't requeue.
Found issue:
Reason: caused by errorHandler handler which was specified on listener level. In some cases error handler return null - which was causing infinite loop (instead rethrowing exception and rollback transaction)

Spring amqp delay messaging with rabbitMQ

I am struggling hard to find out the way for scheduled/Delaying messages in Spring AMQP/Rabbit MQ and found solution in here.But i still with a prolem
about Spring AMQP/Rabbit MQ which can not received any message.
My source as the following:
#Configuration
public class AmqpConfig {
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setAddresses("172.16.101.14:5672");
connectionFactory.setUsername("admin");
connectionFactory.setPassword("admin");
connectionFactory.setPublisherConfirms(true);
return connectionFactory;
}
#Bean
#Scope("prototype")
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
return template;
}
#Bean
CustomExchange delayExchange() {
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
return new CustomExchange("my-exchange", "x-delayed-message", true, false, args);
}
#Bean
public Queue queue() {
return new Queue("spring-boot-queue", true);
}
#Bean
Binding binding(Queue queue, Exchange delayExchange) {
return BindingBuilder.bind(queue).to(delayExchange).with("spring-boot-queue").noargs();
}
#Bean
public SimpleMessageListenerContainer messageContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory());
container.setQueues(queue());
container.setExposeListenerChannel(true);
container.setMaxConcurrentConsumers(1);
container.setConcurrentConsumers(1);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
container.setMessageListener(new ChannelAwareMessageListener() {
public void onMessage(Message message, Channel channel) throws Exception {
byte[] body = message.getBody();
System.err.println("receive msg : " + new String(body));
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false); //确认消息成功消费
}
});
return container;
}
}
#Component
public class Send implements RabbitTemplate.ConfirmCallback{
private RabbitTemplate rabbitTemplate;
#Autowired
public Send(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
this.rabbitTemplate.setConfirmCallback(this);
rabbitTemplate.setMandatory(true);
}
public void sendMsg(String content) {
CorrelationData correlationId = new CorrelationData(UUID.randomUUID().toString());
rabbitTemplate.convertAndSend("my-exchange", "", content, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setHeader("x-delay", 6000);
return message;
}
},correlationId);
System.err.println("delay message send ................");
}
/**
* 回调
*/
#Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
System.err.println(" callback id :" + correlationData);
if (ack) {
System.err.println("ok");
} else {
System.err.println("fail:" + cause);
}
}
}
Is there someone could give a help.
Thanks all.
Delay messaging is nothing to do with Spring amqp, it's a library which will reside with your code, so the library can't hold any message as such. There are two approaches you can try:
Old Approach:
Set the TTL(time to live) header in each message/queue(policy) and then introduce a DLQ to handle it. once the ttl expired your messages will move from DLQ to main queue so that your listener can process it.
Latest Approach:
Recently RabbitMQ came up with RabbitMQ Delayed Message Plugin , using which you can achieve the same and this plugin support available since RabbitMQ-3.5.8.
You can declare an exchange with the type x-delayed-message and then publish messages with the custom header x-delay expressing in milliseconds a delay time for the message. The message will be delivered to the respective queues after x-delay milliseconds
Details:
To use the delayed-messaging feature, declare an exchange with the type x-delayed-message:
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);
Note that we pass an extra header called x-delayed-type, more on it under the Routing section.
Once we have the exchange declared we can publish messages providing a header telling the plugin for how long to delay our messages:
byte[] messageBodyBytes = "delayed payload".getBytes("UTF-8");
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("x-delay", 5000);
AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder().headers(headers);
channel.basicPublish("my-exchange", "", props.build(), messageBodyBytes);
byte[] messageBodyBytes2 = "more delayed payload".getBytes("UTF-8");
Map<String, Object> headers2 = new HashMap<String, Object>();
headers2.put("x-delay", 1000);
AMQP.BasicProperties.Builder props2 = new AMQP.BasicProperties.Builder().headers(headers2);
channel.basicPublish("my-exchange", "", props2.build(), messageBodyBytes2);
In the above example we publish two messages, specifying the delay time with the x-delay header. For this example, the plugin will deliver to our queues first the message with the body "more delayed payload" and then the one with the body "delayed payload".
If the x-delay header is not present, then the plugin will proceed to route the message without delay.
More here: git

Resources