Spring kafka transaction id is wrong at start? - spring-boot

I'm using spring kafka 2.2.2.RELEASE
Our broker has strict authorization verification.
Here is our client configuration :
spring.kafka.producer.transaction-id-prefix=test.
topic : topic
groupId: group
It should generate a transactional id : "test.group.topic0"
Problem:
When I start the application, KafkaMessageListenerContainer#onPartitionsAssigned is called and the transactional id used is "test0".
It seems like in this case, the kafkaTemplate is not considered to be in a listenerContainer.
This is leading to an Unauthorized exception because this transactional id is refused by the broker
Question
Is it normal ?

Thanks; it's a bug; I opened an issue.

Related

Starting Spring Boot application without check for Kafka Server

I have got an application that uses SpringBoot 2.10.0.Release and kafka in the version 2.10.0. The application has got a simple producer and consumer: The sender works with KafkaTemplate and the consumer with KafkaListener.
What I try to achieve is to be able to start the SpringBoot application even if the KafkaServer is not running.
Currently without a running KafkaBroker the application cannot be started with this error message:
org.springframework.context.ApplicationContextException:
Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry';
nested exception is org.apache.kafka.common.errors.TimeoutException
Is there a way to achieve this and if yes could anybody give me hint or a keyword how to manage this?
When running the Spring-Boot application with a KafkaListener, the listener will per default try to listen to Kafka. If the KafkaBroker is invalid or missing, then you will get a org.apache.kafka.common.KafkaException.
You can change the default behaviour of the container factory by setting the autoStartup property to false. One way to do this is by adding autoStartup = "false" element to your KafkaListener annotation:
#KafkaListener(topics = "some_topic", autoStartup = "false")
public void fooEventListener(){
Now your spring boot application will start. You will still get an error when trying to use the KafkaListener if the broker is still down or invalid, but you will now be able to handle the error within your Java code instead of a Spring Boot server crash.
Documentation about KafkaListner autoStartup element.
It have to be mentioned that the error you are receiving (TimeoutException) is not because the broker is down, it is what Kafka will throw if the buffer is full.
The batch records will then be removed from the queue and will not be delivered to the broker. This error will not be the reason for you application using Kafka not to start.

Can I get JMSXDeliveryCount and JMSDestination using org.springframework.messaging.Message

I am using Spring JMS 4.1.2, and I am using org.springframework.messaging.Message as the input payload in the JMS Listener. I am listening to an Active MQ Queue, and I would like to know if it is possible to obtain the JMSXDeliveryCount and JMSDestination using the org.springframework.messaging.Message, or if I have to use the javax.jms.Message class to accomplish this.
Thanks,
Juan
The delivery count property is mapped by default; I just ran a test...
message: GenericMessage [payload=jms test, headers={jms_redelivered=true, JMSXDeliveryCount=2, ...
Standard headers are mapped with a jms_ prefix all; message properties are mapped as-is.
The destination from which the message is received is not currently mapped to the message.
You would need to add a custom JmsHeaderMapper - perhaps a subclass of DefaultJmsHeaderMapper.
I opened a JIRA issue to make this standard.

How to handle transaction involving Spring message/JMS and Database

I have a method that get an invoice and it creates XML and send that XML to a JMS queue and then save the invoice to DB with updated status like 'invoiced'. Below is pseudo code that involves Spring and Hibernate. My question is: Is the failure in hibernate save rollsback Jms sending.or if JMS send failed, how can I roll back saving invoice status? is this comes under distributed transaction management. What are the transactional cases involved here. Thanks.
#Transactional(propagation=Propagation.Required)
void processInvoices(invoice ){
String xml = createXML(invoice);
messageService.sendInvoice(xml );
invoice.setStatus("invoiced");
save(invoice);
}
As per my knowledge and what I understand from your question you want to synchronize hibernate and JMS transaction, For doing this you will need to use JTA to manage transactions across the the both Hibernate and JMS
Read More # Spring synchronising Hibernate and JMS transactions

How to test WSO2 Message Broker with JMeter

I'm having some issues getting JMeter to work against the WSO2 Message Broker using the JMS Publisher. I had JMeter working against ActiveMQ but I'm still new with the tool.
I copied the client libraries over to jmeter wso2mb-2.0.1\client-lib to apache-jmeter-2.9\lib
andes-client-0.13.wso2v4.jar
geronimo-jms_1.1_spec-1.1.0.wso2v1.jar
Settings:
Context Factory : org.wso2.andes.jndi.PropertiesFileInitialContextFactory
Provider Url : amqp://admin:admin#clientID/carbon?brokerlist='tcp://localhost:5672'
Connection Factory : qpidConnectionfactory
...also tried several other values
Destination: dynamicQueues/test
The error I'm getting is on the Connection Factory field.
I've tried several different values all of which generate a naming error like there is a setting missing.
When I leave it blank I get:
javax.naming.NamingException: Expected javax.jms.ConnectionFactory, found org.wso2.andes.jndi.ReadOnlyContext
Does anyone know what I'm missing here?
Suspect it's something simple.
I found the problem.
In short the qpid context factory org.wso2.andes.jndi.PropertiesFileInitialContextFactory does not use fields the same way as the activeMQ context factory org.apache.activemq.jndi.ActiveMQInitialContextFactory.
While ActiveMQ allows you to not use a separate properties file with Jmeter, Qpid does not.
Jmeter JMS Publisher:
Context Factory : org.wso2.andes.jndi.PropertiesFileInitialContextFactory
Provider Url : nameOfYouFile.properties
Connection Factory : qpidConnectionfactory
Destination : <QueuePropertyName>
nameOfYouFile.properties:
connectionfactory.qpidConnectionfactory = amqp://admin:admin#clientID/carbon?brokerlist='tcp://localhost:5672'
queue.JMeterQueue = JMeterQueue
Reference:
Qpid Wiki

How to stop exception propagation if jms broker is down when sending a message with spring integration?

I configured spring xml based interceptor, which sends a jms message to activemq queue on each invokation of some transactional method after it is commited. It's happening with the following xml code.
<jms:outbound-channel-adapter channel="filteredStakesChannel" destination="stakesQueue" delivery-persistent="true" explicit-qos-enabled="true" />
But if the activemq server is down i get connection refused exception, which is propagated and i don't want this to happen even if the jms delivery fails. Is this possible?
Should i use some error-channel?
The simplest solution is to make fileredStakesChannel an Executor channel and the send will run on a different thread.
http://static.springsource.org/spring-integration/reference/html/messaging-channels-section.html#executor-channel
http://static.springsource.org/spring-integration/reference/html/messaging-channels-section.html#channel-configuration-executorchannel
Use the <task/> namespace to define an executor to use.

Resources