I am using Spring AMQP's MessageListenerContainer for recieving messages from RabbitMq Broker . Though I am able to receive message inside the listener , autowiring is not working inside listener .
Here is how I have configured my Listener
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(this.inputQueueMgr
.getRabbitConnectionFactory());
JsonMessageConverter converter = new JsonMessageConverter();
listenerContainer.setMessageConverter(converter);
listenerContainer.setMessageListener(new InputQueueEventDispatcher());
listenerContainer.setQueueNames("queue1");
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
listenerContainer.setPrefetchCount(1);
return listenerContainer;
}
Here is the class where I am listening to the messages from rabbitMq
#Component(value = "inputMessageListner")
public class InputQueueEventDispatcher implements Serializable, MessageListener {
private static final long serialVersionUID = -5391659256992655430L;
#Autowired
private volatile InputQueueManagerImpl inputQueueMgr;
#Autowired
private volatile NotificationQueueManagerImpl notificationManager;
#Value("${input.rabbitmq.events.queue}")
private String queueName;
#Autowired
private SubscriptionRepository subRepository;
#SuppressWarnings({ "unchecked" })
#Override
public void onMessage(Message message) {
try {
String messageContent = new String(message.getBody());
.....
}
The problem is inside onMessage(Message message) all the autowire components are coming as null .
PS-> Please note that I have declared all the autowire instances as #Component and doing appropriate ComponentScan to scan their packages appropriately . Infact these components do get autowired in normal flow but since onMessage(Message message) gets executed on a seperate thread , these values are showing null . Is there any way to enable autowiring here inside listener .
Thanks
You've set a #Component annotation on your listener, but you don't get this bean from the context. Instead, you're creating the instance yourself, using new. So Spring has no way to know that this instance has been created and must be autowired.
Remove that #Component annotation, and change your code to
#Bean
public SimpleMessageListenerContainer listenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setConnectionFactory(this.inputQueueMgr
.getRabbitConnectionFactory());
JsonMessageConverter converter = new JsonMessageConverter();
listenerContainer.setMessageConverter(converter);
listenerContainer.setMessageListener(inputMessageListener());
listenerContainer.setQueueNames("queue1");
listenerContainer.setAcknowledgeMode(AcknowledgeMode.AUTO);
listenerContainer.setPrefetchCount(1);
return listenerContainer;
}
#Bean
public InputQueueEventDispatcher inputMessageListener() {
return new InputQueueEventDispatcher();
}
Now, since the bean is returned from a #Bean-annotated method, Spring will make is a Spring bean and autowire it.
Most probably the other thread you mentioned on your question is not getting that instance from spring's application context. So no injection happens for it.
You should use
#RabbitListener(queues = "queue_name")
on your onMessage method.
But you should also change onMessage method syntax and and
onMessage(Message<String> originalMessage)
that way spring will automatically call that method with message
Related
I am using spring.
I have a configured ObjectMapper for the entire project and I use it to set up a kafka deserializer.
And then I need a custom kafka deserializer to be used in KafkaListener.
I'm configuring KafkaListener via autoconfiguration, not via #Configuration class.
#Component
#RequiredArgsConstructor
public class CustomMessageDeserializer implements Deserializer<MyMessage> {
private final ObjectMapper objectMapper;
#SneakyThrows
#Override
public MyMessage deserialize(String topic, byte[] data) {
return objectMapper.readValue(data, MyMessage.class);
}
}
If i do like this
#KafkaListener(
topics = {"${topics.invite-user-topic}"},
properties = {"value.deserializer=com.service.deserializer.CustomMessageDeserializer"}
)
public void receiveInviteUserMessages(MyMessage myMessage) {}
I received KafkaException: Could not find a public no-argument constructor
But with public no-argument constructor in CustomMessageDeserializer class i am getting NPE because ObjectMapper = null. It creates and uses a new class, not a spring component.
#KafkaListener supports SpEL expressions.
And I think that this problem can be solved using SpEL.
Do you have any idea how to inject spring bean CustomMessageDeserializer with SpEL?
There are no easy ways to do it with SPeL.
Analysis
To get started, see the JavaDoc for #KafkaListener#properties:
/**
*
* SpEL expressions must resolve to a String ...
*/
The value of value.deserializer is used to instantiate the specified deserializer class. Let's follow the call chain:
You specify this value in the #KafkaListener annotation, then you are probably not creating a bean of the ConsumerFactory.class. So Spring creates this bean class itself - see KafkaAutoConfiguration#kafkaConsumerFactory.
Next is the creation of the returned object new DefaultKafkaConsumerFactory(...) as ConsumerFactory<?,?> using the constructor for default delivery expressions keyDeserializer/valueDeserializer = () -> null
This factory is used to create a Kafka consumer (The entry point is the constructor KafkaMessageListenerContainer#ListenerConsumer, then KafkaMessageListenerContainer.this.consumerFactory.createConsumer...)
In the KafkaConsumer constructor, the valueDeserializer object is being created, because it is null (for the default factory of point 2 above):
if (valueDeserializer == null) {
this.valueDeserializer = config.getConfiguredInstance(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, Deserializer.class);
The implementation of config.getConfiguredInstance involves instantiating your deserializer class via a parameterless constructor using reflection and your String "com.service.deserializer.CustomMessageDeserializer" class name
Solutions
To use value.deserializer with your customized ObjectMapper, you must create the ConsumerFactory bean yourself using the setValueDeserializer(...) method. This is also mentioned in the second Important part of the JSON.Mapping_Types.Important documentation
If you don't want to create a ConsumerFactory bean, and also don't have complicated logic in your deserializer (you only have return objectMapper.readValue(data, MyMessage.class);), then register DefaultKafkaConsumerFactoryCustomizer:
#Bean
// inject your custom objectMapper
public DefaultKafkaConsumerFactoryCustomizer customizeJsonDeserializer(ObjectMapper objectMapper) {
return consumerFactory ->
consumerFactory.setValueDeserializerSupplier(() ->
new org.springframework.kafka.support.serializer.JsonDeserializer<>(objectMapper));
}
In this case, you don't need to create your own CustomMessageDeserializer class (remove it) and Spring will automatically parse the message into your MyMessage.
#KafkaListener annotation should also not contains the property properties = {"value.deserializer=com.my.kafka_test.component.CustomMessageDeserializer"}. This DefaultKafkaConsumerFactoryCustomizer bean will automatically be used to configure the default ConsumerFactory<?, ?> (see the implementation of the KafkaAutoConfiguration#kafkaConsumerFactory method)
Here how it works for me:
#KafkaListener(topics = "${solr.kafka.topic}", containerFactory = "batchFactory")
public void listen(List<SolrInputDocument> docs, #Header(KafkaHeaders.BATCH_CONVERTED_HEADERS) List<Map<String, Object>> headers, Acknowledgment ack) throws IOException {...}
And then I have 2 beans defined in my Configuration
#Profile("!test")
#Bean
#Autowired
public ConsumerFactory<String, SolrInputDocument> consumerFactory(KafkaProperties properties) {
Map<String, Object> props = properties.buildConsumerProperties();
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
DefaultKafkaConsumerFactory<String, SolrInputDocument> result = new DefaultKafkaConsumerFactory<>(props);
String validatedKeyDeserializerName = KafkaMessageType.valueOf(keyDeserializerName).toString();
ZiDeserializer<SolrInputDocument> deserializer = ZiDeserializerFactory.getInstance(validatedKeyDeserializerName);
result.setValueDeserializer(deserializer);
return result;
}
#Profile("!test")
#Bean
#Autowired
public ConcurrentKafkaListenerContainerFactory<String, SolrInputDocument> batchFactory(ConsumerFactory<String, SolrInputDocument> consumerFactory) {
ConcurrentKafkaListenerContainerFactory<String, SolrInputDocument> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory);
factory.setBatchListener(true);
factory.setConcurrency(2);
ExponentialBackOffWithMaxRetries backoff = new ExponentialBackOffWithMaxRetries(10);
backoff.setMultiplier(3); // Default is 1.5 but this seems more reasonable
factory.setCommonErrorHandler(new DefaultErrorHandler(null, backoff));
// Needed for manual commits
factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL_IMMEDIATE);
return factory;
}
Note that the interface ZiDeserializer<SolrInputDocument> deserializeris my interface and ZiDeserializerFactory.getInstance(validatedKeyDeserializerName); returns my custom implementation of ZiDeserializer. And ZiDeserializer extends org.apache.kafka.common.serialization.Deserializer. This works for me
I have a spring boot test to check if a kafka consumer listens for a message in specific topic. The kafka listener is triggered when using #SpringBootTest. But I just don't want to load all the classes and I only supplied the listener class like this #SpringBootTest(classes={KafkaConsumerTest.class}).
When only loading the consumer class, the listener has stopped to trigger. Is there something I am missing?
Here is the KafkaTestConsumer class
#Service
public class KafkaTestConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(KafkaTestConsumer.class);
private CountDownLatch latch = new CountDownLatch(1);
private String payload;
#KafkaListener(topics = {"topic"})
public void receive(ConsumerRecord<?, ?> consumerRecord) {
payload = consumerRecord.toString();
latch.countDown();
}
public CountDownLatch getLatch() {
return latch;
}
public void resetLatch() {
latch = new CountDownLatch(1);
}
public String getPayload() {
return payload;
}
}
It would be great to see what is your KafkaConsumerTest, but perhaps you just override the whole auto-configuration with a plain #Configuration.
See more in docs: https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.testing.spring-boot-applications.detecting-configuration
If you want to customize the primary configuration, you can use a nested #TestConfiguration class. Unlike a nested #Configuration class, which would be used instead of your application’s primary configuration, a nested #TestConfiguration class is used in addition to your application’s primary configuration.
I created a spring boot application that sends Messages through a PublishSubscribeChannel. This Channel is "autowired" as SubscribableChannel interface.
I am only subscribing one MessageHandler to this channel, a KafkaProducerMessageHandler.
My problem is that one additional MessageHandler is subscribed and this is an LoggingHandler. It is instantiated with ERROR level. So i see every message logged es error.
I want to know why and where this LoggingHandler is wired (instantiated) and why it is subscribed to the channel - i want to disable it.
(
I debugged around a bit but (was not really helpful):
The LoggingHandler is instantiated and subscribed after the KafkaHandler.
I see this chain EventdrivenConsumer.doStart()<-- ``ConsumerEndpointFactoryBean.initializeEndpoint()<-- ... until reflective calls
)
EDIT
As suggested in comments here is some code (i can't share the whole project). My problem is that the code can't explain the behavior. The LoggingHandler is beeing subscribed to my PublishSubscribeChannel for some unknown reason and it is instantiated with error as level for some unknown reason.
The class that subscribes the KafkaHandler:
#Component
public class EventRelay {
#Autowired
private EventRelay( SubscribableChannel eventBus, #Qualifier( KafkaProducerConfig.KAFKA_PRODUCER ) MessageHandler kafka ) {
eventBus.subscribe( kafka );
}
}
The class that send events is implementing an proprietary interface with many callback methods:
public class PropEvents implements PropClass.IEvents {
private SubscribableChannel eventBus;
private final ObjectMapper om;
private final String userId;
public PropEvents( SubscribableChannel eventBus, ObjectMapper om, String userId ) {
this.eventBus = eventBus;
this.om = om;
this.userId = userId;
}
#Override
public void onLogin( ) {
eventBus.send( new OnLoginMessage(... ) ) );
}
//many other onXYZ methods
}
Here is the Factory that produces instances of PropEvents:
#Configuration
public class EventHandlerFactory {
private final ObjectMapper om;
private final SubscribableChannel eventBus;
#Autowired
public EventHandlerFactory( ObjectMapper om, SubscribableChannel eventBus){
this.om = checkNotNull( om );
this.eventBus = checkNotNull( eventBus );
}
#Bean
#Scope( SCOPE_PROTOTYPE)
public IEvents getEvantHandler(String userId){
if(Strings.isNullOrEmpty(userId)){
throw new IllegalArgumentException( "user id must be set." );
}
return new PropEvents(eventBus, om, userId);
}
}
I appreciate any help with debugging or use tooling (e.g. Eclipse Spring tools does not show any hint to a LoggingHandler Bean) to find where and why a LoggingHandler is instantiated and subscribed to my autowired Channel.
My current workaround is to disable logging for LoggingHandler.
My question at a glance
Why spring instantiates an LoggingHandler with error level and subscribes it to my SubscribableChannel(provided by PublishSubscribeChannel)? How to disable this?
When you #Autowired SubscribableChannel, there should be one in the application context. That might be confusing a bit and mislead, but Spring Integration provides a PublishSubscribeChannel for the global errorChannel: https://docs.spring.io/spring-integration/docs/5.0.2.RELEASE/reference/html/messaging-channels-section.html#channel-special-channels
This one has a LoggingHandler to log error as a default subscriber.
I don't think that it is OK to make your logic based on the errorChannel.
You should consider to declare your own MessageChannel bean and inject it by the particular #Qualifier.
I am getting the following error when trying to autowire two beans using
No qualifying bean of type [javax.jms.ConnectionFactory] is defined:
expected single matching bean but found 2: aConnectionFactory, bConnectionFactory
Description:
Parameter 1 of method jmsListenerContainerFactory in org.springframework.boot.autoconfigure.jms.JmsAnnotationDrivenConfiguration required a single bean, but 2 were found:
- aConnectionFactory: defined by method 'aConnectionFactory' in package.Application
- bConnectionFactory: defined by method 'bConnectionFactory' in package.Application
Action:
Consider marking one of the beans as #Primary, updating the consumer to accept multiple beans, or using #Qualifier to identify the bean that should be consumed
I have this annotation driven configuration:
#SpringBootApplication
#EnableIntegration
#IntegrationComponentScan
public class Application extends SpringBootServletInitializer implements
WebApplicationInitializer {
#Resource(name = "aConnectionFactory")
private ConnectionFactory aConnectionFactory;
#Resource(name = "bConnectionFactory")
private ConnectionFactory bConnectionFactory;
#Bean
public IntegrationFlow jmsInboundFlow() {
return IntegrationFlows
.from(
Jms.inboundAdapter(aConnectionFactory)
.destination(aQueue),
e -> e.poller( Pollers.fixedRate(100,
TimeUnit.MILLISECONDS).maxMessagesPerPoll(100))
).channel("entrypoint")
.get();
}
#Bean
public IntegrationFlow jmsInboundFlowB() {
return IntegrationFlows
.from(
Jms.inboundAdapter(bConnectionFactory)
.destination(bQueue),
e -> e.poller( Pollers.fixedRate(100,
TimeUnit.MILLISECONDS).maxMessagesPerPoll(100))
).channel("entrypoint")
.get();
}
#Bean(name = "aConnectionFactory")
#Profile({"weblogic"})
public ConnectionFactory aConnectionFactory() {
ConnectionFactory factory = null;
JndiTemplate jndi = new JndiTemplate();
try {
factory = (ConnectionFactory) jndi.lookup("jms/ConnectionFactory");
} catch (NamingException e) {
logger.error("NamingException for jms/ConnectionFactory", e);
}
return factory;
}
#Bean(name = "bConnectionFactory")
#Profile({"weblogic"})
public ConnectionFactory bConnectionFactory() {
ConnectionFactory factory = null;
JndiTemplate jndi = new JndiTemplate();
try {
factory = (ConnectionFactory) jndi.lookup("jms/ConnectionFactory");
} catch (NamingException e) {
logger.error("NamingException for jms/ConnectionFactory", e);
}
return factory;
}
}
Any ideas what's wrong in this code? This seems to be straight forward, but specifying the Qualifier doesn't work, I have also tried to use #Resource. What am I missing there?
Any help appreciated.
Nothing wrong with your code.
That is just JmsAnnotationDrivenConfiguration from Spring Boot which doesn't like your two ConnectionFactory beans, but requires only one.
Why just don't follow with that report recommendations and mark one of them with the #Primary?
Looks like you don't use Spring Boot JMS auto-configuration feature, so that would be just straightforward to disable JmsAnnotationDrivenConfiguration: http://docs.spring.io/spring-boot/docs/1.4.1.RELEASE/reference/htmlsingle/#using-boot-disabling-specific-auto-configuration
The problem consist
javax.jms.ConnectionFactory is singleton, you need one object that type!
Solutions for your problem:
If you need two object that create objects and extend ConnectionFactory
them change scope as needed.
try #Scope("singleton") or #Scope("prototype").
if you receive error, make a objects. then use a scope #Scope("singleton")
"Other Two" disfigure the other class that is already using and setting such an.
Ciao, my basic requirement is to have a route where I can send a message and this is put on a JMS Queue. The camel context run in a JavaEE 6 container namely JBoss AS 7.1.1 so it's HornetQ for JMS which ships with it; I start the context via bootstrap singleton but I don't use the camel-cdi. So far I've been using camel-jms component, but now I'm looking to migrate to the camel-sjms if possible because springless.
My question is: what is the proper way to configure the ConnectionFactory for camel-sjms in this JavaEE scenario, please?
With the camel-jms I could put this in the endpoint URL, as simple as .to("jms:myQueue?connectionFactory=#ConnectionFactory"). With the camel-sjms instead it seems to me that I need to create an instance of the SJMSComponent myself, set the connectionFactory, and set this instance in the camel context before starting it.
I have code below for the camel-jms Vs camel-sjms case, and I would like to know if I "migrated" the setting of the ConnectionFactory correctly. Thanks.
For camel-jms this was done as:
#Singleton
#Startup
public class CamelBootstrap {
private CamelContext camelContext;
private ProducerTemplate producerTemplate;
public CamelContext getCamelContext() {
return camelContext;
}
public ProducerTemplate getProducerTemplate() {
return producerTemplate;
}
#PostConstruct
public void init() throws Exception {
camelContext = new DefaultCamelContext();
camelContext.addRoutes(new MyCamelRoutes());
camelContext.start();
producerTemplate = camelContext.createProducerTemplate();
}
}
Nothing special, and in the MyCamelRoutes I could do route configuration using:
.to("jms:myQueue?connectionFactory=#ConnectionFactory")
For camel-sjms now I have to modify the bootstrap singleton with:
#Singleton
#Startup
public class CamelBootstrap {
#Resource(mappedName="java:/ConnectionFactory")
private ConnectionFactory connectionFactory;
private CamelContext camelContext;
private ProducerTemplate producerTemplate;
public CamelContext getCamelContext() {
return camelContext;
}
public ProducerTemplate getProducerTemplate() {
return producerTemplate;
}
#PostConstruct
public void init() throws Exception {
camelContext = new DefaultCamelContext();
SjmsComponent sjms = new SjmsComponent();
sjms.setConnectionFactory(connectionFactory);
camelContext.addComponent("sjms", sjms);
camelContext.addRoutes(new MyCamelRoutes());
camelContext.start();
producerTemplate = camelContext.createProducerTemplate();
}
}
and please notice #Resource for the connectionFactory this is passed as a reference to the SjmsComponent instance, which is passed to the camelContext. And then in the MyCamelRoutes I could use the sjms while do route configuration using:
.to("sjms:myQueue")
The code seems to work correctly in both scenario, but as I understand the configuration of the ConnectionFactory is quite susceptible of performance issue if not done correctly, therefore I prefer to ask if I migrated to the camel-sjms correctly for my JavaEE scenario. Thanks again
Performance issues are likely to happend if you don't do caching/pooling of JMS resources. Caching is typically configured by wrapping a ConnectionFactory in some Caching ConnectionFactory library - or by handing over the control to the application server.
Camel SJMS includes built-in pooling. However, if you have a container managed resource to handle JMS connections, you should probably consider using it. SJMS has some facilities to deal with that, ConncetionResource instead of ConnectionFactory.