Spring Integration: connection to multiple MQ servers by config - spring-boot

I do have a Spring Boot 5 application and I also have it running against one IBM MQ server.
Now we want it to connect to three or more MQ servers. My intention is now to just add XY connection infos to the environment and then I get XY MQConnectionFactory beans and al the other beans that are needed for processing.
At the moment this is what I have:
#Bean
#Qualifier(value="MQConnection")
public MQConnectionFactory getIbmConnectionFactory() throws JMSException {
MQConnectionFactory factory = new MQConnectionFactory();
// seeting all the parameters here
return factory;
}
But this is quite static. Is there an elegant way of doing this?
I stumbled about IntegrationFlow. Is this a possibly working solution?
Thanks for all your tipps!
KR
Solution
Based on Artem Bilan's response I built this class.
#Configuration
public class ConnectionWithIntegrationFlowMulti {
protected static final Logger LOG = Logger.create();
#Value("${mq.queue.jms.sources.queue.queue-manager}")
private String queueManager;
#Autowired
private ConnectionConfig connectionConfig;
#Autowired
private SSLSocketFactory sslSocketFactory;
#Bean
public MessageChannel queureader() {
return new DirectChannel();
}
#Autowired
private IntegrationFlowContext flowContext;
#PostConstruct
public void processBeanDefinitionRegistry() throws BeansException {
Assert.notEmpty(connectionConfig.getTab().getLocations(), "At least one CCDT file locations must be provided.");
for (String tabLocation : connectionConfig.getTab().getLocations()) {
try {
IntegrationFlowRegistration theFlow = this.flowContext.registration(createFlow(tabLocation)).register();
LOG.info("Registered bean flow for %s with id = %s", queueManager, theFlow.getId());
} catch (JMSException e) {
LOG.error(e);
}
}
}
public IntegrationFlow createFlow(String tabLocation) throws JMSException {
LOG.info("creating ibmInbound");
return IntegrationFlows.from(Jms.messageDrivenChannelAdapter(getConnection(tabLocation)).destination(createDestinationBean()))
.handle(m -> LOG.info("received payload: " + m.getPayload().toString()))
.get();
}
public MQConnectionFactory getConnection(String tabLocation) throws JMSException {
MQConnectionFactory factory = new MQConnectionFactory();
// doing stuff
return factory;
}
#Bean
public MQQueue createDestinationBean() {
LOG.info("creating destination bean");
MQQueue queue = new MQQueue();
try {
queue.setBaseQueueManagerName(queueManager);
queue.setBaseQueueName(queueName);
} catch (Exception e) {
LOG.error(e, "destination bean: Error for integration flow");
}
return queue;
}
}

With Spring Integration you can create IntegrationFlow instances dynamically at runtime. For that purpose there is an IntegrationFlowContext with its registration() API. The returned IntegrationFlowRegistrationBuilder as a callback like:
/**
* Add an object which will be registered as an {#link IntegrationFlow} dependant bean in the
* application context. Usually it is some support component, which needs an application context.
* For example dynamically created connection factories or header mappers for AMQP, JMS, TCP etc.
* #param bean an additional arbitrary bean to register into the application context.
* #return the current builder instance
*/
IntegrationFlowRegistrationBuilder addBean(Object bean);
So, your MQConnectionFactory instances can be populated alongside with the other flow, used as references in the particular JMS components and registered as beans, too.
See more info in docs: https://docs.spring.io/spring-integration/docs/5.2.3.RELEASE/reference/html/dsl.html#java-dsl-runtime-flows

If you are fine with creating them statically, you can create the beans as you are now (each having a unique qualifier), but you can access them all dynamically in your services / components by having an #Autowired List<MQConnectionFactory> field or #Autowired Map<String, MQConnectionFactory> field. Spring will automatically populate the fields with all of the beans of type MQConnectionFactory
In the the Map implementation, the String will be the qualifier value.
If you also want to create the beans dynamically based on some properties, etc, it gets a little more complicated. You will need to look into something along the lines of instantiating beans at runtime

Related

spring-kafka: DefaultErrorHandler with DeadLetterPublishingRecoverer(BiFunction) not considered. No DL topic created

In my Spring Boot application using spring-kafka, I am trying to configure an error handler with 2 things:-
Retry message consumption failures a certain times (FixedBackOff) before publishing to a dead letter topic
Create a dead letter topic with a name of my choice
Using
// Version highlights
id 'org.springframework.boot' version '2.7.2'
...
implementation 'org.springframework.kafka:spring-kafka' // 2.8.8
Here is the code I am using based on what I read in Spring docs and reiterated in several articles online:
#Bean
public DefaultErrorHandler byteArrayDefaultErrorHandler(KafkaTemplate<String, byte[]> template) {
var recoverer =
new DeadLetterPublishingRecoverer(
template,
(record, e) -> new TopicPartition("%s.deadLetter".formatted(record.topic()), 0);
);
return new DefaultErrorHandler(recoverer, new FixedBackOff(3000, 3));
}
But the above bean is not considered/used. So, when consumption encounters a failure (currently simulating failure by throwing an exception),
the FixedBackOff is not considered but the default one with 10 attempts back to back is used.
No DL topic is created.
Currently, the consumer config class has minimal stuff:
#Bean public ConsumerFactory<String, byte[]> byteArrayConsumerFactory() { ... }
#Bean public ConcurrentKafkaListenerContainerFactory<String, byte[]> byteArrayListenerContainerFactory() { .. }
#Bean public DefaultErrorHandler byteArrayDefaultErrorHandler(KafkaTemplate<String, byte[]> template) { ...code pasted above... }
And the listener is as follows:
#KafkaListener(
topics = "${app.config.kafka.topic}",
containerFactory = "byteArrayListenerContainerFactory"
)
public void consumeMessage(ConsumerRecord<String, byte[]> record) { ... }
Am at a loss figuring out what I have missed or added something conflicting the wiring. Help figuring out is highly appreciated.
The error handler bean will only be wired in by boot if you are using Boot's auto configured container factory.
Since you are creating your own container factory bean...
#Bean public ConcurrentKafkaListenerContainerFactory<String, byte[]> byteArrayListenerContainerFactory() { .. }
...you must add the error handler yourself - see setCommonErrorHandler().
The framework does not automatically provision the dead letter topic; add a #Bean NewTopic dlt() { ... }.
https://docs.spring.io/spring-kafka/docs/current/reference/html/#configuring-topics

#KafkaListener separate filtering logic for each listener

I need to define a custom filtering strategy for each listener produced by the listener factory.
Currently, I'm using RecordFilterStrategy to do that:
#Bean
ConcurrentKafkaListenerContainerFactory<String, GenericRecord> kafkaListenerContainerFactoryProject() {
ConcurrentKafkaListenerContainerFactory<String, GenericRecord> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
factory.setRecordFilterStrategy(new RecordFilterStrategy<String, GenericRecord>() {
#Override
public boolean filter(ConsumerRecord<String, GenericRecord> consumerRecord) {
return true;
}
});
return factory;
}
But such filtering applies to all listeners produced by this factory. What I need is something like to define the different logic for each listener:
#Component
#SendTo("out")
#KafkaListener(topics = "incoming")
public class TestListener {
#Filter
public boolean filter(){
return true;
}
#KafkaHandler
public TestObject listener(TestObject testObject) {
log.debug("Received Message: " + testObject);
return testObject;
}
}
Does spring-kafka have some tools to do that? Or I need to write such logic on my own?
Thanks in advance!
No, you don't. What you just need is a set of ConcurrentKafkaListenerContainerFactory beans with particular RecordFilterStrategy. Then your #KafkaListener should just specify which factory they are based on:
/**
* The bean name of the {#link org.springframework.kafka.config.KafkaListenerContainerFactory}
* to use to create the message listener container responsible to serve this endpoint.
* <p>If not specified, the default container factory is used, if any.
* #return the container factory bean name.
*/
String containerFactory() default "";

Spring JMS HornetQ user is null

I am trying to connect to a remote HornetQ broker in a spring boot/spring jms application and setup a #JmsListener.
HornetQ ConnectionFactory is being fetched from JNDI registry that HornetQ instance hosts. Everything works fine as long as HornetQ security is turned off but when it is turned on I get this error
WARN o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'jms/MI/Notification/Queue' - trying to recover. Cause: User: null doesn't have permission='CONSUME' on address jms.queue.MI/Notification/Queue
I ran a debug session to figure out that ConnectionFactory instance being returned is HornetQXAConnectionFactory but user and password fields are not set, which I believe is why user is null. I verified that user principal and credentials are set in JNDI properties but somehow it is not being passed on to ConnectionFactory instance. Any help on how I can get this setup working would be greatly appreciated.
This is my jms related config
#Configuration
#EnableJms
public class JmsConfig {
#Bean
public JmsListenerContainerFactory<?> jmsListenerContainerFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setDestinationResolver(destinationResolver());
return factory;
}
#Bean // Serialize message content to json using TextMessage
public MessageConverter jacksonJmsMessageConverter() {
MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter();
converter.setTargetType(MessageType.BYTES);
converter.setTypeIdPropertyName("_type");
return converter;
}
#Value("${jms.jndi.provider.url}")
private String jndiProviderURL;
#Value("${jms.jndi.principal}")
private String jndiPrincipal;
#Value("${jms.jndi.credentials}")
private String jndiCredential;
#Bean
public JndiTemplate jndiTemplate() {
Properties env = new Properties();
env.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
env.put("java.naming.provider.url", jndiProviderURL);
env.put("java.naming.security.principal", jndiPrincipal);
env.put("java.naming.security.credentials", jndiCredential);
return new JndiTemplate(env);
}
#Bean
public DestinationResolver destinationResolver() {
JndiDestinationResolver destinationResolver = new JndiDestinationResolver();
destinationResolver.setJndiTemplate(jndiTemplate());
return destinationResolver;
}
#Value("${jms.connectionfactory.jndiname}")
private String connectionFactoryJNDIName;
#Bean
public JndiObjectFactoryBean connectionFactoryFactory() {
JndiObjectFactoryBean jndiObjectFactoryBean = new JndiObjectFactoryBean();
jndiObjectFactoryBean.setJndiTemplate(jndiTemplate());
jndiObjectFactoryBean.setJndiName(connectionFactoryJNDIName);
jndiObjectFactoryBean.setResourceRef(true);
jndiObjectFactoryBean.setProxyInterface(ConnectionFactory.class);
return jndiObjectFactoryBean;
}
#Bean
public ConnectionFactory connectionFactory(JndiObjectFactoryBean connectionFactoryFactory) {
return (ConnectionFactory) connectionFactoryFactory.getObject();
}
}
JNDI and JMS are 100% independent as they are completely different specifications implemented in potentially completely different ways. Therefore the credentials you use for your JNDI lookup do not apply to your JMS resources. You need to explicitly set the username and password credentials on your JMS connection. This is easy using the JMS API directly (e.g. via javax.jms.ConnectionFactory#createConnection(String username, String password)). Since you're using Spring you could use something like this:
#Bean
public ConnectionFactory connectionFactory(JndiObjectFactoryBean connectionFactoryFactory) {
UserCredentialsConnectionFactoryAdapter cf = new UserCredentialsConnectionFactoryAdapter();
cf.setTargetConnectionFactory((ConnectionFactory) connectionFactoryFactory.getObject());
cf.setUsername("yourJmsUsername");
cf.setPassword("yourJmsPassword");
return cf;
}
Also, for what it's worth, the HornetQ code-base was donated to the Apache ActiveMQ project three and a half years ago now and it lives on as the Apache ActiveMQ Artemis broker. There's been 22 releases since then with numerous new features and bug fixes. I strongly recommend you migrate if at all possible.
Wrap the connection factory in a UserCredentialsConnectionFactoryAdapter.
/**
* An adapter for a target JMS {#link javax.jms.ConnectionFactory}, applying the
* given user credentials to every standard {#code createConnection()} call,
* that is, implicitly invoking {#code createConnection(username, password)}
* on the target. All other methods simply delegate to the corresponding methods
* of the target ConnectionFactory.
* ...

Dynamic bean update once it has instantiated during the Spring Boot start up

I am setting up a bean during the spring boot application startup. I am trying to update the bean using a rest endpoint. The end point in the controller calls the updatePoints(). When I retrieve the data using GET point it still has only the data that was instantiated during the startup. It does not have the updated data inside the bean.
#Component
public class DynamicEntry{
private Map<String, DynamicPoint> dynamicPoints = new HashMap<>();
private DefaultListableBeanFactory beanFactory;
#Autowired
public DynamicEntry(DefaultListableBeanFactory beanFactory){
this.beanFactory = beanFactory;
}
#PostConstruct
void loadPoints(){
//load the dynamicPoints after the spring boots up
}
void updatePoints(String point){
try {
if (!dynamicPoints.containsKey(point)) {
DynamicPoint dynamicPoint = new DynamicPoint(point);
beanFactory.registerSingleton(point, dynamicPoint);
dynamicPoints(point, dynamicPoint);
}
} catch (Exception | Error e) {
e.printStackTrace();
}
}
#Bean
public Map<String, DynamicPoint> dynamicPoints() {
return dynamicPoints;
}
}
You can try refreshing with ConfigurableApplicationContext.refresh():
beanFactory.registerSingleton(point, dynamicPoint);
beanFactory.refresh();
but there will be side effects, e.g. recreation of existing singletons which may lead to your application downtime or response not being sent back to client.
What you are trying to achieve is very non standard. By design singleton bean definitions are processed during startup. You should rethink your approach and don't create singleton beans on the fly. Maybe you need a request or session scoped bean?

NoUniqueBeanDefinitionException in Spring annotation driven configuration

I am getting the following error when trying to autowire two beans using
No qualifying bean of type [javax.jms.ConnectionFactory] is defined:
expected single matching bean but found 2: aConnectionFactory, bConnectionFactory
Description:
Parameter 1 of method jmsListenerContainerFactory in org.springframework.boot.autoconfigure.jms.JmsAnnotationDrivenConfiguration required a single bean, but 2 were found:
- aConnectionFactory: defined by method 'aConnectionFactory' in package.Application
- bConnectionFactory: defined by method 'bConnectionFactory' in package.Application
Action:
Consider marking one of the beans as #Primary, updating the consumer to accept multiple beans, or using #Qualifier to identify the bean that should be consumed
I have this annotation driven configuration:
#SpringBootApplication
#EnableIntegration
#IntegrationComponentScan
public class Application extends SpringBootServletInitializer implements
WebApplicationInitializer {
#Resource(name = "aConnectionFactory")
private ConnectionFactory aConnectionFactory;
#Resource(name = "bConnectionFactory")
private ConnectionFactory bConnectionFactory;
#Bean
public IntegrationFlow jmsInboundFlow() {
return IntegrationFlows
.from(
Jms.inboundAdapter(aConnectionFactory)
.destination(aQueue),
e -> e.poller( Pollers.fixedRate(100,
TimeUnit.MILLISECONDS).maxMessagesPerPoll(100))
).channel("entrypoint")
.get();
}
#Bean
public IntegrationFlow jmsInboundFlowB() {
return IntegrationFlows
.from(
Jms.inboundAdapter(bConnectionFactory)
.destination(bQueue),
e -> e.poller( Pollers.fixedRate(100,
TimeUnit.MILLISECONDS).maxMessagesPerPoll(100))
).channel("entrypoint")
.get();
}
#Bean(name = "aConnectionFactory")
#Profile({"weblogic"})
public ConnectionFactory aConnectionFactory() {
ConnectionFactory factory = null;
JndiTemplate jndi = new JndiTemplate();
try {
factory = (ConnectionFactory) jndi.lookup("jms/ConnectionFactory");
} catch (NamingException e) {
logger.error("NamingException for jms/ConnectionFactory", e);
}
return factory;
}
#Bean(name = "bConnectionFactory")
#Profile({"weblogic"})
public ConnectionFactory bConnectionFactory() {
ConnectionFactory factory = null;
JndiTemplate jndi = new JndiTemplate();
try {
factory = (ConnectionFactory) jndi.lookup("jms/ConnectionFactory");
} catch (NamingException e) {
logger.error("NamingException for jms/ConnectionFactory", e);
}
return factory;
}
}
Any ideas what's wrong in this code? This seems to be straight forward, but specifying the Qualifier doesn't work, I have also tried to use #Resource. What am I missing there?
Any help appreciated.
Nothing wrong with your code.
That is just JmsAnnotationDrivenConfiguration from Spring Boot which doesn't like your two ConnectionFactory beans, but requires only one.
Why just don't follow with that report recommendations and mark one of them with the #Primary?
Looks like you don't use Spring Boot JMS auto-configuration feature, so that would be just straightforward to disable JmsAnnotationDrivenConfiguration: http://docs.spring.io/spring-boot/docs/1.4.1.RELEASE/reference/htmlsingle/#using-boot-disabling-specific-auto-configuration
The problem consist
javax.jms.ConnectionFactory is singleton, you need one object that type!
Solutions for your problem:
If you need two object that create objects and extend ConnectionFactory
them change scope as needed.
try #Scope("singleton") or #Scope("prototype").
if you receive error, make a objects. then use a scope #Scope("singleton")
"Other Two" disfigure the other class that is already using and setting such an.

Resources