we are using xml file to implement apache kafka using spring-integration. for now producer and consumer factories are called using #bean in xml file. but I want to read configurations from java class and use that class in the xml file. How do I do that for producer and consumer?
current version for producer:
<int:chain input-channel="kafka-output-channel">
<int:object-to-json-transformer/>
<int-kafka:outbound-channel-adapter id="kafkaOutboundChannelAdapter"
kafka-template="template"
topic="${topic.name}" />
</int:chain>
<bean id="pf" class="org.springframework.kafka.core.DefaultKafkaProducerFactory">
<constructor-arg>
<map>
<entry key="bootstrap.servers" value="${broker.list}" />
<entry key="key.serializer" value="org.apache.kafka.common.serialization.StringSerializer"/>
<entry key="value.serializer" value="org.apache.kafka.common.serialization.StringSerializer"/>
</map>
</constructor-arg>
</bean>
<!-- Kafka Template -->
<bean id="template" class="org.springframework.kafka.core.KafkaTemplate">
<constructor-arg ref="pf"/>
</bean>
But I want to create class like below and use beans below in the xml file,
#Configuration
public class KafkaProducerConfig {
#Bean
public KafkaTemplate<String, String> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory());
}
#Bean
public ProducerFactory<String, String> producerFactory() {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, appConfig.getBrokersList());
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
return new DefaultKafkaProducerFactory<>(props);
}
}
same thing I want to do for consumer part!
could someone please help?
Nothing is changed here. The XML and Java & Annotation configurations contributes to the same ApplicationContext and bean declared in one place can be references in the other, and wise versa. So, to use that kafkaTemplate bean in the XML, you just need to use exactly this name instead of that template and remove those bean definitions in the XML.
Only the problem that annotation configuration has to be primary: use an AnnotationConfigApplicationContext. Although I see that you use Spring Boot, so everything is OK. Your XML config has to be used on some #Configuration class as a #ImportResource. On the other hand if you have already enough experience with annotations configuration, there might not be a reason to stick with XML config even for Spring Integration. Consider to investigate Java DSL which smoothly replaces whatever you can have in XML: https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl
Please, also pay attention that there is an auto-configuration for Apache Kafka in Spring Boot. So you might not need to configure your template and produce/consumer factories: https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#messaging.kafka
Related
Secton 65.3 of the spring boot manual indicates that I can replace the default ObjectMapper by providing my own. I am not using boot, just a spring WebMVC application that builds to a .war and runs in tomcat.
It instantiates my ObjectMapper but doesn't use it. I used the debugger to trace through why timestamps still come out as numeric and found that it was using a different instance of ObjectMapper. It's not clear to me where it came from, or why this doesn't cause it to only use mine:
#Primary
#Bean
public ObjectMapper localObjectMapper() {
JodaMapper mapper = new JodaMapper();
mapper.setWriteDatesAsTimestamps(false);
mapper.getSerializationConfig().with(SerializationFeature.INDENT_OUTPUT)
.without(SerializationFeature.WRITE_DURATIONS_AS_TIMESTAMPS)
.without(SerializationFeature.WRITE_DATE_KEYS_AS_TIMESTAMPS)
.without(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
return mapper;
}
The above is in a #Configure bean that's definitely getting loaded.
The approach I took above worked fine in Spring 3, just not when I ugpraded to 4.2.2. I have read Jackson Integration Improvements as well, and tried approaches listed there, to the same effect.
--Chris
The way I always did it was:
#Configuration
#EnableWebMvc
public class MyWebMvcConfigurer extends WebMvcConfigurerAdapter {
#Bean
public ObjectMapper localObjectMapper() {
JodaMapper mapper = new JodaMapper();
// ...
return mapper;
}
#Override
public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
converters.add(new MappingJackson2HttpMessageConverter (localObjectMapper())); // use your own ObjectMapper
}
}
One warning, to quote the JavaDoc of WebMvcConfigurer.html#configureMessageConverters:
Note that adding converters to the list, turns off default converter registration. To simply add a converter without impacting default registration, consider using the method extendMessageConverters(java.util.List) instead.
In Spring 4 I've solved with the following xml-configuration
<bean name="jacksonObjectMapper"
class="org.springframework.http.converter.json.Jackson2ObjectMapperFactoryBean">
<property name="featuresToDisable">
<array>
<util:constant
static-field="com.fasterxml.jackson.databind.SerializationFeature.WRITE_DATES_AS_TIMESTAMPS" />
</array>
</property>
</bean>
<mvc:annotation-driven>
<mvc:message-converters register-defaults="true">
<bean
class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter">
<property name="objectMapper" ref="jacksonObjectMapper" />
</bean>
</mvc:message-converters>
</mvc:annotation-driven>
putting it in the Servlet configuration file, usually under
/WEB-INF/spring/appServlet/*.xml
In a redis java client,I found this:
To use it, init a pool:
JedisPool pool = new JedisPool(new JedisPoolConfig(), "localhost");
You can store the pool somewhere statically, it is thread-safe.
I just wondering,with spring,how can I store JedisPool statically.
You don't.
In spring it's preferable to define a JedisPool bean and autowire it wherever necessary.
For example, using xml config:
<bean id="jedisPool" class="redis.clients.jedis.JedisPool">
<constructor-arg>
<bean class="redis.clients.jedis.JedisPoolConfig" />
</consrtuctor-arg>
<constructor-arg value="localhost" />
</bean>
and then, inside your beans:
#Autowire
JedisPool jedisPool;
It's even simpler if you use spring java config - you can use exactly the code you posted to define the pool bean:
#Configuration
public class Configuration {
#Bean
public JedisPool createJedisPool() {
return new JedisPool(new JedisPoolConfig(), "localhost");
}
}
Also you might want to take a look at spring-data - redis
How to configure multiple remote activemq brokers (different IP address) in spring context? Below is the configuration for 1 remote broker. I am using camel to create routes that produce and consume messages from and to different queues in multiple remote brokers. Based on the following routes, how do the system knows which remote broker each queue belongs to?
List item
from("direct:start").to("activemq:queue:outgoingRequests")
List item
from("activemq:queue:incomingOrders").to("log:Events?
showAll=true").to("bean:jmsService")
Spring context for 1 broker
org.camel.routes
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://10.1.11.97:61616" />
</bean>
<bean id="pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory" init-
method="start" destroy-method="stop">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="pooledConnectionFactory"/>
<property name="concurrentConsumers" value="10"/>
</bean>
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig"/>
</bean>
Just add more components with different names
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig"/>
</bean>
<bean id="activemq2" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="myOtherJmsConfig"/>
</bean>
Then simply use the names:
<from uri="activemq:queue:MY.QUEUE"/><!-- get from "1st" ActiveMQ -->
<to uri="activemq2:queue:MY.QUEUE"/> <!-- put to same queue name on other ActiveMQ -->
Actually, you can call them whatever you want, like "EuropeanMarketBroker" or whatever fits in.
I have been trying to achieve this with the difference that my spring configuration is not in xml. It is helpful to know that you can achieve the same outcome by using spring annotations in a few ways.
The key to achieving this is registering the component with the desired name.
For example:
camelContext.addComponent("activemq2", jmsComponentInstance);
There is two ways of achieving this. Namely by creating two beans with qualifiers which identifies them from each other and then wiring those beans and registering them as components. Alternatively (this is preferable) you can create the bean and register the component all at once. Below are examples of both:
1 - Create Bean and Register elsewhere
#Configuration
public class ClassA{
#Bean #Qualifier("activemq2") public JmsComponent createJmsComponent(){
return JmsComponent.jmsComponentAutoAcknowledge(..);//Initialise component from externalised configs
}
}
#Component
public class ClassB{
#Autowired private CamelContext camelContext;
#Autowired #Qualifier("activemq2")
private JmsComponent jmsComponent;
public void someMethod(){
camelContext.addComponent("activemq2", jmsComponent);
}
}
2 - Create Bean and Register in one place within your #Configuration bean.
#Bean #Autowired public JmsComponent createJmsComponent(CamelContext camelContext){
JmsComponent component = JmsComponent.jmsComponentAutoAcknowledge(..);//Initialise component from externalised configs
camelContext.addComponent("activemq2", component);//Add Component to camel context
return component;//Return component instance
}
I addition of the two answers, here is my working solution with the latest SpringBoot using dedicated properties for both broker:
First, I define two Beans for each ConnectionFactory:
// gatewayRouterProperties is a java `record` mapped to the application.yml property file.
// One ConnectionFactory for the onPremise broker
#Bean
public ConnectionFactory jmsConnectionFactoryOnPrem() {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(gatewayRouterProperties.activeMq().brokerOnPrem().url());
activeMQConnectionFactory.setUserName(gatewayRouterProperties.activeMq().brokerOnPrem().user());
activeMQConnectionFactory.setPassword(gatewayRouterProperties.activeMq().brokerOnPrem().pass());
return activeMQConnectionFactory;
}
// Another broker ConnectionFactory for the cloud AWS broker
#Bean
public ConnectionFactory jmsConnectionFactoryAws() {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(gatewayRouterProperties.activeMq().brokerAws().url());
activeMQConnectionFactory.setUserName(gatewayRouterProperties.activeMq().brokerAws().user());
activeMQConnectionFactory.setPassword(gatewayRouterProperties.activeMq().brokerAws().pass());
return activeMQConnectionFactory;
}
And then I just define the two Beans ActiveMQComponent (the same as Peter's answer but using annotations):
#Bean(name = "activemq")
public ActiveMQComponent createActiveMQComponentOnPrem() {
ActiveMQConfiguration amqConfig = new ActiveMQConfiguration();
amqConfig.setConnectionFactory(jmsConnectionFactoryOnPrem());
return new ActiveMQComponent(amqConfig);
}
#Bean(name = "activemq2")
public ActiveMQComponent createActiveMQComponentAws() {
ActiveMQConfiguration amqConfig = new ActiveMQConfiguration();
amqConfig.setConnectionFactory(jmsConnectionFactoryAws());
return new ActiveMQComponent(amqConfig);
}
Note that I am using the bean name attribute and no need to add that manually in CamelContext.
After in my Camel route I just use my activemq components beans like this:
// 'activemq' component => AMQ on-prem
// 'activemq2' component => AMQ AWS
from("activemq:queue:QUEUE.TO.SYNC.TO.AWS")
.routeId("gw-router-route-on-prem-to-aws")
.autoStartup("{{autostart-enabled}}")
.to("activemq2:queue:QUEUE.FROM.ON.PREM")
;
What is difference between these 2 Spring Annotation and XML config
1) Annotation Based
#Configuration
#EnableWebMvc
public class MyWebConfig{
//
}
2) XML Based
<mvc:annotation-driven />
I can not see any other difference than xml and annotation.
and When to use which one ?
#Treydone wrote some examples plus expressed subjective opinion about Java based config being better.
I disagree with this statement, because there is no functional difference between Java based configuration and XML configuration, it's only matter of habit which one will you use. Some say traditional XML namespace config is better, others say Java based configuration (which is in Spring since 3.0) is the next level of IoC in Spring.
BTW Annotation based configuration is not the same as Java based one - you wrote example from the latter, so I assume you are choosing between XML and Java configs.
I think you should read:
Spring reference manual about Java based config basic concepts and Enabling MVC
some blog posts: this and this
and then decide which one is the best for you.
P.S. Annotation based configuration is IMO worse choice than these two as it moves some depedency information directly into ordinary classes.
Annotations based configuration is easier and more readable to build than the equivalent in xml. For instance setting a property as map in xml looks like this:
<property name="maps">
<map>
<entry key="Key 1" value="1" />
<entry key="Key 2" value-ref="PersonBean" />
<entry key="Key 3">
<bean class="com.mkyong.common.Person">
<property name="name" value="mkyongMap" />
<property name="address" value="address" />
<property name="age" value="28" />
</bean>
</entry>
</map>
</property>
In a java configuration file, this looks like this:
Map<String, Object> maps = ...
maps.put()...
....setMaps(maps);
There are many others pros:
Add a bean from an instance of an anonymous inner type
See the errors during the compilation, before starting your Spring context and your tomcat...
Add some conditions in your bean construction
For instance:
#Bean
public ViewResolver internalResourceViewResolver() {
ClassLoader classLoader = getClass().getClassLoader();
if (ClassUtils.isPresent("org.apache.tiles.TilesContainer", classLoader)) {
TilesViewResolver viewResolver = new TilesViewResolver();
return viewResolver;
} else {
InternalResourceViewResolver viewResolver = new InternalResourceViewResolver();
viewResolver.setPrefix("/");
viewResolver.setSuffix(".jsp");
return viewResolver;
}
}
Many others....
I think I've read every question and answer on Spring and autowiring a servlet, both here and at springsource.org, and I still can't get it working.
All I want to do is have the datasource automatically set in my servlets. I understand that the container creates the servlet and not Spring.
Here is code from my test servlet:
package mypackage.servlets;
imports go here...
#Service
public class TestServlet extends HttpServlet
{
private JdbcTemplate _jt;
#Autowired
public void setDataSource(DataSource dataSource)
{
_jt = new JdbcTemplate(dataSource);
}
etc etc
In my applicationContext.xml I have:
<context:annotation-config />
<context:component-scan base-package="mypackage.servlets />
<import resource="datasource.xml" />
and in my datasource.xml:
<jee:jndi-lookup id="dataSource" jndi-name="java:comp/env/jdbc/db" />
If I can't get this working I'll just use WebApplicationContextUtils in the servlet's init method but I'd really like to make this work after all the reading I've been doing.
I'm using Spring 3, Java 1.6.
Thanks,
Paul
You need to replace your Servlets by Spring MVC contollers. Because Spring will not inject anything the classes (servlets) created by someone else then Spring itselfe (except #Configurable).
(To get an very simple example, take a look at the STS Spring Template Project: MVC).
What I wanted to do was get a DataSource reference in my Servlet for free, i.e. not calling a static getDatasource method on some class.
Here's what I learned and how I got it working:
Servlets cannot be configured or autowired by Spring. Servlets are created before Spring's app context is loaded. See issue SPR-7801: https://jira.springsource.org/browse/SPR-7801
What I did was create a DataSource in my applicationContext.xml and export that as a property:
<jee:jndi-lookup id="dataSource" jndi-name="java:comp/env/jdbc/db" />
<bean class="org.springframework.web.context.support.ServletContextAttributeExporter">
<property name="attributes">
<map>
<entry key="myDatasource">
<ref bean="dataSource"/>
</entry>
</map>
</property>
</bean>
In my servlet's init method I read the property:
public void init(ServletConfig config)
{
Object obj = config.getServletContext().getAttribute("myDatasource");
setDataSource((DataSource)obj);
}
public void setDataSource(DataSource datasource)
{
// do something here with datasource, like
// store it or make a JdbcTemplate out of it
}
If I'd been using DAOs instead of hitting the database from the servlets it would have been easy to wire them up for #Autowired by marking them #Configurable, and also be able to use #Transactional and other Spring goodies.