I have a hierarchy of messages. The topmost message in the hierarchy is defined with #JsonTypeInfo and #JsonSubTypes. The class is not under my control. I extended the hierarchy with my own message and to allow Jackson to deserialize my message, customized ObjectMapper. By debugging, I figured out that I need to register subtypes to the "jacksonObjectMapper" bean. So my code is as follows:
#Component
public class JacksonConfig implements BeanPostProcessor {
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (beanName.equals("jacksonObjectMapper")) {
((ObjectMapper) bean).registerSubtypes(new NamedType(ExtendedMessage.class, "ExtendedMessage"));
}
return bean;
}
}
The solution works, but I'm afraid it's a workaround and it won't work after Spring update (e.g. the bean name changes). Is there a documented way to customize the ObjectMapper that is used to deserialize messages?
Construct the deserializer yourself and pass it into the consumer factory, via a constructor or setter.
https://docs.spring.io/spring-kafka/docs/current/reference/html/#prog-json
If you are using Boot's auto-configured factory, use something like this
#Bean
JsonDeserializer deser(ConsumerFactory<Object, Object> cf) {
...
cf.setDeserializer(...);
}
Related
I am using spring.
I have a configured ObjectMapper for the entire project and I use it to set up a kafka deserializer.
And then I need a custom kafka deserializer to be used in KafkaListener.
I'm configuring KafkaListener via autoconfiguration, not via #Configuration class.
#Component
#RequiredArgsConstructor
public class CustomMessageDeserializer implements Deserializer<MyMessage> {
private final ObjectMapper objectMapper;
#SneakyThrows
#Override
public MyMessage deserialize(String topic, byte[] data) {
return objectMapper.readValue(data, MyMessage.class);
}
}
If i do like this
#KafkaListener(
topics = {"${topics.invite-user-topic}"},
properties = {"value.deserializer=com.service.deserializer.CustomMessageDeserializer"}
)
public void receiveInviteUserMessages(MyMessage myMessage) {}
I received KafkaException: Could not find a public no-argument constructor
But with public no-argument constructor in CustomMessageDeserializer class i am getting NPE because ObjectMapper = null. It creates and uses a new class, not a spring component.
#KafkaListener supports SpEL expressions.
And I think that this problem can be solved using SpEL.
Do you have any idea how to inject spring bean CustomMessageDeserializer with SpEL?
There are no easy ways to do it with SPeL.
Analysis
To get started, see the JavaDoc for #KafkaListener#properties:
/**
*
* SpEL expressions must resolve to a String ...
*/
The value of value.deserializer is used to instantiate the specified deserializer class. Let's follow the call chain:
You specify this value in the #KafkaListener annotation, then you are probably not creating a bean of the ConsumerFactory.class. So Spring creates this bean class itself - see KafkaAutoConfiguration#kafkaConsumerFactory.
Next is the creation of the returned object new DefaultKafkaConsumerFactory(...) as ConsumerFactory<?,?> using the constructor for default delivery expressions keyDeserializer/valueDeserializer = () -> null
This factory is used to create a Kafka consumer (The entry point is the constructor KafkaMessageListenerContainer#ListenerConsumer, then KafkaMessageListenerContainer.this.consumerFactory.createConsumer...)
In the KafkaConsumer constructor, the valueDeserializer object is being created, because it is null (for the default factory of point 2 above):
if (valueDeserializer == null) {
this.valueDeserializer = config.getConfiguredInstance(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, Deserializer.class);
The implementation of config.getConfiguredInstance involves instantiating your deserializer class via a parameterless constructor using reflection and your String "com.service.deserializer.CustomMessageDeserializer" class name
Solutions
To use value.deserializer with your customized ObjectMapper, you must create the ConsumerFactory bean yourself using the setValueDeserializer(...) method. This is also mentioned in the second Important part of the JSON.Mapping_Types.Important documentation
If you don't want to create a ConsumerFactory bean, and also don't have complicated logic in your deserializer (you only have return objectMapper.readValue(data, MyMessage.class);), then register DefaultKafkaConsumerFactoryCustomizer:
#Bean
// inject your custom objectMapper
public DefaultKafkaConsumerFactoryCustomizer customizeJsonDeserializer(ObjectMapper objectMapper) {
return consumerFactory ->
consumerFactory.setValueDeserializerSupplier(() ->
new org.springframework.kafka.support.serializer.JsonDeserializer<>(objectMapper));
}
In this case, you don't need to create your own CustomMessageDeserializer class (remove it) and Spring will automatically parse the message into your MyMessage.
#KafkaListener annotation should also not contains the property properties = {"value.deserializer=com.my.kafka_test.component.CustomMessageDeserializer"}. This DefaultKafkaConsumerFactoryCustomizer bean will automatically be used to configure the default ConsumerFactory<?, ?> (see the implementation of the KafkaAutoConfiguration#kafkaConsumerFactory method)
Here how it works for me:
#KafkaListener(topics = "${solr.kafka.topic}", containerFactory = "batchFactory")
public void listen(List<SolrInputDocument> docs, #Header(KafkaHeaders.BATCH_CONVERTED_HEADERS) List<Map<String, Object>> headers, Acknowledgment ack) throws IOException {...}
And then I have 2 beans defined in my Configuration
#Profile("!test")
#Bean
#Autowired
public ConsumerFactory<String, SolrInputDocument> consumerFactory(KafkaProperties properties) {
Map<String, Object> props = properties.buildConsumerProperties();
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
DefaultKafkaConsumerFactory<String, SolrInputDocument> result = new DefaultKafkaConsumerFactory<>(props);
String validatedKeyDeserializerName = KafkaMessageType.valueOf(keyDeserializerName).toString();
ZiDeserializer<SolrInputDocument> deserializer = ZiDeserializerFactory.getInstance(validatedKeyDeserializerName);
result.setValueDeserializer(deserializer);
return result;
}
#Profile("!test")
#Bean
#Autowired
public ConcurrentKafkaListenerContainerFactory<String, SolrInputDocument> batchFactory(ConsumerFactory<String, SolrInputDocument> consumerFactory) {
ConcurrentKafkaListenerContainerFactory<String, SolrInputDocument> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory);
factory.setBatchListener(true);
factory.setConcurrency(2);
ExponentialBackOffWithMaxRetries backoff = new ExponentialBackOffWithMaxRetries(10);
backoff.setMultiplier(3); // Default is 1.5 but this seems more reasonable
factory.setCommonErrorHandler(new DefaultErrorHandler(null, backoff));
// Needed for manual commits
factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL_IMMEDIATE);
return factory;
}
Note that the interface ZiDeserializer<SolrInputDocument> deserializeris my interface and ZiDeserializerFactory.getInstance(validatedKeyDeserializerName); returns my custom implementation of ZiDeserializer. And ZiDeserializer extends org.apache.kafka.common.serialization.Deserializer. This works for me
I'm trying to inject values from the application.properties file in a camel processor but it returns null. I also tried adding the #Component annotation but this breaks the application.
public class MyProcessor implements Processor {
#Value("${myProperty.path}")
public String path;
public void process(Exchange exchange) throws Exception {
System.out.println(path);
}
}
I'm new to both camel and spring. What could I do to read properties from the application.properties file in the processor class?
Property injection with #Value only works with container managed beans.
Therefore, you are on the right track with #Component, but I suspect (as mentioned in the comments) that your MyProcessor is not such a bean.
If you do one of these in your Camel route, then your Processor is NOT such a bean.
.process(MyProcessor.class)
.process(new MyProcessor())
Instead you have to annotate your Processor with #Component, hold an instance variable of it in your Camel Route class and then reference the instance.
.process(myProcessorInstance) <-- variable
The verion of spring boot which I use is 2.1.5.RELEASE.
My project work with redis.For security,I encrypt my redis password.I set value in my application.properties as follows：
spring.redis.password=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
I want to decrypt before spring bean's init,so I want to change the value of RedisProperties's passowrd property.So I customize a BeanPostProcesser like this:
#Component
public class PasswordBeanPostProcessor implements BeanPostProcessor {
#Autowired
private Cryptor cryptor;
#Value("${spring.redis.password}")
private String password;
#Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
log.info("beanName = {}",beanName);
if (bean instanceof RedisProperties) {
RedisProperties redisPropertiesBean = (RedisProperties) bean;
try {
redisPropertiesBean.setPassword(cryptor.decrypt(password));
log.debug(redisPropertiesBean.getPassword());
return redisPropertiesBean;
} catch (Exception ex) {
log.error("redis password decrypt error", ex);
throw new RuntimeException(ex);
}
}
return bean;
}
}
But this didnot work well,when I run my application ,there is no log like this print:
beanName = redisProperties
To make sure there is a bean named redisProperties in my applicationContext,I inject bean RedisProperties to another Bean.It work well ,I can get properties in RedisProperties.
To make my application run success with encrypt password,I decrypt redis's password in another's #PostConstruct method.But I think this way is not graceful,what is the right way?
who can help me，please
Ok, I understood that jasypt can't be used.
However, take a look at its source code which is fairly simple, given the fact you already work with Bean Post Processors which is a fairly advanced stuff in spring / spring boot
It's starter (autoconfig module) is available Here
So you'll see that it has spring.factories that enable some bootstrap and autoconfigurations.
Eventually you'll come to the code that actually handles the encryption
It uses a bean factory post processor - something that kicks in when the bean definitions are ready but the actual beans have not been created yet. This is a hook that micht be relevant to you. Of course the implementation will be different but the "orchestration" is the same...
I am having difficulty with Spring-Camel getting a HeaderFilterStrategy class registered as a Bean so it can be found by the Camel Route. My attempts to annotate the HeaderFilterStrategy custom class seem futile... so how do I register this thing so it gets found at run time?
I have a camel application with a route utilizing a custom HeaderFilterStrategy
The Strategy Class looks like :
public class HeaderFilter implements HeaderFilterStrategy {
#Override
public boolean applyFilterToCamelHeaders(String s, Object o, Exchange exchange) {
return false;
}
#Override
public boolean applyFilterToExternalHeaders(String s, Object o, Exchange exchange) {
return true;
}
}
I register it with camel using a simple registry:
SimpleRegistry registry = new SimpleRegistry();
registry.put("HeaderFilter" ,new HeaderFilter());
.
.
final CamelContext ctx = new DefaultCamelContext(registry);
And I reference it in my Route in
.to("https://myhost/endpoint&headerFilterStrategy=#HeaderFilter")
And all like Ralphy on Christmas night with his trusty Red Rider BB Gun, all is right with the world.
So, now I am trying to take this pure camel app and put it under Spring. I make sure all the appropriate Camel, and Spring-Camel and Spring things are imported.. However, when I attempt to annotate my HeaderStrategy as a Bean for Spring and it fails:
#Component
public class HeaderFilter implements HeaderFilterStrategy {
#Bean
#Override
public boolean applyFilterToCamelHeaders(String s, Object o, Exchange exchange) {
return false;
}
#Override
public boolean applyFilterToExternalHeaders(String s, Object o, Exchange exchange) {
return true;
}
}
Now when I do this, the IDE basically tells me it can't autowire any of the parameters in the method calls becaue there is more than one bean of type String or Object and no beans of type Exchange found..
At Runtime, Camel does attempt to interpret the route, but throws a failure with "No Qualifying bean type of "java.lang.String" available, since this is the first parameter in the method call...
So, How do I get this thing to be able register with annotations correctly? Or manually register this bean without it attempting to autowire? All I need is the class to be registered as a BEAN so it can be found by camel at runtime... Or at least that is what I understand needs to happen... so how the heck to I do this?
I figured it out, I was not properly using the annotationsI added the following to my AppConfig class:
#Configuration
public class AppConfig{
#Bean
public HeaderFilter HeaderFilter(){
return new HeaderFilter();
}
}
I am not sure if the suggestion above will work, but this clearly does.
I'm trying to create a BeanPostProcessor for registering some values to a Map.
The BeanPostProcessor works fine if I'm create the bean instance via xml definition, but if I change the bean definition to #Configuration it is not working.
PostProcessor
public class InstantiationTracingBeanPostProcessor implements BeanPostProcessor {
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
System.out.println("Bean '" + beanName );
return bean;
}
}
Bean Configuration
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
#org.springframework.context.annotation.Configuration
public class Configuration {
#Bean
public #Qualifier("InstantiationTracingBeanPostProcessor")
InstantiationTracingBeanPostProcessor activitiConfigurationBeanPostProcessor() {
return new InstantiationTracingBeanPostProcessor();
}
}
Component scan Configuration
<context:component-scan base-package="xyz.config"/>
<context:annotation-config/>
The application just hangs if I use the above configuration. But if I use xml based configuration as given below it works fine.
<bean class="xyz.bean.InstantiationTracingBeanPostProcessor"/>
What am I doing wrong here?
I got a solution to this after asking another question in spring forum.
The bean factory method should be defined as a static method to make it work.
I thought the way to do this was to annotate your BeanPostProcessor with Component:
#Component
public class InstantiationTracingBeanPostProcessor implements BeanPostProcessor {
//...
}
Which would be automatically handled by Spring during component scan (so no need to add a #Bean-annotated method to the configuration).