Jackson configuration to consume list of records in rabbitmq - spring

I am using spring boot amqp in which I will be consuming a list of Employee objects from a queue. My listener method looks like this:
#RabbitListener(queues = "emp_queue")
public void processAndPortEmployeeData(List<Employee> empList) {
empList.forEach(emp -> { some logic })
}
However, when I try to consume the message, I get a class cast exception: For some reason, I'm getting a LinkedHashMap.
Caused by: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to com.integration.domain.Employee
If I change my listener method to consume a single employee object, it works fine and I'm using the following jackson configurations for it:
#Configuration
#EnableRabbit
public class RabbitConfiguration implements RabbitListenerConfigurer {
#Bean
public MappingJackson2MessageConverter jackson2Converter() {
return new MappingJackson2MessageConverter();
}
#Bean
public DefaultMessageHandlerMethodFactory handlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(jackson2Converter());
return factory;
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(handlerMethodFactory());
}
}
Is there some other jackson configuration that I need to do to consume the list of employee objects?
Thanks a lot!
Sample Input Json message which I will be consuming:
[
{
"name" : "Jasmine",
"age" : "24",
"emp_id" : 1344
},
{
"name" : "Mark",
"age" : "32",
"emp_id" : 1314
}
]

What version of Spring AMQP are you using?
If 1.6 or greater, the framework passes the argument type to the message converter.
Before 1.6 you either need type information in the message headers, or you need to configure the converter with type information.
That said, since the converter created a map, it implies that was what received (not a list).
Please show a sample of the JSON in a message.
EDIT
Note that boot auto-configures the message converter if there's a single bean of that type...
#SpringBootApplication
public class So40491628Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(So40491628Application.class, args);
Resource resource = new ClassPathResource("data.json");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
FileCopyUtils.copy(resource.getInputStream(), baos);
context.getBean(RabbitTemplate.class).send("foo", MessageBuilder.withBody(baos.toByteArray())
.andProperties(MessagePropertiesBuilder.newInstance().setContentType("application/json").build()).build());
Thread.sleep(10000);
context.close();
}
#Bean
public Jackson2JsonMessageConverter converter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public Queue foo() {
return new Queue("foo");
}
#RabbitListener(queues = "foo")
public void listen(List<Employee> emps) {
System.out.println(emps);
}
public static class Employee {
private String name;
private String age;
private int empId;
public String getName() {
return this.name;
}
public void setName(String name) {
this.name = name;
}
public String getAge() {
return this.age;
}
public void setAge(String age) {
this.age = age;
}
public int getEmpId() {
return this.empId;
}
public void setEmpId(int empId) {
this.empId = empId;
}
#Override
public String toString() {
return "Employee [name=" + this.name + ", age=" + this.age + ", empId=" + this.empId + "]";
}
}
}
Result:
[Employee [name=Jasmine, age=24, empId=0], Employee [name=Mark, age=32, empId=0]]

Related

Not able to get error object in JSON format while using #Valid and MessageSource to get display errors in Spring boot

I am currently learning Spring REST and I am trying to build a demo spring boot app. Incase of DTO object has validation error I want to show it as below:
{
"errors": [
{
"code": "first_error_code",
"message": "1st error message"
"field":"field_name"
}
]
}
Where the code in above JSON should display the validation message that I have given in my entity class i.e
#NotEmpty(message = "{name.not.empty}")
String name;
then code should be name.not.empty and message should be taken from messages.properties file.
Now to achieve this, I used several tutorials. Below are the classes:
Main class: (Included MessageSource and LocalValidatorFactoryBean)
#SpringBootApplication
#EnableSwagger2
public class Demo3PathvariableApplication implements WebMvcConfigurer {
#Bean
public MessageSource messageSource() {
ReloadableResourceBundleMessageSource messageSource = new ReloadableResourceBundleMessageSource();
messageSource.setBasename("classpath:messages");
messageSource.setDefaultEncoding("UTF-8");
return messageSource;
}
#Bean
public LocalValidatorFactoryBean validator(MessageSource messageSource) {
LocalValidatorFactoryBean bean = new LocalValidatorFactoryBean();
bean.setValidationMessageSource(messageSource());
return bean;
}
public static void main(String[] args) {
SpringApplication.run(Demo3PathvariableApplication.class, args);
}
/*
* To enable matrix variables, configurePathMatch() method of WebMvcConfigurer
* needs to overriden. Matrix variables are disabled by default and the
* following configuration
*
* urlPathHelper.setRemoveSemicolonContent(false);
*
* should be present in the overriden method to enable the same. see below
* method.
*/
#Override
public void configurePathMatch(PathMatchConfigurer configurer) {
UrlPathHelper urlPathHelper = new UrlPathHelper();
urlPathHelper.setRemoveSemicolonContent(false);
configurer.setUrlPathHelper(urlPathHelper);
}
#Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping("/**").allowedMethods("GET", "POST");
}
/* For Swagger Document Generation */
#Bean
public Docket api() {
return new Docket(DocumentationType.SWAGGER_2).select()
.apis(RequestHandlerSelectors.basePackage("com.infytel.controller")).paths(PathSelectors.any()).build()
.useDefaultResponseMessages(false);
// To scan for RestControllers from this package and For disabling default
// response messages
}
}
Controller class:
#RestController
#RequestMapping("/customers")
#Api(value = "CustomerController, REST APIs that deal with Customer DTO")
public class CustomerController {
#Autowired
private CustomerService customerService;
#PostMapping(consumes = "application/json")
public ResponseEntity createCustomer(#RequestBody #Valid CustomerDTO customer, Errors errors) {
return ResponseEntity.ok(customerService.createCustomer(customer));
}
}
FieldErrorDTO.java:
public class FieldErrorDTO {
private String errorCode;
private String message;
private String field;
public FieldErrorDTO(String errorCode, String message, String field) {
this.errorCode = errorCode;
this.message = message;
this.field = field;
}
//Getter setter
ValidationErrorDTO.java:
public class ValidationErrorDTO {
private List<FieldErrorDTO> fieldErrors = new ArrayList<>();
public ValidationErrorDTO() {
super();
}
public void addFieldError(String errorCode, String message, String field) {
FieldErrorDTO error = new FieldErrorDTO(errorCode, message, field);
fieldErrors.add(error);
}
public List<FieldErrorDTO> getFieldErrors() {
return fieldErrors;
}
public void setFieldErrors(List<FieldErrorDTO> fieldErrors) {
this.fieldErrors = fieldErrors;
}
}
RestErrorHandler .java
#ControllerAdvice
public class RestErrorHandler {
#Autowired
private MessageSource messageSource;
#ResponseStatus(BAD_REQUEST)
#ResponseBody
#ExceptionHandler(MethodArgumentNotValidException.class)
public ValidationErrorDTO processValidationError(MethodArgumentNotValidException ex) {
BindingResult result = ex.getBindingResult();
List<org.springframework.validation.FieldError> fieldErrors = result.getFieldErrors();
return processFieldErrors(fieldErrors);
}
private ValidationErrorDTO processFieldErrors(List<FieldError> fieldErrors) {
ValidationErrorDTO dto = new ValidationErrorDTO();
for (FieldError fieldError : fieldErrors) {
String localizedErrorMessage = resolveLocalizedErrorMessage(fieldError);
dto.addFieldError(fieldError.getCode(), localizedErrorMessage, fieldError.getField());
}
return dto;
}
private String resolveLocalizedErrorMessage(FieldError fieldError) {
Locale currentLocale = LocaleContextHolder.getLocale();
String localizedErrorMessage = messageSource.getMessage(fieldError, currentLocale);
return localizedErrorMessage;
}
}
messages.properties
name.not.empty=Please provide a name.
email.not.valid=Please provide valid email id.
age.adult.only=Age should be more than 18.
Now with all these config, I am able to see below JSON,
{
"fieldErrors": [
{
"errorCode": "NotEmpty",
"message": "Please provide a name.",
"field": "name"
},
{
"errorCode": "Email",
"message": "Please provide valid email id.",
"field": "email"
}
]
}
How do I acheive this requirement, where instead of "errorCode": "NotEmpty", I want show
"errorCode": "name.not.empty"
From CustomerDTO class?
To do so you need to change you processFieldErrors:
First remove "{}" from your anotations:
#NotEmpty(message = "name.not.empty")
String name;
Second:
private ValidationErrorDTO processFieldErrors(List<FieldError> fieldErrors) {
ValidationErrorDTO dto = new ValidationErrorDTO();
for (FieldError fieldError : fieldErrors) {
String localizedErrorMessage = resolveLocalizedErrorMessage(fieldError);
dto.addFieldError(fieldError.getDefaultMessage(), localizedErrorMessage, fieldError.getField());
}
return dto;
}
And third, change your message.getMessage:
private String resolveLocalizedErrorMessage(FieldError fieldError) {
Locale currentLocale = LocaleContextHolder.getLocale();
String localizedErrorMessage = messageSource.getMessage(fieldError.getDefaultMessage(), null, currentLocale);
return localizedErrorMessage;
}
This way you would retrieve the key for the message. In your example it will be :
name.not.empty
Hope this helps

Why does CachePut not work in this example?

I am playing around with the Spring framework and I would like to get my name returned from the cache. After 5 seconds I will update the cache and I hope to receive a new name.... unfortunately this is not working.... why?
#Component
public class Test {
public String name = "peter";
#Cacheable(value = "numCache")
public String getName() {
return name;
}
#Scheduled(fixedRate = 5000)
#CachePut(value = "numCache")
public String setName() {
this.name = "piet";
return name;
}
}
#Component
public class AppRunner implements CommandLineRunner {
public void run(String... args) throws Exception {
Test test = new Test();
while(true) {
Thread.sleep(1000);
System.out.println(test.getName());
}
}
}
#SpringBootApplication
#EnableCaching
#EnableScheduling
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
You are creating an instance of Test yourself with new, you are not autowiring it. I would try like this:
#Component
public class Test {
public String name = "peter";
#Cacheable(value = "numCache")
public String getName() {
return name;
}
#Scheduled(fixedRate = 5000)
#CachePut(value = "numCache")
public String setName() {
this.name = "piet";
return name;
}
}
#Component
public class AppRunner implements CommandLineRunner {
#Autowired private Test test;
public void run(String... args) throws Exception {
while(true) {
Thread.sleep(1000);
System.out.println(test.getName());
}
}
}

Pass list of topics from application yml to KafkaListener

I have the following application.yml:
service:
kafka:
groupId: 345
consumer:
topics:
-
name: response
producer:
topics:
-
name: request1
num-partitions: 5
replication-factor: 1
-
name: request2
num-partitions: 3
replication-factor: 1
How can I access the list of topic names using spel for passing to KafkaListener annotation?
#KafkaListener(topics = "#{'${service.kafka.consumer.topics.name}'}", containerFactory = "kafkaListenerContainerFactory")
public void receive(String payload, #Header(KafkaHeaders.RECEIVED_TOPIC)String topic) {
Use configuration properties and collection projection...
#ConfigurationProperties("service.kafka.producer")
#Component
public class ConfigProps {
List<Topic> topics = new ArrayList<>();
public List<Topic> getTopics() {
return this.topics;
}
public void setTopics(List<Topic> topics) {
this.topics = topics;
}
#Override
public String toString() {
return "ConfigProps [topics=" + this.topics + "]";
}
public static class Topic {
private String name;
private int numPartitions;
private short replicationFactor;
public String getName() {
return this.name;
}
public void setName(String name) {
this.name = name;
}
public int getNumPartitions() {
return this.numPartitions;
}
public void setNumPartitions(int numPartitions) {
this.numPartitions = numPartitions;
}
public short getReplicationFactor() {
return this.replicationFactor;
}
public void setReplicationFactor(short replicationFactor) {
this.replicationFactor = replicationFactor;
}
#Override
public String toString() {
return "Topic [name=" + this.name + ", numPartitions=" + this.numPartitions + ", replicationFactor="
+ this.replicationFactor + "]";
}
}
}
and
#SpringBootApplication
public class So52741016Application {
public static void main(String[] args) {
SpringApplication.run(So52741016Application.class, args);
}
#KafkaListener(groupId = "${service.kafka.groupId}", topics = "#{configProps.topics.![name]}")
public void listener(String in) {
}
#Bean
public SmartLifecycle createTopics(KafkaAdmin admin, ConfigProps props) {
return new SmartLifecycle() {
#Override
public int getPhase() {
return Integer.MIN_VALUE;
}
#Override
public void stop() {
}
#Override
public void start() {
try (AdminClient client = AdminClient.create(admin.getConfig())) {
CreateTopicsResult createTopics = client.createTopics(props.topics.stream()
.map(t -> new NewTopic(t.getName(), t.getNumPartitions(), t.getReplicationFactor()))
.collect(Collectors.toList()));
createTopics.all().get();
}
catch (Exception e) {
// e.printStackTrace();
}
}
#Override
public boolean isRunning() {
return false;
}
#Override
public void stop(Runnable callback) {
}
#Override
public boolean isAutoStartup() {
return true;
}
};
}
}
and
2018-10-10 11:20:25.813 INFO 14979 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer : partitions assigned: [request1-4, request2-0, request1-0, request2-1, request1-1, request2-2, request1-2, request1-3]
Of course, this is only the producer topics, but you can handle them all this way.

Maybe not public or not valid? Using Spring's Websocket and Kafka

As I am trying to consume data from a topic (the topic name is based on user) and during runtime I am trying to consume message from the topic but I am getting the following error.
Caused by:
org.springframework.expression.spel.SpelEvaluationException: EL1008E:
Property or field 'consumerProperties' cannot be found on object of
type 'org.springframework.beans.factory.config.BeanExpressionContext'
- maybe not public or not valid?
Here is my code
#Service
public class kafkaConsumerService {
private SimpMessagingTemplate template;
KafkaConsumerProperties consumerProperties;
#Autowired
public kafkaConsumerService(KafkaConsumerProperties consumerProperties, SimpMessagingTemplate template) {
this.consumerProperties=consumerProperties;
this.template=template;
}
#KafkaListener(topics = {"#{consumerProperties.getTopic()}"})
// #KafkaListener(topics="Chandan3706")
public void consume(#Payload Message message) {
System.out.println("from kafka topic::" + message);
template.convertAndSend("/chat/getMessage", message);
}
}
My KafkaConsumerProperties.class
#Component
#ConfigurationProperties(prefix="kafka.consumer")
public class KafkaConsumerProperties {
private String bootStrap;
private String group;
private String topic;
public String getBootStrap() {
return bootStrap;
}
public void setBootStrap(String bootStrap) {
this.bootStrap = bootStrap;
}
public String getGroup() {
return group;
}
public void setGroup(String group) {
this.group = group;
}
public String getTopic() {
return topic;
}
public void setTopic(String topic) {
this.topic = topic;
}
#Override
public String toString() {
return "KafkaConsumerProperties [bootStrap=" + bootStrap + ", group=" + group + ", topic=" + topic + "]";
}
}
Thanks in advance
Since you don’t provide any bean name for your KafkaConsumerProperties component, the default one is de-capitalized class name. That’s one.
The expression you use in the #KafkaListener is regular bean definition phase expression, therefore a root object is some BeanExpressionContext , but not your listener bean as you try to get access through the property.
Not sure if you need that KafkaConsumerProperties property in this listener, but expression must ask for the kafkaConsumerProperties bean:
#Service
public class kafkaConsumerService {
private SimpMessagingTemplate template;
#Autowired
public kafkaConsumerService(SimpMessagingTemplate template) {
this.template=template;
}
#KafkaListener(topics = {"#{kafkaConsumerProperties.topic}"})
// #KafkaListener(topics="Chandan3706")
public void consume(#Payload Message message) {
System.out.println("from kafka topic::" + message);
template.convertAndSend("/chat/getMessage", message);
}
}
The following code worked for me, notice to #DependsOn("KafkaConsumerProperties") and #Component("KafkaConsumerProperties") annotations.
KafkaConsumerService class:
#Service
#DependsOn("KafkaConsumerProperties")
public class KafkaConsumerService {
#KafkaListener(topics = "#{#KafkaConsumerProperties.getTopic()}")
public void consume(#Payload Message message) {
System.out.println("from kafka topic::" + message);
template.convertAndSend("/chat/getMessage", message);
}
}
KafkaConsumerProperties class:
#Component("KafkaConsumerProperties")
#ConfigurationProperties(prefix="kafka.consumer")
public class KafkaConsumerProperties {
private String topic;
public String getTopic() {
return topic;
}
}

RabbitMQ separate listeners by type

I have POJO which represents a message to Rabbit MQ. There is an integer which is responsible for the type of the message(whether it's update, remove, add and so on):
public class Message {
private String field1;
private String field2;
private Integer type;
...
<some other fields>
}
I have a consumer which accepts such messages in my spring boot app. So in order to handle each type separately, I have to add some switch/case construction in my code.
Are there any more clear solutions for such case?
You can use Spring Integration with a router...
Rabbit Inbound channel adapter -> router ->
Where the router routes to a different service activator (method) based on the type.
EDIT
Here's an example:
#SpringBootApplication
public class So47272336Application {
public static void main(String[] args) {
SpringApplication.run(So47272336Application.class, args);
}
#Bean
public ApplicationRunner runner(RabbitTemplate rabbitTemplate) {
return args -> {
rabbitTemplate.convertAndSend("my.queue", new Domain(1, "foo"));
rabbitTemplate.convertAndSend("my.queue", new Domain(2, "bar"));
rabbitTemplate.convertAndSend("my.queue", new Domain(3, "baz"));
};
}
#Bean
public Queue queue() {
return new Queue("my.queue");
}
#Bean
public IntegrationFlow flow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, "my.queue"))
.route("payload.type", r -> r
.subFlowMapping("1", f -> f.handle("bean", "add"))
.subFlowMapping("2", f -> f.handle("bean", "remove"))
.subFlowMapping("3", f -> f.handle("bean", "update")))
.get();
}
#Bean
public MyBean bean() {
return new MyBean();
}
public static class MyBean {
public void add(Domain object) {
System.out.println("Adding " + object);
}
public void remove(Domain object) {
System.out.println("Removing " + object);
}
public void update(Domain object) {
System.out.println("Updating " + object);
}
}
public static class Domain implements Serializable {
private final Integer type;
private final String info;
public Domain(Integer type, String info) {
this.type = type;
this.info = info;
}
public Integer getType() {
return this.type;
}
public String getInfo() {
return this.info;
}
#Override
public String toString() {
return "Domain [type=" + this.type + ", info=" + this.info + "]";
}
}
}

Resources