Spring Boot - RabbitMQ - Changing message content type from octet-stream to json - spring

I having issue getting from my Byte[] message received in RabbitMQ converted into some object based on a model.
I'm using Spring Boot.
In the application I have implemented RabbitListenerConfigurer:
#SpringBootApplication
class Application : RabbitListenerConfigurer {
...
override fun configureRabbitListeners(registrar: RabbitListenerEndpointRegistrar) {
registrar.messageHandlerMethodFactory = messageHandlerMethodFactory()
}
#Bean
fun messageHandlerMethodFactory(): MessageHandlerMethodFactory {
val messageHandlerMethodFactory = DefaultMessageHandlerMethodFactory()
messageHandlerMethodFactory.setMessageConverter(consumerJackson2MessageConverter())
return messageHandlerMethodFactory
}
#Bean
fun consumerJackson2MessageConverter(): MappingJackson2MessageConverter {
return MappingJackson2MessageConverter()
}
}
I then have a listener (the println is just for the time being for me to check) like:
#RabbitListener(queues = ["success.queue"])
fun receivedSuccessMessage(data: MyModel) {
println(data)
}
And the model itself looks like (again simplified just for testing):
#JsonIgnoreProperties(ignoreUnknown = true)
data class MyModel(
val input: Any,
val metadata: Any
)
Now whenever that queue gets a message the error i get is:
Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot convert from [[B] to [com.models.MyModel] for GenericMessage [payload=byte[1429]
Also to add the message I'm receving into that success queue looks like this:
{"input":"somestuff", "metadata": "somestuff"}
And I also just noticed that the message type is coming in as content_type: application/octet-stream. This is out of my control as thats just what I am receiving from the service that puts the message in there:
Any ideas why this is - I assumed that RabbitListenerConfigurer implementation I have does the conversion for all messages from byte array into whatever model I specify.
Any help/ideas would be appreciated.
Thanks.
UPDATE
Ok I got this working, however I'm quite confused about something:
This is what I added in:
#Autowired
lateinit var connectionFactory : ConnectionFactory
#Bean
fun rabbitListenerContainerFactory(): SimpleRabbitListenerContainerFactory {
val factory = SimpleRabbitListenerContainerFactory()
factory.setConnectionFactory(connectionFactory)
factory.setAfterReceivePostProcessors(SomeBusiness())
return factory
}
And then the actual conversion like this:
class SomeBusiness : MessagePostProcessor {
#Throws(AmqpException::class)
override fun postProcessMessage(message: Message): Message {
println("WE WENT IN NOW")
println()
if (message.messageProperties.contentType == "application/octet-stream") {
message.messageProperties.contentType = "application/json"
}
return message
}
}
However the thing I don't understand is I already have my messageHandlerMethodFactory function which I assumed did all the work around listening to messages. Now that I have rabbitListenerContainerFactory is there any concern of anything conflicting. Or is it perfectly valid, namely I need rabbitListenerContainerFactory as it implements SimpleRabbitListenerContainerFactory which gives me access to the method setAfterReceivePostProcessors.
Again, any explanation would really be appreciated.
Thanks

As you are receiving content_type: application/octet-stream and this can't changed.
You could use Spring Interceptor Or Servlet Filter to convert the content type application/octet-stream to application/json.
Example using HandlerInterceptor for Spring Boot Hope this helps. Thanks

Please find the below source which is working for me,
public void convertByteArrayToObject(Message<?> message) {
Object request = message.getPayload();
Object result = null;
try {
byte[] byteArray = (byte[]) request;
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(byteArray);
result = new ObjectInputStream(byteArrayInputStream).readObject();
} catch (ClassNotFoundException | IOException e) {
LOG.error(e.getMessage());
}
return result;
}

I had the same problem. Solution below helped me to solve it.
Solution:
Spring understand only "contentType", not "content_type". So, just set "contentType" property for your message, for example (in my case message producer was not spring app):
String json = ...;
TextMessage textMessage = session.createTextMessage(json);
textMessage.setStringProperty("contentType", "application/json");
Explanation:
Class MappingJackson2MessageConverter has method canConvertFrom(Message<?> message, #Nullable Class<?> targetClass). That method checks if headers contains supported mime types. And if to dive deeper, we will find that AbstractMessageConverter.getMimeType(MessageHeaders header) has contentTypeResolver, who gets appropriate property. Instance of that object located in CompositeMessageConverterFactory, it's DefaultContentResolver, which gets content type in method resolve(MessageHeaders headers):
Object contentType = headers.get(MessageHeaders.CONTENT_TYPE);
Where public static final String CONTENT_TYPE = "contentType";

Related

Feign ErrorDecoder is not invoked - how to configure feign to use it?

As i understand the decode() method of the feign ErrorDecoder will be called when a request responds with a status code != 2xx. Through debugging my tests i found out that the decode() method of my CustomErrorDecoder is not invoked on e.g. 504 or 404. I tried two ways to configure it:
Either include it as a Bean in the client configuration:
#Bean
public CustomErrorDecoder customErrorDecoder() {
return new CustomErrorDecoder();
}
or write it into the application configuration :
feign:
client:
config:
myCustomRestClientName:
retryer: com.a.b.some.package.CustomRetryer
errorDecoder: com.a.b.some.package.CustomErrorDecoder
Both ways don't invoke the ErrorDecoder. What am I doing wrong? The Bean is beeing instantiated and my CustomErrorDecoder looks like this:
#Component
public class CustomErrorDecoder implements ErrorDecoder {
private final ErrorDecoder defaultErrorDecoder = new Default();
#Override
public Exception decode(String s, Response response) {
Exception exception = defaultErrorDecoder.decode(s, response);
if (exception instanceof RetryableException) {
return exception;
}
if (response.status() == 504) {
// throwing new RetryableException to retry 504s
}
return exception;
}
}
Update:
I have created a minimal reproducible example in this git repo. Please look at the commit history to find 3 ways that I tried.
The problem is that your feign client uses feign.Response as the return type:
import feign.Param;
import feign.RequestLine;
import feign.Response;
public interface TestEngineRestClient {
#RequestLine(value = "GET /{uuid}")
Response getReport(#Param("uuid") String uuid);
}
In this case, Feign delegates its handling to the developer - e.g., you can retrieve HTTP status and a response body and do some stuff with it.
If interested, you can look at the source code of feign.SynchronousMethodHandler, executeAndDecode section.
To fix this, replace Response.class with the desired type in case of the correct response with status code = 2xx (probably some DTO class). I made a PR where I've changed it to String for simplicity.

MockMvc Test does not get to the endpoint for a Multipart file in a RestController

I am calling a service in an orders controller which receives a multipart file and processes it and saving it into a database. I am trying to create a Spring Rest Doc for it but it is not even hitting the endpoint. I am creating a list of orders which is what the service expects. It receives the order as a stream as shown and converts into a stream of orders before saving it into a database. I have shown the main part of the controller and my code for generating the rest docs. When I run the code I get the following exception, it never even hits the endpoint when I set a breakpoint. I also used fileupload() but that did not work either.
Exception is:
Content type = application/json
Body = {"path":"/orders/order_reception","exceptionName":
"MissingServletRequestPartException","message":"Required request part 'uploadFile' is not
present",
"rootExceptionName":"MissingServletRequestPartException",
"rootMessage":"MissingServletRequestPartException: Required request part 'uploadFile' is not present"}
#RestController
#RequestMapping(value = "/orders")
#Validated
class OrderController{
#PostMapping(path = "/order_reception")
public ResponseEntity receiveData(#RequestPart MultipartFile uploadFile,
HttpServletRequest request,
HttpServletResponse response) {
if (!uploadFile.isEmpty()) {
try {
Reader reader = new InputStreamReader(request.getInputStream()));
... save file
return new ResponseEntity<>(HttpStatus.HttpStatus.CREATED);
} catch (Exception e) {
return new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
}
}
return new ResponseEntity(HttpStatus.BAD_REQUEST);
}
#Test
public void sendData() throws Exception {
ObjectMapper mapper = new ObjectMapper();
Order order = repository.getOrder("1233333");
List<Order> orderList = new ArrayList<>():
resourceList.add(order);
MockMultipartFile orderFile = new MockMultipartFile("order-data", "order.json", "application/json",
mapper.writeValueAsString(orderList).getBytes(Charset.defaultCharset()));
mockMvc.perform(multipart("/orders/order_reception")
.file(orderFile))
.andExpect(status().isCreated())
.andDo(document("send-order",
preprocessRequest(prettyPrint()),
preprocessResponse(prettyPrint())));
}
Thank you Marten Deinum, your suggestion that the file name was wrong fixed it.
I simply changed name in the MockMultipartFile( "uploadsFile", ...)

How can I modify default Json error response Spring?

When an item that doesn't exist in my web app is invoked through an URL, Spring responds with a JSON with data like (timestand, status, error, message, path). So, I need to change the structure of this JSON, specificly I need to remove path.
How can I do it?
Where should I implement the customization of the exception in my project?
Best regards to everyone!
Json response to modify
It's pretty easy in Spring MVC applications to handle errors by their types using the #ContollerAdvice class.
You could define your own handler for the exceptions you get on a method calls.
E.g.:
#ControllerAdvice
public class ErrorHandler {
#ExceptionHandler(value = ExceptionToHandle.class)
#ResponseBody
public YourResponse handle(ExceptionToHandle ex) {
return new YourResponse(ex.getMessage());
}
}
Here YourResponse is just a POJO, that could have any structure your want to be presented at the client.
The #ExceptionHandler specifies what types of errors will be handled in the method (including more specific types).
The #ResponseBody says that your returned value will be presented in the JSON format in your response.
You may try something like that:
#RestController
#RequestMapping("/")
public class TestController {
#GetMapping("/exception")
void getException() {
throw new MyCustomException("Something went wrong!");
}
class MyCustomException extends RuntimeException {
MyCustomException(String message) {
super(message);
}
}
class CustomError {
private String message;
private Integer code;
CustomError(String message, Integer code) {
this.message = message;
this.code = code;
}
public String getMessage() {
return message;
}
public Integer getCode() {
return code;
}
}
#ExceptionHandler(MyCustomException.class)
public CustomError handleMyCustomException(Exception ex) {
return new CustomError("Oops! " + ex.getMessage(), HttpStatus.BAD_REQUEST.value());
}
}
Fast and simple, you can just make your own exception and your own error object (which is later turned to json).
If you ask where to put such a thing... Well, you can make a separate class for the exception (and an exception package also), and put a small #ExceptionHandler method inside your controller. If you don't want it to be in the same class, you may delegate it to separate class also; for further and in-depth reading, look up for annotation like #ControllerAdvice.

Can I use Spring WebFlux to implement REST services which get data through Kafka request/response topics?

I'm developing REST service which, in turn, will query slow legacy system so response time will be measured in seconds. We also expect massive load so I was thinking about asynchronous/non-blocking approaches to avoid hundreds of "servlet" threads blocked on calls to slow system.
As I see this can be implemented using AsyncContext which is present in new servlet API specs. I even developed small prototype and it seems to be working.
On the other hand it looks like I can achieve the same using Spring WebFlux.
Unfortunately I did not find any example where custom "backend" calls are wrapped with Mono/Flux. Most of the examples just reuse already-prepared reactive connectors, like ReactiveCassandraOperations.java, etc.
My data flow is the following:
JS client --> Spring RestController --> send request to Kafka topic --> read response from Kafka reply topic --> return data to client
Can I wrap Kafka steps into Mono/Flux and how to do this?
How my RestController method should look like?
Here is my simple implementation which achieves the same using Servlet 3.1 API
//took the idea from some Jetty examples
public class AsyncRestServlet extends HttpServlet {
...
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
String result = (String) req.getAttribute(RESULTS_ATTR);
if (result == null) { //data not ready yet: schedule async processing
final AsyncContext async = req.startAsync();
//generate some unique request ID
String uid = "req-" + String.valueOf(req.hashCode());
//share it to Kafka receive together with AsyncContext
//when Kafka receiver will get the response it will put it in Servlet request attribute and call async.dispatch()
//This doGet() method will be called again and it will send the response to client
receiver.rememberKey(uid, async);
//send request to Kafka
sender.send(uid, param);
//data is not ready yet so we are releasing Servlet thread
return;
}
//return result as html response
resp.setContentType("text/html");
PrintWriter out = resp.getWriter();
out.println(result);
out.close();
}
Here's a short example - Not the WebFlux client you probably had in mind, but at least it would enable you to utilize Flux and Mono for asynchronous processing, which I interpreted to be the point of your question. The web objects should work without additional configurations, but of course you will need to configure Kafka as the KafkaTemplate object will not work on its own.
#Bean // Using org.springframework.web.reactive.function.server.RouterFunction<ServerResponse>
public RouterFunction<ServerResponse> sendMessageToTopic(KafkaController kafkaController){
return RouterFunctions.route(RequestPredicates.POST("/endpoint"), kafkaController::sendMessage);
}
#Component
public class ResponseHandler {
public getServerResponse() {
return ServerResponse.ok().body(Mono.just(Status.SUCCESS), String.class);
}
}
#Component
public class KafkaController {
public Mono<ServerResponse> auditInvalidTransaction(ServerRequest request) {
return request.bodyToMono(TopicMsgMap.class)
// your HTTP call may not return immediately without this
.subscribeOn(Schedulers.single()) // for a single worker thread
.flatMap(topicMsgMap -> {
MyKafkaPublisher.sendMessages(topicMsgMap);
}.flatMap(responseHandler::getServerResponse);
}
}
#Data // model class just to easily convert the ServerRequest (from json, for ex.)
// + ~#constructors
public class TopicMsgMap() {
private Map<String, String> topicMsgMap;
}
#Service // Using org.springframework.kafka.core.KafkaTemplate<String, String>
public class MyKafkaPublisher {
#Autowired
private KafkaTemplate<String, String> template;
#Value("${topic1}")
private String topic1;
#Value("${topic2}")
private String topic2;
public void sendMessages(Map<String, String> topicMsgMap){
topicMsgMap.forEach((top, msg) -> {
if (topic.equals("topic1") kafkaTemplate.send(topic1, message);
if (topic.equals("topic2") kafkaTemplate.send(topic2, message);
});
}
}
Guessing this isn't the use-case you had in mind, but hope you find this general structure useful.
There is several approaches including KafkaReplyingRestTemplate for this problem but continuing your approach in servlet api's the solution will be something like this in spring Webflux.
Your Controller method looks like this:
#RequestMapping(path = "/completable-future", method = RequestMethod.POST)
Mono<Response> asyncTransaction(#RequestBody RequestDto requestDto, #RequestHeader Map<String, String> requestHeaders) {
String internalTransactionId = UUID.randomUUID().toString();
kafkaSender.send(Request.builder()
.transactionId(requestHeaders.get("transactionId"))
.internalTransactionId(internalTransactionId)
.sourceIban(requestDto.getSourceIban())
.destIban(requestDto.getDestIban())
.build());
CompletableFuture<Response> completableFuture = new CompletableFuture();
taskHolder.pushTask(completableFuture, internalTransactionId);
return Mono.fromFuture(completableFuture);
}
Your taskHolder component will be something like this:
#Component
public class TaskHolder {
private Map<String, CompletableFuture> taskHolder = new ConcurrentHashMap();
public void pushTask(CompletableFuture<Response> task, String transactionId) {
this.taskHolder.put(transactionId, task);
}
public Optional<CompletableFuture> remove(String transactionId) {
return Optional.ofNullable(this.taskHolder.remove(transactionId));
}
}
And finally your Kafka ResponseListener looks like this:
#Component
public class ResponseListener {
#Autowired
TaskHolder taskHolder;
#KafkaListener(topics = "reactive-response-topic", groupId = "test")
public void listen(Response response) {
taskHolder.remove(response.getInternalTransactionId()).orElse(
new CompletableFuture()).complete(response);
}
}
In this example I used internalTransactionId as CorrelationId but you can use "kafka_correlationId" that is a known kafka header.

Custom json response for internal exception in spring

While implementing a global exception handler in Spring, I noticed that in case of a not recognized Accept header, Spring would throw it's own internal error. What I need is to return a custom JSON error structure instead. Works fine for application specific exceptions and totally fails for Spring HttpMediaTypeNotAcceptableException.
This code tells me "Failed to invoke #ExceptionHandler method: public java.util.Map RestExceptionHandler.springMalformedAcceptHeaderException()" when I try to request a page with incorrect Accept header. Any other way to return custom JSON for spring internal exceptions?
#ControllerAdvice
public class RestExceptionHandler {
#ExceptionHandler(value = HttpMediaTypeNotAcceptableException.class)
#ResponseBody
public Map<String, String> springMalformedAcceptHeaderException() {
Map<String, String> test = new HashMap<String, String>();
test.put("test", "test");
return test;
}
}
Eventually figured that the only way is to do the json mapping manually.
#ExceptionHandler(value = HttpMediaTypeNotAcceptableException.class)
#ResponseBody
public String springMalformedAcceptHeaderException(HttpServletResponse response) {
// populate errorObj, set response headers, etc
ObjectWriter jsonWriter = new ObjectMapper().writer();
try {
return jsonWriter.writeValueAsString(errorObj);
} catch(Exception e){}
return "Whatever";
}

Resources