Spring Boot Callback after client receives resource? - spring-boot

I'm creating an endpoint using Spring Boot which executes a combination of system commands (java.lang.Runtime API) to generate a zip file to return to the client upon request, here's the code.
#GetMapping(value = "generateZipFile")
public ResponseEntity<Resource> generateZipFile(#RequestParam("id") Integer id) throws IOException {
org.springframework.core.io.Resource resource = null;
//generate zip file using commandline
resource = service.generateTmpResource(id);
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_TYPE, "application/zip")
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"randomFile.zip\"")
.body(resource);
//somehow delete generated file here after client receives it
}
I cannot keep stacking up the files on the server for obvious disk limit reasons, so I'm looking for a way to delete the files as soon as the client downloads them. Is there a solution in Spring Boot for this? I basically need to hook a callback that would do the cleanup after the user receives the resource.
I'm using Spring Boot 2.0.6

You can create a new thread but a best solution would be create a ThreadPoolExecutor in order to manage threads or also Scheduled annotation helps us.
new Thread() {
#Override
public void run() {
service.cleanup(id);
}
}.start();
UPDATED
A best answer, it would be using a Stack combine with Thread.
Here is the solution that I've done.
https://github.com/jjohxx/example-thread

I ended up using a HandlerInterceptorAdapter, afterCompletion was the callback I needed. The only challenge I had to deal with was passing through the id of the resource to cleanup, which I handled by adding a header in my controller method:
#GetMapping(value = "generateZipFile")
public ResponseEntity<Resource> genereateZipFile(#RequestParam("id") Integer id,
RedirectAttributes redirectAttributes) throws IOException {
org.springframework.core.io.Resource resource = myService.generateTmpResource(id);;
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_TYPE, "application/zip")
.header(MyInterceptor.TMP_RESOURCE_ID_HEADER, id.toString())
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"someFile.zip\"")
.body(resource);
}
The interceptor code:
#Component
public class MyInterceptor extends HandlerInterceptorAdapter {
public static final String TMP_RESOURCE_ID_HEADER = "Tmp-ID";
#Autowired
private MyService myService;
#Override
public void afterCompletion(HttpServletRequest request,
HttpServletResponse response,
Object handler,
Exception ex) {
if(response == null || !response.containsHeader(TMP_RESOURCE_ID_HEADER)) return;
String tmpFileId = response.getHeader(TMP_RESOURCE_ID_HEADER);
myService.cleanup(tmpFileId);
}
}
For more information about interceptors see here.

Related

Transferring big files in spring integration

The spring integration flow I wrote has to get files (some of them are as big as 4G) from a rest service and transfer them to a remote shared drive. For downloading them from the rest service I configured this simple component:
#Bean
public HttpRequestExecutingMessagehandler httpDownloader (RestTemplate template){
Expression expr = (new SpelExpressionParser()).parseExpression("payload.url");
HttpRequestExecutingMessagehandler handler = new HttpRequestExecutingMessagehandler (expr, template);
handler.setExpectedResponseType(byte[].class);
handler.setHttpMethod(GET);
return handler;
}
Unfortunately this won't scale meaning for larger files it will eventually throw java.lang.OutOfMemoryError: Java heap space, even if i add more memory with -Xmx or -XXMaxPermSize
So my question is, what to do in order to avoid these problems no matter how big the files will be?
I think I have answered you in some other similar your question that Spring RestTemplate is not designed for streaming response body. It is explained in this SO thread: Getting InputStream with RestTemplate.
One of the solution which may work for your is to write a custom HttpMessageConverter which would return a File object containing data from HTTP response. This article explains how to do that with the ResponseExtractor, but something like FileHttpMessageConverter is not so hard to implement based on experience from that article. See StreamUtils.copy(InputStream in, OutputStream out)
Then you inject this FileHttpMessageConverter into your HttpRequestExecutingMessagehandler - setMessageConverters(List<HttpMessageConverter<?>> messageConverters).
Your service for remote shared drive should already deal with this local temporary file to get that large content without consuming memory.
See also this one about possible approach via WebFlux: https://www.amitph.com/spring-webclient-large-file-download/
Created this starting from ByteArrayHttpMessageConverter class and injected it into the custom RestTemplate I use. But this solution is based on using a File message, which is not quite the streaming I was hoping for.
public class FileCustomConverter extends AbstractHttpMessageConverter<File> {
public FileCustomConverter() {
super(new MediaType[]{MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL});
}
public boolean supports(Class<?> clazz) {
return File.class == clazz;
}
public File readInternal(Class<? extends File> clazz, HttpInputMessage inputMessage) throws IOException {
File outputFile = File.createTempFile(UUID.randomUUID().toString(), ".tmp");
OutputStream outputStream = new FileOutputStream(outputFile);
StreamUtils.copy(inputMessage.getBody(), outputStream);
outputStream.close();
return outputFile;
}
protected Long getContentLength(File bytes, #Nullable MediaType contentType) {
return bytes.length();
}
protected void writeInternal(File file, HttpOutputMessage outputMessage) throws IOException {
InputStream inputStream = new FileInputStream(file);
StreamUtils.copy(inputStream, outputMessage.getBody());
inputStream.close();
}
}

MockMvc Test does not get to the endpoint for a Multipart file in a RestController

I am calling a service in an orders controller which receives a multipart file and processes it and saving it into a database. I am trying to create a Spring Rest Doc for it but it is not even hitting the endpoint. I am creating a list of orders which is what the service expects. It receives the order as a stream as shown and converts into a stream of orders before saving it into a database. I have shown the main part of the controller and my code for generating the rest docs. When I run the code I get the following exception, it never even hits the endpoint when I set a breakpoint. I also used fileupload() but that did not work either.
Exception is:
Content type = application/json
Body = {"path":"/orders/order_reception","exceptionName":
"MissingServletRequestPartException","message":"Required request part 'uploadFile' is not
present",
"rootExceptionName":"MissingServletRequestPartException",
"rootMessage":"MissingServletRequestPartException: Required request part 'uploadFile' is not present"}
#RestController
#RequestMapping(value = "/orders")
#Validated
class OrderController{
#PostMapping(path = "/order_reception")
public ResponseEntity receiveData(#RequestPart MultipartFile uploadFile,
HttpServletRequest request,
HttpServletResponse response) {
if (!uploadFile.isEmpty()) {
try {
Reader reader = new InputStreamReader(request.getInputStream()));
... save file
return new ResponseEntity<>(HttpStatus.HttpStatus.CREATED);
} catch (Exception e) {
return new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
}
}
return new ResponseEntity(HttpStatus.BAD_REQUEST);
}
#Test
public void sendData() throws Exception {
ObjectMapper mapper = new ObjectMapper();
Order order = repository.getOrder("1233333");
List<Order> orderList = new ArrayList<>():
resourceList.add(order);
MockMultipartFile orderFile = new MockMultipartFile("order-data", "order.json", "application/json",
mapper.writeValueAsString(orderList).getBytes(Charset.defaultCharset()));
mockMvc.perform(multipart("/orders/order_reception")
.file(orderFile))
.andExpect(status().isCreated())
.andDo(document("send-order",
preprocessRequest(prettyPrint()),
preprocessResponse(prettyPrint())));
}
Thank you Marten Deinum, your suggestion that the file name was wrong fixed it.
I simply changed name in the MockMultipartFile( "uploadsFile", ...)

Return response messages in spring boot

I am working with spring boot with a h2 database. I would like to return a 201 message when the register is inserted succesfully and a 400 when is duplicated. I am using ResponseEntity to achieve this, fot example , the next is my create method from the Service:
#Override
public ResponseEntity<Object> createEvent(EventDTO eventDTO) {
if (eventRepository.findOne(eventDTO.getId()) != null) {
//THis is a test, I am looking for the correct message
return new ResponseEntity(HttpStatus.IM_USED);
}
Actor actor = actorService.createActor(eventDTO.getActor());
Repo repo = repoService.createRepo(eventDTO.getRepo());
Event event = new Event(eventDTO.getId(), eventDTO.getType(), actor, repo, createdAt(eventDTO));
eventRepository.save(event);
return new ResponseEntity(HttpStatus.CREATED);
}
This is my controller:
#PostMapping(value = "/events")
public ResponseEntity addEvent(#RequestBody EventDTO body) {
return eventService.createEvent(body);
}
But I'm not getting any message in the browser, I am doing different tests with postman and when I consult for all the events, the result is correct, but each time that I make a post I dont get any message in the browser, I am not pretty sure what is the cause of this issue. Any ideas?
The ideal way to send Response to the client is to create DTO/DAO with ResponseEntity in Controller
Controller.java
#PostMapping("/test")
public ResponseEntity<Object> testApi(#RequestBody User user)
{
System.out.println("User: "+user.toString());
return assetService.testApi(user);
}
Service.java
public ResponseEntity testApi(User user) {
if(user.getId()==1)
return new ResponseEntity("Created",HttpStatus.CREATED);
else
return new ResponseEntity("Used",HttpStatus.IM_USED);
// for BAD_REQUEST(400) return new ResponseEntity("Bad Request",HttpStatus.BAD_REQUEST);
}
Tested using Postman
Status 201 Created
Status 226 IM Used
Okay, I really don't feel good that service sending the ResponseEntity but not Controller.You could use #ResponseStatus and ExceptionHandler classes for these cases, like below.
Create a class in exception package
GlobalExceptionHandler.java
#ControllerAdvice
public class GlobalExceptionHandler {
#ResponseStatus(HttpStatus.BAD_REQUEST)
#ExceptionHandler(DataIntegrityViolationException.class) // NOTE : You could create a custom exception class to handle duplications
public void handleConflict() {
}
}
Controller.java
#PostMapping(value = "/events")
#ResponseStatus(HttpStatus.CREATED) // You don't have to return any object this will take care of the status
public void addEvent(#RequestBody EventDTO body) {
eventService.createEvent(body);
}
Now changing the service would look like,
Service.java
#Override
public void createEvent(EventDTO eventDTO) { // No need to return
if (eventRepository.findOne(eventDTO.getId()) != null) {
throw new DataIntegrityViolationException("Already exists"); // you have to throw the same exception which you have marked in Handler class
}
Actor actor = actorService.createActor(eventDTO.getActor());
Repo repo = repoService.createRepo(eventDTO.getRepo());
Event event = new Event(eventDTO.getId(), eventDTO.getType(), actor, repo, createdAt(eventDTO));
eventRepository.save(event);
}

Can I use Spring WebFlux to implement REST services which get data through Kafka request/response topics?

I'm developing REST service which, in turn, will query slow legacy system so response time will be measured in seconds. We also expect massive load so I was thinking about asynchronous/non-blocking approaches to avoid hundreds of "servlet" threads blocked on calls to slow system.
As I see this can be implemented using AsyncContext which is present in new servlet API specs. I even developed small prototype and it seems to be working.
On the other hand it looks like I can achieve the same using Spring WebFlux.
Unfortunately I did not find any example where custom "backend" calls are wrapped with Mono/Flux. Most of the examples just reuse already-prepared reactive connectors, like ReactiveCassandraOperations.java, etc.
My data flow is the following:
JS client --> Spring RestController --> send request to Kafka topic --> read response from Kafka reply topic --> return data to client
Can I wrap Kafka steps into Mono/Flux and how to do this?
How my RestController method should look like?
Here is my simple implementation which achieves the same using Servlet 3.1 API
//took the idea from some Jetty examples
public class AsyncRestServlet extends HttpServlet {
...
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
String result = (String) req.getAttribute(RESULTS_ATTR);
if (result == null) { //data not ready yet: schedule async processing
final AsyncContext async = req.startAsync();
//generate some unique request ID
String uid = "req-" + String.valueOf(req.hashCode());
//share it to Kafka receive together with AsyncContext
//when Kafka receiver will get the response it will put it in Servlet request attribute and call async.dispatch()
//This doGet() method will be called again and it will send the response to client
receiver.rememberKey(uid, async);
//send request to Kafka
sender.send(uid, param);
//data is not ready yet so we are releasing Servlet thread
return;
}
//return result as html response
resp.setContentType("text/html");
PrintWriter out = resp.getWriter();
out.println(result);
out.close();
}
Here's a short example - Not the WebFlux client you probably had in mind, but at least it would enable you to utilize Flux and Mono for asynchronous processing, which I interpreted to be the point of your question. The web objects should work without additional configurations, but of course you will need to configure Kafka as the KafkaTemplate object will not work on its own.
#Bean // Using org.springframework.web.reactive.function.server.RouterFunction<ServerResponse>
public RouterFunction<ServerResponse> sendMessageToTopic(KafkaController kafkaController){
return RouterFunctions.route(RequestPredicates.POST("/endpoint"), kafkaController::sendMessage);
}
#Component
public class ResponseHandler {
public getServerResponse() {
return ServerResponse.ok().body(Mono.just(Status.SUCCESS), String.class);
}
}
#Component
public class KafkaController {
public Mono<ServerResponse> auditInvalidTransaction(ServerRequest request) {
return request.bodyToMono(TopicMsgMap.class)
// your HTTP call may not return immediately without this
.subscribeOn(Schedulers.single()) // for a single worker thread
.flatMap(topicMsgMap -> {
MyKafkaPublisher.sendMessages(topicMsgMap);
}.flatMap(responseHandler::getServerResponse);
}
}
#Data // model class just to easily convert the ServerRequest (from json, for ex.)
// + ~#constructors
public class TopicMsgMap() {
private Map<String, String> topicMsgMap;
}
#Service // Using org.springframework.kafka.core.KafkaTemplate<String, String>
public class MyKafkaPublisher {
#Autowired
private KafkaTemplate<String, String> template;
#Value("${topic1}")
private String topic1;
#Value("${topic2}")
private String topic2;
public void sendMessages(Map<String, String> topicMsgMap){
topicMsgMap.forEach((top, msg) -> {
if (topic.equals("topic1") kafkaTemplate.send(topic1, message);
if (topic.equals("topic2") kafkaTemplate.send(topic2, message);
});
}
}
Guessing this isn't the use-case you had in mind, but hope you find this general structure useful.
There is several approaches including KafkaReplyingRestTemplate for this problem but continuing your approach in servlet api's the solution will be something like this in spring Webflux.
Your Controller method looks like this:
#RequestMapping(path = "/completable-future", method = RequestMethod.POST)
Mono<Response> asyncTransaction(#RequestBody RequestDto requestDto, #RequestHeader Map<String, String> requestHeaders) {
String internalTransactionId = UUID.randomUUID().toString();
kafkaSender.send(Request.builder()
.transactionId(requestHeaders.get("transactionId"))
.internalTransactionId(internalTransactionId)
.sourceIban(requestDto.getSourceIban())
.destIban(requestDto.getDestIban())
.build());
CompletableFuture<Response> completableFuture = new CompletableFuture();
taskHolder.pushTask(completableFuture, internalTransactionId);
return Mono.fromFuture(completableFuture);
}
Your taskHolder component will be something like this:
#Component
public class TaskHolder {
private Map<String, CompletableFuture> taskHolder = new ConcurrentHashMap();
public void pushTask(CompletableFuture<Response> task, String transactionId) {
this.taskHolder.put(transactionId, task);
}
public Optional<CompletableFuture> remove(String transactionId) {
return Optional.ofNullable(this.taskHolder.remove(transactionId));
}
}
And finally your Kafka ResponseListener looks like this:
#Component
public class ResponseListener {
#Autowired
TaskHolder taskHolder;
#KafkaListener(topics = "reactive-response-topic", groupId = "test")
public void listen(Response response) {
taskHolder.remove(response.getInternalTransactionId()).orElse(
new CompletableFuture()).complete(response);
}
}
In this example I used internalTransactionId as CorrelationId but you can use "kafka_correlationId" that is a known kafka header.

Spring MVC #RestController and redirect

I have a REST endpoint implemented with Spring MVC #RestController. Sometime, depends on input parameters in my controller I need to send http redirect on client.
Is it possible with Spring MVC #RestController and if so, could you please show an example ?
Add an HttpServletResponse parameter to your Handler Method then call response.sendRedirect("some-url");
Something like:
#RestController
public class FooController {
#RequestMapping("/foo")
void handleFoo(HttpServletResponse response) throws IOException {
response.sendRedirect("some-url");
}
}
To avoid any direct dependency on HttpServletRequest or HttpServletResponse I suggest a "pure Spring" implementation returning a ResponseEntity like this:
HttpHeaders headers = new HttpHeaders();
headers.setLocation(URI.create(newUrl));
return new ResponseEntity<>(headers, HttpStatus.MOVED_PERMANENTLY);
If your method always returns a redirect, use ResponseEntity<Void>, otherwise whatever is returned normally as generic type.
Came across this question and was surprised that no-one mentioned RedirectView. I have just tested it, and you can solve this in a clean 100% spring way with:
#RestController
public class FooController {
#RequestMapping("/foo")
public RedirectView handleFoo() {
return new RedirectView("some-url");
}
}
redirect means http code 302, which means Found in springMVC.
Here is an util method, which could be placed in some kind of BaseController:
protected ResponseEntity found(HttpServletResponse response, String url) throws IOException { // 302, found, redirect,
response.sendRedirect(url);
return null;
}
But sometimes might want to return http code 301 instead, which means moved permanently.
In that case, here is the util method:
protected ResponseEntity movedPermanently(HttpServletResponse response, String url) { // 301, moved permanently,
return ResponseEntity.status(HttpStatus.MOVED_PERMANENTLY).header(HttpHeaders.LOCATION, url).build();
}
As the redirections are usually needed in a not-straightforward path, I think throwing an exception and handling it later is my favourite solution.
Using a ControllerAdvice
#ControllerAdvice
public class RestResponseEntityExceptionHandler
extends ResponseEntityExceptionHandler {
#ExceptionHandler(value = {
NotLoggedInException.class
})
protected ResponseEntity<Object> handleNotLoggedIn(
final NotLoggedInException ex, final WebRequest request
) {
final String bodyOfResponse = ex.getMessage();
final HttpHeaders headers = new HttpHeaders();
headers.add("Location", ex.getRedirectUri());
return handleExceptionInternal(
ex, bodyOfResponse,
headers, HttpStatus.FOUND, request
);
}
}
The exception class in my case:
#Getter
public class NotLoggedInException extends RuntimeException {
private static final long serialVersionUID = -4900004519786666447L;
String redirectUri;
public NotLoggedInException(final String message, final String uri) {
super(message);
redirectUri = uri;
}
}
And I trigger it like this:
if (null == remoteUser)
throw new NotLoggedInException("please log in", LOGIN_URL);
if you #RestController returns an String you can use something like this
return "redirect:/other/controller/";
and this kind of redirect is only for GET request, if you want to use other type of request use HttpServletResponse

Resources