Spring boot large file upload and download support - spring-boot

I have a spring boot web application which will handle large file (max size of 5g) upload and then save it to s3. The request and response may last for a long time.
So what is the best practice to handle the upload and download like this? How to make a good performance to prevent my server down when download or upload large files?

you can use multipart/form-data
#RequestMapping(value = "/agency/create", method = RequestMethod.POST, consumes = "multipart/form-data")
public ResponseEntity<List<String>> createAgency(
#RequestParam(value = "username", required = true) String username,
#RequestParam(value = "pic1", required = true)MultipartFile pic1File,
MultipartHttpServletRequest request, ModelAndView modelAndView) {
List<String> requestKeys=new ArrayList<String>();
List<String> originalFileName=new ArrayList<String>();
request.getFileNames().forEachRemaining(requestKeys::add);
for(String multipartFile:requestKeys) {
originalFileName.add(request.getFile(multipartFile).getOriginalFilename());
}
storageService.store(pic1File);
return new ResponseEntity<List<String>>(originalFileName, HttpStatus.CREATED);
}

Posting in case someone finds this useful in the future. This works with a REST controller as of Spring Boot 2.4.2.
Class annotations:
#RestController
#RequestMapping("/api")
Method declaration:
#RequestMapping(path = "/file-upload/{depot}/{fileName}", method = {RequestMethod.POST, RequestMethod.PUT})
public ResponseEntity<String> fileUpload(
#PathVariable(name = "depot") String depot,
#PathVariable(name = "fileName") String fileName,
InputStream inputStream,
HttpServletRequest request,
HttpServletResponse response)
The above is the Spring Boot configuration for a REST Controller that worked for me for large file upload. The key was adding InputStream inputStream directly.

Related

Spring cloud feign not upload video file

I have 2 microservices. The first service(video converter) and the second service(integration Spring with Amazon S3 to store video and image on the Amazon S3).
The first service produces files by schedule and after converts File to MultipartFile uploads it to the second service and after to Amazon S3.
But I stack on the issue with Feigh uploading the video.
When I debug, I saw that the file in #RequestPart in the second service is null. But converting was done successfully.
I try adding encoding as in that post File upload spring cloud feign client, but that does not help.
Code sample first microservice:
Upload methods:
#Override
public Response<String> uploadFile(final Object object, final MultipartFile multipartFile) {
final Map<String, Object> s3Url = videoServiceFeign.uploadFile(multipartFile,
object.getId().toString(), object.getIdIds().toString(), object.getName());
object.setS3Url(s3Url.get("object").toString());
return generateSuccessResponse();
}
#Override
public Response<String> uploadFile(final Stream stream, final File file) throws IOException {
final InputStream inputStream = Files.newInputStream(file.toPath());
final MultipartFile multipartFile = new MockMultipartFile(file.getName(), file.getName(),
ContentType.MULTIPART_FORM_DATA.toString(), IOUtils.toByteArray(inputStream));
return this.uploadFile(stream, multipartFile);
}
Feign client:
#FeignClient(name = "video-service", url = "${video-service.ribbon.listOfServers}")
public interface VideoServiceFeign {
#PostMapping(path = "/api/v1/video/upload", consumes = {"multipart/form-data"})
Map<String, Object> uploadFile(#RequestPart("file") MultipartFile multipartFile,
#RequestParam("id") String id,
#RequestParam("ids") String ids,
#RequestParam("name") String name);
}
Upload endpoint in the second service:
#ApiOperation("Upload new video file")
#ApiResponses({
#ApiResponse(code = 200, message = "File uploaded successfully"),
#ApiResponse(code = 403, message = "Access Denied"),
#ApiResponse(code = 500, message = "Internal server error"),
#ApiResponse(code = 503, message = "Gateway timeout")
})
#ResponseStatus(HttpStatus.OK)
#PostMapping(value = "/upload", consumes = {MediaType.APPLICATION_JSON_VALUE, MediaType.MULTIPART_FORM_DATA_VALUE})
public #ResponseBody
Map<String, Object> uploadFile(#RequestPart("file") final MultipartFile file,
#RequestParam("id") final String id,
#RequestParam("ids") final String ids,
#RequestParam("name") final String name
) throws IOException {
return uploadService.uploadFile(id, ids, name, file);
}
What am I missing?
After a few days of researching, I found what I miss, the issue was with that method: public Response<String> uploadFile(final Stream stream, final File file) throws IOException
In the row:
final MultipartFile multipartFile = new MockMultipartFile(file.getName(), file.getName(), ContentType.MULTIPART_FORM_DATA.toString(), IOUtils.toByteArray(inputStream));
I had two issues first is the content type, I use video/mp4 not ContentType.MULTIPART_FORM_DATA.
And the second one is the first param in the constructor of MockMultipartFile I use the file.getName(), but that is incorrect because the docs say that is a field name in the multipart request. And that should be "file".

Unable to upload file due to Content-Type "multipart/form-data" not set for request body of type StandardMultipartFile

I have a remote service A which does the file upload. I have service B which calls the upload API of service A through FeignClient to upload a file
The method definition in Service A is something like
ResponseEntity<?> upload(#RequestPart("file") MultipartFile file) { }
And the method in Service B is
#FeignClient(url = "http://localhost:5000/")
public interface uploadService {
#RequestMapping(method = RequestMethod.POST, value = "/serviceA/upload")
#Headers("Content-Type: multipart/form-data")
void uploadFile(#RequestPart("file") MultipartFile file);
}
I am getting the error
Content-Type "multipart/form-data" not set for request body of type StandardMultipartFile
I have tried most of the suggestions on https://github.com/spring-cloud/spring-cloud-netflix/issues/867 and
https://github.com/OpenFeign/feign-form but nothing works for me
I was able to solve this issue by simply adding consumes = "multipart/form-data" in the RequestMapping. The reason was that I was mixing spring based annotations with open feign annotations. #Headers("Content-Type: multipart/form-data") works with Open feign. Here I am using spring-cloud-openfeign which provides abstraction to Open feign and make it easy to integration with spring framework components.
#FeignClient(url = "http://localhost:5000/")
public interface uploadService {
#RequestMapping(method = RequestMethod.POST, value = "/serviceA/upload" consumes = "multipart/form-data" )
void uploadFile(#RequestPart("file") MultipartFile file);
}
If you have trouble just within the test just use org.springframework.mock.web.MockMultipartFile where you can set contentType as one of argument in construtor.

Camel rest API to provide dynamic download

How can we provide document download using camel API, I need to provide an api using camel rest to response the file as download and I have the logic to create the pdf using apache fop, but i need to get some information how to respond the file as rest response using camel rest.
#RestController
public class MyController {
#Autowired
ICityService cityService;
#RequestMapping(
value = "/pdfreport",
method = RequestMethod.GET,
produces = MediaType.APPLICATION_PDF_VALUE
)
public ResponseEntity<InputStreamResource> citiesReport() throws IOException {
List<City> cities = (List<City>) cityService.findAll();
ByteArrayInputStream bis = GeneratePdfReport.citiesReport(cities);
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Disposition", "inline;
filename = citiesreport.pdf");
return ResponseEntity
.ok()
.headers(headers)
.contentType(MediaType.APPLICATION_PDF)
.body(new InputStreamResource(bis));
}
}

Block number of request using spring webflux

I currently have below reactive service exposed using spring boot 2(Spring webflux)
#RequestMapping(value = "/tasks/v1", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
public Mono taskForUserV1(#RequestParam(value = "userId", required = true) String userId,
#RequestParam(required = false) Map<String, String> userData) {
return service.taskForUserV1(userId, userData);
}
But i want to throw an exception to the client requesting my service if the request are lets say more than 500. How to do it?
you can take a look at Rate Limiter design pattern implemented by resileince4j # https://resilience4j.github.io/resilience4j/#_rate_limiting
RateLimiter rateLimiter = RateLimiter.ofDefaults("testName");
Retrofit retrofit = new Retrofit.Builder()
.addCallAdapterFactory(RateLimiterCallAdapter.of(rateLimiter))
.baseUrl("http://localhost:8080/")
.build();

#RequestBody Annotation not working on Websphere Application Server

I am trying to use #RequestBody annotation in one of my controllers method:
#Auditable(application = AuditApplication.DEP_TRXN, actionCategory = AuditActionCategory.READ_RESPONSE, actionDetail = "Viewed a tracking group.", event = AuditEventType.ACTION)
#RequestMapping(value = "groupView", method = RequestMethod.POST, produces = "application/json")
public #ResponseBody
String ajaxGroupView(#RequestBody String payload, final HttpServletRequest request) throws TrackingServiceException, JsonGenerationException, JsonMappingException,
IOException
{
String requestBody = java.net.URLDecoder.decode(payload, "UTF-8");
return requestBody;
}
When i run this code on Jetty server my 'payload' gets the post data, but when i run the same code on Websphere Application Server -8, payload is 'null'. I have mvc annotation driven turn on. How do i make this code work on Websphere Application Server?
Thanks!

Resources