I have 2 microservices. The first service(video converter) and the second service(integration Spring with Amazon S3 to store video and image on the Amazon S3).
The first service produces files by schedule and after converts File to MultipartFile uploads it to the second service and after to Amazon S3.
But I stack on the issue with Feigh uploading the video.
When I debug, I saw that the file in #RequestPart in the second service is null. But converting was done successfully.
I try adding encoding as in that post File upload spring cloud feign client, but that does not help.
Code sample first microservice:
Upload methods:
#Override
public Response<String> uploadFile(final Object object, final MultipartFile multipartFile) {
final Map<String, Object> s3Url = videoServiceFeign.uploadFile(multipartFile,
object.getId().toString(), object.getIdIds().toString(), object.getName());
object.setS3Url(s3Url.get("object").toString());
return generateSuccessResponse();
}
#Override
public Response<String> uploadFile(final Stream stream, final File file) throws IOException {
final InputStream inputStream = Files.newInputStream(file.toPath());
final MultipartFile multipartFile = new MockMultipartFile(file.getName(), file.getName(),
ContentType.MULTIPART_FORM_DATA.toString(), IOUtils.toByteArray(inputStream));
return this.uploadFile(stream, multipartFile);
}
Feign client:
#FeignClient(name = "video-service", url = "${video-service.ribbon.listOfServers}")
public interface VideoServiceFeign {
#PostMapping(path = "/api/v1/video/upload", consumes = {"multipart/form-data"})
Map<String, Object> uploadFile(#RequestPart("file") MultipartFile multipartFile,
#RequestParam("id") String id,
#RequestParam("ids") String ids,
#RequestParam("name") String name);
}
Upload endpoint in the second service:
#ApiOperation("Upload new video file")
#ApiResponses({
#ApiResponse(code = 200, message = "File uploaded successfully"),
#ApiResponse(code = 403, message = "Access Denied"),
#ApiResponse(code = 500, message = "Internal server error"),
#ApiResponse(code = 503, message = "Gateway timeout")
})
#ResponseStatus(HttpStatus.OK)
#PostMapping(value = "/upload", consumes = {MediaType.APPLICATION_JSON_VALUE, MediaType.MULTIPART_FORM_DATA_VALUE})
public #ResponseBody
Map<String, Object> uploadFile(#RequestPart("file") final MultipartFile file,
#RequestParam("id") final String id,
#RequestParam("ids") final String ids,
#RequestParam("name") final String name
) throws IOException {
return uploadService.uploadFile(id, ids, name, file);
}
What am I missing?
After a few days of researching, I found what I miss, the issue was with that method: public Response<String> uploadFile(final Stream stream, final File file) throws IOException
In the row:
final MultipartFile multipartFile = new MockMultipartFile(file.getName(), file.getName(), ContentType.MULTIPART_FORM_DATA.toString(), IOUtils.toByteArray(inputStream));
I had two issues first is the content type, I use video/mp4 not ContentType.MULTIPART_FORM_DATA.
And the second one is the first param in the constructor of MockMultipartFile I use the file.getName(), but that is incorrect because the docs say that is a field name in the multipart request. And that should be "file".
Related
I have a spring boot web application which will handle large file (max size of 5g) upload and then save it to s3. The request and response may last for a long time.
So what is the best practice to handle the upload and download like this? How to make a good performance to prevent my server down when download or upload large files?
you can use multipart/form-data
#RequestMapping(value = "/agency/create", method = RequestMethod.POST, consumes = "multipart/form-data")
public ResponseEntity<List<String>> createAgency(
#RequestParam(value = "username", required = true) String username,
#RequestParam(value = "pic1", required = true)MultipartFile pic1File,
MultipartHttpServletRequest request, ModelAndView modelAndView) {
List<String> requestKeys=new ArrayList<String>();
List<String> originalFileName=new ArrayList<String>();
request.getFileNames().forEachRemaining(requestKeys::add);
for(String multipartFile:requestKeys) {
originalFileName.add(request.getFile(multipartFile).getOriginalFilename());
}
storageService.store(pic1File);
return new ResponseEntity<List<String>>(originalFileName, HttpStatus.CREATED);
}
Posting in case someone finds this useful in the future. This works with a REST controller as of Spring Boot 2.4.2.
Class annotations:
#RestController
#RequestMapping("/api")
Method declaration:
#RequestMapping(path = "/file-upload/{depot}/{fileName}", method = {RequestMethod.POST, RequestMethod.PUT})
public ResponseEntity<String> fileUpload(
#PathVariable(name = "depot") String depot,
#PathVariable(name = "fileName") String fileName,
InputStream inputStream,
HttpServletRequest request,
HttpServletResponse response)
The above is the Spring Boot configuration for a REST Controller that worked for me for large file upload. The key was adding InputStream inputStream directly.
I would like to send a File object along with custom model object in a single request.
let formData:FormData = new FormData();
let file = this.fileList[0];
formData.append('file', file, file.name);
formData.append('address', JSON.stringify(customObj));
...
this.http.post(fileServeUrl, formData)
My backend is in Spring Rest as below
#RequestMapping(value = "/fileServe",
produces = {"application/json"},
consumes = {MediaType.APPLICATION_JSON_VALUE, MediaType.MULTIPART_FORM_DATA_VALUE},
method = RequestMethod.POST)
ResponseEntity<Image> uploadFile(#RequestPart("file") MultipartFile imageData, #RequestPart("address") Address address) throws IOException {...}
I was able to receive the data if I pass simple String along with File though.
formData.append('file', file, file.name);
formData.append('address', addressText);
Backend
#RequestMapping(value = "/fileServe",
produces = {"application/json"},
consumes = {MediaType.APPLICATION_JSON_VALUE, MediaType.MULTIPART_FORM_DATA_VALUE},
method = RequestMethod.POST)
ResponseEntity<Image> uploadFile(#RequestPart("file") MultipartFile imageData, #RequestPart("address") String addressText) throws IOException {...}
I tried #RequestBody for my custom object but even that didn't work. Any advise please.
The problem with #Requestbody and #RequestPart annotation is that spring use the HttpMessageConverter to take convert the incoming json message into the your object. As you send form data with a file and a text value spring can not convert it into your object. I am afraid you have to pass the value of address seperatetly.
#RequestMapping(value = "/fileupload", headers = ("content-type=multipart/*"), method = RequestMethod.POST)
public ResponseEntity<AjaxResponseBody> upload(#RequestParam("file") MultipartFile file, #RequestParam String name, #RequestParam String postCode) {
AjaxResponseBody result = new AjaxResponseBody();
HttpHeaders headers = new HttpHeaders();
if (!file.isEmpty()) {
try {
Address address = new Address();
address.setName(name);
result.setMsg("ok");
return new ResponseEntity<AjaxResponseBody>(result, headers, HttpStatus.OK);
} catch (Exception e) {
return new ResponseEntity<AjaxResponseBody>(HttpStatus.BAD_REQUEST);
}
} else {
return new ResponseEntity<AjaxResponseBody>(HttpStatus.BAD_REQUEST);
}
}
Expept if you find a way your client app send a file with MimeType of image/jpg and and an address of application/json which allow spring to parse the json and map to your Address object which i couldn't do it.
I wrote some code for related to upload a file using spring, It works fine, Now I am writing integration test cases for that but I am facing some issue
My controller method,
#RequestMapping(value = "/{attributeName}/upload", method = RequestMethod.POST)
#ResponseBody
public Result uploadCompany(HttpServletRequest request,
#RequestParam MultipartFile file, #PathVariable String attributeName,
#RequestParam long dateKey)
throws IOException, PromotionException {
some code
}
Test cases
#Test
public void shouldReturnTrueStatusWhenUploadCompany() throws Exception {
MockMultipartFile file = new MockMultipartFile("company_upload", "company_upload.csv",
MediaType.MULTIPART_FORM_DATA_VALUE, EMPLOYEE_NUMBER_FILE_CONTENT.getBytes(UTF_8));
mockMvc.perform(
MockMvcRequestBuilders.fileUpload(
PROMOTION + StringUtils.replace(ATTRIBUTE_NAME, "{attributeName}", "COMPANY") + "/upload")
.file(file).param("dateKey", "852017") .contentType(MediaType.MULTIPART_FORM_DATA)
.accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk());
}
But I am getting
2017-05-09 13:42:42,506 ERROR [Test worker] INTERNAL_SERVER_ERROR:
org.springframework.web.bind.MissingServletRequestParameterException: Required MultipartFile parameter 'file' is not present
Where am I wrong?
change your line
MockMultipartFile file = new MockMultipartFile("company_upload", "company_upload.csv",
MediaType.MULTIPART_FORM_DATA_VALUE, EMPLOYEE_NUMBER_FILE_CONTENT.getBytes(UTF_8));
to
MockMultipartFile file = new MockMultipartFile("file", "company_upload.csv",
MediaType.MULTIPART_FORM_DATA_VALUE, EMPLOYEE_NUMBER_FILE_CONTENT.getBytes(UTF_8));
or change your controller method declaration to something like this
public Result uploadCompany(HttpServletRequest request,
#RequestParam(value = "company_upload") MultipartFile file, #PathVariable String attributeName,
#RequestParam long dateKey)
When I click on any link the content should be downloaded
But this is what I get.
MastercourseController.java
#RequestMapping(value = { ControllerUriConstant.download_file }, method = RequestMethod.GET)
#ResponseBody
public void downloadingAFileById(#RequestParam("id") String id, Model model, HttpServletRequest request)
throws TechnoShineException, IOException {
String filePath = "D:/dev/testFIle.txt";
long download = Long.parseLong(id);
byte[] b = masterCourseFileFormService.getAllDownloadable(download);
OutputStream outputStream = new FileOutputStream(filePath);
outputStream.write(b);
outputStream.close();
}
MasterCourseService
public byte[] getAllDownloadable(long id) throws TechnoShineException
{
return masterCourseFormUploadDao.getAllDownloadableFiles(id);
}
MasterCourseDao
public byte[] getAllDownloadableFiles(long id) throws TechnoShineException
{
return masterCourseFormUploadMapper.getAllDownloadable(id);
}
MasterCourseMapper
public byte[] getAllDownloadable(long id) throws TechnoShineException;
You are writing the data returned by getAllDownloadable(..) to a hard-coded file. Are you sure that is what you want? I think you want to write the content returned by getAllDownloadable(..) to be written into the response. That can be done by adding a method parameter of the type HttpServletResponse to your mapping and writing into the output stream returned by HttpServletResponse#getOutputStream() and flushing (not closing!) that stream at the end.
Furthermore you have to remove the #ResponseBody annotation as this is meant to be used if the value that is returned by the mapping method returns the data that should directly be sent to the client (i.e. when sending a JSON data object or a string) without passing it to the template engine. As you are not returning anything you can remove this annotation.
Furthermore you have to set the content type of your response by invoking HttpServletResponse#setContentType(contentType: String).
In your case, the invocation would be the following:
response.setContentType("text/plain");
You complete method would look like this:
#RequestMapping(
value = ControllerUriConstant.download_file,
method = RequestMethod.GET
)
public void downloadingAFileById(#RequestParam("id") String id, HttpServletResponse response)
throws TechnoShineException, IOException {
long download = Long.parseLong(id);
byte[] b = masterCourseFileFormService.getAllDownloadable(download);
response.getOutputStream().write(b);
response.getOutputStream().flush();
}
I'm using spring boot, I need to upload a multipart file (jpg or png file). I need to send a (POST request to upload the multi part file using "postman"), can anyone provide a screen shot of "postman" of how to set it up to do that or tell me? Thanks.
method :
#RequestMapping(method = RequestMethod.POST, value = "/upload")
#ResponseBody
ResponseEntity<?> writeUserProfilePhoto(#PathVariable Long user, #RequestPart("file") MultipartFile file) throws Throwable {
byte bytesForProfilePhoto[] = FileCopyUtils.copyToByteArray(file.getInputStream()); //Return an InputStream to read the contents of the file from.
this.crmService.writeUserProfilePhoto(user, MediaType.parseMediaType(file.getContentType()),bytesForProfilePhoto);
HttpHeaders httpHeaders = new HttpHeaders();
URI uriOfPhoto = ServletUriComponentsBuilder.fromCurrentContextPath()
.pathSegment(("/users" + "/{user}" + "/photo").substring(1))
.buildAndExpand(Collections.singletonMap("user", user)).toUri();
httpHeaders.setLocation(uriOfPhoto);
return new ResponseEntity<>(httpHeaders, HttpStatus.CREATED);
}
and this is how I sent the POST request:
my configuration class:
#Configuration
#ConditionalOnClass({ Servlet.class, StandardServletMultipartResolver.class, MultipartConfigElement.class })
#ConditionalOnProperty(prefix = "multipart", name = "enabled", matchIfMissing = true)
#EnableConfigurationProperties(MultipartProperties.class)
public class MultipartAutoConfiguration {
#Autowired
private MultipartProperties multipartProperties = new MultipartProperties();
#Bean
#ConditionalOnMissingBean
public MultipartConfigElement multipartConfigElement() {
return this.multipartProperties.createMultipartConfig();
}
#Bean(name = DispatcherServlet.MULTIPART_RESOLVER_BEAN_NAME)
#ConditionalOnMissingBean(MultipartResolver.class)
public StandardServletMultipartResolver multipartResolver() {
return new StandardServletMultipartResolver();
}
}
The error in postman says
Required MultipartFile parameter 'file' is not present
The method signature looks fine defining file parameter:
ResponseEntity<?> writeUserProfilePhoto(
#PathVariable Long user, #RequestPart("file") MultipartFile file)
throws Throwable
The issue is that when using postman, you're using dog1 as the name of this parameter. Change it to file to match the expected parameter name for the multipart file.
This approach worked for me.
The error in postman says
Required MultipartFile parameter 'file' is not present
The method signature looks fine defining file parameter:
ResponseEntity<?> writeUserProfilePhoto(
#PathVariable Long user, #RequestPart("file") MultipartFile file)
throws Throwable
The issue is that when using postman, you're using dog1 as the name of this parameter. Change it to file to match the expected parameter name for the multipart file.
#Override
public Response uploadImage(String token, MultipartFile file) {
long id=tokenUtil.decodeToken(token);
Optional<User> user=userRepo.findById(id);
if(!user.isPresent()) {
throw new UserException(-5, "User does not exists");
}
UUID uuid=UUID.randomUUID();
String uniqueUserId=uuid.toString();
try {
Files.copy(file.getInputStream(), fileLocation.resolve(uniqueUserId), StandardCopyOption.REPLACE_EXISTING);
user.get().setProfilePic(uniqueUserId);
userRepo.save(user.get());
}catch (IOException e) {
e.printStackTrace();
// TODO: handle exception
}
return ResponseHelper.statusResponse(200, "Profile Pic Uploaded Successfully");
}