Am trying to upload a large file using the 'streaming' Apache Commons File Upload API.
The reason I am using the Apache Commons File Uploader and not the default Spring Multipart uploader is that it fails when we upload very large file sizes (~2GB). I working on a GIS application where such file uploads are pretty common.
The full code for my file upload controller is as follows:
#Controller
public class FileUploadController {
#RequestMapping(value="/upload", method=RequestMethod.POST)
public void upload(HttpServletRequest request) {
boolean isMultipart = ServletFileUpload.isMultipartContent(request);
if (!isMultipart) {
// Inform user about invalid request
return;
}
//String filename = request.getParameter("name");
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
// Parse the request
try {
FileItemIterator iter = upload.getItemIterator(request);
while (iter.hasNext()) {
FileItemStream item = iter.next();
String name = item.getFieldName();
InputStream stream = item.openStream();
if (item.isFormField()) {
System.out.println("Form field " + name + " with value " + Streams.asString(stream) + " detected.");
} else {
System.out.println("File field " + name + " with file name " + item.getName() + " detected.");
// Process the input stream
OutputStream out = new FileOutputStream("incoming.gz");
IOUtils.copy(stream, out);
stream.close();
out.close();
}
}
}catch (FileUploadException e){
e.printStackTrace();
}catch (IOException e){
e.printStackTrace();
}
}
#RequestMapping(value = "/uploader", method = RequestMethod.GET)
public ModelAndView uploaderPage() {
ModelAndView model = new ModelAndView();
model.setViewName("uploader");
return model;
}
}
The trouble is that the getItemIterator(request) always returns an iterator that does not have any items (i.e. iter.hasNext() ) always returns false.
My application.properties file is as follows:
spring.datasource.driverClassName=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:19095/authdb
spring.datasource.username=georbis
spring.datasource.password=asdf123
logging.level.org.springframework.web=DEBUG
spring.jpa.hibernate.ddl-auto=update
multipart.maxFileSize: 128000MB
multipart.maxRequestSize: 128000MB
server.port=19091
The JSP view for the /uploader is as follows:
<html>
<body>
<form method="POST" enctype="multipart/form-data" action="/upload">
File to upload: <input type="file" name="file"><br />
Name: <input type="text" name="name"><br /> <br />
Press here to upload the file!<input type="submit" value="Upload">
<input type="hidden" name="${_csrf.parameterName}" value="${_csrf.token}" />
</form>
</body>
</html>
What might I be doing wrong?
Thanks to some very helpful comments by M.Deinum, I managed to solve the problem. I have cleaned up some of my original post and am posting this as a complete answer for future reference.
The first mistake I was making was not disabling the default MultipartResolver that Spring provides. This ended up in the resolver processing the HttpServeletRequest and thus consuming it before my controller could act on it.
The way to disable it, thanks to M. Deinum was as follows:
multipart.enabled=false
However, there was still another hidden pitfall waiting for me after this. As soon as I disabled default multipart resolver, I started getting the following error when trying to make an upload:
Fri Sep 25 20:23:47 IST 2015
There was an unexpected error (type=Method Not Allowed, status=405).
Request method 'POST' not supported
In my security configuration, I had enabled CSRF protection. That necessitated that I send my POST request in the following manner:
<html>
<body>
<form method="POST" enctype="multipart/form-data" action="/upload?${_csrf.parameterName}=${_csrf.token}">
<input type="file" name="file"><br>
<input type="submit" value="Upload">
</form>
</body>
</html>
I also modified my controller a bit:
#Controller
public class FileUploadController {
#RequestMapping(value="/upload", method=RequestMethod.POST)
public #ResponseBody Response<String> upload(HttpServletRequest request) {
try {
boolean isMultipart = ServletFileUpload.isMultipartContent(request);
if (!isMultipart) {
// Inform user about invalid request
Response<String> responseObject = new Response<String>(false, "Not a multipart request.", "");
return responseObject;
}
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload();
// Parse the request
FileItemIterator iter = upload.getItemIterator(request);
while (iter.hasNext()) {
FileItemStream item = iter.next();
String name = item.getFieldName();
InputStream stream = item.openStream();
if (!item.isFormField()) {
String filename = item.getName();
// Process the input stream
OutputStream out = new FileOutputStream(filename);
IOUtils.copy(stream, out);
stream.close();
out.close();
}
}
} catch (FileUploadException e) {
return new Response<String>(false, "File upload error", e.toString());
} catch (IOException e) {
return new Response<String>(false, "Internal server IO error", e.toString());
}
return new Response<String>(true, "Success", "");
}
#RequestMapping(value = "/uploader", method = RequestMethod.GET)
public ModelAndView uploaderPage() {
ModelAndView model = new ModelAndView();
model.setViewName("uploader");
return model;
}
}
where Response is just a simple generic response type I use:
public class Response<T> {
/** Boolean indicating if request succeeded **/
private boolean status;
/** Message indicating error if any **/
private String message;
/** Additional data that is part of this response **/
private T data;
public Response(boolean status, String message, T data) {
this.status = status;
this.message = message;
this.data = data;
}
// Setters and getters
...
}
If you're using a recent version of spring boot (I'm using 2.0.0.M7) then the property names have changed.
Spring started using technology specific names
spring.servlet.multipart.maxFileSize=-1
spring.servlet.multipart.maxRequestSize=-1
spring.servlet.multipart.enabled=false
If you're getting StreamClosed exceptions caused by multiple implementations being active, then the last option allows you to disable the default spring implementation
Please try to add spring.http.multipart.enabled=false in application.properties file.
I use kindeditor + springboot. When I use (MultipartHttpServletRequest) request. I could get the file, but I use appeche-common-io:upload.parse(request) the return value is null.
public BaseResult uploadImg(HttpServletRequest request,String type){
MultipartHttpServletRequest multipartRequest = (MultipartHttpServletRequest) request;
MultiValueMap<String, MultipartFile> multiFileMap = multipartRequest.getMultiFileMap();
You Can simply add spring properties:
spring.servlet.multipart.max-file-size=20000KB
spring.servlet.multipart.max-request-size=20000KB
here my maximum file size is 20000KB, you can change if required.
Related
I'm trying to upload and save an image in MySQL with SpringBoot using JPA but unfortunately I'm greeted with this error, can anyone tell me what am I missing ?
Error
Field error in object 'institution' on field 'logo': rejected value [org.springframework.web.multipart.support.StandardMultipartHttpServletRequest$StandardMultipartFile#387c2996]; codes [typeMismatch.institution.logo,typeMismatch.logo,typeMismatch.[Ljava.lang.Byte;,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [institution.logo,logo]; arguments []; default message [logo]]; default message [Failed to convert property value of type 'org.springframework.web.multipart.support.StandardMultipartHttpServletRequest$StandardMultipartFile' to required type 'java.lang.Byte[]' for property 'logo'; nested exception is java.lang.IllegalArgumentException: Cannot convert value of type 'org.springframework.web.multipart.support.StandardMultipartHttpServletRequest$StandardMultipartFile' to required type 'java.lang.Byte' for property 'logo[0]': PropertyEditor [org.springframework.beans.propertyeditors.CustomNumberEditor] returned inappropriate value of type 'org.springframework.web.multipart.support.StandardMultipartHttpServletRequest$StandardMultipartFile']]
pom.xml
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.4</version>
</dependency>
Entity
#Lob
private Byte[] logo;
controller
#PostMapping("/institutionAddNew")
public String addNewComp(#ModelAttribute("institution") Institution institution, Model model,
HttpSession session,
#RequestParam CommonsMultipartFile fileUpload) throws Exception {
Integer insId = (Integer) session.getAttribute("instId");
Byte[] byteObjects = new Byte[fileUpload.getBytes().length];
int i = 0;
for (byte b : fileUpload.getBytes()){
byteObjects[i++] = b;
}
institution.setLogo(byteObjects);
cs.addNewInstitution(institution);
model.addAttribute("institutionlist", cs.getAllInstitutions(insId));
return "redirect:/showAllInstitutions";
}
Form
<form:form action="/institutionAddNew" method="post" modelAttribute="institution" enctype="multipart/form-data">
<form:input path="instName" />
<input type="file" name="logo" />
<button type="submit" class="btn btn-primary">Submit</button>
</form:form>
The provided information is insufficient to properly diagnose the problem. You need to provide more information.
But I am guessing that, when the jsp file is sending data to your controller, it is sending the file as a MultiPartFile. But your application is trying to cast it to an byte array. That's why your are getting the exception. The problem does not lie withJPA, the exception should happen before the data enters the controller.
You can use an dto object to pass the data to your controller where the file is a Multipartfile object, then you can prepare the entity object with the dto object data. Here you can read the multipartfile and convert it to an byte array and then save it to your database.
Add another InstitutionParam class like
public class InstitutionParam {
private MultipartFile logo;
}
Keep your Institution class same. Your controller should look like
#PostMapping("/institutionAddNew")
public String addNewComp(#ModelAttribute("institution") InstitutionParam institution, Model model,
HttpSession session) throws Exception {
Integer insId = (Integer) session.getAttribute("instId");
MultipartFile fileUpload = institution.getLogo();
Byte[] byteObjects = new Byte[fileUpload.getBytes().length];
int i = 0;
for (byte b : fileUpload.getBytes()){
byteObjects[i++] = b;
}
Institution entity = new Institution();
entity.setLogo(byteObjects);
// save to db
cs.addNewInstitution(institution);
model.addAttribute("institutionlist", cs.getAllInstitutions(insId));
return "redirect:/showAllInstitutions";
}
I've managed to solve it eventually, I had to change the code. I'm sure there are many ways of doing this, not sure if this approached is the best one but it works for me.
Entity
private Integer instId;
private String fileName;
private String fileType;
#Lob
private byte[] image;
public Institution(Integer instId, String instName, String fileName, String fileType, byte[] image) {
super();
this.instId = instId;
this.instName = instName;
this.fileName = fileName;
this.fileType = fileType;
this.image = image;
}
Service
public Institution storeFile(Integer instId, String name, MultipartFile file) throws Exception {
String fileName = StringUtils.cleanPath(file.getOriginalFilename());
try {
Institution institution = new Institution(instId, name,fileName, file.getContentType(), file.getBytes());
return comrepos.save(institution);
} catch (IOException ex) {
throw new Exception("Could not store file " + fileName, e);
}
}
controller
#PostMapping("/institutionAddNew")
public String uploadImage(#RequestParam("image") MultipartFile file,
#RequestParam("instName") String name,
#RequestParam("instId") Integer instId) {
try {
iservice.storeFile(instId,name,file);
} catch (Exception e) {
e.printStackTrace();
}
return "redirect:/showAllInstitutions";
}
Form
<form:form action="/institutionAddNew" method="post" modelAttribute="institution" enctype="multipart/form-data">
<form:input path="instName" />
<input type="file" name="logo" />
<button type="submit" class="btn btn-primary">Submit</button>
</form:form>
My Controller use " org.apache.commons.fileupload " realized the file UPload.
see it:
#PostMapping("/upload")
public String upload2(HttpServletRequest request) throws Exception {
ServletFileUpload upload = new ServletFileUpload();
FileItemIterator iter = upload.getItemIterator(request);
boolean uploaded = false;
while (iter.hasNext() && !uploaded) {
FileItemStream item = iter.next();
if (item.isFormField()) {
item.openStream().close();
} else {
String fieldName = item.getFieldName();
if (!"file".equals(fieldName)) {
item.openStream().close();
} else {
InputStream stream = item.openStream();
// dosomething here.
uploaded = true;
}
}
}
if (uploaded) {
return "ok";
} else {
throw new BaseResponseException(HttpStatus.BAD_REQUEST, "400", "no file field or data file is empty.");
}
}
and my MockMvc code is
public void upload() throws Exception {
File file = new File("/Users/jianxiaowen/Documents/a.txt");
MockMultipartFile multipartFile = new MockMultipartFile("file", new FileInputStream(file));
HashMap<String, String> contentTypeParams = new HashMap<String, String>();
contentTypeParams.put("boundary", "----WebKitFormBoundaryaDEFKSFMY18ehkjt");
MediaType mediaType = new MediaType("multipart", "form-data", contentTypeParams);
MvcResult mvcResult = mockMvc.perform(MockMvcRequestBuilders.post(baseUrl+"/upload")
.content(multipartFile.getBytes())
.contentType(mediaType)
.header(Origin,OriginValue)
.cookie(cookie))
.andReturn();
logResult(mvcResult);
}
my controller is right , it has successed in my web project,
but I want to test it use MvcMock, it has some mistake, see :
can someOne can help me?
"status":"400","msg":"no file field or data file is empty.","data":null
I don't know why it says my file is empty.
my English is poor, thank you very much if someone can help me.
The MockMvc can be used for integration testing for controllers using Apache Commons Fileupload too!
Import the org.apache.httpcomponents:httpmime into your pom.xml or gradle.properties
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.5.13</version>
</dependency>
Update the code to use MultipartEntityBuilder to build the multipart request on the client, and then serialize the entity into bytes, which is then set in the request content
public void upload() throws Exception {
File file = new File("/Users/jianxiaowen/Documents/a.txt");
String boundary = "----WebKitFormBoundaryaDEFKSFMY18ehkjt";
// create 'Content-Type' header for multipart along with boundary
HashMap<String, String> contentTypeParams = new HashMap<String, String>();
contentTypeParams.put("boundary", boundary); // set boundary in the header
MediaType mediaType = new MediaType("multipart", "form-data", contentTypeParams);
// create a multipart entity builder, and add parts (file/form data)
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
HttpEntity multipartEntity = MultipartEntityBuilder.create()
.addPart("file", new FileBody(file, ContentType.create("text/plain"), file.getName())) // add file
// .addTextBody("param1", "value1") // optionally add form data
.setBoundary(boundary) // set boundary to be used
.build();
multipartEntity.writeTo(outputStream); // or getContent() to get content stream
byte[] content = outputStream.toByteArray(); // serialize the content to bytes
MvcResult mvcResult = mockMvc.perform(
MockMvcRequestBuilders.post(baseUrl + "/upload")
.contentType(mediaType)
.content(content) // finally set the content
.header(Origin,OriginValue)
.cookie(cookie)
).andReturn();
logResult(mvcResult);
}
Can you try the below?
mockMvc.perform(
MockMvcRequestBuilders.multipart(baseUrl+"/upload")
.file(multiPartFile)
).andReturn();
Update:
You need to update the controller to handle the MultipartFile:
#PostMapping("/upload")
public String upload2(#RequestParam(name="nameOfRequestParamWhichContainsFileData")
MultipartFile uploadedFile, HttpServletRequest request) throws Exception {
//the uploaded file gets copied to uploadedFile object.
}
You need not use another library for managing file uploads. You can use the file upload capabilities provided by Spring MVC.
I am working on a file upload controller and I am currently getting the following error when testing in Postman.
{
"timestamp": "2019-04-18T14:53:07.988+0000",
"status": 400,
"error": "Bad Request",
"message": "Required request part 'file' is not present",
"path": "/upload"
}
At the moment my controller is very simple but first I need to overcome this problem.
I have looked at the answers given
[here](upload file springboot Required request part 'file' is not present"upload file springboot Required request part file is not present")!
But unfortunately, anything suggested here did not resolve my problem
Any help with this error would be appreciated
This is my controller:
#Controller
public class UploadController {
#ResponseBody
#RequestMapping(value = "/upload", method = RequestMethod.POST)
public boolean upload(#RequestParam("file") MultipartFile file) throws IOException {
try {
if (!file.isEmpty()) {
return true;
} else {
return false;
}
}
catch(Exception e){
e.printStackTrace();
return false;
}
}
}
It's difficult without knowing how you are sending the data but here's how i solved sending multipart/form-data through a #RestController:
#PostMapping(value = "/foo", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
ResponseEntity fileUpload(#Requestparam("bar") LinkedList<MultipartFile> payload) {
MultipartFile file = payload.get(0)
...
Spring just wouldn't accept anything another than a linked list in my case, but that was form-data sent as an Angular2+ FormData object with field name bar.
As you hasve not mentioned your request model, let it be EarningRequest, so know your model data is:
class EarningRequest{
private FilePart file;
//and other data which you want to add.
//add the getter setters also, so that jackson can map the json to this pojo
}
#RestController
public class UploadController {
#PostMapping(value = "/upload",consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public boolean upload (#ModelAttribute EarningRequest earningRequest){
//earningRequest contains the file
//you can get the filePart as earningRequest.getFile()
return true;
}
}
In postman under "key" I wasn't setting anything. I needed to set this as 'file'. I previously made the assumption all I had to do was click the drop-down and select file.
I will include below all the updated code & a link to the image which explains this better(I couldn't display image here as reputation < 10)
link to postman Image
#RestController
public class UploadController {
#PostMapping("/upload")
#ResponseBody
public boolean upload(#RequestParam("file") MultipartFile file) {
try{
if(file.isEmpty() ==false){
System.out.println("Successfully Uploaded: "+ file.getOriginalFilename());
return true;
}
else{
System.out.println("ERROR");
return false;
}
}
catch(Exception e){
System.out.println(e);
return false;
}
}
}
Intro
I have spring MVC application I'm loading image from controller. For security purpose, I added X-Content-Type-Options:nosniff to my Spring application
By setting the following in springConfig xml <security:content-type-options/>
Problem : after this IE is not loading the images responded by controller. I suspect the content type is not set in the response. Because another site which is responding X-Content-Type-Options:nosniff and Content-Type:image/png; is working fine.
TRY1
I tried to change my controller to set content type. But it is not happening.
#RequestMapping(value = "/getUserImage" , produces = org.springframework.http.MediaType.IMAGE_PNG_VALUE)
public #ResponseBody
void getUserImage(
#RequestParam(value = "userId", required = false) int userId,
HttpServletRequest request, HttpServletResponse response) {
try {
//Get file and add it to response
IOUtils.copy(inputStream, response.getOutputStream());
response.getOutputStream().flush();
response.setContentType(org.springframework.http.MediaType.IMAGE_PNG_VALUE);
response.setHeader("Content-Type","image/png");
response.flushBuffer();
inputStream.close();
} catch (Exception e){
}
}
TRY2
I tried to add response header as the same way in method interceptor but still no luck.
But the same thing working in Chrome and Firefox.
Try this :
#RequestMapping(value = "/image/{personId}")
#ResponseBody
public HttpEntity<byte[]> getPhoto(#PathVariable int personId) {
Person person = this.personService.getPersonById(personId);
if (person != null && person.getProfileThumbnail() != null) {
try {
byte[] image;
try {
image = org.apache.commons.io.FileUtils.readFileToByteArray(new File(msg + "/" + person.getUsername() + "/" + personId + ".png"));
} catch (FileNotFoundException e) {
image = org.apache.commons.io.FileUtils.readFileToByteArray(new File(defaultProfilePath));
}
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.IMAGE_PNG);
headers.setContentLength(image.length);
return new HttpEntity<>(image, headers);
} catch (IOException ignored) {
}
}
}
What I am basically doing is checking if there is an image on File-system for the user, if not then I am loading a default image. Right now it works on all browser, so even if personid is 0, I get default image back, with the else cause, which I have not posted here.
I have a form which a user is to use to upload a file to a local drive and also persist some data to the database. But this error message has come up which I would like to know how to deal with it.
HTTP Status 400 - The request sent by the client was syntactically incorrect.
The controller
#RequestMapping(value = "/main/user/setter/addpage", method =
RequestMethod.POST, params = "save")
public String saveProcess(#ModelAttribute("module") Module module,
#RequestParam("userId") Integer userId,
#RequestParam("name") String name,
#RequestParam("file") MultipartFile file,
BindingResult result, HttpSession session) {
if (result.hasErrors()) {
return "redirect:/main/user/setter/settingpage";
}
else
if(module != null){
try {
MultipartFile filea = module.getFileData();
InputStream inputStream = null;
OutputStream outputStream = null;
if (filea.getSize() > 0) {
inputStream = filea.getInputStream();
outputStream = new FileOutputStream("C:\\Test\\"
+ filea.getOriginalFilename());
System.out.println("Uploaded File Name");
System.out.println(filea.getOriginalFilename());
int readBytes = 0;
byte[] buffer = new byte[8192];
while ((readBytes = inputStream.read(buffer, 0, 8192)) !=
-1) {
outputStream.write(buffer, 0, readBytes);
}
outputStream.close();
inputStream.close();
session.setAttribute("success", "File Uploaded");
session.setAttribute("uploadFile", "C:\\Test\\"
+ filea.getOriginalFilename());
}
} catch (Exception e) {
e.printStackTrace();
}
}
// Delegate to service
moduleService.add(userId, module);
return "redirect:/main/user/setter/settingpage";
}
The form itself
<c:url var="processUrl" value="/main/user/setter/addpage?userId=2" />
<form:form modelAttribute="module" method="POST" action="${processUrl}" name="module"
enctype="multipart/form-data">
<form:label path="fileName">Document Title:</form:label>
<form:input path="fileName" name="name"/><br/>
<form:label path="documentPath">Coursework Sample:</form:label>
<form:input path="documentPath" type="file" name="file" id="file" size="43.9"/><br/>
<form:label path="liveDate">Live Date:</form:label>
<form:input path="liveDate"/><br/>
<input type="submit" name="save" value="Save" id="save"/>
<input type="submit" name="send" value="Send" id="send"/>
</form:form>
I'm trying to save the form first the I will use a separate method to send it.
Your method is declared as such:
public String saveProcess(#ModelAttribute("module") Module module,
#RequestParam("userId") Integer userId,
#RequestParam("name") String name,
#RequestParam("file") MultipartFile file,
BindingResult result, HttpSession session) ...
Assuming that you are trying to apply the BindingResult to the Module instance, you need to change the method definition so that the BindingResult comes right after the Module instance.
public String saveProcess(#ModelAttribute("module") Module module,
BindingResult result,
#RequestParam("userId") Integer userId,
#RequestParam("name") String name,
#RequestParam("file") MultipartFile file,
HttpSession session) ...
See the reference for more details
The Errors or BindingResult parameters have to follow the model object
that is being bound immediately as the method signature might have
more that one model object and Spring will create a separate
BindingResult instance for each of them [...]
See here.
The error HTTP Status 400 - The request sent by the client was syntactically incorrect. can come from a missing multipartResolver in the context (servlet-context.xml)
`<beans:bean id="multipartResolver"
class="org.springframework.web.multipart.commons.CommonsMultipartResolver" />`
For more info, see the replies from SpringMVC-FileUpload - The request sent by the client was syntactically incorrect