I have an implementation similar to this one that allows the caller to ask for a specific file from the filesystem. The file is one that doesn't change frequently and is being downloaded by a mobile application. Is there a way I can change this to use the 304 - Not Modified HTTP status code so the client wouldn't have to download the file when it hasn't changed? If so, would the interface to this method have to change in some way to accept a date? Is there some standard way to implement this?
#RequestMapping(value = "/files/datafile", method = RequestMethod.GET)
public void getDataFile(HttpServletResponse response) {
try {
// get your file as InputStream
InputStream is = ...;
// copy it to response's OutputStream
org.apache.commons.io.IOUtils.copy(is, response.getOutputStream());
response.flushBuffer();
} catch (IOException ex) {
log.info("Error writing file to output stream. Filename was '{}'", fileName, ex);
throw new RuntimeException("IOError writing file to output stream");
}
}
Use Springs ETag Header support.
You just need to add this filter to your web.xml
<filter>
<filter-name>etagFilter</filter-name>
<filter-class>org.springframework.web.filter.ShallowEtagHeaderFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>etagFilter</filter-name>
<servlet-name>petclinic</servlet-name>
</filter-mapping>
Or you do it more sophisticated in your controller:
Using Spring support, but then you need to rewrite it so that it return a ResponseEntity instead of using HttpResponse directly, like described in Spring Reference Chapter 21.14.3 Support for the Cache-Control, ETag and Last-Modified response headers in Controllers.
Of course you could also set the etag header directly to your HttpResponse: RFC 2616 Chapter 14.19 ETag
Related
I have created a Spring Boot 2 demo application with the Spring Initializr and added the controller below:
#Controller
#RequestMapping("/demo")
public class UploadController {
private final static Logger LOG = LoggerFactory.getLogger(UploadController.class);
#PostMapping("/upload")
public ResponseEntity<String> uploadFile(
#RequestParam("metadata") MultipartFile metadata,
#RequestParam("payload") MultipartFile payload) throws IOException {
ObjectMapper mapper = new ObjectMapper();
Map metadataMap = mapper.readValue(metadata.getInputStream(), Map.class);
LOG.info("Received call to upload file {}", metadataMap.get("filename"));
LOG.info("File size: {}", payload.getBytes().length);
LOG.info("File {} successfully uploaded", metadataMap.get("filename"));
return ResponseEntity.ok().build();
}
}
I then added an application.yaml file containing this configuration:
spring:
servlet:
multipart:
max-file-size: 2000000MB
max-request-size: 2000000MB
resolve-lazily: true
My goal is to have the controller parse and log the metadata file before it starts reading the payload file, but the resolve-lazily setting seems to be ignored by Boot: the code inside the controller won't be executed until the whole body is read.
I use the command below to test the controller:
curl -F metadata=#metadata.json -F payload=#payload.bin http://localhost:8080/demo/upload
Is there anything wrong with my code/configuration? Am I getting the meaning of the setting right?
At present, if you want to avoid reading (and buffering) the whole body all at once, I think you will have to provide your own parser, as described in the answers here. What would be really interesting (but generally unnecessary) would be to do so in the form of a new MultipartResolver implementation.
There are two existing implementations documented for interface MultipartResolver, and both supply a function setResolveLazily(boolean) (standard), (commons). I have tried with both, and neither seem to allow for parsing or streaming multipart files or parameters independently.
Default is "false", resolving the multipart elements immediately, throwing corresponding exceptions at the time of the resolveMultipart(javax.servlet.http.HttpServletRequest) call. Switch this to "true" for lazy multipart parsing, throwing parse exceptions once the application attempts to obtain multipart files or parameters.
Despite what it says in the documentation, I have found that once you call resolveMultipart, the entire body is parsed and buffered before the call returns. I know this because I can watch the temp-files being created.
One note about "Is there anything wrong with my code"...
Answer: Yes, because by using #RequestParam you have indirectly asked Spring to resolve your parameters ahead of time, before your controller is ever called. What you should be able to do instead (if the documentation were correct) is request the parameters independently from inside your controller:
Configuration (application.properties):
spring.servlet.multipart.enabled = true
spring.servlet.multipart.resolve-lazily = true
Controller:
#PostMapping(path = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<Void> postUpload(HttpServletRequest rawRequest) {
multipartResolver.setResolveLazily(true); // unclear why this is exists
MultipartHttpServletRequest request = multipartResolver.resolveMultipart(rawRequest);
String p1 = request.getParameter("first-parameter");
String p2 = request.getParameter("second-parameter");
System.out.println("first-parameter="+p1+", second-parameter"+p2);
multipartResolver.cleanupMultipart(request);
return new ResponseEntity<Void>(HttpStatus.ACCEPTED);
}
One useful aspect of resolve-lazily that I have discovered is that it allows you to write your own parser for some rest controllers while using the built-in parser for others (see my answer here). In other words, you don't have to use spring.servlet.multipart.enabled = false to get your parser to work. This is a minor breakthrough relative to other advice that I had seen previously.
I am on a team developing a single page web application with an associated REST API.
I wonder if anyone can help me? I am trying to find a way for our application to return the contents of index.html with a 200 response if certain URLs are accessed. For example the client wants to embed information in the URL but expects the content on index.html to be returned.
For example our single page web application is available on a single context root e.g: http://host:9082/webapp
We have rest endpoints available on http://host:9082/webapp/api/... These endpoints must not return index html, they must only return valid rest responses with the appropriate status code (400, 404, 200, 201 etc)
Java script is served from http://host:9082/webapp/js/... and there are other locations we don't want to fall back to index.html
However, if the client requests http://host:9082/webapp/resource/7/show we want index.html to be returned with status code 200. The client will then extract meaning from the URL to drive other REST requests.
So I tried to write a filter like the following:
#Override
public void doFilter(ServletRequest servletRequest,ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException
{
final HttpServletRequest request = (HttpServletRequest)servletRequest;
final HttpServletResponse response = (HttpServletResponse)servletResponse;
final String requestUri = request.getRequestURI();
if (!excluded(requestUri))
{
request.getRequestDispatcher(INDEX_HTML).forward(request, response);
}
else
{
filterChain.doFilter(servletRequest, servletResponse);
}
}
private boolean excluded(String requestUri)
{
for (String part : mExcludedUriParts)
{
if (requestUri.contains(part))
{
return true;
}
}
return false;
}
and enabled the filter in web.xml as following:
<filter>
<filter-name>FallbackFilter</filter-name>
<filter-class>com....http.filter.internal.FallbackFilter</filter-class>
<init-param>
<param-name>excludedUriParts</param-name>
<param-value>/api/,.js/,.png,.html,/apidocs/,/users/imgs/</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>FallbackFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
However this approach is quite fragile as the deployed needs to change the web.xml to match the available resources on the server which may of course change.
We also considered detecting 404's in the filterChain then modifying the response but Liberty did not allow this as the response has already been committed. We also considered using the request accept headers (i.e text/html) as the basis for whether or not to return index html, but we have other html files so this approach did not work either.
We basically want a way to allow some non existent locations on the server to return index.html with a 200 status code. Ideally we want to be informed of a 404 and control the response.
Is there a way to achieve this using filters or any other mechanism in Liberty?
Many thanks
I'm not certain of this, but if you wrap the response in a ServletResponseWrapper you may be able to intercept PrintWriter.flush() so setting the 404 does not commit the response, then the filter can work with it. There's an example of this used for something else here:
https://www.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.iseries.doc/ae/twbs_jaxrs_handlers_servlet_filters.html
This question already has answers here:
SpringBoot: Large Streaming File Upload Using Apache Commons FileUpload
(5 answers)
Closed 2 years ago.
Spring boot's default MultiPartResolver interface handles the uploading of multipart files by storing them on the local file system. Before the controller method is entered, the entire multipart file must finish uploading to the server.
We are storing all of our uploaded files directly to a database and our servers have a very small disk quota, so if a large file is uploaded, we are seeing an IOExeption - Disk quota exceeded.
Is there a way to get the stream directly from the client's incoming request before Spring's MultiPartResolver stores the file on the local filesystem so the we can stream directly to our db?
You could use apache directly, as described here https://commons.apache.org/proper/commons-fileupload/streaming.html.
#Controller
public class UploadController {
#RequestMapping("/upload")
public String upload(HttpServletRequest request) throws IOException, FileUploadException {
ServletFileUpload upload = new ServletFileUpload();
FileItemIterator iterator = upload.getItemIterator(request);
while (iterator.hasNext()) {
FileItemStream item = iterator.next();
if (!item.isFormField()) {
InputStream inputStream = item.openStream();
//...
}
}
}
}
Make sure to disable springs multipart resolving mechanism.
application.yml:
spring:
http:
multipart:
enabled: false
Actually it is not trivial task. If you would like to write stream from client right to the database, you have to process request manually. There are some libraries, that can make this task simpler. One of them is "Apache Commons FileUpload". Below very simple example, how can you process incoming multipart/form-data request by this library.
#Controller
public class Controller{
#RequestMapping("/upload")
public String upload(HttpServletRequest request){
String boundary = extractBoundary(request);
try {
MultipartStream multipartStream = new MultipartStream(request.getInputStream(),
boundary.getBytes(), 1024, null);
boolean nextPart = multipartStream.skipPreamble();
while(nextPart) {
String header = multipartStream.readHeaders();
if(header.contains("filename")){
//if input is file
OutputStream output = createDbOutputStream();
multipartStream.readBodyData(output);
output.flush();
output.close();
} else {
//if input is not file (text, checkbox etc)
ByteArrayOutputStream output = new ByteArrayOutputStream();
multipartStream.readBodyData(output);
String value = output.toString("utf-8");
//... do something with extracted value
}
nextPart = multipartStream.readBoundary();
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private String extractBoundary(HttpServletRequest request) {
String boundaryHeader = "boundary=";
int i = request.getContentType().indexOf(boundaryHeader)+
boundaryHeader.length();
return request.getContentType().substring(i);
}
}
Header for file field will looks like:
Content-Disposition: form-data; name="fieldName"; filename="fileName.jpg"
Content-Type: image/jpeg
Header for simple field will looks like:
Content-Disposition: form-data; name="fieldName";
Note, that this snippet is just simplified example to show you direction. There is no some details like: extract field name from header, create database output stream etc. You can implement all of this stuff by your own.
Examples of multipart request's field headers you can find in RFC1867. Information about multipart/form-data RFC2388.
I need to have a controller (or other component) that handles all 404 errors and smartly redirects to the correct page (this is based on a table src/ target). I have found a few questions about handling thrown exceptions FROM controllers so I have made the following:
#ControllerAdvice
public class ServiceExceptionHandler extends ResponseEntityExceptionHandler {
#ExceptionHandler(Throwable.class)
#ResponseBody
ResponseEntity<String> handleControllerException(HttpServletRequest req, Throwable ex) {
String slug = req.getRequestURI(); // this is the URL that was not found
URI location=null;
try {
// lookup based on slug ...
location = new URI("http://cnn.com"); // let's say I want to redirect to cnn
} catch (URISyntaxException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
HttpHeaders responseHeaders = new HttpHeaders();
responseHeaders.setLocation(location);
return new ResponseEntity<String>("Page has permanently moved", responseHeaders, HttpStatus.PERMANENT_REDIRECT);
}
}
I have not made any other configuration changes
The two issues with this are:
It only catches exceptions thrown by my other controllers and not 404 errors
It catches ALL types of exceptions and not just 404
Any ideas on how to implement a kind of "catch-all"?
This is probably not the best Spring boot solution, but maybe a similar idea can be extrapolated from it and applied to your application. I had a similar problem when working with Spring 3.x, and here's the solution I came up with.
My scenario was that I had multiple servlets running under the same application context. One of those handled a web interface, and another one handled API calls. I wanted the servlet handling the API calls to handle 404s differently than the UI servlet.
I created a new class which extended DispatcherServlet, and overridden the noHandlerFound method. I wired my new class in my web.xml in the <servlet>tags instead of the default DispatcherServlet class I had there before.
<servlet>
<servlet-name>api-servlet</servlet-name>
<servlet-class>path.to.your.new.DispatcherServletClass</servlet-class>
...
</servlet>
I have an issue using Ajax upload with Spring 3 MVC. I understand that I have to configure multipartResolver bean in spring config, which I've done. Than I can have controller like this
#RequestMapping(value ="/settingsSim")
#ResponseBody
public Map uploadSimSettings(#RequestParam(value="qqfile", required=true) MultipartFile settings) {
Map<String, Object> ret = new HashMap<String, Object>();
return ret;
}
The problem is that when I actually send the request to the server (actually valums Ajax file upload does this for me), I get an Internal server error response and nothing is shown in the logs. I am really scratching my head now, as I cannot figure out the problem.
my solution:
#RequestMapping(value = "/create/upload", method = RequestMethod.POST, consumes="multipart/form-data", produces="application/json")
#ResponseBody()
public String handleImageUpload(#RequestParam(value="qqfile", required=true) MultipartFile[] files,
#ModelAttribute(value="files") List<MultipartFile> filesSession) throws IOException, FileUploadException {
if (files.length > 0) {
filesSession.addAll(Arrays.asList(files));
// store the bytes somewhere
return "{\"success\": true}";
}
else {
return "{\"success\": false}";
}
}
#RequestMapping(value = "/create/upload", method = RequestMethod.POST, consumes="application/octet-stream", produces="application/json")
#ResponseBody()
public String handleImageUploadApplication(HttpServletRequest request,
#ModelAttribute(value="files") List<MultipartFile> filesSession) throws IOException, FileUploadException {
if (request.getInputStream() != null) {
// creamos el fichero temporal
File file = File.createTempFile("file", "valumns",
RepositoryData.getRepositoryData());
FileOutputStream fos = new FileOutputStream(file);
// copiamos contenido
Streams.copy(request.getInputStream(), fos, true);
//TODO:
//filesSession.addAll(Arrays.asList(files));
// store the bytes somewhere
return "{\"success\": true}";
}
else {
return "{\"success\": true}";
}
}
#ExceptionHandler(Exception.class)
#ResponseStatus(value = HttpStatus.SERVICE_UNAVAILABLE)
public void handleException(Exception ex) {
log.error("Ocurrio un error en el album", ex);
}
I had the same problem with the fineuploader (valums), and I tried using request.getInputStream() but did not get it to work.
The #ResponseBody annotation worked but I got the whole body with headers. I thought processing that and stripping off the unwanted chunks was not very elegant.
I looked further and found the solution is this post:
problem with spring ajax file upload
Like it is said, I added the bean configuration for the multipart resolver to my spring configuration
<bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver">
</bean>
After that, I could easily retrieve my file using
public #ResponseBody Map ajaxUploadFile(#RequestParam MultipartFile qqfile) { ... }
Don't forget to add the Apache commons-io.jar and commons-fileupload.jar libraries in your project to get it to work
When using valums plugin I solved this problem by using #RequestBody Spring annotation.
You could rewrite your code as follows:
#RequestMapping(value ="/settingsSim",method=RequestMethod.POST)
#ResponseBody
public Map uploadSimSettings(#RequestBody String body) {
/*
some controller logic
*/
}
Note that the variable body will contain the contents of the uploaded file. Also there is no method declaration in your example which means that your method will be mapped to GET request.
P.S. I also had this "no multipart boundary" problem when parsing request with Apache Commons. HttpServletRequest#getParts() returns just an empty collection.
#Tomas I encountered same issue while using the same jquery plugin. Please change the Content-Type in the plugin code to xhr.setRequestHeader("Content-Type", "multipart/form-data"); on my plugin its line 1203, after this its now showing a stack trace, however I am encountering another issue where the logs are printing :
Sep 8, 2011 9:43:39 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet dispatcher threw exception
org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found
As per my observation the file upload plugin does not send a multipart file but sends a stream. I could get it to work by declaring the controller method to accept filename as request param qqfile and the second parameter as httprequest. I then did further processing using request.getinputstream. Hope that helps!
Regards,
Pradyumna