I need to implement a feature where users can download their personal data in XML or JSON format file.
The file will be generated at run time and I have no idea how to implement this in Spring Boot in the corresponding #RestController of my application.
So far I've only implemented the creation of the appropriate XML or JSON String and at first I thought of sending it as a Response and let the front-end manage the rest but I don't think this is the way to go as a file download is required.
Since your file is created dynamically from a string, you will need to create an output stream and write the string content to it. I haven't tested any of this but I've used similar code in the past.
import org.apache.commons.io.IOUtils;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
#RequestMapping("/downloadDocument")
public void downloadDocument(HttpServletResponse response, HttpServletRequest request) throws IOException {
//jsonPersonal is the string that you're going to create dynamically in your code
final String jsonPersonal = " some json encoded data here";
response.setCharacterEncoding("UTF-8");
response.setHeader("Content-Transfer-Encoding", "binary");
response.setContentType("application/json");
response.setContentLength( jsonPersonal.length());
response.setHeader("Content-Disposition", "attachment");
//this copies the content of your string to the output stream
IOUtils.copy(IOUtils.toInputStream(jsonPersonal), response.getOutputStream());
response.flushBuffer();
}
Related
We have a SpringBoot application and are using Jersey to audit incoming HTTP requests.
We implemented a Jersey ContainerRequestFilter to retrieve the incoming HttpServletRequest
and use the HttpServletRequest's getParameterMap() method to extract both query and form data and place it in our audit.
This aligns with the javadoc for the getParameterMap():
"Request parameters are extra information sent with the request. For
HTTP servlets, parameters are contained in the query string or posted
form data."
And here is the documentation pertaining to the filter:
https://eclipse-ee4j.github.io/jersey.github.io/documentation/latest/user-guide.html#filters-and-interceptors
Upon updating SpringBoot, we found that the getParameterMap() no longer returned form data, but still returned query data.
We found that SpringBoot 2.1 is the last version to support our code. In SpringBoot 2.2 the version of Jersey was updated 2.29, but upon reviewing the release notes we don't see anything related to this.
What changed? What would we need to change to support SpringBoot 2.2 / Jersey 2.29?
Here is a simplified version of our code:
JerseyRequestFilter - our filter
import javax.annotation.Priority;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.Priorities;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerRequestFilter;
import javax.ws.rs.container.ResourceInfo;
import javax.ws.rs.core.Context;
import javax.ws.rs.ext.Provider;
...
#Provider
#Priority(Priorities.AUTHORIZATION)
public class JerseyRequestFilter implements ContainerRequestFilter {
#Context
private ResourceInfo resourceInfo;
#Context
private HttpServletRequest httpRequest;
...
public void filter(ContainerRequestContext context) throws IOException {
...
requestData = new RequestInterceptorModel(context, httpRequest, resourceInfo);
...
}
...
}
RequestInterceptorModel - the map is not populating with form data, only query data
import lombok.Data;
import org.glassfish.jersey.server.ContainerRequest;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ResourceInfo;
...
#Data
public class RequestInterceptorModel {
private Map<String, String[]> parameterMap;
...
public RequestInterceptorModel(ContainerRequestContext context, HttpServletRequest httpRequest, ResourceInfo resourceInfo) throws AuthorizationException, IOException {
...
setParameterMap(httpRequest.getParameterMap());
...
}
...
}
JerseyConfig - our config
import com.xyz.service.APIService;
import io.swagger.jaxrs.config.BeanConfig;
import io.swagger.jaxrs.listing.ApiListingResource;
import io.swagger.jaxrs.listing.SwaggerSerializers;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.wadl.internal.WadlResource;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
...
#Component
public class JerseyConfig extends ResourceConfig {
...
public JerseyConfig() {
this.register(APIService.class);
...
// Access through /<Jersey's servlet path>/application.wadl
this.register(WadlResource.class);
this.register(AuthFilter.class);
this.register(JerseyRequestFilter.class);
this.register(JerseyResponseFilter.class);
this.register(ExceptionHandler.class);
this.register(ClientAbortExceptionWriterInterceptor.class);
}
#PostConstruct
public void init()
this.configureSwagger();
}
private void configureSwagger() {
...
}
}
Full Example
Here are the steps to recreate with our sample project:
download the source from github here:
git clone https://github.com/fei0x/so-jerseyBodyIssue
navigate to the project directory with the pom.xml file
run the project with:
mvn -Prun
in a new terminal run the following curl command to test the web service
curl -X POST \
http://localhost:8012/api/jerseyBody/ping \
-H 'content-type: application/x-www-form-urlencoded' \
-d param=Test%20String
in the log you will see the form parameters
stop the running project, ctrl-C
update the pom's parent version to the newer version of SpringBoot
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.15.RELEASE</version>
to
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.9.RELEASE</version>
run the project again:
mvn -Prun
invoke the curl call again:
curl -X POST \
http://localhost:8012/api/jerseyBody/ping \
-H 'content-type: application/x-www-form-urlencoded' \
-d param=Test%20String
This time the log will be missing the form parameters
Alright, after a ton of debugging code and digging through github repos I found the following:
There is a filter, that reads the body inputstream of the request if it is a POST request, making it unusable for further usage. This is the HiddenHttpMethodFilter. This filter, however, puts the content of the body, if it is application/x-www-form-urlencoded into the requests parameterMap.
See this github issue: https://github.com/spring-projects/spring-framework/issues/21439
This filter was active by default in spring-boot 2.1.X.
Since this behavior is unwanted in most cases, a property was created to enable/disable it and with spring-boot 2.2.X it was deactivated by default.
Since your code relies on this filter, you can enable it via the following property:
spring.mvc.hiddenmethod.filter.enabled=true
I tested it locally and it worked for me.
Edit:
Here is what makes this solution work:
The HiddenHttpMethodFilter calls
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
HttpServletRequest requestToUse = request;
if ("POST".equals(request.getMethod()) && request.getAttribute(WebUtils.ERROR_EXCEPTION_ATTRIBUTE) == null) {
String paramValue = request.getParameter(this.methodParam);
...
request.getParameter checks if the parameters have already been parsed and does so, if not the case.
At this time, the request body input stream has not been called yet, so the request figures to parse the body aswell:
org.apache.catalina.connector.Request#parseParameters
protected void parseParameters() {
parametersParsed = true;
Parameters parameters = coyoteRequest.getParameters();
boolean success = false;
try {
...
// this is the bit that parses the actual query parameters
parameters.handleQueryParameters();
// here usingInputStream is false, and so the body is parsed aswell
if (usingInputStream || usingReader) {
success = true;
return;
}
... // the actual body parsing is done here
The thing is, that usingInputStream in this scenario is false and so the method does not return after parsing query params.
usingInputStream is only set to true when the input stream of the request body is retrieved for the first time. That is only done after we fall off the end of the filterChain and servicing the request. The inputStream is called when jersey initializes the ContainerRequest in org.glassfish.jersey.servlet.WebComponent#initContainerRequest
private void initContainerRequest(
final ContainerRequest requestContext,
final HttpServletRequest servletRequest,
final HttpServletResponse servletResponse,
final ResponseWriter responseWriter) throws IOException {
requestContext.setEntityStream(servletRequest.getInputStream());
...
Request#getInputStream
public ServletInputStream getInputStream() throws IOException {
...
usingInputStream = true;
...
Since the HiddenHttpMethodFilter is the only filter to access the parameters, without this filter the parameters are never parsed until we call request.getParameterMap() in RequestInterceptorModel. But at that time, the inputStream of the request body has already been accessed and so it
I will post this answer, even though #Amir Schnell already posted a working solution. The reason is that I am not quite sure why that solution works. Definitely, I would rather have a solution that just requires adding a property to a property file, as opposed to having to alter code as my solution does. But I am not sure if I am comfortable with a solution that works opposite of how my logic sees it's supposed to work. Here's what I mean. In your current code (SBv 2.1.15), if you make a request, look at the log and you will see a Jersey log
2020-12-15 11:43:04.858 WARN 5045 --- [nio-8012-exec-1] o.g.j.s.WebComponent : A servlet request to the URI http://localhost:8012/api/jerseyBody/ping contains form parameters in the request body but the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using #FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
This has been a known problem with Jersey and I have seen a few people on here asking why they can't get the parameters from the HttpServletRequest (this message is almost always in their log). In your app though, even though this is logged, you are able to get the parameters. It is only after upgrading your SB version, and then not seeing the log, that the parameters are unavailable. So you see why I am confused.
Here is another solution that doesn't require messing with filters. What you can do is use the same method that Jersey uses to get the #FormParams. Just add the following method to your RequestInterceptorModel class
private static Map<String, String[]> getFormParameterMap(ContainerRequestContext context) {
Map<String, String[]> paramMap = new HashMap<>();
ContainerRequest request = (ContainerRequest) context;
if (MediaTypes.typeEqual(MediaType.APPLICATION_FORM_URLENCODED_TYPE, request.getMediaType())) {
request.bufferEntity();
Form form = request.readEntity(Form.class);
MultivaluedMap<String, String> multiMap = form.asMap();
multiMap.forEach((key, list) -> paramMap.put(key, list.toArray(new String[0])));
}
return paramMap;
}
You don't need the HttpServletRequest at all for this. Now you can set your parameter map by calling this method instead
setParameterMap(getFormParameterMap(context));
Hopefully someone can explain this baffling case though.
I am writing a module in Microservices, where I have to write a rest end point to upload large files chunk by chunk and save each chunk in local tmp directory.. Will get input as chunk of video file.
I am using Java11, Jetty server, SpringBoot framework.
I have written a piece of code in controller as below:
#Named
#Singleton
#Path("/chunk")
public class FileUploadChunkController {
#POST
#Produces({MediaType.APPLICATION_JSON})
#Consumes({MediaType.APPLICATION_OCTET_STREAM})
#Path("/uploadvideo")
public Response uploadVideo(#FormDataParam("file")InputStream inputStream, #FormDataParam("file") FormDataContentDisposition contentDisposition , #Context final HttpServletRequest request) throws Exception {
LOGGER.info("Inside method: uploadVideo");
}
When invoking the api with chunk file as data, it is not reaching to the controller POST method.
Could some one help me?
I have spent lot of time but did not get solution for it.
I have created a Spring Boot 2 demo application with the Spring Initializr and added the controller below:
#Controller
#RequestMapping("/demo")
public class UploadController {
private final static Logger LOG = LoggerFactory.getLogger(UploadController.class);
#PostMapping("/upload")
public ResponseEntity<String> uploadFile(
#RequestParam("metadata") MultipartFile metadata,
#RequestParam("payload") MultipartFile payload) throws IOException {
ObjectMapper mapper = new ObjectMapper();
Map metadataMap = mapper.readValue(metadata.getInputStream(), Map.class);
LOG.info("Received call to upload file {}", metadataMap.get("filename"));
LOG.info("File size: {}", payload.getBytes().length);
LOG.info("File {} successfully uploaded", metadataMap.get("filename"));
return ResponseEntity.ok().build();
}
}
I then added an application.yaml file containing this configuration:
spring:
servlet:
multipart:
max-file-size: 2000000MB
max-request-size: 2000000MB
resolve-lazily: true
My goal is to have the controller parse and log the metadata file before it starts reading the payload file, but the resolve-lazily setting seems to be ignored by Boot: the code inside the controller won't be executed until the whole body is read.
I use the command below to test the controller:
curl -F metadata=#metadata.json -F payload=#payload.bin http://localhost:8080/demo/upload
Is there anything wrong with my code/configuration? Am I getting the meaning of the setting right?
At present, if you want to avoid reading (and buffering) the whole body all at once, I think you will have to provide your own parser, as described in the answers here. What would be really interesting (but generally unnecessary) would be to do so in the form of a new MultipartResolver implementation.
There are two existing implementations documented for interface MultipartResolver, and both supply a function setResolveLazily(boolean) (standard), (commons). I have tried with both, and neither seem to allow for parsing or streaming multipart files or parameters independently.
Default is "false", resolving the multipart elements immediately, throwing corresponding exceptions at the time of the resolveMultipart(javax.servlet.http.HttpServletRequest) call. Switch this to "true" for lazy multipart parsing, throwing parse exceptions once the application attempts to obtain multipart files or parameters.
Despite what it says in the documentation, I have found that once you call resolveMultipart, the entire body is parsed and buffered before the call returns. I know this because I can watch the temp-files being created.
One note about "Is there anything wrong with my code"...
Answer: Yes, because by using #RequestParam you have indirectly asked Spring to resolve your parameters ahead of time, before your controller is ever called. What you should be able to do instead (if the documentation were correct) is request the parameters independently from inside your controller:
Configuration (application.properties):
spring.servlet.multipart.enabled = true
spring.servlet.multipart.resolve-lazily = true
Controller:
#PostMapping(path = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<Void> postUpload(HttpServletRequest rawRequest) {
multipartResolver.setResolveLazily(true); // unclear why this is exists
MultipartHttpServletRequest request = multipartResolver.resolveMultipart(rawRequest);
String p1 = request.getParameter("first-parameter");
String p2 = request.getParameter("second-parameter");
System.out.println("first-parameter="+p1+", second-parameter"+p2);
multipartResolver.cleanupMultipart(request);
return new ResponseEntity<Void>(HttpStatus.ACCEPTED);
}
One useful aspect of resolve-lazily that I have discovered is that it allows you to write your own parser for some rest controllers while using the built-in parser for others (see my answer here). In other words, you don't have to use spring.servlet.multipart.enabled = false to get your parser to work. This is a minor breakthrough relative to other advice that I had seen previously.
Right now we are holding file in our postgresql database and mapping that content using byte[] field in our entity. I need to investigate if we could
stream the content data direct from the database to the HTTP output stream, and do the same thing in opposite way so stream binary data from HTTP into database using jpa Blob data type. I know that Blob has methods getBinaryStream and setBinaryStream so its may work, and we do not need hold data into memory.
What I am concern are database transaction, because we are mapping entity into DTO, and the second thing is broken Http request and data may be lost in some point.
Is there are anybody who had any experience with that solution ?
Solution for stream-reading data from BLOBs:
Existing BLOB data are streamed by passing OutputStream (provided by the servlet container) into transactional method which writes entity blob data to the stream from inside transaction. Note that the content type of response is set before writing the data.
Entity class:
public class Attachment {
private java.sql.Blob data;
public java.sql.Blob getData() { return data; }
}
Service method:
#Transactional(readOnly = true)
public void copyContentsTo(long attachmentId, OutputStream outputStream) throws IOException {
Attachment dbAttachment = attachmentRepository.findOne(attachmentId);
try (InputStream is = dbAttachment.getData().getBinaryStream()) {
IOUtils.copy(is, outputStream);
} catch (SQLException e) {
throw new ParameterException("Cannot extract BLOB for attachment #" + attachmentId, e);
}
}
REST API Spring Controller method:
#GetMapping(value = "/api/project-attachment/{attachment-id}/content")
#ResponseStatus(HttpStatus.OK)
public void getAttachmentContent(
#PathVariable("attachment-id") long attachmentId,
HttpServletResponse response,
OutputStream stream) throws IOException {
response.setContentType(getMime(attachmentId));
attachmentService.copyContentsTo(attachmentId, stream);
}
Lucasz, Spring Content for JPA does exactly what you are asking. Designed to make it really easy to create Spring applications that handle content (documents, images, video's etc). It supports a range of backend stores one of which being relational DBs and obviously they uses BLOBs.
This JPA module will stream uploaded files from the request input stream directly to the database and vice versa thus it never stores the entire file in-memory which would obviously cause problems with very large files.
It would save you from having to write ANY of the code in #tequilacat's answer.
Probably worth a look.
I am new to Servlets and Spring Framework.
I try to get media files from directory through Rest Service.
For videos/mp4 I couldn't find anything.
For audio I did this:
Writing mp3 file to response output stream
For images I did this:
#RequestMapping("/tmp/{uuid}")
#ResponseBody
public ResponseEntity<InputStreamResource> getTmp(#PathVariable("uuid") String uuid)
throws IOException {
Path path = Paths.get("/media/psmaster/HDD/TUC-IPS/" + uuid);
String contentType = Files.probeContentType(path);
FileSystemResource file = new FileSystemResource("/media/psmaster/HDD/TUC-IPS/" + uuid);
return ResponseEntity
.ok()
.contentLength(file.contentLength())
.contentType(
MediaType.parseMediaType(contentType))
.body(new InputStreamResource(file.getInputStream()));
}
Can someone please help to figure out the problem?
If you are using Spring 4.2 you can use StreamingResponseBody, Have a look at this post
You can also give Spring Content a look. It allows you to build content services very quickly and easily using similar programming techniques to Spring Data. You can also pair it with Spring Data to additionally store and search metadata for your videos. By defining a single interface and including the appropriate Spring Content dependency in your project you can create a set of REST endpoints that allow you to manage the full lifecycle of a video including streaming.
You can write media using streams and HttpServletResponse:
#RequestMapping(value = "/image/{imgName}", method = RequestMethod.GET)
public void getImageAsByteArray(#PathVariable String imgName , HttpServletResponse response) throws IOException {
InputStream in = servletContext.getResourceAsStream("/WEB-INF/images/" + imgName);
response.setContentType(MediaType.IMAGE_JPEG_VALUE);
IOUtils.copy(in, response.getOutputStream());
}
The example above serves an image file.
Hope this helps