x-www-form-urlencoded Array inconsistently populated in Spring REST call - spring

I am attempting to send a PUT request to a Rest API using x-www-form-urlencoded content. My aim is to send a list of strings in the request similar to this article. I have the following REST controller defined in a Spring Boot application to allow for this:
#RestController
#RequestMapping(value = "/rest/api", produces = MediaType.APPLICATION_JSON_VALUE)
public class RestApiController {
#PutMapping(value = "/{id}", consumes = MediaType.APPLICATION_JSON_VALUE)
private ReturnType putRestApiTypeJson(
#PathVariable("id") String id,
#ModelAttribute PutDataRequest request) {
System.out.println();
return null;
}
#PutMapping(value = "/{id}", consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
private ReturnType putRestApiTypeUrlEncoded(
#PathVariable("id") String id,
#ModelAttribute PutDataRequest request) {
System.out.println();
return null;
}
}
which leverages PutDataRequest defined by:
#Data
#AllArgsConstructor
#NoArgsConstructor
public class PutDataRequest {
Set<String> characters = new HashMap<>();
Set<String> movies = new HashMap<>();
}
I try hitting the rest api controller via curl to perform testing. The Application JSON PUT request receives characters and movies no problem, however the form-urlencoded endpoint does so inconsistently:
// No data populated in PutDataRequest at debug time:
curl -X PUT 'http://localhost:some-port/rest/api' -d 'characters=Some%20Name%26movies=Some%20Title' -H 'Content-Type: application/x-www-form-urlencoded'
// Data populated in PutDataRequest at debug time:
curl -X PUT 'http://localhost:some-port/rest/api?characters=Some%20Name%26movies=Some%20Title' -H 'Content-Type: application/x-www-form-urlencoded'
Can anyone give an insight on why providing the key-value pairs via -d prevents the data from being forwarded to the form-urlencoded PUT endpoint? For context, I run this coded using spring version 5.2.3.RELEASE and spring boot version 2.2.4.RELEASE.

I decided to sidestep Spring in this situation. Instead of relying on Spring to figure out how to marshal the data I wanted, I added a HttpServletRequest to the form-urlencoded method signature and pulled the data out of the request:
#PutMapping(value = "/{id}", consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
private ReturnType putRestApiTypeUrlEncoded(
#PathVariable("id") String id,
#ModelAttribute PutDataRequest data,
HttpServletRequest request) {
String body = request.getRequest().lines()
.map(line -> URLDecoder.decode(line, Charset.defaultCharset()))
.collect(Collectors.joining(System.lineSeparator()));
// manipulate body content to extract desired data
}
I was inspired to do the above by this answer.

Also found another way to get around this error. Turns out PUT and DELETE requests aren't enabled by default, and you need to add an implementation for the formContentFilter method in your Application.java (or wherever you call SpringApplication.run(...) )
Once I added the following to Application.java, I ran again and it worked like magic:
#Bean
#ConditionalOnMissingBean(org.springframework.web.filter.FormContentFilter.class)
#ConditionalOnProperty(prefix="spring.mvc.formcontent.filter", name="enabled", matchIfMissing=true)
public OrderedFormContentFilter formContentFilter() {
return new OrderedFormContentFilter();
}

Related

In Spring Boot - Can you make request mapping that receives the whole message body without any parsing/intervention?

I need to have a method that does not regard/parse the content of request message, just ... pass it along as input parameter to the #PostMapping method.
Is it possible? Because defining parameters like:
#RequestBody byte[] data
or
#RequestBody String text
tell the framework that it suppose to get some xml/json. and I want it to receive plain text + utf-8 encoding.
Some code to clarify:
#RestController
#RequestMapping(path="/abc", method = RequestMethod.POST)
public class NlpController {
#PostMapping(path="/def", consumes="text/plain; charset: UTF-8", produces=MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<Object> processText(#RequestBody String text)
{
...
return ResponseEntity.ok().body(object);
}
}
Trying also:
#RestController
#RequestMapping(path="/abc", method = RequestMethod.POST)
public class NlpController {
#PostMapping(path="/nlp", consumes=MediaType.APPLICATION_JSON_UTF8_VALUE, produces=MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<Object> process(HttpServletRequest request)
{
....
return ResponseEntity.ok().body(article);
}
}
But I get 406 response...
using curl:
curl -v -s -X POST -H "Content-Type:" -H "Content-Type: application/json; charset: utf-8" --data-binary #article.txt localhost:8080/abc/def/
I think you should inject HttpServletRequest as controller method attribute, then you will have acces to request payload.
#PostMapping(path="/something")
public ResponseEntity<Object> processText(HttpServletRequest request) {
// do something with request
}
More info.
406 Not Acceptable
The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/406
If i understand your question correctly you need to load text file directly as input param in spring boot rest call.
You need to modify your code and curl request , please use fllowing code as referance .
#RequestMapping(value = "/abc", method = RequestMethod.POST)
public String ResponseEntity<Object> processText(#RequestParam("file")
MultipartFile file) {
System.out.println("---------loading file----------");
/// Calculation and your logic
return ResponseEntity.ok().body(article);
}
Curl request :
curl -X POST localhost:8080/abc -F "file=#article.txt"
One more issue i can see in your curl request your mapping is abc and you are calling
localhost:8080/abc/def/
Using #RequestParam for multipartfile is a right way?
If using data in memory following code will work for you
#PostMapping(value = "/abc", consumes = "application/json", produces =
"application/json")
ResponseEntity<Object> processText( #RequestBody String input)
throws JSONException {
//
return ResponseEntity.ok().body(article);
}
Short answer: This is not a job for a full blown framework like spring boot. Better use something like spark that can do this with one liner and without any configurations. At least this is the best answer for my humble causes.
Long answer: I could not make spring boot to receive clean body text from a client, not even after many (failed) attempts to tweak the headers / media / consume flag / ... Guess this just (might) not be possible.

Supporting application/json and application/x-www-form-urlencoded simultaneously from Spring's rest controller

Am writing a REST endpoint which needs to support both application/x-www-form-urlencoded and application/json as request body simultaneously. I have made below configuration,
#RequestMapping(method = RequestMethod.POST, produces = { MediaType.APPLICATION_JSON_VALUE }, consumes = {
MediaType.APPLICATION_FORM_URLENCODED_VALUE, MediaType.APPLICATION_JSON_VALUE }, path = Constants.ACCESS_TOKEN_V1_ENDPOINT)
public OAuth2Authorization createAccessTokenPost(
#RequestBody(required = false) MultiValueMap<String, String> paramMap) { ..
While it supports application/x-www-form-urlencoded or application/json individually (when I comment out one content type from consumes = {}), but it does not support both simultaneously. Any ideas ?
So RestControllers by default can handle application/json fairly easily and can create a request pojo from a #RequestBody annotated parameter, while application/x-www-form-urlencoded takes a little more work. A solution could be creating an extra RestController method that has the same mapping endpoint to handle the different kinds of requests that come in (application/json, application/x-www-form-urlencoded, etc). This is because application/x-www-form-urlencoded endpoints need to use the #RequestParam instead of the #RequestBody annotation (for application/json).
For instance if I wanted to host a POST endpoint for /emp that takes either application/json or application/x-www-form-urlencoded as Content-Types and uses a service to do something, I could create Overload methods like so
#Autowired
private EmpService empService;
#PostMapping(path = "/emp", consumes = {MediaType.APPLICATION_FORM_URLENCODED_VALUE})
public ResponseEntity createEmp(final #RequestHeader(value = "Authorization", required = false) String authorizationHeader,
final #RequestParam Map<String, String> map) {
//After receiving a FORM URLENCODED request, change it to your desired request pojo with ObjectMapper
final ObjectMapper mapper = new ObjectMapper();
final TokenRequest tokenRequest = mapper.convertValue(map, CreateEmpRequest.class);
return empService.create(authorizationHeader, createEmpRequest);
}
#PostMapping(path = "/emp", consumes = {MediaType.APPLICATION_JSON_VALUE})
public ResponseEntity createEmp(final #RequestHeader(value = "Authorization", required = false) String authorizationHeader,
final #RequestBody CreateEmpRequest createEmpRequest) {
//Receieved a JSON request, the #RequestBody Annotation can handle turning the body of the request into a request pojo without extra lines of code
return empService.create(authorizationHeader, createEmpRequest);
}
As per my findings, spring does not support content types "application/x-www-form-urlencoded", "application/json" and "application/xml" together.
Reason I figured: Spring processes JSON and XML types by parsing and injecting them into the java pojo marked with #RequestBody spring annotation. However, x-www-form-urlencoded must be injected into a MultiValueMap<> object marked with #RequestBody. Two different java types marked with #RequestBody will not be supported simultaneously, as spring may not know where to inject the payload.
A working solution:
"application/x-www-form-urlencoded" can be supported as it is in the API. That is, it can be injected into spring's MultiValueMap<> using an #RequestBody annotation.
To support JSON and XML on the same method, we can leverage servlet specification and spring's class built on top of them to extract the payload as stream.
Sample code:
import org.springframework.http.HttpInputMessage;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.http.converter.xml.Jaxb2RootElementHttpMessageConverter;
import org.springframework.http.server.ServletServerHttpRequest;
import org.springframework.util.MultiValueMap;
// usual REST service class
#Autowired
private MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter;
#Autowired
private Jaxb2RootElementHttpMessageConverter jaxb2RootElementHttpMessageConverter;
public ResponseEntity<Object> authorizationRequestPost(HttpServletResponse response, HttpServletRequest request,#RequestBody(required = false) MultiValueMap<String, String> parameters) {
// this MultiValueMap<String,String> will contain key value pairs of "application/x-www-form-urlencoded" parameters.
// payload object to be populated
Authorization authorization = null;
HttpInputMessage inputMessage = new ServletServerHttpRequest(request) {
#Override
public InputStream getBody() throws IOException {
return request.getInputStream();
}
};
if (request.getContentType().equals(MediaType.APPLICATION_JSON_VALUE)) {
authorization = (Authorization) mappingJackson2HttpMessageConverter.read(Authorization.class, inputMessage);
}
else if (request.getContentType().equals(MediaType.APPLICATION_XML_VALUE)) {
authorization = (Authorization)jaxb2RootElementHttpMessageConverter.read(Authorization.class, inputMessage);
}
else{
// extract values from MultiValueMap<String,String> and populate Authorization
}
// remaining method instructions
}
Point to note that any custom data type/markup/format can be supported using this approach. Spring's org.springframework.http.converter.HttpMessageConverter<> can be extended to write the parsing logic.
Another possible approach could be an AOP style solution which would execute the same logic: parse payload by extracting it from HttpServlet input stream and inject into the payload object.
A third approach will be to write a filter for executing the logic.
It's not possible to handle application/json and application/x-www-form-urlencoded requests simultaneously with a single Spring controller method.
Spring get application/x-www-form-urlencoded data by ServletRequest.getParameter(java.lang.String), the document said:
For HTTP servlets, parameters are contained in the query string or posted form data.
If the parameter data was sent in the request body, such as occurs with an HTTP POST request, then reading the body directly via getInputStream() or getReader() can interfere with the execution of this method.
So, if your method parameter is annotated with #RequestBody, Spring will read request body and parse it to the method parameter object. But application/x-www-form-urlencoded leads Spring to populate the parameter object by invoking ServletRequest.getParameter(java.lang.String).
Just to make it, the above answer doesn't work as even if you do not annotate MultiValueMap with #RequestBody it would always check for contentType==MediaType.APPLICATION_FORM_URLENCODED_VALUE which again in rest of the cases resolves to 415 Unsupported Media Type.

415 Unsupported Media Type, when NOT sending an optional request body with POST request

I have a REST controller that defines an interface which takes an optional request body.
#RestController
#RequestMapping(ExampleRest.EXAMPLE_URI)
public class ExampleRest {
public static final String EXAMPLE_URI = "/examples";
#RequestMapping(value = "/search", method = POST)
public Page<ExampleDto> search(#RequestBody(required = false) Searchable searchable, Pageable pageable) {
return exampleService.findAll(searchable, pageable);
}
}
The Searchable object contains information to create a JPASpecification. It's pretty much a dto. I would like to make this searchable optional. I understood that #RequestBody(required = false) should do the trick.
I have the following test, where I want to test a request without any request body.
#Test
public void post_NoCriteria_Ok() {
RequestEntity requestEntity = new RequestEntity(HttpMethod.POST, URI.create(ExampleRest.EXAMPLE_URI + "/search"));
ResponseEntity <RestResponsePage<ExampleDto>> response = restTemplate.exchange(requestEntity, new ParameterizedTypeReference <RestResponsePage<ExampleDto>> () {});
Assert.assertEquals(HttpStatus.OK, response.getStatusCode());
}
If I run this test, it keeps failing with this response from the RestController:
<415 Unsupported Media Type,Page 1 of 1 containing UNKNOWN
instances,{Content-Type=[application/json;charset=UTF-8],
Transfer-Encoding=[chunked], Date=[Wed, 13 Sep 2017 10:10:22 GMT]}>
The Code execution does not even enter search method implementation inside of the RestController.
As soon I provide an empty Searchable for the test, it runs through.
Is the implementation of #RequestBody(required = false) buggy, or what am I doing wrong here?
You need to set Content-Type as "application/json" in your request while sending from #Test file.

REST multipart mixed request (file+json) with Spring

I need to send a file alongside with a json to my Spring Controller. I have the following controller class:
#Controller
#RequestMapping("/perform")
public class PerformController {
...
#RequestMapping(path = "gopdf", method = RequestMethod.POST, consumes = { "multipart/mixed" })
#ResponseStatus(HttpStatus.CREATED)
public void handleFileUpload(#RequestPart("file") MultipartFile file, #RequestPart("map") String map, HttpServletResponse response) throws Exception {
...
}
}
But when I curl on my server with the following command :
curl -H "Content-Type: multipart/form-data" -F "map=#map.json; type=application/json" -F "content=#SMP.docx" -X POST localhost:9000/perform/gopdf-i -v
I get 415 unsupported Media Type !
Any clue ?
I've found the solution:
I need to use #RequestParam instead of #RequestPart
#RequestMapping(path = "gopdf", method = RequestMethod.POST, consumes = { "multipart/form-data" })
#ResponseStatus(HttpStatus.OK)
public void handleFileUpload2(#RequestParam("file") MultipartFile file, #RequestParam("map") String jsonMap,
HttpServletResponse response) throws Exceptio
The consumes thing in the other answers didn't do crap for me. The key was getting the specific multipart/* types I wanted to support onto some headers key in the RequestMapping. It was really difficult to figure out, mostly guess work and stare at the Spring source code. I'm kind-of underwhelmed with Spring's support for this, but I have managed to make it work in our Spring Boot App, but only with Tomcat?!? Something called the MultipartResolver chokes when you configure your Boot application to use Jetty...so long Jetty. But I digress...
In my Controller I set up for multipart/mixed or multipart/form-data like...
#RequestMapping(value = "/user/{userId}/scouting_activities", method = RequestMethod.POST,
headers = {"content-type=multipart/mixed","content-type=multipart/form-data"})
public ResponseEntity<String> POST_v1_scouting_activities(
#RequestHeader HttpHeaders headers,
#PathVariable String userId,
#RequestPart(value = "image", required = false) MultipartFile image,
#RequestPart(value = "scouting_activity", required = true) String scouting_activity_json) {
LOG.info("POST_v1_scouting_activities: headers.getContentType(): {}", headers.getContentType());
LOG.info("POST_v1_scouting_activities: userId: {}", userId);
LOG.info("POST_v1_scouting_activities: image.originalFilename: {}, image: {}",
(image!=null) ? image.getOriginalFilename() : null, image);
LOG.info("POST_v1_scouting_activities: scouting_activity_json.getType().getName(): {}, scouting_activity: {}",
scouting_activity_json.getClass().getName(), scouting_activity_json);
return new ResponseEntity<String>("POST_v1_scouting_activities\n", HttpStatus.OK);
}
That headers thing let it uniquely identify the multipart content types it was willing to take a shot at. This lets curls like...
curl -i -X POST 'http://localhost:8080/robert/v1/140218/scouting_activities' \
-H 'Content-type:multipart/mixed' \
-F 'image=#Smile_128x128.png;type=image/png' \
-F 'scouting_activity={
"field": 14006513,
"longitude": -93.2038253,
"latitude": 38.5203231,
"note": "This is the center of Dino Head.",
"scouting_date": "2017-01-19T22:56:04.836Z"
};type=application/json'
...or...
curl -i -X POST 'http://localhost:8080/robert/v1/140218/scouting_activities' \
-H 'Content-type:multipart/form-data' \
-F 'image=#Smile_128x128.png;type=image/png' \
-F 'scouting_activity=#scoutingFrackingCurl.json;type=application/json'
work.
The multipart/mixed for spring webflux(2.1.0) did not work for me. Here is an alternative approach that worked
Working - spring-boot-starter-web/Multipart[] - upload files where
one is the payload, another is a file itself. In my case since the payload was constant, it worked.
Not working - spring-boot-starter-webflux/Flux. The flux is empty. I tried this https://github.com/spring-projects/spring-boot/issues/13268, but it didn't work
It's maybe related to your request mapping annotation. I think accept value is missing to determine what service can accept :
Example :
#RequestMapping(path = "gopdf", method = RequestMethod.POST, consumes = { "multipart/mixed" }, accept = MediaType.MULTIPART_FORM_DATA_VALUE)
Import :
import org.springframework.http.MediaType;
Documentation/API : http://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/http/MediaType.html

Content type 'application/x-www-form-urlencoded;charset=UTF-8' not supported for #RequestBody MultiValueMap

Based on the answer for problem with x-www-form-urlencoded with Spring #Controller
I have written the below #Controller method
#RequestMapping(value = "/{email}/authenticate", method = RequestMethod.POST
, produces = {"application/json", "application/xml"}
, consumes = {"application/x-www-form-urlencoded"}
)
public
#ResponseBody
Representation authenticate(#PathVariable("email") String anEmailAddress,
#RequestBody MultiValueMap paramMap)
throws Exception {
if(paramMap == null || paramMap.get("password") == null) {
throw new IllegalArgumentException("Password not provided");
}
}
the request to which fails with the below error
{
"timestamp": 1447911866786,
"status": 415,
"error": "Unsupported Media Type",
"exception": "org.springframework.web.HttpMediaTypeNotSupportedException",
"message": "Content type 'application/x-www-form-urlencoded;charset=UTF-8' not supported",
"path": "/users/usermail%40gmail.com/authenticate"
}
[PS: Jersey was far more friendly, but couldn't use it now given the practical restrictions here]
The problem is that when we use application/x-www-form-urlencoded, Spring doesn't understand it as a RequestBody. So, if we want to use this
we must remove the #RequestBody annotation.
Then try the following:
#RequestMapping(
path = "/{email}/authenticate",
method = RequestMethod.POST,
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = {
MediaType.APPLICATION_ATOM_XML_VALUE,
MediaType.APPLICATION_JSON_VALUE
})
public #ResponseBody Representation authenticate(
#PathVariable("email") String anEmailAddress,
MultiValueMap paramMap) throws Exception {
if (paramMap == null &&
paramMap.get("password") == null) {
throw new IllegalArgumentException("Password not provided");
}
return null;
}
Note that removed the annotation #RequestBody
answer: Http Post request with content type application/x-www-form-urlencoded not working in Spring
It seems that now you can just mark the method parameter with #RequestParam and it will do the job for you.
#PostMapping( "some/request/path" )
public void someControllerMethod( #RequestParam Map<String, String> body ) {
//work with Map
}
Add a header to your request to set content type to application/json
curl -H 'Content-Type: application/json' -s -XPOST http://your.domain.com/ -d YOUR_JSON_BODY
this way spring knows how to parse the content.
In Spring 5
#PostMapping( "some/request/path" )
public void someControllerMethod( #RequestParam MultiValueMap body ) {
// import org.springframework.util.MultiValueMap;
String datax = (String) body .getFirst("datax");
}
#RequestBody MultiValueMap paramMap
in here Remove the #RequestBody Annotaion
#RequestMapping(value = "/signin",method = RequestMethod.POST)
public String createAccount(#RequestBody LogingData user){
logingService.save(user);
return "login";
}
#RequestMapping(value = "/signin",method = RequestMethod.POST)
public String createAccount( LogingData user){
logingService.save(user);
return "login";
}
like that
Simply removing #RequestBody annotation solves the problem (tested on Spring Boot 2):
#RestController
public class MyController {
#PostMapping
public void method(#Valid RequestDto dto) {
// method body ...
}
}
I met the same problem when I want to process my simple HTML form submission (without using thymeleaf or Spring's form tag) in Spring MVC.
The answer of Douglas Ribeiro will work very well. But just in case, for anyone, like me, who really want to use "#RequestBody" in Spring MVC.
Here is the cause of the problem:
Spring need to ① recognize the "Content-Type", and ② convert the
content to the parameter type we declared in the method's signature.
The 'application/x-www-form-urlencoded' is not supported, because, by
default, the Spring cannot find a proper HttpMessageConverter to do
the converting job, which is step ②.
Solution:
We manually add a proper HttpMessageConverter into the Spring's
configuration of our application.
Steps:
Choose the HttpMessageConverter's class we want to use. For
'application/x-www-form-urlencoded', we can choose
"org.springframework.http.converter.FormHttpMessageConverter".
Add the FormHttpMessageConverter object to Spring's configuration,
by calling the "public void
configureMessageConverters(List<HttpMessageConverter<?>>
converters)" method of the "WebMvcConfigurer" implementation class
in our application. Inside the method, we can add any
HttpMessageConverter object as needed, by using "converters.add()".
By the way, the reason why we can access the value by using "#RequestParam" is:
According to Servlet Specification (Section 3.1.1):
The following are the conditions that must be met before post form
data will be populated to the parameter set: The request is an HTTP
or HTTPS request. 2. The HTTP method is POST. 3. The content type is
application/x-www-form-urlencoded. 4. The servlet has made an initial
call of any of the getParameter family of methods on the request
object.
So, the value in request body will be populated to parameters. But in Spring, you can still access RequestBody, even you can use #RequstBody and #RequestParam at the same method's signature.
Like:
#RequestMapping(method = RequestMethod.POST, consumes = {MediaType.APPLICATION_FORM_URLENCODED_VALUE})
public String processForm(#RequestParam Map<String, String> inputValue, #RequestBody MultiValueMap<String, List<String>> formInfo) {
......
......
}
The inputValue and formInfo contains the same data, excpet for the type for "#RequestParam" is Map, while for "#RequestBody" is MultiValueMap.
I wrote about an alternative in this StackOverflow answer.
There I wrote step by step, explaining with code. The short way:
First: write an object
Second: create a converter to mapping the model extending the AbstractHttpMessageConverter
Third: tell to spring use this converter implementing a WebMvcConfigurer.class overriding the configureMessageConverters method
Fourth and final: using this implementation setting in the mapping inside your controller the consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE and #RequestBody in front of your object.
I'm using spring boot 2.
#PostMapping(path = "/my/endpoint", consumes = { MediaType.APPLICATION_FORM_URLENCODED_VALUE })
public ResponseEntity<Void> handleBrowserSubmissions(MyDTO dto) throws Exception {
...
}
That way works for me
You can try to turn support on in spring's converter
#EnableWebMvc
#Configuration
public class WebConfig implements WebMvcConfigurer {
#Override
public void extendMessageConverters(List<HttpMessageConverter<?>> converters) {
// add converter suport Content-Type: 'application/x-www-form-urlencoded'
converters.stream()
.filter(AllEncompassingFormHttpMessageConverter.class::isInstance)
.map(AllEncompassingFormHttpMessageConverter.class::cast)
.findFirst()
.ifPresent(converter -> converter.addSupportedMediaTypes(MediaType.APPLICATION_FORM_URLENCODED_VALUE));
}
}
Just add an HTTP Header Manager if you are testing using JMeter :

Resources