I have a Rest service where get call if I send multiple invalid/extra & and = characters then also my endpoint does not throw any error. I would like to throw back invalid request error if url contains any extra special character like & or =.
for example:
http://localhost:8080/myservice?rollNo=03456789321&school=Myschool //This is Okay for me
http://localhost:8080/myservice?rollNo=03456789321&school= //should throw error as school is not having value
http://localhost:8080/myservice?rollNo=03456789321&&&&school=Myschool
//should throw error as &&&& is multiple where it should only one
http://localhost:8080/myservice?rollNo=03456789321&= //should throw error as &= is there at end having no sence.
Note that , I am hitting these request from postman , and I have doubt that postman do something with these parameters, cause I am not able to find these extra characters in spring boot while debugging.
Any way through which i can get whole request url in my controller so that I can find out for these charecters comming?
Any built in springboot annotation is there to handle such a cases?
I got my problem solved.
After lot of research , and some observation I came to know that when you pass any number of characters among & and = in request url, the rest client tools like postman , or advanced rest client will refine the url before hitting actual server and remove those extra un-necessary characters. SO if you write multiple &&&& or == charecters in url , it will consider each extra & as blank parameter and will ignore while sending final request, only of those characters which has parameter names besides it it will taken as part of refined request.
you can see in screenshot bellow:
You can Use #RequestParam in your Spring Boot rest Controller
Something of the following
#GetMapping(value = "/myservice")
public boolean doSomething(#RequestParam("rollNo") Integer rollNo , #RequestParam("school") String school) {
doValidation(rollNo,school);
// Do Something
return true;
}
#RequestParam will make sure that your Url need to have these Params rollNo & school. Without it it will throw error.
But if you were to pass an empty string like &school= in your second example. The controller will get an empty String.
You can add a basic validation layer right before you do anything in you controller to handle this condition.
Related
I'm working in a spring boot application, where i'm getting violation as, "number of parameters should be less than 8"
i,m passing all the parameters through request param
I'm passing exactly 8 parameters all are mandatory
any other way to overcome this ?
In general, it looks like bad API design that you have so many request params. Request params are normally only used for things like filter and sorting options, but not to provide any actual data. Instead, use the request body.
Nevertheless, you can also get all parameters as a HashMap using:
#PostMapping("/api/example")
#ResponseBody
public String examplePost(#RequestParam Map<String,String> allParams) {
return "Parameters are " + allParams.entrySet();
}
I have a Spring application that receives a request like http://localhost/foo?email=foo+bar#example.com. This triggers a controller that roughly looks like this:
#RestController
#RequestMapping("/foo")
public class FooController extends Controller {
#GetMapping
public void foo(#RequestParam("email") String email) {
System.out.println(email)
}
}
By the time I can access email, it's been converted to foo bar#example.com instead of the original foo+bar#example.com. According to When to encode space to plus (+) or %20? this should only happen in requests where the content is application/x-www-form-urlencoded. My request has a content type of application/json. The full MIME headers of the request look like this:
=== MimeHeaders ===
accept = application/json
content-type = application/json
user-agent = Dashman Configurator/0.0.0-dev
content-length = 0
host = localhost:8080
connection = keep-alive
Why is Spring then decoding the plus as a space? And if this is the way it should work, why isn't it encoding pluses as %2B when making requests?
I found this bug report about it: https://jira.spring.io/browse/SPR-6291 which may imply that this is fixed on version 3.0.5 and I'm using Spring > 5.0.0. It is possible that I may misinterpreting something about the bug report.
I also found this discussion about RestTemplate treatment of these values: https://jira.spring.io/browse/SPR-5516 (my client is using RestTemplate).
So, my questions are, why is Spring doing this? How can I disable it? Should I disable it or should I encode pluses on the client, even if the requests are json?
Just to clarify, I'm not using neither HTML nor JavaScript anywhere here. There's a Spring Rest Controller and the client is Spring's RestTemplate with UriTemplate or UriComponentsBuilder, neither of which encode the plus sign the way Spring decodes it.
Original Answer
You are mixing 2 things, a + in the body of the request would mean a space when header has application/x-www-form-urlencoded. The body or content of the request would be dependent on the headers but a request can just have a url and no headers and no body.
So the encoding of a URI cannot be controlled by any headers as such
See the URL Encoding section in https://en.wikipedia.org/wiki/Query_string
Some characters cannot be part of a URL (for example, the space) and some other characters have a special meaning in a URL: for example, the character # can be used to further specify a subsection (or fragment) of a document. In HTML forms, the character = is used to separate a name from a value. The URI generic syntax uses URL encoding to deal with this problem, while HTML forms make some additional substitutions rather than applying percent encoding for all such characters. SPACE is encoded as '+' or "%20".[10]
HTML 5 specifies the following transformation for submitting HTML forms with the "get" method to a web server.1 The following is a brief summary of the algorithm:
Characters that cannot be converted to the correct charset are replaced with HTML numeric character references[11]
SPACE is encoded as '+' or '%20'
Letters (A–Z and a–z), numbers (0–9) and the characters '*','-','.' and '_' are left as-is
All other characters are encoded as %HH hex representation with any non-ASCII characters first encoded as UTF-8 (or other specified encoding)
The octet corresponding to the tilde ("~") is permitted in query strings by RFC3986 but required to be percent-encoded in HTML forms to "%7E".
The encoding of SPACE as '+' and the selection of "as-is" characters distinguishes this encoding from RFC 3986.
And you can see the same behaviour on google.com as well from below screenshots
Also you can see the same behaviour in other frameworks as well. Below is an example of Python Flask
So what you are seeing is correct, you are just comparing it with a document which refers to the body content of a request and not the URL
Edit-1: 22nd May
After debugging it seems the decoding doesn't even happen in Spring. I happens in package org.apache.tomcat.util.buf; and the UDecoder class
/**
* URLDecode, will modify the source.
* #param mb The URL encoded bytes
* #param query <code>true</code> if this is a query string
* #throws IOException Invalid %xx URL encoding
*/
public void convert( ByteChunk mb, boolean query )
throws IOException
{
int start=mb.getOffset();
And below is where the conversion stuff actually happens
if( buff[ j ] == '+' && query) {
buff[idx]= (byte)' ' ;
} else if( buff[ j ] != '%' ) {
This means that it is an embedded tomcat server which does this translation and spring doesn't even participate in this. There is no config to change this behaviour as seen in the class code. So you have to live with it
SPR-6291 fixed this problem in v3.0.5 but this remains unresolved in some other cases like SPR-11047 is still unresolved. While SPR-6291's priority was Major, SPR-11047's priority is Minor.
I faced this problem when I was working on REST API in old Spring last year. There are multiple ways we can get data in Spring controller. So two of them are via #RequestParam or #PathVariable annotation
As others mentioned I think its spring's internal issue and does not specifically belong to URL encoding because I was sending data over POST request but it is somewhat encoding problem. But I also agree with others as now it remains problematic only in URL.
So there are two solutions I know:
You can use #PathVariable instead of #RequestParam because as of SPR-6291 this plus sign issue is fixed in #PathVariable and still remains open for #RequestParam as SPR-11047
My version of spring was not even accepting plus sign via #PathVariable annotation, so this is how I overcome the problem (I don't remember it step by step but it will give you hint).
In your case you can get the fields via JS and escape the plus sign before sending a request. Something like this:
var email = document.getElementById("emailField").value;
email = email.replace('+', '%2B');
If you have this request:
http://localhost/foo?email=foo+bar#example.com
then the original is foo bar#example.com. If you say the original should be foo+bar#example.com then the request should be:
http://localhost/foo?email=foo%2Bbar#example.com
So Spring is working as supposed to. Maybe on client you should check if the URI is properly encoded. The client-side URL encoding is responsible for building a correct HTTP request.
See encodeURI() if you generate the request in JavaScript or uriToString() if you generate the request in Spring.
Build your request string (the part after ?), without any encoding, with unencoded values like foo+bar#email.com, and only in the end, before actually using it in GET, encode all of it with whatever is available on the client platform. If you want to use POST then you should encode it according to the MIME type of your choice.
I have a rest controller setup like below
#RequestMapping(value = {"getDataFromSpaceForType/{gridName}/{spaceName}/{dataType}/{fieldNames}/{criteria}"}, method = GET, produces = "application/json")
public EntriesForTypeName getDataFromSpaceForType(#PathVariable Map<String, String> dataRequestVariables) throws Exception {
The last field criteria can contain multiple forward slashes. I am unable to handle such scenario. It breaks with 404 error
Also i dont know how many slashes might come, so it becomes impossible to create multiple methods.
If you know that your URL may contain slashes, all you have to do is to encode the URL before you send/use it.
If your front-end is angularJS then you can do it like this:
How to generate url encoded anchor links with AngularJS?
Or if you use jquery then:
URL Encode a string in jQuery for an AJAX request
When I am passing email address as path variable it is throwing following error
Console --> 2015-02-09 16:30:06,634 WARN - GET request for "http://localhost:8181/abc/users/testabghtmail#gmail.com" resulted in 406 (Not Acceptable); invoking error handler
Exception in thread "main" org.springframework.web.client.HttpClientErrorException: 406 Not Acceptable
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
at org.springframework.web.client.RestTemplate.handleResponseError(RestTemplate.java:607)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:565)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:521)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:439)
at RestClient.main(RestClient.java:35)
I have tried lots of cases, so I finally found the problem with last domain like .com and .org which are internationalize domains. So instead of "testabghtmail#gmail.com" if I pass "testabghtmail#gmail.dom" it will work perfectly fine.
My code is
#RequestMapping(value = "users/{emailId:.*}", method = RequestMethod.GET)
public Object searchUser(#PathVariable("emailId") String emailId){
logger.info("Inside search user --> emailId " + emailId);
return userService.findUserByuserId(emailId);
}
I found no answer to this. I think it's an http rule we can't have domains at last in prameters and can make a request.
So work around to this is just pass a slash at the end of the url and there you go.
Like modify "http://localhost:8181/abc/users/testabghtmail#gmail.com/" with "http://localhost:8181/abc/users/testabghtmail#gmail.com". And thanks to spring rest architecture, it will automatically omit the last slash and you will get "testabghtmail#gmail.com" as a parameter value.
Let me know if you guys find something else.
I think this is not possible, but I need confirmation before throwing it away...
I have a GET REST endpoint with this pattern
/networks/{networkId}/publishers/{publisherId}/ratings
the problem I am facing is, publisherId could have '/' in its id, like the id could be "opt/foo/bar" (we have not control over this id, it is given to us by our clients).
So
/networks/68/publishers/opt/foo/bar/ratings - obviously does not work, getting a url not fond error.
/networks/68/publishers/opt%2ffoo%2fbar/ratings - also does not work. same error.
I know passing it as a query param will work. But I want to know if there is a way to make it work having it as a path param?
Thanks!
URL encoding is the right way to go but it looks like your container is decoding the slash before Jersey receives it.
Assuming you are using Tomcat, you can attempt to persuade Tomcat to allow the encoding, try:
tomcat/bin/setenv.bat
set
CATALINA_OPTS="-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true"
I don't know if other containers have similar issues and settings.
I have not tried this, but theoretically this should work in Jersey:
#Path("/networks/{networkId}/publishers/")
#GET
public String get(#PathParam("networkId") String networkId, #Context UriInfo ui) {
java.util.List<PathSegment> segments = ui.getPathSegments();
// Last segment is "ratings", the rest is your publisherId.
}