The url-string contains a back-slash character that needs to be encoded. The url string is as follows.
String folder = "\\Foo\\Bar\\"; // some folder search path.
String urlString= "http://localhost:8081/certificates/?mypath=%5CFoo%5CBar%5C" // (after encoding)
Here I use Spring RestTemplate to do a GET request. I setup a mock-server to examine the request in detail (mock server setup using Mulesoft, if u must know!).
ResponseEntity<String> responseEntity = api.exchange(urlString, HttpMethod.GET, new HttpEntity<>(new HttpHeaders()), String.class);
Here I use plain vanilla Java URLConnection to perform the request. Attached image with detailed request snapshot.
// 2. Plain vanilla java URLConnection. "result.toString()" has certificate match.
StringBuilder result = new StringBuilder();
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("X-Venafi-Api-Key", apiKey);
conn.setRequestMethod("GET");
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
result.append(line);
}
rd.close();
System.out.println(result.toString());
In the images, you can see that the queryString value is different for these two requests. One of them shows \\ while the other shows %5C, although the parsed parameter value for myPath is still the same.
I am having to deal with an api that seems to work if-and-only-if the queryString looks like the former (i.e. "\\"). Why does the parsed queryString for Spring show "%5C" while this value shows double-backslash for requests originating from plain Java, curl, and even a simple browser?
What baffles me EVEN more, is that just about everything about the two HTTP Requests are IDENTICAL! And yet, why does the queryString/requestUri parse differently for these two requests? Shouldn't it be that a HTTP GET method is completely defined by its header contents and the requestUri? What am I missing to capture in these two GET requests?
Lots of questions. Spent an entire day, but at least I could verify that the way the requestUri/queryString is parsed seems to align with how the remote api-server responds.
Thanks.
Did some digging around the following morning. Turn out, with
ResponseEntity<String> responseEntity = api.exchange(urlString, HttpMethod.GET, new HttpEntity<>(new HttpHeaders()), String.class);
You should NOT have the "urlString" already encoded. The 'exchange' method does that encoding for you under-the-hood.
Related
We have a Controller running on tomcat 8.5.32 which receives a POST request with query params
/{path_param}/issue?title=4&description=5
request body is empty
Then controller redirects this request to Spring Boot microservice with tomcat 9.0.27.
At line
CloseableHttpResponse result = httpClient.execute(request);
request.getURI().getQuery() equals&title=1&description=2
But when it arrives to microservice parameters are duplicated (title=[4,4]&description=[5,5]).
This is the code which redirects request to microservice
private static <T, U> T executePostRequest(String url, U body, HttpServletRequest httpServletRequest, Function<String, T> readValueFunction) {
try (CloseableHttpClient httpClient = HttpClientBuilder.create().build()) {
URIBuilder uriBuilder = new URIBuilder(url);
httpServletRequest.getParameterMap().forEach((k, v) -> Arrays.stream(v).forEach(e -> uriBuilder.addParameter(k, e)));
HttpPost request = new HttpPost(uriBuilder.build());
CloseableHttpResponse result = httpClient.execute(request);
String json = EntityUtils.toString(result.getEntity(), "UTF-8");
handleResultStatus(result, json);
return readValueFunction.apply(json);
} catch (IOException | URISyntaxException e) {
...
}
}
I found that there was similar issue with jetty and it was fixed but did not find anything related to tomcat - and how it can be fixed.
I saw also this topic whith suggestion how to handle duplicated parameters in spring boot but i am wondering if anyone else experienced same issue and how did you resolve it if yes.
It's not a bug, it's a feature present in every servlet container.
The Servlet API does not require for the request parameters to have unique names. If you send a POST request for http://example.com/app/issue?title=1&description=2 with a body of:
title=3&description=4
then each parameter will have multiple values: title will have values 1 and 3, while description will have values 2 and 4 in that order:
Data from the query string and the post body are aggregated into the request
parameter set. Query string data is presented before post body data. For example, if
a request is made with a query string of a=hello and a post body of a=goodbye&a=
world, the resulting parameter set would be ordered a=(hello, goodbye, world).
(Servlet specification, section 3.1)
If you want to copy just the first value of the parameters use:
httpServletRequest.getParameterMap()//
.forEach((k, v) -> uriBuilder.addParameter(k, v[0]));
I can easily get the expected JSON response if I send the following get request from my browser:
http://www.bookandwalk.hu/api/AdminTransactionList?password=XXX&begindate=2016-04-30&enddate=2016-10-12&corpusid=HUBW
I tried to use SPRING BOOT 1.4 to create a small demo app to see how rest calls work in Spring.
So I created a POJO representing my domain object and I requested the list of domain objects by the following method invocation:
String startDate=new SimpleDateFormat("yyyy-MM-dd").format(start.getTime());
String endDate=new SimpleDateFormat("yyyy-MM-dd").format(end.getTime());
UriComponents uri=UriComponentsBuilder.newInstance().scheme("http").host("www.bookandwalk.hu").path("/api/AdminTransactionList").queryParam("password","xxx").queryParam("begindate",startDate).queryParam("enddate",endDate).queryParam("corpusid","HUBW").build().encode();
LOG.log(Level.INFO,"{0} were called as a rest call",uri.toString());
ResponseEntity<List<BandWTransaction>> transResponse =
restTemplate.exchange(uri.toString(),
HttpMethod.GET, null, new ParameterizedTypeReference<List<BandWTransaction>>() {
});
List<BandWTransaction> transactions = transResponse.getBody();
I got the following exception:
org.springframework.web.client.HttpClientErrorException: 404 Not Found
As I logged the uri.toString(), I copied it to my browser to double check the is there any typos in my uri but it was working without any failure.
Does Anybody have idea why the same string works from the browser but not from the code?
It seems that you should specify a user agent header in the request for this webapp. Use a HttpEntity object to set this header.
final HttpHeaders headers = new HttpHeaders();
headers.set("User-Agent", "eltabo");
final HttpEntity<String> entity = new HttpEntity<String>(headers);
ResponseEntity<List<BandWTransaction>> transResponse =
restTemplate.exchange(uri.toString(),
HttpMethod.GET, entity,
new ParameterizedTypeReference<List<BandWTransaction>>() {});
Hope it helps.
I have been trying to use accent characters in URL to call SOLR.
My Url looks like this:
"http://host:8983/solr/principal/select?q=**name:%22Michaël.e%22**"
When fire the URL from browser I get the correct result but when try from RestTempalte.exchange(URI,HttpMethod.GET, entity, String.class)
The log I see on SOLR is showing the accent characters being coverted to "?" as shown below
q=(name:"Micha?.e")
I have set RestTemple request charSet to "UTF-8" it still does the same.
My SOLR is running on Jetty.
You can try to encode HTML characters before calling RestTemplate using URLEncoder
String baseUri = "http://host:8983/solr/principal/select?q=**name:%22";
// TODO get name from somewhere
String name = "Michaël.e";
String encodedName= = URLEncoder.encode(name, "UTF-8");
RestTempalte.exchange(baseUri + encodedName + "%22**",HttpMethod.GET, entity, String.class);
I want to capture the current page and send it to an application that converts it to pdf.
This is what I am using:
FacesContext facesContext=FacesContext.getCurrentInstance();
HttpServletResponse response = (HttpServletResponse)
facesContext.getExternalContext().getResponse();
HttpServletRequest request = (HttpServletRequest) facesContext.getExternalContext().getRequest();
// RequestPrinter.debugString();
response.reset();
// download a pdf file
response.setContentType("application/pdf");
response.setHeader("Content-Disposition","attachment;filename="+new Date().toString()+".pdf");
prince.setVerbose(true);
prince.setLog(logFile);
try{
//getPath() to the page the user is currently on
URL pagePath=new URL(this.getPath());
URLConnection urlConnection = pagePath.openConnection();
urlConnection.setDoOutput(true);
int length = urlConnection.getContentLength();
//Lets use inputStream
BufferedInputStream bis=new BufferedInputStream(urlConnection.getInputStream());
response.setContentLength(length);
//this.getPageUsingJSoup().data().getBytes();
//call prince and pass params for inputstream outputStream
prince.convert(bis,response.getOutputStream());
urlConnection.getInputStream().close();
}catch(MalformedURLException mu){
mu.printStackTrace();
}
catch(IOException ioe){
ioe.printStackTrace();
}
facesContext.responseComplete();
Since the website requires authentication, the pdf generated is the loging error page.
Is there a way to capture the page's content that uses the current user's session?
Thank you in advance.
Just request the page in the same HTTP session as the current request. If your webapp supports URL rewriting (as by default), then just append session ID as jsessionid path fragment:
String sessionId = ((HttpSession) externalContext.getSession()).getId();
InputStream input = new URL("http://localhost:8080/context/page.jsf;jsessionid=" + sessionId).openStream();
// ...
Or if your webapp doesn't accept URL rewriting, but accepts cookies only, then set it as a request cookie the usual way:
URLConnection connection = new URL("http://localhost:8080/context/page.jsf").openConnection();
connection.setRequestProperty("Cookie", "JSESSIONID=" + sessionId);
InputStream input = connection.getInputStream();
// ...
Note that I removed setDoOutput() since you don't seem to be interested in performing a POST request.
I do not know how to capture the page's content using the current user's session, but I can suggest another way to do it - you could move the pdf conversion logic inside a Selenium test-case and use the test-case to navigate and login to the page requiring authentication. After the automated tc has logged in, you could call your pdf conversion logic...?
Yes of course there is. You are sending this content, so you have it. You should store the Content Object. If you dont have it, inspect your byte streams. The content should be there ;)
There of couple of websites which allow you to convert the entire page to pdf and save it as .pdf file. Try out the site http://pdfcrowd.com/ Hope this helps you.
For example, if the post url is:
http://www.wolf.com/pcap/search?stime={stime}&etime=${etime}&bpf=${bpf}
then can we do this:
Map<String, String> vars = new HashMap<String, String>();
vars.put("bpf", bpf);
...
responseString = restTemplate.postForObject(url, null, String.class,vars);
If bpf is a String, then is there a limitation of the size of bpf? Can it be any size?
Unfortunately the answer is: "It depends".
More precisely: as you append the bpf as a parameter to the URL it does not really matter if you are doing a POST or a GET. Sometimes there are restrictions on the length of an URL a server will handle, but that depends on what the server accepts, and cannot be determined from the RestTemplate, which is the client.
For example if the server you send the REST request to is a tomcat, then the maximal value of the complete header (URL, HTTP-Header etc) is by default 8kB for tomcat 6.0 or higher; see e.g. https://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat
Just in case if you have control over the server side, too, you can change the expected interface by not sending the bpf as parameter, but as request body, like:
Map<String, String> vars = new HashMap<String, String>();
// vars.put("bpf", bpf); <--- not needed
responseString = restTemplate.postForObject(url, bpf, String.class, vars);
(and then of course get the bpf on the server from the request body instead).
Otherwise you are out of luck and have to limit the length of the URL. Maybe use a proxy or network sniffer to see what extra Headers are actually send and subtract that from the 8kB limit to get the maximal length of the URL.