We have a Controller running on tomcat 8.5.32 which receives a POST request with query params
/{path_param}/issue?title=4&description=5
request body is empty
Then controller redirects this request to Spring Boot microservice with tomcat 9.0.27.
At line
CloseableHttpResponse result = httpClient.execute(request);
request.getURI().getQuery() equals&title=1&description=2
But when it arrives to microservice parameters are duplicated (title=[4,4]&description=[5,5]).
This is the code which redirects request to microservice
private static <T, U> T executePostRequest(String url, U body, HttpServletRequest httpServletRequest, Function<String, T> readValueFunction) {
try (CloseableHttpClient httpClient = HttpClientBuilder.create().build()) {
URIBuilder uriBuilder = new URIBuilder(url);
httpServletRequest.getParameterMap().forEach((k, v) -> Arrays.stream(v).forEach(e -> uriBuilder.addParameter(k, e)));
HttpPost request = new HttpPost(uriBuilder.build());
CloseableHttpResponse result = httpClient.execute(request);
String json = EntityUtils.toString(result.getEntity(), "UTF-8");
handleResultStatus(result, json);
return readValueFunction.apply(json);
} catch (IOException | URISyntaxException e) {
...
}
}
I found that there was similar issue with jetty and it was fixed but did not find anything related to tomcat - and how it can be fixed.
I saw also this topic whith suggestion how to handle duplicated parameters in spring boot but i am wondering if anyone else experienced same issue and how did you resolve it if yes.
It's not a bug, it's a feature present in every servlet container.
The Servlet API does not require for the request parameters to have unique names. If you send a POST request for http://example.com/app/issue?title=1&description=2 with a body of:
title=3&description=4
then each parameter will have multiple values: title will have values 1 and 3, while description will have values 2 and 4 in that order:
Data from the query string and the post body are aggregated into the request
parameter set. Query string data is presented before post body data. For example, if
a request is made with a query string of a=hello and a post body of a=goodbye&a=
world, the resulting parameter set would be ordered a=(hello, goodbye, world).
(Servlet specification, section 3.1)
If you want to copy just the first value of the parameters use:
httpServletRequest.getParameterMap()//
.forEach((k, v) -> uriBuilder.addParameter(k, v[0]));
Related
I want to be able to extract the List<Payload> from the Mono<List<Payload>> to pass it to a downstream service for processing (or maybe return from the read(RequestParams params) method, instead of it returning void):
#PostMapping("/subset")
public void read(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
....
}
where reader.read(...) is a method on an autowired Spring service utilizing a webClient to get the data from external web service API:
public Mono<List<Payload>> read(String date, String assetClasses, String firmAccounts, String id, String password) {
Flux<Payload> nodes = client
.get()
.uri(uriBuilder -> uriBuilder
.path("/api/subset")
.queryParam("payloads", true)
.queryParam("date", date)
.queryParam("assetClasses", assetClasses)
.queryParam("firmAccounts", firmAccounts)
.build())
.headers(header -> header.setBasicAuth("abc123", "XXXXXXX"))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, response -> {
System.out.println("4xx error");
return Mono.error(new RuntimeException("4xx"));
})
.onStatus(HttpStatus::is5xxServerError, response -> {
System.out.println("5xx error");
return Mono.error(new RuntimeException("5xx"));
})
.bodyToFlux(Payload.class);
Mono<List<Payload>> records = nodes
.collectList();
return records;
}
Doing a blocking result.block() is not allowed in WebFlux and throws an exception:
new IllegalStateException("block()/blockFirst()/blockLast() are blocking, which is not supported in thread ..." ;
What is a proper way to extract the contents of a Mono in WebFlux?
Is it some kind of a subscribe()? What would be the syntax?
Thank you in advance.
There is no "proper way" and that is the entire point. To get the value you need to block, and blocking is bad in webflux for many reasons (that I won't go into right now).
What you should do is to return the publisher all the way out to the calling client.
One of the things that many usually have a hard time understanding is that webflux works with a producer (Mono or Flux) and a subscriber.
Your entire service is also a producer, and the calling client can be seen as the subscriber.
Think of it as a long chain, that starts at the datasource, and ends up in the client showing the data.
A simple rule of thumb is that whomever is the final consumer of the data is the subscriber, everyone else is a producer.
So in your case, you just return the Mono<List<T> out to the calling client.
#PostMapping("/subset")
public Mono<List<Payload>> read(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
return result;
}
While the following does return the value of the Mono observable in the logs:
#PostMapping("/subset")
#ResponseBody
public Mono<ResponseEntity<List<Payload>>> read1(#RequestBody RequestParams params){
Mono<List<Payload>> result = reader.read(params.getDate(), params.getAssetClasses(), params.getFirmAccounts(), params.getUserId(), params.getPassword());
return result
.map(e -> new ResponseEntity<List<PayloadByStandardBasis>>(e, HttpStatus.OK));
}
the understanding I was seeking was a proper way to compose a chain of calls, with WebFlux, whereby a response from one of its operators/legs (materialized as as a result from a webclient call, producing a set of records, as above) could be passed downstream to another operator/leg to facilitate a side effect of saving those records in a DB, or something to that effect.
It would probably be a good idea to model each of those steps as a separate REST endpoint, and then have another endpoint for a composition operation which internally calls each independent endpoint in the right order, or would other design choices be more preferred?
That is ultimately the understanding I was looking for, so if anyone wants to share an example code as well as opinions to better implement the set of steps described above, I'm willing to accept the most comprehensive answer.
Thank you.
I am a newbie to reactive programming in Java. I plan to use spring-webclient instead of restclient as the latter is being decommissioned. I have a situation when I make several http post requests to different endpoints and the response structure is identical. With webclient code as below,
List<Mono<CommonResponse>> monolist = new ArrayList<>();
for(String endpoint : endpoints) {
Mono<CommonResponse> mono = webClient.post()
.uri(URI.create(endPoint))
.body(Mono.just(requestData), RequestData.class)
.retrieve()
.bodyToMono(CommonResponse.class);
monolist.add(mono);
}
I get a mono per request. As the response is common, I would like each mono to be subscribed a common method, but how can I distinguish the endpoints, assuming that the response data is not helping.
Can I pass additional arguments to the method while subscribing?
You can do this in following way. If you have many monos you can treat team as flux which actually means that you have many of Mono. Then you can subscribe all of them with single method. To pass to subscribing method some extra arguments like information about endpoint, you can create extra object with additional information.
Flux<ResponseWithEndpoint> commonResponseFlux = Flux.fromIterable(endpoints)
.flatMap(endpoint -> webClient.post()
.uri(URI.create(endpoint))
.body(Mono.just(requestData), RequestData.class)
.retrieve()
.bodyToMono(CommonResponse.class)
.map(response -> new ResponseWithEndpoint(response, endpoint)));
...
class ResponseWithEndpoint {
CommonResponse commonResponse;
String endpoint;
public ResponseWithEndpoint(CommonResponse commonResponse, String endpoint) {
this.commonResponse = commonResponse;
this.endpoint = endpoint;
}
}
The url-string contains a back-slash character that needs to be encoded. The url string is as follows.
String folder = "\\Foo\\Bar\\"; // some folder search path.
String urlString= "http://localhost:8081/certificates/?mypath=%5CFoo%5CBar%5C" // (after encoding)
Here I use Spring RestTemplate to do a GET request. I setup a mock-server to examine the request in detail (mock server setup using Mulesoft, if u must know!).
ResponseEntity<String> responseEntity = api.exchange(urlString, HttpMethod.GET, new HttpEntity<>(new HttpHeaders()), String.class);
Here I use plain vanilla Java URLConnection to perform the request. Attached image with detailed request snapshot.
// 2. Plain vanilla java URLConnection. "result.toString()" has certificate match.
StringBuilder result = new StringBuilder();
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("X-Venafi-Api-Key", apiKey);
conn.setRequestMethod("GET");
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
result.append(line);
}
rd.close();
System.out.println(result.toString());
In the images, you can see that the queryString value is different for these two requests. One of them shows \\ while the other shows %5C, although the parsed parameter value for myPath is still the same.
I am having to deal with an api that seems to work if-and-only-if the queryString looks like the former (i.e. "\\"). Why does the parsed queryString for Spring show "%5C" while this value shows double-backslash for requests originating from plain Java, curl, and even a simple browser?
What baffles me EVEN more, is that just about everything about the two HTTP Requests are IDENTICAL! And yet, why does the queryString/requestUri parse differently for these two requests? Shouldn't it be that a HTTP GET method is completely defined by its header contents and the requestUri? What am I missing to capture in these two GET requests?
Lots of questions. Spent an entire day, but at least I could verify that the way the requestUri/queryString is parsed seems to align with how the remote api-server responds.
Thanks.
Did some digging around the following morning. Turn out, with
ResponseEntity<String> responseEntity = api.exchange(urlString, HttpMethod.GET, new HttpEntity<>(new HttpHeaders()), String.class);
You should NOT have the "urlString" already encoded. The 'exchange' method does that encoding for you under-the-hood.
I am using Feign with the Apache Http Client and I would like to support the following jax-rs interface:
#POST
#Path("/do_something")
void doSomething(#QueryParam("arg") String arg);
But, ApacheHttpClient uses a RequestBuilder, which converts query parameters for requests without a body/entity into a UrlEncodedFormEntity.
I am converting my APIs to jax-rs, and I do not want to break backwards compatibility. Is there a way to use Feign without adjusting my API? Will the OkHttp or Ribbon clients support POSTs with query params and no body/entity? Is there another java jax-rs client that will support this?
Also, is there a reason why RequestBuilder turns query params into a UrlEncodedFormEntity? Is there an alternative HttpUriRequest builder within the apache-httpclient library that doesn't do this? RequestBuilder's build method has the following lines of code:
if (entity == null && (HttpPost.METHOD_NAME.equalsIgnoreCase(method) || HttpPut.METHOD_NAME.equalsIgnoreCase(method))) {
entity = new UrlEncodedFormEntity(parameters, HTTP.DEF_CONTENT_CHARSET);
} else {
// omitted expected behavior
}
Before switching to Feign, my code constructed a HttpUriRequest with something similar to the following:
URI uri = new URIBuilder()
.setScheme("https")
.setHost("localhost")
.setPath("service/do_something")
.addParameter("arg", "value")
.build();
HttpUriRequest request = new HttpPost(uri);
If you are willing to break the API slightly and maintain support for the #QueryParam, then you could define a request interceptor on the feign client that adds a plain text entity/body to the request:
.requestInterceptor(template -> {
if (template.method().equals(HttpPost.METHOD_NAME) && template.queries().keySet().size() > 0 && template.body() == null) {
template.body(" ");
}
})
Then, your API would change with the following:
#POST
#Consumes(MediaType.TEXT_PLAIN)
#Path("/do_something")
void doSomething(#QueryParam("arg") String arg);
But, this breaks the API since the server now expects/consumes a POST message with a plain text entity/body.
I think the same could be accomplished without the requestInterceptor and with Feign's #Body template:
#POST
#Consumes(MediaType.TEXT_PLAIN)
#Body(" ")
#Path("/do_something")
void doSomething(#QueryParam("arg") String arg);
But, this means that your API would have to include Feign rather than pure jax-rs annotations.
I am calling a micro-service in my rest controller. It works fine when ever there is a successful response from the Micro-service but if there is some error response I fails to pass on the error response back to user. Below is the sample code.
#GetMapping("/{id}/users/all")
public Mono<Employee> findAllProfiles(#PathVariable("id") UUID organisationId,
#RequestHeader(name = "Authorization", required = false) String oauthJwt) {
return webClient.get().uri(prepareUrl("{id}/users/all"), organisationId)
.header("Authorization", oauthJwt).accept(MediaType.APPLICATION_JSON)
.exchange().then(response -> response.bodyToMono(Employee.class));
}
Now if there is any JSON response with error code then web client does not pass on the error response to the controller due to which no information is propagated to the api end user.
You should be able to chain methods from the Mono API. Look for "onError" to see a number of options which allow you to define the behavior when there is an error.
For example, if you wanted to return an "empty" Employee, you could do the following:
.exchange()
.then(response -> response.bodyToMono(Employee.class))
.onErrorReturn(new Employee());