RestTempalete exchange error - spring

I am trying to execute this REST api call as shown below using RESTTemplate
When I run this on browser it runs fine
http://ftc-wbpyrdb201:8080/statdata/InsStatData/_aggrs/getData?avars=%7B%22issuerId%22:19038%7D
But when I use URL in code
http://ftc-wbpyrdb201:8080/statdata/InsStatData/_aggrs/getData?avars={"issuerId":19038}
code is
requestURI="http://ftc-wbpyrdb201:8080/statdata/InsStatData/_aggrs/getData";
UriComponentsBuilder builder =
UriComponentsBuilder.fromUriString(requestURI)
// Add query parameter
.queryParam("avars","{\"issuerId\":19038}");
System.out.println(builder.buildAndExpand(requestURI).toUri());
System.out.println(builder.toUriString());
ResponseEntity<String> responseEntity = restTemplate
.exchange(builder.toUriString() , HttpMethod.GET, null,String.class);
I am getting below error, Please let me know what could be wrong here.
org.springframework.web.client.HttpClientErrorException: 400 Bad Request
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:63) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531) ~[spring-web-4.3.8.RELEASE.jar:4.3.8.RELEASE]

i think you has problem in the query string of the URL, i used your code to print out the uri
System.out.println(builder.toUriString());
the printed string was
http://ftc-wbpyrdb201:8080/statdata/InsStatData/_aggrs/getData?avars=%7B%22issuerId%22:19038%7D
but if you used w3cschool encoder to encode the same url you will get
http://ftc-wbpyrdb201:8080/statdata/InsStatData/_aggrs/getData?avars=%7B%22issuerId%22%3A19038%7D
there is a little difference between the two urls, instead of : the w3school encoder produce %3A

Related

how to retrieve the same error response received by webclient in spring reactive

I reveive a request from client, and to process that I have to make request to azure resource management app.
Now, if the client passes me incorrect info, and if I make direct call from postman to azure API, it returns me very useful info. refer below (below call contains incorrect query params) :
i get response like below in case of incorrect param passed :
{
"error": {
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.MachineLearningServices/workspaces/workspace_XYZ/onlineEndpoints/Endpoint_XYZ' under resource group 'resourceGroupXYZ' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
}
}
Now, I make this query using springboot reactive webclient API.
But I am not sure how to pass the same error back as it contains very useful info. The exception handling methods calls like onErrorReturn etc did not help me here to get the original error msg. :
response = getWebClient()
.post()
.uri(apiUrl)
.headers(h -> authToken)
.retrieve()
.bodyToMono(String.class)
// .onErrorReturn(response)
.block();

parsing HTTP request parameter

I want to pass a physical path from a file server to a web service using Spring boot Rest API
the path I want to send is
E:\\sharedFile\\data
I am sending my parameter as #RequistParam like so
#PostMapping("/loadPicturesIntoDB")
public ResponseEntity<String> loadDB(#RequestParam String path) {
int size = imageService.loadDB(path);
return new ResponseEntity<String>("the database has been loaded with" + size + " files from the provided path ",
HttpStatus.OK);
}
I am getting the following error
2020-10-03 15:18:34.273 INFO 17356 --- [nio-8080-exec-1] o.apache.coyote.http11.Http11Processor : Error parsing HTTP request header
Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in the request target [image/loadPicturesIntoDB?path=%22E:\\sharedFile\\data%22]. The valid characters are defined in RFC 7230 and RFC 3986
at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:486) ~[tomcat-embed-core-9.0.38.jar:9.0.38]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:261) ~[tomcat-embed-core-9.0.38.jar:9.0.38]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.38.jar:9.0.38]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-embed-core-9.0.38.jar:9.0.38]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590) [tomcat-embed-core-9.0.38.jar:9.0.38]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.38.jar:9.0.38]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_231]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_231]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.38.jar:9.0.38]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_231]
Thanks in advance guys
Sending URL in request parameter will be problematic as tomcat by default do not perform encoding.
I would recommend you to send this as part of Request body instead.

Spring Boot Error when redirecting to html-file

I always get a "NoSuchMethodError" in my Spring Boot application when I try to redirect to a html-file and I don't understand where this is coming from. Hope someone can help.
Background:
An external server is sending some information to my server by calling a method via url /resolce.
This method processed the information and shall deliver the information to a html-file via redirecting. The processing of the information works fine, but when it comes to the redirect I get an NoSuchMethodError. I thought the template engine can't find the view, but the configuration seems fine.
Here is the stacktrace:
Exception Processing ErrorPage[errorCode=0, location=/error]
java.lang.NoSuchMethodError: javax.servlet.http.HttpServletRequest.getHttpServletMapping()Ljavax/servlet/http/HttpServletMapping;
at org.apache.catalina.core.ApplicationHttpRequest.setRequest(ApplicationHttpRequest.java:708)
at org.apache.catalina.core.ApplicationHttpRequest.<init>(ApplicationHttpRequest.java:114)
at org.apache.catalina.core.ApplicationDispatcher.wrapRequest(ApplicationDispatcher.java:917)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:358)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)
at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:394)
at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:253)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:175)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
The method:
#PostMapping ( "response" )
public ModelAndView postResponse ( ..., UriComponentsBuilder ucb, HttpServletRequest request ) {
byte [ ] samlResponseAsBytes = DatatypeConverter.parseBase64Binary ( samlResponse );
...
return new ModelAndView ( "redirect:" + ucb.path ( "/delivery" ).build ( ).toString ( ) );
}
Application.properties
...
spring.thymeleaf.prefix = classpath:views/
spring.thymeleaf.suffix = .html
spring.thymeleaf.check-template-location = true
spring.thymeleaf.cache = false
spring.resources.static-locations = classpath:resources/
spring.resources.cache.period = 0
I've tried several locations for the delivery.html but as it is configured it should be under resources/views.
I can upload more informations, if neccessary.
Can someone help me please?

ConnectionClosedException: Premature end of chunk coded message body with apache hhtpclient 4.5.5

I am trying to parse http response from a GET request but it throws following exception.
org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:266) ~[httpcore-4.4.10.jar!/:4.4.10]
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225) ~[httpcore-4.4.10.jar!/:4.4.10]
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184) ~[httpcore-4.4.10.jar!/:4.4.10]
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) ~[httpclient-4.5.6.jar!/:4.5.6]
at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[na:na]
at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[na:na]
at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[na:na]
at java.base/java.io.InputStreamReader.read(InputStreamReader.java:185) ~[na:na]
at java.base/java.io.Reader.read(Reader.java:229) ~[na:na]
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:227) ~[httpcore-4.4.10.jar!/:4.4.10]
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:308) ~[httpcore-4.4.10.jar!/:4.4.10]
My code to parse the response is
String parseResponse(HttpResponse resp) {
try {
return org.apache.http.util.EntityUtils.toString(resp.getEntity());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
I used org.apache.httpcomponents:httpcore:4.5.6
The GET endpoint(spring boot application) which I invoke is as follows
public ResponseEntity<org.springframework.data.domain.
Page<JSONObject>> getList() {
}
I read here https://github.com/jersey/jersey/issues/3629 If I use org.apache.httpcomponents:httpclient:4.5.5 then it should work fine but it did not work for me.
Can someone please give some pointers.
I saw a similar question org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected but it did not help me.
Understood the issue. I was closing CloseableHttpClient and then trying to use the HTTPResponse.
Follwing post helped
https://stackoverflow.com/a/50136644/1053496

How can I Read and Transfer chunks of file with Hadoop WebHDFS?

I need to transfer big files (at least 14MB) from the Cosmos instance of the FIWARE Lab to my backend.
I used the Spring RestTemplate as a client interface for the Hadoop WebHDFS REST API described here but I run into an IO Exception:
Exception in thread "main" org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/<user.name>/<path>?op=open&user.name=<user.name>":Truncated chunk ( expected size: 14744230; actual size: 11285103); nested exception is org.apache.http.TruncatedChunkException: Truncated chunk ( expected size: 14744230; actual size: 11285103)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:580)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:545)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:466)
This is the actual code that generates the Exception:
RestTemplate restTemplate = new RestTemplate();
restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
restTemplate.getMessageConverters().add(new ByteArrayHttpMessageConverter());
HttpEntity<?> entity = new HttpEntity<>(headers);
UriComponentsBuilder builder =
UriComponentsBuilder.fromHttpUrl(hdfs_path)
.queryParam("op", "OPEN")
.queryParam("user.name", user_name);
ResponseEntity<byte[]> response =
restTemplate
.exchange(builder.build().encode().toUri(), HttpMethod.GET, entity, byte[].class);
FileOutputStream output = new FileOutputStream(new File(local_path));
IOUtils.write(response.getBody(), output);
output.close();
I think this is due to a transfer timeout on the Cosmos instance, so I tried to
send a curl on the path by specifying offset, buffer and length parameters, but they seem to be ignored: I got the whole file.
Thanks in advance.
Ok, I found out a solution. I don't understand why, but the transfer succeds if I use a Jetty HttpClient instead of the RestTemplate (and so Apache HttpClient). This works now:
ContentExchange exchange = new ContentExchange(true){
ByteArrayOutputStream bos = new ByteArrayOutputStream();
protected void onResponseContent(Buffer content) throws IOException {
bos.write(content.asArray(), 0, content.length());
}
protected void onResponseComplete() throws IOException {
if (getResponseStatus()== HttpStatus.OK_200) {
FileOutputStream output = new FileOutputStream(new File(<local_path>));
IOUtils.write(bos.toByteArray(), output);
output.close();
}
}
};
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(<hdfs_path>)
.queryParam("op", "OPEN")
.queryParam("user.name", <user_name>);
exchange.setURL(builder.build().encode().toUriString());
exchange.setMethod("GET");
exchange.setRequestHeader("X-Auth-Token", <token>);
HttpClient client = new HttpClient();
client.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL);
client.setMaxConnectionsPerAddress(200);
client.setThreadPool(new QueuedThreadPool(250));
client.start();
client.send(exchange);
exchange.waitForDone();
Is there any known bug on the Apache Http Client for chunked files transfer?
Was I doing something wrong in my RestTemplate request?
UPDATE: I still don't have a solution
After few tests I see that I don't have solved my problems.
I found out that the hadoop version installed on the Cosmos instance is quite old Hadoop 0.20.2-cdh3u6 and I read that WebHDFS doesn't support partial file transfer with length parameter (introduced since v 0.23.3).
These are the headers I received from the Server when I send a GET request using curl:
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: HEAD, POST, GET, OPTIONS, DELETE
Access-Control-Allow-Headers: origin, content-type, X-Auth-Token, Tenant-ID, Authorization
server: Apache-Coyote/1.1
set-cookie: hadoop.auth="u=<user>&p=<user>&t=simple&e=1448999699735&s=rhxMPyR1teP/bIJLfjOLWvW2pIQ="; Version=1; Path=/
Content-Type: application/octet-stream; charset=utf-8
content-length: 172934567
date: Tue, 01 Dec 2015 09:54:59 GMT
connection: close
As you see the Connection header is set to close. Actually, the connection is usually closed each time the GET request lasts more than 120 seconds, even if the file transfer has not been completed.
In conclusion, I can say that Cosmos is totally useless if it doesn't support large file transfer.
Please correct me if I'm wrong, or if you know a workaround.

Resources