I am trying to POST a payload with almost 4-5 MB of data through RestTemplate. Some of the request are passing but some of the requests are not even going to API gateway with this error
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting
for connection from pool
Here is the rest template implementation I am using.
#Component
public class RestTemplateHelper {
#Autowired
private RestTemplate restTemplate;
public RestTemplateHelper() {
}
private <X, Y> Y post(final String url, final String accessToken, final MediaType mediaType, final X requestBody, final Class<Y> responseType, final Map<String, String> extraHeaders, HttpMessageConverter httpMessageConverter, RestTemplate restTemplateFromParam) {
HttpHeaders headers = this.getHeaders(accessToken, mediaType, extraHeaders);
HttpEntity<X> httpEntity = new HttpEntity(requestBody, headers);
ResponseEntity responseEntity;
NonRetriableException nonRetriableException;
try {
RestTemplate restTemplateToUse = this.restTemplate;
if (Objects.nonNull(httpMessageConverter)) {
restTemplateToUse = Objects.nonNull(restTemplateFromParam) ? restTemplateFromParam : restTemplateToUse;
restTemplateToUse.getMessageConverters().add(0, httpMessageConverter);
}
responseEntity = restTemplateToUse.exchange(url, HttpMethod.POST, httpEntity, responseType, new Object[0]);
} catch (ResourceAccessException var14) {
nonRetriableException = new NonRetriableException(var14);
nonRetriableException.setRootException(var14);
throw nonRetriableException;
} catch (RestClientResponseException var15) {
nonRetriableException = new NonRetriableException(var15);
nonRetriableException.setRawStatusCode(var15.getRawStatusCode());
nonRetriableException.setStatusText(var15.getStatusText());
nonRetriableException.setResponseBody(var15.getResponseBodyAsString());
nonRetriableException.setResponseHeaders(var15.getResponseHeaders());
nonRetriableException.setRootException(var15);
throw nonRetriableException;
} catch (Exception var16) {
throw new NonRetriableException(var16);
}
return responseEntity.getBody();
}
}
So I have an API request where I am copying the details directly from postman where it works. I am however getting a bad request error.
#Service
public class GraphApiService {
#Bean
public RestTemplate restTemplate() {
return new RestTemplate();
}
#Autowired
RestTemplate restTemplate;
#Autowired
Constants constants;
private final Logger logger = LoggerFactory.getLogger(this.getClass());
public ResponseEntity<String> getAccessTokenUsingRefreshToken(Credential cred) throws IOException{
try {
//https://learn.microsoft.com/en-us/graph/auth-v2-user
// section 5. Use the refresh token to get a new access token
String url = "url";
JSONObject body = new JSONObject();
body.put("grant_type", "refresh_token");
body.put("client_id", "clientid");
body.put("scope","User.Read offline_access Files.Read Mail.Read Sites.Read.All");
body.put("redirect_uri", "http://localhost");
body.put("client_secret","secret");
body.put("refresh_token", "token");
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
HttpEntity<String> request = new HttpEntity<String>(body.toString(), headers);
ResponseEntity<String> response= restTemplate.postForEntity(url, request,String.class);
return response;
}
catch(HttpClientErrorException e){
logger.error(e.getResponseBodyAsString());
logger.error(e.getMessage());
return null;
}
}
I would appreciate any help. The bad request error message from microsoft graph isn't a descriptive one that will help
You're sending JSON payload with FORM_URLENCODED header.
Either you need to check if API accepts json payload, if so you need to change content-type to application/json or you can post form data as follows.
public ResponseEntity<String> getAccessTokenUsingRefreshToken(Credential cred) throws IOException{
try {
//https://learn.microsoft.com/en-us/graph/auth-v2-user
// section 5. Use the refresh token to get a new access token
String url = "url";
MultiValueMap<String, String> multiValueMap= new LinkedMultiValueMap<String, String>();
multiValueMap.add("grant_type", "refresh_token");
multiValueMap.add("client_id", "clientid");
//.....
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<>(multiValueMap, headers);
ResponseEntity<String> response= restTemplate.postForEntity(url, request, String.class);
return response;
}catch(HttpClientErrorException e){
logger.error(e.getResponseBodyAsString());
logger.error(e.getMessage());
return null;
}
}
I am trying to get the actual json i recieve as response from rest service.
My problem is that i have multiple message converters configured for my RestTemplate like this.
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
RestTemplate restTemplate = builder.build();
restTemplate.setMessageConverters(getMessageConverters());
return restTemplate;
}
private List<HttpMessageConverter<?>> getMessageConverters() {
List<HttpMessageConverter<?>> converters = new LinkedList<>();
StringHttpMessageConverter textConverter = new StringHttpMessageConverter();
converters.add(textConverter);
MappingJackson2HttpMessageConverter jsonConverter = new MappingJackson2HttpMessageConverter();
converters.add(jsonConverter);
HttpMessageConverter formHttpMessageConverter = new FormHttpMessageConverter();
converters.add(formHttpMessageConverter);
return converters;
}
Here is the how i try to retrive the string from the response.
ResponseEntity<String> message = null;
try {
message = restTemplate.postForEntity(
url,
new HttpEntity<>(payload, getHeaders()),
String.class);
return message.getBody();
} catch (HttpClientErrorException e) {
log.error(e.getResponseBodyAsString(), e);
}
My guess is that because i have a Mapping2Jackson2HttpMessageConverter configured, each time i recieve message with content type application/json, it tries to convert it using json converter even if response type class is specified as String.
I'm wondering how do I mock the rest controller for the code below,
public void sendData(ID id, String xmlString, Records record) throws ValidationException{
ClientHttpRequestFactory requestFactory = new
HttpComponentsClientHttpRequestFactory(HttpClients.createDefault());
RestTemplate restTemplate = new RestTemplate(requestFactory);
List<HttpMessageConverter<?>> messageConverters = new ArrayList<>();
messageConverters.add(new StringHttpMessageConverter(Charset.forName("UTF-8")));
restTemplate.setMessageConverters(messageConverters);
MultiValueMap<String,String> header = new LinkedMultiValueMap<>();
header.add("x-api-key",api_key);
header.add("Content-Type",content_type);
header.add("Cache-Control",cache_control);
HttpEntity<String> request = new HttpEntity<>(xmlString, header);
try {
restTemplate.postForEntity(getUri(id,record), request, String.class);
}catch (RestClientResponseException e){
throw new ValidationException("Error occurred while sending a file to some server "+e.getResponseBodyAsString());
}
}
Any suggestion would be helpful.
I tried to do something like this,
#RunWith(MockitoJUnitRunner.class)
public class Safe2RestControllerTest {
private MockRestServiceServer server;
private RestTemplate restTemplate;
private restControllerClass serviceToTest;
#Before
public void init(){
//some code for initialization of the parameters used in controller class
this.server = MockRestServiceServer.bindTo(this.restTemplate).ignoreExpectOrder(true).build();
}
#Test
public void testSendDataToSafe2() throws ValidationException, URISyntaxException {
//some code here when().then()
String responseBody = "{\n" +
" \"responseMessage\": \"Validation succeeded, message
accepted.\",\n" +
" \"responseCode\": \"SUCCESS\",\n" +
" 2\"responseID\": \"627ccf4dcc1a413588e5e2bae7f47e9c::0d86869e-663a-41f0-9f4c-4c7e0b278905\"\n" +
"}";
this.server.expect(MockRestRequestMatchers.requestTo(uri))
.andRespond(MockRestResponseCreators.withSuccess(responseBody,
MediaType.APPLICATION_JSON));
serviceToTest.sendDataToSafe2(id, xmlString, record);
this.server.verify();
}
}
This is the test case what I'm trying to do but it still calling actual rest api
As pointed out by #JBNizet, you should take a look at MockRestServiceServer. It allows you to test Spring components which are using RestTemplate to make HTTP calls.
See MockRestServiceServer and #RestClientTest.
Below is the code snippet; basically, I am trying to propagate the exception when the error code is anything other than 200.
ResponseEntity<Object> response = restTemplate.exchange(url.toString().replace("{version}", version),
HttpMethod.POST, entity, Object.class);
if(response.getStatusCode().value()!= 200){
logger.debug("Encountered Error while Calling API");
throw new ApplicationException();
}
However in the case of a 500 response from the server I am getting the exception
org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:94) ~[spring-web-4.2.3.RELEASE.jar:4.2.3.RELEASE]
Do I really need to wrap the rest template exchange method in try? What would then be the purpose of codes?
You want to create a class that implements ResponseErrorHandler and then use an instance of it to set the error handling of your rest template:
public class MyErrorHandler implements ResponseErrorHandler {
#Override
public void handleError(ClientHttpResponse response) throws IOException {
// your error handling here
}
#Override
public boolean hasError(ClientHttpResponse response) throws IOException {
...
}
}
[...]
public static void main(String args[]) {
RestTemplate restTemplate = new RestTemplate();
restTemplate.setErrorHandler(new MyErrorHandler());
}
Also, Spring has the class DefaultResponseErrorHandler, which you can extend instead of implementing the interface, in case you only want to override the handleError method.
public class MyErrorHandler extends DefaultResponseErrorHandler {
#Override
public void handleError(ClientHttpResponse response) throws IOException {
// your error handling here
}
}
Take a look at its source code to have an idea of how Spring handles HTTP errors.
Spring cleverly treats http error codes as exceptions, and assumes that your exception handling code has the context to handle the error. To get exchange to function as you would expect it, do this:
try {
return restTemplate.exchange(url, httpMethod, httpEntity, String.class);
} catch(HttpStatusCodeException e) {
return ResponseEntity.status(e.getRawStatusCode()).headers(e.getResponseHeaders())
.body(e.getResponseBodyAsString());
}
This will return all the expected results from the response.
You should catch a HttpStatusCodeException exception:
try {
restTemplate.exchange(...);
} catch (HttpStatusCodeException exception) {
int statusCode = exception.getStatusCode().value();
...
}
Another solution is the one described here at the end of this post by "enlian":
http://springinpractice.com/2013/10/07/handling-json-error-object-responses-with-springs-resttemplate
try{
restTemplate.exchange(...)
} catch(HttpStatusCodeException e){
String errorpayload = e.getResponseBodyAsString();
//do whatever you want
} catch(RestClientException e){
//no response payload, tell the user sth else
}
Spring abstracts you from the very very very large list of http status code. That is the idea of the exceptions. Take a look into org.springframework.web.client.RestClientException hierarchy:
You have a bunch of classes to map the most common situations when dealing with http responses. The http codes list is really large, you won't want write code to handle each situation. But for example, take a look into the HttpClientErrorException sub-hierarchy. You have a single exception to map any 4xx kind of error. If you need to go deep, then you can. But with just catching HttpClientErrorException, you can handle any situation where bad data was provided to the service.
The DefaultResponseErrorHandler is really simple and solid. If the response status code is not from the family of 2xx, it just returns true for the hasError method.
I have handled this as below:
try {
response = restTemplate.postForEntity(requestUrl, new HttpEntity<>(requestBody, headers), String.class);
} catch (HttpStatusCodeException ex) {
response = new ResponseEntity<String>(ex.getResponseBodyAsString(), ex.getResponseHeaders(), ex.getStatusCode());
}
A very simple solution can be:
try {
requestEntity = RequestEntity
.get(new URI("user String"));
return restTemplate.exchange(requestEntity, String.class);
} catch (RestClientResponseException e) {
return ResponseEntity.status(e.getRawStatusCode()).body(e.getResponseBodyAsString());
}
If you use pooling (http client factory) or load balancing (eureka) mechanism with your RestTemplate, you will not have the luxury of creating a new RestTemplate per class. If you are calling more than one service you cannot use setErrorHandler because if would be globally used for all your requests.
In this case, catching the HttpStatusCodeException seems to be the better option.
The only other option you have is to define multiple RestTemplate instances using the #Qualifier annotation.
Also - but this is my own taste - I like my error handling snuggled tightly to my calls.
The code of exchange is below:
public <T> ResponseEntity<T> exchange(String url, HttpMethod method,
HttpEntity<?> requestEntity, Class<T> responseType, Object... uriVariables) throws RestClientException
Exception RestClientException has HttpClientErrorException and HttpStatusCodeException exception.
So in RestTemplete there may occure HttpClientErrorException and HttpStatusCodeException exception.
In exception object you can get exact error message using this way: exception.getResponseBodyAsString()
Here is the example code:
public Object callToRestService(HttpMethod httpMethod, String url, Object requestObject, Class<?> responseObject) {
printLog( "Url : " + url);
printLog( "callToRestService Request : " + new GsonBuilder().setPrettyPrinting().create().toJson(requestObject));
try {
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
restTemplate.getMessageConverters().add(new StringHttpMessageConverter());
HttpHeaders requestHeaders = new HttpHeaders();
requestHeaders.setContentType(MediaType.APPLICATION_JSON);
HttpEntity<Object> entity = new HttpEntity<>(requestObject, requestHeaders);
long start = System.currentTimeMillis();
ResponseEntity<?> responseEntity = restTemplate.exchange(url, httpMethod, entity, responseObject);
printLog( "callToRestService Status : " + responseEntity.getStatusCodeValue());
printLog( "callToRestService Body : " + new GsonBuilder().setPrettyPrinting().create().toJson(responseEntity.getBody()));
long elapsedTime = System.currentTimeMillis() - start;
printLog( "callToRestService Execution time: " + elapsedTime + " Milliseconds)");
if (responseEntity.getStatusCodeValue() == 200 && responseEntity.getBody() != null) {
return responseEntity.getBody();
}
} catch (HttpClientErrorException exception) {
printLog( "callToRestService Error :" + exception.getResponseBodyAsString());
//Handle exception here
}catch (HttpStatusCodeException exception) {
printLog( "callToRestService Error :" + exception.getResponseBodyAsString());
//Handle exception here
}
return null;
}
Here is the code description:
In this method you have to pass request and response class. This method will automatically parse response as requested object.
First of All you have to add message converter.
restTemplate.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
restTemplate.getMessageConverters().add(new StringHttpMessageConverter());
Then you have to add requestHeader.
Here is the code:
HttpHeaders requestHeaders = new HttpHeaders();
requestHeaders.setContentType(MediaType.APPLICATION_JSON);
HttpEntity<Object> entity = new HttpEntity<>(requestObject, requestHeaders);
Finally, you have to call exchange method:
ResponseEntity<?> responseEntity = restTemplate.exchange(url, httpMethod, entity, responseObject);
For prety printing i used Gson library.
here is the gradle : compile 'com.google.code.gson:gson:2.4'
You can just call the bellow code to get response:
ResponseObject response=new RestExample().callToRestService(HttpMethod.POST,"URL_HERE",new RequestObject(),ResponseObject.class);
Here is the full working code:
import com.google.gson.GsonBuilder;
import org.springframework.http.*;
import org.springframework.http.converter.StringHttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.client.HttpStatusCodeException;
import org.springframework.web.client.RestTemplate;
public class RestExample {
public RestExample() {
}
public Object callToRestService(HttpMethod httpMethod, String url, Object requestObject, Class<?> responseObject) {
printLog( "Url : " + url);
printLog( "callToRestService Request : " + new GsonBuilder().setPrettyPrinting().create().toJson(requestObject));
try {
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
restTemplate.getMessageConverters().add(new StringHttpMessageConverter());
HttpHeaders requestHeaders = new HttpHeaders();
requestHeaders.setContentType(MediaType.APPLICATION_JSON);
HttpEntity<Object> entity = new HttpEntity<>(requestObject, requestHeaders);
long start = System.currentTimeMillis();
ResponseEntity<?> responseEntity = restTemplate.exchange(url, httpMethod, entity, responseObject);
printLog( "callToRestService Status : " + responseEntity.getStatusCodeValue());
printLog( "callToRestService Body : " + new GsonBuilder().setPrettyPrinting().create().toJson(responseEntity.getBody()));
long elapsedTime = System.currentTimeMillis() - start;
printLog( "callToRestService Execution time: " + elapsedTime + " Milliseconds)");
if (responseEntity.getStatusCodeValue() == 200 && responseEntity.getBody() != null) {
return responseEntity.getBody();
}
} catch (HttpClientErrorException exception) {
printLog( "callToRestService Error :" + exception.getResponseBodyAsString());
//Handle exception here
}catch (HttpStatusCodeException exception) {
printLog( "callToRestService Error :" + exception.getResponseBodyAsString());
//Handle exception here
}
return null;
}
private void printLog(String message){
System.out.println(message);
}
}
Thanks :)
To extedend #carcaret answer a bit....
Consider your response errors are returned by json message. For example the API may return 204 as status code error and a json message as error list. In this case you need to define which messages should spring consider as error and how to consume them.
As a sample your API may return some thing like this, if error happens:
{ "errorCode":"TSC100" , "errorMessage":"The foo bar error happend" , "requestTime" : "202112827733" .... }
To consume above json and throw a custom exception, you can do as below:
First define a class for mapping error ro object
//just to map the json to object
public class ServiceErrorResponse implements Serializable {
//setter and getters
private Object errorMessage;
private String errorCode;
private String requestTime;
}
Now define the error handler:
public class ServiceResponseErrorHandler implements ResponseErrorHandler {
private List<HttpMessageConverter<?>> messageConverters;
#Override
public boolean hasError(ClientHttpResponse response) throws IOException {
return (response.getStatusCode().is4xxClientError() ||
response.getStatusCode().is5xxServerError());
}
#Override
public void handleError(ClientHttpResponse response) throws IOException {
#SuppressWarnings({ "unchecked", "rawtypes" })
HttpMessageConverterExtractor<ServiceErrorResponse> errorMessageExtractor =
new HttpMessageConverterExtractor(ServiceErrorResponse.class, messageConverters);
ServiceErrorResponse errorObject = errorMessageExtractor.extractData(response);
throw new ResponseEntityErrorException(
ResponseEntity.status(response.getRawStatusCode())
.headers(response.getHeaders())
.body(errorObject)
);
}
public void setMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
this.messageConverters = messageConverters;
}
}
The custom Exception will be:
public class ResponseEntityErrorException extends RuntimeException {
private ResponseEntity<ServiceErrorResponse> serviceErrorResponseResponse;
public ResponseEntityErrorException(ResponseEntity<ServiceErrorResponse> serviceErrorResponseResponse) {
this.serviceErrorResponseResponse = serviceErrorResponseResponse;
}
public ResponseEntity<ServiceErrorResponse> getServiceErrorResponseResponse() {
return serviceErrorResponseResponse;
}
}
To use it:
RestTemplateResponseErrorHandler errorHandler = new
RestTemplateResponseErrorHandler();
//pass the messageConverters to errror handler and let it convert json to object
errorHandler.setMessageConverters(restTemplate.getMessageConverters());
restTemplate.setErrorHandler(errorHandler);
This is how to handle exceptions in Rest Template
try {
return restTemplate.exchange("URL", HttpMethod.POST, entity, String.class);
}
catch (HttpStatusCodeException e)
{
return ResponseEntity.status(e.getRawStatusCode()).headers(e.getResponseHeaders())
.body(e.getResponseBodyAsString());
}
Here is my POST method with HTTPS which returns a response body for any type of bad responses.
public String postHTTPSRequest(String url,String requestJson)
{
//SSL Context
CloseableHttpClient httpClient = HttpClients.custom().setSSLHostnameVerifier(new NoopHostnameVerifier()).build();
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
//Initiate REST Template
RestTemplate restTemplate = new RestTemplate(requestFactory);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
//Send the Request and get the response.
HttpEntity<String> entity = new HttpEntity<String>(requestJson,headers);
ResponseEntity<String> response;
String stringResponse = "";
try {
response = restTemplate.postForEntity(url, entity, String.class);
stringResponse = response.getBody();
}
catch (HttpClientErrorException e)
{
stringResponse = e.getResponseBodyAsString();
}
return stringResponse;
}
I fixed it by overriding the hasError method from DefaultResponseErrorHandler class:
public class BadRequestSafeRestTemplateErrorHandler extends DefaultResponseErrorHandler
{
#Override
protected boolean hasError(HttpStatus statusCode)
{
if(statusCode == HttpStatus.BAD_REQUEST)
{
return false;
}
return statusCode.isError();
}
}
And you need to set this handler for restemplate bean:
#Bean
protected RestTemplate restTemplate(RestTemplateBuilder builder)
{
return builder.errorHandler(new BadRequestSafeRestTemplateErrorHandler()).build();
}
Read about global exception handling in global exception handler add the below method. this will work.
#ExceptionHandler( {HttpClientErrorException.class, HttpStatusCodeException.class, HttpServerErrorException.class})
#ResponseBody
public ResponseEntity<Object> httpClientErrorException(HttpStatusCodeException e) throws IOException {
BodyBuilder bodyBuilder = ResponseEntity.status(e.getRawStatusCode()).header("X-Backend-Status", String.valueOf(e.getRawStatusCode()));
if (e.getResponseHeaders().getContentType() != null) {
bodyBuilder.contentType(e.getResponseHeaders().getContentType());
}
return bodyBuilder.body(e.getResponseBodyAsString());
}
There is also an option to use TestRestTemplate. It is very useful for integration and E2E tests, when you need to validate all status codes manually (for example in negative test-cases).
TestRestTemplate is fault-tolerant. This means that 4xx and 5xx do not result in an exception being thrown and can instead be detected via the response entity and its status code.
Try using #ControllerAdvice. This allows you to handle the exception only once and have all 'custom' handled exceptions in one place.
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/bind/annotation/ControllerAdvice.html
example
#ControllerAdvice
public class MyExceptionHandler extends ResponseEntityExceptionHandler{
#ExceptionHandler(MyException.class)
protected ResponseEntity<Object> handleMyException(){
MyException exception,
WebRequest webRequest) {
return handleExceptionInternal(
exception,
exception.getMessage(),
exception.getResponseHeaders(),
exception.getStatusCode(),
webRequest);
}