I am trying to POST a payload with almost 4-5 MB of data through RestTemplate. Some of the request are passing but some of the requests are not even going to API gateway with this error
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting
for connection from pool
Here is the rest template implementation I am using.
#Component
public class RestTemplateHelper {
#Autowired
private RestTemplate restTemplate;
public RestTemplateHelper() {
}
private <X, Y> Y post(final String url, final String accessToken, final MediaType mediaType, final X requestBody, final Class<Y> responseType, final Map<String, String> extraHeaders, HttpMessageConverter httpMessageConverter, RestTemplate restTemplateFromParam) {
HttpHeaders headers = this.getHeaders(accessToken, mediaType, extraHeaders);
HttpEntity<X> httpEntity = new HttpEntity(requestBody, headers);
ResponseEntity responseEntity;
NonRetriableException nonRetriableException;
try {
RestTemplate restTemplateToUse = this.restTemplate;
if (Objects.nonNull(httpMessageConverter)) {
restTemplateToUse = Objects.nonNull(restTemplateFromParam) ? restTemplateFromParam : restTemplateToUse;
restTemplateToUse.getMessageConverters().add(0, httpMessageConverter);
}
responseEntity = restTemplateToUse.exchange(url, HttpMethod.POST, httpEntity, responseType, new Object[0]);
} catch (ResourceAccessException var14) {
nonRetriableException = new NonRetriableException(var14);
nonRetriableException.setRootException(var14);
throw nonRetriableException;
} catch (RestClientResponseException var15) {
nonRetriableException = new NonRetriableException(var15);
nonRetriableException.setRawStatusCode(var15.getRawStatusCode());
nonRetriableException.setStatusText(var15.getStatusText());
nonRetriableException.setResponseBody(var15.getResponseBodyAsString());
nonRetriableException.setResponseHeaders(var15.getResponseHeaders());
nonRetriableException.setRootException(var15);
throw nonRetriableException;
} catch (Exception var16) {
throw new NonRetriableException(var16);
}
return responseEntity.getBody();
}
}
Related
I am trying to hit a restful endpoint from my springboot application using restTemplate.exchange and restTemplate.postForEntity and I am getting 400 Bad request [no body] exception.
I tried every possible solution to figure out the issue. From the same code I used HttpURLConnection to hit the endpoint and it works fine. I am able to get valid response. I also tried hitting the endpoint from Postman and it works fine
Below is endpoint code
#RestController
#RequestMapping("auth")
#Slf4j
public class AuthController {
#PostMapping(value = "/as/resourceOwner",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<Token> getToken(#RequestParam(required = false)MultiValuedMap<?, ?> params) {
log.info("token endpoint");
return new ResponseEntity<>(new Token(), HttpStatus.OK);
}
}
The below code with HttpURLConnection class works fine
public class Test {
public static void main(String[] args) {
try {
URL url = new URL("http://localhost:8080/fc-services-mock-auth/fmrco/as/resourceOwner");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setDoOutput(true);
urlConnection.setRequestMethod("POST");
urlConnection.setRequestProperty("Content-Type", MediaType.APPLICATION_FORM_URLENCODED_VALUE);
OutputStream writer = urlConnection.getOutputStream();
writer.write("a=a&b=b".getBytes(StandardCharsets.UTF_8));
writer.flush();
writer.close();
BufferedReader reader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
} catch (Exception e) {
System.out.println(e);
}
}
}
But the below code with restTemplate does not work
try {
HttpEntity httpEntity = new HttpEntity<>(createBody(), createHeaders());
ResponseEntity<String> response = restTemplate.exchange(new URI(endpointConfig.getTokenServiceUrl()),
HttpMethod.POST, httpEntity, String.class);
HttpEntity<MultiValueMap<String, String>> request =
new HttpEntity<>(createBody(), createHeaders());
ResponseEntity<TokenResponse> tokenResponse = restTemplate.postForEntity(
new URI(endpointConfig.getTokenServiceUrl()),
request,
TokenResponse.class);
logger.debug(" token process completed");
return tokenResponse.getBody();
} catch (Exception e) {
throw new TokenException(" token error: ", e);
}
private HttpHeaders createHeaders() {
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Type", MediaType.APPLICATION_FORM_URLENCODED_VALUE);
return headers;
}
private MultiValueMap createBody() {
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.add("client_id", applicationConfig.getTokenClientId());
map.add("client_secret", applicationConfig.getTokenClientSecret());
map.add("grant_type", "password");
map.add("username", applicationConfig.getTokenUsername());
map.add("password", applicationConfig.getTokenPassword());
return map;
}
Can anyone please tell me what is wrong here?
Additionally I have written another GET restful endpoint like below and while hitting the endpoint using exchange I still get the same error.
#GetMapping(value = "test", produces = MediaType.TEXT_PLAIN_VALUE)
public ResponseEntity<String> test() {
return new ResponseEntity<>("Hello", HttpStatus.OK);
}
try {
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.TEXT_PLAIN));
HttpEntity<MultiValueMap<String, String>> entity =
new HttpEntity<>(null, headers);
ResponseEntity<String> response = restTemplate.exchange(
"http://localhost:8080/context/path",
HttpMethod.GET, entity, String.class);
} catch (Exception e) {
System.out.println(e);
}
Also, intentionally I tried making a POST call to the get endpoint to see if it returns 405 Method not allowed. But to my wonder it still returned 400 Bad Request no body.
I finally figured it out. The restTemplate is configured to use a proxy. I removed the proxy and it is working fine now.
You can try something like:-
public void fetchAuthenticationToken() {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
headers.setAccept(List.of(MediaType.APPLICATION_JSON));
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.add("client_id", applicationConfig.getTokenClientId());
map.add("client_secret", applicationConfig.getTokenClientSecret());
map.add("grant_type", "client_credentials");
map.add("username", applicationConfig.getTokenUsername());
map.add("password", applicationConfig.getTokenPassword());
HttpEntity<MultiValueMap<String, String>> entity = new HttpEntity<>(map, headers);
String tokenEndPoint = "{your endpoints URL}";
ResponseEntity<TokenResponse> responseEntity = testRestTemplate.exchange(tokenEndPoint,
HttpMethod.POST, entity, TokenResponse.class, new HashMap<String, String>());
assertThat(responseEntity.getStatusCode()).isEqualTo(HttpStatus.OK);
TokenResponse token = responseEntity.getBody();
assertThat(token).isNotNull();
System.out.println("Auth Token value is "+ token)
}
The service provider supplies me with the header data: grant_type, Content-Type. And the body data: grant_type, username and password. Which I use in Postman where it generates OK token. But in the Spring application it generates an error HttpClientErrorException $ BadRequest: 400 Bad Request.
I have the class to set the body data:
public class BodyToken {
private String grant_type = "password";//set body data
private String username = "User";//set body data
private String password = "123";//set body data
private String access_token;
#JsonGetter("access_token")
public String getAccess_token() {
return access_token;
}
public void setAccess_token(String access_token) {
this.access_token = access_token;
}
#JsonGetter("grant_type")
public String getGrant_type() {
return grant_type;
}
#JsonGetter("username")
public String getUsername() {
return username;
}
#JsonGetter("password")
public String getPassword() {
return password;
}
}
This is the controller where the header data is set:
#PostMapping("/TokenGeneration")
#ResponseBody
public BodyToken TokenGeneration() throws IOException {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.set("grant_type", "password");//set header data
headers.set("Content-Type", "application/x-www-form-urlencoded");//set header data
HttpEntity request = new HttpEntity(headers);
headers.add("User-Agent", "Spring's RestTemplate" );
ResponseEntity<BodyToken> response = restTemplate.exchange(
"https://sw/token",
HttpMethod.POST,
request,
BodyToken.class
);
try {
return response.getBody();
} catch (Exception e) {
BodyToken body = new BodyToken();
log.info(e.getMessage());
return body;
}
}
OK was solved with using the Class MultiValueMap and LinkedMultiValueMap. The credentials are added to this new object and it is sent together with the request:
#PostMapping("/TokenGeneration")
#ResponseBody
public BodyToken TokenGeneration() throws IOException {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.set("grant_type", "password");//set header data
headers.set("Content-Type", "application/x-www-form-urlencoded");//set header data
MultiValueMap<String, String> body = new LinkedMultiValueMap<String, String>();//line solution
body.add("grant_type", "password");//line solution
body.add("username", "user");//line solution
body.add("password", "123");//line solution
HttpEntity request = new HttpEntity(body, headers);//and I add this body to HttpEntity
headers.add("User-Agent", "Spring's RestTemplate" );
ResponseEntity<BodyToken> response = restTemplate.exchange(
"https://sw/token",
HttpMethod.POST,
request,
BodyToken.class
);
try {
return response.getBody();
} catch (Exception e) {
BodyToken body = new BodyToken();
log.info(e.getMessage());
return body;
}
}
I'm sending a response to another web service to create an user. If the user already exists it sends back the 409 response. I'm using RestTemplate like so:
#PostMapping("/todos/{toDoNoteId}/users")
public ResponseEntity <String> postUser(#RequestBody User user, #PathVariable int toDoNoteId, UriComponentsBuilder builder)throws HttpMessageNotReadableException, ParseException{
RestTemplate restTemplate = new RestTemplate();
final String uri = "http://friend:5000/users";
try {
ResponseEntity<String> result = restTemplate.postForEntity(uri, user, String.class);
return result;
}
catch (HttpClientErrorException ex) {
return ResponseEntity.status(ex.getRawStatusCode()).headers(ex.getResponseHeaders())
.body(ex.getResponseBodyAsString());
}
}
While catching an exception somewhat works (in the catch block i can access the status code and body), is there a way to access it without exceptions something similar like this:
#PostMapping("/todos/{toDoNoteId}/users")
public ResponseEntity <String> postUser(#RequestBody User user, #PathVariable int toDoNoteId, UriComponentsBuilder builder)throws HttpMessageNotReadableException, ParseException{
RestTemplate restTemplate = new RestTemplate();
final String uri = "http://friend:5000/users";
ResponseEntity<String> result = restTemplate.postForEntity(uri, user, String.class);
if(result.getStatusCode()=="409"){
// do something
}
else{
// do something else
}
return result;
}
Have you been check the ExceptionHandler? When exception throws, ExceptionHandler handles it.
For example:
#ControllerAdvice()
public class CustomExceptionHandler {
private static final Logger logger = LogManager.getLogger("CustomExceptionHandler");
#ExceptionHandler(YourException.class)
public ResponseEntity handleYourException(HttpServletRequest request, YourException ex) {
return ResponseEntity.ok("");
}
#ExceptionHandler(Exception.class)
public ResponseEntity handleException(HttpServletRequest request, Exception ex) {
logExp("Exception", request, ex);
//return new ResponseEntity<>();
return null;
}
}
You can create your own custom resttemplate and define exception handler. Here is a code snippet.
#Configuration
public class CustomRestTemplate extends RestTemplate {
#Autowired
private CustomErrorHandler customErrorHandler;
#PostConstruct
public void init() {
this.setErrorHandler(customErrorHandler);
}
}
#Component
public class CustomErrorHandler implements ResponseErrorHandler {
#Override
public boolean hasError(ClientHttpResponse response) throws IOException {
if(response.getStatusCode() != "409"){
return true;
}else {
return false;
}
}
#Override
public void handleError(ClientHttpResponse response) throws IOException {
String responseBody = response.getBody();//Pls read from InputStream and create write into String
JSONObject jsonObj = new JSONObject(result);
JSONArray jsonArray = new JSONArray();
jsonObj.put("status", response.getStatusCode());
jsonObj.put("body", responseBody );
jsonArray.put(jsonObj);
responseString = jsonArray.get(0).toString();
throw new MyException(responseString );
}
}
class MyException throw RuntimeException {
public MyException (String message) {
super(message);
}
}
So, your class will changed to
#PostMapping("/todos/{toDoNoteId}/users")
public ResponseEntity <String> postUser(#RequestBody User user, #PathVariable int toDoNoteId, UriComponentsBuilder builder)throws HttpMessageNotReadableException, ParseException{
CustomRestTemplate restTemplate = new CustomRestTemplate ();
final String uri = "http://friend:5000/users";
ResponseEntity<String> result = restTemplate.postForEntity(uri, user, String.class);
return result
}
With Spring-boot 1.5.10.RELEASE, I am getting response.body as null.
Here is how I am using RestTemplate
RestTemplate restTemplate = new RestTemplate();
List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>();
interceptors.add(new LoggingRequestInterceptor());
restTemplate.setInterceptors(interceptors);
String url = "http://someurl/Commands";
MultiValueMap<String, String> params = new LinkedMultiValueMap<>();
params.add("cmd", "{\"operation\":\"getSomeDetails\"}}");
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<>(params, headers);
ResponseEntity<String> response = restTemplate.postForEntity(url, request, String.class);
System.out.println("This is always null: " + response.getBody());
While above program always prints null,
following interceptor prints valid response body
public class LoggingRequestInterceptor implements ClientHttpRequestInterceptor {
final static Logger log = LoggerFactory.getLogger(LoggingRequestInterceptor.class);
#Override
public ClientHttpResponse intercept(final HttpRequest request, final byte[] body,
final ClientHttpRequestExecution execution) throws IOException {
traceRequest(request, body);
ClientHttpResponse response = execution.execute(request, body);
traceResponse(response);
return response;
}
private void traceResponse(ClientHttpResponse response) throws IOException {
StringBuilder inputStringBuilder = new StringBuilder();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(response.getBody(), "UTF-8"));
String line = bufferedReader.readLine();
while (line != null) {
inputStringBuilder.append(line);
inputStringBuilder.append('\n');
line = bufferedReader.readLine();
}
log.debug("============================response begin==========================================");
log.debug("Status code : {}", response.getStatusCode());
log.debug("Status text : {}", response.getStatusText());
log.debug("Headers : {}", response.getHeaders());
log.debug("Response body: {}", inputStringBuilder.toString());
log.debug("=======================response end=================================================");
}
}
Although the accepted answer has the reason, I believe the solution is also necessary.
Spring has a BufferingClientHttpRequestFactory that acts as a wrapper to Rest Template's default SimpleClientHttpRequestFactory.
It can be passed to a Rest Template during creation. This forces the Rest Template to make interceptors use a copy of the response rather than destroying it.
ClientHttpRequestFactory factory = new BufferingClientHttpRequestFactory(new SimpleClientHttpRequestFactory());
RestTemplate restTemplate = new RestTemplate(factory);
Source :
http://objectpartners.com/2018/03/01/log-your-resttemplate-request-and-response-without-destroying-the-body/
You're consuming the response body in traceResponse; that's your problem. Also, please update your question to be specific; "all latest" means nothing. What's latest today isn't so tomorrow.
Below code will resolve issue.
#Bean public RestTemplate restTemplate() {
final RestTemplate restTempate = new RestTemplate(new BufferingClientHttpRequestFactory(new
SimpleClientHttpRequestFactory()));
final List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>();
interceptors.add(new LogHttpInterceptor());
restTempate.setInterceptors(interceptors);
return restTemplate;}
While log interceptor will be like below
public class LogHttpInterceptor implements ClientHttpRequestInterceptor {
final static Logger log = LoggerFactory.getLogger(LogHttpInterceptor.class);
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
traceRequest(request, body);
ClientHttpResponse response = execution.execute(request, body);
traceResponse(response);
return response;
}
private void traceRequest(HttpRequest request, byte[] body) throws IOException {
log.info("===========================================================================request begin");
log.debug("URI : {}", request.getURI());
log.debug("Method : {}", request.getMethod());
log.debug("Headers : {}", request.getHeaders() );
log.debug("Request body: {}", new String(body, "UTF-8"));
log.info("=============================================================================request end");
}
private void traceResponse(ClientHttpResponse response) throws IOException {
StringBuilder inputStringBuilder = new StringBuilder();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(response.getBody(), "UTF-8"));
String line = bufferedReader.readLine();
while (line != null) {
inputStringBuilder.append(line);
inputStringBuilder.append('\n');
line = bufferedReader.readLine();
}
log.info("==========================================================================response begin");
log.debug("Status code : {}", response.getStatusCode());
log.debug("Status text : {}", response.getStatusText());
log.debug("Headers : {}", response.getHeaders());
log.debug("Response body: {}", inputStringBuilder.toString());
log.info("===========================================================================response end");
}
Let me know if doesn't work
After some time of searching I tried to use HttpComponentsClientHttpRequestFactory() instead of SimpleClientHttpRequestFactory()
ClientHttpRequestFactory factory = new BufferingClientHttpRequestFactory(new HttpComponentsClientHttpRequestFactory());
That solved the issue for me.
Create your RestTemplate like this
#Bean
public RestTemplate interceptedRestTemplate() {
RestTemplate restTemplate = new RestTemplate(new BufferingClientHttpRequestFactory(
new SimpleClientHttpRequestFactory()
));
restTemplate.setInterceptors(List.of(<i>your interceptor</i>));
return restTemplate;
}
worked for me.
I'm getting this json response with some timeout message in the beginning and the class throwing Not valid jSON exception.
Json Response
failed to connect: timeout I should not be here {Valid json}
but when I test through postman , I don't see that frustrating message.
I'm using spring RestTemplate` to call the service
final HttpEntity<String> entity = new HttpEntity<String>(headers);
final ResponseEntity<String> exchange = restTemplate.exchange(uri.toUriString(),
HttpMethod.GET, entity, String.class);
Even though I increase timeout values, it's the same
#PostConstruct
public void init() {
restTemplate = new RestTemplate(clientHttpRequestFactory());
restTemplate.setErrorHandler(new DefaultResponseErrorHandler() {
protected boolean hasError(HttpStatus statusCode) {
return false;
}
});
}
private ClientHttpRequestFactory clientHttpRequestFactory() {
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(10000);
factory.setConnectTimeout(10000);
return factory;
}
This is my camel route
from("direct:processRequest").process(new RecipientListBean()).recipientList(header("recipients"))
.aggregationStrategy(aggregationStrategy)
.parallelProcessing()
.to("bean:transformerBean");