Currently I want to call a service using WebClient which returns csv content with response header as text/csv. I want to something like below and convert the CSV response to POJO.
#Data
public class Address {
String name;
String street;
String id;
String city;
}
public class SOQLBulkJobResultResponse<T> {
List<T> records;
}
//Read SalesForceBulkAPIReponse
public SOQLBulkJobResultResponse<T> getJobResult(UriComponents uriComponents, final Class<T> clazz) {
URI uri = UriComponentsBuilder.fromUriString(baseUrl).uriComponents(uriComponents).build().toUri();
ParameterizedTypeReference<SOQLBulkJobResultResponse<T>> typeReference =
ParameterizedTypeReference.forType(ResolvableType.forClassWithGenerics(SOQLBulkJobResultResponse.class, clazz).getType());
log.info("Calling out: " + uriComponents);
return Optional.ofNullable(this.webClient.get()
.uri(uri)
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.retrieve()
.bodyToMono(typeReference)
.retry(1)
.share()
.block())
.orElseThrow(() -> new IllegalStateException("No response from /queryResponse endpoint for URI: " + uri));
}
// Get CSV Data from API
#Bean
public WebClientCustomizer webClientCustomizer() {
HttpClient httpClient = HttpClient.create().responseTimeout(Duration.ofSeconds(responseTimeoutSec))
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectionTimeoutSec * 1000);
return webClientBuilder -> webClientBuilder
.codecs(
configurer -> {
ObjectMapper csvDecoderObjectMapper = new ObjectMapper();
csvDecoderObjectMapper.configure(DeserializationFeature.FAIL_ON_IGNORED_PROPERTIES, false);
configurer.customCodecs().registerWithDefaultConfig(new Jackson2JsonDecoder(csvDecoderObjectMapper,new MediaType("text", "csv")));
ObjectMapper encoderObjectMapper = new ObjectMapper();
encoderObjectMapper.setPropertyNamingStrategy(PropertyNamingStrategies.UPPER_CAMEL_CASE);
ObjectMapper decoderObjectMapper = new ObjectMapper();
decoderObjectMapper.setPropertyNamingStrategy(PropertyNamingStrategies.UPPER_CAMEL_CASE);
configurer.defaultCodecs().jackson2JsonEncoder(new Jackson2JsonEncoder(encoderObjectMapper, MediaType.APPLICATION_JSON));
configurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(decoderObjectMapper, MediaType.APPLICATION_JSON));
configurer.defaultCodecs().maxInMemorySize(10 * 1024 * 1024);
})
.filter(WebClientFilter.handleErrors())
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
}
As I understand Jackson2JsonDecoder is not right decoder for CSV content. Need solutions/suggestion here.
Related
Getting a null when i make call to post endpoint using spring webclient
I tried using webclient post end point. Got null instead og Object as return type
final int size = 16 * 1024 * 1024;
final ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(size))
.build();
#Bean
public WebClient webClient() {
return WebClient
.builder()
.exchangeStrategies(strategies)
.build();
}
Object = countryz = webClient.post()
.uri(new URI("https://countriesnow.space/api/v0.1/countries/population"))
.contentType(MediaType.APPLICATION_JSON)
.accept(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromObject(country))
.retrieve().bodyToMono(Object.class).block();
Create Weblcient Bean
#Bean
public WebClient webClient() {
final int size = 16 * 1024 * 1024;
final ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(size))
.build();
return WebClient.builder()
.exchangeStrategies(strategies)
.build();
}
In your service class
#Autowired
private WebClient webClient;
Object countryz = webClient.post()
.uri("https://countriesnow.space/api/v0.1/countries/population")
.header("cache-control", "no-cache")
.header("content-type", "application/json")
.body(BodyInserters.fromObject(Collections.singletonMap("country", "nigeria")))
.retrieve().bodyToMono(Object.class).block();
I am trying to POST a payload with almost 4-5 MB of data through RestTemplate. Some of the request are passing but some of the requests are not even going to API gateway with this error
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting
for connection from pool
Here is the rest template implementation I am using.
#Component
public class RestTemplateHelper {
#Autowired
private RestTemplate restTemplate;
public RestTemplateHelper() {
}
private <X, Y> Y post(final String url, final String accessToken, final MediaType mediaType, final X requestBody, final Class<Y> responseType, final Map<String, String> extraHeaders, HttpMessageConverter httpMessageConverter, RestTemplate restTemplateFromParam) {
HttpHeaders headers = this.getHeaders(accessToken, mediaType, extraHeaders);
HttpEntity<X> httpEntity = new HttpEntity(requestBody, headers);
ResponseEntity responseEntity;
NonRetriableException nonRetriableException;
try {
RestTemplate restTemplateToUse = this.restTemplate;
if (Objects.nonNull(httpMessageConverter)) {
restTemplateToUse = Objects.nonNull(restTemplateFromParam) ? restTemplateFromParam : restTemplateToUse;
restTemplateToUse.getMessageConverters().add(0, httpMessageConverter);
}
responseEntity = restTemplateToUse.exchange(url, HttpMethod.POST, httpEntity, responseType, new Object[0]);
} catch (ResourceAccessException var14) {
nonRetriableException = new NonRetriableException(var14);
nonRetriableException.setRootException(var14);
throw nonRetriableException;
} catch (RestClientResponseException var15) {
nonRetriableException = new NonRetriableException(var15);
nonRetriableException.setRawStatusCode(var15.getRawStatusCode());
nonRetriableException.setStatusText(var15.getStatusText());
nonRetriableException.setResponseBody(var15.getResponseBodyAsString());
nonRetriableException.setResponseHeaders(var15.getResponseHeaders());
nonRetriableException.setRootException(var15);
throw nonRetriableException;
} catch (Exception var16) {
throw new NonRetriableException(var16);
}
return responseEntity.getBody();
}
}
Method in question
#GetMapping("/all")
public Mono<ResponseEntity<String>> getSomeData(#RequestHeader String someId) {
...some code
}
Tried to call the consume the endpoint with this method:
#Autowired
WebClient.Builder webClient;
String someString = webClient.
.get()
.uri(someUrl)
.header("someId", "someString")
.retrieve()
.bodyToMono(String.class)
.block();
I got a status 415 with Unsupported media type with "Content type '' not supported"
How do I use webClientBuilder to set my id header?
You just need to set the correct content-type. If your controller expects it to be "plain/text" you might have to set that explicitly within your requesting client. 415 does indicate a miss match.
As mentioned by #Alex you are autowiring builder instead look for the concrete implementation of WebClient. Please check my WebClient config bean. But that is not the actual issue.
When you are sending body with webClient you have to use
.body(...)
so for sending plain text body where controller is expecting plain body you need something like below:
.body(BodyInserters.fromProducer(Mono.just("random body"), String.class))
and when controller is expecing an object is request you need to use something like this
.body(BodyInserters.fromProducer(Mono.just(new Greet("Hello there this is the body of post request")), Greet.class))
Greet.java
public static class Greet {
String name;
public Greet() {
}
public Greet(String name) {
this.name = name;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
Configuration of WebCLient
#Configuration
class WebClientConfig {
#Bean
WebClient webClient() {
return WebClient.builder().baseUrl("http://localhost:8080/").build();
}
}
#RequestMapping("/sample")
#RestController
static class SampleComntroller {
private final WebClient webClient;
#Autowired
SampleComntroller(WebClient webClient) {
this.webClient = webClient;
}
#GetMapping(value = "/main-get")//, consumes = MediaType.APPLICATION_JSON_VALUE)
public Mono<String> helloGet(#RequestHeader(name = "someId") String someId) {
return Mono.just("Hello, Spring!, get, response with header is=>" + someId);
}
#PostMapping(value = "/main-post-plain-string", consumes = MediaType.TEXT_PLAIN_VALUE)
public Mono<String> helloPost(#RequestHeader(name = "someId") String someId, #RequestBody String body) {
return Mono.just("Hello, Spring!, post, response with header is=>" + someId + " and random body " + UUID.randomUUID().toString());
}
#PostMapping(value = "/main-post-object", consumes = MediaType.APPLICATION_JSON_VALUE)
public Mono<String> helloPostObject(#RequestHeader(name = "someId") String someId, #RequestBody Greet greet) {
return Mono.just("Hello, Spring!, post, response with header is=>" + someId + " " + greet.getName() + " " + UUID.randomUUID().toString());
}
#GetMapping("/delegate-get")
public String delegateGet() {
return webClient
.get()
.uri("/sample/main-get")
.header("someId", "178A-0E88-get")
.retrieve().bodyToMono(String.class).block();
}
#PostMapping("/delegate-post")
public String delegatePost() {
return webClient
.post()
.uri("/sample/main-post-plain-string")
.body(BodyInserters.fromProducer(Mono.just("random body"), String.class))
.header("someId", "178A-0E88-post")
.retrieve()
.bodyToMono(String.class).block();
}
#PostMapping("/delegate-post-object")
public String delegatePostObject() {
return webClient
.post()
.uri("/sample/main-post-object")
.body(BodyInserters.fromProducer(Mono.just(new Greet("Hello there this is the body of post request")), Greet.class))
.header("someId", "178A-0E88-post")
.retrieve()
.bodyToMono(String.class).block();
}
}
The service provider supplies me with the header data: grant_type, Content-Type. And the body data: grant_type, username and password. Which I use in Postman where it generates OK token. But in the Spring application it generates an error HttpClientErrorException $ BadRequest: 400 Bad Request.
I have the class to set the body data:
public class BodyToken {
private String grant_type = "password";//set body data
private String username = "User";//set body data
private String password = "123";//set body data
private String access_token;
#JsonGetter("access_token")
public String getAccess_token() {
return access_token;
}
public void setAccess_token(String access_token) {
this.access_token = access_token;
}
#JsonGetter("grant_type")
public String getGrant_type() {
return grant_type;
}
#JsonGetter("username")
public String getUsername() {
return username;
}
#JsonGetter("password")
public String getPassword() {
return password;
}
}
This is the controller where the header data is set:
#PostMapping("/TokenGeneration")
#ResponseBody
public BodyToken TokenGeneration() throws IOException {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.set("grant_type", "password");//set header data
headers.set("Content-Type", "application/x-www-form-urlencoded");//set header data
HttpEntity request = new HttpEntity(headers);
headers.add("User-Agent", "Spring's RestTemplate" );
ResponseEntity<BodyToken> response = restTemplate.exchange(
"https://sw/token",
HttpMethod.POST,
request,
BodyToken.class
);
try {
return response.getBody();
} catch (Exception e) {
BodyToken body = new BodyToken();
log.info(e.getMessage());
return body;
}
}
OK was solved with using the Class MultiValueMap and LinkedMultiValueMap. The credentials are added to this new object and it is sent together with the request:
#PostMapping("/TokenGeneration")
#ResponseBody
public BodyToken TokenGeneration() throws IOException {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.set("grant_type", "password");//set header data
headers.set("Content-Type", "application/x-www-form-urlencoded");//set header data
MultiValueMap<String, String> body = new LinkedMultiValueMap<String, String>();//line solution
body.add("grant_type", "password");//line solution
body.add("username", "user");//line solution
body.add("password", "123");//line solution
HttpEntity request = new HttpEntity(body, headers);//and I add this body to HttpEntity
headers.add("User-Agent", "Spring's RestTemplate" );
ResponseEntity<BodyToken> response = restTemplate.exchange(
"https://sw/token",
HttpMethod.POST,
request,
BodyToken.class
);
try {
return response.getBody();
} catch (Exception e) {
BodyToken body = new BodyToken();
log.info(e.getMessage());
return body;
}
}
This call works as expected and makes the POST successfully:
public class MyService implements IMyService {
private final WebClient webClient;
private final String url;
MyService(#Qualifier("web-client") WebClient webClient,
String url) {
this.webClient = webClient;
this.url = url;
}
#SneakyThrows
#Override
public void execute(Long jobId) {
MultiValueMap<String, String> requestParms = new LinkedMultiValueMap<>();
requestParms.add("arguments", "--batchJobId=" + jobId.toString());
HttpEntity<MultiValueMap<String, String>> requestEntity =
new HttpEntity<>(requestParms, null);
final WebClient.ResponseSpec responseSpec = webClient.post()
.uri(new URI(url + "/tasks/executions"))
.body(BodyInserters.fromMultipartData(requestParms))
.exchange()
.block();
}
}
Inside the configuration class:
#Bean
#Qualifier("web-client")
public WebClient getWebClient() {
return WebClient.builder()
.filter(basicAuthentication("user", "pass"))
.filter(printLnFilter())
.build();
}
private ExchangeFilterFunction printLnFilter() {
return (request, next) -> {
System.out.println("\n\n" + request.method().toString().toUpperCase() + ":\n\nURL:"
+ request.url().toString() + ":\n\nHeaders:" + request.headers().toString() + "\n\nAttributes:"
+ request.attributes() + "\n\n");
return next.exchange(request);
};
}
In the example above, we see the URL, Attributes, and Headers logged and the Http call success fully made. However, just removing the block() call results in no call ever being made, no logs:
// No call made
final WebClient.ResponseSpec responseSpec = webClient.post()
.uri(new URI(url + "/tasks/executions"))
.body(BodyInserters.fromMultipartData(requestParms))
.exchange();
That's because it's non blocking...
From Spring Docs:
Simply put, WebClient is an interface representing the main entry
point for performing web requests.
It has been created as a part of the Spring Web Reactive module and
will be replacing the classic RestTemplate in these scenarios. The new
client is a reactive, non-blocking solution that works over the
HTTP/1.1 protocol.
It's an implementation using the Reactive Streams concept through the Project Reactor implementation