Getting a null when i make call to post endpoint using spring webclient
I tried using webclient post end point. Got null instead og Object as return type
final int size = 16 * 1024 * 1024;
final ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(size))
.build();
#Bean
public WebClient webClient() {
return WebClient
.builder()
.exchangeStrategies(strategies)
.build();
}
Object = countryz = webClient.post()
.uri(new URI("https://countriesnow.space/api/v0.1/countries/population"))
.contentType(MediaType.APPLICATION_JSON)
.accept(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromObject(country))
.retrieve().bodyToMono(Object.class).block();
Create Weblcient Bean
#Bean
public WebClient webClient() {
final int size = 16 * 1024 * 1024;
final ExchangeStrategies strategies = ExchangeStrategies.builder()
.codecs(codecs -> codecs.defaultCodecs().maxInMemorySize(size))
.build();
return WebClient.builder()
.exchangeStrategies(strategies)
.build();
}
In your service class
#Autowired
private WebClient webClient;
Object countryz = webClient.post()
.uri("https://countriesnow.space/api/v0.1/countries/population")
.header("cache-control", "no-cache")
.header("content-type", "application/json")
.body(BodyInserters.fromObject(Collections.singletonMap("country", "nigeria")))
.retrieve().bodyToMono(Object.class).block();
Related
Currently I want to call a service using WebClient which returns csv content with response header as text/csv. I want to something like below and convert the CSV response to POJO.
#Data
public class Address {
String name;
String street;
String id;
String city;
}
public class SOQLBulkJobResultResponse<T> {
List<T> records;
}
//Read SalesForceBulkAPIReponse
public SOQLBulkJobResultResponse<T> getJobResult(UriComponents uriComponents, final Class<T> clazz) {
URI uri = UriComponentsBuilder.fromUriString(baseUrl).uriComponents(uriComponents).build().toUri();
ParameterizedTypeReference<SOQLBulkJobResultResponse<T>> typeReference =
ParameterizedTypeReference.forType(ResolvableType.forClassWithGenerics(SOQLBulkJobResultResponse.class, clazz).getType());
log.info("Calling out: " + uriComponents);
return Optional.ofNullable(this.webClient.get()
.uri(uri)
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.retrieve()
.bodyToMono(typeReference)
.retry(1)
.share()
.block())
.orElseThrow(() -> new IllegalStateException("No response from /queryResponse endpoint for URI: " + uri));
}
// Get CSV Data from API
#Bean
public WebClientCustomizer webClientCustomizer() {
HttpClient httpClient = HttpClient.create().responseTimeout(Duration.ofSeconds(responseTimeoutSec))
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectionTimeoutSec * 1000);
return webClientBuilder -> webClientBuilder
.codecs(
configurer -> {
ObjectMapper csvDecoderObjectMapper = new ObjectMapper();
csvDecoderObjectMapper.configure(DeserializationFeature.FAIL_ON_IGNORED_PROPERTIES, false);
configurer.customCodecs().registerWithDefaultConfig(new Jackson2JsonDecoder(csvDecoderObjectMapper,new MediaType("text", "csv")));
ObjectMapper encoderObjectMapper = new ObjectMapper();
encoderObjectMapper.setPropertyNamingStrategy(PropertyNamingStrategies.UPPER_CAMEL_CASE);
ObjectMapper decoderObjectMapper = new ObjectMapper();
decoderObjectMapper.setPropertyNamingStrategy(PropertyNamingStrategies.UPPER_CAMEL_CASE);
configurer.defaultCodecs().jackson2JsonEncoder(new Jackson2JsonEncoder(encoderObjectMapper, MediaType.APPLICATION_JSON));
configurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(decoderObjectMapper, MediaType.APPLICATION_JSON));
configurer.defaultCodecs().maxInMemorySize(10 * 1024 * 1024);
})
.filter(WebClientFilter.handleErrors())
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
}
As I understand Jackson2JsonDecoder is not right decoder for CSV content. Need solutions/suggestion here.
I am trying to POST a payload with almost 4-5 MB of data through RestTemplate. Some of the request are passing but some of the requests are not even going to API gateway with this error
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting
for connection from pool
Here is the rest template implementation I am using.
#Component
public class RestTemplateHelper {
#Autowired
private RestTemplate restTemplate;
public RestTemplateHelper() {
}
private <X, Y> Y post(final String url, final String accessToken, final MediaType mediaType, final X requestBody, final Class<Y> responseType, final Map<String, String> extraHeaders, HttpMessageConverter httpMessageConverter, RestTemplate restTemplateFromParam) {
HttpHeaders headers = this.getHeaders(accessToken, mediaType, extraHeaders);
HttpEntity<X> httpEntity = new HttpEntity(requestBody, headers);
ResponseEntity responseEntity;
NonRetriableException nonRetriableException;
try {
RestTemplate restTemplateToUse = this.restTemplate;
if (Objects.nonNull(httpMessageConverter)) {
restTemplateToUse = Objects.nonNull(restTemplateFromParam) ? restTemplateFromParam : restTemplateToUse;
restTemplateToUse.getMessageConverters().add(0, httpMessageConverter);
}
responseEntity = restTemplateToUse.exchange(url, HttpMethod.POST, httpEntity, responseType, new Object[0]);
} catch (ResourceAccessException var14) {
nonRetriableException = new NonRetriableException(var14);
nonRetriableException.setRootException(var14);
throw nonRetriableException;
} catch (RestClientResponseException var15) {
nonRetriableException = new NonRetriableException(var15);
nonRetriableException.setRawStatusCode(var15.getRawStatusCode());
nonRetriableException.setStatusText(var15.getStatusText());
nonRetriableException.setResponseBody(var15.getResponseBodyAsString());
nonRetriableException.setResponseHeaders(var15.getResponseHeaders());
nonRetriableException.setRootException(var15);
throw nonRetriableException;
} catch (Exception var16) {
throw new NonRetriableException(var16);
}
return responseEntity.getBody();
}
}
I have a Spring Boot application that is creating a request to an external system. The external system is responding after some time, 3-4 minutes. I would like to keep the connection open until i receive an response from the remote API. I tried using webflux, i tried setup the connection timeout for my application in application.yml file. I could make the application to wait for a response more than 2 minutes. Some code that i have tried
#Configuration
public class RestConfigurations {
#Bean(name = "restTemplate")
public RestTemplate getRestTemplate(RestTemplateBuilder restTemplateBuilder) {
RestTemplate restTemplate = new RestTemplate(getClientHttpRequestFactory());
// Error handler
restTemplate.setErrorHandler(new CustomRestTemplateErrorHandler());
// Media converters
MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
converter.setSupportedMediaTypes(Arrays.asList(
MediaType.APPLICATION_JSON,
MediaType.TEXT_PLAIN,
MediaType.TEXT_HTML));
restTemplate.getMessageConverters().add(converter);
return restTemplate;
}
private ClientHttpRequestFactory getClientHttpRequestFactory() {
int connectionTimeout = 15*60000; // milliseconds
int socketTimeout = 15*60000; // milliseconds
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(connectionTimeout)
.setConnectionRequestTimeout(connectionTimeout)
.setSocketTimeout(socketTimeout)
.build();
CloseableHttpClient client = HttpClientBuilder
.create()
.setDefaultRequestConfig(config)
.setKeepAliveStrategy(connectionKeepAliveStrategy())
.build();
return new HttpComponentsClientHttpRequestFactory(client);
}
public HttpComponentsClientHttpRequestFactory getHttpClientFactory() {
HttpComponentsClientHttpRequestFactory httpClientFactory = new
HttpComponentsClientHttpRequestFactory(
HttpClients.createDefault()
);
httpClientFactory.setConnectTimeout(15 * 600000);
httpClientFactory.setReadTimeout(15 * 600000);
httpClientFactory.setConnectionRequestTimeout(15 * 600000);
return httpClientFactory;
}
public ClientHttpRequestFactory createRequestFactory(){
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(400);
connectionManager.setDefaultMaxPerRoute(200);
RequestConfig requestConfig = RequestConfig
.custom()
.setConnectionRequestTimeout(5000)
.setSocketTimeout(10000)
.build();
CloseableHttpClient httpClient = HttpClients
.custom()
.setConnectionManager(connectionManager)
.setKeepAliveStrategy(connectionKeepAliveStrategy())
.setDefaultRequestConfig(requestConfig)
.build();
return new HttpComponentsClientHttpRequestFactory(httpClient);
}
private ConnectionKeepAliveStrategy connectionKeepAliveStrategy() {
return (response,context)-> {
HeaderElementIterator it = new BasicHeaderElementIterator(
response.headerIterator(HTTP.CONN_KEEP_ALIVE));
while (it.hasNext()) {
HeaderElement he = it.nextElement();
String param = he.getName();
String value = he.getValue();
if ( value != null && param.equalsIgnoreCase("timeout")){
try {
return Long.parseLong(value) * 999999999;
} catch (NumberFormatException exception) {
exception.printStackTrace();
}
}
}
return 15 * 60000;
} ;
}
#Bean(name = "webClient")
public WebClient webClient() {
TcpClient tcpClient = TcpClient
.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 60000 * 15)//15 minutes
.doOnConnected(connection -> {
connection.addHandlerLast(new ReadTimeoutHandler(15, TimeUnit.MINUTES));
connection.addHandlerLast(new WriteTimeoutHandler(15, TimeUnit.MINUTES));
});
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient).wiretap(true)))
.build();
}
None of this configs worked. I tried using okhttpclient like this.
public class OkHttpClientFactoryImpl implements OkHttpClientFactory {
#Override
public OkHttpClient.Builder createBuilder(boolean disableSslValidation) {
OkHttpClient.Builder builder = new OkHttpClient.Builder();
ConnectionPool okHttpConnectionPool = new ConnectionPool(50, 30, TimeUnit.SECONDS);
builder.connectionPool(okHttpConnectionPool);
builder.connectTimeout(20, TimeUnit.MINUTES);
builder.readTimeout(20, TimeUnit.MINUTES);
builder.writeTimeout(20, TimeUnit.MINUTES);
builder.retryOnConnectionFailure(false);
return builder;
}
}
#Bean
#Qualifier("OKSpringCommonsRestTemplate")
public ClientHttpRequestFactory createOKCommonsRequestFactory() {
OkHttpClientFactoryImpl httpClientFactory = new OkHttpClientFactoryImpl();
OkHttpClient client = httpClientFactory.createBuilder(false).build();
return new OkHttp3ClientHttpRequestFactory(client);
}
It did not work. I don't know what could cause the connection to close other than the things i've setup.
After receiving an answer, i tried :
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(url))
.timeout(Duration.ofMinutes(10))
.header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(json))
.build();
HttpClient client= HttpClient.newHttpClient();
try {
client.send(request, HttpResponse.BodyHandlers.ofString());
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
The result is the same. After 2 minutes i get an error saying java.io.IOException: HTTP/1.1 header parser received no bytes
I found the problem. It was in fact because of the server on the other end of the call. Express server is closing the connection, by default, after 2 minutes.
Increase Client Idle Timeout at server/LB to a higher number.
I have a Spring OAuth 2 server based on Spring Boot 1.5 (Spring Security v4) which generates customized tokens and a few resource servers who communicate with this authorization server, making use of /oauth/check_token endpoint by configuration of RemoteTokenServices.
All the logic related to storing/retrieving tokens on Authorization server side is done with JdbcTokenStore.
I am building a new Spring Boot 2 application which is build with Spring webflux module and trying to implement client_credentials flow with existing Authorization Server using Spring Security 5.1.1.
I found that support for resource servers was added in 5.1.0.RC1 (https://spring.io/blog/2018/08/21/spring-security-5-1-0-rc1-released#oauth2-resource-servers) and updated in 5.1.0.RC2 (https://spring.io/blog/2018/09/10/spring-security-5-1-0-rc2-released#oauth2-resource-server) but looks like it's only possible to configure it with JWT support.
I might be messing up with concepts here but looking for more info and a way to configure all these components together.
I'm in same situation as you.I solve that problem in next way, maybe it can help you:
spring-boot-starter-parent.version: 2.1.1
spring-cloud-dependencies.version: Greenwich.R1
Security configuration:
#EnableWebFluxSecurity
public class SecurityConfig {
#Autowired
private ReactiveAuthenticationManager manager; //custom implementation
#Bean
SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http) {
return http
.authorizeExchange()
.pathMatchers("/role").hasRole("ADMIN")
.pathMatchers("/test").access(new HasScope("server")) //custom implementation
.anyExchange().authenticated()
.and()
.httpBasic().disable()
.oauth2ResourceServer()
.jwt()
.authenticationManager(manager)
.and().and()
.build();
}
}
ReactiveAuthorizationManager (HasScope) implementation:
Helper which allow search for scopes in authentication object
public class HasScope implements ReactiveAuthorizationManager<AuthorizationContext> {
public HasScope(String...scopes) {
this.scopes = Arrays.asList(scopes);
}
private final Collection<String> scopes;
#Override
public Mono<AuthorizationDecision> check(Mono<Authentication> authentication, AuthorizationContext object) {
return authentication
.flatMap(it -> {
OAuth2Authentication auth = (OAuth2Authentication) it;
Set<String> requestScopes = auth.getOAuth2Request().getScope();
boolean allow = requestScopes.containsAll(scopes);
return Mono.just(new AuthorizationDecision(allow));
});
}
}
ReactiveAuthenticationManager implementation:
That is the main component in configuration which create OAuth2Authentication. There is a problem with response for wrong access_token, it returns only status code without body response.
#Component
public class ReactiveAuthenticationManagerImpl implements ReactiveAuthenticationManager {
private final ResourceServerProperties sso;
private final WebClient.Builder webClient;
private final ObjectMapper objectMapper;
private AuthoritiesExtractor authoritiesExtractor = new FixedAuthoritiesExtractor();
public ReactiveAuthenticationManagerImpl(ResourceServerProperties sso,
#Qualifier("loadBalancedWebClient") WebClient.Builder webClient, ObjectMapper objectMapper) {
this.sso = sso;
this.webClient = webClient;
this.objectMapper = objectMapper;
}
#Override
public Mono<Authentication> authenticate(Authentication authentication) {
return Mono.just(authentication)
.cast(BearerTokenAuthenticationToken.class)
.flatMap(it -> getMap(it.getToken()))
.flatMap(result -> Mono.just(extractAuthentication(result)));
}
private OAuth2Authentication extractAuthentication(Map<String, Object> map) {
Object principal = getPrincipal(map);
OAuth2Request request = getRequest(map);
List<GrantedAuthority> authorities = authoritiesExtractor.extractAuthorities(map);
UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(principal, "N/A", authorities);
token.setDetails(map);
return new OAuth2Authentication(request, token);
}
private Object getPrincipal(Map<String, Object> map) {
if (map.containsKey("principal")) {
try {
//that is the case for user authentication
return objectMapper.convertValue(map.get("principal"), UserPrincipal.class);
} catch (IllegalArgumentException ex) {
//that is the case for client authentication
return objectMapper.convertValue(map.get("principal"), String.class);
}
}
return null;
}
#SuppressWarnings({"unchecked"})
private OAuth2Request getRequest(Map<String, Object> map) {
Map<String, Object> request = (Map<String, Object>) map.get("oauth2Request");
String clientId = (String) request.get("clientId");
Set<String> scope = new LinkedHashSet<>(request.containsKey("scope") ?
(Collection<String>) request.get("scope") : Collections.emptySet());
return new OAuth2Request(null, clientId, null, true, new HashSet<>(scope),
null, null, null, null);
}
private Mono<Map<String, Object>> getMap(String accessToken) {
String uri = sso.getUserInfoUri();
return webClient.build().get()
.uri(uri)
.accept(MediaType.APPLICATION_JSON)
.header("Authorization", "Bearer " + accessToken)
.exchange()
.flatMap(it -> it.bodyToMono(new ParameterizedTypeReference<Map<String, Object>>() {}))
.onErrorMap(InvalidTokenException.class, mapper -> new InvalidTokenException("Invalid token: " + accessToken));
}
This call works as expected and makes the POST successfully:
public class MyService implements IMyService {
private final WebClient webClient;
private final String url;
MyService(#Qualifier("web-client") WebClient webClient,
String url) {
this.webClient = webClient;
this.url = url;
}
#SneakyThrows
#Override
public void execute(Long jobId) {
MultiValueMap<String, String> requestParms = new LinkedMultiValueMap<>();
requestParms.add("arguments", "--batchJobId=" + jobId.toString());
HttpEntity<MultiValueMap<String, String>> requestEntity =
new HttpEntity<>(requestParms, null);
final WebClient.ResponseSpec responseSpec = webClient.post()
.uri(new URI(url + "/tasks/executions"))
.body(BodyInserters.fromMultipartData(requestParms))
.exchange()
.block();
}
}
Inside the configuration class:
#Bean
#Qualifier("web-client")
public WebClient getWebClient() {
return WebClient.builder()
.filter(basicAuthentication("user", "pass"))
.filter(printLnFilter())
.build();
}
private ExchangeFilterFunction printLnFilter() {
return (request, next) -> {
System.out.println("\n\n" + request.method().toString().toUpperCase() + ":\n\nURL:"
+ request.url().toString() + ":\n\nHeaders:" + request.headers().toString() + "\n\nAttributes:"
+ request.attributes() + "\n\n");
return next.exchange(request);
};
}
In the example above, we see the URL, Attributes, and Headers logged and the Http call success fully made. However, just removing the block() call results in no call ever being made, no logs:
// No call made
final WebClient.ResponseSpec responseSpec = webClient.post()
.uri(new URI(url + "/tasks/executions"))
.body(BodyInserters.fromMultipartData(requestParms))
.exchange();
That's because it's non blocking...
From Spring Docs:
Simply put, WebClient is an interface representing the main entry
point for performing web requests.
It has been created as a part of the Spring Web Reactive module and
will be replacing the classic RestTemplate in these scenarios. The new
client is a reactive, non-blocking solution that works over the
HTTP/1.1 protocol.
It's an implementation using the Reactive Streams concept through the Project Reactor implementation