I am using caffeine cache to store an authorisation token that has been obtained using webClient WebFlux. I have set the expireAfterWrite to a hardcoded value in the application.yml file as follows:
spring:
cache:
cache-names: accessTokens
caffeine:
spec: expireAfterWrite=100m
The token is obtained using a WebClient with Spring WebFlux as below code depicts:
#Autowired
var cacheManager: CacheManager? = null
override fun getAuthToken(): Mono<AccessToken> {
val map = LinkedMultiValueMap<String, String>()
map.add("client_id", clientId)
map.add("client_secret", clientSecret)
map.add("grant_type", "client_credentials")
var cachedVersion = this.cacheManager?.getCache("accessTokens");
if (cachedVersion?.get("tokens") != null) {
val token = cachedVersion.get("tokens")
return Mono.just(token?.get() as AccessToken)
} else {
return webClient.post()
.uri("/client-credentials/token")
.body(BodyInserters.fromFormData(map))
.retrieve()
.onStatus(HttpStatus::is5xxServerError) {
ClientLogger.logClientErrorResponse(it, errorResponse)
}
.onStatus(HttpStatus::is4xxClientError) {
ClientLogger.logClientErrorResponse(it, errorResponse)
}
.bodyToMono(AccessToken::class.java)
.doOnNext { response ->
// Set here the expiration time of the cache based on
// response.expiresIn
this.cacheManager?.getCache("accessTokens")?.put("tokens", response) }
.log()
}
}
I am storing the token after the data is emitted/returned successfully within the .doOnNext() method but i need to be able to set the expiration time or refresh the hardcoded expiration time of the cache based on the expiresIn property that is part of the response object,
.doOnNext { response ->
// Set here the expiration time of the cache based on
// response.expiresIn
this.cacheManager?.getCache("accessTokens")?.put("tokens", response)
}
Any ideas would be much appreciated.
// Policy to set the lifetime based on when the entry was created
var expiresAfterCreate = new Expiry<String, AccessToken>() {
public long expireAfterCreate(String credentials, AccessToken token, long currentTime) {
Duration duration = token.expiresIn();
return token.toNanos();
}
public long expireAfterUpdate(String credentials, AccessToken token,
long currentTime, long currentDuration) {
return currentDuration;
}
public long expireAfterRead(String credentials, AccessToken token,
long currentTime, long currentDuration) {
return currentDuration;
}
});
// CompletableFuture-based cache
AsyncLoadingCache<String, AccessToken> cache = Caffeine.newBuilder()
.expireAfter(expiresAfterCreate)
.buildAsync((credentials, executor) -> {
Mono<AccessToken> token = retrieve(credentials);
return token.toFuture();
});
// Get from cache, loading if absent, and converts to Mono
Mono<AccessToken> getAuthToken() {
var token = cache.get(credentials);
return Mono.fromFuture(token);
}
Related
I have a spring boot application that uses Retrofit to make requests to a secured server.
My endpoints:
public interface ServiceAPI {
#GET("/v1/isrcResource/{isrc}/summary")
Call<ResourceSummary> getResourceSummaryByIsrc(#Path("isrc") String isrc);
}
public interface TokenServiceAPI {
#FormUrlEncoded
#POST("/bbcb6b2f-8c7c-4e24-86e4-6c36fed00b78/oauth2/v2.0/token")
Call<Token> obtainToken(#Field("client_id") String clientId,
#Field("scope") String scope,
#Field("client_secret") String clientSecret,
#Field("grant_type") String grantType);
}
Configuration class:
#Bean
Retrofit tokenAPIFactory(#Value("${some.token.url}") String tokenUrl) {
Retrofit.Builder builder = new Retrofit.Builder()
.baseUrl(tokenUrl)
.addConverterFactory(JacksonConverterFactory.create());
return builder.build();
}
#Bean
Retrofit serviceAPIFactory(#Value("${some.service.url}") String serviceUrl, TokenServiceAPI tokenAPI) {
OkHttpClient okHttpClient = new OkHttpClient.Builder()
.addInterceptor(new ServiceInterceptor(clientId, scope, clientSecret, grantType, apiKey, tokenAPI))
.build();
Retrofit.Builder builder = new Retrofit.Builder()
.baseUrl(repertoireUrl)
.client(okHttpClient)
.addConverterFactory(JacksonConverterFactory.create());
return builder.build();
}
Interceptor to add the Authorization header to every request
public class ServiceInterceptor implements Interceptor {
public ServiceInterceptor(String clientId,
String scope,
String clientSecret,
String grantType,
String apiKey,
TokenServiceAPI tokenAPI) {
this.clientId = clientId;
this.scope = scope;
this.clientSecret = clientSecret;
this.grantType = grantType;
this.apiKey = apiKey;
this.tokenAPI = tokenAPI;
}
#Override
public Response intercept(Chain chain) throws IOException {
Request newRequest = chain.request().newBuilder()
.addHeader(AUTHORIZATION_HEADER, getToken())
.addHeader(API_KEY_HEADER, this.apiKey)
.build();
return chain.proceed(newRequest);
}
private String getToken() throws IOException {
retrofit2.Response<Token> tokenResponse = repertoireTokenAPI.obtainToken(clientId, scope, clientSecret, grantType).execute();
String accessToken = "Bearer " + tokenAPI.body().getAccessToken();
return accessToken;
}
}
This is working as expected, the problem is that the token is being requested for every request rather than using the existing valid one. How can one store the token somewhere and re-use it? I was wondering if Retrofit had a built-in solution.
a possible option with caching:
add caffeiene
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</dependency>
add #Cacheable("your-token-cache-name") on the method returning the token, looks like getToken above
add max cache size and expiration configuration in application.yml
e.g. 500 entries and 10 minutes for configuration below
spring.cache.cache-names=your-token-cache-name
spring.cache.caffeine.spec=maximumSize=500,expireAfterAccess=600s
example from: https://www.javadevjournal.com/spring-boot/spring-boot-with-caffeine-cache/
can some one help me with a correct pattern for stateful service in SpringWebflux? I have a REST service which communicates with an external API and needs to fetch auth token from that API during the first call and cache it to reuse in all next calls. Currently I'm having a code which works, but concurrent calls cause multiple token requests. Is there a way to handle concurrency?
#Service
#RequiredArgsConstructor
public class ExternalTokenRepository {
private final WebClient webClient;
private Object cachedToken = null;
public Mono<Object> getToken() {
if (cachedToken != null) {
return Mono.just(cachedToken);
} else {
return webClient.post()
//...
.exchangeToMono(response -> {
//...
return response.bodyToMono(Object.class)
})
.doOnNext(token -> cachedToken = token)
}
}
}
UPDATED: Token I receive have some expiration and I need to refresh it after some time. Refresh request should be call only once too.
You can initialize Mono in the constructor and use cache operator:
#Service
public class ExternalTokenRepository {
private final Mono<Object> cachedToken;
public ExternalTokenRepository(WebClient webClient) {
this.cachedToken = webClient.post()
//...
.exchangeToMono(response -> {
//...
return response.bodyToMono(Object.class);
})
.cache(); // this is the important part
}
public Mono<Object> getToken() {
return cachedToken;
}
}
UPDATE: cache operator also supports TTL based on the return value: https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#cache-java.util.function.Function-java.util.function.Function-java.util.function.Supplier-
I am trying to validate and log form data that goes through Spring Cloud Gateway. I have tried a few methods and encounter a few problems and I could not read it properly. I have tried:
#Component
public class GatewayRequestFilter {
#Bean
public GlobalFilter apply() {
return (exchange, chain) -> {
MediaType contentType = exchange.getRequest().getHeaders().getContentType();
ModifyRequestBodyGatewayFilterFactory.Config modifyRequestConfig = new ModifyRequestBodyGatewayFilterFactory.Config();
/// Method 1
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
modifyRequestConfig.setRewriteFunction(String.class, String.class, (exchange1, originalRequestBody) -> {
validateAndAuditLog(exchange1, originalRequestBody);
return Mono.just(originalRequestBody);
});
}
/// Method 2
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
return exchange.getMultipartData().flatMap(originalRequestBody -> {
validateAndAuditLog(exchange1, originalRequestBody);
return chain.filter(exchange);
});
}
/// Method 3:
/// https://github.com/spring-cloud/spring-cloud-gateway/issues/1307#issuecomment-553910834
return new ModifyRequestBodyGatewayFilterFactory().apply(modifyRequestConfig).filter(exchange, chain);
};
}
}
For the 1st and 3rd method, if I set inClass as String.class then I can see data in some kind of http format. The problem is that I don't know how to parse it into hashMap or LinkedMultiValueMap to access each of value using key. Here is the output I get:
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-text"
text
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-file"; filename="simple-file"
Content-Type: application/octet-stream
Simple file
----------------------------162653831591335516327921--
If I change inClass as Object.class then there is another error:
{
"timestamp": "2020-04-03T02:37:57.096+0000",
"path": "/tc/test/test",
"status": 500,
"error": "Internal Server Error",
"message": "Content type 'multipart/form-data;boundary=--------------------------537619313111072161580699' not supported for bodyType=java.lang.Object",
"requestId": "0592497a-1"
}
For the 2nd method I can get data in LinkedMultiValueMap which is good because I can read each data using key value and I can also get uploaded files name, but the problem is that, it hang for 10s before pass the request to down stream.
Anyone has any idea what should I do to read or modify form data that goes through Spring Cloud Gateway?
Rewriting the answer with example.
Basic approach is defined here, though it needs lot of refinement to work for multi-part.
https://developpaper.com/question/how-to-modify-the-request-parameters-of-multipart-form-data-format-in-spring-cloud-gateway/
For any approach to work once you read the data, you need to set a modified request object to exchange downstream to be processed again. Setting the new multi-part object downstream is bit tricky because there is not a straightforward way to convert string->multi-part->string.
Here is a sample code based on the approach. Note that this for now works only if multi-part contains form fields and not file type fields, because in later case we are dealing with a stream, which can be embedded anywhere within the entire multi-part request, and it is not possible to modify such request without blocking calls, which the netty does not allow.
private final List<HttpMessageReader<?>> messageReaders = HandlerStrategies.withDefaults().messageReaders();
public GatewayFilter apply(Config config) {
return new OrderedGatewayFilter((exchange, chain) -> {
ServerRequest serverRequest = ServerRequest.create(exchange, messageReaders);
// get modified body from original body o
Mono<MultiValueMap<String, String>> modifiedBody = serverRequest.bodyToMono(String.class).flatMap(o -> {
// create mock request to read body
SynchronossPartHttpMessageReader synchronossReader = new SynchronossPartHttpMessageReader();
MultipartHttpMessageReader reader = new MultipartHttpMessageReader(synchronossReader);
MockServerHttpRequest request = MockServerHttpRequest.post("").contentType(exchange.getRequest().getHeaders().getContentType()).body(o);
Mono<MultiValueMap<String, Part>> monoRequestParts = reader.readMono(MULTIPART_DATA_TYPE, request, Collections.emptyMap());
// modify parts
return monoRequestParts.flatMap(requestParts -> {
Map<String, List<String>> modifedBodyArray = requestParts.entrySet().stream().map(entry -> {
String key = entry.getKey();
LOGGER.info(key);
List<String> entries = entry.getValue().stream().map(part -> {
LOGGER.info("{}", part);
// read the input part
String input = ((FormFieldPart) part).value();
// return the modified input part
return new String(modifyRequest(config, exchange, key, input));
}).collect(Collectors.toList());
return new Map.Entry<String, List<String>>() {
#Override
public String getKey() {
return key;
}
#Override
public List<String> getValue() {
return entries;
}
#Override
public List<String> setValue(List<String> param1) {
return param1;
}
};
}).collect(Collectors.toMap(k -> k.getKey(), k -> k.getValue()));
return Mono.just(new LinkedMultiValueMap<String, String>(modifedBodyArray));
});
});
// insert the new modified body
BodyInserter bodyInserter = BodyInserters.fromPublisher(modifiedBody, new ParameterizedTypeReference<MultiValueMap<String, String>>() {});
HttpHeaders headers = new HttpHeaders();
headers.putAll(exchange.getRequest().getHeaders());
// the new content type will be computed by bodyInserter
// and then set in the request decorator
headers.remove(HttpHeaders.CONTENT_LENGTH);
CachedBodyOutputMessage outputMessage = new CachedBodyOutputMessage(exchange, headers);
return bodyInserter.insert(outputMessage, new BodyInserterContext())
.then(Mono.defer(() -> {
ServerHttpRequest decorator = decorate(exchange, headers, outputMessage);
return chain.filter(exchange.mutate().request(decorator).build());
}));
}, RouteToRequestUrlFilter.ROUTE_TO_URL_FILTER_ORDER + 1);
}
// some of the helper methods
private String modifyRequest(Config config, ServerWebExchange exchange, String key, String input) {
// do your thing in here !!!
return input;
}
private ServerHttpRequestDecorator decorate(ServerWebExchange exchange, HttpHeaders headers, CachedBodyOutputMessage outputMessage) {
return new ServerHttpRequestDecorator(exchange.getRequest()) {
#Override
public HttpHeaders getHeaders() {
long contentLength = headers.getContentLength();
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.putAll(headers);
if (contentLength > 0) {
httpHeaders.setContentLength(contentLength);
} else {
// TODO: this causes a 'HTTP/1.1 411 Length Required' // on httpbin.org
httpHeaders.set(HttpHeaders.TRANSFER_ENCODING, "chunked");
}
return httpHeaders;
}
#Override
public Flux<DataBuffer> getBody() {
return outputMessage.getBody();
}
};
}
I have Spring Boot microservice, and sending large payload using swagger. At the server I get only 15000 chars and reset 2000 chars are not read.
How can I use ReadBodyPredicateFactory to cache the body message text?
I am using springcloudgateway and added filters. In the filter in apply method I am trying to read the payload json using
DefaultServerRequest serverRequest = new DefaultServerRequest(exchange);
body = serverRequest.bodyToMono(String.class).toFuture().get();
Sometimes it hangs.
I tried with Flux and then i get only half message
Flux body = request.getBody();
body.subscribe(buffer -> {
try {
System.out.println("byte count:" +
buffer.readableByteCount());
byte[] bytes = new byte[buffer.readableByteCount()];
buffer.read(bytes);
DataBufferUtils.release(buffer);
String bodyString = new String(bytes, StandardCharsets.UTF_8);
sb.append(bodyString);
} catch (Exception e) {
e.printStackTrace();
}
Recently, I needed the similar thing in my application and I've found that it can be achieved by Spring Cloud Gateway built-in caching in ServerWebExchangeUtils
Before filters that use request content in some business cases, I created a filter that only forces content caching:
#Component
class CachingRequestBodyFilter extends AbstractGatewayFilterFactory<CachingRequestBodyFilter.Config> {
public CachingRequestBodyFilter() {
super(Config.class);
}
public GatewayFilter apply(final Config config) {
return (exchange, chain) -> ServerWebExchangeUtils.cacheRequestBody(exchange,
(serverHttpRequest) -> chain.filter(exchange.mutate().request(serverHttpRequest).build()));
}
public static class Config {
}
}
In any of the subsequent filters, we can extract the content of the request body, as below:
// some ReadRequestBodyFilter filter
public GatewayFilter apply(final Config config) {
return (exchange, chain) -> {
final var cachedBody = new StringBuilder();
final var cachedBodyAttribute = exchange.getAttribute(CACHED_REQUEST_BODY_ATTR);
if (!(cachedBodyAttribute instanceof DataBuffer)) {
// caching gone wrong error handling
}
final var dataBuffer = (DataBuffer) cachedBodyAttribute;
cachedBody.append(StandardCharsets.UTF_8.decode(dataBuffer.asByteBuffer()).toString());
final var bodyAsJson = cachedBody.toString();
// some processing
return chain.filter(exchange);
};
}
Then the gateway configuration would look like this:
spring:
cloud:
gateway:
routes:
- [...]
filters:
- CachingRequestBodyFilter
- ReadRequestBodyFilter
I'm new to Spring boot and reactive programming.
I'm using spring webflux webclient for an external api service. I need to fetch the auth token and set it in the header
WebClient.builder()
.baseUrl(baseUrl)
.filter((request, next) -> {
return next.exchange(request)
.flatMap((Function<ClientResponse, Mono<ClientResponse>>) clientResponse -> {
if (clientResponse.statusCode().value() == 401) {
return authenticate().map(token -> {
Token accessToken = authenticate().block();
ClientRequest retryRequest = ClientRequest.from(request).header("Authorisation", "Bearer " + accessToken.getAccessToken()).build();
return next.exchange(retryRequest);
}).
} else {
return Mono.just(clientResponse);
}
});
})
.defaultHeader("Authorization", "Bearer " + authToken.getAccessToken())
.build();
private Mono<Token> authenticate() {
MultiValueMap<String, String> params = new LinkedMultiValueMap();
params.add("client_id", clientId);
params.add("client_secret", clientSecret);
params.add("grant_type", "password");
params.add("username", username);
params.add("password", password);
WebClient client = WebClient.create(baseUrl);
return client
.post()
.uri(tokenUri)
.accept(MediaType.APPLICATION_JSON)
.contentType(MediaType.APPLICATION_FORM_URLENCODED)
.syncBody(params)
.retrieve()
.bodyToMono(Token.class);
}
private static class Token {
#JsonProperty("access_token")
private String accessToken;
public String getAccessToken() { return accessToken; }
}
During the application startup, I'll fetch the access token and set it in the webclient builder. I've created a filter to handle authentication failures after token expiry. But the above code throws error because I've used block() which is not supposed to be used in a reactor thread. How else can I handle it? I'm using oauth2 resource owner password grant flow. Is there is any other way to handle the flow?
Hi I had the same issue (Adding a retry all requests of WebClient) which looks like you have reused.
but here flatmap is your friend, if you have a Mono<Mono<T>> you can flatten it with flatMap
builder.baseUrl("http://localhost:8080")
//sets the header before the exchange
.filter(((request, next) -> tokenProvider.getAccessToken()
.map(setBearerTokenInHeader(request))
.flatMap(next::exchange)))
//do the exchange
.filter((request, next) -> next.exchange(request)
.flatMap(clientResponse -> {
if (clientResponse.statusCode().value() == 401) {
//If unauthenicated try again
return authenticate()
.flatMap(Token::getAccessToken)
.map(setBearerTokenInHeader(request))
.flatMap(next::exchange);
} else {
return Mono.just(clientResponse);
}
}))
.build();
private Function<String, ClientRequest> setBearerTokenInHeader(ClientRequest request) {
return token -> ClientRequest.from(request).header("Bearer ", token).build();
}
I know this is an old thread, but I could not find any other working examples for the initial question
Basically, I was not able to write a working code from the above examples...
With the main task: Use WebClient instance to get protected resource by providing Bearer token. The Bearer token can be requested by a separate request.
The Mono authenticate() should work fine to get a new token.
WebClient client2 = WebClient.builder()
.baseUrl(SERVER_URL)
.filter((request, next) -> {
return next.exchange(request)
.flatMap( clientResponse -> {
if (clientResponse.statusCode().value() == 401) {
return authenticate().map(token -> {
Token accessToken = authenticate().block();
ClientRequest retryRequest = ClientRequest.from(request).header("Authorisation", "Bearer " + accessToken.getAccessToken()).build();
return next.exchange(retryRequest);
});
} else {
return Mono.just(clientResponse);
}
});
})
.defaultHeader("Authorization", "Bearer " + token.getAccessToken())
.build();
For the above example was not able to replace the ".block()" with flatMap()
And the second example
WebClient client3 = WebClient.builder().baseUrl("http://localhost:8080")
//sets the header before the exchange
.filter(((request, next) -> tokenProvider.getAccessToken()
.map(setBearerTokenInHeader(request))
.flatMap(next::exchange)))
//do the exchange
.filter((request, next) -> next.exchange(request)
.flatMap(clientResponse -> {
if (clientResponse.statusCode().value() == 401) {
//If unauthenicated try again
return authenticate()
.flatMap(Token::getAccessToken)
.map(setBearerTokenInHeader(request))
.flatMap(next::exchange);
} else {
return Mono.just(clientResponse);
}
}))
.build();
Not sure what is the "tokenProvider.getAccessToken()" and ".flatMap(Token::getAccessToken)" won't accept
Due to
class Token {
String token = "";
public String getAccessToken() { return token; }
}
Sorry I'm new to this. If you had a working example please share in this thread