I have Resilience4j version: 1.7.1, Kotlin version: 1.7.0, Kotlin Coroutines: 1.6.1.
I'd like to use RateLimiter and Retry in kotlin code, but documentations don't contain information how to use Kotlin Flow with them.
I have a simple code:
suspend main() {
val rateLimiterConfig = RateLimiterConfig.custom()
.limitForPeriod(2)
.limitRefreshPeriod(Duration.ofSeconds(1))
.timeoutDuration(Duration.ofSeconds(2))
.build()
val rateLimiter = RateLimiter.of("rate-limiter", rateLimiterConfig)
val retryConfig = RetryConfig.custom<Any>()
.maxAttempts(3)
.retryExceptions(Exception::class.java)
.build()
val retry = Retry.of("retry", retryConfig)
coroutineScope {
flowOf(1,2,3,4,5,6,7,8)
.rateLimiter(rateLimiter)
.retry(retry)
.map { async { process(it) } }
.toList().awaitAll()
}
}
suspend fun process(num: Int): Int {
println("time: ${getTime()}, value: $num")
if(num >= 8) throw Exception()
delay(1000)
return num * num
}
And I don't have any limiting or retry.
If run this code with printing time(mm:ss.SSS) and incoming value, I have this:
time: 46:26.488,value: 7
time: 46:26.488,value: 4
time: 46:26.488,value: 3
time: 46:26.488,value: 1
time: 46:26.488,value: 6
time: 46:26.488,value: 5
time: 46:26.488,value: 8
time: 46:26.488,value: 2
Exception in thread "main" java.lang.Exception
at MainKt.process(Main.kt:165)
at MainKt$main$2$1$1.invokeSuspend(Main.kt:142)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
How does it work?
I think this is what you want:
coroutineScope {
flowOf(1,2,3,4,5,6,7,8)
.rateLimiter(rateLimiter)
.map { process(it) }
.retry(retry)
.toList()
}
1. Retries
retry from Resilience4j uses Flow.retryWhen under the hood. To make it work you have to use it after .map invocation. Also, .retry operator will retry the whole flow, not only the failed operation.
kotlinx.coroutines docs:
Retries collection of the given flow when an exception occurs in the upstream flow and the predicate returns true.
This operator is transparent to exceptions that occur in downstream flow and does not retry on exceptions that are thrown to cancel the flow.
2. Rate limiting
Using async { } and then .awaitAll kinda parallelizes the whole process, so rateLimiter won't be able to do its job. Just do .map { process(it) }.
Related
From time to time Spring REST function fails with: "kotlinx.coroutines.JobCancellationException: MonoCoroutine was cancelled".
It is suspend function which calls another service using spring-webflux client. There are multiple suspend functions in my rest class. Looks like this problem occurs when multiple requests arrive to the same time. But may be not :-)
Application runs on Netty server.
Example:
#GetMapping("/customer/{id}")
suspend fun getCustomer(#PathVariable #NotBlank id: String): ResponseEntity<CustomerResponse> =
withContext(MDCContext()) {
ResponseEntity.status(HttpStatus.OK)
.body(customerService.aggregateCustomer(id))
}
Service call:
suspend fun executeServiceCall(vararg urlData: Input) = webClient
.get()
.uri(properties.url, *urlData)
.retrieve()
.bodyToMono(responseTypeRef)
.retryWhen(
Retry.fixedDelay(properties.retryCount, properties.retryBackoff)
.onRetryExhaustedThrow { _, retrySignal ->
handleRetryException(retrySignal)
}
.filter { it is ReadTimeoutException || it is ConnectTimeoutException }
)
.onErrorMap {
// throw exception
}
.awaitFirstOrNull()
Part of Stack Trace:
Caused by: kotlinx.coroutines.JobCancellationException: MonoCoroutine was cancelled; job="coroutine#1":MonoCoroutine{Cancelling}#650774ce
at kotlinx.coroutines.JobSupport.cancel(JobSupport.kt:1578)
at kotlinx.coroutines.Job$DefaultImpls.cancel$default(Job.kt:183)
at kotlinx.coroutines.reactor.MonoCoroutine.dispose(Mono.kt:122)
at reactor.core.publisher.FluxCreate$SinkDisposable.dispose(FluxCreate.java:1033)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.disposeResource(MonoCreate.java:313)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.cancel(MonoCreate.java:300)
I am struggling with proper coroutine usage on cache handling using spring boot #Cacheable with ehcache on two methods:
calling another service using webclient:
suspend fun getDeviceOwner(correlationId: String, ownerId: String): DeviceOwner{
webClient
.get()
.uri(uriProvider.provideUrl())
.header(CORRELATION_ID, correlationId)
.retrieve()
.onStatus(HttpStatus::isError) {response ->
Mono.error(
ServiceCallExcpetion("Call failed with: ${response.statusCode()}")
)
}.awaitBodyOrNull()
?: throw ServiceCallExcpetion("Call failed with - response is null.")
}
calling db using r2dbc
suspend fun findDeviceTokens(ownerId: UUID, deviceType: String) {
//CoroutineCrudRepository.findTokens
}
What seems to be working for me is calling from:
suspend fun findTokens(data: Data): Collection<String> = coroutineScope {
val ownership = async(Dispatchers.IO, CoroutineStart.LAZY) { service.getDeviceOwner(data.nonce, data.ownerId) }.await()
val tokens = async(Dispatchers.IO, CoroutineStart.LAZY) {service.findDeviceTokens(ownership.ownerId, ownership.ownershipType)}
tokens.await()
}
#Cacheable(value = ["ownerCache"], key = "#ownerId")
fun getDeviceOwner(correlationId: String, ownerId: String)= runBlocking(Dispatchers.IO) {
//webClientCall
}
#Cacheable("deviceCache")
override fun findDeviceTokens(ownerId: UUID, deviceType: String) = runBlocking(Dispatchers.IO) {
//CoroutineCrudRepository.findTokens
}
But from what I am reading it's not good practise to use runBlocking.
https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine
Would it block the main thread or the thread which was designated by the parent coroutine?
I also tried with
#Cacheable(value = ["ownerCache"], key = "#ownerId")
fun getDeviceOwnerAsync(correlationId: String, ownerId: String) = GlobalScope.async(Dispatchers.IO, CoroutineStart.LAZY) {
//webClientCall
}
#Cacheable("deviceCache")
override fun findDeviceTokensAsync(ownerId: UUID, deviceType: String) = GlobalScope.async(Dispatchers.IO, CoroutineStart.LAZY) {
//CoroutineCrudRepository.findTokens
}
Both called from suspended function without any additional coroutineScope {} and async{}
suspend fun findTokens(data: Data): Collection<String> =
service.getDeviceOwnerAsync(data.nonce,data.ownerId).await()
.let{service.findDeviceTokensAsync(it.ownerId, it.ownershipType).await()}
I am reading that using GlobalScope is not good practise either due to possible endless run of this coroutine when something stuck or long response (in very simple words). Also in this approach, using GlobalScope, when I tested negative scenarios and external ms call resulted with 404(on purpose) result was not stored in the cache (as I excepted) but for failing CoroutineCrudRepository.findTokens call (throwing exception) Deferred value was cached which is not what I wanted. Storing failing exececution results is not a thing with runBlocking.
I tried also #Cacheable("deviceCache", unless = "#result.isCompleted == true && #result.isCancelled == true")
but it also seems to not work as I would imagine.
Could you please advice the best coroutine approach with correct exception handling for integrating with spring boot caching which will store value in cache only on non failing call?
Although annotations from Spring Cache abstraction are fancy, I also, unfortunately, haven't found any official solution for using them side by side with Kotlin coroutines.
Yet there is a library called spring-kotlin-coroutine that claims to solve this issue. Though, never tried as it doesn't seem to be maintained any longer - the last commit was pushed in May 2019.
For the moment I've been using CacheManager bean and managing the aforementioned manually. I found that a better solution rather than blocking threads.
Sample code with Redis as a cache provider:
Dependency in build.gradle.kts:
implementation("org.springframework.boot:spring-boot-starter-data-redis-reactive")
application.yml configuration:
spring:
redis:
host: redis
port: 6379
password: changeit
cache:
type: REDIS
cache-names:
- account-exists
redis:
time-to-live: 3m
Code:
#Service
class AccountService(
private val accountServiceApiClient: AccountServiceApiClient,
private val redisCacheManager: RedisCacheManager
) {
suspend fun isAccountExisting(accountId: UUID): Boolean {
if (getAccountExistsCache().get(accountId)?.get() as Boolean? == true) {
return true
}
val account = accountServiceApiClient.getAccountById(accountId) // this call is reactive
if (account != null) {
getAccountExistsCache().put(account.id, true)
return true
}
return false
}
private fun getAccountExistsCache() = redisCacheManager.getCache("account-exists") as RedisCache
}
In the Kotlin Coroutines context, every suspend function has 1 additional param of type kotlin.coroutines.Continuation<T>, that's why the org.springframework.cache.interceptor.SimpleKeyGenerator generates always a wrong key. Also, the CacheInterceptor does not know anything about suspend functions, so, it stores a COROUTINE_SUSPENDED object instead of the actual value, without evaluating the suspended wrapper.
You can check this repository https://github.com/konrad-kaminski/spring-kotlin-coroutine, they added Cache support for Coroutines, the specific Cache support implementation is here -> https://github.com/konrad-kaminski/spring-kotlin-coroutine/blob/master/spring-kotlin-coroutine/src/main/kotlin/org/springframework/kotlin/coroutine/cache/CoroutineCacheConfiguration.kt.
Take a look at CoroutineCacheInterceptor and CoroutineAwareSimpleKeyGenerator,
Hope this fixes your issue
Using spring reactive WebClient, I consume an API and in case of response with 500 status I need to retry with exponential backoff. But in Mono class, I don't see any retryBackoff with Predicate as input parameter.
This is the kind of function I search for:
public final Mono<T> retryBackoff(Predicate<? super Throwable> retryMatcher, long numRetries, Duration firstBackoff)
Right now my implementation is as following (I don't have retry with backOff mechanism):
client.sendRequest()
.retry(e -> ((RestClientException) e).getStatus() == 500)
.subscribe();
You might want to have a look at the reactor-extra module in the reactor-addons project. In Maven you can do:
<dependency>
<groupId>io.projectreactor.addons</groupId>
<artifactId>reactor-extra</artifactId>
<version>3.2.3.RELEASE</version>
</dependency>
And then use it like this:
client.post()
.syncBody("test")
.retrieve()
.bodyToMono(String.class)
.retryWhen(Retry.onlyIf(ctx -> ctx.exception() instanceof RestClientException)
.exponentialBackoff(firstBackoff, maxBackoff)
.retryMax(maxRetries))
Retry.onlyIf is now deprecated/removed.
If anyone is interested in the up-to-date solution:
client.post()
.syncBody("test")
.retrieve()
.bodyToMono(String.class)
.retryWhen(Retry.backoff(maxRetries, minBackoff).filter(ctx -> {
return ctx.exception() instanceof RestClientException && ctx.exception().statusCode == 500;
}))
It's worth mentioning that retryWhen wraps the source exception into the RetryExhaustedException. If you want to 'restore' the source exception you can use the reactor.core.Exceptions util:
.onErrorResume(throwable -> {
if (Exceptions.isRetryExhausted(throwable)) {
throwable = throwable.getCause();
}
return Mono.error(throwable);
})
I'm not sure, what spring version you are using, in 2.1.4 I have this:
client.post()
.syncBody("test")
.retrieve()
.bodyToMono(String.class)
.retryBackoff(numretries, firstBackoff, maxBackoff, jitterFactor);
... so that's exactly what you want, right?
I'm currently trying it with Kotlin Coroutines + Spring WebFlux:
It seems the following is not working:
suspend fun ClientResponse.asResponse(): ServerResponse =
status(statusCode())
.headers { headerConsumer -> headerConsumer.addAll(headers().asHttpHeaders()) }
.body(bodyToMono(DataBuffer::class.java), DataBuffer::class.java)
.retryWhen {
Retry.onlyIf { ctx: RetryContext<Throwable> -> (ctx.exception() as? WebClientResponseException)?.statusCode in retryableErrorCodes }
.exponentialBackoff(ofSeconds(1), ofSeconds(5))
.retryMax(3)
.doOnRetry { log.error("Retry for {}", it.exception()) }
)
.awaitSingle()
AtomicInteger errorCount = new AtomicInteger();
Flux<String> flux =
Flux.<String>error(new IllegalStateException("boom"))
.doOnError(e -> {
errorCount.incrementAndGet();
System.out.println(e + " at " + LocalTime.now());
})
.retryWhen(Retry
.backoff(3, Duration.ofMillis(100)).jitter(0d)
.doAfterRetry(rs -> System.out.println("retried at " + LocalTime.now() + ", attempt " + rs.totalRetries()))
.onRetryExhaustedThrow((spec, rs) -> rs.failure())
);
We will log the time of errors emitted by the source and count them.
We configure an exponential backoff retry with at most 3 attempts and no jitter.
We also log the time at which the retry happens, and the retry attempt number (starting from 0).
By default, an Exceptions.retryExhausted exception would be thrown, with the last failure() as a cause. Here we customize that to directly emit the cause as onError.
I'm looking to eagerly cache the results of a Reactor Mono. It's scheduled to be updated in cache every 10 minutes, but since the Mono is only evaluated when subscribed to, the task doesn't actually refresh the cache.
Example:
#Scheduled(fixedRate = 10 * 60 * 1000 + 3000)
fun getMessage(): Mono<String> {
return Mono.just("Hello")
.map { it.toUpperCase() }
.cache(Duration.ofMinutes(10))
}
You need to store your Mono somewhere, otherwise each invocation of the method (through the Scheduled or directly) will return a different instance.
Perhaps as a companion object?
Here is how I would do it naïvely in Java:
protected Mono<String> cached;
//for the scheduler to periodically eagerly refresh the cache
#Scheduled(fixedRate = 10 * 60 * 1000 + 3000)
void refreshCache() {
this.cached = Mono.just("Hello")
.map { it.toUpperCase() }
.cache(Duration.ofMinutes(10));
this.cached.subscribe(v -> {}, e -> {}); //swallows errors during refresh
}
//for users
public Mono<String> getMessage() {
return this.cached;
}
I am working with Vert.x 2.x (http://vertx.io) which makes extensive use of asynchronous callbacks. These quickly become unwieldy with typical nesting/callback hell issues.
I have considered both Scala Futures/Promises (which I think would be the defacto approach) and also Reactive Extensions (RxScala).
From my testing I have found some interesting performance results.
My testing is pretty basic, I'm just issuing a bunch of HTTP requests (via weighttp) to a Vert.x verticle that makes an asynchronous call across the Vert.x eventbus, and processes a response that is then returned in an HTTP 200 response.
What I have found is the following (performance here is measured in terms of HTTP requests per second):
Async Callback performance = 68,305 rps
Rx performance = 64,656 rps
Future/Promises performance = 61,376 rps
The test conditions were:
Mac Pro OS X Yosemite 10.10.2
Oracle JVM 1.8U25
weighttp version 0.3
Vert.x 2.1.5
Scala 2.10.4
RxScala 0.23.0
4 x Web Service Verticle Instances
4 x Backend Service Verticle Instances
The test command was
weighttp -n 1000000 -c 128 -7 8 -k "localhost:8888"
The figures above are the average of five test runs less best and worst result. Note the results are very consistent around the presented average (no more than a few hundred rps deviation).
Is there any known reason why the above might be happening - i.e. Rx > Futures in pure requests per second?
Reactive Extensions in my opinion are superior as they can do so much more but given the standard approach to async callbacks typically seems to go down the Futures/Promises track I'm surprised at the performance hit.
EDIT: Here is the Web Service Verticle
class WebVerticle extends Verticle {
override def start() {
val port = container.env().getOrElse("HTTP_PORT", "8888").toInt
val approach = container.env().getOrElse("APPROACH", "ASYNC")
container.logger.info("Listening on port: " + port)
container.logger.info("Using approach: " + approach)
vertx.createHttpServer.requestHandler { req: HttpServerRequest =>
approach match {
case "ASYNC" => sendAsync(req, "hello")
case "FUTURES" => sendWithFuture("hello").onSuccess { case body => req.response.end(body) }
case "RX" => sendWithObservable("hello").doOnNext(req.response.end(_)).subscribe()
}
}.listen(port)
}
// Async callback
def sendAsync(req: HttpServerRequest, body: String): Unit = {
vertx.eventBus.send("service.verticle", body, { msg: Message[String] =>
req.response.end(msg.body())
})
}
// Rx
def sendWithObservable(body: String) : Observable[String] = {
val subject = ReplaySubject[String]()
vertx.eventBus.send("service.verticle", body, { msg: Message[String] =>
subject.onNext(msg.body())
subject.onCompleted()
})
subject
}
// Futures
def sendWithFuture(body: String) : Future[String] = {
val promise = Promise[String]()
vertx.eventBus.send("service.verticle", body, { msg: Message[String] =>
promise.success(msg.body())
})
promise.future
}
}
EDIT: Here is the Backend Verticle
class ServiceVerticle extends Verticle {
override def start(): Unit = {
vertx.eventBus.registerHandler("service.verticle", { msg: Message[String] =>
msg.reply("Hello Scala")
})
}
}