I could easily chain coroutine Flows by doing:
val someFlow = flow { //Some logic that my succeed or throw error }
val anotherFlow = flow { // Another logic that my succeed or throe error }
val resultingFlow = someFlow.flatmapLatest(anotherFlow)
but what if I want to individually be able to retry someFlow and anotherFlow where if someFlow already succeed to return a value but anotherFlow failed, I want to retry anotherFlow by using the value from someFlow (The value return when it succeeded).
Whats the best way to do this?
You could use the retryWhen operator on anotherFlow like this:
val someFlow = flow {
//Some logic that my succeed or throw error
}
val anotherFlow = flow {
// Another logic that my succeed or throe error
}
.retryWhen { cause, attempt ->
if (cause is IOException) { // retry on IOException
emit(“Some value”) // emit anything you want before retry
delay(1000) // delay for one second before retry
true
} else { // do not retry otherwise
false
}
}
val resultingFlow = someFlow.flatmapLatest(anotherFlow)
Just be careful because you could end up retrying forever. Use the attempt argument to check the number of times you have retried.
Here is the retryWhen operator official documentation: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/retry-when.html
Have you considered using zip?
I haven't tested or anything but this might be worth a try.
val someFlow = flow {}
val anotherFlow = flow {}
someFlow.zip(anotherFlow) { some, another ->
if(another is Resource.Error)
repository.fetchAnother(some)
else
some
}.collect {
Log.d(TAG, it)
}
Related
I have a kotlinjs app. I handle a particular event (dropping of data onto a component) like this:
onEvent {
drop = { event ->
GlobalScope.async {
//...
dropTask(y, data)
}
}
}
// ...
// this function has to be a suspend function because model's is
private suspend fun dropTask(y: Int, taskId: TaskId) {
// ... prepare data
model.insertBefore(taskId!!, insertBefore?.id)
}
// ... Model's function is defined like this:
suspend fun insertBefore(taskToInsert: TaskId, taskBefore: TaskId?) {
val (src, _) = memory.find(taskToInsert)
// ... and finally, the find function is:
fun find(taskId: TaskId): Pair<Task?, Int> {
// ...
return if (task != null) {
// ...
} else {
throw Exception("Couldn't find task with id $taskId!!")
}
}
The issue is that the Exception gets thrown, but isn't reported anywhere.
I have tried:
a) Installing a CoroutineExceptionHandler into the GlobalScope.async (i.e.:
val handler = CoroutineExceptionHandler { _, e ->
console.log("Caught exception: $e")
}
GlobalScope.async(handler) {
...but this never gets called. This would be relatively clean if I could make it work. It would be even nicer if this was default behavior for kotlinjs, so that exceptions weren't accidentally unreported.
b) Calling await:
drop = { event ->
GlobalScope.launch {
GlobalScope.async() {
// ...
dropTask(y, data)
}.await()
}
}
This does result in the exception being logged to the console, but it's so ugly. It's not possible to call .await() outside of a suspend function or coroutine, so for this particular event handler I have to wrap the async call in a launch. I must be doing something wrong. Anybody have a better pattern that I should be using?
I have a service that calls a dependency via REST. Service and dependency are part of a microservice architecture, so I'd like to use resilience patterns. My goals are:
Have a circuit-breaker to protect the dependency when it's struggling
Limit the time the call can run. The service has an SLA and has to answer in a certain time. On timeout we use the fallback value.
Limit the number of concurrent calls to the dependency. Usually the rate of calls is low and the responses are fast, but we want to protect the dependency against bursts and queue requests inside the service.
Below is my current code. It works, but ideally I'd like to use the TimeLimiter and Bulkhead classes as they seem to be built to work together.
How can I write this better?
#Component
class FooService(#Autowired val circuitBreakerRegistry: CircuitBreakerRegistry)
{
...
// State machine to take load off the dependency when slow or unresponsive
private val circuitBreaker = circuitBreakerRegistry
.circuitBreaker("fooService")
// Limit parallel requests to dependency
private var semaphore = Semaphore(maxParallelRequests)
// The protected function
private suspend fun makeHttpCall(customerId: String): Boolean {
val client = webClientProvider.getCachedWebClient(baseUrl)
val response = client
.head()
.uri("/the/request/url")
.awaitExchange()
return when (val status = response.rawStatusCode()) {
200 -> true
204 -> false
else -> throw Exception(
"Foo service responded with invalid status code: $status"
)
}
}
// Main function
suspend fun isFoo(someId: String): Boolean {
try {
return circuitBreaker.executeSuspendFunction {
semaphore.withPermit {
try {
withTimeout(timeoutMs) {
makeHttpCall(someId)
}
} catch (e: TimeoutCancellationException) {
// This exception has to be converted because
// the circuit-breaker ignores CancellationException
throw Exception("Call to foo service timed out")
}
}
}
} catch (e: CallNotPermittedException) {
logger.error { "Call to foo blocked by circuit breaker" }
} catch (e: Exception) {
logger.error { "Exception while calling foo service: ${e.message}" }
}
// Fallback
return true
}
}
Ideally I'd like to write something like the docs describe for Flows:
// Main function
suspend fun isFoo(someId: String): Boolean {
return monoOf(makeHttpCall(someId))
.bulkhead(bulkhead)
.timeLimiter(timeLimiter)
.circuitBreaker(circuitBreaker)
}
You could also use Resilience4j's Bulkhead instead of your own Semaphore and Resilience4j's TimeLimiter.
You can stack you CircuitBreaker with bulkhead.executeSuspendFunction and timelimiter.executeSuspendFunction.
im breaking my mind around how to do this in RX.
The actual usecase is mapping of LowerLevelEvent(val userId: String) to HigherLevelEvent(val user: User), where the User is provided by observable, so it can emit n times, so example output
LowerLevelEvent1(abc) -> HigherLevelEvent1(userAbc(nameVariation1)
LowerLevelEvent2(abc) -> HigherLevelEvent2(userAbc(nameVariation1)
LowerLevelEvent3(abc) -> HigherLevelEvent3(userAbc(nameVariation1)
LowerLevelEvent4(abc) -> HigherLevelEvent4(userAbc(nameVariation1)
HigherLevelEvent4(userAbc(nameVariation2)
HigherLevelEvent4(userAbc(nameVariation3)
So my naive solution was to use combineLatest. So while userId is not changed user observable is subscribed, i.e. not resubscribed when new lowerLevelEmits & its userId is not changed
val _lowerLevelEventObservable: Observable<LowerLevelEvent> = lowerLevelEventObservable
.replayingShare()
val _higherLevelEventObservable: Observable<HigherLevelEvent> = Observables
.combineLatest(
_lowerLevelEventObservable,
_lowerLevelEventObservable
.map { it.userId }
.distinctUntilChanged()
.switchMap { userRepository.findByIdObservable(it)
) { lowerLevelEvent, user -> createHigherLevelInstance... }
However this has glitch issues, since both sources in combineLatest originate from same observable.
Then I thought about
lowerLevelObservable.
.switchMap { lowerLevelEvent ->
userRepository.findByIdObservable(lowerLevelEvent.userId)
.map { user -> createHigherLevelInstance... }
}
This however can break if lowerLevelObservable emits fast, and since user observable can take some time, given lowerLevelX event can be skipped, which I cannot have. Also it resubscribes user observable each emit, which is wasteful since it wont change most likely
So, maybe concatMap? That has issue of that the user observable doesnt complete, so concatMap wouldnt work.
Anyone have a clue?
Thanks a lot
// Clarification:
basically its mapping of A variants (A1, A2..) to A' variants (A1', A2'..) while attaching a queried object to it, where the query is observable so it might reemit after the mapping was made, so AX' needs to be reemited with new query result. But the query is cold and doesnt complete
So example A1(1) -> A1'(user1), A2(1) -> A2'(user1), A3(1) -> A3'(user1) -- now somebody changes user1 somewhere else in the app, so next emit is A3'(user1')
Based on the comments you have made, the below would work in RxSwift. I have no idea how to translate it to RxJava. Honestly though, I think there is a fundamental misuse of Rx here. Good luck.
How it works: If it's allowed to subscribe it will, otherwise it will add the event to a buffer for later use. It is allowed to subscribe if it currently isn't subscribed to an inner event, or if the inner Observable it's currently subscribed to has emitted an element.
WARNING: It doesn't handle completions properly as it stands. I'll leave that to you as an exercise.
func example(lowerLevelEventObservable: Observable<LowerLevelEvent>, userRepository: UserRepository) {
let higherLevelEventObservable = lowerLevelEventObservable
.flatMapAtLeastOnce { event in // RxSwift's switchLatest I think.
Observable.combineLatest(
Observable.just(event),
userRepository.findByIdObservable(event.userId),
resultSelector: { (lowLevelEvent: $0, user: $1) }
)
}
.map { createHigherLevelInstance($0.lowLevelEvent, $0.user) }
// use higherLevelEventObservable
}
extension ObservableType {
func flatMapAtLeastOnce<U>(from fn: #escaping (E) -> Observable<U>) -> Observable<U> {
return Observable.create { observer in
let disposables = CompositeDisposable()
var nexts: [E] = []
var disposeKey: CompositeDisposable.DisposeKey?
var isAllowedToSubscribe = true
let lock = NSRecursiveLock()
func nextSubscription() {
isAllowedToSubscribe = true
if !nexts.isEmpty {
let e = nexts[0]
nexts.remove(at: 0)
subscribeToInner(e)
}
}
func subscribeToInner(_ element: E) {
isAllowedToSubscribe = false
if let key = disposeKey {
disposables.remove(for: key)
}
let disposable = fn(element).subscribe { innerEvent in
lock.lock(); defer { lock.unlock() }
switch innerEvent {
case .next:
observer.on(innerEvent)
nextSubscription()
case .error:
observer.on(innerEvent)
case .completed:
nextSubscription()
}
}
disposeKey = disposables.insert(disposable)
}
let disposable = self.subscribe { event in
lock.lock(); defer { lock.unlock() }
switch event {
case let .next(element):
if isAllowedToSubscribe == true {
subscribeToInner(element)
}
else {
nexts.append(element)
}
case let .error(error):
observer.onError(error)
case .completed:
observer.onCompleted()
}
}
_ = disposables.insert(disposable)
return disposables
}
}
}
I'm new to reactive programming. I expect to see
test provider started
Beat 1000
Beat 2000
in logs but there is only test provider started and no Beat or on complete messages. Looks like I miss something
#Service
class ProviderService {
#PostConstruct
fun start(){
val hb: Flux<HeartBeat> = Flux.interval(Duration.ofSeconds(1)).map { HeartBeat(it) }
val provider = Provider("test", hb)
}
}
////////////////////////
open class Provider(name: String, heartBests: Flux<HeartBeat>) {
companion object {
val log = LoggerFactory.getLogger(Provider::class.java)!!
}
init {
log.info("$name provider started")
heartBests.doOnComplete { log.info("on complete") }
heartBests.doOnEach { onBeat(it.get().number) }
}
fun onBeat(n: Number){
log.info("Beat $n")
}
}
/////
class HeartBeat(val number: Number)
three pretty common mistakes here:
operators like doOnEach return a new Flux instance with the added behavior, so you need to (re)assign to a variable or use a fluent style
nothing happens until you subscribe() (or a variant of it. blockXXX do also subscribe under the hood for instance...)
such a pipeline is fully asynchronous, and runs on a separate Thread due to the time dimension of the source, interval. As a result, control would immediately return in init even if you had subscribed, potentially causing the main thread and then the app to exit.
In your code lambda from 'doOnComplete' has been never called, because you created infinite stream. Method 'doOnEach' as 'map' is intermediate operations (like map in streams), its doesn't make a call.
And you have another mistake, reactive suggests "fluent pattern".
Try this simple example:
import reactor.core.publisher.Flux
import java.time.Duration
fun main(args: Array<String>) {
val flux = Flux.interval(Duration.ofSeconds(1)).map { HeartBeat(it) }
println("start")
flux.take(3)
.doOnEach { println("on each $it") }
.map { println("before map");HeartBeat(it.value * 2) }
.doOnNext { println("on next $it") }
.doOnComplete { println("on complete") }
.subscribe { println("subscribe $it") }
Thread.sleep(5000)
}
data class HeartBeat(val value: Long)
I'm using promisekit 3.0 to help chain alamofire callbacks in a clean way. The objective is to start with a network call, with a promise to return an array of urls.
Then, I'm looking to execute network calls on as many of those urls as needed to find the next link i'm looking for. As soon as this link is found, I can pass it to the next step.
This part is where I'm stuck.
I can pick an arbitrary index in the array that I know has what I want, but I can't figure out the looping to keep it going until the right information is returned.
I tried learning from this obj-c example, but i couldn't get it working in swift.
https://stackoverflow.com/a/30693077/1079379
He's a more tangible example of what i've done.
Network.sharedInstance.makeFirstPromise(.GET, url: NSURL(string: fullSourceLink)! )
.then { (idArray) -> Promise<AnyObject> in
let ids = idArray as! [String]
//how do i do that in swift? (from the example SO answer)
//PMKPromise *p = [PMKPromise promiseWithValue: nil]; // create empty promise
//only thing i could do was feed it the first value
var p:Promise<AnyObject> = Network.sharedInstance.makePromiseRequestHostLink(.POST, id: ids[0])
//var to hold my eventual promise value, doesn't really work unless i set it to something first
var goodValue:Promise<AnyObject>
for item in ids {
//use continue to offset the promise from before the loop started
continue
//hard part
p = p.then{ returnValue -> Promise<AnyObject> in
//need a way to check if what i get is what i wanted then we can break the loop and move on
if returnValue = "whatIwant" {
goodvalue = returnValue
break
//or else we try again with the next on the list
}else {
return Network.sharedInstance.makeLoopingPromise(.POST, id: item)
}
}
}
return goodValue
}.then { (finalLink) -> Void in
//do stuck with finalLink
}
Can someone show me how to structure this properly, please?
Is nesting promises like that anti-pattern to avoid? In that case, what is the best approach.
I have finally figured this out with a combination of your post and the link you posted. It works, but I'll be glad if anyone has input on a proper solution.
func download(arrayOfObjects: [Object]) -> Promise<AnyObject> {
// This stopped the compiler from complaining
var promise : Promise<AnyObject> = Promise<AnyObject>("emptyPromise")
for object in arrayOfObjects {
promise = promise.then { _ in
return Promise { fulfill, reject in
Service.getData(stuff: object.stuff completion: { success, data in
if success {
print("Got the data")
}
fulfill(successful)
})
}
}
}
return promise
}
The only thing I'm not doing is showing in this example is retaining the received data, but I'm assuming you can do that with the results array you have now.
The key to figuring out my particular issue was using the "when" function. It keeps going until all the calls you inputted are finished. The map makes it easier to look at (and think about in my head)
}.then { (idArray) -> Void in
when(idArray.map({Network.sharedInstance.makePromiseRequest(.POST, params: ["thing":$0])})).then{ link -> Promise<String> in
return Promise { fulfill, reject in
let stringLink:[String] = link as! [String]
for entry in stringLink {
if entry != "" {
fulfill(entry)
break
}
}
}
}.then {
}
}