I was doing this code lab
https://developer.android.com/codelabs/android-room-with-a-view-kotlin#13
and having a question
class WordsApplication : Application() {
// No need to cancel this scope as it'll be torn down with the process
val applicationScope = CoroutineScope(SupervisorJob())
// Using by lazy so the database and the repository are only created when they're needed
// rather than when the application starts
val database by lazy { WordRoomDatabase.getDatabase(this, applicationScope) }
val repository by lazy { WordRepository(database.wordDao()) }
}
private class WordDatabaseCallback(
private val scope: CoroutineScope
) : RoomDatabase.Callback() {
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
INSTANCE?.let { database ->
scope.launch {
var wordDao = database.wordDao()
// Delete all content here.
wordDao.deleteAll()
// Add sample words.
var word = Word("Hello")
wordDao.insert(word)
word = Word("World!")
wordDao.insert(word)
// TODO: Add your own words!
word = Word("TODO!")
wordDao.insert(word)
}
}
}
}
this is the code I found, as you can see, it is directly calling scope.launch(...)
my question is that:
isn't all the Room operations supposed to run in non-UI scope? Could someone help me to understand this? thanks so much!
Is CoroutineScope(SupervisorJob()) runs in Main scope?
No. By default CoroutineScope() uses Dispatchers.Default, as can be found in the documentation:
CoroutineScope() uses Dispatchers.Default for its coroutines.
isn't all the Room operations supposed to run in non-UI scope?
I'm not very familiar specifically with Room, but generally speaking it depends if the operation is suspending or blocking. You can run suspend functions from any dispatcher/thread. deleteAll() and insert() functions in the example are marked as suspend, therefore you can run them from both UI and non-UI threads.
Related
I have a requirement, where we want to asynchronously handle some upstream request/payload via coroutine. I see that there are several ways to do this, but wondering which is the right approach -
Provide explicit spring service class that implements CoroutineScope
Autowire singleton scope-context backed by certain defined thread-pool dispatcher.
Define method local CoroutineScope object
Following on this question, I'm wondering whats the trade-off if we define method local scopes like below -
fun testSuspensions(count: Int) {
val launchTime = measureTimeMillis {
val parentJob = CoroutineScope(Dispatchers.IO).launch {
repeat(count) {
this.launch {
process() //Some lone running process
}
}
}
}
}
Alternative approach to autowire explicit scope object backed by custom dispatcher -
#KafkaListener(
topics = ["test_topic"],
concurrency = "1",
containerFactory = "someListenerContainerConfig"
)
private fun testKafkaListener(consumerRecord: ConsumerRecord<String, ByteArray>, ack: Acknowledgment) {
try {
this.coroutineScope.launch {
consumeRecordAsync(consumerRecord)
}
} finally {
ack.acknowledge()
}
}
suspend fun consumeRecordAsync(record: ConsumerRecord<String, ByteArray>) {
println("[${Thread.currentThread().name}] Starting to consume record - ${record.key()}")
val statusCode = initiateIO(record) // Add error-handling depending on kafka topic commit semantics.
// Chain any-other business logic (depending on status-code) as suspending functions.
consumeStatusCode(record.key(), statusCode)
}
suspend fun initiateIO(record: ConsumerRecord<String, ByteArray>): Int {
return withContext(Dispatchers.IO) { // Switch context to IO thread for http.
println("[${Thread.currentThread().name}] Executing network call - ${record.key()}")
delay(1000 * 2) // Simulate IO call
200 // Return status-code
}
}
suspend fun consumeStatusCode(recordKey: String, statusCode: Int) {
delay(1000 * 1) // Simulate work.
println("[${Thread.currentThread().name}] consumed record - $recordKey, status-code - $statusCode")
}
Autowiring bean as follows in some upstream config class -
#Bean(name = ["testScope"])
fun defineExtensionScope(): CoroutineScope {
val threadCount: Int = 4
return CoroutineScope(Executors.newFixedThreadPool(threadCount).asCoroutineDispatcher())
}
It depends on what your goal is. If you just want to avoid the thread-per-request model, you can use Spring's support for suspend functions in controllers instead (by using webflux), and that removes the need from even using an external scope at all:
suspend fun testSuspensions(count: Int) {
val execTime = measureTimeMillis {
coroutineScope {
repeat(count) {
launch {
process() // some long running process
}
}
}
}
// all child coroutines are done at this point
}
If you really want your method to return immediately and schedule coroutines that outlive it, you indeed need that extra scope.
Regarding option 1), making custom classes implement CoroutineScope is not encouraged anymore (as far as I understood). It's usually suggested to use composition instead (declare a scope as a property instead of implementing the interface by your own classes). So I would suggest your option 2.
I would say option 3) is out of the question, because there is no point in using CoroutineScope(Dispatchers.IO).launch { ... }. It's no better than using GlobalScope.launch(Dispatchers.IO) { ... } (it has the same pitfalls) - you can read about the pitfalls of GlobalScope in its documentation.
The main problem being that you run your coroutines outside structured concurrency (your running coroutines are not children of a parent job and may accumulate and hold resources if they are not well behaved and you forget about them). In general it's better to define a scope that is cancelled when you no longer need any of the coroutines that are run by it, so you can clean rogue coroutines.
That said, in some circumstances you do need to run coroutines "forever" (for the whole life of your application). In that case it's ok to use GlobalScope, or a custom application-wide scope if you need to customize things like the thread pool or exception handler. But in any case don't create a scope on the spot just to launch a coroutine without keeping a handle to it.
In your case, it seems you have no clear moment when you wouldn't care about the long running coroutines anymore, so you may be ok with the fact that your coroutines can live forever and are never cancelled. In that case, I would suggest a custom application-wide scope that you would wire in your components.
I have a suspend function that calls POST request to the server. I want to configure some text in the activity to show the information I received from the server.
suspend fun retrieveInfo():String
I tried calling inside onCreate, onResume but crashes runtime.
runBlocking {
retrieveInfo()
}
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.google.augmentedimage/com.google.AugmentedImageActivity}: android.os.NetworkOnMainThreadException
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3086)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3229)
at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
Where am I suppose to put these suspend calls (in which part of lifecycle of activity)? Should I be using something other than runBlocking?
By default runBlocking runs the suspending code block in the thread runBlocking was called on.
So if you called runBlocking from the Activity callback your suspending block will be executed on the main thread, from which you cannot access network (query a server).
You need to switch a dispatcher in your coroutine block for that call. The simplest fix for your code would be to move the execution to the Dispatchers.IO.
runBlocking {
withContext(Dispatchers.IO) {
retrieveInfo()
}
}
That being said, I suggest two things (not related directly to your question):
Read Coroutines on Android (this part and the following ones)
2. Don't use runBlocking for your case, but define a correct job and use job.launch{}
If you want to write in activity:
class MyActivity : AppCompatActivity() {
private val scope = CoroutineScope(newSingleThreadContext("name"))
fun doSomething() {
scope.launch { ... }
}
}
I'm trying to achieve functionality: I have a rest endpoint that calls code that execution can take a lot of time. My idea to improve experience for now is to wrap that piece of code as a new thread, wait for completion or for some max time to elapse and return an appropriate message. Wrapped code should be completed even through endpoint already send message back. Current implementation looks like this:
private const val N = 1000
private const val MAX_WAIT_TIME = 5000
#RestController
#RequestMapping("/long")
class SomeController(
val service: SomeService,
) {
private val executor = Executors.newFixedThreadPool(N)
#PostMapping
fun longEndpoint(#RequestParam("someParam") someParam: Long): ResponseEntity<String> {
val submit = executor.submit {
service.longExecution(someParam)
}
val start = System.currentTimeMillis()
while (System.currentTimeMillis() - start < MAX_WAIT_TIME) {
if (submit.isDone)
return ResponseEntity.ok("Done")
}
return ResponseEntity.ok("Check later")
}
}
First question is - waiting on while for time seems wrong, we don't release thread, can it be improved?
More important question - how to rewrite it to Kotlin coroutines?
My attempt, simple without returning as soon as task is done, looked like this:
#PostMapping
fun longEndpoint(#RequestParam("someParam") someParam: Long): ResponseEntity<String> = runBlocking {
val result = async {
withContext(Dispatchers.Default) {
service.longExecution(someParam)
}
}
delay(MAX_WAIT_TIME)
return#runBlocking ResponseEntity.ok(if(result.isCompleted) "Done" else "Check later")
}
But even through correct string is returned, answer is not send until longExecution is done. How to fix that, what am I missing? Maybe coroutines are bad application here?
There are several problems with your current coroutines attempt:
you are launching your async computation within runBlocking's scope, so the overall endpoint method will wait for child coroutines to finish, despite your attempt at return-ing before that.
delay() will always wait for MAX_WAIT_TIME even if the task is done quicker than that
(optional) you don't have to use runBlocking at all if your framework supports async controller methods (Spring WebFlux does support suspend functions in controllers)
For the first problem, remember that every time you launch a coroutine that should outlive your function, you have to use an external scope. coroutineScope or runBlocking are not appropriate in these cases because they will wait for your child coroutines to finish.
You can use the CoroutineScope() factory function to create a scope, but you need to think about the lifetime of your coroutine and when you want it cancelled. If the longExecution function has a bug and hangs forever, you don't want to leak the coroutines that call it and blow up your memory, so you should cancel those coroutines somehow. That's why you should store the scope as a variable in your class and cancel it when appropriate (when you want to give up on those operations).
For the second problem, using withTimeout is very common, but it doesn't fit your use case because you want the task to keep going even after you timeout waiting for it. One possible solution would be using select clauses to either wait until the job is done, or wait for some specified maximum time:
// TODO call scope.cancel() somewhere appropriate (when this component is not needed anymore)
val scope = CoroutineScope(Job())
#PostMapping
fun longEndpoint(#RequestParam("someParam") someParam: Long): ResponseEntity<String> {
val job = scope.launch {
longExecution()
}
val resultText = runBlocking {
select {
job.onJoin() { "Done" }
onTimeout(MAX_WAIT_TIME) { "Check later" }
}
}
return ResponseEntity.ok(resultText)
}
Note: I'm using launch instead of async because you don't seem to need the return value of longExecution here.
If you want to solve the problem #3 too, you can simply declare your handler suspend and remove runBlocking around the select:
// TODO call scope.cancel() somewhere appropriate (when this component is not needed anymore)
val scope = CoroutineScope(Job())
#PostMapping
suspend fun longEndpoint(#RequestParam("someParam") someParam: Long): ResponseEntity<String> {
val job = scope.launch {
longExecution()
}
val resultText = select {
job.onJoin() { "Done" }
onTimeout(MAX_WAIT_TIME) { "Check later" }
}
return ResponseEntity.ok(resultText)
}
Note that this requires spring-boot-starter-webflux instead of spring-boot-starter-web.
Your implementation always waits for MAX_WAIT_TIME. This might work:
#PostMapping
fun longEndpoint(#RequestParam("someParam") someParam: Long): ResponseEntity<String> = runBlocking {
try {
withTimeout(MAX_WAIT_TIME) {
async {
withContext(Dispatchers.Default) {
service.longExecution(someParam)
}
}
}
} catch (ex: CancellationException) {
return#runBlocking ResponseEntity.ok("Check later")
}
return#runBlocking ResponseEntity.ok("Done")
}
Although I'm not sure if there will be any unwanted side effects because it seems that this will cancel the coroutine when it reaches MAX_WAIT_TIME. Read more about it here:
Cancellation and timeouts
Update 2: I suspect the question gets upvoted because of the possible solution that I describe. Highlighted it for clarity.
Update 1: This question gets a lot of views. If you think the question can be enhanced with the situation in which you encountered the error yourself, please briefly describe your situation in the comments so we can make this Q&A more valuable. And if you have a solution to your version of the problem, please add it as an answer.
I want to update the UI after doing async background work using Task.detached and an async function.
However, I get a build error Reference to captured var 'a' in concurrently-executing code error during build.
I tried some things and turning the variable into a let constant before updating the UI is the only thing that works. Why do I need to make a let constant before being able to update the UI? Are there alternatives?
class ViewModel: ObservableObject {
#Published var something: String?
init() {
Task.detached(priority: .userInitiated) {
await self.doVariousStuff()
}
}
private func doVariousStuff() async {
var a = "a"
let b = await doSomeAsyncStuff()
a.append(b)
something = a /* Not working,
Gives
- runtime warning `Publishing changes from
background threads is not allowed; make sure to
publish values from the main thread (via operators
like receive(on:)) on model updates.`
or, if `something` is #MainActor:
- buildtime error `Property 'something' isolated
to global actor 'MainActor' can not be mutated
from this context`
*/
await MainActor.run {
something = a
} /* Not working,
Gives buildtime error "Reference to captured var 'a'
in concurrently-executing code" error during build
*/
DispatchQueue.main.async {
self.something = a
} /* Not working,
Gives buildtime error "Reference to captured var 'a'
in concurrently-executing code" error during build
*/
/*
This however, works!
*/
let c = a
await MainActor.run {
something = c
}
}
private func doSomeAsyncStuff() async -> String {
return "b"
}
}
Make your observable object as main actor, like
#MainActor // << here !!
class ViewModel: ObservableObject {
#Published var something: String?
init() {
Task.detached(priority: .userInitiated) {
await self.doVariousStuff()
}
}
private func doVariousStuff() async {
var a = "a"
let b = await doSomeAsyncStuff()
a.append(b)
something = a // << now this works !!
}
private func doSomeAsyncStuff() async -> String {
return "b"
}
}
Tested with Xcode 13 / iOS 15
In short, something has to be modified from the main thread and only Sendable types can be passed from one actor to another. Let's dig in the details.
something has to be modified from the main thread. This is because #Published properties in an ObservableObject have to be modified from the main thread. The documentation for this is lacking (if anyone finds a link to the official documentation I'll update this answer). But as the subscriber of an ObservableObject is probably a SwiftUI View, it makes sense. Apple could have decided that a View subscribes and receives events on the main thread, but this would hide the fact that it is dangerous to send UI update events from multiple threads.
Only Sendable types can be passed from one actor to another. There are two ways to solve this. First we can make a Sendable. Second we can make sure not to pass a across actor boundaries and have all code run on the same actor (in this case it has to be the Main Actor as it is guaranteed to run on the main thread).
Let's see how to make a sendable and study the case of:
await MainActor.run {
something = a
}
The code in doVariousStuff() function can run from any actor; let's call it Actor A. a belongs to Actor A and it has to be sent to the Main Actor. As a does not conform to Sendable, the compiler does not see any guarantee that a will not be changed while a is read on the Main Actor. This is not allowed in the Swift concurrency model. To give the compiler that guarantee, a has to be Sendable. One way to do that is to make it constant. Which is why this works:
let c = a
await MainActor.run {
something = c
}
Even if it could be improved to:
await MainActor.run { [a] in
something = a
}
Which captures a as a constant. There are other Sendable types, details can be found here https://docs.swift.org/swift-book/LanguageGuide/Concurrency.html#ID649.
The other way to solve this is to make all code run on the same actor. The easiest way to do that is to mark ViewModel with #MainActor as suggested by Asperi. This will guarantee that doVariousStuff() runs from the Main Actor, so it can set something. As a side note, a then belongs to the Main Actor so (even if it is pointless) await MainActor.run { something = a } would work.
Note that actors are not threads. Actor A can run from any thread. It can start on one thread and then continue on another after any await. It could even run partially on the main thread. What is important is that one actor can only ever run from one thread at a time. The only exception to the rule that any actor can run from any thread is for the Main Actor which only runs on the main thread.
You can use #State and .task as follows:
struct ContentView: View {
#State var result = ""
var body: some View {
HStack {
Text(result)
}
.task {
result = await Something.doSomeAsyncStuff()
}
}
}
The task is started when view appears and is cancelled when it disappears. Also if you use .task(id:) it will restart (also cancelling previous task) when the value of id changes.
The async func can go in a few different places, usually somewhere so it can tested independently.
struct Something {
static func doSomeAsyncStuff() async -> String {
return "b"
}
}
I'm in the process of rewriting one little WPF-App I wrote to make use of ReactiveUI, to get a feeling about the library.
I really like it so far!
Now I've stumbled upon the Throttle method and want to use it when applying a filter to a collection.
This is my ViewModel:
namespace ReactiveUIThrottle
{
public class MainViewModel : ReactiveObject
{
private string _filter;
public string Filter { get => _filter; set => this.RaiseAndSetIfChanged(ref _filter, value); }
private readonly ReactiveList<Person> _persons = new ReactiveList<Person>();
private readonly ObservableAsPropertyHelper<IReactiveDerivedList<Person>> _filteredPersons;
public IReactiveDerivedList<Person> Persons => _filteredPersons.Value;
public MainViewModel()
{
Filter = string.Empty;
_persons.AddRange(new[]
{
new Person("Peter"),
new Person("Jane"),
new Person("Jon"),
new Person("Marc"),
new Person("Heinz")
});
var filterPersonsCommand = ReactiveCommand.CreateFromTask<string, IReactiveDerivedList<Person>>(FilterPersons);
this.WhenAnyValue(x => x.Filter)
// to see the problem
.Throttle(TimeSpan.FromMilliseconds(2000), RxApp.MainThreadScheduler)
.InvokeCommand(filterPersonsCommand);
_filteredPersons = filterPersonsCommand.ToProperty(this, vm => vm.Persons, _persons.CreateDerivedCollection(p => p));
}
private async Task<IReactiveDerivedList<Person>> FilterPersons(string filter)
{
await Task.Delay(500); // Lets say this takes some time
return _persons.CreateDerivedCollection(p => p, p => p.Name.Contains(filter));
}
}
}
The filtering itself works like a charm, also the throttling, when using the GUI.
However, I'd like to unittest the behavior of the filtering and this is my first attempt:
[Test]
public void FilterPersonsByName()
{
var sut = new MainViewModel();
sut.Persons.Should().HaveCount(5);
sut.Filter = "J";
sut.Persons.Should().HaveCount(2);
}
This test fails because the collection still has 5 people.
When I get rid of the await Task.Delay(500) in FilterPersons then the test will pass, but takes 2 seconds (from the throttle).
1) Is there a way to have the throttle be instant within the test to speed up the unittest?
2) How would I test the async behavior in my filter?
I'm using ReactiveUI 7.x
Short answers:
Yes, by making sure you're using CurrentThreadScheduler.Instance when running under test
Instead of using CurrentThreadScheduler, use a TestScheduler and manually advance it
The longer answer is that you need to ensure your unit tests can control the scheduler being used by your System Under Test (SUT). By default, you'll generally want to use CurrentThreadScheduler.Instance to make things happen "instantly" without any need to advance the scheduler manually. But when you want to write tests that do validate timing, you use a TestScheduler instead.
If, as you seem to be, you're using RxApp.*Scheduler, take a look at the With extension method, which can be used like this:
(new TestScheduler()).With(sched => {
// write test logic here, and RxApp.*Scheduler will resolve to the chosen TestScheduler
});
I tend to avoid using the RxApp ambient context altogether for the same reason I avoid all ambient contexts: they're shared state and can cause trouble as a consequence. Instead, I inject an IScheduler (or two) into my SUT as a dependency.