i'd like to make a daemon in Vala which only executes a task every X seconds.
I was wondering which would be the best way:
Thread.usleep() or Posix.sleep()
GLib.MainLoop + GLib.Timeout
other?
I don't want it to eat too many resources when it's doing nothing..
If you spend your time sleeping in a system call, there's won't be any appreciable difference from a performance perspective. That said, it probably makes sense to use the MainLoop approach for two reasons:
You're going to need to setup signal handlers so that your daemon can die instantaneously when it is given SIGTERM. If you call quit on your main loop by binding SIGTERM via Posix.signal, that's probably going to be a more readable piece of code than checking that the sleep was successful.
If you ever decide to add complexity, the MainLoop will make it more straight forward.
You can use GLib.Timeout.add_seconds the following way:
Timeout.add_seconds (5000, () => {
/* Do what you want here */
// Continue this "loop" every 5000 ms
return Source.CONTINUE;
// Or remove it
return Source.REMOVE;
}, Priority.LOW);
Note: The Timeout is set as Priority.LOW as it runs in background and should give priority to others tasks.
Related
I've been attempting to do a bit of performance review on an app I have, it's a back end Kotlin app that just pulls in some data, does a bit of data transformation and dumps it out, nothing too fancy. One thing that caught my eye was the final bit of execution where we dump our final data onto a queue, at first I noticed when we start up the app the final network call takes a very long time at first, sometimes over a second. Normally we run this network call in a coroutine to stop that last call blocking everything but I started trying to time the coroutine and the network call separately and got some odd results, from what I can see the coroutine takes can take forever to launch/complete compared to the network call. It's entirely possible I'm not recording things correctly but this is the general timing approach I have:
val coroutineTime - Instant.now().toEpochMillis()
GlobalScope.launch {
executionTime = measureTimeMillis { <--DO Message Sending -->}
totalTime = Instant.now().toEpochMillis() - coroutineTime
// Log out execution Time and total time
}
Now here what I'll see is something like
- totalTime = ~800ms
- executionTime = ~150ms
These aren't one-offs either, I have multiple of these processes going on at once ( up to 10 threads I think) and the first total times will always take significantly longer than the actual executionTime/network call. Eventually after a new dozen messages the overhead will calm down and these times will become equivalent at about 15ms, but having nearly 700ms overhead on coroutine start up seems insane to me.
Is this normal/expected behavior? I've tested this in a separate app and see similar but less extreme results where the first coroutine will take about 70ms to boot up, I'm struggling to find any other examples of this type of discussion outside of kotlin being used in android development.
As a first note, it's almost never a good idea to use the GlobalScope unless you really know what you're doing. This is why it was marked as delicate API. You should instead use a scope that is appropriately closed (following the lifecycle of whatever component launches this work).
Now, AFAIK, this GlobalScope runs on the default dispatcher, so maybe this is due to a cold start of that default thread pool. Later, it could also be a problem to use this dispatcher for network calls depending on the amount of concurrent coroutines you have. It would be more appropriate to use Disptachers.IO instead for IO bound work (or a custom thread pool).
It still doesn't explain the cold start, but I would first change that before investigating.
This is expected behavior if you use coroutines inappropriately ;-)
My guess is that your message sending is a blocking operation. By default GlobalScope.launch() dispatches coroutines with Dispatchers.Default which is designed to perform CPU-intensive operations, it has a limited number of threads and you should never block when using it. If you do you may run out of threads and coroutines will need to wait until some blocking operations will finish.
If you need to run blocking or IO code, you should use Dispatchers.IO instead:
GlobalScope.launch(Dispatchers.IO) {
I was facing similar issue, I have a function that loads some data from shared prefs, makes some calculations on the data (all this done in Dispatcher.Default), and return the result on Dispatcher.Main. I measured how long it took the Coroutine to actually start executing the block inside Dispatchers.Main.launch { } after calculations are done(time from tag2 to tag3 below), and got about 950ms (!!) Here is the function :
fun someName() {
CoroutineScope(Dispatchers.Default).launch {
val time = System.currentTimeMillis()
//load data and calculations
Log.d("tag2", "load and calculations took " + (System.currentTimeMillis() - time))
CoroutineScope(Dispatchers.Main.immediate).launch {
Log.d("tag3", "reached main thread code " + (System.currentTimeMillis() - time))
//do something
Log.d("tag4", "do something took " + (System.currentTimeMillis() - time))
}
}
}
But then I realized this happens while app launch, and main thread is busy creating all the UI, so even with .immediate it takes time until main thread will get to execute the dispatched code... then I tried to run this function after app already started and waiting, and found that from tag2 to tag 3 takes about 1ms (!!) (with .immediate). So looks like when dispatching something on Coroutine, when thread isn't busy it will start immediately
The following simple Promise is vowed and I am not allowed to break it.
my $my_promise = start {
loop {} # or sleep x;
'promise response'
}
say 'status : ', $my_promise.status; # status : Planned
$my_promise.break('promise broke'); # Access denied to keep/break this Promise; already vowed
# in block <unit> at xxx line xxx
Why is that?
Because the Promise is vowed, you cannot change it: only something that actually has the vow, can break the Promise. That is the intent of the vow functionality.
What are you trying to achieve by breaking the promise as you showed? Is it to stop the work being done inside of the start block? Breaking the Promise would not do that. And the vow mechanism was explicitly added to prevent you from thinking it can somehow stop the work inside a start block.
If you want work inside a start block to be interruptible, you will need to add some kind of semaphore that is regularly checked, for instance:
my int $running = 1;
my $my_promise = start {
while $running {
# do stuff
}
$running
}
# do other stuff
$running = 0;
await $my_promise;
Hope this made sense.
The reason why you cannot directly keep/break Promise from outside or stop it on Thread Pool are explained here in Jonathans comment.
Common misuse of Promises comes from timeout pattern.
await Promise.anyof(
start { sleep 4; say "finished"; },
Promise.in( 1 )
);
say "moving on...";
sleep;
This will print "finished". And when user realize that the next logical step for him is to try to kill obsolete Promise. While the only correct way to solve it is to make Promise aware that its work is no longer needed. For example through periodically checking some shared variable.
Things gets complicated if you have blocking code on Promise (for example database query) that runs for too long and you want to terminate it from main thread. That is not doable on Promises. All you can do is to ensure Promise will run in finite time (for example on MySQL by setting MAX_EXECUTION_TIME before running query). And then you have choice:
You can grind your teeth and patiently wait for Promise to finish. For example if you really must disconnect database in main thread.
Or you can move on immediately and allow "abandoned" Promise to finish on its own, without ever receiving its result. In this case you should control how many of those Promises can stack up in background by using Semaphore or running them on dedicated ThreadPoolScheduler.
I need to perform data analysis on files in a directory as they come in.
I'd like to know, if it is better,
to implement an event listener on the directory, and start the analysis process when activated. Then having the program go into sleep forever: while(true), sleep(1e10), end
or to have a loop polling for changes and reacting.
I personally prefer the listeners way, as one is able to start the analysis twice on two new files coming in NEARLY the same time but resulting in two events. While the other solution might just handle the first one and after that finds the second new data.
Additional idea for option 1: Hiding the matlab GUI by calling frames=java.awt.Frame.getFrames and setting frames(index).setVisible(0) on the index matching the com.mathworks.mde.desk.MLMainFrame-frame. (This idea is taken from Yair Altman)
Are there other ways to realize such things?
In this case, (if you are using Windows), the best way is to use the power of .NET.
fileObj = System.IO.FileSystemWatcher('c:\work\temp');
fileObj.Filter = '*.txt';
fileObj.EnableRaisingEvents = true;
addlistener(fileObj,'Changed',#eventhandlerChanged);
There are different event types, you can use the same callback for them, or different ones:
addlistener(fileObj, 'Changed', #eventhandlerChanged );
addlistener(fileObj, 'Deleted', #eventhandlerChanged );
addlistener(fileObj, 'Created', #eventhandlerChanged );
addlistener(fileObj, 'Renamed', #eventhandlerChanged );
Where eventhandlerChanged is your callback function.
function eventhandlerChanged(source,arg)
disp('TXT file changed')
end
There is no need to use sleep or polling. If your program is UI based, then there is nothing else to do, when the user closes the figure, the program has ended. The event callbacks are executed exactly like button clicks. If your program is script-like, you can use an infinite loop.
More info in here: http://www.mathworks.com/help/matlab/matlab_external/working-with-net-events-in-matlab.html
I have two threads of execution like,
Routine 1 {
// do something
}
Routine 2 {
// do something
}
Is it possible to pause execution of routine 2 from routine 1 for few seconds and how can it possible ?
It is not possible to control the execution of one goroutine from another. Goroutines are cooperative. They don't dominate each other.
What you could do is put points in routine 2 where it checks whether it's allowed to proceed. Such as
// do stuff
select {
case <-wait:
<-resume
default:
}
Then routine 1 could tell routine 1 could send a signal to routine 2 telling it to wait:
wait <- true
// whatever stuff goes here
resume <- true
Why do you want to pause the goroutine? That might help answer your question better. It is best to start from a place of what you are trying to do rather than how you want to do it. That way, you can find out how to achieve what you actually want in the language, rather than being given poor substitutes for the method of achieving it that you'd originally imagined.
From one thread, it is not possible to control another thread implicitly. You can do like this, define a bool and based on that you can pause by time.Sleep(2*1e9).
Ryan Tomayko touched off quite a fire storm with this post about using Unix process control commands.
We should be doing more of this. A lot more of this. I'm talking about fork(2), execve(2), pipe(2), socketpair(2), select(2), kill(2), sigaction(2), and so on and so forth. These are our friends. They want so badly just to help us.
I have a bit of code (a delayed_job clone for DataMapper that I think would fit right in with this, but I'm not clear on how to take advantage of the listed commands. Any Ideas on how to improve this code?
def start
say "*** Starting job worker #{#name}"
t = Thread.new do
loop do
delay = Update.work_off(self)
break if $exit
sleep delay
break if $exit
end
clear_locks
end
trap('TERM') { terminate_with t }
trap('INT') { terminate_with t }
trap('USR1') do
say "Wakeup Signal Caught"
t.run
end
end
Ahh yes... the dangers of "We should do more of this" without explaining what each of those do and in what circumstances you'd use them. For something like delayed_job you may even be using fork without knowing that you're using fork. That said, it really doesn't matter. Ryan was talking about using fork for preforking servers. delayed_job would use fork for turning a process into a daemon. Same system call, different purposes. Running delayed_job in the foreground (without fork) vs in the background (with fork) will result in a negligible performance difference.
However, if you write a server that accepts concurrent connections, now Ryan's advice is right on the money.
fork: creates a copy of the original process
execve: stops executing the current file and begins executing a new file in the same process (very useful in rake tasks)
pipe: creates a pipe (two file descriptors, one for read, one for write)
socketpair: like a pipe, but for sockets
select: let's you wait for one or more of multiple file descriptors to be ready with a timeout
kill: used to send a signal to a process
sigaction: lets you change what happens when a process receives a signal
5 months later, you can view my solution at http://github.com/antarestrader/Updater. Look at lib/updater/fork_worker.rb