I have a golang function which recursively steps through a json string and replaces custom references with the json document they are referencing. I just noticed that I forgot to handle cyclic references, whose occurrence will lead to endless recursion.
Before I fix this, I'd like to write a test for this case (using ginkgo/gomega), so I can verify that my solution works and I will notice if I ever break it and run into this problem again.
But how do I do something like if this function call does not return within <timeout>, abort it and fail the test?
Gomega's Eventually has a timeout, but it doesn't abort the function if it is already running, so it will block forever in this case.
I found this example for how to check for a timeout using select and channels, but from what I understood, it is not possible to terminate a goroutine from outside - so my function will continue to run in the background, eating up resources?
What is the best way to check for infinite recursion?
You can't abort a running function. The function has to support abortion, idiomatically through a context.Context or a channel. If you want to support timeout or abortion, you have to change / refactor your function. And the function itself has to support this, e.g. it has to monitor the context.Context and return early if cancellation was requested. For details and example, see Terminating function execution if a context is cancelled
See related:
cancel a blocking operation in Go
Cancelling user specific goroutines
Related
So I a attempting to shut down a long running function if something takes too long, maybe is just a solution to treating the symptoms rather than cause, but in any case for my situation it didn't really worked out.
I did it like this:
func foo(abort <- chan struct{}) {
for {
select{
case <-abort:
return
default:
///long running code
}
}
}
And in separate function I have which after some time closes the passed chain, which it does, if I cut the body returns the function. However if there is some long running code, it does not affect the outcome it simply continues the work as if nothing has happened.
I am pretty new to GO, but it feels like it should work, but it does not. Is there anything I am missing. After all routers frameworks have timeout function, after which whatever is running is terminated. So maybe this is just out of curiosity, but I would really want how to od it.
your code only checks whether the channel was closed once per iteration, before executing the long running code. There's no opportunity to check the abort chan after the long running code starts, so it will run to completion.
You need to occasionally check whether to exit early in the body of the long running code, and this is more idiomatically accomplished using context.Context and WithTimeout for example: https://pkg.go.dev/context#example-WithTimeout
In your "long running code" you have to periodically check that abort channel.
The usual approach to implement that "periodically" is to split the code into chunks each of which completes in a reasonably short time frame (given that the system the process runs on is not overloaded).
After executing each such chunk you check whether the termination condition holds and then terminate execution if it is.
The idiomatic approach to perform such a check is "select with default":
select {
case <-channel:
// terminate processing
default:
}
Here, the default no-op branch is immediately taken if channel is not ready to be received from (or closed).
Some alogrithms make such chunking easier because they employ a loop where each iteration takes roughly the same time to execute.
If your algorithm is not like this, you'd have to chunk it manually; in this case, it's best to create a separate function (or a method) for each chunk.
Further points.
Consider using contexts: they provide a useful framework to solve the style of problems like the one you're solving.
What's better, the fact they can "inherit" one another allow one to easily implement two neat things:
You can combine various ways to cancel contexts: say, it's possible to create a context which is cancelled either when some timeout passes or explicitly by some other code.
They make it possible to create "cancellation trees" — when cancelling the root context propagates this signal to all the inheriting contexts — making them cancel what other goroutines are doing.
Sometimes, when people say "long-running code" they do not mean code actually crunching numbers on a CPU all that time, but rather the code which performs requests to slow entities — such as databases, HTTP servers etc, — in which case the code is not actually running but sleeping on the I/O to deliver some data to be processed.
If this is your case, note that all well-written Go packages (of course, this includes all the packages of the Go standard library which deal with networked services) accept contexts in those functions of their APIs which actually make calls to such slow entities, and this means that if you make your function to accept a context, you can (actually should) pass this context down the stack of calls where applicable — so that all the code you call can be cancelled in the same way as yours.
Further reading:
https://go.dev/blog/pipelines
https://blog.golang.org/advanced-go-concurrency-patterns
For example, a defer function is implemented within a webhook's logic (the webhook is written in Golang). The webhook times out after taking too long to complete. In this case, will the defer function still get executed?
The request timeout is set via context.WithTimeout for the webhook.
It's usually a good idea to clearly mention what programming language you're talking about, although I can figure it out.
In Go, the defer functor (whatever you want to call it) will be executed when the method returns, no matter how it completes, unless you manage to kill the executing process in a drastic manner, and perhaps even then.
I want to access Redis and do a GET on two machines simultaneously. I'll use the first result I get and cancel the other call.
What I'm wondering is whether cancelFunc() breaks a go function immediately or whether it will wait for a specific event to happen before it notices the signal. The documentation just says:
Calling the CancelFunc cancels the child and its children, removes the parent's reference to the child, and stops any associated timers.
That doesn't say whether it happens immediately or whether a special cancellation point needs to be reached.
Any documentation I missed about that particular point?
Canceling a context simply closes its Done channel, it doesn't "kill" a goroutine or anything like that. Code that executes under the context has to do a select to notice the closure of the Done channel and abandon whatever work it's doing.
However in many cases this work is already done for you. If the redis client method you're using to make requests accepts a context, then it's safe to assume that if that context is canceled, the client will handle the cancellation and the method will immediately return an error of context.Canceled — all you need to do is handle that error by quitting what you're doing.
I'm using gin-gonic as HTTP handler. I want to prerender some graphical resources after my users make POST request. For this, I put a middleware that assign a function (with a timer inside) to a map[string]func() and call this function directly after assignation.
The problem is, when the user make two subsequent request, the function is called twice.
Is there any way to clear function ref and/or his currently running call like a clearInterval or clearTimeout in Javascript ?
Thanks
No; whatever function you've scheduled to run as a goroutine needs to either return or call runtime.Goexit.
If you're looking for a way to build cancellation into your worker, Go provides a primitive to handle that, which is already part of any HTTP request - contexts. Check out these articles from the Go blog:
Concurrency Patterns: Context
Pipelines and cancellation
I suppose your rendering function is calling into a library, so you don't have control over the code where the bulk of the time is spent. If you do have such control, just pass a channel into the goroutine, periodically check if the channel is closed, and just return from the goroutine if that happens.
But actually I would recommend a different, and simpler, solution: keep track (in a map) of the file names (or hashes) of the files that are currently being processed, and check that map before launching a second one.
I'm trying to figure out how to handle events using coroutines (in Lua). I see that a common way of doing it seems to be creating wrapper functions that yield the current coroutine and then resume it when the thing you're waiting for has occured. That seems like a nice solution, but what about these problems? :
How do you wait for multiple events at the same time, and branch depending on which one comes first? Or should the program be redesigned to avoid such situations?
How to cancel the waiting after a certain period? The event loop can have timeout parameters in its socket send/receive wrappers, but what about custom events?
How do you trigger the coroutine to change its state from outside? For example, I would want a function that when called, would cause the coroutine to jump to a different step, or start waiting for a different event.
EDIT:
Currently I have a system where I register a coroutine with an event, and the coroutine gets resumed with the event name and info as parameters every time the event occurs. With this system, 1 and 2 are not issues, and 3 can solved by having the coro expect a special event name that makes it jump to the different step, and resuming it with that name as an arg. Also custom objects can have methods to register event handlers the same way.
I just wonder if this is considered the right way to use coroutines for event handling. For example, if I have a read event and a timer event (as a timeout for the read), and the read event happens first, I have to manually cancel the timer. It just doesn't seem to fit the sequential nature or handling events with coroutines.
How do you wait for multiple events at the same time, and branch depending on which one comes first?
If you need to use coroutines for this, rather than just a Lua function that you register (for example, if you have a function that does stuff, waits for an event, then does more stuff), then this is pretty simple. coroutine.yield will return all of the values passed to coroutine.resume when the coroutine is resumed.
So just pass the event, and let the script decide for itself if that's the one it's waiting for or not. Indeed, you could build a simple function to do this:
function WaitForEvents(...)
local events = {...}
assert(#... ~= 0, "You must pass at least one parameter")
do
RegisterForAnyEvent(coroutine.running()) --Registers the coroutine with the system, so that it will be resumed when an event is fired.
local event = coroutine.yield()
for i, testEvt in ipairs(events) do
if(event == testEvt) then
return
end
end
until(false)
end
This function will continue to yield until one of the events it is given has been fired. The loop assumes that RegisterForAnyEvent is temporary, registering the function for just one event, so you need to re-register every time an event is fired.
How to cancel the waiting after a certain period?
Put a counter in the above loop, and leave after a certain period of time. I'll leave that as an exercise for the reader; it all depends on how your application measures time.
How do you trigger the coroutine to change its state from outside?
You cannot magic a Lua function into a different "state". You can only call functions and have them return results. So if you want to skip around within some process, you must write your Lua function system to be able to be skippable.
How you do that is up to you. You could have each set of non-waiting commands be a separate Lua function. Or you could just design your wait states to be able to skip ahead. Or whatever.