I'm using the pion/webrtc Go library in my project and found this problem that the callback-based API the library provides (which mirrors the JavaScript API of WebRTC) can be awkward to use in Go.
For example, doing the following
conn.OnTrack(func(...) { ... })
conn.OnICEConnectionStateChange(func(...) { ... })
is typical in JavaScript, but in Go, this has a few problems:
This API makes it easy to introduce data race, if the callbacks are called in parallel.
The callback-based API propagates to other part of the codebase and makes everything takes callbacks.
What's the conventional way to handle this situation in Go? I'm new to Go and I read that synchronous API is preferred in Go because Goroutines are cheap. So perhaps one possible design is to use a channel to synchronize the callbacks:
msgChan := make(chan Msg)
// or use a separate channel for each type of event?
conn.OnTrack(func(...) {
msgChan <- onTrackMsg
})
conn.OnICEConnectionStateChange(func(...) {
msgChan <- onStateChangeMsg
})
for {
msg := <-msgChan
// do something depending on the type of msg
}
I think forcing synchronization with channels basically mimics the single-threaded nature of JavaScript.
Anyway, how do people usually model event-driven workflow in Go?
No need for a channel. Just wrap your async/callback code in a single function that waits for a response, and use a WaitGroup (you could use a channel here instead, but a WaitGroup is much easier):
func DoSomething() (someType, error) {
var result SomeType
var err error
wg := sync.WaitGroup{}
wg.Add(1)
StartAsyncProcess(func() {
// This is the call back that gets called eventually
defer wg.Done()
result = /* Set the result */
err = /* and/or set the error */
})
wg.Wait() // Wait until the callback is called, and exits
return result, err // And finally return our values
}
You may need/wish to add additional locks or synchronization in the callback, if necessary in your case, if your callback relies on or modifies shared state.
Related
Assuming that I have a function that sends web requests to an API endpoint, I would like to add a timeout to the client so that if the call is taking too long, the operation breaks either by returning an error or panicing the current thread.
Another assumption is that, the client function (the function that sends web requests) comes from a library and it has been implemented in a synchronous way.
Let's have a look at the client function's signature:
func Send(params map[string]string) (*http.Response, error)
I would like to write a wrapper around this function to add a timeout mechanism. To do that, I can do:
func SendWithTimeout(ctx context.Context, params map[string]string) (*http.Response, error) {
completed := make(chan bool)
go func() {
res, err := Send(params)
_ = res
_ = err
completed <- true
}()
for {
select {
case <-ctx.Done():
{
return nil, errors.New("Cancelled")
}
case <-completed:
{
return nil, nil // just to test how this method works
}
}
}
}
Now when I call the new function and pass a cancellable context, I successfully get a cancellation error, but the goroutine that is running the original Send function keeps on running to the end.
Since, the function makes an API call meaning that establishing socket/TCP connections are actually involved in the background, it is not a good practice to leave a long-running API behind the scene.
Is there any standard way to interrupt the original Send function when the context.Done() is hit?
This is a "poor" design choice to add context support to an existing API / implementation that did not support it earlier. Context support should be added to the existing Send() implementation that uses it / monitors it, renaming it to SendWithTimeout(), and provide a new Send() function that takes no context, and calls SendWithTimeout() with context.TODO() or context.Background().
For example if your Send() function makes an outgoing HTTP call, that may be achieved by using http.NewRequest() followed by Client.Do(). In the new, context-aware version use http.NewRequestWithContext().
If you have a Send() function which you cannot change, then you're "out of luck". The function itself has to support the context or cancellation. You can't abort it from the outside.
See related:
Terminating function execution if a context is cancelled
Is it possible to cancel unfinished goroutines?
Stopping running function using context timeout in Golang
cancel a blocking operation in Go
What is the default mode in which network requests are executed in GoColly? Since we have the Async method in the collector I would assume that the default mode is synchronous.
However, I see no particular difference when I execute these 8 requests in the program other than I need to use Wait for async mode. It seems as if the method only controls how the program is executed (the other code) and the requests are always asynchronous.
package main
import (
"fmt"
"github.com/gocolly/colly/v2"
)
func main() {
urls := []string{
"http://webcode.me",
"https://example.com",
"http://httpbin.org",
"https://www.perl.org",
"https://www.php.net",
"https://www.python.org",
"https://code.visualstudio.com",
"https://clojure.org",
}
c := colly.NewCollector(
colly.Async(true),
)
c.OnHTML("title", func(e *colly.HTMLElement) {
fmt.Println(e.Text)
})
for _, url := range urls {
c.Visit(url)
}
c.Wait()
}
The default collection is synchronous.
The confusing bit is probably the collector option colly.Async() which ignores the actual param. In fact the implementation at the time of writing is:
func Async(a ...bool) CollectorOption {
return func(c *Collector) {
c.Async = true // uh-oh...!
}
}
Based on this issue, it was done this way for backwards compatibility, so that (I believe) you can pass an option with no param at it'll still work, e.g.:
colly.NewCollector(colly.Async()) // no param, async collection
If you remove the async option altogether and instantiate with just colly.NewCollector(), the network requests will be clearly sequential — i.e. you can also remove c.Wait() and the program won't exit right away.
Which ctx should I use in run parameter of hystrix.Do function of hystrix-go package? The ctx from the upper level, or context.Background()?
Thanks.
package main
import(
"context"
"github.com/myteksi/hystrix-go/hystrix"
)
func tb(ctx context.Context)error{
return nil
}
func ta(ctx context.Context){
hystrix.Do("cbName", func()error{
// At this place, the ctx parameter of function tb,
// Should I use ctx from ta function, or context.Background()?
return tb(ctx)
}, nil)
}
func main(){
ta(context.Background())
}
If you're using contexts, it seems to me like you should using hystrix.DoC. There's no reason to use anything than whatever context passed through, since Do is synchronous, and you would like whatever cancellations, deadlines (and whatever else is attached to your context) to be preserved inside this code.
func ta(ctx context.Context) {
err := hystrix.DoC(ctx, "cbName", func(ctx context.Context) error {
... code that uses ctx here.
}, nil)
// handle err, which may be a hystrix error.
}
It's hard to say if this is actually different from calling hystrix.Do, but this potentially allows hystrix to use your context, to add deadlines/cancellations itself.
Always use the context.Context coming from the upper level as a parameter wherever you can. It allows an end to end mechanism to control request, all the caller has to do is cancel, or invoke timeout on the initial ctx, and it will work for the complete request path.
The initial context passed can depend on your requirement. If you're not sure about what context to use initially, context.TODO can be a good option till you're sure.
I reuse the http client connection to make external calls to a single endpoint. An excerpt of the program is shown below:
var AppCon MyApp
func New(user, pass string, platformURL *url.URL, restContext string) (*MyApp, error) {
if AppCon == (MyApp{}) {
AppCon = MyApp{
user: user,
password: pass,
URL: platformURL,
Client: &http.Client{Timeout: 30 * time.Second},
RESTContext: restContext,
}
cj, err := cookiejar.New(nil)
if err != nil {
return &AppCon, err
}
AppCon.cookie = cj
}
return &AppCon, nil
}
// This is an example only. There are many more functions which accept *MyApp as a pointer.
func(ma *MyApp) GetUser(name string) (string, error){
// Return user
}
func main(){
for {
// Get messages from a queue
// The message returned from the queue provide info on which methods to call
// 'm' is a struct with message metadata
c, err := New(m.un, m.pass, m.url)
go func(){
// Do something i.e c.GetUser("123456")
}()
}
}
I now have the requirement to set up a client connections with different endpoints/credentials received via queue messages.
The problem I foresee is I can't just simply modify AppCon with the new endpoint details since a pointer to MyApp is returned, resulting in resetting c. This can impact a goroutine making a HTTP call to an unintended endpoint. To make matters non trivial, the program is not meant to have awareness of the endpoints (I was considering using a switch statement) but rather receive what it needs via queue messages.
Given the issues I've called out are correct, are there any recommendations on how to solve it?
EDIT 1
Based on the feedback provided, I am inclined to believe this will solve my problem:
Remove the use of a Singleton of MyApp
Decouple the http client from MyApp which will enable it for reuse
var httpClient *http.Client
func New(user, pass string, platformURL *url.URL, restContext string) (*MyApp, error) {
AppCon = MyApp{
user: user,
password: pass,
URL: platformURL,
Client: func() *http.Client {
if httpClient == nil {
httpClient = &http.Client{Timeout: 30 * time.Second}
}
return httpClient
}()
RESTContext: restContext,
}
return &AppCon, nil
}
// This is an example only. There are many more functions which accept *MyApp as a pointer.
func(ma *MyApp) GetUser(name string) (string, error){
// Return user
}
func main(){
for {
// Get messages from a queue
// The message returned from the queue provide info on which methods to call
// 'm' is a struct with message metadata
c, err := New(m.un, m.pass, m.url)
// Must pass a reference
go func(c *MyApp){
// Do something i.e c.GetUser("123456")
}(c)
}
}
Disclaimer: this is not a direct answer to your question but rather an attempt to direct you to a proper way of solving your problem.
Try to avoid a singleton pattern for you MyApp. In addition, New is misleading, it doesn't actually create a new object every time. Instead you could be creating a new instance every time, while preserving the http client connection.
Don't use constructions like this: AppCon == (MyApp{}), one day you will shoot in your leg doing this. Use instead a pointer and compare it to nil.
Avoid race conditions. In your code you start a goroutine and immediately proceed to the new iteration of the for loop. Considering you re-use the whole MyApp instance, you essentially introduce a race condition.
Using cookies, you make your connection kinda stateful, but your task seems to require stateless connections. There might be something wrong in such an approach.
I'm learning go and ran into some design issues while developing web app. The app has main route "/" where user can submit a simple form. With those form values I am calling external API and unmarshaling response into some struct. Now from here I want to make another call based on retrieved values to another external API and I'm not sure what's the proper way of doing this. Here is a snippet for better understandment:
func main() {
http.HandleFunc("/", mainHandler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func mainHandler(w http.ResponseWriter, r *http.Request) {
//renders form template
//makes post and retrieves data from api
//here with retrieved data I want to make another call to different API,
// but mainHandler would get too big and complex. I'm not sure how should I pass this data to
// another handler or redirect to another handler with this data.
}
The handlers' semantics should be designed to match the desired HTTP behavior, regardless of the code complexity. If you want to handle a single client request by doing a bunch of stuff, that should be a single handler. If the handler becomes too complex, break it up. Handlers are just functions and can be broken up exactly like any other function - by extracting some part of it into another function and calling that new function. To take you pseudocode:
func mainHandler(w http.ResponseWriter, r *http.Request) {
err := renderTemplate(w)
if err != nil { ... }
err, data := postToApi()
if err != nil { ... }
err, data2 := postToApi2(data)
if err != nil { ... }
}
There's no reason for those functions to be handlers themselves or to get the client involved with a redirect. Just break up your logic the way you normally break up logic - it doesn't matter that it's an HTTP handler.
Hi golearner, in the mainHandler just render the form and make another handler kinda "/formaction" to handle the form, in that way you can easily organize your code.