I'm running a websocket client and want to pass the response(s) from the client to a channel that i can work with in my main file. Currently, the channel just returns a nil value once and then nothing else. I seem to have an issue when passing a value to the channel. Any help? Here is what I've done so far
package main
import (
"context"
"fmt"
"kraken_client/stored_data"
"kraken_client/ws_client"
"os"
"os/signal"
"strings"
"sync"
"syscall"
)
func main() {
// check if in production or testing mode & find base curency
var testing bool = true
args := os.Args
isTesting(args, &testing, &stored_data.Base_currency)
// go routine handler
comms := make(chan os.Signal, 1)
signal.Notify(comms, os.Interrupt, syscall.SIGTERM)
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
var wg sync.WaitGroup
// set ohlc interval and pairs
OHLCinterval := 5
pairs := []string{"BTC/" + stored_data.Base_currency, "EOS/" + stored_data.Base_currency}
// create ws connections
pubSocket, err := ws_client.ConnectToServer("public", testing)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// listen to websocket connections
ch := make(chan interface{})
wg.Add(1)
go pubSocket.PubListen(ctx, &wg, ch, testing)
// subscribe to a stream
pubSocket.SubscribeToOHLC(pairs, OHLCinterval)
go func() {
for c := range ch {
fmt.Println(c)
}
}()
<-comms
cancel()
wg.Wait()
defer close(ch)
}
Here is how the PubListen function works
func (socket *Socket) PubListen(ctx context.Context, wg *sync.WaitGroup, ch chan interface{}, testing bool) {
defer wg.Done()
defer socket.Close()
var res interface{}
socket.OnTextMessage = func(message string, socket Socket) {
//log.Println(message)
res = pubJsonDecoder(message, testing) // this function decodes the message and returns an interface
log.Println(res) // this is printing the correctly decoded value.
}
ch <- res
log.Println(res) // does not print a value
log.Println(ch) // does not print a value
<-ctx.Done()
log.Println("closing public socket")
return
}
What am I doing wrong?
The code in the question executes the statement ch <- res once from PubListen before the res is set by the OnTextMessage function.
To send a value to ch on each message, move the line ch <- res to the OnTextMessage function. That function is called once for each message.
func (socket *Socket) PubListen(ctx context.Context, wg *sync.WaitGroup, ch chan interface{}, testing bool) {
defer wg.Done()
defer socket.Close()
socket.OnTextMessage = func(message string, socket Socket) {
res := pubJsonDecoder(message, testing)
ch <- res
log.Println(res)
}
<-ctx.Done()
log.Println("closing public socket")
return
}
Related
I've been trying to look through existing posts about closing a channel, but I can't seem to find exactly what I'm looking for. I have two channels inside of a struct. After making the channels, I tried to run defer close(channelName) immediately after, but the channels immediately closed and I got an error. If I don't deal with closing the channels, the code runs fine, but doesn't shut down gracefully and I have to use ctrl z to suspend the program, since it's still running after using ctrl c. Here are the important parts of the code:
type WebsocketClient struct {
pubSocket ws_client.Socket
privSocket ws_client.Socket
pubChan chan interface{}
privChan chan interface{}
}
type KrakenClient struct {
WebSocket WebsocketClient
Testing bool
}
func (client *KrakenClient) initChannels() {
client.WebSocket.pubChan = make(chan interface{})
client.WebSocket.privChan = make(chan interface{})
//defer close(client.WebSocket.pubChan)
//defer close(client.WebSocket.privChan)
}
func (client *KrakenClient) InitWebSocketClient(wg *sync.WaitGroup, testing bool) {
client.initTesting(testing)
client.initChannels()
client.startWebSocketConnection(wg)
}
func (client *KrakenClient) PubDecoder(wg *sync.WaitGroup, ctx context.Context) {
wg.Add(1)
defer wg.Done()
defer client.WebSocket.pubSocket.Close()
if err := PubSocketGuard(client.WebSocket); err != nil { // guard clause checker. makes sure i'm actually using a public WebSocket
panic(err)
}
var res interface{}
ws_client.ReceiveLocker(&client.WebSocket.pubSocket)
client.WebSocket.pubSocket.OnTextMessage = func(message string, socket ws_client.Socket) {
res = ws_client.PubJsonDecoder(message, client.Testing)
client.WebSocket.pubChan <- res
}
ws_client.ReceiveUnlocker(&client.WebSocket.pubSocket)
<-ctx.Done()
log.Println("closing public socket")
return
}
func (client *KrakenClient) PubListen(wg *sync.WaitGroup, ctx context.Context, ohlcMap *types.OHLCVals) {
wg.Add(1)
defer wg.Done()
for c := range client.WebSocket.pubChan {
switch v := c.(type) {
// More code here. Not important to the channel problem
}
<-ctx.Done()
}
func main() {
var testing bool = true
comms := make(chan os.Signal, 1)
signal.Notify(comms, os.Interrupt, syscall.SIGTERM)
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
var wg sync.WaitGroup
kraken := &kraken_client.KrakenClient{}
kraken.InitWebSocketClient(&wg, testing)
go kraken.PubDecoder(&wg, ctx)
kraken.SubscribeToOHLC(&wg, []string{"BTC/USD"}, 5)
ohlcMap := types.OHLCVals{}
go kraken.PubListen(&wg, ctx, &ohlcMap)
<-comms
cancel()
wg.Wait()
}
In summary, struct KrakenClient has a type WebsocketClient, which holds 2 WebSockets and 2 channels. Calling the InitWebSocketClient func on the KrakenClient struct creates the two channels and connects to the WS server. Once connected to the server, I start to unmarshal all responses and send those responses to a channel. I then subscribe to a specific endpoint, and start to "listen" to the responses (actually go through the data and add it to the ohlcMap variable that's passed to the listen func depending on certain circumstances). I just don't understand where I should be closing the channels. Do I need to create a Close function on my KrakenClient struct that defers when the channels are closed? If so, where would it even go? Appreciate any help!
You must call defer in main func because when you call defer in initChannels func it's immediately close your channels after making.
The defer always execute when the function (initChannels in your code) has been return.
you can write a closing function for close the channels and call it on main func like as:
func (client *KrakenClient) closeChannels() {
close(client.WebSocket.pubChan)
close(client.WebSocket.privChan)
}
func (client *KrakenClient) initChannels() {
client.WebSocket.pubChan = make(chan interface{})
client.WebSocket.privChan = make(chan interface{})
}
func main() {
...
kraken.InitWebSocketClient(&wg, testing)
defer kraken.closeChannels()
...
}
I try to send an error in the channel on recovery
Why this error is not sent to the channel?
package main
import (
"fmt"
"sync"
"errors"
)
func main() {
var wg sync.WaitGroup
wg.Add(1)
batchErrChan := make(chan error)
go func(errchan chan error) {
defer func() {
if r := recover(); r != nil {
errchan <- errors.New("recover err")
}
close(batchErrChan)
wg.Done()
}()
panic("ddd")
}(batchErrChan)
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
}()
wg.Wait()
}
https://play.golang.org/p/0ytunuYDWZU
I expect "err in range" to be printed, but it is not. Why?
Your program ends before the goroutine gets a chance to print the message. Try waiting to it:
...
done:=make(chan struct{})
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
close(done)
}()
wg.Wait()
<-done
}
I have a go function processing that use two distinct goroutines. produce will push some data into a channel and consume will read these data. Here is an example:
type MyObject struct{
...
}
func processing() {
var wg sync.WaitGroup
dataChannel := make(chan MyObject, 5)
wg.Add(2)
go produce(wg, dataChannel)
go consume(wg, dataChannel)
wg.Wait()
}
func produce (wg *sync.WaitGroup, dataChannel chan MyObject){
for{
// Produce data in dataChannel
}
}
func consume (wg *sync.WaitGroup, dataChannel chan MyObject){
for{
// Consume data from dataChannel
}
}
I want my processing function to be started and stoped by an HTTP call. So I am looking to do something as follow:
func main() {
// echo instance
e := echo.New()
e.GET("/", startProcessing)
e.Logger.Fatal(e.Start(":8099"))
}
func startProcessing(c echo.Context) error{
command := c.QueryParam("command")
if(command == "start"){
processing()
}else if(command == "stop"){
if (/* ? processing is running ? */){
/* ? stop processing process? */
}
}
}
What is the correct way to do this with Go?
Here how to start and stop a function using context, try this:
package main
import (
"context"
"fmt"
"sync"
"time"
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
dataChannel := make(chan MyObject, 5)
wg.Add(2)
go produce(ctx, &wg, dataChannel)
go consume(&wg, dataChannel)
time.Sleep(1 * time.Second)
cancel() // cancel when we are finished consuming data
wg.Wait()
}
func produce(ctx context.Context, wg *sync.WaitGroup, dataChannel chan MyObject) {
defer wg.Done()
i := 1
for {
select {
case <-ctx.Done():
close(dataChannel)
return // returning not to leak the goroutine
case dataChannel <- MyObject{i}:
i++
time.Sleep(250 * time.Millisecond)
}
}
}
func consume(wg *sync.WaitGroup, dataChannel chan MyObject) {
defer wg.Done()
for v := range dataChannel {
fmt.Println(v)
}
}
type MyObject struct {
i int
}
For HTTP you need to do it yourself!
It needs to have some concurrent safe ID or map or something to keep track of how many functions you called and then call a cancel() to stop it.
How to I go about implementing the aggregation pattern in Go, I have to send a bunch of http request concurrently where each go routine will call the endpoint and send the response status on a channel. Now on the main calling function I will range through the channel and display all the responses.
The problem is how do I unblock the channel ?? - I cannot close the channel from the go routines as it will be closed before the complete work is done
package main
import (
"fmt"
"net/http"
"sync"
"time"
"golang.org/x/net/context"
)
func main() {
var wg sync.WaitGroup
wg.Add(10)
c := make(chan string, 100)
ctx := context.Background()
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
for i := 1; i <= 10; i++ {
go SendHttpRequest(ctx, c, &wg)
}
for v := range c {
fmt.Println(v)
}
wg.Wait()
}
func SendHttpRequest(ctx context.Context, c chan string, wg *sync.WaitGroup) {
//defer wg.Done()
client := http.Client{}
req, err := http.NewRequest("POST", "https://jsonplaceholder.typicode.com/posts/1", nil)
if err != nil {
panic(err)
}
req.WithContext(ctx)
res, _ := client.Do(req)
select {
case <-time.After(1 * time.Microsecond):
c <- res.Status
case <-ctx.Done():
c <- "599 ToLong"
}
if res != nil {
defer res.Body.Close()
}
//close(c)
defer wg.Done()
}
Use the WaitGroup
go func(){
wg.Wait()
close(c)
}()
for v := range c {
fmt.Println(v)
}
// Don't bother with wg.Wait() here
In this kind of situation use a generator and idiomatic early defer patterns:
import (
"fmt"
"errors"
"net/http"
"sync"
"time"
"golang.org/x/net/context"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel() // defer early context cancel
for v := range requests(ctx) {
fmt.Println(v)
}
}
// requests generator (handling synchro)
func requests(ctx context.Context)<-chan string {
c := make(chan string/*, 100*/) // No need for buffer, do it on the fly
go func(){
defer close(c) // defer early chan close, will also check goroutine ending
var wg sync.WaitGroup
defer wg.Wait() // defer early wait
wg.Add(10)
for i := 1; i <= 10; i++ {
go func() {
defer wg.Done() // defer early goroutine waitgroup done
if status, err := SendHttpRequest(ctx, c); err != nil {
c <- status
}
}()
}
}
return c
}
// SendHttpRequest looks more conventional, no goroutines, no syncro (waitgroup not spread)
func SendHttpRequest(ctx context.Context) (status string, err error) {
client := http.Client{}
req, err := http.NewRequest("POST", "https://jsonplaceholder.typicode.com/posts/1", nil)
if err != nil {
return
}
req.WithContext(ctx)
res, err := client.Do(req)
if err != nil {
if errors.Is(err, context.Canceled) { // check that request was not cancelled by context cancel trigger
status = "599 ToLong"
}
return
}
defer res.Body.Close() // defer early response body close (in case of no error)
status = res.Status
return
}
I have the following golang program;
package main
import (
"fmt"
"net/http"
"time"
)
var urls = []string{
"http://www.google.com/",
"http://golang.org/",
"http://yahoo.com/",
}
type HttpResponse struct {
url string
response *http.Response
err error
status string
}
func asyncHttpGets(url string, ch chan *HttpResponse) {
client := http.Client{}
if url == "http://www.google.com/" {
time.Sleep(500 * time.Millisecond) //google is down
}
fmt.Printf("Fetching %s \n", url)
resp, err := client.Get(url)
u := &HttpResponse{url, resp, err, "fetched"}
ch <- u
fmt.Println("sent to chan")
}
func main() {
fmt.Println("start")
ch := make(chan *HttpResponse, len(urls))
for _, url := range urls {
go asyncHttpGets(url, ch)
}
for i := range ch {
fmt.Println(i)
}
fmt.Println("Im done")
}
Run it on Playground
However when I run it; it hangs (ie the last part that ought to print Im done doesnt run.)
Here's the terminal output;;
$ go run get.go
start
Fetching http://yahoo.com/
Fetching http://golang.org/
Fetching http://www.google.com/
sent to chan
&{http://www.google.com/ 0xc820144120 fetched}
sent to chan
&{http://golang.org/ 0xc82008b710 fetched}
sent to chan
&{http://yahoo.com/ 0xc82008b7a0 fetched}
The problem is that ranging over a channel in a for loop will continue forever unless the channel is closed. If you want to read precisely len(urls) values from the channel, you should loop that many times:
for i := 0; i < len(urls); i++ {
fmt.Println(<-ch)
}
Another good dirty devious trick would be to use sync.WaitGroup and increment it per goroutine and then monitor it with a Wait and after its done it will close your channel allowing the next blocks of code to run, the reason I am offering you this approach is because it gets away from using a static number in a loop like len(urls) so that you can have a dynamic slice that might change and what not.
The reason Wait and close are in their own goroutine is so that your code can reach the for loop to range over your channel
package main
import (
"fmt"
"net/http"
"time"
"sync"
)
var urls = []string{
"http://www.google.com/",
"http://golang.org/",
"http://yahoo.com/",
}
type HttpResponse struct {
url string
response *http.Response
err error
status string
}
func asyncHttpGets(url string, ch chan *HttpResponse, wg *sync.WaitGroup) {
client := http.Client{}
if url == "http://www.google.com/" {
time.Sleep(500 * time.Millisecond) //google is down
}
fmt.Printf("Fetching %s \n", url)
resp, err := client.Get(url)
u := &HttpResponse{url, resp, err, "fetched"}
ch <- u
fmt.Println("sent to chan")
wg.Done()
}
func main() {
fmt.Println("start")
ch := make(chan *HttpResponse, len(urls))
var wg sync.WaitGroup
for _, url := range urls {
wg.Add(1)
go asyncHttpGets(url, ch, &wg)
}
go func() {
wg.Wait()
close(ch)
}()
for i := range ch {
fmt.Println(i)
}
fmt.Println("Im done")
}