If I use a chan anywhere on main or func home, the application runs, but it doesn't really work. No errors thrown, however, it won't work.
If I remove the channels references, it goes back to working.
Either by using a chan in a struct or a global channel the application stops working.
In a GET request, it returns h.Message from func home
by adding any channel in the code, GET request wont return the message.
https://play.golang.org/p/-ZVcLhZRRRG
package main
import (
"fmt"
"net/http"
"github.com/gorilla/websocket"
// _ "github.com/go-sql-driver/mysql"
)
type WMessage struct {
Message string `json:"message"`
ch chan string
}
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
CheckOrigin: func(r *http.Request) bool {
return true
},
}
var Chann chan string
func (h *WMessage) home(w http.ResponseWriter, r *http.Request) {
h.Message = "hey this is the message from home"
fmt.Fprintln(w, h.Message)
fmt.Println(<-h.ch)
}
func main() {
hom := &WMessage{}
hom.ch = make(chan string)
hom.ch <- "message sent"
http.HandleFunc("/", hom.home)
err := http.ListenAndServe(":3000", nil)
if err != nil {
fmt.Println("there was an error -> ", err)
}
}
Unbuffered channels are blocking in nature, i.e. writing to channel (hom.ch <- "message sent") and reading from channel (fmt.Println(<-h.ch)) block the go-routine.
In your case, http server is not running because hom.ch <- "message sent" blocks the execution. That's why GET request is not working.
One simple solution can be to make it buffered channel instead.
Related
I have a websocket server in Go using the Gorilla websocket package. At this stage, I will have only one server serving 5 clients. I am getting some messages from upstream into the WebSocket server. My intention is to NOT BROADCAST all the messages to the connected clients. I would like to send only one copy of the message to the connected clients in a round robin fashion. It doesn't matter which client gets it as long as there is only one that gets it.
My attempted solution
I have a simple Go server, created a Pool of clients (websocket connections) that I am receiving. However, I do not see any options to round robin the messages as I mentioned above. All my clients are getting the message. How can I send only one copy of the message to the connected clients instead of broadcasting to all.
Discalimer
The code I have is taken from online sources and modified to my requirement. I am relatively new to Go and Websockets. Is this something even possible using Websockets?
main.go
package main
import (
"fmt"
"net/http"
"github.com/realtime-chat-go-react/backend/pkg/websocket"
)
func serveWs(pool *websocket.Pool, w http.ResponseWriter, r *http.Request) {
fmt.Println("WebSocket Endpoint Hit")
conn, err := websocket.Upgrade(w, r)
if err != nil {
fmt.Fprintf(w, "%+v\n", err)
}
client := &websocket.Client{
Conn: conn,
Pool: pool,
}
pool.Register <- client
client.Read()
}
func setupRoutes() {
pool := websocket.NewPool()
go pool.Start()
http.HandleFunc("/ws", func(w http.ResponseWriter, r *http.Request) {
serveWs(pool, w, r)
})
}
func main() {
setupRoutes()
err := http.ListenAndServe(":8080",nil)
if err != nil {
fmt.Println(err)
}
}
websocket.go
package websocket
import (
"log"
"net/http"
"github.com/gorilla/websocket"
)
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
var wsList []*websocket.Conn
func Upgrade(w http.ResponseWriter, r *http.Request) (*websocket.Conn, error) {
upgrader.CheckOrigin = func(r *http.Request) bool { return true }
conn, err := upgrader.Upgrade(w, r, nil)
wsList = append(wsList, conn) //Creating a list here to store all websocket clients.
if err != nil {
log.Println(err)
return nil, err
}
return conn, nil
}
pool.go
package websocket
import "fmt"
type Pool struct {
Register chan *Client
Unregister chan *Client
Clients map[*Client]bool
Broadcast chan Message
}
func NewPool() *Pool {
return &Pool{
Register: make(chan *Client),
Unregister: make(chan *Client),
Clients: make(map[*Client]bool),
Broadcast: make(chan Message),
}
}
func (pool *Pool) Start() {
for {
select {
case client := <-pool.Register:
pool.Clients[client] = true
fmt.Println("Size of Connection Pool: ", len(pool.Clients))
for client, _ := range pool.Clients {
fmt.Println(client)
client.Conn.WriteJSON(Message{Type: 1, Body: "New User Joined..."})
}
break
case client := <-pool.Unregister:
delete(pool.Clients, client)
fmt.Println("Size of Connection Pool: ", len(pool.Clients))
for client, _ := range pool.Clients {
client.Conn.WriteJSON(Message{Type: 1, Body: "User Disconnected..."})
}
break
case message := <-pool.Broadcast: //This is where I need to modify the code but not sure how
fmt.Println("Sending message to all clients in Pool")
for client, _ := range pool.Clients {
if err := client.Conn.WriteJSON(message); err != nil {
fmt.Println(err)
return
}
}
}
}
}
client.go
package websocket
import (
"fmt"
"log"
"sync"
"github.com/gorilla/websocket"
)
type Client struct {
ID string
Conn *websocket.Conn
Pool *Pool
mu sync.Mutex
}
type Message struct {
Type int `json:"type"`
Body string `json:"body"`
}
func (c *Client) Read() {
defer func() {
c.Pool.Unregister <- c
c.Conn.Close()
}()
for {
messageType, p, err := c.Conn.ReadMessage()
if err != nil {
log.Println(err)
return
}
message := Message{Type: messageType, Body: string(p)}
c.Pool.Broadcast <- message
fmt.Printf("Message Received: %+v\n", message)
}
}
Modify the pool to store clients in a slice instead of a map. Add field to record index of the previous client used.
type Pool struct {
Register chan *Client
Unregister chan *Client
Clients []*Client
Broadcast chan Message
PrevClientIndex int
}
Round robin instead of broadcasting:
case message := <-pool.Broadcast:
if len(pool.Clients) == 0 {
continue
}
pool.PrevClientIndex++
if pool.PrevClientIndex >= len(pool.Clients) {
pool.PrevClientIndex = 0
}
client := pool.Clients[pool.PrevClientIndex]
if err := client.Conn.WriteJSON(message); err != nil {
// handle error
...
Register appends to the slice:
case client := <-pool.Register:
pool.Clients = append(pool.Clients, client)
...
Unregister removes the client from the slice:
case client := <-pool.Unregister:
j := 0
for _, c := range pool.Clients {
if c != client {
c.Clients[j] = c
j++
}
}
pool.Clients = pool.Clients[:j]
...
I'm using a channel to pass messages from an HTTP handler:
package server
import (
"bytes"
"errors"
"io/ioutil"
"log"
"net/http"
)
type Server struct {}
func (s Server) Listen() chan interface{} {
ch := make(chan interface{})
http.HandleFunc("/", handle(ch))
go http.ListenAndServe(":8080", nil)
return ch
}
func handle(ch chan interface{}) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
b, err := ioutil.ReadAll(r.Body)
defer r.Body.Close()
if err != nil {
ch <- errors.New(string(500))
return
}
w.Write([]byte("Hello World"))
log.Print("about to pass to handler channel")
ch <- bytes.NewBuffer(b)
log.Print("passed to handler channel")
}
}
When I make a request to the server running on port 8080, the thread blocks on this line:
ch <- bytes.NewBuffer(b)
Why is this happening? If you notice, I'm running the listener in a goroutine. I also figured that HTTP handles happen in a separate thread. If I delete the above line, the thread becomes unblocked and the program works as expected. What am I doing wrong?
To clarify, I want to be able to pass the body of a POST request to a channel. Help.
UPDATE:
I'm reading from the channel on the main thread:
listenerChan := n.Listen()
go SendRequest("POST", "http://localhost:8080", []byte("hello"))
for listenedMsg := range listenerChan {
log.Print("listened message>>>> ", listenedMsg)
}
But the thread still blocks on the same line. For clarification, there is nothing wrong with how im sending the request. If I remove the channel send line above, the thread doesnt block.
Because the channel is unbuffered, the send operation blocks until there's someone who is ready to receive from them. Making the channel buffered will only defer the blocking, so you always need some reading goroutine.
Update to your update: the control flow of the program would go like this:
Server starts listening
main sends the request and waits for the response
Server receives the request and tries to write to the channel
main reads from the channel
4 may happen only after 2, which is blocked by 3 which is blocked because 4 is not happening yet. A classical deadlock.
I think #bereal gave a good explanation about using an unbuffered or synchronous channel.
Another way to make things work is to make the channel buffered by changing the line that creates the channel to:
ch := make(chan interface{}, 1) // added the 1
This will prevent the function from being blocked.
I added missing parts in your code and run it, everything works well. I don't see any block. Here's the code:
package main
import (
"bytes"
"errors"
"io/ioutil"
"log"
"net/http"
"time"
)
type Server struct{}
func (s *Server) Listen() chan interface{} {
ch := make(chan interface{})
http.HandleFunc("/", handle(ch))
go http.ListenAndServe(":8080", nil)
return ch
}
func handle(ch chan interface{}) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
b, err := ioutil.ReadAll(r.Body)
defer r.Body.Close()
if err != nil {
ch <- errors.New(string(500))
return
}
w.Write([]byte("Hello World"))
log.Print("about to pass to handler channel")
ch <- bytes.NewBuffer(b)
log.Print("passed to handler channel")
}
}
// SendRequest send request
func SendRequest(method string, url string, data []byte) {
tr := &http.Transport{
MaxIdleConns: 10,
IdleConnTimeout: 30 * time.Second,
DisableCompression: true,
}
client := &http.Client{Transport: tr}
reader := bytes.NewReader(data)
req, err := http.NewRequest(method, url, reader)
if err != nil {
panic(err)
}
client.Do(req)
}
func main() {
n := new(Server)
listenerChan := n.Listen()
go SendRequest("POST", "http://localhost:8080", []byte("hello"))
for listenedMsg := range listenerChan {
log.Print("listened message>>>> ", listenedMsg)
}
}
And the output are:
2018/06/28 17:22:10 about to pass to handler channel
2018/06/28 17:22:10 passed to handler channel
2018/06/28 17:22:10 listened message>>>> hello
I've created a simple websocket that publishes a JSON stream. I't works fine most of the time except for few cases where I think while looping through the clients to send them message, it gets hung up on a client that is being disconnected abnormally. What measure can I add to this code to mitigate it?
Client.go
import (
"github.com/gorilla/websocket"
)
type client struct {
socket *websocket.Conn
send chan *Message
}
func (c *client) read() {
defer c.socket.Close()
for {
_, _, err := c.socket.ReadMessage()
if err != nil {
log.Info("Websocket: %s", err)
break
}
}
}
func (c *client) write() {
defer c.socket.Close()
for msg := range c.send {
err := c.socket.WriteJSON(msg)
if err != nil {
break
}
}
}
Stream.go
import (
"net/http"
"github.com/gorilla/websocket"
)
const (
socketBufferSize = 1024
messageBufferSize = 256
)
var upgrader = &websocket.Upgrader{
ReadBufferSize: socketBufferSize,
WriteBufferSize: socketBufferSize,
}
type Stream struct {
Send chan *Message
join chan *client
leave chan *client
clients map[*client]bool
}
func (s *Stream) Run() {
for {
select {
case client := <-s.join: // joining
s.clients[client] = true
case client := <-s.leave: // leaving
delete(s.clients, client)
close(client.send)
case msg := <-s.Send: // send message to all clients
for client := range s.clients {
client.send <- msg
}
}
}
}
func (s *Stream) ServeHTTP(w http.ResponseWriter, res *http.Request) {
socket, err := upgrader.Upgrade(w, res, nil)
if err != nil {
log.Error(err)
return
}
defer func() {
socket.Close()
}()
client := &client{
socket: socket,
send: make(chan *Message, messageBufferSize),
}
s.join <- client
defer func() { s.leave <- client }()
go client.write()
client.read()
}
See the Gorilla Chat Application for an example of how to avoid blocking on a client.
The key parts are:
Use a buffered channel for sending to the client. Your application is already doing this.
Send to the client using select/default to avoid blocking. Assume that the client is blocked on write when the client cannot immediately receive a message. Close the client's channel in this situation to cause the client's write loop to exit.
Write with a deadline.
I think this question has been asked before (and probably more than once) but I can't find it...
Im learning Go, and I wanted to extend the classical web server example by sending a channel to the "handler".
I have this standard thing:
func hello(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Hello world!")
}
func main() {
http.HandleFunc("/", hello)
http.ListenAndServe(":8000", nil)
}
And now I would like the "hello" function to be able to write stuff on a channel, for someone to consume... The way I have done with "normal" functions is to create a channel:
c := make(chan string)
and pass c in the call to the function. Something like:
dosomething(c)
But... how would I go about doing that if I want "hello" to get access to the channel c?
There are two other ways to do this (other than exporting your channels as in the previous answer).
The first is to use a function to return another handler function. When the function is returned, it will create a closure around the channel.
func makeHello(logger chan string) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
logger <- r.Host
io.WriteString(w, "Hello world!")
}
}
The second is to use a struct which holds the channel as a member and use pointer receiver methods to handle the request...
type DataPasser struct {
logs chan string
}
func (p *DataPasser) handleHello(w http.ResponseWriter, r *http.Request) {
p.logs <- r.URL.String()
io.WriteString(w, "Hello world")
}
This is a full working example (just hit /1 and /2 to see the two examples)
package main
import (
"fmt"
"io"
"net/http"
)
func main() {
// METHOD 1
logs := make(chan string)
go logLogs(logs)
handleHello := makeHello(logs)
// METHOD 2
passer := &DataPasser{logs: make(chan string)}
go passer.log()
http.HandleFunc("/1", handleHello)
http.HandleFunc("/2", passer.handleHello)
http.ListenAndServe(":9999", nil)
}
// METHOD 1
func makeHello(logger chan string) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
logger <- r.Host
io.WriteString(w, "Hello world!")
}
}
func logLogs(logger chan string) {
for item := range logger {
fmt.Println("1. Item", item)
}
}
// METHOD 2
type DataPasser struct {
logs chan string
}
func (p *DataPasser) handleHello(w http.ResponseWriter, r *http.Request) {
p.logs <- r.URL.String()
io.WriteString(w, "Hello world")
}
func (p *DataPasser) log() {
for item := range p.logs {
fmt.Println("2. Item", item)
}
}
There a few ways to solve this problem, the simplest is to define an exported channel in a package and import said package where ever you want to use the channel.
package mychannel
var Chan = make(chan string)
Server.go
package main
import (
"fmt"
"net/http"
//"strings"
"encoding/json"
"io/ioutil"
"strconv"
"net"
"bufio"
)
type Message struct {
Text string
}
func Unmarshal(data []byte, v interface{}) error
func main() {
//http.HandleFunc("/", handler)
server,_ := net.Listen("tcp", ":" + strconv.Itoa(8080))
if server == nil {
panic("couldn't start listening: ")
}
conns := clientConns(server)
for {
go handleConn(<-conns)
}
}
func clientConns(listener net.Listener) chan net.Conn {
ch := make(chan net.Conn)
i := 0
go func() {
for {
client, _ := listener.Accept()
if client == nil {
fmt.Printf("couldn't accept: ")
continue
}
i++
fmt.Printf("%d: %v <-> %v\n", i, client.LocalAddr(), client.RemoteAddr())
ch <- client
}
}()
return ch
}
func handleConn(client net.Conn) {
b := bufio.NewReader(client)
fmt.Println("Buffer")
for {
line, err := b.ReadBytes('\n')
if err != nil { // EOF, or worse
break
}
client.Write(line)
}
}
Client.go
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
"strings"
"flag"
//"io"
// "net"
// "net/rpc"
// "sync"
)
func Unmarshal(data []byte, v interface{}) error
func Marshal(v interface{}) ([]byte, error)
type Message struct {
Text string
}
func main(){
var flagtext = flag.String("flagtext", "Hello!", "Flag")
flag.Parse()
var text string
text = *flagtext
m := Message{text}
var m1 Message
b, err := json.Marshal(m)
if err == nil{
resp, err := http.Post("http://127.0.0.1:8080","application/json", strings.NewReader(string(b)))
if err != nil{
log.Fatal("Error while post: %v",err)
}
fmt.Println(resp)
err = json.Unmarshal(b, &m1)
}
}
Error I get when I run client.go is this:
Error while post: %vmalformed HTTP status code "/"
Though, the server registers a channel for each post, it shows a malformed HTTP status code. Is it because I'm listening in the wrong channel? I'm confused why this error is occurring.
This line in the server code:
client.Write(line)
sends the request line back to the client. Since the client is posting something like GET / HTTP/1.1, this means that the server is responding with something like GET / HTTP/1.1, instead of something like HTTP/1.1 200 OK. The error-message you're seeing is because / appears in the status-code position.
In server.go it seems you are trying to write your own HTTP server from the TCP socket level up. This is unnecessary work - take the easy route and use the built-in HTTP server API.
The general outline of such a server is like this:
package main
import (
"fmt"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
and is described further in this article. More documentation is in net/http.