How to reuse http2 stream in golang?
Hi all, recently I tried many ways to reuse a stream to send http2 data but failed, and I didn't find many related solutions in http2, can anyone give me some hints or demos?
client := http.Client{
Transport: &http2.Transport{
DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, error) {
return net.DialTimeout(network, addr, time.Second)
},
AllowHTTP: true,
},
}
resp, err := client.Get(path)
I tried to use resp as client and write to this resp to send data to server. Is it right way to do it?
In HTTP/2 streams are short lived, and correspond to a request/response exchange. In effect the end of the response closes the stream. Issuing another request via client.Get will automatically open a new stream on the connection, if the underlying connection is using HTTP/2.
Related
I have been trying to set up a WebSocket connection over TLS (so with encryption). I use Golang with Gorilla. A WebSocket connection is implemented as an initial HTTP connection that gets upgraded to the WebSocket protocol connection. The code is like this:
func wsEndpoint(w http.ResponseWriter, r *http.Request) {
// upgrade
ws, err := upgrader.Upgrade(w, r, nil)
if err != nil {
//...
}
log.Println("Client Connected")
err = ws.WriteMessage(1, []byte("Hi Client!"))
if err != nil {
//...
}
// listen indefinitely for new messages coming
}
Then we set up the routing:
func main() {
//...
setupRoutes()
log.Fatal(http.ListenAndServe(":8080", nil))
}
Does it suffice to change the last line to:
...http.ListenAndServeTLS(...)
so in other words to use TLS to establish the first connection?
Does this approach suffice to secure the entire communication over WebSocket from start till the end? Should I be certain that all packet transmission within the connection duration is also protected by TLS? If not, how to set it up in Golang / Gorilla framework?
Use http.ListenAndServeTLS to encrypt the underlying network connections used for the HTTP protocol and the WebSocket protocol.
The approach secures the entire communication on the underlying network connection including all WebSocket traffic.
The Gorilla server code uses the network connection provided by the net/http server. The Gorilla server code does create new network connections.
http.ListenAndServeTLS is a helper function that calls lower-level functions and methods. It also works to call those lower-level functions and methods directly.
One approach to this problem would be to set up a reverse proxy like nginx with certbot to generate certificates.
Here's how it would work
[ Client ] ----------> [ nginx ] --------------> [ golang server ]
Encrypted Not encrypted
I am using grpc in my project, if i have a grpc service call helloService, should i use GetNewHelloServiceClient to get a new client in every function? Or just get once in start program?
// for example:
c.GET("/hello", SayHello)
func SayHello() {
c := pb.GetNewHelloServiceClient()
res, err := c.SayHello(context.Background(), &request)
if err != nil {
return
}
fmt.print(res.Hello)
}
Create a gRPC client just once.
Lots of networking concepts in go are designed for reuse: http clients, http transports, sql.DB connection pools etc. They are all go-routine safe & should only be created once but reused many times.
I am very new to Go and have found myself working with sockets as my first project. This is a redundant question, but I have failed to understand how
to send a websocket update to a specific client in Go (using Gorilla).
The broad problem that I am trying to solve is - Building a typeahead using websockets and a search engine like ES/Lucene. I have maintained a bunch of indexes on my search engine and have a Go wrapper around it. When I started working on using websockets in Go, I have been finding almost all the examples showing broadcasting mechanism. When I tried to dig into this and tried to modify the example given in Gorilla's github repo based on the examples given in this thread and in this answer, I don't seem to understand connections and how does that fit in client.go
Ideally, the way I would like to see this working is -
A socket connection between the client and server is established
Upon the client sending inputs via the socket, the server fetches it and throws into into a channel (Go channel)
The indexing wrapper checks for this channel, and once there is something to fetch, the index is retrieved and written back to the socket
How can the server uniquely identify the Client?
I have used the examples given on Gorilla's Github repo
From my codebase hub.go has the following
type Hub struct {
// Registered clients.
clients map[*Client]bool
// Inbound messages from the clients.
broadcast chan []byte
// Register requests from the clients.
register chan *Client
// Unregister requests from clients.
unregister chan *Client
connections map[string]*connection
}
func newHub() *Hub {
return &Hub{
broadcast: make(chan []byte),
register: make(chan *Client),
unregister: make(chan *Client),
clients: make(map[*Client]bool),
connection: make(map[*Client]bool), // is this alright?
}
}
func (h *Hub) run() {
for {
select {
case client := <-h.register:
h.clients[client] = true
case client := <-h.unregister:
if _, ok := h.clients[client]; ok {
delete(h.clients, client)
close(client.send)
}
case message := <-h.broadcast:
for client := range h.connections {
select {
case client.send <- message:
default:
close(client.send)
delete(h.connections, client)
}
}
}
}
}
and I am unsure with what I should be adding to client.go
type Client struct {
// unique ID for each client
// id string
// Hub object
hub *Hub
// The websocket connection.
conn *websocket.Conn
// Buffered channel of outbound messages.
send chan []byte
// connection --> (what should the connection property be?)
connection string
}
Please note - I will be adding an Id field within the Client struct. How can I proceed from here?
The chat example shows how to implement broadcast. The chat example is not a good starting point for an application if broadcast is not required.
To send a message to a specific websocket connection, simply write to the connection using NextWriter or WriteMessage. These methods do not support concurrent writers, so you may need to use a mutex or goroutine to ensure a single writer.
The simple approach for finding a specific *websocket.Connection is to pass *websocket.Connection to the code that needs it. If the application needs to associate other state with a connection, then define a type to hold that state and pass a pointer to that around:
type Client struct {
conn *websocket.Conn
mu sync.Mutex
...
}
The Hub can be modified to send messages to specific connection, but it's a roundabout path if broadcast is not needed. Here's how to do it:
Add ID field to client:
ID idType // replace idType with int, string, or whatever you want to use
Change the Gorilla hub field from connections map[*connection]bool to connections map[idType]*connection.
Define a message type containing the message data and the ID of the target client:
type message struct {
ID idtype
data []byte
}
Replace the hub broadcast field with:
send chan message
Change the hub for loop to:
for {
select {
case client := <-h.register:
h.clients[client.ID] = client
case client := <-h.unregister:
if _, ok := h.clients[client.ID]; ok {
delete(h.clients, client.ID)
close(client.send)
}
case message := <-h.send:
if client, ok := h.clients[message.ID]; ok {
select {
case client.send <- message.data:
default:
close(client.send)
delete(h.connections, client)
}
}
}
Send messages to a specific client by creating a message with the appropriate ID:
hub.send <- message{ID: targetID, data: data}
I'm wondering if it's a good idea to push data from gRPC server to a client. Basically I want to use a pub/sub pattern with gRPC.
The way I do is that I return a response stream on the server implementation that I never close. Then, the client has a never ending go routine in charge of reading this stream.
Here is an example:
service Service {
rpc RegularChanges (Void) returns (stream Change) {}
}
On the server side:
func (self *MyServiceImpl) RegularChanges(in *pb.Void, stream pb.Service_RegularChangesServer) error {
for {
d, err := time.ParseDuration("1s")
if err != nil {
log.Fatalf("Cannot parse duration")
break;
}
time.Sleep(d)
stream.Send(&pb.Change{Name:"toto", Description:"status changed"})
}
return nil
}
On client:
for {
change, err := streamChanges.Recv()
if err != nil {
log.Fatalf("Error retrieving change")
} else {
log.Println(change)
}
}
I just began with go and gRPC but I know it's based on HTTP2, hence it should support pushing datas. However, I'm not sure this is the way gRPC should be used.
gRPC is intended to be used in this way.
You should still consider how the client should behave on failures and how you may want to re-balance across backends. If your connection is going across the Internet, you may also want to enable keepalive to detect connection breakages by providing KeepaliveParams to the client and server.
I use code.google.com/p/go.net/websocket in server, so client can get notification from server.
however, It seems after client connected to server, if there is no any data tranfer between client and server, server will return EOF error at websocket.JSON.Receive(), it looks like a timeout mechanism.
And I have search in Google, it seems websocket protocol has a ping-pong heartbeat to maintain the connection, I want to ask whether code.google.com/p/go.net/websocket support this ping protocol or not?
What should I do if I want keep connection between client and server alive?
Here's working drop-in solution for gorilla/websocket package.
func keepAlive(c *websocket.Conn, timeout time.Duration) {
lastResponse := time.Now()
c.SetPongHandler(func(msg string) error {
lastResponse = time.Now()
return nil
})
go func() {
for {
err := c.WriteMessage(websocket.PingMessage, []byte("keepalive"))
if err != nil {
return
}
time.Sleep(timeout/2)
if(time.Since(lastResponse) > timeout) {
c.Close()
return
}
}
}()
}
As recently as 2013, the go.net websocket library does not support (automatic) keep-alive messages. You have two options:
Implement an "application level" keep-alive by periodically having your application send a message down the pipe (either direction should work), that is ignored by the other side.
Move to a different websocket library that does support keep-alives (like this one) Edit: it looks like that library has been superseded by Gorilla websockets.