How to set SetKeepAlivePeriod on a *tls.Conn - go

I want to increase the keep alive period of my TCP connection for both HTTP and HTTPS requests.
For HTTP requests this can be done like this:
package main
import (
"fmt"
"io"
"log"
"net"
"net/http"
"time"
)
func main() {
server := &http.Server{Addr: ":8080", Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Hello, World!")
})}
server.ConnState = func(conn net.Conn, state http.ConnState) {
if state == http.StateNew {
if err := conn.(*net.TCPConn).SetKeepAlivePeriod(1000 * time.Second); err != nil {
fmt.Println("Could not set keep alive period", err)
} else {
fmt.Println("update keep alive period")
}
}
}
log.Fatal(server.ListenAndServe())
}
For HTTPS requests this cannot be done via server.ConnState because the net.Conn that will be passed inside the function is a *tls.Conn. This connection does not expose a function like SetKeepAlivePeriod or gives access to the underlying *net.TCPConn.
func main() {
server := &http.Server{Addr: ":8080", Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Hello, World!")
})}
server.ConnState = func(conn net.Conn, state http.ConnState) {
if state == http.StateNew {
tlsConn := conn.(*tls.Conn)
// how to set SetKeepAlivePeriod
}
}
log.Fatal(server.ListenAndServeTLS("../example.crt", "../example.key"))
}
How can I set the keep alive period for tls connections?

There are (at least) two ways to do it:
Use a net.ListenConfig:
The net.ListenConfig object has a KeepAlive time.Duration field. When non-zero, this will be used to set the keep-alive on accepted connections (eg: for TCP on posix).
You can pass the listener to ServeTLS:
server := &http.Server{...}
lc := net.ListenConfig{KeepAlive: 1000 * time.Second}
ln, err := lc.Listen(context.Background(), "tcp", ":8080")
if err != nil {
panic(err)
}
defer ln.Close()
log.Fatal(server.ServeTLS(ln, "../example.crt", "../example.key"))
As mentioned, accepted TCP connections will automatically have keep-alive enabled and the period set to the specified value.
Use a tls.Config callback:
You can access the net.Conn underlying the tls.Conn by setting the tls.Config GetConfigForClient or GetCertificate callback.
It does not matter which one you're using as long as you return nil to make the TLS code fall back to the default behavior. The important part is to get access to the tls.ClientHelloInfo which has a .Conn field pointing to the underlying connection. This will the net.TCPConn.
setTCPKeepAlive := func(clientHello *tls.ClientHelloInfo) (*tls.Config, error) {
// Check that the underlying connection really is TCP.
if tcpConn, ok := clientHello.Conn.(*net.TCPConn); ok {
if err := tcpConn.SetKeepAlivePeriod(1000 * time.Second); err != nil {
fmt.Println("Could not set keep alive period", err)
} else {
fmt.Println("update keep alive period")
}
} else {
fmt.Println("TLS over non-TCP connection")
}
// Make sure to return nil, nil to let the caller fall back on the default behavior.
return nil, nil
}
tlsConfig := &tls.Config{
...
GetConfigForClient: setTCPKeepAlive,
...
}
server := &http.Server{
Addr: ":8080",
TLSConfig: tlsConfig,
}
server.ListenAndServeTLS("../example.crt", "../example.key")

Related

Proper way for cancelling accept(block) and close listener in Go [duplicate]

I've been trying to find a way to stop a listening server in Go gracefully. Because listen.Accept blocks it is necessary to close the listening socket to signal the end, but I can't tell that error apart from any other errors as the relevant error isn't exported.
Can I do better than this? See FIXME in the code below in serve()
package main
import (
"io"
"log"
"net"
"time"
)
// Echo server struct
type EchoServer struct {
listen net.Listener
done chan bool
}
// Respond to incoming connection
//
// Write the address connected to then echo
func (es *EchoServer) respond(remote *net.TCPConn) {
defer remote.Close()
_, err := io.Copy(remote, remote)
if err != nil {
log.Printf("Error: %s", err)
}
}
// Listen for incoming connections
func (es *EchoServer) serve() {
for {
conn, err := es.listen.Accept()
// FIXME I'd like to detect "use of closed network connection" here
// FIXME but it isn't exported from net
if err != nil {
log.Printf("Accept failed: %v", err)
break
}
go es.respond(conn.(*net.TCPConn))
}
es.done <- true
}
// Stop the server by closing the listening listen
func (es *EchoServer) stop() {
es.listen.Close()
<-es.done
}
// Make a new echo server
func NewEchoServer(address string) *EchoServer {
listen, err := net.Listen("tcp", address)
if err != nil {
log.Fatalf("Failed to open listening socket: %s", err)
}
es := &EchoServer{
listen: listen,
done: make(chan bool),
}
go es.serve()
return es
}
// Main
func main() {
log.Println("Starting echo server")
es := NewEchoServer("127.0.0.1:18081")
// Run the server for 1 second
time.Sleep(1 * time.Second)
// Close the server
log.Println("Stopping echo server")
es.stop()
}
This prints
2012/11/16 12:53:35 Starting echo server
2012/11/16 12:53:36 Stopping echo server
2012/11/16 12:53:36 Accept failed: accept tcp 127.0.0.1:18081: use of closed network connection
I'd like to hide the Accept failed message, but obviously I don't want to mask other errors Accept can report. I could of course look in the error test for use of closed network connection but that would be really ugly. I could set a flag saying I'm about to close and ignore errors if that was set I suppose - Is there a better way?
I would handle this by using es.done to send a signal before it closes the connection. In addition to the following code you'd need to create es.done with make(chan bool, 1) so that we can put a single value in it without blocking.
// Listen for incoming connections
func (es *EchoServer) serve() {
for {
conn, err := es.listen.Accept()
if err != nil {
select {
case <-es.done:
// If we called stop() then there will be a value in es.done, so
// we'll get here and we can exit without showing the error.
default:
log.Printf("Accept failed: %v", err)
}
return
}
go es.respond(conn.(*net.TCPConn))
}
}
// Stop the server by closing the listening listen
func (es *EchoServer) stop() {
es.done <- true // We can advance past this because we gave it buffer of 1
es.listen.Close() // Now it the Accept will have an error above
}
Check some "is it time to stop" flag in your loop right after the accept() call, then flip it from your main, then connect to your listening port to get server socket "un-stuck". This is very similar to the old "self-pipe trick".
Something among these lines might work in this case, I hope:
// Listen for incoming connections
func (es *EchoServer) serve() {
for {
conn, err := es.listen.Accept()
if err != nil {
if x, ok := err.(*net.OpError); ok && x.Op == "accept" { // We're done
log.Print("Stoping")
break
}
log.Printf("Accept failed: %v", err)
continue
}
go es.respond(conn.(*net.TCPConn))
}
es.done <- true
}
Here's a simple way that's good enough for local development.
http://www.sergiotapia.me/how-to-stop-your-go-http-server/
package main
import (
"net/http"
"os"
"github.com/bmizerany/pat"
)
var mux = pat.New()
func main() {
mux.Get("/kill", http.HandlerFunc(kill))
http.Handle("/", mux)
http.ListenAndServe(":8080", nil)
}
func kill(w http.ResponseWriter, r *http.Request) {
os.Exit(0)
}

Setting idletimeout in Go

I have a function in go which is handling connections which are coming through tcp and handled via ssh. I am trying to set an idle timeout by creating struct in the connection function.
Use case - a customer should be able to make a connection and upload/download multiple files
Reference - IdleTimeout in tcp server
Function code:
type Conn struct {
net.Conn
idleTimeout time.Duration
}
func HandleConn(conn net.Conn) {
var err error
rAddr := conn.RemoteAddr()
session := shortuuid.New()
config := LoadSSHServerConfig(session)
blocklistItem := blocklist.GetBlockListItem(rAddr)
if blocklistItem.IsBlocked() {
conn.Close()
atomic.AddInt64(&stats.Stats.BlockedConnections, 1)
return
}
func (c *Conn) Read(b []byte) (int, error) {
err := c.Conn.SetReadDeadline(time.Now().Add(c.idleTimeout))
if err != nil {
return 0, err
}
return c.Conn.Read(b)
}
sConn, chans, reqs, err := ssh.NewServerConn(conn, config)
if err != nil {
if err == io.EOF {
log.Errorw("SSH: Handshaking was terminated", log.Fields{
"address": rAddr,
"error": err,
"session": session})
} else {
log.Errorw("SSH: Error on handshaking", log.Fields{
"address": rAddr,
"error": err,
"session": session})
}
atomic.AddInt64(&stats.Stats.AuthorizationFailed, 1)
return
}
log.Infow("connection accepted", log.Fields{
"user": sConn.User(),
})
if user, ok := users[session]; ok {
log.Infow("SSH: Connection accepted", log.Fields{
"user": user.LogFields(),
"clientVersion": string(sConn.ClientVersion())})
atomic.AddInt64(&stats.Stats.AuthorizationSucceeded, 1)
// The incoming Request channel must be serviced.
go ssh.DiscardRequests(reqs)
// Key ID: sConn.Permissions.Extensions["key-id"]
handleServerConn(user, chans)
log.Infow("connection finished", log.Fields{"user": user.LogFields()})
log.Infow("checking connections", log.Fields{
//"cc": Stats.AcceptedConnections,
"cc2": &stats.Stats.AcceptedConnections})
// Remove connection from local cache
delete(users, session)
} else {
log.Infow("user not found from memory", log.Fields{"username": sConn.User()})
}
}
This code is coming from the Listen function:
func Listen() {
listener, err := net.Listen("tcp", sshListen)
if err != nil {
panic(err)
}
if useProxyProtocol {
listener = &proxyproto.Listener{
Listener: listener,
ProxyHeaderTimeout: time.Second * 10,
}
}
for {
// Once a ServerConfig has been configured, connections can be accepted.
conn, err := listener.Accept()
if err != nil {
log.Errorw("SSH: Error accepting incoming connection", log.Fields{"error": err})
atomic.AddInt64(&stats.Stats.FailedConnections, 1)
continue
}
// Before use, a handshake must be performed on the incoming net.Conn.
// It must be handled in a separate goroutine,
// otherwise one user could easily block entire loop.
// For example, user could be asked to trust server key fingerprint and hangs.
go HandleConn(conn)
}
}
Is that even possible to set a deadline for only the connections which have been idle for 20 secinds (no upload/downloads).
EDIT 1 : Following #LiamKelly's suggestions, I have made the changes in the code. Now the code is like
type SshProxyConn struct {
net.Conn
idleTimeout time.Duration
}
func (c *SshProxyConn) Read(b []byte) (int, error) {
err := c.Conn.SetReadDeadline(time.Now().Add(c.idleTimeout))
if err != nil {
return 0, err
}
return c.Conn.Read(b)
}
func HandleConn(conn net.Conn) {
//lines of code as above
sshproxyconn := &SshProxyConn{nil, time.Second * 20}
Conn, chans, reqs, err := ssh.NewServerConn(sshproxyconn, config)
//lines of code
}
But now the issue is that SSH is not happening. I am getting the error "Connection closed" when I try to do ssh. Is it still waiting for "conn" variable in the function call?
Is that even possible to set a deadline for only the connections which have been idle for 20 [seconds]
Ok so first a general disclaimer, I am going to assume go-protoproxy implements the Conn interface as we would expected. Also as you hinted at before, I don't think you can put a a struct method inside another function (I also recommend renaming it something unique to prevent Conn vs net.Conn confusion).
type SshProxyConn struct {
net.Conn
idleTimeout time.Duration
}
func (c *SshProxyConn) Read(b []byte) (int, error) {
err := c.Conn.SetReadDeadline(time.Now().Add(c.idleTimeout))
if err != nil {
return 0, err
}
return c.Conn.Read(b)
}
func HandleConn(conn net.Conn) {
This makes is more clear what your primary issue is, which you passed the normal net.Conn to your SSH server, not your wrapper class. So
sConn, chans, reqs, err := ssh.NewServerConn(conn, config)
should be EDIT
sshproxyconn := &SshProxyConn{conn, time.Second * 20}
Conn, chans, reqs, err := ssh.NewServerConn(sshproxyconn , config)

write health check endpoints if the service has no HTTP server

I want to write health check endpoints for 2 different services, but the problem is they have no HTTP server.
if I can write health check endpoints how can I proceed. or is it mandatory to have an HTTP server to work on health check endpoints with Golang.
Yes, you can add an HTTP health check handler to your application with something like this. Then, in the service that's performing the health check, just make sure it knows which port to run the HTTP checks against.
package main
import "net/http"
func main() {
// Start the health check endpoint and make sure not to block
go func() {
_ = http.ListenAndServe(":8080", http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte("ok"))
},
))
}()
// Start my application code
}
Alternatively, if you need to expose your health check route at a separate path, you can do something like this.
http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte("ok"))
})
_ = http.ListenAndServe(":8080", nil)
Updated
If you want to check the health of a go-routine, you can do something like this.
package main
func main() {
crashed := make(chan struct{})
go func() {
defer close(crashed)
}()
select {
case <-crashed:
// Do something now that the go-routine crashed
}
}
It's not mandatory to have an HTTP server.
You can ping the IP address of your service server. For example I use ping repo:
package main
import (
"fmt"
"time"
"github.com/go-ping/ping"
)
func main() {
t := time.NewTicker(5 * time.Second)
for {
select {
case <-t.C:
err := checkService("google", "216.239.38.120")
if err != nil {
fmt.Println("notif to email, error:", err.Error())
time.Sleep(1 * time.Hour) // to not spam email
}
}
}
}
func checkService(name string, ip string) error {
p, err := ping.NewPinger(ip)
if err != nil {
return err
}
p.Count = 3
p.Timeout = 5 * time.Second
err = p.Run()
if err != nil {
return err
}
stats := p.Statistics()
if stats.PacketLoss == 100 {
return fmt.Errorf("service %s down", name)
}
fmt.Printf("stats: %#v\n", stats)
return nil
}

WebSocket Server that feeds messages to clients in a round robin fashion

I have a websocket server in Go using the Gorilla websocket package. At this stage, I will have only one server serving 5 clients. I am getting some messages from upstream into the WebSocket server. My intention is to NOT BROADCAST all the messages to the connected clients. I would like to send only one copy of the message to the connected clients in a round robin fashion. It doesn't matter which client gets it as long as there is only one that gets it.
My attempted solution
I have a simple Go server, created a Pool of clients (websocket connections) that I am receiving. However, I do not see any options to round robin the messages as I mentioned above. All my clients are getting the message. How can I send only one copy of the message to the connected clients instead of broadcasting to all.
Discalimer
The code I have is taken from online sources and modified to my requirement. I am relatively new to Go and Websockets. Is this something even possible using Websockets?
main.go
package main
import (
"fmt"
"net/http"
"github.com/realtime-chat-go-react/backend/pkg/websocket"
)
func serveWs(pool *websocket.Pool, w http.ResponseWriter, r *http.Request) {
fmt.Println("WebSocket Endpoint Hit")
conn, err := websocket.Upgrade(w, r)
if err != nil {
fmt.Fprintf(w, "%+v\n", err)
}
client := &websocket.Client{
Conn: conn,
Pool: pool,
}
pool.Register <- client
client.Read()
}
func setupRoutes() {
pool := websocket.NewPool()
go pool.Start()
http.HandleFunc("/ws", func(w http.ResponseWriter, r *http.Request) {
serveWs(pool, w, r)
})
}
func main() {
setupRoutes()
err := http.ListenAndServe(":8080",nil)
if err != nil {
fmt.Println(err)
}
}
websocket.go
package websocket
import (
"log"
"net/http"
"github.com/gorilla/websocket"
)
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
var wsList []*websocket.Conn
func Upgrade(w http.ResponseWriter, r *http.Request) (*websocket.Conn, error) {
upgrader.CheckOrigin = func(r *http.Request) bool { return true }
conn, err := upgrader.Upgrade(w, r, nil)
wsList = append(wsList, conn) //Creating a list here to store all websocket clients.
if err != nil {
log.Println(err)
return nil, err
}
return conn, nil
}
pool.go
package websocket
import "fmt"
type Pool struct {
Register chan *Client
Unregister chan *Client
Clients map[*Client]bool
Broadcast chan Message
}
func NewPool() *Pool {
return &Pool{
Register: make(chan *Client),
Unregister: make(chan *Client),
Clients: make(map[*Client]bool),
Broadcast: make(chan Message),
}
}
func (pool *Pool) Start() {
for {
select {
case client := <-pool.Register:
pool.Clients[client] = true
fmt.Println("Size of Connection Pool: ", len(pool.Clients))
for client, _ := range pool.Clients {
fmt.Println(client)
client.Conn.WriteJSON(Message{Type: 1, Body: "New User Joined..."})
}
break
case client := <-pool.Unregister:
delete(pool.Clients, client)
fmt.Println("Size of Connection Pool: ", len(pool.Clients))
for client, _ := range pool.Clients {
client.Conn.WriteJSON(Message{Type: 1, Body: "User Disconnected..."})
}
break
case message := <-pool.Broadcast: //This is where I need to modify the code but not sure how
fmt.Println("Sending message to all clients in Pool")
for client, _ := range pool.Clients {
if err := client.Conn.WriteJSON(message); err != nil {
fmt.Println(err)
return
}
}
}
}
}
client.go
package websocket
import (
"fmt"
"log"
"sync"
"github.com/gorilla/websocket"
)
type Client struct {
ID string
Conn *websocket.Conn
Pool *Pool
mu sync.Mutex
}
type Message struct {
Type int `json:"type"`
Body string `json:"body"`
}
func (c *Client) Read() {
defer func() {
c.Pool.Unregister <- c
c.Conn.Close()
}()
for {
messageType, p, err := c.Conn.ReadMessage()
if err != nil {
log.Println(err)
return
}
message := Message{Type: messageType, Body: string(p)}
c.Pool.Broadcast <- message
fmt.Printf("Message Received: %+v\n", message)
}
}
Modify the pool to store clients in a slice instead of a map. Add field to record index of the previous client used.
type Pool struct {
Register chan *Client
Unregister chan *Client
Clients []*Client
Broadcast chan Message
PrevClientIndex int
}
Round robin instead of broadcasting:
case message := <-pool.Broadcast:
if len(pool.Clients) == 0 {
continue
}
pool.PrevClientIndex++
if pool.PrevClientIndex >= len(pool.Clients) {
pool.PrevClientIndex = 0
}
client := pool.Clients[pool.PrevClientIndex]
if err := client.Conn.WriteJSON(message); err != nil {
// handle error
...
Register appends to the slice:
case client := <-pool.Register:
pool.Clients = append(pool.Clients, client)
...
Unregister removes the client from the slice:
case client := <-pool.Unregister:
j := 0
for _, c := range pool.Clients {
if c != client {
c.Clients[j] = c
j++
}
}
pool.Clients = pool.Clients[:j]
...

Golang H2C Server doesn't write full body. Chrome says: "Server reset stream". Only occurs over http2 connection

Most Recent Version of Go (1.153)
Below is the code for reproducibility. Please try to access https://easy-dp.ngrok.io to see the issue.
Here's what I did:
Create a Reverse Proxy accessing Gzipped/ Br encoded Content
Request a publicly available URL, I just grabbed Google Analytics
Attempt to encode and decode the response via an http2 connection with a proxy.modifyresponse function
Watch as content is dropped.
However, this only occurs under the following conditions:
Under SSL, like with https://easy-dp.ngrok.io
When running a proxy.ModifyResponse function
Decompressing and re-compressing the body (for example, just reading and rewriting the body to new bytes works)
package main
import (
"bytes"
"compress/gzip"
"fmt"
"golang.org/x/net/http2"
"golang.org/x/net/http2/h2c"
"io/ioutil"
"net/http"
"net/http/httputil"
"strconv"
"time"
)
func ForwardAnalytics(req *http.Request) {
req.URL.Scheme = "https"
req.URL.Host = "www.google-analytics.com"
req.Host = "www.google-analytics.com"
req.URL.Path = "/analytics.js"
req.Header.Set("Accept-Encoding", "gzip")
}
func ModifyAnalytics(r *http.Response) error {
bytesFromBody, err := ioutil.ReadAll(r.Body)
defer r.Body.Close()
if err != nil {
return nil
}
if r.Header.Get("Content-Encoding") == "gzip" {
gzipReader, err := gzip.NewReader(bytes.NewBuffer(bytesFromBody))
if err != nil {
return nil
}
defer gzipReader.Close()
readableBytes, err := ioutil.ReadAll(gzipReader)
var b bytes.Buffer
gzipWriter, err := gzip.NewWriterLevel(&b, gzip.DefaultCompression)
if err != nil {
return nil
}
defer gzipWriter.Close()
writtenLen, err := gzipWriter.Write(readableBytes)
fmt.Println("Wrote ", writtenLen)
if err != nil {
return nil
}
r.ContentLength = int64(len(readableBytes))
r.Header.Set("Content-Length", strconv.FormatInt(int64(len(readableBytes)), 10))
r.Body = ioutil.NopCloser(&b)
return nil
} else {
return nil
}
}
func handleProxy(w http.ResponseWriter, req *http.Request) {
proxy := httputil.ReverseProxy{
Director: ForwardAnalytics
}
proxy.ModifyResponse = ModifyAnalytics
proxy.ServeHTTP(w, req)
}
func main() {
h2s := &http2.Server{
IdleTimeout: 20 * time.Second,
}
mux := http.NewServeMux()
mux.HandleFunc( "/", handleProxy)
s := &http.Server{
ReadHeaderTimeout: 20 * time.Second,
ReadTimeout: 10 * time.Second,
WriteTimeout: 30 * time.Second,
Addr: "localhost:8456",
Handler: h2c.NewHandler(mux, h2s),
}
s.ListenAndServe()
}
What did you expect to see?
I expect to see the ability to open the bytes, modify them, and update the response body on an H2C connection
What did you see instead?
Two things of note happen:
Chrome gives a nice little error that expands upon what's going on
{"params":{"description":"Server reset stream.","net_error":"ERR_HTTP2_PROTOCOL_ERROR","stream_id":5},"phase":0,"source":{"id":1493828,"start_time":"732370299","type":1},"time":"732375561","type":224},
Under the normal http connection, there's no problem, but under the https connection the script may or may not print out to a certain length. Sometimes it doesn't print at all, sometimes it prints about 30%.
This is a cross browser issue.
The Content-Length header indicates the size of the entity body in the message, in bytes. The size includes any content encodings (the Content-Length of a gzip-compressed text file will be the compressed size, not the original size).
src
I thought I had tried this but kept running into ERR_CONTENT_LENGTH_MISMATCH because of how I was closing my gzip writer. Related Question
Final handler looked like this:
if r.Header.Get("Content-Encoding") == "gzip" {
gzipReader, err := gzip.NewReader(bytes.NewBuffer(bytesFromBody))
if err != nil {
return nil
}
defer gzipReader.Close()
readableBytes, err := ioutil.ReadAll(gzipReader)
var b bytes.Buffer
gzipWriter, err := gzip.NewWriterLevel(&b, gzip.DefaultCompression)
if err != nil {
return nil
}
writtenLen, err := gzipWriter.Write(readableBytes)
gzipWriter.Close() // This was the culprit. It needed to be closed here
fmt.Println("Wrote ", writtenLen)
if err != nil {
return nil
}
r.ContentLength = int64(b.Len())
r.Header.Set("Content-Length", strconv.FormatInt(int64(b.Len()), 10))
r.Body = ioutil.NopCloser(&b)
return nil
}

Resources