golang: serving net.Conn using a gin router - go

I have a function that handles an incoming TCP connection:
func Handle(conn net.Conn) error {
// ...
}
Also, I have an initialized gin router with implemented handles:
router := gin.New()
router.GET(...)
router.POST(...)
The router.Run(addr) call will start a separate HTTP server on the addr.
Is there any way to handle incoming connections inside the Handle function using this router without running a separate HTTP server?

Create a net.Listener implementation that accepts connections by receiving on a channel:
type listener struct {
ch chan net.Conn
addr net.Addr
}
// newListener creates a channel listener. The addr argument
// is the listener's network address.
func newListener(addr net.Addr) *listener {
return &listener{
ch: make(chan net.Conn),
addr: addr,
}
}
func (l *listener) Accept() (net.Conn, error) {
c, ok := <-l.ch
if !ok {
return nil, errors.New("closed")
}
return c, nil
}
func (l *listener) Close() error { return nil }
func (l *listener) Addr() net.Addr { return l.addr }
Handle connections by sending to the channel:
func (l *listener) Handle(c net.Conn) error {
l.ch <- c
return nil
}
Here's how to tie it all together:
Create the listener:
s := newListener(someAddr)
Configure the Gin engine as usual.
router := gin.New()
router.GET(...)
router.POST(...)
Run the net/http server in a goroutine using the channel listener and the Gin engine:
err := http.Serve(s, router)
if err != nil {
// handle error
}
In your dispatching code, call the s.Handle(c) to pass the connection to the net/http server and then on to the Gin engine.

For those who have a similar task - handle TCP connections from multiple ports using a single router, here's a workaround that I eventually found. Instead of running an HTTP server on a port, I run it with a UNIX socket using router.RunUnix(socketName). The full solution consists of three steps:
Run a HTTP server to listen through a UNIX socket using router.RunUnix(socketName)
Inside the Handle function read the incoming bytes from the connection and send them to the socket. After that, read the HTTP response from the socket and write it into the connection.
Close the connection.

Related

Connections stuck at CLOSE_WAIT in golang server

I am using gorilla mux to create a golang server to support a simple health GET endpoint.
The endpoint responds with a status of ok whenver the server is up.
I see a lot of connections (over 400) in CLOSE_WAIT state on one system.
This does not happen on other systems with the same code.
Output of netstat (9003 is my server port):
tcp 164 0 ::1:9003 ::1:60702 CLOSE_WAIT -
tcp 164 0 ::1:9003 ::1:44472 CLOSE_WAIT -
tcp 164 0 ::1:9003 ::1:31504 CLOSE_WAIT -
This seems to imply that I have a connection I need to close.
Most of the questions I read online seem to suggest that open connections pertain to the client not issuing a response.body.close() after a GET.
As per https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/, I could add read/write timeouts on server side but I would like to understand the root cause of CLOSE_WAITS before adding the improvements.
Am I missing any close on the server side?
My code is below:
import "github.com/gorilla/mux"
...
func (server *Srvr) healthHandler(w http.ResponseWriter, r *http.Request) {
resp := map[string]string{"status": "ok"}
respJSON, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "Error creating JSON response %s", err)
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write(respJSON)
}
// Load initializes the servers
func Load(port string) *Srvr {
srvrPort := ":" + port
log.Infof("Will listen on port %s", srvrPort)
serverMux := mux.NewRouter()
srvr := &Srvr{Port: port, Srv: &http.Server{Addr: srvrPort, Handler: serverMux}}
serverMux.HandleFunc("/api/v1.0/health", srvr.healthHandler).Methods("GET")
return srvr
}
// Run starts the server
func (server *Srvr) Run() {
log.Info("Starting the server")
// Starting a server this way to allow for shutdown.
// https://stackoverflow.com/questions/39320025/how-to-stop-http-listenandserve
err := server.Srv.ListenAndServe()
if err != http.ErrServerClosed {
log.Fatalf("ListenAndServe(): %s", err)
}
}
// Main resides outside the server package
func main() {
srvr := server.Load("9003")
// Now that all setup is done successfully, lets start the server
go srvr.Run()
// An unrelated forever loop executes below for different business logic
for {
glog.Info("Evaluation iteration begins now")
...
time.Sleep(time.Duration(evalFreq) * time.Minute)
}
}

Wait for gin HTTP server to start

We are using gin to expose some REST APIs in production. Now I have to do some stuff once the HTTP server starts.
I am not very familiar with channels, but given below code is what I'm trying to do. Once the startHTPPRouter() starts the HTTP service, I want to send a signal to main(). Based on that signal I want to do some other stuffs.
Please let me know what wrong I'm doing in the given below code.
func startHTTPRouter(routerChannel chan bool){
router := gin.New()
// Many REST API routes definitions
router.Run("<port>")
routerChannel <- true // Is this gonna work ? Because Run() again launches a go routine for Serve()
}
func main() {
routerChannel := make(chan bool)
defer close(routerChannel)
go startHTTPRouter(routerChannel )
for {
select {
case <-routerChannel:
doStuff() // Only when the REST APIs are available.
time.Sleep(time.Second * 5)
default:
log.Info("Waiting for router channel...")
time.Sleep(time.Second * 5)
}
}
}
gin.New().Run() is blocking API. gin server is not returned until exit.
func startHTTPRouter(routerChannel chan bool) {
router := gin.New()
router.Run("<port>")
routerChannel <- true // Is this gonna work ? Because Run() again launches a go routine for Serve()
}
Below is gin'Run() API. https://github.com/gin-gonic/gin/blob/master/gin.go
// Run attaches the router to a http.Server and starts listening and serving HTTP requests.
// It is a shortcut for http.ListenAndServe(addr, router)
// Note: this method will block the calling goroutine indefinitely unless an error happens.
func (engine *Engine) Run(addr ...string) (err error) {
defer func() { debugPrintError(err) }()
address := resolveAddress(addr)
debugPrint("Listening and serving HTTP on %s\n", address)
err = http.ListenAndServe(address, engine)
return
}

grpc go : how to know in server side, when client closes the connection

I am using grpc go
i have an rpc which looks roughly like this
196 service MyService {
197 // Operation 1
198 rpc Operation1(OperationRequest) returns (OperationResponse) {
199 option (google.api.http) = {
200 post: "/apiver/myser/oper1"
201 body: "*"
202 };
203 }
Client connects by using grpc.Dial() method
When a client connects, the server does some book keeping. when the client disconnects, the bookkeeping needs to be removed.
is there any callback that can be registered which can be used to know that client has closed the session.
Based on your code, it's an unary rpc call, the client connect to server for only one time, send a request and get a response. The client will wait for the response until timeout.
In server side streaming, you can get the client disconnect from
<-grpc.ServerStream.Context.Done()
signal.
With that above, you can implement your own channel in a go routine to build your logic. Use select statement as:
select {
case <-srv.Context().Done():
return
case res := <-<YOUR OWN CHANNEL, WITH RECEIVED RESQUEST OR YOUR RESPONSE>
....
}
I provide some detailed code here
In client streaming, besides the above signal, you can check whether the server can receive the msg:
req, err := grpc.ServerStream.Recv()
if err == io.EOF {
break
} else if err != nil {
return err
}
Assuming that the server is implemented in go, there's an API on the *grpc.ClientConn that reports state changes in the connection.
func (cc *ClientConn) WaitForStateChange(ctx context.Context, sourceState connectivity.State) bool
https://godoc.org/google.golang.org/grpc#ClientConn.WaitForStateChange
These are the docs on each of the connectivity.State
https://github.com/grpc/grpc/blob/master/doc/connectivity-semantics-and-api.md
If you need to expose a channel that you can listen to for the client closing the connection then you could do something like this:
func connectionOnState(ctx context.Context, conn *grpc.ClientConn, states ...connectivity.State) <-chan struct{} {
done := make(chan struct{})
go func() {
// any return from this func will close the channel
defer close(done)
// continue checking for state change
// until one of break states is found
for {
change := conn.WaitForStateChange(ctx, conn.GetState())
if !change {
// ctx is done, return
// something upstream is cancelling
return
}
currentState := conn.GetState()
for _, s := range states {
if currentState == s {
// matches one of the states passed
// return, closing the done channel
return
}
}
}
}()
return done
}
If you only want to consider connections that are shutting down or shutdown, then you could call it like so:
// any receives from shutdownCh will mean the state Shutdown
shutdownCh := connectionOnState(ctx, conn, connectivity.Shutdown)
as the github issue:link
you can do like this
err = stream.Context().Err()
if err != nil {
break
}

Gorilla websocket error: close 1007 Illegal UTF-8 Sequence

I'm trying to implement a websocket proxy server for GlassFish. If I try to connect more than one client I'm getting error:
ReadMessage Failed: websocket: close 1007 Illegal UTF-8 Sequence.
I'm sure the GlassFish server sending right data, because the same server works properly with another proxy server implemented with node.js.
func GlassFishHandler(conn *websocket.Conn){
defer conn.Close()
conn.SetReadDeadline(time.Now().Add(1000 * time.Second))
conn.SetWriteDeadline(time.Now().Add(1000 * time.Second))
fmt.Println("WS-GOLANG PROXY SERVER: Connected to GlassFish")
for {
messageType, reader, err := conn.NextReader()
if err != nil {
fmt.Println("ReadMessage Failed: ", err) // <- error here
} else {
message, err := ioutil.ReadAll(reader)
if (err == nil && messageType == websocket.TextMessage){
var dat map[string]interface{}
if err := json.Unmarshal(message, &dat); err != nil {
panic(err)
}
// get client destination id
clientId := dat["target"].(string)
fmt.Println("Msg from GlassFish for Client: ", dat);
// pass through
clients[clientId].WriteMessage(websocket.TextMessage, message)
}
}
}
}
Summing up my comments as an answer:
When you are writing to the client, you are taking the clientId from the GlassFish message, fetching the client from a map, and then writing to it - basically clients[clientId].WriteMessage(...).
While your map access can be thread safe, writing is not, as this can be seen as:
// map access - can be safe if you're using a concurrent map
client := clients[clientId]
// writing to a client, not protected at all
client.WriteMessage(...)
So what's probably happening is that two separate goroutines are writing to the same client at the same time. You should protect your client from it by adding a mutex in the WriteMessage method implementation.
BTW actually instead of protecting this method with a mutex, a better, more "go-ish" approach would be to use a channel to write the message, and a goroutine per client that consumes from the channel and writes to the actual socket.
So in the client struct I'd do something like this:
type message struct {
msgtype string
msg string
}
type client struct {
...
msgqueue chan *message
}
func (c *client)WriteMessage(messageType, messageText string) {
// I'm simplifying here, but you get the idea
c.msgqueue <- &message{msgtype: messageType, msg: messageText}
}
func (c *client)writeLoop() {
go func() {
for msg := ragne c.msgqueue {
c.actuallyWriteMessage(msg)
}
}()
}
and when creating a new client instance, just launch the write loop

RPC from both client and server in Go

Is it actually possible to do RPC calls from a server to a client with the net/rpc package in Go? If no, is there a better solution out there?
I am currently using thrift (thrift4go) for server->client and client->server RPC functionality. By default, thrift does only client->server calls just like net/rpc. As I also required server->client communication, I did some research and found bidi-thrift. Bidi-thrift explains how to connect a java server + java client to have bidirectional thrift communication.
What bidi-thrift does, and it's limitations.
A TCP connection has an incomming and outgoing communication line (RC and TX). The idea of bidi-thrift is to split RS and TX and provide these to a server(processor) and client(remote) on both client-application and server-application. I found this to be hard to do in Go. Also, this way there is no "response" possible (the response line is in use). Therefore, all methods in the service's must be "oneway void". (fire and forget, call gives no result).
The solution
I changed the idea of bidi-thrift and made the client open two connections to the server, A and B. The first connection(A) is used to perform client -> server communication (where client makes the calls, as usual). The second connection(B) is 'hijacked', and connected to a server(processor) on the client, while it is connected to a client(remote) on the server. I've got this working with a Go server and a Java client. It works very well. It's fast and reliable (just like normal thrift is).
Some sources.. The B connection (server->client) is set up like this:
Go server
// factories
framedTransportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
// create socket listener
addr, err := net.ResolveTCPAddr("tcp", "127.0.0.1:9091")
if err != nil {
log.Print("Error resolving address: ", err.Error(), "\n")
return
}
serverTransport, err := thrift.NewTServerSocketAddr(addr)
if err != nil {
log.Print("Error creating server socket: ", err.Error(), "\n")
return
}
// Start the server to listen for connections
log.Print("Starting the server for B communication (server->client) on ", addr, "\n")
err = serverTransport.Listen()
if err != nil {
log.Print("Error during B server: ", err.Error(), "\n")
return //err
}
// Accept new connections and handle those
for {
transport, err := serverTransport.Accept()
if err != nil {
return //err
}
if transport != nil {
// Each transport is handled in a goroutine so the server is availiable again.
go func() {
useTransport := framedTransportFactory.GetTransport(transport)
client := worldclient.NewWorldClientClientFactory(useTransport, protocolFactory)
// Thats it!
// Lets do something with the connction
result, err := client.Hello()
if err != nil {
log.Printf("Errror when calling Hello on client: %s\n", err)
}
// client.CallSomething()
}()
}
}
Java client
// preparations for B connection
TTransportFactory transportFactory = new TTransportFactory();
TProtocolFactory protocolFactory = new TBinaryProtocol.Factory();
YourServiceProcessor processor = new YourService.Processor<YourServiceProcessor>(new YourServiceProcessor(this));
/* Create thrift connection for B calls (server -> client) */
try {
// create the transport
final TTransport transport = new TSocket("127.0.0.1", 9091);
// open the transport
transport.open();
// add framing to the transport layer
final TTransport framedTransport = new TFramedTransport(transportFactory.getTransport(transport));
// connect framed transports to protocols
final TProtocol protocol = protocolFactory.getProtocol(framedTransport);
// let the processor handle the requests in new Thread
new Thread() {
public void run() {
try {
while (processor.process(protocol, protocol)) {}
} catch (TException e) {
e.printStackTrace();
} catch (NullPointerException e) {
e.printStackTrace();
}
}
}.start();
} catch(Exception e) {
e.printStackTrace();
}
I came across rpc2 which implements it. An example:
Server.go
// server.go
package main
import (
"net"
"github.com/cenkalti/rpc2"
"fmt"
)
type Args struct{ A, B int }
type Reply int
func main(){
srv := rpc2.NewServer()
srv.Handle("add", func(client *rpc2.Client, args *Args, reply *Reply) error{
// Reversed call (server to client)
var rep Reply
client.Call("mult", Args{2, 3}, &rep)
fmt.Println("mult result:", rep)
*reply = Reply(args.A + args.B)
return nil
})
lis, _ := net.Listen("tcp", "127.0.0.1:5000")
srv.Accept(lis)
}
Client.go
// client.go
package main
import (
"fmt"
"github.com/cenkalti/rpc2"
"net"
)
type Args struct{ A, B int }
type Reply int
func main(){
conn, _ := net.Dial("tcp", "127.0.0.1:5000")
clt := rpc2.NewClient(conn)
clt.Handle("mult", func(client *rpc2.Client, args *Args, reply *Reply) error {
*reply = Reply(args.A * args.B)
return nil
})
go clt.Run()
var rep Reply
clt.Call("add", Args{5, 2}, &rep)
fmt.Println("add result:", rep)
}
RPC is a (remote) service. Whenever some computer requests a remote service then it is acting as a client asking the server to provide the service. Within this "definition" the concept of a server calling client RPC has no well defined meaning.

Resources