In Golang, if we want to write a socket server, we can write like this :
listen, err := net.Listen("tcp", "****")
for {
conn, err := listen.Accept()
...
}
net.Listen() include create socket, bind, listen, and uses epoll in implementation.
in CPP, if we want to write a server, we can chose to use select, poll or epoll freely , so my question is: in Golang, how can I write a server using select or poll instead of `epoll.
If you are familiar with system calls used during connection creation; you will find the syscalls package useful for what you need to do where you can choose to use these system calls as you need. It consists of all the system calls you will need.
I also found this example gist which you can reference for making your own implementation using poll or select.
Related
I am using buffio.Writer in GoLang as follows.
conn, err := net.Dial("tcp", address) // address is of form ip:port
w := bufio.NewWriter(conn)
w.WriteByte(code) // code is a byte
w.Write(data) // data is a byte buffer
w.Flush()
I am trying to modify the above code so that the write() events have a deadline: when the deadline is passed, the write() event will return irrespective of the fact that it was successful or not.
In GoLang its possible to have a deadline, if the conn (connection object) is directly used for writing using conn.SetWriteDeadline(time.Now().Add(n * time.Second)). However, when I use bufifo.writer object, which is essentially a wrapper around conn for buffered IO, there is no API to set a deadline.
While its possible to use conn.SetWriteDeadline(time.Now().Add(n * time.Second)) and use conn.Write(b), its very inefficient since it doesn't buffer the write events (thus a lot of context switches)
Is there a way in GoLang where I can use buffered IO with a writeDeadline()?
Thanks
There are two cases to note here.
If you want to have per write() deadline, then its not possible to use buffering. When you use buffering, then the actual write() is triggered when the buffer is full. So technically its not possible to know when your write() is completed. In this case, you are essentially using conn.write() and you can use conn.SetWriteDeadline(time.Now().Add(n * time.Second)).
In the second case, as #icza has mentioned in the comment, you can set the deadline in the underlying conn object, and the buffio.writer() wrapper will adhere to this rule. While this is semantically correct, it doesn't provide the networking abstraction you want.
How to call RPC endpoints using ethclient.Client ( https://github.com/ethereum/go-ethereum )?
Some methods don't have wrappers, and, as far as i can see, calling it directly is impossible
e.g.
client, err := ethclient.Dial(url)
// ok
client.BalanceAt(...)
// incorrect code, trying to access private field `c *rpc.Client`
client.c.Call("debug_traceTransaction", ...)
The only way i can think of is spinning up totally separate rpc client and keep both running at all times.
Is this the only way?
The ethclient.Dial function (which you mentioned) uses the rpc.DialContext function underneath, and the package also provides an ethclient.NewClient function to create a new ethclient.Client with an existing rpc connection.
A possible solution could be to create a new rpc connection, then pass it to the ethclient.Client, so you're using one connection, but can use the RPC connection itself and the eth client as well.
Something like this:
rpcClient, err := rpc.DialContext(ctx, url)
ethClient := ethclient.NewClient(rpcClient)
// use the ethClient
ethClient.BalanceAt(...)
// access to rpc client
rpcClient.Call(...)
I have been using GORM for my application based on AWS lambda. I used gorm.Open() for every Handler function,
db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{
Logger: logger.Default.LogMode(logger.Info),
})
so can someone help me confirm that does gorm.Open(...) automatically close the connection or not? Or I must use generic database interface bellow?
// Get generic database object sql.DB to use its functions
sqlDB, err := db.DB()
// Ping
sqlDB.Ping()
// Close
sqlDB.Close()
// Returns database statistics
sqlDB.Stats()
A gorm.DB object is intended to be reused, like a sql.DB handle. You rarely have to explicitly close these objects. Just create it once and reuse it.
gorm.DB contains a sql.DB which uses a connection pool to manage the connections. If it is closed, it will stop accepting new queries, wait for running queries to finish and close all connections.
When serving a bidirectional stream in gRPC in golang, the canonical stream handler looks something like this:
func (s *MyServer) MyBidiRPC(stream somepb.MyServer_MyBidiServer) error {
for {
data, err := stream.Recv()
if err == io.EOF {
return nil // clean close
}
if err != nil {
return err // some other error
}
// do things with data here
}
}
Specifically, when the handler for the bidi RPC returns, that is the signal to consider the server side closed.
This is a synchronous programming model -- the server stays blocked inside this goroutine (created by the grpc library) while waiting for messages from the client.
Now, I would like to unblock this Recv() call (which ends up calling RecvMsg() on an underlying grpc.ServerStream,) and return/close the stream, because the server process has decided that it is done with this client.
Unfortunately, I can find no obvious way to do this:
There's no Close() or CloseSend() or CloseRecv() or Shutdown()-like function on the bidi server interface generated for my service
The context inside the stream, which I can get at with stream.Context(), doesn't expose user-accessible the cancel function
I can't find a way to pass in a context on the "starting side" for a new connection accepted by the grpc.Server, where I could inject my own cancel function
I could close the entire grpc.Server by calling Stop(), but that's not what I want to do -- only this particular client connection (grpc.ServerStream) should be finished.
I could send a message to the client that makes the client in turn shut down the conection. However, this doesn't work if the client has fallen off the network, which would be solved with a timeout, which has to be pretty long to be generally robust. I want it now because I'm impatient, and, more importantly, at scale, dangling unresponsive clients can be a high cost.
I could (perhaps) dig through the grpc.ServerStream with reflection until I find the transportStream, and then dig out the cancel function out of that and call it. Or dig through the stream.Context() with reflection, and make my own cancel function reference to call. Neither of these seem well advised for future maintainers.
But surely these can't be the only options? Deciding that a particular client no longer needs to be connected is not magic space-alien science. How do I close this stream such that the Recv() call un-blocks, from the server process side, without involving a round-trip to the client?
Unfortunately I don't think there is a great way to do what you are asking. Depending on your goal, I think you have two options:
Run Recv in a goroutine and return from the bidi handler when you need it to return. This will close the context and unblock Recv. This is obviously suboptimal, as it requires care because you now have code executing outside the scope of the handler's execution. It is, however, the closest answer I can seem to find.
If you are trying to mitigate the impact of misbehaving clients by instituting timeouts, you might be able to offload the work of this to the framework with KeepaliveEnforcementPolicy and/or KeepaliveParams. This is probably preferable if this aligns with the reason you are hoping to close the connection, but otherwise isn't of much use.
I'm writing a TCP server which simultaneously accepts multiple connections from mobile devices and some WiFi devices (IOT). The connections needs to be maintained once established, with the 30 seconds timeout if there is no heartbeat received. So it is something like the following:
// clientsMap map[string] conn
func someFunction() {
conn, err := s.listener.Accept()
// I store the conn in clientsMap
// so I can access it, for brevity not
// shown here, then:
go serve(connn)
}
func serve(conn net.Conn) {
timeoutDuration := 30 * time.Second
conn.SetReadDeadline(time.Now().Add(timeoutDuration))
for {
msgBuffer := make([]byte, 2048)
msgBufferLen, err := conn.Read(msgBuffer)
// do something with the stuff
}
}
So there is one goroutine for each client. And each client, once connected to the server, is pending on the read. The server then processes the stuff read.
The problem is that I sometimes need to read things off one client, and then pass data to another (Between a mobile device and a WiFi device). I have stored the connections in clientsMap. So I can always access that. But since each client is handled by one goroutine, shall I be passing the data from one client to another by using a channel? But if the goroutine is blocked waiting for a pending read, how do I make it also wait for data from a channel? Or shall I just obtain the connection for the other party from the clientsMap and write to it?
The documentation for net.Conn clearly states:
Multiple goroutines may invoke methods on a Conn simultaneously.
So yes, it is okay to simply Write to the connections. You should take care to issue a single Write call per message you want to send. If you call Write more than once you risk interleaving messages from different mobile devices. This implies calling Write directly and not via some other API (in other words don't wrap the connection). For instance, the following would not be safe:
json.NewEncoder(conn).Encode(myValue) // use json.Marshal(myValue) instead
io.Copy(conn, src) // use io.ReadAll(src) instead