I'm studying Go for network programming. The problem is Go documentation is too simple. For example， I don't know when to use net.DialTCP, and when to use TCPListener object to AcceptTCP, what's the difference? How about client communicate with another client? Not client to server.
Connecting
In Go, you use the Dial function from net to connect to a remote machine.
net.Dial("tcp","google.com:80")
net.Dial("udp","tracker.thepiratebay.org:6969")
net.Dial("ip","kremvax.su")
net.Dial("unix","/dev/log")
This gives you an abstract Conn object that represents the connection you just established. Conn implements the ReadWriteCloser interface from io and a couple of other functions. You can use this object to send and receive data.
Listening
To listen, i.e. open a port, you use the Listen function from net. Calling Listen gives you a Listener object. Use Accept to accept incoming connections. Accept returns another Conn object that can be used as above.
ls, err := net.Listen("tcp",":1337")
if err != nil {
// port probably blocked, insert error handling here
}
conn, err := ls.Accept()
if err != nil {
// error handling
}
conn.Write("Hello, world!")
DialTCP and ListenTCP
These functions give you more control over TCP connections. I suggest you to only use them if they are definitly needed for your program as Dial and Listen are simpler, more generic and easily allow you to adapt your program to other types of network connections.
net.DialTCP is used on the client side to create a connection to remote server.
net.TCPListener.AcceptTCP is used on the server side to accept new connection (possibly initiated by net.DialTCP if client is written in Go). Note that listener may accept multiple connections, one by one, thus serving multiple clients at once (e.g. each in different goroutine).
Depending on whether you are writing client or server, you use net.DialTCP or net.TCPListener
Maybe you should learn about network programming in general first? Then these would make more sense I think.
Related
I am working on an application that tries to send some data to a remote server. Once I get the hostname, I get a connection by resolving the hostname using net.Dialer.DialContext. Once, I resolve the hostname, I keep on using conn.Write method to write data to the connection.
conn, err := d.DialContext(ctx, string(transport), addr)
_, err := client.conn.Write([]byte(msg))
Error faced: I observed that due to some issues, I was not able to ping my server. Surprisingly, conn obtained from DialContext did not complain while doing conn.Write and it kept writing to the same connection.
Can someone help me in how to modify my writing methods in order to get an error in case the destination server is not reachable?
From this UDP connection example
the best a "connected" UDP socket can do to simulate a send failure is to save the ICMP response, and return it as an error on the next write.
So try and (for testing) make a second conn.Write, to confirm that you would indeed get an error this time.
Is it possible to use a context with the Dialer returned by the SOCKS5 function from net/proxy library?
If a SOCKS5 proxy Dialer blocks while establishing the connection, an HTTP client using the proxy Dialer could get stuck without a way to abort the connection.
Under the hood of golang.org/x/net/proxy.SOCKS5 it uses golang.org/x/net/internal/socks.Dialer, which has an exported method DialContext, and since Go 1.7, http.Transport supports a custom DialContext as a field. So you can cast the dialer to an interface with method DialContext and set it to a custom transport.
dc := dialer.(interface {
DialContext(ctx context.Context, network, addr string) (net.Conn, error)
})
httpTransport.DialContext = dc.DialContext
playground: https://play.golang.org/p/tSi9IZ-2Zqg
Note: while this is valid Go code, some would argue it relies on implementation details that the package does not want to export (hidden in internal). I think it is best to send an issue to Go team to ask for export it; if it is too much an issue for you, you can either copy the code and preserve it from changing, or use http.Client.Timeout or write start every http.Do concurrently and wraps a select over it for context like dealing with any blocking operation.
I have a grpc server and client that works as expected most of the time, but do get a "transport is closing" error occasionally:
rpc error: code = Unavailable desc = transport is closing
I'm wondering if it's a problem with my setup. The client is pretty basic
connection, err := grpc.Dial(address, grpc.WithInsecure(), grpc.WithBlock())
pb.NewAppClient(connection)
defer connection.Close()
and calls are made with a timeout like
ctx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
defer cancel()
client.MyGRPCMethod(ctx, params)
One other thing I'm doing is checking the connection to see if it's either open, idle or connecting, and reusing the connection if so. Otherwise, redialing.
Nothing special configuration is happening with the server
grpc.NewServer()
Are there any common mistakes setting up a grpc client/server that I might be making?
After much search, I have finally come to an acceptable and logical solution to this problem.
The root-cause is this: The underlying TCP connection is closed abruptly, but neither the gRPC Client nor Server are 'notified' of this event.
The challenge is at multiple levels:
Kernel's management of TCP sockets
Any intermediary load-balancers/reverse-proxies (by Cloud Providers or otherwise) and how they manage TCP sockets
Your application layer itself and it's networking requirements - whether it can reuse the same connection for future requests not
My solution turned out to be fairly simple:
server = grpc.NewServer(
grpc.KeepaliveParams(keepalive.ServerParameters{
MaxConnectionIdle: 5 * time.Minute, // <--- This fixes it!
}),
)
This ensures that the gRPC server closes the underlying TCP socket gracefully itself before any abrupt kills from the kernel or intermediary servers (AWS and Google Cloud Load Balancers both have larger timeouts than 5 minutes).
The added bonus you will find here is also that any places where you're using multiple connections, any leaks introduced by clients that forget to Close the connection will also not affect your server.
My $0.02: Don't blindly trust any organisation's (even Google's) ability to design and maintain API. This is a classic case of defaults-gone-wrong.
One other thing I'm doing is checking the connection to see if it's either open, idle or connecting, and reusing the connection if so. Otherwise, redialing.
grpc will manage your connections for you, reconnecting when needed, so you should never need to monitor it after creating it unless you have very specific needs.
"transport is closing" has many different reasons for happening; please see the relevant question in our FAQ and let us know if you still have questions: https://github.com/grpc/grpc-go#the-rpc-failed-with-error-code--unavailable-desc--transport-is-closing
I had about the same issue earlier this year . After about 15 minuets I had servers close the connection.
My solution which is working was to create my connection with grpc.Dial once on my main function then create the pb.NewAppClient(connection) on each request. Since the connection was already created latency wasn't an issue. After the request was done I closed the client.
I am trying to build a BitTorrent client in go. I need to make UDP requests to connect to the various trackers. For this I use the net package and do this:
net.Dial("udp", "udp://hostname:1337/announce")
I get a "too many colons in address" error.
If I try this:
net.Dial("udp", "hostname:1337/announce")
I get a "nodename nor servname provided, or not known" error.
How do I fix this?
So you'll need to send it to the IP address and port as provided by the .torrent metafile (announce field).
And once you open the net.Conn you can conn.Write() to the socket and similarly conn.Read()
So you've just about gotten i:
conn, err := net.Dial("udp", announceAddr:Port)
When connecting with HTTP, yeah you use the /announce endpoint, but not with UDP
The specs explain how many bytes to read and write (it is fixed at first, but later dynamic when it comes to reading the peer list). I've found this link, rather, the most useful: https://github.com/naim94a/udpt/wiki/The-BitTorrent-UDP-tracker-protocol
In Go/Golang, once a connection object (Conn) is created with the following code:
conn, err := net.Dial("tcp","33.33.33.33:444")
if err != nil {
// good connection
}
I would like to preserve the conn value for later on verifying if the connection is active. I dont want to re-connect from time to time to check the connection as it causes various TIME_WAITs on the OS, so overall my requirements are:
create a connection
preserve the connection object
capture if the connection drops for any reason
do not send or receive any data
Any thoughts on how to achieve this ? Is there a way to capture that the connection is discontinued without sending or receiving data or reconnecting ?
I don't think it is possible to do without performing an operation. If it is infrequently used, when you try to read you may get an error if the client (or some proxy) closed the connection. If that happens then reconnect and retry.
Many protocols will bake in a heartbeat mechanism to facilitate this kind of thing. Then you can read constantly (with SetDeadline if you want) and know within a heartbeat frame that something went wrong.
For example, I use a redis client that supports connection pooling. When I retrieve an idele connection from the pool, I immediately perform a PING operation. If that succeeds, I know the connection is ready to use. If not, I get another idle one, or connect anew.