In Go/Golang, once a connection object (Conn) is created with the following code:
conn, err := net.Dial("tcp","33.33.33.33:444")
if err != nil {
// good connection
}
I would like to preserve the conn value for later on verifying if the connection is active. I dont want to re-connect from time to time to check the connection as it causes various TIME_WAITs on the OS, so overall my requirements are:
create a connection
preserve the connection object
capture if the connection drops for any reason
do not send or receive any data
Any thoughts on how to achieve this ? Is there a way to capture that the connection is discontinued without sending or receiving data or reconnecting ?
I don't think it is possible to do without performing an operation. If it is infrequently used, when you try to read you may get an error if the client (or some proxy) closed the connection. If that happens then reconnect and retry.
Many protocols will bake in a heartbeat mechanism to facilitate this kind of thing. Then you can read constantly (with SetDeadline if you want) and know within a heartbeat frame that something went wrong.
For example, I use a redis client that supports connection pooling. When I retrieve an idele connection from the pool, I immediately perform a PING operation. If that succeeds, I know the connection is ready to use. If not, I get another idle one, or connect anew.
Related
I have a verticle, which consumes a message from the event bus and processes it. I have a question as to when the JDBC connection should be closed. There are 2 approaches
Closing the connection once the message is processed. But this will be very expensive because I will open/close connection every time.
Trust that vertx will close the connection when the verticle is stopped/undeployed (which is literally never) and that there wont be any memory leaks as long as the connection is open. I will open the connection in the start() method, so that whenever there is a message it available.
On the other hand, If I have an elastic search backend and I am using the elastic search SDK, which has a specific method to close the client, when should the connection be really closed?
Use a connection pool, that will take away most of the cost of closing/opening connections. When using a connection pool, closing the connection returns it to the connection pool for re-use.
The basic usage pattern is:
try (Connection connection = dataSource.getConnection()) {
// use connection
}
At the end of the block the connection is closed, which - if dataSource has a connection pool - will make it available for re-use.
You can always put your clean up code in Stop() method of Verticle interface. It will be called when the verticle starts it's un-deploy procedure.
See Vert.x Docs
I have a grpc server and client that works as expected most of the time, but do get a "transport is closing" error occasionally:
rpc error: code = Unavailable desc = transport is closing
I'm wondering if it's a problem with my setup. The client is pretty basic
connection, err := grpc.Dial(address, grpc.WithInsecure(), grpc.WithBlock())
pb.NewAppClient(connection)
defer connection.Close()
and calls are made with a timeout like
ctx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
defer cancel()
client.MyGRPCMethod(ctx, params)
One other thing I'm doing is checking the connection to see if it's either open, idle or connecting, and reusing the connection if so. Otherwise, redialing.
Nothing special configuration is happening with the server
grpc.NewServer()
Are there any common mistakes setting up a grpc client/server that I might be making?
After much search, I have finally come to an acceptable and logical solution to this problem.
The root-cause is this: The underlying TCP connection is closed abruptly, but neither the gRPC Client nor Server are 'notified' of this event.
The challenge is at multiple levels:
Kernel's management of TCP sockets
Any intermediary load-balancers/reverse-proxies (by Cloud Providers or otherwise) and how they manage TCP sockets
Your application layer itself and it's networking requirements - whether it can reuse the same connection for future requests not
My solution turned out to be fairly simple:
server = grpc.NewServer(
grpc.KeepaliveParams(keepalive.ServerParameters{
MaxConnectionIdle: 5 * time.Minute, // <--- This fixes it!
}),
)
This ensures that the gRPC server closes the underlying TCP socket gracefully itself before any abrupt kills from the kernel or intermediary servers (AWS and Google Cloud Load Balancers both have larger timeouts than 5 minutes).
The added bonus you will find here is also that any places where you're using multiple connections, any leaks introduced by clients that forget to Close the connection will also not affect your server.
My $0.02: Don't blindly trust any organisation's (even Google's) ability to design and maintain API. This is a classic case of defaults-gone-wrong.
One other thing I'm doing is checking the connection to see if it's either open, idle or connecting, and reusing the connection if so. Otherwise, redialing.
grpc will manage your connections for you, reconnecting when needed, so you should never need to monitor it after creating it unless you have very specific needs.
"transport is closing" has many different reasons for happening; please see the relevant question in our FAQ and let us know if you still have questions: https://github.com/grpc/grpc-go#the-rpc-failed-with-error-code--unavailable-desc--transport-is-closing
I had about the same issue earlier this year . After about 15 minuets I had servers close the connection.
My solution which is working was to create my connection with grpc.Dial once on my main function then create the pb.NewAppClient(connection) on each request. Since the connection was already created latency wasn't an issue. After the request was done I closed the client.
I am using C3P0NativeJdbcExtractor to extract the native JDBC connection as below.
public Connection getNativeConnection() throws SQLException{
C3P0NativeJdbcExtractor nativeJbdc;
nativeJbdc = new C3P0NativeJdbcExtractor();
return nativeJbdc.getNativeConnection(dataSource.getConnection());
}
Note that the data source here is obtained of a C3P0 Connection Pool. When I do a Connection.close() returned on this method, it is actually closing the connection instead of returning to the pool.
However if we close the unwrapped connection, then it is returned to the Pool.
Is there is a reason to why closing the wrapped connection here is failing to return the connection to the pool?
A connection pool like c3p0, holds a collection of physical ('native') connections created by a JDBC driver. When you ask it for a connection, it wraps that physical connection in a proxy, also known as the logical connection.
That proxy will intercept certain methods like Connection.close(). For close() instead of closing the connection, it invalidates the logical connection so it behaves as a closed connection, and it returns the physical connection to the connection pool.
Your code extracts the physical connection from the logical connection, and returns that instead, so if you call close() on that, you actually close the connection to the database instead of returning it to the pool.
You should almost never have a reason to extract the native connection like that. The only reason is when you need access to driver-specific features. You should try to use standard JDBC as much as possible, and only unwrap to access driver-specific features when you really need to.
When you call close(), make sure you call close() on the logical connection that you received from the connection pool, not on the unwrapped physical connection.
I'm studying Go for network programming. The problem is Go documentation is too simple. For example， I don't know when to use net.DialTCP, and when to use TCPListener object to AcceptTCP, what's the difference? How about client communicate with another client? Not client to server.
Connecting
In Go, you use the Dial function from net to connect to a remote machine.
net.Dial("tcp","google.com:80")
net.Dial("udp","tracker.thepiratebay.org:6969")
net.Dial("ip","kremvax.su")
net.Dial("unix","/dev/log")
This gives you an abstract Conn object that represents the connection you just established. Conn implements the ReadWriteCloser interface from io and a couple of other functions. You can use this object to send and receive data.
Listening
To listen, i.e. open a port, you use the Listen function from net. Calling Listen gives you a Listener object. Use Accept to accept incoming connections. Accept returns another Conn object that can be used as above.
ls, err := net.Listen("tcp",":1337")
if err != nil {
// port probably blocked, insert error handling here
}
conn, err := ls.Accept()
if err != nil {
// error handling
}
conn.Write("Hello, world!")
DialTCP and ListenTCP
These functions give you more control over TCP connections. I suggest you to only use them if they are definitly needed for your program as Dial and Listen are simpler, more generic and easily allow you to adapt your program to other types of network connections.
net.DialTCP is used on the client side to create a connection to remote server.
net.TCPListener.AcceptTCP is used on the server side to accept new connection (possibly initiated by net.DialTCP if client is written in Go). Note that listener may accept multiple connections, one by one, thus serving multiple clients at once (e.g. each in different goroutine).
Depending on whether you are writing client or server, you use net.DialTCP or net.TCPListener
Maybe you should learn about network programming in general first? Then these would make more sense I think.
I am working in an environment where we get production issues from time to time related to Oracle connections. We use ODP.NET from ASP.NET applications, and we suspect the firewall closes connections that have been in the connection pool too long.
Sometimes we get an "ORA-12571: TNS packet writer failure" error, and sometimes we get "ORA-03135: connection lost contact."
I was wondering if someone has run into this and/or has an understanding of the difference between the 2 errors.
Using a mobile phone analogy:
ORA-12571 (Failure) Means call is dropped.
ORA-03135 (Connection Lost) Other party hung up.
My understanding is that 3135 occurs when a connection is lost. This doesn't tell you why the connection was lost, though. It may have been terminated by the server because the server failed to recieve a response to a probe for a certain amount of time, and assumed that the connection was dead. Or (I'm not sure about this) the exact reverse of that: the client failed to recieve a probe response from the server for a certain amount of time, so it assumed the connection was lost. The "certain amount of time" is cotrolled by SQLNET.EXPIRE_TIME=[minutes] in sqlnet.ora.
As for 12571, my (again vague) understanding is that there was a sudden failure to send a packet during communication with the server, and that this is typically caused by some software or hardware interfering with the connection (either by design, or by error). For instance, if you pull out your ethernet cable and then try to execute a query, you'll probably get this. Or if a firewall or anti-malware application decides to block the traffic.