Currently, I am using http GET requests to connect to a server.
Get request is calling from inside a thread. For each Get request one thread is being used but problem is for each Get request, one connection is established. Thus if there are 10000 Get request then 10000 connections will be established. However, I want first to establishe a TLS connection between me and the server, then create a thread and from that thread I want to send Get over that already established connection.
e.g.
for {
1. establish a tls connection
2. create thread go func()
}
func() {
resp, err := http.Get(url) // should be over already established tls connection
}
Firstly, just to clarify, Go has goroutines, not threads. Goroutines are co-operative light-weight processes. The Go runtime time-slices a number of goroutines onto the available operating system threads. This thread abstraction is an important feature of Go.
On the question of making many concurrent GET requests using goroutines, it is best initially to let the standard API handle multiplexing of requests onto connections for you. There may be a large number of requests and a smaller number of supporting connections. Provided you use keep-alive connections, you should not need to care about the details.
For control over proxies, TLS configuration, keep-alives, compression, and other settings, create a Transport
HTTP/1.1 handles keep-alive connections with both http and https. This is a benefit when many requests are made to the same server. As long as you don't force it to close each connection after each request, you will get a lot of benefit from keep-alive connections in your case.
The usual mantra applies: don't optimise prematurely. Describe what you need to do as clearly as possible. Benchmark it (Go has a useful micro-benchmark tool). Do this before you decide whether or how to optimise the performance.
Related
I'm using this amazing SSE Server in Golang (https://github.com/r3labs/sse) on Heroku.
There is a timeout limit there: https://devcenter.heroku.com/articles/request-timeout#long-polling-and-streaming-responses:
If you’re sending a streaming response, such as with server-sent events, you’ll need to detect when the client has hung up, and make sure your app server closes the connection promptly. If the server keeps the connection open for 55 seconds without sending any data, you’ll see a request timeout.
I know in the WebSocket world there is the concept of Keep-Alive ping.
I'm trying with this code:
go func() {
for range time.Tick(time.Second * 1) {
fmt.Println("a second")
sseServer.Publish("test", &sse.Event{Data: []byte("ping")})
}
}()
using a simple server like this:
httpServer := &http.Server{
//...
ReadTimeout: "10s",
WriteTimeout: "10s",
}
but it doesn't work. The call is closed after 10 seconds (and 10 pings).
I think it will fail on Heroku too.
Where am I wrong?
Can we change the timeout for these SSE calls only?
Is there another way?
UPDATE
I don't need accurate detection of disconnected client, I don't care at all. I'm using it to refresh a dashboard when something happens on server.
I prefer not to use WebSocket for something so easy.
I think I have not explained myself well, because I would like to use ping not so much for detecting disconnected clients but because I would like the connection not to be interrupted (as is happening on Heroku).
THE SITUATION RIGHT NOW
LOCALLY
If I remove the ReadTimeout field totally on Windows but also on Docker linux container the connection does not stop and everything works fine.
ON HEROKU
Since on Heroku the connection drops every 55 seconds for that timeout I told you in the first post I tried that loop with that very simple code and it works: the SSE calls are not closed anymore!
THE REMAINING ISSUE
How to have in any case a default ReadTimeout for all other calls (not SSE); I think it's best practice to set a default ReadTimeout.
How to do?
You do not need the ReadTimeout; the server will never read anything from the client after the initial EventSource/Server Sent Events (SSE) connection.
Thus, it is not a best practice to set a default read timeout with an SSE connection, because that read timeout will always get hit. You can't ever send more data back up through the initial SSE GET request.
You should think about SSE as basically a GET request that simply never closes, because that's almost literally what it is. That means that it works great through most proxy servers, and where it doesn't (where the proxy server applies its own timeouts), the client side will automatically reconnect, which is actually a very nice feature that is not found in websockets (although most websocket client libraries do implement it).
You might want to read through this article to learn some more of the great (and not-so-great) things about SSE: https://www.smashingmagazine.com/2018/02/sse-websockets-data-flow-http2/
With regards to your other question, you're probably looking for your HTTP routing library that will allow you to apply timeouts to some GET requests, and not others, but the question is why; if you are trying to protect from a resource drain, you should apply that protection evenly across all endpoints.
With using HTTP, I can develop server-to-server communication code easily. For example,
At Server1,
func1()
{
curl("https://server2/func2");
}
[route("func3")]
func3()
{
// finished!
}
At Server2,
[route("func2")]
func2()
{
curl("https://server1/func3");
}
Using these two (curl and http response code), I don't need to write to socket connection or disconnection or socket accept. Yes, it is convenient.
However, HTTP is heavy for me. Performance matters. I don't need HTTP nor SSL. HTTP requests involves one time TCP connection and disconnect for each request.
I am looking for a simple networking module like this:
Internally, If the server-to-server communication does not exists, establish one just-in-time, then send a message. If it exists, reuse it.
Internally, If the server-to-server communication is unused for a long time, disconnect it.
A server simply sends a message to other server, regardless of the connection exists or not.
There is no relay or broker process between servers. Servers communicate directly.
I found ZeroMQ. ZeroMQ provides many networking patterns with fancy name, for example, Titanic or Espresso. I looked for which fulfills what are described above in ZeroMQ, but I cannot find it. Lazy Pirate seems to be a similar one, but it involves connection and disconnection statements.
With ZeroMQ, can I do it?
Just starting to use Spring Webflux Webclient,Just wanted to know what is the default KeepAlive time for the Http Connection ? Is there a way to increase the keepAlive Time? In our Rest Service we get a request probably every five minutes,The request takes long time to process .It takes time between 500 seconds-- 10 second. However in load test if I send frequent requests the processing time is less than 250ms.
Spring WebFlux WebClient is an HTTP client API that wraps actual HTTP libraries - so configuration like connection management, timeouts, etc. are configured at the library level directly and behavior might change depending on the chosen library.
The default library with WebClient is Reactor Netty.
Many HTTP clients (and this is the case with Reactor Netty) are maintaining HTTP connections in a connection pool to reuse them. Clients usually acquire a new connection to a remote host, use it to send/receive information and then put it back in the connection pool. This is very useful since sometimes acquiring a new connection can be costly. This seems to be really costly in your case.
HTTP clients leave those unused connections in the pool, but what about keepAlive time?
Most clients leave those connections in the pool as long as possible and test them before acquiring them to see if they're still valid or listen to server events asynchronously to remove them from the pool (I believe Reactor Netty does that). So ultimately, the server is in control and decides when to close connections if they're inactive.
Now your problem description might suggest that connecting to that remote host is very costly, but it could be also the remote host taking a long time to respond to your requests (for example, it might be operating on an empty cache and needs to calculate a lot of things).
How connect to https://api.push.apple.com using http2 with persistent connection ?
Persistent connection is to avoid rapid connection and disconnection:
APNs treats rapid connection and disconnection as a denial-of-service attack
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Chapters/APNsProviderAPI.html
Is writing a client in c using https://nghttp2.org the only solution?
(If that question should be ask in another StackExchange website, please do tell me)
Non-persistent connections are a relic of the past. They were used in HTTP/1.0, but HTTP/1.1 already moved to a model where the connections were persistent by default, and HTTP/2 (also being multiplexed) continues on that model of connections being persistent by default.
Independently on the language you are using to develop your applications, any HTTP/2 compliant client will, by default, use persistent connections.
You only need to use the HTTP/2 client library in a way that you don't explicitly close the connection after every request you make.
Typically these libraries employ a connection pool that keeps the connections open, typically until an idle timeout fires.
When your application makes HTTP requests, the library will pick an open connection and send the request. When the response arrives the library will not close the connection but instead put it back into the pool for the next usage.
Just study how the library you want to use allows you to make multiple requests without closing the connection.
I also met this question!
If the connection be idle for a long time (about 1 hour), then function poll catches no socket status changed. It always returns 0 even as on_frame_send_callback was invoked.
Is there anyone can figure out the problem?
I have a service, foo, running on machine A. I need to access that service from machine B. One way is to launch a web server on A and do it via HTTP; code running under web server on A accesses foo and returns the results. Another is to write socket server on A; socket server access service foo and returns the result.
HTTP connection initiation and handshake is expensive; sockets can be written, but I want to avoid that. What other options are available for high performance remote calls?
HTTP is just the protocol over the socket. If you are using TCP/IP networks, you are going to be using a socket. The HTTP connection initiation and handshake are not the expensive bits, it's TCP initiation that's really the expensive bit.
If you use HTTP 1.1, you can use persistent connections (Keep-Alive), which drastically reduces this cost, closer to that of keeping a persistent socket open.
It all depends on whether you want/need the higher-level protocol. Using HTTP means you will be able to do things like consume this service from a lot more clients, while writing much less documentation (if you write your own protocol, you will have to document it). HTTP servers also supports things like authentication, cookies, logging, out of the box. If you don't need these sorts of capabilities, then HTTP might be a waste. But I've seen few projects that don't need at least some of these things.
Adding to the answer of #Rob, as the question is not precisely pointing to an application or performance boundaries, it would be good to look into the options available in a broader context, which is Inter process communication.
The wikipedia page cleanly lists down the options available and would be a good place to start with.
What technology are you going to use? Let me answer for Java world.
If your request rate is below 100/sec, you should not care about optimizations and use most versatile solution - HTTP.
Well-written asynchronous server like Netty-HTTP can easily handle 1000 requests per second on medium-level hardware.
If you need more or have constrained resources, you can go to binary format. Most popular one out there is Google Protobuf(multilanguage) + Netty (Java).
What you should know about HTTP performance:
Http can use Keep-Alive which removes reconnection cost for every request
Http adds traffic overhead for every request and response - around 50-100 bytes.
Http client and server consumes additional CPU for parsing HTTP headers - that is noticeable after abovementioned 100 req/sec
Be careful when selecting technology. Even in 21 century it is hard to find well-written HTTP server and client.