I have a multi-user Ruby on Rails web application that can interact with an FTP server via AJAX. The application allows the user to browse an FTP site. Javascript makes an AJAX call which communicates with a server-script that returns a list of files and directories within a given directory.
This works fine. However, each time a directory listing is requested, the server must re-establish a connection with the FTP server, which takes a lot of time. I'm looking for a way to leave the FTP connection open for until some number of timeout seconds.
I could probably do this using threads (though, I'm completely open to other ideas) or some fancy connection-pooling scheme (perhaps via a daemon that manages this).
What are some ways I could persist and regain reference to connections in my ruby source?
Someone suggested using a "Connection: Keep-Alive" header, but I don't see how that would help in this case.
Not a complete answer, but if you did have some sort of daemon or something managing the connection, you could use TCP keepalives to keep the control connection alive for an extended period of time.
FTP uses two connections. A control connection is established client-to-server, and data connections are established server-to-client for each request. So each directory listing or GET would prompt another data connection to be opened for the duration of the request.
You shouldn't worry about keeping lots of listening sockets open because the data connections are negotiated over the control connection just prior to being established. (Also the data connections could be made client-to-server instead of server-to-client by using passive mode if you want, but it's still a separate connection.)
Either way, I think the source of sluggishness is more to do with closing and reopening the control connection (and authenticating) for each request. I think if you have some process that keeps the control connection open using TCP keepalives (SO_KEEPALIVE socket option), you'll see a big improvement.
Related
In a Webserver for basic static website non-blocking event-driven, I don't understand the mechanics I should implement for a "new client".
When a browser connects to my socket, I get the clientfd from accept and answer with a HTTP response, but when the browser is reloaded, should it create a new connection and answer, or should it reuse the same connection and just send the new response?
I use poll to handle multiple fds, but when I reload the page its the same connection (for me this makes sense) but then I open a new tab, and it's still the same connection (It only does accept once). I'm not finding any documentation on this, and I don't have a way to test with multiple client's if it reuses the same one every time.
You can't reuse a connection from another client, new connections must always be accepted as new connections. It doesn't matter what kind of server application you're writing.
However, if the client passes the header Connection: keep-alive you should not close the connection once the response is finished, but keep the connection open for future requests from the same client.
I hope i understand correctly,
but anyway, What i personally do is create a map of sockets, each socket is a client.
Every time a socket disconnects, it's being removed from that map... and so on...
Whether to use a new connection is the browser's choice. You don't get much of a choice.
However, you can tell the browser that you don't allow it to reuse a connection, if you send Connection: close in the response. In this case, the browser is forced to open a new connection for the next request. This is the only control you have.
If you want to test several connections at the same time, you could open several different browsers, or you could use a different program, such as some HTTP load testing tool (there are many). You could also send it a web page with many images; browsers should try to download all the images using several connections at the same time.
A web server doesn't create clients. A web server has clients -- new clients trying to connect, and existing clients communicating on the sockets that it has already opened.
To handle new clients, a web server should pretty much be calling accept all the time, unless it's already handling the maximum number of clients that it's configured to handle.
As soon as you get a new connection from accept, hand it off to other threads to process and call accept again.
How connect to https://api.push.apple.com using http2 with persistent connection ?
Persistent connection is to avoid rapid connection and disconnection:
APNs treats rapid connection and disconnection as a denial-of-service attack
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Chapters/APNsProviderAPI.html
Is writing a client in c using https://nghttp2.org the only solution?
(If that question should be ask in another StackExchange website, please do tell me)
Non-persistent connections are a relic of the past. They were used in HTTP/1.0, but HTTP/1.1 already moved to a model where the connections were persistent by default, and HTTP/2 (also being multiplexed) continues on that model of connections being persistent by default.
Independently on the language you are using to develop your applications, any HTTP/2 compliant client will, by default, use persistent connections.
You only need to use the HTTP/2 client library in a way that you don't explicitly close the connection after every request you make.
Typically these libraries employ a connection pool that keeps the connections open, typically until an idle timeout fires.
When your application makes HTTP requests, the library will pick an open connection and send the request. When the response arrives the library will not close the connection but instead put it back into the pool for the next usage.
Just study how the library you want to use allows you to make multiple requests without closing the connection.
I also met this question!
If the connection be idle for a long time (about 1 hour), then function poll catches no socket status changed. It always returns 0 even as on_frame_send_callback was invoked.
Is there anyone can figure out the problem?
The documentation says:
If you wish to re-use a connection across multiple HTTP requests without automatically closing it you can use ::new instead of ::start. request will automatically open a connection to the server if one is not currently open. You can manually close the connection with finish.
But it's ::start that seems like a way of reusing a connection, isn't it? Additionally, how can I check if connection is open or still open?
I have a client trying to upload multiple files to FTP server in passive mode.
The client sends PASV command and the server responds with the relevant ip and port.
Is it possible to send multiple files on this one data connection? or the client need to send the PASV command and get a new port for each file?
Since the only indicator of the end of file is the close of the connection and because you cannot transfer any more data after the connection has closed, you will not be able to transfer more than one file using the same data connection.
But, maybe you tried to ask a different question, that is if is possible to have multiple data transfers (and thus multiple data connections) after a single PASV command? I can see nothing in RFC959 which directly would prevent this and reusing the same target port on the server. And because access would be done from different source ports on the client this should also not give problems with TCP connection states. But, in practice you will probably see problems because if you try to use this from the client side, because lots of servers create the listener only for a single data connection. So you better precede each data transfer with a new PASV command, like existing clients do.
Is this chat using "long polling" or "http streaming" ?
http://go-mono.com/moonlight/chat.aspx
It's not anything that simple. It uses http://www.mibbit.com/chat, which is a full IRC client written in Javascript and Java. Blog at http://blog.mibbit.com/.
Edit: Here's your answer.
The first part I got working was the communications between browser and server. That’s done using 2 XMLHttpRequests. The first one is simply to send data from browser to server. It utilizes keep-alive, to minimise new connections.
The second XHR is the ‘receive lazy polling’ one. It connects to the server, and the server holds it open until there are messages available, or a timeout expires. This one is also keep-alive, so the next request goes down the same connection.
What you end up with is 2 connections held open to the server, with packets (json in this case), and some http headers from time to time.
To make sure the server would scale, I wrote a custom webserver in java using nio. It handles all of the connections in a single thread and as I say, scales to tens of thousands of connections.
If the client requests a new connection, it sends a request to the webserver, which then connects out, and starts proxying etc. It also runs an ident server in the case of irc connections so that an irc server can identify individual browsers. I looked at existing frameworks etc to do this sort of thing, but I valued learning how it all works, and thought that my use case may be specific enough to be able to optimise more than general frameworks can.