I am working on an FTP client for kicks and I am trying to understand the workflow of data connections.
As I understand, the initial (command) connection is permanent until you quit. However, I am unsure of the data connection - is it re-initiated per-command? So you call PORT ... or PASV, get a second connection, do a LIST, get the results, connection closes, start over?
Also, do you need to call PASV (or PORT ...) again after each connection closes? It seems that when I try to test some things out using a passive connection, I cannot re-connect to the same port after the first command has returned the results and closed the data connection. I can keep calling PASV -> Data Connect -> Run Command -> Get Results -> Data Connection closed -> PASV, but it seems like it's not how it's meant to run?
Also, if someone has a good material on FTP that is more terse than the RFC I really appreciate it.
You have to open a new connection every time. It's only the closing of the connection, how you (or the server) can tell that the transfer completed (at least in the common "stream mode").
You cannot even reuse the local/remote port number combination, as when a TCP connection is closed, it enters TIME_WAIT mode and the port number combination cannot be used for some time. So for two immediately consecutive transfers you have to use a different port number combination anyway.
Refer to RFC 959, section 3.3. Data management:
Reuse of the Data Connection: When using the stream mode of data
transfer the end of the file must be indicated by closing the
connection. This causes a problem if multiple files are to be
transfered in the session, due to need for TCP to hold the
connection record for a time out period to guarantee the reliable
communication. Thus the connection can not be reopened at once.
There are two solutions to this problem. The first is to
negotiate a non-default port. The second is to use another
transfer mode.
A comment on transfer modes. The stream transfer mode is
inherently unreliable, since one can not determine if the
connection closed prematurely or not. The other transfer modes
(Block, Compressed) do not close the connection to indicate the
end of file. They have enough FTP encoding that the data
connection can be parsed to determine the end of the file.
Thus using these modes one can leave the data connection open
for multiple file transfers.
See also:
Why does FTP passive mode require a port range as opposed to only one port?
How many data channel ports do I need for an FTPS server running behind NAT?
Related
Is there a way to distinguish between a lost connection and an expected timeout on reading from a serial port (/dev/tty... file descriptor) in golang 1.16 on a linux host?
All I get from two different implementations of "serial" for go are a error string holding only "EOF" in both scenarios. I tried os.IsTimeout() but there is no type associated with that error from io.file.
I need to know if I lost connection so I can reconnect (close and reopen the port). Without it is reconnecting on every timeout.
Can someone hint me to a way to distinguish these to scenarios or a different approach for handling a disconnected device?
I'm writing a DLL for a purchased software.
The software will perform multi-threaded calculations on certain tasks.
My job is to output the relative result into a database.
However, due to the limited support of the software, it is kind of difficult to do multi-threaded output of the data.
The key problem is that there is no info on the last execution of the DLL function.
Therefore, the database connection will not be closed.
So may I ask if I leave the connection open and terminate the process, what would be the potential problems?
My platform is winserver 2008, and PostgreSQL 10.
I don't understand the background information you are giving, but I can answer the question:
If a PostgreSQL client process dies without closing the database (and TCP) connection, the PostgreSQL server process (“backend process”) that servers this connection will not realize this immediately.
Of course, as soon as the server tries to communicate to the client, e.g. to return some results, TCP it will notice that the partner has gone away and will return an error.
However, often the backend process is idle, waiting for the client to send the next request. In this case, it would never notice that its partner has died. This could eventually cause max_connections to be exhausted with dead connections.
Because this is a common problem in networking, TCP provides the “keepalive” functionality: when a connection has been idle for a while (2 hours by default), the operating system will send a so-called “keepalive packet” and wait for a response from the other side. Sending keepalive packets is repeated several times (5 times by default) in short intervals (1 second by default), and if no response is received, the connection is closed by the operating system, the backend process receives an error message and terminates.
PostgreSQL provides parameters with which you can configure these settings on the server side: tcp_keepalives_idle, tcp_keepalives_count and tcp_keepalives_interval. If you set tcp_keepalives_idle to a shorter value, dead connections will be detected and removed faster, at the cost of some little added network traffic.
I am relatively new to JMeter however I have been doing Performance testing for almost a decade.
I am working with a proprietary TCP protocol that sends a keep alive periodically - through the existing TCP connection.
I am struggling to understand how I can fork the JMeter 'thread group' to handle a TCP Keep alive received over the same TCP session.
Any ideas?
Thank you brainstrust!
edit: I'm using the TCPsampler and have read the help page. I'll try to provide some more detail shortly about what's happening and how the protocol is written.
edit2: Unfortunately because it's a propriety protocol I cannot reveal the exact nature of the protocol itself but it's largely irrelevant to the problem I'm facing.
Basically, I use the 1st TCP sampler to 'start/authenticate' the session with the server. This is configured the following options:
1. TCPClient classname: LengthPrefixedBinaryTCPClientImpl (my protocol is implemented this standard way)
2. Re-use connection ON.
3. Close connection OFF.
4. Set NoDelay OFF.
5. SO_Linger: nothing
6. Text to send: my hex code for the protocol (this is correct)
I get the response from the first TCP request and then I want to start interacting, however in the session, the server sends a keep alive mid-stream, so occassionally when I send a request, I get an unexpected keep alive response instead (it's an open stream of data).
This is what I would like to solve.
I attempted to use a recursive test fragment, so that on KeepAlive response, it would send the request again however one cannot recurse the test fragments (it throws a Java error on Run attempt).
I hope this gives more context! Thank you for your patience (I'm a newbie SO user!)
Please check the below options if it helps with you sceario:-
If "Re-use connection" is selected, connections are shared between
Samplers in the same thread, provided that the exact same host name
string and port are used. Different hosts/port combinations will use
different connections, as will different threads. If both of "Re-use
connection" and "Close connection" are selected, the socket will be
closed after running the sampler. On the next sampler, another socket
will be created. You may want to close a socket at the end of each
thread loop.
If an error is detected - or "Re-use connection" is not selected - the
socket is closed. Another socket will be reopened on the next sample.
The following properties can be used to control its operation:
tcp.status.prefix text that precedes a status numbertcp.status.suffix
text that follows a status numbertcp.status.properties name of
property file to convert status codes to messagestcp.handler Name of
TCP Handler class (default TCPClientImpl) - only used if not specified
on the GUI
For more details:-https://jmeter.apache.org/usermanual/component_reference.html#TCP_Sampler
The Properties file of a file watcher contains many attributes. I was not able to find the use of certain attributes. The attributes are
FTPModeBinary and
FTPConnectionPASV
these take true or false as its value.
Can anyone please help me to understand the use of these attributes?
"Binary" (as opposed to "text") transfers the file exactly as-is without doing any end-of-line conversions (like LF to CR/LF).
"Passive" means that the FTP server always opens a listening socket for the data connection and the client connects to it. (FTP uses a separate TCP channel for the actual data transfer.) Traditionally, FTP would have the sender open a connection to the receiver but this began breaking when people started residing behind NAT firewalls as there's no easy way to open a connection back to the client in that case. Without passive, it's actually possible to instruct two FTP servers to transfer a file between themselves without ever passing through the client.
Why does Apache-Commons-Net's FTPClient sometimes make the wrong computation for the port number in the PORT command? This is in active mode. For example FTPClient it could send out
PORT <some>,<ip>,<address>,<here>,235,181 when in fact the port number used is 60340. What's the cause for this wrong computation?
This could happen on version 3.3.
I know ftpClient.enterLocalPassiveMode(); could solve this, but I want to know the part where the active mode doesn't work as expected.
From your comments, I assume you mistake an FTP control connection with a data connection.
I assume that the 60340 is local port of the FTP control connection. When opening data connection, 60341 is assigned (hence the PORT ...,235,181).
Reasoning: In an FTP active mode, the client opens listening port for the expected data connection, which it then reports to the server via PORT command over an existing control connection. If the server cannot connect to the port, no TCP/IP packet can ever come to that port. As you claim that the "two machines still communicate at port 60340", it must be the control connection. There cannot be any communication on port, if the connection failed ("Can't open data connection").
The actual cause of the "Can't open data connection" error is likely that you are behind a firewall, so the server cannot connect back to the client. What is a common nowadays. That's what passive mode is good for.