TLS/SSL session resume on FTP transfer connection with OpenSSL - session

I'm open source developer implementing FTP client (WinSCP).
I'm trying to resume TLS/SSL session from the FTP control socket on the transfer socket.
Some FTP servers started to require this.
E.g. vsftpd:
https://scarybeastsecurity.blogspot.com/2009/02/vsftpd-210-released.html
I'm using OpenSSL to implement SSL layer.
I've tried the obvious way to implement the session resume, i.e. to use SSL_get1_session and SSL_set_session, like here:
https://www.linuxjournal.com/article/5487
Though it does not work. I'm still not able to connect to any FTP server requiring TLS session resume (like the vsftpd).
I have suspicion that the problem may be due to in my case, there are two parallel TLS connections, which cannot share the same TLS session. Which is different to the example on linuxjournal.com, where the first connection is closed before the other is opened.
I have also tried several ways to clone the session, e.g. using i2d_SSL_SESSION/d2i_SSL_SESSION. Didn't help either.
I'm really stuck here.
Thanks in advance for any help.

Using the SSL_get1_session and the SSL_set_session worked in the end. I must have used them incorrectly when trying the first time.
Once the TLS/SSL session on the control connection is established, use SSL_get1_session to retrieve the session.
I specifically do it from a callback set by the SSL_set_info_callback, when where & SSL_ST_CONNECT.
But for TLS 1.3 (SSL_version >= TLS1_3_VERSION), I had to use SSL_CTX_set_session_cache_mode with SSL_SESS_CACHE_CLIENT | SSL_SESS_CACHE_NO_INTERNAL_STORE | SSL_SESS_CACHE_NO_AUTO_CLEAR, and use a callback set by SSL_CTX_sess_set_new_cb.
Call the SSL_set_session with the reference to the control connection session, when setting up TLS/SSL session for the data connection.

You must specifically enable client session caching on your SSL_CTX object with:
SSL_CTX_set_session_cache_mode(ctx, SSL_SESS_CACHE_CLIENT);
You may also need to increase the default session cache timeout (the default is 300 seconds), using SSL_CTX_set_timeout().
(You must also be creating your SSL objects from the same SSL_CTX object).

Related

Oracle JDBC intermittent connection reset SQLRecoverableException [duplicate]

I am getting the following error trying to read from a socket. I'm doing a readInt() on that InputStream, and I am getting this error. Perusing the documentation this suggests that the client part of the connection closed the connection. In this scenario, I am the server.
I have access to the client log files and it is not closing the connection, and in fact its log files suggest I am closing the connection. So does anybody have an idea why this is happening? What else to check for? Does this arise when there are local resources that are perhaps reaching thresholds?
I do note that I have the following line:
socket.setSoTimeout(10000);
just prior to the readInt(). There is a reason for this (long story), but just curious, are there circumstances under which this might lead to the indicated error? I have the server running in my IDE, and I happened to leave my IDE stuck on a breakpoint, and I then noticed the exact same errors begin appearing in my own logs in my IDE.
Anyway, just mentioning it, hopefully not a red herring. :-(
There are several possible causes.
The other end has deliberately reset the connection, in a way which I will not document here. It is rare, and generally incorrect, for application software to do this, but it is not unknown for commercial software.
More commonly, it is caused by writing to a connection that the other end has already closed normally. In other words an application protocol error.
It can also be caused by closing a socket when there is unread data in the socket receive buffer.
In Windows, 'software caused connection abort', which is not the same as 'connection reset', is caused by network problems sending from your end. There's a Microsoft knowledge base article about this.
Connection reset simply means that a TCP RST was received. This happens when your peer receives data that it can't process, and there can be various reasons for that.
The simplest is when you close the socket, and then write more data on the output stream. By closing the socket, you told your peer that you are done talking, and it can forget about your connection. When you send more data on that stream anyway, the peer rejects it with an RST to let you know it isn't listening.
In other cases, an intervening firewall or even the remote host itself might "forget" about your TCP connection. This could happen if you don't send any data for a long time (2 hours is a common time-out), or because the peer was rebooted and lost its information about active connections. Sending data on one of these defunct connections will cause a RST too.
Update in response to additional information:
Take a close look at your handling of the SocketTimeoutException. This exception is raised if the configured timeout is exceeded while blocked on a socket operation. The state of the socket itself is not changed when this exception is thrown, but if your exception handler closes the socket, and then tries to write to it, you'll be in a connection reset condition. setSoTimeout() is meant to give you a clean way to break out of a read() operation that might otherwise block forever, without doing dirty things like closing the socket from another thread.
Whenever I have had odd issues like this, I usually sit down with a tool like WireShark and look at the raw data being passed back and forth. You might be surprised where things are being disconnected, and you are only being notified when you try and read.
You should inspect full trace very carefully,
I've a server socket application and fixed a java.net.SocketException: Connection reset case.
In my case it happens while reading from a clientSocket Socket object which is closed its connection because of some reason. (Network lost,firewall or application crash or intended close)
Actually I was re-establishing connection when I got an error while reading from this Socket object.
Socket clientSocket = ServerSocket.accept();
is = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
int readed = is.read(); // WHERE ERROR STARTS !!!
The interesting thing is for my JAVA Socket if a client connects to my ServerSocket and close its connection without sending anything is.read() is being called repeatedly.It seems because of being in an infinite while loop for reading from this socket you try to read from a closed connection.
If you use something like below for read operation;
while(true)
{
Receive();
}
Then you get a stackTrace something like below on and on
java.net.SocketException: Socket is closed
at java.net.ServerSocket.accept(ServerSocket.java:494)
What I did is just closing ServerSocket and renewing my connection and waiting for further incoming client connections
String Receive() throws Exception
{
try {
int readed = is.read();
....
}catch(Exception e)
{
tryReConnect();
logit(); //etc
}
//...
}
This reestablises my connection for unknown client socket losts
private void tryReConnect()
{
try
{
ServerSocket.close();
//empty my old lost connection and let it get by garbage col. immediately
clientSocket=null;
System.gc();
//Wait a new client Socket connection and address this to my local variable
clientSocket= ServerSocket.accept(); // Waiting for another Connection
System.out.println("Connection established...");
}catch (Exception e) {
String message="ReConnect not successful "+e.getMessage();
logit();//etc...
}
}
I couldn't find another way because as you see from below image you can't understand whether connection is lost or not without a try and catch ,because everything seems right . I got this snapshot while I was getting Connection reset continuously.
Embarrassing to say it, but when I had this problem, it was simply a mistake that I was closing the connection before I read all the data. In cases with small strings being returned, it worked, but that was probably due to the whole response was buffered, before I closed it.
In cases of longer amounts of text being returned, the exception was thrown, since more then a buffer was coming back.
You might check for this oversight. Remember opening a URL is like a file, be sure to close it (release the connection) once it has been fully read.
I had the same error. I found the solution for problem now. The problem was client program was finishing before server read the streams.
I had this problem with a SOA system written in Java. I was running both the client and the server on different physical machines and they worked fine for a long time, then those nasty connection resets appeared in the client log and there wasn't anything strange in the server log. Restarting both client and server didn't solve the problem. Finally we discovered that the heap on the server side was rather full so we increased the memory available to the JVM: problem solved! Note that there was no OutOfMemoryError in the log: memory was just scarce, not exhausted.
Check your server's Java version. Happened to me because my Weblogic 10.3.6 was on JDK 1.7.0_75 which was on TLSv1. The rest endpoint I was trying to consume was shutting down anything below TLSv1.2.
By default Weblogic was trying to negotiate the strongest shared protocol. See details here: Issues with setting https.protocols System Property for HTTPS connections.
I added verbose SSL logging to identify the supported TLS. This indicated TLSv1 was being used for the handshake.
-Djavax.net.debug=ssl:handshake:verbose:keymanager:trustmanager -Djava.security.debug=access:stack
I resolved this by pushing the feature out to our JDK8-compatible product, JDK8 defaults to TLSv1.2. For those restricted to JDK7, I also successfully tested a workaround for Java 7 by upgrading to TLSv1.2. I used this answer: How to enable TLS 1.2 in Java 7
I also had this problem with a Java program trying to send a command on a server via SSH. The problem was with the machine executing the Java code. It didn't have the permission to connect to the remote server. The write() method was doing alright, but the read() method was throwing a java.net.SocketException: Connection reset. I fixed this problem with adding the client SSH key to the remote server known keys.
In my case was DNS problem .
I put in host file the resolved IP and everything works fine.
Of course it is not a permanent solution put this give me time to fix the DNS problem.
In my experience, I often encounter the following situations;
If you work in a corporate company, contact the network and security team. Because in requests made to external services, it may be necessary to give permission for the relevant endpoint.
Another issue is that the SSL certificate may have expired on the server where your application is running.
I've seen this problem. In my case, there was an error caused by reusing the same ClientRequest object in an specific Java class. That project was using Jboss Resteasy.
Initially only one method was using/invoking the object ClientRequest (placed as global variable in the class) to do a request in an specific URL.
After that, another method was created to get data with another URL, reusing the same ClientRequest object, though.
The solution: in the same class was created another ClientRequest object and exclusively to not be reused.
In my case it was problem with TSL version. I was using Retrofit with OkHttp client and after update ALB on server side I should have to delete my config with connectionSpecs:
OkHttpClient.Builder clientBuilder = new OkHttpClient.Builder();
List<ConnectionSpec> connectionSpecs = new ArrayList<>();
connectionSpecs.add(ConnectionSpec.COMPATIBLE_TLS);
// clientBuilder.connectionSpecs(connectionSpecs);
So try to remove or add this config to use different TSL configurations.
I used to get the 'NotifyUtil::java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:...' message in the Apache Console of my Netbeans7.4 setup.
I tried many solutions to get away from it, what worked for me is enabling the TLS on Tomcat.
Here is how to:
Create a keystore file to store the server's private key and
self-signed certificate by executing the following command:
Windows:
"%JAVA_HOME%\bin\keytool" -genkey -alias tomcat -keyalg RSA
Unix:
$JAVA_HOME/bin/keytool -genkey -alias tomcat -keyalg RSA
and specify a password value of "changeit".
As per https://tomcat.apache.org/tomcat-7.0-doc/ssl-howto.html
(This will create a .keystore file in your localuser dir)
Then edit server.xml (uncomment and edit relevant lines) file (%CATALINA_HOME%apache-tomcat-7.0.41.0_base\conf\server.xml) to enable SSL and TLS protocol:
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" keystorePass="changeit" />
I hope this helps

Stale connection with Pheanstalk

I'm using beanstalkd to offload some work to other machines. The setup is a bit unusual, the server is on the internet (public ip) but the consumers are behind adsl lines on some peoples homes. So there is a linux server as client going out through a dynamic ip and connecting to the server to get a job. It's all PHP and I'm using pheanstalk library.
Everything runs smoothly for some time, but then the adsl changes the IP (every 24h hours the provider forces a disconnect-reconnect) the client just hangs, never to go out of "reserve".
I thought that putting a timeout on the reserve would help it, but it didn't. As it seems, the client issues a command and blocks, it never checks the timeout. It just issues a reserve-with-timeout (instead of a simple reserve) and it is the servers responsibility to return a TIME_OUT as the timeout occurs. The problem is, the connection is broken (but the TCP/IP doesn't know about that yet until any of the sides try to talk to the other side) and if the client blocked reading, it will never return.
The library seems to have support for some kind of timeouts locally (for example when trying to connect to server), but it does not seem to contemplate this scenario.
How could I detect the stale connection and force a reconnect? Is there some kind of keepalive on the protocol (and on the pheanstalk itself)?
Thanks!
You could try to close each connection right after the request is answered and reopen a new connection each time.
There is no close() function but you deleting the Pheanstaly Object with unset($pheanstalk) will close it.
This explanation is quite helpful:
Pheanstalk (PHP client for beanstalk) - how do connections work?
I haven't tried it yet, but I came up with the idea of connecting to the beanstalk server through an SSH tunnel. We can enable the ServerAliveCountMax and ServerAliveInterval options on the tunnel, so that a network or server failure will cause the tunnel to close. This should then cause the pheanstalk client to report an error.

X11 remote application timeout

I am in need of a way to decrease the timeout my X server has on remote applications. Currently X11 will keep an application on the display for a very long time (> 30min) after removing the Ethernet connection. I am needing to timeout within 10-30 seconds of loss of communication with the application.
I am running a standard Xorg server with no modification made to it. I have tried numerous methods for doing this. I have tried using the -to option on the X server but this does not seem to have any effect. I have also tried messing with the TCP properties using sysctl. I have set the tcp_keepalive_* properties to values which should give me the timeout needed but this also does not seem to have an effect on the timeout.
Also, the remote applications are not using SSH tunneling to connect to the server. It is an open sever on a secure connection so tunneling is not needed. The timeout mechanism must be done on the server side as I have no control over the applications.
Anyone have any ideas how to get the needed behavior from the X server?
The X server doesn't have client timeouts. Anything you see that looks like one is TCP's doing, not X's.
If you're lucky, the application you're talking to responds to the _NET_WM_PING protocol (most modern toolkits do this for you internally). If you can at least control the window manager you're using, you could modify it to send ping messages to all your running apps and blow them away with XKillClient if they don't respond promptly.

How to reach used cipher key of current SSL connection under Firefox?

I would like have one quick question. Is there any addon for Firefox or tool how to get session key generated from master secret during SSL handshake by which is encoded symmetrically whole client/server communication? I need it due to decoding of communication (POST/GET/etc..) via Wireshark or PCAP library. As I can see Firebug is showing decrypted communication so I hope there exist some proper ways how to reach this session key :)
Thank you all for a help.
I have good news for you. You can actually get the Master-Key data that you need from both Firefox and Chrome. And you can use the output file in Wireshark to decrypt the SSL/TLS traffic without the need for the private key from the SSL/TLS server. Check out "Method 2" here: http://www.root9.net/2012/11/ssl-decryption-with-wireshark-private.html
As a tip, if you don't want to reboot your machine just open a command prompt and run:
set SSLKEYLOGFILE=c:\sslKeyLogFile.txt
"C:\Program Files (x86)\Mozilla Firefox\firefox.exe"
Since Firefox is being launched from the same session that you added the environment variable in, it will launch with that variable set. Otherwise a restart of Windows will be required after setting it in the System settings dialogs.
I also want to point out that the answer from Chris wasn't necessarily wrong, this is a fairly new feature. It didn't make it into release until Wireshark 1.6.
If you want to use Wireshark then the pre master secret will be of no use for you (you refer to it as 'cipher key' in your question).
Wireshark can only decrypt traffic if you specify the RSA private key of the server, which doesn't change on every connection unlike the pre master secret. However, you can't get that through your browser or anything else for obvious reasons.
If you want to decrypt SSL traffic I suggest using an intermediate proxy instead, like Fiddler. It does not passively capture traffic but proxies the traffic, which enables it to actually decrypt the data sent and received.

SSL Session reuse with SChannel (Windows)

I have a program that use schannel.dll to make a SSL connection to remote server. But I'd like to disconnect from server often and reconnect without the need to renegotiate shared key again. I know that SSL support this, but don't know how to make it with SChannel.
Can you help me?
Here are some links that can help:
Creating a Secure Connection Using Schannel: http://msdn.microsoft.com/en-us/library/aa374782(VS.85).aspx
SSL session reuse - how to find if supported?: http://royontechnology.blogspot.com/2008/01/how-to-find-out-if-server-supports-ssl.html
If you want reconnects to work it's important to use the same
credential handle for each connection. You'll also need to pass in the
same targetname string when calling InitializeSecurityContext. Pretty
much everything else is automatic.

Resources