We have a golang http.client sending a request every 5th second to a golang http server with mTLS (client certificate is verified towards CA on server-side), without creating client or transport at every request. The client certificate is valid for 5h and renewal starts after 4h. Renewal is always happening promptly after 4h.
What we observe is that the connection is established and kept open for the entire 5 hours, causing "certificate expired" after 5h in which the connection is closed and then re-created with the renewed client certificate. After that communication is fine for another 5h.
Behavior is not happening when setting client's http.Transport's DisableKeepAlives: true (meaning the renewal process in application is fine) but would like to avoid creating a new connection for every request. I.e. I'm looking for a behavior in which connections are re-established at a periodic interval, say every 20th second or when a certificate renewal is detected, even if connection is deemed "active".
Related
I use mailkit for my ASP.Net Core application and create a SMTPClient object which I then connect to Office365 and authenticate my user with a username and password. How long can this connection be open until it expires or needs to be reauthenticated? Also, is there a way to keep the connection alive without sending an email at the expiration time?
To keep a client connection to a server alive (whether it be using the ImapClient, Pop3Client, or the SmtpClient), you can use the NoOp() or NoOpAsync() methods to send a command to the server that does "nothing" but to let the server know that the client is still there and wants the connection to remain alive. At least in theory this will work.
That said, it's likely not considered good netiquette to keep an SMTP connection alive for any lengthy period of time.
Most SMTP servers are going to expect clients to connect, flush their outbound message queues (by sending each message that is queued up) and then immediately disconnecting and some of the bigger free mail servers may ignore NOOP commands and disconnect the client anyway after some set period of time.
For example, the IMAP specification states that servers should keep a connection alive for at least 30 minutes after the client's last command, but in practice, GMail will only keep that connection alive for less than 10 minutes.
The SMTP specification, as far as I can remember off the top of my head, makes no such suggestion as far as how long to keep the connection alive after the client's last command, so some servers may require more frequent NOOP commands than others and some may even have a max period of time that they'll allow you to keep that connection alive for regardless of whether you send those NOOP commands.
TL;DR:
The NOOP command (via the NoOp() or NoOpAsync() methods) will theoretically keep the connection alive if sent frequently enough, but I would not depend on this working reliably.
I've read a lot of related topics in the net, but I still don't have an answer to my question.
Is it possible to implement flow described below?
Proxy receive request.
If request is encrypted and proxy cert is trusted then intercept.
If request is not encrypted, then intercept.
If request is encrypted and proxy cert is NOT trusted then pass it through without interception.
This behaviour should be default for all traffic going through the proxy.
It'd be also really nice to be able to get all possible info for passing encrypted requests (src and dst ip addresses etc.). Basically the same info which I can get with fiddler.
Not really. The main problem is that mitmproxy can not know if proxy cert is trusted by the client or not.
In the SSL/TLS protocol client starts with the CLIENT_HELLO and in response the server (in this case motmproxy) sends back the SERVER_HELLO message containing the generated server certificate.
The client now checks if the received server certificate is trusted. If not the connection is terminated. As far as I know the SSL/TLS spec does not define how to do so. Sems clients end back an SSL_ALERT message, other simply drop the connection, and a third group continues the SSL/TLS handshake but have certain internal values set in a way that always let the handshake fail.
There is a mitmproxy script that tries to identify connections that were not successful and then if the client asks for the same domain a second time bypasses interception.
Of course this requires that the client resends requests which is not always the case.
https://github.com/sociam/x-ray/blob/master/mitmproxy/examples/tls_passthrough.py
I use Caddy (a webserver written in GO), I have TLS 1.0-TLS 1.2 allowed, and GO supports only the tickets option of session resumption (the TLS session data is stored on the client side).
Now I'm not quite sure about when the TLS resumption should occur.
Regardless of ticket lifetime (which I think is a week by default) and Session Ticket Encryption Key (which are rotated every 10 hours and Caddy "remembers" the last 4) - so non of those should be the issue.
As far as I understand the session resumption should occur whenever the ticket is still valid and the browser hasn't been restarted, which means that even if I changed my IP address, the TLS ticket should still work.
But that's not the case for me; when I access the webserver (using both Chrome and Firefox) I get a TLS ticket, which remains valid throughout surfs to that webserver, but when I change my IP address (either by proxy or changing WIFI) the ticket is not accepted on the server side and a full TLS handshake is made, in which I get a new ticket.
So my main question: does TLS session resumption only work within a TCP session and whenever a new TCP session is started the former TLS ticket becomes invalid?
... does TLS session resumption only work within a TCP session
Given that there is almost always only a single TLS handshake in the TCP connection that would not make much sense.
Session resumption with tickets works as long as the client sends a session ticket and the server has the necessary secrets to extract the information from the ticket. The server might implement additional restrictions though, like encoding the client IP in the ticket and checking if it is still the same.
Is a refresh security type(ssl) command needed or not after we deploy renewed certs into keystore. I usually execute this command after certs renewal, but I saw cases where this command is not executed and still the channels picked up the new certs. Are the certs which are put into the keystore are coming into effect immediately or we need to perform ssl refresh.
To ensure that all connections in or out of the queue manager are using current information in the key store you should always run the REFRESH SECURITY TYPE(SSL) after any updates are made (for example to the private key or even adding new trusted CA certs).
The first time any MQ process needs to access the key store, the process will read the key store and cache it in memory. In a default configuration the connections into the queue manager are handled by processes that keep running so they would hold onto that key store cache after the first inbound connection to the queue manager which requires the key store. In a default configuration outbound channel connections use separate runmqchl processes that start when the channel is started so they always read the current key store each time.
This means that in some cases things may work without a REFRESH SECURITY TYPE(SSL) for example:
If a SDR channel with MCATYPE(PROCESS) starts, a new runmqchl process is started and will read the key store fresh and pick up any changes.
If no inbound TLS connection has been received since the last time the queue manager was started then even if you make changes to the key store they will be picked up the first time an inbound TLS connection is received.
Morag Hughson gave a presentation at Capitalware's MQ Technical Conference v2.0.1.4 titled "MQ SSL/TLS Channels Including V8 changes". The notes portion of page 10 of the PDF have a good summary of what REFRESH SECURITY TYPE(SSL) does.
Refreshing SSL on WebSphere MQ - Notes
This feature was introduced in WebSphere MQ V6.
The SSL environment set up to run SSL channels in a channel process has a cached view of the key repository made at initialization time.
If you make changes to your key repository, i.e. add, remove or update
certificates, for example, because your are replacing a certificate
that is about to expire, this cached view needs to be refreshed in
order for the SSL channels to start using the new certificates.
In order to refresh this cached view of the SSL environment, without disrupting any non-SSL channels, use the REFRESH SECURITY TYPE(SSL)
command. This will stop all the SSL channels on the queue manager, new
cached view(s) of the key repository will be made and all the sending
type channels will be started again. Receiving type channels will get
restarted as the partner end retries the connection.
Non-SSL channels will be unaffected by this command and will continue to run.
Also use this command to pick up other changes, such as a new Key Repository locations, or new LDAP CRL/ARL locations.
IBM documents this in the IBM MQ Knowledge Center, below are two pages with information on the topic:
IBM MQ v7.5 KC page "Refreshing SSL or TLS security"
About this task
When a channel is secured using Secure Sockets Layer (SSL) or TLS, the
digital certificates and their associated private keys are stored in
the key repository. A copy of the key repository is held in memory
while a channel is running. If you make a change to the key
repository, you can refresh the copy of the key repository that is
held in memory without restarting the channel.
When you refresh the cached copy of the key repository, all SSL or TLS
channels that are currently running are updated:
Sender, server, and cluster-sender channels that use SSL or TLS are allowed to complete the current batch of messages. The channels then
run the SSL handshake again with the refreshed view of the key
repository.
All other channel types that use SSL or TLS are stopped. If the partner end of the stopped channel has retry values defined, the
channel retries and runs the SSL handshake again. The new SSL
handshake uses the refreshed view of the contents of the key
repository, the location of the LDAP server to be used for the
Certificate Revocation Lists, and the location of the key repository.
In the case of server-connection channel, the client application loses
its connection to the queue manager and has to reconnect to continue.
MQ v7.5 KC page "REFRESH SECURITY" states:
Usage notes for REFRESH SECURITY
When you issue the REFRESH SECURITY TYPE(SSL) MQSC command, all
running SSL channels are stopped and restarted. Sometimes SSL channels
can take a long time to shut down and this means that the refresh
operation takes some time to complete. There is a time limit of 10
minutes for an SSL refresh to complete (or 1 minute on z/OS), so it
can potentially take 10 minutes for the command to finish. This can
give the appearance that the refresh operation has "frozen". The
refresh operation will fail with an MQSC error message of AMQ9710 or
PCF error MQRCCF_COMMAND_FAILED if the timeout is exceeded before all
channels have stopped. This is likely to happen if the following
conditions are true:
The queue manager has many SSL channels running simultaneously when the refresh command is invoked
The channels are handling large numbers of messages
If a refresh fails under these conditions, retry the command later
when the queue manager is less busy. In the case where many channels
are running, you can choose to stop some of the channels manually
before invoking the REFRESH command.
...
When using TYPE(SSL):
On platforms other than z/OS, the command updates all SSL channels regardless of whether a security refresh is needed.
If a refresh is to be performed, the command updates all SSL channels currently running, as follows:
Sender, server and cluster-sender channels using SSL are allowed to complete the current batch. In general they then run the SSL handshake
again with the refreshed view of the SSL key repository. However, you
must manually restart a requester-server channel on which the server
definition has no CONNAME parameter.
All other channel types using SSL are stopped with a STOP CHANNEL MODE(FORCE) STATUS(INACTIVE) command. If the partner end of the
stopped message channel has retry values defined, the channel retries
and the new SSL handshake uses the refreshed view of the contents of
the SSL key repository, the location of the LDAP server to be used for
Certification Revocation Lists, and the location of the key
repository. In the case of a server-connection channel, the client
application loses its connection to the queue manager and has to
reconnect in order to continue.
Without timeout? Or is there even a timeout?
There is no limit on the lifetime of a connected jid. For command line bots, it is a good practice to send periodic ping packets to the server, just to make sure opened socket doesn't drop after some period of inactivity.
In case your client is connected from browser and suppose the user refreshes the browser without disconnecting from the jabber server. User can still use saved (via cookie/session) jid,sid,rid combination to reconnect with previously opened session. However, bosh connection manager will drop the connection after "X" seconds of inactivity.
XMPP does not say anything about having or not a timeout. So, in theory, you XMPP Session could last as long as the TCP connection is established.
You are free to implement a timeout in your client or server though...