I have a client which will establish TLS connection to backend service.
There are two kind of scenarios that I encounter.
Direct network: client--->server
In this environment, The client connect directly to server as below code.
var d tls.Dialer
//...
d.Config = &tls.Config{
//...
}
//...
c1 := d.Dial("tcp", addr)
Proxy network: client--->proxy--->server
In this environment, The client is behind a http proxy, client need leverage proxy http tunnel to forward traffic between client and server.
I use golang.org/x/net/proxy in client to connect proxy, as proxy is http proxy, client should use net.Dialer to connect proxy via tcp.
dailer, err := proxy.FromURL(proxy, &net.Dialer{
Timeout: TCP_CONNECT_TIMEOUT,
KeepAlive: TCP_KEEPALIVE_TIMEOUT,
})
c2 := dailer.Dial("tcp", addr)
In case1, client start a TLS connection, in network traffic packets, client trigger TCP connection, after 3 way handshakes, client send Client Hello to server.
In case2, client first use TCP to connect http proxy (ex, 10.0.0.1:8080), next, send CONNECT to proxy, then proxy return Connection Established, however, client do NOT send Client Hello to server.
For case2, I do not know how and where to implement to send Client Hello in client?
Thanks in advance.
Related
To comply with company guidelines of utilizing only HTTP2, how can I establish a new TLS with the underlying connection of the HTTP2 request?
I have successfully used the Hijack method on the http.ResponseWriter to retrieve the conn object from an HTTP1.1 request, and with the help of the conn object and TLS configuration, I have established a TLS server. My goal is to establish a TLS server in a similar manner, but with incoming HTTP2 requests.
Code snippet for HTTP1.1 request
hijacker, ok := w.(http.Hijacker)
if !ok {
return nil, errors.New("hijacking not supported")
}
clientConn, _, err := hijacker.Hijack()
if err != nil {
return nil, errors.New("hijacking failed")
}
tlsConn := tls.Server(clientConn, &tlsConfig)
tlsConn.Write([]byte("hello there"))
We are trying to set up a secure server that can handle different certificates based on the hostname. To do this, Service A sends information about the certificate to Service B through HTTP 1.1. Service B uses the certificate information to create a new secure server (TLS). However, if the incoming request is using HTTP2, this process is not possible as HTTP2 does not support retrieving the underlying connection.
The reason for this setup is to allow Service A, which acts as a proxy, to communicate with Service B in a secure manner. The client's original request to Service A may not trust Service B's certificate, so Service A sends the host information to Service B first, allowing B to create a secure server using the proper certificate. Service A then forwards the client's request to this new secure server which is still using the same underlying connection.
I am trying to implement a Socks5 server that could relay both HTTP and HTTPS traffic.
As the RFC1928 mentions, the following steps to establish a connection and forward the data must be taken :
Client sends a greeting message to the proxy.
Client & proxy authentication (assuming it is successful).
Client sends a request to the proxy to connect to the destination.
The proxy connects to the destination and sends back a response to the client to indicate a successful open tunnel.
The proxy reads the data from the client and forwards it to the destination.
The proxy reads the data from the destination and forwards it to the client.
So far, the proxy works as it should. It is able to relay HTTP traffic using its basic data forwarding mechanism. However, any request from the client to an HTTPS website will be aborted because of SSL/TLS encryption.
Is there another sequence/steps that should be followed to be able to handle SSL/TLS (HTTPS) traffic?
The sequence you have described is correct, even for HTTPS. When the client wants to send a request to an HTTPS server through a proxy, it will request the proxy to connect to the target server's HTTPS port, and then once the tunnel is established, the client will negotiate a TLS handshake with the target server, then send an (encrypted) HTTP request and receive an (encrypted) HTTP response. The tunnel is just a passthrough of raw bytes, the proxy has no concept of any encryption between the client and server. It doesn't care what the bytes represent, its job is just to pass them along as-is.
Can you configure the IMAP client in mailkit to connect via a proxy? if so, how? I'm guessing it would have to be a socks proxy.
Each of MailKit's *Client implementations has a Connect method that takes a Socket. This is how you'd connect via a proxy.
First, you'd open a Socket and connect it to your proxy, send the appropriate commands, and then pass that socket off to MailKit.
Considering the example from the Go gRPC code base:
func main() {
// Set up a connection to the server.
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewGreeterClient(conn)
// Contact the server and print out its response.
name := defaultName
if len(os.Args) > 1 {
name = os.Args[1]
}
r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name: name})
if err != nil {
log.Fatalf("could not greet: %v", err)
}
log.Printf("Greeting: %s", r.Message)
}
When consuming a gRPC service from another service what should the scope of the connection (conn) be? I assume it should have affinity with the scope of the request being handled be the consumer service, but I have yet to find any documentation around this. Should I be using a connection pool here?
E.G.
gRPC consumer service receives request
establish connection to gRPC service (either directly or via pool)
make n requests to gRPC service
close gRPC connection (or release back to the pool)
From experience, gRPC client connections should be re-used for the lifetime of the client application as they are safe for concurrent use. Furthermore, one of the key features of gRPC is rapid response from remote procedural calls, which would not be achieved if you have to reconnect on every request received.
Nonetheless, it is highly recommended to use some kind of gRPC load balancing along with these persistent connections. Otherwise, a lot of the load may end up on a few long-lived grpc client-server connections. Load Balancing options include:
A gRPC connection pool on client side combined with a server side TCP (Layer 4) load balancer. This will create a pool of client connections initially, and re-use this pool of connections for subsequent gRPC requests. This is the easier route to implement in my opinion. See Pooling gRPC Connections for an example of grpc connection pooling on grpc client side which uses the grpc-go-pool library.
HTTP/2(Layer 7) load balancer with gRPC support for load balancing requests. See gRPC Load Balancing which gives an overview of different grpc load balancing options. nginx recently added support for gRPC load balancing.
My websocket server listens on port 8080 with no proxy.
Most of the time I'm getting requests with the Upgrade Websocket header and it works fine.
Sometimes I'm getting HTTP CONNECT requests.
Is this a valid request?
Does it means that there is a proxy server between the client and the server?
How my server is suppose to respond to the CONNECT request?
Thanks
You are getting CONNECT requests because you are likely to have configured your browser to use a proxy. If you directed your browser to use port 8080 on your local IP address, it will assume there is a proxy and that means when you ask for a secure connection, the browser leads with CONNECT.
You will need to add support for SSL/TLS tunnelling to your server to deal with this.