Low level TLS handshake? - go

I'd like to intercept ALPN selection and select the one I want instead of the first common one between the client and the server.
Code:
// we have some ALPN protocols and certificates for TLS/SSL
tlsConfig := &tls.Config {
Certificates: serverCertificates,
NextProtos : serverALPN,
}
// a simple TCP server
server, err := net.Listen("tcp", serverHost+":"+serverPort)
...
for {
// we accept a connection
conn, err := ln.Accept()
...
// wrap it in TLS
tlsConn := tls.Server(conn, &tlsConfig)
// and pass it to the handler
go handleConnection(tlsConn)
}
So far we didn't do any TLS handshaking, so the connection is idle, pure. In func handleConnection(conn *tls.Conn) we would err := conn.Handshake() to perform the handshaking automatically for us, but how to do it manually? I didn't seem to find any info on that in tls module documentation. I'd like to do something like this:
// we communicate with client until he's told us all the ALPNs they support
state := conn.HandshakeUntilALPN()
// we got the list, now we use some algorithm to choose the ALPN
ALPN := myALPNAlgorithm(state.listOfALPNsFromClient)
// after that we tell the client which algo we've chosen
conn.SendALPN(ALPN)
// and we continue the handshake
conn.ContinueHandshake()
...
Of course it's silly pseudo-pseudo code, but hopefully you get the idea :)
Ideally, I'd like to know that for both the Server and the Client, if it's possible at all.

the answer is in the crypto library:
https://pkg.go.dev/crypto/tls
these papers might be useful:
https://developpaper.com/the-principle-of-https-and-golang-specifies-the-https-cipher-suite/
and
https://eli.thegreenplace.net/2021/go-https-servers-with-tls/
You will need to build your own server and client. TLSlistenandserve is an abstraction. you will have to build your own listenandserve.

Related

go grpc error: code = Unavailable desc = connection closed before server preface received [duplicate]

I deployed DgraphAlpha and DgraphZero in docker. I am connecting to Dgraph as described in the documentation.
func newClient() *dgo.Dgraph {
d, err := grpc.Dial("localhost:9080", grpc.WithInsecure())
if err != nil {
log.Fatal(err)
}
return dgo.NewDgraphClient(
api.NewDgraphClient(d),
)
}
And the client is created successfully, but when I try to search
txn := i.dgraphClient.NewReadOnlyTxn()
defer txn.Discard(context.Background())
dgraphQuery := "search here"
response, err := txn.Query(context.Background(), dgraphQuery)
if err != nil {
// error here
}
I get an error:
rpc error: code = Unavailable desc = connection closed before server preface received
This error does not always occur, at unexpected moments, and because of this it is difficult for me to determine its root. Has anyone encountered something similar? What could be the problem?
Beside other transitory causes, a common cause of this error is the server running with TLS enabled and the client attempting to connect without TLS.
Make sure you correctly configured TLS options on the client:
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{myCertificate},
RootCAs: myCAPool,
}
tlsOpt := grpc.WithTransportCredentials(credentials.NewTLS(tlsConfig))
conn, err := grpc.DialContext(ctx, "<connection_string>", tlsOpt)
Make also sure you are actually using client certificates on the client connection.
It may be some issue with timing. This may happens more often in the first requests? Do you have any log on the Dgraph side?
Consider:
Use dial option WithBlock() to ensure you have a connection
Use DialContext and use a context with timeout to avoid wait a lot
Be aware that Insecure dial option is deprecated
Deprecated: use WithTransportCredentials and insecure.NewCredentials() instead.
About the error:
https://grpc.github.io/grpc/core/md_doc_statuscodes.html
Status code 14
This seems a transient error.
You may retry after some time. There are some retriers that can be used on this case like:
https://pkg.go.dev/github.com/eapache/go-resiliency/retrier

Is a gRPC client/stub usage across goroutines

I'm experimenting with Protobuf and gRPC and was going through the Go basics tutorial: https://www.grpc.io/docs/languages/go/basics/
From what I could see in the documentation about the ClientConn it seems that it is safe to use the same ClientConn concurrently. But I can't anything about the stub/client that one creates...
Say I was receiving data via HTTP POST requests and I wanted to forward that data via gRPC as protobuf and then respond to the HTTP request. Should I create a client/stub every time I process a HTTP request, or can I create the client/stub when setting up the HTTP server and pass in the client/stub when setting up HTTP Handlers?
Example: I assume this is okay
var opts []grpc.DialOption
conn, err := grpc.Dial("127.0.0.1:1234", opts...)
if err != nil {
log.Panic(err)
}
defer conn.Close()
grpcService := service.Service{GRPC: conn}
http.HandleFunc("/", util.ValidateRequest(grpcService.HandleRoot))
// listen to port
http.ListenAndServe(fmt.Sprintf("%s:%d", viper.GetString("server.address"), viper.GetInt32("server.port")), nil)
but what about this
var opts []grpc.DialOption
conn, err := grpc.Dial("127.0.0.1:1234", opts...)
if err != nil {
log.Panic(err)
}
defer conn.Close()
client := pb.NewEventShipperClient(conn)
grpcService := service.Service{GRPC: conn, Client: client}
http.HandleFunc("/", util.ValidateRequest(grpcService.HandleRoot))
// listen to port
http.ListenAndServe(fmt.Sprintf("%s:%d", viper.GetString("server.address"), viper.GetInt32("server.port")), nil)
Thread safety ("is it safe to run this concurrently") is covered in the Generated-code reference:
Thread-safety: note that client-side RPC invocations and server-side RPC handlers are thread-safe and are meant to be run on concurrent goroutines. But also note that for individual streams, incoming and outgoing data is bi-directional but serial; so e.g. individual streams do not support concurrent reads or concurrent writes (but reads are safely concurrent with writes).
So, yes, methods on the struct returned by pb.NewEventShipperClient(conn) can be called concurrently.
You can also check this yourself by looking at the implementation of pb.NewEventShipperClient. Currently (I guess this may change) the only member variable will be an interface (cc grpc.ClientConnInterface - implemented by *ClientConn) with each method using the connection to Invoke a gRPC call.
Please note, however, that you need to be careful in your implementation of service.Service to ensure it is also threadsafe and note the caveat regarding streams.

False positives during asynchronous TCP scanning through the Socks5 Proxy relay

I am learning golang and wanted to build a TCP port scanner with SOCKS5 proxies as a relay for mass scanning.
Although all of the S5 proxies are being checked for every target scan, sometimes there are some False positives - and I cannot find the reason why.
Preparing proxyDialer:
func create_socks5_tcp_dialer(socks5_addr string) proxy.Dialer {
//socks5_dialer_tcp, err := proxy.SOCKS5("tcp", socks5_addr, nil, proxy.Direct)
socks5_dialer_tcp, err := proxy.SOCKS5("tcp", socks5_addr, nil, &net.Dialer{Timeout: 5 * time.Second, KeepAlive: 5 * time.Second})
if err != nil {
fmt.Println("Error connecting to proxy:", err)
}
return socks5_dialer_tcp
}
Validating socks5 address:
func socks5_validator(socks5_addr, vps_opened, vps_closed string) (bool, string) {
/* Check if SOCKS5 proxy is valid.
1. Connect to the open port on the server under my control using proxy.
2. Connect to the closed port on the server under my control using proxy.
- If both checks are true then, SOCKS5 proxy is considered as valid.
- If one of the check is false, SOCKS5 proxy is considered as invalid.
3. Returns true/false and s5_addr.
*/
// Create SOCKS5 dialer
socks5_dialer_tcp := create_socks5_tcp_dialer(socks5_addr)
// Make connection using SOCKS5 proxy to the opened port on the vps.
conn_1, err := socks5_dialer_tcp.Dial("tcp", vps_opened)
// If it was successful and not generate any error then check1 is passed.
if err == nil {
//fmt.Println("CHECK 1: PASSED")
conn_1.Close()
// If error was generated then check is not passed and do not make check2.
} else {
//fmt.Println("CHECK 1: NOT PASSED")
return false, socks5_addr
}
// Make connection using SOCKS5 proxy to the closed port on the vps.
conn_2, err := socks5_dialer_tcp.Dial("tcp", vps_closed)
// If it was unsuccessful and error was generated then check2 is passed.
if err != nil {
//fmt.Println("CHECK 2: PASSED")
// If both checks were passed then return false.
return true, socks5_addr
// If error was not generated then check2 is not passed.
} else {
//fmt.Println("CHECK 2: NOT PASSED")
conn_2.Close()
return false, socks5_addr
}
}
Port scanning
s5_dialer_tcp := create_socks5_tcp_dialer(socks5_addr)
// Scan target using s5
conn, err := s5_dialer_tcp.Dial("tcp", target)
if err != nil {
//open
} else {
//closed
}
My question is:
Do I correctly scan TCP services through the SOCKS5 proxy and do I validate this proxy properly?
Link to the full code:
https://github.com/Karmaz95/crimson_prober
I don't think that these are actual false positives. Instead you are having the wrong assumptions of how these proxies will work: You assume that if the single check for a specific port open (connect success) and a specific port closed (connect failure) on a specific server at a specific time succeeds, then the proxy can be used to reliably check many arbitrary ports on arbitrary servers at arbitrary times.
This assumption is likely not valid, especially given that you seem to use proxies which are outside of your control.
A common behavior of such proxies is that they provide only restricted access, i.e. common ports like HTTP and HTTPS will work while other ports will be blocked. Proxies might also employ rate limiting, so they will simply deny access through the proxy after a while. And free proxies available in some lists often cease to work after a while.

Testing bidirectional rpc on golang

I was following some of the tutorial for creating bidirectional grpc client and server. Client will pass some value and when last maximum value changed on server it'll response client with current max. Finally I'd like to write down some of the test cases but I have no experience with testing scenarios that's why I'm not sure if I'm doing the correct thing or not.
func TestClientConnection(t *testing.T) {
creds, _ := credentials.NewClientTLSFromFile("../server-cert.pem", "")
conn, err := grpc.Dial(address, grpc.WithTransportCredentials(creds))
if err != nil {
t.Error("Had problem with connection, NOT PASSED")
}
defer conn.Close()
c := proto.NewHerdiusServerClient(conn)
stream, err := c.CheckMax(context.Background())
if err != nil {
t.Error("Had problem with stream, NOT PASSED")
return
}
err = stream.Send(&proto.MaxRequest{Val: int32(10)})
err = stream.Send(&proto.MaxRequest{Val: int32(12)})
err = stream.Send(&proto.MaxRequest{Val: int32(13)})
err = stream.Send(&proto.MaxRequest{Val: int32(9)})
if err != nil {
t.Error("Had problem with stream, NOT PASSED")
return
}
return
}
Right now when I test this scenario wiht go test it passes but I also want to test if something received from server side.
My second question was If I want to tear this test to different scenarios for example to check is server connected or is stream connected or it received response from server side, how can I do that? Should I create another class to retrieve connection and streaming and use on test functions?
Create a contest with timeout context.WithTimeout, and after sending your data call Recv on the stream. Check if you receive anything within the timeout.
The specifics depend on the protocol here - you may need a goroutine to Recv if you have to send server data at the same time.
As for your second question, the Go philosophy is to have clear, explicit, readable tests for each scenario. It's OK if some code is duplicated. It's much more important that each test in isolation is readable and understandable. In cases where the tests are very repetitive one should use table driven tests, but in the cases you describe that sounds like separate tests to me.
It's useful to have tests that "build up" functionality. One test to test connection, the other connection and sending, yet another connection and sending and receiving. This way when tests fail, by rerunning them individually you can very quickly isolate the problem even before you look at the tests' code.

Updating receive and send message size for grpc in golang

I have grpc server written in Go and I am trying to update receive and send message size to 20MB instead of the default 4MB with the following code
var s *grpc.Server
s = grpc.NewServer(grpc.MaxRecvMsgSize(1024*1024*20), grpc.MaxSendMsgSize(1024*1024*20))
pb.RegisterProductServer(s,mysrv)
But the above doesn't seem to be working as I still get an error when I tried to call from a client received message larger than max (5807570 vs. 4194304)"
Not sure whats overriding the size
I haven't had the chance to test this yet but have you tried adding the same options from the client stub? The same options can be attached as dial options:
maxMsgSize := 1024*1024*20
conn, err := grpc.Dial(address, grpc.WithDefaultCallOptions(grpc.MaxRecvMsgSize(maxMsgSize), grpc.MaxSendMsgSize(maxMsgSize)))
if err != nil {
// ...
}
defer conn.close()
client := pb.NewProductClient(conn)
// ...
I don't know anything about your use case but if your response data can be delivered piece wise then the streaming APIs might be useful.

Resources