I am learning networking with Go.
I wrote the code because I wanted to test the timeout of the current dial, but it doesn't work as I intended.
package socket
import (
"net"
"testing"
"time"
)
func TestDialTimeout(t *testing.T) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
go func() {
for {
conn, err := listener.Accept()
if err != nil {
t.Error(err)
}
t.Logf("[server] connection success from %q", conn.RemoteAddr())
done := make(chan struct{})
go func(conn net.Conn) {
time.Sleep(time.Second * 10)
conn.Close()
done <- struct{}{}
}(conn)
<-done
}
}()
front, err := net.Dial("tcp", listener.Addr().String())
t.Logf("[front] connection request from %q to %q", front.LocalAddr(), front.RemoteAddr())
if err != nil {
t.Fatal(err)
}
t.Logf("[front] connection success to %q", front.RemoteAddr())
dialer := net.Dialer{Timeout: time.Second * 5}
conn, err := dialer.Dial("tcp", listener.Addr().String())
t.Logf("[conn] connection request from %q to %q", conn.LocalAddr(), conn.RemoteAddr())
if err == nil {
conn.Close()
t.Error("connection did not time out")
}
t.Logf("[conn] connection success to %q", conn.RemoteAddr())
nErr, ok := err.(net.Error)
if !ok {
t.Fatal(err)
}
if !nErr.Timeout() {
t.Fatal("error is not a timeout")
}
listener.Close()
}
Since the server first connects with one client, I think that a timeout should be taken for the next client to be connected.
But printed message is below.
=== RUN TestDialTimeout
timeout_test.go:18: [server] connection success from "127.0.0.1:58517"
timeout_test.go:30: [front] connection request from "127.0.0.1:58517" to "127.0.0.1:58516"
timeout_test.go:34: [front] connection success to "127.0.0.1:58516"
timeout_test.go:38: [conn] connection request from "127.0.0.1:58518" to "127.0.0.1:58516"
timeout_test.go:41: connection did not time out
timeout_test.go:43: [conn] connection success to "127.0.0.1:58516"
timeout_test.go:47: <nil>
--- FAIL: TestDialTimeout (0.00s)
FAIL
No error occurs on the second dial.
Even the time.Sleep() doesn't seem to work.
I don't know what I did wrong.
Could you please give me some advice?
Related
Implementing an ssh proxy in Go, errors out with bad packet length, these are the errors with ssh in debug mode:
debug1: SSH2_MSG_KEXINIT sent
Bad packet length 1231976033.
ssh_dispatch_run_fatal: Connection to ::1 port 8080: message authentication code incorrect
Code:
func handleSSH(conn net.Conn, r *bufio.Reader, protocol string) {
target, err := url.Parse("ssh://localhost:3333")
if err != nil {
fmt.Println("Error parsing target", err)
conn.Close()
return
}
targetConn, err := net.Dial("tcp", target.Host)
if err != nil {
fmt.Println("error dialing SSH target:", err)
conn.Close()
return
}
defer targetConn.Close()
var wg sync.WaitGroup
wg.Add(2)
go func() {
_, err := io.Copy(targetConn, conn)
if err != nil {
fmt.Println("error copying data to target:", err)
}
wg.Done()
}()
go func() {
_, err := io.Copy(conn, targetConn)
if err != nil {
fmt.Println("error copying data from target:", err)
}
wg.Done()
}()
wg.Wait()
conn.Close()
}
// EDIT
func connection(conn net.Conn) {
r := bufio.NewReader(conn)
protocol, err := r.ReadString('\n')
if err != nil {
fmt.Println("Error reading first line", err)
conn.Close()
return
}
if protocol[0:3] == "SSH" {
handleSSH(conn, r, protocol)
}
}
func main() {
ln, err := net.Listen("tcp", ":8080")
if err != nil {
panic(err)
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
panic(err)
}
go connection(conn)
}
}
EDIT: added code for relevant information on how the connection is initiated and reproduce the error.
My best guess is the ssh negotiation process is being interrupted, and things goes out of sync.
The code is reading the first line from the client and checks the kind of protocol in order to call the appropriate handler:
protocol, err := r.ReadString('\n')
...
if protocol[0:3] == "SSH" {
handleSSH(conn, r, protocol)
}
}
But the code fails to forward the already read bytes to the connected server. These bytes are in protocol and are given to handleSSH. But it fails to send these bytes to the connected server once the connection is established. Instead it only copies new data between client and server.
This means the server does not get the first line from the client. It therefore likely complains about a protocol error with something like Invalid SSH identification string. which gets forwarded to the client and misinterpreted as valid data from an SSH connection.
I have a go api project where I also run a worker (RabbitMQ). I just discovered a problem that my worker and my http listen and serve do not work together. The moment I run the worker, the port of api is not reached.
Here is what my code looks like.
app.go
func (a *App) StartWorker() {
connection, err := amqp091.Dial(os.Getenv("AMQP_URL"))
if err != nil {
panic(err)
}
defer connection.Close()
consumer, err := events.NewConsumer(connection, database.GetDatabase(a.Database))
if err != nil {
panic(err)
}
consumer.Listen(os.Args[1:])
}
func (a *App) Run(addr string) {
logs := log.New(os.Stdout, "my-service", log.LstdFlags)
server := &http.Server{
Addr: addr,
Handler: a.Router,
ErrorLog: logs,
IdleTimeout: 120 * time.Second, // max time for connections using TCP Keep-Alive
ReadTimeout: 5 * time.Second,
WriteTimeout: 10 * time.Second,
}
go func() {
if err := server.ListenAndServe(); err != nil {
logs.Fatal(err)
}
}()
// trap sigterm or interrupt and gracefully shutdown the server
c := make(chan os.Signal)
signal.Notify(c, os.Interrupt)
signal.Notify(c, os.Kill)
sig := <-c
logs.Println("Recieved terminate, graceful shutdown", sig)
tc, _ := context.WithTimeout(context.Background(), 30*time.Second)
server.Shutdown(tc)
}
here is my
consumer.go
// NewConsumer returns a new Consumer
func NewConsumer(conn *amqp.Connection, db *mongo.Database) (Consumer, error) {
consumer := Consumer{
conn: conn,
db: db,
}
err := consumer.setup()
if err != nil {
return Consumer{}, err
}
return consumer, nil
}
// Listen will listen for all new Queue publications
// and print them to the console.
func (consumer *Consumer) Listen(topics []string) error {
ch, err := consumer.conn.Channel()
if err != nil {
return err
}
defer ch.Close()
if err != nil {
return err
}
msgs, err := ch.Consume("update.package.rating", "", true, false, false, false, nil)
if err != nil {
return err
}
forever := make(chan bool)
go func() {
for msg := range msgs {
switch msg.RoutingKey {
case "update.package.rating":
worker.RatePackage(packageRepo.NewPackagesRepository(consumer.db), msg.Body)
}
// acknowledege received event
log.Printf("Received a message: %s", msg.Body)
}
}()
log.Printf("[*] Waiting for message [Exchange, Queue][%s, %s]. To exit press CTRL+C", getExchangeName(), "update.package.rating")
<-forever
return nil
}
main.go
func main() {
start := app.App{}
start.StartApp()
start.StartWorker()
start.Run(":3006")
}
the port 3006 is not reached.
I am using gin-gonic to serve my http request.
Any help is welcomed.
I had a similar problem while using gin framework.Solved the issue by running my consumer inside a go routine.I invoked my consumer like below.
go notificationCallback.ConsumeBankTransaction()
and both the server and the rabbitmq consumer run seamlessly.Still monitoring performance to see if it is robust and resilient enough.
I'm writing a Go websocket server and I want to graceful stop the connections when my server goes down.
I have a map of active connections stored in the following variable:
var connections = make(map[string]*websocket.Conn)
My main function looks like this:
func main() {
// ... stuff ....
gracefulStop := make(chan os.Signal)
signal.Notify(gracefulStop, syscall.SIGTERM)
signal.Notify(gracefulStop, syscall.SIGINT)
signal.Notify(gracefulStop, syscall.SIGQUIT)
signal.Notify(gracefulStop, syscall.SIGKILL)
signal.Notify(gracefulStop, syscall.SIGHUP)
go func() {
sig := <-gracefulStop
log.Printf("Exiting from process due to %+v", sig)
log.Println("Closing all websocket connections")
for id, conn := range connections {
closeConnection(id, conn)
}
os.Exit(0)
}()
r := mux.NewRouter()
r.HandleFunc("/{id}", wsHandler)
err := http.ListenAndServe(fmt.Sprintf(":%d", *argPort), r)
if err != nil {
log.Println("Could not start http server")
log.Println(err)
}
}
closeConnection does 4 things:
conn.Close()
sets conn as nil
removes the id from the map
calls an AWS Lambda function
The same function is called as a defer function inside the wsHandler function, so if a client disconnects by its own, I execute function in the handler.
It's all working nicely, except that when I ctrl+c the server my closeConnection function is called two times per client, one in the graceful stop handler and the other in the wsHandler defer function.
I tried to check in my closeConnection function if the connection is still defined in connections, but it returns true both of the times.
I thought that it was due to the fact that they are called two times because they are in different goroutines, so I replaced the for loop above with just a time.Sleep(2 * time.Second), but in this case nothing happens (the closeConnection inside the wsHandler defer function is not even called).
This is what I mean:
go func() {
sig := <-gracefulStop
log.Printf("Exiting from process due to %+v", sig)
log.Println("Closing all websocket connections")
// for chargeboxIdentity, conn := range connections {
// chargeboxDisconnected(chargeboxIdentity, conn)
// }
time.Sleep(2 * time.Second)
os.Exit(0)
}()
EDIT: Here is the closeConnection function:
func closeConnection(id string, conn *websocket.Conn) {
_, ok := connections[id]
log.Println(ok)
log.Printf("%s (%s) disconnected", id, conn.RemoteAddr())
conn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, ""))
time.Sleep(300 * time.Millisecond)
conn.Close()
conn = nil
delete(connections, id)
request := LambdaPayload{ID: id}
payload, err := json.Marshal(request)
if err != nil {
log.Println("Could not create payload for lambda call")
log.Println(err)
return
}
_, err = client.Invoke(&lambda.InvokeInput{FunctionName: aws.String(lambdaPrefix + "MainDisconnect"), Payload: payload})
if err != nil {
log.Println("Disconnect Lambda returned an error")
log.Println(err)
}
}
EDIT: Here's the wsHandler function:
func wsHandler(w http.ResponseWriter, r *http.Request) {
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Println("Could not upgrade websocket connection")
log.Println(err)
return
}
vars := mux.Vars(r)
if !clientConnected(vars["id"], conn) {
return
}
defer closeConnection(vars["id"], conn)
for {
msgType, msg, err := conn.ReadMessage()
if err != nil {
break
}
log.Printf("%s sent: %s", vars["id"], string(msg))
// ... stuff ...
}
}
Summary: I'm running into a race condition during testing where my server is not reliably ready to serve requests before making client requests against it. How can I block only until the listener is ready, and still maintain composable public APIs without requiring users to BYO net.Listener?
We see the following error as the goroutine that spins up our (blocking) server in the background isn't listening before we call client.Do(req) in the TestRun test function.
--- FAIL: TestRun/Server_accepts_HTTP_requests (0.00s)
/home/matt/repos/admission-control/server_test.go:64: failed to make a request: Get https://127.0.0.1:37877: dial tcp 127.0.0.1:37877: connect: connection refused
I'm not using httptest.Server directly as I'm attempting to test the blocking & cancellation characteristics of my own server componenent.
I create an httptest.NewUnstartedServer, clone its *tls.Config into a new http.Server after starting it with StartTLS(), and then close it, before calling *AdmissionServer.Run(). This also has the benefit of giving me a *http.Client with the matching RootCAs configured.
Testing TLS is important here as the daemon this exposes lives in a TLS-only environment.
func newTestServer(ctx context.Context, t *testing.T) *httptest.Server {
testHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "OK")
})
testSrv := httptest.NewUnstartedServer(testHandler)
admissionServer, err := NewServer(nil, &noopLogger{})
if err != nil {
t.Fatalf("admission server creation failed: %s", err)
return nil
}
// We start the test server, copy its config out, and close it down so we can
// start our own server. This is because httptest.Server only generates a
// self-signed TLS config after starting it.
testSrv.StartTLS()
admissionServer.srv = &http.Server{
Addr: testSrv.Listener.Addr().String(),
Handler: testHandler,
TLSConfig: testSrv.TLS.Clone(),
}
testSrv.Close()
// We need a better synchronization primitive here that doesn't block
// but allows the underlying listener to be ready before
// serving client requests.
go func() {
if err := admissionServer.Run(ctx); err != nil {
t.Fatalf("server returned unexpectedly: %s", err)
}
}()
return testSrv
}
// Test that we can start a minimal AdmissionServer and handle a request.
func TestRun(t *testing.T) {
testSrv := newTestServer(context.TODO(), t)
t.Run("Server accepts HTTP requests", func(t *testing.T) {
client := testSrv.Client()
req, err := http.NewRequest(http.MethodGet, testSrv.URL, nil)
if err != nil {
t.Fatalf("request creation failed: %s", err)
}
resp, err := client.Do(req)
if err != nil {
t.Fatalf("failed to make a request: %s", err)
}
// Later sub-tests will test cancellation propagation, signal handling, etc.
For posterity, this is our composable Run function, that listens in a goroutine and then blocks on our cancellation & error channels in a for-select:
type AdmissionServer struct {
srv *http.Server
logger log.Logger
GracePeriod time.Duration
}
func (as *AdmissionServer) Run(ctx context.Context) error {
sigChan := make(chan os.Signal, 1)
defer close(sigChan)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
// run in goroutine
errs := make(chan error)
defer close(errs)
go func() {
as.logger.Log(
"msg", fmt.Sprintf("admission control listening on '%s'", as.srv.Addr),
)
if err := as.srv.ListenAndServeTLS("", ""); err != nil && err != http.ErrServerClosed {
errs <- err
as.logger.Log(
"err", err.Error(),
"msg", "the server exited",
)
return
}
return
}()
// Block indefinitely until we receive an interrupt, cancellation or error
// signal.
for {
select {
case sig := <-sigChan:
as.logger.Log(
"msg", fmt.Sprintf("signal received: %s", sig),
)
return as.shutdown(ctx, as.GracePeriod)
case err := <-errs:
as.logger.Log(
"msg", fmt.Sprintf("listener error: %s", err),
)
// We don't need to explictly call shutdown here, as
// *http.Server.ListenAndServe closes the listener when returning an error.
return err
case <-ctx.Done():
as.logger.Log(
"msg", fmt.Sprintf("cancellation received: %s", ctx.Err()),
)
return as.shutdown(ctx, as.GracePeriod)
}
}
}
Notes:
There is a (simple) constructor for an *AdmissionServer: I've left it out for brevity. The AdmissionServer is composable and accepts a *http.Server so that it can be plugged into existing applications easily.
The wrapped http.Server type that we create a listener from doesn't itself expose any way to tell if its listening; at best we can try to listen again and catch the error (e.g. port already bound to another listener), which does not seem robust as the net package doesn't expose a useful typed error for this.
You can just attempt to connect to the server before starting the test suite, as part of the initialization process.
For example, I usually have a function like this in my tests:
// waitForServer attempts to establish a TCP connection to localhost:<port>
// in a given amount of time. It returns upon a successful connection;
// ptherwise exits with an error.
func waitForServer(port string) {
backoff := 50 * time.Millisecond
for i := 0; i < 10; i++ {
conn, err := net.DialTimeout("tcp", ":"+port, 1*time.Second)
if err != nil {
time.Sleep(backoff)
continue
}
err = conn.Close()
if err != nil {
log.Fatal(err)
}
return
}
log.Fatalf("Server on port %s not up after 10 attempts", port)
}
Then in my TestMain() I do:
func TestMain(m *testing.M) {
go startServer()
waitForServer(serverPort)
// run the suite
os.Exit(m.Run())
}
I am creating a GoLang application and clients are android phones. I am able to handle connections. If user closes the android application connection is dropped with EOF
My problem is, if client just turn off wifi network connection is still alive.
Here is my code
func main() {
fmt.Println("Starting server...")
connection, err := net.Listen("tcp", ":4406")
if err != nil {
fmt.Println(err)
}
defer connection.Close()
manager := ClientManager{
clients: make(map[*Client]bool),
broadcast: make(chan []byte),
register: make(chan *Client),
}
go manager.start()
for {
connection, _ := connection.Accept()
if err != nil {
fmt.Println(err)
}
client := &Client{socket: connection, data: make(chan []byte), uuid: connection.RemoteAddr().String()}
manager.register <- client
go manager.receive(client)
go handleConnection(client)
}
}
Handeling connections
func handleConnection(client *Client) {
conn := client.socket
defer conn.Close()
notify := make(chan error)
go func() {
buf := make([]byte, 1024)
for {
n, err := conn.Read(buf)
if err != nil {
notify <- err
return
}
if n > 0 {
fmt.Println("unexpected data: %s", buf[:n])
}
}
}()
for {
select {
case err := <-notify:
if err != nil {
fmt.Println("connection dropped message", err)
return
}
case <-time.After(time.Second * 1):
fmt.Println("timeout 1, still alive")
}
}
}
When remote wifi is off (cable removed) I want to disconnect the user. I tried to read a byte and every second and it is reading it. I sent a byte and it is sent as well.