I do have a process receiving data from second local process. I need to test if connection errors are handled well AND if it automatically reconnects and keep receiving data after a disconnection.
To do so I am trying to make it disconnect abruptly or put the TCP connection in an error state from a unit test.
As seen in this question to check if a connection is closed I am checking for data to come and test if it returns an error.
I am not sure how to:
close the connection ungracefully
make it be in an error state
This is the essence of my data receiver:
import (
"bufio"
"encoding/json"
"fmt"
"io"
"net"
)
type Message struct {
ID string `json:"id"`
}
func ReceiveData(listener Listener) {
var tcpConn net.Conn
var addr string = "127.0.0.1:9999"
tcpConn, err := net.Dial("tcp", addr)
socketReader := bufio.NewReader(tcpConn)
decoder := json.NewDecoder(socketReader)
for {
var msg Message
if err := decoder.Decode(&msg); err == io.EOF {
listener.ProcessUpdate(Message{}, fmt.Errorf("Received EOF"), nil)
tcpConn = nil
return
} else if err != nil {
listener.ProcessUpdate(Message{}, nil, fmt.Errorf("Error decoding message: %s", err.Error()))
tcpConn = nil
return
}
// process message
_ = msg
// Test disconnection
// This does not disconnect:
// tcpConn = nil
// This does but gracefully:
// tcpConn.Close()
}
I am not mocking the TCP connection as I'd like to try with the real data producer. If is needed I'll look at it.
A solution is to set a deadline to the TCP connection itself:
tcpConn.SetDeadline(time.Now())
Later this will trigger a timeout error which can be caught with:
err := decoder.Decode(&msg);
if err != nil {
if neterr, ok := err.(net.Error); ok && neterr.Timeout() {
fmt.Errorf("TCP timeout : %s", err.Error())
} else {
fmt.Errorf("Received error decoding message: %s", err.Error())
}
}
Related
I have a simple UDP struct I made in Go, and when I run the following code, it blocks forever. For reference, the address of my server is running on the same computer running the client, but the server listens on a different port and address than the one that is bound to by the client.
var client Clientee
client.Create("the address of my server")
err, messages := client.Read() // <-- HERE IT BLOCKS FOREVER
if err != nil { panic(err) }
fmt.Printf("Messages: %s", messages)
Here's the part of the code that declares my struct:
package controllers
import (
"fmt"
"net"
"time"
)
const (
BUF_SIZE = 1024
CLIENT_PORT = "4097"
SERVER_PORT = "4096"
)
type Clientee struct {
ClServerAddr *net.UDPAddr
ClLocalAddr *net.UDPAddr
ClConn *net.UDPConn
ClWasShutdown bool
}
// Initialize and connect the Clientee
func (c *Clientee) Create(hostAddr string) error {
var err error
c.ClWasShutdown=false
// Resolve the server's address
c.ClServerAddr, err = net.ResolveUDPAddr("udp", hostAddr+":"+SERVER_PORT)
if err != nil { return err }
fmt.Println("Server addr = ",c.ClServerAddr.String())
// Resolve our local address
c.ClLocalAddr, err = net.ResolveUDPAddr("udp", ":"+CLIENT_PORT)
if err != nil { return err }
// Create the connection
c.ClConn, err = net.ListenUDP("udp", c.ClLocalAddr)
if err != nil { return err }
// Pause
time.Sleep(time.Millisecond*200)
return nil
}
// Send a message to the Server
func (c *Clientee) Send(msg string) error {
_, err := c.ClConn.WriteToUDP([]byte(msg), c.ClServerAddr)
if err!=nil { return err }
return nil
}
// Read messages from the Server
func (c *Clientee) Read() (error, string) {
bfr:=make([]byte, BUF_SIZE) // Make the buffer
n, addr, err := c.ClConn.ReadFromUDP(bfr)
if err!=nil { return err, "" }
// If the message doesn't come from the server, don't return it
if addr.String()!=c.ClServerAddr.String() {
return nil, ""
}
return nil, string(bfr[0:n])
}
// Close the Connection.
func (c *Clientee) Close() error {
return c.ClConn.Close()
}
ReadFromUDP will block until something is received. The docs refer you to ReadFrom which says "ReadFrom implements the PacketConn ReadFrom method.". Looking at the PacketConn docs you will find the following:
ReadFrom reads a packet from the connection, copying the payload into
p. It returns the number of bytes copied into p and the return address
that was on the packet. It returns the number of bytes read (0 <= n <=
len(p)) and any error encountered. Callers should always process the n > 0
bytes returned before considering the error err.
ReadFrom can be made to time out and return an Error with Timeout() == true after a
fixed time limit; see SetDeadline and SetReadDeadline.
So if you do not want this to block (as per the comments) then you can either:
Use SetDeadline or SetReadDeadline to set a deadline.
Run ReadFromUDP within a goroutine and handle data as it's received (which may include putting received data onto a channel)
Note: Calling ReadFromUDP with a zero length buffer may not block but this depends upon the operating system implementation so is probably not something to rely on.
I'm doing some tests based on the idea of pwnat, it introduced a method for NAT traversal without 3rd party: the server sends ICMP echo request packets to the fixed address(for example, 3.3.3.3) where no echo replies won't be returned from, the client, pretending to be a hop on the Internet, sends an ICMP Time Exceeded packet to the server, expect the NAT in the front of the server to forward the ICMP time exceeded message to the server.
After I pinged to 3.3.3.3, then I run the code below in 192.168.1.100 to listen ICMP messages in Go:
package main
import (
"fmt"
"golang.org/x/net/icmp"
"golang.org/x/net/ipv4"
)
func main() {
c, err := icmp.ListenPacket("ip4:icmp", "0.0.0.0")
if err != nil {
fmt.Println("listen error", err)
}
rb := make([]byte, 1500)
for {
n, _, err := c.ReadFrom(rb)
if err != nil {
fmt.Printf("read err: %s\n", err)
}
reply, err := icmp.ParseMessage(1, rb[:n])
if err != nil {
fmt.Println("parse icmp err:", err)
return
}
switch reply.Type {
case ipv4.ICMPTypeTimeExceeded:
if _, ok := reply.Body.(*icmp.TimeExceeded); ok {
// internet header(20 bytes) plus the first 64 bits of the original datagram's data
//fmt.Println("recv id ", binary.BigEndian.Uint16(timeExceed.Data[22:24]))
fmt.Printf("ttl exceeded\n")
}
default:
}
}
}
and a program which runs in 192.168.2.100 to send forged time exceeded message to 192.168.1.100:
package main
import (
"errors"
"fmt"
"golang.org/x/net/icmp"
"golang.org/x/net/ipv4"
"net"
"os"
)
func sendTtle(host string) error {
conn, err := net.Dial("ip4:icmp", host)
if err != nil {
return err
}
// original IP header
h := ipv4.Header{
Version: 4,
Len: 20,
TotalLen: 20 + 8,
TTL: 64,
Protocol: 1,
}
h.Src = net.ParseIP(host)
h.Dst = net.ParseIP("3.3.3.3")
iph, err := h.Marshal()
if err != nil {
fmt.Println("ip header error", err)
return err
}
// 8 bytes of original datagram's data
echo := icmp.Message{
Type: ipv4.ICMPTypeEcho, Code: 0,
Body: &icmp.Echo{
ID: 3456, Seq: 1,
}}
oriReq, err := echo.Marshal(nil)
if err != nil {
return errors.New("Marshal error")
}
data := append(iph, oriReq...)
te := icmp.Message{
Type: ipv4.ICMPTypeTimeExceeded,
Code: 0,
Body: &icmp.TimeExceeded{
Data: data,
}}
if buf, err := te.Marshal(nil); err == nil {
fmt.Println("sent")
if _, err := conn.Write(buf); err != nil {
return errors.New("write error")
}
} else {
return errors.New("Marshal error")
}
return nil
}
func main() {
argc := len(os.Args)
if argc < 2 {
fmt.Println("usage: prpgram + host")
return
}
if err := sendTtle(os.Args[1]); err != nil {
fmt.Println("failed to send TTL exceeded message: ", err)
}
}
the problem is 192.168.1.100 cannot receive the message. What're the possible reasons?
Your code has no problem. If you run your code in the same network(I mean no NAT/router involvement), the program will receive time exceeded message as expected. The reason is the theory pwnat uses doesn't work nowadays.
First, you didn't get the identifier of the echo request sent by
192.168.2.100 to 3.3.3.3, the identifier will be uniquely
mapped to an external query ID by NAPT(if any) so that it can route
future ICMP Echo Replies with the same query ID to the sender. According to rfc 3022 ICMP error packet modifications section,
In a NAPT setup, if the IP message embedded within ICMP happens to be
a TCP, UDP or ICMP Query packet, you will also need to modify the
appropriate TU port number within the TCP/UDP header or the Query
Identifier field in the ICMP Query header.
Second, according to rfc 5508:
If a NAT device receives an ICMP Error packet from the private realm,
and the NAT does not have an active mapping for the embedded payload,
the NAT SHOULD silently drop the ICMP Error packet.
So the forged time exceeded message wouldn't get through. Here is more details about this.
I was performing some RPC tests, and stumbled across a problem I can't seem to solve. In my testing I create three separate RPC servers, all of which I try to close and shutdown. However upon performing my last test (TestRpcCodecServerClientComm), it seems my client connection is connecting to the first RPC server I started (I know this because I at some point attached IDs to the RPCHandlers), even though I attempted everything I could to make sure it was shutdown. Though the code is not there I have attempted to inspect every single error I could, but that did not bring about anything.
rpc.go
package rbot
import (
"io"
"net"
"net/rpc"
"net/rpc/jsonrpc"
)
func RpcCodecClientWithPort(port string) (rpc.ClientCodec, error) {
conn, err := net.Dial("tcp", "localhost:"+port)
if err != nil {
return nil, err
}
return jsonrpc.NewClientCodec(conn), nil
}
func RpcCodecServer(conn io.ReadWriteCloser) rpc.ServerCodec {
return jsonrpc.NewServerCodec(conn)
}
rpc_test.go
package rbot
import (
"errors"
"fmt"
"net"
"net/rpc"
"testing"
)
type RPCHandler struct {
RPCServer net.Listener
conn rpc.ServerCodec
done chan bool
TestPort string
stop bool
GotRPC bool
}
func (r *RPCHandler) SetupTest() {
r.stop = false
r.GotRPC = false
r.done = make(chan bool)
r.TestPort = "5556"
}
// TODO: Create separate function to handle erroring
func (r *RPCHandler) CreateRPCServer() error {
rpc.RegisterName("TestMaster", TestAPI{r})
var err error
r.RPCServer, err = net.Listen("tcp", ":"+r.TestPort)
if err != nil {
return err
}
go func() {
for {
conn, err := r.RPCServer.Accept()
if err != nil || r.stop {
r.done <- true
return
}
r.conn = RpcCodecServer(conn)
rpc.ServeCodec(r.conn)
}
}()
return nil
}
func (r *RPCHandler) CloseRPCServer() error {
r.stop = true
if r.conn != nil {
err := r.conn.Close()
if err != nil {
fmt.Println(err)
}
}
err := r.RPCServer.Close()
<-r.done
return err
}
type TestAPI struct {
t *RPCHandler
}
func (tapi TestAPI) Send(msg string, result *string) error {
if msg == "Got RPC?" {
tapi.t.GotRPC = true
return nil
}
return errors.New("Didn't receive right message")
}
// Check if we can create and close an RPC server successfully using the RPC server codec.
func TestRpcCodecServer(t *testing.T) {
r := RPCHandler{}
r.SetupTest()
err := r.CreateRPCServer()
if err != nil {
t.Fatalf("Could not create rpc server! %s:", err.Error())
}
err = r.CloseRPCServer()
if err != nil {
t.Fatalf("Could not close RPC server! %s:", err.Error())
}
}
// Check if we can create a client without erroring.
func TestRpcCodecClientWithPortt(t *testing.T) {
r := RPCHandler{}
r.SetupTest()
r.CreateRPCServer()
defer r.CloseRPCServer()
RPCClient, err := RpcCodecClientWithPort(r.TestPort)
defer RPCClient.Close()
if err != nil {
t.Fatalf("Could not create an RPC client! %s:", err.Error())
}
}
// Let's double check and make sure our server and client can speak to each other
func TestRpcCodecServerClientComm(t *testing.T) {
r := RPCHandler{}
r.SetupTest()
r.CreateRPCServer()
defer r.CloseRPCServer()
RPCCodec, _ := RpcCodecClientWithPort(r.TestPort)
RPCClient := rpc.NewClientWithCodec(RPCCodec)
defer RPCClient.Close()
var result string
err := RPCClient.Call("TestMaster.Send", "Got RPC?", &result)
if err != nil {
t.Fatalf("Error while trying to send RPC message: %s", err.Error())
}
if !r.GotRPC {
t.Fatalf("Could not send correct message over RPC")
}
}
Not sure if I'm just mishandling the connection or something of the like, any help would be much appreciated.
For the Record The RPC api does receive the correct string message
While not the source of your problems, your test configuration has a few race conditions which you should take care of before they cause problems. Always check for issues with the -race option. You should also let the OS allocate the port so you don't run into conflicts. See for example how httptest.Server works.
Your failure here is that you're not creating a new rpc.Server for each test, you're reusing the rpc.DefaultServer. The first call to CreateRPCServer registers a TestAPI under the name TestMaster. Each subsequent call uses the already registered instance.
If you create a new rpc.Server each time you setup the test and register a new TestAPI, the final test will pass.
srv := rpc.NewServer()
srv.RegisterName("TestMaster", testAPI)
...
// and then use srv to handle the new connection
srv.ServeCodec(RpcCodecServer(conn))
So far I've been doing this:
import (
_redis "gopkg.in/redis.v3"
"strconv"
"time"
)
type Redis struct {
Connector *_redis.Client
PubSub *_redis.PubSub
}
var redis *Redis = nil
func NewRedis() bool {
if redis == nil {
redis = new(Redis)
redis.Connector = _redis.NewClient(&_redis.Options{
Addr: config.RedisHostname + ":" + strconv.FormatInt(config.RedisPort, 10),
Password: "",
DB: 0,
})
Logger.Log(nil, "Connected to Redis")
err := redis.Init()
if err != nil {
Logger.Fatal(nil, "Cannot setup Redis:", err.Error())
return false
}
return true
}
return false
}
func (this *Redis) Init() error {
pubsub, err := this.Connector.Subscribe("test")
if err != nil {
return err
}
defer pubsub.Close()
this.PubSub = pubsub
for {
msgi, err := this.PubSub.ReceiveTimeout(100 * time.Millisecond)
if err != nil {
Logger.Error(nil, "PubSub error:", err.Error())
err = this.PubSub.Ping("")
if err != nil {
Logger.Error(nil, "PubSub failure:", err.Error())
break
}
continue
}
switch msg := msgi.(type) {
case *_redis.Message:
Logger.Log(nil, "Received", msg.Payload, "on channel", msg.Channel)
}
}
return nil
}
My Connector is a redis.Client, it's working because I was able to publish messages as well.
When I run my program, I get the following error:
PubSub error: WSARecv tcp 127.0.0.1:64505: i/o timeout
Do you have any idea of what I'm doing wrong ? I'm using this package: https://github.com/go-redis/redis
Some things to note:
(implementation detail) When redis goes into PubSub mode, the only thing that happens on that socket afterwards is PubSub events, which is why PubSub in go-redis is abstracted into its own type
A PubSub client can potentially subscribe to multiple topics in a single subscriber, hence why there are subscribe/unsubscribe events throughout.
the interface has both Receive() and ReceiveTimeout(duration) methods, both of which return the next event on the wire; which can be subscribe/unsubscribe events and message events; (you don't necessarily know which) the only difference between them that Receive blocks forever until there's a new message, and ReceiveTimeout will error on timeout.
With that in mind, unless you have messages far more than 10/second consistently (in other words, <100 milliseconds between messages), it's inefficient to use that short of a timeout; and I'd argue that due to golang having goroutines, you should almost never use ReceiveTimeout for real applications, or use a sufficiently long timeout like a minute.
with that in mind, your receive loop should look like:
for {
msgi, err := this.PubSub.Receive()
if err != nil {
Logger.Error(nil, "PubSub error:", err.Error())
return err
}
switch msg := msgi.(type) {
case *_redis.Message:
Logger.Log(nil, "Received", msg.Payload, "on channel", msg.Channel)
default:
Logger.Log(nil, "Got control message", msg)
}
}
If your application really warranted using a timeout, then you should use a type assertion to assert the *net.OpError that signifies a timeout and distinguish it from other more serious errors.
i have a problem with my golang server in which i'm using websockets.
The server opens the connection and the client could connect to it, but the problem is that when the server starts sending the data to the client, the client connection is closed after a small period of time. i suggest that the problem is with the server and not with the client because i tried to connect to the server with another web client, and it's the same issue. I didn't understand the cause ! Can someone help me?
server.go:
func Echo(ws *websocket.Conn) {
fmt.Println("Echoing")
for {
msg := MessageReceived{Name: "OrderCommand", Nbmsg: 3}
if err := websocket.JSON.Send(ws, msg); err != nil {
fmt.Println("Can't send")
break
}
//os.Exit(0)
}
}
func checkError(err error) {
if err != nil {
Log("Fatal error ", err.Error())
os.Exit(1)
}
}
func main() {
http.HandleFunc("/save", saveHandler)
http.Handle("/", websocket.Handler(Echo))
err:= http.ListenAndServe(":8081", nil)
checkError(err)
}
and client.go:
import (
"code.google.com/p/go.net/websocket"
"fmt"
"log"
)
func main() {
origin := "http://localhost/"
url := "ws://localhost:8081/echo"
ws, err := websocket.Dial(url, "", origin)
if err != nil {
log.Fatal(err)
}
var msg = make([]byte, 512)
var n int
if n, err = ws.Read(msg); err != nil {
log.Fatal(err)
}
fmt.Printf("Received: %s.\n", msg[:n])
}
Your problem, as others have pointed out, is that you must receive a message as well.
Currently, when someone connects, your program will step into the for-loop and start bombarding the client with messages. Probably not the intended behaviour of an echo server.
First you want to Receive a message, then Send a reply:
func Echo(ws *websocket.Conn) {
fmt.Println("Echoing")
msg := new(MessageReceived)
for {
// The server blocks here until a message from the client is received
websocket.JSON.Receive(ws, &msg)
fmt.Printf("Received message: %+v\n", msg)
// Reencode the same message and send it back
if err := websocket.JSON.Send(ws, msg); err != nil {
fmt.Println("Can't send echo")
break
}
}
}
A full working version can be found at Playground: http://play.golang.org/p/nQ3fJ5Nb0I
Since it uses websockets, you must compile it on your local computer.
Why using ws.Read when you can use websocket.JSON.Receive to deserialize the message?
Here are the server: http://play.golang.org/p/NZ6VJ4daGm
and the client: http://play.golang.org/p/rkJVKGhrGk (that I have changed to receive 10 messages before exiting).
The string "Can't send" will be printed by the server once the client closes the websocket connection.