How to read data from Telnet session - go

I'm trying to create a tool to connect to a network device by Telnet and send some commands (Expect-like with certain additional requirements) using go-telnet.
To the moment I managed to create a connection and send commands with something like this:
func main() {
var loginBuffer = [6]byte{'r', 'o', 'o', 't', '\r', '\n'}
var login = loginBuffer[:]
conn, err := telnet.DialTo("10.10.10.2:23")
if nil != err {
fmt.Println(err)
}
defer conn.Close()
conn.Write(login)
}
Using Wireshark I can see the device responding, however I cannot read any response data. Guess I'm using Read() in a wrong way, not sure.
Would appreciate a working example or an explanation of how to capture and process response data in this case.

Thanks everyone, who spared their time to answer my question. I managed to identify the problem:
Every time I created a read buffer it was too big (1024 bytes) so the program was waiting for it to fill up. Now I'm using a cycle reading to a 1 byte buffer.
It seems, I also needed some criterion for the function to stop reading and proceed with sending commands.
Here is the working piece of code:
// Thin function reads from Telnet session. "expect" is a string I use as signal to stop reading
func ReaderTelnet(conn *telnet.Conn, expect string) (out string) {
var buffer [1]byte
recvData := buffer[:]
var n int
var err error
for {
n, err = conn.Read(recvData)
fmt.Println("Bytes: ", n, "Data: ", recvData, string(recvData))
if n <= 0 || err != nil || strings.Contains(out, expect) {
break
} else {
out += string(recvData)
}
}
return out
}
//convert a command to bytes, and send to Telnet connection followed by '\r\n'
func SenderTelnet(conn *telnet.Conn, command string) {
var commandBuffer []byte
for _, char := range command {
commandBuffer = append(commandBuffer, byte(char))
}
var crlfBuffer [2]byte = [2]byte{'\r', '\n'}
crlf := crlfBuffer[:]
fmt.Println(commandBuffer)
conn.Write(commandBuffer)
conn.Write(crlf)
}
func main() {
conn, err := telnet.DialTo("10.10.10.2:23")
if nil != err {
fmt.Println(err)
}
fmt.Print(ReaderTelnet(conn, "Login"))
SenderTelnet(conn, "root")
fmt.Print(ReaderTelnet(conn, "Password"))
SenderTelnet(conn, "root")
fmt.Print(ReaderTelnet(conn, ">"))
}

Where is your read operation from connection ?
I think you need to call conn.read(buffer) to read from the connection and write it to buffer
https://godoc.org/github.com/reiver/go-telnet#Conn.Read
The examples are not helping much with hte package that you are using. May be the following example taken out of different telnet go package would be more helpful.
https://github.com/ziutek/telnet/blob/master/examples/unix-cisco/main.go

Related

Streaming Stdout and Stderr over SSH, manipulate the stream and then print to local Stdout, and Stderr

I'm performing a bunch of operations over SSH on a remote machine and I'm streaming its stdout and stderr and then consuming it to by a writer, which writes to the local stdout and stderr, along with byte buffers.
Just before the writer consumes it, I want to perform a series of string manipulations on it and then write to my screen and buffer. Up to this point, it all works fine and dandy.
My issue is now it's not a stream anymore, it hangs and then outputs the whole glob in one chunk. I want it to be real time, so I put channels in my go routines but with no improvement. Below are my functions, let me know if you can spot a reason why, or possibly a better way of achieving this.
// sending
func handleStdStream(filters []string, replaceFilters map[string]string, pipe io.Reader, readers chan io.Reader) {
if filters != nil {
// filters exist
// read first 8 bytes
res := readPipe(8, pipe)
// get each line from the resulting streamed output
for _, str := range strings.Split(res, "\n") {
if str != "" {
out := lineFilterAndReplace(str, filters, replaceFilters)
// instantiate an io.Reader obj from the given string
outReader := strings.NewReader(out)
readers <- outReader
}
}
} else {
// filters dont exist
if len(replaceFilters) > 0 {
res := readPipe(8, pipe)
for _, str := range strings.Split(res, "\n") {
if str != "" {
out := lineReplace(str, replaceFilters)
// instantiate an io.Reader obj from the given string
outReader := strings.NewReader(out)
readers <- outReader
}
}
} else {
readers <- pipe
}
}
}
// recieving
outReaders := make(chan io.Reader)
go handleStdStream(outFilters, replaceFilters, stdoutIn, outReaders)
go func() {
for {
pipe := <-outReaders
_, errStdout = io.Copy(outWriter, pipe)
}
// _, errStdout = io.Copy(outWriter, stdoutIn)
}()
I don't think you need channels or goroutines to accomplish this. The Writer and Reader interfaces are already streaming; you sip bytes from a Reader continuously until you hit EOF or an error and you hand off bytes to a Writer continuously until you're done or you get an error. On its own, processing a stream does not require any concurrency, so doing this sequentially in a single goroutine is quite appropriate.
You shouldn't ignore error returns. If a function or method returns an error value, you need to check it. In the case of I/O, you usually need to stop reading from a Reader when it returns an error and you usually need to stop writing to a Writer when it returns an error. In the case of a Reader you also have to check for the special "error" value io.EOF.
I think using Scanner from the bufio package is better than trying to do your own buffering/splitting. By default, Scanner splits input on newlines (Unix-style LF or DOS-style CRLF). It also gets rid of the need to check for io.EOF, provided you only interact with the Reader through the Scanner.
Consider the following version of handleStdStream:
func handleStdStream(filters []string, replaceFilters map[string]string, pipe io.Reader, w io.Writer) error {
scanner := bufio.NewScanner(pipe)
for scanner.Scan() {
str := scanner.Text()
if str == "" {
continue
}
out := ""
if len(filters) != 0 {
out = lineFilterAndReplace(str, filters, replaceFilters)
} else {
out = lineReplace(str, replaceFilters)
}
if _, err := w.Write([]byte(out)); err != nil {
return err
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
}
You would use it like this:
err := handleStdStream(filters, replaceFilters, pipe, outWriter)
if err != nil {
// do something, like printing the error to a log or stderr
}

Golang reading from serial

I'm trying to read from a serial port (a GPS device on a Raspberry Pi).
Following the instructions from http://www.modmypi.com/blog/raspberry-pi-gps-hat-and-python
I can read from shell using
stty -F /dev/ttyAMA0 raw 9600 cs8 clocal -cstopb
cat /dev/ttyAMA0
I get well formatted output
$GNGLL,5133.35213,N,00108.27278,W,160345.00,A,A*65
$GNRMC,160346.00,A,5153.35209,N,00108.27286,W,0.237,,290418,,,A*75
$GNVTG,,T,,M,0.237,N,0.439,K,A*35
$GNGGA,160346.00,5153.35209,N,00108.27286,W,1,12,0.67,81.5,M,46.9,M,,*6C
$GNGSA,A,3,29,25,31,20,26,23,21,16,05,27,,,1.11,0.67,0.89*10
$GNGSA,A,3,68,73,83,74,84,75,85,67,,,,,1.11,0.67,0.89*1D
$GPGSV,4,1,15,04,,,34,05,14,040,21,09,07,330,,16,45,298,34*40
$GPGSV,4,2,15,20,14,127,18,21,59,154,30,23,07,295,26,25,13,123,22*74
$GPGSV,4,3,15,26,76,281,40,27,15,255,20,29,40,068,19,31,34,199,33*7C
$GPGSV,4,4,15,33,29,198,,36,23,141,,49,30,172,*4C
$GLGSV,3,1,11,66,00,325,,67,13,011,20,68,09,062,16,73,12,156,21*60
$GLGSV,3,2,11,74,62,177,20,75,53,312,36,76,08,328,,83,17,046,25*69
$GLGSV,3,3,11,84,75,032,22,85,44,233,32,,,,35*62
$GNGLL,5153.35209,N,00108.27286,W,160346.00,A,A*6C
$GNRMC,160347.00,A,5153.35205,N,00108.27292,W,0.216,,290418,,,A*7E
$GNVTG,,T,,M,0.216,N,0.401,K,A*3D
$GNGGA,160347.00,5153.35205,N,00108.27292,W,1,12,0.67,81.7,M,46.9,M,,*66
$GNGSA,A,3,29,25,31,20,26,23,21,16,05,27,,,1.11,0.67,0.89*10
$GNGSA,A,3,68,73,83,74,84,75,85,67,,,,,1.11,0.67,0.89*1D
$GPGSV,4,1,15,04,,,34,05,14,040,21,09,07,330,,16,45,298,34*40
(I've put some random data in)
I'm trying to read this in Go. Currently, I have
package main
import "fmt"
import "log"
import "github.com/tarm/serial"
func main() {
config := &serial.Config{
Name: "/dev/ttyAMA0",
Baud: 9600,
ReadTimeout: 1,
Size: 8,
}
stream, err := serial.OpenPort(config)
if err != nil {
log.Fatal(err)
}
buf := make([]byte, 1024)
for {
n, err := stream.Read(buf)
if err != nil {
log.Fatal(err)
}
s := string(buf[:n])
fmt.Println(s)
}
}
But this prints malformed data. I suspect that this is due to the buffer size or the value of Size in the config struct being wrong, but I'm not sure how to get those values from the stty settings.
Looking back, I think the issue is that I'm getting a stream and I want to be able to iterate over lines of the stty, rather than chunks. This is how the stream is outputted:
$GLGSV,3
,1,09,69
,10,017,
,70,43,0
69,,71,3
2,135,27
,76,23,2
32,22*6F
$GLGSV
,3,2,09,
77,35,30
0,21,78,
11,347,,
85,31,08
1,30,86,
72,355,3
6*6C
$G
LGSV,3,3
,09,87,2
4,285,30
*59
$GN
GLL,5153
.34919,N
,00108.2
7603,W,1
92901.00
,A,A*6A
The struct you get back from serial.OpenPort() contains a pointer to an open os.File corresponding to the opened serial port connection. When you Read() from this, the library calls Read() on the underlying os.File.
The documentation for this function call is:
Read reads up to len(b) bytes from the File. It returns the number of bytes read and any error encountered. At end of file, Read returns 0, io.EOF.
This means you have to keep track of how much data was read. You also have to keep track of whether there were newlines, if this is important to you. Unfortunately, the underlying *os.File is not exported, so you'll find it difficult to use tricks like bufio.ReadLine(). It may be worth modifying the library and sending a pull request.
As Matthew Rankin noted in a comment, Port implements io.ReadWriter so you can simply use bufio to read by lines.
stream, err := serial.OpenPort(config)
if err != nil {
log.Fatal(err)
}
scanner := bufio.NewScanner(stream)
for scanner.Scan() {
fmt.Println(scanner.Text()) // Println will add back the final '\n'
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
Change
fmt.Println(s)
to
fmt.Print(s)
and you will probably get what you want.
Or did I misunderstand the question?
Two additions to Michael Hamptom's answer which can be useful:
line endings
You might receive data that is not newline-separated text. bufio.Scanner uses ScanLines by default to split the received data into lines - but you can also write your own line splitter based on the default function's signature and set it for the scanner:
scanner := bufio.NewScanner(stream)
scanner.Split(ownLineSplitter) // set custom line splitter function
reader shutdown
You might not receive a constant stream but only some packets of bytes from time to time. If no bytes arrive at the port, the scanner will block and you can't just kill it. You'll have to close the stream to do so, effectively raising an error. To not block any outer loops and handle errors appropriately, you can wrap the scanner in a goroutine that takes a context. If the context was cancelled, ignore the error, otherwise forward the error. In principle, this can look like
var errChan = make(chan error)
var dataChan = make(chan []byte)
ctx, cancelPortScanner := context.WithCancel(context.Background())
go func(ctx context.Context) {
scanner := bufio.NewScanner(stream)
for scanner.Scan() { // will terminate if connection is closed
dataChan <- scanner.Bytes()
}
// if execution reaches this point, something went wrong or stream was closed
select {
case <-ctx.Done():
return // ctx was cancelled, just return without error
default:
errChan <- scanner.Err() // ctx wasn't cancelled, forward error
}
}(ctx)
// handle data from dataChan, error from errChan
To stop the scanner, you would cancel the context and close the connection:
cancelPortScanner()
stream.Close()

NTP detection using golang, payload coming up empty

I'm working on detecting NTP using golang and the gopacket package. I'm using a pcap I downloaded from wireshark. I've got the following code for opening PCAPs and handling them :
func (d *DPI) readPCAP(pcapFile string) (*pcap.Handle, error) {
// Open file instead of device
handle, err := pcap.OpenOffline(pcapFile)
if err != nil {
return nil, err
}
return handle, nil
}
And this is the code I'm writing to perform the actual detection
func TestNTP(t *testing.T) {
dpi := newDPI()
handle, _ := dpi.readPCAP("data/pcap/NTP_sync.pcap")
var filter = "udp and port 123"
dpi.setFilter(handle,filter)
ntpPackets := 0
for packet := range dpi.getPacketChan(handle) {
fmt.Println("stuff: ",packet.ApplicationLayer().Payload())
if dpi.detectNTP(packet) == 1 {
ntpPackets++
} else {
fmt.Println(" Output : ", dpi.detectNTP(packet))
}
}
fmt.Println(" Total ntp packets ", ntpPackets)
}
The Payload content in the ApplicationLayer is coming up empty and I'm unable to figure out why this is happening.
Example screenshot when I print out the ApplicationLayer itself :
https://i.gyazo.com/6257f298a09e7403bbc0be5b8ac84ccc.png
Example screenshot when I print out the Payload :
https://i.gyazo.com/7f4abd449025f5d65160fdbecffa8181.png
Could use some help figuring out what I'm doing wrong.
Thanks!
Reading through the golang soure code, I came across this :
// NTP packets do not carry any data payload, so the empty byte slice is retured.
// In Go, a nil slice is functionally identical to an empty slice, so we
// return nil to avoid a heap allocation.
func (d *NTP) Payload() []byte {
return nil
}
So, apparently it's not supposed to carry a Payload. I've managed to perform the detection using layers.

How can I keep reading using net Conn Read method

I'm creating a simple chat server as a personal project to learn net package and some concurrency in go. My 1st idea is to make the server print whatever is send using nc command echo -n "hello" | nc -w1 -4 localhost 2016 -p 61865. However after the 1st read my code ignores the subsequent messages.
func (s *Server) messageReader(conn net.Conn) {
defer conn.Close()
buffer := make([]byte, 1024)
for {
//read buff
blen, err := conn.Read(buffer)
if err != nil {
log.Fatal(err)
}
message := string(buffer[:blen])
if message == "/quit" {
fmt.Println("quit command received. Bye.")
return
}
if blen > 0 {
fmt.Println(message)
buffer = buffer[:0]
}
}
}
// Run Start up the server. Manages join and leave chat
func (s *Server) Run() {
// Listen on port TCP 2016
listener, err := net.Listen("tcp", ":2016")
if err != nil {
log.Fatal(err)
}
defer listener.Close()
for {
//wait for connection
conn, err := listener.Accept()
if err != nil {
log.Fatal(err)
}
go s.messageReader(conn)
}
}
If I send a new message from a new client it prints without problems but if I send another one it does nothing. What am I missing do I need to reset the Conn or close it and spawn a new one?
After printing your message, you slice buffer down to zero length. You can't read any data into a zero-length slice. There's no reason to re-slice your read buffer at all.
You also need to handle the read bytes before checking for errors, as io.EOF can be returned on a successful read.
You shouldn't use log.Fatal in the server's read loop, as that calls os.Exit
A working messageReader body might look like:
defer conn.Close()
buffer := make([]byte, 1024)
for {
n, err := conn.Read(buffer)
message := string(buffer[:n])
if message == "/quit" {
fmt.Println("quit command received. Bye.")
return
}
if n > 0 {
fmt.Println(message)
}
if err != nil {
log.Println(err)
return
}
}
You should note though that because you're not using any sort of framing protocol here, you can't guarantee that each conn.Read returns a complete or single message. You need to have some sort of higher-level protocol to delimit messages in your stream.

Go: Server should block until a message from the client is received

I'm building some server/client application in Go (the language is new to me). I searched a lot and read a whole bunch of different examples but there is still one thing I can't find. Lets say I have a single server client up and running. The client will send some kind of a message to the server and vice versa. Encoding and decoding is done by the package gob.
This example is not my application, it is only a quick example:
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type Message struct {
Sender string
Receiver string
Command uint8
Value int64
}
func (message *Message) Set(sender string, receiver string, command uint8, value int64) *Message {
message.Sender = sender
message.Receiver = receiver
message.Command = command
message.Value = value
return message
}
func main() {
var network bytes.Buffer // Stand-in for a network connection
enc := gob.NewEncoder(&network) // Will write to network.
dec := gob.NewDecoder(&network) // Will read from network.
message := new(Message).Set("first", "second", 10, -1)
err := enc.Encode(*message) // send message
if err != nil {
log.Fatal("encode error:", err)
}
var m Message
err = dec.Decode(&m) // receice message
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Printf("%q %q %d %d\n", m.Sender, m.Receiver, m.Command, m.Value)
}
This works fine, but I want the server to block until a new message is received so I can put the receiving process inside a infinite for loop inside a goroutine.
Something like that:
for {
// The server blocks HERE until a message from the client is received
fmt.Println("Received message:")
// Decode the new message
var m Message
err = dec.Decode(&m) // receice message
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Printf("%q %q %d %d\n", m.Sender, m.Receiver, m.Command, m.Value)
}
The gob decoder blocks until it has read a full message or there's an error. The read loop in the question works as is.
working example on the playground
add a length header to the raw tcp stream.
that means, send a 4-bytes-length-header information to server before send the real load. and in server side read 4 bytes, allocate buffer, full read total message, and finally decode.
assume you have a tcp connection conn, in server side we could have:
func getInt(v []byte) int {
var r uint
r = 0
r |= uint(v[0]) << 24
r |= uint(v[1]) << 16
r |= uint(v[2]) << 8
r |= uint(v[3]) << 0
return int(r)
}
buf := make([]byte, 4)
_, err := io.ReadFull(conn, buf)
if err != nil {
return
}
length := getInt(buf)
buf = make([]byte, length)
_, err = io.ReadFull(conn, buf)
if err != nil {
return
}
//do gob decode from `buf` here
you may know client side refer the the server side source I think.

Resources