I'm trying to build a ISO8583 Client using a Golang,
when using java, i don't have any problem when creating client.
But when trying creating a client using golang (i'm just starting learning golang btw), i can't send a message to the server. Can someone help me, why i can't send a message?
I've tried to send a SIGN IN Message, The client and server already connected, but the message i send not readable by the server.
This My Code
package main
import (
"bufio"
"fmt"
"net"
"os"
"time"
"github.com/ideazxy/iso8583"
)
type ISOSignIn struct {
Bit3 *iso8583.Numeric `field:"3" length:"6" encode:"bcd"`
Bit7 *iso8583.Numeric `field:"7" length:"10" encode:"ascii`
Bit11 *iso8583.Numeric `field:"11" length:"6" encode:"rbcd`
Bit32 *iso8583.Llnumeric `field:"32" length:"11" encode:"ascii`
Bit70 *iso8583.Numeric `field:"70" length:"3" encode:"ascii`
}
func main() {
testIso()
}
func testIso() {
data := ISOSignIn{
Bit3: iso8583.NewNumeric("001111"),
Bit7: iso8583.NewNumeric("0913110004"),
Bit11: iso8583.NewNumeric("000001"),
Bit32: iso8583.NewLlnumeric("9999"), // Client ID
Bit70: iso8583.NewNumeric("301"), // Login Code
}
msg := iso8583.NewMessage("0800", data)
msg.MtiEncode = iso8583.BCD
b, err := msg.Bytes()
if err != nil {
fmt.Println(err.Error())
}
fmt.Printf("% x\n", b)
tcpClientNew(b)
}
func tcpClientNew(b []byte) {
tcpAddr, err := net.ResolveTCPAddr("tcp", "192.168.100.5:12346")
if err != nil {
println("ResolveTCPAddr failed:", err.Error())
os.Exit(1)
}
conn, err := net.DialTCP("tcp", nil, tcpAddr)
if err != nil {
println("Dial failed:", err.Error())
os.Exit(1)
}
timeoutDuration := 30 * time.Second
_, err = conn.Write(b)
if err != nil {
println("Write to server failed:", err.Error())
os.Exit(1)
}
conn.SetReadDeadline(time.Now().Add(timeoutDuration))
bufReader := bufio.NewReader(conn)
resp, _ := bufReader.ReadByte()
fmt.Print("Message from server: " + string(resp))
conn.Close()
}
Server Already Connected
<log realm="Server-A.server.session/192.168.100.1:32218" at="Mon Jan 07 09:37:15.747 WIB 2019">
<session-start/>
</log>
<log realm="channel/192.168.100.1:32218" at="Mon Jan 07 09:37:19.034 WIB 2019" lifespan="3287ms">
<receive>
<peer-disconnect/>
</receive>
</log>
<log realm="Server-A.server.session/192.168.100.1:32218" at="Mon Jan 07 09:37:19.035 WIB 2019">
<session-end/>
</log>
Output from Client Terminal :
GOROOT=/Users/ivanaribanilia/Applications/go
GOPATH=/Users/ivanaribanilia/Project/Workspace-Github/Project-Go/pclient
/Users/ivanaribanilia/Applications/go/bin/go build -i -o /Users/ivanaribanilia/Project/Workspace-Github/Project-Go/pclient/build/pclient /Users/ivanaribanilia/Project/Workspace-Github/Project-Go/pclient/src/github.com/ivanj4u/pclient/main.go
/Users/ivanaribanilia/Project/Workspace-Github/Project-Go/pclient/build/pclient
08 00 22 20 00 01 00 00 00 00 00 11 11 30 39 31 33 31 31 30 30 30 34 30 30 30 30 30 31 30 34 31 31 31 34
Message from server:
Process finished with exit code 0
I expect a response from the server, so i can develop other message like INQUIRY or PAYMENT.
Thank you
ReadByte reads and returns a single byte. If no byte is available,
returns an error.
seems that what you read from server is only one byte, which is a white-space char.
The server and client should make a protocol when to close the connection. Thus, if server don't close the conn actively, client should read all bytes from server and close the connection. Like this:
recvBuf := make([]byte, 1024)
n, err := bufReader.Read(recvBuf)
for err == nil {
println("Recv data from server:", string(recvBuf[:n]))
n, err = bufReader.Read(recvBuf)
}
if err != io.EOF {
println("recv from server failed, err:", err)
}
conn.Close()
Or if the protocol defines the client should close the connection when received a certain byte, client can use ReadBytes() and close the connection actively.
func (b *Reader) ReadBytes(delim byte) ([]byte, error)
ReadBytes reads until the first occurrence of delim in the input,
returning a slice containing the data up to and including the
delimiter. If ReadBytes encounters an error before finding a
delimiter, it returns the data read before the error and the error
itself (often io.EOF). ReadBytes returns err != nil if and only if the
returned data does not end in delim.
Related
I have []byte of zip file. I have to unzip it without creating a new file, and get a []byte of that unzipped file. Please help me to do that.
I am making an API call and the response I get is the []byte in zipped format - I am trying to unzip it - and use it's content for creating a new zip file. So unzip - rezip.
Language: Golang
Code I've used:
func UnzipBytes(zippedBytes []byte) ([]byte, error) {
reader := bytes.NewReader(zippedBytes)
zipReader, err := zlib.NewReader(reader)
if err != nil {
return nil, err
}
defer zipReader.Close()
p, err := ioutil.ReadAll(zipReader)
if err != nil {
return nil, err
}
return p, nil
}
I get an error saying "zlib: invalid header"
The code that was initially used to zip the []byte
buffer := new(bytes.Buffer)
zipWriter := zip.NewWriter(buffer)
zipFile, err := zipWriter.Create(file.name)
_, err = zipFile.Write(file.content)
Hex dump of the []byte - the zippedBytes
00059350 78 b4 5b 0d 2b 81 c2 87 35 76 1b 11 4a ec 07 d1 |x.[.+...5v..J...|
00059360 76 77 a2 e1 3b d9 12 e2 51 d4 c5 bd 4b 2f 09 da |vw..;...Q...K/..|
00059370 f7 21 c7 26 73 1f 8e da f0 ff a3 52 f6 e2 00 e6 |.!.&s......R....|
You used zip.Writer to compress the data. You must close it by calling its Writer.Close() method. And you must use zip.Reader to read it, and use Reader.Open() with the same name you used when compressed it (file.name).
This is how it could look like:
func UnzipBytes(name string, zippedBytes []byte) ([]byte, error) {
reader := bytes.NewReader(zippedBytes)
zipReader, err := zip.NewReader(reader, int64(len(zippedBytes)))
if err != nil {
return nil, err
}
f, err := zipReader.Open(name)
if err != nil {
panic(err)
}
p, err := ioutil.ReadAll(f)
if err != nil {
return nil, err
}
return p, nil
}
Testing it:
filename := "test.txt"
filecontent := []byte("line1\nline2")
buffer := new(bytes.Buffer)
zipWriter := zip.NewWriter(buffer)
zipFile, err := zipWriter.Create(filename)
if err != nil {
panic(err)
}
if _, err = zipFile.Write(filecontent); err != nil {
panic(err)
}
if err = zipWriter.Close(); err != nil {
panic(err)
}
decoded, err := UnzipBytes(filename, buffer.Bytes())
fmt.Println(err)
fmt.Println(string(decoded))
This will output (try it on the Go Playground):
<nil>
line1
line2
If you don't know the name when decompressing, you may see all files in the Reader.Files header field. You may choose to open the first file:
func UnzipBytes(zippedBytes []byte) ([]byte, error) {
reader := bytes.NewReader(zippedBytes)
zipReader, err := zip.NewReader(reader, int64(len(zippedBytes)))
if err != nil {
return nil, err
}
if len(zipReader.File) == 0 {
return nil, nil // No file to open / extract
}
f, err := zipReader.File[0].Open()
if err != nil {
panic(err)
}
p, err := ioutil.ReadAll(f)
if err != nil {
return nil, err
}
return p, nil
}
This outputs the same. Try this one on the Go Playground.
I have pasted a minimally reproducible test code. In short, with SetMaxOpenConns set to 10, the program hangs forever after 10. I found this relevant thread from way back when, but it seems resolved and tested: https://github.com/golang/go/issues/6593
Note that by commenting out the SetMaxOpenConns the code runs normally.
What am I doing wrong? or should I open a new issue?
1 package main
2
3 import (
4 "database/sql"
5 "log"
6 "time"
7 _ "github.com/lib/pq"
8 )
9
10 func main(){
11 // Establish db connection
12 db, err := sql.Open("postgres", "host=0.0.0.0 port=5432 user=postgres password=password dbname=test sslmode=disable")
13 if err != nil {
14 log.Fatal(err)
15 }
16
17 db.SetMaxOpenConns(10) // commenting this line will resolve the problem
18 db.SetMaxIdleConns(10)
19 db.SetConnMaxLifetime(10 * time.Second)
20
21 // Query more than max open; note that hangs forever
22 for i:=0; i<12; i++ {
23 rows, err := Query(db)
24 if err != nil {
25 log.Fatal(err)
26 }
27 log.Println(i)
28 log.Println(rows)
29 }
30 }
31
32 func Query(db *sql.DB) (*sql.Rows, error){
33 stmt, err := db.Prepare("SELECT * FROM test;")
34 if err != nil {
35 log.Fatal(err)
36 }
37
38 defer stmt.Close()
39
40 rows, err := stmt.Query()
41 if err != nil {
42 log.Fatal(err)
43 }
44
45 return rows, nil
46 }
You need to either fully iterate through the result set with rows.Next and/or call rows.Close(); as per the docs:
Close closes the Rows, preventing further enumeration. If Next is called and returns false and there are no further result sets, the Rows are closed automatically and it will suffice to check the result of Err. Close is idempotent and does not affect the result of Err.
Something like:
for i:=0; i<12; i++ {
rows, err := Query(db)
if err != nil {
log.Fatal(err)
}
log.Println(i)
log.Println(rows)
if err = rows.Close(); err != nil {
panic(err)
}
}
For this to be useful you need to iterate through the rows (see the example in the docs).
The connection to the database will remain in use until the result set is closed (at which point it is returned to the pool). Because you are doing this in a loop you will end up with 10 active result sets and when you call Query() again the sql package will wait for a connection to become available (which will never happen).
Note that because your query has no parameters (and you are only using the stmt once) calling Prepare has no benefit; the following is simpler and will have the same result:
func Query(db *sql.DB) (*sql.Rows, error) {
return db.Query("SELECT * FROM test;")
}
I'm trying to get data from the UDP server but nothing shows in my client so basically, I send bytes [0x00 0x01 0x02 0x03 ] and the server will respond with data so the bytes are being sent successfully I don't know what's going on but the server is 100% fine when I send data using another app I get data back
conn2, err := net.Dial("udp", "192.168.1.240:2010")
if err != nil {
log.Println("error dialing remote addr", err)
}
request, err := hex.DecodeString("00010203")
if err != nil {
panic(err)
}
n, err := conn2.Write(request)
if err != nil {
return fmt.Errorf("unable to send request to weblist")
}
log.Printf("\033[33mWeb Server\033[0m : %d bytes sent successfully\n", n)
var buffer []byte = make([]byte, 1500)
for {
// Read from server
n, err := conn2.Read(buffer)
if err != nil {
panic(err)
}
//log.Printf("\033[33mWeb Server\033[0m : %s replied with %d bytes\n", addr.String(), n)
encodedStr := hex.EncodeToString(buffer[0:n])
log.Printf("\033[33mWeb Server\033[0m : %s\n", encodedStr)
}
Same code that runs fine in python
def main():
socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
remote_address = ('192.168.1.240', 2010)
logging = logging
socket.sendto(bytes.fromhex('01 02 03 04'), remote_address)
logging.debug('packet successfully sent')
while True:
readable, writable, exceptional = select.select([socket], [], [], 2)
for s in readable:
data, addr = s.recvfrom(1024)
if data[0:4] == bytes.fromhex('01 02 03 04'):
packet = bytes.fromhex('01 02 03 04')
auth_key = bytes(data[4:5] + data[5:6] + data[6:7] + data[7:8])
So I'm trying to get the string representation of a JSON message in Golang. I just want to receive the messagepack encoded JSON, modify some values and send it back.
I haven't found an easy way to do it. Most of the times, I can't know in advance what is the structure of the JSON (apart from having JSON structure), so want I want to do is to receive the binary message. Decode it as a JSON, print it as a string to the standard output, modify the content, convert it to MessagePack again and send it back.
I've been looking at these two packages, but I don't know how to properly use them for a simple task like that:
https://godoc.org/github.com/vmihailenco/msgpack
https://godoc.org/github.com/ugorji/go/codec
So I will receive something like this:
DF 00 00 00 01 A7 6D 65 73 73 61 67 65 A3 48 69 21
I want to print this:
{"message": "Hi!"}
Modify the "Hi!":
{"message": "Hello Sir!"}
Send it as messagepack:
DF 00 00 00 01 A7 6D 65 73 73 61 67 65 AA 48 65 6C 6C 6F 20 53 69 72 21
Current Python code I'm trying to port to Golang:
def decode_msgpack(jsonData):
packedStuff = 0
for key in jsonData.keys():
if type(jsonData[key]) is bytes:
packedStuff += 1
try:
jsonData[key] = umsgpack.unpackb(jsonData[key])
except umsgpack.InvalidStringException:
try:
jsonData[key] = umsgpack.unpackb(jsonData[key], allow_invalid_utf8=True)
except umsgpack.InsufficientDataException:
print("[!] InsufficientDataException")
jsonData[key] = base64.b64encode(jsonData[key]).decode('utf-8')
else:
jsonData[key] = base64.b64encode(jsonData[key]).decode('utf-8')
if packedStuff > 0:
return decode_msgpack(jsonData)
else:
return jsonData
Using the codec library and assuming that {"message": "Hi"} is a map, the code would look something like this.
package main
import (
"fmt"
"github.com/ugorji/go/codec"
)
func main() {
var data []byte
original := map[string]string{"message": "Hi!"}
enc := codec.NewEncoderBytes(&data, new(codec.MsgpackHandle))
if err := enc.Encode(&original); err != nil {
panic(err)
}
fmt.Printf("Encoded: ")
for _, b := range data {
fmt.Printf("%X ", b)
}
fmt.Printf("\n")
decoded := make(map[string]string)
dec := codec.NewDecoderBytes(data, new(codec.MsgpackHandle))
if err := dec.Decode(&decoded); err != nil {
panic(err)
}
fmt.Printf("Decoded: %v\n", decoded)
decoded["message"] = "Hello Sir!"
/* reinitialize the encoder */
enc = codec.NewEncoderBytes(&data, new(codec.MsgpackHandle))
if err := enc.Encode(&decoded); err != nil {
panic(err)
}
fmt.Printf("Encoded: ")
for _, b := range data {
fmt.Printf("%X ", b)
}
fmt.Printf("\n")
}
That said, if you get {"message": "Hi"} as a JSON string, you can use codec to decode the JSON into a map, update the map and then re-encode it as msgpack.
The best way is to first decode it, make your changes via Go structs and then re-encode it.
data := []byte(`{"message": "Hi!"}`)
var p map[string]interface{}
// Decode into Struct
if err := json.Unmarshal(data, &p); err != nil {
// TODO: handle err
}
// Modify contents
p["message"] = "Hello Sir!"
// Encode from struct
newData, err := json.Marshal(p)
if err != nil {
// TODO: Handle err
}
fmt.Println(string(newData))
I have a test function which both creates a server and spawns a goroutine acting as a client. Now, simply sending a message from the client to the server works, but if I want to create an exchange, they seem to deadlock since the test never runs to completion (if no r/w deadlines are set). For example, I want the client to send a message to the server, the server to copy that message and send it back to the client, and then the client to verify that the received message was identical. Here is my test code:
func TestSendAwait(t *testing.T) {
m := "Hello World"
go func() {
conn, err := net.Dial("tcp", testingAddr)
if err != nil {
t.Fatal(err)
}
defer conn.Close()
t.Log("client connected to server") // DEBUG
conn.SetDeadline(time.Now().Add(2 * time.Second))
conn.Write([]byte(m))
conn.SetDeadline(time.Now().Add(2 * time.Second))
buf, err := ioutil.ReadAll(conn)
if err != nil {
t.Fatal(err)
}
t.Log(string(buf))
}()
ln, err := net.Listen("tcp", testingAddr)
if err != nil {
t.Fatal(err)
}
defer ln.Close()
t.Log("server started") // DEBUG
conn, err := ln.Accept()
if err != nil {
t.Fatal(err)
}
defer conn.Close()
t.Log("server received connection") // DEBUG
buf, err := ioutil.ReadAll(conn)
if err != nil {
t.Fatal(err)
}
t.Logf("server read buffer: %v", buf) // DEBUG
_, err = conn.Write(buf)
if err != nil {
t.Fatal(err)
}
t.Log("server wrote to connection") // DEBUG
}
The deadlines are set on the connection because otherwise the deadlock would be indefinite. The output is as follows:
transmission_test.go:42: server started
transmission_test.go:24: client connected to server
transmission_test.go:49: server received connection
transmission_test.go:32: read tcp 127.0.0.1:41164->127.0.0.1:9090: i/o timeout
transmission_test.go:55: server read buffer: [72 101 108 108 111 32 87 111 114 108 100]
transmission_test.go:61: server wrote to connection
Process finished with exit code 1
I don't understand why the client is unable to read and exits, and only then the server decides to send data down the socket? This happens even if I increase the read deadline in the client.
The program blocks on the call to ioutil.ReadAll. This function reads until io.EOF or some other error is returned.
One fix is to shutdown write after writing data to the connection. This will cause read on the peer to return io.EOF and for ioutil.ReadAll to return successfully.
conn.Write(data)
cw, ok := conn.(interface{ CloseWrite() error })
if !ok {
// handle error
}
cw.CloseWrite()
playground example
The program in the question does not guarantee that the listener is opened before the connection is dialed or that client will print print the received message. The playground example corrects these issues.
Another approach is to frame the messages in some way:
Write newline or some other byte sequence not allowed in message after message. Read until this byte sequence is found.
Write message length before message. Read length and then specified number of bytes.