Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 1 year ago.
Improve this question
I want array of json from sql rows. when i try to do marshal on each struct after scanning each row, its returning weird values like [123 34 105 100 34 ..]
type Org struct {
Id int `json:"id"`
Name string `json:"name"`
}
res, err := db.Query("select id,name from organization")
if err != nil {
// fmt.Print("err in query")
panic(err)
}
// var orgArray []Org
defer res.Close()
for res.Next() {
var org Org
fmt.Println(&org.Id, &org.Name, "PRINT ADDRESS BEFORE SCAN")
// 0xc0001c0648 0xc0001c0650 PRINT ADDRESS BEFORE SCAN
err = res.Scan(&org.Id, &org.Name)
fmt.Println(org.Id, org.Name, org, "PRINT VALUES AFTER SCAN")
// 1535 TestOrg {1535 TestOrg} PRINT VALUES AFTER SCAN
b, err := json.Marshal(org)
if err != nil {
panic(err)
}
fmt.Println(b)
//[123 34 105 100 34 58 49 53 51 55 44 34 110 97 109 101 34 58 34 98 114 97 110 100 32 69 104 71 74 89 34 125]
}
whats the problem here?
json.Marshal response is bytes array, convert to string before printing
package main
import (
"encoding/json"
"fmt"
)
type Abc struct {
A string `json:"a"`
B string `json:"b"`
}
func main() {
d := Abc{A: "aaa", B: "bbbb"}
a, _ := json.Marshal(d)
fmt.Println(string(a))
}
json.Marshal returns a byte array - []byte.
The Println prints b out as such. The array of integers (byte values) you see is how byte arrays are printed out in Go.
Use string(b) to print a string. fmt.Println(string(b)), etc.
Related
I run this code in my mac. I test it and it is ok. But i compile it and start it at aws ecs. I get this error
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x55fff8]
goroutine 6 [running]:
go.sia.tech/siad/types.NewCurrency(0xc0000d5640)
/Users/jaslin.wang/go/pkg/mod/go.sia.tech/siad#v1.5.8/types/currency.go:57 +0x18
There is my code. It get error when call this line 80
74 amount, _ := new(big.Int).SetString(output.Value, 10)
75
76 addr := types.UnlockHash{}
77 addr.LoadString(output.Address)
78
79 sco := types.SiacoinOutput{
80 Value: types.NewCurrency(amount),
81 UnlockHash: addr,
82 }
The NewCurrency function is
56 func NewCurrency(b *big.Int) (c Currency) {
57 if b.Sign() < 0 {
58 build.Critical(ErrNegativeCurrency)
59 } else {
60 c.i = *b
61 }
62 return
63 }
There is my compile command
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build
Thanks very much!
2022/08/30 03:30:05 output.Value is 340900000000000000000000000.000000000000000000000000000000000000000000000000000000
2022-08-30T11:30:05.726+08:00 2022/08/30 03:30:05 ok is false
2022-08-30T11:30:05.726+08:00 2022/08/30 03:30:05 amount is <nil>
sry to bother you guys. I don't know why output.Value is 340900000000000000000000000.000000000000000000000000000000000000000000000000000000. It should be 340900000000000000000000000. But i think change it to 340900000000000000000000000 will fix this issue. Thank all of you!
You need to check whether big.SetString() actually parsed the value. Try running this code to see it happen... Presumably SetString() is not parsing your value because you're using a big.Int() and passing it a string with floating point decimals (even if they are zeroes).
package main
import (
"fmt"
"math/big"
)
type Currency struct {
i big.Int
}
func main() {
for _, v := range []string{"1234567878768376294872638", "boom!"} {
amount, ok := new(big.Int).SetString(v, 10)
if !ok {
fmt.Println("Expecting this will go boom!")
}
x := NewCurrency(amount)
fmt.Println(x)
}
}
func NewCurrency(b *big.Int) (c Currency) {
if b.Sign() < 0 {
fmt.Printf("Error negative currency")
panic("panicking!")
} else {
c.i = *b
}
return
}
Given the following code (copied from here):
1 package main
2
3 import (
4 "fmt"
5
6 "github.com/pkg/errors"
7
8 "go.uber.org/zap"
9 )
10
11 func main() {
12 logger, _ := zap.NewProduction()
13 defer logger.Sync()
14
15 sugar := logger.Sugar()
16 result, err := caller1()
17 if err != nil {
18 sugar.Error(err)
19 return
20 }
21 fmt.Println("Result: ", result)
22 }
23
24 func caller1() (int, error) {
25 err := caller2()
26 if err != nil {
27 return 0, err
28 }
29 return 1, nil
30 }
31
32 func caller2() error {
33 doSomething()
34 return caller3()
35 }
36
37 func caller3() error {
38 return errors.New("failed")
39 }
40
41 func doSomething() {}
Since we are using github.com/pkg/errors package, I know that to log the stack trace, just use sugar.Errorf("%+v", err):
$ go run stacktrace.go 2>&1 | jq
{
"level": "error",
"ts": 1606501191.539652,
"caller": "zap/stacktrace.go:18",
"msg": "failed\nmain.caller3\n\t/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:38\nmain.caller2\n\t/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:34\nmain.caller1\n\t/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:25\nmain.main\n\t/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:16\nruntime.main\n\t/usr/local/Cellar/go/1.15.2/libexec/src/runtime/proc.go:204\nruntime.goexit\n\t/usr/local/Cellar/go/1.15.2/libexec/src/runtime/asm_amd64.s:1374",
"stacktrace": "main.main\n\t/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:18\nruntime.main\n\t/usr/local/Cellar/go/1.15.2/libexec/src/runtime/proc.go:204"
}
The thing is the stack trace is put under the msg key and the one under the stacktrace is the stack trace of the logging line:
main.main
/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:18
runtime.main
/usr/local/Cellar/go/1.15.2/libexec/src/runtime/proc.go:204
not the stack trace of the error:
main.caller3
/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:38
main.caller2
/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:34
main.caller1
/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:25
main.main
/Users/quanta/go/src/github.com/quantonganh/zap/stacktrace.go:16
runtime.main
/usr/local/Cellar/go/1.15.2/libexec/src/runtime/proc.go:204
runtime.goexit
/usr/local/Cellar/go/1.15.2/libexec/src/runtime/asm_amd64.s:1374
Question: how can I log the "real" stack trace under the stacktrace key?
I have looked into the source code and I think we cannot do that.
Log messages are formatted and put under the msg key.
The stack trace is taken from runtime.CallersFrames at the time of logging.
The reason I asked this question is before using %+v to get the complete call stack, I just looked at the "stacktrace" value and wondering:
Why it is missing?
Why it just point to the logging line instead of pointing to the actual error?
I'm not able to understand how message framing using fixed size prefix length header works.
It's said that a fixed size byte array is going to contain the length for message to be sent. But how would you define a fixed size byte array, specifically in Golang.
Say this is my message:
Hello
Its length is 5.
So if I want to send this through a tcp stream, to make sure I receive all the message on the other end I'd have to tell it how many bytes it should read.
A simple header would be length:message:
5:Hello // [53 58 104 101 108 108 111]
But if message length grows 10x each time, there are going to be more bytes. So header size is dynamic that way.
36:Hello, this is just a dumb question. // [51 54 58 72 101 108 108 111 44 32 116 104 105 115 32 105 115 32 106 117 115 116 32 97 32 100 117 109 98 32 113 117 101 115 116 105 111 110 46]
So here 36 takes 2 bytes.
One approach I come to think of is to consider a maximum message length for the protocol. Say 10KB = 10240 bytes. Then add leading 0's to message length. This way I'm sure that I'm going to have a fixed 5 bytes header.
Would this work for all cases?
If yes, what if I have a message more than 10KBs, should I split it into 2 messages?
If not, what are other solutions?
I want to implement the solutions in Golang.
UPDATE 1:
I read about Endians, although I wasn't able to understand what they do that causes fixed length bytes. But I found an example in python and tried to write it in go this way:
Client:
const maxLengthBytes = 8
conn, err := net.Dial("tcp", "127.0.0.1:9999")
if err != nil {
fmt.Println(err)
return
}
message := "Hello, this is just a dumb question"
bs := make([]byte, maxLengthBytes)
binary.LittleEndian.PutUint64(bs, uint64(len(text)))
bytes := append(bs, []byte(text)...)
conn.Write(bytes)
Server:
listener, err := net.ListenTCP("tcp", &net.TCPAddr{Port: 9999})
if err != nil {
fmt.Println(err)
return
}
for {
tcp, err := listener.AcceptTCP()
if err != nil {
fmt.Println(err)
continue
}
go Reader(tcp)
}
func Reader(conn *net.TCPConn) {
foundLength := false
messageLength := 0
for {
if !foundLength {
var b = make([]byte, maxLengthBytes)
read, err := conn.Read(b)
if err != nil {
fmt.Println(err)
continue
}
if read != 8 {
fmt.Println("invalid header")
continue
}
foundLength = true
messageLength = int(binary.LittleEndian.Uint64(b))
} else {
var message = make([]byte, messageLength)
read, err := conn.Read(message)
if err != nil {
fmt.Println(err)
continue
}
if read != messageLength {
fmt.Println("invalid data")
continue
}
fmt.Println("Received:", string(message))
foundLength = false
messageLength = 0
}
}
}
Please refer to my answer in this post
TCP client for Android: text is not received in full
Basically, you have to define how the data stored/formatted.
For example:
We store prefix length as int32 (4 bytes) with little endian. It's different from yours.
With your solution, the length is a string, it's hard to fix the length.
For your solution, you have to use fixed length string. For example: 10 characters, and add leading zero.
For your questions.
It doesn't work for all cases with just prefix length. It has its limitation, for example if we use int32 as the prefix length, the length of message must be less than Integer32.max, right?
Yes, we have to split or even combine (please refer my explanation in above link).
We have many ways to deal with length limitation if it's our concern (actually, almost application protocols has it maximum request size).
You could use one more bit to indicate, whether or not the message exceeds max length to resolve it, right?
So I'm trying to get the string representation of a JSON message in Golang. I just want to receive the messagepack encoded JSON, modify some values and send it back.
I haven't found an easy way to do it. Most of the times, I can't know in advance what is the structure of the JSON (apart from having JSON structure), so want I want to do is to receive the binary message. Decode it as a JSON, print it as a string to the standard output, modify the content, convert it to MessagePack again and send it back.
I've been looking at these two packages, but I don't know how to properly use them for a simple task like that:
https://godoc.org/github.com/vmihailenco/msgpack
https://godoc.org/github.com/ugorji/go/codec
So I will receive something like this:
DF 00 00 00 01 A7 6D 65 73 73 61 67 65 A3 48 69 21
I want to print this:
{"message": "Hi!"}
Modify the "Hi!":
{"message": "Hello Sir!"}
Send it as messagepack:
DF 00 00 00 01 A7 6D 65 73 73 61 67 65 AA 48 65 6C 6C 6F 20 53 69 72 21
Current Python code I'm trying to port to Golang:
def decode_msgpack(jsonData):
packedStuff = 0
for key in jsonData.keys():
if type(jsonData[key]) is bytes:
packedStuff += 1
try:
jsonData[key] = umsgpack.unpackb(jsonData[key])
except umsgpack.InvalidStringException:
try:
jsonData[key] = umsgpack.unpackb(jsonData[key], allow_invalid_utf8=True)
except umsgpack.InsufficientDataException:
print("[!] InsufficientDataException")
jsonData[key] = base64.b64encode(jsonData[key]).decode('utf-8')
else:
jsonData[key] = base64.b64encode(jsonData[key]).decode('utf-8')
if packedStuff > 0:
return decode_msgpack(jsonData)
else:
return jsonData
Using the codec library and assuming that {"message": "Hi"} is a map, the code would look something like this.
package main
import (
"fmt"
"github.com/ugorji/go/codec"
)
func main() {
var data []byte
original := map[string]string{"message": "Hi!"}
enc := codec.NewEncoderBytes(&data, new(codec.MsgpackHandle))
if err := enc.Encode(&original); err != nil {
panic(err)
}
fmt.Printf("Encoded: ")
for _, b := range data {
fmt.Printf("%X ", b)
}
fmt.Printf("\n")
decoded := make(map[string]string)
dec := codec.NewDecoderBytes(data, new(codec.MsgpackHandle))
if err := dec.Decode(&decoded); err != nil {
panic(err)
}
fmt.Printf("Decoded: %v\n", decoded)
decoded["message"] = "Hello Sir!"
/* reinitialize the encoder */
enc = codec.NewEncoderBytes(&data, new(codec.MsgpackHandle))
if err := enc.Encode(&decoded); err != nil {
panic(err)
}
fmt.Printf("Encoded: ")
for _, b := range data {
fmt.Printf("%X ", b)
}
fmt.Printf("\n")
}
That said, if you get {"message": "Hi"} as a JSON string, you can use codec to decode the JSON into a map, update the map and then re-encode it as msgpack.
The best way is to first decode it, make your changes via Go structs and then re-encode it.
data := []byte(`{"message": "Hi!"}`)
var p map[string]interface{}
// Decode into Struct
if err := json.Unmarshal(data, &p); err != nil {
// TODO: handle err
}
// Modify contents
p["message"] = "Hello Sir!"
// Encode from struct
newData, err := json.Marshal(p)
if err != nil {
// TODO: Handle err
}
fmt.Println(string(newData))
I want to unmarshal a geojson string into a suitable struct type.
I have three different geojson strings that I want to unmarshal into the same struct:
var jsonBlobPointString = []byte(`{"Type":"Point", "Coordinates":[1.1,2.0]}`)
var jsonBlobLineString = []byte(`{"Type":"LineString", "Coordinates":[[1.1,2.0],[3.0,6.3]]}`)
var jsonBlobPolygonString = []byte(`{"Type":"Polygon", "Coordinates":[[[1.1,2.0],[3.0,6.3],[5.1,7.0],[1.1,2.0]]]}`)
I came up with a struct type that I´m not totally happy with:
type GeojsonType struct {
Type string
Coordinates interface{}
}
See this link for complete example:
http://play.golang.org/p/Bt-51BX__A
I would rather not use interface{} for Coordinates.
I would instead use somehting that give me some validation for example Coordinates [] float64 for Point
and Coordinates[][] float64 for LineString.
Is it possible to create a struct type so that Point, LineString and Polygon all can be represented in Coordinates without using interface?
What you want is to create 3 different types of object from the same json dictionary.
As far as I know that isn't possible, however you can use the RawMessage type to delay the json decoding and use a bit of pre-processing like this
package main
import (
"encoding/json"
"fmt"
)
type Point struct {
Coordinates []float64
}
type Line struct {
Points [][]float64
}
type Polygon struct {
Lines [][][]float64
}
type GeojsonType struct {
Type string
Coordinates json.RawMessage
Point Point
Line Line
Polygon Polygon
}
var jsonBlob = []byte(`[
{"Type":"Point", "Coordinates":[1.1,2.0]},
{"Type":"LineString", "Coordinates":[[1.1,2.0],[3.0,6.3]]},
{"Type":"Polygon", "Coordinates":[[[1.1,2.0],[3.0,6.3],[5.1,7.0],[1.1,2.0]]]}
]`)
func main() {
var geojsonPoints []GeojsonType
err := json.Unmarshal(jsonBlob, &geojsonPoints)
if err != nil {
fmt.Println("error:", err)
}
// Postprocess the coordinates
for i := range geojsonPoints {
t := &geojsonPoints[i]
switch t.Type {
case "Point":
err = json.Unmarshal(t.Coordinates, &t.Point.Coordinates)
case "LineString":
err = json.Unmarshal(t.Coordinates, &t.Line.Points)
case "Polygon":
err = json.Unmarshal(t.Coordinates, &t.Polygon.Lines)
default:
panic("Unknown type")
}
if err != nil {
fmt.Printf("Failed to convert %s: %s", t.Type, err)
}
fmt.Printf("%+v\n", t)
}
}
Which prints
&{Type:Point Coordinates:[91 49 46 49 44 50 46 48 93] Point:{Coordinates:[1.1 2]} Line:{Points:[]} Polygon:{Lines:[]}}
&{Type:LineString Coordinates:[91 91 49 46 49 44 50 46 48 93 44 91 51 46 48 44 54 46 51 93 93] Point:{Coordinates:[]} Line:{Points:[[1.1 2] [3 6.3]]} Polygon:{Lines:[]}}
&{Type:Polygon Coordinates:[91 91 91 49 46 49 44 50 46 48 93 44 91 51 46 48 44 54 46 51 93 44 91 53 46 49 44 55 46 48 93 44 91 49 46 49 44 50 46 48 93 93 93] Point:{Coordinates:[]} Line:{Points:[]} Polygon:{Lines:[[[1.1 2] [3 6.3] [5.1 7] [1.1 2]]]}}
Based on Nick Craig-Wood answer I built the following Marshal/UnMarshal functions
package geojson
//https://stackoverflow.com/questions/15719532/suitable-struct-type-for-unmarshal-of-geojson
import (
"encoding/json"
)
type Point struct {
Coordinates []float64
}
type Line struct {
Points [][]float64
}
type Polygon struct {
Lines [][][]float64
}
type Geojson struct {
Type string `json:"type"`
Coordinates json.RawMessage `json:"coordinates"`
Point Point `json:"-"`
Line Line `json:"-"`
Polygon Polygon `json:"-"`
}
func (g *Geojson) UnmarshalJSON(b []byte) error {
type Alias Geojson
aux := (*Alias)(g)
err := json.Unmarshal(b, &aux)
if err != nil {
return err
}
switch g.Type {
case "Point":
err = json.Unmarshal(g.Coordinates, &g.Point.Coordinates)
case "LineString":
err = json.Unmarshal(g.Coordinates, &g.Line.Points)
case "Polygon":
err = json.Unmarshal(g.Coordinates, &g.Polygon.Lines)
}
g.Coordinates = []byte(nil)
return err
}
func (g Geojson) MarshalJSON() ([]byte, error) {
var raw json.RawMessage
var err error
switch g.Type {
case "Point":
raw, err = json.Marshal(&g.Point.Coordinates)
case "LineString":
raw, err = json.Marshal(&g.Line.Points)
case "Polygon":
raw, err = json.Marshal(&g.Polygon.Lines)
}
if err != nil {
return nil, err
}
g.Coordinates = raw
type Alias Geojson
aux := (*Alias)(&g)
return json.Marshal(aux)
}