In my Go program I am encoding []byte data with gob
buf := new(bytes.Buffer)
enc := gob.NewEncoder(buf)
//data is []byte
buf.Reset()
enc.Encode(data)
but getting 'gob decoder attempting to decode into a non-pointer' when I am trying to decoded
buf := new(bytes.Buffer)
d := gob.NewDecoder(buf)
d.Decode(data)
log.Printf("%s", d)
Gob requires you to pass a pointer to decode.
In your case, you would do:
d.Decode(&data)
reason being, it may have to modify the slice (ie: to make it bigger, to fit the decoded array)
Related
I am making a go program where I need to write a gob to a file. I used the .String() method to convert the gob to a string.
var network bytes.Buffer
encoder := gob.NewEncoder(&network)
_ = encoder.Encode(valueToEncode)
gobString := network.String()
then I will write the gob to a file, and later I will retrieve it and send it to this program:
var filebytes = []byte(file) //i think that these two lines are the issue
network := bytes.NewBuffer(filebytes)
decoder := gob.NewDecoder(network)
var decoded interface{}
_ := decoder.Decode(&decoded)
but when i run this, it gives me this error:
gob: encoded unsigned integer out of range
I think the issue is with the first two lines of the decoder program. So what should I put to properly decode the gob?
EDIT:
What I want is a .UnString() method for the gobString. How can i achieve that?
The encoding/gob generates binary data from Go values. The result is not for textual representation, so you should not treat it as a string, but as a series of bytes, e.g. []byte.
That said, do not use Buffer.String() but rather Buffer.Bytes() if you must obtain the encoded data.
Here's an example encoding and decoding a string value using encoding/gob:
// ENCODE
var network bytes.Buffer
encoder := gob.NewEncoder(&network)
valueToEncode := "Hello, 世界"
if err := encoder.Encode(valueToEncode); err != nil {
panic(err)
}
gobData := network.Bytes() // Save / serialize this byte slice
// DECODE
network2 := bytes.NewBuffer(gobData)
decoder := gob.NewDecoder(network2)
var decoded string
if err := decoder.Decode(&decoded); err != nil {
panic(err)
}
fmt.PrintTln(decoded)
Output (try it on the Go Playground):
Hello, 世界
Also note that if you intend to write the encoded data into a network connection or a file, you don't need bytes.Buffer, you can directly encode to those. If you must use bytes.Buffer, you may also use its Buffer.WriteTo() method to write its contents into an io.Writer (such as a file or network connection).
I need to send struct data with byte slice data type during socket communication.
type A struct {
header []byte
body []byte
}
So I wrote the following source code to convert the structure to bytes.
var a A
a.header = byte slice data...
a.body = byte slice data...
buf := new(bytes.Buffer)
binary.Write(buf, binary.BigEndian, a)
However, I get an error with the binary.Write function showing the following error:
binary.Write: invalid type main.A
I have found that fixed arrays solve the problem. But since the length of the data is constantly changing, I have to use a slice rather than a fixed array.
Is there a way to solve this problem?
If you write a variable length of byte slice, the other end would not know how many bytes it needs to read. You have to communicate the length too.
So one way to send a byte slice is to first write the length (number of bytes) using a fixed-size type, e.g. int32 or int64. Then simply write the byte slice.
For example:
var w io.Writer // This represents your connection
var a A
if err := binary.Write(w, binary.LittleEndian, int32(len(a.header))); err != nil {
// Handle error
}
if _, err := w.Write(a.header); err != nil {
// Handle error
}
You may use the same logic to send a.body too.
On the other end, this is how you could read it:
var r io.Reader // This represents your connection
var a A
var size int32
if err := binary.Read(r, binary.LittleEndian, &size); err != nil {
// Handle error
}
a.header = make([]byte, size)
if _, err := io.ReadFull(r, a.header); err != nil {
// Handle error
}
Try a working example on the Go Playground.
If you have to transfer more complex structs, consider using the encoding/gob which handles sending slices with ease. For an example and some insights, see Efficient Go serialization of struct to disk.
I am new to Golang. I am developing a service which reads bytes from remote address over TCP. The problem is that I can not change encoding of bytes I read. I want to convert the bytes I read to ISO-8859-9 string. Here is part of reading code.
conn, err := net.Dial("tcp", constant.ConnectHost+":"+constant.ConnectPort)
checkError(err)
defer conn.Close()
reader := bufio.NewReader(conn)
textproc := textproto.NewReader(reader)
bytes, err := textproc.R.ReadBytes(constant.EndTextDelimiter)
checkError(err)
msg := string(bytes[:])
Code works fine. But the encoding is different than I want. It is a problem for receiving service. Any suggestion?
charmap.ISO8859_9.NewEncoder().Bytes() function wants UTF-8 format to encode. I was getting error when I try to encode my bytes. Because my incoming bytes are in 8859-9 format and I was trying to convert them directly. First I decode the bytes to UTF-8 format. I did my process, at the end I encoded this UTF-8 bytes to ISO8859-9 unicode using encoder. Here is the new code.
//main package
bytes, err := textproc.R.ReadBytes(constant.EndTextDelimiter)
checkError(err)
msg := encoder.DecodeISO8859_9ToUTF8(bytes)
//..........
// Process that string, create struct Then convert struct to json bytes
// Then encode that bytes
json := encoder.EncodeUTF8ToISO8859_9(bytes)
//encoder package
package encoder
import "golang.org/x/text/encoding/charmap"
func DecodeISO8859_9ToUTF8(bytes []byte) string {
encoded, _ := charmap.ISO8859_9.NewDecoder().Bytes(bytes)
return string(encoded[:])
}
func EncodeUTF8ToISO8859_9(bytes []byte) string {
encoded, _ := charmap.ISO8859_9.NewEncoder().Bytes(bytes)
return string(encoded[:])
}
How to read a unit8 from []byte without creating a bytes.Buffer. The value has been written to the buffer like this,
buf := new(bytes.Buffer)
binary.Write(buf, binary.BigEndian, uint32(1))
binary.Write(buf, binary.BigEndian, uint8(1))
b := buf.Bytes()
While decoding, it can easily be done for uint32, like the following...
len := binary.BigEndian.Uint32(b[:4])
But for the uint8, the only way to retrieve the value that I could come up with, is to create a buffer and then read the first byte,
buf := new(bytes.Buffer)
_, err := buf.Write(b[4:5])
// error handling ...
id = buf.ReadByte()
It seems like there's no method in the encoding/binary pkg for uint8 value retrieval. And I guess there's probably some good reason behind it.
Question: Is there any other way to read uint8 from that []byte without creating a Buffer??
Us an index expression to get an single uint8 from a slice.
len := binary.BigEndian.Uint32(b[:4])
id := b[4] // <-- index expression
Note that byte is an alias for uint8.
Here's an elegant way that I found,
var len uint32
var id uint8
binary.Read(buf, binary.BigEndian, &len)
binary.Read(buf, binary.BigEndian, &id)
// this method doesn't need you to create a buffer for each value read
I'm fairly new to the Go language and having a hard time achieving the following: I'm receiving a base64 string (basically, an encoded image) and need to transform it to the binary form on the server.
func addOrUpdateUserBase64(w http.ResponseWriter, r *http.Request, params martini.Params) {
c := appengine.NewContext(r)
sDec, _ := b64.StdEncoding.DecodeString(r.Body)
...
This is not working, because DecodeString expects a string... how do I transform request.Body into a string? Any tips are very much appreciated!
Do not use base64.StdEncoding.DecodeString, instead set up a decoder directly from the r.Body
dec := base64.NewDecoder(base64.StdEncoding, r.Body)` // dec is an io.Reader
now use dec, e.g. dump to a bytes.Buffer like
buf := &bytes.Buffer{}
n, err := io.copy(buf, dec)
which will decode r.Body into buf or copy directly to a http.Response or a file.
Or use Peter's method below if keeping all in memory is okay.
func (*Encoding) Decode
func (enc *Encoding) Decode(dst, src []byte) (n int, err error)
Decode decodes src using the encoding enc. It writes at most
DecodedLen(len(src)) bytes to dst and returns the number of bytes
written. If src contains invalid base64 data, it will return the
number of bytes successfully written and CorruptInputError. New line
characters (\r and \n) are ignored.
And one more option would be just casting r.Body to a string :
//Edit, fixed the code to work with an io.Reader
import "io/ioutil"
..........
if body, err := ioutil.ReadAll(r.Body); err == nil {
sDec, _ := b64.StdEncoding.DecodeString(string(body))
}