I have following code which decodes base64 and then encodes it to hex.
doc_id := "Can35qPeFkm9Xgmp9+aj3g=="
base64_decode, err := base64.StdEncoding.DecodeString(doc_id)
if err != nil {
log.Fatal("error:", err)
}
base64_decoded := fmt.Sprintf("%q", base64_decode)
fmt.Printf("base_decoded %v\n", base64_decoded)
src := []byte(base64_decoded)
fmt.Println(src)
hex_encode := make([]byte, hex.EncodedLen(len(src)))
hex.Encode(hex_encode, src)
hex_encoded := fmt.Sprintf("%s", hex_encode)
fmt.Printf("hex_encoded %v", hex_encoded)
where doc_id is base64 format.
base64_decoded is its decoded value.
I have to encode it to hex, so i pass it to src.
The problem is when i pass the identifier base64_decoded to src i get wrong when i pass in the value that base64_decoded is holding i get correct answer.
for example:
if i get base64_decoded value as "\x11z\xc0[d~\xfcK\xb1\xf8\x11z\xc0[d~"
if i pass its value which is "\x11z\xc0[d~\xfcK\xb1\xf8\x11z\xc0[d~", i get correct answer 117ac05b647efc4bb1f8117ac05b647e
if i pass the variable holding "\x11z\xc0[d~\xfcK\xb1\xf8\x11z\xc0[d~" i get wrong answer 225c7831317a5c7863305b647e5c7866634b5c7862315c7866385c7831317a5c7863305b647e22dn
Has it got something with this assignment base64_decoded := fmt.Sprintf("%q", base64_decode)
what am i doing wrong
Use the following code:
doc_id := "Can35qPeFkm9Xgmp9+aj3g=="
base64_decode, err := base64.StdEncoding.DecodeString(doc_id)
if err != nil {
log.Fatal("error:", err)
}
fmt.Println(hex.EncodeToString(base64_decode))
Directly encode the bytes in base64_decode as hex.
The call to fmt.Sprintf("%q", base64_decode) returns a string with the bytes escaped per Go's string literal rules. In the general case, the bytes in the returned string are not equal to the bytes in base64_decode. For example, the input byte 0x11 is escaped to the four bytes \x11.
what happens is that the conversion that fmt.Sprintf("%q", base64_decode) makes to base64_decode, in this process to convert some bytes in the memory are lost.
here you below I leave an example: https://play.golang.org/p/pdQBp7NsvQM
doc_id := "Can35qPeFkm9Xgmp9+aj3g=="
var_encode := b64.StdEncoding.EncodeToString([]byte(doc_id))
fmt.Println(var_encode)
var_decode , _ := b64.StdEncoding.DecodeString(var_encode)
fmt.Println(string(var_decode))
Related
I am making a go program where I need to write a gob to a file. I used the .String() method to convert the gob to a string.
var network bytes.Buffer
encoder := gob.NewEncoder(&network)
_ = encoder.Encode(valueToEncode)
gobString := network.String()
then I will write the gob to a file, and later I will retrieve it and send it to this program:
var filebytes = []byte(file) //i think that these two lines are the issue
network := bytes.NewBuffer(filebytes)
decoder := gob.NewDecoder(network)
var decoded interface{}
_ := decoder.Decode(&decoded)
but when i run this, it gives me this error:
gob: encoded unsigned integer out of range
I think the issue is with the first two lines of the decoder program. So what should I put to properly decode the gob?
EDIT:
What I want is a .UnString() method for the gobString. How can i achieve that?
The encoding/gob generates binary data from Go values. The result is not for textual representation, so you should not treat it as a string, but as a series of bytes, e.g. []byte.
That said, do not use Buffer.String() but rather Buffer.Bytes() if you must obtain the encoded data.
Here's an example encoding and decoding a string value using encoding/gob:
// ENCODE
var network bytes.Buffer
encoder := gob.NewEncoder(&network)
valueToEncode := "Hello, 世界"
if err := encoder.Encode(valueToEncode); err != nil {
panic(err)
}
gobData := network.Bytes() // Save / serialize this byte slice
// DECODE
network2 := bytes.NewBuffer(gobData)
decoder := gob.NewDecoder(network2)
var decoded string
if err := decoder.Decode(&decoded); err != nil {
panic(err)
}
fmt.PrintTln(decoded)
Output (try it on the Go Playground):
Hello, 世界
Also note that if you intend to write the encoded data into a network connection or a file, you don't need bytes.Buffer, you can directly encode to those. If you must use bytes.Buffer, you may also use its Buffer.WriteTo() method to write its contents into an io.Writer (such as a file or network connection).
I need to send struct data with byte slice data type during socket communication.
type A struct {
header []byte
body []byte
}
So I wrote the following source code to convert the structure to bytes.
var a A
a.header = byte slice data...
a.body = byte slice data...
buf := new(bytes.Buffer)
binary.Write(buf, binary.BigEndian, a)
However, I get an error with the binary.Write function showing the following error:
binary.Write: invalid type main.A
I have found that fixed arrays solve the problem. But since the length of the data is constantly changing, I have to use a slice rather than a fixed array.
Is there a way to solve this problem?
If you write a variable length of byte slice, the other end would not know how many bytes it needs to read. You have to communicate the length too.
So one way to send a byte slice is to first write the length (number of bytes) using a fixed-size type, e.g. int32 or int64. Then simply write the byte slice.
For example:
var w io.Writer // This represents your connection
var a A
if err := binary.Write(w, binary.LittleEndian, int32(len(a.header))); err != nil {
// Handle error
}
if _, err := w.Write(a.header); err != nil {
// Handle error
}
You may use the same logic to send a.body too.
On the other end, this is how you could read it:
var r io.Reader // This represents your connection
var a A
var size int32
if err := binary.Read(r, binary.LittleEndian, &size); err != nil {
// Handle error
}
a.header = make([]byte, size)
if _, err := io.ReadFull(r, a.header); err != nil {
// Handle error
}
Try a working example on the Go Playground.
If you have to transfer more complex structs, consider using the encoding/gob which handles sending slices with ease. For an example and some insights, see Efficient Go serialization of struct to disk.
I am new to Golang. I am developing a service which reads bytes from remote address over TCP. The problem is that I can not change encoding of bytes I read. I want to convert the bytes I read to ISO-8859-9 string. Here is part of reading code.
conn, err := net.Dial("tcp", constant.ConnectHost+":"+constant.ConnectPort)
checkError(err)
defer conn.Close()
reader := bufio.NewReader(conn)
textproc := textproto.NewReader(reader)
bytes, err := textproc.R.ReadBytes(constant.EndTextDelimiter)
checkError(err)
msg := string(bytes[:])
Code works fine. But the encoding is different than I want. It is a problem for receiving service. Any suggestion?
charmap.ISO8859_9.NewEncoder().Bytes() function wants UTF-8 format to encode. I was getting error when I try to encode my bytes. Because my incoming bytes are in 8859-9 format and I was trying to convert them directly. First I decode the bytes to UTF-8 format. I did my process, at the end I encoded this UTF-8 bytes to ISO8859-9 unicode using encoder. Here is the new code.
//main package
bytes, err := textproc.R.ReadBytes(constant.EndTextDelimiter)
checkError(err)
msg := encoder.DecodeISO8859_9ToUTF8(bytes)
//..........
// Process that string, create struct Then convert struct to json bytes
// Then encode that bytes
json := encoder.EncodeUTF8ToISO8859_9(bytes)
//encoder package
package encoder
import "golang.org/x/text/encoding/charmap"
func DecodeISO8859_9ToUTF8(bytes []byte) string {
encoded, _ := charmap.ISO8859_9.NewDecoder().Bytes(bytes)
return string(encoded[:])
}
func EncodeUTF8ToISO8859_9(bytes []byte) string {
encoded, _ := charmap.ISO8859_9.NewEncoder().Bytes(bytes)
return string(encoded[:])
}
I am trying to use the go-skeltrack library with some depth images I have (Not using freenect). For that I need to modify the provided example by replacing the kinect images by my own. For that I have to read an image and convert it later to an []uint16 variable. The code which I tried is:
file, err := os.Open("./images/4.png")
if err != nil {
fmt.Println("4.png file not found!")
os.Exit(1)
}
defer file.Close()
fileInfo, _ := file.Stat()
var size int64 = fileInfo.Size()
bytes := make([]byte, size)
// read file into bytes
buffer := bufio.NewReader(file)
_, err = buffer.Read(bytes)
integerImage := binary.BigEndian.Uint16(bytes)
onDepthFrame(integerImage)
Where onDepthFrame is a function which has the form
func onDepthFrame(depth []uint16).
But I am getting the following error while compiling:
./skeltrackOfflineImage.go:155: cannot use integerImage (type uint16) as type []uint16 in argument to onDepthFrame
Which of course refers to the fact that I generated a single integer instead of an array. I am quite confused about the way that Go data types conversion works. Please help!
Thanks in advance for your help.
Luis
binary.BigEndian.Uint16 converts two bytes (in a slice) to a 16-bit value using big endian byte order. If you want to convert bytes to a slice of uint16, you should use binary.Read:
// This reads 10 uint16s from file.
slice := make([]uint16, 10)
err := binary.Read(file, binary.BigEndian, slice)
It sounds like you're looking to get raw pixels. If that's the case, I don't recommend reading the file as binary directly. It means you would need to parse the file format yourself since image files contain more information than just the raw pixel values. There are already tools in the image package to deal with that.
This code should get you on the right track. It reads RGBA values, so it ends up with a 1D array of uint8's of length width * height * 4, since there are four values per pixel.
https://play.golang.org/p/WUgHQ3pRla
import (
"bufio"
"fmt"
"image"
"os"
// for decoding png files
_ "image/png"
)
// RGBA attempts to load an image from file and return the raw RGBA pixel values.
func RGBA(path string) ([]uint8, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
img, _, err := image.Decode(bufio.NewReader(file))
if err != nil {
return nil, err
}
switch trueim := img.(type) {
case *image.RGBA:
return trueim.Pix, nil
case *image.NRGBA:
return trueim.Pix, nil
}
return nil, fmt.Errorf("unhandled image format")
}
I'm not entirely sure where the uint16 values you need should come from, but presumably it's data per pixel, so the code should be very similar to this except the switch on trueim should likely check for something other than image.RGBA. Take a look at the other image types in https://golang.org/pkg/image
I'm fairly new to the Go language and having a hard time achieving the following: I'm receiving a base64 string (basically, an encoded image) and need to transform it to the binary form on the server.
func addOrUpdateUserBase64(w http.ResponseWriter, r *http.Request, params martini.Params) {
c := appengine.NewContext(r)
sDec, _ := b64.StdEncoding.DecodeString(r.Body)
...
This is not working, because DecodeString expects a string... how do I transform request.Body into a string? Any tips are very much appreciated!
Do not use base64.StdEncoding.DecodeString, instead set up a decoder directly from the r.Body
dec := base64.NewDecoder(base64.StdEncoding, r.Body)` // dec is an io.Reader
now use dec, e.g. dump to a bytes.Buffer like
buf := &bytes.Buffer{}
n, err := io.copy(buf, dec)
which will decode r.Body into buf or copy directly to a http.Response or a file.
Or use Peter's method below if keeping all in memory is okay.
func (*Encoding) Decode
func (enc *Encoding) Decode(dst, src []byte) (n int, err error)
Decode decodes src using the encoding enc. It writes at most
DecodedLen(len(src)) bytes to dst and returns the number of bytes
written. If src contains invalid base64 data, it will return the
number of bytes successfully written and CorruptInputError. New line
characters (\r and \n) are ignored.
And one more option would be just casting r.Body to a string :
//Edit, fixed the code to work with an io.Reader
import "io/ioutil"
..........
if body, err := ioutil.ReadAll(r.Body); err == nil {
sDec, _ := b64.StdEncoding.DecodeString(string(body))
}