I'm fairly new to the Go language and having a hard time achieving the following: I'm receiving a base64 string (basically, an encoded image) and need to transform it to the binary form on the server.
func addOrUpdateUserBase64(w http.ResponseWriter, r *http.Request, params martini.Params) {
c := appengine.NewContext(r)
sDec, _ := b64.StdEncoding.DecodeString(r.Body)
...
This is not working, because DecodeString expects a string... how do I transform request.Body into a string? Any tips are very much appreciated!
Do not use base64.StdEncoding.DecodeString, instead set up a decoder directly from the r.Body
dec := base64.NewDecoder(base64.StdEncoding, r.Body)` // dec is an io.Reader
now use dec, e.g. dump to a bytes.Buffer like
buf := &bytes.Buffer{}
n, err := io.copy(buf, dec)
which will decode r.Body into buf or copy directly to a http.Response or a file.
Or use Peter's method below if keeping all in memory is okay.
func (*Encoding) Decode
func (enc *Encoding) Decode(dst, src []byte) (n int, err error)
Decode decodes src using the encoding enc. It writes at most
DecodedLen(len(src)) bytes to dst and returns the number of bytes
written. If src contains invalid base64 data, it will return the
number of bytes successfully written and CorruptInputError. New line
characters (\r and \n) are ignored.
And one more option would be just casting r.Body to a string :
//Edit, fixed the code to work with an io.Reader
import "io/ioutil"
..........
if body, err := ioutil.ReadAll(r.Body); err == nil {
sDec, _ := b64.StdEncoding.DecodeString(string(body))
}
Related
I am following this documentation and was trying to implement a simple AES encryption and decryption with using GoLang. For plain text it is working fine however, for UUID it is not working. Excepting a resolution of this and why this is happening. Here is my sample code
package main
import (
"crypto/aes"
"encoding/hex"
"fmt"
)
func main() {
key := "thisis32bitlongpassphraseimusing"
pt := "a30a1777-e9f4-ed45-4755-add00172ebae"
c := EncryptAES([]byte(key), pt)
fmt.Println(pt)
fmt.Println(c)
DecryptAES([]byte(key), c)
}
func EncryptAES(key []byte, plaintext string) string {
c, err := aes.NewCipher(key)
CheckError(err)
out := make([]byte, len(plaintext))
c.Encrypt(out, []byte(plaintext))
return hex.EncodeToString(out)
}
func DecryptAES(key []byte, ct string) {
ciphertext, _ := hex.DecodeString(ct)
c, err := aes.NewCipher(key)
CheckError(err)
pt := make([]byte, len(ciphertext))
c.Decrypt(pt, ciphertext)
s := string(pt[:])
fmt.Println("DECRYPTED:", s)
}
func CheckError(err error) {
if err != nil {
panic(err)
}
}
And here is the output
a30a1777-e9f4-ed45-4755-add00172ebae
e0f32a5bcf576754da4206cc967157ae0000000000000000000000000000000000000000
DECRYPTED: a30a1777-e9f4-ed
As you can see in the remaining last part of the UUID is disappearing. I have attached as a snap which says it didn't decrypt the last part properly. Does anyone know reasoning behind this? I have seen a close question like that but not and exactly one.
If you look at the page for the AES block cipher you'll find out that aes.NewCipher returns a Block as mentioned by Jake in the comments.
Now if you go to that page you'll see that this page points out various modes that you can use to create a real, secure cipher out of a block cipher. A block cipher only handles blocks of data, which are always 128 bits / 16 bytes in the case of AES. So this is precisely why there are all those zero's after the ciphertext, it encrypted 16 bytes and that was that. Note that ciphertext should always look as randomized bytes.
Unfortunately it doesn't directly list the authenticated (AEAD) modes, so please take a look here as well. That said, you can see that CTR mode is in there including examples, and that's the main idea missing from the question + my answer that you've linked to. Nothing in your code shows a mode of operation, and certainly not counter mode.
The problem with your code is that AES encryption requires the input to be a multiple of the block size (16 bytes for AES-128), but the UUID you're trying to encrypt ("a30a1777-e9f4-ed45-4755-add00172ebae")is 36 bytes long and it is causing the cipher to error.
One way to fix this issue is by padding the plaintext so that its length is a multiple of the block size before encryption. The Go standard library has a package called crypto/padding that provides functions for adding padding to plaintext.
You can modify your EncryptAES function like this:
func EncryptAES(key []byte, plaintext string) string {
c, err := aes.NewCipher(key)
CheckError(err)
plaintextBytes := []byte(plaintext)
// Add padding to plaintext
blockSize := c.BlockSize()
padding := blockSize - (len(plaintextBytes) % blockSize)
padtext := bytes.Repeat([]byte{byte(padding)}, padding)
plaintextBytes = append(plaintextBytes, padtext...)
out := make([]byte, len(plaintextBytes))
c.Encrypt(out, plaintextBytes)
return hex.EncodeToString(out)
}
Then in the DecryptAES function, you can remove the padding before decrypting the ciphertext like this:
func DecryptAES(key []byte, ct string) {
ciphertext, _ := hex.DecodeString(ct)
c, err := aes.NewCipher(key)
CheckError(err)
pt := make([]byte, len(ciphertext))
c.Decrypt(pt, ciphertext)
//Remove padding
padLen := int(pt[len(pt)-1])
s := string(pt[:len(pt)-padLen])
fmt.Println("DECRYPTED:", s)
}
As for padding schemes, you might want to try padding scheme like pkcs#5 or pkcs#7.
I am making a go program where I need to write a gob to a file. I used the .String() method to convert the gob to a string.
var network bytes.Buffer
encoder := gob.NewEncoder(&network)
_ = encoder.Encode(valueToEncode)
gobString := network.String()
then I will write the gob to a file, and later I will retrieve it and send it to this program:
var filebytes = []byte(file) //i think that these two lines are the issue
network := bytes.NewBuffer(filebytes)
decoder := gob.NewDecoder(network)
var decoded interface{}
_ := decoder.Decode(&decoded)
but when i run this, it gives me this error:
gob: encoded unsigned integer out of range
I think the issue is with the first two lines of the decoder program. So what should I put to properly decode the gob?
EDIT:
What I want is a .UnString() method for the gobString. How can i achieve that?
The encoding/gob generates binary data from Go values. The result is not for textual representation, so you should not treat it as a string, but as a series of bytes, e.g. []byte.
That said, do not use Buffer.String() but rather Buffer.Bytes() if you must obtain the encoded data.
Here's an example encoding and decoding a string value using encoding/gob:
// ENCODE
var network bytes.Buffer
encoder := gob.NewEncoder(&network)
valueToEncode := "Hello, 世界"
if err := encoder.Encode(valueToEncode); err != nil {
panic(err)
}
gobData := network.Bytes() // Save / serialize this byte slice
// DECODE
network2 := bytes.NewBuffer(gobData)
decoder := gob.NewDecoder(network2)
var decoded string
if err := decoder.Decode(&decoded); err != nil {
panic(err)
}
fmt.PrintTln(decoded)
Output (try it on the Go Playground):
Hello, 世界
Also note that if you intend to write the encoded data into a network connection or a file, you don't need bytes.Buffer, you can directly encode to those. If you must use bytes.Buffer, you may also use its Buffer.WriteTo() method to write its contents into an io.Writer (such as a file or network connection).
I am new to Golang. I am developing a service which reads bytes from remote address over TCP. The problem is that I can not change encoding of bytes I read. I want to convert the bytes I read to ISO-8859-9 string. Here is part of reading code.
conn, err := net.Dial("tcp", constant.ConnectHost+":"+constant.ConnectPort)
checkError(err)
defer conn.Close()
reader := bufio.NewReader(conn)
textproc := textproto.NewReader(reader)
bytes, err := textproc.R.ReadBytes(constant.EndTextDelimiter)
checkError(err)
msg := string(bytes[:])
Code works fine. But the encoding is different than I want. It is a problem for receiving service. Any suggestion?
charmap.ISO8859_9.NewEncoder().Bytes() function wants UTF-8 format to encode. I was getting error when I try to encode my bytes. Because my incoming bytes are in 8859-9 format and I was trying to convert them directly. First I decode the bytes to UTF-8 format. I did my process, at the end I encoded this UTF-8 bytes to ISO8859-9 unicode using encoder. Here is the new code.
//main package
bytes, err := textproc.R.ReadBytes(constant.EndTextDelimiter)
checkError(err)
msg := encoder.DecodeISO8859_9ToUTF8(bytes)
//..........
// Process that string, create struct Then convert struct to json bytes
// Then encode that bytes
json := encoder.EncodeUTF8ToISO8859_9(bytes)
//encoder package
package encoder
import "golang.org/x/text/encoding/charmap"
func DecodeISO8859_9ToUTF8(bytes []byte) string {
encoded, _ := charmap.ISO8859_9.NewDecoder().Bytes(bytes)
return string(encoded[:])
}
func EncodeUTF8ToISO8859_9(bytes []byte) string {
encoded, _ := charmap.ISO8859_9.NewEncoder().Bytes(bytes)
return string(encoded[:])
}
In my Go program I am encoding []byte data with gob
buf := new(bytes.Buffer)
enc := gob.NewEncoder(buf)
//data is []byte
buf.Reset()
enc.Encode(data)
but getting 'gob decoder attempting to decode into a non-pointer' when I am trying to decoded
buf := new(bytes.Buffer)
d := gob.NewDecoder(buf)
d.Decode(data)
log.Printf("%s", d)
Gob requires you to pass a pointer to decode.
In your case, you would do:
d.Decode(&data)
reason being, it may have to modify the slice (ie: to make it bigger, to fit the decoded array)
I am trying to use the go-skeltrack library with some depth images I have (Not using freenect). For that I need to modify the provided example by replacing the kinect images by my own. For that I have to read an image and convert it later to an []uint16 variable. The code which I tried is:
file, err := os.Open("./images/4.png")
if err != nil {
fmt.Println("4.png file not found!")
os.Exit(1)
}
defer file.Close()
fileInfo, _ := file.Stat()
var size int64 = fileInfo.Size()
bytes := make([]byte, size)
// read file into bytes
buffer := bufio.NewReader(file)
_, err = buffer.Read(bytes)
integerImage := binary.BigEndian.Uint16(bytes)
onDepthFrame(integerImage)
Where onDepthFrame is a function which has the form
func onDepthFrame(depth []uint16).
But I am getting the following error while compiling:
./skeltrackOfflineImage.go:155: cannot use integerImage (type uint16) as type []uint16 in argument to onDepthFrame
Which of course refers to the fact that I generated a single integer instead of an array. I am quite confused about the way that Go data types conversion works. Please help!
Thanks in advance for your help.
Luis
binary.BigEndian.Uint16 converts two bytes (in a slice) to a 16-bit value using big endian byte order. If you want to convert bytes to a slice of uint16, you should use binary.Read:
// This reads 10 uint16s from file.
slice := make([]uint16, 10)
err := binary.Read(file, binary.BigEndian, slice)
It sounds like you're looking to get raw pixels. If that's the case, I don't recommend reading the file as binary directly. It means you would need to parse the file format yourself since image files contain more information than just the raw pixel values. There are already tools in the image package to deal with that.
This code should get you on the right track. It reads RGBA values, so it ends up with a 1D array of uint8's of length width * height * 4, since there are four values per pixel.
https://play.golang.org/p/WUgHQ3pRla
import (
"bufio"
"fmt"
"image"
"os"
// for decoding png files
_ "image/png"
)
// RGBA attempts to load an image from file and return the raw RGBA pixel values.
func RGBA(path string) ([]uint8, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
img, _, err := image.Decode(bufio.NewReader(file))
if err != nil {
return nil, err
}
switch trueim := img.(type) {
case *image.RGBA:
return trueim.Pix, nil
case *image.NRGBA:
return trueim.Pix, nil
}
return nil, fmt.Errorf("unhandled image format")
}
I'm not entirely sure where the uint16 values you need should come from, but presumably it's data per pixel, so the code should be very similar to this except the switch on trueim should likely check for something other than image.RGBA. Take a look at the other image types in https://golang.org/pkg/image