I am using an IV in cipher.NewOFB, but my encrypted file never gets prefixed with it. I followed the golang examples at https://golang.org/pkg/crypto/cipher/, but can't seem to figure out why the prefix isn't being considered.
Does anyone see what the problem is?
func generateRandomIV(length int) []byte {
iv := make([]byte, aes.BlockSize)
if _, err := io.ReadFull(rand.Reader, iv); err != nil {
panic(err)
}
return iv
}
func encryptFile(filename, keystring string) error {
readFile, err := os.Open(filename)
iv := generateRandomIV(aes.BlockSize)
outFile, err := os.OpenFile(filename+".enc", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
panic(err)
}
defer readFile.Close()
defer outFile.Close()
key := []byte(keystring)
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
fmt.Println("IV:", iv)
writer := &cipher.StreamWriter{S: cipher.NewOFB(block, iv), W: outFile}
if _, err := io.Copy(writer, readFile); err != nil {
return err
}
return nil
}
Add the IV prefix yourself or pre-share the IV. If you prefix it you will have the remove it and apply it on decryption.
How an IV is shared is not part of the encryption standard, it is a developer choice. Prefixing the IV is common but not required or the only way, it is however a good choice.
Related
I have an encryption in Ruby with aes-256-gcm
require 'openssl'
key = "972ec8dd995743d981417981ac2f30db"
iv = "6a825c25ea74"
auth_data = "73f6828fc5be"
plaintext = "John Doe play foo bar"
cipher = OpenSSL::Cipher.new('aes-256-gcm')
cipher.encrypt
cipher.iv = iv
cipher.key = key
cipher.auth_data = auth_data
cipherText = cipher.update(plaintext) + cipher.final
authTag = cipher.auth_tag
hexString = (cipherText + iv + authTag).unpack('H*').first
the hextString result looks like
fa03a24cad007ceaadc34c22edff943cb58fe514ed36613832356332356561373425f6bc5724b956daae151c8d78a21263
I want to decrypt it in Go
key := "972ec8dd995743d981417981ac2f30db"
iv := "6a825c25ea74"
authData := "73f6828fc5be"
hexString, _ := hex.DecodeString("fa03a24cad007ceaadc34c22edff943cb58fe514ed36613832356332356561373425f6bc5724b956daae151c8d78a21263")
block, err := aes.NewCipher([]byte(key))
if err != nil {
panic(err.Error())
}
aesgcm, err := cipher.NewGCM(block)
if err != nil {
panic(err.Error())
}
plaintext, err := aesgcm.Open(nil, []byte(iv), hexString, []byte(authData))
if err != nil {
panic(err.Error())
}
I got cipher: message authentication failed.
and Also I don't get the point about authData in golang, I can't fine it in here https://golang.org/pkg/crypto/cipher/#NewGCM
The nonce (iv) does not belong in the middle of the ciphertext. The hex-encoded output you want here from the ruby example is only the cipherText + authTag. Since the nonce must be sent along with the ciphertext, it is common to prefix the ciphertext with the nonce if you so choose, but you must trim that off before deciphering the message. (Also note that your key, iv and auth_data values appear to be hex strings, but they are being used as raw bytes which may be adding to some of the confusion).
Re-arranging the bytes of the message to prepend the nonce, gives us this example: https://play.golang.org/p/YV5FugSyM5_G
key := []byte("972ec8dd995743d981417981ac2f30db")
authData := []byte("73f6828fc5be")
msg, err := hex.DecodeString("366138323563323565613734fa03a24cad007ceaadc34c22edff943cb58fe514ed25f6bc5724b956daae151c8d78a21263")
block, err := aes.NewCipher([]byte(key))
if err != nil {
log.Fatal(err)
}
aesgcm, err := cipher.NewGCM(block)
if err != nil {
log.Fatal(err)
}
sz := aesgcm.NonceSize()
nonce, cipherText := msg[:sz], msg[sz:]
pt, err := aesgcm.Open(nil, nonce, cipherText, authData)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%q\n", pt)
"John Doe play foo bar"
This is what worked for me. I originally used the attr_encrypted gem to encrypt the information.
package main
import (
"crypto/aes"
"crypto/cipher"
"encoding/base64"
"fmt"
"log"
)
func main() {
// data: "7621276423"
// encrypted_data: "Hz1HXFTfSyucPvy3iDoY1F4O5YmAx2skRa4="
// encrypted_data_iv: "Ezk8f3+944gs4x5E"
// license_key: "iUPGMBmppYA92kbciS5fIUe7gRcx6G025haOeAmEjU4="
cipherText, err := base64.StdEncoding.DecodeString("Hz1HXFTfSyucPvy3iDoY1F4O5YmAx2skRa4=")
nonce, err := base64.StdEncoding.DecodeString("Ezk8f3+944gs4x5E")
key, err := base64.StdEncoding.DecodeString("iUPGMBmppYA92kbciS5fIUe7gRcx6G025haOeAmEjU4=")
block, err := aes.NewCipher([]byte(key))
if err != nil {
log.Fatal(err)
}
aesgcm, err := cipher.NewGCM(block)
if err != nil {
log.Fatal(err)
}
pt, err := aesgcm.Open(nil, nonce, cipherText, nil)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%q\n", pt)
}
// "123456789"
I need to write multicast listener on Go. I faces the problem of twicing packets when I read it. It seems that I need to set IP_MULTICAST_LOOP to false. It is not easy to do in Go.
I found this post. It seems that it should work. But I still get copies of the same host. How should it be done?
ipAddr, err := net.ResolveUDPAddr("udp", groupAddress)
if err != nil {...}
iface, err := net.InterfaceByName("en0")
if err != nil {...}
conn, err := net.ListenPacket("udp4", groupAddress)
if err != nil {...}
pc := ipv4.NewPacketConn(conn)
if err := pc.JoinGroup(iface, ipAddr); err != nil {...}
if err := pc.SetMulticastLoopback(false); err != nil {...}
if loop, err := pc.MulticastLoopback(); err == nil {...}
buf := make([]byte, 1024)
for {
n, _, addr, err := pc.ReadFrom(buf)
if err != nil {...}
fmt.Printf("recv from %v: [%s] \n", addr, buf[:n])
}
The simplest way is to use the ListenMulticastUDP wrapper in the net package, as actually already explained in the other SO answer you point to, How to set IP_MULTICAST_LOOP on multicast UDPConn in Golang.
If you follow the implementation of ListenMulticastUDP(), you will see that at a certain point it calls setIPv4MulticastLoopback(fd, false).
If you need something more advanced, the documentation of ListenMulticastUDP() suggests to look at https://godoc.org/golang.org/x/net/ipv4 and https://godoc.org/golang.org/x/net/ipv6, which document extensively how to do multicast in Go.
Here is some minimal code (tested on MacOS, but platform-independent) that shows how to use ListenMulticastUDP():
func main() {
// MDNS (https://en.wikipedia.org/wiki/Multicast_DNS)
groupAddress := "224.0.0.251:5353"
ifaceName := "en0"
if err := run(groupAddress, ifaceName); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
func run(groupAddr string, ifaceName string) error {
iface, err := net.InterfaceByName(ifaceName)
if err != nil {
return err
}
gaddr, err := net.ResolveUDPAddr("udp", groupAddr)
if err != nil {
return err
}
conn, err := net.ListenMulticastUDP("udp", iface, gaddr)
if err != nil {
return err
}
buf := make([]byte, 1024)
for {
n, addr, err := conn.ReadFromUDP(buf)
if err != nil {
return err
}
fmt.Printf("recv %4d bytes from %v\n", n, addr)
}
}
I'm trying to create E2E Encryption for my software, but the decryption are very unstable, sometime can successfully decrypt, sometime got cipher: message authentication failed, here's my encrypt & decrypt code
func Encrypt(data []byte, passphrase string) ([]byte, error) {
// create aes.NewCipher from hashed md5 passphrase
block, _ := aes.NewCipher([]byte(createHash(passphrase)))
// NewGCM returns the given 128-bit, block cipher wrapped in
// Galois Counter Mode with the standard nonce length.
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
// initialize slice with length of nonce that must be passed to Seal and Open.
nonce := make([]byte, gcm.NonceSize())
if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
return nil, err
}
ciphertext := gcm.Seal(nonce, nonce, data, nil)
return ciphertext, nil
}
func Decrypt(data []byte, passphrase string) ([]byte, error) {
// create md5 byte slice
key := []byte(createHash(passphrase))
// just `reverse` algorithm with passphrase until return
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
nonceSize := gcm.NonceSize()
nonce, ciphertext := data[:nonceSize], data[nonceSize:]
plaintext, err := gcm.Open(nil, nonce, ciphertext, nil)
if err != nil {
return nil, err
}
return plaintext, nil
}
the encrypted binary value are transferred via http :
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
decrypt, err := Decrypt(body, r.Passphrase)
what i already try is to check, is ioutil.ReadAll read content correctly, or something wrong with decryptor
sorry, the problem was not in encryption/decryption, but in http server for transferring the chipertext, and already fixes now https://github.com/codenoid/GoTral-Server/commit/493c7f654753cae36f074c1c5f382953e227d295
How do I make this not require a file seek? Basically, I am doing a hash, and then re-reading the file. This is not optimal. How can I optimize by either using TeeReader or another method to read in chunks so Hashing and Writing content can happen without duplication of reading.
Also, do I need to specify content length myself?
// PUT method
func (c *Client) PutFileOld(filename string, noLen bool) error {
file, err := os.Open(filename)
if err != nil {
return err
}
defer file.Close()
hasher := md5.New()
if _, err := io.Copy(hasher, file); err != nil {
log.Fatal("Could not compute MD5")
}
// Lazy way to go back to the beginning since the reader has consumed our bytes
// and we have to compute the hash
file.Seek(0, 0)
c.MD5 = hex.EncodeToString(hasher.Sum(nil)[:16])
log.Printf("Uploading to: %s", fmt.Sprintf("%s/%s", c.baseURL, filename))
baseURL, err := url.Parse(fmt.Sprintf("%s/%s", c.baseURL, filename))
if err != nil {
return err
}
log.Printf("MD5: %s - file: %s\n", c.MD5, filename)
req, err := http.NewRequest(http.MethodPut, baseURL.String(), bufio.NewReader(file))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/octet-stream")
req.Header.Set("Content-Md5", c.MD5)
fi, _ := file.Stat()
// Not sure if this is needed, or if Go sets it automatically
req.ContentLength = fi.Size()
res, err := c.httpClient.Do(req)
if err != nil {
return err
}
dump, err := httputil.DumpResponse(res, true)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%q\n", dump)
c.StatusCode = res.StatusCode
defer res.Body.Close()
return nil
}
I'm trying to download a remote file over ssh
The following approach works fine on shell
ssh hostname "tar cz /opt/local/folder" > folder.tar.gz
However the same approach on golang giving some difference in output artifact size. For example the same folders with pure shell produce artifact gz file 179B and same with go script 178B.
I assume that something has been missed from io.Reader or session got closed earlier. Kindly ask you guys to help.
Here is the example of my script:
func executeCmd(cmd, hostname string, config *ssh.ClientConfig, path string) error {
conn, _ := ssh.Dial("tcp", hostname+":22", config)
session, err := conn.NewSession()
if err != nil {
panic("Failed to create session: " + err.Error())
}
r, _ := session.StdoutPipe()
scanner := bufio.NewScanner(r)
go func() {
defer session.Close()
name := fmt.Sprintf("%s/backup_folder_%v.tar.gz", path, time.Now().Unix())
file, err := os.OpenFile(name, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
panic(err)
}
defer file.Close()
for scanner.Scan() {
fmt.Println(scanner.Bytes())
if err := scanner.Err(); err != nil {
fmt.Println(err)
}
if _, err = file.Write(scanner.Bytes()); err != nil {
log.Fatal(err)
}
}
}()
if err := session.Run(cmd); err != nil {
fmt.Println(err.Error())
panic("Failed to run: " + err.Error())
}
return nil
}
Thanks!
bufio.Scanner is for newline delimited text. According to the documentation, the scanner will remove the newline characters, stripping any 10s out of your binary file.
You don't need a goroutine to do the copy, because you can use session.Start to start the process asynchronously.
You probably don't need to use bufio either. You should be using io.Copy to copy the file, which has an internal buffer already on top of any buffering already done in the ssh client itself. If an additional buffer is needed for performance, wrap the session output in a bufio.Reader
Finally, you return an error value, so use it rather than panic'ing on regular error conditions.
conn, err := ssh.Dial("tcp", hostname+":22", config)
if err != nil {
return err
}
session, err := conn.NewSession()
if err != nil {
return err
}
defer session.Close()
r, err := session.StdoutPipe()
if err != nil {
return err
}
name := fmt.Sprintf("%s/backup_folder_%v.tar.gz", path, time.Now().Unix())
file, err := os.OpenFile(name, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
return err
}
defer file.Close()
if err := session.Start(cmd); err != nil {
return err
}
n, err := io.Copy(file, r)
if err != nil {
return err
}
if err := session.Wait(); err != nil {
return err
}
return nil
You can try doing something like this:
r, _ := session.StdoutPipe()
reader := bufio.NewReader(r)
go func() {
defer session.Close()
// open file etc
// 10 is the number of bytes you'd like to copy in one write operation
p := make([]byte, 10)
for {
n, err := reader.Read(p)
if err == io.EOF {
break
}
if err != nil {
log.Fatal("err", err)
}
if _, err = file.Write(p[:n]); err != nil {
log.Fatal(err)
}
}
}()
Make sure your goroutines are synchronized properly so output is completeky written to the file.