I'm working with this example to try to encrypt data using AES-256. However, when I use the password WeakPasswordForTesting as input, I get an error: crypto/aes: invalid key size. I get the impression that it has to do with my plain password length. How can I use this example code to encrypt the data using a short plain-text password?
func Encrypt(password []byte, plainSource []byte) ([]byte, error) {
key, _ := hex.DecodeString(string(password))
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
// The IV needs to be unique, but not secure. Therefore it's common to
// include it at the beginning of the ciphertext.
ciphertext := make([]byte, aes.BlockSize+len(plainSource))
iv := ciphertext[:aes.BlockSize]
if _, err := io.ReadFull(rand.Reader, iv); err != nil {
panic(err)
}
stream := cipher.NewCTR(block, iv)
stream.XORKeyStream(ciphertext[aes.BlockSize:], plainSource)
return ciphertext, nil
}
The Advanced Encryption Standard (AES) is a block cipher which expects a key of a fixed size. The key length is in the name of the cipher: AES-256 means a key length of 256 bits.
As your password is not a key which conforms to this spec, you need to implement a process to make it so. A common mechanism is a key derivation function, which takes an input passphrase of entropy significantly lower than the desired key length, and pads it with various data to produce a key of the required length. The PBKDF family (password-based key derivation function) is an example of this, as is scrypt.
You should not implement a naïve mechanism to pad your input data in this way, as the security of a good cryptosystem is entirely dependent on the quality of the key, and it is highly unlikely any homegrown solution will be resistant to a variety of attacks.
This question on crypto.SE may be interesting regarding a more thorough assessment of implementation recommendations using PBKDF2.
Related
Lets say I've this base64 encrypted text: (Please understand this is all test data)
0Ns8cv7CHb0Pn/L1RPQEJjAaPJbBuhJxf9RCm3k/qG6DoNgd06jSgUwiaM/ym1Xc9DOBMyZvMNZQKwv/cgr9WGP4ogmqNguaM/zIOIh/GpMGCbTh+5UmP3x+La5eQKCmhim+nxVIBVzcaepUGUyjyPncu2RPQB8maA2AjgmqFb2tk3rcRozj8rTjcQcykxk2i7omhmt1FHBj4YBJoT4oc6B9bCUj8nhIZPCOj8T1z7A3MvZJwSmMa5rFnnozMCQ3no7lxORJQnN9d7gkLge1BAGwrxbXZapkuBx/juqTK3sJ46oiPDCdB0IzZ6Q1PZTrJPcWghOlQwfvHfKgu8kHtW2y2QskP+1uzVQBainXxcCHX7ZBq7Vhdl7n3WQjjok3P/A/6ELJa6qI9eL/NTfEavr7m0PVyrm6Zci4piK2Gjdz/LJZWEKbX6/wUq3pnDClQXAv1DV6g8P46H0p2ZCb0/WsbyZmgxuWYMx/KV8236IUQwY4ylyxnhCHpgHZAeM/CC9qgnMw6zjEtud+uWCx3iWZpQdOmxvaEVMQOQFte/PkUkXgH/9Z5nqJWoftKRcQSyC2+gl5ORgjA1GjbFcB5cr36DFj4su+whgKfyDHtCVVqFK+lIkcu2uLJQX7MwImuvKEAY48raCt8eE3qj2dfjK+dNhOfB8TbU4qAsRTVh7FtD0Hfto7sdVRLWAZRnqo7tHqD0R1LWkXg/VhxFGMnJkUxdJonlEFYLadUTI2PIr4acnpHcr6P91dTU+onXdFaYumeJrayrxBAujmXmAzZki+KrYyviORMlxmpK87ZJ2jP0psRx0hD+SlnHnX3o5nJVMCEPy1Gh9lbU3F4O2PtZ5edc7GEq8TFMoAXtRqL3qg2m/vfyrgzjbU9127pHvk4HNpF0ow9PlC3oi827npONncormAml5Ii+4sz5KF8aN1JEiqt5jvrRnbAOOcOCdCysl1jyDyapchdxcHKk/wmzFVrykkaehfoYYDIAMqzPaBCmGFjZ53eXwvjP2XrKaJDvS2DbIkHnT+G0/lv58fYzAGwDlyxqA/oetN+Y5WolmS6gJVaws4qSYKaqVmIdz4pP7xOQNP22qnnjAoB49A0aKpQfGkL3W8zppB3Qa3rtvK0piPemX3HbNLAdKFFBEh1+AEbYNyUE/EjdkkVEwIDayw6L3WurLTytPX+05kKSkfG9AmJzqYmd1y50UQ5GxQu7V65nRL73LBoVWxkUO/T+rnyILmQ8dh91HuMgXdLlH41gfiRzk4axZr7LlSnsXVODoBYUHLOZ273WtFnixmKYCyzdI7gPnwN/BPBpVScpymKO0RZcQjFEp3TBOpF14W0+2RqVl/d+OSmXzeA3j7yvZVeUU59KNSMZzebBI4U9kTu0koHDBJzNHItRkrqsjw3NobpK6JvWjWXatPU9ytJ0gX2smtWGuuzjr/asV0X4omKmovOCllMYh+xRICkSIYB8C6rZ/a8ToVusmA62E6W+61t27e5wN4TCrtq9w3e12kNhZ73T/l2GHKa1gEqV6hBFm/VBEgxdiCtJeerqkbuotmJVMzFpJ+s+NM+xfw2w8uyKUjJFX75nwwI4idhbTl
I've this IV: db64fa140a888b41
And this secret key: 62448f7b7128e8b55224dd667b7c1a7e
If I use this site https://www.devglan.com/online-tools/aes-encryption-decryption to decrypt it using CBC, 256 key size and base64 decrypt, I get the correct string which is:
"{\"totaldistance\":7.116600000000001,\"totaltraveltime\":9,\"fare\":{\"vehicle_list\":[{\"name\":\"Electric Limousine\",\"category_image\":\"image-1667907996681.png\",\"seat_count\":\"4\",\"unique_category_id\":6,\"available_for\":\"2\",\"schedule_before\":\"20\",\"schedule_upto\":\"10800\",\"nearest_driver\":{\"fare\":52,\"min_fare\":35},\"schedule_only\":true},{\"name\":\"Limousine\",\"category_image\":\"image-1667908122302.png\",\"seat_count\":\"4\",\"unique_category_id\":7,\"available_for\":\"2\",\"schedule_before\":\"20\",\"schedule_upto\":\"10800\",\"nearest_driver\":{\"fare\":48,\"min_fare\":32},\"schedule_only\":true},{\"name\":\"XL Limousine\",\"category_image\":\"image-1667907858244.png\",\"seat_count\":\"6\",\"unique_category_id\":13,\"available_for\":\"2\",\"schedule_before\":\"20\",\"schedule_upto\":\"10800\",\"nearest_driver\":{\"fare\":59,\"min_fare\":40},\"schedule_only\":true},{\"name\":\"Ameera Limousine\",\"category_image\":\"image-1667907336189.png\",\"seat_count\":\"4\",\"unique_category_id\":16,\"available_for\":\"2\",\"schedule_before\":\"20\",\"schedule_upto\":\"60\",\"nearest_driver\":{\"fare\":48,\"min_fare\":32},\"schedule_only\":true}]}}"
But whenever I use my CBC decrypt function with the same input, I get this decrypted text:
"��(|���\u000bҖ�\b�Dvltraveltime\\\":9,\\\"fare\\\":{\\\"vehicle_list\\\":[{\\\"name\\\":\\\"Electric Limousine\\\",\\\"category_image\\\":\\\"image-1667907996681.png\\\",\\\"seat_count\\\":\\\"4\\\",\\\"unique_category_id\\\":6,\\\"available_for\\\":\\\"2\\\",\\\"schedule_before\\\":\\\"20\\\",\\\"schedule_upto\\\":\\\"10800\\\",\\\"nearest_driver\\\":{\\\"fare\\\":52,\\\"min_fare\\\":35},\\\"schedule_only\\\":true},{\\\"name\\\":\\\"Limousine\\\",\\\"category_image\\\":\\\"image-1667908122302.png\\\",\\\"seat_count\\\":\\\"4\\\",\\\"unique_category_id\\\":7,\\\"available_for\\\":\\\"2\\\",\\\"schedule_before\\\":\\\"20\\\",\\\"schedule_upto\\\":\\\"10800\\\",\\\"nearest_driver\\\":{\\\"fare\\\":48,\\\"min_fare\\\":32},\\\"schedule_only\\\":true},{\\\"name\\\":\\\"XL Limousine\\\",\\\"category_image\\\":\\\"image-1667907858244.png\\\",\\\"seat_count\\\":\\\"6\\\",\\\"unique_category_id\\\":13,\\\"available_for\\\":\\\"2\\\",\\\"schedule_before\\\":\\\"20\\\",\\\"schedule_upto\\\":\\\"10800\\\",\\\"nearest_driver\\\":{\\\"fare\\\":59,\\\"min_fare\\\":40},\\\"schedule_only\\\":true},{\\\"name\\\":\\\"Ameera Limousine\\\",\\\"category_image\\\":\\\"image-1667907336189.png\\\",\\\"seat_count\\\":\\\"4\\\",\\\"unique_category_id\\\":16,\\\"available_for\\\":\\\"2\\\",\\\"schedule_before\\\":\\\"20\\\",\\\"schedule_upto\\\":\\\"60\\\",\\\"nearest_driver\\\":{\\\"fare\\\":47,\\\"min_fare\\\":31},\\\"schedule_only\\\":true}]}}\"\u000b\u000b\u000b\u000b\u000b\u000b\u000b\u000b\u000b\u000b\u000b"
I'm using this decrypt function:
func DecryptCBC(key, ciphertext []byte, iv string) (plaintext []byte, err error) {
var block cipher.Block
ciphertext, err = b64.StdEncoding.DecodeString(string(ciphertext))
if block, err = aes.NewCipher(key); err != nil {
return
}
if len(ciphertext) < aes.BlockSize {
fmt.Printf("ciphertext too short")
return
}
ciphertext = ciphertext[aes.BlockSize:]
cbc := cipher.NewCBCDecrypter(block,[]byte(iv))
cbc.CryptBlocks(ciphertext, ciphertext)
plaintext = ciphertext
return
}
What am I doing wrong?
I have tried not passing the vector and do this:
iv := ciphertext[0:aes.BlockSize]
But it's returning the same result.
The encrypted text is returned from a third party API, which also provides the secret key and token from a different API.
The fact that the ciphertext is correctly decrypted by entering it, the IV, and the key into the appropriate boxes on the linked website tells me that the IV is not transmitted with the ciphertext, and therefore that decryption should start at the first byte of ciphertext rather than the 16th. So remove the line
ciphertext = ciphertext[aes.BlockSize:]
and I'd expect you'll get the correct answer. Because the hex-encoded IV and key are used as is rather than hex-decoded, there is at most only half as much entropy in each. I'm not aware of any weakness of having a 256-bit AES key with only 128 bits of entropy but I'd rather not see an IV with only 64 bits of entropy. In any event, this is something that's decided by the encrypting party and not necessarily something you can control.
I would like to encrypt a string in Go using AES-256, without any GCM processing, to compare against MQL4. I encounter issues when I try to encrypt special characters or numbers. Should I be pre-processing my plaintext somehow? I am new to Go so any help would be appreciated; my code is below this explanation.
If I encrypt the plaintext "This is a secret" and then decrypt the ciphertext (encoded to hex), I get the same result (i.e. "This is a secret"). pt is the variable name of the plaintext in the code below.
If I try to encrypt "This is a secret; 1234", the ciphertext has a group of zeroes at the end, and when I decrypt I only get "This is a secret". Similar ciphertext in MQL4 does not have zeroes at the end and decrypts correctly.
If I try to encrypt only "1234", I get build errors, stemming from "crypto/aes.(*aesCipherAsm).Encrypt(0xc0000c43c0, 0xc0000ac058, 0x4, 0x4, 0xc0000ac070, 0x4, 0x8)
C:/Program Files/Go/src/crypto/aes/cipher_asm.go:60 +0x125"
Here is my code:
package main
import (
"crypto/aes"
"encoding/hex"
"fmt"
)
func main() {
// cipher key
key := "thisis32bitlongpassphraseimusing"
// plaintext
pt := "This is a secret"
// pt := "This is a secret; 1234" // zeroes in ciphertext
// pt := "1234" // doesn't build
c := EncryptAES([]byte(key), pt)
// plaintext
fmt.Println(pt)
// ciphertext
fmt.Println(c)
// decrypt
DecryptAES([]byte(key), c)
}
func EncryptAES(key []byte, plaintext string) string {
c, err := aes.NewCipher(key)
CheckError(err)
out := make([]byte, len(plaintext))
c.Encrypt(out, []byte(plaintext))
return hex.EncodeToString(out)
}
func DecryptAES(key []byte, ct string) {
ciphertext, _ := hex.DecodeString(ct)
c, err := aes.NewCipher(key)
CheckError(err)
pt := make([]byte, len(ciphertext))
c.Decrypt(pt, ciphertext)
s := string(pt[:])
fmt.Println("DECRYPTED:", s)
}
func CheckError(err error) {
if err != nil {
panic(err)
}
}
You're creating a raw AES encryptor here. AES can only encrypt precisely 16 bytes of plaintext, producing exactly 16 bytes of cipher text. Your first example "This is a secret" is exactly 16 bytes long, so it works as expected. Your second example is too long. Only the first 16 bytes are being encrypted. The third example is too short and you're likely running into uninitialized memory.
The specific characters in your text are irrelevant. Encryption is performed on raw bytes, not letters.
In order to encrypt larger (or smaller) blocks of text, you need to use a block cipher mode on top of AES. Common modes are GCM, CBC, and CTR, but there are many others. In most cases, when someone says "AES" without any qualifier, they mean AES-CBC. (GCM is becoming much more popular, and it's a great mode, but it's not so popular that it's assumed quite yet.)
I don't know anything about MQL4, but I assume you're trying to reimplement CryptEncode? I don't see any documentation on how they do the encryption. You need to know what mode they use, how they derive their key, how they generate (and possibly encode) their IV, whether they include an HMAC or other auth, and more. You need to know exactly how they implement whatever they mean by "CRYPT_AES256." There is no one, standard answer to this.
MQL4 only supports a very specific implementation of AES encryption and unless you use the correct settings in your other code you will not achieve compatibility between the two platforms.
Specifically you need to ensure the following are implemented:
Padding Mode: Zeros
Cipher Mode: ECB (so no IV)
KeySize: 256
BlockSize: 128
You also need to remember in MQL4 that encryption/decryption is a two stage process (to AES256 then to BASE64).
You can try the online AES encryption/decryption tool to verify your results available here: The online toolbox
I have some large files I would like to AES encrypt before sending over the wire or saving to disk. While it seems possible to encrypt streams, there seems to be warnings against doing this and instead people recommend splitting the files into chunks and using GCM or crypto/nacl/secretbox.
Processing streams of data is more difficult due to the authenticity requirement. We can’t encrypt-then-MAC: by it’s nature, we usually don’t know the size of a stream. We can’t send the MAC after the stream is complete, as that usually is indicated by the stream being closed. We can’t decrypt a stream on the fly, because we have to see the entire ciphertext in order to check the MAC. Attempting to secure a stream adds enormous complexity to the problem, with no good answers. The solution is to break the stream into discrete chunks, and treat them as messages.
https://leanpub.com/gocrypto/read
Files are segmented into 4KiB blocks. Each block gets a fresh random 128 bit IV each time it is modified. A 128-bit authentication tag (GHASH) protects each block from modifications.
https://nuetzlich.net/gocryptfs/forward_mode_crypto/
If a large amount of data is decrypted it is not always possible to buffer all decrypted data until the authentication tag is verified. Splitting the data into small chunks fixes the problem of deferred authentication checks but introduces a new one. The chunks can be reordered... ...because every chunk is encrypted separately. Therefore the order of the chunks must be encoded somehow into the chunks itself to be able to detect rearranging any number of chunks.
https://github.com/minio/sio
Can anyone with actual cryptography experience point me in the right direction?
Update
I realized after asking this question that there is a difference between simply not being able to fit the whole byte stream into memory (encrypting a 10GB file) and the byte stream also being an unknown length that could continue long past the need for the stream's start to be decoded (an 24-hour live video stream).
I am mostly interested in large blobs where the end of the stream can be reached before the beginning needs to be decoded. In other words, encryption that does not require the whole plaintext/ciphertext to be loaded into memory at the same time.
As you've already discovered from your research, there isn't much of an elegant solution for authenticated encryption of large files.
There are traditionally two ways to approach this problem:
Split the file into chunks, encrypt each chunk individually and let each chunk have its own authentication tag. AES-GCM would be the best mode to use for this. This method causes file size bloating proportionate to the size of the file. You'll also need a unique nonce for each chunk. You also need a way to indicate where chunks begin/end.
Encrypt using AES-CTR with a buffer, call Hash.Write on an HMAC for each buffer of encrypted data. The benefit of this is that encrypting can be done in one pass. The downside is that decryption requires one pass to validate the HMAC and then another pass to actually decrypt. The upside here is that the file size remains the same, plus roughly ~48 or so bytes for the IV and HMAC result.
Neither is ideal, but for very large files (~2GB or more), the second option is probably preferred.
I have included an example of encryption in Go using the second method below. In this scenario, the last 48 bytes are the IV (16 bytes) and the result of the HMAC (32 bytes). Note the HMACing of the IV also.
const BUFFER_SIZE int = 4096
const IV_SIZE int = 16
func encrypt(filePathIn, filePathOut string, keyAes, keyHmac []byte) error {
inFile, err := os.Open(filePathIn)
if err != nil { return err }
defer inFile.Close()
outFile, err := os.Create(filePathOut)
if err != nil { return err }
defer outFile.Close()
iv := make([]byte, IV_SIZE)
_, err = rand.Read(iv)
if err != nil { return err }
aes, err := aes.NewCipher(keyAes)
if err != nil { return err }
ctr := cipher.NewCTR(aes, iv)
hmac := hmac.New(sha256.New, keyHmac)
buf := make([]byte, BUFFER_SIZE)
for {
n, err := inFile.Read(buf)
if err != nil && err != io.EOF { return err }
outBuf := make([]byte, n)
ctr.XORKeyStream(outBuf, buf[:n])
hmac.Write(outBuf)
outFile.Write(outBuf)
if err == io.EOF { break }
}
outFile.Write(iv)
hmac.Write(iv)
outFile.Write(hmac.Sum(nil))
return nil
}
Using HMAC after encryption is a valid method. However, HMAC can be pretty slow, especially if SHA-2 is used. You could actually do the same with GMAC, the underlying MAC of GCM. It may be tricky to find an implementation but GMAC is over the ciphertext, so you can simply perform it separately if you really want. There are other methods as well such as Poly1305 with AES as used for TLS 1.2 and 1.3.
For GCM (or CCM or EAX or any other authenticated cipher) you need to authenticate the order of the chunks. You could do this by creating a separate file encryption key and then using the nonce input (the 12 byte IV) to indicate the number of the chunk. This will solve the storage of the IV and make sure that the chunks are in order. You can generate the file encryption key using a KDF (if you have a unique way to indicate the file) or by wrapping a random key with a master key.
I am verifying the identity of the sender of a piece of data. I am provided the RSA public key in a PEM format and I know the data is passed through the SHA256 hashing function. The equivalent verification on the node.js platform:
Ticket.prototype.verify = function (ticket) {
if (!ticket) return null;
var pubkey = fs.readFileSync('/etc/SCAMP/auth/ticket_verify_public_key.pem');
var parts = ticket.split(',');
if (parts[0] != '1') return null;
var sig = new Buffer(parts.pop().replace(/-/g,'+').replace(/_/g,'/'), 'base64');
var valid = crypto.createVerify('sha256').update( new Buffer(parts.join(',')) ).verify( pubkey, sig )
Which can verify:
1,3063,21,1438783424,660,1+20+31+32+34+35+36+37+38+39+40+41+42+43+44+46+47+48+50+53+56+59+60+61+62+67+68+69+70+71+75+76+80+81+82+86+87+88+102+104+105+107+109+110+122+124,PcFNyWjoz_iiVMgEe8I3IBfzSlUcqUGtsuN7536PTiBW7KDovIqCaSi_8nZWcj-j1dfbQRA8mftwYUWMhhZ4DD78-BH8MovNVucbmTmf2Wzbx9bsI-dmUADY5Q2ol4qDXG4YQJeyZ6f6F9s_1uxHTH456QcsfNxFWh18ygo5_DVmQQSXCHN7EXM5M-u2DSol9MSROeBolYnHZyE093LgQ2veWQREbrwg5Fcp2VZ6VqIC7yu6f_xYHEvU0-ZsSSRMAMUmhLNhmFM4KDjl8blVgC134z7XfCTDDjCDiynSL6b-D-
by splitting on the last ,. The left side of the split is the ticket data I care about, the right side is the signature which I need to verify before I can use the ticket data.
I have tried to port the logic to go:
func TestSigVerification(t *testing.T) {
block, _ := pem.Decode(signingPubKey)
if block == nil {
t.Errorf("expected to block to be non-nil CERTIFICATE", block)
}
key, err := x509.ParsePKIXPublicKey(block.Bytes)
if err != nil {
t.Errorf("could not parse PKIXPublicKey: `%s`", key)
}
rsaPubKey, ok := key.(*rsa.PublicKey)
if !ok {
t.Errorf("couldn't cast to rsa.PublicKey!")
}
ticket,_ := ParseTicketBytes(fullTicketBytes)
h := sha256.New()
h.Write(ticketBytes)
digest := h.Sum(nil)
err = rsa.VerifyPKCS1v15(rsaPubKey, crypto.SHA256, digest, ticket.Signature)
if err != nil {
t.Errorf("could not verify ticket: `%s` (digest: `%v`)", err, digest )
}
}
But I'm pretty sure VerifyPKCS1v15 is not equivalent to node's crypto.createVerify and this test case fails. What should I be using? How can I use the public key to decrypt the signature and get the sha256? once I have the decrypted sha256 value I could just do a basic comparison with the sha256 I have generated.
Here's a runnable playground example: http://play.golang.org/p/COx2OG-AiA
Though I couldn't get it to work, I suspect the issue is that you'll need to convert the sig from base64 into bytes via the base64 encoding. See this example here:
http://play.golang.org/p/bzpD7Pa9mr (especially lines 23 to 28, where they have to encode the sig from bytes to base64 string to print it, then feed the byte version into the sig check, indicating that you have to use the byte version and not base64 string)
Which I stumbled across on this post:
Signing and decoding with RSA-SHA in GO
I've found that golang generally expects bytes everywhere in byte encoding. I tried to decode your sig string from base64 to bytes however, even after replacing the '-' with '+' and the '_' with '/' it still won't work, for reasons unknown to me:
http://play.golang.org/p/71IiV2z_t8
At the very least this seems to indicate that maybe your sig is bad? If it isn't valid base64? I think if you can find a way to solve this the rest should work!
I tried running your code and it doesn't look like your ticket is well formed. It's not long enough. Where did you get the value from?
Double check that ticket -- that may just be enough to get you going.
I am developing a client-side app in Go that relies on AES CFB. The server-side is written in C. My problem is that Go's AES CFB implementation appears to differ from many others (including OpenSSL). I wrote this to test my theory:-
package main
import (
"fmt"
"encoding/hex"
"crypto/cipher"
"crypto/aes"
)
func encrypt_aes_cfb(plain, key, iv []byte) (encrypted []byte) {
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
encrypted = make([]byte, len(plain))
stream := cipher.NewCFBEncrypter(block, iv)
stream.XORKeyStream(encrypted, plain)
return
}
func decrypt_aes_cfb(encrypted, key, iv []byte) (plain []byte) {
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
plain = make([]byte, len(encrypted))
stream := cipher.NewCFBDecrypter(block, iv)
stream.XORKeyStream(plain, encrypted)
return
}
func main() {
plain := []byte("Hello world.....")
key := []byte("01234567890123456789012345678901")
iv := []byte("0123456789012345")
enc := encrypt_aes_cfb(plain, key, iv)
dec := decrypt_aes_cfb(enc, key, iv)
fmt.Println("Key: ", hex.EncodeToString(key))
fmt.Println("IV: ", hex.EncodeToString(iv))
fmt.Println("Enc: ", hex.EncodeToString(enc))
fmt.Println("In: ", hex.EncodeToString(plain))
fmt.Println("Out: ", hex.EncodeToString(dec))
}
When this is run, it appears to work perfectly, however, if the encrypted bytes are pasted into another AES implementation and decrypted using the same key and IV, the plaintext is corrupted (except for the first Byte). http://aes.online-domain-tools.com/ provides a simple means to test this. Any suggestions why this might be happening and how I can resolve it?
Thanks
Steve
(Firstly, an obligatory warning: CFB mode is a sign of homegrown crypto. Unless you're implementing OpenPGP you should be using an AE mode like AES-GCM or NaCl's secretbox. If you're forced to use CFB mode, I hope that you're authenticating ciphertexts with an HMAC at least.)
With that aside, CFB mode in Go is there for OpenPGP support. (OpenPGP uses both a tweaked CFB mode called OCFB, and standard CFB mode in different places.) The Go OpenPGP code appears to interoperate with other implementations at least.
Nick is correct that test vectors are missing in the Go crypto package. The testing was coming from the OpenPGP code, but packages should stand alone and so I'll add tests to crypto/cipher with the test vectors from section F.3.13 of [1].
My best guess for the source of any differences is that CFB is parameterised by a chunk size. This is generally a power of two number of bits up to the block size of the underlying cipher. If the chunk size isn't specified then it's generally taken to be the cipher block size, which is what the Go code does. See [1], section 6.3. A more friendly explanation is given by [2].
Small chunk sizes were used in the dark ages (the late 90s) when people worried about things like cipher resync in the face of ciphertext loss. If another implementation is using CFB1 or CFB8 then it'll be very different from Go's CFB mode and many others. (Go's code does not support smaller chunk sizes.)
[1] http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38a.pdf
[2] http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Cipher_feedback_.28CFB.29
I investigated this with the following inputs because I was unsure of the bit/byte order for both inputs and outputs :
Key: 00000000000000000000000000000000
IV: 00000000000000000000000000000000
Enc: 66
In: 00
Out: 00
http://play.golang.org/p/wl2y1EE6lK
Which matches the tool you provided, and then this :
Key: 00000000000000000000000000000000
IV: 00000000000000000000000000000000
Enc: 66e94b
In: 000000
Out: 000000
http://play.golang.org/p/DNC42m2oU5
Which doesn't match the tool :
6616f9
http://aes.online-domain-tools.com/link/63687gDNzymApefh/
The first byte matches, which indicates there may be a feedback issue.
So I checked the Go package's code and I think there is a bug here :
func (x *cfb) XORKeyStream(dst, src []byte) {
for len(src) > 0 {
if x.outUsed == len(x.out) {
x.b.Encrypt(x.out, x.next)
x.outUsed = 0
}
if x.decrypt {
// We can precompute a larger segment of the
// keystream on decryption. This will allow
// larger batches for xor, and we should be
// able to match CTR/OFB performance.
copy(x.next[x.outUsed:], src)
}
n := xorBytes(dst, src, x.out[x.outUsed:])
if !x.decrypt {
copy(x.next[x.outUsed:], dst) // BUG? `dst` should be `src`
}
dst = dst[n:]
src = src[n:]
x.outUsed += n
}
}
EDIT
After a second look at CFB mode it seems that Go's code is fine, so yeah it may be the other implementations are wrong.