I'm working on a Ruby project that is interacting with a webservice that I'm exchanging some encrypted data with.
I am having a very hard time decrypting something I get back from the webservice in Ruby, although in the .NET side, it's working fine, and a number of other web-based or desktop-based tools can deal with this.
The encryption method was 3DES with ECB and no padding.
Below is a test script I have been working on. I've tried everything I can think of to get these strings unpacked correctly, but to no avail.
require 'openssl'
require 'base64'
def cipher(key, encrypted)
key = key.unpack('a2'*32).map{|x| x.hex}.pack('c'*32)
encrypted = encrypted.unpack('a2'*32).map{|x| x.hex}.pack('c'*32)
OpenSSL::Cipher::ciphers.select{|c| c.include? 'des3' }.map do |cipher_name|
begin
cipher = OpenSSL::Cipher.new(cipher_name)
cipher.padding = 0
cipher.decrypt
cipher.key=key
plain = cipher.update(encrypted) + cipher.final
p "Cipher #{cipher_name} success: #{plain} #{plain.class} #{plain.length} #{plain.encoding.to_s}"
plain
rescue => e
p "Cipher #{cipher_name} failed #{e}"
nil
end
end
end
key = '202FA9B21843D7022B6466DB68327E1F'
encrypted = 'ff6f07e270ebd5c0878c67c999d87ebf'
res1 = cipher key, encrypted
key = '49CE85147B24123718AB3F4539AB1A21'
encrypted = '995604ed8016da8897f1875ebd725529'
res2 = cipher key, encrypted
p res1 == res2 ? "SUCCESS" : "FAIL"
# In both cases, the correct output should be '25588015543912470222703296730936'
A 3DES key is 24-bytes, use a full length key.
3DES uses triple encryption with essentially a 24-byte key. 202FA9B21843D7022B6466DB68327E1F is hex encoded 16-byte key.
Try repeating the first 8-bytes of the key:
202FA9B21843D7022B6466DB68327E1F202FA9B21843D702
Some 3DES implementations will repeat 8-bytes of a 16-byte key but relying on such implementation details is not a good idea.
Note: 3DES actually uses a 168-bit key because the LSb of each byte is not used. Further because there are actually three DES calls the security is only 112-bits. Additionally DES has some weak keys. There are two common modes, ede and ded, in an effort to facilitate moving from DES to 3DES thus adding more confusion.
Finally: Move from 3DES to AES in CBC mode with a random IV. Please don't continue poor security practices.
Related
If I understand GCM mode correctly, it is supposed to provide not only encryption, but also authentication of the ciphertext. However when I use Ruby's OpenSSL implementation to encrypt data with AES-256-GCM mode, it will happily decrypt the data even if I tamper with the auth_tag. Am I missing something here or is the implementation indeed broken?
require 'openssl'
# ALICE encrypts some secret data
data = 'secret'
cipher = OpenSSL::Cipher.new('aes-128-gcm')
cipher.encrypt
key = cipher.random_key
iv = cipher.random_iv
cipher.auth_data = 'auth_data'
ciphertext = cipher.update(data) + cipher.final
auth_tag = cipher.auth_tag
# EVE tampers with the auth tag, e.g. dropping the last 10 bytes
auth_tag = auth_tag[0..-11]
# BOB decrypts the ciphertext
cipher = OpenSSL::Cipher.new('aes-128-gcm')
cipher.decrypt
cipher.key = key
cipher.iv = iv
cipher.auth_tag = auth_tag
cipher.auth_data = 'auth_data'
data = cipher.update(ciphertext) + cipher.final
# BOB is very sad because no error is raised!
I am using OS X built in OpenSSL version:
% openssl version
OpenSSL 0.9.8zg 14 July 2015
GCM supports multiple sizes for the authentication tag. The authentication tag is shortened in these versions by removing bytes from the right. That's exactly what your attacker seems to be doing.
Now the API is dangerously unstable. First of all, (16 - 10) * 8 = 48, which is not a valid size for AES-GCM (according to NIST SP 800-38D. Furthermore, the authentication tag size should be an input or configuration parameter of the cipher. At the very least the API should warn users to check the authentication tag size themselves instead of allowing just any authentication tag input.
So yes, you are missing something and yes, I would say the implementation - or at the very least the documentation - is broken; good catch.
I'm Node developer, but every once in awhile I get to play around with ERB templates. I really love pulling out as much ruby as I can in these templates when I can and this idea caught my eye especially.
I have this configuration value, which should be encrypted, but is coming in plain text. The program would decrypt it like so:
var crypto = require('crypto');
var decipher = crypto.createDecipher('aes256', 'e20jhciwjf90u2r9u9ujj');
var decrypted = crypto.update('4ufujj90u19ru90u109u') + crypto.final();
I was wondering how I might go about creating an encrypted string for the above to decrypt using ruby?
So far I have:
require 'openssl'
cipher = OpenSSL::Cipher::Cipher.new('aes256');
cipher.key= 'e20jhciwjf90u2r9u9ujj'
encrypted = cipher.update('my cat is yellow and very pretty.') + cipher.final
Two problems I have:
I often get a Key length not long enough error on the ruby side.
Ruby outputs a bunch of crazy hex, whereas node seems to always take/want utf8.
Am I encrypting/decrypting safely?
Is there a way to universally translate/work laterally with these two APIs?
AES-256 uses a key of 256bits, and by default ruby uses utf-8 encoding, so each ansi character is 8bit long. So the key string must be 32 bytes.
Explicitly use an aes mode (e.g. aes-256-cbc).
Set the same iv(initial vector) on both sides
I finally succeeded with the above methods.
Here is my code:
Ruby side:
require 'openssl'
require 'base64'
cipher = OpenSSL::Cipher.new('aes-256-cbc')
cipher.encrypt
cipher.iv = 'a'*16;
cipher.key = '01234567890123456789012345678901' # should be 32 characters, 32*8=256 bits
enc = Base64.strict_encode64(cipher.update('01234567890123456789012345678901') + cipher.final)
puts enc
Javascript side:
var encrypted = new Buffer(base64Data, 'base64');
var crypto = require('crypto');
var decipher = crypto.createDecipheriv('aes-256-cbc', '01234567890123456789012345678901', 'aaaaaaaaaaaaaaaa');
var dec = decipher.update(encrypted);
console.log(Buffer.concat([dec, decipher.final()]));
I need to decrypt text encrypted using AES/CBC/PKCS5Padding scheme. The encrypted text I got was generated using Coldfusion.
CFML example below:
<table border="1" cellpadding="5" cellspacing="0">
<tr bgcolor="c0c0c0">
<th>Decrypted string</th>
<th>3DESKey</th>
</tr>
<cfset variables.algorithm ="AES/CBC/PKCS5Padding">
<cfset variables.seed ="C610297CE8570750">
<cfset variables.password = "Vza0O49SHpIe/mR4+4jHXhApmKhEyl5O2nzzDxVNQbo=">
<cfset variables.decryptedString = Decrypt(variables.password, generate3DesKey("#variables.seed#"), "#variables.algorithm#", "Base64")>
<cfoutput>
<tr>
<td>#variables.decryptedString#</td>
<td><cfoutput>#generate3DesKey("variables.seed")#</cfoutput></td>
</tr>
</cfoutput>
</table>
Output is:
Decrypted String: Name322big563
3DESKey: QzYxMDI5N0NFODU3MDc1MA==
I tried with ruby:
require 'openssl'
require 'base64'
string = "Vza0O49SHpIe/mR4+4jHXhApmKhEyl5O2nzzDxVNQbo="
def decrypt(cpass)
des = OpenSSL::Cipher::Cipher.new('AES-256-CBC')
des.decrypt
des.key = 'C610297CE8570750'
return des.update(Base64.decode64(cpass)) + des.final
end
decrypted = decrypt(string)
puts "decrypted string: #{decrypted}"
I get key length too short (OpenSSL::Cipher::CipherError)
The problem is I don't know the key but only the seed used C610297CE8570750, because the key returned by the CFML script is base64 but I need a hex key.
I tried also with OpenSSL::Cipher::AES256.new(:CBC) same error.
require 'openssl'
require 'base64'
# decryption
aes = OpenSSL::Cipher::AES256.new(:CBC)
aes.decrypt
aes.padding = 1 # actually it's on by default
aes.key = "QzYxMDI5N0NFODU3MDc1MA=="
aes.iv = "C610297CE8570750"
aes.update(Base64::decode64("Vza0O49SHpIe/mR4+4jHXhApmKhEyl5O2nzzDxVNQbo="))+aes.final
Any idea?
EDIT:
As hinted by #Leigh, need to use AES-128-CBC, so I did this:
require 'openssl'
require 'base64'
string = "Vza0O49SHpIe/mR4+4jHXhApmKhEyl5O2nzzDxVNQbo="
def decrypt(cpass)
des = OpenSSL::Cipher::Cipher.new('AES-128-CBC')
des.decrypt
des.key = 'C610297CE8570750'
return des.update(Base64.decode64(cpass)) + des.final
end
decrypted = decrypt(string)
puts "decrypted string: #{decrypted}"
actually seems to kinda work (...ish).
decrypted string: ▒▒.ϥD▒▒ ▒▒▒▒▒Name322big563
any idea what's still wrong?
(Expanded from comments)
but I need a hex key
Then convert it from base64 to hex. In CF, you can use the BinaryEncode() and BinaryDecode functions:
binaryEncode(binaryDecode("QzYxMDI5N0NFODU3MDc1MA==", "base64"), "hex")
Looks like there are a few other problems:
The CF code generates a 128 bit key, but the ruby code is using AES 256. It needs to use AES 128.
The CF code is generating a random IV. The Ruby code is using a totally different iv. With CBC mode, both sides must use the same iv to get the expected results. "Decrypting with the incorrect IV causes the first block of plaintext to be corrupt ...", which is why your decrypted value is off. To resolve it, the Ruby code should use the same iv that was used to encrypt.
Update:
When CF generates the IV automatically (as it does here), it prepends that IV to the encrypted value:
When ColdFusion creates an IV automatically, it generates a secure,
random IV and prepends this to the encrypted data. When ColdFusion
decrypts the data, this IV is recovered and used. It is
cryptologically important that the IV varies between encryptions. This
is why the encrypted value changes when you repeatedly encrypt the
same string with an algorithm that uses an IV, like
DES/CBC/PKCS5Padding. Unlike the encryption key, it is not necessary
for the IV to be kept secret.
So the IV value can be extracted by removing the first "block" of the encrypted binary. The block size depends on the algorithm. For AES, it is 16. I do not know the exact Ruby code, but in CF you could extract the IV like so:
blockSize = 16;
rawBinary = binaryDecode(encryptedString, "base64");
// IV is always the first block
ivBytes = arraySlice(rawBinary, 1, blockSize);
// Remaining bytes are the encrypted value
dataBytes = arraySlice(rawBinary, blockSize+1, arrayLen(rawBinary)-blockSize);
Unless I'm very much mistaken, this is a problem I encountered years ago.
PHP Encryption Code Converted to ColdFusion
While trying to setup an interoperable encryption system, I met a weird situation during a light "proof-of-concept".
I wrote the following code in Ruby to:
create an encrypted file from a dummy text file on my file system
decrypt the encrypted file
compare with the original file and check if they are the same
Here is the code:
require 'openssl'
require 'base64'
# Read the dummy file
data = File.read("test.txt")
# Create an encrypter
cipher = OpenSSL::Cipher::AES.new(256, :CBC)
cipher.encrypt
key = "somethingreallyreallycomplicated"
cipher.key = key
# Encrypt and save to a file
encrypted = cipher.update(data) + cipher.final
open "encrypted.txt", "w" do |io| io.write Base64.encode64(encrypted) end
# Create a decrypter
decipher = OpenSSL::Cipher::AES.new(256, :CBC)
decipher.decrypt
decipher.key = key
# Decrypt and save to a file
encrypted_data = Base64.decode64(File.read("encrypted.txt"))
plain = decipher.update(encrypted_data) + decipher.final
open "decrypted.txt", "w" do |io| io.write plain end
# Compare original message and decrypted message
puts data == plain #=> true
Everything works fine, this script outputs "true"
Then I tried to use the openssl command-line to decrypt my file with the following command:
openssl aes-256-cbc -d -a -in encrypted.txt -k somethingreallyreallycomplicated
But I got: bad magic number
How come?
You need to use the -K (upper case) and -iv options on the command line to specify key and IV explicitly as a string of hex digits. If you use -k (lower case), OpenSSL will derive key and IV from the password using a key derivation function. When OpenSSL derives a key, it will also use a "salted" ciphertext format which is incompatible with the plain blockwise CBC you are expecting.
Note that in your Ruby code, you are using the first 256 bits (32 bytes) of an ASCII string directly as a key, which is almost certainly not what you want for a real world application where security is an issue. You should use a (randomly generated) binary key, or derive a key from a password using a key derivation function such as PBKDF2, bcrypt or scrypt.
I need to mimic what MySQL does when encrypting and decrypting strings using built-in functions AES_ENCRYPT() and AES_DECRYPT().
I have read a couple of blog posts and apparently MySQL uses AES 128-bit encryption for those functions. On top of that, since this encryption requires a 16-bit key, MySQL pads the string with x0 chars (\0s) until it's 16-bit in size.
The algorithm in C from MySQL source code is spotted here.
Now I need to replicate what MySQL does in a Rails application, but every single thing I tried, doesn't work.
Here's a way to replicate the behavior I am getting:
1) Create a new Rails app
rails encryption-test
cd encryption-test
2) Create a new scaffolding
script/generate scaffold user name:string password:binary
3) Edit your config/database.yml and add a test MySQL database
development:
adapter: mysql
host: localhost
database: test
user: <<user>>
password: <<password>>
4) Run the migration
rake db:migrate
5) Enter console, create an user and update its password from MySQL query
script/console
Loading development environment (Rails 2.2.2)
>> User.create(:name => "John Doe")
>> key = "82pjd12398JKBSDIGUSisahdoahOUASDHsdapdjqwjeASIduAsdh078asdASD087asdADSsdjhA7809asdajhADSs"
>> ActiveRecord::Base.connection.execute("UPDATE users SET password = AES_ENCRYPT('password', '#{key}') WHERE name='John Doe'")
That's where I got stuck. If I attempt to decrypt it, using MySQL it works:
>> loaded_user = User.find_by_sql("SELECT AES_DECRYPT(password, '#{key}') AS password FROM users WHERE id=1").first
>> loaded_user['password']
=> "password"
However if I attempt to use OpenSSL library, there's no way I can make it work:
cipher = OpenSSL::Cipher::Cipher.new("AES-128-ECB")
cipher.padding = 0
cipher.key = key
cipher.decrypt
user = User.find(1)
cipher.update(user.password) << cipher.final #=> "########gf####\027\227"
I have tried padding the key:
desired_length = 16 * ((key.length / 16) + 1)
padded_key = key + "\0" * (desired_length - key.length)
cipher = OpenSSL::Cipher::Cipher.new("AES-128-ECB")
cipher.key = key
cipher.decrypt
user = User.find(1)
cipher.update(user.password) << cipher.final #=> ""|\e\261\205:\032s\273\242\030\261\272P##"
But it really doesn't work.
Does anyone have a clue on how can I mimic the MySQL AES_ENCRYPT() and AES_DECRYPT() functions behavior in Ruby?
Thanks!
For future reference:
According to the blog post I sent before, here's how MySQL works with
the key you provide AES_ENCRYPT / DECRYPT:
"The algorithm just creates a 16 byte
buffer set to all zero, then loops
through all the characters of the
string you provide and does an
assignment with bitwise OR between the
two values. If we iterate until we
hit the end of the 16 byte buffer, we
just start over from the beginning
doing ^=. For strings shorter than 16
characters, we stop at the end of the
string."
I don't know if you can read C, but here's the mentioned snippet:
http://pastie.org/425161
Specially this part:
bzero((char*) rkey,AES_KEY_LENGTH/8); /* Set initial key */
for (ptr= rkey, sptr= key; sptr < key_end; ptr++,sptr++)
{
if (ptr == rkey_end)
ptr= rkey; /* Just loop over tmp_key until we used all key */
*ptr^= (uint8) *sptr;
}
So I came up with this method (with a help from Rob Biedenharn, from ruby forum):
def mysql_key(key)
final_key = "\0" * 16
key.length.times do |i|
final_key[i%16] ^= key[i]
end
final_key
end
That, given a string returns the key MySQL uses when encrypting and decrypting. So all you need now is:
def aes(m,k,t)
(aes = OpenSSL::Cipher::AES128.new("ECB").send(m)).key = k
aes.update(t) << aes.final
end
def encrypt(key, text)
aes(:encrypt, key, text)
end
def decrypt(key, text)
aes(:decrypt, key, text)
end
To use openssl lib, built into ruby, and then you can make the two "final" methods:
def mysql_encrypt(s, key)
encrypt(mysql_key(key), s)
end
def mysql_decrypt(s, key)
decrypt(mysql_key(key), s)
end
And you're set! Also, complete code can be found in this Gist:
http://gist.github.com/84093
:-)
Generally you don't want to pad the key, you pad/unpad the data to be encrypted/decrypted. That could be another source of problems. I suggest using test data of a complete number of blocks to eliminate this possibility.
Also, I suspect the key for the OpenSSL API requires a "literal" key, not an ASCII representation of the key as you have in your code.
Given the paucity of the OpenSSL ruby docs and if you speak a little Java, you may want to prototype in JRuby with the BouncyCastle provider - this is something that I've done to good effect when working with TwoFish (not present in OpenSSL API).
EDIT: I re-read your comment about padding the key. You have some bits/bytes confusion in your question, and I'm not sure how this applies in any case since your posted key is 89 characters (712 bits) in length. Perhaps you should try with a 128 bit key/password to eliminate this padding phenomenon?
Incidentally, MySQL devs should be spanked for weak crypto, there are better ways to stretch passwords than by simply padding with zero bytes :(
If you don't mind using an openssl implementation attr_encrypted is a gem that will allow drop-in encryption on most classes, ActiveRecord or not. It unfortunately will not be compatible with MySQL's AES_EN/DECRYPT functions though.