I'm working on implementing a protocol and per the specification I need to encrypt / decrypt data using AES-128-CCM and AES-128-GCM. My question is: how do you use CCM mode while specifying the Cipher#auth_data? All the examples I find online are using GCM mode which seems to be working, while CCM does not. The exception I encounter is a very generic OpenSSL::Cipher::CipherError with no additional details.
The example code I have below works when algo is either option (AES-128-CCM or AES-128-GCM), and no auth_data is nil, or when algo is AES-128-GCM and the auth_data is non-nil.
I'm using Ruby v2.5.5 and OpenSSL v2.1.2.
algo = 'AES-128-CCM' # AES-128-GCM works
auth_data = OpenSSL::Random.random_bytes(32) # nil works
cipher = OpenSSL::Cipher.new(algo).encrypt
if algo == 'AES-128-CCM'
cipher.auth_tag_len = 16
cipher.iv_len = 11
elsif algo == 'AES-128-GCM'
cipher.iv_len = 12
end
key = cipher.random_key
iv = cipher.random_iv
# has to be done in this order for unknown reasons
if algo == 'AES-128-CCM'
encrypted = cipher.update('Hello World') + cipher.final
cipher.auth_data = auth_data unless auth_data.nil?
elsif algo == 'AES-128-GCM'
cipher.auth_data = auth_data unless auth_data.nil?
encrypted = cipher.update('Hello World') + cipher.final
end
auth_tag = cipher.auth_tag
cipher = OpenSSL::Cipher.new(algo).decrypt
if algo == 'AES-128-CCM'
cipher.auth_tag_len = 16
cipher.iv_len = 11
elsif algo == 'AES-128-GCM'
cipher.iv_len = 12
end
cipher.key = key
cipher.iv = iv
cipher.auth_tag = auth_tag
cipher.update(encrypted) # crashes when auth_data != nil and algo == AES-128-CCM
You've switched the order of the authentication data and plaintext around in your call for CCM.
encrypted = cipher.update('Hello World') + cipher.final
cipher.auth_data = auth_data unless auth_data.nil?
In general, the additional authenticated data needs to be processed before the plaintext data. There are some tricks around that for GCM (which may be implemented or not) but not for CCM. EAX mode does allow any order of the arguments as it was explicitly made to be a more flexible implementation of an authenticating cipher.
Related
I am using Ruby's Open SSL bindings to do AES-256 encryption. I can encrypt a non-empty string. However, when attempting to encrypt an empty string, Ruby raises an exception complaining that the data must not be empty. How can I encrypt an empty string using Ruby's OpenSSL bindings?
Code to reproduce the problem
require "openssl"
KEY = OpenSSL::Cipher::Cipher.new("aes-256-cbc").random_key
def encrypt(plaintext)
cipher = OpenSSL::Cipher::Cipher.new("aes-256-cbc")
cipher.encrypt
iv = cipher.random_iv
cipher.iv = iv
cipher.key = KEY
ciphertext = cipher.update(plaintext) # <- ArgumentError here
ciphertext << cipher.final
[iv, ciphertext]
end
def decrypt(iv, ciphertext)
cipher = OpenSSL::Cipher::Cipher.new("aes-256-cbc")
cipher.decrypt
cipher.iv = iv
cipher.key = KEY
plaintext = cipher.update(ciphertext)
plaintext << cipher.final
plaintext
end
p decrypt(*encrypt("foo")) # "foo"
p decrypt(*encrypt(""))
# /tmp/foo.rb:11:in `update': data must not be empty (ArgumentError)
# from /tmp/foo.rb:11:in `encrypt'
# from /tmp/foo.rb:27:in `<main>'
Versions
ruby-2.2.2p95
OpenSSL::VERSION is "1.1.0"
Microsoft SQL Server 2014 (12.0.2000.8)
Why do I want to encrypt empty strings?
I am writing an ETL program to migrate data from one database to a SqlServer database. Certain columns from the source database must be encrypted before writing them to the destination database. The source columns may contain any data, including empty strings. The destination columns are usually non-nullable. The destination columns will be decrypted by .net code.
Goal #1: No information about the encrypted field, including whether or not it even exists, should be recoverable without properly decrypting it. An encrypted empty string should be indistinguishable from any other encrypted data.
Goal #2: The .net code that will decrypt these values should not need to handle empty strings specially.
If I can get openssl to encrypt empty strings, I will achieve both of these goals.
Workaround - Don't encrypt empty strings
I could just not encrypt empty strings, passing them through.
def encrypt(plaintext)
return plaintext if plaintext.empty?
...
end
def decrypt(iv, ciphertext)
return ciphertext if ciphertext.empty?
...
end
This has the disadvantages of exposing information, and also of requiring cooperating code to be written on the .net side.
Workaround - Add some constant to the plaintext
I could add some constant string to the plaintext before encryption, and remove it after decryption:
PLAINTEXT_SUFFIX = " "
def encrypt(plaintext)
plaintext += PLAINTEXT_SUFFIX
...
end
def decrypt(iv, ciphertext)
...
plaintext.chomp(PLAINTEXT_SUFFIX)
end
This hides whether the data exists or not, but still requires cooperating .net code.
As suggested by #ArtjomB, it's as simple as not calling Cipher#update with the empty string. The value returned by Cipher#final then properly encrypts an empty string.
require "openssl"
KEY = OpenSSL::Cipher::Cipher.new("aes-256-cbc").random_key
def encrypt(plaintext)
cipher = OpenSSL::Cipher::Cipher.new("aes-256-cbc")
cipher.encrypt
iv = cipher.random_iv
cipher.iv = iv
cipher.key = KEY
ciphertext = ""
ciphertext << cipher.update(plaintext) unless plaintext.empty?
ciphertext << cipher.final
[iv, ciphertext]
end
def decrypt(iv, ciphertext)
cipher = OpenSSL::Cipher::Cipher.new("aes-256-cbc")
cipher.decrypt
cipher.iv = iv
cipher.key = KEY
plaintext = cipher.update(ciphertext)
plaintext << cipher.final
end
p decrypt(*encrypt("foo")) # "foo"
p decrypt(*encrypt("")) # ""
If you can use DBMS provided encryption functions, then, MySQL AES_ENCRYPT, seems to be able to encrypt blank string.
For example:
UPDATE some_table
SET some_column = AES_ENCRYPT('',UNHEX('F3229A0B371ED2D9441B830D21A390C3'));
It's AES-128 by default, I am guessing that will be a problem as you need AES-256. Also, not sure which DBMS you are using and whether that DBMS has encryption functions.
I have two scripts for encrypting and decrypting messages using AES.
Here's encrypt.rb:
require 'openssl'
require 'base64'
require 'digest'
KEY = 'sekrit_key'
MESSAGE = "My Name is Rabbit and I don't know anything!"
cipher = OpenSSL::Cipher::AES256.new(:CFB)
cipher.encrypt
cipher.key = KEY
# hexdigest the IV to make sure encode64 doesn't fuck up
iv = Digest::MD5.hexdigest(cipher.random_iv)
cipher.iv = iv
encrypted = cipher.update(secret_message)+cipher.final
puts Base64.urlsafe_encode64(iv+'|'+encrypted.encode)
... and decrypt.rb:
require 'openssl'
require 'base64'
KEY = 'sekrit_key'
encrypted_message = STDIN.read.strip
parts = Base64.urlsafe_decode64(encrypted_message ).split('|')
iv = parts[0]
encrypted = parts[1]
decipher = OpenSSL::Cipher::AES256.new(:CFB)
decipher.decrypt
decipher.key = KEY
decipher.iv = iv
message = decipher.update(encrypted)+decipher.final
if message.eql?("My Name is Rabbit and I don't know anything!")
print '.'
else
puts
puts encrypted_message
puts message
end
When I now continuosly run both scripts, My output is cut off quite often!
$ while true; do ruby encrypt.rb | ruby decrypt.rb; done
....
YmZhNDg2ODJjNGZiOGIzZTcyMzAwYzMxZWUwNWI0Y2V8w2aJk930EL3gh3rfQsd2B3xZKy5wjoCzlZoYHBgmv6m51ZwAWQHGtCJoNRg=
My Name is Rabbi
..
YjMwNDQxOGRjMjg4NGEzOThmM2IwNGFiZDBiZTQxZGZ8OfLyjGQGKV3PPUpCvfL08IDuk7M7d3w7fj6F5Rql94jkRdwaCuuMfedqtFk=
My Name is Rabbit and
..
OWUxYzFlZWU5MTc4NGZjYWYxYzZiOGEwOTBjOGMxYzJ8g7I4X_Dt6K9ufByMhLBGlpoYCv8vlR0lTBqP-zS647tmmFh81rXdR8T-UkM=
My Name i
....
# and so on
Why are so many of my messages cut off?
Update: When using a fixed IV in encrypt.rb (eg. substitute cipher.random_iv with KEY) instead of a randomly generated, the problem does not occur.
The problem stems from the fact that you're treating binary data as strings. iv as well as encrypted.encode are binary and you're concatenating them with "|" (a string). Both iv as well as the message may contain the pipe character which will cause problems when splitting. In general, it's best to base64 both parts separately.
Here's working code:
encrypt.rb
require 'openssl'
require 'base64'
require 'digest'
KEY = 'sekrit_key123456' * 2
MESSAGE = "My Name is Rabbit and I don't know anything!"
cipher = OpenSSL::Cipher::AES256.new(:CFB)
cipher.encrypt
cipher.key = KEY
iv = cipher.random_iv
encrypted = cipher.update(MESSAGE)+cipher.final
puts Base64.strict_encode64(iv)+'|'+Base64.strict_encode64(encrypted.encode)
decrypt.rb
require 'openssl'
require 'base64'
KEY = 'sekrit_key123456' * 2 # key needs to be the right length
encrypted_message = STDIN.read.strip
parts = encrypted_message.split('|')
iv = Base64.strict_decode64(parts[0])
encrypted = Base64.strict_decode64(parts[1])
decipher = OpenSSL::Cipher::AES256.new(:CFB)
decipher.decrypt
decipher.key = KEY
decipher.iv = iv
message = decipher.update(encrypted)+decipher.final
if message.eql?("My Name is Rabbit and I don't know anything!")
print '.'
else
puts
puts encrypted_message
puts message
end
Also notice that random_iv already assigns iv to cipher, so you don't need to (see source under http://apidock.com/ruby/v1_9_3_125/OpenSSL/Cipher/random_iv ).
Why is there a difference in blowfish encryption between Crypt::CBC (perl) and OpenSSL (ruby)?
Perl
use Crypt::CBC;
my $cipher = Crypt::CBC->new( -key => 'length32length32length32length32', -cipher => 'Blowfish' );
my $ciphertext = $cipher->encrypt_hex('test');
# ciphertext is 53616c7465645f5f409c8b8eb353823c06d9b50537c92e19
Ruby
require "rubygems"
require "openssl"
cipher = OpenSSL::Cipher::Cipher.new("bf-cbc")
cipher.encrypt
cipher.key = "length32length32length32length32"
result = cipher.update("test") << cipher.final
ciphertext = result.unpack("H*").first
# ciphertext is 16f99115a09e0464
Crypt::CBC seems to be prepending Salted__ to the output by default. Can you explain what is going on that is so different between these? Is there a way to make OpenSSL behave in a similar way to Crypt::CBC?
Crypt::CBC (perl) uses its own method to randomize the salt and initialization vector. Plus in the case of Blowfish it uses a key length of 56 as mentioned above.
Using the perl code from your example:
Perl
use Crypt::CBC;
my $cipher = Crypt::CBC->new( -key => 'length32length32length32length32', -cipher => 'Blowfish' );
my $ciphertext = $cipher->encrypt_hex('test');
# 53616c7465645f5f409c8b8eb353823c06d9b50537c92e19
To decode this using ruby (OpenSSL) requires a little tweaking to extract the key and initialization vector:
Ruby
require 'openssl'
# Hex string to decode(from above)
string = '53616c7465645f5f409c8b8eb353823c06d9b50537c92e19'
# Pack Hex
string = [string].pack('H*')
# Some Config
pass = 'length32length32length32length32'
key_len = 56;
iv_len = 8;
desired_len = key_len + iv_len;
salt_re = /^Salted__(.{8})/
#Extract salt
salt = salt_re.match(string)
salt = salt.captures[0]
data = '';
d = '';
while (data.length < desired_len)
d = Digest::MD5::digest("#{d}#{pass}#{salt}");
data << d;
end
#Now you have extracted your key and initialization vector
key = data.slice(0..key_len-1)
iv = data.slice(key_len .. -1)
# Trim string of salt
string = string[16..-1]
cipher = OpenSSL::Cipher::Cipher.new "bf-cbc"
cipher.decrypt
cipher.key_len = key_len
cipher.key = key
cipher.iv = iv
puts cipher.update(string) << cipher.final
# test
Turns out the blowfish key size defaults are different between these two. OpenSSL defaults to 16, Crypt::Blowfish defaults to 56.
You can override key size in Crypt::CBC by specifying -keysize => n, but unfortunately there does not appear to be a way to override the key size in OpenSSL.
The Openssl library defaults to 16 byte Blowfish key sizes, so for
compatibility with Openssl you may wish to set -keysize=>16
http://metacpan.org/pod/Crypt::CBC
Perl (specify keysize)
my $cipher = Crypt::CBC->new(
-key => 'length32length32length32length32',
-keysize => 16,
-cipher => 'Blowfish'
);
I have some testdata key/text/encrypted from an API provider and am now trying to yield the same encrypted result with the function below, but my result diverts from the provided one in the last 16 of 241 digits. Do you have an idea, what the reason may be?
I ensured, that 'bf-ecb' is the right mode, and experimented with url-encoding, but so far without success.
require 'openssl'
def encrypt(key, data)
cipher = OpenSSL::Cipher::Cipher.new('bf-ecb').send(:encrypt)
cipher.key = key
result = cipher.update(data) << cipher.final
hexed = ''
result.each_byte { |c| hexed << '%02x' % c }
hexed.upcase
end
UPDATE
Also trying to decrypt the example result results in an OpenSSL::Cipher::CipherError "bad decrypt"
If your last 16 digits (=128 bits) are incorrect then it is likely there is a problem with the last block. Probably this is a problem with padding, your encryption is using one form of padding while your decryption is expecting a different padding. I suggest that you explicitly specify the padding at both sides. PKCS5 or PKCS7 are the usual choice. Faulty padding will also explain the "bad decrypt" error message.
It was indeed a problem with the padding. I worked around it with deactivating it and implementing it by myself.
So far it works.
This is how it looks like:
require 'openssl'
def encrypt(key,data)
cipher = OpenSSL::Cipher::Cipher.new "bf-ecb"
cipher.padding = 0
cipher.key = key
cipher.encrypt
enhex(cipher.update padd data)
end
def decrypt(key,data,len)
cipher = OpenSSL::Cipher::Cipher.new "bf-ecb"
cipher.padding = 0
cipher.key = key
cipher.decrypt
(cipher.update dehex(data)).slice(0,len)
end
def enhex(data)
hexed = ''
data.each_byte { |c| hexed << '%02x' % c }
hexed.upcase
end
def dehex(data)
data.scan(/../).map{ |b| b.to_i(16) }.pack('C*')
end
def padd(data)
data + " "*(8 - (data.length % 8))
end
You can simply do the blowfish encryption with ecb cipher mode just like this:
def blowfish_encrypt(key,data)
cipher = OpenSSL::Cipher::Cipher.new("bf-ecb").send :encrypt
cipher.key = key
cipher.update(data) << cipher.final
end
And you don't need to care about padding in this case.
I came across this blog post which describes how they encrypt their ID's. For educational and entertainment purposes, I decided to try this out. I can't quite get it, though.
As described in this blog post, I'm base 36 encoding the 3DES encrypted 64 bit padded value and keep getting the resulting string to be 25 characters long. How is it possible to get 13 characters, like the site claims, without using some stream cipher -- the blog claims to use a 3DES block cipher.
Here's the code I'm using:
require 'openssl'
SECRET_KEY = "secret"
ENCRYPTION_ALGO = "DES-EDE3-CBC"
def base36encode(s)
s.unpack('H*')[0].to_i(16).to_s 36
end
def base36decode(s36)
[s36.to_i(36).to_s(16)].pack 'H*'
end
def num_to_bits(n, bit_count=64)
#Array.new(bit_count) { |i| (n)[i] }.reverse!
sprintf('%064b', n).split("")
end
def bits_to_string(bits)
[bits.join("")].pack("B*")
end
def num_to_binstring(n, bit_count=64)
bits_to_string(num_to_bits(n, bit_count))
end
def binstring_to_num(str)
#elements = str.unpack("N*")
#(elements[0] << 32) | elements[1]
#
ans = 0
str.each_byte do |i|
ans = ans * 256 + i
end
ans
end
def encrypt(message, password)
cipher = OpenSSL::Cipher::Cipher.new(ENCRYPTION_ALGO)
cipher.encrypt
cipher.pkcs5_keyivgen(password)
ciphertext = cipher.update(message)
ciphertext << cipher.final
end
def decrypt(message, password)
cipher = OpenSSL::Cipher::Cipher.new(ENCRYPTION_ALGO)
cipher.decrypt
cipher.pkcs5_keyivgen(password)
decryptedtext = cipher.update(message)
decryptedtext << cipher.final
end
id = 12345678
puts "Encrypting: \"#{id}\""
num_string = num_to_binstring(id)
encrypted = encrypt(num_string, SECRET_KEY)
encoded = base36encode(encrypted).upcase
puts "Encrypted and encoded to: \"#{encoded}\" that's size is: #{encoded.length}\n"
decoded = base36decode(encoded.downcase)
decrypted = decrypt(decoded, SECRET_KEY)
string_num = binstring_to_num(decrypted)
puts "Decoded and decrypted to: \"#{string_num}\""
# ---- OUTPUT ---
# Encrypting: "12345678"
# Encrypted and encoded to: "49OMDVRHHMM24DVMODQU4X7JY" that's size is: 25
# Decoded and decrypted to: "12345678"
What size is the output of the encryption before you base36 encode? My guess is that your code is encrypting in CBC mode (given that your code mentions pkcs5) and you're getting the IV and ciphertext out, a total of 16 bytes. I infer from the blog post that the author is using ECB mode, what you might think of as "raw" 3DES, which is usually wrong for most situations but seems plausible here.