I'm trying to get a ruby implementation of an encryption lib that's apparently popular in the Java world -- PBEWithMD5AndDES
Does anyone know how to use openssl or another open source gem to perform encryption/decryption that's compatible with this format?
Updated:
I used a gem chilkat to implement it but it is paid, i need an opensource solution.
I know it is super old but I had the same problem and just solved it so here it goes
to encrypt, where salt is your salt sting, passkey is your password key string and iterations is number of iterations you want to use
def encrypt_account_number
cipher = OpenSSL::Cipher::Cipher.new("DES")
cipher.encrypt
cipher.pkcs5_keyivgen passkey, salt,iterations,digest
encrypted_account_number = cipher.update(account_number)
encrypted_account_number << cipher.final
Base64.encode64(encrypted_account_number )
end
def decrypt_account_number
cipher = OpenSSL::Cipher::Cipher.new("DES")
base_64_code = Base64.decode64(account_number)
cipher.decrypt
cipher.pkcs5_keyivgen passkey, salt,iterations,digest
decrypted_account_number = cipher.update base_64_code
decrypted_account_number << cipher.final
decrypted_account_number
end
You don't need to actually implement PBEWithMD5andDES assuming ruby has a DES implementation. What you need to implement is the key derivation function ( who you get a key out of a password) and then feed that derived key to DES with the appropriate mode and padding.
Thankfully, the key derivation function is not particularly security critical in implementation, so you can do it yourself safely enough. According to the rfc, PBEwithMD5AndDES is actually the PBKDF1 ( a ker derivation function) used with DES in CBC mode .
PBKDF1 does not look that hard to implement . Looks like you can do it with a for loop and an md5 call.
Note that you may still get some odd results because of the possibility of a different padding scheme being used in Java and Ruby. I assume that the spec one is pkcs 1.5 padding, but at a quick glance, I can't confirm this
5.1 PBKDF1
PBKDF1 applies a hash function, which shall be MD2 [6], MD5 [19] or
SHA-1 [18], to derive keys. The length of the derived key is bounded
by the length of the hash function output, which is 16 octets for MD2
and MD5 and 20 octets for SHA-1. PBKDF1 is compatible with the key
derivation process in PKCS #5 v1.5.
PBKDF1 is recommended only for compatibility with existing
applications since the keys it produces may not be large enough for
some applications.
PBKDF1 (P, S, c, dkLen)
Options: Hash underlying hash function
Input: P password, an octet string
S salt, an eight-octet string
c iteration count, a positive integer
dkLen intended length in octets of derived key,
a positive integer, at most 16 for MD2 or
MD5 and 20 for SHA-1
Output: DK derived key, a dkLen-octet string
Steps:
1. If dkLen > 16 for MD2 and MD5, or dkLen > 20 for SHA-1, output
"derived key too long" and stop.
2. Apply the underlying hash function Hash for c iterations to the
concatenation of the password P and the salt S, then extract
the first dkLen octets to produce a derived key DK:
T_1 = Hash (P || S) ,
T_2 = Hash (T_1) ,
...
T_c = Hash (T_{c-1}) ,
DK = Tc<0..dkLen-1>
3. Output the derived key DK.
For what its' worth, I'm posting my python code, which actually works (I have tons of encrypted values which were done using org.jasypt.util.text.BasicTextEncryptor, I needed to decrypt them.)
import base64
import hashlib
from Crypto.Cipher import DES
"""
Note about PBEWithMD5AndDES in java crypto library:
Encrypt:
Generate a salt (random): 8 bytes
<start derived key generation>
Append salt to the password
MD5 Hash it, and hash the result, hash the result ... 1000 times
MD5 always gives us a 16 byte hash
Final result: first 8 bytes is the "key" and the next is the "initialization vector"
(there is something about the first 8 bytes needing to be of odd paraity, therefore
the least significant bit needs to be changed to 1 if required. We don't do it,
maybe the python crypto library does it for us)
<end derived key generation>
Pad the input string with 1-8 bytes (note: not 0-7, so we always have padding)
so that the result is a multiple of 8 bytes. Padding byte value is same as number of
bytes being padded, eg, \x07 if 7 bytes need to be padded.
Use the key and iv to encrypt the input string, using DES with CBC mode.
Prepend the encrypted value with the salt (needed for decrypting since it is random)
Base64 encode it -> this is your result
Decrypt:
Base64 decode the input message
Extract the salt (first 8 bytes). The rest is the encoded text.
Use derived key generation as in Encrypt above to get the key and iv
Decrypt the encoded text using key and iv
Remove padding -> this is your result
(I only have implemented decrypt here since that's all I needed,
but encrypt should be straighforward as well)
"""
def get_derived_key(password, salt, count):
key = password + salt
for i in range(count):
m = hashlib.md5(key)
key = m.digest()
return (key[:8], key[8:])
def decrypt(msg, password):
msg_bytes = base64.b64decode(msg)
salt = msg_bytes[:8]
enc_text = msg_bytes[8:]
(dk, iv) = get_derived_key(password, salt, 1000)
crypter = DES.new(dk, DES.MODE_CBC, iv)
text = crypter.decrypt(enc_text)
# remove the padding at the end, if any
return re.sub(r'[\x01-\x08]','',text)
I've updated python script from user3392439, with encrypt support. Wish it helpful.
import base64
import hashlib
import re
import os
from Crypto.Cipher import DES
"""
Note about PBEWithMD5AndDES in java crypto library:
Encrypt:
Generate a salt (random): 8 bytes
<start derived key generation>
Append salt to the password
MD5 Hash it, and hash the result, hash the result ... 1000 times
MD5 always gives us a 16 byte hash
Final result: first 8 bytes is the "key" and the next is the "initialization vector"
(there is something about the first 8 bytes needing to be of odd paraity, therefore
the least significant bit needs to be changed to 1 if required. We don't do it,
maybe the python crypto library does it for us)
<end derived key generation>
Pad the input string with 1-8 bytes (note: not 0-7, so we always have padding)
so that the result is a multiple of 8 bytes. Padding byte value is same as number of
bytes being padded, eg, \x07 if 7 bytes need to be padded.
Use the key and iv to encrypt the input string, using DES with CBC mode.
Prepend the encrypted value with the salt (needed for decrypting since it is random)
Base64 encode it -> this is your result
Decrypt:
Base64 decode the input message
Extract the salt (first 8 bytes). The rest is the encoded text.
Use derived key generation as in Encrypt above to get the key and iv
Decrypt the encoded text using key and iv
Remove padding -> this is your result
(I only have implemented decrypt here since that's all I needed,
but encrypt should be straighforward as well)
"""
def get_derived_key(password, salt, count):
key = password + salt
for i in range(count):
m = hashlib.md5(key)
key = m.digest()
return (key[:8], key[8:])
def decrypt(msg, password):
msg_bytes = base64.b64decode(msg)
salt = msg_bytes[:8]
enc_text = msg_bytes[8:]
(dk, iv) = get_derived_key(password, salt, 1000)
crypter = DES.new(dk, DES.MODE_CBC, iv)
text = crypter.decrypt(enc_text)
# remove the padding at the end, if any
return re.sub(r'[\x01-\x08]','',text)
def encrypt(msg, password):
salt = os.urandom(8)
pad_num = 8 - (len(msg) % 8)
for i in range(pad_num):
msg += chr(pad_num)
(dk, iv) = get_derived_key(password, salt, 1000)
crypter = DES.new(dk, DES.MODE_CBC, iv)
enc_text = crypter.encrypt(msg)
return base64.b64encode(salt + enc_text)
def main():
msg = "hello, world"
passwd = "mypassword"
s = encrypt(msg, passwd)
print s
print decrypt(s, passwd)
if __name__ == "__main__":
main()
For #cooljohny
I do not really recall how this code works, but I'm 99% sure it does. I wrote it by carefully digging through the spec from the Java implementation of pbewithmd5anddes, and testing this python against that. I delivered exactly this code to a client, and it worked fine for them. I changed the constants before pasting it here, but that's all. You should be able to confirm that it produces the same encrypted output as the Java lib, and then replicate it in ruby. Good luck!
import base64
from Crypto.Cipher import DES
from passlib.utils.pbkdf2 import pbkdf1
password = 'xxxxxxx'
iterations = 22
salt_bytes = [19,15,78,45,34,90,12,11]
# convert saltBytes to a string
salt_string = ''.join([chr(a) for a in salt_bytes])
# a sample request
raw_data = '''{"something":"to","encrypt":"here"}'''
# from the standard...
padding_value = (8 - (raw_data.__len__() % 8))
padding_data = chr(padding_value) * padding_value
padded_data = raw_data + padding_data
# 22 iterations, 16 is the # of bytes in an md5 digest
pbkres = pbkdf1(password, salt_string, iterations, 16, 'md5')
# split the digest into two 8-byte halves
# this gives the DES secret key and initializing vector
des_key, iv = pbkres[0:8], pbkres[8:16]
# encrypt with DES
cipher = DES.new(des_key, DES.MODE_CBC, iv)
cmsg = cipher.encrypt(padded_data)
# and base64 encode
base64.b64encode(cmsg)
Related
I am trying to replicate the encryption result from here in Ruby using OpenSSL: https://emvlab.org/descalc/?key=18074F7ADD44C903&iv=18074F7ADD44C903&input=4E5A56564F4C563230313641454E5300&mode=cbc&action=Encrypt&output=25C843BA5C043FFFB50F76E43A211F8D
Original string = "NZVVOLV2016AENS"
String converted to hexadecimal = "4e5a56564f4c563230313641454e53"
iv = "18074F7ADD44C903"
key = "18074F7ADD44C903"
Expected result = "9B699B4C59F1444E8D37806FA9D15F81"
Here is my ruby code:
require 'openssl'
require "base64"
include Base64
iv = "08074F7ADD44C903"
cipher = "08074F7ADD44C903"
def encode(string)
puts "Attempting encryption - Input: #{string}"
encrypt = OpenSSL::Cipher.new('DES-CBC')
encrypt.encrypt
encrypt.key = ["18074F7ADD44C903"].pack('H*') #.scan(/../).map{|b|b.hex}.pack('c*')
encrypt.iv = ["18074F7ADD44C903"].pack('H*')
result = encrypt.update(string) + encrypt.final
puts "Raw output: #{result.inspect}"
unpacked = result.unpack('H*')[0]
puts "Encrypted key is: #{unpacked}"
puts "Encrypted Should be: 9B699B4C59F1444E8D37806FA9D15F81"
return unpacked
end
res = encode("NZVVOLV2016AENS")
Output:
Encrypted key is: 9b699b4c59f1444ea723ab91e89c023a
Encrypted Should be: 9B699B4C59F1444E8D37806FA9D15F81
Interestingly, the first half of the result is correct, and the last 16 digits are incorrect.
The web site uses Zero padding by default, while the Ruby code uses PKCS#7 padding by default.
Ruby does not seem to support Zero padding, so disable the default padding and implement Zero padding yourself.
Zero padding pads to the next full block size with 0x00 values. The block size for DES is 8 bytes. If the last block of the plaintext is already filled, no padding is done:
def zeroPad(string, blocksize)
len = string.bytes.length
padLen = (blocksize - len % blocksize) % blocksize
string += "\0" * padLen
return string
end
In the encode() function (which should better be called encrypt() function) the following lines must be added before encryption:
encrypt.padding = 0 # disable PKCS#7 padding
string = zeroPad(string, 8) # enable Zero padding
The modified Ruby code then gives the same ciphertext as the web site.
Note that DES is insecure, also it' s insecure to use the key as IV (as well as a static IV). Furthermore, Zero padding is unreliable in contrast to PKCS#7 padding.
I'm having troubles to extract the IV generated with the encrypt method from encrypted_strings library for a specific password I provide. From the documentation, I see that this method generates a key and iv based on a password using a C library that calls the same method as openssl to generate the key and iv: EVP_BytesToKey.
What I'm trying to do is to be able to print the IV for any password I specify so I can port the encryption to another language.
Can you think of any method to extract/print this IV vector from a password?
These are the details of the algorithm, mode and padding this library uses:
ALGO: DES-EDE3
MODE: CBC
PADDING: PKCS5
The ruby script below prints out the encrypted message but no clue which iv was used.
#!/usr/bin/ruby
require 'encrypted_strings'
data = 'Whackabad'
password = 'bAJLyifeUJUBFWdHzVbykfDmPHtLKLMzViHW9aHGmyTLD8hGYZ'
encrypted_data = data.encrypt(:symmetric, :password => password)
printf "Data: #{data}\n"
printf "Encrypted Data: #{encrypted_data}"
I tried to use openssl as it allows me to print the iv and key generated using -p option but it uses a PKCS7 padding instead of PKCS5. So if I run the command below, doesn't print the same encrypted string as the ruby code above.
echo -n 'Whackabad' | openssl enc -des-ede3-cbc -nosalt -a -k bAJLyifeUJUBFWdHzVbykfDmPHtLKLMzViHW9aHGmyTLD8hGYZ
NOTE:
-a: base64 encode, -k: password, and echo -n: removes the new line from the string so its exactly the same size as the ruby in string.
If I add -nopad option, I don't know how to pad the output to get exactly the same encrypted result.
Any help would be much appreciated
PKCS7 padding is basically the same as PKCS5. The reason you get a different result on the command line is that it only uses a single hash iteration, where the function used by encrypted_strings does 2048 iterations by default.
The function used, EVP_BytesToKey is described in the OpenSSL wiki, which include details of the algorithm. Reproducing it in Ruby might look something like this (using MD5 and 2048 iterations):
def hash(d, count)
count.times do
d = OpenSSL::Digest.digest('md5', d)
end
d
end
password = 'bAJLyifeUJUBFWdHzVbykfDmPHtLKLMzViHW9aHGmyTLD8hGYZ'
bytes = ''
last = ''
# For des-ede3-cbc, 24 byte key + 8 byte IV = 32 bytes.
while bytes.length < 32
last = hash(last + password, 2048)
bytes << last
end
key = bytes[0...24]
iv = bytes[24..-1]
You can use these values to decrypt the result of your code (add require 'base64' first):
# This is the result of your code:
encrypted_data = "AEsDXVcgh2jsTjlDgh+REg=="
# enrypted_strings produces base64 encoded results, so we decode first
encrypted_data = Base64.decode64(encrypted_data)
cipher = OpenSSL::Cipher.new('des-ede3-cbc')
cipher.decrypt
cipher.key = key
cipher.iv = iv
plain = cipher.update(encrypted_data) + cipher.final
puts plain #=> "Whackabad"
I am trying to create a bitcoin address in ruby according to the documentation of bitcoin wiki (bitcoin creation according bitcoin wiki).
Starting point is just some random string which emulates the output of ripmed160.
Unfortunately I don't quite succeed in doing so, here is my code:
require 'base58_gmp'
tx_hash = "a8d0c0184dde994a09ec054286f1ce581bebf46446a512166eae7628734ea0a5"
ripmed160 = tx_hash[0..39]
ripmed160_with_pre = "00" + ripmed160
sha1 = Digest::SHA256.hexdigest ripmed160_with_pre
sha2 = Digest::SHA256.hexdigest sha1
bin_address = Integer("0x" + ripmed160_with_pre + sha2[0..7])
bitcoin_address = "1" + Base58GMP.encode(bin_address, 'bitcoin') # => "1GPcbTYDBwJ42MfKkedxjmJ3nrgoaNd2Sf"
I get something that looks like a bitcoin address but it is not recognised by blockchain.info so I guess it is invalid.
Can you please help me to make that work.
When you calculate the SHA256 checksum, make sure to calculate it over the actual bytes of the previous step, not the hex encoding of those bytes:
# First convert to actual bytes.
bytes = [ripmed160_with_pre].pack('H*')
# Now calculate the first hash over the raw bytes, and
# return the raw bytes again for the next hash
# (note: digest not hexdigest).
sha1 = Digest::SHA256.digest bytes
# Second SHA256, using the raw bytes from the previous step
# but this time we can use hexdigest as the rest of the code
# assumes hex encoded strings
sha2 = Digest::SHA256.hexdigest sha1
I am trying to encrypt the same text using AES w/ the same key and iv. I use the bash approach and ruby's openssl stdlib, and I b64 encode the encrypted result. But the results are different! I am trying to understand why. Here is what I did:
Ruby (1.9.3-p448)
data = "foobar"
key = ["a8f9f5c5b4c42c75e255a73b6be311e1"].pack("H*")
cipher = OpenSSL::Cipher::AES.new(128, :CBC)
cipher.encrypt
cipher.key = key
iv = cipher.random_iv
enc = cipher.update(data) + cipher.final
Base64.strict_encode64(enc)
cipher_key_hex = key.each_byte.map { |b| b.to_s(16) }.join
iv_hex = iv.each_byte.map { |b| b.to_s(16) }.join
Bash
The -iv parameter is set to the iv_hex value calculated above.
$ echo -n 'foobar' > foobar.txt
$ openssl enc -aes-128-cbc -base64 -in foobar.txt -K a8f9f5c5b4c42c75e255a73b6be311e1 -iv 5dc7ff87987c5a25ae73433f11bd6fa
=========================================
I double checked the IV is the same in both approaches.
Ruby output: rbRIW7iTsNC6LttBQ+Amzg==
Bash output: CcSs1+ssEu8FYkcspLeCWw==
What am I missing? This is baffling.
The issue is that b.to_s will not return two characters per byte. This means that any byte value less than 16 will be converted into one byte, and if those bytes are concatenated then the hexadecimal digits of different bytes will be combined, resulting in an invalid IV or key. This fact is hidden by the cryptographic API, which may pad the keys and IV to a correct size, instead of rejecting them (as it should).
You can use unpack("H*") instead to perform the correct encoding of the bytes to hexadecimal. This is also nicely symmetric to pack("H*") used for decoding the hexadecimal values.
I have been using the following code in my app for the past year and have 200k record using this code:
options = { :algorithm => 'aes-256-cbc', :value => "changethis", :key => "secretkey" }
cipher = OpenSSL::Cipher::Cipher.new(options[:algorithm])
cipher.send(:encrypt)
cipher.pkcs5_keyivgen(options[:key])
result = cipher.update(options[:value])
result << cipher.final
# => "x\xED\x14s\xFD\x0E\x97\xC5\x996[M\x1E\x94\xDEI"
I am required (by business) to refactor the pkcs5_keyivgen part, to do it correctly: For example,
options = { :algorithm => 'aes-256-cbc', :value => "changethis", :key => "secretkey" }
cipher = OpenSSL::Cipher::Cipher.new(options[:algorithm])
cipher.send(:encrypt)
cipher.key = '' # ??? 1) How does pkcs5_keyivgen in above code generate key, or does it just use my options[:key]
cipher.iv = '' # ??? 2) How does pkcs5_keyivgen in above code generate iv
result = cipher.update(options[:value])
result << cipher.final
I have to figure out how pkcs5_keyivgen sets key and iv. ideas here? We are using ruby-1.9.3-p286 and encryptor-1.1.3
I saw this question and this question, but they haven't help me solve the problem.
Was trying to solve this problem, but I think there is no easy solution or i just can't see one). Pkcs5_keyivgen() is deprecated and implements non-standard pass key derivation for AES 256.
From this docs and this source code
Pkcs5_keyivgen (pass, SALT = nil, num = 2048, digest = "MD5") -> nil
Generates some key and IV from salt and pass. No salt in your case. Generation method is defined in v1.5 PKCS #5 (deprecated)
So you are looking for "Password Based Key Derivation Function". PBKDF1
Pkcs5_keyivgen() function calls EVP_BytesToKey() from Openssl and EVP_BytesToKey() generates key bytes for larger key size in a non-standard way
So MD5 generates hash of size EVP_MAX_MD_SIZE (16 + 20) // 16 for MD5
But AES key(32) + IV(16) sizes > EVP_MAX_MD_SIZE, so AES 256 will need multiple hashes to generate random key and IV. Here is source code of algorithm in C
And here is nice pseudo-code explanation of the EVP_BytesToKey()
If you really want to re-implement PBKDF1 here is also RTC2898 standard for PBKDF1
But i don't think that it is a good idea to implement crypto yourself