I am trying to replicate the encryption result from here in Ruby using OpenSSL: https://emvlab.org/descalc/?key=18074F7ADD44C903&iv=18074F7ADD44C903&input=4E5A56564F4C563230313641454E5300&mode=cbc&action=Encrypt&output=25C843BA5C043FFFB50F76E43A211F8D
Original string = "NZVVOLV2016AENS"
String converted to hexadecimal = "4e5a56564f4c563230313641454e53"
iv = "18074F7ADD44C903"
key = "18074F7ADD44C903"
Expected result = "9B699B4C59F1444E8D37806FA9D15F81"
Here is my ruby code:
require 'openssl'
require "base64"
include Base64
iv = "08074F7ADD44C903"
cipher = "08074F7ADD44C903"
def encode(string)
puts "Attempting encryption - Input: #{string}"
encrypt = OpenSSL::Cipher.new('DES-CBC')
encrypt.encrypt
encrypt.key = ["18074F7ADD44C903"].pack('H*') #.scan(/../).map{|b|b.hex}.pack('c*')
encrypt.iv = ["18074F7ADD44C903"].pack('H*')
result = encrypt.update(string) + encrypt.final
puts "Raw output: #{result.inspect}"
unpacked = result.unpack('H*')[0]
puts "Encrypted key is: #{unpacked}"
puts "Encrypted Should be: 9B699B4C59F1444E8D37806FA9D15F81"
return unpacked
end
res = encode("NZVVOLV2016AENS")
Output:
Encrypted key is: 9b699b4c59f1444ea723ab91e89c023a
Encrypted Should be: 9B699B4C59F1444E8D37806FA9D15F81
Interestingly, the first half of the result is correct, and the last 16 digits are incorrect.
The web site uses Zero padding by default, while the Ruby code uses PKCS#7 padding by default.
Ruby does not seem to support Zero padding, so disable the default padding and implement Zero padding yourself.
Zero padding pads to the next full block size with 0x00 values. The block size for DES is 8 bytes. If the last block of the plaintext is already filled, no padding is done:
def zeroPad(string, blocksize)
len = string.bytes.length
padLen = (blocksize - len % blocksize) % blocksize
string += "\0" * padLen
return string
end
In the encode() function (which should better be called encrypt() function) the following lines must be added before encryption:
encrypt.padding = 0 # disable PKCS#7 padding
string = zeroPad(string, 8) # enable Zero padding
The modified Ruby code then gives the same ciphertext as the web site.
Note that DES is insecure, also it' s insecure to use the key as IV (as well as a static IV). Furthermore, Zero padding is unreliable in contrast to PKCS#7 padding.
I'm trying to populate TreeView with data from os.listdir(path).
All is ok until I read a directory name with a non-utf character. In my case 0xf6 which is not utf8.
As I'm running on Windows the charset from os.listdir() is Windows-1252 or ANSI.
How can I solve this problem to achieve correct display in TreeView?
Here some of my code:
def fill_tree(treeview, node):
if treeview.set(node, "type") != 'directory':
return
path = treeview.set(node, "fullpath")
# Delete the possibly 'dummy' node present.
treeview.delete(*treeview.get_children(node))
parent = treeview.parent(node)
for p in os.listdir(path):
ptype = None
p = os.path.join(path, p)
if os.path.isdir(p):
ptype = 'directory'
fname = os.path.split(p)[1].decode('cp1252').encode('utf8')
if ptype == 'directory':
oid = treeview.insert(node, 'end', text=fname, values=[p, ptype])
treeview.insert(oid, 0, text='dummy')
Regards
Göran
The UnicodeDecodeError is due to passing byte strings when the function is expecting Unicode strings. Python 2 attempts to implicitly decode byte strings to Unicode. Use Unicode strings explicitly instead. os.listdir(unicode_path) will return Unicode string, for example os.listdir(u'.').
I am trying to create a bitcoin address in ruby according to the documentation of bitcoin wiki (bitcoin creation according bitcoin wiki).
Starting point is just some random string which emulates the output of ripmed160.
Unfortunately I don't quite succeed in doing so, here is my code:
require 'base58_gmp'
tx_hash = "a8d0c0184dde994a09ec054286f1ce581bebf46446a512166eae7628734ea0a5"
ripmed160 = tx_hash[0..39]
ripmed160_with_pre = "00" + ripmed160
sha1 = Digest::SHA256.hexdigest ripmed160_with_pre
sha2 = Digest::SHA256.hexdigest sha1
bin_address = Integer("0x" + ripmed160_with_pre + sha2[0..7])
bitcoin_address = "1" + Base58GMP.encode(bin_address, 'bitcoin') # => "1GPcbTYDBwJ42MfKkedxjmJ3nrgoaNd2Sf"
I get something that looks like a bitcoin address but it is not recognised by blockchain.info so I guess it is invalid.
Can you please help me to make that work.
When you calculate the SHA256 checksum, make sure to calculate it over the actual bytes of the previous step, not the hex encoding of those bytes:
# First convert to actual bytes.
bytes = [ripmed160_with_pre].pack('H*')
# Now calculate the first hash over the raw bytes, and
# return the raw bytes again for the next hash
# (note: digest not hexdigest).
sha1 = Digest::SHA256.digest bytes
# Second SHA256, using the raw bytes from the previous step
# but this time we can use hexdigest as the rest of the code
# assumes hex encoded strings
sha2 = Digest::SHA256.hexdigest sha1
I'm trying to format a UUIDv4 into a url friendly string. The typical format in base16 is pretty long and has dashes:
xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx
To avoid dashes and underscores I was going to use base58 (like bitcoin does) so each character fully encode sqrt(58).floor = 7 bits.
I can pack the uuid into binary with:
[ uuid.delete('-') ].pack('H*')
To get 8-bit unsigned integers its:
binary.unpack('C*')
How can i unpack every 7-bits into 8-bit unsigned integers? Is there a pattern to scan 7-bits at a time and set the high bit to 0?
require 'base58'
uuid ="123e4567-e89b-12d3-a456-426655440000"
Base58.encode(uuid.delete('-').to_i(16))
=> "3fEgj34VWmVufdDD1fE1Su"
and back again
Base58.decode("3fEgj34VWmVufdDD1fE1Su").to_s(16)
=> "123e4567e89b12d3a456426655440000"
A handy pattern to reconstruct the uuid format from a template
template = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'
src = "123e4567e89b12d3a456426655440000".each_char
template.each_char.reduce(''){|acc, e| acc += e=='-' ? e : src.next}
=> "123e4567-e89b-12d3-a456-426655440000"
John La Rooy's answer is great, but I just wanted to point out how simple the Base58 algorithm is because I think it's neat. (Loosely based on the base58 gem, plus bonus original int_to_uuid function):
ALPHABET = "123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ".chars
BASE = ALPHABET.size
def base58_to_int(base58_val)
base58_val.chars
.reverse_each.with_index
.reduce(0) do |int_val, (char, index)|
int_val + ALPHABET.index(char) * BASE ** index
end
end
def int_to_base58(int_val)
''.tap do |base58_val|
while int_val > 0
int_val, mod = int_val.divmod(BASE)
base58_val.prepend ALPHABET[mod]
end
end
end
def int_to_uuid(int_val)
base16_val = int_val.to_s(16)
[ 8, 4, 4, 4, 12 ].map do |n|
base16_val.slice!(0...n)
end.join('-')
end
uuid = "123e4567-e89b-12d3-a456-426655440000"
int_val = uuid.delete('-').to_i(16)
base58_val = int_to_base58(int_val)
int_val2 = base58_to_int(base58_val)
uuid2 = int_to_uuid(int_val2)
printf <<END, uuid, int_val, base_58_val, int_val2, uuid2
Input UUID: %s
Input UUID as integer: %d
Integer encoded as base 58: %s
Integer decoded from base 58: %d
Decoded integer as UUID: %s
END
Output:
Input UUID: 123e4567-e89b-12d3-a456-426655440000
Input UUID as integer: 24249434048109030647017182302883282944
Integer encoded as base 58: 3fEgj34VWmVufdDD1fE1Su
Integer decoded from base 58: 24249434048109030647017182302883282944
Decoded integer as UUID: 123e4567-e89b-12d3-a456-426655440000
I'm trying to get a ruby implementation of an encryption lib that's apparently popular in the Java world -- PBEWithMD5AndDES
Does anyone know how to use openssl or another open source gem to perform encryption/decryption that's compatible with this format?
Updated:
I used a gem chilkat to implement it but it is paid, i need an opensource solution.
I know it is super old but I had the same problem and just solved it so here it goes
to encrypt, where salt is your salt sting, passkey is your password key string and iterations is number of iterations you want to use
def encrypt_account_number
cipher = OpenSSL::Cipher::Cipher.new("DES")
cipher.encrypt
cipher.pkcs5_keyivgen passkey, salt,iterations,digest
encrypted_account_number = cipher.update(account_number)
encrypted_account_number << cipher.final
Base64.encode64(encrypted_account_number )
end
def decrypt_account_number
cipher = OpenSSL::Cipher::Cipher.new("DES")
base_64_code = Base64.decode64(account_number)
cipher.decrypt
cipher.pkcs5_keyivgen passkey, salt,iterations,digest
decrypted_account_number = cipher.update base_64_code
decrypted_account_number << cipher.final
decrypted_account_number
end
You don't need to actually implement PBEWithMD5andDES assuming ruby has a DES implementation. What you need to implement is the key derivation function ( who you get a key out of a password) and then feed that derived key to DES with the appropriate mode and padding.
Thankfully, the key derivation function is not particularly security critical in implementation, so you can do it yourself safely enough. According to the rfc, PBEwithMD5AndDES is actually the PBKDF1 ( a ker derivation function) used with DES in CBC mode .
PBKDF1 does not look that hard to implement . Looks like you can do it with a for loop and an md5 call.
Note that you may still get some odd results because of the possibility of a different padding scheme being used in Java and Ruby. I assume that the spec one is pkcs 1.5 padding, but at a quick glance, I can't confirm this
5.1 PBKDF1
PBKDF1 applies a hash function, which shall be MD2 [6], MD5 [19] or
SHA-1 [18], to derive keys. The length of the derived key is bounded
by the length of the hash function output, which is 16 octets for MD2
and MD5 and 20 octets for SHA-1. PBKDF1 is compatible with the key
derivation process in PKCS #5 v1.5.
PBKDF1 is recommended only for compatibility with existing
applications since the keys it produces may not be large enough for
some applications.
PBKDF1 (P, S, c, dkLen)
Options: Hash underlying hash function
Input: P password, an octet string
S salt, an eight-octet string
c iteration count, a positive integer
dkLen intended length in octets of derived key,
a positive integer, at most 16 for MD2 or
MD5 and 20 for SHA-1
Output: DK derived key, a dkLen-octet string
Steps:
1. If dkLen > 16 for MD2 and MD5, or dkLen > 20 for SHA-1, output
"derived key too long" and stop.
2. Apply the underlying hash function Hash for c iterations to the
concatenation of the password P and the salt S, then extract
the first dkLen octets to produce a derived key DK:
T_1 = Hash (P || S) ,
T_2 = Hash (T_1) ,
...
T_c = Hash (T_{c-1}) ,
DK = Tc<0..dkLen-1>
3. Output the derived key DK.
For what its' worth, I'm posting my python code, which actually works (I have tons of encrypted values which were done using org.jasypt.util.text.BasicTextEncryptor, I needed to decrypt them.)
import base64
import hashlib
from Crypto.Cipher import DES
"""
Note about PBEWithMD5AndDES in java crypto library:
Encrypt:
Generate a salt (random): 8 bytes
<start derived key generation>
Append salt to the password
MD5 Hash it, and hash the result, hash the result ... 1000 times
MD5 always gives us a 16 byte hash
Final result: first 8 bytes is the "key" and the next is the "initialization vector"
(there is something about the first 8 bytes needing to be of odd paraity, therefore
the least significant bit needs to be changed to 1 if required. We don't do it,
maybe the python crypto library does it for us)
<end derived key generation>
Pad the input string with 1-8 bytes (note: not 0-7, so we always have padding)
so that the result is a multiple of 8 bytes. Padding byte value is same as number of
bytes being padded, eg, \x07 if 7 bytes need to be padded.
Use the key and iv to encrypt the input string, using DES with CBC mode.
Prepend the encrypted value with the salt (needed for decrypting since it is random)
Base64 encode it -> this is your result
Decrypt:
Base64 decode the input message
Extract the salt (first 8 bytes). The rest is the encoded text.
Use derived key generation as in Encrypt above to get the key and iv
Decrypt the encoded text using key and iv
Remove padding -> this is your result
(I only have implemented decrypt here since that's all I needed,
but encrypt should be straighforward as well)
"""
def get_derived_key(password, salt, count):
key = password + salt
for i in range(count):
m = hashlib.md5(key)
key = m.digest()
return (key[:8], key[8:])
def decrypt(msg, password):
msg_bytes = base64.b64decode(msg)
salt = msg_bytes[:8]
enc_text = msg_bytes[8:]
(dk, iv) = get_derived_key(password, salt, 1000)
crypter = DES.new(dk, DES.MODE_CBC, iv)
text = crypter.decrypt(enc_text)
# remove the padding at the end, if any
return re.sub(r'[\x01-\x08]','',text)
I've updated python script from user3392439, with encrypt support. Wish it helpful.
import base64
import hashlib
import re
import os
from Crypto.Cipher import DES
"""
Note about PBEWithMD5AndDES in java crypto library:
Encrypt:
Generate a salt (random): 8 bytes
<start derived key generation>
Append salt to the password
MD5 Hash it, and hash the result, hash the result ... 1000 times
MD5 always gives us a 16 byte hash
Final result: first 8 bytes is the "key" and the next is the "initialization vector"
(there is something about the first 8 bytes needing to be of odd paraity, therefore
the least significant bit needs to be changed to 1 if required. We don't do it,
maybe the python crypto library does it for us)
<end derived key generation>
Pad the input string with 1-8 bytes (note: not 0-7, so we always have padding)
so that the result is a multiple of 8 bytes. Padding byte value is same as number of
bytes being padded, eg, \x07 if 7 bytes need to be padded.
Use the key and iv to encrypt the input string, using DES with CBC mode.
Prepend the encrypted value with the salt (needed for decrypting since it is random)
Base64 encode it -> this is your result
Decrypt:
Base64 decode the input message
Extract the salt (first 8 bytes). The rest is the encoded text.
Use derived key generation as in Encrypt above to get the key and iv
Decrypt the encoded text using key and iv
Remove padding -> this is your result
(I only have implemented decrypt here since that's all I needed,
but encrypt should be straighforward as well)
"""
def get_derived_key(password, salt, count):
key = password + salt
for i in range(count):
m = hashlib.md5(key)
key = m.digest()
return (key[:8], key[8:])
def decrypt(msg, password):
msg_bytes = base64.b64decode(msg)
salt = msg_bytes[:8]
enc_text = msg_bytes[8:]
(dk, iv) = get_derived_key(password, salt, 1000)
crypter = DES.new(dk, DES.MODE_CBC, iv)
text = crypter.decrypt(enc_text)
# remove the padding at the end, if any
return re.sub(r'[\x01-\x08]','',text)
def encrypt(msg, password):
salt = os.urandom(8)
pad_num = 8 - (len(msg) % 8)
for i in range(pad_num):
msg += chr(pad_num)
(dk, iv) = get_derived_key(password, salt, 1000)
crypter = DES.new(dk, DES.MODE_CBC, iv)
enc_text = crypter.encrypt(msg)
return base64.b64encode(salt + enc_text)
def main():
msg = "hello, world"
passwd = "mypassword"
s = encrypt(msg, passwd)
print s
print decrypt(s, passwd)
if __name__ == "__main__":
main()
For #cooljohny
I do not really recall how this code works, but I'm 99% sure it does. I wrote it by carefully digging through the spec from the Java implementation of pbewithmd5anddes, and testing this python against that. I delivered exactly this code to a client, and it worked fine for them. I changed the constants before pasting it here, but that's all. You should be able to confirm that it produces the same encrypted output as the Java lib, and then replicate it in ruby. Good luck!
import base64
from Crypto.Cipher import DES
from passlib.utils.pbkdf2 import pbkdf1
password = 'xxxxxxx'
iterations = 22
salt_bytes = [19,15,78,45,34,90,12,11]
# convert saltBytes to a string
salt_string = ''.join([chr(a) for a in salt_bytes])
# a sample request
raw_data = '''{"something":"to","encrypt":"here"}'''
# from the standard...
padding_value = (8 - (raw_data.__len__() % 8))
padding_data = chr(padding_value) * padding_value
padded_data = raw_data + padding_data
# 22 iterations, 16 is the # of bytes in an md5 digest
pbkres = pbkdf1(password, salt_string, iterations, 16, 'md5')
# split the digest into two 8-byte halves
# this gives the DES secret key and initializing vector
des_key, iv = pbkres[0:8], pbkres[8:16]
# encrypt with DES
cipher = DES.new(des_key, DES.MODE_CBC, iv)
cmsg = cipher.encrypt(padded_data)
# and base64 encode
base64.b64encode(cmsg)