Related
ORIGINAL QUESTION:
I am currently trying to write a library in rust - to be compiled to WASM - for converting a bip39 mnemonic passphrase into an Arweave JWK. I am currently using tiny-bip39 and RSA.
When generating a private key using RSA as per the example given on RSA I want to seed the rng based on the mnemonic passphrase I have passed into the function. I tried achieving this by simply getting the seed from the mnemonic object generated by tiny-bip39, however this seems to generate a &[u8] with a length of 64. However, Seed is defined as [u8; 32], and without having to write my own rng, I cannot figure out how to use a len 64 seed.
#[wasm_bindgen]
pub fn get_key_from_mnemonic(phrase: &str) {
let mnemonic = Mnemonic::from_phrase(phrase, Language::English).unwrap();
assert_eq!(phrase, mnemonic.phrase());
let seed = Seed::new(&mnemonic, "");
let seed_bytes = seed.as_bytes();
let mut rng = ChaCha12Rng::from_seed(seed_bytes);
[...]
}
Is there a cryptographically secure rng that allows for len 64 seed?
I tried simply trying into, but that did not seem to work, which makes sense.
let seed_bytes: <ChaCha12Rng as SeedableRng>::Seed = seed.as_bytes().try_into().unwrap();
EDIT:
I came up with a solution that seem to work in every way except the random number generation.
let mnemonic = Mnemonic::from_phrase(phrase, Language::English).unwrap();
assert_eq!(phrase, mnemonic.phrase());
let seed = Seed::new(&mnemonic, "");
let seed_bytes = seed.as_bytes();
let mut seed_buf: [u8; 32] = Default::default();
let mut hmac_drgb = HmacDRBG::<Sha256>::new(&seed_bytes, &[], &[]);
hmac_drgb.generate_to_slice(&mut seed_buf, None);
let mut chacha = ChaCha20Rng::from_seed(seed_buf);
let modulus_length = 4098;
let rsa_private_key = RsaPrivateKey::new(&mut chacha, modulus_length).unwrap();
let der = rsa_private_key.to_pkcs1_der().unwrap();
let jwk = JWK {
modulus: der.private_key().modulus.as_bytes().to_vec(),
public_exponent: der.private_key().public_exponent.as_bytes().to_vec(),
private_exponent: der.private_key().private_exponent.as_bytes().to_vec(),
prime1: der.private_key().prime1.as_bytes().to_vec(),
prime2: der.private_key().prime2.as_bytes().to_vec(),
exponent1: der.private_key().exponent1.as_bytes().to_vec(),
exponent2: der.private_key().exponent2.as_bytes().to_vec(),
coefficient: der.private_key().coefficient.as_bytes().to_vec(),
};
As I am trying to rewrite some of the functionality provided by arweave-mnemonic-keys, I have tried to go through all of the dependencies, figuring out which rust modules I need, and think I have managed to figure out everything except how to generate the random numbers for the RSA algorithm.
I have tried looking through the node-forge/lib/rsa.js file, and found this snippet:
function generateRandom(bits, rng) {
var num = new BigInteger(bits, rng);
// force MSB set
var bits1 = bits - 1;
if(!num.testBit(bits1)) {
num.bitwiseTo(BigInteger.ONE.shiftLeft(bits1), op_or, num);
}
// align number on 30k+1 boundary
num.dAddOffset(31 - num.mod(THIRTY).byteValue(), 0);
return num;
}
However, I am not sure how to reproduce this in rust. So far I have tried to use ChaCha8Rng, ChaCha12Rng, ChaCha20Rng, and Pcg64, none of which produces the wanted result.
It depends on the CSPRNG. If you were seeding an HMAC DRBG using HMAC-SHA-512, then this would be a perfectly normal amount of input. However, in your case, the CSPRNG is ChaCha, which is configured to have a 256-bit key.
If this mnemonic has been generated from a CSPRNG and has sufficient entropy, then all you need is a simple, straightforward key derivation function like HKDF. You can use HKDF with SHA-256 or SHA-512, with the seed as the input keying material, no salt, and an output keying material which is 32 bytes in size. Then, you can use that to seed your CSPRNG.
You will also need an info string, which is usually some text string for the purpose. I like to use a version number to make things future proof, so you could use something like "v1 PRNG seed".
My recommendation here is that since you have a 64-byte input seed, that HKDF using SHA-512 is best, since it avoids losing entropy if you end up needing to seed other data. Also, while ChaCha12Rng is the default, ChaCha20Rng is more conservative and would probably be appropriate for generating a long term key.
I'm having trouble encrypting a value from a third party vendor I am using.
Their instructions are as follows:
1) Convert the encryption password to a byte array.
2) Convert the value to be encrypted to a byte array.
3) The entire length of the array is inserted as the first four bytes onto the front
of the first block of the resultant byte array before encryption.
4) Encrypt the value using AES with:
1. 256-bit key size,
2. 256-bit block size,
3. Encryption Mode ECB, and
4. an EMPTY initialization vector.
5) After encryption, you should now have a byte array that holds the encrypted value.
6) Convert each byte to a HEX format and string all the HEX values together.
7) The final result is a string of HEX values. This is the final encrypted value to be passed.
The length of the final value will always be an even number.
EXAMPLE:
Given the following input values:
plainText: 2017/02/07 22:46
secretKey: ABCD1234FGHI5678
The following string will be produced:
D6281D5BE6CD6E79BB41C039F4DD020FBEC9D290AD631B2598A6DFF55C68AD04
What I've tried so far...
plain_text = "2017/02/07 22:46"
secret_key = "ABCD1234FGHI5678"
plain_text_byte_array = plain_text.bytes
plain_text_byte_array.unshift(0).unshift(0).unshift(0).unshift(16) # I found a Java example in their documentation and this is what they do. They prepend their byte array with 16, 0, 0, 0
secret_byte_array = secret_key.bytes
secret_byte_array = secret_byte_array.concat([0, 0, 0,...]) # also from their java example, they append the secret_byte array with 16 0's in order to get its length to 32
cipher = OpenSSL::Cipher::AES256.new(:ECB)
cipher.key = secret_byte_array.pack("C*")
encrypted = cipher.update(plain_text_byte_array.pack("C*")) + cipher.final
p encrypted.unpack("H*").first.to_s.upcase
# Result is:
# "84A0E5DCA7D704C41332F86E707DDAC244A1A87C38A906145DE4060D2BC5C8F4"
As you can see my result is off from the actual result which should be
"D6281D5BE6CD6E79BB41C039F4DD020FBEC9D290AD631B2598A6DFF55C68AD04"
Does anyone know if I am missing something or doing something strange. Their instructions were difficult for me to parse through so maybe I'm missing something. Thank you for any help anyone can provide! (I've tried a ton of different variations on what you see above). I just need some guidance or at least someone to tell me I'm not crazy for not understanding their instructions.
I managed to reproduce their result - the process they've used is extremely convoluted and as far from elegant as it could possibly be. I've attached a far more descriptive explanation of the steps required to achieve their result, and the C# source code I used to do so.
Convert the password to a byte array. The byte array must be 32 bytes in length, and, if the password is not long enough, should be right-padded with 0 bytes. Thus their password, hex-encoded, becomes 4142434431323334464748493536373800000000000000000000000000000000.
Convert the value to be encrypted to a byte array. This one is simple enough, just encode with UTF-8.
The entire length of the array is inserted as the first four bytes onto the front of the first block of the resultant byte array before encryption. This is stupid and serves no purpose, but take the length of the byte array from step 2 as an unsigned 32-bit integer and convert to a little endian byte array. Prefix this to the array from step 2.
Encrypt the value using AES. Uhm. No don't do that. Encrypt the value with Rijndael, using a 256-bit block size, 256-bit key size, ECB mode and zero's for padding.
The rest is easy, just convert the result of encryption to hex.
The code I used to achieve this result is below, in C#. I don't know Ruby all that well sorry.
// 1. Convert the encryption password to a byte array.
byte[] passwordBytesOriginal = Encoding.UTF8.GetBytes("ABCD1234FGHI5678");
byte[] passwordBytes = new byte[32];
Array.Copy(passwordBytesOriginal, 0, passwordBytes, 0, passwordBytesOriginal.Length);
// 2. Convert the value to be encrypted to a byte array.
byte[] valueBytes = Encoding.UTF8.GetBytes("2017/02/07 22:46");
// 3. The entire length of the array is inserted as the first four bytes onto the front
// of the first block of the resultant byte array before encryption.
byte[] valueLengthAsBytes = BitConverter.GetBytes((uint)valueBytes.Length);
byte[] finalPlaintext = new byte[valueBytes.Length + valueLengthAsBytes.Length];
Array.Copy(valueLengthAsBytes, 0, finalPlaintext, 0, valueLengthAsBytes.Length);
Array.Copy(valueBytes, 0, finalPlaintext, valueLengthAsBytes.Length, valueBytes.Length);
// 4. Encrypt the value using AES...
byte[] ciphertext;
using (RijndaelManaged rijn = new RijndaelManaged())
{
rijn.BlockSize = 256;
rijn.KeySize = 256;
rijn.Key = passwordBytes;
rijn.Mode = CipherMode.ECB;
rijn.Padding = PaddingMode.Zeros;
var encryptor = rijn.CreateEncryptor();
ciphertext = encryptor.TransformFinalBlock(finalPlaintext, 0, finalPlaintext.Length);
}
// 5., 6., 7...
string result = BitConverter.ToString(ciphertext).Replace("-", "").ToUpper();
Console.WriteLine(result); // D6281D5BE6CD6E79BB41C039F4DD020FBEC9D290AD631B2598A6DFF55C68AD04
Based on Luke's excellent answer here is the Ruby version. I had to use the ruby-mcrypt gem and install the mcrypt library locally using brew install libmcrypt.
It is as Luke's answer points out that the secret key should be right padded with 0's. Here is my code:
plain_text = "2017/02/07 22:46"
secret_text = "ABCD1234FGHI5678"
answer = "D6281D5BE6CD6E79BB41C039F4DD020FBEC9D290AD631B2598A6DFF55C68AD04"
def format_byte_arrays(plain, secret)
zero_byte_array = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
length_array = [16, 0, 0, 0]
plain_bytes = length_array.concat(plain.bytes)
secret_bytes = secret.bytes.concat(zero_byte_array)
[plain_bytes, secret_bytes]
end
plain_bytes, secret_bytes = format_byte_arrays(plain_text, secret_text)
final_plain, final_secret = [plain_bytes.pack("C*"), secret_bytes.pack("C*")]
cipher = Mcrypt.new("rijndael-256", :ecb, final_secret, nil, :zeros)
encrypted = cipher.encrypt(final_plain)
result = encrypted.unpack("H*").first.to_s.upcase
The result will be the correct answer.
I have an app in Rails with following methods to encrypt and decrypt a text and communicate with java clients.
def encrypt(string, key)
cipher = OpenSSL::Cipher::AES.new(128, :CBC)
cipher.encrypt
cipher.padding = 1
cipher.key = hex_to_bin(Digest::SHA1.hexdigest(key)[0..32])
cipher_text = cipher.update(string)
cipher_text << cipher.final
return bin_to_hex(cipher_text).upcase
end
def decrypt(encrypted, key)
encrypted = hex_to_bin(encrypted.downcase)
cipher = OpenSSL::Cipher::AES.new(128, :CBC)
cipher.decrypt
cipher.padding = 1
cipher.key = hex_to_bin(Digest::SHA1.hexdigest(key)[0..32])
d = cipher.update(encrypted)
d << cipher.final
rescue Exception => exc
end
def hex_to_bin(str)
[str].pack "H*"
end
def bin_to_hex(str)
str.unpack('C*').map{ |b| "%02X" % b }.join('')
end
I need do the same in Elixir for phoenix framework. Since I'm new to Elixir I couldn't find a way for that.
I Found that Elixir uses Erlang's :crypto Module for that. In documentations there was no method for AES CBC encryption.
The block_encrypt/4 function from the Erlang crypto module is the function you want. Unlike the Ruby OpenSSL bindings, the Erlang code doesn’t handle padding, so you will need to do that yourself before encrypting (and remove it after decrypting).
NOTE: As of Erlang v23, the block_encrypt/4 and block_decrypt/4 functions (and their /3 sisters) are deprecated and will be removed from the Erlang crypto module in Erlang v24. The new API functions that have replaced them are crypto_one_time/4 and crypto_one_time/5 and these functions should be used for all new Erlang/Elixir programs. The new API functions support IVs and other improvements over the old functions.
However, unless this is just a toy app for learning purposes, I would recommend not doing this kind of crypto stuff yourself if you can avoid it. Rather you should find a higher level API that takes care of the various details where you can go wrong. I have listed some potential issues with your code as it is below, as well as a suggestion of what to do instead.
The padding that OpenSSL uses (sometimes called PKCS7 padding) is fairly simple. First you need to work out how many bytes you need to add to your data to make the length into a multiple of the block size (16 for AES). Then you simply add that many bytes of that value to the end. For example if your data was 14 bytes long then you would need to add two bytes, and each of those bytes would have the value 0x02 (2 bytes each with value 2). Note that you always add padding, so if your data is already a multiple of 16 byte then you add another 16 bytes (all with value 0x10).
To strip the padding you simply look at the value of the last byte and remove that many bytes from the end (you should probably check that the padding is correct too, i.e. all the bytes have the expected value).
Here is a simple implementation in Elixir (there may be a better / clearer / more idiomatic way to do this):
# These will need to be in a module of course
def pad(data, block_size) do
to_add = block_size - rem(byte_size(data), block_size)
data <> to_string(:string.chars(to_add, to_add))
end
def unpad(data) do
to_remove = :binary.last(data)
:binary.part(data, 0, byte_size(data) - to_remove)
end
You can now use these along with the :crypto.block_encrypt function to get AES CBC encryption like your Ruby code:
# BAD, don't do this!
# This is just to reproduce your code, where you are not using
# an initialisation vector.
#zero_iv to_string(:string.chars(0, 16))
#aes_block_size 16
def encrypt(data, key) do
:crypto.block_encrypt(:aes_cbc128, key, #zero_iv, pad(data, #aes_block_size))
end
def decrypt(data, key) do
padded = :crypto.block_decrypt(:aes_cbc128, key, #zero_iv, data)
unpad(padded)
end
Some issues
Here are some potential problems with your code. This is not an exhaustive list, just some things I noticed (I am not an expert in crypto).
No authentication. Unless you’re checking the authentication in another method before the code you show, then you don’t have any authentication of the messages. This is very bad. You are exposing yourself to potential padding oracle attacks (where an attacker could decrypt the messages) and things like bit-flipping attacks, where an attacker can send specially modified messages that your code might not recognise as bad, and cause some undesired action to take place.
You should be using something like HMAC. But even if you decide to use a HMAC, there are still several questions you need to work out. Where does the HMAC key come from? Can we use the same key for encryption and authentication? Do we calculate the HMAC over the plaintext or the ciphertext? Should it cover the IV as well?
No Initialisation Vector. CBC mode should make use of an initialisation vector, or IV. In the Ruby OpenSSL bindings if you don’t specify one it just uses zero bytes (which is why we needed to create the #zero_iv in the code above. Each message should have its own IV. This can just be a random series of bytes, and doesn’t need to be kept secret (it can just be sent prepended to the ciphertext).
Weak key generation. I could be wrong with this one, but since you are calculating the SHA1 hash of the provided key argument to use as the encryption/decryption key it suggest that this argument is actually a password. If this is the case then you should be using a better key derivation function (and if not then what’s the purpose of the hashing?). If you are using an easy for a human to remember password (or a single hash of one) you could be vulnerable to brute force attacks where an attacker tries lots of dictionary words as the key.
You should be using a proper key derivation function, such as PBKDF2. Even then you will still have complications since you might need two keys (encryption and authentication), so you need to work out how to generate them both.
What to use instead
If possible you should look for a higher level library that takes into account these factors and provides a simpler API. I would recommend Libsodium, which has bindings for many languages including Ruby, Elixir, Erlang, and Java/Android.
I'd recommend not using CBC mode directly but use GCM mode as this will provide authentication as well.
In Elixir (for a 256bit AES key)
# Gen once (see also https://hexdocs.pm/plug/Plug.Crypto.KeyGenerator.html#content)
k = :crypto.strong_rand_bytes(32)
# Gen every time you encrypt a message
iv = :crypto.strong_rand_bytes(32)
{ct, tag} = :crypto.block_encrypt(:aes_gcm, k, iv, {"AES128GCM", msg})
payload = Base.encode16(iv <> tag <> ct)
To decrypt:
<<iv::binary-32, tag::binary-16, ct::binary>> = Base.decode16!(payload)
:crypto.block_decrypt(:aes_gcm, k, iv, {"AES128GCM", ct, tag})
Here is what I use for ECB, CBC should be the same with the added need to pass the previous block cipher in the accumulator. Don't forget that you also need to write a function to pad the term to 16 byte blocks(the ruby seems to do that automatically).
Key = "12345678"
AES_ECB_Encrypt =
fun Crypt(<<Block:16/binary, Rest/binary>>, Acc) ->
NewAcc = erlang:iolist_to_binary( [Acc, crypto:block_encrypt(aes_ecb, Key, Block)] ),
Crypt(Rest, NewAcc);
Crypt(_, Acc) ->
Acc
end,
AES_ECB_Encrypt(<<"hello00000000000">>, <<>>)
JOSE.JWA component from JOSE package has block_decrypt/4 and block_encrypt/4 functions.
iex> JOSE.JWA.crypto_supports()
[ciphers: [aes_cbc: 128, aes_cbc: 192, aes_cbc: 256, aes_ecb: 128, aes_ecb: 192,
aes_ecb: 256, aes_gcm: 128, aes_gcm: 192, aes_gcm: 256,
chacha20_poly1305: 256],
hashs: [:md5, :poly1305, :sha, :sha256, :sha384, :sha512, :shake256],
public_keys: [:ec_gf2m, :ecdh, :ecdsa, :ed25519, :ed25519ph, :ed448, :ed448ph,
:rsa, :x25519, :x448], rsa_crypt: [:rsa1_5, :rsa_oaep, :rsa_oaep_256],
rsa_sign: [:rsa_pkcs1_padding, :rsa_pkcs1_pss_padding]]
Thanks #matt, I wrote my AES_ECB in Elixir.
Hope that it helps you, CBC should be the same.
def encrypt(data, key) do
:crypto.block_encrypt(:aes_ecb, key, pad(data, #aes_block_size))
end
# PKCS5Padding
defp pad(data, block_size) do
to_add = block_size - rem(byte_size(data), block_size)
data <> :binary.copy(<<to_add>>, to_add)
end
def decrypt(data, key) do
padded = :crypto.block_decrypt(:aes_ecb, key, data)
unpad(padded)
end
defp unpad(data) do
to_remove = :binary.last(data)
:binary.part(data, 0, byte_size(data) - to_remove)
end
I was working interchangeably with Node's crypto library and Ruby's OpenSSL library.
The challenge I was coming across was that I could encrypt usingaes256 in both libraries.
However, in node using the crypto.createDecipher('aes256', key) I could have a key that was less than 32 bits long, but ruby would throw an error saying the key is not long enough when using:
cipher = OpenSSL::Cipher.new 'aes256'
cipher.encrypt
key = 'geeses'
I also don't have to set an initialization vector for node, but ruby seems to set one under the covers. I'm pretty new to this crypto stuff, what's going on here?
While #mscdex answers is perfectly, I want to add how to get a cipher with a specific key when your algorithm does not require Initialization Vector using the crypto.createCipheriv or crypto.createDecipheriv
Taking the case of AES-256-ECB where chaining is not done and hence IV is not used.
You can pass empty Buffer as IV.
var data = "plaintext";
const key = crypto.randomBytes(32);
var iv = new Buffer('');
var cipher = crypto.createCipheriv('AES-256-ECB',key,iv);
var encrypted = cipher.update(data,'utf8','base64');
encrypted += cipheriv.final('base64');
console.log('encrypted AES-256-ECB',encrypted);
And decrypt fairly simply using the same pattern:
var decipheriv = crypto.createDecipheriv('AES-256-ECB',key,iv);
var decryptediv = decipheriv.update(encrypted,'base64','utf8');
decryptediv += decipheriv.final('utf8');
console.log('decrypted base64 aes-256 ',decryptediv);
When you use crypto.createDecipher(), the value you pass as the second argument is a password from which a key and IV will be derived (using one iteration of MD5 hashing). This is accomplished by using EVP_BytesToKey() to create those two values. OpenSSL knows the correct lengths both values need to be because the cipher is also passed to EVP_BytesToKey().
So most likely the Ruby function is more analogous to node's crypto.createDecipheriv() which accepts both a key and an IV (which need to be the right lengths for the cipher).
As a programming assignment in a Cryptography course I have the following problem:
Read a video file, divide it in 1KB blocks, grab the last block, get it's SHA256 sum, append that sum to the second to last block, get the SHA256 sum of the resulting block, and so on and so forth... The answer to the problem is the last SHA256 sum you get from this chain. The answer yielded by this algorithm applied to a certain video is this SHA256 sum: '5b96aece304a1422224f9a41b228416028f9ba26b0d1058f400200f06a589949'.
I understand the problem, but I cannot solve it using Ruby.
This is my Ruby code:
require 'digest/sha2'
def chunker
video, array = File.new('video.mp4', 'r'), []
(0..video.size/1024).each { |i| array[i] = video.read 1024 }
array
end
video_chunks, sha, digest = chunker, '', Digest::SHA2.new
video_chunks.reverse_each { |chunk| sha = (digest << chunk+sha).to_s }
puts sha
I'm basically dividing the video into 1024 byte chunks, then traversing it in reverse, getting the SHA256 sum of (currentBlock + lastSha) and saving it to a variable, which I output at the end of this reverse traversal.
This does not work.
The SHA256 sum of the first chunk (which doesn't have any past sha appended to it) is 'f2e208617302c6b089f52b6f27f78a7171b4424c1191989bbf86ed5ab0cbccee', I know this from a Java program which does the exact same problem. That sum is correct. But the second SHA256 sum, which is the SHA265 result of the appending of 'f2e2...' to the second to last block should be '34b6...' and it is outputting another thing. The problem occurs in the code "digest << chunk+sha". Somehow, when appending, something happens and the resulting sha is incorrect.
Any ideas? :(
The sha should not be generated via .to_s, you need the binary string version. In addition you are feeding more and more blocks into the same digest, whilst your exercise is specifically about a process for doing the same thing but under your own control (i.e. in your own code).
So instead of maintaining a digest object, and calling .to_s on it to fetch each sub-hash, you should calculate the hash fresh each time using the Digest::SHA2.digest( data ) class method
Try this instead:
video_chunks, sha = chunker, ''
video_chunks.reverse_each { |chunk| sha = Digest::SHA2.digest( chunk+sha ) }
# Convert to hex:
puts sha.unpack('H*').first