I am trying to understand the proof of work algorithm. I computed a block header (which includes the nonce):
"02000000aaf8ab82362344f49083ee4edef795362cf135293564c4070000000000000000c009bb6222e9bc4cdb8f26b2e8a2f8d163509691a4038fa692abf9a474c9b21476800755c02e17181fe6c1c3"
I have to apply SHA256 to this twice. The correct answer is supposed to be:
"00000000000000001354e21fea9c1ec9ac337c8a6c0bda736ec1096663383429"
I tried pack, unpack, hex, etc., but I can't get this output. What is the correct Ruby code to convert the input to the output using SHA256?
header_hex = "02000000aaf8ab82362344f49083ee4edef795362cf135293564c4070000000000000000c009bb6222e9bc4cdb8f26b2e8a2f8d163509691a4038fa692abf9a474c9b21476800755c02e17181fe6c1c3"
# Decode header hex into binary string
header = [header_hex].pack("H*")
# Apply SHA256 twice
require "digest"
d1 = Digest::SHA256.digest(header)
d2 = Digest::SHA256.digest(d1)
# Convert to hex
result = d2.unpack("H*").join
# => "293438636609c16e73da0b6c8a7c33acc91e9cea1fe254130000000000000000"
Oops, for some reason the result is somewhat "backwards". Perhaps it is a byte-ordering issue? Let's try that again with the binary data reversed:
result = d2.reverse.unpack("H*").join
# => "00000000000000001354e21fea9c1ec9ac337c8a6c0bda736ec1096663383429"
Bingo!
Edit: Just to clarify, this was a trial-and-error solution. I don't have any special insight into the proof of work algorithm!
Related
My current project has me converting Ruby into Node.js. I've never worked with Ruby before this and so I'm still trying to learn all the syntactical differences. Currently, I'm a bit confused about this snippet:
myUri = Addressable::URI.parse(original_path)
idx = original_path.index(myUri.path)
hit.props[:path] = original_path[idx..original_path.length-1]
else
hit.is_invalid = true
So, I understand that we are parsing the original_path.
Then, we are getting the index of the myUri.path inside of the original_path, which I believe will return a number.
The next line is what's confusing me. I know I'm setting [:path] to something, but I don't understand what. I think that it's a modified version of original_path but I'm not understanding how its being modified.
original_path[idx..original_path.length-1] accesses string using a range.
See a range exaple
(1..4).to_a
# [1,2,3,4]
Range string/array access takes a part of the array/string that is between range boundaries. In this case - between idx (that is the beginning of myUri.path) and the end of the string.
See below:
a = 'abcd'
# => "abcd"
a[1..a.size-1]
# => "bcd"
Docs
I can't figure out what I am doing wrong here trying to decrypt a string of hex values with a given key using ruby's OpenSSL cipher AES-128-CTR.
I am using the gem hex_string to convert my hex to bytes
ctrkey = "36f18357be4dbd77f050515c73fcf9f2"
ciphertext3 = "69dda8455c7dd4254bf353b773304eec0ec7702330098ce7f7520d1cbbb20fc3\
88d1b0adb5054dbd7370849dbf0b88d393f252e764f1f5f7ad97ef79d59ce29f5f51eeca32eabedd9afa9329"
cipher2 = OpenSSL::Cipher.new('AES-128-CTR')
cipher2.decrypt
ctrkey = ctrkey.to_byte_string
cipher2.key = ctrkey
iv = cipher2.random_iv
cipher2.iv = iv
ciphertext3 = ciphertext3.to_byte_string
plain = cipher2.update(ciphertext3) + cipher2.final
puts "plaintext of Q3: #{plain}"
I know I am missing something small because I have similar code implementing AES-128-CBC. Do I need to have a counter that increments the IV for each block of 128 bytes in the ciphertext?
No, you're not missing something small, you are missing something huge.
Instead of using the same IV as used for encryption, you are generating a new one. For CTR, if the IV is random then each counter value is different, resulting in random looking output.
Often the IV (or nonce in the case of CTR) is prefixed to the ciphertext. For CTR that may be fewer bytes than 16 - although that is still the most probable size to try.
I am trying to generate a file in ruby that has a specific size. The content doesn't matter.
Here is what I got so far (and it works!):
File.open("done/#{NAME}.txt", 'w') do |f|
contents = "x" * (1024*1024)
SIZE.to_i.times { f.write(contents) }
end
The problem is: Once I zip or rar this file the created archive is only a few kb small. I guess thats because the random data in the file got compressed.
How do I create data that is more random as if it were just a normal file (for example a movie file)? To be specific: How to create a file with random data that keeps its size when archived?
You cannot guarantee an exact file size when compressing. However, as you suggest in the question, completely random data does not compress.
You can generate a random String using most random number generators. Even simple ones are capable of making hard-to-compress data, but you would have to write your own string-creation code. Luckily for you, Ruby comes with a built-in library that already has a convenient byte-generating method, and you can use it in a variation of your code:
require 'securerandom'
one_megabyte = 2 ** 20 # or 1024 * 1024, if you prefer
# Note use 'wb' mode to prevent problems with character encoding
File.open("done/#{NAME}.txt", 'wb') do |f|
SIZE.to_i.times { f.write( SecureRandom.random_bytes( one_megabyte ) ) }
end
This file is not going to compress much, if at all. Many compressors will detect that and just store the file as-is (making a .zip or .rar file slightly larger than the original).
For a given string size N and compression method c (e.g., from the rubyzip, libarchive or seven_zip_ruby gems), you want to find a string str such that:
str.size == c(str).size == N
I'm doubtful that you can be assured of finding such a string, but here's a way that should come close:
Step 0: Select a number m such that m > N.
Step 1: Generate a random string s with m characters.
Step 2: Compute str = c(str). If str.size <= N, increase m and repeat Step 1; else go to Step 3.
Step 3: Return str[0,N].
As a programming assignment in a Cryptography course I have the following problem:
Read a video file, divide it in 1KB blocks, grab the last block, get it's SHA256 sum, append that sum to the second to last block, get the SHA256 sum of the resulting block, and so on and so forth... The answer to the problem is the last SHA256 sum you get from this chain. The answer yielded by this algorithm applied to a certain video is this SHA256 sum: '5b96aece304a1422224f9a41b228416028f9ba26b0d1058f400200f06a589949'.
I understand the problem, but I cannot solve it using Ruby.
This is my Ruby code:
require 'digest/sha2'
def chunker
video, array = File.new('video.mp4', 'r'), []
(0..video.size/1024).each { |i| array[i] = video.read 1024 }
array
end
video_chunks, sha, digest = chunker, '', Digest::SHA2.new
video_chunks.reverse_each { |chunk| sha = (digest << chunk+sha).to_s }
puts sha
I'm basically dividing the video into 1024 byte chunks, then traversing it in reverse, getting the SHA256 sum of (currentBlock + lastSha) and saving it to a variable, which I output at the end of this reverse traversal.
This does not work.
The SHA256 sum of the first chunk (which doesn't have any past sha appended to it) is 'f2e208617302c6b089f52b6f27f78a7171b4424c1191989bbf86ed5ab0cbccee', I know this from a Java program which does the exact same problem. That sum is correct. But the second SHA256 sum, which is the SHA265 result of the appending of 'f2e2...' to the second to last block should be '34b6...' and it is outputting another thing. The problem occurs in the code "digest << chunk+sha". Somehow, when appending, something happens and the resulting sha is incorrect.
Any ideas? :(
The sha should not be generated via .to_s, you need the binary string version. In addition you are feeding more and more blocks into the same digest, whilst your exercise is specifically about a process for doing the same thing but under your own control (i.e. in your own code).
So instead of maintaining a digest object, and calling .to_s on it to fetch each sub-hash, you should calculate the hash fresh each time using the Digest::SHA2.digest( data ) class method
Try this instead:
video_chunks, sha = chunker, ''
video_chunks.reverse_each { |chunk| sha = Digest::SHA2.digest( chunk+sha ) }
# Convert to hex:
puts sha.unpack('H*').first
As a personal challenge I'm trying to implement the SIMON block cipher in Ruby. I'm running into some issues finding the best way to work with the data. The full code related to this question is located at: https://github.com/Rami114/Personal/blob/master/Simon/Simon.rb
SIMON requires both XOR, shift and circular shift operations, the last of which is forcing me to work with BigNums so I can perform the left circular shift with math rather than a more complex/slower double loop on byte arrays.
Is there a better way to convert a string to a BigNum and back again.
String -> BigNum (where N is 64 and pt is a string of plaintext)
pt = pt.chars.each_slice(N/8).map {|x| x.join.unpack('b*')[0].to_i(2)}.to_a
So I break the string into individual characters, slice into N-sized arrays (the word size in SIMON) and unpack each set into a BigNum. That appears to work fine and I can convert it back.
Now my SIMON code is currently broken, but that's more the math I think/hope and not the code. The conversion back is (where ct is an array of bignums representing the ciphertext):
ct.map { |x| [x.to_s(2).rjust(128,'0')].pack('b*') }.join
I seem to have to right-justify pad the string as bignums are of undefined width so I have no leading 0s. Unfortunately the pack requires the defined with to have sensible output.
Is this a valid method of conversion? Is there a better way? I'm not sure on either count and hoping someone here can help out.
E: For #torimus, the circular shift implementation I'm using (From link above)
def self.lcs (bytes, block_size, shift)
((bytes << shift) | (bytes >> (block_size - shift))) & ((1<< block_size)-1)
end
If you would be equally happy with unpack('B*') with msb first binary numbers (which you could well be if all your processing is circular), then you could also use .unpack('Q>') instead of .unpack('B*')[0].to_i(2) for generating pt:
pt = "qwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNM1234567890!#"
# Your version (with 'B' == msb first) for comparison:
pt_nums = pt.chars.each_slice(N/8).map {|x| x.join.unpack('B*')[0].to_i(2)}.to_a
=> [8176115190769218921, 8030025283835160424, 7668342063789995618, 7957105551900562521,
6145530372635706438, 5136437062280042563, 6215616529169527604, 3834312847369707840]
# unpack to 64-bit unsigned integers directly
pt_nums = pt.unpack('Q>8')
=> [8176115190769218921, 8030025283835160424, 7668342063789995618, 7957105551900562521,
6145530372635706438, 5136437062280042563, 6215616529169527604, 3834312847369707840]
There are no native 128-bit pack/unpacks to return in the other direction, but you can use Fixnum to solve this too:
split128 = 1 << 64
ct = pt # Just to show round-trip
ct.map { |x| [ x / split128, x % split128 ].pack('Q>2') }.join
=> "\x00\x00\x00\x00\x00\x00\x00\x00qwertyui . . . " # truncated
This avoids a lot of the temporary stages on your code, but at the expense of using a different byte coding - I don't know enough about SIMON to comment whether this is adaptable to your needs.