Have ready C# code to split integer into 2 bytes as you can see below, Needs to re-write same in Ruby-
int seat2 = 65000;
// Split into two bytes
byte seats = (byte)(seat2 & 0xFF); // lower byte
byte options = (byte)((seat2 >> 8) & 0xFF); // upper byte
Below is the output above
Output Seats => 232
options => 253
// Merge back into integer
seat2 = (options << 8) | seats;
Please suggest anyone has any solution to rewrite the above in Ruby
The code you wrote would work well in Ruby with very few modifications.
You could simply try:
seat2 = 65000
seat2 & 0xFF
# => 232
(seat2 >> 8) & 0xFF
# => 253
An alternative would be to use pack and unpack:
[65000].pack('S').unpack('CC')
# => [232, 253]
[232, 253].pack('CC').unpack('S')
# => [65000]
I believe the most idiomatic way for binary transformations in Ruby is Array#pack and String#unpack (like in Eric's answer).
Also you have an option to use Numeric#divmod with 256(2^8, byte size):
> upper, lower = 65000.divmod(256)
# => [253, 232]
> upper
# => 253
> lower
# => 232
In this case, to have correct bytes, your Integer should not exceed 65535 (2^16-1).
Another one:
lower, upper = 65000.digits(256)
Related
I am working with a micro controller which calculates the CRC32 checksum of data I upload to it's flash memory on the fly. This can in turn be used to verify that the upload was correct, by verifying the resulting checksum after all data is uploaded.
The only problem is that the Micro Controller reverses the bit order of the input bytes when it's run through the otherwise standard crc32 calculation. This in turn means I need to reverse every byte in the data on the programming host in order to calculate the CRC32 sum to verify. As the programming host is somewhat constrained, this is quite slow.
I figure that if it's possible to modify the CRC32 lookuptable so I can do the lookup without having to reverse the bit order, the verification algorithm would run many times faster. But I seem unable to figure out a way to do this.
To clarify the byte reversal, I need to change the input bytes following way:
01 02 03 04 -> 80 40 C0 20
It's a lot easier to see the reversal in binary representation of course:
00000001 00000010 00000011 00000100 ->
10000000 01000000 11000000 00100000
Edit
Here is the PoC Python code I use to verify the correctness of the CRC32 calculation, however this reverses each byte (a.e the slow way).
EDIT2
I've also included my failed attempt to generate a permutated lookup table, and using a standard LUT CRC32 algorithm.
The code spits out the correct reference CRC value first, and then the wrong LUT calculated CRC afterwards.
import binascii
CRC32_POLY = 0xEDB88320
def reverse_byte_bits(x):
'''
Reverses the bit order of the giveb byte 'x' and returns the result
'''
x = ((x<<4) & 0xF0)|((x>>4) & 0x0F)
x = ((x<<2) & 0xCC)|((x>>2) & 0x33)
x = ((x<<1) & 0xAA)|((x>>1) & 0x55)
return x
def reverse_bits(ba, blen):
'''
Reverses all bytes in the given array of bytes
'''
bar = bytearray()
for i in range(0, blen):
bar.append(reverse_byte_bits(ba[i]))
return bar
def crc32_reverse(ba):
# Reverse all bits in the
bar = reverse_bits(ba, len(ba))
# Calculate the CRC value
return binascii.crc32(bar)
def gen_crc_table_msb():
crctable = [0] * 256
for i in range(0, 256):
remainder = i
for bit in range(0, 8):
if remainder & 0x1:
remainder = (remainder >> 1) ^ CRC32_POLY
else:
remainder = (remainder >> 1)
# The correct index for the calculated value is the reverse of the index
ix = reverse_byte_bits(i)
crctable[ix] = remainder
return crctable
def crc32_revlut(ba, lut):
crc = 0xFFFFFFFF
for x in ba:
crc = lut[x ^ (crc & 0xFF)] ^ (crc >> 8)
return ~crc
# Reference test which gives the correct CRC
test = bytearray([1, 2, 3, 4, 5, 6, 7, 8])
crcrev = crc32_reverse(test)
print("0x%08X" % (crcrev & 0xFFFFFFFF))
# Test using permutated lookup table, but standard CRC32 LUT algorithm
lut = gen_crc_table_msb()
crctst = crc32_revlut(test, lut)
print("0x%08X" % (crctst & 0xFFFFFFFF))
Does anyone have any hints to how this could be done?
By reversing the logic of which way the crc "streams", the reverse in the main calculation can be avoided. So instead of crc >> 8 there would be crc << 8 and instead of XORing with the bottom byte of the crc for the LUT index we take the top. Like this:
def reverse_dword_bits(x):
'''
Reverses the bit order of the given dword 'x' and returns the result
'''
x = ((x<<16) & 0xFFFF0000)|((x>>16) & 0x0000FFFF)
x = ((x<<8) & 0xFF00FF00)|((x>>8) & 0x00FF00FF)
x = ((x<<4) & 0xF0F0F0F0)|((x>>4) & 0x0F0F0F0F)
x = ((x<<2) & 0xCCCCCCCC)|((x>>2) & 0x33333333)
x = ((x<<1) & 0xAAAAAAAA)|((x>>1) & 0x55555555)
return x
def gen_crc_table_msb():
crctable = [0] * 256
for i in range(0, 256):
remainder = i
for bit in range(0, 8):
if remainder & 0x1:
remainder = (remainder >> 1) ^ CRC32_POLY
else:
remainder = (remainder >> 1)
# The correct index for the calculated value is the reverse of the index
ix = reverse_byte_bits(i)
crctable[ix] = reverse_dword_bits(remainder)
return crctable
def crc32_revlut(ba, lut):
crc = 0xFFFFFFFF
for x in ba:
crc = lut[x ^ (crc >> 24)] ^ ((crc << 8) & 0xFFFFFFFF)
return reverse_dword_bits(~crc)
In the SEG-D seismic data format, some header parameters are formatted as three bytes, two’s complement, signed binary. All values are big-endian.
Using String#unpack, Ruby core can only convert 16-bit and 32-bit values, but not 24-bit.
How can I get the binary values converted into integers in the following two’s complement way:
"\x00\x00\x00" => 0x000000 (0)
"\x00\x00\x01" => 0x000001 (1)
"\x7F\xFF\xFF" => 0x7FFFFF (8388607)
"\xFF\xFF\xFF" => -0x000001 (-1)
"\x80\x00\x00" => -0x800000 (-8388608)
Convert the fist byte as signed 8-bit (two’s complement) and the second and third as unsigned 16-bit.
# String to be converted (-8388608)
bin = "\x80\x00\x00"
# Convert as signed 8-bit and unsigned 16-bit (big-endian)
values = bin.unpack("c S>")
# Add with the first byte weighted
converted = values[0] * 2**16 + values[1]
Alternate version of the last line using bitwise operations shift and OR (probably more efficient):
converted = values[0] << 16 | values[1]
I would like to get a clue on how I can get Ruby to work with byte arrays
The below code is C#:
int t = (GetTime() / 60) //t is time in seconds divided by 60s (1 min)
byte[] myArray = new byte[64];
myArray[0] = (byte)(t >> 24);
myArray[1] = (byte)(t >> 16);
Any idea how I can get this to work in Ruby?
One way would be to work with arrays of integers and use Array#pack to pack the result into a binary string. E.g.
[65, 66, 67].pack('C*')
Returns ABC
Another way would be to manipulate a string directly when the encoding is set to "ASCII-8BIT"
Ruby can do bitwise operations and you can use a normal array, so I don't see a problem.
I don't use C# at the moment so I can't check if the results are the same.
t = Time.now.to_i / 60 #t is time in seconds divided by 60s (1 min)
myArray = []
myArray[0] = t >> 24
myArray[1] = t >> 16
p myArray #=>[1, 360]
Given a Ruby Float value, e.g.,
f = 12.125
I'd like to wind up a 3-element array containing the floating-point number's sign (1 bit), exponent (11 bits), and fraction (52 bits). (Ruby's floats are the IEEE 754 double-precision 64-bit representation.)
What's the best way to do that? Bit-level manipulation doesn't seem to be Ruby's strong point.
Note that I want the bits, not the numerical values they correspond to. For instance, getting [0, -127, 1] for the floating-point value of 1.0 is not what I'm after -- I want the actual bits in string form or an equivalent representation, like ["0", "0ff", "000 0000 0000"].
The bit data can be exposed via Arrays pack as Float doesn't provide functions internally.
str = [12.125].pack('D').bytes.reverse.map{|n| "%08b" %n }.join
=> "0100000000101000010000000000000000000000000000000000000000000000"
[ str[0], str[1..11], str[12..63] ]
=> ["0", "10000000010", "1000010000000000000000000000000000000000000000000000"]
This is a bit 'around about the houses' to pull it out from a string representation. I'm sure there is a more efficient way to pull the data from the original bytes...
Edit The bit level manipulation tweaked my interest so I had a poke around. To use the operations in Ruby you need to have an Integer so the float requires some more unpacking to convert into a 64 bit int. The big endian/ieee754 documented representation is fairly trivial. The little endian representation I'm not so sure about. It's a little odd, as you are not on complete byte boundaries with an 11 bit exponent and 52 bit mantissa. It's becomes fiddly to pull the bits out and swap them about to get what resembles little endian, and not sure if it's right as I haven't seen any reference to the layout. So the 64 bit value is little endian, I'm not too sure how that applies to the components of the 64bit value until you store them somewhere else, like a 16bit int for the mantissa.
As an example for an 11 bit value from little > big, The kind of thing I was doing was to shift the most significant byte left 3 to the front, then OR with the least significant 3 bits.
v = 0x4F2
((v & 0xFF) << 3) | ( v >> 8 ))
Here it is anyway, hopefully its of some use.
class Float
Float::LITTLE_ENDIAN = [1.0].pack("E") == [1.0].pack("D")
# Returns a sign, exponent and mantissa as integers
def ieee745_binary64
# Build a big end int representation so we can use bit operations
tb = [self].pack('D').unpack('Q>').first
# Check what we are
if Float::LITTLE_ENDIAN
ieee745_binary64_little_endian tb
else
ieee745_binary64_big_endian tb
end
end
# Force a little end calc
def ieee745_binary64_little
ieee745_binary64_little_endian [self].pack('E').unpack('Q>').first
end
# Force a big end calc
def ieee745_binary64_big
ieee745_binary64_big_endian [self].pack('G').unpack('Q>').first
end
# Little
def ieee745_binary64_little_endian big_end_int
#puts "big #{big_end_int.to_s(2)}"
sign = ( big_end_int & 0x80 ) >> 7
exp_a = ( big_end_int & 0x7F ) << 1 # get the last 7 bits, make it more significant
exp_b = ( big_end_int & 0x8000 ) >> 15 # get the 9th bit, to fill the sign gap
exp_c = ( big_end_int & 0x7000 ) >> 4 # get the 10-12th bit to stick on the front
exponent = exp_a | exp_b | exp_c
mant_a = ( big_end_int & 0xFFFFFFFFFFFF0000 ) >> 12 # F000 was taken above
mant_b = ( big_end_int & 0x0000000000000F00 ) >> 8 # F00 was left over
mantissa = mant_a | mant_b
[ sign, exponent, mantissa ]
end
# Big
def ieee745_binary64_big_endian big_end_int
sign = ( big_end_int & 0x8000000000000000 ) >> 63
exponent = ( big_end_int & 0x7FF0000000000000 ) >> 52
mantissa = ( big_end_int & 0x000FFFFFFFFFFFFF ) >> 0
[ sign, exponent, mantissa ]
end
end
and testing...
def printer val, vals
printf "%-15s sign|%01b|\n", val, vals[0]
printf " hex e|%3x| m|%013x|\n", vals[1], vals[2]
printf " bin e|%011b| m|%052b|\n\n", vals[1], vals[2]
end
floats = [ 12.125, -12.125, 1.0/3, -1.0/3, 1.0, -1.0, 1.131313131313, -1.131313131313 ]
floats.each do |v|
printer v, v.ieee745_binary64
printer v, v.ieee745_binary64_big
end
TIL my brain is big endian! You'll note the ints being worked with are both big endian. I failed at bit shifting the other way.
Use frexp from the Math module. From the doc:
fraction, exponent = Math.frexp(1234) #=> [0.6025390625, 11]
fraction * 2**exponent #=> 1234.0
The sign bit is easy to find on its own.
I'm trying to send a very long binary integer over UDP (on the order of 200 bits). When I try to use Array's pack method, it complains the string I'm trying to convert is too large.
Am I going about this the wrong way?
ruby-1.8.7-p352 :003 > [0b1101001010101101111010100101010011010101010110010101010101010010010101001010101010101011101010101010101111010101010101010101].pack('i')
RangeError: bignum too big to convert into `unsigned long'
from (irb):3:in `pack'
from (irb):3
This number is supposed to represent a DNS query packet (this is for a homework assignment; we're not allowed to use any DNS libraries).
You need to break apart your number into smaller pieces. Probably best is to encode 32 bits at a time:
> num = 0b1101001010101101111010100101010011010101010110010101010101010010010101001010101010101011101010101010101111010101010101010101
=> 17502556204775004286774747314501014869
> low_1 = num & 0xFFFFFFFF
=> 2864534869
> low_2 = (num >> 32) & 0xFFFFFFFF
=> 625650362
> low_3 = (num >> 64) & 0xFFFFFFFF
=> 1297454421
> low_4 = (num >> 96) & 0xFFFFFFFF
=> 220913317
> (low_4 << 96) + (low_3 << 64) + (low_2 << 32) + low_1
=> 17502556204775004286774747314501014869
> msg = [low_4, low_3, low_2, low_1].pack("NNNN")
=> "\r*\336\245MU\225U%J\252\272\252\275UU"
> msg.unpack("NNNN").inject {|sum, elem| (sum << 32) + elem}
=> 17502556204775004286774747314501014869
I prefer 32 bits here because you pack these in Network Byte Order, which makes interopation with other platforms much easier. The pack() method doesn't provide a network byte order 64-bit integer. (Which isn't too surprising, since POSIX doesn't provide a 64-bit routine.)
Ruby 1.9.3 works normally.
irb(main):001:0> [0b1101001010101101111010100101010011010101010110010101010101010010010101001010101010101011101010101010101111010101010101010101].pack('i')
=> "UU\xBD\xAA"