Ruby - Unpack array with mixed types - ruby

I am trying to use unpack to decode a binary file. The binary file has the following structure:
ABCDEF\tFFFABCDEF\tFFFF....
where
ABCDEF -> String of fixed length
\t -> tab character
FFF -> 3 Floats
.... -> repeat thousands of times
I know how to do it when types are all the same or with only numbers and fixed length arrays, but I am struggling in this situation. For example, if I had a list of floats I would do
s.unpack('F*')
Or if I had integers and floats like
[1, 3.4, 5.2, 4, 2.3, 7.8]
I would do
s.unpack('CF2CF2')
But in this case I am a bit lost. I was hoping to use a format string such `(CF2)*' with brackets, but it does not work.
I need to use Ruby 2.0.0-p247 if that matters
Example
ary = ["ABCDEF\t", 3.4, 5.6, 9.1, "FEDCBA\t", 2.5, 8.9, 3.1]
s = ary.pack('P7fffP7fff')
then
s.scan(/.{19}/)
["\xA8lf\xF9\xD4\x7F\x00\x00\x9A\x99Y#33\xB3#\x9A\x99\x11", "A\x80lf\xF9\xD4\x7F\x00\x00\x00\x00 #ff\x0EAff"]
Finally
s.scan(/.{19}/).map{ |item| item.unpack('P7fff') }
Error: #<ArgumentError: no associated pointer>
<main>:in `unpack'
<main>:in `block in <main>'
<main>:in `map'
<main>:in `<main>'

You could read the file in small chunks of 19 bytes and use 'A7fff' to pack and unpack. Do not use pointers to structure ('p' and 'P'), as they need more than 19 bytes to encode your information.
You could also use 'A6xfff' to ignore the 7th byte and get a string with 6 chars.
Here's an example, which is similar to the documentation of IO.read:
data = [["ABCDEF\t", 3.4, 5.6, 9.1],
["FEDCBA\t", 2.5, 8.9, 3.1]]
binary_file = 'data.bin'
chunk_size = 19
pattern = 'A7fff'
File.open(binary_file, 'wb') do |o|
data.each do |row|
o.write row.pack(pattern)
end
end
raise "Something went wrong. Please check data, pattern and chunk_size." unless File.size(binary_file) == data.length * chunk_size
File.open(binary_file, 'rb') do |f|
while record = f.read(chunk_size)
puts '%s %g %g %g' % record.unpack(pattern)
end
end
# =>
# ABCDEF 3.4 5.6 9.1
# FEDCBA 2.5 8.9 3.1
You could use a multiple of 19 to speed up the process if your file is large.

When dealing with mixed formats that repeat, and are of a known fixed size, it is often easier to split the string first,
Quick example would be:
binary.scan(/.{LENGTH_OF_DATA}/).map { |item| item.unpack(FORMAT) }
Considering your above example, take the length of the string including the tab character (in bytes), plus the size of a 3 floats. If your strings are literally 'ABCDEF\t', you would use a size of 19 (7 for the string, 12 for the 3 floats).
Your final product would look like this:
str.scan(/.{19}/).map { |item| item.unpack('P7fff') }
Per example:
irb(main):001:0> ary = ["ABCDEF\t", 3.4, 5.6, 9.1, "FEDCBA\t", 2.5, 8.9, 3.1]
=> ["ABCDEF\t", 3.4, 5.6, 9.1, "FEDCBA\t", 2.5, 8.9, 3.1]
irb(main):002:0> s = ary.pack('pfffpfff')
=> "\xE8Pd\xE4eU\x00\x00\x9A\x99Y#33\xB3#\x9A\x99\x11A\x98Pd\xE4eU\x00\x00\x00\x00 #ff\x0EAffF#"
irb(main):003:0> s.unpack('pfffpfff')
=> ["ABCDEF\t", 3.4000000953674316, 5.599999904632568, 9.100000381469727, "FEDCBA\t", 2.5, 8.899999618530273, 3.0999999046325684]
The minor differences in precision is unavoidable, but do not worry about it, as it comes from the difference of a 32-bit float and 64-bit double (what Ruby used internally), and the precision difference will be less than is significant for a 32-bit float.

Related

Increae a version's last part

What is the best way to increase the last part of a version by 1 (without the use of libraries).
Examples:
Gem::Version.new("2.0.31") to Gem::Version.new("2.0.32")
Gem::Version.new("2.0.3.0") to Gem::Version.new("2.0.3.1")
If the output is a string, that's ok too
Use Gem::Version#segments:
Gem::Version.new("2.0.31").
segments.
dup.
tap { |segments| segments.push(segments.pop.succ) }.
join('.')
#⇒ "2.0.32"
Here's a helper method I've used as part of rake tasks before, for version bumping.
Feel free to extend it as per your needs - e.g. if you want to add a forth number; or support versions named alpha-x or pre-x; or support an option for 'bump the last digit' (regardless of how many digits there are).
However, I'd be inclined not to support a 'bump the last digit' option, as this is quite ambiguous.
def bump(version, type)
major, minor, patch = version.split('.').map(&:to_i)
bumped = case type
when :major
[major + 1, 0, 0]
when :minor
[major, minor + 1, 0]
when :patch
[major, minor, patch + 1]
else
raise "Don't know how to bump for #{type}"
end
bumped.join('.')
end
bump("2.1.31", :patch) #=> "2.1.32"
bump("2.1.31", :minor) #=> "2.2.0"
bump("2.1.31", :major) #=> "3.0.0"

How to convert negative integers to binary in Ruby

Question 1: I cannot find a way to convert negative integers to binary in the following way. I am supposed to convert it like this.
-3 => "11111111111111111111111111111101"
I tried below:
sprintf('%b', -3) => "..101" # .. appears and does not show 111111 bit.
-3.to_s(2) => "-11" # This just adds - to the binary of the positive integer 3.
Question 2: Interestingly, if I use online converter, it tells me that binary of -3 is "00101101 00110011".
What is the difference between "11111111111111111111111111111101" and "00101101 00110011"?
Packing then unpacking will convert -3 to 4294967293 (232 - 3):
[-3].pack('L').unpack('L')
=> [4294967293]
sprintf('%b', [-3].pack('L').unpack('L')[0])
# => "11111111111111111111111111111101"
sprintf('%b', [3].pack('L').unpack('L')[0])
# => "11"
Try:
> 32.downto(0).map { |n| -3[n] }.join
#=> "111111111111111111111111111111101
Note: This applies to negative number's only.

Ruby convert string array to string

I have a ruby string array value and i want to get it as string value. I am using ruby with chef recipe. Running in windows platform. Code-
version_string = Mixlib::ShellOut.new('some.exe -version').run_command
Log.info(version.stdout.to_s)
extract_var = version_string.stdout.to_s.lines.grep(/ver/)
Log.info('version:'+ extract_var.to_s)
output is coming-
version 530
[2016-06-08T07:03:49+00:00] INFO: version ["version 530\r\n"]
I want to extract 530 string only.
long time no see since Rot :)
You can use some Chef helper methods and regular expressions to make this a little easier.
output = shell_out!('saphostexec.exe -version', cwd: 'C:\\Program Files\\hostctrl\\exe').stdout
if output =~ /kernel release\s+(\d+)/
kernel_version = $1
else
raise "unable to parse kernel version"
end
Chef::Log.info(kernel_version)
As you want val = 720 and not val = "720" you can write
val = strvar.first.to_i
#=> 720
You can return the first series of digits found as an integer from the current_kernel string with String#[regexp] :
current_kernel[/\d+/].to_i
#=> 720

How to parse ruby BigDecimal inspect?

In the following code:
x = BigDecimal(10)
s = x.inspect # "#<BigDecimal:6fe4790,'0.1E2',9(36)>"
Is there a way to parse s and get the original value ? The reason is that I have some text files with BigDecimal written in them using inspect, and I need to parse these values.
You .to_s to get the value in string. .inspect will print the object
x = BigDecimal(10)
x.to_s
# => "0.1E2"
The documentation for BigDecimal#inspect is incomplete. Consider the following:
require 'bigdecimal`
BigDecimal.new("1.2345").inspect
#=> "#<BigDecimal:7fb06a110298,'0.12345E1',18(18)>"
...
BigDecimal.new("1.234567890").inspect
#=> "#<BigDecimal:7fb06a16ab58,'0.123456789E1',18(27)>"
BigDecimal.new("1.2345678901").inspect
#=> "#<BigDecimal:7fb06a14a6a0,'0.1234567890 1E1',27(27)>"
BigDecimal.new("1.23456789012").inspect
#=> "#<BigDecimal:7fb06a1393a0,'0.1234567890 12E1',27(27)>"
BigDecimal.new("1.234567890123").inspect
#=> "#<BigDecimal:7fb06a123780,'0.1234567890 123E1',27(27)>"
It can be seen from the source code for inspect that, if there are more than 10 significant digits, each 10 characters are separate by a space (for readability, presumably):
BigDecimal.new("123.456789012345678901234567").inspect
#=> "#<BigDecimal:7fb06a0ac8b0,'0.1234567890 1234567890 1234567E3',36(36)>"
I suggest retrieving the string representation of the BigDecimal value as follows:
str = "#<BigDecimal:7fb06a14a6a0,'0.1234567890 1E1',27(27)>"
str.delete(' ').split(?')[1]
#=> "0.12345678901E1"
We are not finished. We must still convert the string we extract to a numerical object. We cannot use BigDecimal#to_f, however, if the value is large in absolute value:
"1.23456789012345678".to_f
#=> 1.2345678901234567
The safest course of action is to return a BigDecimal object, using the method BigDecimal::new, which takes two arguments:
the value to be converted to a BigDecimal object, which can be an Integer, Float, Rational, BigDecimal, or String. If a String, which is what we will supply, "spaces are ignored and unrecognized characters terminate the value" (similar to "123.4cat".to_f #=> 123.4).
the number of significant digits. If omitted or zero, the number of significant digits is determined from the value. I will omit this argument. (For example, BigDecimal.new("0.1234E2").precs #=> [18, 18], where the array contains the current and maximum numbers of significant digits.
Note the second argument is required if the first is a Float or Rational, else it is optional.
We therefore can write:
require 'bigdecimal'
def convert(str)
BigDecimal.new(str.delete(' ').split(?')[1])
end
convert "#<BigDecimal:7facd39d7ee8,'0.1234E4',9(18)>"
#=> #<BigDecimal:7facd39c7de0,'0.1234E4',9(18)>
convert "#<BigDecimal:7facd39b7be8,'0.1234E2',18(18)>"
#=> #<BigDecimal:7facd39ae610,'0.1234E2',18(18)>
convert "#<BigDecimal:7facd3990638,'0.1234E0',9(18)>"
#=> #<BigDecimal:7facd3980aa8,'0.1234E0',9(18)>
convert "#<BigDecimal:7facd3970e28,'0.1234E-2',9(18)>"
#=> #<BigDecimal:7facd39625d0,'0.1234E-2',9(18)>
v = convert "#<BigDecimal:7fb06a123780,'0.1234567890 123E1',27(27)>"
#=> #<BigDecimal:7fb069851d78,'0.1234567890 123E1',27(27)>
An easy way to see if the BigDecimal object can be converted to a float without loss of accuracy is:
def convert_bd_to_float(bd)
f = bd.to_f
(bd==BigDecimal.new(f.to_s)) ? f : nil
end
convert_bd_to_float BigDecimal.new('1234567890123456')
#=> 1.234567890123456e+15
convert_bd_to_float BigDecimal.new('12345678901234567')
#=> nil
"#<BigDecimal:6fe4790,'0.1E2',9(36)>"[/(?<=').+(?=')/]
# => "0.1E2"
I don't know which version of Ruby you are using, so I checked some MRI source code for BigDecimal:
2000 /* Returns debugging information about the value as a string of comma-separated
2001 * values in angle brackets with a leading #:
2002 *
2003 * BigDecimal.new("1234.5678").inspect ->
2004 * "#<BigDecimal:b7ea1130,'0.12345678E4',8(12)>"
2005 *
2006 * The first part is the address, the second is the value as a string, and
2007 * the final part ss(mm) is the current number of significant digits and the
2008 * maximum number of significant digits, respectively.
2009 */
2010 static VALUE
2011 BigDecimal_inspect(VALUE self)
2012 {
2013 ENTER(5);
2014 Real *vp;
2015 volatile VALUE obj;
2016 size_t nc;
2017 char *psz, *tmp;
2018
2019 GUARD_OBJ(vp, GetVpValue(self, 1));
2020 nc = VpNumOfChars(vp, "E");
2021 nc += (nc + 9) / 10;
2022
2023 obj = rb_str_new(0, nc+256);
2024 psz = RSTRING_PTR(obj);
2025 sprintf(psz, "#<BigDecimal:%"PRIxVALUE",'", self);
2026 tmp = psz + strlen(psz);
2027 VpToString(vp, tmp, 10, 0);
2028 tmp += strlen(tmp);
2029 sprintf(tmp, "',%"PRIuSIZE"(%"PRIuSIZE")>", VpPrec(vp)*VpBaseFig(), VpMaxPrec(vp)*VpBaseFig());
2030 rb_str_resize(obj, strlen(psz));
2031 return obj;
2032 }
2033
So, what you want seems to be the second part of the inspect-string, 0.1E2 in your case, equal to 10. The comment above is quite clear, this should be the full numeric value of the object. Simple regex will be enough.
another option:
"#<BigDecimal:95915c4,'0.1E2',9(27)>".split(",")[1].tr! "'", ''
=> "0.1E2"

Round up to the nearest tenth?

I need to round up to the nearest tenth. What I need is ceil but with precision to the first decimal place.
Examples:
10.38 would be 10.4
10.31 would be 10.4
10.4 would be 10.4
So if it is any amount past a full tenth, it should be rounded up.
I'm running Ruby 1.8.7.
This works in general:
ceil(number*10)/10
So in Ruby it should be like:
(number*10).ceil/10.0
Ruby's round method can consume precisions:
10.38.round(1) # => 10.4
In this case 1 gets you rounding to the nearest tenth.
If you have ActiveSupport available, it adds a round method:
3.14.round(1) # => 3.1
3.14159.round(3) # => 3.142
The source is as follows:
def round_with_precision(precision = nil)
precision.nil? ? round_without_precision : (self * (10 ** precision)).round / (10 ** precision).to_f
end
To round up to the nearest tenth in Ruby you could do
(number/10.0).ceil*10
(12345/10.0).ceil*10 # => 12350
(10.33 + 0.05).round(1) # => 10.4
This always rounds up like ceil, is concise, supports precision, and without the goofy /10 *10.0 thing.
Eg. round up to nearest hundredth:
(10.333 + 0.005).round(2) # => 10.34
To nearest thousandth:
(10.3333 + 0.0005).round(3) # => 10.334
etc.

Resources