With the same byte array, the result of performing a checksum using .NET Core 3.0's SHA512Managed class is different than the result of using the built-in shasum app in MacOS.
I've triple-checked the byte array is unmodified/identical by iterating over every byte position using File.ReadAllBytes().
Process in code:
var stream = asm.GetManifestResourceStream(resourceName);
var data = new byte[stream.Length];
stream.Read(data, 0, (int)stream.Length);
var sha = new SHA512Managed();
var checksum = sha.ComputeHash(data);
At the command line:
shasum -a512 ./myfile.dat
I'm receiving a good (expected) checksum value from the shasum utility, the iHex app from the AppStore, and the website "passwordsgenerator.net/sha512-hash-generator/".
I'm receiving a bad value from the .NET code above and from the a different website "online-convert.com". Both of which produce the identical bad result!
Is there more than one implementation of the SHA512 algorithm? Since I'm comparing byte arrays, there should be no nonsense involving character encoding, right? Is there something about compiling or reading back an embedded resource in MacOS I should know about?
Any tips or pointers would be greatly appreciated!
Related
I use charles and got a protobuf http message from other iOS applications. Now I want to genereate the same http packet but the output is not the same.
My protobuf file:
message TTCreateConversationBody
{
repeated uint32 imUid1 = 2;
}
I'm using objective-c:
TTCreateConversationBody *body = [TTCreateConversationBody new];
GPBUInt32Array *arr = [[GPBUInt32Array alloc] initWithCapacity:2];
[arr addValue:123123];
[arr addValue:9999999];
body.imUid1Array = arr;
and my output, charles decode it as a length-delimited string:
it's raw data and mine:
8A-26-10-08-01-10-AE-F7-81-80-9F-03-10-D4-E4-82-F0-D2-01
8A-26-10-08-01-12-0C-F9-F6-C3-9D-FA-02-AE-F7-81-80-9F-03
What's the correct protobuf file format?
They're actually both valid... ish.
This comes down to "packed" fields; without "packed", your two integers are encoded as
[header, varint][value][header, varint][value]
[10][AE-F7-81-80-9F-03][10][D4-E4-82-F0-D2-01]
where-as with "packed", it becomes
[header, string][length][value][value]
[12][0C][F9-F6-C3-9D-FA-02][AE-F7-81-80-9F-03]
note: the actual values look very different in the two runs... I'm assuming that is accidental.
To quote from the specification:
Protocol buffer parsers must be able to parse repeated fields that were compiled as packed as if they were not packed, and vice versa. This permits adding [packed=true] to existing fields in a forward- and backward-compatible way.
So: serializers should write the layout that is defined by whether your data is "packed" or not, but decoders must be able to handle it either way. Some libraries, when encountering data that should be "packed": determine which layout will actually be shorter, and make the final decision based on that. In reality, this can be approximated to "use packed encoding whenever there's at least two items".
I'm trying to save and load the states of Matrices (using Matrix) during the execution of my program with the functions dump and load from Marshal. I can serialize the matrix and get a ~275 KB file, but when I try to load it back as a string to deserialize it into an object, Ruby gives me only the beginning of it.
# when I want to save
mat_dump = Marshal.dump(#mat) # serialize object - OK
File.open('mat_save', 'w') {|f| f.write(mat_dump)} # write String to file - OK
# somewhere else in the code
mat_dump = File.read('mat_save') # read String from file - only reads like 5%
#mat = Marshal.load(mat_dump) # deserialize object - "ArgumentError: marshal data too short"
I tried to change the arguments for load but didn't find anything yet that doesn't cause an error.
How can I load the entire file into memory? If I could read the file chunk by chunk, then loop to store it in the String and then deserialize, it would work too. The file has basically one big line so I can't even say I'll read it line by line, the problem stays the same.
I saw some questions about the topic:
"Ruby serialize array and deserialize back"
"What's a reasonable way to read an entire text file as a single string?"
"How to read whole file in Ruby?"
but none of them seem to have the answers I'm looking for.
Marshal is a binary format, so you need to read and write in binary mode. The easiest way is to use IO.binread/write.
...
IO.binwrite('mat_save', mat_dump)
...
mat_dump = IO.binread('mat_save')
#mat = Marshal.load(mat_dump)
Remember that Marshaling is Ruby version dependent. It's only compatible under specific circumstances with other Ruby versions. So keep that in mind:
In normal use, marshaling can only load data written with the same major version number and an equal or lower minor version number.
I have a 16 length string that I would like to broadcast as the identifier, which the app uses to do certain actions.
Relatively new to the different beacon layouts, so would love to get the right opinion. I was thinking of doing it the following ways
hex encode the string and use it as identifier in eddystone url layout
But the 16 length takes up 32 bytes and doesn't help
Another option is to use altbeacon library like below.
byte[] dataBytes = new String("16 length string").getBytes()
Identifier identifier = Identifier.fromBytes(dataBytes, 0, dataBytes.length, false);
I am not sure either works mainly because of the 16 length string. Is there a better way to achieve or do it at all in first place?
I've been trying to get this to work, but I'm very frustrated at this point. I am a beginner in this field, so maybe I'm just making mistakes.
What I need to do is to take in a website .html and store it into a txt file. Now the problem is that this website is in Russian (encoding windows-1251) and Silverlight only supports 3 encodings. So in order to bypass that limitation, I got my hands on an encoding class that transfers the stream into a byte array and then tries to pull the correctly encoded string from the text. The problem with this is that
1) I try to ensure that webClient recieves a Unicode encoded stream, because the other ones do not seem to create a retrievable string, but it still doesn't seem to work.
WebClient wc = new WebClient();
wc.Encoding = System.Text.Encoding.Unicode;
wc.DownloadStringCompleted += new DownloadStringCompletedEventHandler(wc_LoadCompleted);
wc.DownloadStringAsync(new Uri(site));
2) I fear that when I store the html into a txt file using streamWriter, the encoding is, yet again, somehow screwed up.
3) The encoding class is not doing its job.
Encoding rus = Encoding.GetEncoding(1251);
Encoding eng = Encoding.Unicode;
byte[] bytes = rus.GetBytes(string);
textBlock1.Text = eng.GetString(bytes);
Can anyone offer any help on this matter? This huge detriment to my project. Thanks in advance,
Since you want to handle an encoding alien to Silverlight you should start with downloading using OpenReadAsync and OpenReadCompleted.
Now you should be able to take the Stream provided by the event args Result property and supply it directly to the encoding component you have acquired to generate the correct string result.
I am attempting to decrypt a number encrypted by another program that uses the BouncyCastle library for Java.
In Java, I can set the key like this: key = Hex.decode("5F3B603AFCE22359");
I am trying to figure out how to represent that same step in Ruby.
To get Integer — just str.hex. You may get byte array in several ways:
str.scan(/../).map(&:hex)
[str].pack('H*').unpack('C*')
[str].pack('H*').bytes.to_a
See other options for pack/unpack and examples (by codeweblog).
For a string str:
"".tap {|binary| str.scan(/../) {|hn| binary << hn.to_i(16).chr}}