Implementing a SNMP v1 decoder and working with some Wireshark captures, I can see that sometimes length field of a BER if coded with one byte and other times with two bytes.
Reading BER rules, if more significative bit is setted to 1, then the length value must be extended with next byte to represent values bigger than 255.
So, if firt byte is 0x81, and next byte is 0x9F, then the extended Length field should take the 0x9F value... OK
My question is:
If second byte is 0x9F, the more significative bit is 1 again.
Wireshark only takes two bytes for this length.
Why in this case size of Length is only two bytes?
Length fields are restricted to 2 bytes?
Thanks.
According to the BER rule, the length field can be multiple bytes (much more than 2),
http://en.wikipedia.org/wiki/KLV
Length Field
Following the bytes for the Key are bytes for the
Length field which will tell you how many bytes follow the length
field and make up the Value portion. There are four kinds of encoding
for the Length field: 1-byte, 2-byte, 4-byte and Basic Encoding Rules
(BER). The 1-, 2-, and 4-byte variants are pretty straightforward:
make an unsigned integer out of the bytes, and that integer is the
number of bytes that follow.
BER length encoding is a bit more
complicated but the most flexible. If the first byte in the length
field does not have the high bit set (0x80), then that single byte
represents an integer between 0 and 127 and indicates the number of
Value bytes that immediately follows. If the high bit is set, then the
lower seven bits indicate how many bytes follow that themselves make
up a length field.
For example if the first byte of a BER length field
is binary 10000010, that would indicate that the next two bytes make
up an integer that then indicates how many Value bytes follow.
Therefore a total of three bytes were taken up to specify a length.
"If second byte is 0x9F, the more significative bit is 1 again." Is that a question? Only the first byte in the bytes determines how many following bytes are used to determine the length. So you never need to care about the most significant bit of the second byte. Never.
How Wireshark represents the bytes is not very critical. Unless Wireshark shows you a wrong value for length, you should not pay much attention to it.
In ASN.1 BER Length Encoding Rules:
a) If the number of content octets <= 127 then length octet encodes number of content octets.
b) Else the most signicant bit of the rst length octet is set to 1 and other 7 bits describe number of length octets following.
c) The following length octets encode the length of content octets in Big Endian byte order.
Example:
Length 126: 01111110
Length 127: 01111111
Length 128: 10000001 10000000
Length 1031: 10000010 00000100 00000111
Number | MSB of 1st Byte | Bytes to represent the Number | BE Binary
128 | 1 | 0000001 (=1) | 10000000 (=128+0+0+0+0+0+0+0)
1031 | 1 | 0000010 (=2) | 00000100 00000111 (=1024+0+0+0+0+0+0+4+2+1)
I'd add that SNMP (usually) uses UDP datagram for transport, which are limited to 65535 bytes, that is 0xffff. Exactly 2 bytes are needed at most to encode the length.
Related
https://developers.google.com/protocol-buffers/docs/encoding?hl=eng#structure
enter image description here
I understand why drop msb from "value".
However, I don't know why msb is dropped on the "key" side.
? 000 1000: key
1 001 0110: value (msb: 1)
0 000 0001: value (msb: 0 = end)
"key" has a byte after it, so the number to enter msb is 1 (1 000 1000)
or
I think there is no reason to drop msb because "key" is 1 byte.(0000 1000)
but It dropped msb and msb bit is 0.
Why is the value 0 even though it dropped msb?
You need to think of the key, in isolation, as an integer, and varint encode that integer by itself using varint encoding. The integer shown (b1000) only needs 4 bits, so in varint encoding that is a single byte with a 0 MSB. But keys can be much much larger. Think of field number 1708 - the number (not including the wire type) is b11010101100 - so that's 11 bits for the number, plus 3 bits for the wire type, so we have 14 bits of data to encode. Varint is a 7-bit encoding, so we can store that in 2 bytes: the first will have the MSB 1, and the second will have the MSB 0.
Or in other words: any MSB when discussing the key is there to help you find where the key stops being encoded. It has nothing to do with the payload that comes afterwards.
According to the RFC 6455 specification about websocket's.
Data frame structure is follows:
frame-fin ; 1 bit in length
frame-rsv1 ; 1 bit in length
frame-rsv2 ; 1 bit in length
frame-rsv3 ; 1 bit in length
frame-opcode ; 4 bits in length
frame-masked ; 1 bit in length
frame-payload-length ; either 7, 7+16,
; or 7+64 bits in
; length
[ frame-masking-key ] ; 32 bits in length
frame-payload-data ; n*8 bits in
; length, where
; n >= 0
So the minimum length of byte array to hold a frame would be 224 bytes (56 bits)? As I read on internet to represent a bit in byte array we need 4 bytes (1000).
How do I mask data? And what data should I mask? Only frame-payload-data or all the frame except the mask key?
The frame-masking-key field is only present when the frame is masked, which is only done for frames sent by a client to a server. And the frame-payload-data is optional; a frame may be empty, containing no data. Therefore the minimum length of a frame in the client-to-server direction is (1+1+1+1+4+1+7+32)=48 bits or 6 bytes, and the minimum length of a frame in the server-to-client direction is (1+1+1+1+4+1+7)=16 bits or 2 bytes.
Those would be frames that carry no payload. Obviously frames that carry payload data will require additional space.
As I read on internet to represent a bit in byte array we need 4 bytes
(1000).
Umm, no, each byte holds 8 bits. It might be convenient within a program to use larger data units to represent bit values, but that is completely independent of the format that is used in the actual frame.
How do I mask data? And what data should I mask? Only frame-payload-data
or all the frame except the mask key?
You mask by XOR-ing the frame-masking-key over the frame-payload-data. This is described in section 5.3 of RFC 6455.
In a base64 digits you could save up to 6 bits (2**6 == 64).
Which means that you could fit 3 bytes in 4 base64 digits.
64**4 == 2**24
That's why:
0x000000 == 'AAAA'
0xFFFFFF == '////'
This means that a random string of 3 bytes is equivalent to a base64 string of 4 characters.
However if I am converting a number of bytes which is not a multiple of 3 in a base64 string, I will not be able to generate all the combination of the base64 string.
Let's take an example:
If I want a random 7 characters base64 string, I would need to generate 42 random bits (64**7 == 2**42).
If I am using urandom to get 5 random bytes I will get only 40 bits (5*8) and if ask for 6 I will get 48 bits (6*8).
Can I ask for 6 bytes and use a mask to short it down to 5 or will it break my random repartition?
One solution:
hex(0x123456789012 & 0xFFFFFFFFFF)
'0x3456789012'
Another one:
hex(0x123456789012 >> 8)
'0x1234567890'
What do you think?
base64 strings with 7 characters of length
is an encode of a file with 5 bytes ( 40 bits: no less, no more )
40%6 = 4
base64 needs to add 2 more bits, and then, with 42 bits, 42%6=0,
the encode is possible; but, beware:
" If I want a random 7 characters base64 string, I would need to
generate 42 random bits (64**7 == 2**42). "
the 2 additional bits are not random, and are constants; are zero, indeed.
The cardinal number of your key-space doesn't change: is 2**40 = 1099511627776, not (64**7 == 2**42).
(64**7 == 2**42) is the cardinal number of the key-space that contains all possible combinations of 64 chars with length of 7; but, with the last two bits fixed (to zero, in this case, but doesn't matter the value) you don't have all possible combinations.
6 random bytes (48 bits), or 42 random bits, increase your original key-space; you should use 5 random bytes (40 bits), and send it to base64
In RSA algorithm, it is given that plain text size must be less than or equal to ( key size - 11 ).
why it is like that. what is that 11 actually...?
This is only a requirement when PKCS1 padding is used. "Naked" RSA, insecure though it is, can do stuff up to the key size in length, and OAEP padding depends on the hash algorithm you're using.
From the PKCS1 spec:
Steps:
1. Length checking: If mLen > k - 11, output "message too long" and
stop.
2. EME-PKCS1-v1_5 encoding:
a. Generate an octet string PS of length k - mLen - 3 consisting
of pseudo-randomly generated nonzero octets. The length of PS
will be at least eight octets.
b. Concatenate PS, the message M, and other padding to form an
encoded message EM of length k octets as
EM = 0x00 || 0x02 || PS || 0x00 || M.
As per that at least 11 bytes are prepended to the plaintext. 0x00 and 0x02 and then PS, which has a length of "at least eight octets" and then 0x00. So 3 + 8 = 11.
I'm using Ruby and I'm communicating with a network endpoint that requires the formatting of a 'header' prior to sending the message itself.
The first field in the header must be the message length which is defined as a 2 binary byte message length in network byte order.
For example, my message is 1024 in length. How do I represent 1024 as binary two-bytes?
The standard tools for byte wrangling in Ruby (and Perl and Python and ...) are pack and unpack. Ruby's pack is in Array. You have a length that should be two bytes long and in network byte order, that sounds like a job for the n format specifier:
n | Integer | 16-bit unsigned, network (big-endian) byte order
So if the length is in length, you'd get your two bytes thusly:
two_bytes = [ length ].pack('n')
If you need to do the opposite, have a look at String#unpack:
length = two_bytes.unpack('n').first
See Array#pack.
[1024].pack("n")
This packs the number as the network-order byte sequence \x04\x00.
The way this works is that each byte is 8 binary bits. 1024 in binary is 10000000000. If we break this up into octets of 8 (8 bits per byte), we get: 00000100 00000000.
A byte can represent (2 states) ^ (8 positions) = 256 unique values. However, since we don't have 256 ascii-printable characters, we visually represent bytes as hexadecimal pairs, since a hexadecimal digit can represent 16 different values and 16 * 16 = 256. Thus, we can take the first byte, 00000100 and break it into two hexadecimal quads as 0000 0100. Translating binary to hex gives us 0x04. The second byte is trivial, as 0000 0000 is 0x00. This gives us our hexadecimal representation of the two-byte string.
It's worth noting that because you are constrained to a 2-byte (16-bit) header, you are limited to a maximum value of 11111111 11111111, or 2^16 - 1 = 65535 bytes. Any message larger than that cannot accurately represent its length in two bytes.