I'm new to writing dissectors in 'C' and I came across the need to read 8 bytes timestamp from a packet.
I'm trying the following code:
g_print("offset=%d, starttime=0x%08x\n", offset, tvb_get_letoh64(tvb, offset));
and I get:
offset=8, starttime=0x0362ea14
which is only 4 bytes out of the 8 I was expecting.
How can I read it so the output would be:
offset=8, starttime=0x14ea620305779840
I also tried reading it using:
g_print("offset=%d, starttime=0x%08x\n", offset, tvb_get_bits64(tvb, 64, 32, ENC_LITTLE_ENDIAN));
g_print("offset=%d, starttime=0x%08x\n", offset, tvb_get_bits64(tvb, 64, 64, ENC_LITTLE_ENDIAN));
and it printed the 4 first bytes of the timestamp and the 2nd call printed the last 4 bytes. I'm missing something very basic...
2nd question, ok, let's assume I get the value right and convert it into nstime_t, How can I format this into a Date\time format, something like:
YYYY-MM-DDZHH:MM:SS:MMMM
Thank you so much!
What output do you get with this?
g_print("offset=%d, starttime=0x%08lx\n", offset, tvb_get_letoh64(tvb, offset));
As for your 2nd question, what is the meaning of these 8 bytes? Maybe you can declare your hf variable using FT_ABSOLUTE_TIME and use something like proto_tree_add_time(), proto_tree_add_time_item(), proto_tree_add_time_format_value() or proto_tree_add_time_format()?
Related
I have come across the following code when dealing with Go readers to limit the number of bytes read from a remote client when sending a file through multipart upload (e.g. in Postman).
r.Body = http.MaxBytesReader(w, r.Body, 32<<20+1024)
If I am not mistaken, the above notation should represent 33555456 bytes, or 33.555456 MB (32 * 2 ^ 20) + 1024. Or is this number not correct?
What I don't understand is:
why did the author use it like this? Why using 20 and not some other number?
why the author used the notation +1024 at all? Why didn't he write 33 MB instead?
would it be OK to write 33555456 directly as int64?
If I am not mistaken, the above notation should represent 33555456 bytes, or 33.555456 MB (32 * 2 ^ 20) + 1024. Or is this number not correct?
Correct. You can trivially check it yourself.
fmt.Println(32<<20+1024)
Why didn't he write 33 MB instead?
Because this number is not 33 MB. 33 * 1024 * 1024 = 34603008
would it be OK to write 33555456 directly as int64?
Naturally. That's what it likely is reduced to during compilation anyway. This notation is likely easier to read, once you figure out the logic behind 32, 20 and 1024.
Ease of reading is why I almost always (when not using ruby) write constants like "50 MB" as 50 * 1024 * 1024 and "30 days" as 30 * 86400, etc.
I am new to ETL and I was assigned with a task on sanitizing some sensitive information before giving the data to a client.
I am using HUE web client with Impala.
What I want to do is:
For example, a column info like '京客隆（三里屯店）', then I need to transform it into something like '京XXX店）' .
My query is:
select '京客隆（三里屯店）', concat(substr('京客隆（三里屯店）', 1, 3), 'XXX', substr('京客隆（三里屯店）', char_length('京客隆（三里屯店）') -6, 6));
But I get gibberish in the output:
'京客隆（三里屯店）' | concat(substr('京客隆（三里屯店）', 1, 3), 'xxx', substr('京客隆（三里屯店）', char_length('京客隆（三里屯店）') - 6, 6))
京客隆（三里屯店） | 京XXX�店�
The problem is that :
select '京客隆（三里屯店）', substr('京客隆（三里屯店）', char_length('京客隆（三里屯店）') -3 , 3);
output: 京客隆（三里屯店） ��
doesn't get the correct characaters. Why is that? I pasted the string in python shell and I can get the correct characters if I only take the last 3 bytes.
It turns out that I misunderstood the function substr.
substr(STRING a, INT start [, INT len]) :
It takes characters starting from (including) INT start. So for example my string '京客隆（三里屯店）' is 27 bytes long in total, and each utf8 char takes 3 bytes here. I need to take the last 3 bytes, which is the ） , then I need to write:
substr('京客隆（三里屯店）, 27 - 2 ,3 ) .
It then gets the 25, 26, 27 3 bytes and display the char ） correctly.
Updated:
I was told to use :
SELECT regexp_replace('京客隆（三里屯店）', '(.)(.*)(.{2})', '\\1***\\3');
works like an charm :P.
I'm trying to print the first 5 lines from a set of large (>500MB) csv files into small headers in order to inspect the content more easily.
I'm using Ruby code to do this but am getting each line padded out with extra Chinese characters, like this:
week_num type ID location total_qty A_qty B_qty count㌀㐀ऀ猀漀爀琀愀戀氀攀ऀ㤀㜀ऀ䐀䔀开伀渀氀礀ऀ㔀㐀㜀㈀ ㌀ऀ㔀㐀㜀㈀ ㌀ऀ ऀ㤀㈀㔀㌀ഀ
44 small 14 A 907859 907859 0 550360㐀ऀ猀漀爀琀愀戀氀攀ऀ㐀㈀ऀ䐀䔀开伀渀氀礀ऀ㌀ ㈀㜀㐀ऀ㌀ ㈀
The first few lines of input file are like so:
week_num type ID location total_qty A_qty B_qty count
34 small 197 A 547203 547203 0 91253
44 small 14 A 907859 907859 0 550360
41 small 421 A 302174 302174 0 18198
The strange characters appear to be Line 1 and Line 3 of the data.
Here's my Ruby code:
num_lines=ARGV[0]
fh = File.open(file_in,"r")
fw = File.open(file_out,"w")
until (line=fh.gets).nil? or num_lines==0
fw.puts line if outflag
num_lines = num_lines-1
end
Any idea what's going on and what I can do to simply stop at the line end character?
Looking at input/output files in hex (useful suggestion by #user1934428)
Input file - each character looks to be two bytes.
Output file - notice the NULL (00) between each single byte character...
Ruby version 1.9.1
The problem is an encoding mismatch which is happening because the encoding is not explicitly specified in the read and write parts of the code. Read the input csv as a binary file "rb" with utf-16le encoding. Write the output in the same format.
num_lines=ARGV[0]
# ****** Specifying the right encodings <<<< this is the key
fh = File.open(file_in,"rb:utf-16le")
fw = File.open(file_out,"wb:utf-16le")
until (line=fh.gets).nil? or num_lines==0
fw.puts line
num_lines = num_lines-1
end
Useful references:
Working with encodings in Ruby 1.9
CSV encodings
Determining the encoding of a CSV file
I am processing one binary file in which I want to retrieve first 4 bytes, next 4 bytes, again 4 bytes and then rest of the file contents except last 8 bytes of last line.
I have tried with this line file.read.scan(/(.{4})(.{4})(.{4})(.*\w)(.{8})/).each do |a,b,c,d,e| but after some iterations this regex starts from some line in between with first 4 bytes, next 4 bytes, next 4 bytes pattern. Because of this my condition check fails.
I want to do following.
Read first 4 bytes of first line of file, then bytes 5 to 7, then bytes 8 to 11, then rest of the file content except last 8 bytes of last line of the file.
What could be the regex for this in Ruby?
Use #read instead of a regexp:
f = File.open(file_name,"rb")
chunk1 = f.read(4)
chunk2 = f.read(3)
chunk3 = f.read(4)
chunk4 = f.read(f.size - (4 + 3 + 4 + 8))
How about:
/(.{4})(.{3})(.{4})(.*).{8}/m
In Ruby, how do I replace, say, the 7th byte of a file with another byte?
Use binwrite method from IO class
IO.binwrite("testfile", [0x0D].pack("C"), 7) # => 1
# File could contain: "This is0two\nThis is line three\nAnd so on...\n"
0x0D is 13
Also you may need to know about pack method