What do these bytes do? - format
This is the hexdump of a black 1x1 PNG made in Gimp and exported with minimal information:
89 50 4E 47 0D 0A 1A 0A 00 00 00 0D 49 48 44 52
00 00 00 01 00 00 00 01 08 02 00 00 00 90 77 53
DE 00 00 00 0C 49 44 41 54 08 D7 63 60 60 60 00
00 00 04 00 01 27 34 27 0A 00 00 00 00 49 45 4E
44 AE 42 60 82
Now after reading the specification I am quite sure what most of them mean, except for bytes 30-34 between the IHDR and IDAT chunk: 90 77 53 DE
Can someone enlighten me?
Those numbers are the CRC checksum for the previous chunk. See in the official specification: 5 Datastream structure for a general overview, and in particular 5.3 Chunk layout.
A CRC is calculated for, and appended to each separate chunk:
A four-byte CRC (Cyclic Redundancy Code) calculated on the preceding bytes in the chunk, including the chunk type field and chunk data fields, but not including the length field. The CRC can be used to check for corruption of the data. The CRC is always present, even for chunks containing no data.
Here is your 1x1 pixel image, annotated byte for byte. Right after the data of each of the chunks IHDR, IDAT, and IEND is a CRC for the preceding data.
File: test.png
89 50 4E 47 0D 0A 1A 0A
Header 0x89 "PNG" CR LF ^Z LF checks out okay
===========
00 00 00 0D
49 48 44 52
00 00 00 01 00 00 00 01 08 02 00 00 00
90 77 53 DE
block: "IHDR", 13 bytes [49484452]
Width: 1
Height: 1
Bit depth: 8
Color type: 2 = Color
(Bits per pixel: 8)
(Bytes per pixel: 3)
Compression method: 0
Filter method: 0
Interlace method: 0 (none)
CRC: 907753DE
===========
00 00 00 0C
49 44 41 54
08 D7 63 60 60 60 00 00 00 04 00 01
27 34 27 0A
block: "IDAT", 12 bytes [49444154]
expanded result: 4 (as expected)
(Row 0 Filter:0)
decompresses into
00 00 00 00
CRC: 2734270A
===========
00 00 00 00
49 45 4E 44
AE 42 60 82
block: "IEND", 0 bytes [49454E44]
CRC: AE426082
The IDAT data decompresses into four 0's: the first one is the row filter (0, meaning 'none') and the next 3 bytes are Red, Green, Blue values for the one single pixel.
Related
"zlib: inflate error = -3 (data error)" with custom png implementation
I'm implementing a PNG encoder in VHDL for learning purposes. It works with image sizes from 1x1 to 4x4. At the image size of 5x5 there is a behaviour I can't understand: When encoding raw data with values 0...24, the encoding works. However, when using raw data with values 255...231, it generates a broken image. Input values 0...24: > hexdump -C png_encoder/gen/test_img_no_compression_5x5.png 00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR| 00000010 00 00 00 05 00 00 00 05 08 00 00 00 00 a8 04 79 |...............y| 00000020 39 00 00 00 4c 49 44 41 54 78 01 00 04 00 fb ff |9...LIDATx......| 00000030 00 00 01 02 00 04 00 fb ff 03 04 00 05 00 04 00 |................| 00000040 fb ff 06 07 08 09 00 04 00 fb ff 00 0a 0b 0c 00 |................| 00000050 04 00 fb ff 0d 0e 00 0f 00 04 00 fb ff 10 11 12 |................| 00000060 13 00 04 00 fb ff 00 14 15 16 01 02 00 fd ff 17 |................| 00000070 18 0b a4 01 2d d5 1f a2 6d 00 00 00 00 49 45 4e |....-...m....IEN| 00000080 44 ae 42 60 82 |D.B`.| 00000085 > pngcheck -vv png_encoder/gen/test_img_no_compression_5x5.png File: png_encoder/gen/test_img_no_compression_5x5.png (133 bytes) chunk IHDR at offset 0x0000c, length 13 5 x 5 image, 8-bit grayscale, non-interlaced chunk IDAT at offset 0x00025, length 76 zlib: deflated, 32K window, superfast compression row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth): 0 0 0 0 0 (5 out of 5) chunk IEND at offset 0x0007d, length 0 No errors detected in png_encoder/gen/test_img_no_compression_5x5.png (3 chunks, -432.0% compression). Input values 255...231: > hexdump -C png_encoder/gen/test_img_no_compression_5x5.png 00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR| 00000010 00 00 00 05 00 00 00 05 08 00 00 00 00 a8 04 79 |...............y| 00000020 39 00 00 00 4c 49 44 41 54 78 01 00 04 00 fb ff |9...LIDATx......| 00000030 00 ff fe fd 00 04 00 fb ff fc fb 00 fa 00 04 00 |................| 00000040 fb ff f9 f8 f7 f6 00 04 00 fb ff 00 f5 f4 f3 00 |................| 00000050 04 00 fb ff f2 f1 00 f0 00 04 00 fb ff ef ee ed |................| 00000060 ec 00 04 00 fb ff 00 eb ea e9 01 02 00 fd ff e8 |................| 00000070 e7 6a 21 17 bc 9a 17 87 e7 00 00 00 00 49 45 4e |.j!..........IEN| 00000080 44 ae 42 60 82 |D.B`.| 00000085 > pngcheck -vv png_encoder/gen/test_img_no_compression_5x5.png File: png_encoder/gen/test_img_no_compression_5x5.png (133 bytes) chunk IHDR at offset 0x0000c, length 13 5 x 5 image, 8-bit grayscale, non-interlaced chunk IDAT at offset 0x00025, length 76 zlib: deflated, 32K window, superfast compression row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth): zlib: inflate error = -3 (data error) (0 out of 5) ERRORS DETECTED in png_encoder/gen/test_img_no_compression_5x5.png How to interpret the error message zlib: inflate error = -3 (data error)? I read https://www.zlib.net/zlib_how.html, but didn't find more specific information. My first guess was the row filters are incorrect, but since both files are structured the same, this is unlikely. Is there something wrong with the ADLER32 calculation in the second case (possibly some overflow)?
It was an overflow in the ADLER32 checksum calculation. Specifically, there were two 16 bit numbers added and truncated before applying the modulo with 65521. Unfortunately my ADLER32 unittest didn't catch it, yet. However, the error message was shown several times during the implementation and I was always not sure about the cause. If anybody could elaborate the error message or explain how to get a better error message, I would be glad to hear it.
Why smppcxx sample_app isn't working and returns "SMPP error: Invalid command_length" and crashes?
I took sample_app from smppcxx library and changed the settings to:
const std::string ipaddr = "194.228.174.1";
const Smpp::Uint16 port = 9111;
const Smpp::SystemId sysid("MaxiTipSMPP");
const Smpp::Password pass(<actual_password>);
const Smpp::SystemType systype("");
const Smpp::Uint8 infver = 0x34;
const Smpp::ServiceType servtype("");
const Smpp::Address srcaddr("234567");
const Smpp::Address dstaddr("420606752839");
const std::string msgtext = "Hello smsc";
The code called is:
Socket sd;
sd.connect(ipaddr.c_str(), port);
send_bind(sd);
read_bind_resp(sd);
//send_enquire_link(sd);
//read_enquire_link_resp(sd);
send_submit_sm(sd);
read_submit_sm_resp(sd);
Smpp::Uint32 seqnum = read_deliver_sm(sd);
send_deliver_sm_resp(sd, seqnum);
//send_data_sm(sd);
//read_data_sm_resp(sd);
//seqnum = read_deliver_sm(sd);
//send_deliver_sm_resp(sd, seqnum);
send_unbind(sd);
read_unbind_resp(sd);
and the problem happens in read_submit_sm_resp(sd) (or in read_enquire_link_resp(sd) if uncommented):
Buffer buf;
buf = read_smpp_pdu(sd, buf);
std::cout << "\nRead a submit sm resp\n";
Smpp::hex_dump(&buf[0], buf.size(), std::cout);
Smpp::SubmitSmResp pdu;
std::cout << "read_submit_sm_resp buf.size() is " << buf.size() << std::endl;
pdu.decode(&buf[0]);
std::string sid = pdu.message_id();
printf("response message_id: \"%s\"\n", sid.c_str());
on line
pdu.decode(&buf[0]);
, why? The application crashes. I expected the code to work as is, but it just doesn't.
There is the output:
Sending a bind transceiver
00 00 00 2a 00 00 00 09 00 00 00 00 00 00 00 01 ...*............
4d 61 78 69 54 69 70 53 4d 50 50 00 MaxiTipSMPP.password
Read a bind response
00 00 00 15 80 00 00 09 00 00 00 00 00 00 00 01 ................
53 4d 53 43 00 SMSC.
read_bind_resp buf.size() is 21
response system_id: "SMSC"
Sending a submit sm
00 00 00 3d 00 00 00 04 00 00 00 00 00 00 00 01 ...=............
00 00 00 32 33 34 35 36 37 00 01 01 34 32 30 36 ...234567...4206
30 36 37 35 32 38 33 39 00 00 00 00 00 00 01 00 06752839........
00 00 0a 48 65 6c 6c 6f 20 73 6d 73 63 ...Hello smsc
Read a submit sm resp
00 00 00 a4 00 00 00 05 00 00 00 00 00 00 00 01 ................
00 01 01 39 39 39 30 33 30 00 01 01 34 32 30 36 ...999030...4206
30 36 37 35 32 38 33 39 00 04 00 00 00 00 00 00 06752839........
00 00 47 69 64 3a 66 62 32 37 37 66 62 34 33 66 ..Gid:fb277fb43f
63 31 34 36 66 30 39 61 39 31 37 37 32 63 37 63 c146f09a91772c7c
31 33 64 65 35 62 20 64 6f 6e 65 20 64 61 74 65 13de5b done date
3a 31 37 30 32 30 36 30 35 30 37 30 34 20 73 74 :170206050704 st
61 74 3a 55 4e 44 45 4c 49 56 00 1e 00 21 66 62 at:UNDELIV...!fb
32 37 37 66 62 34 33 66 63 31 34 36 66 30 39 61 277fb43fc146f09a
39 31 37 37 32 63 37 63 31 33 64 65 35 62 00 04 91772c7c13de5b..
27 00 01 05 '...
read_submit_sm_resp buf.size() is 164
SMPP error: Invalid command_length
I added an output and it tells that size is 164 and I see 164 bytes and in bind response, which works without problems, there is size 21 and I see 21 bytes, should I fix the decode function somehow?
Smpp::SubmitSmResp::decode(const Smpp::Uint8* buff)
{
Response::decode(buff);
Smpp::Uint32 len = Response::command_length();
Smpp::Uint32 offset = 16;
const char* err = "Bad length in submit_sm_resp";
if(len < offset)
throw Error(err);
const Smpp::Char* sptr = reinterpret_cast<const Smpp::Char*>(buff);
message_id_ = &sptr[offset];
offset += message_id_.length() + 1;
if(len < offset)
throw Error(err);
Header::decode_tlvs(buff + offset, len - offset);
}
I still think that the library should work as is, so I guess that maybe I should change some setting or something. Did anyone have the same problem? Any idea what to do? The only thing I want is to send smses, about max 100 a day...
I managed to fix it. 1) Have only 1 open connection instead of opening a new connection for every sms to send. Provider banned me, because of too many connection openings. 2) The flow isn't send->read->send->read->.., but reading of the responses should be done asynchronously and so the client must parse the responses to understand, what type of response was received. 3) The connection should be maintained with send_enquire_link. I guess that programming this gateway for sending smses was far over the scope of 1 manday for a person like me that didn't know anything about smpp. I finished the task in about 3 days with quite a lot of work overnight. Why do I add that? Because the main problem was my approach because of the assigned time => I thought that there should be a simple solution for 1 manday task...
Which Registry Editor Version to initiate reg file with when editing the Windows registry (if needed at all)?
According to Microsoft knowledge base article 310516, one should initiate .reg files editing the Windows registry with either Windows Registry Editor Version 5.00 for Windows 2000, Windows XP, and Windows Server 2003 or REGEDIT4 for Windows 95, Windows 98 and Windows NT 4.0. When on Windows 7, is it correct to initiate with Windows Registry Editor Version 5.00 or should one state version 6 instead? Is it in fact needed at all when running a .bat file instead? For current version click in Registry Editor in menu Help on menu item About Registry Editor.
There are used mainly two standard applications to process Windows registry from within a batch file: reg.exe regedit.exe On Windows x64 both exist as 32-bit and as 64-bit applications whereby the 32-bit versions access the registry like 32-bit applications which means registry redirector is active and registry keys for 64-bit applications are not visible for 32-bit REG and REGEDIT. It is in general better to use REG instead of REGEDIT for getting (read access) or setting (write access) just a few keys or values in batch files. The usage of REGEDIT requires always administrator privileges since Windows Vista even when just used to export keys into a registry file (read access) or when importing data from a registry file (write access) to HKEY_CURRENT_USER (short: HKCU). By using REG many operations like QUERY (read access) or ADD (write access) to HKCU don't need administrator privileges. Of course a write access to HKEY_LOCAL_MACHINE (short: HKLM) requires administrator privileges as also other not often used operations. Running in a command prompt window reg /? shows the available operations. reg query /? and reg add /? executed in a console window displays syntax and options to get (query) or set (add) data in Windows registry. A *.reg file is interpreted on import only as file with valid registry data if the first line of the file contains REGEDIT4 or Windows Registry Editor Version 5.00 This is not the version of registry editor or the Windows version. It is a version information for format of the registry data in the file itself. So it is a registry file format version. The first line of a registry file must have one of those two strings. An attempt importing a *.reg file without this header line results in an error. Windows 2000 (has also regedt32.exe) and later versions of Windows support registry files in both format versions, 4 and 5. Windows 95, 98, Millennium and NT4 support only registry files in format 4. So it depends nowadays (year 2016) mainly on the data which format to use for a registry file. The main difference is the supported encoding of text data. Registry files in format REGEDIT4 can be only text files with 1 byte per character using the code page defined in Windows Locales and Languages settings for non Unicode aware applications which is Windows-1252 for North American and Western European countries. Therefore registry files in format REGEDIT4 cannot contain Unicode characters with a code value greater 255 (decimal, 0xFF hexadecimal). Registry files in format Windows Registry Editor Version 5.00 support text encoded with more than 1 byte per character according to Unicode standard. Therefore *.reg files in format Windows Registry Editor Version 5.00 are created by REGEDIT as UTF-16 Little Endian encoded text files with Byte Order Mark (FF FE). So it does not matter which file format to use if the data to import / export is binary or text using only ASCII characters (code value < decimal 128) as 99.99 % of all registry data. For non ASCII characters it is already important to know which code page is used by Windows command interpreter as this is usually an OEM code page like code page 437 (US, Canada) or code page 850 (Western Europe) being different from GUI code page Windows-1252. Running in a command prompt window chcp or mode con outputs the default code page used on current computer by Windows command interpreter for console. If it is really necessary to support also Unicode strings, it is necessary to use format Windows Registry Editor Version 5.00. Let us look on a hypothetical Unicode example: In Windows registry under key HKCU\RegFormat there are two Unicode strings defined: A string of type REG_SZ with name Two Not Equal 3 with Unicode string value: 2 ≠ 3 A multi-line string of type REG_MULTI_SZ with name Formulas with Unicode string value: 2 µs × 3 = 6 µs A = π × r² See the Microsoft articles Registry Value Types, Distributing Registry Changes and Data Types in the Registry for details about registry value types. Exporting key HKCU\RegFormat in format Windows Registry Editor Version 5.00 being the standard format since Windows 2000 results in a UTF-16 LE encoded text file with 596 bytes with content: Windows Registry Editor Version 5.00 [HKEY_CURRENT_USER\RegFormat] "Two Not Equal 3"="2 ≠ 3" "Formulas"=hex(7):32,00,20,00,b5,00,73,00,20,00,d7,00,20,00,33,00,20,00,3d,00,\ 20,00,36,00,20,00,b5,00,73,00,00,00,41,00,20,00,3d,00,20,00,c0,03,20,00,d7,\ 00,20,00,72,00,b2,00,00,00,00,00 The byte stream of this file is displayed in a hexadecimal viewer/editor as: 00000000h: FF FE 57 00 69 00 6E 00 64 00 6F 00 77 00 73 00 ; ÿþW.i.n.d.o.w.s. 00000010h: 20 00 52 00 65 00 67 00 69 00 73 00 74 00 72 00 ; .R.e.g.i.s.t.r. 00000020h: 79 00 20 00 45 00 64 00 69 00 74 00 6F 00 72 00 ; y. .E.d.i.t.o.r. 00000030h: 20 00 56 00 65 00 72 00 73 00 69 00 6F 00 6E 00 ; .V.e.r.s.i.o.n. 00000040h: 20 00 35 00 2E 00 30 00 30 00 0D 00 0A 00 0D 00 ; .5...0.0....... 00000050h: 0A 00 5B 00 48 00 4B 00 45 00 59 00 5F 00 43 00 ; ..[.H.K.E.Y._.C. 00000060h: 55 00 52 00 52 00 45 00 4E 00 54 00 5F 00 55 00 ; U.R.R.E.N.T._.U. 00000070h: 53 00 45 00 52 00 5C 00 52 00 65 00 67 00 46 00 ; S.E.R.\.R.e.g.F. 00000080h: 6F 00 72 00 6D 00 61 00 74 00 5D 00 0D 00 0A 00 ; o.r.m.a.t.]..... 00000090h: 22 00 54 00 77 00 6F 00 20 00 4E 00 6F 00 74 00 ; ".T.w.o. .N.o.t. 000000a0h: 20 00 45 00 71 00 75 00 61 00 6C 00 20 00 33 00 ; .E.q.u.a.l. .3. 000000b0h: 22 00 3D 00 22 00 32 00 20 00 60 22 20 00 33 00 ; ".=.".2. .`" .3. 000000c0h: 22 00 0D 00 0A 00 22 00 46 00 6F 00 72 00 6D 00 ; ".....".F.o.r.m. 000000d0h: 75 00 6C 00 61 00 73 00 22 00 3D 00 68 00 65 00 ; u.l.a.s.".=.h.e. 000000e0h: 78 00 28 00 37 00 29 00 3A 00 33 00 32 00 2C 00 ; x.(.7.).:.3.2.,. 000000f0h: 30 00 30 00 2C 00 32 00 30 00 2C 00 30 00 30 00 ; 0.0.,.2.0.,.0.0. 00000100h: 2C 00 62 00 35 00 2C 00 30 00 30 00 2C 00 37 00 ; ,.b.5.,.0.0.,.7. 00000110h: 33 00 2C 00 30 00 30 00 2C 00 32 00 30 00 2C 00 ; 3.,.0.0.,.2.0.,. 00000120h: 30 00 30 00 2C 00 64 00 37 00 2C 00 30 00 30 00 ; 0.0.,.d.7.,.0.0. 00000130h: 2C 00 32 00 30 00 2C 00 30 00 30 00 2C 00 33 00 ; ,.2.0.,.0.0.,.3. 00000140h: 33 00 2C 00 30 00 30 00 2C 00 32 00 30 00 2C 00 ; 3.,.0.0.,.2.0.,. 00000150h: 30 00 30 00 2C 00 33 00 64 00 2C 00 30 00 30 00 ; 0.0.,.3.d.,.0.0. 00000160h: 2C 00 5C 00 0D 00 0A 00 20 00 20 00 32 00 30 00 ; ,.\..... . .2.0. 00000170h: 2C 00 30 00 30 00 2C 00 33 00 36 00 2C 00 30 00 ; ,.0.0.,.3.6.,.0. 00000180h: 30 00 2C 00 32 00 30 00 2C 00 30 00 30 00 2C 00 ; 0.,.2.0.,.0.0.,. 00000190h: 62 00 35 00 2C 00 30 00 30 00 2C 00 37 00 33 00 ; b.5.,.0.0.,.7.3. 000001a0h: 2C 00 30 00 30 00 2C 00 30 00 30 00 2C 00 30 00 ; ,.0.0.,.0.0.,.0. 000001b0h: 30 00 2C 00 34 00 31 00 2C 00 30 00 30 00 2C 00 ; 0.,.4.1.,.0.0.,. 000001c0h: 32 00 30 00 2C 00 30 00 30 00 2C 00 33 00 64 00 ; 2.0.,.0.0.,.3.d. 000001d0h: 2C 00 30 00 30 00 2C 00 32 00 30 00 2C 00 30 00 ; ,.0.0.,.2.0.,.0. 000001e0h: 30 00 2C 00 63 00 30 00 2C 00 30 00 33 00 2C 00 ; 0.,.c.0.,.0.3.,. 000001f0h: 32 00 30 00 2C 00 30 00 30 00 2C 00 64 00 37 00 ; 2.0.,.0.0.,.d.7. 00000200h: 2C 00 5C 00 0D 00 0A 00 20 00 20 00 30 00 30 00 ; ,.\..... . .0.0. 00000210h: 2C 00 32 00 30 00 2C 00 30 00 30 00 2C 00 37 00 ; ,.2.0.,.0.0.,.7. 00000220h: 32 00 2C 00 30 00 30 00 2C 00 62 00 32 00 2C 00 ; 2.,.0.0.,.b.2.,. 00000230h: 30 00 30 00 2C 00 30 00 30 00 2C 00 30 00 30 00 ; 0.0.,.0.0.,.0.0. 00000240h: 2C 00 30 00 30 00 2C 00 30 00 30 00 0D 00 0A 00 ; ,.0.0.,.0.0..... 00000250h: 0D 00 0A 00 ; .... The first 2 bytes being displayed with code page Windows-1252 as ÿþ are the two bytes of the UTF-16 LE BOM which is not displayed by text editors in text editing mode. But exporting key HKCU\RegFormat in format REGEDIT4 by clicking on file type drop down list in export dialog and selecting Win9x/NT4 registry files results in an ANSI (more precise: Windows-1252) encoded text file with just 180 bytes with content: REGEDIT4 [HKEY_CURRENT_USER\RegFormat] "Two Not Equal 3"="2 ? 3" "Formulas"=hex(7):32,20,b5,73,20,d7,20,33,20,3d,20,36,20,b5,73,00,41,20,3d,20,\ c0,20,d7,20,72,b2,00,00 The byte stream of this file is displayed in a hexadecimal viewer/editor as: 00000000h: 52 45 47 45 44 49 54 34 0D 0A 0D 0A 5B 48 4B 45 ; REGEDIT4....[HKE 00000010h: 59 5F 43 55 52 52 45 4E 54 5F 55 53 45 52 5C 52 ; Y_CURRENT_USER\R 00000020h: 65 67 46 6F 72 6D 61 74 5D 0D 0A 22 54 77 6F 20 ; egFormat].."Two 00000030h: 4E 6F 74 20 45 71 75 61 6C 20 33 22 3D 22 32 20 ; Not Equal 3"="2 00000040h: 3F 20 33 22 0D 0A 22 46 6F 72 6D 75 6C 61 73 22 ; ? 3".."Formulas" 00000050h: 3D 68 65 78 28 37 29 3A 33 32 2C 32 30 2C 62 35 ; =hex(7):32,20,b5 00000060h: 2C 37 33 2C 32 30 2C 64 37 2C 32 30 2C 33 33 2C ; ,73,20,d7,20,33, 00000070h: 32 30 2C 33 64 2C 32 30 2C 33 36 2C 32 30 2C 62 ; 20,3d,20,36,20,b 00000080h: 35 2C 37 33 2C 30 30 2C 34 31 2C 32 30 2C 33 64 ; 5,73,00,41,20,3d 00000090h: 2C 32 30 2C 5C 0D 0A 20 20 63 30 2C 32 30 2C 64 ; ,20,\.. c0,20,d 000000a0h: 37 2C 32 30 2C 37 32 2C 62 32 2C 30 30 2C 30 30 ; 7,20,72,b2,00,00 000000b0h: 0D 0A 0D 0A ; .... What are the differences? REGEDIT first converted the string of type REG_SZ from Unicode to code page Windows-1252 and then exported the data. The result is character ? instead of not equal sign with Unicode value U+2260 in string of type REG_SZ because this character is not available in code page Windows-1252. The multi-line string has just halve of the hexadecimal values (just 1 byte per character instead of 2 bytes per character). The small PI with Unicode value U+03C0 is stored with value C0 (just low byte) and is therefore now the character À as this character has code value 0xC0 in code page Windows-1252. Importing the data from file in format Windows Registry Editor Version 5.00 results in getting same data in Windows registry as before. But importing the data from file in format REGEDIT4 results in: A string of type REG_SZ with name Two Not Equal 3 with ANSI string value: 2 ? 3 A multi-line string of type REG_MULTI_SZ with name Formulas with Unicode string value: 2 µs × 3 = 6 µs A = À × r² The characters µ and × and ² have a code value < 255 (decimal) and can be therefore correct stored also in registry file of format REGEDIT4. But ≠ and π can't be correct encoded in format 4. A registry file of format Windows Registry Editor Version 5.00 must not be itself a Unicode encoded text file. It is also possible to have a Windows-1252 encoded registry file with content: Windows Registry Editor Version 5.00 [HKEY_CURRENT_USER\RegFormat] "Two Not Equal 3"=hex(1):32,00,60,22,33,00 "Formulas"=hex(7):32,00,20,00,b5,00,73,00,20,00,d7,00,20,00,33,00,20,00,3d,00,\ 20,00,36,00,20,00,b5,00,73,00,00,00,41,00,20,00,3d,00,20,00,c0,03,20,00,d7,\ 00,20,00,72,00,b2,00,00,00,00,00 being displayed in a hexadecimal viewer/editor as: 00000000h: 57 69 6E 64 6F 77 73 20 52 65 67 69 73 74 72 79 ; Windows Registry 00000010h: 20 45 64 69 74 6F 72 20 56 65 72 73 69 6F 6E 20 ; Editor Version 00000020h: 35 2E 30 30 0D 0A 0D 0A 5B 48 4B 45 59 5F 43 55 ; 5.00....[HKEY_CU 00000030h: 52 52 45 4E 54 5F 55 53 45 52 5C 52 65 67 46 6F ; RRENT_USER\RegFo 00000040h: 72 6D 61 74 5D 0D 0A 22 54 77 6F 20 4E 6F 74 20 ; rmat].."Two Not 00000050h: 45 71 75 61 6C 20 33 22 3D 68 65 78 28 31 29 3A ; Equal 3"=hex(1): 00000060h: 33 32 2C 30 30 2C 36 30 2C 32 32 2C 33 33 2C 30 ; 32,00,60,22,33,0 00000070h: 30 0D 0A 22 46 6F 72 6D 75 6C 61 73 22 3D 68 65 ; 0.."Formulas"=he 00000080h: 78 28 37 29 3A 33 32 2C 30 30 2C 32 30 2C 30 30 ; x(7):32,00,20,00 00000090h: 2C 62 35 2C 30 30 2C 37 33 2C 30 30 2C 32 30 2C ; ,b5,00,73,00,20, 000000a0h: 30 30 2C 64 37 2C 30 30 2C 32 30 2C 30 30 2C 33 ; 00,d7,00,20,00,3 000000b0h: 33 2C 30 30 2C 32 30 2C 30 30 2C 33 64 2C 30 30 ; 3,00,20,00,3d,00 000000c0h: 2C 5C 0D 0A 20 20 32 30 2C 30 30 2C 33 36 2C 30 ; ,\.. 20,00,36,0 000000d0h: 30 2C 32 30 2C 30 30 2C 62 35 2C 30 30 2C 37 33 ; 0,20,00,b5,00,73 000000e0h: 2C 30 30 2C 30 30 2C 30 30 2C 34 31 2C 30 30 2C ; ,00,00,00,41,00, 000000f0h: 32 30 2C 30 30 2C 33 64 2C 30 30 2C 32 30 2C 30 ; 20,00,3d,00,20,0 00000100h: 30 2C 63 30 2C 30 33 2C 32 30 2C 30 30 2C 64 37 ; 0,c0,03,20,00,d7 00000110h: 2C 5C 0D 0A 20 20 30 30 2C 32 30 2C 30 30 2C 37 ; ,\.. 00,20,00,7 00000120h: 32 2C 30 30 2C 62 32 2C 30 30 2C 30 30 2C 30 30 ; 2,00,b2,00,00,00 00000130h: 2C 30 30 2C 30 30 0D 0A 0D 0A ; ,00,00.... Importing this registry file produces nevertheless in Windows registry the Unicode strings: 2 ≠ 3 2 µs × 3 = 6 µs A = À × r² Using binary representation hex(1) for a value of type REG_SZ is not standard. This example was created manually by me to demonstrate that it is possible to import a Unicode string of type REG_SZ also with a non Unicode encoded registry file which is easier to produce from within a batch file. But if the name of a key or value contains itself a Unicode character with a code value greater U+00FF, it is really necessary to use registry file format Windows Registry Editor Version 5.00 with registry file being also Unicode encoded using 16-bit Unicode Transformation Format with little-endian format. See function SHRegWriteUSValue for hex(?) ... REG_? registry type table. By the way: I wrote once the UltraEdit script GetStringFromRegHex.js to get the hexadecimal values of type hex(2) ... REG_EXPAND_SZ and hex(7) ... REG_MULTI_SZ selected in a registry file opened in text editor UltraEdit displayed as text because the hexadecimal values are really hard to "read as text" for a human. The command line %SystemRoot%\System32\regedit.exe /a /e "%USERPROFILE%\Desktop\EntireWindowsRegistry.reg" exports the entire Windows registry to Windows desktop in ANSI using registry file format 4. The same command line without option /a results in an export of entire Windows registry in Unicode using registry file format 5.
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts]"Segoe UI (TrueType)"="""Segoe UI Bold (TrueType)"="""Segoe UI Bold Italic (TrueType)"="""Segoe UI Italic (TrueType)"="""Segoe UI Light (TrueType)"="""Segoe UI Semibold (TrueType)"="""Segoe UI Symbol (TrueType)"=""[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontSubstitutes]"Segoe UI"="NEW-FONT-NAME"
Visual Leak Detector reporting strange leaks in CRT module of VC++
I've just now installed Visual Leak Detector (2.3) on Windows 8. I tested it with blank CRT program (in Visual Studio 2012) that does nothing.
#include <vld.h>
int main(int argc, char** argv)
{
return 0;
}
When I run it VLD reports strange leaks in vc++ crt module:
Visual Leak Detector Version 2.3 installed.
WARNING: Visual Leak Detector detected memory leaks!
---------- Block 31 at 0x0000000052C07530: 70 bytes ----------
Call Stack:
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdenvp.c (127): my_application.exe!_setenvp + 0x27 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (223): my_application.exe!__tmainCRTStartup + 0x5 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (164): my_application.exe!mainCRTStartup
0x00000000FAF8167E (File and line number not available): KERNEL32.DLL!BaseThreadInitThunk + 0x1A bytes
0x00000000FD8CC3F1 (File and line number not available): ntdll.dll!RtlUserThreadStart + 0x21 bytes
Data:
20 B5 C0 52 50 00 00 00 50 92 C0 52 50 00 00 00 ...RP... P..RP...
20 91 DD E1 F6 07 00 00 7E 00 00 00 02 00 00 00 ........ ~.......
12 00 00 00 00 00 00 00 1F 00 00 00 FD FD FD FD ........ ........
50 52 4F 43 45 53 53 4F 52 5F 4C 45 56 45 4C 3D PROCESSO R_LEVEL=
36 00 FD FD FD FD 6....... ........
---------- Block 40 at 0x0000000052C075D0: 72 bytes ----------
Call Stack:
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdenvp.c (127): my_application.exe!_setenvp + 0x27 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (223): my_application.exe!__tmainCRTStartup + 0x5 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (164): my_application.exe!mainCRTStartup
0x00000000FAF8167E (File and line number not available): KERNEL32.DLL!BaseThreadInitThunk + 0x1A bytes
0x00000000FD8CC3F1 (File and line number not available): ntdll.dll!RtlUserThreadStart + 0x21 bytes
Data:
F0 94 C0 52 50 00 00 00 20 76 C0 52 50 00 00 00 ...RP... .v.RP...
20 91 DD E1 F6 07 00 00 7E 00 00 00 02 00 00 00 ........ ~.......
14 00 00 00 00 00 00 00 28 00 00 00 FD FD FD FD ........ (.......
53 45 53 53 49 4F 4E 4E 41 4D 45 3D 43 6F 6E 73 SESSIONN AME=Cons
6F 6C 65 00 FD FD FD FD ole..... ........
---------- Block 41 at 0x0000000052C07620: 67 bytes ----------
Call Stack:
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdenvp.c (127): my_application.exe!_setenvp + 0x27 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (223): my_application.exe!__tmainCRTStartup + 0x5 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (164): my_application.exe!mainCRTStartup
0x00000000FAF8167E (File and line number not available): KERNEL32.DLL!BaseThreadInitThunk + 0x1A bytes
0x00000000FD8CC3F1 (File and line number not available): ntdll.dll!RtlUserThreadStart + 0x21 bytes
Data:
D0 75 C0 52 50 00 00 00 D0 96 C0 52 50 00 00 00 .u.RP... ...RP...
20 91 DD E1 F6 07 00 00 7E 00 00 00 02 00 00 00 ........ ~.......
0F 00 00 00 00 00 00 00 29 00 00 00 FD FD FD FD ........ ).......
53 79 73 74 65 6D 44 72 69 76 65 3D 43 3A 00 FD SystemDr ive=C:..
FD FD FD ........ ........
---------- Block 43 at 0x0000000052C07670: 65 bytes ----------
Call Stack:
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdenvp.c (127): my_application.exe!_setenvp + 0x27 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (223): my_application.exe!__tmainCRTStartup + 0x5 bytes
f:\dd\vctools\crt_bld\self_64_amd64\crt\src\crt0.c (164): my_application.exe!mainCRTStartup
0x00000000FAF8167E (File and line number not available): KERNEL32.DLL!BaseThreadInitThunk + 0x1A bytes
0x00000000FD8CC3F1 (File and line number not available): ntdll.dll!RtlUserThreadStart + 0x21 bytes
Data:
D0 96 C0 52 50 00 00 00 C0 76 C0 52 50 00 00 00 ...RP... .v.RP...
20 91 DD E1 F6 07 00 00 7E 00 00 00 02 00 00 00 ........ ~.......
0D 00 00 00 00 00 00 00 2B 00 00 00 FD FD FD FD ........ +.......
54 45 4D 50 3D 46 3A 5C 54 45 4D 50 00 FD FD FD TEMP=F:\ TEMP....
FD ........ ........
Visual Leak Detector detected 48 memory leaks (6044 bytes).
Largest number used: 15094 bytes.
Total allocations: 25276 bytes.
Visual Leak Detector is now exiting.
There are not much details about this on net however in this msdn forum a comment says:
That's not really a 'leak' so much as 'preparing your environment for
execution'. It's making a writable copy of the process's environment
for programs which expect it, and they will be released when the
process exits. You can safely ignore the report.
However, I want to suppress these lines from appearing it to report (If they genuinely are not leaks)
Has anyone experienced this and know how to sort out?
Easiest solution apparently is to add StartDisabled=yes in the vld.ini file and then explicitly enable it from the first line of main() . Sure, you'll also miss "memory leaks" from your global objects but that's usually equally harmless.
I found this was bug in VLD 2.3. Details of bug are here. I just downloaded v2.4rc2 (vld-2.4rc2-setup.exe) and this issue not seems appearing anymore. (I had downloaded v2.3 because it has been marked stable)
Trying to extract pixel values from a given PNG image
Trying to understand PNG format.
Consider this PNG Image:
The Image is taken from here
In Hex Editor , it looks like this:
89 50 4E 47 0D 0A 1A 0A 00 00 00 0D 49 48 44 52 00 00 00 80 00 00 00 44 08 02 00 00 00
C6 25 AA 3E 00 00 00 C2 49 44 41 54 78 5E ED D4 81 06 C3 30 14 40 D1 B7 34 DD FF FF 6F
B3 74 56 EA 89 12 6C 28 73 E2 AA 34 49 03 87 D6 FE D8 7B 89 BB 52 8D 3B 87 FE 01 00 80
00 00 10 00 00 02 00 40 00 00 08 00 00 01 00 20 00 00 04 00 80 00 00 10 00 00 02 00 40
00 00 08 00 00 01 00 20 00 00 00 D4 5E 6A 64 4B 94 F5 98 7C D1 F4 92 5C 5C 3E CF 9C 3F
73 71 58 5F AF 8B 79 5B EE 96 B6 47 EB F1 EA D1 CE B6 E3 75 3B E6 B9 95 8D C7 CE 03 39
C9 AF C6 33 93 7B 66 37 CF AB BF F9 C9 2F 08 80 00 00 10 00 00 02 00 40 00 00 08 00 00
01 00 20 00 00 04 00 80 00 00 10 00 00 02 00 40 00 00 08 00 00 01 00 20 00 00 8C 37 DB
68 03 20 FB ED 96 65 00 00 00 00 49 45 4E 44 AE 42 60 82
Equivalent characters:
‰PNG........IHDR...€...D.....Æ%ª>...ÂIDATx^íÔ..Ã0.#Ñ·4Ýÿÿo³tVê‰.l(sâª4I.‡ÖþØ{‰
»R.;‡þ..€.......#....... ....€.......#....... ...Ô^jdK”õ˜|Ñô’\\>Ïœ?sqX_¯
‹y[î–¶GëñêÑζãu;湕.ÇÎ.9ɯÆ3“{f7Ï«¿ùÉ/.€.......#....... ....€.......#....... ..Œ7Ûh.
ûí–e....IEND®B`‚
The same is shown in following Screenshot of the HEX Editor:
I am trying to reverse engineer this image to extract the header part and the RGB pixel values. I read about the PNG and also here , and so far I have noted the following about this Image:
The IHDR chunk must appear FIRST. It contains:
Width: 4 bytes
Height: 4 bytes
Bit depth: 1 byte
Color type: 1 byte
Compression method: 1 byte
Filter method: 1 byte
Interlace method: 1 byte
Below I am starting reading the HEX Data in sequence:
1- First 8-bytes: This is the 8-Byte signature
89 50 4E 47 0D 0A 1A 0A
Equivalently this is : %PNG as can be seen in HEX Editor
A valid PNG image must contain an IHDR chunk, one or more IDAT chunks, and an IEND chunk.
2- Chunk: Length
00 00 00 0D
3-Chunk: Chunk Type
49 48 44 52
Which is IHDR.
http://www.w3.org/TR/PNG-Chunks.html
4- Chunk: Width of the Image (in Decimal 128)
00 00 00 80
5- Chunk: Height of the image (in Decimal 68)
00 00 00 44
6- Chunk: BIT DEPTH (1 byte )
08
7- Chunk: Color Type
02
8- Compression method
00
9- Filter method:
00
10- Interlace method:
00
11- What is the following data?
C6 25 AA 3E 00 00 00 C2
12-- IDAT
49 44 41 54
13- What is this data (after IDAT):
78 5E ED D4 81 06 C3 30 14 40 D1 B7 34 DD FF FF 6F B3 74 56 EA 89 12 6C 28 73 E2 AA 34 49 03 87 D6 FE D8 7B 89 BB 52 8D 3B 87 FE 01 00 80 00 00 10 00 00 02 00 40 00 00 08 00 00 01 00 20 00 00 04 00 80 00 00 10 00 00 02 00 40 00 00 08 00 00 01 00 20 00 00 00 D4 5E 6A 64 4B 94 F5 98 7C D1 F4 92 5C 5C 3E CF 9C 3F 73 71 58 5F AF 8B 79 5B EE 96 B6 47 EB F1 EA D1 CE B6 E3 75 3B E6 B9 95 8D C7 CE 03 39 C9 AF C6 33 93 7B 66 37 CF AB BF F9 C9 2F 08 80 00 00 10 00 00 02 00 40 00 00 08 00 00 01 00 20 00 00 04 00 80 00 00 10 00 00 02 00 40 00 00 08 00 00 01 00 20 00 00 8C 37 DB 68 03 20 FB ED 96 65 00 00 00 00
14- IEND:
49 45 4E 44
15- Last 4 bytes
AE 42 60 82
What are these ?
Can some help me understand, points 11, 13 and 15 above? And where are the Pixel values? The Image is having (128 x 68 pixels)
Purpose of knowing these details:
Once I know these details, I will generate my own 16 bit PNG image. I already have pixel values, so my job would be to introduce headers etc.
I dont know if there is software that can perform this job.
UPDATE
I understand now because of compression, I would not be able to locate the pixel values.
I got the idea that I can write a file in OpenCV and save it as png. Well now my direct question is: I have a binary file having gray-scale 16 bit-pixel values. Can I write this in OpenCV as 16 bit PNG ?
Although it might be interesting to learn about what PNG Images actually are, and how the image is actually represented in the file, you don't need to know this to generate a PNG file. Note that PNG uses lossless compression, which means you won't get two bytes per pixel. You can generate your image in a program and output it in PNG format using many of the libraries that there are out there. For example, you can make your image in OpenCV and then output it with imWrite. One of the parameters can make it output to a PNG. If you have the gray-scale 16-bit pixel values, then you can put them into a Mat. Then convert that to an IplImage: Converting cv::Mat to IplImage* Then you can output it to a file.
Just for completeness (eboix's answer is right on the spot) 11- What is the following data? C6 25 AA 3E 00 00 00 C2 Each chunk ends with a CRC (4 bytes), and starts with 4 bytes that tell its length. So, C6 25 AA 3E is the CRC of the previous chunk (IHDR) and 00 00 00 C2 (194) is the length of the following (IDAT) chunk. In the same way, the last 4 bytes is the CRC of the IEND chunk.
I did not look too carefully but from looking at the structure... Q11. C6 25 AA 3E = CRC32 00 00 00 C2 = Size of next chunk Q13. check the png spec's you referred to earlier that looks like the IDAT chunk you allready know the compression applied to it! Q15. AE 42 60 82 = CRC32