When I am building a native image of my application with quarkus, I get an executable that has a size of 150MB (the associated jar has a size of 12MB I doubt that Substrate VM make up for all that space). Is this normal or am I doing something wrong ? Is there a way to investigate like logging what is loaded in the native image ?
$ du -sh target/placeholder-1.0.0-SNAPSHOT-runner 1.9m mar. 06 juil. 2021 00:13:58
150M target/placeholder-1.0.0-SNAPSHOT-runner
$ du -sh target/placeholder-1.0.0-SNAPSHOT.jar mar. 06 juil. 2021 00:14:32
12M target/placeholder-1.0.0-SNAPSHOT.jar
Yes, the size of the binary can be larger than the jar file.
It mostly consists of 2 parts: code of your application compiled to binary and the "image heap" the preinitialized data structures and components of your application.
For example, if your application initializes any classes at build time, then the values of the static fields of these classes are serialized into the image heap. This can help startup of the app, because it doesn't need to initialize these again, but it can also make the binary larger.
There are reporting options you can enable for the native image build process and tools that help you make sense of the contents of the image.
From the article linked above, you can use the -H:+DashboardAll option and the dashboard tool here (it is hosted on github and works offline): https://www.graalvm.org/docs/tools/dashboard/?ojr=dashboard
And then it can visualize what takes space for example like this:
https://miro.medium.com/max/2400/1*mWIhq53ALPiI2GP-IQkYoQ.png
Related
I've been generating payloads on Metasploit and I've been experimenting with the different templates and one of the templates you can have your payload as is exe-small. The type of payload I've been generating is a windows/meterpreter/reverse_tcp and just using the normal exe template it has a file size around 72 KB however exe-small outputs a payload the size of 2.4kb. Why is this? And how could I apply this to my programming?
The smallest possible PE file is just 97 bytes - and it does nothing (just return).
The smallest runnable executable today is 133 bytes, because Windows requires kernel32 being loaded. Executing a PE file with no imports is not possible.
At that size it can already download payload from the Internet by specifying an UNC path in the import table.
To achieve such a small executable, you have to
implement in assembler, mainly to get rid of the C runtime
decrease the file alignment which is 1024 by default
remove the DOS stub that prints the message "This program cannot be run in DOS mode"
Merge some of the PE parts into the MZ header
Remove the data directory
The full description is available in a larger research blog post called TinyPE.
For EXE's this small, the most space typically is used for the icon. Typically the icon has various sizes and color schemes contained, which you could get rid of, if you do not care having an "old, rusty" icon, or no icon at all.
There is also some 4k of space used, when you sign the EXE.
As an example for a small EXE, see never10 by grc. There is a details page which highlights the above points:
https://www.grc.com/never10/details.htm
in the last paragraph:
A final note: I'm a bit annoyed that “Never10” is as large as it is at
85 kbyte. The digital signature increases the application's size by
4k, but the high-resolution and high-color icons Microsoft now
requires takes up 56k! So without all that annoying overhead, the app
would be a respectable 25k. And, yes, of course I wrote it in
assembly language.
Disclaimer: I am not affiliated with grc in any way.
The is little need for an executable to be big, except when it contains what I call code spam, code not actually critical to the functionality of the program/exe. This is valid for other files too. Look at a manually written HTML page compared to one written in FrontPage. That's spamcode.
I remember my good old DOS files that were all KB in size and were performing practically any needed task in the OS. One of my .exes (actually .com) was only 20 bytes in size.
Just think of it this way: just as in some situations a large majority of the files contained in a Windows OS can be removed and still the OS can function perfectly, it's the same with the .exe files: large parts of the code is either useless, or has different than relevant-to-objective purpose or are intentionally added (see below).
The peak of this aberration is the code added nowdays in the .exe files of some games that use advanced copy protection, which can make the files as large as dozens of MB. The actually code needed to run the game is practically under 10% of the full code.
A file size of 72 KB as in your example can be pretty sufficient to do practically anything to a windows OS.
To apply this to your programming, as in make very small .exes, keep things simple. Don't add unnecessary code just for the looks of it or by thinking you will use that part of the program/code at a point.
I'm currently trying to get information about the CPU load and RAM usage out of an PowerPC with QNX running on it. The idea is to write that information in a text file with a time stamp over a certain amount of time, but this ain't my problem here once I have the information as a "standard value". My programm will be in C++ and I already did this kind of program for Windows (via PDH API). Maybe you have page like this but for QNX? Probably I'm looking for the wrong keywords.
Can you help me with this problem? Any kind of direction would be most welcome as I'm new to QNX and this kind of programming. Thanks a lot!
You will work with the /proc filesystem.
From the command line you can check the size of the memory space of the process that has process ID = 1234 by:
ls -l /proc/1234/as
"as" stands for "address space" and the size of this virtual file will indicate a good estimate of the memory used by the process in question, 1236992 bytes in this example:
-rw-r--r-- 1 root root 1236992 Aug 21 21:25 as
To get the same value programmatically you will need to use the stat() function on the /proc/PID/as file.
You can refer the following page in the documentation for a more detailed explanation of the same:
http://www.qnx.com/developers/docs/660/index.jsp?topic=%2Fcom.qnx.doc.neutrino.cookbook%2Ftopic%2Fs3_procfs_pid_directories.html
In order to get the CPU time (system/user) used by the process you can use the DCMD_PROC_INFO devctly() on the /proc/PID/as file. You will need to refer the "utime" and "stime" members of the debug_process_t structure passed to the devctl().
You can find a detailed explanation and sample code on the following page in the QNX documentation:
http://www.qnx.com/developers/docs/660/index.jsp?topic=%2Fcom.qnx.doc.neutrino.cookbook%2Ftopic%2Fs3_procfs_DCMD_PROC_INFO.html
I am using Wp7IsolatedStorageExplorer to get some small files (say max 20 bytes) stored in IsolatedStorage.But every time I download file the Filesize remains 12288 bytes(for small files) .Is IsolatedStorageExplorer appending something at last OR Is it How small Files are stored in IsolatedStorage by default ?
Thanks
vaysage.
Initially I thought this might be an indication of the underlying FAT implementation.
However, having looked a bit deeper and having looked at your answers to comments, my guess is that this is just a UI issue in the IsolatedStorageExplorer - if you look at the source http://wp7explorer.codeplex.com/SourceControl/changeset/view/63791#1114123 - then it seems to use 12288 as a chunk size for its networking layers.
I have created an image, using Platform Builder, for Windows CE6.
As per the legal agreement, I then 'licensed' the NK.BIN image file, again using Platform Builder, using a purchased run-time key that came with 100 licenses.
How can I test that this process has worked?
What is 'different' in the image? Is there a command/action that can be performed to identify a legally stamped CE6 image?
If you are trying to determine if the NK.bin has been built with the correct PID then Stampbin should do the trick:
http://msdn.microsoft.com/en-us/library/ee504718.aspx
Otherwise a search on Viewbin seems to be positive as well:
http://msdn.microsoft.com/en-us/library/ms938075.aspx:
You can use Viewbin.exe tool like "viewbin -t nk.bin" which will give an output comprised of PID[0] - PID[9]. If all these values are 0x00000000 then it means your runtime image is not stamped, otherwise it is stamped.
I have a VS 2005 application using C++ . It basically importing a large XML of around 9 GB into the application . After running for more than 18 hrs it gave an exception 0xc0000006 In page error. THe virtual memory consumed is 2.6 GB (I have set the 3GB) flag.
Does any one have a clue as to what caused this error and what could be the solution
Instead of loading the whole file into the memory you can use SAX parsers to load only a part of the file to the memory.
9Gb seems overly large to read in. I would say that even 3Gb is too large in one go.
Is your OS 64bit?
What is the maximum pagefile size set to?
How much RAM do you have?
Were you running this in debug or release mode?
I would suggest that you try to reading the XML in smaller chunks.
Why are you trying to read in such a large file in one go?
I would imagine that your application took so long to run before failing as it started to copy the file into virtual memory, which is basically a large file on the hard disk. Thus the OS is reading the XML from the disk and writing it back onto a different area of disk.
**Edit - added text below **
Having had a quick peek at Expat XML parser it does look as if you're running into problems with stack or event handling, most likely you are adding too much to the stack.
Do you really need 3Gb of data on the stack? At a guess I would say that you are trying to process a XML database file, but I can't imagine that you have a table row that is so large.
I think that really you should use it to search for key areas and discard what is not wanted.
I know nothing other than what I have just read about Expat XML Parser but would suggest that you are not using it in the most efficient manner.