I'm trying to understand how does golang's garbage collector work when the golang code is compiled and I guess when using go run as well. I figure go run is a little more straight forward and just runs the garbage collector along with the .go files you are running. But is the garbage collector compiled into the binaries as well when compiling to an executable?
The compiled object files do not contain any garbage collector "code".
When a program is run with go run, the go command will compile your sources, create and start an executable binary in a temp folder. See below.
When an application is compiled and linked into an executable binary, a go runtime is also included in the executable which is loaded when the binary is started. This runtime provides the garbage collector amongst other services such as runtime reflection and stacktrace information. This is the main reason why a simple Hello World application results in like a 2 MB executable binary.
Related
Compiling an executable with gccgo creates a dynamically linked executable, linked against libgo.so. I am using gccgo-11 on ubuntu 20.04, pulled from apt.
The resulting executable is small, as expected. About 50Kb for a basic hello world.
As soon as I start using external modules (in this case viper and gorilla/websocket) the executable size starts creeping back to the original, statically linked size (~9MB).
If I compile with go alone (not using gccgo) the exeutable is about 9MB also.
This look to me, like the modules are compiled separately, and statically, and then linked, statically into the final executable.
Am I overlooking something here? Is there some option that must be enabled?
I would expect the module dependencies to also be compiled by gccgo, and linked dynamically against the shared libgo.so.
How to directly run a c++ file present in read-only storage like CD-drive without making executable files using g++? There must be some arguments for that to work.
The process of a C/C++ program when you make one till you run it:
You write the program's source code.
The compiler comes in here and compiles the source code to object files.
Note: Remember that the program cannot be executed at this stage. It's only an object file. You'd know this if you have worked on bigger size programs, but if you haven't here is how it works. Remember using those header files in your programs? These header files just tell the compiler that there are some things that are not defined in your program. They are somewhere else. So your compile compiles the program to the object file leaving out things that have a prototype (which is in the header files).
This is a very important point. Here a program called 'linker' comes into play. What linker does is to take all the object files created by compiler and combines them into one. Say for example your compiler created a single object file. Now, you're using math library or anything from standard library. The compiler-linker package (often called only compiler) comes with object files for these standard library definitions. So, linker takes your object file and combines it with other object files from the package and then converts it to an executable file. This is the file that you can run. Nothing else is runnable directly.
To run source code the process is explained already, we have to use the g++. Now
What I understand from your question is that you want to know if a program can be run once it's compiled and linked properly (hence an executable has been generated). Answer to that would be yes.
Alternatively, may sound strange, there is an interpreter I know called Cling that can be of use to bypass the compilation of C++ program.
After all C++ is generally seen as a compiled language. However, any programming language can be implemented as a compiler or as an interpreter and Cling happens to be an interactive C++ interpreter based on LLVM and Clang.
Take a thorough look at this
I know very little about DLL's and LIB's other than that they contain vital code required for a program to run properly - libraries. But why do compilers generate them at all? Wouldn't it be easier to just include all the code in a single executable? And what's the difference between DLL's and LIB's?
There are static libraries (LIB) and dynamic libraries (DLL) - but note that .LIB files can be either static libraries (containing object files) or import libraries (containing symbols to allow the linker to link to a DLL).
Libraries are used because you may have code that you want to use in many programs. For example if you write a function that counts the number of characters in a string, that function will be useful in lots of programs. Once you get that function working correctly you don't want to have to recompile the code every time you use it, so you put the executable code for that function in a library, and the linker can extract and insert the compiled code into your program. Static libraries are sometimes called 'archives' for this reason.
Dynamic libraries take this one step further. It seems wasteful to have multiple copies of the library functions taking up space in each of the programs. Why can't they all share one copy of the function? This is what dynamic libraries are for. Rather than building the library code into your program when it is compiled, it can be run by mapping it into your program as it is loaded into memory. Multiple programs running at the same time that use the same functions can all share one copy, saving memory. In fact, you can load dynamic libraries only as needed, depending on the path through your code. No point in having the printer routines taking up memory if you aren't doing any printing. On the other hand, this means you have to have a copy of the dynamic library installed on every machine your program runs on. This creates its own set of problems.
As an example, almost every program written in 'C' will need functions from a library called the 'C runtime library, though few programs will need all of the functions. The C runtime comes in both static and dynamic versions, so you can determine which version your program uses depending on particular needs.
Another aspect is security (obfuscation). Once a piece of code is extracted from the main application and put in a "separated" Dynamic-Link Library, it is easier to attack, analyse (reverse-engineer) the code, since it has been isolated. When the same piece of code is kept in a LIB Library, it is part of the compiled (linked) target application, and this thus harder to isolate (differentiate) that piece of code from the rest of the target binaries.
One important reason for creating a DLL/LIB rather than just compiling the code into an executable is reuse and relocation. The average Java or .NET application (for example) will most likely use several 3rd party (or framework) libraries. It is much easier and faster to just compile against a pre-built library, rather than having to compile all of the 3rd party code into your application. Compiling your code into libraries also encourages good design practices, e.g. designing your classes to be used in different types of applications.
A DLL is a library of functions that are shared among other executable programs. Just look in your windows/system32 directory and you will find dozens of them. When your program creates a DLL it also normally creates a lib file so that the application *.exe program can resolve symbols that are declared in the DLL.
A .lib is a library of functions that are statically linked to a program -- they are NOT shared by other programs. Each program that links with a *.lib file has all the code in that file. If you have two programs A.exe and B.exe that link with C.lib then each A and B will both contain the code in C.lib.
How you create DLLs and libs depend on the compiler you use. Each compiler does it differently.
One other difference lies in the performance.
As the DLL is loaded at runtime by the .exe(s), the .exe(s) and the DLL work with shared memory concept and hence the performance is low relatively to static linking.
On the other hand, a .lib is code that is linked statically at compile time into every process that requests. Hence the .exe(s) will have single memory, thus increasing the performance of the process.
Came across the following paragraph from a page on the MySQL website here:
You can write plugins in C or C++ (or another language that can use C
calling conventions). Plugins are loaded and unloaded dynamically, so
your operating system must support dynamic loading and you must have
compiled the calling application dynamically (not statically). For
server plugins, this means that mysqld must be compiled
dynamically.
What is meant by dynamical compiling? I know about dynamical linking, but I'm not sure what they meant with dynamical compiling.
Also, on Windows 10 (x64), how can I assure that an exe has been compiled dynamically? Is it possible to figure it out from the output of dumpbin? Here's the dumpbin output for mysqld.exe (version 5.7):
Note: I reviewed this old question which did not provide me with that much information. The depends tool it suggests is no longer on Windows.
Compiling dynamically simply means that you are compiling the code such that the compiled output is suitable for dynamic linking.
On Windows, the process of creating as DLL necessarily compiles it such that it's suitable for dynamic linking because DLLs are always dynamically linked.
I believe that most platforms today always compile dynamically and produce relocatable output, even if they're subsequently linked statically.
I have a command-line executable from which I would like to be able load essentially any dynamic library that contains Objective-C classes. In the Project Settings, I have specified that my program supports garbage collection (without requiring it). To this end, I use dlopen to load the libraries, as it's globally more flexible than NSBundle (first off and most importantly, it can load libraries that aren't in a bundle).
However, when I try to load a library or framework that requires garbage collection, dlopen fails and I get this message:
Unable to open /path/to/object: dlopen(/path/to/lib, 2): no
suitable image found. Did find:
/path/to/lib: GC capability mismatch
If my program supports garbage collection, why can't it load libraries that require it? How can I load libraries that require it?
I can't really flag my executable as requiring garbage collection because it will stop working with libraries that don't support it.
Right now I have two versions of my executable, but that's sub-par.
Additional informations: I target Mountain Lion only. My program is a universal binary and it currently handles relaunching as its 32-bits counterpart using the posix_spawn function, however there seems to be no flag controlling garbage collection there.
To load a Garbage Collected library, your executable must be running in a garbage-collected environment to start with. You'll need to either:
Always run under GC
Detect that the library requires GC and relaunch using GC, then retry loading the library. Kinda like how System Preferences will relaunch itself for 32bit panes.
I found the solution to my problem. The Objective-C runtime has several switches triggered by environment variables, including OBJC_DISABLE_GC that can turn off the garbage collector even if the executable wants it on. This switch is taken into account when new images are loaded (see gc_enforcer and how it uses the UseGC static variable), so turning the GC off will actually affect the way libraries are loaded into the process.
My solution, then, is to implement proper retain/release logic into my executable but compile it with -fobjc-gc-only. I then try to load the library normally; if it works, everything's good. If it doesn't, I programmatically relaunch the process with the same arguments, plus OBJC_DISABLE_GC=YES in its environment, so the GC is disabled and the library successfully loads this time.
This snippet excludes error handling.
char disableGC[] = "OBJC_DISABLE_GC=YES";
// resets the current process with the given argv and enables the GC as per useGC
void respawn(char* const argv[], bool useGC)
{
posix_spawnattr_t attributes;
posix_spawnattr_init(&attributes);
posix_spawnattr_setflags(&attributes, POSIX_SPAWN_SETEXEC)
// build the environment variables array
std::vector<char*> environment;
for (char** environ_iter = environ; *environ_iter != 0; environ_iter++)
environment.push_back(*environ_iter);
// disable the GC if we need it off
if (!useGC)
{
environment.push_back(disableGC);
}
environment.push_back(nullptr);
pid_t child;
posix_spawn(&child, argv[0], nullptr, &attributes, argv, environment.data());
}