I want to try making a simple game engine. Just something that handles states, assets, characters/actors and their stats and an inventory. Most of the code I can take from other games I've wrote, but I'm confused on how I then turn it into a static library. Do I need a main.cpp? If so what has to go in it? Under Linux I'm guessing I compile it to .so and add the headers to my include directory and then just link to the .so but what do I do on Windows and Mac?
A .so is not a static library, it's a dynamic one. A static library is, in its most basic, a .o file compiled from a single C file, or a .a file which is simply a collection of .o files.
A static library is different from a shared one in that the object code is linked directly in to the final executable, requiring no dependencies at run time.
Under Unix, the ar(1) command is used to bundle .o files in to a composite .a file. I do not know the comparable utility for Windows.
Once you have the .a file, you will simply need the combination of the .a file and the .h files to build your code. You use the .h files for compiling, and then link against the .a file.
Shared libraries have a specific advantage over static libraries in that if you have multiple, yet different, programs relying on the same libraries, the code from the shared libraries can be shared among all of the programs at the same time, so in that sense they lower the overall impact on the system. Their downside is slower start up times (though that's pretty marginal nowadays). Statically linked libraries can not be shared across independent programs, but if you run the same executable several times, its code will be shared.
Related
I'm building an iOS project that includes a sub-project whose symbols I would like exclude from the product's .dSYM DWARF file.
The situation is that the sub-project (a static library) contains valuable proprietary code that I would not want an attacker to be able to symbolicate, even if they had the dSYM files used for resymbolicate crash reports for the whole app. The subproject covers a very specific domain and is well tested independently, so I'm not worried about being unable to resymbolicate stack traces in that code. However, I do need to be able to resymbolicate crash reports for the rest of the app, so I need a dSYM (as distributing symbols with the app is not an option).
I've already managed to make sure that all of the relevant symbols are stripped from the binary, and setting GCC_GENERATE_DEBUGGING_SYMBOLS=NO removed a lot from the dSYM, but I'm still seeing class-private C++ method names inside the dSYM file. For reference, I'm using clang.
How could I produce a dSYM for my app without compromising the symbols of this sub-project?
With a bog-standard Xcode workflow, this might be difficult. You could probably do something with a shell script phase which moves the static library to a different filename ("hides" it) and then runs dsymutil on your main app binary to create a dSYM. Because dsymutil can't find the static library, it won't be able to include any debug information for those functions. Alternatively, you can create a no-debug-info version of the static library although this will take a little bit more scripting. A static library is really a zip file of object (.o) files -- you need to create a directory, extract the .o files (ar x mylib.a), strip the .o files, then create a new static library (ar q mylib-nodebuginfo.a *.o I think) and put that in place before running dsymutil.
I know no on way to selectively remove debug information from a dSYM once it has been created, though. It's possible to do but I don't think anyone has written a tool like that.
I know ways to get CMake to generate both a Static & Dynamic library simultaneously (avoiding the mess of multiple build trees and dual-compiles), but they mainly only work on OSX & Linux (where you get a dynamic .so/.dylib, and a static .a). Windows is a bit messier, since you get a static .lib and a dynamic .dll AND .lib, and the two .lib's have the same name but are different.
In a project I'm converting over right now, that's done manually through lots of batch files and makefiles that just place the resulting files in separate subdirectories. How can I achieve this similar behavior in CMake? (e.g. How can I make CMake build the same binary twice, once static and once dynamic, but place the results into two different subdirectories).
The question is pretty much in the title: in terms of OS-level implementation, how are shared objects and dlls different?
The reason I ask this is because I recently read this page on extending Python, which states:
Unix and Windows use completely different paradigms for run-time loading of code. Before you try to build a module that can be dynamically loaded, be aware of how your system works.
In Unix, a shared object (.so) file contains code to be used by the program, and also the names of functions and data that it expects to find in the program. When the file is joined to the program, all references to those functions and data in the file’s code are changed to point to the actual locations in the program where the functions and data are placed in memory. This is basically a link operation.
In Windows, a dynamic-link library (.dll) file has no dangling references. Instead, an access to functions or data goes through a lookup table. So the DLL code does not have to be fixed up at runtime to refer to the program’s memory; instead, the code already uses the DLL’s lookup table, and the lookup table is modified at runtime to point to the functions and data.
Could anyone elaborate on that? Specifically I'm not sure I understand the description of shared objects containing references to what they expect to find. Similarly, a DLL sounds like pretty much the same mechanism to me.
Is this a complete explanation of what is going on? Are there better ones? Is there in fact any difference?
I am aware of how to link to a DLL or shared object and a couple of mechanisms (.def listings, dllexport/dllimport) for writing DLLs so I'm explicitly not looking for a how to on those areas; I'm more intrigued as to what is going on in the background.
(Edit: another obvious point - I'm aware they work on different platforms, use different file types (ELF vs PE), are ABI-incompatible etc...)
A Dll is pretty much the same mechanism as used by .so or .dylib (MacOS) files, so it is very hard to explain exactly what the differences are.
The core difference is in what is visible by default from each type of file. .so files export the language (gcc) level linkage - which means that (by default) all C & c++ symbols that are "extern" are available for linking when .so's are pulled in.
It also means that, as resolving .so files is essentially a link step, the loader doesn't care which .so file a symbol comes from. It just searches the specified .so files in some order following the usual link step rules that .a files adhere to.
Dll files on the other hand are an Operating system feature, completely separate to the link step of the language. MSVC uses .lib files for linking both static, and dynamic libraries (each dll file generates a paired .lib file that is used for linking) so the resulting program is fully "linked" (from a language centric point of view) once its built.
During the link stage however, symbols were resolved in the lib's that represents the Dlls, allowing the linker to build the import table in the PE file containing an explicit list of dlls and the entry points referenced in each dll. At load time, Windows does not have to perform a "link" to resolving symbols from shared libraries: That step was already done - the windows loader just loads up the dll's and hooks up the functions directly.
I never put much thought into the size difference between a static library and a dynamic library until I downloaded pre-built libraries of boost today. I found that the static libraries of boost are much much bigger than the dynamic libraries.
For example, the debug multi-threaded boost wave static library is 97.7 mb in size while the same library, but dynamic, is only 1.4 mb in size (including import library and dll)! That is a huge difference. Why is that?
Second question, if I statically link against, let's say, the wave library. Does that mean my executable will balloon in size to more than 97.7 mb?
The static libraries have the full debug symbol information in them. For DLLs that information would be in .pdb files (which I assume would be similar in size to the static libs).
When you link to the static lib, the symbol information will not be copied into the .exe - it will be placed in the .pdb file (if your build is configured to create a .pdb file). The .pdb file does not need to be distributed with the .exe, whether or not the .pdb is created.
In the pre-built library download I get from boostpro.com, I don't get .pdb files for the boost DLLs they provide. if you build the DLLs yourself, you'll probably get the .pdb files (though you might have to set some config option, for which I have no idea what the details are).
update:
Looks like I might be wrong about easily getting .pdb files for the boost DLLs. From http://comments.gmane.org/gmane.comp.lib.boost.build/23246:
> Is there an additional option that I can pass on the command line to
> have the (correctly generated) PDB files also copied into the stage
> directory?
Not at this time. You can only hack
tools/build/v2/tools/package.jam to
add <install-type>PDB everywhere where
<install-type>SHARED_LIB or
<install-type>STATIC_LIB is now
written.
No, just because the LIB file is a certain size, doesn't mean it will add that size to your EXE. In fact, most linkers are smart enough to link in only the stuff that's used. Compare that to a dynamic library, which must contain everything.
Static libraries definitely make your EXE larger, but I always prefer it. Then I don't have to worry about missing or incompatible libraries at run time. (Or at least, I minimize the chances of this.)
Since static libraries do not contain finished binary data, but rather information needed for linker to build binary, this information may be bigger than built binaries.
When some function defined in header file is used in cpp-file, compiler puts its code (either inlines, or simply adds) to resulting object file. This means that there will be a lot of duplicates. It's linker's job to merge them, so static library just waits for linker to be reduced :)
Generally size of executable is usually bigger with static libraries, but size of executable together with dynamic libraries is usually smaller. DLL and EXE are linked separately, so linker cannot know which functionality is needed in DLL and which can be thrown out. In case of static library, linker has such information and can take only those obj-files which are used.
The debug static library contains debug information, which explains the huge size difference.
How exactly do DLL files work? There seems to be an awful lot of them, but I don't know what they are or how they work.
So, what's the deal with them?
What is a DLL?
Dynamic Link Libraries (DLL)s are like EXEs but they are not directly executable. They are similar to .so files in Linux/Unix. That is to say, DLLs are MS's implementation of shared libraries.
DLLs are so much like an EXE that the file format itself is the same. Both EXE and DLLs are based on the Portable Executable (PE) file format. DLLs can also contain COM components and .NET libraries.
What does a DLL contain?
A DLL contains functions, classes, variables, UIs and resources (such as icons, images, files, ...) that an EXE, or other DLL uses.
Types of libraries:
On virtually all operating systems, there are 2 types of libraries. Static libraries and dynamic libraries. In windows the file extensions are as follows: Static libraries (.lib) and dynamic libraries (.dll). The main difference is that static libraries are linked to the executable at compile time; whereas dynamic linked libraries are not linked until run-time.
More on static and dynamic libraries:
You don't normally see static libraries though on your computer, because a static library is embedded directly inside of a module (EXE or DLL). A dynamic library is a stand-alone file.
A DLL can be changed at any time and is only loaded at runtime when an EXE explicitly loads the DLL. A static library cannot be changed once it is compiled within the EXE.
A DLL can be updated individually without updating the EXE itself.
Loading a DLL:
A program loads a DLL at startup, via the Win32 API LoadLibrary, or when it is a dependency of another DLL. A program uses the GetProcAddress to load a function or LoadResource to load a resource.
Further reading:
Please check MSDN or Wikipedia for further reading. Also the sources of this answer.
What is a DLL?
DLL files are binary files that can contain executable code and resources like images, etc. Unlike applications, these cannot be directly executed, but an application will load them as and when they are required (or all at once during startup).
Are they important?
Most applications will load the DLL files they require at startup. If any of these are not found the system will not be able to start the process at all.
DLL files might require other DLL files
In the same way that an application requires a DLL file, a DLL file might be dependent on other DLL files itself. If one of these DLL files in the chain of dependency is not found, the application will not load. This is debugged easily using any dependency walker tools, like Dependency Walker.
There are so many of them in the system folders
Most of the system functionality is exposed to a user program in the form of DLL files as they are a standard form of sharing code / resources. Each functionality is kept separately in different DLL files so that only the required DLL files will be loaded and thus reduce the memory constraints on the system.
Installed applications also use DLL files
DLL files also becomes a form of separating functionalities physically as explained above. Good applications also try to not load the DLL files until they are absolutely required, which reduces the memory requirements. This too causes applications to ship with a lot of DLL files.
DLL Hell
However, at times system upgrades often breaks other programs when there is a version mismatch between the shared DLL files and the program that requires them. System checkpoints and DLL cache, etc. have been the initiatives from M$ to solve this problem. The .NET platform might not face this issue at all.
How do we know what's inside a DLL file?
You have to use an external tool like DUMPBIN or Dependency Walker which will not only show what publicly visible functions (known as exports) are contained inside the DLL files and also what other DLL files it requires and which exports from those DLL files this DLL file is dependent upon.
How do we create / use them?
Refer the programming documentation from your vendor. For C++, refer to LoadLibrary in MSDN.
Let’s say you are making an executable that uses some functions found in a library.
If the library you are using is static, the linker will copy the object code for these functions directly from the library and insert them into the executable.
Now if this executable is run it has every thing it needs, so the executable loader just loads it into memory and runs it.
If the library is dynamic the linker will not insert object code but rather it will insert a stub which basically says this function is located in this DLL at this location.
Now if this executable is run, bits of the executable are missing (i.e the stubs) so the loader goes through the executable fixing up the missing stubs. Only after all the stubs have been resolved will the executable be allowed to run.
To see this in action delete or rename the DLL and watch how the loader will report a missing DLL error when you try to run the executable.
Hence the name Dynamic Link Library, parts of the linking process is being done dynamically at run time by the executable loader.
One a final note, if you don't link to the DLL then no stubs will be inserted by the linker, but Windows still provides the GetProcAddress API that allows you to load an execute the DLL function entry point long after the executable has started.
DLLs (dynamic link libraries) and SLs (shared libraries, equivalent under UNIX) are just libraries of executable code which can be dynamically linked into an executable at load time.
Static libraries are inserted into an executable at compile time and are fixed from that point. They increase the size of the executable and cannot be shared.
Dynamic libraries have the following advantages:
1/ They are loaded at run time rather than compile time so they can be updated independently of the executable (all those fancy windows and dialog boxes you see in Windows come from DLLs so the look-and-feel of your application can change without you having to rewrite it).
2/ Because they're independent, the code can be shared across multiple executables - this saves memory since, if you're running 100 apps with a single DLL, there may only be one copy of the DLL in memory.
Their main disadvantage is advantage #1 - having DLLs change independent your application may cause your application to stop working or start behaving in a bizarre manner. DLL versioning tend not to be managed very well under Windows and this leads to the quaintly-named "DLL Hell".
DLL files contain an Export Table which is a list of symbols which can be looked up by the calling program. The symbols are typically functions with the C calling convention (__stcall). The export table also contains the address of the function.
With this information, the calling program can then call the functions within the DLL even though it did not have access to the DLL at compile time.
Introducing Dynamic Link Libraries has some more information.
http://support.microsoft.com/kb/815065
A DLL is a library that contains code
and data that can be used by more than
one program at the same time. For
example, in Windows operating systems,
the Comdlg32 DLL performs common
dialog box related functions.
Therefore, each program can use the
functionality that is contained in
this DLL to implement an Open dialog
box. This helps promote code reuse and
efficient memory usage.
By using a DLL, a program can be
modularized into separate components.
For example, an accounting program may
be sold by module. Each module can be
loaded into the main program at run
time if that module is installed.
Because the modules are separate, the
load time of the program is faster,
and a module is only loaded when that
functionality is requested.
Additionally, updates are easier to
apply to each module without affecting
other parts of the program. For
example, you may have a payroll
program, and the tax rates change each
year. When these changes are isolated
to a DLL, you can apply an update
without needing to build or install
the whole program again.
http://en.wikipedia.org/wiki/Dynamic-link_library
DLL is a File Extension & Known As “dynamic link library” file format used for holding multiple codes and procedures for Windows programs. Software & Games runs on the bases of DLL Files; DLL files was created so that multiple applications could use their information at the same time.
IF you want to get more information about DLL Files or facing any error read the following post.
https://www.bouncegeek.com/fix-dll-errors-windows-586985/
DLLs (Dynamic Link Libraries) contain resources used by one or more applications or services. They can contain classes, icons, strings, objects, interfaces, and pretty much anything a developer would need to store except a UI.
According to Microsoft
(DLL) Dynamic link libraries are files that contain data, code, or resources needed for the running of applications. These are files that are created by the windows ecosystem and can be shared between two or more applications.
When a program or software runs on Windows, much of how the application works depends on the DLL files of the program. For instance, if a particular application had several modules, then how each module interacts with each other is determined by the Windows DLL files.
If you want detailed explanation, check these useful resources
What are dll files , About Dll files