Case in point, a have a program that needs to be able to load dynamic libraries from runtime created arbitrary source files. The program is licensed under GPL. The program should ship as a package free of any external dependencies, ie. run out of an unzipped package.
Ideally, it should work as a portable, platform independent IDE (different binaries for different targets of course). Targets in mind are OSX 10.8++ and Windows, both in 32 and 64 bit formats. I need it to support C and C++ only.
I envision it as GCC placed in a subdirectory and invoked from a platform-supported script (like .bat files and bash etc.) with the goal of producing a dynamic library that can be loaded right away (ie. a DLL or a dylib) - through a classic commandline. The script will be fed with input / output files and include directories.
Can i use GCC for this - and how can i obtain/create such a package? Would this be legal from a license context? How about the standard library source files?
Any help with this project is hugely welcomed.
Related
Looks like there are tons of webpages online and questions on SO, but I still can't get this to work after several hours of searching.
I'm cross-compiling from Linux to Windows by setting CC/CXX to x86-64 MinGW-w64. Compilation actually works and the exe runs fine on Windows.
With file I get this output:
PE32+ executable (GUI) x86-64 (stripped to external PDB), for MS Windows
I have two resource files (a Windows *.ico file and a *.res ASCII file that contains properties such as VERSIONINFO and the like).
The two resource files are transformed into *.o files with MinGW's windres tool. They look like COFF files now. That's as far as I could get.
What I want to do is link those two resources into the exe.
I don't find any way to instruct the Go linker to include those files during linking (I run go build, not go tool). I've also tried to add the resource post-linking with objcopy following various pages online and other SO questions... but to no avail.
All I want is Windows to recognize those two resources (icon and info) so that the executable behaves like any other on Windows.
If I should instruct the Go linker (go tool instead of go build), then how can I tell it which files to link? I have "simple" *.go files and then C dependencies (GLFW and OpenGL) I link with CGO.
How can this be done on the Linux command-line? I want to add these steps to a broader CI/CD (build workflow/asset pipeline). A Windows GUI tool or similar would not help much.
Thanks!
I don’t know if you are using Fyne, but it sounds like what “fyne package” does (embed icon and metadata).
If you are in Fyne land then you can use their tools, if not maybe check out the source and see for yourself how it’s done? https://github.com/fyne-io/fyne/blob/master/cmd/fyne/internal/commands/package-windows.go
Forgive me before I start, as I'm not a C / C++ etc programmer, a mere PHP one :) but I've been working on projects that use some others sourced from online open source repos, such as svn and git. For some of these projects, I need to install libraries and then run "./configure", "make" and then "make all" (as an example) and I do this on a "build" virtual machine to get the binaries that I need to use within my project.
The ultimate goal of some of my projects is to then take these "compiled" (if that's the correct term) binaries and place them onto a virtual machine which I would then re-distribute (according to licenses etc).
My question is this : when I build these binaries on my build machine, with all the pre-requisities that I need in order to build them in the first place ("build-essential" and "cmake" and "gcc" etc etc) - once the binaries are on my build machine (in /usr/lib for example) are they self-contained to the point that I can merely copy those /usr/lib binary files that the build created and place them in the same folder on the virtual machines that I would distribute, without the build servers having all the build components installed on them?
With all the dependencies that I would need to build the source in the place, would that finally built binary contain them all in itself, or would I have to include them on the distributed servers as well?
Would that work? Is the question a little too general and perhaps it would all depend on what I'm building?
Update from original posting after a couple of responses
I will be distributing the VMs myself, inasmuch as I will build them and then install my projects upon them. Therefore, I know the OS and environment completely. I just don't want to "bloat" them with unnecessary software that's been installed that I don't actually need because the compiled executables I will place on the distributed VMs in for example /usr/local/bin ...
That depends on how you link your program to libraries it depends on. In most cases, the default is to link dynamically, which means that you need to distribute your executable along its deps. You can check out what libraries are required to run the file using ldd command.
Theoretically, you can link everything statically, which means that library code would be compiled into executable. Thus, executable would really be self-contained, but linking statically is not always possible. This depends on actual libraries you are using and probably require playing with ./configure args when building them.
Finally, there are some liraries that always linked dynamically, such as libc. The good thing is that machine you are distributing to would surely have this library. The bad thing is that versions of these libraries may differ, and you might face ABI mismatch.
In short, if your project not huge and there is possibility to link everything statically, go this way. If not, read about AppImage and Docker.
The distribution of built libraries and headers (binary distribution) is a possible way and should work. (I do it in my projects always.)
It is not necessary that all of the libraries you built are installed into /usr/lib. To keep your target machine clean you can install it in other folder to, e.g.
/usr/local/MYLIB/lib/libmylib.so
/usr/local/MYLIB/include/mylib.h
/usr/local/MYOTHERLIB/lib/libmyotherlib.so
/usr/local/MYOTHERLIB/include/libmyotherlib.so
Advantages:
Easy installation, easy remove
All files within one subfolder, no files are missing, no mix with other libs
Disadvantage:
The loader must know the extra search path
So I made this executable program that uses the Windows library and some others (string, ctime, lmcons...) in C++. When it runs on my computer it works great but when I transfer the executable to a computer that does not have some of those libraries on it the program does not run. How do I "add" those libraries in with my code?
1 - You need to identify libraries that need to exist on the system in order to execute your application.
2 - you need to create a package that contain these libraries. It could be an installation or a zip file. Depending on the libraries, sometimes they need to be registered on the system, sometimes just dropped in. If you use install packaging software, you can set up registration [if needed]. If you distribute zip or ftp folder, you may need to supply script file. Sometimes libraries are part of some Microsoft package and this package can be prerequisite to run your application. You may pack it into your installation and have it installed silently. There are many ways as you see.
3 - this is up to you how you want to distribute your application and supporting libraries. But best is when user doesn't have to jump the hoops trying to install your stuff. User should click and forget.
There are many open-source libraries that can be compiled with Visual Studio. I'm porting a program from Linux to Windows, but it depends on a number of libraries. I don't know what the best practices regarding libraries are on Windows.
On Linux, these libraries are typically part of the distribution. To use sqlite on Debian, for example, you need only to install libsqlite3-dev and the include files and libraries (both static and dynamic) are automatically installed and available to your program.
If you need a different version than your distribution supplies, you can compile it in your home directory, install it to ~/include and ~/lib, and set the appropriate environment variables so that your compiler includes those directories in its search path.
What is the best way to use libraries that are distributed as source on Windows? If I link dynamically rather than statically, is there an easy way to copy required DLLs into the output directory to ease redistribution (assuming license requirements are met)?
Option 1 - Projects that have binary distributions for windows / do not build in DevStudio.
E.g. OpenSSL.
Projects like OpenSSL are best downloaded to their own folder and built using their own scripts. OpenSSL typically installs itself to C:\OpenSSL on windows builds, so one done, you can add C:\OpenSSL\include and C:\OpenSSL\lib to your project environment to access the OpenSSL headers and Libs. The actual dll files you will need to copy from C:\OpenSSL\bin into your projects staging folder (normally your SolutionDir\Debug or Release).
Once youve gone through the hassle of building OpenSSL once, you don't want to do it again. Or, if you've downloaded the binary distribution, its best left alone. Just document to others which binary distribution you used so they can set up their Visual Studio build environment appropriately.
Option 2 - Small libraries that are easy to create Visual Studio Projects for (or already have). Lua and sqllite fall into this category.
For projects that are small enough, it is not inconvenient to simply add them to your solution in a sub folder. This way you can get their outputs built directly to the solutions output folder, and you do not have to bundle pre-build binary files in your solution making it far easier to share the project with others.
Option 3 - As an alternative you could create your own standardized folder for the products of open source projects. Create C:\oss\include, c:\oss\lib, c:\oss\bin etc, add these paths to DevStudios lib and include paths, add c:\oss\bin to the systems PATH variable, as you build each OSS project, copy the appropriate files to these locations.
Again, while convenient, this setup makes it diffucult to replicate the build environment on a 2nd PC, so you might want to keep the entire C:\oss tree in source control as well.
On windows the easiest way is to build your own DLLs and include them in the program directory.
Yes it uses a bit more space, but HD are large these days and avoids a lot of headaches of incompatible versions (DLL hell). Windows also suffers a few more wrinkles with versions of libs built with different compilers so shipping your own builds is safest
How exactly do DLL files work? There seems to be an awful lot of them, but I don't know what they are or how they work.
So, what's the deal with them?
What is a DLL?
Dynamic Link Libraries (DLL)s are like EXEs but they are not directly executable. They are similar to .so files in Linux/Unix. That is to say, DLLs are MS's implementation of shared libraries.
DLLs are so much like an EXE that the file format itself is the same. Both EXE and DLLs are based on the Portable Executable (PE) file format. DLLs can also contain COM components and .NET libraries.
What does a DLL contain?
A DLL contains functions, classes, variables, UIs and resources (such as icons, images, files, ...) that an EXE, or other DLL uses.
Types of libraries:
On virtually all operating systems, there are 2 types of libraries. Static libraries and dynamic libraries. In windows the file extensions are as follows: Static libraries (.lib) and dynamic libraries (.dll). The main difference is that static libraries are linked to the executable at compile time; whereas dynamic linked libraries are not linked until run-time.
More on static and dynamic libraries:
You don't normally see static libraries though on your computer, because a static library is embedded directly inside of a module (EXE or DLL). A dynamic library is a stand-alone file.
A DLL can be changed at any time and is only loaded at runtime when an EXE explicitly loads the DLL. A static library cannot be changed once it is compiled within the EXE.
A DLL can be updated individually without updating the EXE itself.
Loading a DLL:
A program loads a DLL at startup, via the Win32 API LoadLibrary, or when it is a dependency of another DLL. A program uses the GetProcAddress to load a function or LoadResource to load a resource.
Further reading:
Please check MSDN or Wikipedia for further reading. Also the sources of this answer.
What is a DLL?
DLL files are binary files that can contain executable code and resources like images, etc. Unlike applications, these cannot be directly executed, but an application will load them as and when they are required (or all at once during startup).
Are they important?
Most applications will load the DLL files they require at startup. If any of these are not found the system will not be able to start the process at all.
DLL files might require other DLL files
In the same way that an application requires a DLL file, a DLL file might be dependent on other DLL files itself. If one of these DLL files in the chain of dependency is not found, the application will not load. This is debugged easily using any dependency walker tools, like Dependency Walker.
There are so many of them in the system folders
Most of the system functionality is exposed to a user program in the form of DLL files as they are a standard form of sharing code / resources. Each functionality is kept separately in different DLL files so that only the required DLL files will be loaded and thus reduce the memory constraints on the system.
Installed applications also use DLL files
DLL files also becomes a form of separating functionalities physically as explained above. Good applications also try to not load the DLL files until they are absolutely required, which reduces the memory requirements. This too causes applications to ship with a lot of DLL files.
DLL Hell
However, at times system upgrades often breaks other programs when there is a version mismatch between the shared DLL files and the program that requires them. System checkpoints and DLL cache, etc. have been the initiatives from M$ to solve this problem. The .NET platform might not face this issue at all.
How do we know what's inside a DLL file?
You have to use an external tool like DUMPBIN or Dependency Walker which will not only show what publicly visible functions (known as exports) are contained inside the DLL files and also what other DLL files it requires and which exports from those DLL files this DLL file is dependent upon.
How do we create / use them?
Refer the programming documentation from your vendor. For C++, refer to LoadLibrary in MSDN.
Let’s say you are making an executable that uses some functions found in a library.
If the library you are using is static, the linker will copy the object code for these functions directly from the library and insert them into the executable.
Now if this executable is run it has every thing it needs, so the executable loader just loads it into memory and runs it.
If the library is dynamic the linker will not insert object code but rather it will insert a stub which basically says this function is located in this DLL at this location.
Now if this executable is run, bits of the executable are missing (i.e the stubs) so the loader goes through the executable fixing up the missing stubs. Only after all the stubs have been resolved will the executable be allowed to run.
To see this in action delete or rename the DLL and watch how the loader will report a missing DLL error when you try to run the executable.
Hence the name Dynamic Link Library, parts of the linking process is being done dynamically at run time by the executable loader.
One a final note, if you don't link to the DLL then no stubs will be inserted by the linker, but Windows still provides the GetProcAddress API that allows you to load an execute the DLL function entry point long after the executable has started.
DLLs (dynamic link libraries) and SLs (shared libraries, equivalent under UNIX) are just libraries of executable code which can be dynamically linked into an executable at load time.
Static libraries are inserted into an executable at compile time and are fixed from that point. They increase the size of the executable and cannot be shared.
Dynamic libraries have the following advantages:
1/ They are loaded at run time rather than compile time so they can be updated independently of the executable (all those fancy windows and dialog boxes you see in Windows come from DLLs so the look-and-feel of your application can change without you having to rewrite it).
2/ Because they're independent, the code can be shared across multiple executables - this saves memory since, if you're running 100 apps with a single DLL, there may only be one copy of the DLL in memory.
Their main disadvantage is advantage #1 - having DLLs change independent your application may cause your application to stop working or start behaving in a bizarre manner. DLL versioning tend not to be managed very well under Windows and this leads to the quaintly-named "DLL Hell".
DLL files contain an Export Table which is a list of symbols which can be looked up by the calling program. The symbols are typically functions with the C calling convention (__stcall). The export table also contains the address of the function.
With this information, the calling program can then call the functions within the DLL even though it did not have access to the DLL at compile time.
Introducing Dynamic Link Libraries has some more information.
http://support.microsoft.com/kb/815065
A DLL is a library that contains code
and data that can be used by more than
one program at the same time. For
example, in Windows operating systems,
the Comdlg32 DLL performs common
dialog box related functions.
Therefore, each program can use the
functionality that is contained in
this DLL to implement an Open dialog
box. This helps promote code reuse and
efficient memory usage.
By using a DLL, a program can be
modularized into separate components.
For example, an accounting program may
be sold by module. Each module can be
loaded into the main program at run
time if that module is installed.
Because the modules are separate, the
load time of the program is faster,
and a module is only loaded when that
functionality is requested.
Additionally, updates are easier to
apply to each module without affecting
other parts of the program. For
example, you may have a payroll
program, and the tax rates change each
year. When these changes are isolated
to a DLL, you can apply an update
without needing to build or install
the whole program again.
http://en.wikipedia.org/wiki/Dynamic-link_library
DLL is a File Extension & Known As “dynamic link library” file format used for holding multiple codes and procedures for Windows programs. Software & Games runs on the bases of DLL Files; DLL files was created so that multiple applications could use their information at the same time.
IF you want to get more information about DLL Files or facing any error read the following post.
https://www.bouncegeek.com/fix-dll-errors-windows-586985/
DLLs (Dynamic Link Libraries) contain resources used by one or more applications or services. They can contain classes, icons, strings, objects, interfaces, and pretty much anything a developer would need to store except a UI.
According to Microsoft
(DLL) Dynamic link libraries are files that contain data, code, or resources needed for the running of applications. These are files that are created by the windows ecosystem and can be shared between two or more applications.
When a program or software runs on Windows, much of how the application works depends on the DLL files of the program. For instance, if a particular application had several modules, then how each module interacts with each other is determined by the Windows DLL files.
If you want detailed explanation, check these useful resources
What are dll files , About Dll files