In this episode of "let's be stupid", we have the following problem: a C++ library has been wrapped with a layer of code that exports its functionality in a way that allows it to be called from C. This results in a separate library that must be linked (along with the original C++ library and some object files specific to the program) into a C program to produce the desired result.
The tricky part is that this is being done in the context of a rigid build system that was built in-house and consists of literally dozens of include makefiles. This system has a separate step for the linking of libraries and object files into the final executable but it insists on using gcc for this step instead of g++ because the program source files all have a .c extension, so the result is a profusion of undefined symbols. If the command line is manually pasted at a prompt and g++ is substituted for gcc, then everything works fine.
There is a well-known (to this build system) make variable that allows flags to be passed to the linking step, and it would be nice if there were some incantation that could be added to this variable that would force gcc to act like g++ (since both are just driver programs).
I have spent quality time with the gcc documentation searching for something that would do this but haven't found anything that looks right, does anybody have suggestions?
Considering such a terrible build system write a wrapper around gcc that exec's gcc or g++ dependent upon the arguments. Replace /usr/bin/gcc with this script, or modify your PATH to use this script in preference to the real binary.
#!/bin/sh
if [ "$1" == "wibble wobble" ]
then
exec /usr/bin/gcc-4.5 $*
else
exec /usr/bin/g++-4.5 $*
fi
The problem is that C linkage produces object files with C name mangling, and that C++ linkage produces object files with C++ name mangling.
Your best bet is to use
extern "C"
before declarations in your C++ builds, and no prefix on your C builds.
You can detect C++ using
#if __cplusplus
Many thanks to bmargulies for his comment on the original question. By comparing the output of running the link line with both gcc and g++ using the -v option and doing a bit of experimenting, I was able to determine that "-lstdc++" was the magic ingredient to add to my linking flags (in the appropriate order relative to other libraries) in order to avoid the problem of undefined symbols.
For those of you who wish to play "let's be stupid" at home, I should note that I have avoided any use of static initialization in the C++ code (as is generally wise), so I wasn't forced to compile the translation unit containing the main() function with g++ as indicated in item 32.1 of FAQ-Lite (http://www.parashift.com/c++-faq-lite/mixing-c-and-cpp.html).
Related
I'm experimenting a bit with building DLLs on windows using MINGW.
A very good summary (in my opinion) can be found at:
https://www.transmissionzero.co.uk/computing/building-dlls-with-mingw/
There is even a basic project which can be used for the purpose of this discussion:
https://github.com/TransmissionZero/MinGW-DLL-Example/releases/tag/rel%2Fv1.1
Note there is a cosmetic mistake in this project which will make it fail out of the box: the Makefile does not create an "obj" directory - Either adjust the Makefile or create it manually.
So here is the real question.
How to change the Windows DLL name so it differs from the actual DLL file name ??
Essentially I'm trying to achieve on Windows, the effect which is very well described here on Linux:
https://www.man7.org/conf/lca2006/shared_libraries/slide4b.html
Initially I tried changing "InternalName" and ""OriginalFilename" in the resource file used to create the DLL but that does not work.
In a second step, I tried adding "-Wl,-soname,SoName.dll" on the command that performs the final link, to change the Windows DLL name.
However, that does not seem to have the expected effect (I'm using MingW 7.3.0, x86_64-posix-seh-rev0).
Two things makes me say that:
1/ The test executable still works (I would expect it to fail, because it tries to locate SoName.dll but can't find it).
2/ "pexports.exe AddLib.dll" produces the output below, where the library name hasn't changed:
LIBRARY "AddLib.dll"
EXPORTS
Add
bar DATA
foo DATA
Am I doing anything wrong ? Are my expectations wrong perhaps ?
Thanks for your help !
David
First of all, I would like to say it's important to use either a .def file for specifying the exported symbols or use __declspec(dllexport) / __declspec(dllimport), but never mix these two methods. There is also another method using the -Wl,--export-all-symbols linker flag, but I think that's ugly and should only be used when quick and dirty is what you want.
It is possible to tell MinGW to use a DLL filename that does not match the library name. In the link step use -o to specify the DLL and use -Wl,--out-implib, to specify the library file.
Let me illustrate by showing how to build chebyshev as a both static and shared library. Its sources consist of only only 2 files: chebyshev.h and chebyshev.c.
Compile
gcc -c -o chebyshev.o chebyshev.c -I. -O3
Create static library
ar cr libchebyshev.a chebyshev.o
Create a .def file (as it wasn't supplied and __declspec(dllexport) / __declspec(dllimport) wasn't used either). Note that this file doesn't contain a line with LIBRARY allowing the linker to specify the DLL filename later.
There are several ways to do this if the .def file wasn't supplied by the project:
3.1. Get the symbols from the .h file(s). This may be hard as sometimes you need to distinguish for example between type definitions (like typedef, enum, struct) and actual functions and variables that need to be exported;
echo "EXPORTS" > chebyshev.def
sed -n -e "s/^.* \**\(chebyshev_.*\) *(.*$/\1/p" chebyshev.h >> chebyshev.def
3.2. Use nm to list symbols in the library file and filter out the type of symbols you need.
echo "EXPORTS" > chebyshev.def
nm -f posix --defined-only -p libchebyshev.a | sed -n -e "s/^_*\([^ ]*\) T .*$/\1/p" >> chebyshev.def
Link the static library into the shared library.
gcc -shared -s -mwindows -def chebyshev.def -o chebyshev-0.dll -Wl,--out-implib,libchebyshev.dll.a libchebyshev.a
If you have a project that uses __declspec(dllexport) / __declspec(dllimport) things are a lot easier. And you can even have the link step generate a .def file using the -Wl,--output-def, linker flag like this:
gcc -shared -s -mwindows -o myproject.dll -Wl,--out-implib,myproject.dll.a -Wl,--output-def,myproject.def myproject.o
This answer is based on my experiences with C. For C++ you really should use __declspec(dllexport) / __declspec(dllimport).
I believe I have found one mechanism to achieve on Windows, the effect described for Linux in https://www.man7.org/conf/lca2006/shared_libraries/slide4b.html
This involves dll_tool
In the example Makefile there was originally this line:
gcc -o AddLib.dll obj/add.o obj/resource.o -shared -s -Wl,--subsystem,windows,--out-implib,libaddlib.a
I simply replaced it with the 2 lines below instead:
dlltool -e obj/exports.o --dllname soname.dll -l libAddLib.a obj/resource.o obj/add.o
gcc -o AddLib.dll obj/resource.o obj/add.o obj/exports.o -shared -s -Wl,--subsystem,windows
Really, the key seems to be the creation with dlltool of an exports file in conjunction with dllname. This exports file is linked with the object files that make up the body of the DLL and it handles the interface between the DLL and the outside world. Note that dlltool also creates the "import library" at the same time
Now I get the expected effect, and I can see that the "Internal DLL name" (not sure what the correct terminology is) has changed:
First evidence:
>> dlltool.exe -I libAddLib.a
soname.dll
Second evidence:
>> pexports.exe AddLib.dll
LIBRARY "soname.dll"
EXPORTS
Add
bar DATA
foo DATA
Third evidence:
>> AddTest.exe
Error: the code execution cannot proceed because soname.dll was not found.
Although the desired effect is achieved, this still seems to be some sort of workaround. My understanding (but I could well be wrong) is that the gcc option "-Wl,-soname" should achieve exactly the same thing. At least it does on Linux, but is this broken on Windows perhaps ??
I have used the C code from the following verbatim: https://wiki.tcl-lang.org/page/Hello+World+as+a+C+extension
/*
* hello.c -- A minimal Tcl C extension.
*/
#include <tcl.h>
static int
Hello_Cmd(ClientData cdata, Tcl_Interp *interp, int objc, Tcl_Obj *const objv[])
{
Tcl_SetObjResult(interp, Tcl_NewStringObj("Hello, World!", -1));
return TCL_OK;
}
/*
* Hello_Init -- Called when Tcl loads your extension.
*/
int DLLEXPORT
Hello_Init(Tcl_Interp *interp)
{
if (Tcl_InitStubs(interp, TCL_VERSION, 0) == NULL) {
return TCL_ERROR;
}
/* changed this to check for an error - GPS */
if (Tcl_PkgProvide(interp, "Hello", "1.0") == TCL_ERROR) {
return TCL_ERROR;
}
Tcl_CreateObjCommand(interp, "hello", Hello_Cmd, NULL, NULL);
return TCL_OK;
}
My command for compiling is nearly verbatim except for the last character, indicating Tcl version 8.6 rather than 8.4, and it compiles without error:
gcc -shared -o hello.dll -DUSE_TCL_STUBS -I$TCLINC -L$TCLLIB -ltclstub86
Then I created the following Tcl program:
load hello.dll Hello
puts "got here"
But when running it with tclsh get the following error:
cannot find symbol "Hello_Init"
while executing
"load ./hello.dll Hello"
(file "hello.tcl" line 1)
So I am essentially following a couple of suggestions from Donal Fellows answer here: cannot find symbol "Embeddedrcall_Init" The OP there however commented that, like me, the suggestion(s) hadn't resolved their issue. One thing that I didn't try from that answer was "You should have an exported (extern "C") function symbol in your library" -- could that be the difference maker? Shouldn't it have been in the example all along then?
At the suggestion of somebody on comp.lang.tcl I found "DLL Export Viewer" but when I run it against the DLL it reports 0 functions found :( What am I doing wrong?
Could it be an issue with MinGW/gcc on Windows, and I need to bite the bullet and do this with Visual Studio? That's overkill I'd like to avoid if possible.
The core of the problem is that your function Hello_Init is not ending up in the global symbol table exported by the resulting DLL. (Some linkers would put such things in as _Hello_Init instead of Hello_Init; Tcl adapts to them transparently.) The symbol must be there for Tcl's load command to work: without it, there's simply no consistent way to tell your extension code what the Tcl_Interp context handle is (which allows it to make commands, variables, etc.)
(If you'd been working with C++, one of the possible problem is a missing extern "C" whose actual meaning is to turn off name mangling. That's probably not the problem here.)
Since you are on Windows — going by the symbols in your DLL, such as EnterCriticalSection and GetLastError — the problem is probably linked to exactly how you are linking. I'm guessing that Tcl is defining your function to have __declspec(dllexport) (assuming you've not defined STATIC_BUILD, which absolutely should not be used when building a DLL) and yet that's not getting respected. Assuming you're using a modern-enough version of GCC… which you probably are.
I'm also going through the process of how to build tcl extensions in C and had exactly the same problem when working though this same example using tcl 8.6.
i.e. I was compiling using MinGW GCC (64-bit), and used the following:
gcc -shared -o hello.dll -DUSE_TCL_STUBS "-IC:\\ActiveTcl\\include" "-LC:\\ActiveTcl\\lib" -ltclstub86
And like the OP I got no compile error, but when loading the dll at a tclsh prompt tcl complained :
'cannot find symbol "Hello_Init"'
I can't say that I understand, but I was able to find a solution that works thanks to some trial and error, and some information on the tcl wiki here
https://wiki.tcl-lang.org/page/Building+Tcl+DLL%27s+for+Windows
In my case I had to adjust the compiler statement to the following
gcc -shared -o hello.dll hello.c "-IC:\\ActiveTcl\\include" "-LC:\\ActiveTcl\\bin" -ltcl86t
Obviously those file paths are specific to my system, but basically
I had to add an explicit reference to the .c file
I had to include the tcl86t dll library from the tcl bin directory
I had to remove the -DUSE_TCL_STUBS flag ( meaning that the references -LC:\\ActiveTcl\\lib and -ltclstub86 could also be removed)
(attempting to use the -DUSE_TCL_STUBS flag caused the compiler to complain with C:\ActiveTcl\lib/tclstub86.lib: error adding symbols: File format not recognized )
This successfully compiled a dll that I could load, and then call the hello function to print my 'Hello World' message.
Something else I stumbled over, and which wasn't immediately obvious:
reading https://www.tcl.tk/man/tcl8.6/TclCmd/load.htm, tcl epxects to find an 'init' function based on a certain naming convention.
if the C extension does not define a package name then the name of that init function will be derived from the dll filename.
This caused a few problems for me (when compiling via Eclipse IDE), as the dll name was being automatically determined from the eclipse projet name.
For example, if I recompile the same example, but call the .dll something else, eg.
gcc -shared -o helloWorldExenstion.dll hello.c "-IC:\\ActiveTcl\\include" "-LC:\\ActiveTcl\\bin" -ltcl86t
Then at tclsh prompt:
% load helloWorldExtension
cannot find symbol "Helloworldextension_Init"
How do I create a custom library in GNU? What I mean is:
When we use #include < stdio.h> and printf
we can compile it with gcc main.c
Now I create my custom headers and .a/.so library files, I know I can set the environment variable C_INCLUDE_PATH and include my header files with #include<> instead of #include"". However, I still have to compile it with
gcc main.c -o program -L/whatever/ -lwahtever
(with set environment variable if using .so)
Is it possible to make it behave like #include< stdio.h> where I don't need to include the paths with corresponding command line arguments?
You actually don't need -L/whatever/, just -lwhatever. The first option supplies the path to your library, but you have already taken care of that with the #include and modifying C_INCLUDE_PATH. The second option tells the linker which library to link your executable with. An example of this is when using functions from the C math library, you #include <math.h>, but to compile, you still need the linker option -lmath. So to answer your question, no. You can remove the first option, but you must leave the second.
I currently invoke clang or gcc as
cc -E -DPREPROCESSING ...
when debugging macros.
It has occurred to me that the define is redundant. Is there an expression I could write in the source to detect when the compiler will stop after preprocessing, and so drop this definition from my build scripts?
#if magic
#define PREPROCESSING
#ending
A look at the docs suggests not, but with luck I'm missing something.
Whatever solution you come up with is going to be compiler-specific, since the C standard does not have anything to say about separate preprocessing.
In gcc, you could implement the magic by adding a custom spec file:
%rename cpp old_cpp
*cpp:
%{E:-DPREPROCESSING} %(old_cpp)
You would need to tell gcc to use this spec file (-specs=/path/to/specfile), unless you compiled your own gcc with the above definition added to the built-in cpp spec. If you are using a Makefile, you could add the -specs option above to your CFLAGS.
(I should add that I don't think this is a particularly good idea. But it is possible.)
I'd be curious to understand if there's any substantial difference in specifying libraries (both shared and static) to gcc/g++ in the two following ways (CC can be g++ or gcc)
CC -o output_executable /path/to/my/libstatic.a /path/to/my/libshared.so source1.cpp source2.cpp ... sourceN.cpp
vs
CC -o output_executable -L/path/to/my/libs -lstatic -lshared source1.cpp source2.cpp ... sourceN.cpp
I can only see a major difference being that passing directly the fully-specified library name would make for a greater control in choosing static or dynamic versions, but I suspect there's something else going on that can have side effects on how the executable is built or will behave at runtime, am I right?
Andrea.
Ok, I can answer myself basing on some experiments and a deeper reading of gcc documentation:
From gcc documentation: http://gcc.gnu.org/onlinedocs/gcc/Link-Options.html
[...] The linker handles an archive file by scanning through it for members which define symbols that have so far been referenced but not defined. But if the file that is found is an ordinary object file, it is linked in the usual fashion. The only difference between using an -l option and specifying a file name is that -l surrounds library with lib' and.a' and searches several directories
This actually answers also to the related doubt about the 3rd option of directly specifying object files on the gcc command line (i.e. in that case all the code in the object files will become part of the final executable, while using archives, only the object files that are really needed will be pulled in).