Is their location hardcoded into gcc code or does gcc just call as and we must have as location in our PATH variable?
And in the latter case, how could we create two completely separate gcc toolchains? I mean, how can we make gcc-A invoke as-A and gcc-B invoke as-B if as-A and as-B are both called as?
Some of the paths (e.g., to cc1) are compiled in. Others (e.g., as) use normal lookup in $PATH. This can vary depending on the options GCC is configured with.
You can tell fairly easily by running with strace, and grepping for exec|stat.
$ strace -f gcc foo.c -o foo |& grep exec
⋮
[pid 24943] execve("/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.6.1/cc1", …
That is a call to cc1 by a compiled-in path, as you can see from the
lack of looking for it. Its also not in $PATH.
[pid 24944] execve("/home/anthony/bin/as", ["as", "--64", "-o", "/tmp/ccCIrcGi.o", "/tmp/ccbw3PkL.s"], [/* 51 vars */]) = -1 ENOENT (No such file or directory)
[pid 24944] execve("/usr/local/bin/as", ["as", "--64", "-o", "/tmp/ccCIrcGi.o", "/tmp/ccbw3PkL.s"], [/* 51 vars */]) = -1 ENOENT (No such file or directory)
[pid 24944] execve("/usr/bin/as", ["as", "--64", "-o", "/tmp/ccCIrcGi.o", "/tmp/ccbw3PkL.s"], [/* 51 vars */]) = 0
That is looking for as in $PATH. You can tell because its trying each
location in $PATH in order.
I've omitted a lot of strace output—even with just stat and exec, its
several pages long.
Running gcc -v will show you some of the compiled-in paths (as part of the configure line).
How could we create two completely separate gcc toolchains?
Compile GCC from source twice, detailed instructions at: Multiple glibc libraries on a single host
Everything is hardcoded and highly coupled as far as I can see, I don't think there is any other decent solution.
Query the GCC search path
You can also query the GCC search path with:
gcc -print-search-dirs | grep -E '^programs' | tr ':' '\n'
sample output:
programs
=/usr/lib/gcc/x86_64-linux-gnu/6/
/usr/lib/gcc/x86_64-linux-gnu/6/
/usr/lib/gcc/x86_64-linux-gnu/
/usr/lib/gcc/x86_64-linux-gnu/6/
/usr/lib/gcc/x86_64-linux-gnu/
/usr/lib/gcc/x86_64-linux-gnu/6/../../../../x86_64-linux-gnu/bin/x86_64-linux-gnu/6/
/usr/lib/gcc/x86_64-linux-gnu/6/../../../../x86_64-linux-gnu/bin/x86_64-linux-gnu/
/usr/lib/gcc/x86_64-linux-gnu/6/../../../../x86_64-linux-gnu/bin/
and a specific program with:
gcc -print-prog-name=cc1
sample output:
/usr/lib/gcc/x86_64-linux-gnu/6/cc1
GCC spec files
It is wort mentioning that what actually determines the final cpp, ld, as are the "spec" files in the GCC source code, see also: What are GCC's passes and invoked programs?
There's an ad-hoc option for that: -B*prefix*, quoting gcc docs:
For each subprogram to be run, the compiler driver first tries the -B prefix, if any. If
that name is not found, or if -B was not specified, the driver tries two standard prefixes,
which are /usr/lib/gcc/ and /usr/local/lib/gcc/. [...]
Related
I am running GnuCOBOL on Windows from command line to compile and link a COBOL program. I have been able to produce a functioning Hello World program.
The COBOL program I now want to build is quite simple, but calls routines that are available in a C library I have built using another build environment. I have exchanged some names and paths for anonymity.
The lib is called mysrc.lib, and is placed at C:\dev\foo. I compile and link with the command
cobc -x -fword-continuation=ok -ffold-call=LOWER c:\test\MULTI\cobsim.cob -o c:\test\MULTI\cobsim.EXE -LC:\dev\foo -lmysrc
It seems to find the lib file, because if I change the name of the lib file to something that doesn't exist, it complains about not finding it.
cobc -x -fword-continuation=ok -ffold-call=LOWER c:\test\MULTI\cobsim.cob -o c:\test\MULTI\cobsim.EXE -LC:\dev\foo -lmysrc2
C:\GnuCOBOL\bin/ld.exe: cannot find -lmysrc2
collect2.exe: error: ld returned 1 exit status
However, when I run the program, it cannot find the routines in mysrc. The first C routine in mysrc that is called from the program cobsim.cob is named dbadr4. When I try to run the resulting cobsim.exe, I get the following error:
libcob: module 'DBADR4' not found
I can also use the flag -fstatic-call to get the link error already when linking.
cobc -x -fword-continuation=ok -ffold-call=LOWER -fstatic-call c:\test\MULTI\cobsim.cob -o c:\test\MULTI\cobsim.EXE -LC:\dev\foo -lmysrc
.\cob14860_0.o:cob14860_0.c:(.text+0x14f): undefined reference to `dbadr4'
It doesn't seem to matter which order I place the switches and arguments. Including or excluding the -ffold-call=LOWER flag has no effect, apart from the link error complaining about not finding the routine in upper or lower case. I have multiple other programs calling these routines without problem, including C programs and this specific COBOL program compiled and linked with another compiler and linker on another platform.
Any suggestions?
EDIT: Adding requested information.
Output when adding -g -v flags (some information removed or altered for anonymity):
cobc (GnuCOBOL) 3.0-rc1.0
Built May 05 2018 21:41:43 Packaged Apr 22 2018 22:26:37 UTC
C version (MinGW) "6.3.0"
loading standard configuration file 'default.conf'
command line: cobc -x -save-temps -fword-continuation=ok -ffold-call=LOWER -fstatic-call -L C:\dev\foo -l mysrc -o C:\test\MULTI\cobsim.EXE -g -v C:\test\MULTI\cobsim.cob
preprocessing: C:\test\MULTI\cobsim.cob -> cobsim.i
return status: 0
parsing: cobsim.i (C:\test\MULTI\cobsim.cob)
return status: 0
translating: cobsim.i -> cobsim.c (C:\test\MULTI\cobsim.cob)
executing: gcc -c -I"C:\GnuCOBOL\include" -g -o "cobsim.o" "cobsim.c"
return status: 0
executing: gcc -Wl,--export-all-symbols -Wl,--enable-auto-import
-Wl,--enable-auto-image-base -o "C:\test\MULTI\cobsim.EXE"
"cobsim.o" -L"C:\GnuCOBOL\lib"
-L"C:\dev\foo" -L/mingw/lib
-lcob -lm -lgmp -L/mingw/lib -lintl -lpdcurses -l"mysrc"
cobsim.o: In function `simple_':
cobsim.c:112: undefined reference to `dbadr4'
And snippets of the program code, including the call to dbadr4:
DATA DIVISION.
WORKING-STORAGE SECTION.
01 SQL005.
05 SQL005-001 PIC X(1) VALUE " ".
01 SQLPAA.
05 SQLPAA-001 USAGE POINTER.
05 SQLPAA-002 USAGE POINTER.
05 SQLPAA-003 USAGE POINTER.
05 SQLPAA-004 USAGE POINTER.
PROCEDURE DIVISION.
MAIN SECTION.
DO-IT.
CALL "DBADR4" USING SQLPAA-001 SQL005
EDIT 2: Missed adding the DBADR4 function declaration.
void dbadr4(void** addr, const int varibl[1])
The COBOL code is originally Embedded SQL/COBOL code, the call to DBADR4 and the SQL005 and SQLPAA data blocks have been automatically generated. The dbadr4 code is originally Ada code, the C routine declaration has been automatically generated. This generated COBOL code calling this generated C code works fine on HP OpenVMS, where we use HP's COBOL compiler and linker, and has done so for many years.
If I have this line in the make file:\
libpqxx_Libs = -L/share/home/cb -lpqxx-2.6.9 -lpq
Does this indicate the compiler to use the lpqxx-2.6.9.so shared object file or does this indciate the compiler to use all the .so in the foler lpqxx-2.6.9? Or is this something else altogether?
Thanks for the help!
-L in this context is an argument to the linker, that adds the specified directory to the list of directories that the linker will search for necessary libraries, e.g. libraries that you've specified using -l.
It isn't a makefile command, even though it's usually seen in makefiles for C projects.
The -L is actually not a makefile command (as you state it in the title of your question).
What actually happens in this line is an assignment of a value to the variable libpqxx_Libs -- nothing more and nothing less. You will have to search in your makefile where that variable is used via $(libpqxx_Libs) or ${libpqxx_Libs}. That is most likely as a argument in a link command, or a compile command that includes linking.
In that context, the meaning of -L and -l can be found in, for example, the gcc man pages, which state that
-llibrary
Use the library named library when linking.
The linker searches a standard list of directories for the li-
brary, which is actually a file named `liblibrary.a'. The linker
then uses this file as if it had been specified precisely by
name.
The directories searched include several standard system direc-
tories plus any that you specify with `-L'.
I have been trying to compile openssl 1.0.0g with the following rpath:
$ORIGIN/../lib64
Everytime I readelf -d apps/openssl, I am getting results like the following depending on what escaping variation I tried:
\RIGIN/../lib64
RIGIN/../lib64
ORIGIN/../lib64
I want to setup my rpath without using external tools like chrpath. Is it at all possible? I will basically accept anything that does not involve using external tools like chrpath (though I would already be done with that).
Ideally, I would like to do it by passing options on the command line (any form of -Wl,-rpath,$ORIGIN/../lib64).
I don't mind editing the generated Makefile, which is what I have been trying last. If only I could get it to print a stupid dollar sign!!! I tried modifying LIBRPATH under the BUILDENV= block with no luck. My best results so far:
LIBRPATH=$$'ORIGIN/../lib64 # result: /../lib64
LIBRPATH=$$$$'ORIGIN/../lib64 # result: 12345<pid>/../lib64
I have read various rpath related questions and tried various escaping and quoting tricks but nothing worked so far!
In your makefile try:
-Wl,-rpath,${ORIGIN}/../lib64
I am assuming that the ORIGIN is a shell variable.
EDIT
I have just found an answer to your question (better late then never):
You need to prevent make from interpolating variables, to do that you need to use $$ (double dolar sign):
-Wl,-rpath,'$$ORIGIN/../lib64'
I know that it works because I have tested it with my own application, enjoy :)
I went the chrpath way.
http://enchildfone.wordpress.com/2010/03/23/a-description-of-rpath-origin-ld_library_path-and-portable-linux-binaries/
It is quite complicated to counter shell expansion of `$$ORIGIN`` in openssl. Sooner or later, it gets expanded because of the dollar sign. If you really want to go this way, you can do it. I have found the following to work with openssl 1.0.1g on Linux. In Makefile.shared, look for this line:
DO_GNU_APP=LDFLAGS="$(CFLAGS) -Wl,-rpath,$(LIBRPATH)"
Replace it with the following. This quoting-fu neutralize the expansion of $. The double $$ is the way to get a single dollar sign in makefiles.
DO_GNU_APP=LDFLAGS="$(CFLAGS) -Wl,-rpath,'"'$$'"ORIGIN/../lib64'"
After compiling:
readelf -d apps/openssl | grep RPATH
0x000000000000000f (RPATH) Library rpath: ['$ORIGIN/../lib64']
OK I spent several hours fighting with this same issue and trying all manner of crazy escaping, at one point I was up to eight $ signs, at which point I decided that there must be another way.
In fact, it appears that there is, at least with GNU ld.
Instead of -Wl,-rpath,\\$$$\$\$\$$$\$\\\\$ or some other elder god invoking monstrosity, just do this:
echo '-rpath=$ORIGIN/../lib64' > rpathorigin
./config -Wl,#$(pwd)/rpathorigin ...
I don't see that ld.gold documents the # flag, and I have no idea about, say, lld. But if you are using GCC and it is invoking BFD ld, the above may just work for you.
Of course, the actual path used with origin should be customized as needed, and I have no opinion on ./config vs ./Configure. But using the response file trick seems to entirely sidestep the shell/make escaping nightmare.
I don't mind editing the generated Makefile, which is what I have been trying last...
I'm not sure you can set it with a shell variable and relative path. I don't think ldd expands the $ORIGIN in $ORIGIN/../lib64. In this case, I think you need to use ldconfig to add $ORIGIN/../lib64 to the library search paths. See finding ldd search path on Server Fault for more details.
Since I'm not sure, I'll provide the instructions anyway. You don't need to change the Makefiles. As a matter of fact, I did not have any luck doing so in the past because things get overwritten, and other things like CFLAGS and LDFLAGS get ignored.
Also see Build OpenSSL with RPATH? Your question and the cited question are different question that converge on similar answers (no duplicates between them). But it provides the OpenSSL dev's position on RPATHs. It was a private email, so I shared the relevant details rather than the whole message.
If you manage to embed $ORIGIN/../lib64 in the ELF section and it works, then please report back. Below, I am using /usr/local/ssl/lib for my RPATH. You should substitute $ORIGIN/../lib64 for /usr/local/ssl/lib.
OpenSSL supports RPATH's out of the box for BSD targets (but not others). From Configure:
# Unlike other OSes (like Solaris, Linux, Tru64, IRIX) BSD run-time
# linkers (tested OpenBSD, NetBSD and FreeBSD) "demand" RPATH set on
# .so objects. Apparently application RPATH is not global and does
# not apply to .so linked with other .so. Problem manifests itself
# when libssl.so fails to load libcrypto.so. One can argue that we
# should engrave this into Makefile.shared rules or into BSD-* config
# lines above. Meanwhile let's try to be cautious and pass -rpath to
# linker only when --prefix is not /usr.
if ($target =~ /^BSD\-/)
{
$shared_ldflag.=" -Wl,-rpath,\$(LIBRPATH)" if ($prefix !~ m|^/usr[/]*$|);
}
The easiest way to do it for OpenSSL 1.0.2 appears to be add it to linker flags during configuration
./config -Wl,-rpath=/usr/local/ssl/lib
You can also edit Configure line and hard code the rpath. For example, I am working on Debian x86_64. So I opened the file Configure in an editor, copied linux-x86_64, named it linux-x86_64-rpath, and made the following change to add the -rpath option:
"linux-x86_64-rpath", "gcc:-m64 -DL_ENDIAN -O3 -Wall -Wl,-rpath=/usr/local/ssl/lib::
-D_REENTRANT::-Wl,-rpath=/usr/local/ssl/lib -ldl:SIXTY_FOUR_BIT_LONG RC4_CHUNK DES_INT DES_UNROLL:
${x86_64_asm}:elf:dlfcn:linux-shared:-fPIC:-m64:.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR):::64",
Above, fields 2 and 6 were changed. They correspond to $cflag and $ldflag in OpenSSL's builds system.
Then, Configure with the new configuration:
$ ./Configure linux-x86_64-rpath shared no-ssl2 no-ssl3 no-comp \
--openssldir=/usr/local/ssl enable-ec_nistp_64_gcc_128
Finally, after make, verify the settings stuck:
$ readelf -d ./libssl.so | grep -i rpath
0x000000000000000f (RPATH) Library rpath: [/usr/local/ssl/lib]
$ readelf -d ./libcrypto.so | grep -i rpath
0x000000000000000f (RPATH) Library rpath: [/usr/local/ssl/lib]
$ readelf -d ./apps/openssl | grep -i rpath
0x000000000000000f (RPATH) Library rpath: [/usr/local/ssl/lib]
Once you perform make install, then ldd will produce expected results:
$ ldd /usr/local/ssl/lib/libssl.so
linux-vdso.so.1 => (0x00007ffceff6c000)
libcrypto.so.1.0.0 => /usr/local/ssl/lib/libcrypto.so.1.0.0 (0x00007ff5eff96000)
...
$ ldd /usr/local/ssl/bin/openssl
linux-vdso.so.1 => (0x00007ffc30d3a000)
libssl.so.1.0.0 => /usr/local/ssl/lib/libssl.so.1.0.0 (0x00007f9e8372e000)
libcrypto.so.1.0.0 => /usr/local/ssl/lib/libcrypto.so.1.0.0 (0x00007f9e832c0000)
...
Don't ask me why but this worked for me in OpenSSL 1.1.1i in getting around the $ sign issue:
\$\$\$$ORIGIN
Example:
./Configure linux-x86_64 '-Wl,-rpath,\$\$\$$ORIGIN'
Alternatively, if this command line hack isn't congruent with you, you can always use chrpath after building as others have suggested:
./Configure linux-x86_64 '-Wl,-rpath,XORIGIN'
make depend
make all
chrpath -r "\$ORIGIN" libssl.so
I've got a binary "CeeloPartyServer" that needs to find libFoundation.so at runtime, on a FreeBSD machine. They're both in the same directory. I compile (on another platform, using a cross compiler) CeeloPartyServer using linker flag -rpath=$ORIGIN.
> readelf -d CeeloPartyServer |grep -i rpath
0x0000000f (RPATH) Library rpath: [$ORIGIN]
> ls
CeeloPartyServer Contents Foundation.framework libFoundation.so
> ./CeeloPartyServer
/libexec/ld-elf.so.1: Shared object "libFoundation.so" not found, required by "CeeloPartyServer"
Why isn't it finding the library when I try to run it?
My exact linker line is: -lm -lmysql -rpath=$ORIGIN.
I am pretty sure I don't have to escape $ or anything like that since my readelf analysis does in fact show that library rpath is set to $ORIGIN. What am I missing?
I'm assuming you are using gcc and binutils.
If you do
readelf -d CeeloPartyServer | grep ORIGIN
You should get back the RPATH line you found above, but you should also see some entries about flags. The following is from a library that I built.
0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../lib]
0x000000000000001e (FLAGS) ORIGIN
0x000000006ffffffb (FLAGS_1) Flags: ORIGIN
If you aren't seeing some sort of FLAGS entries, you probably haven't told the linker to mark the object as requiring origin processing. With binutils ld, you do this by passing the -z origin flag.
I'm guessing you are using gcc to drive the link though, so in that case you will need to pass flag through the compiler by adding -Wl,-z,origin to your gcc link line.
Depending on how many layers this flag passes through before the linker sees it, you may need to use $$ORIGIN or even \$$ORIGIN. You will know that you have it right when readelf shows an RPATH header that looks like $ORIGIN/../lib or similar. The extra $ and the backslash are just to prevent the $ from being processed by other tools in the chain.
\$\ORIGIN if you are using chrpath and \$\$ORIGIN if you are providing directly in LDFLAGS
using ldd CeeloPartyServer to check the dependency .so is starting with ./ or not. (e.g. libFoundation.so and ./libFoundation.so)
For common situation it should be libFoundation.so and without the prefix ./
if ./ prefix is necessary for some uncommon case, make sure the CWD is the same folder with libFoundation.so, and the $ORIGIN would be invalid.
=======
For example:
g++ --shared -Wl,--rpath="\$ORIGIN" ./libFoundation.so -o lib2.so
would got a library lib2.so with ./libFoundation.so
g++ --shared -Wl,--rpath="\$ORIGIN" libFoundation.so -o lib2.so
would got libFoundation.so instead.
I'm trying to use cygwin as a build environment under Windows. I have some dependencies on 3rd party packages, for example, GTK+.
Normally when I build under Linux, in my Makefile I can add a call to pkg-config as an argument to gcc, so it comes out like so:
gcc example.c `pkg-config --libs --cflags gtk+-2.0`
This works fine under Linux, but in cygwin I get:
:Invalid argument
make: *** [example] Error 1
Right now, I am just manually running pkg-config and pasting the output into the Makefile, which is truly terrible. Is there a good way to workaround or fix for this issue?
Make isn't the culprit. I can copy and paste the command line that make uses to call gcc, and that by itself will run gcc, which halts with ": Invalid argument".
I wrote a small test program to print out command line arguments:
for (i = 0; i < argc; i++)
printf("'%s'\n", argv[i]);
Notice the single quotes.
$ pkg-config --libs gtk+-2.0
-Lc:/mingw/lib -lgtk-win32-2.0 -lgdk-win32-2.0 -latk-1.0 -lgdk_pixbuf-2.0 -lpang
owin32-1.0 -lgdi32 -lpangocairo-1.0 -lpango-1.0 -lcairo -lgobject-2.0 -lgmodule-
2.0 -lglib-2.0 -lintl
Running through the test program:
$ ./t `pkg-config --libs gtk+-2.0`
'C:\cygwin\home\smo\pvm\src\t.exe'
'-Lc:/mingw/lib'
'-lgtk-win32-2.0'
'-lgdk-win32-2.0'
'-latk-1.0'
'-lgdk_pixbuf-2.0'
'-lpangowin32-1.0'
'-lgdi32'
'-lpangocairo-1.0'
'-lpango-1.0'
'-lcairo'
'-lgobject-2.0'
'-lgmodule-2.0'
'-lglib-2.0'
'-lintl'
'
Notice the one single quote on the last line. It looks like argc is one greater than it should be, and argv[argc - 1] is null. Running the same test on Linux does not have this result.
That said, is there, say, some way I could have the Makefile store the result of pkg-config into a variable, and then use that variable, rather than using the back-tick operator?
That said, is there, say, some way I could have the Makefile store the result of pkg-config into a variable, and then use that variable, rather than using the back-tick operator?
GTK_LIBS = $(shell pkg-config --libs gtk+-2.0)
Are you sure that you're using the make provided by Cygwin? Use
which make
make --version
to check - this should return "/usr/bin/make" and "GNU Make 3.8 [...]" or something similar.
Hmmm... have you tried
make -d
That will give you some (lots) of debugging output.
My guess would be that cygwin's gcc can't handle -Lc:/mingw/lib. Try translating that to a cygwin path.
GTK_LIBS = $(patsubst -Lc:/%,-L/cygdrive/c/%,$(shell pkg-config --libs gtk+-2.0))
The single quote at the end of the "t" output may be an artifact of CRLF translation. Is your pkg-config a cygwin app? The $(shell) solution I posted earlier may help with this, as GNU make seems to be fairly tolerant of different line ending styles.
I had a similar issue and I found a fix here: http://www.demexp.org/dokuwiki/en:demexp_build_on_windows
Take care to put /usr/bin before /cygwin/c/GTK/bin in your PATH so that you use /usr/bin/pkg-config. This is required because GTK's pkg-config post-processes paths, often transforming them in their Windows absolute paths equivalents. As a consequence, tools under cygwin may not understand those paths.