I'm working on embedded project, which consists of my own code as well as 3rd party libraries and executables. To build all the parts consistently, I've written a script, which sets environment variables for cross-compilation (CC, CXX, CFLAGS, etc.). Among others it sets LDFLAGS to pass the rpath flag to linker. The rpath value contains $ORIGIN token, which must not be expanded and must be seen by linker and written to output binary as is. I then build several needed 3rd party projects using the environment set by the script. The projects uses different build systems (make, CMake, others maybe). Because of this and maybe because of the build scripts written in different ways, the dollar sign is expanded differently. I.e., whatever escaping I try, I get different results in different projects (e.g., $$ORIGIN, RIGIN, empty string), but never I managed to get the same $ORIGIN value in all the binaries. Is there a universal way to escape dollar sign so that it will work the same in at least make and shell, but in any combination?
This is how I've finally solved this problem.
In addition to the previous environment variables, needed to build for my platform, I've added two more:
ORIGIN=$ORIGIN
O=$$O
The former is to workaround shell expansion, and the latter is to workaround makefile expansion. With this fix, variables are resolved to themselves.
Yes, this does not look like an ideal solution, looks more like a hack, but it works so far allowing me to avoid adapting my build environment for every third party project I use.
I've hit the same expansion problem, and here is the adapted version for a bash script.
LDFLAGS="-Wl,-rpath=\$ORIGIN"
LDFLAGS="-lfoo $LDFLAGS"
LDFLAGS="-L. $LDFLAGS"
echo $LDFLAGS
# -L. -lfoo -Wl,-rpath=$ORIGIN # <== correct ORIGIN
ORIGIN='$ORIGIN'
eval echo $LDFLAGS
# -L. -lfoo -Wl,-rpath=$ORIGIN # <== correct ORIGIN
Related
I'm using a $CONFIG_SITE set to $ZPFX/share/config.site file containing:
CPPFLAGS="-I$ZPFX/include $CPPFLAGS"
LDFLAGS="-L$ZPFX/lib $LDFLAGS"
where $ZPFX variable is my custom user prefix, similar to ~/.local.
Now, the problem is that the system iconv.h (under path /usr/include) isn't found because of this, as only the above pre-set (with use of config.site) -I$ZPFX…/-L$ZPFX… are being passed to the test program, as the following lines from config.log are showing:
configure:21831: gcc -o conftest -g -O2 \
-I/home/q/.local/share/zinit/polaris/include \
-L/home/q/.local/share/zinit/polaris/lib \
conftest.c -lxml2 -liconv >&5
/usr/bin/ld: cannot find -liconv
collect2: error: ld returned 1 exit status
($ZPFX is expanded to its value, which is: /home/q/.local/share/zinit/polaris)
The Question: how to append (prepend) any custom ($ZPFX/… in my case) directories to CPPFLAGS/LDFLAGS, preserving their default values? So that my custom libraries are only given higher precedence, and not exclusivity?
What I've tried
As it can be seen, I've tried to prepend to the flags by appending any occurred values at the time of sourcing the config.site file, by "…$CPP…/$LDFLAGS" :
CPPFLAGS="-I$ZPFX/include $CPPFLAGS"
LDFLAGS="-L$ZPFX/lib $LDFLAGS"
However this has no effect.
I'm also waving between not-appending and appending any typically used system-libraries paths: /usr/include and /usr/lib{,64}, however I don't like the idea, because some system might use e.g.: /opt/… for main prefix, making such hack not working at all.
The documentation explains how configure scripts use CONFIG_SITE. The parts most relevant to your question are near the beginning:
If the environment variable CONFIG_SITE is set, configure uses its
value as a space-separated list of shell scripts to read [...].
Otherwise, it reads the shell script prefix/share/config.site if it
exists, then prefix/etc/config.site if it exists. [...]
Site files can be arbitrary shell scripts
You ask:
how to append (prepend) any custom ($ZPFX/… in my case) directories
to CPPFLAGS/LDFLAGS, preserving their default values? So that my
custom libraries are only given higher precedence, and not
exclusivity?
You need to read only a little between the lines to recognize that the mechanism by which configure scripts read site defaults must be via the . command. That the files may contain arbitrary shell code and that they can set shell variables within configure implies that they are going to be parsed and executed as shell code.
The docs also say what configure does by default when you do not provide CONFIG_SITE, so your site config can do that, too, if you wish. And that's what I would do, to start. Specifically, add this at the beginning of $ZPFX/share/config.site:
test -r "${prefix}/share/config.site" && . "${prefix}/share/config.site"
test -r "${prefix}/etc/config.site" && . "${prefix}/etc/config.site"
Now you have the same settings that configure would get when you don't define CONFIG_SITE at all. What remains is to insert your own additional settings, and the code already presented in the question ought to be fine for that:
CPPFLAGS="-I$ZPFX/include $CPPFLAGS"
LDFLAGS="-L$ZPFX/lib $LDFLAGS"
Additionally, you write:
I'm also waving between not-appending and appending any typically used
system-libraries paths: /usr/include and /usr/lib{,64}, however I
don't like the idea, because some system might use e.g.: /opt/… for
main prefix, making such hack not working at all.
I don't like that idea either.
In the first place, /usr/include, /usr/lib, etc. are not just commonly used, they are typical toolchain defaults. You pretty much never need to explicitly specify toolchain defaults, and trying to do so is more likely to cause harm than good. For example, it could cause breakage in cases such as you posit, where the defaults for the toolchain actually being used are different from the usual ones.
But in the second place, your remarks convey an incorrect perspective on what you're doing with site defaults. These are site-specific, which may mean machine specific or may mean more broadly specific to a group or organization, but any way around, you can set only your own site defaults, not those of other sites.
And that leads me to one final point: the site configuration is part of your site, not part of your project. Hopefully this is not a point of confusion for you, but I want to be sure it is clear. You should not be planning to distribute your site configuration file outside your own site, nor to rely on people wanting to build your project at other sites specifically to use a site configuration file to inject build settings.
I want to use the environment variable CPATH for compiling my application with GCC. The GCC manual says that the list of paths assigned to CPATH must be separated by the special character PATH_SEPARATOR, which got defined when GCC was build. PATH_SEPARATOR is usually ';' on Windows and ':' elsewhere.
Since I want to build my code on multiple platforms I need to know the character PATH_SEPARATOR in my makefile (to concatenate the paths assigned to CPATH correctly).
Can I ask my used GCC, with which value for PATH_SEPARATOR it was build?
(Of course I can alternatively try to detect the host OS in GNU make, and speculate on a suitable path separator from there, but this is not exactly the same. (E.g. I expect unwanted problems with CygWin))
A bunch of projects foo-A-B, foo-B-C, foo-A-C etc. each depend on foo-A, foo-B, foo-C etc.
Each of foo-X installs a pkg-config file (foo-X.pc.in) which contains a variable srcdir=#datarootdir#/foo/foo-B. A foo-X-Y project needs to refer to files in Xsrcdir and Ysrcdir.
Currently we do it like this in configure.ac:
PKG_CHECK_MODULES([foo_X], [foo-X])
AC_ARG_VAR(XSRC, "Source directory for foo-X")
AS_IF([test -z "$XSRC"], [XSRC=`pkg-config --variable=srcdir foo-X`])
(so Makefile.am gets to have rules like compile "$XSRC"/file.bar $#). This also lets developers override XSRC on running ./configure.
My question: is there a more canonical way to use non-standard pkg-config variables in autotools configury/makefiles? For e.g. libdir, I see pkg-config sets the variables itself so no configure.ac line is needed apart from PKG_CHECK_MODULES; are there other m4 macros we should be using?
I know this is fairly late, but since somebody was asking me about this recently, I thought it might be worth answering this too.
What you're looking for is PKG_CHECK_VAR, indeed most of that code can be replaced by a single line:
PKG_CHECK_VAR([XSRC], [foo-X], [srcdir], ,
AC_MSG_FAILURE([Unable to find value for XSRC]))
The error message is a bit less clear than the one triggered by PKG_CHECK_MODULES, but it also triggers in case the srcdir variable is not defined.
I wrote some more details as part of my Autotools Mythbuster.
I understand that # suppresses printing of a command in a Makefile...
http://www.gnu.org/software/make/manual/make.html#Echoing
... and I understand that $# is the target name...
http://www.gnu.org/software/make/manual/make.html#Automatic-Variables
... but I can't find any information on what a line like this might mean:
variable=#value#
I'm not trying to fix anything here, just trying to better understand Makefiles.
Update: The "Makefile Subsitutions" section of the GNU autoconf manual explains that it's a value that is substituted by autoconf.
Typically you find this in Makefile.in files, which are processed by configure (which are in turn generated by autoconf) scripts.
In that case #X# will be replaced by the value of a shell variable $X, if configure is told so. If it's not, no occurrence in the input file will be touched by configure, hence leaving the replaceable string as it is. If you ask me these instances indicate slips in the build system.
Is there a way to change the specs file so that it will pass -march=native if nothing is specified in command line?
Related things in the default specs file is:
*cc1:
%(cc1_cpu)
*cc1_cpu:
%{march=native:%>march=native %:local_cpu_detect(arch) %{!mtune=*:%>mtune=native %:local_cpu_detect(tune)}} %{mtune=native:%>mtune=native %:local_cpu_detect(tune)}
I am not sure how specs works. Simply specifying -march=native before or after %(cc1_cpu) doesn't work. However, this line does take effect because GCC will report error if I put -something_wierd instead of -march=native.
Another thing I noticed is if I put %{march=i386:-something_wierd} before %(cc1_cpu), gcc reports error so looks like -march=i386 is always passed in if nothing is specified, so is there a way to distinguish between nothing specified and -march=i386 in specs file?
BTW, what does %> do? Seems like it is not specified in the documentation.
I am using MinGW's gcc-4.6.2.
Referring to your last question: The gcc 4.6.1 sources (gcc/gcc.c) contain the following comment on %>:
%>S Similar to "%<S", but keep it in the GCC command line.
For the sake of completeness following the comment for %< form the same file:
%<S remove all occurrences of -S from the command line.
Note - this command is position dependent. % commands in the
spec string before this one will see -S, % commands in the
spec string after this one will not.
To answer the first question in short: yes, but ....
... the only generic solution I found has the significant drawback that the -march option will be ignored, so every build is done as if -march=native had been specified. Anyhow there is a workaround to that.
1 The solution (without workaround)
Create a specs-file called let's say specs.nativealways containing:
*cc1_cpu:
%<march=* -march=native %>march=native %:local_cpu_detect(arch) %{!mtune=*:%>mtune=native %:local_cpu_detect(tune)} %{mtune=native:%>mtune=native %:local_cpu_detect(tune)}
When using the specs-file (for example by invoking gcc with the option -specs=specs.nativealways) the build will be done as if -march=native was specified (with the mentioned drawback that any occurrence of option -march=<arch> would have simply been ignored).
2 The workaround
To still by able to override the newly configured default behavior one can use a modified version of the specs-file described above, introducing a new option called -myarch using the same syntax as -march (except for -myarch=native, which won't work, which does not metter as native now is the default).
The modfied specs-file looks like this:
*cc1_cpu:
%<march=* %{myarch=*:%<myarch* -march=%* ; :-march=native %>march=native %:local_cpu_detect(arch) %{!mtune=*:%>mtune=native %:local_cpu_detect(tune)}} %{mtune=native:%>mtune=native %:local_cpu_detect(tune)}
PS: This has been tested with with gcc 4.6.2 on Linux, but should work on MinGW.
While not a direct answer to your question, you can reach a very similar effect by defining CFLAGS and CXXFLAGS in your shell's initialization file. 99% of the Makefiles are sufficiently standard to pick up the environment values and pass the flags to gcc.
*cc1_cpu:
+ %{!march*:-march=native}