The kbuild document says that:
ldflags-y... [applies] only to the kbuild makefile in which they are assigned....[is used] for all ld invocations happening during a recursive build.
while
LDFLAGS... [is used] for all invocations of the linker.
I cannot see any difference between them, given a Makefile m (and we type make m)
suppose LDFLAGS := x, because it applies for all invocations of the linker, then all ld invoked in any build session starting from make m has value x for LDFLAGS
suppose ldflags-y := y , becaus if applies for all recursived invocations of ld, then all ld invoked (by any build session starting from make m) has also value y for LDFLAGS
Is this interpretation correct?
[applies] only to the kbuild makefile in which they are assigned
means that the link flags defined by that variable in a given makefile are used only in rules appearing in the same makefile. They are not inherited from a parent makefile, and they are not transmitted to sub-makes controlled by other makefiles.
[is used] for all ld invocations happening during a recursive build.
This is a statement about the facility in general, not about specific flags designated via any particular use of that facility. It says that you can use an ldflags-y variable in any kbuild makefile, and expect to see the behavior described in the docs -- linker invocations in rules appearing directly in the same makefile will use those flags, but they will not automatically be conveyed to sub-makes in other directories.
On the other hand, this ...
LDFLAGS... [is used] for all invocations of the linker.
... appears in the documentation for architecture makefiles. The point of these, as described in the docs, is to define flags appropriate for the overall build and (substantially) everything in it. This variable should be defined only in such a makefile, and the quotation says that the linker flags defined that way are used in all linker runs during the build, no matter in what makefile the corresponding rule appears (including, notably, in directories that are not in the same tree as the architecture makefile).
The docs go on to say that ldflags-y can be used for further customization, which you should take to indicate that both are applied in linker runs. Reading a bit more closely between the lines, you should expect the (global) LDFLAGS to appear earlier in the linker command than the (local) ldflags-y.
I cannot see any difference between them, given a Makefile m (and we
type make m)
suppose LDFLAGS := x, because it applies for all invocations of the linker, then all ld invoked in any build session starting from
make m has value x for LDFLAGS
Yes and no. A general kbuild makefile m shouldn't provide a definition for LDFLAGS. That belongs only in the appropriate architecture makefile. It's unclear what would happen if a kbuild makefile did so anyway, but I suspect that it would cascade to sub-makes.
Additionally, you shouldn't attempt to make a subtree directly if it is configured for kbuild. The system is not designed for that.
suppose ldflags-y := y , becaus if applies for all recursived invocations of ld, then all ld invoked (by any build session starting
from make m) has also value y for LDFLAGS
No. The ability to use an ldflags-y variable to provide local link-flag customizations applies to all kbuild makefiles. The specific flags provided that way by a particular makefile do not cascade to sub-makes.
The help from a reddit user clarifies the difference: actually ldflags-y applies for ld invocations happen in descending builds, but not for the builds of the current Makefile. In short, LDFLAGS defines the top, then descending builds may custom this value using ldflags-y.
Related
Normally, I would compile a program that requires a specific library, e.g. math, by passing the linker flag after the sources that need it like so:
gcc foo.c -lm
However, it seems that older versions of gcc work equally well with the reverse order (let's call this BAD ORDER):
gcc -lm foo.c
I wouldn't worry about it if some popular open-source projects I'm trying to compile didn't use the latter while my version of gcc (or is it ld that's the problem?) work only in the former case (also, the correct one in my opinion).
My question is: when did the BAD ORDER stop working and why? It seems that not supporting it breaks legacy packages.
when did the BAD ORDER stop working and why? It seems that not supporting it breaks legacy packages.
When?
Not dead sure but I think pre-GCC 4.5. Long ago. Subsequently, the --as-needed option is operative for shared libraries by default,
so like static libraries, they must occur in the linkage sequence later than the objects for which they provide definitions.
This is a change in the default options that the gcc/g++/gfortran etc. tool-driver passes to ld.
Why?
It was considered confusing to inexpert users that static libraries by default has to appear
later that the objects to which they provided definitions while shared libraries by default did
not - the difference between the two typically being concealed by the -l<name> convention
for linking either libname.a or libname.so.
It was perhaps an unforeseen consequence that inexpert users who
had formerly had a lot of luck with the mistaken belief that a GCC
[compile and] link command conforms to the normal Unix pattern:
command [OPTION...] FILE [FILE...]
e.g.
gcc -lthis -lthat -o prog foo.o bar.o
now fare much worse with it.
Could somebody provide a real-world example of using recursively expanded variables (REV)? In the docs or various blog posts people give only useless toy examples, like
foo = $(bar)
bar = $(ugh)
ugh = Huh?
I cannot find a real use for REV besides creating custom functions with $(call). I also found that in the past people were using REV to supply additional parameters to a compiler for specific targets but that trick is considered outdated now because GNU Make has target-specific variables.
Both recursively expanded variables and simply expanded variables recurse their expansions. The major difference is when that expansion happens.
So your example above works just fine with simply expanded variables if you invert the assignments:
ugh := Huh?
bar := $(ugh)
foo := $(bar)
So the major thing that recursively expanded variables get you is the freedom to assign values in whatever order you need (which means you don't need to worry about inclusion order for included makefiles, etc.).
In a project at work we have a dozen or so included makefiles that have inter-dependent relationships. These are expressed through the usage of known-format variable names (e.g. module A generates an A_provides variable, etc.) Modules that need to utilize the things that module A provides can then list $(A_provides) in their makefiles.
With simply expanded variables (which we had been using until somewhat recently) this meant that the inclusion of the generated makefiles required a manually sorted order to force the inclusion of assigning makefiles before consuming makefiles (into other variables).
With recursively expanded variables this order does not matter. (This would not be the case if the variables were used in any immediately evaluated context in these makefiles but luckily they are not, they only set variables that are used later in the main makefiles.)
Well, one simple example are variables which contain commands for recipes; perhaps:
buildc = $(CC) $(CPPFLAGS) $(CFLAGS) -c -o $# $<
%.o: %.c
$(buildc)
I probably wouldn't write a compile rule like this this way, but if you have a much more complex recipe it can be very useful.
Personally I don't consider "additional parameters ... for specific targets" (by which I assume you mean recursively defined variables such as $($*_FLAGS)) to be outdated or obsoleted by target-specific variables, by any stretch. If nothing else recursively defined variables are much more portable than target-specific variables.
It just means that recursively defined variables are set at the time of definition only!
The best example I can find is this
x := foo
y := $(x) bar
x := later
and is equivalent to
y := foo bar
x := later
I have a program foo I'm trying to compile and link and I'm running into a chicken and egg dillemma.
For reasons I'll explain below, Within a given directory I'm forced to add a link to several libraries we build (let's call them libA and libB) regardless of my target. I know I only actually need libA for my program; so after all libs are built and this binary is built I verified with ldd -u -r foo to show that libB is an unused direct dependency.
Being unused I altered the makefiles and flags such that libB is enveloped with -Wl --as-needed and -Wl --no-as-needed. I make rebuild, use ldd again and this time it doesn't show any unused deps. So far so good.
Now the fun part: Since its unused I would expect that if libB is not found/available/built that I should still be able to compile and link foo as long is libA is available. (example: If I did a fresh checkout and only built libA before trying to compile this specific test). But ld errors out with /usr/bin/ld: cannot find -lB
This suggests that ld needs to locate libB even if it won't need any of the symbols it provides? That doesn't seem to make sense. If all symbolic dependencies are already met, why does it even need to look at this other library? (That would explain the problem ld has and why this is not possible)
Is there a way I can say "Hey don't complain if you can't find this library and we shouldn't need to link with it?"
The promised reasons below
For various reasons beyond my control I have to share makeflags with many other tests in this directory due to the projects makefile hierarchy. There is a two level makefile for all these tests that says foo is a phony target, his recipe is make -f generictest.mk target=foo, and the generictest.mk just says that the source file is $(target).C, that this binary needs to use each library we build, specifies relative path to our root directory and then includes root's generic makefile. The root directory generic makefile expands all the other stuff out (flags, options, compiler, auto-gen of dependencies through g++ etc), and most importantly for each statement that said "use libX" in generictest.mk it adds -lX to the flags (or in my case enveloped in as-needed's)
While I'm well aware there are lots of things that are very unideal and/or horribly incorrect in terms of makefile best practices with this, I don't have the authority/physical ability to change it. And compared to the alternative employed in other folders, where others make individual concrete copies of this makefile for each target, I greatly prefer it; because that forces me to edit all of them whenever want to revise our whole make pattern, and yields lot of other typos and problems.
I could certainly create another generictest.mk like file to use for some tests and group together those using each based on actual library needs, but it would be kind of neat if I didn't have to as long as I said "you don't all of them, you need each of them but only if you actually use it".
There's no way that the linker can know that the library is not needed. Even after all your "normal" libraries are linked there are still lots and lots of unresolved symbols: symbols for the C runtime library (printf, etc.). The linker has no idea where those are going to come from.
Personally I'd be surprised if the linker didn't complain, even if every single symbol was already resolved. After all there may be fancy things at work here: weak bindings, etc. which may mean that symbols found later on the link line would be preferred over symbols found earlier (I'm not 100% sure this is possible but I wouldn't be surprised).
As for your situation, if you know that the library is not needed can't you just use $(filter-out ...) on the link command line to get rid of it? You'd have to write your own explicit rule for this with your own recipe, rather than using a default one, but at least you could use all the same variables.
Alternatively it MIGHT be possible to play some tricks with target-specific variables. Declare a target-specific variable for that target that resets the variable containing the "bad library" with a value that doesn't contain it (maybe by using $(filter-out ...) as above), and it will override that value for that target only. There are some subtle gotchas with target-specific variables overriding "more general" variables but I think it would work.
I'm working on re-writing an old build that was originally "designed" (or not, as the case may be) to be recursive. To preface, there will come a day when we'll move to something more modern, expressive, and powerful (eg, scons); however, that day is not now.
As part of this effort I'm in the process of consolidating what should be generic variables/macros & targets/recipes into a few concise rulefiles that'll be included as part of the primary build. Each sub-section of the build will use a small makefile that adds targets & dependencies with little in the way of variables being added in these sub-makefiles. The top level makefile will then include all the makefiles, allowing everything to contribute to the dependency tree.
I must admit I'm not at all confident that people will use good judgement in modifying makefiles. As an example of what I'm worried about:
CFLAGS = initial cflags
all: A.so
%.so %.o:
#echo "${#}: ${CFLAGS} : ${filter 5.o,${^}} ${filter %.c,${^}"
%.c :
true
%.o : %.c
A.so : B.so a1.o a2.o a3.o
B.so : b1.o b2.o b3.o
A.so : CFLAGS += flags specific to building A.so
Provided I didn't screw up copying that example, the situation is thus: A.so would link to B.so, and A.so's objects need special flags to be built; however, B.so and B's objects will inherit the changes to CFLAGS.
I would prefer to have a single target for building most if not all object files, even to the extent of modifying CFLAGS specifically for those those objects that need slightly different flags, in order to promote re-use of more generic targets (makes debugging easier if there's only one target/recipe to worry about).
After I finish re-architecting this build, I'm not at all confident someone won't do something stupid like this; worse, it's likely to pass peer review if I'm not around to review it.
I've been kicking around the idea of doing something like this:
% : CFLAGS = initial cflags
... which would prevent dependency poisoning unless someone then updates it with:
% : CFLAGS += some naive attempt at altering CFLAGS for a specific purpose
However, if there's, just 1000 targets (an extremely conservative estimate), and approximately 1k in memory allocated to variables, then we're up around 1mb of overhead which could significantly impact the time it takes to lookup the CFLAGS value when working through recipes (depending on gmake's architecture of course).
In short, I suppose my question is: what's a sane/good way to prevent dependency poisoning in a makefile? Is there a better strategy than what I've outlined?
edit
If anyone out there attempts to go down the path of scoping variables as described above, I ran into a nuance that wasn't entirely obvious at first.
% : INCLUDES :=
# ...
SOMEVAR := /some/path
% : INCLUDES += -I${SOMEVAR}
...
SOMEVAR :=
When a variable is created using :=, everything to the right of := should be evaluated immediately, whereas if it just used = it would delay evaluation until the target recipe evaluates INCLUDES.
However, SOMEVAR evaluates to nothing when a target recipe is evaluated. If you change the definition to:
% : INCLUDES := whatever
# ...
SOMEVAR := /some/path
% : INCLUDES := ${INCLUDES} -I${SOMEVAR}
...
SOMEVAR :=
... then it forces SOMEVAR to be evaluated immediately instead of delaying evaluation, but INCLUDES doesn't evaluate to its previously scoped definition, rather to the global definition.
$(flavor ...) says INCLUDES is simple, and $(origin ...) returns file; this occurs whether you use := or +=.
In short, if you use += on scoped variables, it'll only use the definition of the variable scoped to that target; it doesn't look at globals. If you use :=, it only uses globals.
If you abstain from unusual characters in your filenames, you can select target-specific variables with variable name substitution:
A.so_CFLAGS = flags specific to building A.so
%.so %.o:
#echo "${#}: ${CFLAGS} ${$#_CFLAGS} : ${filter %.o,${^}} ${filter %.c,${^}}"
Which obviously doesn't propagate the name of the currently built archive to the objects built for that library, but I don't know if this is desired.
This approach has some obvious deficiencies, naturally, like the inability to actually override CFLAGS. However, considering that automake has the same problem to solve and resorts to stupid text substitution, I think a lot of people already failed to find a nice solution here.
As a side-note, you might consider using automake instead of re-engineering it.
I don't know how feasible it is and how sensible is this question here.
Is there any changes that we can make in makefile to recommend GCC inline all the function although the functions are not inlined during the declaration or nowhere in the source file.
There are a few ways you can make gcc inline functions. One of them is the option -finline-functions, which will make gcc inline "simple" functions. The compiler uses some heuristics to determine whether the function is small enough to be inlined. However, the user has some control over this algorithm through -finline-limit. Read the gcc manual to find the actual values you need.
When inlining functions you should remember that obviously not all functions can be inlined (the simplest example being recursive functions) and the compiler can inline only functions defined within the same translation unit. Also, it is worth mentioning that -finline-functions is on by default at -O3, so just -O3 may sometimes be your solution.
In the makefile you will have to add the right options to all calls to gcc. In a well written makefile you'll easily spot variables with other gcc options, where you can simply place your own.
The gcc -finline_functions option sounds like it might do what you want. Here is some documentation. If your makefile defines a CFLAGS variable, that would be the place to put it.