So, I've got that makefile project that has a huge list of object files that need to be compiled.
I already ran into problems on Win32 because the input string is too large. I figured out that instead of passing the files 1-by-1 to the linker, I could read the object files to be linked from a file by passing #filename to the linker.
In my makefile: Is there a method how the dependencies could be read out of a file? Something like that:
main.lib: #dependency_file_name
where dependency_file_name holds the list of needed .obj files needed to create the lib.
Please keep in mind, that putting the contents of the file into a variable also doesn't work since I've got that problem with a too long input string also when using the list of dependencies as a variable.
In addition: the method shall be portable (Linux, Win32)
Edit: currently, the structure of my makefiles is as follows:
# slurp in some global settings
include /path/to/global/settings
all: prepare_target mylib cleanup_target
prepare_target:
# do preparing work (setup temp files etc)
cleanup_target:
# do cleanup work (delete temp files etc)
mylib: \
file1.obj \
file2.obj \
... \
file215.obj
# this file holds the dependencies for each .obj file
include file_with_obj_file_dependencies
There are really 215 C files to be compiled into this lib. As I said: when feeding this list to the linker, I need to do it via a file because the string is too large for the command line.
I'm free to change whatever is needed in the makefile: it's generated from a VisualStudio vcxproj and I own the generation templates.
file_with_obj_dependencies looks as this:
file1.obj: \
file1.c \
file1.h \
file2.obj: \
file2.c \
file2.h \
... \
file215.obj \
file215.c \
file215.h \
On Windows I tend to use cygwin. It's the closest environment I've found to unix. This makes your Makefiles portable. I haven't come across any limiting command-line length limits. Here is an example with over ¾ of a million command-line arguments:
a := 1 2 3 4 5 6 7 8 9 a b c d e f
.PHONY: test
test:
#echo $(foreach A,$a,$(foreach B,$a,$(foreach C,$a,$(foreach D,$a,$(foreach E,$a,$A$B$C$D$E))))) | wc -wc
(the wc is so that you don't have to wait for the 5,000,000 or so characters to scroll by.)
So, for the limited win32 tools, use bash to write the intermediate command-line-parameters file, then pass that to link.exe (say) prefixed with an #.
Something like:
mylib.lib:
echo $^ >$#-tmp
link -o $# #$#-tmp
(Are you sure you need the intermediate file? 215 source files doesn't seem too onerous.)
Related
I am completely new to using autotools so it might be a dumb question but I'll try anyway. I have two pieces of Makefile.am. Except one is working fine and the other is not.
This works fine.
sbin_PROGRAMS = kernel
kernel_SOURCES = \
src/arch/$(host_cpu)/arch_sysdefs.h \
src/arch/$(host_cpu)/boot.asm \
src/arch/$(host_cpu)/cpu.asm \
src/arch/$(host_cpu)/isr.asm \
src/kmain.cpp
But this doesn't. .asm files are completely ignored by generated Makefile.
if HOST_CPU_X86
ASM_EXT = .asm
else
ASM_EXT = .S
endif
sbin_PROGRAMS = kernel
kernel_SOURCES = \
src/arch/$(host_cpu)/arch_sysdefs.h \
src/arch/$(host_cpu)/boot$(ASM_EXT) \
src/arch/$(host_cpu)/cpu$(ASM_EXT) \
src/arch/$(host_cpu)/isr$(ASM_EXT) \
src/kmain.cpp
What I am trying to do is that I want to use different suffixes for assembly files for some CPUs my project is going to support.
I've also added necessary rule to transform .asm to object files.
.asm.o:
yasm -f $(YASM_OUT_FMT) $< -o $#
EDIT: Temporarily overriding .cpp.o rule with echo $(kernel_SOURCES) reveals that $(ASM_EXT) in kernel_SOURCES is substituted correctly. For example src/arch/$(host_cpu)/boot$(ASM_EXT) becomes src/arch/x86_64/boot.asm for x86-64 CPU and src/arch/arm/boot.S for ARM, etc. Also, setting ASM_EXT variable from autoconf.ac doesn't make any difference.
I see a file .missing-syscalls.d which contains
-: /home/ckim/MVP/snake_linux_3.3.mvpe/include/linux/kconfig.h \
include/generated/autoconf.h \
/home/ckim/MVP/snake_linux_3.3.mvpe/arch/sparc/include/asm/types.h \
include/asm-generic/int-ll64.h \
/home/ckim/MVP/snake_linux_3.3.mvpe/arch/sparc/include/asm/bitsperlong.h \
include/asm-generic/bitsperlong.h \
/home/ckim/MVP/snake_linux_3.3.mvpe/arch/sparc/include/asm/unistd.h
I know .d file is only for showing dependancy. But what is '-' as the target?
Hyphen prior to commands in makefiles is used to suppress errors and continue instead of failing like this:
clean:
-rm -f *.o
The same way goes for include syntax in makefiles to suppress the error messages that would otherwise appear if the file isn't available like this:
-include $(SRC:%.c=%.d)
I would assume the .missing-syscalls.d file is used the same way, ignoring non-existing header files it attempts to include.
Either that's an error (from some automated process gone wrong) and that's a literal target of - which defines some extra prereqs (for a target that I can't imagine is ever likely to actually run).
Or that's an attempt to ensure that those files aren't considered as intermediate files by make in case some chain of rules would cause it to consider them as such.
See http://www.gnu.org/software/make/manual/make.html#Chained-Rules for reference.
Ordinarily, a file cannot be intermediate if it is mentioned in the
makefile as a target or prerequisite. However, you can explicitly mark
a file as intermediate by listing it as a prerequisite of the special
target .INTERMEDIATE. This takes effect even if the file is mentioned
explicitly in some other way.
Consider having a makefile, which can generate some files using generating lines listed in a file. For example file 001 using first line, 002 using second line, etc.
This file can be changed (it has it's own dependences, but this doesn't metter).
If some lines in this file changed, appropriate files should be remade. But other files shouldn't.
The solution I found is this one: for every file there is flag-file which content is generating line it was made last time. After remaking file with generating lines, i check all this file, and remove them if line changed. So files which have dependences on removed files will be remade, and other files won't. But this works too slow if use msys-make.
Can you suggest any other solution, which doesn't need many extra calls to file system and executable runs.
If I understand your description correctly, what you're describing is a Makefile which depends on another file that is functionally a Makefile but for unknown reasons uses a different format.
Turn that file into Makefile format and include it into the original Makefile. (If you're using GNU Make.)
We have what may be similar to your problem. We have xml files, say foobar.xml which contains the dependencies for foobar.out
<files>
<file>a</file>
<file>b</file>
<file>c</file>
</files>
We decided to adhere to this simple layout, so we don't need to parse xml. We use makepp (because we got fed up with gmake not noticing dependencies like changed commands). This gives us the builtin &sed command (the expression is actually Perl programming, but as you see, you don't need to get into it much). Here's what we do with three simple substitutions for the three kinds of lines:
%.d: %.xml
&sed 's!<files>!$(stem).out: \\! || s!<file>(.+)</file>!$$1 \\! || s!</files>!!' \
$(input) -o $(output)
include foobar.d
This produces foobar.d which we then include:
foobar.out: \
a \
b \
c \
Stackoverflow is swallowing the last empty line here, which avoids having to worry about the trailing backslash.
as a part of build process I need to process some files with m4. These files are given an '.in' extension and are located not only in the top-level dir, but also in subdirs.
I locate them using find and process them in a shell loop.
Is there some makefile syntax magic to write simpler rule to process them save the output into a file (the '.in' extension stripped) in the same directory as the input file?
This is what I have in my makefile now:
PROCESS_FILES=$(shell find . -name \*.in)
WORK_FILES=$(subst .in,,$(PROCESS_FILES))
$(WORK_FILES): $(PROCESS_FILES)
for file in $(PROCESS_FILES); \
do \
m4 $$file > $${file%.*};\
done
You could write a set of rules to convert files with the suffix .in to files without that suffix.
In the classic notation, that would be something like:
# Not functional — see discussion
.SUFFIXES: .in # Add .in as a suffix
M4 = m4
M4FLAGS =
M4SCRIPT = xyz.m4
.in:
${M4} ${M4FLAGS} ${M4SCRIPT} $< > $*
This adds .in as a recognized suffix, and says that you convert the file with a .in suffix to the file without it using the command specified by the three macros. The $< is the name of the file with the .in extension; the $* is the name of the file without the extension.
Unfortunately though, that notation only works when the files to be converted have names such as xyz.in and need to be converted to xyz. It does not work for a case where xyz.h.in needs to be converted to xyz.h.
It would be possible to specify that the suffixes are .h.in and .h, but then the rule for converting between the two starts with .h.in.h: and make gets confused.
However, GNU make has an alternative notation for defining suffix rules which can accommodate this:
.SUFFIXES: .h.in .h # Add .h.in and .h as suffixes
M4 = m4
M4FLAGS =
M4SCRIPT = xyz.m4
%.h: %.h.in
${M4} ${M4FLAGS} ${M4SCRIPT} $< > $*.h
The %.h: %.h.in line is able to deal with the suffix with two dots. The only nuisance is that for each extension such as .c.in or .y.in or .mk.in, you have to provide similar mappings.
(Tested on Mac OS X 10.7.4. If that makefile is xyz.mk, you can create an empty file xyz.h.in and then run make -n -f xyz.mk xyz.h and you'll see the build would run the command m4 xyz.m4 xyz.h.in > xyz.h.)
If you have a source file path/to/file.in, a rule for making % from %.in, and a dependency on the output file path/to/file, that should be all you need.
Perhaps you want to express your dependencies in terms of the output from find. In GNU Make this is easy; in other dialects, perhaps you want to generate and include something like find -name '*.in' | sed 's/\.in$//' >make.dep
I have an application with several subdirectories, which I want to compile non-recursive. For this I have seperated all sorucefiles from the subdirectories into several variables, which I then use in the final collection of sources. Something like this:
GUI_SOURCEFILES = Gui/MainWindow.cc \
Gui/StatusBar.cc
...
foo_SOURCES = $(GUI_SOURCEFILES) \
$(DATABASE_SOURCEFILES) \
main.cc
Now however this forces me to write Gui/ for all gui sourcefiles and Db\ in front of all database files. I think it should be possible to create this prefix automaticall, but I cannot find a way to do this correctly. I tried the usual make way:
GUI_SOURCEFILES = MainWindow.cc \
StatusBar.cc
...
foo_SOURCES = $(GUI_SOURCEFILES) \
$(patsubst %,Gui/%,$(DATABASE_SOURCEFILES)) \
main.cc
But autotools will not compile this Makefile.am at all.
Is there a way to get autotools to do this for me?
There is no way here, all filenames must be available at automake time, and that precludes certain make-time like functions (non-portable at that).