We have an ActionScript (Flex) project that we build using GNU make. We would like to add an M4 preprocessing step to the build process (e.g., so that we can create an ASSERT() macro that includes file and line numbers).
We are having remarkable difficulty.
Our current strategy is:
Create a directory "src/build" (assuming source code is in src/ and subdirectories).
Within src/build, create a Makefile.
Run make inside src/build.
The desired behavior is, make would then use the rules we write to send the *.as files src/ and its subdirs, creating new *.as files under build. For example:
src/bar.as -> m4 -> src/build/bar.as
src/a/foo.as -> m4 -> src/build/a/foo.as
The obvious make rule would be:
%.as : ../%.as
echo "m4 --args < $< > $#"
This works for bar.as but not a/foo.as, apparently because make is being "smart" about splitting and re-packing directories. make -d reveals:
Trying implicit prerequisite `a/../foo.as'.
Looking for a rule with intermediate file `a/../foo.as'.
but we want the prerequisite to be "../a/foo.as". This (what we don't want) is apparently documented behavior (http://www.gnu.org/software/make/manual/make.html#Pattern-Match).
Any suggestions? Is it possible to write a pattern rule that does what we want?
We've tried VPATH also and it does not work because the generated .as files are erroneously satisfying the dependency (because . is searched before the contents of VPATH).
Any help would be greatly appreciated.
One option is to use a different extension for files that haven't been preprocessed. Then you can have them in the same directory without conflict.
As Anon also said, your source code is no longer Flex - it is 'to be preprocessed Flex'. So, use an extension such as '.eas' (for Extended ActionScript) for the source code, and create a 'compiler' script that converts '.eas' into '.as' files, which can then be processed as before.
You may prefer to have the Extended ActionScript compiler do the whole compilation job - taking the '.eas' direct to the compiled form.
The main thing to be wary of is ensuring that '.eas' files are considered before the derived '.as' files. Otherwise, your changes in the '.eas' files will not be picked up, leading to hair-tearing and other undesirable behaviours (head banging, as in 'banging head against wall', for example) as you try to debug code that hasn't changed even though the source has.
Related
I'm slowly losing my mind here. First, let me describe what it is I'm trying to do. We have a compiler that spews out weirdly formatted dependency files. To get these makefiles into a format GNU Make can understand, they need to be processed by a Perl script first. Technically, the Perl script doesn't convert the input dependency files it gets passed; instead it creates a new, properly formatted dependency file for each input dependency file.
Now, in order for GNU Make to know which translation units need recompiling and which don't, it obviously must have seen those dependency files before trying to make the translation unit targets, so we have the following line in our master makefile:
include $(PROCESSED_EXISTING_DEPENDENCY_FILES)
where $(PROCESSED_EXISTING_DEPENDENCY_FILES) is a list of all converted dependency files. My idea was to (ab-)use an automatically generated makefile whose recipe not only builds that makefile but also triggers the creation of all dependency files mentioned in the $(PROCESSED_EXISTING_DEPENDENCY_FILES) list and include that makefile just before including the converted dependency files. To ensure that the conversion takes place, the parent process of our Make process will delete the automatically created makefile first (we have a Perl wrapper process controlling GNU Make). The relevant part in the master makefile would look like this:
# Phony target that creates processed dependency files.
CONVERTED_EXISTING_DEPENDENCY_FILES :
<recipe here>
$(PRE_CONVERTED_DEPENDENCY_FILE_INCLUSION_HOOK) : CONVERTED_EXISTING_DEPENDENCY_FILES
$(info $(TARGET_BUILD_MESSAGE_PREFIX) Building $(notdir $#) ...)
$(file >$#,# Automatically generated makefile that gets included before including the existing, converted dependency files.)
$(file >>$#,$(DOLLAR)(info Including pre-converted-dependency-files-inclusion hook file ...))
$(file >>$#,)
include $(PRE_CONVERTED_DEPENDENCY_FILE_INCLUSION_HOOK)
include $(PROCESSED_EXISTING_DEPENDENCY_FILES)
We're already using the same basic principle in several other cases, and so far this has worked perfectly fine, but for some reason when I try this, GNU Make gets lost in an infinite loop where it will continuously re-revaluate the master makefile, include all other makefiles and then go back to re-revaluating the master makefile again.
The $(PRE_CONVERTED_DEPENDENCY_FILE_INCLUSION_HOOK) does get created, and if there are any dependency files to be converted, they are processed, too, but I'm still at a loss as to what causes this infinite loop in Make. We are using GNU Make 4.2.1 for Windows on a Windows 10 (64 bit) system.
I recommend you rework your model completely to avoid any recipes that know how to build included files, and instead follow the model for auto-dependency generation described in this post (based on how automake handles dependency generation).
Then add in the postprocessing step directly into the same recipe that generates the dependency files, rather than having a separate rule that does it. I don't think it's necessary to have two separate rules because you really don't want the intermediate step here: you just want to generate the make prerequisites definitions... similar to how normally we wouldn't have separate rules for preprocessing, compiling, assembling object files: one rule does that even though there are multiple steps involved.
I'm working on a rather large project in pure C, and for various reasons we are not using CMake.
We have a good system which uses various shell commands to require very little maintanence. It will automatically find new headers and C files and add the headers to the dependencies and will compile the C files and include them in the output. However, we've gotten to the point where any changes we make to the header files becomes a problem because it requires a recompilation of the entire project, rather than just the C files that include it directly or indirectly.
I've worked on a system to ensure that the only header files which are added as dependencies are ones which are required by the C file itself (recursively up the tree).
I had thought I would be able to solve this myself, but it seems that make has rather inconsistent rules for expansion of variables.
The solution I came up with was to try to use ag and get all the includes out of the file. This works as intended, and I pipe it into tr so that I can ensure that no newlines are added, thereby messing up make. The issue that I'm having though is in determining which file to search in. This is the recipe that it's using for the section in question:
$(GAMEOBJDIR)/%.o : $(GAMEDIR)/%.c $(shell ag -o '(?<=(^#include "))(.*?)(?=("$$))' $(GAMEDIR)/%.c | tr '\n' ' ')
$(CC) $(CFLAGS) $(if $(RELEASE),$(RELFLAGS),$(DBFLAGS)) -c -o $# $<
The problem is that $(GAMEDIR)/%.c is expanding to src/game/%.c and not src/game/<filename>.c. I'm unsure how to fix this issue based on the expansion rules of make.
Once I can figure this out I'll be able to make sure this walks up the chain of header files too, but until I can get this figured out I have no reason to work on that.
Make's rules for expansion are completely consistent... but sometimes they aren't what people want. That's not the same thing :)
Rules for expansion are explained in the manual. In your case you're working with a prerequisite list which means the variables are expanded as the makefile is parsed. That includes the shell command and all other variables. Because you are working with a pattern rule, it means that at the time the makefile is parsed it doesn't match any particular file, it's just defining the pattern rule itself. So, % can't be expanded here. It will be expanded later, when make starts walking the dependency graph and trying to locate ways to build targets.
In general your method cannot work efficiently. You could defer the expansion of these parts of the prerequisites list using Secondary Expansion. However, that means every time make wants to TRY to use this pattern it will run these commands--that will just be slow.
Have you considered using a more standard way to manage detecting prerequisites? Try reading this description for example to see if it would work for you.
Here is my problem: I have been using Java for many years and enjoy having many directories separating different areas of the code. For my current project I am writing Fortran code, which should compile under Windows and Unix/Linux. For Windows, I am using Eclipse/Photran with MinGW/gfortran tools to set up Makefiles.
Here is the desired project structure (deep nesting tree-like Java-like would be even nicer)
dir1/src/*.f95
dir1/make/Makefile_lib1.any
dir1/make/Makefile_lib1.win
dir1/make/Makefile_lib1.unix
dir2/src/*.f
dir2/make/Makefile_lib2.any
dir2/make/Makefile_lib2.win
dir2/make/Makefile_lib2.unix
...
dir_main/src/*.f or *.f95
dir_main/make/Makefile_main.any
dir_main/make/Makefile_main.win
dir_main/make/Makefile_main.unix
I would like to call make Makefile_main.unix, which would set up any Unix-specific variables and then include Makefile_main.any, Makefile_lib1.any, ...
(similar for making on Windows)
I got to the stage where I can see all source files in a given directory, e.g.
SRCS := $(wildcard $(SRC_DIR)/*.$(SRC_EXT))
Now I am struggling with how to make all dependencies as in Fortran 95 each source generates *.o and *.mod.
Is there a way to switch between directories when compiling so that all targets/dependencies do not have dir-path in their names? Note that I am calling make from some other service directory where the Eclipse project lives. Any suggestions how to proceed?
I really do not want to do the usual Fortran style of having just one directory with
all the mess together with the code.
There are two major strategies you can take.
You can place a makefile in each subdirectory and have it support targets like all, clean etc, then use recursive make invocations from the top-level makefile to make the same target (e.g. all) in every subdirectory.
Alternatively, you can handle it all in one make invocation, without recursing, but then you'll have to work with relative paths containing subdirectory names. Personally I don't see a problem with it, and I've maintained a system of makefiles based on this approach.
Here is what you can do in your case, assuming that SRC is set correctly to the list of relative paths to every source you need to compile.
# This replaces the SRC_EXT suffix with .o in each filename
OBJ = $(SRC:%.$(SRC_EXT)=%.o)
$(BINARY_NAME): $(OBJ)
...link command...
%.o: %.$(SRC_EXT)
...compile command...
My Makefile.am includes a file (with various defined variables), for example:
include make.config
...
The problem is that this file is in turn generated by a tool (i.e. config.generator.sh) based on some input file (i.e. input.dat). The straightforward and wrong idea would be to add a rule to generate make.config:
make.config : input.dat
config.generator.sh input.dat > make.config
include make.config
...
Although this content is perfectly working makefile on its own without automake, the idea is doomed with automake. The make.config file is included by automake before I even have a chance to execute make (and it fails as the file is not yet generated):
automake: cannot open < make.config: No such file or directory
Is there a way to postpone effect of include directive until make is run (possibly by using another directive)?
There is probably a way to simply run arbitrary commands before any makefile generation is done (i.e. AC_CONFIG_COMMANDS*). But the question is more complicated because the config.generator.sh is supposed to use executables which are in turn also generated during the same build process (so there is a dependency chain which logically has to be managed by makefiles from the same project). The documentation simply confirms the logic without providing alternatives.
The solution is described in this email of Automake's mailing list.
The idea is to use include directives inside small regular "wrapper" makefile and include Automake-generated Makefile into it (note the upper case M). Because makefile is not an Automake template, the include works as expected triggering builds for non-existing files.
Note that:
By default make utility will search for makefile first (not for Makefile) making this approach working seamlessly.
It is still recommended to specify all rules inside Makefile.am and keep the "wrapper" makefile simple. The rules for non-existing files will naturally come from the generated Makefile anyway.
I've come across the same annoying problem today when moving my OCaml project to Autotools. My solution is to use autoconf's substitution to go around automake. For the above example, I'd add a substitution to configure.ac:
AC_SUBST([include_make_config], ["include make.config"])
and adjust Makefile.am, replacing the include directive with the autoconf variable reference:
make.config : input.dat
config.generator.sh input.dat > make.config
#include_make_config#
...
automake doesn't touch the #include_make_config# line so it gets carried over into the generated Makefile.in. When autoconf takes over, it substitutes the variable with include make.config in the final Makefile.
Note: I use this with OCaml's ocamldep dependency generator.
I have a somewhat complicated Makefile which runs perl scripts and other tools and generates some 1000 files. I would like to edit/modify some of those generated files after all files are generated. So I thought I can simply add a new rule to do so like this:
(phony new rule): $LIST_OF_FILES_TO_EDIT
file_modifier ...
however, the point here is some of those generated files which I'd like to edit ($LIST_OF_FILES_TO_EDIT) are used in the same make process to generate a long list of files. So I have to wait to make sure those files are no longer needed in the make process before I can go ahead and edit them. But I don't know how to do that. Not to mention that it is really hard to find out what files are generated by the help of $LIST_OF_FILES_TO_EDIT.
If it was possible to mention in the Makefile that this rule should be only run as the last rule, then my problem would be solved. but as far as I know this is not possible. So anyone has an idea?
Some points:
List of files to edit ($LIST_OF_FILES_TO_EDIT) is determined dynamically (not known before make process)
I am not sure I have picked a good title for this question. :)
1) If you're going to modify the files like that, it might behoove you to give the targets different names, like foo_unmodified and foo_modified, so that the Make's dependency handling will take care of this.
2) If your phony new rule is the one you invoke on the command line ("make phonyNewRule"), then Make will build whatever else it's going to build before executing the file_modifier command. If you want to build targets not on that list, you could do it this way:
(phony new rule): $(LIST_OF_FILES_TO_EDIT) $(OTHER_TARGETS)
file_modifier ...
3) If your dependencies are set up correctly, you can find out which targets depend on $(LIST_OF_FILES_TO_EDIT), but it's not very tidy. You could just touch one of the files, run make, see which targets it built, repeat for all files. You could save a little time by using Make arguments: "make -n -W foo1 -W foo2 -W foo3 ... -W foo99 all". This will print the commands Make would run-- I don't know of any way to get it to tell you which targets it would rebuild.