I have a makefile that is erroneously generating a required makefile include file. The included file does not initially exist, but there is a rule to make it. The rule runs successfully, but because of a bug, the required include file is not created where expected (and thus make is unable to include it). However, instead of make failing (due to the fact that the file still can't be included), make completes successfully.
The following is my makefile1.mak file.
include myfile.mak
default:
#echo hi
myfile.mak:
#echo hello
When I execute 'make -f makefile1.mak', I get:
makefile1.mak:1: myfile.mak: No such file or directory
hello
hi
Of course, I finally figured out that my code to generate myfile.mak was not generating it correctly, but the actual makefiles that I'm using are 100s of lines long, so we didn't notice that the include wasn't happening for quite a while (it was a very subtle build issue that was introduced).
So, my question is - is there any way to get make to fail on the above example?
Add a line to the rule:
myfile.mak:
do various things to build myfile.mak
test -f $#
Related
What principles should be followed in order not to rebuild some object in Makefile every time?
I only know the most primitive case where we can split the compilation into several steps: creating object modules and linking them. But what to do in more difficult cases? Let's say I have a set of input files and expected output files to test. I want to make it so that only tests on files with wrong output or changed files are re-run.
TEST_INPUT_FILES = $(wildcard $(TEST_DIR)/, *in)
TEST_OUTPUT_FILES = $(wildcard $(TEST_DIR)/, *out)
The above shows how I create lists for each group of files. And in general, how can I be sure that when one file is changed, tests will be passed on this file? Any advice or literature on this topic will be useful, I couldn’t find the answer myself, because everyone disassembles only the banal assembly case
Make creates files from other files using the shell and any program in the shell environment, should the target files not exist or be out of date.
What you'd do is have Make rules running the test with the tested program and any input files, including expected output, then create a test report file. If you want to rerun a failed test you'd remove (clean) the test report prior running the test.
# Make report from test program and inputs
$(REPORT): $(TEST) $(TEST_INPUT) $(TEST_EXPECTED_OUTPUT)
# Remove old report, if any
rm -f $#
# Run test creating report on success
$^
You can also do this by adding report to .DELETE_ON_ERROR: https://www.gnu.org/software/make/manual/make.html#Special-Targets
Consider the following (MCVE of a) Makefile:
my_target: prepare test.bin
prepare:
echo >test.dat
%.bin: %.dat
cp $? $#
If you run make in a clean directory, it fails:
echo >test.dat
make: *** No rule to make target 'test.bin', needed by 'my_target'. Stop.
Run it again and it succeeds:
echo >test.dat
cp test.dat test.bin
What seems to happen is that the rule to make *.bin from *.dat only recognises that it knows how to make test.bin if test.dat exists before anything is executed, even though according to the output it has already created test.dat before it tries to create test.bin.
This is inconvenient for me as I have to prepare a few files first (import them from a different, earlier part of the build).
Is there a solution? Perhaps some way to allow the rules to be (re)evaluated in the light of the files which are now present?
There are a number of issues with your makefile. However based on your comments I'm inclined to assume that the MCVE here is just a little too "M" and it's been reduced so much that it has a number of basic problems. So I won't discuss them, unless you want me to.
The issue here is that you're creating important files without indicating to make that that's what you're doing. Make keeps internally a cache of the contents of directories that it's worked with, for performance reasons, and that cache is only updated when make invokes a rule that it understands will modify it.
Here your target is prepare but the recipe actually creates a completely different file, test.dat. So, make doesn't modify its internal cache of the directory contents and when it checks the cache to see if the file test.dat exists, it doesn't.
You need to be sure that your makefile is written such that it doesn't trick make: if a recipe creates a file foo then the target name should be foo, not bar.
This happens for wildcard targets, like %.bin. They get evaluated at the first pass. You could add an explicit target of test.bin. Or, follow the advice of tkausl and have test.dat depend on prepare (a phony target). In this case, you don't need the double dependency anymore:
my_target: test.bin
you have to write
test.dat: prepare
or (when when you want to stay with wildcards)
%.dat: prepare
#:
Usually, you might want to create and use .stamp files instead of a prepare target.
I am trying to port an existing code into a larger project. The larger project has a main Makefile with Makefiles in each sub-directory. I am sure the path below tells you all about how it is setup. I want to port my code to
/WORKING_DIRECTORY/Drivers/Char/example
And here is the content:
sansari#ubuntu:~/WORKING_DIRECTORY/drivers/char/examples$ ls
hello1.c Makefile
My question is 1- Should modify this local Makefile or the main one? I am setting up to modify this one, but I am not sure.
2- My other question is if I modify this local file, can I just run make from here and validate my configuration instead of running make for the entire project? I know that make only updates the files that are changed; however I feel better when I clean the build environment before each make. I have run into situations, which this alone fixed my issue.
Just as background, I did try to include the make file of my target project, the one I am trying to import here with -f command. What I did was: make -f Makefile -f ../mytarget/core/Makefile
But I ran into some issues with make not doing some of the normal things it does in the primary project. For instance, there was an include statement with a relative path to a header file, which make gave me an error about not seeing it. So now I am abandoning that strategy for the time being.
#Ahmad Masoud - Hey man, thanks. Here is the Makefile. Hey man, the link is exactly what I needed. I think it will address my other questions also. You see, I cross compiled this code, and when I flashed my phone, I get the following for
uname -r: 1|root#hltespr:/lib/modules # uname -r
uname -r
3.4.0-g7e6fbf7-dirty
And I have been wondering what "dirty" means and where it comes from. If you know please tell me. The link you sent, states that perhaps make would insert the Linux kernel version there? I ask this since, modprobe does not work when I try to load my module. Instead insmod works, and I can validate that my module is in. My main issue now is that I don't know how to execute the file to make sure it runs. I only know how to run the file using modprobe, and I can not use it. It gives me the following error:
1|root#hltespr:/ # modprobe /lib/modules/hello1.ko
modprobe /lib/modules/hello1.ko
modprobe: can't change directory to '3.4.0-g7e6fbf7-dirty': No such file or directory
Update as of 06/20/15 -I put in include /home/sansari/mytree2/tbt/makefile in my module's make file. I get the following error: makefile:3: *** missing separator. Stop.
#Ahmad - This is an update as of 062415. Thanks for the info. My goal is to get make to look into this external directory, collect all the source files and build them for me. What would you suggest? I am stuck because as it stands, I know make looks into my examples directory, but no other changes I make to that local make file in the examples directory shows up in make. For instance I tried adding ($warning ....) and #echo messages, but even they do not show up.
Update on 070215- Thanks for the previous comments and support. I feel I really should reopen this thread since I did not explain the goal in detail, and now I feel I can describe it better, and hopefully the resolution will help other. I issue the command:
TARGET=msm8974 PLATFORM=msm8974 make drivers/char/examples
But I get a message stating: Nothing to be done, while I have added a number of tasks to do. Below is my make file, and I'll elaborate on what I have added right after:
lib_tbt := ../../../m/shahin/tbt
lib_daemon := ../../../m/shahin/daemon
lib_lib := ../../../m/shahin/lib
lib_tasks := ../../../m/shahin/tasks
lib_tbt_driver := ../../../m/shahin/tbt_driver
lib_tbt_make := ../../../m/shahin/tbt/make
lib_tbt_msm_common := ../../../m/shahin/tbt/platform/msm8974/common
lib_tbt_msm8974 := ../../../m/shahin/tbt/platform/msm8974
lib_asm_generic = ../../../m/shahin/tbt/platform/msm8974/include/asm-generic
$(warning This is what is in lib_asm_generic $(lib_asm_generic))
#include $(lib_tbt_make)/macros.mk
.PHONY: all $(lib_tbt)
$(lib_tbt) $(lib_daemon) $(lib_lib) $(lib_tasks) $(lib_tbt_driver) $(lib_tbt_make) $(lib_tbt_msm_common) $(lib_tbt_msm8974) $(lib_asmgeneric) :
$(MAKE) --directory=$#
$(lib_*): $(MAKE) --directory=$#
obj-$(CONFIG_EXAMPLES) += hello1.o
_
Initially I only had the obj-$(CONFIG_EXAMPLES) += hello1.ostatement in my make file. I then proceeded to add the directory variables at the top of my makefile and added the $(lib_*): $(MAKE) --directory=$# directing make to compile what is in the directory. I believe that is what it does. Please let me know if I am mistaken. And although this same make file proceeds to create object files when I put it a different directory within my project, it won't do so when it is in a device driver directory. And I do not understand why. The other directory is the /external directory and it is at the top of the tree. But that should not matter right. What I have done was to first make sure I can compile a hello program in my device driver directory called /examples. I now want to add more source code to this section. I believe the correct term is module? I also want to know if I should copy of the source files to the /examples directory or referencing them via the path is ok. That is should I move the source code directory under /examples directory or not?
It is a LOT simpler than that if you are using a kernel that uses Kbuild.
Highly recommend reading
http://www.tldp.org/LDP/lkmpg/2.6/html/x181.html
Situation A - Your source is a sub-tree of the kernel source
You would NOT modify the top-level Makefile, just ensure that ~/WORKING_DIRECTORY/drivers/char/examples/Makefile and ~/WORKING_DIRECTORY/drivers/char/examples/KBuild are set up correctly/normally. THEN at the top-level of the kernel build directory (assuming you have a separate build directory) you would type:
foo#bar:~/build-dir$ make drivers/char/examples
The kernel top-level makefile then builds just that sub-tree. You can try it out on any part of the kernel, for example:
foo#bar:~/build-dir$ make fs
NOTE: build-dir can be the same as the kernel source directory
Situation B - You are building an external module
Then use the normal module KBuild / Makefile process.
P.S.
If you post your makefile / Kbuild then I may be able to help with the actual build processing.
For a project of mine I am automatically generating makefiles and including them, like this:
all:
#echo 'SUCCESS is $(SUCCESS)'
clean:
rm depend.mk
depend.mk:
#echo 'Creating $#'
#echo 'SUCCESS := 1' > $#
.PHONY: all clean
include depend.mk
This works, but the include line generates a warning message:
$ make
Makefile:13: depend.mk: No such file or directory
Creating depend.mk
SUCCESS is 1
I would like to silence that first warning line saying that depend.mk doesn't exist. I know it doesn't exist since I have a rule written to generate it, so the warning is unnecessary (unless of course there isn't a rule for it). I do NOT want make to ignore the error where the included file doesn't exist and there is no rule for it, so prefixing include with a - to ignore the error will not work for me. I'd like something similar to bash's convention of piping stderr to /dev/null like some_cmd 2>/dev/null but for including in make.
The sample above is a very simplified example of this case. In my actual project there are a lot of automatically generated makefiles (via clang's automatic dependency generation) being included, meaning a fresh run of make will flood my screen with these warning messages.
Is anything like this possible, or am I just going to have to deal with the annoying warning messages?
I've encountered and (re-re-re-re-)solved this problem a number of times myself. Really, the problem is in the thinking surrounding when the dependency files are generated and used.
This link has the detailed description of the "resolution": http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
Basically it comes down to the fact that dependency files are really only necessary for rebuilding, not the initial building of your library/executable. Resultantly you don't need to have a rule for generating dependency files up front (which is in fact less efficient), you instead should generate them during the object file step as intermediate files marked precious (so they're created and tracked as side-effect files that should never be cleaned up automatically). Subsequent builds will then have the files available, which is exactly what you were trying to achieve overall. You can then make it a "-include" on the dependency files, with the foreknowledge that your object file build step will fail if the dependency file generation fails, giving an immediate error, as you've mentioned is preferred, rather than an obscure and indirect one much later.
I've actually done a couple rather large build systems implementing this method, and it does work quite well, including ones that used non-GNU toolchains. To an outside user it appears identical, but internally it performs more efficiently and isn't hiding potentially important errors.
I tried many (many!) things to see if I could prevent or redirect the error message. No luck.
But when I tried -include (include with a leading dash), it didn't give an error, and make with clean, all, depend.mk and 'default' all worked properly and as expected.
Is there a particular reason you didn't want to use the -include variant? Seems to do exactly what you're looking for, and doesn't alter how the Makefile works in any way, just doesn't show the error during the first pass through the Makefile.
Part of my Makefile:
CPUDEPS=./mydeps.cpu
(...)
deps: $(CPUDEPS)
$(CPUDEPS): $(CCFILES)
#echo [DEPS] CPU
$(CMDECHO)makedepend -Y -s'# CPU sources dependencies generated with "make deps"' \
-w4096 -f- -- $(CFLAGS) -- $^ 2> /dev/null > $(CPUDEPS)
(...)
sinclude $(CPUDEPS)
Problem 1: includes are done during the first phase of processing, targets during the second phase; so, if ./mydeps.cpu doesn't exist and I "make deps", I get first the error
Makefile:335: ./mydeps.cpu: No such file or directory
I hide the error using sinclude instead of include, but the problem is still there: the old file is included, not the just-generated-one. Have to run it twice to include the updated file. This is because make does a two-phase processing; is there any way to tell make to complete the target deps before parsing the includes?
Problem 2: even if the file ./mydeps.cpu doesn't exist and make deps actually creates it, I always get a "make: Nothing to do for deps". This doesn't happen with other targets. I don't understand why and how to avoid it.
Problem 1 is non-existant: before building a target, make automatically rebuilds makefiles (with implicit rules if no explicit rule is provided). So having a rule for the makefile ensures that will always be up to date, there is no need to run deps twice. Additionally, since CPUDEPS is a makefile, it will be updated automatically before any other rule is run, so dependencies will always be updated if necessary and make deps is not needed. You can probably notice this by yourself by observing the [DEPS] line being echoed if any of the CCFILES becomes more recent that the dependency file.
For Problem 2, adding anything to the recipe ensures that make doesn't complain about having nothing to do. If there is nothing else, you can use something like #echo OK to give feedback to the user, or a simple #true if you prefer totally silent makes.
What you are trying to achieve is useless: you can use the dependencies file that was created during the previous build. That's enough.
The main reasoning behind that rule is:
if you haven't changed any of your files, then the dependencies file is up-to-date, and there's nothing to build.
if you have changed anything, even very deep into your #include chain, on an existing file that were used by previous build, then the dependencies file have already caught it. You'll rebuild what is needed.
if you change something in a new file (you add that file!) then it was not used by previous build, and not listed in dependencies. But if you really want to use it, then you have to modify at least one of your other files that was used before, and you're back on the previous case.
The solution is to create the dependencies file during the normal process of the compilation, and to optionally include it (with sinclude) if it is present.