we currently generate a dependency file for every .o. But when do an incremental build, Make reads from the dependency file for dependencies for each .o. Is Make checking the time stamp of these dependent files and compare it with the .o? If so, is it possible to cache the status of dependencies to avoid too much I/O hit because of duplicated status checks for each object file?
for example,
a.o: h1.h h2.h
gcc...
b.o: h1.h h2.h
gcc...
If we cache the status of h1.h and h2.h when it builds a.o, do we save two checks when build b.o?
I am not familiar with the make system, but is currently looking for ways to improve its performance on a large legacy C project.
Use strace for that purpose:
strace -e trace=stat make --touch
Output of the first run (full build):
...
stat("a.o", 0x7fff70c35f00) = -1 ENOENT (No such file or directory)
stat("h1.h", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("h2.h", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
touch a.o
stat("b.o", 0x7fff70c35f00) = -1 ENOENT (No such file or directory)
touch b.o
And the second run (incremental build):
...
stat("a.o", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("h1.h", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("h2.h", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("b.o", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
make: Nothing to be done for `all'.
As you can see, GNU Make caches timestamps avoiding unnecessary stat syscalls. However, I guess, things are not so good in case of using recursive make.
Related
Trying to get rid of Ubuntu's apport by clearing /proc/sys/kernel/core_pattern using
sh -c ': > /proc/sys/kernel/core_pattern' does not work.
It looks like the O_TRUNC flag is ignored when writing to the /proc filesystem:
echo nonsense >| /proc/sys/kernel/core_pattern
strace sh -c ': > /proc/sys/kernel/core_pattern # do not call apport'
...
openat(AT_FDCWD, "/proc/sys/kernel/core_pattern", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
...
close(3) = 0
cat /proc/sys/kernel/core_pattern
nonsense
I get an empty file when doing this in regular filesystems instead of the /proc filesystem.
Is that a kernel bug or a feature, perhaps even a documented one?
Edit: Clearing this setting through sysctl does not work:
sysctl kernel.core_pattern=""
sysctl: malformed setting "kernel.core_pattern="
It seems the sysctl program is unable to clear any kernel parameters while man core explicitly describes that an empty value is used to disable the mechanism.
Yes, echo >| /proc/sys/kernel/core_pattern works instead, but the object of this question is to find out whether this is a kernel bug, not to find a workaround.
This is because the O_TRUNC flag of the open* family of syscalls merely updates the size of the inode associated with the opened file. This change is performed right after finding the inode and before finalizing the struct file that is then used by the kernel for any actual operation on the opened file. Truncation is performed before the call to any ->open() file_operations handler implemented by whichever kernel module/driver/subsystem (like for example the sysctl subsystem) and is therefore transparent to the handler.
In other words, the file_operations handlers of the virtual sysctl files (e.g. /proc/sys/kernel/*) merely see a file with a 0 (zero) size (->i_size field of struct inode), they do not know whether this was the result of a truncation or a "normal" open, nor they should need such information.
Since sysctl files (just as the near totality of procfs files) do not really bother tracking sizes for understandable reasons, their functionality is only implemented in terms of read and write system calls (which also do not update the size in any way).
Indeed, using : > PATH will merely do open + close, while a simple echo > PATH will write a newline character after opening, thus you observe two different outcomes. You would observe the same behavior as : > PATH using truncate -s 0 PATH, though this time the truncation is done explicitly after opening through ftruncate (at least on my system).
man core explicitly describes that an empty value is used to disable the mechanism
[...]
Is that a kernel bug or a feature, perhaps even a documented one?
Human readable/writable files under procfs are usually designed to work in a line-oriented fashion, so I would assume that the term "empty" here simply means that the value of the option is empty as a result of writing an empty line to the file. If anything, I would call this an undocumented feature rather than a bug.
Here's some example traces on my system:
root#xxx:~# cat /proc/sys/kernel/core_pattern
|/usr/share/apport/apport %p %s %c %d %P %E
root#xxx:~# strace -f -e openat,write,close,dup2 sh -c ': > /proc/sys/kernel/core_pattern'
...
openat(AT_FDCWD, "/proc/sys/kernel/core_pattern", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
close(1) = 0
dup2(3, 1) = 1
close(3) = 0
...
+++ exited with 0 +++
root#xxx:~# cat /proc/sys/kernel/core_pattern
|/usr/share/apport/apport %p %s %c %d %P %E
root#xxx:~#
root#xxx:~# cat /proc/sys/kernel/core_pattern
|/usr/share/apport/apport %p %s %c %d %P %E
root#xxx:~# strace -f -e openat,write,close,dup2 sh -c 'echo > /proc/sys/kernel/core_pattern'
...
openat(AT_FDCWD, "/proc/sys/kernel/core_pattern", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
close(1) = 0
dup2(3, 1) = 1
close(3) = 0
write(1, "\n", 1) = 1
...
+++ exited with 0 +++
root#xxx:~# cat /proc/sys/kernel/core_pattern
root#xxx:~#
If you want to take a look at the actual implementation of open/read/write/close for sysfs files you can check /kernel/sysctl.c. There are different tables present for different sysctl facilities e.g. kernel, vm, etc.
I'm missing something about implicit rules. Here's the Makefile (GNU Make 4.2.1)
heimdall /tmp 1670> cat Makefile
PARTS= a b c
.SECONDEXPANSION:
data/events2: $$(patsubst %,$$(#D)/%.ppd,$(PARTS))
/bin/ls -l $^
%/events2: $$(patsubst %,$$(#D)/%.ppd,$(PARTS))
/bin/ls -l $^
Here are the cooked up data to illustrate the situation:
heimdall /tmp 1671> ls -1 data data1
data:
a.ppd
b.ppd
c.ppd
data1:
a.ppd
b.ppd
c.ppd
Here is make using an explicit rule, which works like I'd expect.
heimdall /tmp 1672> make data/events2
/bin/ls -l data/a.ppd data/b.ppd data/c.ppd
-rw-rw-r-- 1 bennett None 0 Feb 4 12:19 data/a.ppd
-rw-rw-r-- 1 bennett None 0 Feb 4 12:19 data/b.ppd
-rw-rw-r-- 1 bennett None 0 Feb 4 12:19 data/c.ppd
And finally, this:
heimdall /tmp 1673> make data1/events2
make: *** No rule to make target 'data1/events2'. Stop.
Why doesn't the implicit rule match? I feel like I've missed something fundamental.
Thanks.
-E
%/events2: $$(patsubst %,$$(#D)/%.ppd,$(PARTS))
Is not a pattern rule that would match in your sample structure. From the docs:
% in a prerequisite of a pattern rule stands for the same stem that was matched by the % in the target. In order for the pattern rule to apply, its target pattern must match the file name under consideration and all of its prerequisites (after pattern substitution) must name files that exist or can be made. These files become prerequisites of the target.
However in your target % would be matching data1. But there isn't actually any % to match on prerequisite side as those present are oft patsubst function and directory (stem) is referred to as $(#D).
I've tried to write such rule like this using foreach function:
%/events2: $(foreach part,$(PARTS), %/$(part).ppd)
/bin/ls -l $^
If you wanted to stick with patsubst, this should work as well:
%/events2: $(patsubst %,\%/%.ppd,$(PARTS))
/bin/ls -l $^
Not that % is used for directory name matching the one in target and it's escaped with \ to make it through patsubst unscathed.
Either way seems to have gone well with GNU make yielding:
$ make data1/events2
/bin/ls -l data1/a.ppd data1/b.ppd data1/c.ppd
-rw-r--r-- 1 ondrej users 0 Feb 4 22:00 data1/a.ppd
-rw-r--r-- 1 ondrej users 0 Feb 4 22:00 data1/b.ppd
-rw-r--r-- 1 ondrej users 0 Feb 4 22:00 data1/c.ppd
When I preprocess a C++ file like this:
g++ -E source.cpp
the preprocessed file still contains a lot of preprocessor instructions like these:
# 1 "/usr/include/features.h" 1 3 4
# 367 "/usr/include/features.h" 3 4
# 1 "/usr/include/x86_64-linux-gnu/sys/cdefs.h" 1 3 4
I don't need them. How can I get a preprocessed file without these instructions?
You can use the -P option. It prevents GCC from creating these line options:
g++ -E -P source.cpp
The follow makefile is created for generate the difference from two files,
file1 and file2:
.PHONY: patch
patch:
diff file1 file2 > file12.patch
The content of file1:
xxx
and file2:
xxx
yyy
But when I issued make patch, error comes:
diff file1 file2 > file12.patch
Makefile:3: recipe for target 'patch' failed
make: *** [patch] Error 1
However, patch could be generated in terminal by:
diff file1 file2 > file12.patch.
Content of file12.patch:
1a2
> yyy
Surprisingly, I went back to check the folder, the make patch did produce the right patch file.
What I cannot understand is the make error? e.g. This error certainly will stop the make process, skip all the commands afterward.
Could someone explain this make behavior? Thank you!
Your quoted makefile above cannot be create: recipes for .PHONY targets are ignored. It must be something like:
.PHONY: patch
patch:
diff file1 file2 > file12.patch
Make determines that a command succeeded or failed by looking at its exit code. If the exit code is 0 then make assumes the command succeeded. If it's non-0 make assumes it failed.
The error you're seeing implies that the diff command exited with a non-0 exit code.
The manual for diff says:
An exit status of 0 means no differences were found, 1 means some
differences were found, and 2 means trouble.
So, since your diff does find some differences it exits with a code of 1, which make interprets as failing. You'll probably want to change your recipe to something like:
.PHONY: patch
patch:
diff file1 file2 > file12.patch || [ $? -eq 1 ]
so if the diff fails you further check to see if the exit code was 2 or not.
How i could save le preprocessor in output file with specific name as x or y ?
I tried the command line :
gcc -E -o pgcd.c x.o
But it don't seem being the solution.
ps: the file doesn't exist before the compilation, i just would save the preprocessor in a file with the name i defined.
Thank you for any help.
gcc -E file.c
will preprocess file.c and write the preprocessed source code to the
standard output (console). So to save the preprocessed output, redirect
the standard output to a file of your choice:
gcc -E file.c > somefile
It is a bad idea for somefile to have an .o extension. GCC and other
tools interpret the .o extension as meaning that the file contains object
code, as output by compilation. Preprocessing file.c does not produce
object code. It just produces preprocessed source code, which you might later compile.
The conventional file extension for preprocessed C source code is .i
(.ii for preprocessed C++ source code). Therefore
gcc -E file.c > file.i
is the appropriate choice.
You will discover that file.i contains preprocessor line-markers, e.g.
# 1 "file.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "file.c"
# 1 "/usr/include/stdio.h" 1 3 4
# 27 "/usr/include/stdio.h" 3 4
...
...
If you don't want these line-markers to appear in the output, add the -P
option:
gcc -E -P file.c > file.i
The GCC -save-temps flag can also be used to save the processed file in *.i files (although this differs from -E in that compilation of the files are not halted after the pre-processor stage).