I'm trying to optimize, and also improve features of, the Makefile of the Linux man-pages.
Before, it only had make uninstall and make install, both of which ran a small shell script that run in a single thread, which took between 5 and 10 seconds on my laptop (i7-8850H 6C/12T).
I added the possibility of installing only some section (e.g., man3/ only) with make install-man3.
That also made it possible to install everything in parallel, thus allowing make -j install.
Using multiple threads improves times drastically, but now using a single thread to install also drastically increases the time to install. For comparison, these are the times on my laptop (make does the same as make uninstall && make install):
Old Makefile:
~/src/linux/man-pages$ time sudo make >/dev/null
real 0m7.509s
user 0m5.269s
sys 0m2.614s
The times with the old makefile, varied a lot, between 5 and 10 seconds.
New Makefile:
~/src/linux/man-pages$ time sudo make >/dev/null
real 0m27.883s
user 0m18.257s
sys 0m10.735s
~/src/linux/man-pages$ time sudo make -j2 >/dev/null
real 0m4.636s
user 0m8.514s
sys 0m1.475s
~/src/linux/man-pages$ time sudo make -j4 >/dev/null
real 0m2.582s
user 0m8.400s
sys 0m1.369s
~/src/linux/man-pages$ time sudo make -j >/dev/null
real 0m2.071s
user 0m9.203s
sys 0m1.704s
These new times are very consistent.
If I run make -n I get very small times, so I think it's not make overhead, but actual work being done:
~/src/linux/man-pages$ time sudo make -n >/dev/null
real 0m1.086s
user 0m1.116s
sys 0m0.117s
~/src/linux/man-pages$ time sudo make -nj >/dev/null
real 0m0.652s
user 0m1.269s
sys 0m0.146s
The current (new) Makefile I'm working with is (after removing some uninteresting parts):
Makefile:
SHELL := /bin/bash -Eeuo pipefail
MAKEFLAGS += --no-print-directory
MAKEFLAGS += --silent
ROOTDIR := $(CURDIR)
DESTDIR :=
prefix := /usr/local
datarootdir := $(prefix)/share
MANDIR := $(ROOTDIR)
mandir := $(datarootdir)/man
MAN1DIR := $(MANDIR)/man1
MAN2DIR := $(MANDIR)/man2
MAN3DIR := $(MANDIR)/man3
MAN4DIR := $(MANDIR)/man4
MAN5DIR := $(MANDIR)/man5
MAN6DIR := $(MANDIR)/man6
MAN7DIR := $(MANDIR)/man7
MAN8DIR := $(MANDIR)/man8
man1dir := $(mandir)/man1
man2dir := $(mandir)/man2
man3dir := $(mandir)/man3
man4dir := $(mandir)/man4
man5dir := $(mandir)/man5
man6dir := $(mandir)/man6
man7dir := $(mandir)/man7
man8dir := $(mandir)/man8
man1ext := \.1
man2ext := \.2
man3ext := \.3
man4ext := \.4
man5ext := \.5
man6ext := \.6
man7ext := \.7
man8ext := \.8
INSTALL := install
INSTALL_DATA := $(INSTALL) -m 644
INSTALL_DIR := $(INSTALL) -m 755 -d
RM := rm -f
RMDIR := rmdir --ignore-fail-on-non-empty
.PHONY: all
all:
$(MAKE) uninstall;
$(MAKE) install;
%/:
#echo ' INSTALL $#';
$(INSTALL_DIR) $#;
.PHONY: install
install: install-man
.PHONY: uninstall remove
uninstall remove: uninstall-man
################################################################################
# man
MAN1PAGES := $(shell find $(MAN1DIR) -type f | grep '$(man1ext)$$' | sort)
MAN2PAGES := $(shell find $(MAN2DIR) -type f | grep '$(man2ext)$$' | sort)
MAN3PAGES := $(shell find $(MAN3DIR) -type f | grep '$(man3ext)$$' | sort)
MAN4PAGES := $(shell find $(MAN4DIR) -type f | grep '$(man4ext)$$' | sort)
MAN5PAGES := $(shell find $(MAN5DIR) -type f | grep '$(man5ext)$$' | sort)
MAN6PAGES := $(shell find $(MAN6DIR) -type f | grep '$(man6ext)$$' | sort)
MAN7PAGES := $(shell find $(MAN7DIR) -type f | grep '$(man7ext)$$' | sort)
MAN8PAGES := $(shell find $(MAN8DIR) -type f | grep '$(man8ext)$$' | sort)
man1pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN1PAGES))
man2pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN2PAGES))
man3pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN3PAGES))
man4pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN4PAGES))
man5pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN5PAGES))
man6pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN6PAGES))
man7pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN7PAGES))
man8pages := $(patsubst $(MANDIR)/%,$(DESTDIR)$(mandir)/%,$(MAN8PAGES))
manpages := $(man1pages) \
$(man2pages) \
$(man3pages) \
$(man4pages) \
$(man5pages) \
$(man6pages) \
$(man7pages) \
$(man8pages)
$(manpages): $(DESTDIR)$(mandir)/%: $(MANDIR)/%
#echo ' INSTALL $#';
$(INSTALL_DATA) -T '$(MANDIR)/$*' '$#';
INSTALL_MANn := install-man1 \
install-man2 \
install-man3 \
install-man4 \
install-man5 \
install-man6 \
install-man7 \
install-man8
.PHONY: $(INSTALL_MANn)
$(INSTALL_MANn): install-%: | installdirs-%
$(MAKE) '_install-$*';
.PHONY: install-man
install-man: $(INSTALL_MANn)
#:;
.PHONY: _install-man1
_install-man1: $(man1pages)
.PHONY: _install-man2
_install-man2: $(man2pages)
.PHONY: _install-man3
_install-man3: $(man3pages)
.PHONY: _install-man4
_install-man4: $(man4pages)
.PHONY: _install-man5
_install-man5: $(man5pages)
.PHONY: _install-man6
_install-man6: $(man6pages)
.PHONY: _install-man7
_install-man7: $(man7pages)
.PHONY: _install-man8
_install-man8: $(man8pages)
INSTALLDIRS_MANn := installdirs-man1 \
installdirs-man2 \
installdirs-man3 \
installdirs-man4 \
installdirs-man5 \
installdirs-man6 \
installdirs-man7 \
installdirs-man8
.PHONY: $(INSTALLDIRS_MANn)
$(INSTALLDIRS_MANn): installdirs-%: $(DESTDIR)$(mandir)/%/ | installdirs-man
#:;
.PHONY: installdirs-man
installdirs-man: $(DESTDIR)$(mandir)/
#:;
uninstall_man1pages := $(addprefix uninstall-,$(wildcard $(man1pages)))
uninstall_man2pages := $(addprefix uninstall-,$(wildcard $(man2pages)))
uninstall_man3pages := $(addprefix uninstall-,$(wildcard $(man3pages)))
uninstall_man4pages := $(addprefix uninstall-,$(wildcard $(man4pages)))
uninstall_man5pages := $(addprefix uninstall-,$(wildcard $(man5pages)))
uninstall_man6pages := $(addprefix uninstall-,$(wildcard $(man6pages)))
uninstall_man7pages := $(addprefix uninstall-,$(wildcard $(man7pages)))
uninstall_man8pages := $(addprefix uninstall-,$(wildcard $(man8pages)))
uninstall_manpages := $(uninstall_man1pages) \
$(uninstall_man2pages) \
$(uninstall_man3pages) \
$(uninstall_man4pages) \
$(uninstall_man5pages) \
$(uninstall_man6pages) \
$(uninstall_man7pages) \
$(uninstall_man8pages)
.PHONY: $(uninstall_manpages)
$(uninstall_manpages): uninstall-%:
#echo ' RM $*';
$(RM) '$*';
UNINSTALL_MANn := uninstall-man1 \
uninstall-man2 \
uninstall-man3 \
uninstall-man4 \
uninstall-man5 \
uninstall-man6 \
uninstall-man7 \
uninstall-man8
uninstall-man1: $(uninstall_man1pages)
uninstall-man2: $(uninstall_man2pages)
uninstall-man3: $(uninstall_man3pages)
uninstall-man4: $(uninstall_man4pages)
uninstall-man5: $(uninstall_man5pages)
uninstall-man6: $(uninstall_man6pages)
uninstall-man7: $(uninstall_man7pages)
uninstall-man8: $(uninstall_man8pages)
.PHONY: $(UNINSTALL_MANn)
$(UNINSTALL_MANn): uninstall-%:
#echo ' RMDIR $(DESTDIR)$(mandir)/$*/';
$(RMDIR) '$(DESTDIR)$(mandir)/$*/' 2>/dev/null ||:;
.PHONY: uninstall-man
uninstall-man: $(UNINSTALL_MANn)
#echo ' RMDIR $(DESTDIR)$(mandir)/';
$(RMDIR) '$(DESTDIR)$(mandir)/' ||:;
My question is:
Why is it so slow to run it in a single thread? 6 times slower than 2 threads??
EDIT: Show --trace
Makefile:54: target 'all' does not exist
make uninstall;
Makefile:176: target 'uninstall-/usr/local/share/man/man1/getent.1' does not exist
echo ' RM /usr/local/share/man/man1/getent.1';
RM /usr/local/share/man/man1/getent.1
rm -f '/usr/local/share/man/man1/getent.1';
...XXX...
Makefile:176: target 'uninstall-/usr/local/share/man/man1/time.1' does not exist
echo ' RM /usr/local/share/man/man1/time.1';
RM /usr/local/share/man/man1/time.1
rm -f '/usr/local/share/man/man1/time.1';
Makefile:200: update target 'uninstall-man1' due to: uninstall-/usr/local/share/man/man1/getent.1 ...XXX... uninstall-/usr/local/share/man/man1/time.1
echo ' RMDIR /usr/local/share/man/man1/';
RMDIR /usr/local/share/man/man1/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man1/' 2>/dev/null ||:;
Makefile:176: target 'uninstall-/usr/local/share/man/man2/accept.2' does not exist
echo ' RM /usr/local/share/man/man2/accept.2';
RM /usr/local/share/man/man2/accept.2
rm -f '/usr/local/share/man/man2/accept.2';
...XXX...
Makefile:176: target 'uninstall-/usr/local/share/man/man2/writev.2' does not exist
echo ' RM /usr/local/share/man/man2/writev.2';
RM /usr/local/share/man/man2/writev.2
rm -f '/usr/local/share/man/man2/writev.2';
Makefile:200: update target 'uninstall-man2' due to: uninstall-/usr/local/share/man/man2/accept.2 ...XXX... uninstall-/usr/local/share/man/man2/writev.2
echo ' RMDIR /usr/local/share/man/man2/';
RMDIR /usr/local/share/man/man2/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man2/' 2>/dev/null ||:;
Makefile:176: target 'uninstall-/usr/local/share/man/man3/a64l.3' does not exist
echo ' RM /usr/local/share/man/man3/a64l.3';
RM /usr/local/share/man/man3/a64l.3
rm -f '/usr/local/share/man/man3/a64l.3';
...XXX...
Makefile:176: target 'uninstall-/usr/local/share/man/man3/ynl.3' does not exist
echo ' RM /usr/local/share/man/man3/ynl.3';
RM /usr/local/share/man/man3/ynl.3
rm -f '/usr/local/share/man/man3/ynl.3';
Makefile:200: update target 'uninstall-man3' due to: uninstall-/usr/local/share/man/man3/a64l.3 ...XXX... uninstall-/usr/local/share/man/man3/ynl.3
echo ' RMDIR /usr/local/share/man/man3/';
RMDIR /usr/local/share/man/man3/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man3/' 2>/dev/null ||:;
Makefile:176: target 'uninstall-/usr/local/share/man/man4/cciss.4' does not exist
echo ' RM /usr/local/share/man/man4/cciss.4';
RM /usr/local/share/man/man4/cciss.4
rm -f '/usr/local/share/man/man4/cciss.4';
...XXX...
Makefile:176: target 'uninstall-/usr/local/share/man/man4/zero.4' does not exist
echo ' RM /usr/local/share/man/man4/zero.4';
RM /usr/local/share/man/man4/zero.4
rm -f '/usr/local/share/man/man4/zero.4';
Makefile:200: update target 'uninstall-man4' due to: uninstall-/usr/local/share/man/man4/cciss.4 ...XXX... uninstall-/usr/local/share/man/man4/zero.4
echo ' RMDIR /usr/local/share/man/man4/';
RMDIR /usr/local/share/man/man4/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man4/' 2>/dev/null ||:;
Makefile:176: target 'uninstall-/usr/local/share/man/man5/acct.5' does not exist
echo ' RM /usr/local/share/man/man5/acct.5';
RM /usr/local/share/man/man5/acct.5
rm -f '/usr/local/share/man/man5/acct.5';
...XXX...
Makefile:176: target 'uninstall-/usr/local/share/man/man5/wtmp.5' does not exist
echo ' RM /usr/local/share/man/man5/wtmp.5';
RM /usr/local/share/man/man5/wtmp.5
rm -f '/usr/local/share/man/man5/wtmp.5';
Makefile:200: update target 'uninstall-man5' due to: uninstall-/usr/local/share/man/man5/acct.5 ...XXX... uninstall-/usr/local/share/man/man5/wtmp.5
echo ' RMDIR /usr/local/share/man/man5/';
RMDIR /usr/local/share/man/man5/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man5/' 2>/dev/null ||:;
Makefile:176: target 'uninstall-/usr/local/share/man/man6/intro.6' does not exist
echo ' RM /usr/local/share/man/man6/intro.6';
RM /usr/local/share/man/man6/intro.6
rm -f '/usr/local/share/man/man6/intro.6';
Makefile:200: update target 'uninstall-man6' due to: uninstall-/usr/local/share/man/man6/intro.6
echo ' RMDIR /usr/local/share/man/man6/';
RMDIR /usr/local/share/man/man6/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man6/' 2>/dev/null ||:;
Makefile:176: target 'uninstall-/usr/local/share/man/man7/address_families.7' does not exist
echo ' RM /usr/local/share/man/man7/address_families.7';
RM /usr/local/share/man/man7/address_families.7
rm -f '/usr/local/share/man/man7/address_families.7';
...XXX...
Makefile:176: target 'uninstall-/usr/local/share/man/man7/xattr.7' does not exist
echo ' RM /usr/local/share/man/man7/xattr.7';
RM /usr/local/share/man/man7/xattr.7
rm -f '/usr/local/share/man/man7/xattr.7';
Makefile:200: update target 'uninstall-man7' due to: uninstall-/usr/local/share/man/man7/address_families.7 ...XXX... uninstall-/usr/local/share/man/man7/xattr.7
echo ' RMDIR /usr/local/share/man/man7/';
RMDIR /usr/local/share/man/man7/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man7/' 2>/dev/null ||:;
Makefile:176: target 'uninstall-/usr/local/share/man/man8/iconvconfig.8' does not exist
echo ' RM /usr/local/share/man/man8/iconvconfig.8';
RM /usr/local/share/man/man8/iconvconfig.8
rm -f '/usr/local/share/man/man8/iconvconfig.8';
...XXX...
Makefile:176: target 'uninstall-/usr/local/share/man/man8/zic.8' does not exist
echo ' RM /usr/local/share/man/man8/zic.8';
RM /usr/local/share/man/man8/zic.8
rm -f '/usr/local/share/man/man8/zic.8';
Makefile:200: update target 'uninstall-man8' due to: uninstall-/usr/local/share/man/man8/iconvconfig.8 ...XXX... uninstall-/usr/local/share/man/man8/zic.8
echo ' RMDIR /usr/local/share/man/man8/';
RMDIR /usr/local/share/man/man8/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/man8/' 2>/dev/null ||:;
Makefile:205: update target 'uninstall-man' due to: uninstall-man1 uninstall-man2 uninstall-man3 uninstall-man4 uninstall-man5 uninstall-man6 uninstall-man7 uninstall-man8
echo ' RMDIR /usr/local/share/man/';
RMDIR /usr/local/share/man/
rmdir --ignore-fail-on-non-empty '/usr/local/share/man/' ||:;
make install;
Makefile:154: update target 'installdirs-man' due to: /usr/local/share/man/
:;
Makefile:150: update target 'installdirs-man1' due to: /usr/local/share/man/man1/
:;
Makefile:115: target 'install-man1' does not exist
make '_install-man1';
Makefile:100: update target '/usr/local/share/man/man1/getent.1' due to: /home/user/src/linux/man-pages/man1/getent.1
echo ' INSTALL /usr/local/share/man/man1/getent.1';
INSTALL /usr/local/share/man/man1/getent.1
install -m 644 -T '/home/user/src/linux/man-pages/man1/getent.1' '/usr/local/share/man/man1/getent.1';
...XXX...
Makefile:100: update target '/usr/local/share/man/man1/time.1' due to: /home/user/src/linux/man-pages/man1/time.1
echo ' INSTALL /usr/local/share/man/man1/time.1';
INSTALL /usr/local/share/man/man1/time.1
install -m 644 -T '/home/user/src/linux/man-pages/man1/time.1' '/usr/local/share/man/man1/time.1';
Makefile:150: update target 'installdirs-man2' due to: /usr/local/share/man/man2/
:;
Makefile:115: target 'install-man2' does not exist
make '_install-man2';
Makefile:100: update target '/usr/local/share/man/man2/accept.2' due to: /home/user/src/linux/man-pages/man2/accept.2
echo ' INSTALL /usr/local/share/man/man2/accept.2';
INSTALL /usr/local/share/man/man2/accept.2
install -m 644 -T '/home/user/src/linux/man-pages/man2/accept.2' '/usr/local/share/man/man2/accept.2';
...XXX...
Makefile:100: update target '/usr/local/share/man/man2/writev.2' due to: /home/user/src/linux/man-pages/man2/writev.2
echo ' INSTALL /usr/local/share/man/man2/writev.2';
INSTALL /usr/local/share/man/man2/writev.2
install -m 644 -T '/home/user/src/linux/man-pages/man2/writev.2' '/usr/local/share/man/man2/writev.2';
Makefile:150: update target 'installdirs-man3' due to: /usr/local/share/man/man3/
:;
Makefile:115: target 'install-man3' does not exist
make '_install-man3';
Makefile:100: update target '/usr/local/share/man/man3/a64l.3' due to: /home/user/src/linux/man-pages/man3/a64l.3
echo ' INSTALL /usr/local/share/man/man3/a64l.3';
INSTALL /usr/local/share/man/man3/a64l.3
install -m 644 -T '/home/user/src/linux/man-pages/man3/a64l.3' '/usr/local/share/man/man3/a64l.3';
...XXX...
Makefile:100: update target '/usr/local/share/man/man3/ynl.3' due to: /home/user/src/linux/man-pages/man3/ynl.3
echo ' INSTALL /usr/local/share/man/man3/ynl.3';
INSTALL /usr/local/share/man/man3/ynl.3
install -m 644 -T '/home/user/src/linux/man-pages/man3/ynl.3' '/usr/local/share/man/man3/ynl.3';
Makefile:59: target '/usr/local/share/man/man4/' does not exist
echo ' INSTALL /usr/local/share/man/man4/';
INSTALL /usr/local/share/man/man4/
install -m 755 -d /usr/local/share/man/man4/;
Makefile:150: update target 'installdirs-man4' due to: /usr/local/share/man/man4/
:;
Makefile:115: target 'install-man4' does not exist
make '_install-man4';
Makefile:100: update target '/usr/local/share/man/man4/cciss.4' due to: /home/user/src/linux/man-pages/man4/cciss.4
echo ' INSTALL /usr/local/share/man/man4/cciss.4';
INSTALL /usr/local/share/man/man4/cciss.4
install -m 644 -T '/home/user/src/linux/man-pages/man4/cciss.4' '/usr/local/share/man/man4/cciss.4';
...XXX...
Makefile:100: update target '/usr/local/share/man/man4/zero.4' due to: /home/user/src/linux/man-pages/man4/zero.4
echo ' INSTALL /usr/local/share/man/man4/zero.4';
INSTALL /usr/local/share/man/man4/zero.4
install -m 644 -T '/home/user/src/linux/man-pages/man4/zero.4' '/usr/local/share/man/man4/zero.4';
Makefile:59: target '/usr/local/share/man/man5/' does not exist
echo ' INSTALL /usr/local/share/man/man5/';
INSTALL /usr/local/share/man/man5/
install -m 755 -d /usr/local/share/man/man5/;
Makefile:150: update target 'installdirs-man5' due to: /usr/local/share/man/man5/
:;
Makefile:115: target 'install-man5' does not exist
make '_install-man5';
Makefile:100: update target '/usr/local/share/man/man5/acct.5' due to: /home/user/src/linux/man-pages/man5/acct.5
echo ' INSTALL /usr/local/share/man/man5/acct.5';
INSTALL /usr/local/share/man/man5/acct.5
install -m 644 -T '/home/user/src/linux/man-pages/man5/acct.5' '/usr/local/share/man/man5/acct.5';
...XXX...
Makefile:100: update target '/usr/local/share/man/man5/wtmp.5' due to: /home/user/src/linux/man-pages/man5/wtmp.5
echo ' INSTALL /usr/local/share/man/man5/wtmp.5';
INSTALL /usr/local/share/man/man5/wtmp.5
install -m 644 -T '/home/user/src/linux/man-pages/man5/wtmp.5' '/usr/local/share/man/man5/wtmp.5';
Makefile:59: target '/usr/local/share/man/man6/' does not exist
echo ' INSTALL /usr/local/share/man/man6/';
INSTALL /usr/local/share/man/man6/
install -m 755 -d /usr/local/share/man/man6/;
Makefile:150: update target 'installdirs-man6' due to: /usr/local/share/man/man6/
:;
Makefile:115: target 'install-man6' does not exist
make '_install-man6';
Makefile:100: update target '/usr/local/share/man/man6/intro.6' due to: /home/user/src/linux/man-pages/man6/intro.6
echo ' INSTALL /usr/local/share/man/man6/intro.6';
INSTALL /usr/local/share/man/man6/intro.6
install -m 644 -T '/home/user/src/linux/man-pages/man6/intro.6' '/usr/local/share/man/man6/intro.6';
Makefile:59: target '/usr/local/share/man/man7/' does not exist
echo ' INSTALL /usr/local/share/man/man7/';
INSTALL /usr/local/share/man/man7/
install -m 755 -d /usr/local/share/man/man7/;
Makefile:150: update target 'installdirs-man7' due to: /usr/local/share/man/man7/
:;
Makefile:115: target 'install-man7' does not exist
make '_install-man7';
Makefile:100: update target '/usr/local/share/man/man7/address_families.7' due to: /home/user/src/linux/man-pages/man7/address_families.7
echo ' INSTALL /usr/local/share/man/man7/address_families.7';
INSTALL /usr/local/share/man/man7/address_families.7
install -m 644 -T '/home/user/src/linux/man-pages/man7/address_families.7' '/usr/local/share/man/man7/address_families.7';
...XXX...
Makefile:100: update target '/usr/local/share/man/man7/xattr.7' due to: /home/user/src/linux/man-pages/man7/xattr.7
echo ' INSTALL /usr/local/share/man/man7/xattr.7';
INSTALL /usr/local/share/man/man7/xattr.7
install -m 644 -T '/home/user/src/linux/man-pages/man7/xattr.7' '/usr/local/share/man/man7/xattr.7';
Makefile:59: target '/usr/local/share/man/man8/' does not exist
echo ' INSTALL /usr/local/share/man/man8/';
INSTALL /usr/local/share/man/man8/
install -m 755 -d /usr/local/share/man/man8/;
Makefile:150: update target 'installdirs-man8' due to: /usr/local/share/man/man8/
:;
Makefile:115: target 'install-man8' does not exist
make '_install-man8';
Makefile:100: update target '/usr/local/share/man/man8/iconvconfig.8' due to: /home/user/src/linux/man-pages/man8/iconvconfig.8
echo ' INSTALL /usr/local/share/man/man8/iconvconfig.8';
INSTALL /usr/local/share/man/man8/iconvconfig.8
install -m 644 -T '/home/user/src/linux/man-pages/man8/iconvconfig.8' '/usr/local/share/man/man8/iconvconfig.8';
...XXX...
Makefile:100: update target '/usr/local/share/man/man8/zic.8' due to: /home/user/src/linux/man-pages/man8/zic.8
echo ' INSTALL /usr/local/share/man/man8/zic.8';
INSTALL /usr/local/share/man/man8/zic.8
install -m 644 -T '/home/user/src/linux/man-pages/man8/zic.8' '/usr/local/share/man/man8/zic.8';
Makefile:119: update target 'install-man' due to: install-man1 install-man2 install-man3 install-man4 install-man5 install-man6 install-man7 install-man8
:;
EDIT: Remove echo lines
I removed echo lines, and got a very important improvement (around 30% faster in both single- and multi-threaded mode):
~/src/linux/man-pages$ time sudo make >/dev/null
real 0m20.190s
user 0m11.629s
sys 0m7.177s
~/src/linux/man-pages$ time sudo make -j2 >/dev/null
real 0m2.917s
user 0m5.469s
sys 0m1.046s
~/src/linux/man-pages$ time sudo make -j >/dev/null
real 0m1.434s
user 0m5.906s
sys 0m1.170s
EDIT: Remove actual work lines
~/src/linux/man-pages$ time sudo make >/dev/null
real 0m7.260s
user 0m5.445s
sys 0m2.110s
~/src/linux/man-pages$ time sudo make -j2 >/dev/null
real 0m1.754s
user 0m2.798s
sys 0m0.411s
~/src/linux/man-pages$ time sudo make -j >/dev/null
real 0m1.229s
user 0m3.059s
sys 0m0.402s
If instead of the echo lines, I remove the lines that do the actual work, I get the rest of the improvement.
It looks like most of the time is actually happening at work, but then how does it improve so much by using -j2? Maybe I'm keeping the cache hot?
Related
I ran a particular Makefile on my Lubuntu machine, and it created a couple directories of files in my /usr/local/lib directory.
Tonight I'm trying to create a similar setup in a cygwin window on my Windows 10 machine. Same Makefile, but the /usr/local/lib directory ends up empty. So when I run a different Makefile later on that wants to call these /usr/local/lib libraries, it fails.
What could I be doing wrong? Is there an environment variable that needs to be set such that the /usr/loal/lib directory can be written to during the make?
Thanks.
Update: I just figured out that I can install Ubuntu as an application in Windows https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/ , and my makefile install is working great now. No problems at all, so it appears to be a cygwin problem only. That being said, here is my make file:
#! /usr/bin/make -f
# -*- make -*-
#
# Main gcc Makefile.
# This makefile is maintained by Greg King <greg.king4#verizon.net>.
# Goals that are supported by the cc65 package
.PHONY: all bins libs docs samples tests clean zap
.PHONY: uninstall install install-bins install-libs install-docs install-samps
# If SYS is defined on this makefile's command-line, then we want it to go
# to "samples" and "tests", but not to the other rules. So, we disable a
# feature of GNU make that would have given ${SYS} to every sub-make.
#MAKEOVERRIDES=
# (That trick has been disabled.)
# To compile with custom make-options, set them here; for example:
#MAKEOPTS = -j 2 CFLAGS=-O4 CC=clang
MAKEOPTS =
# The install prefix and directories
prefix = /usr/local
exec_prefix = $(prefix)
bindir = $(exec_prefix)/bin
datadir = $(prefix)/share
docdir = $(datadir)/doc
libdir = $(exec_prefix)/lib
CC65_DOC = $(docdir)/cc65
CC65_HOME = $(libdir)/cc65
CA65_INC = $(CC65_HOME)/asminc
CC65_INC = $(CC65_HOME)/include
LD65_CFG = $(CC65_HOME)/cfg
LD65_LIB = $(CC65_HOME)/lib
LD65_OBJ = $(CC65_HOME)/obj
# Programs
MKDIR = mkdir -m 755
# BSD-like install-script/-program
INSTALL = make/install-sh
INSTALL_DATA = $(INSTALL) -c -m 644
INSTALL_PROG = $(INSTALL) -c -m 755
INSTALL_STRIP = $(INSTALL_PROG) -s
# This file-name extension is needed on DOS/Windows systems.
ifdef COMSPEC
EXT = .exe
endif
# Rules
# The sample and library-test programs must be compiled for only one platform
# at a time. So, those rules are done automatically only when a user names
# a system on the command-line. (A user can do those rules with their
# defaults by putting "all samples tests" on the command-line.)
#
all: bins libs docs $(SYS:%=samples tests)
bins:
#$(MAKE) -C src -f make/gcc.mak $(MAKEOPTS) \
CA65_INC=\\\"${CA65_INC}/\\\" CC65_INC=\\\"${CC65_INC}/\\\" \
LD65_CFG=\\\"${LD65_CFG}/\\\" LD65_LIB=\\\"${LD65_LIB}/\\\" \
LD65_OBJ=\\\"${LD65_OBJ}/\\\"
libs:
#$(MAKE) -C libsrc
# This rule won't try to generate HTML files
# if a host system doesn't have LinuxDoc Tools.
docs:
#if linuxdoc -B check doc/index >/dev/null 2>&1; \
then $(MAKE) -C doc $(MAKEOPTS) html; \
else echo '"LinuxDoc Tools" is not installed; skipping HTML documentation.'; \
fi
# Some platforms cannot compile all of the sample and library-test programs.
# So, these rules ignore errors.
samples:
-#$(MAKE) -k -C samples prefix=$(prefix) $(SYS:%=SYS=%)
tests:
-#$(MAKE) -k -C testcode/lib prefix=$(prefix) $(SYS:%=SYS=%)
clean zap:
#$(MAKE) -C src -f make/gcc.mak $#
#$(MAKE) -C libsrc $#
#$(MAKE) -C doc $#
#$(MAKE) -C samples $#
# #$(MAKE) -C testcode/lib $# $(SYS:%=SYS=%)
uninstall: install-test
cd $(bindir) && $(RM) ar65${EXT} ca65${EXT} cc65${EXT} cl65${EXT} \
co65${EXT} da65${EXT} ld65${EXT} od65${EXT} grc65${EXT} ca65html
$(RM) -R $(CC65_HOME) $(CC65_DOC)
install: install-test install-dirs install-bins install-libs install-docs
#echo
#echo 'If you installed the files into non-default directories, then'
#echo 'you might need to export some shell environment variables:'
#echo
#echo 'CC65_HOME=$(CC65_HOME)'
#echo ' or'
#echo 'CA65_INC=$(CA65_INC)'
#echo 'CC65_INC=$(CC65_INC)'
#echo 'LD65_CFG=$(LD65_CFG)'
#echo 'LD65_LIB=$(LD65_LIB)'
#echo 'LD65_OBJ=$(LD65_OBJ)'
#echo
#if [ -x $(bindir)/grc${EXT} ]; then \
echo 'grc was renamed to grc65; but, a grc command is in your binaries directory.'; \
echo "If that command is an old copy of CC65's program,"; \
echo 'then you should use a "${MAKE} erase-grc" command to remove it.'; \
fi
.PHONY: install-test
install-test:
#if [ `id -u` != 0 ]; then \
echo; \
echo 'If you are denied permission to install or uninstall this package,'; \
echo 'then you will need to do "make/gcc.mak install" or "make/gcc.mak uninstall"'; \
echo 'as either the root user or an administrator.'; \
echo; \
fi 2>/dev/null
.PHONY: install-dirs
install-dirs: $(bindir) $(datadir) $(docdir) $(libdir) \
$(CC65_DOC) $(CC65_HOME) \
$(CA65_INC) $(CC65_INC) \
$(CC65_INC)/em $(CC65_INC)/geos $(CC65_INC)/joystick \
$(CC65_INC)/mouse $(CC65_INC)/sys $(CC65_INC)/tgi \
$(LD65_CFG) $(LD65_LIB) $(LD65_OBJ) \
$(CC65_HOME)/emd $(CC65_HOME)/joy $(CC65_HOME)/mou \
$(CC65_HOME)/ser $(CC65_HOME)/tgi
$(bindir) $(datadir) $(docdir) $(libdir) \
$(CC65_DOC) $(CC65_HOME) \
$(CA65_INC) $(CC65_INC) \
$(LD65_CFG) $(LD65_LIB) $(LD65_OBJ):
$(MKDIR) -p $# || $(MKDIR) $#
$(CC65_HOME)/% $(CC65_INC)/% $(CC65_DOC)/%:
$(MKDIR) $#
install-bins:
for f in ar65 ca65 cc65 cl65 co65 da65 ld65 od65 grc65 sp65; \
do $(INSTALL_STRIP) src/$$f/$$f${EXT} $(bindir) || exit $$?; \
done
$(INSTALL_PROG) src/ca65html/ca65html $(bindir)
install-libs:
for f in asminc/*.inc; \
do $(INSTALL_DATA) $$f $(CA65_INC) || exit $$?; \
done
for f in include/*.h; \
do $(INSTALL_DATA) $$f $(CC65_INC) || exit $$?; \
done
for d in em geos joystick mouse sys tgi; \
do for f in include/$$d/*.h; \
do $(INSTALL_DATA) $$f $(CC65_INC)/$$d || exit $$?; \
done || exit $$?; \
done
for f in libsrc/*.lib; \
do $(INSTALL_DATA) $$f $(LD65_LIB) || exit $$?; \
done
for f in libsrc/*-*.o; \
do $(INSTALL_DATA) $$f $(LD65_OBJ) || exit $$?; \
done
for f in src/ld65/cfg/[!g]*-*.cfg; \
do $(INSTALL_DATA) $$f $(LD65_CFG) || exit $$?; \
done
for d in emd joy mou ser tgi; \
do for f in libsrc/*.$$d; \
do $(INSTALL_DATA) $$f $(CC65_HOME)/$$d || exit $$?; \
done || exit $$?; \
done
install-docs:
for f in src/ca65/macpack/*.mac; \
do $(INSTALL_DATA) $$f $(CC65_DOC) || exit $$?; \
done
for f in readme.1st compile.txt CREDITS BUGS internal.txt newvers.txt; \
do $(INSTALL_DATA) doc/$$f $(CC65_DOC) || exit $$?; \
done
if [ -f doc/index.htm* ]; \
then for f in doc/*.htm*; \
do $(INSTALL_DATA) $$f $(CC65_DOC) || exit $$?; \
done; \
fi
install-samps: ${addprefix $(CC65_DOC)/, $(shell find samples -type d)}
#$(MAKE) -C samples zap
for d in `find samples -type d`; \
do for f in $$d/*; \
do if [ -f $$f ]; \
then $(INSTALL_DATA) $$f $(CC65_DOC)/$$d || exit $$?; \
fi; \
done || exit $$?; \
done
erase-grc:
$(RM) $(bindir)/grc${EXT}
Currently, I am working on a makefile that takes a parameter "CLASS=xxx" and then compiles and does stuff with that value.
In the end, it runs an application ($APP) on a bunch of files.
I enter this command:
make default CLASS=Test_UART
and the makefile processes it thusly:
pc: $(APP)
make -C BUILDENV CLASS=$(CLASS) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes
../classCvt/classCvt <./Applications/$(CLASS).class> ./Applications/$(CLASS).ujc
time -p ./$(APP) ./Applications/$(CLASS).ujc `cat filelist.txt`
Hence it calls a makefile in my BUILDENV folder which does the following:
#USAGE: make -C <PATH_TO_THIS_FILES_PARENT_DIR> CLASS=<MY_JAVA_FILE_NAME_WITHOUT_JAVA_EXTENSION> OUT=<OUTPUT_FILE_NAME>
SELF := $(dir $(CURDIR)/$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)))
CLASS ?= PLEASE_SPECIFY_CLASS_PARAM
DIR := $(PWD)#print working directory
CCVT ?= $(SELF)/../../classCvt/classCvt
TOBIN ?= $(SELF)/../../classCvt/tobin
OUTPUT ?=
### Comment: Defining CMD.
ifeq ($(BUILD), just_filelist)
CMD = echo
else
ifeq ($(BUILD), PC)
CMD = echo
else
ifeq ($(BUILD), unopt)
CMD = $(TOBIN)
else
### Comment: Optimized CMD = tobin -c ccvt
CMD = $(TOBIN) -c $(CCVT)
endif
endif
endif
ifeq ($(OUTPUT), )
OUT = &1
else
OUT = $(DIR)/$(OUTPUT)
endif
ifeq ($(SKIPSELF), yes)
MYCLASS =
else
MYCLASS = $(DIR)/Applications/$(CLASS).class
endif
all:
CLASSPATH=$(SELF)/RT/real:$(SELF)/RT/fake:$(DIR) javac $(DIR)/Applications/$(CLASS).java
find $(SELF)/RT/real -iname "*.class" -type f > $(SELF)/files
ls $(DIR)/Applications/*.class | grep -v "$(CLASS)\.class" >> $(SELF)/files || true
cat $(SELF)/files | xargs $(CMD) $(MYCLASS) >$(OUT)
rm -f $(SELF)/files
What I would like to do is give a command like:
make default CLASS=Test1,Test2,Test3
and the makefile to process it for the 3 classes and put the given classes in a .txt and the default classes in a different .txt, something like this like this:
pc: $(APP)
make -C BUILDENV default_classes BUILD=list_default_classes OUTPUT=list_default_classes.txt
# make -C BUILDENV given_classes BUILD=list_given_classes OUTPUT=list_given_classes.txt CLASS=$(CLASS) SKIPSELF=yes
../classCvt/classCvt `cat list_given_classes.txt`./Applications/$(CLASS).ujc
#here the list_given_classes should now contain the .ujc files
time -p ./$(APP) `cat list_given_classes.txt` `cat list_default_classes.txt`
and for the makefile in the BUILDENV, I expect something like:
default_classes:
CLASSPATH=$(SELF)/RT/real:$(SELF)/RT/fake:$(DIR)
find $(SELF)/RT/real -iname "*.class" -type f > $(SELF)/files
ls $(DIR)/Applications/*.class | grep -v "$(CLASS1)\.class" "$(CLASS2)\.class">> $(SELF)/files || true
cat $(SELF)/files | xargs $(CMD) >$(OUT)
rm -f $(SELF)/files
given_classes:
javac $(DIR)/Applications/$(CLASS).java
find $(SELF)/RT/real -iname "*.class" -type f > $(SELF)/files
ls $(DIR)/Applications/*.class | grep -v "$(CLASS)\.class" >> $(SELF)/files || true
cat $(SELF)/files | xargs $(CMD) $(MYCLASS) >$(OUT)
rm -f $(SELF)/files
However, I'm not sure how to do this for a CLASS parameter containing multiple classes.
I'm thinking to try and parse the Test1,Test2,Test3 value into a list of 1,2,3 and then iterating over it. But no clue if this is a good way and even on how to do it.
What do you guys suggest?
Pretty way:
pc: $(APP)
define BUILD_CLASS
pc: pc-$(CLASS_SPLIT)
pc-$(CLASS_SPLIT):
make -C BUILDENV CLASS=$(CLASS_SPLIT) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes
../classCvt/classCvt <./Applications/$(CLASS_SPLIT).class> ./Applications/$(CLASS_SPLIT).ujc
time -p ./$(APP) ./Applications/$(CLASS_SPLIT).ujc `cat filelist.txt`
endef
CLASSES := $(shell echo $(CLASS) | tr ',' ' ')
$(foreach CLASS_SPLIT, $(CLASSES), $(eval $(BUILD_CLASS)))
Simple way:
pc: $(APP)
$(foreach C, $(shell echo $(CLASS) | tr ',' ' '), \
make -C BUILDENV CLASS=$(C) BUILD=just_filelist OUTPUT=filelist.txt SKIPSELF=yes && \
../classCvt/classCvt <./Applications/$(C).class> ./Applications/$(C).ujc && \
time -p ./$(APP) ./Applications/$(C).ujc `cat filelist.txt` &&) true
I would like to swap two files using a function / macro when running the uninstall target in my Makefile.
Here is what I have which is only leaving one of the files and removing the other two:
swap = mv $(1) $(3); mv $(2) $(1); mv $(3) $(2)
uninstall:
#$(call swap,`printf "%s\n" $(FILE).* | sort -r | sed -n 1p`,\
$(FILE),$(shell mktemp $(FILE).XXX))
If you know what files to swap you could try something like the following. I assume the pairs to swap are fooXXX, barXXX.
define SWAP_files
swap_$(1):
#temp=$$(shell mktemp temp.XXXXXX); \
to_swap_with=$$(patsubst foo%,bar%,$(1)); \
mv $(1) $$$$temp; \
mv $$$$to_swap_with $(1); \
mv $$$$temp $$$$to_swap_with
uninstall: swap_$(1)
endef
$(foreach foo,$(FOOS),$(eval $(call SWAP_files,$(foo))))
Foreach fooXXX file it instantiates a rule:
swap_fooXXX:
#temp=$(shell mktemp temp.XXXXXX); \
to_swap_with=$(patsubst foo%,bar%,fooXXX); \
mv fooXXX $$temp; \
mv $$to_swap_with fooXXX; \
mv $$temp $$to_swap_with
and a rule:
uninstall: swap_fooXXX
I'm trying to add an extra dependency to a rule in a parallel makefile. I might have found a way, but I'm in doubt. (I haven't written the original makefile and I'm not an expert in make.)
The original makefile looks like this:
VER = busybox-1.16.2
URL = http://busybox.net/downloads/$(VER).tar.bz2
export KBUILD_OUTPUT = $(ROOTDIR)/user/busybox/build-$(VER)
all: build-$(VER)/.config depmod.pl
$(MAKE) -C build-$(VER)
build-$(VER)/.config: $(ROOTDIR)/config/.config
mkdir -p build-$(VER)
sed -n \
-e '/_CROSS_COMPILER_PREFIX=/s:=.*:="$(CROSS_COMPILE)":' \
-e '/CONFIG_USER_BUSYBOX_/s:CONFIG_USER_BUSYBOX_:CONFIG_:p' \
$< > $#.uclinux-dist.new
set -e ; \
if [ ! -e $# ] || ! cmp -s $#.uclinux-dist.new $#.uclinux-dist.old ; then \
cp $#.uclinux-dist.new $#.uclinux-dist.old ; \
cp $#.uclinux-dist.old $# ; \
yes "" | $(MAKE) -C $(VER) oldconfig ; \
fi
depmod.pl: $(VER)/examples/depmod.pl
ln -sf $< $#
I want to add a 'download' rule to the make.
$(VER)/: $(VER).tar.bz2
tar -jxvf $(VER).tar.bz2
touch $#
$(VER).tar.bz2:
wget $(URL)
touch $#
This rule must be executed before anything else. The parallel build prevent constructs like
all: |$(VER)/ build-$(VER)/.config depmod.pl
(This works in single threaded builds.)
My solution so far is this:
VER = busybox-1.18.5
URL = http://busybox.net/downloads/$(VER).tar.bz2
export KBUILD_OUTPUT = $(ROOTDIR)/user/busybox/build-$(VER)
all: build-$(VER)/.config depmod.pl
$(MAKE) -C build-$(VER)
$(VER)/: $(VER).tar.bz2
tar -jxvf $(VER).tar.bz2
touch $#
$(VER).tar.bz2:
wget $(URL)
touch $#
build-$(VER)/.config: $(ROOTDIR)/config/.config | $(VER)/
mkdir -p build-$(VER)
sed -n \
-e '/_CROSS_COMPILER_PREFIX=/s:=.*:="$(CROSS_COMPILE)":' \
-e '/CONFIG_USER_BUSYBOX_/s:CONFIG_USER_BUSYBOX_:CONFIG_:p' \
$< > $#.uclinux-dist.new
set -e ; \
if [ ! -e $# ] || ! cmp -s $#.uclinux-dist.new $#.uclinux-dist.old ; then \
cp $#.uclinux-dist.new $#.uclinux-dist.old ; \
cp $#.uclinux-dist.old $# ; \
yes "" | $(MAKE) -C $(VER) oldconfig ; \
fi
depmod.pl: $(VER)/examples/depmod.pl
ln -sf $< $#
$(VER)/examples/depmod.pl: | $(VER)/
Problem is, I don't really know what kind of magic the depmod.pl rule is. Is it executed correctly, now that I've added an explicit empty rule?
I'd hate to answer my own question, but I think I've found the answer.
The dependency of the depmod.pl rule is not real/relevant in the original script.
The code:
depmod.pl: $(VER)/examples/depmod.pl
ln -sf $< $#
Ought to be written:
depmod.pl:
ln -sf $(VER)/examples/depmod.pl $#
So my extra empty rule makes no difference. In the new script, the dependency almost makes sence though.
I am getting an error during compilation of a source code. I searched it but fail to find name of package which fix the problem.
Error
ls: cannot access /lib/modules/3.5.0-17-generic/source: No such file or directory
source code of make file
# Makefile for bcm_wimax module
# Customized for ZTE AX226
# Makefile recreated be Minhazul Haq Shawon
KERNEL_VER := $(shell uname -r)
KDIR := /lib/modules/$(KERNEL_VER)/build
KSRC := /lib/modules/$(KERNEL_VER)/source
INSTALL_DIR := /lib/modules/$(KERNEL_VER)/kernel/drivers/staging/bcm/
TARGET_DRV := bcm_wimax
EXTRAFLAGS := -Wall O=$(KDIR)
PWD:= $(shell pwd)
obj-m = $(TARGET_DRV).o
$(TARGET_DRV)-objs :=InterfaceDld.o InterfaceIdleMode.o InterfaceInit.o InterfaceRx.o \
InterfaceIsr.o InterfaceMisc.o InterfaceTx.o \
CmHost.o IPv6Protocol.o Qos.o Transmit.o\
Bcmnet.o DDRInit.o HandleControlPacket.o\
LeakyBucket.o Misc.o sort.o Bcmchar.o hostmibs.o PHSModule.o\
led_control.o nvm.o vendorspecificextn.o
default:
$(MAKE) $(EXTRAFLAGS) -C $(KSRC) SUBDIRS=$(PWD) modules
rm -f *.o *.mod.* .*.cmd
install:
cp $(TARGET_DRV).ko $(INSTALL_DIR)
clean:
find . -name \*.o -exec rm -rf '{}' ';'
find . -name .\*.o.cmd -exec rm -rf '{}' ';'
find . -name \*.*~ -exec rm -rf '{}' ';'
find . -name \*.*.bak -exec rm -rf '{}' ';'
rm -f *.ko *.o *.mod.* .*.cmd
rm -fr .tmp_versions
rm -rf Module.symvers
You have to compile kernel source first.