I come from the NodeJS world so I consider the Makefile as the "scripts" part in an npm package.json, which may be wrong (or not ?) to do so.
So my idea is to automate repetitive actions when installing a new dependency by typing:
make install github.com/stretchr/testify
And find a way to get the github.com/stretchr/testify parameter without having to use the heavy parameter name-value declaration FOO=bar (=> make install DEP=github.com/stretchr/testify) generally suggested.
So, following this answer, I tried this:
install %:
go get $*
godep save ./...
git add Godeps vendor
git commit -m "godep: add $*"
but unsuccessfully: it runs go get without any param and git commit -m "godep: add".
Trials
1 - When I do that:
install %:
echo $*
I see my "github.com/stretchr/testify".
2 - When I do that:
install %:
go get ${*}
it loops twice and first run go get without any param, then runs go get github.com/stretchr/testify (as wished).
It looks like ${*} represents an "array" of params parsing characters groups after the target, the first one being the space between install and github.com/stretchr/testify and the second one being github.com/stretchr/testify.
You cannot use explicit targets and patterns in the same rule, so your rule % install: will not work.
You can do this with GNU make using the CMDGOALS variable, but it's very hackish and error-prone and I don't recommend it.
ARG := $(filter-out install,$(MAKECMDGOALS))
install:
go get $(ARG)
godep save ./...
git add Godeps vendor
git commit -m "godep: add $(ARG)"
As you can see you'll need to add handling for situations where there are no other arguments, or where there are more than one other argument, and of course you can't add any more targets without putting them in the filter-out list, etc.
Just... not a good way to do it IMO.
Why don't you do something like this instead:
install-%:
go get $*
godep save ./...
git add Godeps vendor
git commit -m "godep: add $*"
Then run:
make install-github.com/stretchr/testify
Make variables can probably do what you want:
host> cat Makefile
install:
go get $(P)
godep save ./...
git add Godeps vendor
git commit -m "godep: add $(P)"
host> make install P=github.com/stretchr/testify
But using make just for this is probably overkill. It is much more than a scripting language.
Related
I have a makefile that includes another makefile. This happens very early in the makefile. Something like:
include $(SOME_SDK)/Makefile.defines
However SOME_SDK is a submodule.
I would like to run git submodule update --init --recursive before getting to the point where the include happens.
Ideally, this should not be part of any target.
How can I do that?
You cannot have this command as part of some target, because the include directive is fulfilled before any target is calculated.
But you can have any shell command (or script) executed at any place of the Makefile, even before the include.
dummy := $(shell git submodule update --init --recursive)
include $(SOME_SDK)/Makefile.defines
In my case I have requirements target, which installs needed Python packages and test, which runs tests and depends on previous one.
Installing dependencies is a long operation and I want it to be executed only when requirements.txt changes. How can I achieve that?
Here is a simplified example of Makefile, that I have now:
.PHONY: test requirements
requirements: requirements.txt
pip install -r $<
test: tests/ | requirements
py.test $^
As #user1034749 pointed out, Make compares the modification times of files. If you want it to know when requirements.txt has been modified since the last installation, you must give it a file whose modification time is the same as the time of the last installation, so that it can compare the two. In other words, you must have a dummy file and modify it whenever you perform the installation. You can call it anything you like, but I will call it "installation":
.PHONY: test
installation: requirements.txt
pip install -r $<
touch $#
test: tests/ | installation
py.test $^
I know the process of installing from source are.
./configure
make
make install
But why "make" before /etc/cups/cupsd.conf, why not just do "make install"?
My understanding so far is "make" only compile the source into executable file, and "make install" actually place them into executable PATH folder, am I right?
If we want to install executable on the machine, can we just do
./configure
make install
Instead of 3 steps shown above.
When you run make, you're instructing it to essentially follow a set of build steps for a particular target. When make is called with no parameters, it runs the first target, which usually simply compiles the project. make install maps to the install target, which usually does nothing more than copy binaries into their destinations.
Frequently, the install target depends upon the compilation target, so you can get the same results by just running make install. However, I can see at least one good reason to do them in separate steps: privilege separation.
Ordinarily, when you install your software, it goes into locations for which ordinary users do not have write access (like /usr/bin and /usr/local/bin). Often, then, you end up actually having to run make and then sudo make install, as the install step requires a privilege escalation. This is a "Good Thing™", because it allows your software to be compiled as a normal user (which actually makes a difference for some projects), limiting the scope of potential damage for a badly-behaving build procedure, and only obtains root privileges for the install step.
make without parameters takes the ./Makefile (or ./makefile) and builds the first target. By convention, this may be the all target, but not necessarily. make install builds the special target, install. By convention, this takes the results of make all, and installs them on the current computer.
Not everybody needs make install. For example, if you build some a web app to be deployed on a different server, or if you use a cross-compiler (e.g. you build an Android application on a Linux machine), it makes no sense to run make install.
In most cases, the single line ./configure && make all install will be equivalent to the three-step process you describe, but this depends on the product, on your specific needs, and again, this is only by a convention.
There are times I want to try to compile code changes but not deploy those changes. For instance, if I'm hacking the Asterisk C code base, and I want to make sure the changes I'm making still compile, I'll save and run make. However, I don't want to deploy those changes because I'm not done coding.
For me, running make is just a way to make sure I don't end up with too many compile errors in my code to where I have trouble locating them. Perhaps more experienced C programmers don't have that problem, but for me, limiting the number of changes between compiles helps reduce the number of possible changes that may have completely trashed my build, and this makes debugging easier.
Lastly, this also helps give me a stopping point. If I want to go to lunch, I know that someone can restart the application in it's currently working state without having to come find me, since only make install would copy the binaries over to the actual application folder.
There may very well be other reasons, but this is my reason for embracing the fact that the two commands are separated. As others have said, if you want them combined, you can combine them using your shell.
A lot of software these days will do the right thing with only make install.
In those that won't, the install target doesn't have a dependency on the compiled binaries.
So to play safe, most people use make && make install or a variation thereof just to be safe.
A simple Makefile example (real example), from https://github.com/jarun/googler#installation.
Most of the comments are added by me
make install PREFIX=YOUR_own_path
use zsh's autocomple to see what you can chose and you will
know many things!!
PREFIX ?= /usr/local
# These two are the same:
# FOO ?= bar
# ifeq ($(origin FOO), undefined)
# FOO = bar
# endif
# ---
BINDIR = $(DESTDIR)$(PREFIX)/bin
MANDIR = $(DESTDIR)$(PREFIX)/share/man/man1
DOCDIR = $(DESTDIR)$(PREFIX)/share/doc/googler
# the cmamnd `make YOUR_target_name`
# Call a specific target in ./Makefile (or ./makefile), which
# contains such pairs :
# targets:
# ^I shell_command_line_1
# ...
# ^I shell_command_line_n
# `make` can be regarded as using the default target: the first one in
# Makefile, which usually named `all`
# .PHONY: all install uninstall disable-self-upgrade
.PHONY: second all install uninstall disable-self-upgrade
# In terms of `Make`, whenever you ask `make <phony_target>`,
# it will run, independent from the state of what files you have,
# because a `phony target` is marked as always out-of-date
all:
echo "hi, this is the 'all' target"
my_first:
echo "hi, this is the first target"
second:
echo "hi, this is the 2nd target"
# the target `install` can usually be found in Makefile. You can change it to `buy` or others
install:
# from tldr: `install` command : Copy files and set attributes.
# -m --mode= set mode
# -d --dirctory
install --mode=755 -d $(BINDIR)
install -m755 -d $(MANDIR)
install -m755 -d $(DOCDIR)
gzip --to-stdout googler.1 > googler.1.gz
install -m755 googler $(BINDIR)
install -m644 googler.1.gz $(MANDIR)
install -m644 README.md $(DOCDIR)
rm -f googler.1.gz
# same as above
buy:
# from tldr: `install` command : Copy files and set attributes.
# -m --mode= set mode
# -d --dirctory
install --mode=755 -d $(BINDIR)
install -m755 -d $(MANDIR)
install -m755 -d $(DOCDIR)
gzip --to-stdout googler.1 > googler.1.gz
install -m755 googler $(BINDIR)
install -m644 googler.1.gz $(MANDIR)
install -m644 README.md $(DOCDIR)
rm -f googler.1.gz
uninstall:
rm -f $(BINDIR)/googler
rm -f $(MANDIR)/googler.1.gz
rm -rf $(DOCDIR)
# Ignore below if you don't use apt or others package managers to install this
# Disable the self-upgrade mechanism entirely. Intended for packagers
#
# We assume that sed(1) has the -i option, which is not POSIX but seems common
# enough in modern implementations.
disable-self-upgrade:
sed -i.bak 's/^ENABLE_SELF_UPGRADE_MECHANISM = True$$/ENABLE_SELF_UPGRADE_MECHANISM = False/' googler
I use a lot of commands providing exposing a sort of API via subcommands. For instance,
git push
bzr push
apt-get install
After a while, I get tired of writing git push, git commit, git something... because I know the only thing I'm using to 'commit', 'push', etc. is git at this moment.
Having played with languages providing a REPL (Ruby, Python, etc.) I was missing the convenience of typing.
$ git pus... ## arrgh!
$ from git import *
$ push ## yes!
I see a clear symmetry between commands like git and namespaces or modules in the forementined programming languages.
So, the question is: what does it take to have support for namespaces in SHELL? or vice-versa, What does it take to have these language replace the SHELL?
This would be extremely hard to do generically. However, you can approximate something similar...
#!/usr/bin/env bash
cmd="${1-echo}"
history -r
while read -p"$cmd\$ " -e -r -a input ; do
history -s -- "${input[#]}"
"$cmd" "${input[#]}"
done
Invoke with the name of the command, e.g.
./wrapper apt-get
And now when you say e.g. install foo you will actually execute apt-get install foo.
Additional improvements, such as more intelligent tab completion, are possible but would certainly require command-specific code.
I'm trying to setup a parallel CMake-based build for my source tree, but when I issue
$ cmake .
$ make -j2
I get:
jobserver unavailable: using -j1. Add '+' to parent make rule
as a warning. Does anyone have an idea if it is possible to fix it somehow?
In the generated Makefile, when calling into a sub-make it needs to either use $(MAKE) (not just 'make') or else precede the line with a +. That is, a rule should look like this:
mysubdir:
$(MAKE) -C mysubdir
or like this:
mysubdir:
+make -C mysubdir
If you don't do it one of those two ways, make will give you that warning.
I don't know anything about cmake, so maybe it's generating Makefiles that aren't correct. Or maybe you did something incorrectly on your end.
In my case (with CMake 3.5.2) the trivial cd build && cmake .. && make -j5 works just fine.
But, I do get the jobserver unavailable error when building custom targets (as dependencies of other targets) via the cmake --build . --target foo idiom.
Like this:
add_custom_target(buildroot
COMMAND ${CMAKE_COMMAND} --build . --target install
COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc
— so that the user can make deb and it Just Works. CMake will regenerate makefiles if needed, run the compilation, install everything exactly as with make install, and then run my custom scripts to package up the populated buildroot into whatever shape or form I need.
Sure enough, I'd like to make -j15 deb — but that fails.
Now, as explained on the mailing list by CMake devs, the root cause lies, surprisingly (or not), within GNU Make; there is a workaround.
The root cause is that make will not pass its jobserver environment to child processes it thinks aren't make.
To illustrate, here's a process tree (ps -A f) branch:
…
\_ bash
\_ make -j15 deb
\_ make -f CMakeFiles/Makefile2 deb
\_ make -f CMakeFiles/buildroot.dir/build.make CMakeFiles/buildroot.dir/build
\_ /usr/bin/cmake --build . --target install ⦿
\_ /usr/bin/gmake install
…
At ⦿ point, make drops jobserver environment, ultimately causing single-threaded compilation.
The workaround which worked great for me, as given away in the linked email, is to prefix all custom commands with +env. Like this:
add_custom_target(buildroot
#-- this ↓↓↓ here -- https://stackoverflow.com/a/41268443/531179
COMMAND +env ${CMAKE_COMMAND} --build . --target install
COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc
In the end, this appears in the rule for buildroot in the appropriate makefile (CMake generates a bunch of them), and causes GNU Make to behave properly and respect -j.
Hope this helps.
As pointed out by #Carlo Wood in his comment to this answer, trying to convince cmake to add + to the beginning of the command in the cmake-generated makefile is not possible.
A work-around I found is to shield underlying make command from the make flags coming from cmake. This can be done by setting environment variable MAKEFLAGS to empty string for the custom command:
COMMAND ${CMAKE_COMMAND} -E env
MAKEFLAGS=
make <your target and make options>
Hope this helps.