I have a package that uses autotools for build/install/etc. It is a Python module written in C++. (This isn't important except to know that the python scripts I have to install aren't part of what's being built, i.e. they're not a xxxxx_SOURCES primary.) This module is used solely for some "unit tests" for a driver that our team distributes in a HW solution. For various reasons, I have need to "install" the python unit tests with the system.
Since these python scripts aren't part of the module, I'm managing the installation separately in the Makefile. Here's what I have in my Makefile.am:
EXTRA_DIST = setupenv.sh bootstrap tests
dist-hook:
rm -rf $$(find $(distdir)/tests -name \*.swp -o -name \*.pyc)
install-exec-hook:
mkdir -p $(prefix)/unit_tests/unittest2
for f in tests/*.py; do \
cp $$f $(prefix)/unit_tests; \
done
for f in tests/unittest2/*.py; do \
cp $$f $(prefix)/unit_tests/unittest2; \
done
uninstall-hook:
rm -r $(prefix)/unit_tests
This works just fine except during rpmbuild for the module. The install-exec-hook rule isn't written correctly to make the directory where the rpm process redirects it. That is, when mkdir - $(prefix)/... is executed, the script literally tries to make the dir: /opt/oursw/.... How should this rule be rewritten so that rpmbuild puts them in the same place as the installation for the module?
You want to add $(DESTDIR) as a prefix to all those directories. That's standard for autotools installs.
As an aside, you also might want to look at the _SCRIPTS suffix, which allows "make install" to copy your scripts in for you the "official" way, removing the need for this hack in the first place.
Related
I have a simple JUnit test that I successfully run inside Eclipse on macOS Mojave. The class makes a JNA call to a dynamic library, so I had to set the Runtime environment variable DYLD_LIBRARY_PATH
When I try to run mvn test both inside or outsite of Eclipse, they fail.
The reason, so I learned, is macOS' System Integrity Protection which wipes all DYLD variables. However it must be possible to set them, somehow, since the JUnit test inside Eclipse works as designed.
I tried to "hack" the mvn shell script which ar the very end executes:
exec "$JAVACMD" \
$MAVEN_OPTS \
$MAVEN_DEBUG_OPTS \
-classpath "${CLASSWORLDS_JAR}" \
"-Dclassworlds.conf=${MAVEN_HOME}/bin/m2.conf" \
"-Dmaven.home=${MAVEN_HOME}" \
"-Dlibrary.jansi.path=${MAVEN_HOME}/lib/jansi-native" \
"-Dmaven.multiModuleProjectDirectory=${MAVEN_PROJECTBASEDIR}" \
${CLASSWORLDS_LAUNCHER} "$#"
by setting the DYLD_LIBRARY_PATH immediately before the exec command, without success.
Also tried to patch $JAVACMD to set the variable first, no luck.
What options did I miss (short of disabling SIP)?
I'm not looking for a general solution, running mvn is my goal
Similar questions:
Why isn't DYLD_LIBRARY_PATH being propagated here?
LD_LIBRARY_PATH and DYLD_LIBRARY_PATH not imported on OS X
How to set LD_LIBRARY_PATH/DYLD_LIBRARY_PATH on macos
I found a solution, I don't like it, but it seems to work. The JNA call went out to load additional code that uses the #executable_path annotation. Running Java, the executable is the JVM, not the original application and thus the JNA call would fail.
Linking the whole program directory (and cleaning up thereafter) did the trick for me. My call to mvn looks like this:
#!/bin/bash
# Ugly hack to make mvn play ball
JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
for onelib in $(ls /Applications/HCL\ Notes.app/Contents/MacOS)
do
echo "Linking /Applications/HCL\ Notes.app/Contents/MacOS/${onelib}"
sudo ln -s /Applications/HCL\ Notes.app/Contents/MacOS/${onelib} ${JAVA_HOME}/jre/bin/${onelib}
done
# Maven test
mvn test
# Cleanup
sudo find $JAVA_HOME/jre/bin -maxdepth 1 -type l -delete
YMMV
I just want to use GNU make to compress some files.
So I wrote the Makefile as follows:
lib.tar.lzma: $(shell find ~/lib -name "*")
rm -f lib.tar.lzma
tar -cavf lib.tar.lzma -C ~/ lib/
However, after I run make, it automatically compile the c++ source code in that directory.
How can I stop it from compiling them? I just want to compress them.
Update:
I got the following error:
<builtin>: recipe for target '/home/xxx/lib/app' failed
It seems a built-in recipe.
(We don't know your entire Makefile and your full file tree, so this is only a guess; I assume that you have shown us a fragment of your much bigger Makefile)
However, after I run make, it automatically compile the c++ source code in that directory.
This is probably happening because your $(shell find ~/lib -name "*") is expanded to something containing your object files. Since they are in your dependencies their source file is recompiled if it is newer. BTW you might want to use instead $(shell cd .. ; find lib -name "*") or if lib has no subdirectory even $(wildcard ../lib/*)
You probably don't need any dependency for that lib.tar.lzma target, so just have:
lib.tar.lzma:
rm -f lib.tar.lzma
tar -cavf lib.tar.lzma -C ~/ lib/
BTW, that -C ~/ perhaps should be -C $$HOME since make use /bin/sh to run commands, and that POSIX shell don't know about ~ ; perhaps a -C .. might be better ...
Perhaps you might write some shell script make-backup.sh to do a more clever tar and you would then code
lib.tar.lzma: make-backup.sh
./make-backup.sh $#
However, perhaps you do have dependencies (e.g. if you need to archive some generated files). Then you need to list them explicitly and wisely (you certainly don't want to depend on all the files; perhaps only the source ones). Also, you might not need to archive any object files *.o, if you have some (but YMMV).
I recommend using make --trace or remake -x to debug your Makefile.
BTW, having a Makefile only for a backup is useless; write a shell script instead.
I also strongly recommend using some version control system (like git) if you don't use any. Notice that git has an archive subcommand which might be a more clever backup.
In my case I have requirements target, which installs needed Python packages and test, which runs tests and depends on previous one.
Installing dependencies is a long operation and I want it to be executed only when requirements.txt changes. How can I achieve that?
Here is a simplified example of Makefile, that I have now:
.PHONY: test requirements
requirements: requirements.txt
pip install -r $<
test: tests/ | requirements
py.test $^
As #user1034749 pointed out, Make compares the modification times of files. If you want it to know when requirements.txt has been modified since the last installation, you must give it a file whose modification time is the same as the time of the last installation, so that it can compare the two. In other words, you must have a dummy file and modify it whenever you perform the installation. You can call it anything you like, but I will call it "installation":
.PHONY: test
installation: requirements.txt
pip install -r $<
touch $#
test: tests/ | installation
py.test $^
I know the process of installing from source are.
./configure
make
make install
But why "make" before /etc/cups/cupsd.conf, why not just do "make install"?
My understanding so far is "make" only compile the source into executable file, and "make install" actually place them into executable PATH folder, am I right?
If we want to install executable on the machine, can we just do
./configure
make install
Instead of 3 steps shown above.
When you run make, you're instructing it to essentially follow a set of build steps for a particular target. When make is called with no parameters, it runs the first target, which usually simply compiles the project. make install maps to the install target, which usually does nothing more than copy binaries into their destinations.
Frequently, the install target depends upon the compilation target, so you can get the same results by just running make install. However, I can see at least one good reason to do them in separate steps: privilege separation.
Ordinarily, when you install your software, it goes into locations for which ordinary users do not have write access (like /usr/bin and /usr/local/bin). Often, then, you end up actually having to run make and then sudo make install, as the install step requires a privilege escalation. This is a "Good Thing™", because it allows your software to be compiled as a normal user (which actually makes a difference for some projects), limiting the scope of potential damage for a badly-behaving build procedure, and only obtains root privileges for the install step.
make without parameters takes the ./Makefile (or ./makefile) and builds the first target. By convention, this may be the all target, but not necessarily. make install builds the special target, install. By convention, this takes the results of make all, and installs them on the current computer.
Not everybody needs make install. For example, if you build some a web app to be deployed on a different server, or if you use a cross-compiler (e.g. you build an Android application on a Linux machine), it makes no sense to run make install.
In most cases, the single line ./configure && make all install will be equivalent to the three-step process you describe, but this depends on the product, on your specific needs, and again, this is only by a convention.
There are times I want to try to compile code changes but not deploy those changes. For instance, if I'm hacking the Asterisk C code base, and I want to make sure the changes I'm making still compile, I'll save and run make. However, I don't want to deploy those changes because I'm not done coding.
For me, running make is just a way to make sure I don't end up with too many compile errors in my code to where I have trouble locating them. Perhaps more experienced C programmers don't have that problem, but for me, limiting the number of changes between compiles helps reduce the number of possible changes that may have completely trashed my build, and this makes debugging easier.
Lastly, this also helps give me a stopping point. If I want to go to lunch, I know that someone can restart the application in it's currently working state without having to come find me, since only make install would copy the binaries over to the actual application folder.
There may very well be other reasons, but this is my reason for embracing the fact that the two commands are separated. As others have said, if you want them combined, you can combine them using your shell.
A lot of software these days will do the right thing with only make install.
In those that won't, the install target doesn't have a dependency on the compiled binaries.
So to play safe, most people use make && make install or a variation thereof just to be safe.
A simple Makefile example (real example), from https://github.com/jarun/googler#installation.
Most of the comments are added by me
make install PREFIX=YOUR_own_path
use zsh's autocomple to see what you can chose and you will
know many things!!
PREFIX ?= /usr/local
# These two are the same:
# FOO ?= bar
# ifeq ($(origin FOO), undefined)
# FOO = bar
# endif
# ---
BINDIR = $(DESTDIR)$(PREFIX)/bin
MANDIR = $(DESTDIR)$(PREFIX)/share/man/man1
DOCDIR = $(DESTDIR)$(PREFIX)/share/doc/googler
# the cmamnd `make YOUR_target_name`
# Call a specific target in ./Makefile (or ./makefile), which
# contains such pairs :
# targets:
# ^I shell_command_line_1
# ...
# ^I shell_command_line_n
# `make` can be regarded as using the default target: the first one in
# Makefile, which usually named `all`
# .PHONY: all install uninstall disable-self-upgrade
.PHONY: second all install uninstall disable-self-upgrade
# In terms of `Make`, whenever you ask `make <phony_target>`,
# it will run, independent from the state of what files you have,
# because a `phony target` is marked as always out-of-date
all:
echo "hi, this is the 'all' target"
my_first:
echo "hi, this is the first target"
second:
echo "hi, this is the 2nd target"
# the target `install` can usually be found in Makefile. You can change it to `buy` or others
install:
# from tldr: `install` command : Copy files and set attributes.
# -m --mode= set mode
# -d --dirctory
install --mode=755 -d $(BINDIR)
install -m755 -d $(MANDIR)
install -m755 -d $(DOCDIR)
gzip --to-stdout googler.1 > googler.1.gz
install -m755 googler $(BINDIR)
install -m644 googler.1.gz $(MANDIR)
install -m644 README.md $(DOCDIR)
rm -f googler.1.gz
# same as above
buy:
# from tldr: `install` command : Copy files and set attributes.
# -m --mode= set mode
# -d --dirctory
install --mode=755 -d $(BINDIR)
install -m755 -d $(MANDIR)
install -m755 -d $(DOCDIR)
gzip --to-stdout googler.1 > googler.1.gz
install -m755 googler $(BINDIR)
install -m644 googler.1.gz $(MANDIR)
install -m644 README.md $(DOCDIR)
rm -f googler.1.gz
uninstall:
rm -f $(BINDIR)/googler
rm -f $(MANDIR)/googler.1.gz
rm -rf $(DOCDIR)
# Ignore below if you don't use apt or others package managers to install this
# Disable the self-upgrade mechanism entirely. Intended for packagers
#
# We assume that sed(1) has the -i option, which is not POSIX but seems common
# enough in modern implementations.
disable-self-upgrade:
sed -i.bak 's/^ENABLE_SELF_UPGRADE_MECHANISM = True$$/ENABLE_SELF_UPGRADE_MECHANISM = False/' googler
I'm trying to setup a parallel CMake-based build for my source tree, but when I issue
$ cmake .
$ make -j2
I get:
jobserver unavailable: using -j1. Add '+' to parent make rule
as a warning. Does anyone have an idea if it is possible to fix it somehow?
In the generated Makefile, when calling into a sub-make it needs to either use $(MAKE) (not just 'make') or else precede the line with a +. That is, a rule should look like this:
mysubdir:
$(MAKE) -C mysubdir
or like this:
mysubdir:
+make -C mysubdir
If you don't do it one of those two ways, make will give you that warning.
I don't know anything about cmake, so maybe it's generating Makefiles that aren't correct. Or maybe you did something incorrectly on your end.
In my case (with CMake 3.5.2) the trivial cd build && cmake .. && make -j5 works just fine.
But, I do get the jobserver unavailable error when building custom targets (as dependencies of other targets) via the cmake --build . --target foo idiom.
Like this:
add_custom_target(buildroot
COMMAND ${CMAKE_COMMAND} --build . --target install
COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc
— so that the user can make deb and it Just Works. CMake will regenerate makefiles if needed, run the compilation, install everything exactly as with make install, and then run my custom scripts to package up the populated buildroot into whatever shape or form I need.
Sure enough, I'd like to make -j15 deb — but that fails.
Now, as explained on the mailing list by CMake devs, the root cause lies, surprisingly (or not), within GNU Make; there is a workaround.
The root cause is that make will not pass its jobserver environment to child processes it thinks aren't make.
To illustrate, here's a process tree (ps -A f) branch:
…
\_ bash
\_ make -j15 deb
\_ make -f CMakeFiles/Makefile2 deb
\_ make -f CMakeFiles/buildroot.dir/build.make CMakeFiles/buildroot.dir/build
\_ /usr/bin/cmake --build . --target install ⦿
\_ /usr/bin/gmake install
…
At ⦿ point, make drops jobserver environment, ultimately causing single-threaded compilation.
The workaround which worked great for me, as given away in the linked email, is to prefix all custom commands with +env. Like this:
add_custom_target(buildroot
#-- this ↓↓↓ here -- https://stackoverflow.com/a/41268443/531179
COMMAND +env ${CMAKE_COMMAND} --build . --target install
COMMENT "Populating buildroot..."
)
add_dependencies(deb buildroot)
add_dependencies(rpm buildroot) #... etc
In the end, this appears in the rule for buildroot in the appropriate makefile (CMake generates a bunch of them), and causes GNU Make to behave properly and respect -j.
Hope this helps.
As pointed out by #Carlo Wood in his comment to this answer, trying to convince cmake to add + to the beginning of the command in the cmake-generated makefile is not possible.
A work-around I found is to shield underlying make command from the make flags coming from cmake. This can be done by setting environment variable MAKEFLAGS to empty string for the custom command:
COMMAND ${CMAKE_COMMAND} -E env
MAKEFLAGS=
make <your target and make options>
Hope this helps.