I've written a scons build chain form a little C project, but I'm afraid users won't like to be told "You should install SCons first. Besides, it's really cool!" (expecially my professor, as he's kind of from the old guard).
Is there a way I can set up a Makefile that will wrap scons, not requiring it to be installed on the target system?
After looking for such a solution some time ago, I ended up writing a Makefile for this purpose.
Because SCons also comes as a drop-in userspace package scons-local (see the download page), one can fetch in and run it. Here is a dissectioned and commented version of my Makefile, which I also uploaded as a gist.
all: get_scons
#$(SCONS_EXE)
↑ The default action depends on scons being available, and simply runs the scons command (set later in the script) (the # symbol prevents make from printing the command)
SCONS_VERSION=2.3.4
scons-local-%.tar.gz:
curl -L http://sourceforge.net/projects/scons/files/scons-local/$(SCONS_VERSION)/scons-local-$(SCONS_VERSION).tar.gz > scons-local-$(SCONS_VERSION).tar.gz
touch scons-local-$(SCONS_VERSION).tar.gz
scons-local: scons-local-$(SCONS_VERSION).tar.gz
mkdir -p scons-local
tar xzf scons-local-$(SCONS_VERSION).tar.gz --directory scons-local
touch scons-local
↑ Set up the rules for fetching the tarball and unpack it into the scons-local directory
NATIVE_SCONS=$(strip $(shell which scons 2>/dev/null))
ifeq ($(NATIVE_SCONS),)
SCONS_EXE=python2 ./scons-local/scons.py
get_scons: scons-local
#echo "Couldn't find an installation of SCons, using a local copy"
else
SCONS_EXE=$(NATIVE_SCONS)
get_scons:
#echo "Found SCons installation at $(SCONS_EXE)"
endif
↑ Look for the scons executable in the search path (using the which command): if it is available, set up the get-scons target to simply print it is available. If, instead, it is not available, create the get-scons target instructing it to depend on the scons-local target defined earlier.
clean:
$(SCONS_EXE) -c
rm -rf scons-local
rm -f scons-local-*.tar.gz
.PHONY: all clean get_scons
↑ Finally, set-up the clean target that delegates to scons and deletes the local installation afterwards. The .PHONY rule tells make that the following rules do not correspond to files being created.
At this point, one could add more proxy rules of the kind:
mytarget: get_scons
#$(SCONS_EXE) mytarget
Which will invoke scons with the corresponding target.
Hope this is useful, feel free to correct me in case there's something wrong (I'm actually not a Makefile expert, and I'm trying not to become one by using SCons instead :P )
Related
Is there a way to have a 'watch' target in my Makefile which would keep looping and rebuilding the project every time the source file gets changed?
I have this for my Latex project:
.PHONY : monitor
monitor:
while true; do \
inotifywait -e modify -q *.tex *.cls; make all; \
done
Interesting arguments:
-q for quiet
-r for recursive (if you want to watch the whole src folder)
-e to list specific events (if your editor does more file operations and retriggers the build way too often)
--exclude to exclude some (if your src folder contains build artifacts) to make sure the build itself will not retrigger this loop (which would be equivalent of an infinite loop without any delays)
More arguments here (inotify tools are amazing):
https://linux.die.net/man/1/inotifywait
Depending on your distribution you might have to install separate package, on my Debian I had to do
sudo apt-get install inotify-tools
Every now and then a new tarball or a new xyHub/Lab-repository needs to be built. They usually come with a Makfile or an Autotools/CMake/XY-Generator provides one on the fly. As the maintainers most likely use another operating system or distribution than the one I am currently running, the assumptions that went into their Makefiles usually do not fit my filesystem hierarchy (lib vs. lib64, bin vs. sbin, /usr/lib vs. /lib and so on). As the final command in the build sequence usually is
sudo make install
it is quite annoying to move thousands of files to the correct place. Or even worse determine which files of my distribution were overwritten. Here GNU Makes dry run mode comes in very handy. Running
sudo make -n install
first, saves me the trouble of cleaning up my file system, by just printing all the commands from all active GNU Make recepies without executing them. In case of a handwritten or Autotools-generated Makfile this works as intended. If the Makefile contains something like:
#PREFIX is environment variable, but if it is not set, then set default value
ifeq ($(PREFIX),)
PREFIX := /usr/local
endif
install: unixlib.a
install -d $(DESTDIR)$(PREFIX)/lib/
install -m 644 unixlib.a $(DESTDIR)$(PREFIX)/lib/
install -d $(DESTDIR)$(PREFIX)/include/
install -m 644 unixlib.h $(DESTDIR)$(PREFIX)/include/
I would see exactly what would happen. Every install/cp/mv-command with the full path information would be printed. If I made a mistake with the install prefix in the configure step I can see it there. If the default in the Makefile is weird because it comes from another OS, I would see it there.
Now in case of a CMake-generated Makefile this is different. Doing
mkdir build && cd build
cmake ..
make
sudo make -n install
only produces output that ends in
...
make -f CMakeFiles/Makefile2 preinstall
/usr/bin/cmake -E cmake_echo_color --switch= --cyan "Install the project..."
/usr/bin/cmake -P cmake_install.cmake
As these commands get not executed, just printed, I do not get all the cp/mv/mkdir/install/etc-commands that I would like to see first, before I let the Makefile touch the file system.
Is there a way to get the list of commands that would be executed from the install target in a CMake-generated Makefile as it is the case with handwritten or Autotools-generated ones?
Is there a way to get the list of commands that would be executed from the install target.
Actually, the core part of installation process is contained in the file cmake_install.cmake (which is created in the build directory). This file is processed as CMake script using cmake -P flow of the cmake executable.
The script cmake_install.cmake performes installation of files with install command. Semantic of the install command, used by the script, differs from the one described in documentation: internally, CMake uses some undocumented features of the command.
But it shouldn't be so hard to understand cmake_install.cmake script in general and deduce paths from it.
I'm trying to have CMake either run three bash commands or a bash script. However, I can't seem to get it to work.
The bash commands are:
cd ${CMAKE_SOURCE_DIR}/dependencies/library
make
cd ${CMAKE_BINARY_DIR}
Essentially, I would like CMake to build the library in that directory if it does not already exist.
Here's the CMake code I tried:
if(NOT "${CMAKE_SOURCE_DIR}/dependencies/library/lib.o")
execute_process(COMMAND cd ${CMAKE_SOURCE_DIR}/dependencies/library)
execute_process(COMMAND make)
execute_process(COMMAND cd ${CMAKE_BINARY_DIR})
endif(NOT "${CMAKE_SOURCE_DIR}/dependencies/library/lib.o")
However, it's not building anything. What am I doing wrong?
Also, while I'm here asking this: should the third command, to move to the binary folder, be included?
Thanks!
execute_process() is executed during configure time. But you want this to run at build time, thus add_custom_command() and add_custom_target() is what you're looking for.
In this special case you want to generate an output file, so you should go for add_custom_command() (both are essentially the same, but command produces one or multiple output files, while target does not.
The cmake snippet for this should look something like the following:
add_custom_command(
OUTPUT ${CMAKE_SOURCE_DIR}/dependencies/library/lib.o
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/dependencies/library
COMMAND make
)
You then have to add the output file in another target as dependency, and everything should (hopefully) work as expected.
You can also add DEPENDS statements to the add_custom_command() call to rebuild the object file in case some input sources have changed.
From the (slightly) outdated documentation on pyrocksdb, it says:
"If you do not want to call make install export the following enviroment variables:"
$ export CPLUS_INCLUDE_PATH=${CPLUS_INCLUDE_PATH}:`pwd`/include
$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:`pwd`
$ export LIBRARY_PATH=${LIBRARY_PATH}:`pwd`
But the installation instructions for RocksDB do not seem to mention any sort of install target!
Is there an accepted procedure for installing RocksDB from source?
My thoughts are to just copy the contents of the include directory from the rocksdb directory into somewhere like /usr/local/include and copy the librocksdb.so and librocksdb.a files into /usr/local/lib. Is this an acceptable method?
Note: The method of exporting environment variables was less preferable to me, as I built rocksdb in a directory inside my home folder--I am hoping for a cleaner solution (interpret that how you want).
RocksDB recently has make install. If you use the latest version, you should be able to do make install in RocksDB.
There is no install target in the current Makefile.
This breaks the long-established conventions for writing Makefiles (or pretty-much every other build system...); it should be considered a defect.
Without spending a lot of time analysing I can't be sure, but the install target should be something like:
prefix=/usr/local
bindir=$(prefix)/bin
# Normally you'd write a macro for this; 'lib' for 32-bit, 'lib64' for 64...
libdir=$(prefix)/lib64
includedir=$(prefix)/include
# Define this to be the directory(s) the headers are installed into.
# This should not include the 'include' element:
# include/rocksdb/stuff -> rocksdb/stuff
HEADER_DIRS=...
# Define this so all paths are relative to both the $CWD/include directory...
# so include/rocksdb/foo.h -> HEADER_FILES=rocksdb/foo.h
HEADER_FILES=...
.PHONY: install
install: $(TOOLS) $(LIBRARY) $(SHARED) $(MAKEFILES)
mkdir -p $(DESTDIR)$(bindir)
mkdir -p $(DESTDIR)$(libdir)
mkdir -p $(DESTDIR)$(includedir)
for tool in $(TOOLS); do \
install -m 755 $$tool $(DESTDIR)$(bindir); \
done
# No, libraries should NOT be executable on Linux.
install -m 644 $(LIBRARY) $(DESTDIR)$(libdir)
install -m 644 $(SHARED3) $(DESTDIR)$(libdir)
ln -s $(SHARED3) $(DESTDIR)$(libdir)/$(SHARED2)
ln -s $(SHARED2) $(DESTDIR)$(libdir)/$(SHARED1)
for header_dir in $(HEADER_DIRS); do \
mkdir -p $(DESTDIR)$(includedir)/$$header_dir; \
done
for header in $(HEADER_FILES); do \
install -m 644 include/$$header $(DESTDIR)$(includedir)/$$header; \
done
This will then allow you to install the files into /usr/local, by simply doing:
make install
However, the reason it's so heavily parameterised, is so you can change the destination folder, without having to modify the Makefile. For example, to change the destination to /usr, you simply do:
make prefix=/usr install
Alternatively, if you'd like to test the installation process, without messing with your filesystem, you could do:
make DESTDIR=/tmp/rocksdb_install_test prefix=/usr install
This would put the files into /tmp/rocksdb_install_test/usr which you can then check to see if they're where you want them to be... when you're happy, you can just do rm -Rf /tmp/rocksdb_install_test to cleanup.
The variables I've used are essential for packaging with RPM or DEB.
I an use ubuntu 16.04
DEBUG_LEVEL=0 make shared_lib install-shared
In this way, the installation is already generated in the production mode.
If you want to save time, you can specify the quantities of processors used in the process by passing -j[n], in my case, -j4
DEBUG_LEVEL=0 make -j4 shared_lib install-shared
In the case of ubuntu, this is sufficient, but in the case of ubuntu for docker, you should specify where the lib was installed.
export LD_LIBRARY_PATH=/usr/local/lib
Hope this helps.
Kemper
GNU make automatically removes intermediate files created by implicit rules, by calling rm filename at the end. This obviously doesn't work if one of the targets was actually a directory. Take the following example:
.PHONY: all
all: test.target
%.target: tempdir.%
touch $#
tempdir.%:
mkdir -p $#
make -n reveals the action plan:
mkdir -p tempdir.test
touch test.target
rm tempdir.test
Is it possible to get GNU make to correctly dispose of intermediate directories? Perhaps by changing rm to rm -rf?
There is no way to make this happen. Although GNU make prints the command "rm", really internally it's running the unlink(2) system call directly and not invoking a shell command. There is no way to configure or modify the command that GNU make runs (except by changing the source code of course).
However, I feel I should point out that it's just not going to work to use a directory as a normal prerequisite of a target. GNU make uses time-last-modified comparison to tell when targets are up to date or not, and the time-last-modified of a directory does not follow the standard rules. The TLM of a directory is updated every time a file (or subdirectory) in that directory is created, deleted, or renamed. This means you will created the directory, then have a bunch of files that depend on it: the first one is built and has timestamp N. The last one is built and has timestamp N+x. That also sets the directory's timestamp to N+x. Then the next time you run make, it will notice that the first one has an older timestamp (N) than one of its prerequisites (the directory, at N+x), and rebuild.
And this will happen forever, until it can build the remaining "out of date" prerequisites fast enough that their timestamp is not newer than the directory.
And, if you were to drop a temporary file or editor backup file or something in that directory, it would start all over again.
Just don't do it.
Some people use an explicit shell command to create directories. Some people create them as a side-effect of the target creation. Some people use order-only prerequisites to ensure they're created on time.