How to optimize Makefile - makefile

I write a ugly copy/paste way created Makefile:
all: download install
install: \
${EXTERNAL_MODELS_LOCAL}/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 \
${EXTERNAL_MODELS_LOCAL}/resnet50_weights_tf_dim_ordering_tf_kernels.h5 \
${EXTERNAL_MODELS_LOCAL}/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 \
${EXTERNAL_MODELS_LOCAL}/squeezenet_weights_tf_dim_ordering_tf_kernels.h5:
ln -s ${EXTERNAL_MODELS_ROOT}/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 $#
${EXTERNAL_MODELS_LOCAL}/resnet50_weights_tf_dim_ordering_tf_kernels.h5:
ln -s ${EXTERNAL_MODELS_ROOT}/resnet50_weights_tf_dim_ordering_tf_kernels.h5 $#
${EXTERNAL_MODELS_LOCAL}/inception_v3_weights_tf_dim_ordering_tf_kernels.h5:
ln -s ${EXTERNAL_MODELS_ROOT}/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 $#
download: $(EXTERNAL_MODELS_ROOT)/ \
$(EXTERNAL_MODELS_ROOT)/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 \
$(EXTERNAL_MODELS_ROOT)/resnet50_weights_tf_dim_ordering_tf_kernels.h5 \
$(EXTERNAL_MODELS_ROOT)/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 \
$(EXTERNAL_MODELS_ROOT)/squeezenet_weights_tf_dim_ordering_tf_kernels.h5:
wget https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 \
-O $#
$(EXTERNAL_MODELS_ROOT)/resnet50_weights_tf_dim_ordering_tf_kernels.h5:
wget https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/resnet50_weights_tf_dim_ordering_tf_kernels.h5 \
-O $#
$(EXTERNAL_MODELS_ROOT)/inception_v3_weights_tf_dim_ordering_tf_kernels.h5:
wget https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 \
-O $#
Biggest part skipped ,but looks the same. Is it possible to optimize this boilerplate?

A mixture of make variables, make automatic variables (e.g. $<, $#), make functions (e.g. addsuffix, addprefix) and pattern rules, maybe:
RHOST := https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/
H5STEM := squeezenet resnet50 inception_v3
H5 := $(addsuffix _weights_tf_dim_ordering_tf_kernels.h5,$(H5STEM))
H5LOCAL := $(addprefix $(EXTERNAL_MODELS_LOCAL)/,$(H5))
H5ROOT := $(addprefix $(EXTERNAL_MODELS_ROOT)/,$(H5))
.PHONY: install download
install: $(H5LOCAL)
download: $(H5ROOT)
$(EXTERNAL_MODELS_LOCAL)/%.h5: $(EXTERNAL_MODELS_ROOT)/%.h5
ln -s $< $#
$(EXTERNAL_MODELS_ROOT)/%.h5:
wget $(RHOST)/$*.h5 -O $#
And of course, if there was a way to automatically discover the list of remote *.h5 files, it would be even better. But some more information is needed to imagine how to do it (ssh, curl, wget... ?). The make shell function would be the starting point, of course:
H5 := $(shell <the-command-that-lists-the-remote-h5-files>)

Related

Makefile does not invoke action after touching the target file

Here is my Makefile:
NAME := libftprintf.a
LIB := ar rcs
CC := gcc
CFLAGS := -Wall -Wextra -Werror
SRCS_DIR := ./sources/
HDRS_DIR := ./headers/
OBJS_DIR := ./objectives/
SRCS_FILES := ft_conv.c \
ft_eval_hex.c \
ft_eval_number.c \
ft_eval_string.c \
ft_parsers.c \
ft_strs_join.c \
ft_eval_char.c \
ft_eval_int.c \
ft_eval_percent.c \
ft_handler.c \
ft_printf.c
HDRS_FILES := ft_conv.h \
ft_eval_hex.h \
ft_eval_number.h \
ft_eval_string.h \
ft_parsers.h \
ft_strs_join.h \
ft_eval_char.h \
ft_eval_int.h \
ft_eval_percent.h \
ft_handler.h \
ft_printf.h
OBJS_FILES := $(SRCS_FILES:.c=.o)
SRCS := $(addprefix $(SRCS_DIR),$(SRCS_FILES))
HDRS := $(addprefix $(HDRS_DIR),$(HDRS_FILES))
OBJS := $(addprefix $(OBJS_DIR),$(OBJS_FILES))
LIBFT_DIR := ./libft/
LIBFT_NAME := libft.a
LIBFT := $(addprefix $(LIBFT_DIR),$(LIBFT_NAME))
RM := rm -rf
all: $(NAME)
bonus: all
bonus_one: all
bonus_two: all
$(NAME): $(OBJS)
$(LIB) $(NAME) $(OBJS) $(LIBFT)
$(OBJS_DIR)%.o: $(SRCS_DIR)%.c $(HDRS) Makefile | $(OBJS_DIR) subsystem
$(CC) $(CFLAGS) -c $< -o $# -I $(HDRS_DIR) -I $(LIBFT_DIR)
subsystem:
#$(MAKE) -C $(LIBFT_DIR)
$(OBJS_DIR):
mkdir $(OBJS_DIR)
clean:
#$(MAKE) -C $(LIBFT_DIR) clean
$(RM) $(OBJS_DIR)
fclean: clean
#$(MAKE) -C $(LIBFT_DIR) fclean
$(RM) $(NAME)
re: fclean all
.PHONY: all subsystem bonus bonus_one bonus_two clean fclean re
The problem is so: if I do make all, then touch libftprintf.a I would expect that make all will rebuild libftprintf.a since it was changed and target all depends on that file. However, make does nothing and I can't understand this behavior.
Also, there is one more minor issue: during make all I create a directory objectives where I store all .o files. Calling make fclean and make all entirely rebuilds the target, however calling make re results in an error:
rm -rf ./objectives/
rm -rf *libft objectives here*
rm -rf libft.a
rm -rf libftprintf.a
make: *** No rule to make target `objectives', needed by `objectives/ft_conv.o'. Stop.
If I call make re right after this error, the target builds as always. Also, if I change re target to this, I do not receive any errors:
re:
#$(MAKE) fclean
#$(MAKE) all
Could not find solutions to my problems anywhere on the Internet.
Touching libprintf.a won't cause anything to rebuild. Make rebuilds targets that are out of date. Out of date means that either the target doesn't exist, or some prerequisite of the target is newer than the target.
After you run make then libprintf.a is up to date, which means it's newer than all its prerequisites. Running touch libprintf.a just makes it even newer than its prerequisites than it was before, so make still considers it up to date.
If you want to rebuild libprintf.a you need to touch (or delete) one of its prerequisites, not the target itself.
The reason for your second issue seems to be related to the makefile in the libft subdirectory. It seems like you're using the --no-print-directories option here: you should avoid using that, at least as long as you're debugging, so that you can see where make is going and which makefile it's running.

Why is Makefile not including the header?

I'm trying to make a library for my project but I am very very new to Makefiles. I tried several configurations and adding -I but none worked.
I have the following three:
libft/
../includes/
....libft.h
../lst
....ft_lstnew.c
....ft_lstadd_front.c
....ft_lstadd_back.c
.... [...]
../src
....ft_isalpha.c
....ft_isalnum.c
.... [...]
And the following makefile:
NAME=libft.a
LIBSO=libft.so
CC=gcc
CFLAGS=-Wall -Wextra -Werror
SRC_DIR=src/
BONUS_DIR=lst/
OBJ_DIR=obj/
SRC_FILES= ft_bzero.c \
ft_isalmun.c \
ft_isalpha.c \
ft_isascii.c \
ft_isdigit.c \
ft_isprint.c \
ft_memchr.c \
ft_memcpy.c \
ft_memmove.c \
ft_memset.c \
ft_strchr.c \
ft_strlcat.c \
ft_strlcpy.c \
ft_strlen.c \
ft_strncmp.c \
ft_strrchr.c \
ft_tolower.c \
ft_toupper.c
BONUS_FILES=ft_lstadd_back.c \
ft_lstadd_front.c \
ft_lstdelone.c \
ft_lstclear.c \
ft_lstiter.c \
ft_lstlast.c \
ft_lstmap.c \
ft_lstnew.c \
ft_lstsize.c
SRC_PATH=$(addprefix $(SRC_DIR), $(SRC_FILES))
BONUS_PATH=$(addprefix $(BONUS_DIR), $(BONUS_FILES))
SRC_NAMES=$(SRC_FILES:.c=.o)
BONUS_NAMES=$(BONUS_FILES:.c=.o)
SRC_PATH_O=$(addprefix $(SRC_DIR), $(SRC_NAMES))
BONUS_PATH_O=$(addprefix $(BONUS_DIR), $(BONUS_NAMES))
HDR_NAME=libft.h
HDR_DIR=includes/
HDR= $(addprefix $(HDR_DIR),$(HDR_NAME))
all: $(NAME)
$(NAME): $(SRC_PATH_O)
ar rc $# $<
ranlib $#
$(OBJ_DIR):
mkdir $#
$(OBJ_DIR)%.o: $(SRC_DIR)%.c $(HDR_NAME)
$(CC) $(CFLAGS) -c $< -o $# -I $(HDR)
clean:
rm -rf $(OBJ_DIR)
fclean: clean
rm -f $(NAME)
re: fclean all
.PHONY: all clean fclean re
And I keep getting this each time I type make on the terminal:
gcc -Wall -Wextra -Werror -c -o src/ft_bzero.o src/ft_bzero.c
src/ft_bzero.c:1:10: fatal error: libft.h: No such file or directory
1 | #include "libft.h"
| ^~~~~~~~~
compilation terminated.
make: *** [<builtin>: src/ft_bzero.o] Error 1
Am I missing something? It's literally my first time.
Yes, you're missing some things.
First, look at the command line make shows:
gcc -Wall -Wextra -Werror -c -o src/ft_bzero.o src/ft_bzero.c
Note that the output here is not right for the recipe of the pattern rule you created: there's no -I option, and the object file is being written to src/ not obj/.
From this you should realize that your pattern rule is not being used at all, and instead make is using its built-in rule for building object files.
Why isn't your pattern rule being used? Let's look at it:
$(OBJ_DIR)%.o: $(SRC_DIR)%.c $(HDR_NAME)
what is this after variable expansion?
obj/%.o: src/%.c libft.h
This pattern (like all patterns) can only match if ALL the prerequisites either already exist or can be built. The src/%.c exists, after the pattern substitution. What about libft.h? No, that doesn't exist. What does exist is includes/libft.h but that's not the same thing.
So, this rule fails to match and make goes back to using its default rules.
If you want to say that every object file depends on that header, you have to use the correct path to the header file when you write the pattern.
Next, this is wrong:
$(CC) $(CFLAGS) -c $< -o $# -I $(HDR)
What is $(HDR)? It's the name of the file: include/libft.h. You don't include header file names with -I; you include directories that headers are looked for in. So you need $(HDR_DIR) here instead.

makefile C code compile and link in one step but want two separate steps

My currently working makefile uses gcc to compile and link in one step. It is 600 lines long so I have cut it down to just show you the 'compile and link' and hex stages (very cut down code here!)
$(PROGRAM_ELF): \
$(BSP_DIR)/install/lib/$(CONFIGURATION)/libmetal.a \
$(BSP_DIR)/install/lib/$(CONFIGURATION)/libmetal-gloss.a \
$(BSP_DIR)/metal.$(LINK_TARGET).lds
mkdir -p $(dir $#)
$(MAKE) -C $(SRC_DIR) $(basename $(notdir $#)) \
PORT_DIR=$(PORT_DIR) \
AR=$(RISCV_AR) \
CC=$(RISCV_GCC) \
CXX=$(RISCV_GXX) \
ASFLAGS="$(RISCV_ASFLAGS)" \
CCASFLAGS="$(RISCV_CCASFLAGS)" \
CFLAGS="$(RISCV_CFLAGS)" \
CXXFLAGS="$(RISCV_CXXFLAGS)" \
XCFLAGS="$(RISCV_XCFLAGS)" \
LDFLAGS="$(RISCV_LDFLAGS)" \
LDLIBS="$(RISCV_LDLIBS)" \
PROJ_SRC="$(PROJ_SRC)"
$(PROGRAM_HEX): \
$(PROGRAM_ELF)
$(RISCV_OBJCOPY) -O ihex $(PROGRAM_ELF) $#
mv $(PROGRAM_HEX) $(PROGRAM_TMP)
$(RISCV_OBJCOPY) -O verilog $(PROGRAM_ELF) $#
cp $(PROGRAM_HEX) $(PROGRAM_MEM)
mv $(PROGRAM_TMP) $(PROGRAM_HEX)
However, I need the 'compile and link stage' to be in 2 steps now as I'll be using a different compiler which has separate compile and link exes. How would I do this ? So the above would need to be split into 2. Examples online are a bit vague.
at its simplest, you use the gcc (or g++) -c option to compile the source without linking. This will generate an object file - which you can use in the linker state. Here is a simple example:
SOURCE = test.cpp
OBJECT = test.o
OUTPUT = run
all:
#g++ $(SOURCE) -c -o $(OBJECT) <------ Compile test.cpp, produces test.o
#g++ $(OBJECT) -o $(OUTPUT) <------ Links test.o into executable `run`
clean:
#$(RM) -rf $(OBJECT) $(OUTPUT)
That's it...
I did this which seemed to work:
#list of all files
PROJ_SRC = $(SRC_DIR)/plsi2c_riscv.c \
$(SRC_DIR)/SimSpi.c \
etc
#assume BUILD_FOLDER defined already
OBJ_FILES = $(addprefix $(BUILD_FOLDER)/,$(PROJ_SRC:.c=.o))
$(BUILD_FOLDER)/%.o: %.c
#Create the folder structure for the output file
#mkdir -p $(dir $#)
$(MY_CC) $(CCFLAGS) $< -o $#
$(BUILD_FOLDER)/example: $(OBJ_FILES)
mkdir -p $(dir $#)
#echo Linking $(notdir $#)
$(MY_LINK) $(LFLAGS) $^ $(LDLIBS) -o $#

Running a Makefile on Raspberry Pi3, installation problems

let me start off by saying this is an entirely new world to me so forgive any improper use of terminology.
The problem in a nutshell, I was given the file which is used to run a GUI for controlling a motor. The UI was created in Qt Creator. After numerous google searches and trial and error I learned how to run the Makefile on the Raspberry Pi3 device. This code is in working order and should not need any modifications to the code itself persay, it is just a matter of installing the right packages and typing the correct commands.
These are the steps I am getting to and the resulting output:
pi#raspberrypi:~/Documents/BasicInterface $ make
cp source/Collection.py build/Collection.py
cp source/main_release.py build/main_release.py
cp source/utilities.py build/utilities.py
cp source/Calibration.py build/Calibration.py
cp source/main_debug.py build/main_debug.py
cp source/archive.py build/archive.py
cp source/MotorControl.py build/MotorControl.py
cp source/MainCanvas.py build/MainCanvas.py
cp source/Report.py build/Report.py
cp source/MainWindow.py build/MainWindow.py
cp source/LoadCell.py build/LoadCell.py
cp source/Analysis.py build/Analysis.py
pyrcc5 resources/resources_rc.qrc -o build/resources_rc.py
pyuic5 resources/Ui_MainWindow.ui > build/Ui_MainWindow.py
pyuic5 resources/Ui_About.ui > build/Ui_About.py
python3 -m zipapp -p "/usr/bin/env python3" build -m main_release:main -o main_release.pyz
This is the Makefile file:
SRC = $(wildcard source/*.py)
OBJ = $(subst source,build,$(SRC))
OBJ += build/resources_rc.py
OBJ += build/Ui_MainWindow.py
OBJ += build/Ui_About.py
RCS = $(wildcard resources/embed/*)
RCS += $(wildcard resources/dynamic/*)
PYTHON = python3
FLAGS = -m zipapp -p "/usr/bin/env python3" build
all: release
release: main_release.pyz
debug: main_debug.pyz
install: main_release.pyz
.PRECIOUS: $(OBJ)
.PHONY: clean
.PHONY: install
%.pyz: $(OBJ)
$(PYTHON) $(FLAGS) -m $(basename $#):main -o $#
build/%.py: source/%.py
cp $< $#
build/%.py: resources/%.qrc $(RCS)
pyrcc5 $< -o $#
build/%.py: resources/%.ui
pyuic5 $< > $#
resources/dynamic/about.html: support/writeAbout.sh
./support/writeAbout.sh resources/dynamic/about.html
clean:
-rm $(OBJ)
-rm main_*.pyz
-rm -rf build/__pycache__
install:
mkdir -p $(HOME)/.irisEngineering
install main_release.pyz $(HOME)/.irisEngineering/AdhesionTester.pyz
install resources/embed/icon.svg $(HOME)/.irisEngineering/
envsubst <support/AdhesionTester.desktop >$(HOME)/Desktop/AdhesionTester.desktop
Basically it is not creating the directory (unless I'm looking in the wrong spot) and I do not see the application anywhere. I'm not sure if the information I have provided will suffice in helping me, but please let me know if any additional information is needed.

Makefile for compiling Markdown files to a single PDF

I am trying to use a Makefile for compiling a PDF when any of a number of Markdown files change:
# Compile report
source := draft
output := dist
sources := $(wildcard $(source)/*.md)
objects := $(patsubst %.md,%.pdf,$(subst$(source),$(output),$(sources)))
all: $(objects)
report-print.md: $(source)/%.md
cat draft/*.md | pandoc \
--variable geometry:a4paper \
--number-sections \
--toc \
--f markdown \
-s \
-o dist/report-print.pdf \
.PHONY : clean
clean:
rm -f $(output)/*.pdf
I get an error:
make: *** No rule to make target `dist/01-title.pdf', needed by `all'. Stop.
The file draft/01-title.md is one of of the source files.
You don't have a rule for creating one .pdf file from one .md file. Which is fine, because that's not what you want to do. You want to create a single pdf file from all the .md files (as I understand it). So, ditch all the objects stuff; you don't need to create all those individual pdf files.
There are a number of other minor problems: you aren't creating the same filename as your target (report-print.md vs. $(output)/report-print.pdf), you should use automatic variables, etc.)
Your makefile will simply be:
source := draft
output := dist
sources := $(wildcard $(source)/*.md)
all: $(output)/report-print.pdf
$(output)/report-print.pdf: $(sources)
cat $^ | pandoc \
--variable geometry:a4paper \
--number-sections \
--toc \
--f markdown \
-s \
-o $#
.PHONY : clean
clean:
rm -f $(output)/*.pdf

Resources