Copy files recursively with GNU Make - bash

I got a directory where store my project assets, say:
+-- assets
| +-- styles
| | +-- child
| | | +-- child.css
| | +-- main.css
| +-- font.ttf
| +-- image.png
As you can see, they are several file types (this is only a sample, may have many more types) and the tree depth may vary.
How can i copy all this files into a single (flat) location? Something like this:
+-- assets
| +-- child.css
| +-- font.ttf
| +-- image.png
| +-- main.css
I tried use pattern rules but i'm pretty new in GNU Make, so nothing seems to work.
I got the recursive list of source files using the Bash globstar functionality, and modify it by Make's string manipulation functions:
BR_ASSETS := /home/user/project/assets/
BR_APP_ASSETS := /home/user/project/build/assets/
SOURCES := $(shell ls $(BR_ASSETS)**/*.*)
TARGETS := $(addprefix $(BR_APP_ASSETS),$(notdir $(SOURCES)))
What next?
Many thanks in advance.

all:
.PHONY: all
BR_ASSETS := /home/user/project/assets/
BR_APP_ASSETS := /home/user/project/build/assets/
SOURCES := $(shell find $(BR_ASSETS) -type f)
$(info SOURCES=$(SOURCES))
TARGETS := $(addprefix $(BR_APP_ASSETS),$(notdir $(SOURCES)))
# Generate rules to map sources into targets
$(foreach s,$(SOURCES),$(foreach t,$(filter %$(notdir $s),$(TARGETS)),$(info New rule: $t: $s)$(eval $t: $s)))
# All targets have the same recipe.
$(TARGETS):; $(if $(wildcard $(#D)),,mkdir -p $(#D) &&) cp $^ $#
all: $(TARGETS)
clean:; rm -rf $(BR_APP_ASSETS)
.PHONY: clean
Environment before running this Makefile:
$ find /home/user/project
/home/user/project
/home/user/project/assets
/home/user/project/assets/font.ttf
/home/user/project/assets/img.png
/home/user/project/assets/styles
/home/user/project/assets/styles/child
/home/user/project/assets/styles/child/child.css
/home/user/project/assets/styles/main.css
/home/user/project/build
Run this Makefile:
$ make -f Makefile.sample
SOURCES=/home/user/project/assets/font.ttf /home/user/project/assets/img.png /home/user/project/assets/styles/child/child.css /home/user/project/asset
s/styles/main.css
New rule: /home/user/project/build/assets/font.ttf: /home/user/project/assets/font.ttf
New rule: /home/user/project/build/assets/img.png: /home/user/project/assets/img.png
New rule: /home/user/project/build/assets/child.css: /home/user/project/assets/styles/child/child.css
New rule: /home/user/project/build/assets/main.css: /home/user/project/assets/styles/main.css
mkdir -p /home/user/project/build/assets && cp /home/user/project/assets/font.ttf /home/user/project/build/assets/font.ttf
cp /home/user/project/assets/img.png /home/user/project/build/assets/img.png
cp /home/user/project/assets/styles/child/child.css /home/user/project/build/assets/child.css
cp /home/user/project/assets/styles/main.css /home/user/project/build/assets/main.css
Environment after running this Makefile:
$ find /home/user/project
/home/user/project
/home/user/project/assets
/home/user/project/assets/font.ttf
/home/user/project/assets/img.png
/home/user/project/assets/styles
/home/user/project/assets/styles/child
/home/user/project/assets/styles/child/child.css
/home/user/project/assets/styles/main.css
/home/user/project/build
/home/user/project/build/assets
/home/user/project/build/assets/child.css
/home/user/project/build/assets/font.ttf
/home/user/project/build/assets/img.png
/home/user/project/build/assets/main.css

Related

GNU Makefile uses the same source file for all object files

File structure:
cpp
| Makefile
|
| obj
|___include
| | a.cuh
| | b.cuh
| | c.cuh
|
|___src
| | a.cu
| | b.cu
| | c.cu
I don't have much experience with GNU make. The following was written based on different search results on Stackoverflow. The $# variable correctly gets the name of each object file from the list, however $< variable only gets the first item in the list of source file names (as per the manual, but that is what all the stack overflow answers I found are using 1, 2, 3).
NVCC=nvcc
LIB = lib.dll
SRC_DIR = src
INC_DIR = include
OBJ_DIR = obj
CU_FILES = $(wildcard $(SRC_DIR)/*.cu)
CUH_FILES = $(wildcard $(INC_DIR)/*.cuh)
CUO_FILES = $(addprefix $(OBJ_DIR)/,$(notdir $(CU_FILES:.cu=.obj)))
$(LIB): $(CUO_FILES)
$(NVCC) --shared $^ -o $#
$(CUO_FILES): $(CU_FILES) $(CUH_FILES)
$(NVCC) -dc $< -o $#
What you wrote is probably not what you want. Your rule:
$(CUO_FILES): $(CU_FILES) $(CUH_FILES)
$(NVCC) -dc $< -o $#
means that each object file depends on all source files ad all header files.
What you need here is a pattern rule saying that obj/foo.obj depends on src/foo.cu and include/foo.cuh:
obj/%.obj: src/%.cu include/%.cuh
$(NVCC) -dc $< -o $#
Note that you can simplify a bit the CUO_FILES assignment:
CUO_FILES = $(patsubst $(SRC_DIR)/%.cu,$(OBJ_DIR)/%.obj,$(CU_FILES))

compiling *.C and *.S sources from same directory 'make'

I have been working with a project that only compiles C sources but I've found I need some assembler too, I'm reticent to inline the asm code in a C file as GCC may not interpret it correctly.
My predecesor created the makefile for the project (apologies, it's fairly long):
# Compiler and options
CC := sparc-rtems-gcc
# The application software binary
TARGET := icu_asw
# Source and build directories
SRCDIR := src
BUILDDIR := obj
TARGETDIR := bin
HDSWROOT := ../../hdsw
BSWTOOLS := ../../bsw/sw/tools
SRCEXT := c
DEPEXT := d
OBJEXT := o
MRAMEXT := img.elf
# Flags, libraries and includes
CFLAGS := -std=gnu99 -Wall -Wextra -g
LIBDRV := $(HDSWROOT)/lib/libdrv.a
INCFLAGS := -I$(HDSWROOT)/include -I$(HDSWROOT)/osal/rtems
# Debug flags
DEBUGFLAGS = -DLOGERROR=1 -DLOGWARN=1 -DLOGDEBUG=1 -DLOGINFO=1 -DMAKECHECKS=1
NODEBUGFLAGS = -DLOGERROR=1 -DLOGWARN=0 -DLOGDEBUG=0 -DLOGINFO=0 -DMAKECHECKS=1
#-----------------------------------------------------------------------
# Build instructions
#-----------------------------------------------------------------------
SOURCES := $(shell find $(SRCDIR) -type f -name *.$(SRCEXT))
OBJECTS := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES:.$(SRCEXT)=.$(OBJEXT)))
# Default make
all: $(TARGET)
# Remake
remake: cleaner all
# Clean only objects
clean:
#$(RM) -rf $(BUILDDIR)
# Full clean (objects and binaries)
cleaner: clean
#$(RM) -rf $(TARGETDIR)
# Pull in dependency info for *existing* .o files
-include $(OBJECTS:.$(OBJEXT)=.$(DEPEXT))
# Link (uses an order-only prerequisite for the directories so that they
# don't affect the use of the $^)
$(TARGET): $(OBJECTS) | directories
$(CC) -o $(TARGETDIR)/$(TARGET) $^ $(LIBDRV)
# Make the build and target directories
directories:
#mkdir -p $(TARGETDIR)
#mkdir -p $(BUILDDIR)
# Compile
$(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
#mkdir -p $(dir $#)
$(CC) $(CFLAGS) $(INCFLAGS) $(NODEBUGFLAGS) -c -o $# $<
#$(CC) $(CFLAGS) $(INCDEP) -MM $(SRCDIR)/$*.$(SRCEXT) > $(BUILDDIR)/$*.$(DEPEXT)
#cp -f $(BUILDDIR)/$*.$(DEPEXT) $(BUILDDIR)/$*.$(DEPEXT).tmp
#sed -e 's|.*:|$(BUILDDIR)/$*.$(OBJEXT):|' < $(BUILDDIR)/$*.$(DEPEXT).tmp > $(BUILDDIR)/$*.$(DEPEXT)
#sed -e 's/.*://' -e 's/\\$$//' < $(BUILDDIR)/$*.$(DEPEXT).tmp | fmt -1 | sed -e 's/^ *//' -e 's/$$/:/' >> $(BUILDDIR)/$*.$(DEPEXT)
#rm -f $(BUILDDIR)/$*.$(DEPEXT).tmp
# Non-File Targets
.PHONY: all remake clean cleaner
I want to also bring in and compile two .S files, so, I edited the following line
SOURCES := $(shell find $(SRCDIR) -type f -name *.$(SRCEXT) -or -name *.$(ASMEXT))
To bring in the .S files, then I edited the OBJECTS to also include the ASM sources ("*.S")
OBJECTS := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES:.$(SRCEXT)=.$(OBJEXT)),$(SOURCES:.$(ASMEXT)=.$(OBJEXT)))
But when recompiling with 'make all' I'm getting:
$ make all
make: *** No rule to make target `obj/asi_access.S', needed by `icu_asw'. Stop.
I don't suppose someone could spot where I am going wrong? I think I have not correctly added to the OBJECTS line!
Thanks
The expression $(var:.ext1=.ext2) does not filter by .ext1, i.e.
$(SOURCES:.$(SRCEXT)=.$(OBJEXT)) $(SOURCES:.$(ASMEXT)=.$(OBJEXT))
gives for a test source list the following result
a.o b.o c.S a.c b.c c.o
I.e. you duplicated your files and you have source files in the OBJECTS definition.
The following would be a correct approach:
OBJECTS := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%, \
$(patsubst %.$(SRCEXT),%.$(OBJEXT), \
$(patsubst %.$(ASMEXT),%.$(OBJEXT),$(SOURCES)) \
) \
)
UPDATE: you should consider to use 2 separate object lists, so that you can apply different rules for them, e.g.
SOURCES_C := $(filter %.$(SRCEXT),$(SOURCES))
OBJECTS_C := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES_C:%.$(SRCEXT)=%.$(OBJEXT)))
SOURCES_ASM := $(filter %.$(ASMEXT),$(SOURCES))
OBJECTS_ASM := $(patsubst $(SRCDIR)/%,$(BUILDDIR)/%,$(SOURCES_ASM:%.$(ASMEXT)=%.$(OBJEXT)))
$(OBJECTS_C): $(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
.... C compiler recipe ....
$(OBJECTS_ASM): $(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(ASMEXT)
.... assembler recipe ....
$(TARGET): $(OBJECTS_C) $(OBJECTS_ASM) | directories

will $# work with vpath

SRC_DIRS += $(CMM_DIR)
SRC_DIRS += $(ABC_DIR)
VPATH = $(SRC_FILES)
tear_abc123: $(OBJ)/ur23.o
$(CC) $(CFLAGS) -c $(subst tear_,,$#.c) $#.c $#_compiletime.c -o $#
In this, $#_compiletime.c (say tear_abc123_compiletime.c) gets generated before $(CC). make throws an error that abc123.c is not found. where abc123.c is present in $(ABC_DIR). Why vpath is could not find the c file.
The main problem is that VPATH works only for Prerequisites. And it does not work from the 'command' part of the target.
To make VPATH work you should specify list of sources in Prerequisites and reference it in the command using $^.
Please see the following example (for file tree as below)
.
|-- abc
| |-- abc123.c
| |-- tear_abc123.c
| `-- tear_abc123_compiletime.c
`-- Makefile
Makefile contains
SRC_DIRS := abc
VPATH := $(SRC_DIRS)
TARGET := tear_abc123
SOURCE_FILES := $(TARGET).c $(TARGET)_compiletime.c $(subst tear_,,$(TARGET).c)
tear_abc123: $(SOURCE_FILES)
#echo "SOURCE_FILES := $(SOURCE_FILES)"
#echo "prerequisites = $^"
The output will be
$ make
SOURCE_FILES := tear_abc123.c tear_abc123_compiletime.c abc123.c
prerequisites = abc/tear_abc123.c abc/tear_abc123_compiletime.c abc/abc123.c
As you can see
the SOURCE_FILES really contains only source files names
the prerequisites is automatically substituted with correct path

Makefile: use multiple makefiles

I have a project that has many makefiles inside it. Because sometimes I need to compile only some parts of my project so I decided to have one makefile in each directory. But when I want to run the main makefile that includes all other ones to compile my project it gives me lots of errors.
This is my project tree:
root 'makefile'
-folder1 'foo1.c foo2.c makefile'
-folder2 'makefile'
--folder2-1 'foo3.c foo4.c makefile'
--folder2-2 'foo5.c makefile'
Each makefile in the very last directory has been included to it's upper makefile. But it doesn't run the main makefile. I'm really new to GNU make and I want to have something like this but I don't know how. I just wrote some make scripts but it doesn't work.
I just want to have something that I can compile the projects parts separately or compile the whole project using multiple makefiles.
any way, I show you something and hope can help you:
all:
#$(MAKE) -C subfolder
or if you want to use *.mk as your sub makefile, you can write:
all:
#$(MAKE) -C busfolder -f somename.mk
I show you my example, my DIR looks like:
TOPDIR-- Makefile
|
|-- debug
| |-- debug.c
| |-- debug.h
| |-- debug.mk
| |-- instrument.c
| `-- uart_print.c
|-- driver
| |-- driver.c
| |-- driver_ddi.c
| |-- driver_ddi.h
| |-- driver.h
| `-- driver.mk
|-- include
| `-- common.h
|-- Makefile
|-- mw
| |-- manager.c
| `-- mw.mk
|-- root
| |-- main.c
| `-- root.mk
and my TOP makefile looks like:
MAKE_DIR = $(PWD)
ROOT_DIR := $(MAKE_DIR)/root
DRV_DIR := $(MAKE_DIR)/driver
INCLUDE_DIR := $(MAKE_DIR)/include
DEBUG_DIR := $(MAKE_DIR)/debug
INC_SRCH_PATH :=
INC_SRCH_PATH += -I$(ROOT_DIR)
INC_SRCH_PATH += -I$(DRV_DIR)
INC_SRCH_PATH += -I$(INCLUDE_DIR)
INC_SRCH_PATH += -I$(DEBUG_DIR)
LIB_SRCH_PATH :=
LIB_SRCH_PATH += -L$(MAKE_DIR)/libs
COLOR_ON = color
COLOR_OFF =
CC = $(COLOR_ON)gcc
#CC = $(COLOR_OFF)gcc
LD = ld
LINT = splint
LIBS := -ldriver -ldebug -lmw -lm -lpthread
CFLAGS :=
CFLAGS += $(INC_SRCH_PATH) $(LIB_SRCH_PATH)
CFLAGS += -Wall -O -ggdb -Wstrict-prototypes -Wno-pointer-sign -finstrument-functions -fdump-rtl-expand
CFLAGS += -DDEBUG -D_REENTRANT
LDFLAGS :=
export MAKE_DIR CC LD CFLAGS LDFLAGS LIBS LINT INC_SRCH_PATH
all:
#$(MAKE) -C debug -f debug.mk
#$(MAKE) -C driver -f driver.mk
#$(MAKE) -C mw -f mw.mk
#$(MAKE) -C root -f root.mk
.PHONY: clean
clean:
#$(MAKE) -C debug -f debug.mk clean
#$(MAKE) -C driver -f driver.mk clean
#$(MAKE) -C mw -f mw.mk clean
#$(MAKE) -C root -f root.mk clean
it will call sub DIR *.mk during the compile. The sub DIR makefile, I just write a simple example for you reference:
LIB = $(MAKE_DIR)/libs/yourmodulename.a
SRCS = $(wildcard *.c)
OBJS = $(patsubst %.c, %.o, $(SRCS))
$(LIB): $(OBJS)
#mkdir -p ../libs
#$(AR) cr $# $^
#echo " Archive $(notdir $#)"
$(OBJS): $(SRCS)
#$(CC) $(CFLAGS) -c $^
#echo " CC $(OBJS)"
.PHONY: clean
clean:
#$(RM) -f $(LIB) $(OBJS)
#$(RM) -f *.expand
#echo " Remove Objects: $(OBJS)"
for the makefile, which generate the final target file(like a.out) is little bit different, because I use the sub makefile to generate LIB file, and I use a root.mk to generate final target:
PROG = ../prog/DEMO
SRCS = $(wildcard *.c)
OBJS = $(patsubst %.c, %.o, $(SRCS))
$(PROG): $(SRCS)
#mkdir -p ../prog
#$(CC) $^ $(CFLAGS) -Wl,-Map=$(PROG).map $(LIBS) -o $#
#echo " Generate Program $(notdir $(PROG)) from $^"
.PHONY: clean
clean:
#$(RM) -f $(OBJS) $(PROG)
#$(RM) -f *.expand
#$(RM) -rf ../prog ../libs
#echo " Remove Objects: $(OBJS)"
#echo " Remove Libraries: $(notdir $(PROG))"

recursive clean in a Makefile

I'm trying to create a Makefile which can build/clean recursively when it's called. I have the build working, but I'm getting errors with the clean command.
My directory structure is something like:
main
|
+-- Makefile
|
+-- common
| |
| +-- gcas_debug
| | |
| | +-- Makefile
| + -- gcas_nvdata
| |
| +-- Makefile
+-- gcinit
+-- Makefile
When I call make in the main directory it goes through and builds everything as I desire, but when I call make clean it does with:
mike#mike-VirtualBox:~/iCOM/framework$ make clean
for d in common/gcas_debug common/gcas_nvdata; \
do \
make --directory=$f clean; \
done
make: the `-C' option requires a non-empty string argument
Usage: make [options] [target] ...
Options:
-b, -m Ignored for compatibility.
...
I don't understand why the clean command isn't working... Any suggestions?
Full (main) Makefile:
lib_gcas_debug := common/gcas_debug
lib_gcas_nvdata := common/gcas_nvdata
libraries := $(lib_gcas_debug) $(lib_gcas_nvdata)
DESTDIR=$(PWD)/output
bindir=
gc_init := gcinit
EXE := $(gc_init)/$(gc_init)
.PHONY: all $(gc_init) $(libraries)
all: $(gc_init)
$(gc_init) $(libraries):
$(MAKE) --directory=$#
$(gc_init): $(libraries)
install: $(gc_init)
install -d -m 0755 $(DESTDIR)$(bindir)
install -m 0755 $(EXE) $(DESTDIR)$(bindir)
clean:
for d in $(libraries); \
do \
$(MAKE) --directory=$$f clean; \
done
EDIT: If I swap the for d with for $d I get instead:
mike#mike-VirtualBox:~/iCOM/framework$ make clean
for in common/gcas_debug common/gcas_nvdata; \
do \
make --directory= clean; \
done
/bin/sh: -c: line 0: syntax error near unexpected token `common/gcas_debug'
/bin/sh: -c: line 0: `for in common/gcas_debug common/gcas_nvdata; \'
make: *** [clean] Error 1
This worked for me:
SUBDIRS := foo bar baz
.PHONY: subdirs $(SUBDIRS) clean all
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(MAKECMDGOALS)
clean: $(SUBDIRS)
rm -rf dist
dist: $(SUBDIRS) dist/.build_marker
dist/.build_marker:
mkdir -p dist
for d in $(SUBDIRS) ; do cp $$d/dist/* dist ; done
touch dist/.build_marker
I only need to use the for loop in the dist target to copy files. This allows make -j build parallelism, not to mention simpler-to-read Makefiles.
Read the for loop again:
for d in [...]; do make --directory=$f
Didn't you mean $d, as in
for d in [...]; do make --directory=$d
So, the Makefile should look:
for d in common/gcas_debug common/gcas_nvdata; \
do \
make --directory=$d clean; \
done

Resources