I'm trying to package a driver. However, when I compile the driver I don't get the output file CorXtiumCLMX4k.ko as expected, while the Makefile does contain the lines:
obj-m := CorXtiumCLMX4k.o
CorXtiumCLMX4k-objs := $(GDRV_OBJS) $(TARGETLIB)
(Btw any idea what this -objs is for? In the kernel documentation they mention the prefix -y… but I also tried it without success)
The compilation does not give any error at that step… just no .ko file is produced:
salsera_xtium> make: Entering directory '/build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux'
salsera_xtium> make -C /nix/store/n862ppa5sy8bmnhs801cskcisd6z7fd0-linux-5.4.225-dev/lib/modules/5.4.225/build M=/build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux IROOT=/build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux/../../../../.. modules
salsera_xtium> make[1]: Entering directory '/nix/store/n862ppa5sy8bmnhs801cskcisd6z7fd0-linux-5.4.225-dev/lib/modules/5.4.225/build'
salsera_xtium> CC [M] /build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux/GDrv_Linux.o
salsera_xtium> CC [M] /build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux/GDrv_LinuxUtil.o
salsera_xtium> CC [M] /build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux/GDrv_Util.o
salsera_xtium> CC [M] /build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux/memmanag.o
salsera_xtium> make[1]: Leaving directory '/nix/store/n862ppa5sy8bmnhs801cskcisd6z7fd0-linux-5.4.225-dev/lib/modules/5.4.225/build'
salsera_xtium> make: Leaving directory '/build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux'
salsera_xtium> cp: cannot stat '/build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/src/xtium/xtium_cl_mx4/driver/kernel/linux/CorXtiumCLMX4k.ko': No such file or directory
(note that make takes /nix/…/lib instead of /lib because I'm using Nix so I patched the path accordingly… also I do compile before other libraries that are not shown in this log, see below)
Any idea what could go wrong? I tried to compile with kernel 5.4 (supposed to be supported according to the doc…) and a more recent kernel, none of them work.
Debug informations
If it can be of any help, here is the list of files in the current folder before the compilation starts:
copysources corenv.h CorXtiumCLMX4k26_x86_64_a gdrv.h GDrv_kernel.h GDrv_KernelThread.h GDrv_Linux.c GDrv_LinuxUtil.c GDrv_LinuxUtil.h GDrv_OS.h GDrv_Util.c Makefile sdrv.h sources
At the end, I have:
copysources cormgrlibk.c gdrv.h GDrv_KernelThread.h GDrv_LinuxUtil.c GDrv_OS.h Makefile Module.symvers serialk.c
corenv.h CorXtiumCLMX4k26_x86_64_a GDrv_kernel.h GDrv_Linux.c GDrv_LinuxUtil.h GDrv_Util.c memmanag.c sdrv.h sources
The file CorXtiumCLMX4k26_x86_64_a seems to be some sort of static binary blob:
$ file CorXtiumCLMX4k26_x86_64_a
CorXtiumCLMX4k26_x86_64_a: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), with debug_info, not stripped
$ ldd CorXtiumCLMX4k26_x86_64_a
ldd: warning: you do not have execution permission for `./CorXtiumCLMX4k26_x86_64_a'
not a dynamic executable
(note that I also tried to rename it as CorXtiumCLMX4k.o_shipped with 'TARGETLIB = CorXtiumCLMX4k.o' without success)
Note that latter in the build process (yeah, it continues even after errors), I also get an error:
salsera_xtium> /nix/store/039g378vc3pc3dvi9dzdlrd0i4q93qwf-binutils-2.39/bin/ld: warning: /build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/Xtium-CL_MX4/lib/libXtiumCLMX4L_1_0_a has a section extending past end of file
salsera_xtium> /nix/store/039g378vc3pc3dvi9dzdlrd0i4q93qwf-binutils-2.39/bin/ld: error: /build/va8hwnknyg6cgzqydl3gm64j8z0vyq2i-source/DALSA/Xtium-CL_MX4/lib/libXtiumCLMX4L_1_0_a: ELF section name out of range
but I'm not sure if this error is because I'm missing some compilation flags like pic, if the file is just bad, or if it's just that I first need to compile the .ko file properly…
the Makefile looks like:
#-----------------------------------------------------------------------------
# MAKEFILE
# Copyright(C) Teledyne DALSA Inc 2015
# All rights reserved.
#
# Description:
# Makefile for distributed version of the CorXtiumCLMX4 kernel module for Linux
#
#-----------------------------------------------------------------------------
#
# Determine the kernel version being compiled for.
CORKERNEL_MAJOR:=$(shell uname -r | cut -d '.' -f 1)
CORKERNEL_MINOR:=$(shell uname -r | cut -d '.' -f 2)
#
# Kernel version-specific dependencies
# (mainly needed for 2.4 kernel support).
#
ifeq ($(CORKERNEL_MAJOR), 2)
ifeq ($(CORKERNEL_MINOR), 6)
# 2.6 kernel
TARGET_OPTIONS = -DHOSTPC -DPOSIX_HOSTPC -DNEW_CXM_INTERFACE -DEXPORT_SYMTAB
#TARGET_OPTIONS = -DHOSTPC -DPOSIX_HOSTPC -DNEW_CXM_INTERFACE -DEXPORT_SYMTAB -DDEBUG_PRINT
SAPSRCROOT := $(IROOT)/sapera
else
# Not 2.6 kernel - not supported (PCIe hardware).
$(error $(shell echo "Kernel version must be 2.6 or higher to support PCIe hardware"))
endif
else ifeq ($(CORKERNEL_MAJOR), 3)
# 2.6 (or higher kernel)
TARGET_OPTIONS = -DHOSTPC -DPOSIX_HOSTPC -DNEW_CXM_INTERFACE -DEXPORT_SYMTAB
#TARGET_OPTIONS = -DHOSTPC -DPOSIX_HOSTPC -DNEW_CXM_INTERFACE -DEXPORT_SYMTAB -DDEBUG_PRINT
SAPSRCROOT := $(IROOT)/sapera
endif
#
# API Version control information to be embedded.
#
ifndef CORARCH
CORARCH := $(shell uname -m | sed -e s/i.86/i386/ -e s/x86_64/x86_64/ )
endif
TARGETNAME = CorXtiumCLMX4k
TARGET = $(TARGETNAME).ko
ifeq ($(CORARCH), x86_64)
TARGETLIB = $(TARGETNAME)26_x86_64_a
CORARCH_OPTIONS= -Dx86_64
else
TARGETLIB = $(TARGETNAME)26_i386_a
CORARCH_OPTIONS= -D__i386__
endif
#
# Include Path for the kernel module.
#
I_CORSERIALCOMMON= $(SAPSRCROOT)/driver/corserial/linux/common
CORSERIALDEV= $(SAPSRCROOT)/driver/corserial/linux/kernel
CORMGRK= $(SAPSRCROOT)/driver/cormanager/kernel
XCELERA_HS_PX8_INC= $(IROOT)/xtium/xtium_cl_mx4/include
CURDIR = $(IROOT)/xtium/xtium_cl_mx4/driver/kernel/linux
INC_COMMON_PATH = -I$(CURDIR) -I$(KERNEL_INCLUDEDIR) -I$(CURDIR) -I$(SAPSRCROOT)/include \
-I$(XCELERA_HS_PX8_INC) -I$(CORMGRK) -I$(I_CORSERIALCOMMON) -I$(CORSERIALDEV)
DISTINSTALLDIR=$(IROOT)/../Xtium-CL_MX4/bin
RUNNINGKERNEL=$(shell uname -r)
INSTALLDIR= /lib/modules/$(RUNNINGKERNEL)/misc
#
# Source Path for the kernel module.
#
#
# Object definitions: Specific files for this library.
# GDrv objects.
GDRV_OBJS = GDrv_Linux.o \
GDrv_LinuxUtil.o \
GDrv_Util.o \
memmanag.o \
cormgrlibk.o \
serialk.o
#
# 2.6 kernel build section
#
INC_PATH = -I$(CURDIR) $(INC_COMMON_PATH)
CFLAGS_MODULE += $(INC_PATH) $(TARGET_OPTIONS) $(CORKERNEL_CONFIG_OPTIONS) \
$(CORARCH_OPTIONS)
EXTRA_CFLAGS += -Wno-parentheses -Wno-missing-braces -Wno-unknown-pragmas \
-Wno-cast-qual -Wno-unused-function -Wno-unused-label
#
# As of kernel 2.6.28 (and later) - versioned module symbol dependencies need to be specified
# explicitly - using absolute paths - to the Module.symvers files for the modules whose
# symbols we depend on.
# (Actually broken in an early 2.6.28 but later fixed).
#
# Prior to 2.6.28 - the kernel build system and module loader would resolve all symbols automatically
#
CORKVER := $(shell uname -r | cut -d '.' -f 3)
CORKTEMP := 28
COMPAREKVER := $(shell echo ${CORKVER}\>=${CORKTEMP} | bc )
ifeq ($(COMPAREKVER),1)
KBUILD_EXTRA_SYMBOLS= $(CORMGRK)/Module.symvers $(CORSERIALDEV)/Module.symvers
else
$(shell cat $(CORMGRK)/Module.symvers > $(CURDIR)/Module.symvers)
$(shell cat $(CORSERIALDEV)/Module.symvers >> $(CURDIR)/Module.symvers)
endif
#
# 2.6 kernel build section
#
ifneq ($(KERNELRELEASE),)
obj-m := CorXtiumCLMX4k.o
CorXtiumCLMX4k-objs := $(GDRV_OBJS) $(TARGETLIB)
#
# Perform any specific kernel configuration difference checks.
#
CORKERNEL_INCLUDEDIR = /lib/modules/$(shell uname -r)/build/include
ifeq ($(shell $(SAPSRCROOT)/corcfgtest.sh remap_pfn_range $(CORKERNEL_INCLUDEDIR)),1)
CORKERNEL_CONFIG_OPTIONS += -DUSE_REMAP_PFN_RANGE
endif
else
KERNELDIR ?= /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
RES := $(shell ./copysources)
default:
$(MAKE) -C $(KERNELDIR) M=$(PWD) IROOT=$(PWD)/../../../../.. modules
endif
all: $(TARGET)
clean:
-$(RM) $(OBJS) $(GDRV_OBJS) $(TARGET)
distclean:
-$(RM) $(OBJS) $(GDRV_OBJS)
install: all
cp $(TARGET) $(INSTALLDIR)
distinstall: all
cp $(TARGET) $(DISTINSTALLDIR)
(note that I also tried to turn else ifeq ($(CORKERNEL_MAJOR), 3) in else since kernel 5 is bigger than kernel 3… still no success).
If you want the full compilation log (including the sub libraries), see here.
I'm trying to build Git on OS X. Git depends on libidn2. libidn2 2.0.0 fails:
make all-am
make[3]: Entering directory '/Users/jwalton/Build-Scripts/libidn2-2.0.0/lib'
CC idna.lo
CC lookup.lo
CC decode.lo
CC register.lo
CC bidi.lo
CC version.lo
CC error.lo
CC punycode.lo
CC free.lo
CC data.lo
CC tr46map.lo
CC tables.lo
CC context.lo
CCLD libidn2.la
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: no name list
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: no name list
copying selected object files to avoid basename conflicts...
ar: temporary file: No such file or directory
Makefile:1027: recipe for target 'libidn2.la' failed
make[3]: *** [libidn2.la] Error 1
make[3]: Leaving directory '/Users/jwalton/Build-Scripts/libidn2-2.0.0/lib'
Makefile:955: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/Users/jwalton/Build-Scripts/libidn2-2.0.0/lib'
Makefile:1065: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/Users/jwalton/Build-Scripts/libidn2-2.0.0'
Makefile:974: recipe for target 'all' failed
make: *** [all] Error 2
My recipe to build libidn2 finds all Makefiles and it fixes AR and ARFLAGS:
if [[ "$IS_DARWIN" -ne "0" ]]; then
for mfile in $(find "$PWD" -name 'Makefile'); do
echo "Fixing Makefile $mfile"
sed -i "" 's|AR = ar|AR = /usr/bin/libtool|g' "$mfile"
sed -i "" 's|ARFLAGS = cr|ARFLAGS = -static -o|g' "$mfile"
done
fi
I've confirmed it fixes all the Makefiles:
$ find $PWD -name Makefile -exec grep 'AR =' {} \; | egrep -iv 'amtar|char'
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
AR = /usr/bin/libtool
ac_ct_AR = /usr/bin/libtool
I also tried AR="/usr/bin/libtool" ARFLAGS="-static -o" ./configure and make AR="/usr/bin/libtool" ARFLAGS="-static -o", but the choices are not honored. It produces additional failed builds.
Makefile debugging support is fairly crappy, so I'm not sure how to proceed. make -d fails even the simplest tasks, like providing a filename and line number. remake -d produces the same unhelpful results. The large project written by someone else aggravates the situation.
$ make -d
...
Putting child 0x7f9bf0c45cf0 (libidn2.la) PID 75531 on the chain.
Live child 0x7f9bf0c45cf0 (libidn2.la) PID 75531
CCLD libidn2.la
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: no name list
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: no name list
copying selected object files to avoid basename conflicts...
ar: temporary file: No such file or directory
Reaping losing child 0x7f9bf0c45cf0 PID 75531
Makefile:1027: recipe for target 'libidn2.la' failed
Here's line 1027, which is not the call to ar:
libidn2.la: $(libidn2_la_OBJECTS) $(libidn2_la_DEPENDENCIES) $(EXTRA_libidn2_la\
_DEPENDENCIES)
$(AM_V_CCLD)$(libidn2_la_LINK) -rpath $(libdir) $(libidn2_la_OBJECTS) $\
(libidn2_la_LIBADD) $(LIBS)
How can I locate where ar is being called in the collection of Makefiles?
To a simple understanding of a failing recipe you can:
echo the rule (i.e. echo $(AM_V_CCLD)$(libidn2_la_LINK) ...)
add a second command which fails to stop the make
If you need additional help please reports here the make output after these changes.
In your case the use of makefile option --debug=m can help
i have been trying to set up Caffe on my macbook for three days now... maybe you can help me?
i followed several tutorials including these:
the official berkeley vision caffe installation guide
https://gist.github.com/kylemcdonald/0698c7749e483cd43a0e
https://eddiesmo.wordpress.com/2016/12/20/how-to-set-up-caffe-environment-and-pycaffe-on-os-x-10-12-sierra/
but whenever i try to run
make all
i get this error:
LD -o .build_release/lib/libcaffe.so.1.0.0
clang: warning: argument unused during compilation: '-pthread'
Undefined symbols for architecture x86_64:
"cv::String::deallocate()", referenced from:
caffe::WindowDataLayer<float>::load_batch(caffe::Batch<float>*) in window_data_layer.o
..............
..............
..............
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [.build_release/lib/libcaffe.so.1.0.0] Error 1
here's what i did so far:
i was originally on mac os 10.11 el capitan with an outdated xcode (7.3). in order to update xcode i had to upgrade to Sierra (mac os 10.12)
i have xcode 8 now, but downgraded the xcode command line tools to 7.3.2. (i can't remember why but it was recommended for Caffe or maybe CUDA)
i don't have an NVIDIA graphics card but still installed CUDA (from what i read online it's a prerequisite for caffe)... i tried cuda 7 (as recommended on the berkeley vision site) and currently cuda 8
right now i'm using CUDA-8
i used homebrew to install opencv as recommended online (opencv 2.4.13.2)
i use openBLAS
i use protobuf 3.3.0
i added opencv_highgui opencv_imgproc opencv_core opencv_imgcodecs to LIBRARIES in Makefile
i realize that the linker errors have to do with libc++ vs libstdc++ ! i manually checked all kinds of dependencies with otool -L to see if any still used libstdc++ but it seems they should all be built with libc++.
any hints or advice?
pls help!
thank you!
Makefile
PROJECT := caffe
CONFIG_FILE := Makefile.config
# Explicitly check for the config file, otherwise make -k will proceed anyway.
ifeq ($(wildcard $(CONFIG_FILE)),)
$(error $(CONFIG_FILE) not found. See $(CONFIG_FILE).example.)
endif
include $(CONFIG_FILE)
BUILD_DIR_LINK := $(BUILD_DIR)
ifeq ($(RELEASE_BUILD_DIR),)
RELEASE_BUILD_DIR := .$(BUILD_DIR)_release
endif
ifeq ($(DEBUG_BUILD_DIR),)
DEBUG_BUILD_DIR := .$(BUILD_DIR)_debug
endif
DEBUG ?= 0
ifeq ($(DEBUG), 1)
BUILD_DIR := $(DEBUG_BUILD_DIR)
OTHER_BUILD_DIR := $(RELEASE_BUILD_DIR)
else
BUILD_DIR := $(RELEASE_BUILD_DIR)
OTHER_BUILD_DIR := $(DEBUG_BUILD_DIR)
endif
# All of the directories containing code.
SRC_DIRS := $(shell find * -type d -exec bash -c "find {} -maxdepth 1 \
\( -name '*.cpp' -o -name '*.proto' \) | grep -q ." \; -print)
# The target shared library name
LIBRARY_NAME := $(PROJECT)
LIB_BUILD_DIR := $(BUILD_DIR)/lib
STATIC_NAME := $(LIB_BUILD_DIR)/lib$(LIBRARY_NAME).a
DYNAMIC_VERSION_MAJOR := 1
DYNAMIC_VERSION_MINOR := 0
DYNAMIC_VERSION_REVISION := 0
DYNAMIC_NAME_SHORT := lib$(LIBRARY_NAME).so
#DYNAMIC_SONAME_SHORT := $(DYNAMIC_NAME_SHORT).$(DYNAMIC_VERSION_MAJOR)
DYNAMIC_VERSIONED_NAME_SHORT := $(DYNAMIC_NAME_SHORT).$(DYNAMIC_VERSION_MAJOR).$(DYNAMIC_VERSION_MINOR).$(DYNAMIC_VERSION_REVISION)
DYNAMIC_NAME := $(LIB_BUILD_DIR)/$(DYNAMIC_VERSIONED_NAME_SHORT)
COMMON_FLAGS += -DCAFFE_VERSION=$(DYNAMIC_VERSION_MAJOR).$(DYNAMIC_VERSION_MINOR).$(DYNAMIC_VERSION_REVISION)
##############################
# Get all source files
##############################
# CXX_SRCS are the source files excluding the test ones.
CXX_SRCS := $(shell find src/$(PROJECT) ! -name "test_*.cpp" -name "*.cpp")
# CU_SRCS are the cuda source files
CU_SRCS := $(shell find src/$(PROJECT) ! -name "test_*.cu" -name "*.cu")
# TEST_SRCS are the test source files
TEST_MAIN_SRC := src/$(PROJECT)/test/test_caffe_main.cpp
TEST_SRCS := $(shell find src/$(PROJECT) -name "test_*.cpp")
TEST_SRCS := $(filter-out $(TEST_MAIN_SRC), $(TEST_SRCS))
TEST_CU_SRCS := $(shell find src/$(PROJECT) -name "test_*.cu")
GTEST_SRC := src/gtest/gtest-all.cpp
# TOOL_SRCS are the source files for the tool binaries
TOOL_SRCS := $(shell find tools -name "*.cpp")
# EXAMPLE_SRCS are the source files for the example binaries
EXAMPLE_SRCS := $(shell find examples -name "*.cpp")
# BUILD_INCLUDE_DIR contains any generated header files we want to include.
BUILD_INCLUDE_DIR := $(BUILD_DIR)/src
# PROTO_SRCS are the protocol buffer definitions
PROTO_SRC_DIR := src/$(PROJECT)/proto
PROTO_SRCS := $(wildcard $(PROTO_SRC_DIR)/*.proto)
# PROTO_BUILD_DIR will contain the .cc and obj files generated from
# PROTO_SRCS; PROTO_BUILD_INCLUDE_DIR will contain the .h header files
PROTO_BUILD_DIR := $(BUILD_DIR)/$(PROTO_SRC_DIR)
PROTO_BUILD_INCLUDE_DIR := $(BUILD_INCLUDE_DIR)/$(PROJECT)/proto
# NONGEN_CXX_SRCS includes all source/header files except those generated
# automatically (e.g., by proto).
NONGEN_CXX_SRCS := $(shell find \
src/$(PROJECT) \
include/$(PROJECT) \
python/$(PROJECT) \
matlab/+$(PROJECT)/private \
examples \
tools \
-name "*.cpp" -or -name "*.hpp" -or -name "*.cu" -or -name "*.cuh")
LINT_SCRIPT := scripts/cpp_lint.py
LINT_OUTPUT_DIR := $(BUILD_DIR)/.lint
LINT_EXT := lint.txt
LINT_OUTPUTS := $(addsuffix .$(LINT_EXT), $(addprefix $(LINT_OUTPUT_DIR)/, $(NONGEN_CXX_SRCS)))
EMPTY_LINT_REPORT := $(BUILD_DIR)/.$(LINT_EXT)
NONEMPTY_LINT_REPORT := $(BUILD_DIR)/$(LINT_EXT)
# PY$(PROJECT)_SRC is the python wrapper for $(PROJECT)
PY$(PROJECT)_SRC := python/$(PROJECT)/_$(PROJECT).cpp
PY$(PROJECT)_SO := python/$(PROJECT)/_$(PROJECT).so
PY$(PROJECT)_HXX := include/$(PROJECT)/layers/python_layer.hpp
# MAT$(PROJECT)_SRC is the mex entrance point of matlab package for $(PROJECT)
MAT$(PROJECT)_SRC := matlab/+$(PROJECT)/private/$(PROJECT)_.cpp
ifneq ($(MATLAB_DIR),)
MAT_SO_EXT := $(shell $(MATLAB_DIR)/bin/mexext)
endif
MAT$(PROJECT)_SO := matlab/+$(PROJECT)/private/$(PROJECT)_.$(MAT_SO_EXT)
##############################
# Derive generated files
##############################
# The generated files for protocol buffers
PROTO_GEN_HEADER_SRCS := $(addprefix $(PROTO_BUILD_DIR)/, \
$(notdir ${PROTO_SRCS:.proto=.pb.h}))
PROTO_GEN_HEADER := $(addprefix $(PROTO_BUILD_INCLUDE_DIR)/, \
$(notdir ${PROTO_SRCS:.proto=.pb.h}))
PROTO_GEN_CC := $(addprefix $(BUILD_DIR)/, ${PROTO_SRCS:.proto=.pb.cc})
PY_PROTO_BUILD_DIR := python/$(PROJECT)/proto
PY_PROTO_INIT := python/$(PROJECT)/proto/__init__.py
PROTO_GEN_PY := $(foreach file,${PROTO_SRCS:.proto=_pb2.py}, \
$(PY_PROTO_BUILD_DIR)/$(notdir $(file)))
# The objects corresponding to the source files
# These objects will be linked into the final shared library, so we
# exclude the tool, example, and test objects.
CXX_OBJS := $(addprefix $(BUILD_DIR)/, ${CXX_SRCS:.cpp=.o})
CU_OBJS := $(addprefix $(BUILD_DIR)/cuda/, ${CU_SRCS:.cu=.o})
PROTO_OBJS := ${PROTO_GEN_CC:.cc=.o}
OBJS := $(PROTO_OBJS) $(CXX_OBJS) $(CU_OBJS)
# tool, example, and test objects
TOOL_OBJS := $(addprefix $(BUILD_DIR)/, ${TOOL_SRCS:.cpp=.o})
TOOL_BUILD_DIR := $(BUILD_DIR)/tools
TEST_CXX_BUILD_DIR := $(BUILD_DIR)/src/$(PROJECT)/test
TEST_CU_BUILD_DIR := $(BUILD_DIR)/cuda/src/$(PROJECT)/test
TEST_CXX_OBJS := $(addprefix $(BUILD_DIR)/, ${TEST_SRCS:.cpp=.o})
TEST_CU_OBJS := $(addprefix $(BUILD_DIR)/cuda/, ${TEST_CU_SRCS:.cu=.o})
TEST_OBJS := $(TEST_CXX_OBJS) $(TEST_CU_OBJS)
GTEST_OBJ := $(addprefix $(BUILD_DIR)/, ${GTEST_SRC:.cpp=.o})
EXAMPLE_OBJS := $(addprefix $(BUILD_DIR)/, ${EXAMPLE_SRCS:.cpp=.o})
# Output files for automatic dependency generation
DEPS := ${CXX_OBJS:.o=.d} ${CU_OBJS:.o=.d} ${TEST_CXX_OBJS:.o=.d} \
${TEST_CU_OBJS:.o=.d} $(BUILD_DIR)/${MAT$(PROJECT)_SO:.$(MAT_SO_EXT)=.d}
# tool, example, and test bins
TOOL_BINS := ${TOOL_OBJS:.o=.bin}
EXAMPLE_BINS := ${EXAMPLE_OBJS:.o=.bin}
# symlinks to tool bins without the ".bin" extension
TOOL_BIN_LINKS := ${TOOL_BINS:.bin=}
# Put the test binaries in build/test for convenience.
TEST_BIN_DIR := $(BUILD_DIR)/test
TEST_CU_BINS := $(addsuffix .testbin,$(addprefix $(TEST_BIN_DIR)/, \
$(foreach obj,$(TEST_CU_OBJS),$(basename $(notdir $(obj))))))
TEST_CXX_BINS := $(addsuffix .testbin,$(addprefix $(TEST_BIN_DIR)/, \
$(foreach obj,$(TEST_CXX_OBJS),$(basename $(notdir $(obj))))))
TEST_BINS := $(TEST_CXX_BINS) $(TEST_CU_BINS)
# TEST_ALL_BIN is the test binary that links caffe dynamically.
TEST_ALL_BIN := $(TEST_BIN_DIR)/test_all.testbin
##############################
# Derive compiler warning dump locations
##############################
WARNS_EXT := warnings.txt
CXX_WARNS := $(addprefix $(BUILD_DIR)/, ${CXX_SRCS:.cpp=.o.$(WARNS_EXT)})
CU_WARNS := $(addprefix $(BUILD_DIR)/cuda/, ${CU_SRCS:.cu=.o.$(WARNS_EXT)})
TOOL_WARNS := $(addprefix $(BUILD_DIR)/, ${TOOL_SRCS:.cpp=.o.$(WARNS_EXT)})
EXAMPLE_WARNS := $(addprefix $(BUILD_DIR)/, ${EXAMPLE_SRCS:.cpp=.o.$(WARNS_EXT)})
TEST_WARNS := $(addprefix $(BUILD_DIR)/, ${TEST_SRCS:.cpp=.o.$(WARNS_EXT)})
TEST_CU_WARNS := $(addprefix $(BUILD_DIR)/cuda/, ${TEST_CU_SRCS:.cu=.o.$(WARNS_EXT)})
ALL_CXX_WARNS := $(CXX_WARNS) $(TOOL_WARNS) $(EXAMPLE_WARNS) $(TEST_WARNS)
ALL_CU_WARNS := $(CU_WARNS) $(TEST_CU_WARNS)
ALL_WARNS := $(ALL_CXX_WARNS) $(ALL_CU_WARNS)
EMPTY_WARN_REPORT := $(BUILD_DIR)/.$(WARNS_EXT)
NONEMPTY_WARN_REPORT := $(BUILD_DIR)/$(WARNS_EXT)
##############################
# Derive include and lib directories
##############################
CUDA_INCLUDE_DIR := $(CUDA_DIR)/include
CUDA_LIB_DIR :=
# add <cuda>/lib64 only if it exists
ifneq ("$(wildcard $(CUDA_DIR)/lib64)","")
CUDA_LIB_DIR += $(CUDA_DIR)/lib64
endif
CUDA_LIB_DIR += $(CUDA_DIR)/lib
INCLUDE_DIRS += $(BUILD_INCLUDE_DIR) ./src ./include
ifneq ($(CPU_ONLY), 1)
INCLUDE_DIRS += $(CUDA_INCLUDE_DIR)
LIBRARY_DIRS += $(CUDA_LIB_DIR)
LIBRARIES := cudart cublas curand
endif
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5 opencv_highgui opencv_imgproc opencv_core pthread
# handle IO dependencies
USE_LEVELDB ?= 1
USE_LMDB ?= 1
USE_OPENCV ?= 1
ifeq ($(USE_LEVELDB), 1)
LIBRARIES += leveldb snappy
endif
ifeq ($(USE_LMDB), 1)
LIBRARIES += lmdb
endif
ifeq ($(USE_OPENCV), 1)
LIBRARIES += opencv_core opencv_highgui opencv_imgproc
ifeq ($(OPENCV_VERSION), 3)
LIBRARIES += opencv_imgcodecs
endif
endif
PYTHON_LIBRARIES ?= boost_python python2.7
WARNINGS := -Wall -Wno-sign-compare
##############################
# Set build directories
##############################
DISTRIBUTE_DIR ?= distribute
DISTRIBUTE_SUBDIRS := $(DISTRIBUTE_DIR)/bin $(DISTRIBUTE_DIR)/lib
DIST_ALIASES := dist
ifneq ($(strip $(DISTRIBUTE_DIR)),distribute)
DIST_ALIASES += distribute
endif
ALL_BUILD_DIRS := $(sort $(BUILD_DIR) $(addprefix $(BUILD_DIR)/, $(SRC_DIRS)) \
$(addprefix $(BUILD_DIR)/cuda/, $(SRC_DIRS)) \
$(LIB_BUILD_DIR) $(TEST_BIN_DIR) $(PY_PROTO_BUILD_DIR) $(LINT_OUTPUT_DIR) \
$(DISTRIBUTE_SUBDIRS) $(PROTO_BUILD_INCLUDE_DIR))
##############################
# Set directory for Doxygen-generated documentation
##############################
DOXYGEN_CONFIG_FILE ?= ./.Doxyfile
# should be the same as OUTPUT_DIRECTORY in the .Doxyfile
DOXYGEN_OUTPUT_DIR ?= ./doxygen
DOXYGEN_COMMAND ?= doxygen
# All the files that might have Doxygen documentation.
DOXYGEN_SOURCES := $(shell find \
src/$(PROJECT) \
include/$(PROJECT) \
python/ \
matlab/ \
examples \
tools \
-name "*.cpp" -or -name "*.hpp" -or -name "*.cu" -or -name "*.cuh" -or \
-name "*.py" -or -name "*.m")
DOXYGEN_SOURCES += $(DOXYGEN_CONFIG_FILE)
##############################
# Configure build
##############################
# Determine platform
UNAME := $(shell uname -s)
ifeq ($(UNAME), Linux)
LINUX := 1
else ifeq ($(UNAME), Darwin)
OSX := 1
OSX_MAJOR_VERSION := $(shell sw_vers -productVersion | cut -f 1 -d .)
OSX_MINOR_VERSION := $(shell sw_vers -productVersion | cut -f 2 -d .)
endif
# Linux
ifeq ($(LINUX), 1)
CXX ?= /usr/bin/g++
GCCVERSION := $(shell $(CXX) -dumpversion | cut -f1,2 -d.)
# older versions of gcc are too dumb to build boost with -Wuninitalized
ifeq ($(shell echo | awk '{exit $(GCCVERSION) < 4.6;}'), 1)
WARNINGS += -Wno-uninitialized
endif
# boost::thread is reasonably called boost_thread (compare OS X)
# We will also explicitly add stdc++ to the link target.
LIBRARIES += boost_thread stdc++
VERSIONFLAGS += -Wl,-soname,$(DYNAMIC_VERSIONED_NAME_SHORT) -Wl,-rpath,$(ORIGIN)/../lib
endif
# OS X:
# clang++ instead of g++
# libstdc++ for NVCC compatibility on OS X >= 10.9 with CUDA < 7.0
ifeq ($(OSX), 1)
CXX := /usr/bin/clang++
ifneq ($(CPU_ONLY), 1)
CUDA_VERSION := $(shell $(CUDA_DIR)/bin/nvcc -V | grep -o 'release [0-9.]*' | tr -d '[a-z ]')
ifeq ($(shell echo | awk '{exit $(CUDA_VERSION) < 7.0;}'), 1)
CXXFLAGS += -stdlib=libstdc++
LINKFLAGS += -stdlib=libstdc++
endif
# clang throws this warning for cuda headers
WARNINGS += -Wno-unneeded-internal-declaration
# 10.11 strips DYLD_* env vars so link CUDA (rpath is available on 10.5+)
OSX_10_OR_LATER := $(shell [ $(OSX_MAJOR_VERSION) -ge 10 ] && echo true)
OSX_10_5_OR_LATER := $(shell [ $(OSX_MINOR_VERSION) -ge 5 ] && echo true)
ifeq ($(OSX_10_OR_LATER),true)
ifeq ($(OSX_10_5_OR_LATER),true)
LDFLAGS += -Wl,-rpath,$(CUDA_LIB_DIR)
endif
endif
endif
# gtest needs to use its own tuple to not conflict with clang
COMMON_FLAGS += -DGTEST_USE_OWN_TR1_TUPLE=1
# boost::thread is called boost_thread-mt to mark multithreading on OS X
LIBRARIES += boost_thread-mt
# we need to explicitly ask for the rpath to be obeyed
ORIGIN := #loader_path
VERSIONFLAGS += -Wl,-install_name,#rpath/$(DYNAMIC_VERSIONED_NAME_SHORT) -Wl,-rpath,$(ORIGIN)/../../build/lib
else
ORIGIN := \$$ORIGIN
endif
# Custom compiler
ifdef CUSTOM_CXX
CXX := $(CUSTOM_CXX)
endif
# Static linking
ifneq (,$(findstring clang++,$(CXX)))
STATIC_LINK_COMMAND := -Wl,-force_load $(STATIC_NAME)
else ifneq (,$(findstring g++,$(CXX)))
STATIC_LINK_COMMAND := -Wl,--whole-archive $(STATIC_NAME) -Wl,--no-whole-archive
else
# The following line must not be indented with a tab, since we are not inside a target
$(error Cannot static link with the $(CXX) compiler)
endif
# Debugging
ifeq ($(DEBUG), 1)
COMMON_FLAGS += -DDEBUG -g -O0
NVCCFLAGS += -G
else
COMMON_FLAGS += -DNDEBUG -O2
endif
# cuDNN acceleration configuration.
ifeq ($(USE_CUDNN), 1)
LIBRARIES += cudnn
COMMON_FLAGS += -DUSE_CUDNN
endif
# NCCL acceleration configuration
ifeq ($(USE_NCCL), 1)
LIBRARIES += nccl
COMMON_FLAGS += -DUSE_NCCL
endif
# configure IO libraries
ifeq ($(USE_OPENCV), 1)
COMMON_FLAGS += -DUSE_OPENCV
endif
ifeq ($(USE_LEVELDB), 1)
COMMON_FLAGS += -DUSE_LEVELDB
endif
ifeq ($(USE_LMDB), 1)
COMMON_FLAGS += -DUSE_LMDB
ifeq ($(ALLOW_LMDB_NOLOCK), 1)
COMMON_FLAGS += -DALLOW_LMDB_NOLOCK
endif
endif
# CPU-only configuration
ifeq ($(CPU_ONLY), 1)
OBJS := $(PROTO_OBJS) $(CXX_OBJS)
TEST_OBJS := $(TEST_CXX_OBJS)
TEST_BINS := $(TEST_CXX_BINS)
ALL_WARNS := $(ALL_CXX_WARNS)
TEST_FILTER := --gtest_filter="-*GPU*"
COMMON_FLAGS += -DCPU_ONLY
endif
# Python layer support
ifeq ($(WITH_PYTHON_LAYER), 1)
COMMON_FLAGS += -DWITH_PYTHON_LAYER
LIBRARIES += $(PYTHON_LIBRARIES)
endif
# BLAS configuration (default = ATLAS)
BLAS ?= atlas
ifeq ($(BLAS), mkl)
# MKL
LIBRARIES += mkl_rt
COMMON_FLAGS += -DUSE_MKL
MKLROOT ?= /opt/intel/mkl
BLAS_INCLUDE ?= $(MKLROOT)/include
BLAS_LIB ?= $(MKLROOT)/lib $(MKLROOT)/lib/intel64
else ifeq ($(BLAS), open)
# OpenBLAS
LIBRARIES += openblas
else
# ATLAS
ifeq ($(LINUX), 1)
ifeq ($(BLAS), atlas)
# Linux simply has cblas and atlas
LIBRARIES += cblas atlas
endif
else ifeq ($(OSX), 1)
# OS X packages atlas as the vecLib framework
LIBRARIES += cblas
# 10.10 has accelerate while 10.9 has veclib
XCODE_CLT_VER := $(shell pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | grep 'version' | sed 's/[^0-9]*\([0-9]\).*/\1/')
XCODE_CLT_GEQ_7 := $(shell [ $(XCODE_CLT_VER) -gt 6 ] && echo 1)
XCODE_CLT_GEQ_6 := $(shell [ $(XCODE_CLT_VER) -gt 5 ] && echo 1)
ifeq ($(XCODE_CLT_GEQ_7), 1)
BLAS_INCLUDE ?= /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/$(shell ls /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ | sort | tail -1)/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/Headers
else ifeq ($(XCODE_CLT_GEQ_6), 1)
BLAS_INCLUDE ?= /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Headers/
LDFLAGS += -framework Accelerate
else
BLAS_INCLUDE ?= /Users/sarah/caffe/OpenBLAS
#/System/Library/Frameworks/vecLib.framework/Versions/Current/Headers/
LDFLAGS += -framework Accelerate
#vecLib
endif
endif
endif
INCLUDE_DIRS += $(BLAS_INCLUDE)
LIBRARY_DIRS += $(BLAS_LIB)
LIBRARY_DIRS += $(LIB_BUILD_DIR)
# Automatic dependency generation (nvcc is handled separately)
CXXFLAGS += -MMD -MP
# Complete build flags.
COMMON_FLAGS += $(foreach includedir,$(INCLUDE_DIRS),-I$(includedir))
CXXFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS)
NVCCFLAGS += -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)
# mex may invoke an older gcc that is too liberal with -Wuninitalized
MATLAB_CXXFLAGS := $(CXXFLAGS) -Wno-uninitialized
LINKFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS)
USE_PKG_CONFIG ?= 0
ifeq ($(USE_PKG_CONFIG), 1)
PKG_CONFIG := $(shell pkg-config opencv --libs)
else
PKG_CONFIG :=
endif
LDFLAGS += $(foreach librarydir,$(LIBRARY_DIRS),-L$(librarydir)) $(PKG_CONFIG) \
$(foreach library,$(LIBRARIES),-l$(library))
PYTHON_LDFLAGS := $(LDFLAGS) $(foreach library,$(PYTHON_LIBRARIES),-l$(library))
# 'superclean' target recursively* deletes all files ending with an extension
# in $(SUPERCLEAN_EXTS) below. This may be useful if you've built older
# versions of Caffe that do not place all generated files in a location known
# to the 'clean' target.
#
# 'supercleanlist' will list the files to be deleted by make superclean.
#
# * Recursive with the exception that symbolic links are never followed, per the
# default behavior of 'find'.
SUPERCLEAN_EXTS := .so .a .o .bin .testbin .pb.cc .pb.h _pb2.py .cuo
# Set the sub-targets of the 'everything' target.
EVERYTHING_TARGETS := all py$(PROJECT) test warn lint
# Only build matcaffe as part of "everything" if MATLAB_DIR is specified.
ifneq ($(MATLAB_DIR),)
EVERYTHING_TARGETS += mat$(PROJECT)
endif
##############################
# Define build targets
##############################
.PHONY: all lib test clean docs linecount lint lintclean tools examples $(DIST_ALIASES) \
py mat py$(PROJECT) mat$(PROJECT) proto runtest \
superclean supercleanlist supercleanfiles warn everything
all: lib tools examples
lib: $(STATIC_NAME) $(DYNAMIC_NAME)
everything: $(EVERYTHING_TARGETS)
linecount:
cloc --read-lang-def=$(PROJECT).cloc \
src/$(PROJECT) include/$(PROJECT) tools examples \
python matlab
lint: $(EMPTY_LINT_REPORT)
lintclean:
# $(RM) -r $(LINT_OUTPUT_DIR) $(EMPTY_LINT_REPORT) $(NONEMPTY_LINT_REPORT)
docs: $(DOXYGEN_OUTPUT_DIR)
# cd ./docs ; ln -sfn ../$(DOXYGEN_OUTPUT_DIR)/html doxygen
$(DOXYGEN_OUTPUT_DIR): $(DOXYGEN_CONFIG_FILE) $(DOXYGEN_SOURCES)
$(DOXYGEN_COMMAND) $(DOXYGEN_CONFIG_FILE)
$(EMPTY_LINT_REPORT): $(LINT_OUTPUTS) | $(BUILD_DIR)
# cat $(LINT_OUTPUTS) > $#
# if [ -s "$#" ]; then \
cat $#; \
mv $# $(NONEMPTY_LINT_REPORT); \
echo "Found one or more lint errors."; \
exit 1; \
fi; \
$(RM) $(NONEMPTY_LINT_REPORT); \
echo "No lint errors!";
$(LINT_OUTPUTS): $(LINT_OUTPUT_DIR)/%.lint.txt : % $(LINT_SCRIPT) | $(LINT_OUTPUT_DIR)
# mkdir -p $(dir $#)
# python $(LINT_SCRIPT) $< 2>&1 \
| grep -v "^Done processing " \
| grep -v "^Total errors found: 0" \
> $# \
|| true
test: $(TEST_ALL_BIN) $(TEST_ALL_DYNLINK_BIN) $(TEST_BINS)
tools: $(TOOL_BINS) $(TOOL_BIN_LINKS)
examples: $(EXAMPLE_BINS)
py$(PROJECT): py
py: $(PY$(PROJECT)_SO) $(PROTO_GEN_PY)
$(PY$(PROJECT)_SO): $(PY$(PROJECT)_SRC) $(PY$(PROJECT)_HXX) | $(DYNAMIC_NAME)
# echo CXX/LD -o $# $<
$(Q)$(CXX) -shared -o $# $(PY$(PROJECT)_SRC) \
-o $# $(LINKFLAGS) -l$(LIBRARY_NAME) $(PYTHON_LDFLAGS) \
-Wl,-rpath,$(ORIGIN)/../../build/lib
mat$(PROJECT): mat
mat: $(MAT$(PROJECT)_SO)
$(MAT$(PROJECT)_SO): $(MAT$(PROJECT)_SRC) $(STATIC_NAME)
# if [ -z "$(MATLAB_DIR)" ]; then \
echo "MATLAB_DIR must be specified in $(CONFIG_FILE)" \
"to build mat$(PROJECT)."; \
exit 1; \
fi
# echo MEX $<
$(Q)$(MATLAB_DIR)/bin/mex $(MAT$(PROJECT)_SRC) \
CXX="$(CXX)" \
CXXFLAGS="\$$CXXFLAGS $(MATLAB_CXXFLAGS)" \
CXXLIBS="\$$CXXLIBS $(STATIC_LINK_COMMAND) $(LDFLAGS)" -output $#
# if [ -f "$(PROJECT)_.d" ]; then \
mv -f $(PROJECT)_.d $(BUILD_DIR)/${MAT$(PROJECT)_SO:.$(MAT_SO_EXT)=.d}; \
fi
runtest: $(TEST_ALL_BIN)
$(TOOL_BUILD_DIR)/caffe
$(TEST_ALL_BIN) $(TEST_GPUID) --gtest_shuffle $(TEST_FILTER)
pytest: py
cd python; python -m unittest discover -s caffe/test
mattest: mat
cd matlab; $(MATLAB_DIR)/bin/matlab -nodisplay -r 'caffe.run_tests(), exit()'
warn: $(EMPTY_WARN_REPORT)
$(EMPTY_WARN_REPORT): $(ALL_WARNS) | $(BUILD_DIR)
# cat $(ALL_WARNS) > $#
# if [ -s "$#" ]; then \
cat $#; \
mv $# $(NONEMPTY_WARN_REPORT); \
echo "Compiler produced one or more warnings."; \
exit 1; \
fi; \
$(RM) $(NONEMPTY_WARN_REPORT); \
echo "No compiler warnings!";
$(ALL_WARNS): %.o.$(WARNS_EXT) : %.o
$(BUILD_DIR_LINK): $(BUILD_DIR)/.linked
# Create a target ".linked" in this BUILD_DIR to tell Make that the "build" link
# is currently correct, then delete the one in the OTHER_BUILD_DIR in case it
# exists and $(DEBUG) is toggled later.
$(BUILD_DIR)/.linked:
# mkdir -p $(BUILD_DIR)
# $(RM) $(OTHER_BUILD_DIR)/.linked
# $(RM) -r $(BUILD_DIR_LINK)
# ln -s $(BUILD_DIR) $(BUILD_DIR_LINK)
# touch $#
$(ALL_BUILD_DIRS): | $(BUILD_DIR_LINK)
# mkdir -p $#
$(DYNAMIC_NAME): $(OBJS) | $(LIB_BUILD_DIR)
# echo LD -o $#
$(Q)$(CXX) -shared -o $# $(OBJS) $(VERSIONFLAGS) $(LINKFLAGS) $(LDFLAGS)
# cd $(BUILD_DIR)/lib; rm -f $(DYNAMIC_NAME_SHORT); ln -s $(DYNAMIC_VERSIONED_NAME_SHORT) $(DYNAMIC_NAME_SHORT)
$(STATIC_NAME): $(OBJS) | $(LIB_BUILD_DIR)
# echo AR -o $#
$(Q)ar rcs $# $(OBJS)
$(BUILD_DIR)/%.o: %.cpp | $(ALL_BUILD_DIRS)
# echo CXX $<
$(Q)$(CXX) $< $(CXXFLAGS) -c -o $# 2> $#.$(WARNS_EXT) \
|| (cat $#.$(WARNS_EXT); exit 1)
# cat $#.$(WARNS_EXT)
$(PROTO_BUILD_DIR)/%.pb.o: $(PROTO_BUILD_DIR)/%.pb.cc $(PROTO_GEN_HEADER) \
| $(PROTO_BUILD_DIR)
# echo CXX $<
$(Q)$(CXX) $< $(CXXFLAGS) -c -o $# 2> $#.$(WARNS_EXT) \
|| (cat $#.$(WARNS_EXT); exit 1)
# cat $#.$(WARNS_EXT)
$(BUILD_DIR)/cuda/%.o: %.cu | $(ALL_BUILD_DIRS)
# echo NVCC $<
$(Q)$(CUDA_DIR)/bin/nvcc $(NVCCFLAGS) $(CUDA_ARCH) -M $< -o ${#:.o=.d} \
-odir $(#D)
$(Q)$(CUDA_DIR)/bin/nvcc $(NVCCFLAGS) $(CUDA_ARCH) -c $< -o $# 2> $#.$(WARNS_EXT) \
|| (cat $#.$(WARNS_EXT); exit 1)
# cat $#.$(WARNS_EXT)
$(TEST_ALL_BIN): $(TEST_MAIN_SRC) $(TEST_OBJS) $(GTEST_OBJ) \
| $(DYNAMIC_NAME) $(TEST_BIN_DIR)
# echo CXX/LD -o $# $<
$(Q)$(CXX) $(TEST_MAIN_SRC) $(TEST_OBJS) $(GTEST_OBJ) \
-o $# $(LINKFLAGS) $(LDFLAGS) -l$(LIBRARY_NAME) -Wl,-rpath,$(ORIGIN)/../lib
$(TEST_CU_BINS): $(TEST_BIN_DIR)/%.testbin: $(TEST_CU_BUILD_DIR)/%.o \
$(GTEST_OBJ) | $(DYNAMIC_NAME) $(TEST_BIN_DIR)
# echo LD $<
$(Q)$(CXX) $(TEST_MAIN_SRC) $< $(GTEST_OBJ) \
-o $# $(LINKFLAGS) $(LDFLAGS) -l$(LIBRARY_NAME) -Wl,-rpath,$(ORIGIN)/../lib
$(TEST_CXX_BINS): $(TEST_BIN_DIR)/%.testbin: $(TEST_CXX_BUILD_DIR)/%.o \
$(GTEST_OBJ) | $(DYNAMIC_NAME) $(TEST_BIN_DIR)
# echo LD $<
$(Q)$(CXX) $(TEST_MAIN_SRC) $< $(GTEST_OBJ) \
-o $# $(LINKFLAGS) $(LDFLAGS) -l$(LIBRARY_NAME) -Wl,-rpath,$(ORIGIN)/../lib
# Target for extension-less symlinks to tool binaries with extension '*.bin'.
$(TOOL_BUILD_DIR)/%: $(TOOL_BUILD_DIR)/%.bin | $(TOOL_BUILD_DIR)
# $(RM) $#
# ln -s $(notdir $<) $#
$(TOOL_BINS): %.bin : %.o | $(DYNAMIC_NAME)
# echo CXX/LD -o $#
$(Q)$(CXX) $< -o $# $(LINKFLAGS) -l$(LIBRARY_NAME) $(LDFLAGS) \
-Wl,-rpath,$(ORIGIN)/../lib
$(EXAMPLE_BINS): %.bin : %.o | $(DYNAMIC_NAME)
# echo CXX/LD -o $#
$(Q)$(CXX) $< -o $# $(LINKFLAGS) -l$(LIBRARY_NAME) $(LDFLAGS) \
-Wl,-rpath,$(ORIGIN)/../../lib
proto: $(PROTO_GEN_CC) $(PROTO_GEN_HEADER)
$(PROTO_BUILD_DIR)/%.pb.cc $(PROTO_BUILD_DIR)/%.pb.h : \
$(PROTO_SRC_DIR)/%.proto | $(PROTO_BUILD_DIR)
# echo PROTOC $<
$(Q)protoc --proto_path=$(PROTO_SRC_DIR) --cpp_out=$(PROTO_BUILD_DIR) $<
$(PY_PROTO_BUILD_DIR)/%_pb2.py : $(PROTO_SRC_DIR)/%.proto \
$(PY_PROTO_INIT) | $(PY_PROTO_BUILD_DIR)
# echo PROTOC \(python\) $<
$(Q)protoc --proto_path=$(PROTO_SRC_DIR) --python_out=$(PY_PROTO_BUILD_DIR) $<
$(PY_PROTO_INIT): | $(PY_PROTO_BUILD_DIR)
touch $(PY_PROTO_INIT)
clean:
#- $(RM) -rf $(ALL_BUILD_DIRS)
#- $(RM) -rf $(OTHER_BUILD_DIR)
#- $(RM) -rf $(BUILD_DIR_LINK)
#- $(RM) -rf $(DISTRIBUTE_DIR)
#- $(RM) $(PY$(PROJECT)_SO)
#- $(RM) $(MAT$(PROJECT)_SO)
supercleanfiles:
$(eval SUPERCLEAN_FILES := $(strip \
$(foreach ext,$(SUPERCLEAN_EXTS), $(shell find . -name '*$(ext)' \
-not -path './data/*'))))
supercleanlist: supercleanfiles
# \
if [ -z "$(SUPERCLEAN_FILES)" ]; then \
echo "No generated files found."; \
else \
echo $(SUPERCLEAN_FILES) | tr ' ' '\n'; \
fi
superclean: clean supercleanfiles
# \
if [ -z "$(SUPERCLEAN_FILES)" ]; then \
echo "No generated files found."; \
else \
echo "Deleting the following generated files:"; \
echo $(SUPERCLEAN_FILES) | tr ' ' '\n'; \
$(RM) $(SUPERCLEAN_FILES); \
fi
$(DIST_ALIASES): $(DISTRIBUTE_DIR)
$(DISTRIBUTE_DIR): all py | $(DISTRIBUTE_SUBDIRS)
# add proto
cp -r src/caffe/proto $(DISTRIBUTE_DIR)/
# add include
cp -r include $(DISTRIBUTE_DIR)/
mkdir -p $(DISTRIBUTE_DIR)/include/caffe/proto
cp $(PROTO_GEN_HEADER_SRCS) $(DISTRIBUTE_DIR)/include/caffe/proto
# add tool and example binaries
cp $(TOOL_BINS) $(DISTRIBUTE_DIR)/bin
cp $(EXAMPLE_BINS) $(DISTRIBUTE_DIR)/bin
# add libraries
cp $(STATIC_NAME) $(DISTRIBUTE_DIR)/lib
install -m 644 $(DYNAMIC_NAME) $(DISTRIBUTE_DIR)/lib
cd $(DISTRIBUTE_DIR)/lib; rm -f $(DYNAMIC_NAME_SHORT); ln -s $(DYNAMIC_VERSIONED_NAME_SHORT) $(DYNAMIC_NAME_SHORT)
# add python - it's not the standard way, indeed...
cp -r python $(DISTRIBUTE_DIR)/python
-include $(DEPS)
Makefile.config
# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_50,code=compute_50
BLAS := blas
BLAS_INCLUDE := /Users/sarah/caffe/OpenBLAS
BLAS_LIB := /Users/sarah/caffe/OpenBLAS
PYTHON_INCLUDE := /usr/local/lib/python2.7/site-packages/numpy/core/include/ /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/include/python2.7
PYTHON_LIB := /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := /usr/local/include /Developer/NVIDIA/CUDA-8.0/include/ $(PYTHON_INCLUDE) /usr/local/opt/opencv3/include
LIBRARY_DIRS := /usr/local/lib /usr/lib /Developer/NVIDIA/CUDA-8.0/lib/ $(PYTHON_LIB) /usr/local/opt/opencv3/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
TEST_GPUID := 0
Just a couple of hints and guesses.
This isn't necessarily a libc++/libstd++ inconsistency.
opencv_imgcodecs is only available in OpenCV 3+, so that may make some problems.
If you checked the opencv libraries and they were all linked to libc++, check if this command ld -lopencv_highgui, and for the other libraries as well, won't tell you the library is not found.
Another thing to check is making sure you only have one OpenCV installation, sometimes a conflicting copy may cause these problems.
You may also try to compile a standalone OpenCV example code, to make sure this is a problem with Caffe.
opencv_imgcodecs is only available in OpenCV 3+, so that may make some problems.
One last thing: remove everything, enable the cpu only mode to get rid of CUDA dependency, and this time use the latest version of everything, specifically: Latest Xcode command line tools and OpenCV3.
Posting your Makefile in your question could help, too.
You can set LDFLAGS before execute make all
$ LDFLAGS="`pkg-config --libs protobuf` `pkg-config --libs opencv`"
$ make all
If a file exists, I want to add a target to build. If the file does not exist, I want the target to be skipped.
an example:
FILENAME = f
TARGETS := normal
ifneq($(shell stat test_$(FILENAME).c), "")
TARGETS += test
endif
all: $(TARGETS)
normal:
#echo normal
test:
#echo test
I'm not sure the $(shell stat ...) part even works, but the bigger problem is that make with any file test_f.c in the current folder gives:
Makefile:4: *** multiple target patterns. Stop.
Removing the ifneq ... endif block makes the target normal. How can I only run the target test if test_f.c exists?
What you can do is generate a string variable (let's call it OPTIONAL) such that when 'test_f.c' exists, OPTIONAL=test; otherwise, OPTIONAL=_nothing_. And then add OPTIONAL as a prerequisite of all. e.g.:
FILENAME = f
TARGETS = normal
OPTIONAL = $(if $(wildcard test_f.c), test, )
all: $(TARGETS) $(OPTIONAL)
normal:
#echo normal
test:
#echo test
You can also iterate over targets with for loop
.PHONY: all
RECIPES = one
all: RECIPES += $(if $(wildcard test_f.c), two, )
all:
for RECIPE in ${RECIPES} ; do \
$(MAKE) $${RECIPE} ; \
done
one:
$(warning "One")
two:
$(warning "Two")
> make
for RECIPE in one ; do \
/Applications/Xcode.app/Contents/Developer/usr/bin/make ${RECIPE} ; \
done
makefile:11: "One"
make[1]: `one' is up to date.
> touch test_f.c
> make
for RECIPE in one two ; do \
/Applications/Xcode.app/Contents/Developer/usr/bin/make ${RECIPE} ; \
done
makefile:11: "One"
make[1]: `one' is up to date.
makefile:14: "Two"
make[1]: `two' is up to date.
I'm trying to create a Makefile which can build/clean recursively when it's called. I have the build working, but I'm getting errors with the clean command.
My directory structure is something like:
main
|
+-- Makefile
|
+-- common
| |
| +-- gcas_debug
| | |
| | +-- Makefile
| + -- gcas_nvdata
| |
| +-- Makefile
+-- gcinit
+-- Makefile
When I call make in the main directory it goes through and builds everything as I desire, but when I call make clean it does with:
mike#mike-VirtualBox:~/iCOM/framework$ make clean
for d in common/gcas_debug common/gcas_nvdata; \
do \
make --directory=$f clean; \
done
make: the `-C' option requires a non-empty string argument
Usage: make [options] [target] ...
Options:
-b, -m Ignored for compatibility.
...
I don't understand why the clean command isn't working... Any suggestions?
Full (main) Makefile:
lib_gcas_debug := common/gcas_debug
lib_gcas_nvdata := common/gcas_nvdata
libraries := $(lib_gcas_debug) $(lib_gcas_nvdata)
DESTDIR=$(PWD)/output
bindir=
gc_init := gcinit
EXE := $(gc_init)/$(gc_init)
.PHONY: all $(gc_init) $(libraries)
all: $(gc_init)
$(gc_init) $(libraries):
$(MAKE) --directory=$#
$(gc_init): $(libraries)
install: $(gc_init)
install -d -m 0755 $(DESTDIR)$(bindir)
install -m 0755 $(EXE) $(DESTDIR)$(bindir)
clean:
for d in $(libraries); \
do \
$(MAKE) --directory=$$f clean; \
done
EDIT: If I swap the for d with for $d I get instead:
mike#mike-VirtualBox:~/iCOM/framework$ make clean
for in common/gcas_debug common/gcas_nvdata; \
do \
make --directory= clean; \
done
/bin/sh: -c: line 0: syntax error near unexpected token `common/gcas_debug'
/bin/sh: -c: line 0: `for in common/gcas_debug common/gcas_nvdata; \'
make: *** [clean] Error 1
This worked for me:
SUBDIRS := foo bar baz
.PHONY: subdirs $(SUBDIRS) clean all
subdirs: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# $(MAKECMDGOALS)
clean: $(SUBDIRS)
rm -rf dist
dist: $(SUBDIRS) dist/.build_marker
dist/.build_marker:
mkdir -p dist
for d in $(SUBDIRS) ; do cp $$d/dist/* dist ; done
touch dist/.build_marker
I only need to use the for loop in the dist target to copy files. This allows make -j build parallelism, not to mention simpler-to-read Makefiles.
Read the for loop again:
for d in [...]; do make --directory=$f
Didn't you mean $d, as in
for d in [...]; do make --directory=$d
So, the Makefile should look:
for d in common/gcas_debug common/gcas_nvdata; \
do \
make --directory=$d clean; \
done