Hadoop n00b here, just started playing around with Hadoop Pipes. I'm getting linker errors while compiling a simple WordCount example using hadoop-0.20.203 (current most recent version) that did not appear for the same code in hadoop-0.20.2
Linker errors of the form: undefined reference to `EVP_sha1' in HadoopPipes.cc.
EVP_sha1 (and all of the undefined references I get) are part of the openssl library which HadoopPipes.cc from hadoop-0.20.203 uses, but hadoop-0.20.2 does not.
I've tried adjusting my makefile to link to the ssl libraries, but I'm still out of luck. Any ideas would be greatly appreciated. Thanks!
PS, here is my current makefile:
CC = g++
HADOOP_INSTALL = /usr/local/hadoop-0.20.203.0
SSL_INSTALL = /usr/local/ssl
PLATFORM = Linux-amd64-64
CPPFLAGS = -m64 -I$(HADOOP_INSTALL)/c++/$(PLATFORM)/include -I$(SSL_INSTALL)/include
WordCount: WordCount.cc
$(CC) $(CPPFLAGS) $< -Wall -Wextra -L$(SSL_INSTALL)/lib -lssl -lcrypto -L$(HADOOP_INSTALL)/c++/$(PLATFORM)/lib -lhadooppipes -lhadooputils -lpthread -g -O2 -o $#
The actual program I'm using can be found at http://cs.smith.edu/dftwiki/index.php/Hadoop_Tutorial_2.2_--_Running_C%2B%2B_Programs_on_Hadoop
Had same problem here: answer is to add -lcrypto to the compile command line:
http://grokbase.com/p/hadoop.apache.org/common-user/2011/06/re-linker-errors-with-hadoop-pipes/09zqdt5grdudu7no7q6k3gfcynpy
Here is a patch to fix the build process:
diff --git src/examples/pipes/Makefile.in src/examples/pipes/Makefile.in
index 17efa2a..1d8af8e 100644
--- src/examples/pipes/Makefile.in
+++ src/examples/pipes/Makefile.in
## -233,7 +233,7 ## AM_CXXFLAGS = -Wall -I$(HADOOP_UTILS_PREFIX)/include \
-I$(HADOOP_PIPES_PREFIX)/include
LDADD = -L$(HADOOP_UTILS_PREFIX)/lib -L$(HADOOP_PIPES_PREFIX)/lib \
- -lhadooppipes -lhadooputils
+ -lhadooppipes -lhadooputils -lcrypto
# Define the sources for each program
You just need to make some changes to your Makefile. The libraries that natively accompany hadoop seem not to do it. You'll need to "re-make" them and change your linked path.
A comprehensive answer to this can be found at http://goo.gl/y5iGZF.
Related
I have a project that builds with
autoreconf --install
./configure
make
and contains configure.ac and Makefile.am
The project builds 2 libraries and one executable that dynamically links them. One library is cpp, other and the executable are pure C. I want to link one of the libs (cpp one) statically and keep c library to link dynamically. My simplified Makefile.am is
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = include doc po
AM_CPPFLAGS = -Iinclude
AM_CPPFLAGS += -D__STRICT_ANSI__
AM_CPPFLAGS += -DNDEBUG
WARNFLAGS = -Wall
AM_CFLAGS = $(WARNFLAGS) $(OPENMP_CXXFLAGS) -std=c11
AM_CXXFLAGS = $(WARNFLAGS) -std=c++11
localedir = $(datadir)/locale
DEFS = -DLOCALEDIR=\"$(localedir)\" #DEFS#
lib_LTLIBRARIES = libmy.la libmy-settings.la
libmy_la_SOURCES = src/sharedlib.c src/sharedlib.h
libmy_settings_la_SOURCES = src/staticlib.cpp src/staticlib.h
bin_PROGRAMS = myapp
myapp_SOURCES = myapp.c
myapp_LDADD = libmy.la libmy-settings.la
The libtool generates both .a and .so files in ./.libs/ but Libtool prefers .so. In the last make command I see gcc -Wall -std=c11 -g -O2 -o .libs/myapp myapp.o ./.libs/libmy.so ./.libs/libmy-settings.so. If I replace it with gcc -Wall -std=c11 -g -O2 -o .libs/myapp myapp.o ./.libs/libmy.so ./.libs/libmy-settings.a - it works and gives me that I'm trying to achieve. The question is: how to achieve it automatically wit help of Makefile.am?
As I understand, I can't just add myapp_LDFLAGS=-lmy-settings.a bcs it won't replace shared linking argument but add a new one. And I can't remove the libmy-settings.la from lib_LTLIBRARIES or myapp_LDADD bcs it won't generate libmy-settings.a or libmysettings.so in ./.libs/ at all. Any clues?
Since libmy-settings does not need to be a separately installable library, you have two pretty straightfoward alternatives:
Add its sources to myapp_SOURCES, and leave libmy-settings.la completely out of the picture. That the additional sources are located in a different directory and written in a different (autotools-supported) language should not be a problem. This approach may be to your advantage in terms of the Autotools choosing the correct linker driver and options without your intervention, too.
Build libmy-settings.la as a utility library instead of an installable one. This is achieved by marking it noinst:
lib_LTLIBRARIES = libmy.la
noinst_LTLIBRARIES = libmy-settings.la
I have this very simple makefile:
P = hello_world.exe
OBJECTS = main.o
CFLAGS = -g -Wall -O3
LDLIBS =
CC = clang
$(P): $(OBJECTS)
When I run make it will compile main.c but it will not link to hello_world.exe. Shouldn't that be happening automatically?
My environment is cygwin 64bit.
The output of make -p is here: http://pastebin.com/qbr0sRXL
There's no default rule for .exe files that I'm aware of (or can find in that output).
You'll need to write one yourself.
If your output was hello_world and you had a hello_world.c/hello_world.cpp source file and also a main.c/main.cpp file then your makefile as written would work I believe (since the default %: %.o rule would apply and your added prerequisite would be added to the hello_world prerequisite list).
I am trying to install QCL-0.6.4 from this source, but I keep getting errors, when I try it with the make command in the terminal.
I came along this thread about installing QCL on OSX, but when trying to adjust the Makefile I always come across this errors:
extern.cc:84:18: error: variable length array of non-POD element type 'tComplex'
(aka 'complex<double>')
tComplex u[dim][dim];
^
extern.cc:193:9: error: variable length array of non-POD element type 'term'
term t[dim];
^
extern.cc:224:9: error: variable length array of non-POD element type 'term'
term t[dim];
Any help on this would be highly appreciated.
There are a few issues at play here which you need to overcome to get this compiling on OSX. My instructions below assume that you are running on El Capitan (10.11.1 in my instance), but you may get some milage out of them for different versions.
Firstly, Xcode currently uses Apple's LLVM Compiler as the default C++ compiler. However, this doesn't support some of GCC's extensions, such as support for non-POD variable length arrays.
To get around this, I installed and compile with GCC: if you haven't already, install Homebrew, and then install the latest GCC compiler with:
$ brew install gcc
At the time of writing, this will install GCC v5.2.0.
That should fix your initial problem, but you will instantly hit others!
The next issue is that the included libqc.a will need recompiling for x86_64. So you will need to modify the file <base_dir>/qc/Makefile with the following changes:
...
# Add:
CXX = /usr/local/Cellar/gcc/5.2.0/bin/g++-5
CXXFLAGS = $(ARCHOPT) -c -pedantic -Wall $(DEBUG) $(PRGOPT)
...
Then rebuild libqc.a:
$ cd qc; make clean; make
If all goes well, you should have a shiny new libqc.a.
Finally, modify the main Makefile <base_dir>/Makefile with the following changes:
...
# Comment out:
#PLOPT = -DQCL_PLOT
#PLLIB = -L/usr/X11/lib -lplotter
...
# Comment out:
#RLOPT = -DQCL_USE_READLINE
#RLLIB = -lreadline
#RLLIB = -lreadline -lncurses
...
# Comment out:
#CXX = g++
#CPP = $(CC) -E
#CXXFLAGS = -c $(ARCHOPT) $(DEBUG) $(PLOPT) $(RLOPT) $(IRQOPT) $(ENCOPT) -I$(QCDIR) -DDEF_INCLUDE_PATH="\"$(QCLDIR)\""
#LDFLAGS = $(ARCHOPT) -L$(QCDIR) $(DEBUG) $(PLLIB) -lm -lfl -lqc $(RLLIB)
# Add:
CXX = /usr/local/Cellar/gcc/5.2.0/bin/g++-5
CPP = $(CC) -E
CXXFLAGS = -c $(ARCHOPT) $(DEBUG) $(PLOPT) $(RLOPT) $(IRQOPT) $(ENCOPT) -I$(QCDIR) -DDEF_INCLUDE_PATH="\"$(QCLDIR)\""
LDFLAGS = $(ARCHOPT) -L$(QCDIR) $(DEBUG) $(PLLIB) -lm -ll -lqc $(RLLIB) -lc++
...
This should now allow you to build the main application as per the instructions:
$ make clean; make; make install
I'm a newby in cppuTest and actually I'm trying to build the ./examples within the CppuTest root directory. Source and Test files are compiled without problems but I'm stucking on the final linking stage where I get this error:
C:\CppUTest\cpputest-3.7.1\examples>make
compiling AllTests.cpp
compiling CircularBufferTest.cpp
compiling EventDispatcherTest.cpp
compiling HelloTest.cpp
compiling MockDocumentationTest.cpp
compiling PrinterTest.cpp
compiling CircularBuffer.cpp
compiling EventDispatcher.cpp
compiling Printer.cpp
compiling hello.c
Building archive lib/libCppUTestExamples.a
a - objs/ApplicationLib/CircularBuffer.o
a - objs/ApplicationLib/EventDispatcher.o
a - objs/ApplicationLib/Printer.o
a - objs/ApplicationLib/hello.o
Linking CppUTestExamples_tests
../lib/libCppUTest.a(UtestPlatform.cpp.obj): In function `PThreadMutexCreate':
c:/CppUTest/cpputest-3.7.1/src/Platforms/Gcc/UtestPlatform.cpp:248: undefined reference to `_imp__pthread_mutex_init'
../lib/libCppUTest.a(UtestPlatform.cpp.obj): In function `PThreadMutexLock':
c:/CppUTest/cpputest-3.7.1/src/Platforms/Gcc/UtestPlatform.cpp:255: undefined reference to `_imp__pthread_mutex_lock'
../lib/libCppUTest.a(UtestPlatform.cpp.obj): In function `PThreadMutexUnlock':
c:/CppUTest/cpputest-3.7.1/src/Platforms/Gcc/UtestPlatform.cpp:260: undefined reference to `_imp__pthread_mutex_unlock'
../lib/libCppUTest.a(UtestPlatform.cpp.obj): In function `PThreadMutexDestroy':
c:/CppUTest/cpputest-3.7.1/src/Platforms/Gcc/UtestPlatform.cpp:266: undefined reference to `_imp__pthread_mutex_destroy'
collect2.exe: error: ld returned 1 exit status
make: *** [CppUTestExamples_tests] Error 1
I'm using MinGW on Windows 7. The MinGW contains also the pthread.a library. My makefil look as follows:
#---------
#
# CppUTest Examples Makefile
#
#----------
#Set this to # to keep the makefile quiet
ifndef SILENCE
SILENCE = #
endif
#--- Inputs ----#
COMPONENT_NAME = CppUTestExamples
CPPUTEST_HOME = ..
CPPUTEST_USE_EXTENSIONS = Y
CPP_PLATFORM = Gcc
CFLAGS = -Dmalloc=cpputest_malloc -Dfree=cpputest_free
CPPFLAGS =
GCOVFLAGS = -fprofile-arcs -ftest-coverage
LDFLAGS = -lpthread
#USER_LIBS = -lpthread
# This line is overriding the default new macros. This is helpful
# when using std library includes like <list> and other containers
# so that memory leak detection does not conflict with stl.
CPPUTEST_MEMLEAK_DETECTOR_NEW_MACRO_FILE = -include ApplicationLib/ExamplesNewOverrides.h
SRC_DIRS = \
ApplicationLib
TEST_SRC_DIRS = \
AllTests
INCLUDE_DIRS =\
.\
ApplicationLib\
$(CPPUTEST_HOME)/include\
include $(CPPUTEST_HOME)/build/MakefileWorker.mk
As you can see the pthread lib is given to the linker with the LDFLAGS....
somebody similar experience? or maybe knows where the problem is?
Will be thankful for any tips!
thx to #Keith Marshall and #MadScientist,
so instead of
LDFLAGS = -lpthread
I used:
LD_LIBRARIES += -lpthread
and placed this line directly before:
include $(CPPUTEST_HOME)/build/MakefileWorker.mk
Now it works.
As can be seen in the Catalogue of Built-In Rules:
Linking a single object file
n is made automatically from n.o by running the linker (usually called
ld) via the C compiler. The precise recipe used is:
$(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS)
and Variables Used by Implicit Rules:
LDFLAGS
Extra flags to give to compilers when they are supposed to invoke the linker,
ld, such as -L. Libraries (-lfoo) should be added to the LDLIBS variable
instead.
So in this case -lpthread should be set or added to LDLIBS, not LDFLAGS.
When I tried to build a project in Linux, I got Error: undefined symbol clock_gettime. So I figured out that I needed to add -lrt to the build command (gcc). However, now it won't compile in OS X: ld: library not found for -lrt. I don't know exactly where this function is getting called as it's in statically linked code, but it seemed to be working just fine in OS X without librt. The linked code probably uses an alternative behind #if __APPLE__ or something.
Is there any way that I can instruct gcc to only link librt if it's needed, or if it exists? If not, how do I create a Makefile with OS-specific commands? I'm not using autoconf or anything like it.
The Makefile is rather complex, but here's the operative part:
CC := g++
# In this line, remove -lrt to compile on OS X
LFLAGS := -lpthread -lrt
CFLAGS := -c -Wall -Iboost_build -Ilibtorrent_build/include -Iinc
OBJDIR := obj
SRCDIR := src
SRC := $(wildcard $(SRCDIR)/*.cpp)
OBJS := $(patsubst $(SRCDIR)/%.cpp,$(OBJDIR)/%.o,$(SRC))
# Note that libtorrent is built with a modified jamfile to place the
# libtorrent.a file in a consistent location; otherwise it ends up somewhere
# dependent on build environment.
all : $(OBJS) libtorrent_build boost_build
$(CC) -o exec $(LFLAGS) \
$(OBJS) \
libtorrent_build/bin/libtorrent.a \
boost_build/stage/lib/libboost_system.a
You could try this:
LFLAGS := -lpthread
OS := $(shell uname -s)
ifeq ($(OS),Linux)
LFLAGS += -lrt
endif
If you google the problem you will find a good set of solutions, one is posted as a comment at the question, another is here:
Here
gettimeofday() could be a betteer solution, if you are compiling code not writed by you bear in mind that clock_gettime function is NOT provided under OS X and you need to modify the code.
Hope that can help you,
pedr0