In a project I have an already created shared library called, let's say, 'myshare'. The library is already created as 'libmyshare.so.1.0.0' and its location is relative to the makefile at a relative address of 'adir/lib'.
In the makefile I want to create symbolic links for the shared library with the names 'libmyshare.so.1' and 'libmyshare.so'. In order to do this in the makefile I have these lines:
MYSHAREL = ./adir/lib/libmyshare.so.1.0.0
MYSHAREL1 = ./adir/lib/libmyshare.so.1
MYSHAREL2 = ./adir/lib/libmyshare.so
MYSHARELIBS = $(MYSHAREL) $(MYSHAREL1) $(MYSHAREL2)
.PHONY: all
all: myexetarget
myexetarget : various_files $(MYSHARELIBS)
various_commands_to_make_target
$(MYSHAREL1): $(MYSHAREL)
#echo "Creating symbolic link $(MYSHAREL1)"
#ln -s $(MYSHAREL) $(MYSHAREL1)
$(MYSHAREL2): $(MYSHAREL)
#echo "Creating symbolic link $(MYSHAREL2)"
#ln -s $(MYSHAREL) $(MYSHAREL2)
When I run the 'make' command the links to the shared library are always created even when they already exist and even when the date of the shared library is prior to the date of already created symbolic links. Furthermore when I try to actually link to the shared library in my target, despite an -L./adir/lib and -lmyshare options among the linker options ( not shown above ) the link fails with:
/usr/bin/ld: cannot find -lmyshare
Both these problems lead me to believe that I am not specifying the name or relative location of the shared library correctly for the makefile processing to understand it. Does anybody know what I am doing wrong ?
Your use of ln -s is probably the cause of your problem. If, from your top directory you execute:
ln -s ./adir/lib/libmyshare.so.1.0.0 ./adir/lib/libmyshare.so.1
(which is what your make rule does) a symbolic link is created in ./adir/lib, named libmyshare.so.1 and pointing to ./adir/lib/libmyshare.so.1.0.0:
$ cd ./adir/lib
$ ls -l
libmyshare.so.1 -> ./adir/lib/libmyshare.so.1.0.0
libmyshare.so.1.0.0
So, ./adir/lib/libmyshare.so.1 actually points to ./adir/lib/adir/lib/libmyshare.so.1.0.0 that does not exist.
You can replace your ln -s command by:
#ln -sr $(MYSHAREL) $(MYSHAREL1)
Same for MYSHAREL2. The -r option of ln create symbolic links relative to link location.
Related
I'm bundling a .cpp program that depends on libxml, but g++ cannot find it. I've done the appropriate steps to make libxml available, however the directory structure is slightly different than the source expects. I want to know about nix-based solutions to this problem.
The compiler (from a nix-shell session) says that libxml is not in my system path:
fatal error: libxml/encoding.h: No such file or directory
#include <libxml/encoding.h>
However, it should be. In my .nix file I made sure to included libxml2
buildInputs = [ ... libxml2 ...];
and my nix-shell environment confirms that the library is present:
NIX_CFLAGS_COMPILE= [...] -isystem /nix/store/zf1nyqyx2zd6y944ln2rxnhd5m4265n4-libxml2-2.9.9-dev/include [...]
If I look in that directory, I find the search path is off. The path to the file is actually (relative to NIX_CFLAGS_COMPILE path):
libxml2/libxml/encoding.h
I found that I can compensate for this off-by-one path error by adding the following option to my compile command:
-isystem /nix/store/zf1nyqyx2zd6y944ln2rxnhd5m4265n4-libxml2-2.9.9-dev/include/libxml2
(The same path as the NIX_CFLAGS_COMPILE, but starting at the libxml2 directory.)
Given that information, I am wondering about a nix-based solution to this problem. The source is not mine, so changing them is the last thing I want to do. I see two other options.
First, I could add the path in the makefile that is triggered during buildPhase. However, I'm not exactly sure how to simply grab that path. For instance, inheriting libxml2 in my derivation makes the libxm2-2.9.9-bin directory available, when I need the libxml2-2.9.9-dev. I suppose I could grep or sed on the NIX_CFLAGS_COMPILE path, but I'd rather not.
My preferred solution would be to make a simple libxml derivation based off the libxml2 and just add that to my buildInputs, but that seems like it might not be simple either.
Try running pkg-config --cflags libxml-2.0 to get the appropriate compiler options.
If everything is configured correctly, it will find the appropriate /nix/store/*-libxml2-2.9.9-dev/lib/pkgconfig/libxml-2.0.pc file on your computer, and extract the needed compiler options from that.
You could invoke GCC using a one-liner like this:
g++ program.cpp $(pkg-config --libs --cflags libxml-2.0) -o program
This is a general solution that doesn't just apply to Nix, but you should be able to get it to work in Nix by ensuring you have the pkg-config utility on your PATH and ensuring the proper environment variables are set so that it can find libxml2. Nix probably has a bunch of shell scripts or something to help you set up your environment in that way, because this is a common need.
I solved this by defining a libxml attribute in my derivation and adding that to my buildInputs. The new library is just a link to the old, but with path corrected for. Here's the relevant parts of the derivation:
{ stdenv, libxml2, ... } : stdenv.mkDerivation
rec
{ buildInputs = [ libxml ];
libxml = stdenv.mkDerivation {
name = "libxml" ;
system = builtins.currentSystem;
outputs = [ "bin" "dev" "out" ];
phases = ["buildPhase"];
buildPhase =
''echo "my command out = $out"
echo ${libxml2.dev}
mkdir -p $out
mkdir -p $dev
mkdir -p $bin
ln -s ${libxml2.dev}/include/libxml2 $dev/include
ln -s ${libxml2.dev}/lib $dev/lib
''; };
}
With GNU Make, I want to take a list of files and create symbolic links of them in another directory (with the same filenames), but also overwrite any existing link with the same filename.
existing_files = $(wildcard dir1/dir2/*.txt)
# The next line shows where I would want to put the symbolic links
symlinks = $(wildcard new_dir1/new_dir2/*.txt)
make_some_links:
# Remove previous symlinks if they share name as existing_files
ifeq ($(notdir $(existing_files)), $(notdir $(symlinks)))
$(info Removing previous symlinks)
#rm $(symlinks)
endif
# Loop to link here
As an aside, does the above condition attempt to match each element of each list, or does it succeed if only 1 of the elements match?
I can't quite figure out how to loop through in Make, including with $(foreach), so I wrote the following loop to show what I mean in something like Python with the same variable names.
for i in len($(existing_files)):
#ln -s $(existing_files)[i] $(symlinks)[i]
Here, the first element in $(list_files) is linked to the first element in $(symlinks). Any insight on how to write this loop in Make, or if there is straight-up a better way to approach this would be very helpful. Thanks.
Let's take this in stages.
First, to create a symbolic link, and remove a preexisting link of that name, if there is one:
ln -fs filename linkname
Now to make a list of the existing files:
existing_files = $(wildcard dir1/dir2/*.txt)
So far, so good. Let's suppose this gives us dir1/dir2/red.txt dir1/dir2/green.txt.
# The next line shows where I would want to put the symbolic links
symlinks = $(wildcard new_dir1/new_dir2/*.txt)
That will give you a list of the things that already exist in that directory, which is probably not what you intend. We must construct the list of links we want from the list of files we have:
filenames := $(notdir $(existing_files))
symlinks := $(addprefix new_dir1/new_dir2/, $(filenames))
Now for a rule or rules to build the symlinks. We could write two explicit rules:
new_dir1/new_dir2/red.txt: dir1/dir2/red.txt
ln -fs dir1/dir2/red.txt new_dir1/new_dir2
new_dir1/new_dir2/green.txt: dir1/dir2/green.txt
ln -fs dir1/dir2/green.txt new_dir1/new_dir2
but that is horribly redundant, and besides we don't know the file names beforehand. First we can remove some of the redundancy by defining a variable and using the automatic variale $<:
DEST_DIR := new_dir1/new_dir2
$(DEST_DIR)/red.txt: dir1/dir2/red.txt
ln -fs $< $(DEST_DIR)
$(DEST_DIR)/green.txt: dir1/dir2/green.txt
ln -fs $< $(DEST_DIR)
Now we can see how to replace these rules with a pattern rule:
$(DEST_DIR)/%.txt: dir1/dir2/%.txt
ln -fs $< $(DEST_DIR)
Now all we need is a master rule that requires the links:
.PHONY: make_some_links
make_some_links: $(symlinks)
In most cases, when you start designing shell loops in make recipes, you are missing important features of make. Make will naturally "loop" over the targets you declare to reach the goals you ask it to reach. And make offers many ways to factorize similar rules.
But in your case where the targets are symbolic links there are several issues to consider:
If you declare the links as regular targets and if some exist already but point to other files than the ones you want, make could consider them as up-to-date and skip them, instead of replacing them. You must thus find a way to force make to re-create all the links or, at least, those that do not point to the correct file. And the last modification times are not sufficient for this.
Even with the -f option ln will not replace an exiting directory (or a link to a directory), instead it will create the link inside the directory. You must thus first delete any directory (or link to a directory) that conflicts with a target link.
The ln command shall be used with the -sr options to create symbolic links relative to link location.
The target directory (new_dir1/new_dir2) shall exist before you try to create the first link.
If you don't care re-creating links that are already correct, things are easy:
declare a phony dummy target (force) and make it a prerequisite of all your links to force make re-creating them,
before creating your links delete existing links, files or directories that conflict with them,
use the proper ln options,
declare the target directory as an order-only prerequisite of the links and add a rule to create it if it is missing.
The following should do the job:
source_dir := dir1/dir2
link_dir := new_dir1/new_dir2
existing_files := $(wildcard $(source_dir)/*.txt)
symlinks := $(patsubst $(source_dir)/%,$(link_dir)/%,$(existing_files))
.PHONY: all force
all: $(symlinks)
$(link_dir)/%: $(source_dir)/% force | $(link_dir)
rm -rf $#
ln -sr $< $#
force:;
$(link_dir):
mkdir -p $#
If you want to avoid re-creating the existing correct links, things are more difficult because make needs to know which ones are correct and which ones are not, and this cannot be based on last modification times... But as I suspect that the performance impact of re-creating the existing correct links is negligible I suggest that we don't design a complex solution to a non-existing problem.
I am developing in XCode on Mac OS X El Capitan 10.11.4. My project mixes C, C++11 and Embedded Python.
My project works as I can invoke a Python script and return the data to C++ by embedding python. In my project I use absolute paths as pythons search path to load the script from 'C'.
XCode project/
-- Python.framework/
-- python/
---- mypython.py
-- python_interface.c
-- main.cpp
My questions are:
Q1: I have brew Python available but this isn't seen from XCode, instead the system supplied one is. This isn't a problem for now but I would eventually like to know how to point to my chosen installation.
I have bundled the brew Python.framework into my project so it links successfully. I know it doesn't invoke this because if I specify my module path incorrectly it complains the system python can not find it. Also, 'system(which python)' reports '/usr/bin/python'.
Q2: How do I specify relative search paths to python within XCode, i.e. to locate my local python module from 'C' code within my project?
Q3: How do I determine the absolute path of python 'requests' installation at runtime? My python module imports this and it could be different than what I specify.
Currently, I use Py_GetPath and Py_SetPath to indicate these using absolute paths, i.e. '/usr/local/lib/python2.7/site-packages' for 'requests'.
I know how to locate the module path within python itself (Find path of module without importing in Python) but this is not what I want to do. I need to know the path before my script is run.
As mentioned I am embedding python, so I am making the call from 'C' to my python script (see https://docs.python.org/2/extending/embedding.html?highlight=embedded#pure-embedding).
I have found the following links that show how to get the path of the executable. What I would like is the path of the project and use relative paths from that to locate my python module.
Relative Paths Not Working in Xcode C++
Programmatically retrieving the absolute path of an OS X command-line app
I have found this posting; Relative imports for the billionth time. My search path from 'C' code to my python script is relative but I believe this post is mainly about python scripts importing other modules relative to each other.
Q4: The result of adding or linking Frameworks to an XCode project is the same. In my case the Python.framework appears in the 'Project navigator' and it is added to the 'Link Binaries With Libraries' section. Yet the following two articles indicate that there is a difference between the two. It is not the size of the executable as I have tried both methods and this remains the same.
https://developer.apple.com/library/ios/recipes/xcode_help-structure_navigator/articles/Adding_a_Framework.html
This states;
"The frameworks you add this way are third-party-built bundles containing the framework’s object code and runtime resources. See related articles for information about linking to a framework without adding it to the project."
and contains a link to "Linking to a Library or Framework", whose url seems to contradict this (AddingaLibrarytoaTarget).
https://developer.apple.com/library/ios/recipes/xcode_help-project_editor/Articles/AddingaLibrarytoaTarget.html#//apple_ref/doc/uid/TP40010155-CH17
Thanks.
Some answers to my questions
I reorganised my project and created my own Makefile using various sources on google. This was because I wanted to port my project as I couldn't answer the above questions. I had worked with Makefiles some years before but I am relatively new to XCode.
To create my Makefile these are the resources I used;
https://www3.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html
C-library not linking using gcc/g++
https://www.daniweb.com/programming/software-development/threads/124637/makefile-for-c-one-file-programs
https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
http://www.puxan.com/web/blog/HowTo-Write-Generic-Makefiles
https://www.gnu.org/software/make/manual/
http://nuclear.mutantstargoat.com/articles/make/#building-c-c-programs
https://www.gnu.org/software/make/manual/html_node/index.html
The next step is to automate the process of generating Makefiles;
http://www.ifnamemain.com/posts/2014/Mar/13/autoconf_automake/
New Makefile project structure
project/
-- Debug/ or Release/
-- Makefile
-- obj/
---- target .o files
-- python/
---- .py files
-- src/
---- C/C++ files incl headers
Makefile
# https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
# $# Contains the target file name.
# $< Contains the first dependency file name.
# $^ The names of all the prerequisites, with spaces between them.
# Produces a Release build by default, or Debug build with ‘make debug’
EXEC = $(BUILD_DIR)/my_exe_name
CCFLAGS=-c -Wall
CXXFLAGS=-c -Wall -std=c++11
BINARY_DIR=Obj
BUILD_DIR=Release
PYTHON_CCFLAGS=$(shell python2.7-config --cflags)
PYTHON_LDFLAGS=$(shell python2.7-config --ldflags)
CPP_FILES := $(wildcard src/*.cpp)
C_FILES := $(wildcard src/*.c)
CPP_OBJ_FILES := $(addprefix $(BINARY_DIR)/,$(notdir $(CPP_FILES:.cpp=.o)))
C_OBJ_FILES := $(addprefix $(BINARY_DIR)/,$(notdir $(C_FILES:.c=.o)))
C_HEADERS = $(filter-out $(addsuffix .h, $(basename $(SOURCE_FILES))), $(wildcard src/*.h))
CPP_HEADERS = $(filter-out $(addsuffix .h, $(basename $(SOURCE_FILES))), $(wildcard src/*.hpp))
### Take the BUILD_DIR as a dependency but ignore it’s timestamp
### as it will change whenever something is written into it
$(EXEC): $(C_OBJ_FILES) $(CPP_OBJ_FILES) | ${BUILD_DIR}
#echo [ $(EXEC) ]
$(CXX) $(LD_FLAGS) $(PYTHON_LDFLAGS) -o $(EXEC) $^
### The objects in the BINARY_DIR folder require the BINARY_DIR to be created
### as well as any changes to the other dependencies
$(BINARY_DIR)/%.o: src/%.cpp $(CPP_HEADERS) $(C_HEADERS) | ${BINARY_DIR}
#echo [ make $# ]
$(CXX) $(CXXFLAGS) $(C11_FLAGS) -o $# $<
$(BINARY_DIR)/%.o: src/%.c $(C_HEADERS) | ${BINARY_DIR}
#echo [ make $# ]
$(CC) $(CCFLAGS) $(PYTHON_CCFLAGS) -o $# $<
${BINARY_DIR}:
mkdir $(BINARY_DIR)
$(BUILD_DIR):
mkdir $(BUILD_DIR)
### clean the targets. We can have either Debug or Release but I don’t know
### how to specify either in one line, i.e. we don’t want to do ‘make clean debug’
### to specify the Debug folder
.PHONY: clean cleanmsg cleanrel cleandeb
clean: cleanmsg cleanrel cleandeb
rm -r -f $(BINARY_DIR)
cleanmsg:
#echo Cleaning product and all .o files
cleanrel:
rm -r -f Release
cleandeb:
rm -r -f Debug
# Debug build
# Ensure these are last. I had them below the original decl of the flags
# and they were always set! Now you have to ‘make debug’ for it to take effect
.PHONY: debug
debug: CCFLAGS += -DDEBUG -g
debug: CXXFLAGS += -DDEBUG -g
debug: BUILD_DIR=Debug
debug: $(EXEC)
Using XCode with new Makefile project
To continue using XCode to develop my project I created an XCode project over my Makefile project. See http://hiltmon.com/blog/2015/08/01/simple-c-plus-plus-from-makefiles-to-xcode-builds/
Answers
By doing this I am able to somewhat answer the above questions. Q3 is still unanswered.
A1: Create a Makefile project and within it state the platform specified python installation. This way you do not need to bundle it into your XCode project.
See https://docs.python.org/2/extending/embedding.html?highlight=embedded#compiling-and-linking-under-unix-like-systems.
A2: Make the XCode project relative so your product resides in your project. Not the best answer if you want your exe installed in /usr/local/bin.
See How to change output directory for a target (gp_coder's answer).
My Makefile stores the exe in either ./Release/ (make), or ./Debug/ (make debug). So does XCode but with different leading directories that are several levels deep. To change this, simply specify the current dir '.' in XCode for the Targets "Project Settings>Advanced>Custom>Products" destination directory, this will then match the Makefile.
My python search path is set to
:../python:/usr/local/lib/python2.7/site-packages
'../' because my exe is in ./Debug or./Release and python/ is relative to that.
However, if you were to invoke the exe from the cmdline one dir up it wouldn't work. e.g. ./Release/my_exe_name. This is because it is taking the current dir and using the search path in relation to that.
It would be better to store and set the absolute path. See Where to store application data (non-user specific) on Linux
I posted a question on this before I realised the answer; How to set relative project path in XCode when target is /usr/local/bin
The second path is for my python requests directory. Q3 refers to this.
A3: Don't know. Still have to work out how to pass my installed python module path into my C code at runtime BEFORE invoking python, as this makes up my python search path. This value changes based on your installation.
A4: This wasn't really a question but a confusion. Specify this in your Makefile to avoid bundling.
I will start by saying that I am new to gcc and makefiles. I have a .so file on the desktop (~/Desktop) called lib.so. I want to link my program (called myProgram) to it. What I wrote in my makefile is:
g++ myProgram.o -L ~/Desktop -l lib -o myProgram
When I run make I get an error:
/usr/bin/ld: cannot find -llib
I also tried -l lib.so and got the same error.
What is the correct way to link?
Two solutions:
Rename the file to libsomething.so, then use -l something. The linker automatically wraps the name with lib prefix and .so suffix (or .a suffix for static libraries).
Use the option -l :lib.so. When you prefix the name with :, the linker uses the name as given.
These are explained in the ld man page.
This question already has answers here:
How to link using GCC without -l nor hardcoding path for a library that does not follow the libNAME.so naming convention?
(3 answers)
Closed 3 years ago.
I have to link my code to a shared library without the lib prefix. (say, foo.so) The first problem is -l option does not find the file. So I tried directly including this file to the last compilation like this:
gcc a a.o /PATH/TO/FOO/foo.so
But in this case, a is hard linked to foo.so as an absolute path as seen in "ldd a":
/PATH/TO/FOO/foo.so
In the final deployment both files would end up being in the same folder, so this should be normal link, not the absolute path. How can I do this?
Assuming an ELF platform, if you can rebuild foo.so:
- the best fix is to simply name it libfoo.so
- the next best fix is to set SONAME on it:
gcc -Wl,-soname,foo.so -o foo.so foo.o
when you later link with:
gcc -o a.out a.o /path/to/foo.so
only the SONAME will be recorded as a dependency, not a full /path/to/foo.so.
If you can't rebuild foo.so, then do this:
rm -f foo.so && ln -s /path/to/foo.so foo.so &&
gcc -o a.out a.o ./foo.so && rm -f foo.so
-Wl,-rpath,. --> to use current directory for searching lib files. (even if not found in compilation, ok at run-time)
instead of -llibrary --> use library.so.
This seems to work correctly. Hope anyone finds this useful.