Automatic dependency detection not working in GFortran - makefile

I the GCC Wiki it is stated that support for auto-detection of dependencies has been available since version 4.6:
Support the generation of Makefile dependencies via the -M... flags of GCC; you may need to specify additionally the -cpp option. The dependencies take modules, Fortran's include, and CPP's #include into account. Note: Using -M for the module path is no longer supported, use -J instead.
In my program I have two Fortran files: module_1.f08 and main.f08, where main uses module_1. I am using the following command to try to auto-detect dependencies of main:
gfortran -cpp -M main.f08
If module_1 has been already compiled, the command above returns a list of dependencies as expected, though if module_1 has not been compiled yet, I get an error message instead saying that module_1.mod does not exist.
The way I'm seeing this is that every time a new module is added to the code, it has to be compiled separately before running make all (or we might run make all before using the module in any other file, then use the module and compile again) or else any dependency of it might be compiled before the module itself and a compilation error will be returned.
Another thing is that dependency files have to be created gradually and one-by-one as the project grows, and if .mod files and dependency files got deleted at some point (with a make clean command for example), there will be no way to generate dependency files all at once using the auto-detection feature.
Is there a way to get around these limitations? Is there a way for auto-detection to work even if .mod files do not exist yet?

To start with, you need to add snippets to your Makefile to actually use the dependency generation feature. Additionally, you can use the -MD option to generate dependency files automatically for each target, so you don't need a special target to regenerate your dependencies. For an example project like yours above with a main.f90 that uses a module defined in mod1.f90 a simple Makefile using dependencies could look like:
FC := gfortran
FFLAGS := -O2 -g
LIBS := # Needed libs like -lopenblas
SRCS := mod1.f90 main.f90
OBJS := ${SRCS:f90=o}
DEPS := ${OBJS:o=d}
myprog: $(OBJS)
$(FC) $(FFLAGS) -o $# $^ $(LIBS)
.PHONY: clean
clean:
-rm -f $(OBJS) *.mod
-include $(DEPS)
%.o: %.f90
$(FC) $(FFLAGS) -cpp -MD -c -o $# $<
When you run make you'll see that it generates files main.d and mod1.d containing the dependencies for the corresponding source file.
A (minor?) problem here is that your SRCS variable containing your list of source files must be listed in an order that allows the files to be compiled from left to right before you have any .d files. So the dependency stuff as it's done here doesn't help with ordering a build before the .d files have been generated. (Thus I'd recommend distributing the .d files as part of the source package.)

Related

REPAST HPC WSL Installation leads to boost no being found

When trying to run Repast HPC executables I get the following error:
./main.exe: error while loading shared libraries: libboost_mpi-mt.so.1.61.0: cannot open shared object file: No such file or directory ./main.exe: error while loading shared libraries: libboost_mpi-mt.so.1.61.0: cannot open shared object
In the make file I define the link to boost in an env file with the below:
BOOST_INCLUDE= -I$(HOME)/sfw/Boost/Boost_1.61/include/
BOOST_LIB_DIR= -L$(HOME)/sfw/Boost/Boost_1.61/lib/
This is linked to a make file with the following make file
include ./env
.PHONY: clean
clean:
rm -f *.o
rm -f *.exe
.PHONY: compile
compile:
$(MPICXX) $(BOOST_INCLUDE) -c Main.cpp -o Main.o
$(MPICXX) $(BOOST_LIB_DIR) -o main.exe Main.o $(BOOST_LIBS)
.PHONY: all
all: clean compile
Boost in tsnow#DESKTOP-IF7CEHL:~/sfw/Boost/Boost_1.61/lib$ so it should link well across.
Any help appreciated, just trying to get a tutorial running.
We typically don't link repast to boost, but rather use rpath to let the model executable know where the boost and repast libraries are. For example in your Makefile:
REPAST_LIB_DIR = $(REPAST_HOME)/lib
BOOST_LIB_DIR = $(HOME)/theta/sfw/boost-1.66.0/lib
RPATHS += -Wl,-rpath -Wl,$(REPAST_LIB_DIR) -Wl,-rpath -Wl,$(BOOST_LIB_DIR)
and then add $(RPATHS) after the list of libraries during linking.
model : $(EXEC_OBJECTS)
$(CXXLD) -fopenmp -dynamic $^ $(LIBS) $(RPATHS) -o $(NAME)
Some of those args might not be appropriate for WSL, but hopefully the RPATHS part makes sense.
So, if after your include directive, you add
RPATHS=-Wl,-rpath -Wl,$(BOOST_LIB_DIR)
and update the second line of your compile target with
$(MPICXX) $(BOOST_LIB_DIR) -o main.exe Main.o $(BOOST_LIBS) $(RPATHS)
The location of your boost libs is compiled into main.exe, and it should find it.
The command:
export LD_LIBRARY_PATH=/home/tsnow/sfw/Boost/Boost_1.61/lib/
allowed boost to be found.

Makefile library prerequisite

I have a makefile, which I am using to cross-compile for and embeded ARM platform with gcc. Specifcally, I am using arm-none-eabi-gcc, but the same appiles to avr-gcc, msp430-gcc, etc. Typically when using make+gcc (and not cross compiling) I list libs as prerequisite as follows:
programA.elf: programA.o foo.o -lm ...etc
programB.elf: programB.o bar.o -lftdi ...etc
%.elf:
gcc $(LDFLAGS) -o $# $^
Make handles this "-lsyntax" very nicely, and its very convienient if you are building multiple progams/targets and want to have a generic rule for linking. The problem I have run into durring cross-compiling is that arm-none-eabi-gcc obviously has a different libm.a than my system's gcc libm.so (for example), but Make doesn't know whats going on here and keeps trying to use the x86 libm instead of the ARM base one. I can get things to work by adding the line:
.LIBPATTERNS = /usr/lib/arm-none-eabi/newlib/lib%.a
but it seems kinda clunky and exposes anyone wanting to compile the project to knowing a little more about the toolchain's install locations than is normally expected.
My question is: "Is there a better convention to list a binary's lib dependencies I should be using here that wont break when cross-compiling?"
This can be done. But a general solution is complex. I have Makefiles which build arm, x86, and c67 executables from a single set of sources. The page you reference eludes to the key: VPATH. I suggest a separate subdirectory for each architecture. The following is not working code, but it gives the idea
all: arm/pgma x86/pgma
vpath %.c $(CURDIR)
arm x86:
mkdir -p $#
arm/pgma: arm/main.o arm/sub.o | arm
x86/pgma: x86/main.o x86/sub.o more.o | x86
arm/%: CC=arm-none-eabi-gcc
arm/%: CFLAGS += -march=armv7-a -mtune=corex-a8
x86/%: CC=gcc
arm/%: VPATH = /usr/lib/arm-none-eabi/newlib
# Notice, VPATH not needed for x86 since it is the native host
This entire concept can be extend to build dependency file is each subdirectory as well debug and release variants. I have not tried this with the -lfoo, but it should work. E.g.,
arm/pgma: arm/main.o arm/sub.o -lmylib | arm

How to create a non-transitive dependency list (GCC)

Is there a way to generate dependencies of a C(++) source file similar to using the -MM option of GCC that only include the direct dependencies of said file, i.e. only the files directly included by this source file?
More context on why I'm looking for this functionality - maybe there is a completely different solution to my problem: I have a generic makefile with auto-detection of dependencies that suffices my needs but is slow. The basic structure is as follows:
Full dependencies of main.cpp are retrieved with gcc -MM
All *.h dependencies for which a corresonding *.cpp exists are changed to *.o dependencies
the altered dependencies are included in the makefile
All *.o targets are built, dependencies are retrieved with gcc -MM and included
All *.o targets are linked to create the executable
So far, this makefile has worked fine but -as said before- it is slow. I analyzed its execution path for one project and included all the generated dependencies by hand to try and optimize its speed. The result was by removing all transitive dependencies, the makefile retained its functionality but got much faster (also reflected in the number of lines of the debug output of make -d).
First of all, the method you are using is slightly confusing. All .h files used in the compilation of one .cpp file must be kept in its dependencies, and the automatic collection of *.o files shouldn't be that slow. I'd advise to go with classic -MM and to build the list of cpp files that are to be compiled by hand. automake does it that way, and if there was a really reliable way of figuring out the list of compilation units automatically, these guys would have found it :-).
Nevertheless, the -H option of the gcc helps you. It prints the names of all used files to stderr, with . prefixed for the level of inclusion. So,
cpp -I $< >/dev/null | sed -n -e 's/^\. //p'
should do the trick. Caveat: If a header file is included deeper in the hierarchy first and then later in the main file, it is not found. However, your dependency system should be able to handle that if you keep the 1-to-1 of .h and .cpp files.

How to do a Makefile using a specific libraries in different paths?

The issue I'm having is that I need to compile my code using specific libraries that are in different path locations. I need to use -lncurses library from ./ramdisk/libs path, the problem is that this directory also cointains a version of lthr library that I don't want to be linked. The makefile is pulling both libraries from the same location which is not what I want. I can't change the contents of these library directories in the filesystem, so I need to find a way to tell the Makefile to link lncurses library from path A and link lthr library from path B instead using the lthr from path A.
Any suggestions?
CC=icc
NCE=-L./ramdisk/libs
CFLAGS+=-I$(ROOTDIR)/../../include
#LDFLAGS=-static -lthr
$(DESTDIR)/nce: nce
mkdir -p $(DESTDIR)
$(INSTALL) -m 777 nce $(DESTDIR)
nce: nce.c
$(CC) $(CFLAGS) nce.c $(LDFLAGS) -o nce -lthr $(NCE) -lncurses
You can (probably) bypass the search by giving the full path to the library archive. So instead of specifying -lncurses, you might try ./ramdisk/libs/libncurses.a (or whatever). You didn't specify whether it was a shared lib or not, and I'm not entirely sure that this works for shared libraries, but probably worth a try.
[edit]
Since this is a shared lib issues, maybe something like:
CC=icc
THR=/full/path/to/wherever/libthr/lives
NCE=/full/path/to/ramdisk/libs
CFLAGS+=-I$(ROOTDIR)/../../include
LDFLAGS=-static
nce: nce.c
$(CC) $(CFLAGS) nce.c $(LDFLAGS) -o nce -L$(THR) -W,-rpath=$(THR) -lthr -L$(NCE) -W,-rpath=$(NCE) -lncurses
I'm kind of shooting in the dark here as I'm not familiar with icc, but the idea is to make sure the linker puts thr's path on the runtime linker's search path before the one on the ramdisk so that thr gets found there first.
You could copy the remote library to a local working directory.
ncurses would source from one location while thr would source from another.

multi package makefile example for go

I'm trying to setup a multi package go project something like
./main.go
./subpackage1/sub1_1.go
./subpackage1/sub1_2.go
./subpackage2/sub2_1.go
./subpackage2/sub2_2.go
where main.go imports both subpackage1 and subpackage2. And subpackage2 imports subpackage1.
Ive been looking around for go makefile examples but I can't find anything that supports this kind of set-up. Any idea?
Install godag then run:
gd -o myapp
It will automatically build a Directed Acyclic Graph (DAG) of all the dependencies in your src/ directory, then compile and link each package in the proper order.
Much easier than manually maintaining a Makefile, especially since $(GOROOT)/src/Make.* has changed in recent versions of Go (there is no longer a Make.$(GOARCH)). Also useful:
gd clean removes object files.
gd -test runs your automated tests (see testing package).
gd -dot=myapp.dot generates a graph of your package imports you can visualize using GraphViz.
Something like this should work
# Makefile
include $(GOROOT)/src/Make.$(GOARCH)
all:main
main:main.$O
$(LD) -Lsubpackage1/_obj -Lsubpackage2/_obj -o $# $^
%.$O:%.go subpackage1 subpackage2
$(GC) -Isubpackage1/_obj -Isubpackage2/_obj -o $# $^
subpackage1:
$(MAKE) -C subpackage1
subpackage2:
$(MAKE) -C subpackage2
.PHONY:subpackage1 subpackage2
# subpackage1/Makefile
TARG=subpackage1
GOFILES=sub1_1.go sub1_2.go
include $(GOROOT)/src/Make.$(GOARCH)
include $(GOROOT)/src/Make.pkg
# subpackage2/Makefile
TARG=subpackage2
GOFILES=sub2_1.go sub2_2.go
include $(GOROOT)/src/Make.$(GOARCH)
include $(GOROOT)/src/Make.pkg
GC+=-I../subpackage1/_obj
LD+=-L../subpackage1/_obj
sub2_1.$O sub2_2.$O:subpackage1
subpackage1:
$(MAKE) -C ../subpackage1
.PHONY:subpackage1
If you don't install the subpackages you need to explicitly set the include path. The supplied makefile Make.pkg is mainly to build packages, which is why it's only included in the subpackage makefile.
hello world with a Makefile and a test (Googles Groupes : golang-nuts)
Check out https://github.com/banthar/Go-SDL which is an actively maintained multi-package go project using Makefiles.
I notice some of these the answers use the obsolete Make.$(GOARCH) include. So hopefully the above link will be stabler than trying to stay on top of Google's changing API in an answer here.

Resources