I am trying to run a very simple openmp fortran 90 code, using windows 7 and gfortran. Here is my code
PROGRAM HELLO
! USE omp_lib
IMPLICIT NONE
INTEGER OMP_GET_MAX_THREADS
INTEGER OMP_GET_NUM_THREADS
INTEGER OMP_GET_THREAD_NUM
write(6,"(a, i3)") " OpenMP max threads: ", OMP_GET_MAX_THREADS()
!$OMP PARALLEL
write(6,"(2(a,i3))") " OpenMP: N_threads = ", &
& OMP_GET_NUM_THREADS()," thread = ", OMP_GET_THREAD_NUM()
!$OMP END PARALLEL
END PROGRAM
And this is what I'm compiling with
gfortran -fopenmp -g -J"bin" test.f90 -o test
And this is what happens:
gfortran -fopenmp -g -J"bin" test.f90 -o test
gfortran.exe: error: libgomp.spec: No such file or directory
I believe I haven't set up my environment variables correctly, and I wanted to know if there is a way I can do this programmatically, or if anyone knows how they are supposed to be set up, assuming that is the problem. Any help is greatly appreciated.
Related
I'm compiling a Fortran code to obtain shared library .so. In the code it uses a module. The compiling has no problem while it reports undefined symbol error when opening the .so library.
Code mesh_map.F are as follows:
#include "cfx5ext.h"
dllexport(mesh_map)
SUBROUTINE mesh_map (CZ, DZ, IZ, LZ, RZ)
USE EXTRA_FLUID2
USE ISO_C_BINDING
USE IFPORT
IMPLICIT NONE
CHARACTER*(1) CZ(*)
DOUBLE PRECISION DZ(*)
INTEGER IZ(*)
LOGICAL LZ(*)
REAL RZ(*)
MAP_STATUS = 1
END
extra_fluid2.f90:
Module EXTRA_FLUID2
INTEGER :: map_status = 0
end module EXTRA_FLUID2
Compiling command:
ifort -c extra_fluid2.f90
/home/xxx/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/ifort -fpic -assume 2underscore -check uninit -warn declarations -diag-error 6717 -ftz -O2 -fp-speculation=safe -fp-model=precise -fp-model=source -fimf-arch-consistency=true -qno-opt-dynamic-align -fpe0 -fomit-frame-pointer -real-size 32 -integer-size 32 -I/usr/ansys_inc/v192/CFX/include -o linux-amd64/ifort/mesh_map.o -c mesh_map.F
-lrt/xxx/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/ifort -shared -o ./linux-amd64/ifort/libmesh_map.so linux-amd64/ifort/mesh_map.o extra_fluid2.o
When I check the .so library using:
ldd -r libmesh_map.so
The result shows as:
undefined symbol: extra_fluid2_mp_map_status__ (./libmesh_map.so)
How can I fix this? Thanks.
You must use -assume 2underscore in both compilations, using/not using the option cannot be mixed.
Also extra_fluid2.f90 should be compiled with option -fpic. It's generally a bad idea to use different sets of flags for files that are supposed to go into the same executable or shared object.
I compile a Fortran 90 code with mpif90 compiler with two different makefiles, the first one looks like;
FC = mpif90
FFLAGS = -Wall -ffree-line-length-none
FOPT = -O3
all: ParP2S.o ParP2S
ParP2S.o: ParP2S.f90
$(FC) $(FFLAGS) $(FOPT) ParP2S.f90 -c
ParP2S: ParP2S.o
$(FC) $(FFLAGS) $(FOPT) ParP2S.o -o ParP2S
clean:
rm -f *.o* rm -f *.o*
the second makefile looks very similar, I just added the -fopenmp flag;
FC = mpif90
FFLAGS = -Wall -ffree-line-length-none -fopenmp
FOPT = -O3
all: ParP2S.o ParP2S
ParP2S.o: ParP2S.f90
$(FC) $(FFLAGS) $(FOPT) ParP2S.f90 -c
ParP2S: ParP2S.o
$(FC) $(FFLAGS) $(FOPT) ParP2S.o -o ParP2S
clean:
rm -f *.o* rm -f *.o*
The second makefile is for a hybrid (MPI with OpenMP) version of the code. For now, I have exactly the same code but compiled with these different makefiles. In the second case, the code is more than 100 times slower. Any comments in what I am doing wrong?
edit 1: I am not running multi-threaded tasks. In fact, the code does not have any OpenMP directives, it is just the pure MPI code but compiled with a different makefile. Nevertheless, I did try running after the following commands (see below) and it didn't helped.
export MV2_ENABLE_AFFINITY=0
export OMP_NUM_THREADS=1
export OMP_PROC_BIND=true
mpirun -np 2 ./ParP2S
edit 2: I am using gcc version 4.9.2 (I know there was a bug with vectorization with fopenmp in an older version). I thought the inclusion of the -fopenmp flag could be inhibiting the compiler optimizations, however, after reading the interesting discussion (May compiler optimizations be inhibited by multi-threading?) I am not sure if this is the case. Furthermore, as my code does not have any OpenMP directives, I don't see why the code compiled with -fopenmp should be that slower.
edit3: When I run without -fopenmp (first makefile) it takes about 0.2 seconds without optimizations (-O0) and 0.08 seconds with optimizations (-O3), but including the flag -fopenmp it takes about 11 seconds with -O3 or -O0.
It turned out that the problem was really task affinity, as suggested by Vladimir F and Gilles Gouaillardet (thank you very much!).
First I realized I was running MPI with OpenMPI version 1.6.4 and not MVAPICH2, so the command export MV2_ENABLE_AFFINITY=0 has no real meaning here. Second, I was (presumably) taking care of the affinity of different OpenMP threads by setting
export OMP_PROC_BIND=true
export OMP_PLACES=cores
but I was not setting the correct bindings for the MPI processes, as I was incorrectly launching the application as
mpirun -np 2 ./Par2S
and it seems that, with OpenMPI version 1.6.4, a more appropriate way to do it is
mpirun -np 2 -bind-to-core -bycore -cpus-per-proc 2 ./hParP2S
The command -bind-to-core -bycore -cpus-per-proc 2 assures 2 cores for my application (see https://www.open-mpi.org/doc/v1.6/man1/mpirun.1.php and also Gilles Gouaillardet's comments on Ensure hybrid MPI / OpenMP runs each OpenMP thread on a different core). Without it, both MPI processes were going to one single core, which was the reason for the poor efficiency of the code when the flag -fopenmp was used in the Makefile.
Apparently, when running pure MPI code compiled without the -fopenmp flag different MPI processes go automatically to different cores, but with -fopenmp one needs to specify the bindings manually as described above.
As a matter of completeness, I should mention that there is no standard for setting the correct task affinity, so my solution will not work on e.g. MVAPICH2 or (possibly) different versions of OpenMPI. Furthermore, running nproc MPI processes with nthreads each in ncores CPUs would require e.g.
export OMP_PROC_BIND=true
export OMP_PLACES=cores
export OMP_NUM_THREADS=nthreads
mpirun -np nproc -bind-to-core -bycore -cpus-per-proc ncores ./hParP2S
where ncores=nproc*nthreads.
ps: my code has an MPI_all_to_all. The condition where more than one MPI process are on one single core (no hyperthreading) calling this subroutine should be the reason why the code was about 100 times slower.
I'm having a hard time debugging a compilation error in gfortran. My compiler is:
gfortran -v
Thread model: posix
gcc version 4.9.2 (x86_64-posix-seh-rev4, Built by MinGW-W64 project)
I can't include the real problem because it's too big. Below I'll discuss a model problem of what's happening:
I have one big module, let's call it ops, that I'd like to break down into several modules (e.g. add, subtract, multiply, divide, etc). To do this, I "use" modules "add" and "subtract" in module "ops", make the routines from "add" and "subtract" public, and then call the routines in "add" and "subtract" from the main program, which only uses "ops".
In this simple example (illustrated below in a simple program), using both modules works fine, and there are no errors. But when I do this with my big program, I get an error when trying to use the "subtract" module (I've also included what I mean in the program comments).
Here's the sample of the scenario:
module add_mod
private
public :: add
interface add; module procedure add1; end interface
contains
subroutine add1(a,b,c)
integer,intent(inout) :: a
integer,intent(in) :: b,c
a = b + c
end subroutine
end module
module subtract_mod
private
public :: subtract
interface subtract; module procedure subtract1; end interface
contains
subroutine subtract1(a,b,c)
integer,intent(inout) :: a
integer,intent(in) :: b,c
a = b - c
end subroutine
end module
module ops
use add_mod
! use subtract_mod ! Removing comment causes error...
private
public :: init
public :: add
! public :: subtract ! Removing comment causes error...
contains
subroutine init(a,b,c)
implicit none
integer,intent(inout) :: a,b,c
a = 0; b = 0; c = 0
end subroutine
end module
program test
use ops
implicit none
integer :: a,b,c
b = 1; c = 1
call add(a,b,c)
write(*,*) 'a,b,c = ',a,b,c
! call subtract(a,b,c) ! want to call...
write(*,*) 'a,b,c = ',a,b,c
end program
The error that I'm getting (in terms of the simplified example) is:
\add.o
\subtract.o
\subs.o
\main.o
gfortran: error: main.o: No such file or directory
make: *** [main] Error 1
The error from my big example looks like
C:\\Users\\charl\\Documents\\obj\parametricStudy.o
gfortran: error: C:\\Users\\charl\\Documnts\\obj\parametricStudy.o: No such file or directory
make: *** [C:\\Users\\charl\\Documents\\MOONS] Error 1
Where "parametricStudy" is the main program and "MOONS" is the name of the target/executable.
This really does not look like an error in terms of finding the executable, it looks like the executable is not made because of some other issue, but I don't have enough information to find out why. I've tried including all of the debugging flags I can think of:
fimplicit-none -Wuninitialized -cpp -fopenmp -Wall -Wextra -fbacktrace -fcheck=all -O0 -Og
Sorry for the long introduction, but here's my question: How can I debug this? I'm sorry if this was a poorly conveyed question, but I can't re-produce this error in a simple case, so I'm not sure how else to pose the question. Any help on a solution, debugging advice or improving this question is greatly appreciated!
UPDATE:
After seeing the comment about the compilation error typo, I tried reversing the order of the used modules. That is, now I tried using "subtract" alone, which works fine, but then try using "add" (together) and I get the same error. I'm now beginning to think that this is a compiler bug or something.
UPDATE 2:
I uploaded the code to github. It can be downloaded at
https://github.com/charliekawczynski/MOONS
The makefile is in the directory
\MOONS\makefiles\MOONS
And the output should show up in out_dir. To compile this, the $TARGET_DIR$ and $SRC_DIR$ must be defined. Please note that right now it is defined with the variable $USER$.
After changing the directory, I see a different misspelling (which seems to change when I make small changes in the code, e.g. removing some warnings related to unused variables).
UPDATE 3:
I've been removing more warnings related to un-used routines and variables, and now I see that the error still exists, but (as far as I see) there is no typo. Here is the new output:
gfortran -o
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\MOONS -J"
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\mod" -fimplicit-none -Wuninitialized -cpp -D_DOUBLE_PRECISION_ -D_FFT_RADIX2_ -fopenmp -Wall -Wextra -fbacktrace -fcheck=all -O0 -Og -D_DEBUG_DEL_ -D_DEBUG_INTERP_ -D_DEBUG_APPLYBCS_
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\IO_tools.o
...
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\init_Ufield.o
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\init_Pfield.o
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\energySolver.o
...
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\MOONS.o
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\richardsonExtrapolation.o
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\convergenceRate.o
C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\parametricStudy.o
gfortran: error: C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\obj\init_Pfield.: No such file or directory
make: *** [C:\\Users\\charl\\Documents\\GitHub\\MOONS\\out_dir\\MOONS] Error 1
C:\Users\charl\Documents\GitHub\MOONS\makefiles\MOONS>
I'm going to try a newer compiler and see if this fixes the problem
I used cpu_time, but apparently that gives the total time for all threads. I used omp_get_wtime, but get an output in the negatives which is not correct, and also mpi_wtime for which I am now getting a core dump (and for which earlier I was getting just 0.000000000). The relevant code is as follows:
real*8 tbeg, tend
....
!$omp sections private (ie, tbeg, tend)
!$omp section
tbeg = omp_get_wtime()
do ie=1, E
call rmul(u, A, B, dudr, duds, dudt, ie)
enddo
tend = omp_get_wtime()
!Step 4: Print results
print *, tend-tbeg
!$omp end section
!$omp section
....
!$omp end section
!$omp end sections
My compile option is:
gfortran -Ofast -c mult.f -o mult.o -mcmodel=large -I/usr/lib/openmpi/include -fopenmp
gfortran -o baseline ../lib/performance_test.o mult.o ../lib/rose.o -lcuda -lcudart -L/usr/local/cuda-5.0/lib64 -lcublas -lgomp -lmpi_f77
I finally managed to reproduce your issue (with some difficulties, but I've got it). And I'm pretty sure that the bottom line is that you forgot two things in your code:
To include either the OpenMP header include 'omp_lib.h' or better the OpenMP module use omp_lib
To forbid implicit variables declaration implicit none
Although the latter isn't strictly speaking an error, it's definitely a good habit to take and which would have spare you the actual issue coming from the former, since you would have had the following message from the compiler:
tbeg = omp_get_wtime()
1 Error: Function 'omp_get_wtime' at (1) has no IMPLICIT type
So what happened is that you implicitly declared omp_get_wtime as a function returning a single precision floating point variable whereas it actually returns a double precision one. So the return value was truncated and you were having garbage.
Just add the right header and use omp_get_wtime() as you have in you code snippet, and everything should be all-right.
For some reason, I cant make OpenMP work with CodeBlocks and GFortran. I tried another IDE (Geany) and it works properly, but with CodeBlocks it does not. I must say that I included the -fopenmp option in CodeBlocks.
Here is a simple code that I tested.
program test
implicit none
integer i
!$OMP PARALLEL DO DEFAULT(PRIVATE)
Do i=1,10
Write(*,*) i
end do
!$OMP END PARALLEL DO
end program test
And here is the build log:
-------------- Build: Debug in TEST (compiler: GNU Fortran Compiler)---------------
gfortran.exe -Jobj\Debug\ -Wall -g -fopenmp -c "C:\Users\tilter\Dropbox\Public\Code\Nova pasta\TEST\main.f95" -o obj\Debug\main.o
gfortran.exe -o bin\Debug\TEST.exe obj\Debug\main.o
obj\Debug\main.o: In function `test':
C:/Users/tilter/Dropbox/Public/Code/Nova pasta/TEST/main.f95:5: undefined reference to `GOMP_parallel_start'
C:/Users/tilter/Dropbox/Public/Code/Nova pasta/TEST/main.f95:5: undefined reference to `GOMP_parallel_end'
obj\Debug\main.o:C:/Users/tilter/Dropbox/Public/Code/Nova pasta/TEST/main.f95:6: undefined reference to `GOMP_parallel_start'
obj\Debug\main.o:C:/Users/tilter/Dropbox/Public/Code/Nova pasta/TEST/main.f95:6: undefined reference to `GOMP_parallel_end'
obj\Debug\main.o:C:/Users/tilter/Dropbox/Public/Code/Nova pasta/TEST/main.f95:8: undefined reference to `omp_get_num_threads'
obj\Debug\main.o:C:/Users/tilter/Dropbox/Public/Code/Nova pasta/TEST/main.f95:8: undefined reference to `omp_get_thread_num'
collect2: ld returned 1 exit status
Process terminated with status 1 (0 minutes, 0 seconds)
6 errors, 0 warnings (0 minutes, 0 seconds)
I don't know what I'm missing here...
I just restarted Windows 7 and it fixed the problem.