Compiling a fortran90 files with different parameters each time - shell

I am recently working on a fortran90 program which calculate the time needed and result of some mathematics calculation. Here is the code:
program loops
use omp_lib
implicit none
integer, parameter :: N=729
integer, parameter :: reps=1000
real(kind=8), allocatable :: a(:,:), b(:,:), c(:)
integer :: jmax(N)
real(kind=8) :: start1,start2,end1,end2
integer :: r
allocate(a(N,N), b(N,N), c(N))
call init1()
start1 = omp_get_wtime()
do r = 1,reps
call loop1()
end do
end1 = omp_get_wtime()
call valid1();
print *, "Total time for ",reps," reps of loop 1 = ", end1-start1
call init2()
start2 = omp_get_wtime()
do r = 1,reps
call loop2()
end do
end2 = omp_get_wtime()
call valid2();
print *, "Total time for ",reps," reps of loop 2 = ", end2-start2
contains
subroutine init1()
implicit none
integer :: i,j
do i = 1,N
do j = 1,N
a(j,i) = 0.0
b(j,i) = 3.142*(i+j)
end do
end do
end subroutine init1
subroutine init2()
implicit none
integer :: i,j,expr
do i = 1,N
expr = mod(i,3*(i/30)+1)
if (expr == 0) then
jmax(i) = N
else
jmax(i) = 1
end if
c(i) = 0.0
end do
do i = 1,N
do j = 1,N
b(j,i) = dble(i*j+1)/dble(N*N)
end do
end do
end subroutine init2
subroutine loop1()
implicit none
integer :: i,j
!$OMP PARALLEL DO DEFAULT(NONE), PRIVATE(i,j), SHARED(a,b), SCHEDULE(type,chunksize)
do i = 1,N
do j = N,i,-1
a(j,i) = a(j,i) + cos(b(j,i))
end do
end do
!$OMP END PARALLEL DO
end subroutine loop1
subroutine loop2()
implicit none
integer :: i,j,k
real (kind=8) :: rN2
rN2 = 1.0 / dble (N*N)
!$OMP PARALLEL DO DEFAULT(NONE), PRIVATE(i,j,k), SHARED(rN2,c,b,jmax), SCHEDULE(type,chunksize)
do i = 1,N
do j = 1, jmax(i)
do k = 1,j
c(i) = c(i) + k * log(b(j,i)) *rN2
end do
end do
end do
!$OMP END PARALLEL DO
end subroutine loop2
subroutine valid1()
implicit none
integer :: i,j
real (kind=8) :: suma
suma= 0.0
do i = 1,N
do j = 1,N
suma = suma + a(j,i)
end do
end do
print *, "Loop 1 check: Sum of a is ", suma
end subroutine valid1
subroutine valid2()
implicit none
integer i
real (kind=8) sumc
sumc= 0.0
do i = 1,N
sumc = sumc + c(i)
end do
print *, "Loop 2 check: Sum of c is ", sumc
end subroutine valid2
end program loops
In the line !$OMP PARALLEL DO DEFAULT(NONE), PRIVATE(i,j), SHARED(a,b), SCHEDULE(type,chunksize) and !$OMP PARALLEL DO DEFAULT(NONE), PRIVATE(i,j,k), SHARED(rN2,c,b,jmax), SCHEDULE(type,chunksize).
As I want to perform the task of different schedule case to see the different results, so I need to change this part SCHEDULE(type,chunksize), with different schedule type and different chunksize. For example, in this case, the schedule type is static and chunksize is 1.
Say if I have type of (static, a, b, c) and chunksize (1,2,3,4,5,6,7). As I am new to fortran so I wonder is it possible to compile and run the code for all case in once without the fact that I have to change the parameters manually everytime, i.e it compiles and runs to give the result of first case e.g (static,1), it then compiles and runs the file again but with the parameters changed automatically that gives another result. For instance, (static,2)...(b,4) etc.
I heard that we can create a script file to perform such task, but I not am sure what exactly I need to do for this.
Thank you so much.

You may want to investigate the use of the preprocessor. I'm speaking from experience with gfortran, but I believe this applies (almost) all other compilers as well even though it is outside the scope of the Fortran standard.
If you name your source file with a capital F in the suffix, i.e. file.F, file.F90, file.F95 etc, your file will be preprocessed with the C preprocessor before being compiled. That may sound complicated, but cutting this down to what you need, this means that if you compile your code with a command like
$ gfortran -DCHUNK_SIZE=1 mySource.F90
then all occurrences of CHUNK_SIZE (with qualifiers which are not essential to your problem) will be replaced by 1. More technically, CHUNK_SIZE becomes a macro defined to expand to 1. So if you replace SCHEDULE(type,chunksize) with SCHEDULE(type,CHUNK_SIZE) in your source file, you can repeatedly invoke the compiler with different values, -DCHUNK_SIZE=1, -DCHUNK_SIZE=2 etc, and get the result that you described. The same can be done for type.
Now you may want to change the function names accordingly as well. One way would be to add a few preprocessor statements near the top of your file declaring a few macros, namely
#ifdef __GFORTRAN__
#define PASTE2(a,b) a/**/b
#define FUNC_NAME_WITH_CHUNK_SIZE(fn) PASTE2(PASTE2(fn,_),CHUNK_SIZE)
#else
#define FUNC_NAME_WITH_CHUNK_SIZE(fn) fn ## _ ## CHUNK_SIZE
#endif
#define LOOP1 FUNC_NAME_WITH_CHUNK_SIZE(loop1)
#define LOOP2 FUNC_NAME_WITH_CHUNK_SIZE(loop2)
and replace loop1 with LOOP1 etc. You could do this from the command line as before, but since these rules are not supposed to change between compilations, it makes sense to keep these in the source file. I think the only part that is not self-explanatory is the use of ## and /**/ between #ifdef and #endif. This is how one does string concatenation with the preprocessor, and because gfortran uses the way C preprocessors did it before the language was standardized, it gets exceptional treatment, see e.g. this answer for some info on these operators. The purpose of this operation is to replace LOOP1 with loop1_<CHUNK_SIZE>, where <CHUNK_SIZE> is filled in from the command line. Feel free to follow any other conventions for naming these functions.
If you want to call these functions from another translation unit, you will have to process the function names in the same way, of course. In order to make your life easier, you may want to research the #include statement. Detailing this would take us too far here, but the idea is that you put all your includes into a file (conventionally named <something>.inc in the Fortran-world with <something> replaced that makes sense to you) and use #include "<something>.inc in all source files to obtain the same macro definitions.

Related

OpenACC constant parameters

I am wondering what is the proper way to handle constants in OpenACC kernels.
For example, in the following code
module vecaddmod
implicit none
integer, parameter :: n = 100000
!$acc declare create(n)
contains
subroutine vecaddgpu(r, a, b)
real, dimension(:) :: r, a, b
integer :: i
!$acc update self(n)
!$acc data present(n)
!$acc kernels loop copyin(a(1:n),b(1:n)) copyout(r(1:n))
do i = 1, n
r(i) = a(i) + b(i)
enddo
!$acc end data
end subroutine vecaddgpu
end module vecaddmod
program main
use vecaddmod
implicit none
integer :: i, errs, argcount
real, dimension(:), allocatable :: a, b, r, e
character*10 :: arg1
allocate( a(n), b(n), r(n), e(n) )
do i = 1, n
a(i) = i
b(i) = 1000*i
enddo
! compute on the GPU
call vecaddgpu( r, a, b )
! compute on the host to compare
do i = 1, n
e(i) = a(i) + b(i)
enddo
! compare results
errs = 0
do i = 1, n
if( r(i) /= e(i) )then
errs = errs + 1
endif
enddo
print *, errs, ' errors found'
if( errs ) call exit(errs)
end program main
n is declared as a constant on CPU in a module, and it is used as the range in the loop. nvfortran warns me about Constant or Parameter used in data clause. Is the above example the proper way to handle this? Can I take advantage of the constant memory on GPU, such that I don't need to copy it from CPU to GPU for each kernel launch?
Thanks.
The compiler will replace parameters with the literal value so no need to put them in data regions.
module vecaddmod
implicit none
integer, parameter :: n = 100000
contains
subroutine vecaddgpu(r, a, b)
real, dimension(:) :: r, a, b
integer :: i
!$acc kernels loop copyin(a(1:n),b(1:n)) copyout(r(1:n))
do i = 1, n
r(i) = a(i) + b(i)
enddo
end subroutine vecaddgpu
end module vecaddmod
...
% nvfortran -acc -Minfo=accel test.f90
vecaddgpu:
11, Generating copyin(a(:100000)) << "n" is replaced with 100000
Generating copyout(r(:100000))
Generating copyin(b(:100000))
12, Loop is parallelizable
Generating Tesla code
12, !$acc loop gang, vector(128) ! blockidx%x threadidx%x

Disrepancy in results between OpenMP/OpenACC implementation and gcc/PGI compilers

I have a larger Fortran program that I am trying to convert so that the computationally intensive part will run on an NVidia GPU using OpenMP and/or OpenACC. During development I had some issues to understand how variables declared in a module can be used within subroutines that are executed on the GPU (and some of them also on the CPU). Therefore, I created a small example and worked on that, by experimenting and adding the corresponding OpenMP and OpenACC directives. I have included the three files that comprise my example at the end of this message.
Just as I thought that I had understood things and that my example program works, I noticed the following:
I compile the program with gcc 10.2 using the OpenMP directives:
gfortran -O3 -fopenmp -Wall -Wextra test_link.f90 parameters.f90 common_vars.f90 -o test_link
The results are as expected, i.e. all elements of array XMO are 1, of DCP are 2, of IS1 are 3 and of IS2 are 24.
I compile the program with PGI compiler 19.10 community edition using the OpenACC directives:
pgfortran -O4 -acc -ta=tesla,cc35 -Minfo=all,mp,accel -Mcuda=cuda10.0 test_link.f90 common_vars.f90 parameters.f90 -o test_link
The results are the same as above.
I compile the program with gcc 10.2 using the OpenACC directives:
gfortran -O3 -fopenacc -Wall -Wextra test_link.f90 parameters.f90 common_vars.f90 -o test_link
The results for arrays XMO, DCP and IS1 are correct, but all elements of IS2 are 0. It is easy to verify that variable NR has a value of 0 to get this result.
My understanding is that the OpenMP and OpenACC version of my example are equivalent, but I cannot figure out why the OpenACC version works only for the PGI compiler and not for gcc.
If possible, please provide solutions that do not require changes in the code but only in the directives. As I mentioned, my original code is much larger, contains many more module variables and calls many more subroutines in the code to be executed on the GPU. Changes in that code will be much more difficult to do and obviously I would prefer to do that only if really necessary.
Thank you in advance!
The files of my example follow.
File parameters.f90
MODULE PARAMETERS
IMPLICIT NONE
INTEGER, PARAMETER :: MAX_SOURCE_POSITIONS = 100
END MODULE PARAMETERS
File common_vars.f90
MODULE COMMON_VARS
USE PARAMETERS
IMPLICIT NONE
!$OMP DECLARE TARGET TO(NR)
INTEGER :: NR
!$ACC DECLARE COPYIN(NR)
END MODULE COMMON_VARS
File test_link.f90
SUBROUTINE TEST()
USE COMMON_VARS
IMPLICIT NONE
!$OMP DECLARE TARGET
!$ACC ROUTINE SEQ
INTEGER I
I = NR
END SUBROUTINE TEST
PROGRAM TEST_LINK
USE COMMON_VARS
USE PARAMETERS
IMPLICIT NONE
INTERFACE
SUBROUTINE TEST()
!$OMP DECLARE TARGET
!$ACC ROUTINE SEQ
END SUBROUTINE TEST
END INTERFACE
REAL :: XMO(MAX_SOURCE_POSITIONS), DCP(MAX_SOURCE_POSITIONS)
INTEGER :: IS1(MAX_SOURCE_POSITIONS), IS2(MAX_SOURCE_POSITIONS)
INTEGER :: X, Y, Z, MAX_X, MAX_Y, MAX_Z, ISOUR
MAX_X = 3
MAX_Y = 4
MAX_Z = 5
NR = 6
!$OMP TARGET UPDATE TO(NR)
!$OMP TARGET MAP(TOFROM:IS1,IS2,DCP,XMO)
!$OMP TEAMS DISTRIBUTE PARALLEL DO COLLAPSE(3)
!$ACC UPDATE DEVICE(NR)
!$ACC PARALLEL LOOP GANG WORKER COLLAPSE(3) INDEPENDENT &
!$ACC COPY(IS1,IS2,DCP,XMO)
DO X = 1, MAX_X
DO Y = 1, MAX_Y
DO Z = 1, MAX_Z
ISOUR = (X - 1)*MAX_Y*MAX_Z + (Y - 1)*MAX_Z + Z
XMO(ISOUR) = 1.0
DCP(ISOUR) = 2.0
IS1(ISOUR) = 3
IS2(ISOUR) = 4 * NR
CALL TEST()
ENDDO ! End of z loop
ENDDO ! End of y loop
ENDDO ! End of x loop
!$ACC END PARALLEL LOOP
!$OMP END TEAMS DISTRIBUTE PARALLEL DO
!$OMP END TARGET
DO X = 1, MAX_X
DO Y = 1, MAX_Y
DO Z = 1, MAX_Z
ISOUR = (X - 1)*MAX_Y*MAX_Z + (Y - 1)*MAX_Z + Z
WRITE(*, *) 'ISOUR = ', ISOUR, 'XMO = ', XMO(ISOUR), 'DCP = ', DCP(ISOUR), 'IS1 = ', IS1(ISOUR), 'IS2 = ', IS2(ISOUR)
ENDDO ! End of z loop
ENDDO ! End of y loop
ENDDO ! End of x loop
END PROGRAM TEST_LINK

OpenMP FFTW with Fortran not thread safe

I am trying to use the FFTW with openMP and Fortran, but I get wrong results when executing in parallel, which also change their values every execution step, displaying typical behaviour when parallelisation goes wrong.
I am aiming for a simple 3d real-to-complex transformation. Following the FFTW tutorial, I took all but the call to dfftw_execute_dft_r2c() out of the parallel region, but it doesn't seem to work.
I use FFTW 3.3.8, configured with ./configure --enable-threads --enable-openmp --enable-mpi and compile my code with gfortran program.f03 -o program.o -I/usr/include -fopenmp -lfftw3_omp -lfftw3 -g -Wall.
This is how my program looks like:
program use_fftw
use,intrinsic :: iso_c_binding
use omp_lib
implicit none
include 'fftw3.f03'
integer, parameter :: dp=kind(1.0d0)
integer, parameter :: Nx = 10
integer, parameter :: Ny = 5
integer, parameter :: Nz = 5
real(dp), parameter :: pi = 3.1415926d0
real(dp), parameter :: physical_length_x = 20.d0
real(dp), parameter :: physical_length_y = 10.d0
real(dp), parameter :: physical_length_z = 10.d0
real(dp), parameter :: lambda1 = 0.5d0
real(dp), parameter :: lambda2 = 0.7d0
real(dp), parameter :: lambda3 = 0.9d0
real(dp), parameter :: dx = physical_length_x/real(Nx,dp)
real(dp), parameter :: dy = physical_length_y/real(Ny,dp)
real(dp), parameter :: dz = physical_length_z/real(Nz,dp)
integer :: void, nthreads
integer :: i, j, k
real(dp):: d
complex(dp), allocatable, dimension(:,:,:) :: arr_out
real(dp), allocatable, dimension(:,:,:) :: arr_in
integer*8 :: plan_forward
allocate(arr_in( 1:Nx, 1:Ny, 1:Nz)); arr_in = 0
allocate(arr_out(1:Nx/2+1, 1:Ny, 1:Nz)); arr_out = 0
!------------------------------
! Initialize fftw stuff
!------------------------------
! Call before any FFTW routine is called outside of parallel region
void = fftw_init_threads()
if (void==0) then
write(*,*) "Error in fftw_init_threads, quitting"
stop
endif
nthreads = omp_get_num_threads()
call fftw_plan_with_nthreads(nthreads)
! plan execution is thread-safe, but plan creation and destruction are not:
! you should create/destroy plans only from a single thread
call dfftw_plan_dft_r2c_3d(plan_forward, Nx, Ny, Nz, arr_in, arr_out, FFTW_ESTIMATE)
!--------------------------------
! Start parallel region
!--------------------------------
!$OMP PARALLEL PRIVATE( j, k, d)
! Fill array with wave
! NOTE: wave only depends on x so you can plot it later.
!$OMP DO
do i = 1, Nx
d = 2.0*pi*i*dx
do j = 1, Ny
do k = 1, Nz
arr_in(i,j,k) = cos(d/lambda1)+sin(d/lambda2)+cos(d/lambda3)
enddo
enddo
enddo
!$OMP END DO
call dfftw_execute_dft_r2c(plan_forward, arr_in, arr_out)
!$OMP END PARALLEL
!-----------------
! print results
!-----------------
do i=1, Nx/2+1
do j=1, Ny
do k=1, Nz
write(*,'(F12.6,A3,F12.6,A3)',advance='no') real(arr_out(i,j,k)), " , ", aimag(arr_out(i,j,k)), " ||"
enddo
write(*,*)
enddo
write(*,*)
enddo
deallocate(arr_in, arr_out)
! destroy plans is not thread-safe; do only with single
call dfftw_destroy_plan(plan_forward)
end program use_fftw
I also tried moving the initialisation part of FFTW (void = fftw_init_threads(); call fftw_plan_with_nthreads(nthreads); call dfftw_plan_dft_r2c_3d(...) into the parallel region, using a !$OMP SINGLE block and synchronising with a barrier afterwards, but the situation didn't improve.
Can anyone help me?
EDIT: I was able to test my program on another system, the problem remains. So the issue apparently isn't in my implementation of openmp or FFTW, but somewhere in the program itself.
You should normally call fftw execute routines outside of the parallel region. They have their own parallel regions inside them and they will take care of running the transform in parallel with that many threads as you requested during planning. They will re-use your existing OpenMP threads.
You can also call them inside a parallel region, but on different arrays, not on the same arrays! And then your plan should be planned to use 1 thread. Each thread would then preform a 2D transform of a slice of the array, for example.
The thread-safety means you can call the routines concurrently, but each for different data.

What will be the proper assigning of variables (private and shared) in the parallelized do loop of the given subroutine GAUSSLEG?

I am new about openmp. I am trying to parallelize do loop in subroutine GAUSSLEG. Variables Xg, Wg and Ng are taken from module matric. I am getting the unexpected results. I am confused about proper assigning of variables(private and shared). Can somebody help me ?
SUBROUTINE GAUSSLEG(f,a,b,s)
USE OMP_LIB
USE MATRIC , ONLY : XG ,WG , NG
IMPLICIT DOUBLE PRECISION(A-H,O-Z)
external f
xm = 0.5d0*(b+a)
xl = 0.5d0*(b-a)
s = 0.d0
!$omp parallel do reduction ( + : s) default(none)
!$omp private(j) shared(xm,xl,wg,xg,ng,dx)
do j=1,ng
dx = xl*xg(j)
s = s + wg(j)*(func(xm+dx)+func(xm-dx))
end do
!$omp end parallel do
s = xl*s/2.0
return
END
Hi, I have used the subroutine gaussleg to calculate the integration of sin(x) from 0 to pi, I get the same result (2.5464790894) whether i make dx private or shared but the exact result is 2.0. I have also tried by putting xl*xg(j) directly and removing dx, still getting same result as above.Without -openmp option in the compilation, i get the exact result 2.0.This is whole program.
MODULE MATRIC
IMPLICIT NONE
INTEGER , PARAMETER :: NG = 40
DOUBLE PRECISION , PARAMETER :: PI=2.0D0*ACOS(0.0D0)
DOUBLE PRECISION :: XG(60) , WG(60)
END MODULE MATRIC
program gauss
use matric, only : xg,wg,pi
implicit none
double precision :: x1,x2,a,b,ans
external :: f
x1 = -1.0d0 ; x2 = 1.0d0
a = 0.0 ; b = PI
call gauleg(x1,x2)
call gaussleg(f,a,b,ans)
write(*,*)ans
end program gauss
!function to be integrated
double precision function f(x)
implicit none
double precision, intent(in) :: x
f = sin(x)
end function f
SUBROUTINE GAUSSLEG(func,a,b,ss)
USE OMP_LIB
USE MATRIC , ONLY : XG ,WG , NG
double precision,intent(in) :: a , b
double precision,intent(out)::ss
double precision :: xm , xl , dx
integer :: j
double precision,external::func
xm = 0.5d0*(b+a)
xl = 0.5d0*(b-a)
ss = 0.d0
!$OMP PARALLEL DO REDUCTION( + : ss) default(none) &
!$OMP PRIVATE(j,dx) SHARED(xm,xl,xg,wg)
do j=1,ng
dx = xl*xg(j)
ss = ss + wg(j)*(func(xm+dx)+func(xm-dx))
end do
!$OMP END PARALLEL DO
ss = xl*ss/2.0
return
END
Your code includes a canonical data race. You have declared dx shared, then written
dx = xl*xg(j)
so that all threads can update the same, shared, variable, without any co-ordination. I think, but it is your responsibility to check this, that you can make dx private and have each thread look after its own value of the variable.
Incidentally. DO NOT USE implicit typing, you're just asking for trouble. Asking for trouble while you are trying to learn how to use OpenMP is just, well, asking for more trouble. USE implicit none. And don't respond Oh, I'm just updating an existing codebase which uses implicit typing. If that's what you are doing, do it properly.
Got exact results in the following way.
SUBROUTINE QGAUSSP(func,a,b,ss)
USE OMP_LIB
USE MATRIC , ONLY : XG ,WG , NG
implicit none
double precision, intent(in) :: a , b
double precision, intent(out):: ss
double precision :: xm , xl , dx , xgd , wgd
double precision :: s(NG)
integer :: j,tid
double precision,external::func
xm = 0.5d0*(b+a)
xl = 0.5d0*(b-a)
ss = 0.d0
!$omp parallel do private(j,xgd,wgd,dx) shared(xm,xl,xg,wg,s) num_threads(15)
do j=1,ng
xgd=xg(j)
wgd=wg(j)
dx = xl*xgd
s(j)=wgd*(func(xm+dx)+func(xm-dx))
end do
!$omp end parallel do
ss=sum(s) *xl/2.0
return
END

Fortran Syntax Basics?

So I am trying to get my Fortran 95 code to work just for basic function and program definitions. I get practically an error for every line saying "Unexpected" or "Unclassified". I wonder if it is my compiler (gfortran used in cygwin terminal) or if I am supposed to put something at the beginning of the file? Here it is if anyone can tell me anything.
double precision :: pi = 3.14159265359
PROGRAM Diffraction
write (*,*) sinc(0)
write (*,*) sinc(pi)
write (*,*) 1_Slit(0, 1, 550E-9)
end PROGRAM Diffraction
function SINC(angle) result(sinc)
double precision :: sinc
double precision :: angle
if angle == 0.0 then
sinc == 1
else
sinc = (sin(angle)/angle)
endif
end function SINC
function I(angle, d, wl) result(I)
double precision :: I_0 = 0.01
double precision :: angle, d, wl, I
A = (d * pi)/wl
B = SIN(angle)
I = I_0 * (SINC(A*B)**f2)
return
end function I
The way I compile is:
gfortran Diffraction.f95
Generally speaking, it is a good idea to put all definitions in the main program or a module. So your "floating" definitions are a bit odd.
Your program should start with PROGRAM [name] followed by your used modules. In your case, there are no such modules. After this, it is good practice to write IMPLICIT NONE. This means, that no variables have a predefined type. Otherwise, every variable starting with I to N would be of type INTEGER and every other variable would be of type REAL.
The next part is the variable definition part, where your variables are defined. (The first line in your example.)
After this, the main part is following, where you execute your code.
The final part is the CONTAINS part, where your functions and subroutines are placed, which can use every variable, which is defined in the program (but this would be bad praxis...).
So your example (with some corrections) would look like:
PROGRAM Diffraction
IMPLICIT NONE
double precision :: pi = 3.14159265359d0
write (*,*) sinc(0.d0)
write (*,*) sinc(pi)
write (*,*) one_slit(0.d0, 1.d0, 550.d-9)
CONTAINS
function SINC(angle) result(snc)
double precision :: snc
double precision :: angle
if (angle == 0.d0) then
snc = 1.d0
else
snc = (sin(angle)/angle)
endif
end function SINC
function one_slit(angle, d, wl) result(I)
double precision :: I0 = 0.01d0, A, B
double precision :: angle, d, wl, I, f2=2.d0
A = (d * pi)/wl
B = SIN(angle)
I = I0 * (SINC(A*B)**f2)
end function one_slit
end PROGRAM Diffraction

Resources