parallel fortran program willl sleep at a certain time - parallel-processing

Question

Recently I had to change a serial program written in fortran to a parallel version to make it faster to get the result. But I met some problems.
I'm using ubuntu os and gfortran compiler, and as for the parallel API, I'm using OpeMP. In the previous (serial) version, I use many modules to share the data, but in openmp version, I make the variables threadprivate attribute, and some of these variables has allocatable attribute. In the previous version, I allocate the space for the variable before the do-loop, but in the openmp version, if I do the same, the program will report an error as invalid memory reference, although I give it the threadprivate attribute. So I allocate the variable in the loop and deallocate it also in the loop. And I make that do-loop in a parallel region. It give no error and the program can run. But there is another problem. As it runs about 800min cpu time, and I use ps -ux command to see the status of this parallel program, its status changes from Rl to Sl. I search for the meaning of S, and it represent
Interruptible sleep (waiting for an event to complete)
So why this problem appears? Is it because I frequently alloc and free the space? The following is the sample code:
module variables
real, dimension(:), allocatable, save :: a
real, dimension(:,:), allocatable, save :: b
!$omp threadprivate(a,b)
integer, parameter :: n=100
contains
subroutine alloc_var
integer :: status
allocate(a(100),stat=status)
allocate(b(100:100),stat=status)
end subroutine
subroutine free_var
integer :: status
deallocate(a,stat=status)
deallocate(b,stat=status)
end subroutine
end module
for other subroutines, there are some using variable a and b.
subroutine cal_sth
use variables, only a
...
end subroutine
for the serial version main program
program main
implicit none
external :: cal_sth
use variables, only alloc_var,free_var
integer :: i, j
call alloc_var
do j=1, count1
...
other expresion ...
do i=1, count2
call cal_sth
end do
end do
call free_var
end program
for parallel region,
program main
implicit none
external :: cal_sth
use variables, only alloc_var, free_var
integer :: i,j
!$omp parallel do private(i,j)
do j=1, count1
...
other expression ...
do i=1, count2
call alloc_var
call cal_sth
if (logical expression) then
call free_var
cycle
end if
call free_var
end do
end do
end program

Answer 1

Either split the combined parallel do directive and rewrite the parallel loop so:
!$omp parallel
call alloc_var
!$omp do
do i=1, count
call cal_sth
end do
!$omp end do
call free_var
!$omp end parallel
or use dedicated parallel regions as per Gilles' comment:
program main
implicit none
external :: cal_sth
use variables, only alloc_var, free_var
integer :: i
!$omp parallel
call alloc_var
!$omp end parallel
...
!$omp parallel do
do i=1, count
call cal_sth
end do
!$omp end parallel do
...
! other OpenMP regions
...
!$omp parallel
call free_var
!$omp end parallel
end program

Answer 2

With your updated code, I think you have two different path to explore for improving performance:
The memory allocation: As previously mentioned, the calls to alloc_var and free_var only need to be made in a parallel region, but definitely not necessarily inside the do loop. By splitting the parallel do into a parallel, then a do, it gives you room for calling alloc_var prior to entering the loop, and calling free_var after exiting it. And the potential early exit from the inner loop, possibly necessitating a release / re-allocation of the memory isn't by itself a constraint preventing you from doing this. (see the code below for an example on how this can be done)
The scheduling: the early terminations of some of you inner iterations might translate into some load imbalance between threads. This could explain the waiting times you experiment. Explicitly setting the scheduling to dynamic might permit to reduce this effect and improve performance. This will need to be experimented a bit with to find the best scheduling policy to apply, but dynamic seems a good starting point.
So here is your code as it could look like once these two idea implemented:
program main
implicit none
external :: cal_sth
use variables, only alloc_var, free_var
integer :: i,j
!$omp parallel schedule(dynamic)
call alloc_var
!$omp do private(i,j)
do j=1, count1
...
other expression ...
do i=1, count2
call cal_sth
if (logical expression) then
!uncomment these only if needed for some reasons
!call free_var
!call alloc_var
cycle
end if
end do
end do
!$omp end do
call free_var
!$omp end parallel
end program