Problem with using MPI_SENDRECV with 4 threads - parallel-processing

Question

As a minimal problem, I'm trying to send an integer between 4 processors: 0 -> 3 (rank 0 sends to and receives from rank 3), 2 -> 1, 1 -> 2, 3 -> 0. It never finishes execution and hangs, probably waiting for the response from other threads.
I'm compiling the code with mpif90 ... and running with mpiexec -np 4 .... Below is the minimal snippet:
program sendrecv
implicit none
include "mpif.h"
integer :: foo, bar
integer :: mpi_rank, mpi_size, ierr
integer :: mpi_sendto, mpi_recvfrom
integer :: istat(MPI_STATUS_SIZE), status, i
call MPI_INIT(ierr)
call MPI_COMM_SIZE(MPI_COMM_WORLD, mpi_size, ierr)
call MPI_COMM_RANK(MPI_COMM_WORLD, mpi_rank, ierr)
print *, "SENDING..."
if (mpi_rank .eq. 0) then
mpi_sendto = 3; mpi_recvfrom = 3
else if (mpi_rank .eq. 1) then
mpi_sendto = 2; mpi_recvfrom = 2
else if (mpi_rank .eq. 2) then
mpi_sendto = 1; mpi_recvfrom = 1
else
mpi_sendto = 0; mpi_recvfrom = 0
end if
foo = mpi_rank
do i = 1, 5
foo = mpi_rank
call MPI_SENDRECV(foo, 1,&
& MPI_INTEGER, mpi_sendto, mpi_rank * 10 + i,&
& bar, 1,&
& MPI_INTEGER, mpi_recvfrom, mpi_rank * 10 + i,&
& MPI_COMM_WORLD, istat, ierr)
end do
print *, "...DONE"
call MPI_FINALIZE(ierr)
end
I don't really understand why this program hangs, maybe I'm missing something or doing something really wrong. If I understand correctly, MPI_SENDRECV is just non-blocking send and recv with two wait-s. In that case, say, if rank=0 sends to rank=3 it shouldn't have any problem receiving from it, right?
I tried sending/receiving from different threads, i.e., doing this:
if (mpi_rank .eq. 0) then
mpi_sendto = 1; mpi_recvfrom = 3
else if (mpi_rank .eq. 1) then
mpi_sendto = 2; mpi_recvfrom = 0
else if (mpi_rank .eq. 2) then
mpi_sendto = 3; mpi_recvfrom = 1
else
mpi_sendto = 0; mpi_recvfrom = 2
end if
still not working.
UPD As it was pointed out, tags should be the same when doing SENDRECV, however In case when doing this call within a loop, similar tags don't help much (see modified code). Old version:
call MPI_SENDRECV(foo, 1,&
& MPI_INTEGER, mpi_sendto, 200,&
& bar, 1,&
& MPI_INTEGER, mpi_recvfrom, 100,&
& MPI_COMM_WORLD, status, ierr)
UPD#2 Actually, if anyone is interested, I found a discussion exactly about the problem I have on why SENDRECV-s may deadlock sometimes.

Answer 1

The term "thread" is misleading here, you should talk about MPI task or MPI process (both are equivalent).
The root cause is a tag mismatch. You send with tag 200 but receive with tag 100.
Also, you should use istat instead of status as the status argument of MPI_Sendrecv().
Here is how you can fix your program
call MPI_SENDRECV(foo, 1,&
& MPI_INTEGER, mpi_sendto, 200,&
& bar, 1,&
& MPI_INTEGER, mpi_recvfrom, 200,&
& MPI_COMM_WORLD, istat, ierr)