I wrote the following block to read from an external data file:
open(unit=338,file='bounnodes.dat',form='formatted')
DO I=1,NQBOUN
DO J=1,NUMBOUNNODES(I)
read(338,2001) NODEBOUN(i,j)
write(6,*) 'BOUNDARY NODES', NODEBOUN(i,j)
ENDDO
ENDDO
2001
FORMAT(32I5)
As far as I understood, this should read a 2 x 32 array from bounnodes.dat.
However, I get an error end-of-file during read and it prints the first column.
I tried to read a 32 x 2 array using the same code, and it reads 32 elements of the first column, but outputs 0s for the next column.
Can you please explain what is happening? Is my formatting wrong?
Every read statement in Fortran advances to the next record. This means a new line in normal text files. Try this:
DO I=1,NQBOUN
DO J=1,NUMBOUNNODES(I)
read(338,2001,advance='no') NODEBOUN(i,j)
write(*,*) 'BOUNDARY NODES', NODEBOUN(i,j)
ENDDO
read(338,*)
ENDDO
where NQBOUN is number of rows and NUMBOUNNODES(I) is number of columns in a row. (I have allway problems, what is 32x2 vs. 2x32)
You can make it even shorter, using the implied do
DO I=1,NQBOUN
read(338,2001) ( NODEBOUN(i,j) , j=1,NUMBOUNNODES(I) )
write(*,*) ( 'BOUNDARY NODES', NODEBOUN(i,j) , j=1,NUMBOUNNODES(I) )
ENDDO
or even
DO I=1,NQBOUN
read(338,2001) NODEBOUN(i,:)
write(*,*) 'BOUNDARY NODES', NODEBOUN(i,1:NUMBOUNNODES(I))
ENDDO
All of these use Fortran 90 features.
Related
I'm trying to generate a matrix full of random numbers, of dimensions (mysize)x(Np), where Np is a given number (say,5). Mysize comes from the number of processes. And then each processor should take its own column, of size 1xNp.
My first idea was to create a vector data type but then I realized this can be done by using the contiguous data type. The code runs with no errors, but it prints only some of the numbers that were supposed to be there. I'm quite sure the problem lies on the indicators (type_column, MPI_REAL, etc.) and how should they actually be but I can't figure this out. Anyone care to help?
The code:
program main
use mpi
integer :: ierr,myrank,mysize
integer :: Np=5, type_column
integer*8, external :: seedgen
real*8, dimension(:,:), allocatable :: x_init
real*8, dimension(:), allocatable :: block_x
character(4) :: rank
integer, dimension(1) :: new_seed
call MPI_INIT(ierr)
call MPI_COMM_RANK(MPI_COMM_WORLD,myrank,ierr)
call MPI_COMM_SIZE(MPI_COMM_WORLD,mysize,ierr)
allocate(block_x(0:Np-1))
if (myrank==0) then
allocate(x_init(0:Np-1,0:mysize-1))
new_seed = seedgen(myrank)
write(*,*) new_seed !This is to check if the seed changes each time
call random_seed(put=new_seed)
call random_number(x_init)
open(unit=1111, file='./x_tot.txt')
write(1111,*) x_init
close(1111) !Up to this point, everything works
end if
call MPI_TYPE_CONTIGUOUS(Np,MPI_REAL,type_column,ierr)
call MPI_TYPE_COMMIT(type_column,ierr)
call MPI_SCATTER(x_init(:,myrank),Np,MPI_REAL,block_x,Np,type_column,0,MPI_COMM_WORLD,ierr)
write(rank,'(I4)') myrank
write(*,*) 'I, process ',myrank,' received ', block_x(0:Np-1)
open(unit=myrank, file='./x_teste_'//trim(adjustl(rank))//'.txt')
write(myrank,*) block_x
close(myrank)
call MPI_TYPE_FREE(type_column,ierr)
call MPI_FINALIZE(ierr)
end program main
!!! You may ignore what is down below, just a function to generate random seeds.
function seedgen(myrank)
use iso_fortran_env
implicit none
integer(kind=int64) :: seedgen
integer, intent(IN) :: myrank
integer :: s
call system_clock(s)
seedgen = abs( mod((s*181)*((myrank-83)*359), 104729) )
end function seedgen
The output (after running with 3 processors, thus mysize=3)
I, process 0 received 0.247272870046176
0.386432141459887 8.816221414742263E-316 0.000000000000000E+000
0.000000000000000E+000
I, process 1 received 2.737792569698344E-008
I, process 2 received 5.231628354959370E-002
0.454932876242927 3.330771999738651E-315 0.000000000000000E+000
0.000000000000000E+000
3.352416881721526E+270 5.298272496856962E-315 0.000000000000000E+000
0.000000000000000E+000
The matrix x_tot:
0.247272870046176 0.386432141459887 0.473788032900533
0.239586263133600 0.851724848335892 5.231628354959370E-002
0.454932876242927 0.702720716936168 0.559915585253771
0.605745282251549 0.253298270763062 0.809867899324171
0.590174190311136 0.125210425650182 8.138975171285148E-002
So you can see that some numbers are the good ones, the rest is garbage.
Using MPI_DOUBLE as suggested above and changing the writing as below worked fine. Thanks again
open(unit=myrank, file='./x_teste_'//trim(adjustl(rank))//'.txt')
do i=0,Np-1
write(myrank,*) block_x(i)
end do
close(myrank)
In my Fortran90 code I have an outer cycle with several nested loops. In order to speed up my code, I tried to use OpenMP on the outer loop, but I have a very strange problem: when I use more than 1 thread the program runs slower than using OMP with 1 thread which is in turn slower than using the original serial program (in terms of wall clock time. I tried with 1, 2, 3 or 4 threads). In all the cases, however, I get the right result.
I conducted several tests on my code and at last I noticed that the problem is in one subroutine, because if I comment the call to that routine my parallel program works as expected, i.e. the bigger the number of threads, the lower the wall clock time.
Now, that routine takes in input 4 vectors, "ks1", "ks2", "ket1", "ket2" and performs the union between "ks1" and "ks2" obtaining "kstot". Then it creates 2 new vectors, "ket1tot" and "ket2tot", where ket1tot(i) is equal to ket1(j) if ks1tot(i) is equal to ks1(j), otherwise ket1tot(i)=0. The same for ket2tot.
Then combining the values stored in vectors "ks1tot", "ket1tot", "ket2tot" I compute which lines of a vector (matFC) contain the values I need and, by multiplication of that values, I obtain the final result (FCtot).
So I turn that routine in a simple program, adding some initial lines in order to mimic the real program. I mean I added:
1) a loop (on i) that mimics the outer loop of the real program I am trying to parallelize;
2) I implemented the fact that each thread works on a different file (so I should not have a false sharing problem)
3) I added another loop (on k) that mimics that I call the routine several times.
Here is the code (the part that constitutes the original subroutine that gives me problems is indicated in the text):
program evaluatefc
#ifdef _OPENMP
use omp_lib
#endif
implicit none
integer::i,ii,j,jj,jjj,k,sizeks1,sizeks2,sizec,sizekstot,NR,NR1,maxnq
integer::line,ierr,fileunit,mythread,nfreqtot
real*8::FCtot,time1,time2
integer,allocatable,dimension(:)::ks1,ket1,ks2,ket2
integer,dimension(:),allocatable::c,kstot,ket1tot,ket2tot
real*8,allocatable,dimension(:)::matFC
character*15,allocatable,dimension(:)::matfileFC
character::fileFC*15
real*4::tstarting,tending
! This program was originally a subroutine
! that takes in input 4 vectors, ks1, ks2, ket1, ket2
!---------------------------------------------------------------------------
! I initialize some values that in the original subroutine were computed by
!the main program
allocate(matfileFC(3),stat=ierr)
matfileFC(1)='filea.dat'
matfileFC(2)='fileb.dat'
matfileFC(3)='filec.dat'
sizeks1=2
sizeks2=2
maxnq=11
allocate(ks1(sizeks1),stat=ierr)
allocate(ket1(sizeks1),stat=ierr)
allocate(ks2(sizeks2),stat=ierr)
allocate(ket2(sizeks2),stat=ierr)
nfreqtot=42
NR1=nfreqtot*(maxnq**2)+nfreqtot
NR=nfreqtot*(maxnq**2)
allocate(matFC(NR),stat=ierr)
!Call two intrinsic to evaluate CPU and wall clock time
call cpu_time(time1)
tstarting=secnds(0.0)
!$OMP PARALLEL DO &
!$OMP DEFAULT(NONE) &
!$OMP firstprivate(sizeks1,sizeks2,maxnq,matfileFC,NR,NR1) &
!$OMP PRIVATE(i,ii,j,jj,k,ierr,mythread,fileunit,c,sizec,line,sizekstot) &
!$OMP PRIVATE(jjj,ket1,ks1,ket1tot,kstot,ket2,ks2,ket2tot,FCtot,matFC,fileFC)
do ii=1,3
#ifdef _OPENMP
mythread=OMP_GET_THREAD_NUM()
#else
mythread=10
#endif
fileFC=matfileFC(ii)
! Read some lines of a given file.
fileunit=50+mythread
open(unit=fileunit,name=fileFC,status='old',form='formatted')
read(fileunit,*)!Do not read first line
jjj=0
do jj=1,NR1-1
if(mod(jj,(maxnq**2+1)).eq.0) then
read(fileunit,*)
else
jjj=jjj+1
read(fileunit,*)j,k,i,matFC(jjj)
! I actually need only the fourth valor of the line to be stored
endif
enddo
close(fileunit)
do k=1,10000000
! Again I initialize the abovementioned values that in the actual
! subroutine are computed by the main program
ks1(1)=mod(k,30)+1
ks1(2)=mod(k,30)+2
ks2(1)=mod(k,17)+1
ks2(2)=mod(k,17)+3
ket1(1)=mod(k,2)
ket1(2)=mod(k,3)
ket2(1)=mod(k,5)
ket2(2)=mod(k,7)
sizec=sizeks1+sizeks2
allocate(c(sizec),stat=ierr)
do i=1,sizeks1
c(i)=ks1(i)
enddo
do i=sizeks1+1,sizec
c(i)=ks2(i-sizeks1)
enddo
sizekstot=sizec
do i=1,sizeks1
do j=1,sizeks2
if(ks1(i).eq.ks2(j)) then
sizekstot=sizekstot-1
endif
enddo
enddo
allocate(kstot(sizekstot),stat=ierr)
jjj=1
i=1
jj=0
do i=1,sizec-1
jjj=jjj+1
do j=jjj,sizec
if(c(i).eq.c(j)) then
exit
elseif(c(i).ne.c(j).and.j.eq.sizec) then
jj=jj+1
kstot(jj)=c(i)
endif
enddo
enddo
kstot(sizekstot)=c(sizec)
allocate(ket1tot(sizekstot),stat=ierr)
do i=1,sizekstot
ket1tot(i)=0
enddo
allocate(ket2tot(sizekstot),stat=ierr)
do i=1,sizekstot
ket2tot(i)=0
enddo
do i=1,sizekstot
do j=1,sizeks1
if(kstot(i).eq.ks1(j))then
ket1tot(i)=ket1(j)
endif
enddo
enddo
do i=1,sizekstot
do j=1,sizeks2
if(kstot(i).eq.ks2(j))then
ket2tot(i)=ket2(j)
endif
enddo
enddo
FCtot=1
do i=1,sizekstot
line=(kstot(i)-1)*(maxnq)**2+ket1tot(i)*(maxnq)+ket2tot(i)+1
FCtot=matFC(line)*FCtot
enddo
deallocate(c,stat=ierr)
deallocate(kstot,stat=ierr)
deallocate(ket1tot,stat=ierr)
deallocate(ket2tot,stat=ierr)
enddo
enddo
!$OMP END PARALLEL DO
call cpu_time(time2)
tending=secnds(tstarting)
write(*,*)
write(*,*)'CPU time is:'
write(*,*)time2-time1
write(*,*)
write(*,*)'Wall clock time is:'
write(*,*)tending
end program
Still, I get the same problem, i.e. the wall clock time using 4 thread is bigger than using 1 thread.
For example I get (in seconds):
type Wtime CPU time
1 thread 20.37 20.37
4 thread 31.26 91.61
serial 19.64 19.64
I am aware that the call to the OMP library introduces an overhead and in fact the 1-thread OMP program is slower than the serial one. But I cannot understand why the 4-thread OMP code is slower.
I use Intel fortran compiler 2013 on Linux.
Any suggestions?
Thank you for any time you can dedicate to this problem.
Ok, I fixed my own problem.
Thank you all for your suggestions, in particular #Jorge Bellón and #High Performance Mark.
As their comments said, the problem was actually the high number of allocation/deallocation. If I move the allocations out of the loops or at least if I put them right after the first loop, I get the "normal" OpenMP behaviour, i.e. the bigger the number of threads, the lower the wall clock time.
For the example above the wall clock time using 4 threads is now about 7 seconds.
Thank you all for your help.
Now I am data-processing 100,000 files by using Fortran. These data are generated by HPC using MPI I/O. Now I can just figure out the following ways to read the raw, which is not efficient. Is it possible that read every to read ut_yz(:,J,K), at one one time insteading of reading one by one? Thanks
The old code is as follows and the efficiency is not so high.
OPEN(10,FILE=trim(filename)//".dat",FORM='UNFORMATTED',&
ACCESS='DIRECT', RECL=4, STATUS='OLD')
!,CONVERT='big_endian'
COUNT = 1
DO K=1,nz
DO J=1,ny
DO I=1,nxt
READ(10,REC=COUNT) ut_yz(I,J,K)
COUNT = COUNT + 1
ENDDO
ENDDO
ENDDO
CLOSE(10)
The desired one is
OPEN(10,FILE=trim(filename)//".dat",FORM='UNFORMATTED', RECL=4, STATUS='OLD')
!,CONVERT='big_endian'
COUNT = 1
DO K=1,nz
DO J=1,ny
READ(10,REC=COUNT) TEMP(:)
COUNT = COUNT + 153
ut_yz(:,J,K)=TEMP(:)
ENDDO
ENDDO
CLOSE(10)
However, it always fails. Can anyone make a comment on this? Thanks.
Direct IO read will read a single record, if I am not mistaken. Thus, in your new code version you need to increase the record length accordingly:
inquire(iolength=rl) ut_yz(:,1,1)
open(10, file=trim(filename)//'.dat', form='UNFORMATTED', recl=rl, status='OLD', action='READ')
count = 1
do k=1,nz
do j=1,ny
read(10, rec=count) ut_yz(:,j,k)
count = count + 1
end do
end do
close(10)
Of course, in this example you could also read the complete array at once, which should be the fastest option:
inquire(iolength=rl) ut_yz
open(10, file=trim(filename)//'.dat', form='UNFORMATTED', recl=rl, status='OLD', action='READ')
read(10, rec=1) ut_yz
close(10)
I have an excel output in the tab-delimited format:
temperature H2O CO2 N2 NH3
10 2.71539E+12 44374931376 7410673406 2570.560804
20 2.34216E+12 38494172272 6429230649 3148.699673
30 2.04242E+12 33759520581 5639029060 3856.866413
40 1.75491E+12 29172949817 4882467457 4724.305292
.
.
.
I need to convert these numbers to FORMAT(1X,F7.0,2X,1P4E11.3) readable for another code.
This is what I've come up with:
program fixformat
real temp, neuts(4)
integer i,j
character header
open(11,file='./unformatted.txt',status='old')
open(12,file='./formatted.txt',status='unknown')
read(11,*) header
write(12,*) header
do i = 1, 200
read(11,*) temp, (neuts(j),j=1,4)
write(12,23) temp, (neuts(j),j=1,4)
end do
23 FORMAT(1X,F7.0,2X,1P4E11.3)
close(11)
close(12)
return
end
I keep getting this error:
Fortran runtime error: Bad real number in item 1 of list input
Is there any other way to convert the data to that format?
You need a character string, not a single character for the header
character(80) header
other than that you program works for me. Make sure you have the right number of lines in your loop
Do i=1,200
Adjust 200 to the real number of your data lines.
If for some reason you still cannot read even a single line, you can also use the format:
read(11,'(f2.0,4(1x,f11.0))') temp, (neuts(j),j=1,4)
because the tab is just a character you can easily skip.
Notes:
Unformatted and formatted means something completely different in Fortran. Unformatted is what you may know as "binary".
Use some indentation and blank lines for your programs to make them readable.
There is no reason to explicitly use status=unknown. Just don't put anything there. In your case status=replace may be more appropriate.
The FORMAT statement is quite obsolete, in modern Fortran we use format strings:
write(12,'(1X,F7.0,2X,1P4E11.3)') temp, (neuts(j),j=1,4)
There is absolutely no reason for your return before the end. Returns is for early return from a procedure. Some put stop before the end program, but it is superfluous.
To read tab delimited data, I'd use a simple algorithm like the one below. NOTE: This is assuming that there is no tab character in any of your fields.
integer :: error_code, delim_index, line_index
character*500 :: data_line, field_data_string
double precision :: dp_value
Open(Unit=1001,File="C:\\MY\\PATH\\Data.txt")
DO
Read(UNIT=1001,End=106, FMT='(A)' ) data_line
line_length = LEN(TRIM(data_line))
delim_index = SCAN(data_line, achar(9) )
line_index = 0
DO WHILE ( delim_index .NE. 0 )
line_index = line_index + delim_index
IF (delim_index .EQ. 1 ) THEN ! found a NULL (no value), so skip
GOTO 101
END IF
field_data_string = data_line( (line_index-delim_index+1) : line_index )
READ( field_data_string, FMT=*, ERR=100) dp_value
PRINT *, "Is a double precision ", dp_value
GOTO 101
100 Continue
PRINT *, "Not a double precision"
101 Continue
IF ( (line_index+1) .GT. line_length ) THEN
GOTO 104 ! found end of line prematurely
END IF
delim_index = SCAN( data_line( line_index + 1 : ), achar(9) )
END DO
field_data_string = data_line( line_index + 1 : )
READ( field_data_string, FMT=*, ERR=102) dp_value
PRINT *, "Is a double precision ", dp_value
GOTO 103
102 Continue
PRINT *, "Not a double precision"
103 Continue
PRINT *, "Is a double precision ", dp_value
104 Continue
END DO
104 Continue
PRINT *, "Error opening file"
105 Continue
Close(1001)
I have recently learned how to work with basic files in Fortran
and I assumed it was as simple as:
open(unit=10,file="data.dat")
read(10,*) some_variable, somevar2
close(10)
So I can't understand why this function I wrote is not working.
It compiles fine but when I run it it prints:
fortran runtime error:end of file
Code:
Function Load_Names()
character(len=30) :: Staff_Name(65)
integer :: i = 1
open(unit=10, file="Staff_Names.txt")
do while(i < 65)
read(10,*) Staff_Name(i)
print*, Staff_Name(i)
i = i + 1
end do
close(10)
end Function Load_Names
I am using Fortran 2008 with gfortran.
A common reason for the error you report is that the program doesn't find the file it is trying to open. Sometimes your assumptions about the directory in which the program looks for files at run-time will be wrong.
Try:
using the err= option in the open statement to write code to deal gracefully with a missing file; without this the program crashes, as you have observed;
or
using the inquire statement to figure out whether the file exists where your program is looking for it.
You can check when a file has ended. It is done with the option IOSTAT for read statement.
Try:
Function Load_Names()
character(len=30) :: Staff_Name(65)
integer :: i = 1
integer :: iostat
open(unit=10, file="Staff_Names.txt")
do while(i < 65)
read(10,*, IOSTAT=iostat) Staff_Name(i)
if( iostat < 0 )then
write(6,'(A)') 'Warning: File containts less than 65 entries'
exit
else if( iostat > 0 )then
write(6,'(A)') 'Error: error reading file'
stop
end if
print*, Staff_Name(i)
i = i + 1
end do
close(10)
end Function Load_Names
Using Fortran 2003 standard, one can do the following to check if the end of file is reached:
use :: iso_fortran_env
character(len=1024) :: line
integer :: u1,stat
open (newunit=u1,action='read',file='input.dat',status='old')
ef: do
read(u1,'A',iostat=stat) line
if (stat == iostat_end) exit ef ! end of file
...
end do ef
close(u1)
Thanks for all your help i did fix the code:
Function Load_Names(Staff_Name(65))!Loads Staff Names
character(len=30) :: Staff_Name(65)
integer :: i = 1
open(unit=10, file="Staff_Names.txt", status='old', action='read')!opens file for reading
do while(i < 66)!Sets Set_Name() equal to the file one string at a time
read(10,*,end=100) Staff_Name(i)
i = i + 1
end do
100 close(10)!closes file
return!returns Value
end Function Load_Names
I needed to change read(10,*) to read(10,*,END=100)
so it knew what to do when it came to the end the file
as it was in a loop I assume.
Then your problem was that your file was a row vector, and it was likely
giving you this error immediately after reading the first element, as #M.S.B. was suggesting.
If you have a file with a NxM matrix and you read it in this way (F77):
DO i=1,N
DO j=1,M
READ(UNIT,*) Matrix(i,j)
ENDDO
ENDDO
it will load the first column of your file in the first row of your matrix and will give you an error as soon as it reaches the end of the file's first column, because the loop enforces it to read further lines and there are no more lines (if N<M when j=N+1 for example). To read the different columns you should use an implicit loop, which is why your solution worked:
DO i=1,N
READ(UNIT,*) (Matrix(i,j), j=1,M)
ENDDO
I am using GNU Fortran 5.4.0 on the Ubuntu system 16.04. Please check your file if it is the right one you are looking for, because sometimes files of the same name are confusing, and maybe one of them is blank. As you may check the file path if it is in the same working directory.