I am writing a FORTRAN program that reads data from a text file and writing it to the console. the data file looks something like this
1234567890123456 123456.789 987654.321 673647.890 654356.890
6172876534567890 768909.098 234543.890 654321.908 987890.090
I have the following lines of FORTRAN code that reads data and just writes them to the console
OPEN(1,FILE='data.txt')
READ(1,'(I16,3F9.3)') A ,B, C, D
WRITE (*, '(I16,3F9.3)') A,B,C,D
CLOSE(1)
Instead of getting displayed as the same values in the text file, the following is the output
1234567890123456*********89987.656 0.322
6172876534567890*********98234.547 0.891
Can you please help me with this.
Thanks much
List-directed IO (i.e., *) is easier, especially on input. Nevertheless, there are times to use full IO control so that is worth understanding. On input, the data items and descriptors must line up by column. For input, in Fw.d, the d doesn't matter if you have a decimal point in the data item. The fields must be wide enough on both input and output. There need to be enough descriptors, of types which match the variables and the data items. Compare to this example program:
program test_read
implicit none
integer, parameter :: VLI_K = selected_int_kind (18)
integer, parameter :: DR_K = selected_real_kind (14)
integer (VLI_K) :: i
real (DR_K) :: a, b, c, d
open (unit=15, file="data.txt", status='old', &
access='sequential', form='formatted', action='read' )
read (15, 110) i, a, b, c, d
110 format (I16, 4(1X, F10.0) )
write (*, 120) i, a, b, c, d
120 format ( I18, 4 (2X, F12.3) )
read (15, *) i, a, b, c, d
write (*, 120) i, a, b, c, d
end program test_read
I had the hardest time ever trying to use read, but finally...
If you want to read a matrix stored in a .txt file use this:
program FILEREADER
real, dimension(:,:), allocatable :: x
integer :: n,m
open (unit=99, file='array.txt', status='old', action='read')
read(99, *), n
read(99, *), m
allocate(x(n,m))
do I=1,n,1
read(99,*) x(I,:)
write(*,*) x(I,:)
enddo
end
And the "array.txt" file must be like this for instance (And placed in the same folder of the main):
4
3
0.0 1.0 2.0
3.0 4.0 5.0
6.0 7.0 8.0
9.0 10.0 11.0
Hope it works for everyone out there
Slight modification to the #Andrés Argüello Guillén answer.
Unlike most other solutions, my code does not force you to specify in advance the number of rows and columns.
CHARACTER(128) :: buffer
integer strlen, rows, cols
real, dimension(:,:), allocatable :: x
OPEN (1, file = 'matrix.txt', status='old', action='read')
!Count the number of columns
read(1,'(a)') buffer !read first line WITH SPACES INCLUDED
REWIND(1) !Get back to the file beginning
strlen = len(buffer) !Find the REAL length of a string read
do while (buffer(strlen:strlen) == ' ')
strlen = strlen - 1
enddo
cols=0 !Count the number of spaces in the first line
do i=0,strlen
if (buffer(i:i) == ' ') then
cols=cols+1
endif
enddo
cols = cols+1
!Count the number of rows
rows = 0 !Count the number of lines in a file
DO
READ(1,*,iostat=io)
IF (io/=0) EXIT
rows = rows + 1
END DO
REWIND(1)
print*, 'Number of rows:', rows
print*, 'Number of columns:', cols
allocate(x(rows,cols))
do I=1,rows,1
read(1,*) x(I,:)
write(*,*) x(I,:)
enddo
CLOSE (1)
matrix.txt
0.0 1.0 2.0
3.0 4.0 5.0
6.0 7.0 8.0
I used fixed format because the editing and inspecting of input files having fixed column structure is easier than that of zigzag data.
My problem was how the Fortran run-time reader procedures interpret the presence and absence of decimal dots. I am not sure that my solution was the best but I read the data lines as character arrays, split them to fields having length 12 characters then I read the fields by read(*) statements.
The reason is that you're specifying a width that is too small for the real numbers. Usually when the width doesn't fit, fortran will display asterisks, which happens in your case.
You have 9 digits, but you'll need at least 10, since the comma takes up a column as well.
So replacing 3F9.3 with 3F10.3 should do the trick.
It is usually better to read data in non fixed format. And to leave some leading spaces so that numbers can fit when writing them out.
integer(8) :: i
real(4) :: x, y, z
open(unit=1, file='data.txt')
read(1,*)i, x, y, z
write(*,'(i16, 3f11.3)')i, x, y, z
end
Related
I'm just getting started with Julia, and I'm trying to read an unformatted FORTRAN file and store the data in arrays that are shaped in a particular way. I'm not sure how to accomplish this using Julia.
I've found the Julia package FortranFiles, which provides a direct way to read unformatted FORTRAN files using Julia. The file I'm trying to read looks like:
1 integer:
[nzones]
nzones*3 integers (brackets indicate one record):
[idim1,jdim1,kdim1,idim2,jdim2,kdim2,...,
idim_nzones,jdim_nzones,kdim_nzones]
series of nzones datasets:
[xvalues1,yvalues1,zvalues1](floating point values) for 1st zone
[xvalues1,yvalues1,zvalues1](floating point values) for 2nd zone
...,
[xvalues1,yvalues1,zvalues1](floating point values) for last zone
where the first line represents the number of zones and the lines that follow represent a grid dimension in each i, j, and k directions. Following these first two records are the x, y, and z coordinates, which are Float64s, for each i, j, and k point in a zone, and I would like to shape the arrays as x(1:im,1:jm,1:km,m), y(1:im,1:jm,1:km,m), and z(1:im,1:jm,1:km,m) where im, jm, and km are the imax,jmax, and kmax extents listed for each zone. Here's what I have so far:
using FortranFiles
fname = "my_file"
fid = FortranFile(fname)
#fread fid nblks::Int32
#fread fid ni::(Int32,nblks) nj::(Int32,nblks) nk::(Int32,nblks)
Here's where I'm getting hung up. For each zone I have x, y, and z coordinate arrays which should all be rank 4 arrays. For the x array, I want to store all of the x coordinates where x[1,1,1,1] refers to an x coordinate value at i=1, j=1, k=1, and zone =1 and x[end, end, end, end] refers to an x coordinate value at i = imax, j=jmax, k=kmax, and for the last zone listed (i.,e. zone = nblks). Then I want to create similar arrays for the y and z coordinate values.
Something like:
for m = 1:nblks
im = ni[m]
jm = nj[m]
km = nk[m]
#fread fid x::(Float64,im,jm,km,m) y::(Float64,im,jm,km,m) z::(Float64,im,jm,km,m)
end
However, I get a FortranFilesError: attempting to read beyond record end when trying this approach.
It appears that my issue is somewhat related to how Julia reads unformatted binary data, which is different from how FORTRAN's read works on the same data.
In FORTRAN, I could do something like:
integer, dimension (:), allocatable :: idim, jdim, kdim
integer :: nblks, fid, ios
fid = 10
open(unit=fid,form='unformatted', file='my_file',status='old',iostat=ios)
if( ios /= 0 ) then
write(*,*) '*** Error reading file ***'
stop
end if
read(fid) nblks
allocate( idim(nblks), jdim(nblks), kdim(nblks) )
read(fid) ( idim(m), jdim(m), kdim(m), m = 1, nblks )
close(fid)
...
However in Julia, I need to keep track of the file pointer's position, and realize that each record is preceded and followed by a 4-byte integer. I haven't been able to find a way to read each zone's i, j, & k extents directly into three separate arrays like can be done in FORTRAN (since the record is probably parsed line by line), but an alternative in Julia is to just read the entire record into a single nblk*3 element vector, and then reshape this vector afterwards:
fid = open("my_file")
skip(fid,4)
nblks = read(fid,Int32)
skip(fid,8)
dims = Array{Int32}(undef,3*nblks)
read!(fid,dims)
ni, nj, nk = [Array{Int32}(undef,nblks) for i in 1:3]
for m in 1:nblks
ni[m] = dims[3*m-2]
nj[m] = dims[3*m-1]
nk[m] = dims[3*m]
end
I am writing a program which computes the LU decomposition of a matrix, with partial pivoting, and I would like the function to output several (2 or 3) matrices without running the program several times to output each one individually, which is a waste of time since it gets me everything I want in one run. Is there a way of doing this? For example, here is my function using Doolittle's algorithm, for square matrix which don't need pivoting. I want my output to be matrix l and u at once, but I know no means of doing that.
function lu_d(aa) result(l)
real, dimension (:,:) :: aa !input matrix
real, dimension (size(aa,1), size(aa,2)) :: a !keeping input variable intact
real, dimension (size(a,1), size(a,2)) :: l , u !lower and upper matrices
integer :: i,j,k !index
real :: s !auxiliar variable
a=aa
do j=1 , size(a,2)
u(1,j)=a(1,j)
end do
l(1,1)=1
do j=2, size(a,2)
l(1,j)=0
end do
do i=2, size(a,1)
l(i,1)=a(i,1)/u(1,1)
u(i,1)=0
do j=2, i-1
s=0
u(i,j)=0
do k=1, j-1
s=s+l(i,k)*u(k,j)
end do
l(i,j)=(a(i,j)-s)/u(j,j)
end do
l(i,i)=1
do j=i, size(a,2)
s=0
l(i,j)=0
do k=1, i-1
s=s+l(i,k)*u(k,j)
end do
u(i,j)=a(i,j)-s
end do
end do
end function
You could switch from using a function to using a subroutine. This way you can output values for multiple arrays in the arguments list. Additionally using the
INTENT definition when declaring variables in the subroutine, e.g.:
REAL,INTENT(IN)::a declares a and does not allow its values to be altered inside the subroutine/function
REAL,INTENT(OUT)::b declares b and disregards any values it has coming into the subroutine/function
REAL,INTENT(INOUT)::c this is the case by default, if you don't write anything.
I will assume you need the output to be l and u (rather than m), in which case the structure would look something like the one below. Note that l and m should either be declared in the main program and their size defined with respect to aa (as in the first case shown below) OR declared with an allocatable size in the main program, passed to the subroutine without being allocated and allocated within the subroutine (second example). The latter may require you to put the subroutine in a module so that the interfaces are handled properly.
First example:
SUBROUTINE lu_d(aa,l,m)
implicit none
real,intent(in):: a(:,:)
real,intent(out):: l(:,:), m(:,:)
integer:: i,j,k
real:: s
<operations>
RETURN
END SUBROUTINE lud_d
Second example:
SUBROUTINE lu_d(aa,l,m)
implicit none
real,intent(in):: a(:,:)
real,allocatable,intent(out):: l(:,:), m(:,:)
integer:: i,j,k,size_a1,size_a2
real:: s
size_a1=size(aa,1)
size_a2=size(aa,2)
allocate( l(size_a1,size_a2), m(size_a1,size_a2))
<operations>
RETURN
END SUBROUTINE lud_d
I have the following program and below the program an input data file, which contains 10 lines of different data. I want to read this data randomly not sequentially, for example, it will maybe read line 3 then maybe line 5, not like number 1 2 3 4... Then these numbers I want to print randomly.
program rand
implicit none
integer::i, ok
real(kind=8) , allocatable , dimension(:):: s
integer, parameter:: nstep = 1, natom = 10
integer:: seed, rand
open(unit=2,file="fort.2",status="old",action="read")
allocate(s(natom),stat=ok)
if(ok/=0)then
print*,"problem allocating position array"
end if
do i=1,natom
read(2,*)s(i)
print*,i=(rand(seed))
end do
end program rand
Input file:
1.004624
1.008447
1.028897
1.001287
0.9994195
1.036111
0.9829285
1.029622
1.005867
0.9372157
As suggested by #IanBush in a comment, and also by #Sazzad in his answer, a reasonable approach is to read the whole file into an array as your program is already doing. However, simply shuffing does not seem to me to lead to a random printing. It is just a new order. That is the reason while I am proposing this solution.
Random means that the same number can be printed many times while other are not printed at all, if the number of print is limited. And as I can see your problem is how to select randomly. Since you show some effort, here is a modified version of your program
program rand
implicit none
integer::i, ok, idx
real(kind=8) , allocatable , dimension(:):: s
integer, parameter:: nstep = 1, natom = 10
integer:: seed!, rand
real(kind = 8) :: randNum
!
!
open(unit=2,file="fort.2",status="old",action="read")
!
!
allocate(s(natom),stat=ok)
if(ok/=0)then
print*,"problem allocating position array"
end if
!
do i=1,natom
read(2,*)s(i)
!print*,i=(rand(seed))
end do
!
CALL random_seed() ! Initialize a pseudo-random number sequence
! to the default state. For serious program, do not use the default
! use for example the program on the website of gnu fortran
! https://gcc.gnu.org/onlinedocs/gfortran/RANDOM_005fSEED.html
!
do i=1,natom !you can and should change natom here to something else
CALL random_number(randNum)
idx = int(randNum*natom) + 1
print*,'element at ',idx,': ', s(idx)
end do
end program rand
This difference is that the printing is commented in your original program and there is a new loop to print randomly. You will see that some numbers will be printed more than once. To give each number a chance to be printed, you should set a large number of iteration inf the printing loop.
In this answer, I used the default seed for the random number which is not a good idea. On the web site of gnu fortran ( link ) you can find a good approach of initializing the random seed. It is a good programming habit if the reproducibility is not a concern.
General algorithm looks like,
Read all or N lines from file in lines[N]
Create an array index[N] = {1, 2, ... N}
Shuffle index array with simple shuffle algorithms
Traverse index[i] for each i up to size and output line[i]
You have to convert it in your language yourself
I am trying to write a function in Fortran that multiplies a number of matrices with different weights and then adds them together to form a single matrix. I have identified that this process is the bottleneck in my program (this weighting will be made many times for a single run of the program, with different weights). Right now I'm trying to make it run faster by switching from Matlab to Fortran. I am a newbie at Fortran so I appreciate all help.
In Matlab the fastest way I have found to make such a computation looks like this:
function B = weight_matrices()
n = 46;
m = 1800;
A = rand(n,m,m);
w = rand(n,1);
tic;
B = squeeze(sum(bsxfun(#times,w,A),1));
toc;
The line where B is assigned runs in about 0.9 seconds on my machine (Matlab R2012b, MacBook Pro 13" retina, 2.5 GHz Intel Core i5, 8 GB 1600 MHz DDR3). It should be noted that for my problem, the tensor A will be the same (constant) for the whole run of the program (after initialization), but w can take any values. Also, typical values of n and m are used here, meaning that the tensor A will have a size of about 1 GB in memory.
The clearest way I can think of writing this in Fortran is something like this:
pure function weight_matrices(w,A) result(B)
implicit none
integer, parameter :: n = 46
integer, parameter :: m = 1800
double precision, dimension(num_sizes), intent(in) :: w
double precision, dimension(num_sizes,msize,msize), intent(in) :: A
double precision, dimension(msize,msize) :: B
integer :: i
B = 0
do i = 1,n
B = B + w(i)*A(i,:,:)
end do
end function weight_matrices
This function runs in about 1.4 seconds when compiled with gfortran 4.7.2, using -O3 (function call timed with "call cpu_time(t)"). If I manually unwrap the loop into
B = w(1)*A(1,:,:)+w(2)*A(2,:,:)+ ... + w(46)*A(46,:,:)
the function takes about 0.11 seconds to run instead. This is great and means that I get a speedup of about 8 times compared to the Matlab version. However, I still have some questions on readability and performance.
First, I wonder if there is an even faster way to perform this weighting and summing of matrices. I have looked through BLAS and LAPACK, but can't find any function that seems to fit. I have also tried to put the dimension in A that enumerates the matrices as the last dimension (i.e. switching from (i,j,k) to (k,i,j) for the elements), but this resulted in slower code.
Second, this fast version is not very flexible, and actually looks quite ugly, since it is so much text for such a simple computation. For the tests I am running I would like to try to use different numbers of weights, so that the length of w will vary, to see how it affects the rest of my algorithm. However, that means I quite tedious rewrite of the assignment of B every time. Is there any way to make this more flexible, while keeping the performance the same (or better)?
Third, the tensor A will, as mentioned before, be constant during the run of the program. I have set constant scalar values in my program using the "parameter" attribute in their own module, importing them with the "use" expression into the functions/subroutines that need them. What is the best way to do the equivalent thing for the tensor A? I want to tell the compiler that this tensor will be constant, after init., so that any corresponding optimizations can be done. Note that A is typically ~1 GB in size, so it is not practical to enter it directly in the source file.
Thank you in advance for any input! :)
Perhaps you could try something like
do k=1,m
do j=1,m
B(j,k)=sum( [ ( (w(i)*A(i,j,k)), i=1,n) ])
enddo
enddo
The square brace is a newer form of (/ /), the 1d matrix (vector). The term in sum is a matrix of dimension (n) and sum sums all of those elements. This is precisely what your unwrapped code does (and is not exactly equal to the do loop you have).
I tried to refine Kyle Vanos' solution.
Therefor I decided to use sum and Fortran's vector-capabilities.
I don't know, if the results are correct, because I only looked for the timings!
Version 1: (for comparison)
B = 0
do i = 1,n
B = B + w(i)*A(i,:,:)
end do
Version 2: (from Kyle Vanos)
do k=1,m
do j=1,m
B(j,k)=sum( [ ( (w(i)*A(i,j,k)), i=1,n) ])
enddo
enddo
Version 3: (mixed-up indices, work on one row/column at a time)
do j = 1, m
B(:,j)=sum( [ ( (w(i)*A(:,i,j)), i=1,n) ], dim=1)
enddo
Version 4: (complete matrices)
B=sum( [ ( (w(i)*A(:,:,i)), i=1,n) ], dim=1)
Timing
As you can see, I had to mixup the indices to get faster execution times. The third solution is really strange because the number of the matrix is the middle index, but this is necessary for memory-order-reasons.
V1: 1.30s
V2: 0.16s
V3: 0.02s
V4: 0.03s
Concluding, I would say, that you can get a massive speedup, if you have the possibility to change order of the matrix indices in arbitrary order.
I would not hide any looping as this is usually slower. You can write it explicitely, then you'll see that the inner loop access is over the last index, making it inefficient. So, you should make sure your n dimension is the last one by storing A is A(m,m,n):
B = 0
do i = 1,n
w_tmp = w(i)
do j = 1,m
do k = 1,m
B(k,j) = B(k,j) + w_tmp*A(k,j,i)
end do
end do
end do
this should be much more efficient as you are now accessing consecutive elements in memory in the inner loop.
Another solution is to use the level 1 BLAS subroutines _AXPY (y = a*x + y):
B = 0
do i = 1,n
CALL DAXPY(m*m, w(i), A(1,1,i), 1, B(1,1), 1)
end do
With Intel MKL this should be more efficient, but again you should make sure the last index is the one which changes in the outer loop (in this case the loop you're writing). You can find the necessary arguments for this call here: MKL
EDIT: you might also want to use some parallellization? (I don't know if Matlab takes advantage of that)
EDIT2: In the answer of Kyle, the inner loop is over different values of w, which is more efficient than n times reloading B as w can be kept in cache (using A(n,m,m)):
B = 0
do i = 1,m
do j = 1,m
B(j,i)=0.0d0
do k = 1,n
B(j,i) = B(j,i) + w(k)*A(k,j,i)
end do
end do
end do
This explicit looping performs about 10% better as the code of Kyle which uses whole-array operations. Bandwidth with ifort -O3 -xHost is ~6600 MB/s, with gfortran -O3 it's ~6000 MB/s, and the whole-array version with either compiler is also around 6000 MB/s.
I know this is an old post, however I will be glad to bring my contribution as I played with most of the posted solutions.
By adding a local unroll for the weights loop (from Steabert's answer ) gives me a little speed-up compared to the complete unroll version (from 10% to 80% with different size of the matrices). The partial unrolling may help the compiler to vectorize the 4 operations in one SSE call.
pure function weight_matrices_partial_unroll_4(w,A) result(B)
implicit none
integer, parameter :: n = 46
integer, parameter :: m = 1800
real(8), intent(in) :: w(n)
real(8), intent(in) :: A(n,m,m)
real(8) :: B(m,m)
real(8) :: Btemp(4)
integer :: i, j, k, l, ndiv, nmod, roll
!==================================================
roll = 4
ndiv = n / roll
nmod = mod( n, roll )
do i = 1,m
do j = 1,m
B(j,i)=0.0d0
k = 1
do l = 1,ndiv
Btemp(1) = w(k )*A(k ,j,i)
Btemp(2) = w(k+1)*A(k+1,j,i)
Btemp(3) = w(k+2)*A(k+2,j,i)
Btemp(4) = w(k+3)*A(k+3,j,i)
k = k + roll
B(j,i) = B(j,i) + sum( Btemp )
end do
do l = 1,nmod !---- process the rest of the loop
B(j,i) = B(j,i) + w(k)*A(k,j,i)
k = k + 1
enddo
end do
end do
end function
I have a data approximately a million record, each record have 6 floating point number. I want to find sets of records who share identical six values, and ideally I want to do it in Fortran since the rest of processing is done in Fortran. What would be the recommended approach for this? At the end i want to have mapping from original index to new index which is condensed version of these dataset without duplicate. Each record has other attributes and i am interested in aggregating those for groups based on the six attributes.
I tried to find those sets by exporting output as csv, import it into MS Access, then a query that finds those sets took 10 seconds or so to run. I wrote a code which does http://rosettacode.org/wiki/Remove_duplicate_elements#Fortran this ("linear search"?), but with million record it didnt complete after 10 min or so, i just abandoned this approach.
Approach I am thinking now is adapting ranking/sorting routine from slatec or orderpack which i assume do better than my crude code. But I am wondering if such things are already done and i can download, or if there is better approach for this.
EDIT:
I said "finding duplicate", but i actually need mapping from original data records to this reduced sets. I want to have mapping array like imap(1:n), where imap(1), imap(4), imap(5) has same values if those 6 float pt. values in original record 1, 4 and 5 are the same. Hope this is not too much a deviation from what I said originally...
This is what I ended up doing... I took code mrgrnk from ORDERPACK , and adapted for my purpose. The subroutine findmap below appears to be doing what I wanted it to do.
module fndmap
use m_mrgrnk, only:mrgrnk
implicit none
contains
subroutine findmap(stkprm, stkmap )
! given 2-d real array stkprm, find a mapping described below:
!
! (identical records are assigned with same index)
! stkmap(i) == stkmap(j) iff stkprm(:,i) == stkprm(:,j)
! (order conserved)
! if i < j and stkmap(i) /= stkmap(j), then stkmap(i) < stkmap(j)
! (new index are contiguous)
! set(stkmap) == {1,2,..,maxval(stkmap)}
!
real,dimension(:,:),intent(in) :: stkprm
integer,dimension(:), intent(out) :: stkmap
integer, dimension(size(stkprm,2)) :: irngt
integer, dimension(size(stkprm,2)) :: iwork
integer :: nrec, i, j
nrec = size(stkprm,2)
! find rank of each record, duplicate records kept
call ar_mrgrnk(stkprm, irngt)
! construct iwork array, which has index of original array where the
! record are identical, and the index is youguest
i = 1
do while(i<=nrec)
do j=i+1,nrec
if (any(stkprm(:,irngt(i))/=stkprm(:,irngt(j)))) exit
enddo
iwork(irngt(i:j-1)) = minval(irngt(i:j-1))
i = j
enddo
! now construct the map, where stkmap(i) shows index of new array
! with duplicated record eliminated, original order kept
j = 0
do i=1,nrec
if (i==iwork(i)) then
j = j+1
stkmap(i) = j
else
stkmap(i) = stkmap(iwork(i))
endif
enddo
end subroutine
recursive subroutine ar_mrgrnk(xdont, irngt)
! behaves like mrgrnk of ORDERPACK, except that array is 2-d
! each row are ranked by first field, then second and so on
real, dimension(:,:), intent(in) :: xdont
integer, dimension(:), intent(out), target :: irngt
integer, dimension(size(xdont,2)) :: iwork
integer :: nfld,nrec
integer :: i, j
integer, dimension(:), pointer :: ipt
nfld=size(xdont,1)
nrec=size(xdont,2)
! rank by the first field
call mrgrnk(xdont(1,:), irngt)
! if there's only one field, it's done
if (nfld==1) return
! examine the rank to see if multiple record has identical
! values for the first field
i = 1
do while(i<=nrec)
do j=i+1,nrec
if (xdont(1,irngt(i))/=xdont(1,irngt(j))) exit
enddo
! if one-to-one, do nothing
if (j-1>i) then
! if many-to-one,
! gather those many, and rank them
call ar_mrgrnk(xdont(2:,irngt(i:j-1)),iwork)
! rearrange my rank based on those fields to the right
ipt => irngt(i:j-1)
ipt = ipt(iwork(1:j-i))
endif
i = j
enddo
if(associated(ipt)) nullify(ipt)
end subroutine
end module