USe the following dummy code to replicate the issue.
program pp
implicit none
real*8,dimension(45) :: refPoints
refPoints(:) = (/ -1.0 , 1.0 , 1.0 , -1.0 , -1.0 , 1.0 , 1.0 , -1.0 , 0.0 , 1.0 , 0.0 , -1.0 , 0.0 , 1.0 , 0.0 , -1.0 , -1.0 , 1.0 , 1.0 , -1.0 , 0.0 , 1.0 , 0.0 ,-1.0 , 0.0 , 0.0 , 0.0 , 0.0 , 0.0 , 1.0, 1.0, 2.0 , 3.0, 34.0, 35.0, 25.0, 1.0, 50.0, 5.0, 55.0 , 1.0 , 2.0, 3.0, 4.0, 5.0/)
end program pp
PGF90-S-0285-Source line too long (pp.f90: 6)
PGF90-S-0023-Syntax error - unbalanced parentheses (pp.f90: 6)
0 inform, 0 warnings, 2 severes, 0 fatal for pp
132 columns is limit the F90 standard and going beyond this limit is undefined behavior. While a pain, you'll be better off in the long run by getting your code into compliance by adding continuations.
Related
I want to run functions in parallel. These functions are executed many times in a loop.
coordSys = SharedArray{Bool}([true,false,true,true]);
dir = SharedArray{Int8}([1,2,3,2]);
load = SharedArray{Float64}([8,-7.5,7,-8.5]);
L = SharedArray{Float64}([400,450,600,500]);
r = SharedArray{Float64}([0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0]);
Obviously these vectors will be huge, but for simplicity I just put this limited size.
Operation without parallel computing:
function unifLoad(coordSys,dir,load,L,ri)
if coordSys == true
if dir == 1
Q = [load;0;0];
elseif dir == 2
Q = [0;load;0];
elseif dir == 3
Q = [0;0;load];
end
q = ri*Q; #matrix multiplication
P = q[1]*L/2;
V = q[2]*L/2;
M = -q[3]*L*L/12;
f = [P;V;M];
else
f = [1.0;1.0;1.0];
end
return f
end
running the loop:
var = zeros(12)
for i = 1:length(L)
var[3*(i-1)+1:3*i] = unifLoad(coordSys[i],dir[i],load[i],L[i],r[3*(i-1)+1:3*i,:]);
end
The returned value is:
var
12-element Array{Float64,1}:
0.0
0.0
-1.06667e5
1.0
1.0
1.0
2100.0
0.0
-0.0
0.0
2125.0
-0.0
Operation with parallel computing
I've been trying to implement the same function in parallel, but without getting the same results.
# addprocs(3)
#everywhere function unifLoad_Parallel(coordSys,dir,load,L,ri)
if coordSys == true
if dir == 1
Q = [load;0;0];
elseif dir == 2
Q = [0;load;0];
elseif dir == 3
Q = [0;0;load];
end
q = ri*Q; # Matrix multiplication (ri -> Array 3x3)
P = q[1]*L/2;
V = q[2]*L/2;
M = -q[3]*L*L/12;
f = [P;V;M];
else
f = [1.0;1.0;1.0];
end
return f
end
running the parallel loop:
var_parallel = SharedArray{Float64}(12);
#parallel for i = 1:length(L)
var_parallel[3*(i-1)+1:3*i] = unifLoad_Parallel(coordSys[i],dir[i],load[i],L[i],r[3*(i-1)+1:3*i,:]);
end
The returned value is:
var_parallel
12-element SharedArray{Float64,1}:
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
On my Julia 0.6.3 the parallel code returns the same result so I am unable to reproduce the problem (also I do not encounter the issue #SalchiPapa reports).
However, I would like to note that this code actually should work faster with threads (I assume that the real problem is much larger). Here is the code you could use (I used an equivalent implementation to your which is a bit shorter - but the only significantly relevant change is that I wrap it in a function which provides dramatic performance gains). The crucial issue that all arrays except var are shared but only read. And var is written but only once at each entry and not read from. This is the case where it is safe to use threading which has a lower overhead.
Here is an example code (you have to define JULIA_NUM_TREADS environment variable before starting Julia and set it to number of threads you want - most probably 4 is what you want):
using Base.Threads
function experiment()
coordSys = [true,false,true,true];
dir = [1,2,3,2];
load = [8,-7.5,7,-8.5];
L = [400,450,600,500];
r = [0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0];
unifLoad(coordSys,dir,load,L,r, i) =
coordSys ? load * L * r[3*(i-1)+1:3*i, dir] .* [0.5, 0.5, -L/12] : [1.0, 1.0, 1.0]
var = zeros(12)
#threads for i = 1:length(L)
var[3*(i-1)+1:3*i] = unifLoad(coordSys[i],dir[i],load[i],L[i],r,i);
end
var
end
Also here is a bit simplified code for parallel processing using similar ideas:
coordSys = SharedArray{Bool}([true,false,true,true]);
dir = SharedArray{Int8}([1,2,3,2]);
load = SharedArray{Float64}([8,-7.5,7,-8.5]);
L = SharedArray{Float64}([400,450,600,500]);
r = SharedArray{Float64}([0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0
0.0 0.0 1.0; 0.0 -1.0 0.0; 1.0 0.0 0.0]);
#everywhere unifLoad(coordSys,dir,load,L,r,i) =
coordSys ? load * L * r[3*(i-1)+1:3*i, dir] .* [0.5, 0.5, -L/12] : [1.0, 1.0, 1.0]
vcat(pmap(i -> unifLoad(coordSys[i],dir[i],load[i],L[i],r,i), 1:length(L))...)
Here pmap is mostly used to simplify the code so that you do not need #sync.
I am wondering why this matrix multiplication is not working in my Scala program, versus the result I am receiving when using Python. I am using the matrix multiplication algorithm described by this math: Matrix Multiplication where I have two matrices a = n x m and b = m x p. The code that I have written for this algorithm is (each matrix is a 2d array of doubles):
def dot(other: Matrix2D): Matrix2D ={
if (this.shape(1) != other.shape(0)){
throw new IndexOutOfBoundsException("Matrices were not the right shape! [" + this.shape(1) + " != " + other.shape(0) + "]")
}
val n = this.shape(1) //returns the number of columns, shape(0) returns number of rows
var a = matrix.clone()
var b = other.matrix.clone()
var c = Array.ofDim[Double](this.shape(0), other.shape(1))
for(i <- 0 until c.length){
for (j <- 0 until c(0).length){
for (k <- 0 until n){
c(i)(j) += a(i)(k) * b(k)(j)
}
}
}
Matrix2D(c)
}
The Input I put into both the Scala and Python code is:
a = [[1.0 1.0 1.0 1.0 0.0 0.0 0.0]
[1.0 1.0 0.0 1.0 0.0 0.0 0.0 ]
[1.0 1.0 1.0 1.0 1.0 1.0 1.0 ]
[1.0 0.0 0.0 0.0 1.0 1.0 1.0 ]
[1.0 0.0 0.0 0.0 1.0 0.0 1.0 ]
[1.0 0.0 0.0 0.0 0.0 0.0 0.0 ]]
b = [[0.0 0.0 0.0 ]
[0.0 -0.053430398509053074 0.021149859549078387 ]
[0.0 -0.010785871994186721 0.04942555653681449 ]
[0.0 0.04849323245519227 -0.0393881161667335 ]
[0.0 -0.03871752673999099 0.05228579488821056 ]
[0.0 0.07935206375269452 0.06511344235965408 ]
[0.0 -0.02462677123918247 1.723607966539059E-4 ]]
The output I receive from this function is:
[[0.0 -0.015723038048047533 0.031187299919159375]
[0.0 -0.0049371660538608045 -0.018238256617655116]
[0.0 2.84727725473527E-4 0.14875889796367792 ]
[0.0 0.01600776577352106 0.11757159804451854 ]
[0.0 -0.06334429797917346 0.05245815568486446 ]
[0.0 0.0 0.0 ]]
compared to python's numpy.dot algorithm:
[[ 0. -0.01572304 0.0311873 ]
[ 0. -0.00493717 -0.01823826]
[ 0. -0.01572304 0.0311873 ]
[ 0. 0.08912777 0.07801112]
[ 0. 0.00977571 0.01289768]
[ 0. 0.08912777 0.07801112]]
I am wondering why this algorithm doesn't completely fill the output algorithm that I need...I've been messing with the for loops and such and have not been able to figure out whats wrong.
Can you show your Python code?
I tried this in Numpy and get the same as your Scala code:
import numpy as np
a = np.array([[1.0,1.0,1.0,1.0,0.0,0.0,0.0],
[1.0, 1.0, 0.0, 1.0, 0.0,0.0,0.0 ],
[1.0, 1.0, 1.0, 1.0, 1.0,1.0,1.0 ],
[1.0, 0.0, 0.0, 0.0, 1.0 ,1.0,1.0 ],
[1.0, 0.0, 0.0, 0.0, 1.0, 0.0,1.0 ],
[1.0, 0.0, 0.0, 0.0, 0.0, 0.0,0.0 ]])
b=np.array([[0.0 ,0.0 ,0.0 ],
[0.0 ,-0.053430398509053074 ,0.021149859549078387 ],
[0.0 ,-0.010785871994186721, 0.04942555653681449 ],
[0.0 , 0.04849323245519227 ,-0.0393881161667335 ],
[0.0 ,-0.03871752673999099 , 0.05228579488821056 ],
[0.0 , 0.07935206375269452 , 0.06511344235965408 ],
[0.0 ,-0.02462677123918247 ,1.723607966539059E-4 ]])
print a.dot(b)
prints:
[[ 0. -0.01572304 0.0311873 ]
[ 0. -0.00493717 -0.01823826]
[ 0. 0.00028473 0.1487589 ]
[ 0. 0.01600777 0.1175716 ]
[ 0. -0.0633443 0.05245816]
[ 0. 0. 0. ]]
I have a file containing the string "proto" which is repeated 384 times. Each "proto" should be edited according to 384 different labels existing in another textfile. For insance, if the content of the second text file is a, sp, .. (each label exists in a newline), so in my textfile the first "proto" should be changed to "a", the second to "sp" and so on. How should I do that?
The original file is a 384 repetitions of :
~o <VecSize> 39 <MFCC_0_D_A>
~h "proto"
<BeginHMM>
<NumStates> 5
<State> 2
<Mean> 39
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
<Variance> 39
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
<State> 3
<Mean> 39
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
<Variance> 39
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
<State> 4
<Mean> 39
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
<Variance> 39
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
<TransP> 5
0.0 1.0 0.0 0.0 0.0
0.0 0.6 0.4 0.0 0.0
0.0 0.0 0.6 0.4 0.0
0.0 0.0 0.0 0.7 0.3
0.0 0.0 0.0 0.0 0.0
<EndHMM>
and the second file contains 384 labels, such as: a, ap, k12, sp, ... So as you see, I have just one form of "proto". I hope it is now clear.
Assuming textfile with replacement patterns has exact 384 lines you can use this awk command:
awk 'FNR==NR{a[++i]=$0; next} /proto/{sub(/proto/, a[++j])} 1' replacement.txt file.txt
This might work for you (GNU sed):
sed 's|.*|s/proto/&/|' replacementFile | sed -e ':a;$!{N;ba}' -f - textFile
This generates a substitution command for every replacement in the replacementFile and reads this command set into sed script which has slurped the textFile into memory.
N.B. This assumes that there is a replacement for every pattern in the textFile.
I have a text file with RGB data in the form of:
[Pixel 0,0] [Pixel 1,0] [Pixel 2,0]...
[Pixel 0,1] [Pixel 1,1] [Pixel 2,2]...
...
With an input of:
0.0 0.0 0.0 <-- this would be Pixel 0,0
1.0 0.0 0.0
1.0 0.9 0.0
I can create the flag of Germany in size 3x1 with:
%load the data to myData
Germany = reshape(myData,3,1,3);
image(Germany)
The 1px-wide pattern works good as show in picture, however, the goal is to be able to create multiple patterns, e.g. the Germany flag in 3x3 followed by Romania flag in 3x3 or any other pattern of any length and doing that! is where I can not find the proper way to reshape the matrix.
The input that should create the second example shown in picture is this:
|========= Germany Flag ==========| [ Blue ] [ Yellow ] [ Red ]
Black -> 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.9 0.0 1.0 0.0 0.0
Red -> 1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.9 0.0 1.0 0.0 0.0
Yellow-> 1.0 0.9 0.0 1.0 0.9 0.0 1.0 0.9 0.0 0.0 0.0 1.0 1.0 0.9 0.0 1.0 0.0 0.0
Any help is appreciated
Update: Asked by Marcin, the input files are literal as I explained above.
This is the content of the GermanyRomania.txt file:
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.9 0.0 1.0 0.0 0.0
1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.9 0.0 1.0 0.0 0.0
1.0 0.9 0.0 1.0 0.9 0.0 1.0 0.9 0.0 0.0 0.0 1.0 1.0 0.9 0.0 1.0 0.0 0.0
With that file I must create the 2nd pattern in picture (German+Romania Flag), there is ALL the RGB info required to do it.
I don't think you can achieve what you want by simply using the reshape function.
We must take into account that Matlab stores matrices in column-major order (you can read more about it here).
Therefore, before we can use the reshape function, we must have the data matrix in the following format:
[Pixel 0,0]
[Pixel 0,1]
...
[Pixel 1,0]
[Pixel 1,1]
...
[Pixel n,n]
Here's a possible solution:
# data stores the input
height = size(data, 1)
width = size(data, 2)
vertical_data_cell = mat2cell(data, height, 3 * ones(1, width / 3))'
vertical_data = cell2mat(vertical_data_cell)
flags = reshape(vertical_data, height, width / 3, 3)
image(flags)
Note that we make the matrix transformation on lines 4 and 5.
And here is the result for the input you provided:
It also works with different heights.
Here's the input for the flags of Germany, Argentina and Portugal.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.46 0.66 0.85 0.46 0.66 0.85 0.46 0.66 0.85
1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 1.0 1.0 1.0 0.98 0.75 0.29 1.0 1.0 1.0
1.0 0.9 0.0 1.0 0.9 0.0 1.0 0.9 0.0 0.46 0.66 0.85 0.46 0.66 0.85 0.46 0.66 0.85
0.0 1.0 0.0 0.0 1.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0
0.0 1.0 0.0 1.0 0.9 0.0 1.0 0.9 0.0 1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0
0.0 1.0 0.0 0.0 1.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0 1.0 0.0 0.0
And this is the result:
using bash script.
example text file is:
<Mixture> 1 1.0000
<Mean> 39
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
<Variance> 39
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
<TransP> 3
now i WANT TO replace the sequence of zeros with some other array which is matlab variable.i.e I want it to be as follows
<Mixture> 1 1.0000
<Mean> 39
-3.928404e+00 -2.833521e+00 1.418979e+00 -8.560805e+00 -4.987627e+00 -5.422057e+00 -2.148949e+00 -4.035418e+00 1.111476e+00 -4.360466e+00 -2.430894e+00 -3.562756e+00 2.149250e+01 4.148085e-03 3.280356e-01 1.993892e-01 4.226578e-01 3.070029e-01 3.061973e-01 2.200327e-01 2.279716e-01 1.851751e-01 2.630977e-01 2.530313e-01 1.584220e-01 -7.420680e-02 -6.129631e-02 6.845896e-03 1.414505e-02 5.905741e-02 7.828339e-02 6.020883e-02 7.195320e-02 4.838301e-02 4.507983e-02 6.380487e-02 7.884157e-02 6.597416e-02 -1.690722e-02
<Variance> 39
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
<TransP> 3
Assuming $matlabvar contains the data you received from Matlab:
sed "s/ 0\.0\( 0\.0\)*/ $matlabvar/" input.txt > output.txt
This is regular expression match and substitution.
s/old/new/ - replace text that's matched by the old regular expression with the new text
0 - matches 0
\. - matches .
\(...\) - this groups the contained elements
* - this matches 0 or more of the preceding expression (the grouped construct).
So it matches one 0.0 followed by any number of additional 0.0s. This is then replaced with $matlabvar.
You can learn all about regular expressions at regular-expressions.info.
<Mean> 39
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
<Variance> 39
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
<State> 3
0.000e+0 1.000e+0 0.000e+0 0.000e+0 0.000e+0
0.000e+0 5.000e-1 5.000e-1 0.000e+0 0.000e+0
0.000e+0 0.000e+0 5.000e-1 5.000e-1 0.000e+0
0.000e+0 0.000e+0 0.000e+0 5.000e-1 5.000e-1
0.000e+0 0.000e+0 0.000e+0 0.000e+0 0.000e+0
that is My total text file
Unfortunately
The suggested command will replace the zeros in the above matrix form also but i do not want this.Also another question is how to replace a general sequence (we do not know how pattern is) with another general sequence. I think it should be done only by the line numbers but here another problem is we do not know how many lines the sequence is. so is there any way?