I have a large matrix which I have stored in the following format, given the matrix A;
A =
1 0 3
5 1 -2
0 0 7
3 vectors;
NVPN = [1 3 4 7] - I arbitrarily put a 1 in the first column, then from the second onwards it is a cumulatively summing the number of non-zero elements per column.
NNVI = [1 2 2 1 2 3] - row index of each non-zero element.
CONT = [1 5 1 3 -2 7] - value of each non-zero element.
I now need to perform matrix*matrix multiplication and matrix*vector multiplication. Does anyone know if the are any FORTRAN libraries, which I can amend to fit my problem, to do this above?
Thanks in advance
The MATMUL function allows you to perform matrix products, which is defined in the section 13.7.70 of the FORTRAN 90 standard. See also: GCC reference.
There is already a topic on sparse matrix libraries here.
Related
I (new to PyMC3) want to extend the model proposed in the PyMC3 example A Hierarchical model for Rugby prediction by making the latent variables for attack and defence strength autoregressive. I am unsure about how to use 2-dimensional data and the shape parameter of the GaussianRandomWalk class (explained below).
Edit 1: I did not find explicit documentation of multi-dimensional usage, but I found this comment by fonnesbeck among the PyMC3 github issues:
[...] I think most people would expect a vector of variables, which implies that the first dimension is the number of variable elements and the remaining dimension(s) the size of each variable.
As defined below, I use the time index as my 1st dimension. I tried to switch axis, which yields the same result. So, my current model is:
with pm.Model() as model:
home = pm.Normal('home', 0, .0001)
intercept = pm.Normal('intercept', 0, .0001)
tau_att = pm.Exponential('tau_att', 1./.02)
tau_def = pm.Exponential('tau_def', 1./.02)
atts = pm.GaussianRandomWalk('atts', tau_att**-2, shape=[T, num_teams])
defs = pm.GaussianRandomWalk('defs', tau_def**-2, shape=[T, num_teams])
home_theta = tt.exp(intercept + home + atts[:, home_team] + defs[:, away_team])
away_theta = tt.exp(intercept + atts[:, away_team] + defs[:, home_team])
home_points = pm.Poisson('home_points', mu=home_theta, observed=observed_home_goals)
away_points = pm.Poisson('away_points', mu=away_theta, observed=observed_away_goals)
The input is a two dimensional array, with rows being the time steps and columns containing the home or away goals for all teams that played in that time step. Assuming the following mock data
home_score away_score home_team away_team i_home i_away t
0 1 0 Arsenal Liverpool 0 1 1
1 1 1 Liverpool Burnley 1 2 1
2 2 4 Burnley Arsenal 2 0 1
3 0 3 Liverpool Arsenal 1 0 2
4 1 1 Burnley Liverpool 2 1 2
5 5 0 Arsenal Burnley 0 2 2
observed_home_goals (similar to observed_away_goals) would look like this:
[[1 1 2]
[0 1 5]]
and the corresponding team index would look like this:
[[0 1 2]
[1 2 0]]
meaning that in time step 1, teams [0 1 2] shot [1 1 2] goals respectively.
Fitting the model does not throw errors, sampling however yields zero estimates for all parameters. I tried to browse the distributions/timeseries.py source code about how the shape parameter would be used for multiple dimensions in the GaussianRandomWalk class.
My question is, if the model definition would actually work as intended for the 2-dimensional time series data. I am not sure, if I index the atts and defs variables correctly.
Edit 2: I ended up building the time series manually, similar to Javier's solution, which seems to work fine!
An algorithm that need process a matrix n x m that is scalable.
E.g. I have a time series of 3 seconds containing the values: 2,1,4.
I need to decompose it to take a 3 x 4 matrix, where 3 is the number of elements of time series and 4 the maximum value. The resulting matrix that would look like this:
1 1 1
1 0 1
0 0 1
0 0 1
Is this a bad solution or is it only considered a data entry problem?
The question is,
do I need to distribute information from each row of the matrix for various elements without losing the values?
I have a matrix M=[4 3 2 1;1 2 3 4]. I want to append different size matrices at each iteration:
M=[4 3 2 1;1 2 3 4];
for i=1:t
newM=createNewMatrix;
M=[M;newM];
end
newM can be [] or a Nx4 matrix. This is very slow though. What is the fastest way to do this?
Update
Pre-allocating would look like this?
M=zeros(200000,4)
start=1
M(1:2,:)=M=[4 3 2 1;1 2 3 4];
for i=1:t
newM=createNewMatrix;
size_of_newM=size(newM,1);
finish=start+size_of_newM-1;
M(start:finish,:)=newM;
start=finish;
end
Like suggested, preallocation gives the most boost.
Using cell arrays is another good approach and could be implemented like this:
M = cell(200000, 1);
M{1} = [4 3 2 1; 1 2 3 4];
for t=2:200000
i = randi(3)-1;
M{t}=rand(i,4);
end
MC = vertcat(M{:});
In principle you generate a cell array with arbitrary long arrays in each cell and then concatenate them afterwards.
This worked for me nearly twice as fast as your preallocation update. On the other hand, this still was only around one second for the example with 200k iterations...
I've been searching for an algorithm for the solution of all possible matrices of dimension 'n' that can be obtained with two arrays, one of the sum of the rows, and another, of the sum of the columns of a matrix. For example, if I have the following matrix of dimension 7:
matriz= [ 1 0 0 1 1 1 0
1 0 1 0 1 0 0
0 0 1 0 1 0 0
1 0 0 1 1 0 1
0 1 1 0 1 0 1
1 1 1 0 0 0 1
0 0 1 0 1 0 1 ]
The sum of the columns are:
col= [4 2 5 2 6 1 4]
The sum of the rows are:
row = [4 3 2 4 4 4 3]
Now, I want to obtain all possible matrices of "ones and zeros" where the sum of the columns and the rows fulfil the condition of "col" and "row" respectively.
I would appreciate ideas that can help solve this problem.
One obvious way is to brute-force a solution: for the first row, generate all the possibilities that have the right sum, then for each of these, generate all the possibilities for the 2nd row, and so on. Once you have generated all the rows, you check if the sum of the columns is right. But this will take a lot of time. My math might be rusty at this time of the day, but I believe the number of distinct possibilities for a row of length n of which k bits are 1 is given by the binomial coefficient or nchoosek(n,k) in Matlab. To determine the total number of possibilities, you have to multiply this number for every row:
>> n = 7;
>> row= [4 3 2 4 4 4 3];
>> prod(arrayfun(#(k) nchoosek(n, k), row))
ans =
3.8604e+10
This is a lot of possibilities to check! Doing the same for the columns gives
>> col= [4 2 5 2 6 1 4];
>> prod(arrayfun(#(k) nchoosek(n, k), col))
ans =
555891525
Still a large number, but 'only' a factor 70 smaller.
It might be possible to improve this brute-force method a little bit by seeing if the later rows are already constrained by the previous rows. If in your example, for a particular combination of the first two rows, both rows have a 1 in the second column, the rest of this column should all be 0, since the sum must be 2. This reduces the number of possibilities for the remaining rows a bit. Implementing such checks might complicate things a bit, but they might make the difference between a calculation that takes 2 days or one that takes just 1 hour.
An optimized version of this might alternatively generate rows and columns, and start with those for which the number of possibilities is the lowest. I don't know if there is a more elegant solution than this brute-force method, I would be interested to hear one.
i want to create 1D vector in matlab from given matrix,for this i have implemented following algorithm ,which use trivial way
% create one dimensional vector from 2D matrix
function [x]=one_dimensional(b,m,n)
k=1;
for i=1:m
for t=1:n
x(k)=b(i,t);
k=k+1;
end
end
x;
end
when i run it using following example,it seems to do it's task fine
b=[2 1 3;4 2 3;1 5 4]
b =
2 1 3
4 2 3
1 5 4
>> one_dimensional(b,3,3)
ans =
2 1 3 4 2 3 1 5 4
but generally i know that,arrays are not good way to use in matlab,because it's performance,so what should be effective way for transformation matrix into row/column vector?i am just care about performance.thanks very much
You can use the (:) operator...But it works on columns not rows so you need to transpose using the 'operator before , for example:
b=b.';
b(:)'
ans=
2 1 3 4 2 3 1 5 4
and I transposed again to get a row output (otherwise it'll the same vector only in column form)
or also, this is an option (probably a slower one):
reshape(b.',1,[])