Say I have a IxJ matrix of values,
V= [1,4;2,5;3,6];
and a IxR matrix X of indexes,
X = [1 2 1 ; 1 2 2 ; 2 1 2];
I want to get a matrix Vx that is IxR such that for each row i, I want to read R times a (potentially) different column of V, which are given by the numbers in each corresponding column in X.
Vx(i,r) = V(i,X(i,r)).
For instance in this case it would be
Vx = [1,4,1;2,5,5;6,3,6];
Any help to do this fast, (without any looping) is much appreciated!
So what you want to achieve is using vectorization to achieve speed. This is one of the major strength of MATLAB. What you want is a matrix (index in the following code) whose elements are linear indexes that will be used to pick out value from the source matrix(V in your case). The first two lines of codes are doing exactly the same thing as sub2ind, turning subscripts to linear indexes. I'm coding this way so the logic of index conversion is clear.
[m,n] = ndgrid(1:size(X,1),1:size(X,2));
index = m + (X-1)*size(X,1);
Vx = V(index);
You can use bsxfun for an efficient solution -
N = size(V,1)
Vx = V(bsxfun(#plus,[1:N]',(X-1)*N))
Sample run -
>> V
V =
1 4
2 5
3 6
>> X
X =
1 2 1
1 2 2
2 1 2
>> N = size(V,1);
Vx = V(bsxfun(#plus,[1:N]',(X-1)*N))
Vx =
1 4 1
2 5 5
6 3 6
Another method would be to use repmat combined with sub2ind. sub2ind takes in row and column locations and the output are column-major linear indices that you can use to vectorize access into a matrix. Specifically, you want to build a 2D matrix of row indices and column indices which is the same size as X where the column indices are exactly specified as X but the row indices are the same for each row that we're concerned with. Concretely, the first row of this matrix will be all 1s, the next row all 2s, etc. To build this row matrix, first generate a column vector that goes from 1 up to as many rows as there are X and replicate this for as many columns as there are in X. With this new matrix and X, use sub2ind to generate column-major linear indices to finally index V to produce the matrix Vx:
subs = repmat((1:size(X,1)).', [1 size(X,2)]); %'
ind = sub2ind(size(X), subs, X);
Vx = V(ind);
Related
I have two uint16 3D (GPU) arrays A and B in MATLAB, which have the same 2nd and 3rd dimension. For instance, size(A,1) = 300 000, size(B,1) = 2000, size(A,2) = size(B,2) = 20, and size(A,3) = size(B,3) = 100, to give an idea about the orders of magnitude. Actually, size(A,3) = size(B,3) is very big, say ~ 1 000 000, but the arrays are stored externally in small pieces cut along the 3rd dimension. The point is that there is a very long loop along the 3rd dimension (cfg. MWE below), so the code inside of it needs to be optimized further (if possible). Furthermore, the values of A and B can be assumed to be bounded way below 65535, but there are still hundreds of different values.
For each i,j, and d, the rows A(i,:,d) and B(j,:,d) represent multisets of the same size, and I need to find the size of the largest common submultiset (multisubset?) of the two, i.e. the size of their intersection as multisets. Moreover, the rows of B can be assumed sorted.
For example, if [2 3 2 1 4 5 5 5 6 7] and [1 2 2 3 5 5 7 8 9 11] are two such multisets, respectively, then their multiset intersection is [1 2 2 3 5 5 7], which has the size 7 (7 elements as a multiset).
I am currently using the following routine to do this:
s = 300000; % 1st dim. of A
n = 2000; % 1st dim. of B
c = 10; % 2nd dim. of A and B
depth = 10; % 3rd dim. of A and B (corresponds to a batch of size 10 of A and B along the 3rd dim.)
N = 100; % upper bound on the possible values of A and B
A = randi(N,s,c,depth,'uint16','gpuArray');
B = randi(N,n,c,depth,'uint16','gpuArray');
Sizes_of_multiset_intersections = zeros(s,n,depth,'uint8'); % too big to fit in GPU memory together with A and B
for d=1:depth
A_slice = A(:,:,d);
B_slice = B(:,:,d);
unique_B_values = permute(unique(B_slice),[3 2 1]); % B is smaller than A
% compute counts of the unique B-values for each multiset:
A_values_counts = permute(sum(uint8(A_slice==unique_B_values),2,'native'),[1 3 2]);
B_values_counts = permute(sum(uint8(B_slice==unique_B_values),2,'native'),[1 3 2]);
% compute the count of each unique B-value in the intersection:
Sizes_of_multiset_intersections_tmp = gpuArray.zeros(s,n,'uint8');
for i=1:n
Sizes_of_multiset_intersections_tmp(:,i) = sum(min(A_values_counts,B_values_counts(i,:)),2,'native');
end
Sizes_of_multiset_intersections(:,:,d) = gather(Sizes_of_multiset_intersections_tmp);
end
One can also easily adapt above code to compute the result in batches along dimension 3 rather than d=1:depth (=batch of size 1), though at the expense of even bigger unique_B_values vector.
Since the depth dimension is large (even when working in batches along it), I am interested in faster alternatives to the code inside the outer loop. So my question is this: is there a faster (e.g. better vectorized) way to compute sizes of intersections of multisets of equal size?
Disclaimer : This is not a GPU based solution (Don't have a good GPU). I find the results interesting and want to share, but I can delete this answer if you think it should be.
Below is a vectorized version of your code, that makes it possible to get rid of the inner loop, at the cost of having to deal with a bigger array, that might be too big to fit in the memory.
The idea is to have the matrices A_values_counts and B_values_counts be 3D matrices shaped in such a way that calling min(A_values_counts,B_values_counts) will calculate everything in one go due to implicit expansion. In the background it will create a big array of size s x n x length(unique_B_values) (Probably most of the time too big)
In order to go around the constraint on the size, the results are calculated in batches along the n dimension, i.e. the first dimension of B:
tic
nBatches_B = 2000;
sBatches_B = n/nBatches_B;
Sizes_of_multiset_intersections_new = zeros(s,n,depth,'uint8');
for d=1:depth
A_slice = A(:,:,d);
B_slice = B(:,:,d);
% compute counts of the unique B-values for each multiset:
unique_B_values = reshape(unique(B_slice),1,1,[]);
A_values_counts = sum(uint8(A_slice==unique_B_values),2,'native'); % s x 1 x length(uniqueB) array
B_values_counts = reshape(sum(uint8(B_slice==unique_B_values),2,'native'),1,n,[]); % 1 x n x length(uniqueB) array
% Not possible to do it all in one go, must split in batches along B
for ii = 1:nBatches_B
Sizes_of_multiset_intersections_new(:,((ii-1)*sBatches_B+1):ii*sBatches_B,d) = sum(min(A_values_counts,B_values_counts(:,((ii-1)*sBatches_B+1):ii*sBatches_B,:)),3,'native'); % Vectorized
end
end
toc
Here is a little benchmark with different values of the number of batches. You can see that a minimum is found around a number of 400 (batch size 50), with a decrease of around 10% in processing time (each point is an average over 3 runs). (EDIT : x axis is amount of batches, not batches size)
I'd be interested in knowing how it behaves for GPU arrays as well!
I have a nX2 matrix A and a 3D matrix K. I would like to take element-wise multiplication specifying 2 indices in 3rd dimension of K designated by each row vector in A and take summation of them.
For instance of a simplified example when n=2,
A=[1 2;3 4];%2X2 matrix
K=unifrnd(0.1,0.1,2,2,4);%just random 3D matrix
L=zeros(2,2);%save result to here
for t=1:2
L=L+prod(K(:,:,A(t,:)),3);
end
Can I get rid of the for loop in this case?
How's this?
B = A.'; %'
L = squeeze(sum(prod(...
reshape(permute(K(:,:,B(:)),[3 1 2]),2,[],size(K,1),size(K,2)),...
1),...
2));
Although your test case is too simple, so I can't be entirely sure that it's correct.
The idea is that we first take all the indices in A, in column-major order, then reshape the elements of K such that the first two dimensions are of size [2, n], and the second two dimensions are the original 2 of K. We then take the product, then the sum along the necessary dimensions, ending up with a matrix that has to be squeezed to get a 2d matrix.
Using a bit more informative test case:
K = rand(2,3,4);
A = randi(4,4,2);
L = zeros(2,3);%save result to here
for t=1:size(A,1)
L = L+prod(K(:,:,A(t,:)),3);
end
B = A.'; %'
L2 = squeeze(sum(prod(reshape(permute(K(:,:,B(:)),[3 1 2]),2,[],size(K,1),size(K,2)),1),2));
Then
>> isequal(L,L2)
ans =
1
With some reshaping magic -
%// Get sizes
[m1,n1,r1] = size(K);
[m2,n2] = size(A);
%// Index into 3rd dim of K; perform reductions and reshape back
Lout = reshape(sum(prod(reshape(K(:,:,A'),[],n2,m2),2),3),m1,n1);
Explanation :
Index into the third dimension of K with a transposed version of A (transposed because we are using rows of A for indexing).
Perform the prod() and sum() operations.
Finally reshape back to a shape same as K but without the third dimension as that was removed in the earlier reduction steps.
I want to convert 127 by 4 matrix to 1 by 4 matrix such that each value in the output row equivalent to average of all values in that particular column.
Simply use the mean function:
A = rand(127,4);
B = mean(A,1); % Average of A along the first dimension
Best,
So I have the following constraints:
How to write this in MATLAB in an efficient way? The inputs are x_mn, M, and N. The set B={1,...,N} and the set U={1,...,M}
I did it like this (because I write x as the follwoing vector)
x=[x_11, x_12, ..., x_1N, X_21, x_22, ..., x_M1, X_M2, ..., x_MN]:
%# first constraint
function R1 = constraint_1(M, N)
ee = eye(N);
R1 = zeros(N, N*M);
for m = 1:M
R1(:, (m-1)*N+1:m*N) = ee;
end
end
%# second constraint
function R2 = constraint_2(M, N)
ee = ones(1, N);
R2 = zeros(M, N*M);
for m = 1:M
R2(m, (m-1)*N+1:m*N) = ee;
end
end
By the above code I will get a matrix A=[R1; R2] with 0-1 and I will have A*x<=1.
For example, M=N=2, I will have something like this:
And, I will create a function test(x) which returns true or false according to x.
I would like to get some help from you and optimize my code.
You should place your x_mn values in a matrix. After that, you can sum in each dimension to get what you want. Looking at your constraints, you will place these values in an M x N matrix, where M is the amount of rows and N is the amount of columns.
You can certainly place your values in a vector and construct your summations in the way you intended earlier, but you would have to write for loops to properly subset the proper elements in each iteration, which is very inefficient. Instead, use a matrix, and use sum to sum over the dimensions you want.
For example, let's say your values of x_mn ranged from 1 to 20. B is in the set from 1 to 5 and U is in the set from 1 to 4. As such:
X = vec2mat(1:20, 5)
X =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
vec2mat takes a vector and reshapes it into a matrix. You specify the number of columns you want as the second element, and it will create the right amount of rows to ensure that a proper matrix is built. In this case, I want 5 columns, so this should create a 4 x 5 matrix.
The first constraint can be achieved by doing:
first = sum(X,1)
first =
34 38 42 46 50
sum works for vectors as well as matrices. If you have a matrix supplied to sum, you can specify a second parameter that tells you in what direction you wish to sum. In this case, specifying 1 will sum over all of the rows for each column. It works in the first dimension, which is the rows.
What this is doing is it is summing over all possible values in the set B over all values of U, which is what we are exactly doing here. You are simply summing every single column individually.
The second constraint can be achieved by doing:
second = sum(X,2)
second =
15
40
65
90
Here we specify 2 as the second parameter so that we can sum over all of the columns for each row. The second dimension goes over the columns. What this is doing is it is summing over all possible values in the set U over all values of B. Basically, you are simply summing every single row individually.
BTW, your code is not achieving what you think it's achieving. All you're doing is simply replicating the identity matrix a set number of times over groups of columns in your matrix. You are actually not performing any summations as per the constraint. What you are doing is you are simply ensuring that this matrix will have the conditions you specified at the beginning of your post to be enforced. These are the ideal matrices that are required to satisfy the constraints.
Now, if you want to check to see if the first condition or second condition is satisfied, you can do:
%// First condition satisfied?
firstSatisfied = all(first <= 1);
%// Second condition satisfied
secondSatisfied = all(second <= 1);
This will check every element of first or second and see if the resulting sums after you do the above code that I just showed are all <= 1. If they all satisfy this constraint, we will have true. Else, we have false.
Please let me know if you need anything further.
I am new to Octave. I have two matrices. I have to compare a particular column of a one matrix with the other(my matrix A is containing more than 5 variables, similarly matrix B is containing the same.) and if elements in column one of matrix A is equal to elements in the second matrix B then I have to use the third column of second matrix B to compute certain values.I am doing this with octave by using for loop , but it consumes a lot of time to do the computation for single day , i have to do this for a year . Because size of matrices is very large.Please suggest some alternative way so that I can reduce my time and computation.
Thank you in advance.
Thanks for your quick response -hfs
continuation of the same problem,
Thank u, but this will work only if both elements in both the rows are equal.For example my matrices are like this,
A=[1 2 3;4 5 6;7 8 9;6 9 1]
B=[1 2 4; 4 2 6; 7 5 8;3 8 4]
here column 1 of first element of A is equal to column 1 of first element of B,even the second column hence I can take the third element of B, but for the second element of column 1 is equal in A and B ,but second element of column 2 is different ,here it should search for that element and print the element in the third column,and am doing this with for loop which is very slow because of larger dimension.In mine actual problem I have given for loop as written below:
for k=1:37651
for j=1:26018
if (s(k,1:2)==l(j,1:2))
z=sin((90-s(k,3))*pi/180) , break ,end
end
end
I want an alternative way to do this which should be faster than this.
You should work with complete matrices or vectors whenever possible. You should try commands and inspect intermediate results in the interactive shell to see how they fit together.
A(:,1)
selects the first column of a matrix. You can compare matrices/vectors and the result is a matrix/vector of 0/1 again:
> A(:,1) == B(:,1)
ans =
1
1
0
If you assign the result you can use it again to index into matrices:
I = A(:,1) == B(:,1)
B(I, 3)
This selects the third column of B of those rows where the first column of A and B is equal.
I hope this gets you started.