Given the following three dimensional hankel matrix:
hankel_matrix = hankel(1:21, 21:999);
hankel_matrix = repmat(hankel_matrix, [1 1 no_of_weighted_composites]);
And the matrix:
composites_collection = permute(reshape(data_composites.', size(data_composites, 2),1,[]),[2 3 1]);
where data_composites is a 999 x 56 matrix.
Essentially what I want to achieve is use the hankel matrix to index the 56 different pages of single rows found in the composites collection.
I had assumed this would be as easy as data_composites_collection(hankel_matrix) but that simply uses the first page of data_composites_collection and repeats this for all 56 pages.
My current implementation is:
function subsequent_hankel_index = createMovingSnapshotsOfValues(~, matrix_of_numbers, data_composites, windows, no_of_weighted_composites)
data_composites_collection = permute(reshape(data_composites.', size(data_composites, 2),1,[]),[2 3 1]);
hankel_index = NaN(260, 999, no_of_weighted_composites, size(windows, 2));
for window_size = 1:size(windows, 2);
for composite = 1:no_of_weighted_composites;
hankel_matrix = hankel(1:windows(window_size), windows(window_size):length(matrix_of_numbers));
desired_row = data_composites_collection(:,:,composite);
[rows, columns] = size(desired_row(hankel_matrix));
hankel_index(1:rows,1:columns,composite,window_size) = desired_row(hankel_matrix);
subsequent_hankel_index = hankel_index;
end
end
end
However, this is incredibly slow for the amount of data I have. I have assumed vectorising the above would greatly help the speed of my programme. Any tips or approaches to the indexing would be greatly appreciated.
Many thanks in advance!
Related
I have 200 vectors; each one has a length of 10000.
I want to fill a matrix such that each line represents a vector.
If your vectors are already stored in an array then you can use vcat( ) here:
A = [rand(10000)' for idx in 1:200]
B = vcat(A...)
Julia stores matrices in column-major order so you are going to have to adapt a bit to that
If you have 200 vectors of length 100000 you should make first an empty vector, a = [], this will be your matrix
Then you have to vcat the first vector to your empty vector, like so
v = your vectors, however they are defined
a = []
a = vcat(a, v[1])
Then you can iterate through vectors 2:200 by
for i in 2:200
a = hcat(a,v[i])
end
And finally transpose a
a = a'
Alternatively, you could do
a = zeros(200,10000)
for i in 1:length(v)
a[i,:] = v[i]
end
but I suppose that wont be as fast, if performance is at all an issue, because as I said, julia stores in column major order so access will be slower
EDIT from reschu's comment
a = zeros(10000,200)
for i in 1:length(v)
a[:,i] = v[i]
end
a = a'
Suppose I have:
A = rand(1,10,3);
B = rand(10,16);
And I want to get:
C(:,1) = A(:,:,1)*B;
C(:,2) = A(:,:,2)*B;
C(:,3) = A(:,:,3)*B;
Can I somehow multiply this in a single line so that it is faster?
What if I create new tensor b like this
for i = 1:3
b(:,:,i) = B;
end
Can I multiply A and b to get the same C but faster? Time taken in creation of b by the loop above doesn't matter since I will be needing C for many different A-s while B stays the same.
Permute the dimensions of A and B and then apply matrix multiplication:
C = B.'*permute(A, [2 3 1]);
If A is a true 3D array, something like A = rand(4,10,3) and assuming that B stays as a 2D array, then each A(:,:,1)*B would yield a 2D array.
So, assuming that you want to store those 2D arrays as slices in the third dimension of output array, C like so -
C(:,:,1) = A(:,:,1)*B;
C(:,:,2) = A(:,:,2)*B;
C(:,:,3) = A(:,:,3)*B; and so on.
To solve this in a vectorized manner, one of the approaches would be to use reshape A into a 2D array merging the first and third dimensions and then performing matrix-muliplication. Finally, to bring the output size same as the earlier listed C, we need a final step of reshaping.
The implementation would look something like this -
%// Get size and then the final output C
[m,n,r] = size(A);
out = permute(reshape(reshape(permute(A,[1 3 2]),[],n)*B,m,r,[]),[1 3 2]);
Sample run -
>> A = rand(4,10,3);
B = rand(10,16);
C(:,:,1) = A(:,:,1)*B;
C(:,:,2) = A(:,:,2)*B;
C(:,:,3) = A(:,:,3)*B;
>> [m,n,r] = size(A);
out = permute(reshape(reshape(permute(A,[1 3 2]),[],n)*B,m,r,[]),[1 3 2]);
>> all(C(:)==out(:)) %// Verify results
ans =
1
As per the comments, if A is a 3D array with always a singleton dimension at the start, you can just use squeeze and then matrix-multiplication like so -
C = B.'*squeeze(A)
EDIT: #LuisMendo points out that this is indeed possible for this specific use case. However, it is not (in general) possible if the first dimension of A is not 1.
I've grappled with this for a while now, and I've never been able to come up with a solution. Performing element-wise calculations is made nice by bsxfun, but tensor multiplication is something which is woefully unsupported. Sorry, and good luck!
You can check out this mathworks file exchange file, which will make it easier for you and supports the behavior you're looking for, but I believe that it relies on loops as well. Edit: it relies on MEX/C++, so it isn't a pure MATLAB solution if that's what you're looking for.
I have to agree with #GJSein, the for loop is really fast
time
0.7050 0.3145
Here's the timer function
function time
n = 1E7;
A = rand(1,n,3);
B = rand(n,16);
t = [];
C = {};
tic
C{length(C)+1} = squeeze(cell2mat(cellfun(#(x) x*B,num2cell(A,[1 2]),'UniformOutput',false)));
t(length(t)+1) = toc;
tic
for i = 1:size(A,3)
C{length(C)+1}(:,i) = A(:,:,i)*B;
end
t(length(t)+1) = toc;
disp(t)
end
for an input matrix
in = [1 1;
1 2;
1 3;
1 4;
2 5;
2 6;
2 7;
3 8;
3 9;
3 10;
3 11];
i want to get the output matrix
out = [1 5 8;
2 6 9;
3 7 10;
4 0 11];
meaning i want to reshape the second input column into an output matrix, where all values corresponding to one value in the first input column are written into one column of the output matrix.
As there can be different numbers of entries for each value in the first input column (here 4 values for "1" and "3", but only 3 for "2"), the normal reshape function is not applicable. I need to pad all columns to the maximum number of rows.
Do you have an idea how to do this matlab-ish?
The second input column can only contain positive numbers, so the padding values can be 0, -x, NaN, ...
The best i could come up with is this (loop-based):
maxNumElem = 0;
for i=in(1,1):in(end,1)
maxNumElem = max(maxNumElem,numel(find(in(:,1)==i)));
end
out = zeros(maxNumElem,in(end,1)-in(1,1));
for i=in(1,1):in(end,1)
tmp = in(in(:,1)==i,2);
out(1:length(tmp),i) = tmp;
end
Either of the following approaches assumes that column 1 of in is sorted, as in the example. If that's not the case, apply this initially to sort in according to that criterion:
in = sortrows(in,1);
Approach 1 (using accumarray)
Compute the required number of rows, using mode;
Use accumarray to gather the values corresponding to each column, filled with zeros at the end. The result is a cell;
Concatenate horizontally the contents of all cells.
Code:
[~, n] = mode(in(:,1)); %//step 1
out = accumarray(in(:,1), in(:,2), [], #(x){[x; zeros(n-numel(x),1)]}); %//step 2
out = [out{:}]; %//step 3
Alternatively, step 1 could be done with histc
n = max(histc(in(:,1), unique(in(:,1)))); %//step 1
or with accumarray:
n = max(accumarray(in(:,1), in(:,2), [], #(x) numel(x))); %//step 1
Approach 2 (using sparse)
Generate a row-index vector using this answer by #Dan, and then build your matrix with sparse:
a = arrayfun(#(x)(1:x), diff(find([1,diff(in(:,1).'),1])), 'uni', 0); %//'
out = full(sparse([a{:}], in(:,1), in(:,2)));
Introduction to proposed solution and Code
Proposed here is a bsxfun based masking approach that uses the binary operators available as builtins for use with bsxfun and as such I would consider this very appropriate for problems like this. Of course, you must also be aware that bsxfun is a memory hungry tool. So, it could pose a threat if you are dealing with maybe billions of elements depending also on the memory available for MATLAB's usage.
Getting into the details of the proposed approach, we get the counts of each ID from column-1 of the input with histc. Then, the magic happens with bsxfun + #le to create a mask of positions in the output array (initialized by zeros) that are to be filled by the column-2 elements from input. That's all you need to tackle the problem with this approach.
Solution Code
counts = histc(in(:,1),1:max(in(:,1)))'; %//' counts of each ID from column1
max_counts = max(counts); %// Maximum counts for each ID
mask = bsxfun(#le,[1:max_counts]',counts); %//'# mask of locations where
%// column2 elements are to be placed
out = zeros(max_counts,numel(counts)); %// Initialize the output array
out(mask) = in(:,2); %// place the column2 elements in the output array
Benchmarking (for performance)
The benchmarking presented here compares the proposed solution in this post against the various methods presented in Luis's solution. This skips the original loopy approach presented in the problem as it appeared to be very slow for the input generated in the benchmarking code.
Benchmarking Code
num_ids = 5000;
counts_each_id = randi([10 100],num_ids,1);
num_runs = 20; %// number of iterations each approach is run for
%// Generate random input array
in = [];
for k = 1:num_ids
in = [in ; [repmat(k,counts_each_id(k),1) rand(counts_each_id(k),1)]];
end
%// Warm up tic/toc.
for k = 1:50000
tic(); elapsed = toc();
end
disp('------------- With HISTC + BSXFUN Masking approach')
tic
for iter = 1:num_runs
counts = histc(in(:,1),1:max(in(:,1)))';
max_counts = max(counts);
out = zeros(max_counts,numel(counts));
out(bsxfun(#le,[1:max_counts]',counts)) = in(:,2);
end
toc
clear counts max_counts out
disp('------------- With MODE + ACCUMARRAY approach')
tic
for iter = 1:num_runs
[~, n] = mode(in(:,1)); %//step 1
out = accumarray(in(:,1), in(:,2), [], #(x){[x; zeros(n-numel(x),1)]}); %//step 2
out = [out{:}];
end
toc
clear n out
disp('------------- With HISTC + ACCUMARRAY approach')
tic
for iter = 1:num_runs
n = max(histc(in(:,1), unique(in(:,1))));
out = accumarray(in(:,1), in(:,2), [], #(x){[x; zeros(n-numel(x),1)]}); %//step 2
out = [out{:}];
end
toc
clear n out
disp('------------- With ARRAYFUN + Sparse approach')
tic
for iter = 1:num_runs
a = arrayfun(#(x)(1:x), diff(find([1,diff(in(:,1).'),1])), 'uni', 0); %//'
out = full(sparse([a{:}], in(:,1), in(:,2)));
end
toc
clear a out
Results
------------- With HISTC + BSXFUN Masking approach
Elapsed time is 0.598359 seconds.
------------- With MODE + ACCUMARRAY approach
Elapsed time is 2.452778 seconds.
------------- With HISTC + ACCUMARRAY approach
Elapsed time is 2.579482 seconds.
------------- With ARRAYFUN + Sparse approach
Elapsed time is 1.455362 seconds.
slightly better, but still uses a loop :(
out=zeros(4,3);%set to zero matrix
for i = 1:max(in(:,1)); %find max in column 1, and loop for that number
ind = find(in(:,1)==i); %
out(1: size(in(ind,2),1),i)= in(ind,2);
end
don't know if you can avoid the loop...
I have two matrices (tri1 and tri2) which represent a Delaunay triangulation. tri1 is the triangulation before inserting a new point, tri2 is the result after adding a new point. Each row has 4 columns. The rows represent tetrahedra.
I would like to calculate a relation between lines from tri1 to tri2. A result could look like this:
result =
1 1
2 2
3 3
4 4
0 0 % tri1(5, :) was not found in tri2 (a lot more lines could be missing)
6 5
7 6
8 7
9 8
10 9
Currently my source code looks like this:
% sort the arrays
[~, idx1] = sort(tri1(:, 1), 'ascend');
[~, idx2] = sort(tri2(:, 1), 'ascend');
stri1 = tri1(idx1, :);
stri2 = tri2(idx2, :);
result = zeros(size(tri1, 1), 2);
% find old cells in new triangulation
deleted = 0;
for ii = 1:size(tri1, 1)
found = false;
for jj = ii-deleted:size(tri2, 1)
if sum(stri1(ii, :) == stri2(jj, :)) == 4 % hot spot according to the profiler
found = true;
break;
end
if (stri1(ii, 1) < stri2(jj, 1)), break, end;
end
if found == false
deleted = deleted + 1;
else
result(idx1(ii), 1) = idx1(ii);
result(idx1(ii), 2) = idx2(jj);
end
end
The above source code gives me the results that I want, but not fast enough. I am not very experienced with MATLAB, I usually work with C++. My question: How can I speed up the comparison of two rows?
Some additional information (just in case):
the number of rows in tri can grow to about 10000
this function will be called once per inserted vertex (about 1000)
I cannot follow your example code completely, but judging from your explanation you want to see whether a row from matrix A occurs in matrix B.
In this case a very efficient implentation is available:
[Lia, Locb] = ismember(A,B,'rows');
Check the doc for more information about this function and see whether it is what you need.
I wish to ask if anybody out there knows how to partition an image into 8 different rows and 1 column? I have tried using mat2cell() and using the demo on their wiki as a reference, I tried partitioning the image into 8 rows, however not all image partition rows are displayed.
If you see the image below, 2, 4, 6, 8 is not displayed. I am also not sure why is it of 16 blocks.
Can somebody help me check my code? I am not really used to the MatLab syntax and language. I trying my best to understand now.
My code for splitting the blocks are as follows:
blockSizeR = 50; % Rows in block.
blockSizeC = 512; % Columns in block.
wholeBlockRows = floor(rows / blockSizeR);
blockVectorR = [blockSizeR * ones(1, wholeBlockRows), rem(rows, blockSizeR)];
wholeBlockCols = floor(columns / blockSizeC);
blockVectorC = [blockSizeC * ones(1, wholeBlockCols), rem(columns, blockSizeC)];
if numberOfColorBands > 1
% It's a color image.
ca = mat2cell(rgbImage, blockVectorR, blockVectorC, numberOfColorBands);
else
ca = mat2cell(rgbImage, blockVectorR, blockVectorC);
end
% Now display all the blocks.
plotIndex = 1;
numPlotsR = size(ca, 1);
numPlotsC = size(ca, 2);
for r = 1 : numPlotsR
for c = 1 : numPlotsC
fprintf('plotindex = %d, c=%d, r=%d\n', plotIndex, c, r);
% Specify the location for display of the image.
subplot(numPlotsR, 1, plotIndex);
% Extract the numerical array out of the cell
% just for tutorial purposes.
rgbBlock = ca{r,c};
imshow(rgbBlock); % Could call imshow(ca{r,c}) if you wanted to.
[rowsB columnsB numberOfColorBandsB] = size(rgbBlock);
% Make the caption the block number.
caption = sprintf('Block #%d of %d\n%d rows by %d columns', ...
plotIndex, numPlotsR*numPlotsC, rowsB, columnsB);
title(caption);
drawnow;
% Increment the subplot to the next location.
plotIndex = plotIndex + 1;
end
end
I am new to MatLab, so is there is a simpler method to do this that I missed out, please do suggest or better still, if there are references that I can refer to. Many thanks (:
If you know the dimensions of your matrix, you can do the math to figure out how to divide the number of rows into 4 equal parts:
e.g. If: size(rockinsMatrix) == [10 20] (a 10row x 20column) matrix,
then you could split it into a set of 4 sub-matrices, two with 3 rows, and 2 with 2 columns.
If you want the matrices in a cell array then you can do that at that time.
I managed to solve already, the error lies in the for loop. I changed the for r = 1 : numPlotsR into r = 1 : (number of rows I want) for c = 1 : numPlotsC into c= 1: 1(as I only want one column), and used subplot(8,1,k) or (8,2,k) where k is the plot index. Just answering this in case anybody encounter such problem in future and want to use my code as a reference. Cheers!