I'm a French student engineer in signal processing field and I do my internship in a neuroscience laboratory. I've to process a lot of data from the brain activity with the help of Matlab, so one of my main subject is to optimize the code. But now I'm stuck in a situation that I can't resolve and I don't find anything about it on the web. I explain my problem :
For example the matrix a :
a = [ 1 2 3 4 5; 6 7 8 9 10;11 12 13 14 15]
Each row is the datas of a signal (so here we have 3 signals), and I want, for each signal/row, cut the vector in blcoks interlinked of the same length.
For instance for the signal 1, S1 = [1 2 3 4 5] I want to extract the bloc S1_1 = [1 2 3], S1_2 = [2 3 4], and S1_3 = [3 4 5] and compute every sub-block.
My first idea was to use nested loop like that :
[nrow ncol] = size(a);
for i = 1 : nrow
for j = 4 : ncol
sub_block = a(i, (j-3):j);
result(i, j-3) = compute(sub_block);
end
end
BUT as I said I have to process a lot of datas, so I want to avoid for-loop. I'm looking for an algorithm wich will be able to remove these for-loop but I don't know how to do...
I saw the function 'reshape' but this on give me sub-block like : S1_1 = [1 2 3], S1_2 = [4 5 6] I can't use it because here in the sub-block S1_2 I have the data from the signal 1 and the signal 2.
Then I saw the function 'blockproc' but I didn't really understand how it process and I'm not really convaince that this one can help me...
So, I hope you understand my problem and that you could help me or indicate me a way to find a solution.
In addition to #Ziyao Wei's suggestion you could alternatively use im2col:
>> S = im2col(a', [3 1])
S =
1 2 3 6 7 8 11 12 13
2 3 4 7 8 9 12 13 14
3 4 5 8 9 10 13 14 15
Where S(:, 3*k-2:3*k) for k = 1:data_rows are the desired sub-blocks of row k of your data (a).
Blockproc seems to be doing a block operation rather than a sliding operation. A bit of digging around gives this:
http://www.mathworks.com/help/images/ref/nlfilter.html
But it seems the image processing toolbox is needed.
Also this might help:
http://dovgalecs.com/blog/matlab-sliding-window-easy-and-painless/
In general, try to search for sliding-window or convolution, and try to see if something shows up.
You could probably find another way of doing your loop with the arrayfun function, but the fact is that it might not necessarily be faster arrayfun can be significantly slower than an explicit loop in matlab. Why?
Thank you a lot for all your (quick) answers ! I didn't expect to get an answer so quickly !
I have the image processing toolbox and your different methods are greats ! I'll have to use the im2col because is the "slower" solution for me and I can remove one for-loop.
Thank you for your help
Related
I am still pretty new at programming and I would appreciate any help on how to approach the following problem:
Given a matrix (3x5)
a = [1 2 3 4 5;
6 7 8 9 10;
11 12 13 14 15;]
I want to iterate through every row
For each row, I want each element to be checked
With each element, I want to store a separate array that holds the element and the next 2 elements.
Ex:
Row 1 = [1 2 3 4 5]
For element 1
return newArray = [1 2 3]
For element 2
return newArray = [2 3 4]
Getting stuck on part 3. How to make the for loop check only up to the next 2 elements and then continue to the next element in the row.
I took a shot at solving what you asked for, but I agree with the others that you need to think more about what you are trying to do and what you want your output to look like. Your request does not sound like something a beginner programmer would realistically use. I am not sure what shape you want to store your "separate array"s in. I have options below for keeping them in a vector or in the original shape of a.
function nexttwo(row, i)
newarray::Vector{Any} = [row[i]]
for j=1:2
i+=1
if length(row) >= i
push!(newarray, row[i])
else
push!(newarray, nothing)
end
end
return newarray
end
function collectnexttwo(a)
result_collection = []
for i in axes(a,1)
for j in axes(a,2)
row = a[i,:]
newarray = nexttwo(row, j)
push!(result_collection, newarray)
end
end
return result_collection
end
function restoreshape(v, a)
permutedims(reshape(v, reverse(size(a))))
end
julia> a = [1 2 3 4 5; 6 7 8 9 10; 11 12 13 14 15;]
3×5 Matrix{Int64}:
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
julia> result = restoreshape(collectnexttwo(a), a)
3×5 Matrix{Any}:
Any[1, 2, 3] Any[2, 3, 4] Any[3, 4, 5] Any[4, 5, nothing] Any[5, nothing, nothing]
Any[6, 7, 8] Any[7, 8, 9] Any[8, 9, 10] Any[9, 10, nothing] Any[10, nothing, nothing]
Any[11, 12, 13] Any[12, 13, 14] Any[13, 14, 15] Any[14, 15, nothing] Any[15, nothing, nothing]
I think that you have some problems with the statement of what you want to achieve. That can often make a programming assignment much harder.
Restating what you have already:
I want to iterate through every row
This is pretty easy
for row = 1:size(a)[1]
...
end
For each row, I want each element to be checked
This is where things begin to get squishy? What do you mean by "checked". Let's assume you have some function called checkElement.
With each element, I want to store a separate array that holds the element and the next 2 elements.
How long do you want that separate array to live? Do you just want to hold 3 elements? Or three elements for every cell of the original (i.e. have a 3x5x3 result for a 3x5 input like you show)
Also, what do you want to do about elements 4 and 5 in each row? What values do you want to use for their "next" elements? You could use missing as a value or NaN. Or you could make the result just not contain the problematic inputs.
If you answer these questions, you are likely to find it much easier to write the code you need.
I have a matrix M=[4 3 2 1;1 2 3 4]. I want to append different size matrices at each iteration:
M=[4 3 2 1;1 2 3 4];
for i=1:t
newM=createNewMatrix;
M=[M;newM];
end
newM can be [] or a Nx4 matrix. This is very slow though. What is the fastest way to do this?
Update
Pre-allocating would look like this?
M=zeros(200000,4)
start=1
M(1:2,:)=M=[4 3 2 1;1 2 3 4];
for i=1:t
newM=createNewMatrix;
size_of_newM=size(newM,1);
finish=start+size_of_newM-1;
M(start:finish,:)=newM;
start=finish;
end
Like suggested, preallocation gives the most boost.
Using cell arrays is another good approach and could be implemented like this:
M = cell(200000, 1);
M{1} = [4 3 2 1; 1 2 3 4];
for t=2:200000
i = randi(3)-1;
M{t}=rand(i,4);
end
MC = vertcat(M{:});
In principle you generate a cell array with arbitrary long arrays in each cell and then concatenate them afterwards.
This worked for me nearly twice as fast as your preallocation update. On the other hand, this still was only around one second for the example with 200k iterations...
I have an image 640x480 img, and I want to replace pixels having values not in this list or array x=[1, 2, 3, 4, 5] with a certain value 10, so that any pixel in img which doesn't have the any of the values in x will be replaced with 10. I already know how to replace only one value using img(img~=1)=10 or multiple values using this img(img~=1 & img~=2 & img~=3 & img~=4 & img~=5)=10 but I when I tried this img(img~=x)=10 it gave an error saying Matrix dimensions must agree. So if anyone could please advise.
You can achieve this very easily with a combination of permute and bsxfun. We can create a 3D column vector that consists of the elements of [1,2,3,4,5], then use bsxfun with the not equals method (#ne) on your image (assuming grayscale) so that we thus create a 3D matrix of 5 slices. Each slice would tell you whether the locations in the image do not match an element in x. The first slice would give you the locations that don't match x = 1, the second slice would give you the locations that don't match x = 2, and so on.
Once you finish this, we can use an all call operating on the third dimension to consolidate the pixel locations that are not equal to all of 1, 2, 3, 4 or 5. The last step would be to take this logical map, which that tells you the locations that are none of 1, 2, 3, 4, or 5 and we'd set those locations to 10.
One thing we need to consider is that the image type and the vector x must be the same type. We can ensure this by casting the vector to be the same class as img.
As such, do something like this:
x = permute([1 2 3 4 5], [3 1 2]);
vals = bsxfun(#ne, img, cast(x, class(img)));
ind = all(vals, 3);
img(ind) = 10;
The advantage of the above method is that the list you want to use to check for the elements can be whatever you want. It prevents having messy logical indexing syntax, like img(img ~= 1 & img ~= 2 & ....). All you have to do is change the input list at the beginning line of the code, and bsxfun, permute and any should do the work for you.
Here's an example 5 x 5 image:
>> rng(123123);
>> img = randi(7, 5, 5)
img =
3 4 3 6 5
7 2 6 5 1
3 1 6 1 7
6 4 4 3 3
6 2 4 1 3
By using the code above, the output we get is:
img =
3 4 3 10 5
10 2 10 5 1
3 1 10 1 10
10 4 4 3 3
10 2 4 1 3
You can most certainly see that those elements that are neither 1, 2, 3, 4 or 5 get set to 10.
Aside
If you don't like the permute and bsxfun approach, one way would be to have a for loop and with an initially all true array, keep logical ANDing the final result with a logical map that consists of those locations which are not equal to each value in x. In the end, we will have a logical map where true are those locations that are neither equal to 1, 2, 3, 4 or 5.
Therefore, do something like this:
ind = true(size(img));
for idx = 1 : 5
ind = ind & img ~= idx;
end
img(ind) = 10;
If you do this instead, you'll see that we get the same answer.
Approach #1
You can use ismember,
which according to its official documentation for a case of ismember(A,B) would output a logical array of the same size as A and with 1's where
any element from B is present in A, 0's otherwise. Since, you are looking to detect "not in the list or array", you need to invert it afterwards, i.e. ~ismember().
In your case, you have img as A and x as B, so ~ismember(img,x) would give you those places where img~=any element in x
You can then map into img to set all those in it to 10 with this final solution -
img(~ismember(img,x)) = 10
Approach #2
Similar to rayryeng's solution, you can use bsxfun, but keep it in 2D which could be more efficient as it would also avoid permute. The implementation would look something like this -
img(reshape(all(bsxfun(#ne,img(:),x(:).'),2),size(img))) = 10
Based on this code and this one I'm familiar with implementing imfilter function.But as you know these kind of codes( using sequential for loops) would be very slow in matlab specially for high resolution images and they are more efficient in other programming languages. In matlab it's better to vectorize your code as much as possible.
Can anyone suggest me a way for vectorizing imfilter implementation?
Note: I know that I can use edit('imfilter') to study the code that developers have used for implementing imfilter function but it's pretty hard for me. I don't understand much of the codes. I'm pretty new to matlab.
Note: I know that some part of the codes introduced as an example could be vectorized very easily fore example padding section in this code could be implemented more easily.
But I'm thinking of a away for vectorizing the main part of the code (part of applying the filter). I mean the parts that are shown in the pictures:
I don't know how to vectorize these parts?
Oh, I forgot to tell that I have written the accepted answer for this question. Is there any way if I also don't want to use conv2 function?
There is a function just for you... it's called im2col. See http://www.mathworks.com/help/images/ref/im2col.html for a description. It allows you to turn "blocks" of the image into "columns" - if you are looking for 3x3 blocks to filter, each column will be 9 elements long. After that, the filter operation can be very simple. Here is an example:
n = 20; m = 30
myImg = rand(n, m)*255;
myImCol = im2col(myImg, [3 3], 'sliding');
myFilter = [1 2 1 2 4 2 1 2 1]';
myFilter = myFilter / sum(myFilter(:)); % to normalize
filteredImage = reshape( myImCol' * myFilter, n-2, m-2);
Didn't use conv2, and didn't use any explicit loops. This does, however, create an intermediate matrix which is a good deal bigger than the image (in this case, almost 9x). That could be a problem in its own right.
Disclaimer: I usually test Matlab code before posting, but could not connect to the license server. Let me know if you run into issues!
edit some further clarifications for you
1) Why reshape with n-2 and m-2? Well - the im2col function only returns "complete" columns for the blocks that it can create. When I create 3x3 blocks, the first one I can make is centered on (2,2), and the last one on (end-1, end-1). Thus the result is a bit smaller than the original image- it's "like padding". This is in fact the exact opposite of what happens when you use conv2 - in that case things get expanded. If you want to avoid that you could first expand your image with
paddedIm = zeros(n+2, m+2);
paddedIm(2:end-1, 2:end-1) = myImg;
and run the filter on the padded image.
2) The difference between 'sliding' and 'distinct' is best explained with an example:
>> M = magic(4)
M =
16 2 3 13
5 11 10 8
9 7 6 12
4 14 15 1
>> im2col(M,[2 2], 'distinct')
ans =
16 9 3 6
5 4 10 15
2 7 13 12
11 14 8 1
xx-- --xx ---- ----
xx-- --xx ---- ----
---- ---- xx-- --xx
---- ---- xx-- --xx
>> im2col(M,[2 2], 'sliding')
ans =
16 5 9 2 11 7 3 10 6
5 9 4 11 7 14 10 6 15
2 11 7 3 10 6 13 8 12
11 7 14 10 6 15 8 12 1
xx-- ---- ---- -xx-
xx-- xx-- ---- -xx- ... etc ...
---- xx-- xx-- ----
---- ---- xx-- ----
As you can see, the 'distinct' option returns non-overlapping blocks: the 'sliding' option returns "all blocks that fit" even though some will overlap.
3) The implementation of conv2 is likely some lower level code for speed - you may know about .mex files which allow you to write your own C code that can be linked with Matlab and gives you a big speed advantage? This is likely to be something like that. They do claim on their website that they use a "straightforward implementation" - so the speed is most likely just a matter of implementing in a fast manner (not "efficient Matlab").
The two inner loops can be vectorized by-
orignalFlip=flipud(fliplr(orignal(i-1:i+1,j-1:j+1)));
temp=orignalFlip.*filter;
but what the problem with 'conv2' ? seems that exactly what you need...
However, you should not doing 4 nested-loops in matlab.
My data is a 2096x252 matrix of double values. I need a for loop or an equivalent which performs the following:
Each time the matrix is reproduced the first array is deleted and the second becomes the first. When the loop runs again, the remaining matrix is reproduced and the first array is deleted and the next becomes the first and so on.
I've tried using repmat but it is too slow and tedious when dealing with large matrices (2096x252).
Example input:
1 2 3 4
3 4 5 6
3 5 7 5
9 6 3 2
Desired output:
1 2 3 4
3 4 5 6
3 5 7 5
9 6 3 2
3 4 5 6
3 5 7 5
9 6 3 2
3 5 7 5
9 6 3 2
9 6 3 2
Generally with Matlab it is much faster to pre-allocate a large array than to build it incrementally. When you know in advance the final size of the large array there's no reason not to follow this general advice.
Something like the following should do what you want. Suppose you have an array in(nrows, ncols); then
indices = [0 nrows:-1:1];
out = zeros(sum(indices),ncols);
for ix = 1:nrows
out(1+sum(indices(1:ix)):sum(indices(1:ix+1)),:) = in(ix:end,:);
end
This worked on your small test input. I expect you can figure out what is going on.
Whether it is the fastest of all possible approaches I don't know, but I expect it to be much faster than building a large matrix incrementally.
Disclaimer:
You'll probably have memory issues with large matrices, but that is not the question.
Now, to the business:
For a given matrix A, the straightforward approach with the for loop would be:
[N, M] = size(A);
B = zeros(sum(1:N), M);
offset = 1;
for i = 1:N
B(offset:offset + N - i, :) = A(i:end, :);
offset = offset + size(A(i:end, :), 1);
end
B is the desired output matrix.
However, this solution is expected to be slow as well, because of the for loop.
Edit: preallocated B instead of dynamically changing size (this optimization should achieve a slight speedup).