Grouping data using loops (signal processing in MATLAB) - algorithm

Question

I am working in MATLAB with a signal data that consist of consecutive dips as shown below. I am trying to write a code which sorts the contents of each dip into a separate group. How should the general structure of such a code look like?
The following is my data. I am only interested in the portion of the signal that lies below a certain threshold d (the red line):
And here is the desired grouping:
Here is an unsuccessful attempt:
k=0; % Group number
for i = 1 : length(signal)
if signal(i) < d
k=k+1;
while signal(i) < d
NewSignal(i, k) = signal(i);
i = i + 1;
end
end
end
The code above generated 310 groups instead of the desired 12 groups.
Any explanation would be greatly appreciated.

Answer 1

Taking Benl generated data you can do the following:
%generate data
x=1:1000;
y=sin(x/20);
for ii=1:9
y=y+-10*exp(-(x-ii*100).^2./10);
end
y=awgn(y,4);
%set threshold
t=-4;
%threshold data
Y = char(double(y<t) + '0'); %// convert to string of zeros and ones
%search for start and ends
This idea is taken from here
[s, e] = regexp(Y, '1+', 'start', 'end');
%and now plot and see that each pair of starts and end
% represents a group
plot(x,y)
hold on
for k=1:numel(s)
line(s(k)*ones(2,1),ylim,'Color','k','LineStyle','--')
line(e(k)*ones(2,1),ylim,'Color','k','LineStyle','-')
end
hold off
legend('Data','Starts','Ends')
Comments: First of all I choose an arbitrary threshold, it is up to you to find the "best" one in your data. Additionally I didn't group the data explicitly but rather this approach gives you the start and end of each epoch with a dip (you might call it group). So you could say that each index is the grouping index. Finally I did not debug this approach for corner cases, when dips fall on starts and ends...

Answer 2

In MATLAB you cannot change the loop index of a for loop. A for loop:
for i = array
loops over each column of array in turn. In your code, 1 : length(signal) is an array, each of its elements is visited in turn. Inside this loop there is a while loop that increments i. However, when this while loop ends and the next iteration of the for loop runs, i is reset to the next item in the array.
This code therefore needs two while loops:
i = 1; % Index
k = 0; % Group number
while i <= numel(signal)
if signal(i) < d
k = k + 1;
while signal(i) < d
NewSignal(i,k) = signal(i);
i = i + 1;
end
end
i = i + 1;
end

Answer 3

Easy, the function you're looking for is bwlabel, which when combined with logical indexing makes this simple.
To start I made some fake data which resembled your data
x=1:1000;
y=sin(x/20);
for ii=1:9
y=y+-10*exp(-(x-ii*100).^2./10);
end
y=awgn(y,4);
plot(x,y)
Then set your threshold and use 'bwlabel'
d=-4;% set the threshold
groupid=bwlabel(y<d);
bwlabel labels connected groups in a black and white image, what we've effectively done here is make a black and white (logical 0 & 1) 1D image in the logical vector y<d. bwlabel returns the number of the region at the index of the region. We're not interested in the 0 region, so to get the x values or y values of the nth region, simply use x(groupid==n), for example with my test data
x_4=x(groupid==4)
y_4=y(groupid==4)
x_4 = 398 399 400 401 402
y_4 = -5.5601 -7.8280 -9.1965 -7.9083 -5.8751