I have to reorder a sequence of elements based on the similarity between each other (expressed by a coefficient) so that each element is the most similar possible to each of its neighbors. I have to find an algorithm rather than a code.
Example with 10 elements and similarity coefficients calculated for each pair of the elements below :
The excel file can be find here : https://1drv.ms/x/s!AtmZN4-kjgrPms99fqgaDwAS_F4uYw
What I have tried :
Find a pair with the highest coefficient. In the example : 0.98 for T3 (left-end) and T5 (right-end)
Find maximum coefficient between the left-end and the remaining elements
Find maximum coefficient between the right-end and the remaining elements
Take the maximum between 2. and 3.
If maximum is 2. add on the left the element corresponding to the maximum coefficient for the left-end. Else, add on the right the element corresponding to the maximum coefficient for the right-end
Repeat points 2 - 6 until no elements left.
Here is the result :
The result isn't bad. One of the disadvantages I see is that 0.99>0.98 is considered in the same way as 0.99>0.01.
The second option I thought about was maximizing the sum of coefficients between all neighbors, but don't really know where to start from. Especially if there are significantly more than 10 elements. More, it could result in a more "flat" order where while having better similarities overall some extremely similar elements could be placed far from each other.
Being really new to this kind of problems I am pretty sure this should be a rather standard issue with existing solutions. Could you please point to those?
Thank you!
After researching I have found that my problem can be seen as the "Travelling Salesman Problem" (TSP). More here : https://en.wikipedia.org/wiki/Travelling_salesman_problem
To apply it you can see "elements" in my example as "cities" in TSP and (1-Similarity coefficient) as "distances".
I have two sparse matrices "Matrix1" and "Matrix2" of the same size p x n.
By sparse matrix I mean that it contains a lot of exactly zero elements.
I want to show the two matrices under the same colormap and a unique colorbar. Doing this in MATLAB is straightforward:
bottom = min(min(min(Matrix1)),min(min(Matrix2)));
top = max(max(max(Matrix1)),max(max(Matrix2)));
subplot(1,2,1)
imagesc(Matrix1)
colormap(gray)
caxis manual
caxis([bottom top]);
subplot(1,2,2)
imagesc(Matrix2)
colormap(gray)
caxis manual
caxis([bottom top]);
colorbar;
My problem:
In fact, when I show the matrix using imagesc(Matrix), it can ignore the noises (or backgrounds) that always appear with using imagesc(10*log10(Matrix)).
That is why, I want to show the 10*log10 of the matrices. But in this case, the minimum value will be -Inf since the matrices are sparse. In this case caxis will give an error because bottom is equal to -Inf.
What do you suggest me? How can I modify the above code?
Any help will be very appreciated!
A very important point is that the minimum value in your matrix will always be 0. Leveraging this, a very simple way to address your problem is to add 1 inside the log operation so that values that map to 0 in the original matrix also map to 0 in the log operation. This avoids the -Inf error that you're encountering. In fact, this is a very common way of visualizing the Fourier Transform if you will. Adding 1 to the logarithm ensures that the transform has no negative values in the output, yet the derivative or its rate of change remains intact as the effect is simply a translation of the curve by 1 unit to the left.
Therefore, simply do imagesc(10*log10(1 + Matrix));, then the minimum is always bounded at 0 while the maximum is unbounded but subject to the largest value that is seen in Matrix.
Does anyone know how to implement the Principal component analysis (PCA) on a m-by-n matrix in matlab for normalization?
Assuming each column is a sample (that is, you have n samples each of dimension m), and it's stored in a matrix A you first have to subtract off the column means:
Amm = bsxfun(#minus,A,mean(A,2));
then you want to do an eigenvalue decomposition on 1/size(Amm,2)*Amm*Amm' (you can use 1/(size(Amm,2)-1) as a scale factor if you want an interpetation as an unbiased covariance matrix) with:
[v,d] = eig(1/size(Amm,2)*Amm*Amm');
And the columns of v are going to be your PCA vectors. The entries of d are going to be your corresponding "variances".
However, if your m is huge then this is not the best way to go because storing Amm*Amm' is not practical. You want to instead compute:
[u,s,v] = svd(1/sqrt(size(Amm,2))*Amm,'econ');
This time u contains your PCA vectors. The entries of s are related to the entries of d by a sqrt.
Note: there's another way to go if m is huge, i.e. computing eig(1/size(Amm,2)*Amm
'*Amm); (notice the switch of transposes as compared to above) and doing a little trickery, but it's a longer explanation so I won't get into it.
Given a large sparse matrix (say 10k+ by 1M+) I need to find a subset, not necessarily continuous, of the rows and columns that form a dense matrix (all non-zero elements). I want this sub matrix to be as large as possible (not the largest sum, but the largest number of elements) within some aspect ratio constraints.
Are there any known exact or aproxamate solutions to this problem?
A quick scan on Google seems to give a lot of close-but-not-exactly results. What terms should I be looking for?
edit: Just to clarify; the sub matrix need not be continuous. In fact the row and column order is completely arbitrary so adjacency is completely irrelevant.
A thought based on Chad Okere's idea
Order the rows from largest count to smallest count (not necessary but might help perf)
Select two rows that have a "large" overlap
Add all other rows that won't reduce the overlap
Record that set
Add whatever row reduces the overlap by the least
Repeat at #3 until the result gets to small
Start over at #2 with a different starting pair
Continue until you decide the result is good enough
I assume you want something like this. You have a matrix like
1100101
1110101
0100101
You want columns 1,2,5,7 and rows 1 and 2, right? That submatrix would 4x2 with 8 elements. Or you could go with columns 1,5,7 with rows 1,2,3 which would be a 3x3 matrix.
If you want an 'approximate' method, you could start with a single non-zero element, then go on to find another non-zero element and add it to your list of rows and columns. At some point you'll run into a non-zero element that, if it's rows and columns were added to your collection, your collection would no longer be entirely non-zero.
So for the above matrix, if you added 1,1 and 2,2 you would have rows 1,2 and columns 1,2 in your collection. If you tried to add 3,7 it would cause a problem because 1,3 is zero. So you couldn't add it. You could add 2,5 and 2,7 though. Creating the 4x2 submatrix.
You would basically iterate until you can't find any more new rows and columns to add. That would get you too a local minimum. You could store the result and start again with another start point (perhaps one that didn't fit into your current solution).
Then just stop when you can't find any more after a while.
That, obviously, would take a long time, but I don't know if you'll be able to do it any more quickly.
I know you aren't working on this anymore, but I thought someone might have the same question as me in the future.
So, after realizing this is an NP-hard problem (by reduction to MAX-CLIQUE) I decided to come up with a heuristic that has worked well for me so far:
Given an N x M binary/boolean matrix, find a large dense submatrix:
Part I: Generate reasonable candidate submatrices
Consider each of the N rows to be a M-dimensional binary vector, v_i, where i=1 to N
Compute a distance matrix for the N vectors using the Hamming distance
Use the UPGMA (Unweighted Pair Group Method with Arithmetic Mean) algorithm to cluster vectors
Initially, each of the v_i vectors is a singleton cluster. Step 3 above (clustering) gives the order that the vectors should be combined into submatrices. So each internal node in the hierarchical clustering tree is a candidate submatrix.
Part II: Score and rank candidate submatrices
For each submatrix, calculate D, the number of elements in the dense subset of the vectors for the submatrix by eliminating any column with one or more zeros.
Select the submatrix that maximizes D
I also had some considerations regarding the min number of rows that needed to be preserved from the initial full matrix, and I would discard any candidate submatrices that did not meet this criteria before selecting a submatrix with max D value.
Is this a Netflix problem?
MATLAB or some other sparse matrix libraries might have ways to handle it.
Is your intent to write your own?
Maybe the 1D approach for each row would help you. The algorithm might look like this:
Loop over each row
Find the index of the first non-zero element
Find the index of the non-zero row element with the largest span between non-zero columns in each row and store both.
Sort the rows from largest to smallest span between non-zero columns.
At this point I start getting fuzzy (sorry, not an algorithm designer). I'd try looping over each row, lining up the indexes of the starting point, looking for the maximum non-zero run of column indexes that I could.
You don't specify whether or not the dense matrix has to be square. I'll assume not.
I don't know how efficient this is or what its Big-O behavior would be. But it's a brute force method to start with.
EDIT. This is NOT the same as the problem below.. My bad...
But based on the last comment below, it might be equivilent to the following:
Find the furthest vertically separated pair of zero points that have no zero point between them.
Find the furthest horizontally separated pair of zero points that have no zeros between them ?
Then the horizontal region you're looking for is the rectangle that fits between these two pairs of points?
This exact problem is discussed in a gem of a book called "Programming Pearls" by Jon Bentley, and, as I recall, although there is a solution in one dimension, there is no easy answer for the 2-d or higher dimensional variants ...
The 1=D problem is, effectively, find the largest sum of a contiguous subset of a set of numbers:
iterate through the elements, keeping track of a running total from a specific previous element, and the maximum subtotal seen so far (and the start and end elemnt that generateds it)... At each element, if the maxrunning subtotal is greater than the max total seen so far, the max seen so far and endelemnt are reset... If the max running total goes below zero, the start element is reset to the current element and the running total is reset to zero ...
The 2-D problem came from an attempt to generate a visual image processing algorithm, which was attempting to find, within a stream of brightnesss values representing pixels in a 2-color image, find the "brightest" rectangular area within the image. i.e., find the contained 2-D sub-matrix with the highest sum of brightness values, where "Brightness" was measured by the difference between the pixel's brighness value and the overall average brightness of the entire image (so many elements had negative values)
EDIT: To look up the 1-D solution I dredged up my copy of the 2nd edition of this book, and in it, Jon Bentley says "The 2-D version remains unsolved as this edition goes to print..." which was in 1999.
Basically I have been trying to forge an understanding of matrix maths over the last few weeks and after reading (and re-reading) many maths heavy articles and documentation I think I have an adequate understanding, but I just wanted to make sure!
The definitions i have ended up with are:
/*
Minor
-----
-A determinant of a sub matrix
-The sub matrix used to calculate a minor can be obtained by removing more then one row/column from the original matrix
-First minors are minors of a sub matrix where only the row and column of a single element have been removed
Cofactor
--------
-The (signed) minor of a single element from a matrix
ie. the minor of element 2,3 is the determinant of the submatrix, of the matrix, defined by removing row 2 and column 3
Determinant
-----------
-1. Choose any single row or column from a Matrix.
2. For each element in the row/column, multiply the value of the element against the First Minor of that element.
3. This result is then multiplied by (-1 raised to the power of the elements row index + its column index) which will give the result of step 2 a sign.
4. You then simply sum all these results to get the determinant (a real number) for the Matrix.
*/
Please let me know of any holes in my understanding?
Sources
http://en.wikipedia.org /Cofactor_(linear_algebra) & /Minor_(linear_algebra) & /Determinant
http://easyweb.easynet.co.uk/~mrmeanie/matrix/matrices.htm
http://www.geometrictools.com/Documentation/LaplaceExpansionTheorem.pdf (the most helpful)
Geometric tools for computer graphics (this may have missing pages, i have the full copy)
Sounds like you understand determinants -- now go forth and write code! Try writing a solver for simultaneous linear equations in 3 or more variables, using Cramer's Rule.
Since you tagged this question 3dgraphics, matrix and vector multiplication might be a good area to explore next. They come up everywhere in 3d graphics programming.