Related
I'm trying to figure out a way to do a certain "reduction"
I have a varying number of matrices of varying size, e.g
1 2 2 2 5 6...70 70
3 7 8 9 7 7...88 89
1 3 4
2 7 7
3 8 8
9 9 9
.
.
44 49 49 49 49 49 49
50 50 50 50 50 50 50
87 87 88 89 90 91 92
What I need to do (and I hope that I'm explaining this clearly enough) is to combine any possible
combination of columns from these matrices, this means that one column might be
1
3
1
2
3
9
.
.
.
44
50
87
Which would reduce down to
1
2
3
9
.
.
.
44
50
87
The reason why I'm doing this is because I need to find the smallest unique combined column
What am I trying to accomplish
For those interested, I'm trying to find the smallest set of gene knockouts
to disable reactions. Here, every matrix represents a reactions, and the columns represent the indices of
the genes that would disable that reaction.
The method may be as brute force as needed, as these matrices rarely become overwhelmingly large,
and the reaction combinations won't be long either
The problem
I can't (as far as I know) create a for loop with an arbitrary number of iterators, and the number of
matrices (reactions to disable) is arbitrary.
Clarification
If I have matrices A,B,C with columns a1,a2...b1,b2...c1...cn what I need
are the columns [a1 b1 c1], [a1, b1, c2], ..., [a1 b1 cn] ... [an bn cn]
Solution
Courtesy of Michael Ohlrogge below.
Extension of his answer, for completeness
His solution ends with
MyProd = product(Array_of_ColGroups...)
Which gets the job done
And picking up where he left off
collection = collect(MyProd); #MyProd is an iterator
merged_cols = Array[] # the rows of 'collection' are arrays of arrays
for (i,v) in enumerate(collection)
# I apologize for this line
push!(merged_cols, sort!(unique(vcat(v...))))
end
# find all lengths so I can find which is the minimum
lengths = map(x -> length(x), merged_cols);
loc_of_shortest = find(broadcast((x,y) -> length(x) == y, merged_cols,minimum(lengths)))
best_gene_combos = merged_cols[loc_of_shortest]
tl;dr - complete solution:
# example matrices
a = rand(1:50, 8,4); b = rand(1:50, 10,5); c = rand(1:50, 12,4);
Matrices = [a,b,c];
toJagged(x) = [x[:,i] for i in 1:size(x,2)];
JaggedMatrices = [toJagged(x) for x in Matrices];
Combined = [unique(i) for i in JaggedMatrices[1]];
for n in 2:length(JaggedMatrices)
Combined = [unique([i;j]) for i in Combined, j in JaggedMatrices[n]];
end
Lengths = [length(s) for s in Combined];
Minima = findin(Lengths, min(Lengths...));
SubscriptsArray = ind2sub(size(Lengths), Minima);
ComboTuples = [((i[j] for i in SubscriptsArray)...) for j in 1:length(Minima)]
Explanation:
Assume you have matrix a and b
a = rand(1:50, 8,4);
b = rand(1:50, 10,5);
Express them as a jagged array, columns first
A = [a[:,i] for i in 1:size(a,2)];
B = [b[:,i] for i in 1:size(b,2)];
Concatenate rows for all column combinations using a list comprehension; remove duplicates on the spot:
Combined = [unique([i;j]) for i in A, j in B];
You now have all column combinations of a and b, as concatenated rows with duplicates removed. Find the lengths easily:
Lengths = [length(s) for s in Combined];
If you have more than two matrices, perform this process iteratively in a for loop, e.g. by using the Combined matrix in place of a. e.g. if you have a matrix c:
c = rand(1:50, 12,4);
C = [c[:,i] for i in 1:size(c,2)];
Combined = [unique([i;j]) for i in Combined, j in C];
Once you have the Lengths array as a multidimensional array (as many dimensions as input matrices, where the size of each dimension is the number of columns in each matrix), you can find the column combinations that correspond to the lowest value (there may well be more than one combination), via a simple ind2sub operation:
Minima = findin(Lengths, min(Lengths...));
SubscriptsArray = ind2sub(size(Lengths), Minima)
(e.g. for a randomized run with 3 input matrices, I happened to get 4 results with the minimal length of 19. The result of ind2sub was ([4,4,3,4,4],[3,3,4,5,3],[1,3,3,3,4])
You can convert this further to a list of "Column Combination" tuples with a (somewhat ugly) list comprehension:
ComboTuples = [((i[j] for i in SubscriptsArray)...) for j in 1:length(Minima)]
# results in:
# 5-element Array{Tuple{Int64,Int64,Int64},1}:
# (4,3,1)
# (4,3,3)
# (3,4,3)
# (4,5,3)
# (4,3,4)
Ok, let's see if I understand this. You've got n matrices and want all combinations with one column from each of the n matrices? If so, how about the product() (for Cartesian product) from the Iterators package?
using Iterators
n = 3
Array_of_Arrays = [rand(3,3) for idx = 1:n] ## arbitrary representation of your set of arrays.
Array_of_ColGroups = Array(Array, length(Array_of_Arrays))
for (idx, MyArray) in enumerate(Array_of_Arrays)
Array_of_ColGroups[idx] = [MyArray[:,jdx] for jdx in 1:size(MyArray,2)]
end
MyProd = product(Array_of_ColGroups...)
This will create an iterator object which you can then loop over to consider the specific combinations of columns.
So I have a rectilinear grid that can be described with 2 vectors. 1 for the x-coordinates of the cell centres and one for the y-coordinates. These are just points with spacing like x spacing is 50 scaled to 10 scaled to 20 (55..45..30..10,10,10..10,12..20,20,20) and y spacing is 60 scaled to 40 scaled to 60 (60,60,60,55..42,40,40,40..40,42..60,60) and the grid is made like this
e.g. x = 1 2 3, gridx = 1 2 3, y = 10 11 12, gridy = 10 10 10
1 2 3 11 11 11
1 2 3 12 12 12
so then cell centre 1 is 1,10 cc2 is 2,10 etc.
Now Im trying to formulate an algorithm to calculate the positions of the cell edges in the x and y direction. So like my first idea was to first get the first edge using x(1)-[x(2)-x(1)]/2, in the real case x(2)-x(1) is equal to 60 and x(1) = 16348.95 so celledge1 = x(1)-30 = 16318.95. Then after calculating the first one I go through a loop and calculate the rest like this:
for aa = 2:length(x)+1
celledge1(aa) = x(aa-1) + [x(aa-1)-celledge(aa-1)]
end
And I did the same for y. This however does not work and my y vector in the area where the edge spacing should be should be 40 is 35,45,35,45... approx.
Anyone have any idea why this doesnt work and can point me in the right direction. Cheers
Edit: Tried to find a solution using geometric alebra:
We are trying to find the points A,B,C,....H. From basic geometry we know:
c1 (centre 1) = [A+B]/2 and c2 = [B+C]/2 etc. etc.
So we have 7 equations and 8 variables. We also know the the first few distances between centres are equal (60,60,60,60) therefore the first segment is 60 too.
B - A = 60
So now we have 8 equations and 8 variables so I made this algorithm in Matlab:
edgex = zeros(length(DATA2.x)+1,1);
edgey = zeros(length(DATA2.y)+1,1);
edgex(1) = (DATA2.x(1)*2-diffx(1))/2;
edgey(1) = (DATA2.y(1)*2-diffy(1))/2;
for aa = 2:length(DATA2.x)+1
edgex(aa) = DATA2.x(aa-1)*2-edgex(aa-1);
end
for aa = 2:length(DATA2.y)+1
edgey(aa) = DATA2.y(aa-1)*2-edgey(aa-1);
end
And I still got the same answer as before with the y spacing going 35,45,35,45 where it should be 40,40,40... Could it be an accuracy error??
Edit: here are the numbers if ur interested and I did the same computation as above only in excel: http://www.filedropper.com/workoutedges
It seems you're just trying to interpolate your data. You can do this with the built-in interp1
x = [30 24 19 16 8 7 16 22 29 31];
xi = interp1(2:2:numel(x)*2, x, 1:(numel(x)*2+1), 'linear', 'extrap');
This just sets up the original data as the even-indexed elements and interpolates the odd indices, including extrapolation for the two end points.
Results:
xi =
Columns 1 through 11:
33.0000 30.0000 27.0000 24.0000 21.5000 19.0000 17.5000 16.0000 12.0000 8.0000 7.5000
Columns 12 through 21:
7.0000 11.5000 16.0000 19.0000 22.0000 25.5000 29.0000 30.0000 31.0000 32.0000
I need this for Lagrange polynomials. I'm curious how one would do this without a for loop. The code currently looks like this:
tj = 1:n;
ti = zeros(n,n-1);
for i = 1:n
ti(i,:) = tj([1:i-1, i+1:end]);
end
My tj is not really just a 1:n vector but that's not important. While this for loop gets the job done, I'd rather use some matrix operation. I tried looking for some appropriate matrices to multiply it with, but no luck so far.
Here's a way:
v = [10 20 30 40]; %// example vector
n = numel(v);
M = repmat(v(:), 1, n);
M = M(~eye(n));
M = reshape(M,n-1,n).';
gives
M =
20 30 40
10 30 40
10 20 40
10 20 30
This should generalize to any n
ti = flipud(reshape(repmat(1:n, [n-1 1]), [n n-1]));
Taking a closer look at what's going on. If you look at the resulting matrix closely, you'll see that it's n-1 1's, n-1 2's, etc. from the bottom up.
For the case where n is 3.
ti =
2 3
1 3
1 2
So we can flip this vertically and get
f = flipud(ti);
1 2
1 3
2 3
Really this is [1, 2, 3; 1, 2, 3] reshaped to be 3 x 2 rather than 2 x 3.
In that line of thinking
a = repmat(1:3, [2 1])
1 2 3
1 2 3
b = reshape(a, [3 2]);
1 2
1 3
2 3
c = flipud(b);
2 3
1 3
1 2
We are now back to where you started when we bring it all together and replace 3's with n and 2's with n-1.
Here's another way. First create a matrix where each row is the vector tj but are stacked on top of each other. Next, extract the lower and upper triangular parts of the matrix without the diagonal, then add the results together ensuring that you remove the last column of the lower triangular matrix and the first column of the upper triangular matrix.
n = numel(tj);
V = repmat(tj, n, 1);
L = tril(V,-1);
U = triu(V,1);
ti = L(:,1:end-1) + U(:,2:end);
numel finds the total number of values in tj which we store in n. repmat facilitates the stacking of the vector tj to create a matrix that is n x n large. After, we use tril and triu so that we extract the lower and upper triangular parts of the matrices without the diagonal. In addition, the rest of the matrix is all zero except for the relevant triangular parts. The -1 and 1 flags for tril and triu respectively extract this out successfully while ensuring that the diagonal is all zero. This creates a column of extra zeroes appearing at the last column when calling tril and the first column when calling triu. The last part is to simply add these two matrices together ignoring the last column of the tril result and the first column of the triu result.
Given that tj = [10 20 30 40]; (borrowed from Luis Mendo's example), we get:
ti =
20 30 40
10 30 40
10 20 40
10 20 30
I have an NxM matrix in MATLAB that I would like to reorder in similar fashion to the way JPEG reorders its subblock pixels:
(image from Wikipedia)
I would like the algorithm to be generic such that I can pass in a 2D matrix with any dimensions. I am a C++ programmer by trade and am very tempted to write an old school loop to accomplish this, but I suspect there is a better way to do it in MATLAB.
I'd be rather want an algorithm that worked on an NxN matrix and go from there.
Example:
1 2 3
4 5 6 --> 1 2 4 7 5 3 6 8 9
7 8 9
Consider the code:
M = randi(100, [3 4]); %# input matrix
ind = reshape(1:numel(M), size(M)); %# indices of elements
ind = fliplr( spdiags( fliplr(ind) ) ); %# get the anti-diagonals
ind(:,1:2:end) = flipud( ind(:,1:2:end) ); %# reverse order of odd columns
ind(ind==0) = []; %# keep non-zero indices
M(ind) %# get elements in zigzag order
An example with a 4x4 matrix:
» M
M =
17 35 26 96
12 59 51 55
50 23 70 14
96 76 90 15
» M(ind)
ans =
17 35 12 50 59 26 96 51 23 96 76 70 55 14 90 15
and an example with a non-square matrix:
M =
69 9 16 100
75 23 83 8
46 92 54 45
ans =
69 9 75 46 23 16 100 83 92 54 8 45
This approach is pretty fast:
X = randn(500,2000); %// example input matrix
[r, c] = size(X);
M = bsxfun(#plus, (1:r).', 0:c-1);
M = M + bsxfun(#times, (1:r).'/(r+c), (-1).^M);
[~, ind] = sort(M(:));
y = X(ind).'; %'// output row vector
Benchmarking
The following code compares running time with that of Amro's excellent answer, using timeit. It tests different combinations of matrix size (number of entries) and matrix shape (number of rows to number of columns ratio).
%// Amro's approach
function y = zigzag_Amro(M)
ind = reshape(1:numel(M), size(M));
ind = fliplr( spdiags( fliplr(ind) ) );
ind(:,1:2:end) = flipud( ind(:,1:2:end) );
ind(ind==0) = [];
y = M(ind);
%// Luis' approach
function y = zigzag_Luis(X)
[r, c] = size(X);
M = bsxfun(#plus, (1:r).', 0:c-1);
M = M + bsxfun(#times, (1:r).'/(r+c), (-1).^M);
[~, ind] = sort(M(:));
y = X(ind).';
%// Benchmarking code:
S = [10 30 100 300 1000 3000]; %// reference to generate matrix size
f = [1 1]; %// number of cols is S*f(1); number of rows is S*f(2)
%// f = [0.5 2]; %// plotted with '--'
%// f = [2 0.5]; %// plotted with ':'
t_Amro = NaN(size(S));
t_Luis = NaN(size(S));
for n = 1:numel(S)
X = rand(f(1)*S(n), f(2)*S(n));
f_Amro = #() zigzag_Amro(X);
f_Luis = #() zigzag_Luis(X);
t_Amro(n) = timeit(f_Amro);
t_Luis(n) = timeit(f_Luis);
end
loglog(S.^2*prod(f), t_Amro, '.b-');
hold on
loglog(S.^2*prod(f), t_Luis, '.r-');
xlabel('number of matrix entries')
ylabel('time')
The figure below has been obtained with Matlab R2014b on Windows 7 64 bits. Results in R2010b are very similar. It is seen that the new approach reduces running time by a factor between 2.5 (for small matrices) and 1.4 (for large matrices). Results are seen to be almost insensitive to matrix shape, given a total number of entries.
Here's a non-loop solution zig_zag.m. It looks ugly but it works!:
function [M,index] = zig_zag(M)
[r,c] = size(M);
checker = rem(hankel(1:r,r-1+(1:c)),2);
[rEven,cEven] = find(checker);
[cOdd,rOdd] = find(~checker.'); %'#
rTotal = [rEven; rOdd];
cTotal = [cEven; cOdd];
[junk,sortIndex] = sort(rTotal+cTotal);
rSort = rTotal(sortIndex);
cSort = cTotal(sortIndex);
index = sub2ind([r c],rSort,cSort);
M = M(index);
end
And a test matrix:
>> M = [magic(4) zeros(4,1)];
M =
16 2 3 13 0
5 11 10 8 0
9 7 6 12 0
4 14 15 1 0
>> newM = zig_zag(M) %# Zig-zag sampled elements
newM =
16
2
5
9
11
3
13
10
7
4
14
6
8
0
0
12
15
1
0
0
Here's a way how to do this. Basically, your array is a hankel matrix plus vectors of 1:m, where m is the number of elements in each diagonal. Maybe someone else has a neat idea on how to create the diagonal arrays that have to be added to the flipped hankel array without a loop.
I think this should be generalizeable to a non-square array.
% for a 3x3 array
n=3;
numElementsPerDiagonal = [1:n,n-1:-1:1];
hadaRC = cumsum([0,numElementsPerDiagonal(1:end-1)]);
array2add = fliplr(hankel(hadaRC(1:n),hadaRC(end-n+1:n)));
% loop through the hankel array and add numbers counting either up or down
% if they are even or odd
for d = 1:(2*n-1)
if floor(d/2)==d/2
% even, count down
array2add = array2add + diag(1:numElementsPerDiagonal(d),d-n);
else
% odd, count up
array2add = array2add + diag(numElementsPerDiagonal(d):-1:1,d-n);
end
end
% now flip to get the result
indexMatrix = fliplr(array2add)
result =
1 2 6
3 5 7
4 8 9
Afterward, you just call reshape(image(indexMatrix),[],1) to get the vector of reordered elements.
EDIT
Ok, from your comment it looks like you need to use sort like Marc suggested.
indexMatrixT = indexMatrix'; % ' SO formatting
[dummy,sortedIdx] = sort(indexMatrixT(:));
sortedIdx =
1 2 4 7 5 3 6 8 9
Note that you'd need to transpose your input matrix first before you index, because Matlab counts first down, then right.
Assuming X to be the input 2D matrix and that is square or landscape-shaped, this seems to be pretty efficient -
[m,n] = size(X);
nlim = m*n;
n = n+mod(n-m,2);
mask = bsxfun(#le,[1:m]',[n:-1:1]);
start_vec = m:m-1:m*(m-1)+1;
a = bsxfun(#plus,start_vec',[0:n-1]*m);
offset_startcol = 2- mod(m+1,2);
[~,idx] = min(mask,[],1);
idx = idx - 1;
idx(idx==0) = m;
end_ind = a([0:n-1]*m + idx);
offsets = a(1,offset_startcol:2:end) + end_ind(offset_startcol:2:end);
a(:,offset_startcol:2:end) = bsxfun(#minus,offsets,a(:,offset_startcol:2:end));
out = a(mask);
out2 = m*n+1 - out(end:-1:1+m*(n-m+1));
result = X([out2 ; out(out<=nlim)]);
Quick runtime tests against Luis's approach -
Datasize: 500 x 2000
------------------------------------- With Proposed Approach
Elapsed time is 0.037145 seconds.
------------------------------------- With Luis Approach
Elapsed time is 0.045900 seconds.
Datasize: 5000 x 20000
------------------------------------- With Proposed Approach
Elapsed time is 3.947325 seconds.
------------------------------------- With Luis Approach
Elapsed time is 6.370463 seconds.
Let's assume for a moment that you have a 2-D matrix that's the same size as your image specifying the correct index. Call this array idx; then the matlab commands to reorder your image would be
[~,I] = sort (idx(:)); %sort the 1D indices of the image into ascending order according to idx
reorderedim = im(I);
I don't see an obvious solution to generate idx without using for loops or recursion, but I'll think some more.
I need to write a program to input a number and output its factorial's prime factorization in the form:
4!=(2^3)*(3^1)
5!=(2^3)*(3^1)*(5^1)
The problem is I still can't figure out how to get that result.
Apparently each first number in brackets is for the ascending prime numbers up until the actual factorial. The second number in brackets is the amount of times the number occurs in the factorial.
What I can't figure out is for example in 5!=(2^3)*(3^1)*(5^1), how does 2 only occur 3 times, 3 only 1 time and 5 only one time in 120 (5!=120).
I have now solved this thanks to the helpful people who commented but I'm now having trouble trying to figure out how could I take a number and get the factorial in this format without actually calculating the factorial.
Every number can be represented by a unique (up to re-ordering) multiplication of prime numbers, called the prime factorization of the number, as you are finding the prime factors that can uniquely create that number.
2^3=8
3^1=3
5^1=5
and 8*3*5=120
But this also means that: (2^3)*(3^1)*(5^1) = 120
It's not saying that 2 occurs 3 times as a digit in the number 120, which it obviously does not, but rather to multiply 2 by 2 by 2, for a total of 3 twos. Likewise for the 3 and 5, which occur once in the prime factorization of 120. The expression which you mention is showing you this unique prime factorization of the number 120. This is one way of getting the prime factorization of a number in Python:
def pf(number):
factors=[]
d=2
while(number>1):
while(number%d==0):
factors.append(d)
number=number/d
d+=1
return factors
Running it you get:
>>> pf(120)
[2, 2, 2, 3, 5]
Which multiplied together give you 120, as explained above. Here's a little diagram to illustrate this more clearly:
Consider e.g. 33!. It's a product of:
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
the factors are:
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2
2 2 2 2
2 2
2
3 3 3 3 3 3 3 3 3 3 3
3 3 3
3
5 5 5 5 5 5
5
7 7 7 7
11 11 11
13 13
17
19
23
29 31
Do you see the pattern?
33! = 2^( 33 div 2 + 33 div 4 + 33 div 8 + 33 div 16 + 33 div 32) *
3^( 33 div 3 + 33 div 9 + 33 div 27) *
5^( 33 div 5 + 33 div 25) *
----
7^( 33 div 7) * 11^( 33 div 11) * 13^( 33 div 13) *
----
17 * 19 * 23 * 29 * 31
Thus, to find prime factorization of n! without doing any multiplications or factorizations, we just need to have the ordered list of primes not greater than n, which we process (with a repeated integer division and a possible summation) in three stages - primes that are smaller or equal to the square root of n; such that are smaller or equal to n/2; and the rest.
Actually with lazy evaluation it's even simpler than that. Assuming primes is already implemented returning a stream of prime numbers in order, in Haskell, factorial factorization is found as
ff n = [(p, sum . takeWhile (> 0) . tail . iterate (`div` p) $ n)
| p <- takeWhile (<= n) primes]
-- Prelude> ff 33
-- [(2,31),(3,15),(5,7),(7,4),(11,3),(13,2),(17,1),(19,1),(23,1),(29,1),(31,1)]
because 33 div 4 is (33 div 2) div 2, etc..
2^3 is another way of writing 23, or two to the third power. (2^3)(3^1)(5^1) = 23 × 3 × 5 = 120.
(2^3)(3^1)(5^1) is just the prime factorization of 120 expressed in plain ASCII text rather than with pretty mathematical formatting. Your assignment requires output in this form simply because it's easier for you to output than it would be for you to figure out how to output formatted equations (and probably because it's easier to process for grading).
The conventions used here for expressing equations in plain text are standard enough that you can directly type this text into google.com or wolframalpha.com and it will calculate the result as 120 for you: (2^3)(3^1)(5^1) on wolframalpha.com / (2^3)(3^1)(5^1) on google.com
WolframAlpha can also compute prime factorizations, which you can use to get test results to compare your program with. For example: prime factorization of 1000!
A naïve solution that actually calculates the factorial will only handle numbers up to 12 (if using 32 bit ints). This is because 13! is ~6.2 billion, larger than the largest number that can be represented in a 32 bit int.
However it's possible to handle much larger inputs if you avoid calculating the factorial first. I'm not going to tell you exactly how to do that because either figuring it out is part of your assignment or you can ask your prof/TAs. But below are some hints.
ab × ac = ab+c
equation (a) 10 = 21 × 51
equation (b) 15 = 31 × 51
10 × 15 = ? Answer using the right hand sides of equations (a) and (b), not with the number 150.
10 × 15 = (21 × 51) × (31 × 51) = 21 × 31 × (51 × 51) = 21 × 31 × 52
As you can see, computing the prime factorization of 10 × 15 can be done without multiplying 10 by 15; You can instead compute the prime factorization of the individual terms and then combine those factorizations.
If you write out the factorial 5!:
1 * 2 * 3 * 4 * 5,
you will notice that there is one non-prime number: 4. 4 can be written as 2 * 2 or 2^2, which is where the extra 2s come from.
Add up all of the occurrences (exponential forms are in parentheses; add exponents for like bases):
2 (2^1) * 3 (3^1) * 4 (2^2) * 5 (5^1), you get the proper answer.
You can use O(n/2 log log n) algorithm using only sums (no need precalc primes).
This is a sieve using relation
f = a * b ~> f^k = a^k * b^k
then, we reduce all initial factors 1 * 2 * 3 * ... * n moving k from big numbers to small numbers.
Using Sieve of Atkin the Will Ness algorithm could be better for very big n if not, I think it could be better
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
int n = atoi(argv[1]);
int *p = (int *) malloc(sizeof(int) * (n + 1));
int i, j, d;
for(i = 0; i <= n; i++)
p[i] = 1;
for(i = n >> 1; i > 1; i--)
if(p[i]) {
for(j = i + i, d = 2; j <= n; j += i, d++) {
if(p[j]) {
p[i] += p[j];
p[d] += p[j];
p[j] = 0;
}
}
}
printf("1");
for(i = 2; i <= n; i++)
if(p[i])
printf(" * %i^%i", i, p[i]);
printf("\n");
return 0;
}