My problem is the following. I have the adjacency matrix Mat for a neural network. I want to randomize this network in the sense that I want to choose 4 notes randomly (say i,j,p,q) such that i and p are connected (which means Mat[p,i] = 1) and j and q are connected AND i and q are not connected (Mat[q,j] = 0)and j and p are not connected. I then connect i and q and j and p and disconnect the previous nodes. In one run, I want to do this 10^6 times.
So far I have two versions, one using a for loop and one recursively.
newmat = copy(Mat)
for trial in 1:Niter
count = 0
while count < 1
i,j,p,q = sample(Nodes,4,replace = false) #Choosing 4 nodes at random
if (newmat[p,i] == 1 && newmat[q,j] == 1) && (newmat[p,j] == 0 && newmat[q,i] == 0)
newmat[p,i] = 0
newmat[q,j] = 0
newmat[p,j] = 1
newmat[q,i] = 1
count += 1
end
end
end
Doing this recursively runs about just as fast until Niter = 10^4 after which I get a Stack Overflow error. How can I improve this?
I assume you are talking about a recursive variant of the for trial in 1:Niter.
To avoid stack overflows like this, a general rule of thumb (in languages without tail recursion elimination) is to not use recursion unless you know the recursion depth will not scale more than logarithmically.
The cases where this is applicable is mostly algorithms that are like tree traversals, with a "naturally occuring" recursive structure. Your case of a simple for loop can be viewed as the degenerate variant of that, with a "linked list" tree, but is not a all natural.
Just don't do it. There's nothing bad about a loop for some sequential processing like this. Julia is an imperative language, after all.
(If you want to do this with a recursive structure for fun or exercise: look up trampolines. They allow you to write code structured as tail recursive, but with the allocation happening by mutation and on the heap.)
Instead of sampling 4 random nodes and hoping they happen to be connected, you can sample the starting nodes p and q, and look for i and j within the nodes that these are connected to. Here's an implementation of that:
function randomizeconnections(adjmatin)
adjmat = copy(adjmatin)
nodes = axes(adjmat, 2)
niter = 10
for trial in 1:niter
p, q = sample(nodes, 2, replace = false)
#views plist, qlist = findall(adjmat[p, :]), findall(adjmat[q, :])
filter!(i -> !in(i, qlist) && i != q, plist)
filter!(j -> !in(j, plist) && j != p, qlist)
if isempty(plist) || isempty(qlist)
#debug "No swappable exclusive target nodes for source nodes $p and $q, skipping trial $trial..."
continue
end
i = rand(plist)
j = rand(qlist)
adjmat[p, i] = adjmat[q, j] = false
adjmat[p, j] = adjmat[q, i] = true
end
adjmat
end
Through the course of randomization, it may happen that two nodes don't have any swappable connections i.e. they may share all their end points or one's ending nodes are a subset of the other's. So there's a check for that in the above code, and the loop moves on to the next iteration in that case.
The line with the findalls in the above code effectively creates adjacency lists from the adjacency matrix on the fly. You can instead do that in one go at the beginning, and work with that adjacency list vector instead.
function randomizeconnections2(adjmatin)
adjlist = [findall(r) for r in eachrow(adjmatin)]
nodes = axes(adjlist, 1)
niter = 10
for trial in 1:niter
p, q = sample(nodes, 2, replace = false)
plist = filter(i -> !in(i, adjlist[q]) && i != q, adjlist[p])
qlist = filter(j -> !in(j, adjlist[p]) && j != p, adjlist[q])
if isempty(plist) || isempty(qlist)
#debug "No swappable exclusive target nodes for source nodes $p and $q, skipping trial $trial..."
continue
end
i = rand(plist)
j = rand(qlist)
replace!(adjlist[p], i => j)
replace!(adjlist[q], j => i)
end
create_adjmat(adjlist)
end
function create_adjmat(adjlist::Vector{Vector{Int}})
adjmat = falses(length(adjlist), length(adjlist))
for (i, l) in pairs(adjlist)
adjmat[i, l] .= true
end
adjmat
end
With the small matriced I tried locally, randomizeconnections2 seems about twice as fast as randomizeconnections, but you may want to confirm whether that's the case with your matrix sizes and values.
Both of these accept (and were tested with) BitMatrix type values as input, which should be more efficient than an ordinary matrix of booleans or integers.
I've tried to find a good way to speed up the code for a problem I've been working on. The basic idea of the code is very simple. There are five inputs:
Four 1xm (for some m < n, they can be different sizes) matrices (A, B, C, D) that are pairwise-disjoint subsets of {1,2,...,n} and one nxn symmetric binary matrix (M). The basic idea for the code is to check an inequality for for every combination of elements and if the inequality holds, return the values that cause it to hold, i.e.:
for a = A
for b = B
for c = C
for d = D
if M(a,c) + M(b,d) < M(a,d) + M(b,c)
result = [a b c d];
return
end
end
end
end
end
I know there has to be a better way to do this. First, since it's symmetric, I can cut down half of the items checked since M(a,b) = M(b,a). I've been researching vectorization, found several functions I'd never heard of with MATLAB (since I'm relatively new), but I can't find anything that will particularly help me with this specific problem. I've thought of other ways to approach the problem, but nothing has been perfected, and I just don't know what to do at this point.
For example, I could possibly split this into two cases:
1) The right hand side is 1: then I have to check that both terms on the left side are 0.
2) The right hand side is 2: then I have to check that at least one term on the left hand side is 0.
But, again, I won't be able to avoid nesting.
I appreciate all the help you can offer. Thank you!
You're asking two questions here: (1) is there a more efficient algorithm to perform this search, and (2) how can I vectorize this in MATLAB. The first one is very interesting to think about, but may be a little beyond the scope of this forum. The second one is easier to answer.
As pointed out in the comments below your question, you can vectorize the for loop by enumerating all of the possibilities and checking them all together, and the answers from this question can help:
[a,b,c,d] = ndgrid(A,B,C,D); % Enumerate all combos
a=a(:); b=b(:); c=c(:); d=d(:); % Reshape from 4-D matrices to vectors
ac = sub2ind(size(M),a,c); % Convert subscript pairs to linear indices
bd = sub2ind(size(M),b,d);
ad = sub2ind(size(M),a,d);
bc = sub2ind(size(M),b,c);
mask = (M(ac) + M(bd) < M(ad) + M(bc)); % Test the inequality
results = [a(mask), b(mask), c(mask), d(mask)]; % Select the ones that pass
Again, this isn't an algorithmic change: it still has the same complexity as your nested for loop. The vectorization may cause it to run faster, but it also lacks early termination, so in certain cases it may be slower.
Since M is binary, we can think about this as a graph problem. i,j in {1..n} correspond to nodes, and M(i,j) indicates whether there is an undirected edge connecting them.
Since A,B,C,D are disjoint, that simplifies the problem a bit. We can approach the problem in stages:
Find all (c,d) for which there exists a such that M(a,c) < M(a,d). Let's call this set CD_lt_a, (the subset of C*D such that the "less than" inequality holds for some a).
Find all (c,d) for which there exists a such that M(a,c) <= M(a,d), and call this set CD_le_a.
Repeat for b, forming CD_lt_b for M(b,d) < M(b,c) and CD_le_b for M(b,d)<=M(b,c).
One way to satisfy the overall inequality is for M(a,c) < M(a,d) and M(b,d) <= M(b,c), so we can look at the intersection of CD_lt_a and CD_le_b.
The other way is if M(a,c) <= M(a,d) and M(b,d) < M(b,c), so look at the intersection of CD_le_a and CD_lt_b.
With (c,d) known, we can go back and find the (a,b).
And so my implementation is:
% 0. Some preliminaries
% Get the size of each set
mA = numel(A); mB = numel(B); mC = numel(C); mD = numel(D);
% 1. Find all (c,d) for which there exists a such that M(a,c) < M(a,d)
CA_linked = M(C,A);
AD_linked = M(A,D);
CA_not_linked = ~CA_linked;
% Multiplying these matrices tells us, for each (c,d), how many nodes
% in A satisfy this M(a,c)<M(a,d) inequality
% Ugh, we need to cast to double to use the matrix multiplication
CD_lt_a = (CA_not_linked * double(AD_linked)) > 0;
% 2. For M(a,c) <= M(a,d), check that the converse is false for some a
AD_not_linked = ~AD_linked;
CD_le_a = (CA_linked * double(AD_not_linked)) < mA;
% 3. Repeat for b
CB_linked = M(C,B);
BD_linked = M(B,D);
CD_lt_b = (CB_linked * double(~BD_linked)) > 0;
CD_le_b = (~CB_linked * double(BD_linked)) < mB;
% 4. Find the intersection of CD_lt_a and CD_le_b - this is one way
% to satisfy the inequality M(a,c)+M(b,d) < M(a,d)+M(b,c)
CD_satisfy_ineq_1 = CD_lt_a & CD_le_b;
% 5. The other way to satisfy the inequality is CD_le_a & CD_lt_b
CD_satisfy_ineq_2 = CD_le_a & CD_lt_b;
inequality_feasible = any(CD_satisfy_ineq_1(:) | CD_satisfy_ineq_2(:));
Note that you can stop here if feasibility is your only concern. The complexity is A*C*D + B*C*D, which is better than the worst-case A*B*C*D complexity of the for loop. However, early termination means your nested for loops may still be faster in certain cases.
The next block of code enumerates all the a,b,c,d that satisfy the inequality. It's not very well optimized (it appends to a matrix from within a loop), so it can be pretty slow if there are many results.
% 6. With (c,d) known, find a and b
% We can define these functions to help us search
find_a_lt = #(c,d) find(CA_not_linked(c,:)' & AD_linked(:,d));
find_a_le = #(c,d) find(CA_not_linked(c,:)' | AD_linked(:,d));
find_b_lt = #(c,d) find(CB_linked(c,:)' & ~BD_linked(:,d));
find_b_le = #(c,d) find(CB_linked(c,:)' | ~BD_linked(:,d));
% I'm gonna assume there aren't too many results, so I will be appending
% to an array inside of a for loop. Bad for performance, but maybe a bit
% more readable for a StackOverflow answer.
results = zeros(0,4);
% Find those that satisfy it the first way
[c_list,d_list] = find(CD_satisfy_ineq_1);
for ii = 1:numel(c_list)
c = c_list(ii); d = d_list(ii);
a = find_a_lt(c,d);
b = find_b_le(c,d);
% a,b might be vectors, in which case all combos are valid
% Many ways to find all combos, gonna use ndgrid()
[a,b] = ndgrid(a,b);
% Append these to the growing list of results
abcd = [a(:), b(:), repmat([c d],[numel(a),1])];
results = [results; abcd];
end
% Repeat for the second way
[c_list,d_list] = find(CD_satisfy_ineq_2);
for ii = 1:numel(c_list)
c = c_list(ii); d = d_list(ii);
a = find_a_le(c,d);
b = find_b_lt(c,d);
% a,b might be vectors, in which case all combos are valid
% Many ways to find all combos, gonna use ndgrid()
[a,b] = ndgrid(a,b);
% Append these to the growing list of results
abcd = [a(:), b(:), repmat([c d],[numel(a),1])];
results = [results; abcd];
end
% Remove duplicates
results = unique(results, 'rows');
% And actually these a,b,c,d will be indices into A,B,C,D because they
% were obtained from calling find() on submatrices of M.
if ~isempty(results)
results(:,1) = A(results(:,1));
results(:,2) = B(results(:,2));
results(:,3) = C(results(:,3));
results(:,4) = D(results(:,4));
end
I tested this on the following test case:
m = 1000;
A = (1:m); B = A(end)+(1:m); C = B(end)+(1:m); D = C(end)+(1:m);
M = rand(D(end),D(end)) < 1e-6; M = M | M';
I like to think that first part (see if the inequality is feasible for any a,b,c,d) worked pretty well. The other vectorized answers (that use ndgrid or combvec to enumerate all combinations of a,b,c,d) would require 8 terabytes of memory for a problem of this size!
But I would not recommend running the second part (enumerating all of the results) when there are more than a few hundred c,d that satisfy the inequality, because it will be pretty damn slow.
P.S. I know I answered already, but that answer was about vectorizing such loops in general, and is less specific to your particular problem.
P.P.S. This kinda reminds me of the stable marriage problem. Perhaps some of those references would contain algorithms relevant to your problem as well. I suspect that a true graph-based algorithm could probably achieve the worst-case complexity as this while additionally offering early termination. But I think it would be difficult to implement a graph-based algorithm efficiently in MATLAB.
P.P.P.S. If you only want one of the feasible solutions, you can simplify step 6 to only return a single value, e.g.
find_a_lt = #(c,d) find(CA_not_linked(c,:)' & AD_linked(:,d), 1, 'first');
find_a_le = #(c,d) find(CA_not_linked(c,:)' | AD_linked(:,d), 1, 'first');
find_b_lt = #(c,d) find(CB_linked(c,:)' & ~BD_linked(:,d), 1, 'first');
find_b_le = #(c,d) find(CB_linked(c,:)' | ~BD_linked(:,d), 1, 'first');
if any(CD_satisfy_ineq_1)
[c,d] = find(CD_satisfy_ineq_1, 1, 'first');
a = find_a_lt(c,d);
b = find_a_le(c,d);
result = [A(a), B(b), C(c), D(d)];
elseif any(CD_satisfy_ineq_2)
[c,d] = find(CD_satisfy_ineq_2, 1, 'first');
a = find_a_le(c,d);
b = find_a_lt(c,d);
result = [A(a), B(b), C(c), D(d)];
else
result = zeros(0,4);
end
If you have access to the Neural Network Toolbox, combvec could be helpful here.
running allCombs = combvec(A,B,C,D) will give you a (4 by m1*m2*m3*m4) matrix that looks like:
[...
a1, a1, a1, a1, a1 ... a1... a2... am1;
b1, b1, b1, b1, b1 ... b2... b1... bm2;
c1, c1, c1, c1, c2 ... c1... c1... cm3;
d1, d2, d3, d4, d1 ... d1... d1... dm4]
You can then use sub2ind and Matrix Indexing to setup the two values you need for your inequality:
indices = [sub2ind(size(M),allCombs(1,:),allCombs(3,:));
sub2ind(size(M),allCombs(2,:),allCombs(4,:));
sub2ind(size(M),allCombs(1,:),allCombs(4,:));
sub2ind(size(M),allCombs(2,:),allCombs(3,:))];
testValues = M(indices);
testValues(5,:) = (testValues(1,:) + testValues(2,:) < testValues(3,:) + testValues(4,:))
Your final a,b,c,d indices could be retrieved by saying
allCombs(:,find(testValues(5,:)))
Which would print a matrix with all columns which the inequality was true.
This article might be of some use.
I have a two-fold homework problem, Implement Karp-Rabin and run it on a test file and the second part:
For the hash values modulo q, explain why it is a bad idea to use q as a power of 2. Can you construct a terrible example e.g. for q=64
and n=15?
This is my implementation of the algorithm:
def karp_rabin(text, pattern):
# setup
alphabet = 'ACGT'
d = len(alphabet)
n = len(pattern)
d_n = d**n
q = 2**32-1
m = {char:i for i,char in enumerate(alphabet)}
positions = []
def kr_hash(s):
return sum(d**(n-i-1) * m[s[i]] for i in range(n))
def update_hash():
return d*text_hash + m[text[i+n-1]] - d_n * m[text[i-1]]
pattern_hash = kr_hash(pattern)
for i in range(0, len(text) - n + 1):
text_hash = update_hash() if i else kr_hash(text[i:n])
if pattern_hash % q == text_hash % q and pattern == text[i:i+n]:
positions.append(i)
return ' '.join(map(str, positions))
...The second part of the question is referring to this part of the code/algo:
pattern_hash = kr_hash(pattern)
for i in range(0, len(text) - n + 1):
text_hash = update_hash() if i else kr_hash(text[i:n])
# the modulo q used to check if the hashes are congruent
if pattern_hash % q == text_hash % q and pattern == text[i:i+n]:
positions.append(i)
I don't understand why it would be a bad idea to use q as a power of 2. I've tried running the algorithm on the test file provided(which is the genome of ecoli) and there's no discernible difference.
I tried looking at the formula for how the hash is derived (I'm not good at math) trying to find some common factors that would be really bad for powers of two but found nothing. I feel like if q is a power of 2 it should cause a lot of clashes for the hashes so you'd need to compare strings a lot more but I didn't find anything along those lines either.
I'd really appreciate help on this since I'm stumped. If someone wants to point out what I can do better in the first part (code efficiency, readability, correctness etc.) I'd also be thrilled to hear your input on that.
There is a problem if q divides some power of d, because then only a few characters contribute to the hash. For example in your code d=4, if you take q=64 only the last three characters determine the hash (d**3 = 64).
I don't really see a problem if q is a power of 2 but gcd(d,q) = 1.
Your implementation looks a bit strange because instead of
if pattern_hash % q == text_hash % q and pattern == text[i:i+n]:
you could also use
if pattern_hash == text_hash and pattern == text[i:i+n]:
which would be better because you get fewer collisions.
The Thue–Morse sequence has among its properties that its polynomial hash quickly becomes zero when a power of 2 is the hash module, for whatever polynomial base (d). So if you will try to search a short Thue-Morse sequence in a longer one, you will have a great lot of hash collisions.
For example, your code, slightly adapted:
def karp_rabin(text, pattern):
# setup
alphabet = '01'
d = 15
n = len(pattern)
d_n = d**n
q = 32
m = {char:i for i,char in enumerate(alphabet)}
positions = []
def kr_hash(s):
return sum(d**(n-i-1) * m[s[i]] for i in range(n))
def update_hash():
return d*text_hash + m[text[i+n-1]] - d_n * m[text[i-1]]
pattern_hash = kr_hash(pattern)
for i in range(0, len(text) - n + 1):
text_hash = update_hash() if i else kr_hash(text[i:n])
if pattern_hash % q == text_hash % q : #and pattern == text[i:i+n]:
positions.append(i)
return ' '.join(map(str, positions))
print(karp_rabin('0110100110010110100101100110100110010110011010010110100110010110', '0110100110010110'))
outputs a lot of positions, although only three of then are proper matches.
Note that I have dropped the and pattern == text[i:i+n] check. Obviously if you restore it, the result will be correct, but also it is obvious that the algorithm will do much more work checking this additional condition than for other q. In fact, because there are so many collisions, the whole idea of algorithm becomes not working: you could almost as effectively wrote a simple algorithm that checks every position for a match.
Also note that your implementation is quite strange. The whole idea of polynomial hashing is to take the modulo operation each time you compute the hash. Otherwise your pattern_hash and text_hash are very big numbers. In other languages this might mean arithmetic overflow, but in Python this will invoke big integer arithmetic, which is slow and once again loses the whole idea of the algorithm.
I have expression consisting of numbers separated with plus and minus sign.
I need to get the maximal result of this expression by putting braces between the numbers.
I'm trying to get polynomial algorithm for this problem, but I need some advice or hint how to achieve it.
I've found something similar here, but I don't know how to modify it.
EDIT:
I was thinking that the idea could be similar like this
getMax(inp)
{
if(|inp| == 1)
return inp[1] // base case
else
val = 0;
for(k=2; k < |inp|; k+=2)
val = max{val, getMax(inp[:k]) + getMax(inp[k+1:])}
}
One strategy is to use dynamic programming to choose the best operation to perform last. This divides the expression in two parts.
If the operation is addition, you call recursively on each part to find the maximum for each part.
If the operation is subtraction, you want to find the maximum on the first part and the minimum on the second part.
Here is some non-memoized code, just to show how the recurrence works (note that i iterates only on the indices of the operations, to choose the best place to break the expression):
import re
def T(s, f1=max, f2=min):
if len(s) == 1:
return int(s[0])
return f1(
T(s[:i], f1, f2)+T(s[i+1:], f1, f2)
if s[i]=='+' else
T(s[:i], f1, f2)-T(s[i+1:], f2, f1)
for i in xrange(1, len(s), 2))
def solve(expr):
return T(re.split('([+-])', expr))
print solve('1-2+1') #0 ((1-2)+1)
print solve('1-22-23') #2 (1-(22-23))
Implementing a bottom-up dynamic programming is a little more tricky, as the ideal order to fill the table is somewhat non-conventional. The easiest way is to make the DP around T(k, i) that denotes "maximum/minimum for expressions of k operands starting at the ith operand". Using Anonymous idea of separating operators and numbers in O and N respectively, an example code would be:
import re
def T(O, N):
n1 = len(N)+1 #maximum expression length
Tmax = [[float('-Inf')]*len(N) for _ in xrange(n1)]
Tmin = [[float('+Inf')]*len(N) for _ in xrange(n1)]
for i, n in enumerate(N): #only the numbers
Tmax[1][i] = Tmin[1][i] = int(n)
for k in xrange(2, n1):
for i in xrange(n1-k):
for j in xrange(1, k):
if (O[i+j-1] == '+'):
Tmax[k][i] = max(Tmax[k][i], Tmax[j][i]+Tmax[k-j][i+j])
Tmin[k][i] = min(Tmin[k][i], Tmin[j][i]+Tmin[k-j][i+j])
else:
Tmax[k][i] = max(Tmax[k][i], Tmax[j][i]-Tmin[k-j][i+j])
Tmin[k][i] = min(Tmin[k][i], Tmin[j][i]-Tmax[k-j][i+j])
return Tmax[len(N)][0]
def solve(expr):
A = re.split('([+-])', expr)
return T(A[1::2], A[::2])
print solve('1+1') #2
print solve('1-2+1') #0 ((1-2)+1)
print solve('1-22-23') #2 (1-(22-23))
Let the operators be O[0], O[1], ..., O[K-1]. Let the numbers be N[0], N[1], ..., N[K]. (There's one more number than operator).
Let M[op, i, j] be the largest value achievable from the sub-expression starting from number i and ending from number j (inclusive, both ends) if op is +, and the smallest value if op is -.
Thus M[+, 0, K] is maximum value the whole expression can take.
M satisfies a recurrence relation:
M[+, i, i] = M[-, i, i] = N[i]
M[+, i, j] = max(M[+, i, k] O[k] M[O[k], k+1, j) for k in i...j-1)
M[-, i, j] = min(M[-, i, k] O[k] M[-O[k], k+1, j) for k in i...j-1)
Here A O[k] B means A + B or A - B depending on O[k], and -O[k] means - if O[k] is +, and + if O[k] is -.
Basically, you're trying to find the best place to split the expression to either maximise or minimise the overall result. When you consider a - operator, you switch from maximising to minimising and vice versa on the right-hand-side.
These recurrence relations can be evaluated using dynamic programming in a direct way, by building a 3 dimensional table for M of size 2 * (K+1) * (K+1), where K is the number of operators.
Overall, this algorithm is O(K^3).