Question
Do you think genetic algorithms worth trying out for the problem below, or will I hit local-minima issues?
I think maybe aspects of the problem is great for a generator / fitness-function style setup. (If you've botched a similar project I would love hear from you, and not do something similar)
Thank you for any tips on how to structure things and nail this right.
The problem
I'm searching a good scheduling algorithm to use for the following real-world problem.
I have a sequence with 15 slots like this (The digits may vary from 0 to 20) :
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
(And there are in total 10 different sequences of this type)
Each sequence needs to expand into an array, where each slot can take 1 position.
1 1 0 0 1 1 1 0 0 0 1 1 1 0 0
1 1 0 0 1 1 1 0 0 0 1 1 1 0 0
0 0 1 1 0 0 0 1 1 1 0 0 0 1 1
0 0 1 1 0 0 0 1 1 1 0 0 0 1 1
The constraints on the matrix is that:
[row-wise, i.e. horizontally] The number of ones placed, must either be 11 or 111
[row-wise] The distance between two sequences of 1 needs to be a minimum of 00
The sum of each column should match the original array.
The number of rows in the matrix should be optimized.
The array then needs to allocate one of 4 different matrixes, which may have different number of rows:
A, B, C, D
A, B, C and D are real-world departments. The load needs to be placed reasonably fair during the course of a 10-day period, not to interfere with other department goals.
Each of the matrix is compared with expansion of 10 different original sequences so you have:
A1, A2, A3, A4, A5, A6, A7, A8, A9, A10
B1, B2, B3, B4, B5, B6, B7, B8, B9, B10
C1, C2, C3, C4, C5, C6, C7, C8, C9, C10
D1, D2, D3, D4, D5, D6, D7, D8, D9, D10
Certain spots on these may be reserved (Not sure if I should make it just reserved/not reserved or function-based). The reserved spots might be meetings and other events
The sum of each row (for instance all the A's) should be approximately the same within 2%. i.e. sum(A1 through A10) should be approximately the same as (B1 through B10) etc.
The number of rows can vary, so you have for instance:
A1: 5 rows
A2: 5 rows
A3: 1 row, where that single row could for instance be:
0 0 1 1 1 0 0 0 0 0 0 0 0 0 0
etc..
Sub problem*
I'de be very happy to solve only part of the problem. For instance being able to input:
1 1 2 3 4 2 2 3 4 2 2 3 3 2 3
And get an appropriate array of sequences with 1's and 0's minimized on the number of rows following th constraints above.
Sub-problem solution attempt
Well, here's an idea. This solution is not based on using a genetic algorithm, but some ideas could be used in going in that direction.
Basis vectors
First of all, you should generate what I think of as the basis vectors. For instance, if your sequence were 3 numbers long rather than 15, the basis vectors would be:
v1 = [1 1 0]
v2 = [0 1 1]
v3 = [1 1 1]
Any solution for sequence length 3 would be a linear combination of these three vectors using only positive integers. In other words, the general solution would be
a*v1 + b*v2 + c*v3
where a, b and c are positive integers. For the sequence [1 2 1], the solution is v1 = 1, v2 = 1, v3 = 0. What you first want to do is find all of the possible basis vectors of length 15. From my rough calculations I think that there are somewhere between 300-400 basis vectors of length 15. I can give you some tips towards generating them if you want.
Finding solutions
Now, what you want to do is sort these basis vectors by their sums/magnitudes. Then in searching for your solution, you start with the basis vectors which have the largest sums. We start with the vectors that have the largest sums because they lead to having less total rows. We also have an array, veccoefs, which contains an entry for the linear coefficient for each basis vector. At the beginning of searching for the solution, all the veccoefs are 0.
So we take the first basis vector (the one with the largest sum/magnitude) and subtract this vector from the sequence until we either create an unsolvable result ( having a 0 1 0 in it for instance) or any of the numbers in the result is negative. We store the number of times we subtract the vector in veccoefs. We use the result after subtracting the basis vector from the sequence as the sequence for the next basis vector. If there are only zeros left in the result, then we stop the loop.
I'm not sure of the efficiency/accuracy of this method, but it might at least give you some ideas.
Other possible solutions
Another idea for solving this is to use the basis vectors and form the problem as an optimization/least squares problem. You form a matrix of the basis vectors such that the basic problem will be minimizing Sum[(Ax - b)^2] where A is the matrix of basis vectors, b is the input sequence, and x are the basis vector coefficients. However, you also want to minimize the number of rows, so you can add a term like x^T*x to the minimization function where x^T is the transpose of x. The hard part in my opinion is finding differentiable terms to add that will encourage integer vector coefficients. If you can think of a way to do that, then optimization could very well be a good way to do this.
Also, you might consider a Metropolis-type Monte Carlo solution. You would choose randomly whether to add a vector, remove a vector, or substitute a vector at each step. The vector to be added/removed/substituted would be chosen randomly. The probability of this change to be accepted would be a ratio of the suitabilities of the solutions before the change and after the change. The suitability could be equal to the difference between the current solution and the sequence, squared and summed, minus the number of rows/basis vectors involved in the solution. You would need to put in appropriate constants to for various terms to try to get the acceptance rate around 50%. I kind of doubt that this will work very well, but I thought that you should still consider it when looking for possible solutions.
GA can be applied to this problem, but it won't be 5 minute task. You need to put several things together, without knowing which implementation of each of them is best.
So:
Solution representation - how you will represent possible solution? Using matrix seems to be most straight forward. Using collection of one dimensional arrays is possible also.
But you have some constrains, so maybe SuperGene concept is worth considering?
You must use proper mutation/crossover operators for given gene representation.
How will you enforce constrains on solutions? Destroying those that are not proper? What if they contain valuable information? Maybe let them stay in population but add some penalty to fitness, so they will contribute to offspring, but won't go into next generations?
Anyway I think that GA can be applied to this problem. Is it worth? Usually GA are not best algorithm, but they are decent algorithm if others fail. I would go with GA, just because it would be most fun but I would look for alternative solution (just in case).
P.S. Personal insight: I was solving N Queens Problem, for 70 < N < 100 (board NxN, N queens). Algorithm was working fine for lower N (maybe it was trying all combination?), but with N in this range, I couldn't find proper solution. Fitness quickly jumped to about 90% of max, but in the end there were always two queens conflicting. But it was very naive implementation.
Related
Given a bitarray such as the following:
C0 C1 C2 C3 C4 C5
**********************************************
P0 * 0 0 1 0 1 0 *
P1 * 0 1 0 0 1 0 *
P2 * 0 0 0 1 1 0 *
P3 * 1 0 0 0 0 1 *
P4 * 0 0 0 0 0 0 *
P5 * 0 0 0 0 0 0 *
P6 * 1 0 0 0 0 0 *
**********************************************
Each row represents a different person P_i, while each column represent a different color C_j. If a given cell A[i][j] is 1, it means that person i would like color j. A person can only get one color, and a color can only be given to one person.
In general, the number of people P > 0, and the number of colors C >= 0.
How can I, time-efficiently, compute the maximal amount of people who can get a color that they want?
The correct answer to the example above would be 5.
Person 6 (P6) only has one wish, so he gets color 0 (C0)
Since C0 is now taken, P3 only has one wish left, so he gets C5.
P0 gets C2, P1 gets C1 and P2 gets C3.
My first idea was a greedy algorithm, that simply favored the person (i.e. row) with the lowest amount of wanted colors. This works for the most part, but is simply too slow for my liking, as it runs in O(P*(P*C)) time, which is equal to O(n^3) when n = P = C. Any ideas to an algorithm (or another data structure) that can solve the problem quicker?
This might be a duplicate of another similar question, but I had trouble with finding the correct name for the type of problem, so bear with me if this is the case.
This is a classical problem known as maximum cardinality bipartite matching . Here, you have a bipartite graph where in one side you have the vertices corresponding to the people and on the other side the vertices corresponding to the colors. An edge between a person and a color exists if there is a one in the corresponding entry in the matrix.
In the general case, the best known algorithms has worst case performance O(E*sqrt(V)), where E is the number of edges in the graph and V is the number of vertices. One such algorithm is called Hopcroft-Karp. I would suggest you to read the Wikipedia explanation that I linked.
I am searching for a method to create, in a fast way a random matrix A with the follwing properties:
A = transpose(A)
A(i,i) = 0 for all i
A(i,j) >= 0 for all i, j
sum(A) =~ degree; the sum of rows are randomly distributed by a distribution I want to specify (here =~ means approximate equality).
The distribution degree comes from a matrix orig, specifically degree=sum(orig), thus I know that matrices with this distribution exist.
For example: orig=[0 12 7 5; 12 0 1 9; 7 1 0 3; 5 9 3 0]
orig =
0 12 7 5
12 0 1 9
7 1 0 3
5 9 3 0
sum(orig)=[24 22 11 17];
Now one possible matrix A=[0 11 5 8, 11 0 4 7, 5 4 0 2, 8 7 2 0] is
A =
0 11 5 8
11 0 4 7
5 4 0 2
8 7 2 0
with sum(A)=[24 22 11 17].
I am trying this for quite some time, but unfortunatly my two ideas didn't work:
version 1:
I switch Nswitch times two random elements: A(k1,k3)--; A(k1,k4)++; A(k2,k3)++; A(k2,k4)--; (the transposed elements aswell).
Unfortunatly, Nswitch = log(E)*E (with E=sum(sum(nn))) in order that the Matrices are very uncorrelated. As my E > 5.000.000, this is not feasible (in particular, as I need at least 10 of such matrices).
version 2:
I create the matrix according to the distribution from scratch. The idea is, to fill every row i with degree(i) numbers, based on the distribution of degree:
nn=orig;
nnR=zeros(size(nn));
for i=1:length(nn)
degree=sum(nn);
howmany=degree(i);
degree(i)=0;
full=rld_cumsum(degree,1:length(degree));
rr=randi(length(full),[1,howmany]);
ff=full(rr);
xx=i*ones([1,length(ff)]);
nnR = nnR + accumarray([xx(:),ff(:)],1,size(nnR));
end
A=nnR;
However, while sum(A')=degree, sum(A) systematically deviates from degree, and I am not able to find the reason for that.
Small deviations from degree are fine of course, but there seem to be systmatical deviations in particulat of the matrices contain in some places large numbers.
I would be very happy if somebody could either show me a fast method for version1, or a reason for the systematic deviation of the distribution in version 2, or a method to create such matrices in a different way. Thank you!
Edit:
This is the problem in matsmath's proposed solution:
Imagine you have the matrix:
orig =
0 12 3 1
12 0 1 9
3 1 0 3
1 9 3 0
with r(i)=[16 22 7 13].
Step 1: r(1)=16, my random integer partition is p(i)=[0 7 3 6].
Step 2: Check that all p(i)<=r(i), which is the case.
Step 3:
My random matrix starts looks like
A =
0 7 3 6
7 0 . .
3 . 0 .
6 . . 0
with the new row sum vector rnew=[r(2)-p(2),...,r(n)-p(n)]=[15 4 7]
Second iteration (here the problem occures):
Step 1: rnew(1)=15, my random integer partition is p(i)=[0 A B]: rnew(1)=15=A+B.
Step 2: Check that all p(i)<=rnew(i), which gives A<=4, B<=7. So A+B<=11, but A+B has to be 15. contradiction :-/
Edit2:
This is the code representing (to the best of my knowledge) the solution posted by David Eisenstat:
orig=[0 12 3 1; 12 0 1 9; 3 1 0 3; 1 9 3 0];
w=[2.2406 4.6334 0.8174 1.6902];
xfull=zeros(4);
for ii=1:1000
rndmat=[poissrnd(w(1),1,4); poissrnd(w(2),1,4); poissrnd(w(3),1,4); poissrnd(w(4),1,4)];
kkk=rndmat.*(ones(4)-eye(4)); % remove diagonal
hhh=sum(sum(orig))/sum(sum(kkk))*kkk; % normalisation
xfull=xfull+hhh;
end
xf=xfull/ii;
disp(sum(orig)); % gives [16 22 7 13]
disp(sum(xf)); % gives [14.8337 9.6171 18.0627 15.4865] (obvious systematic problem)
disp(sum(xf')) % gives [13.5230 28.8452 4.9635 10.6683] (which is also systematically different from [16, 22, 7, 13]
Since it's enough to approximately preserve the degree sequence, let me propose a random distribution where each entry above the diagonal is chosen according to a Poisson distribution. My intuition is that we want to find weights w_i such that the i,j entry for i != j has mean w_i*w_j (all of the diagonal entries are zero). This gives us a nonlinear system of equations:
for all i, (sum_{j != i} w_i*w_j) = d_i,
where d_i is the degree of i. Equivalently,
for all i, w_i * (sum_j w_j) - w_i^2 = d_i.
The latter can be solved by applying Newton's method as described below from a starting solution of w_i = d_i / sqrt(sum_j d_j).
Once we have the w_is, we can sample repeatedly using poissrnd to generate samples of multiple Poisson distributions at once.
(If I have time, I'll try implementing this in numpy.)
The Jacobian matrix of the equation system for a 4 by 4 problem is
(w_2 + w_3 + w_4) w_1 w_1 w_1
w_2 (w_1 + w_3 + w_4) w_2 w_2
w_3 w_3 (w_1 + w_2 + w_4) w_3
w_4 w_4 w_4 (w_1 + w_2 + w_3).
In general, let A be a diagonal matrix where A_{i,i} = sum_j w_j - 2*w_i. Let u = [w_1, ..., w_n]' and v = [1, ..., 1]'. The Jacobian can be written J = A + u*v'. The inverse is given by the Sherman--Morrison formula
A^-1*u*v'*A^-1
J^-1 = (A + u*v')^-1 = A^-1 - -------------- .
1 + v'*A^-1*u
For the Newton step, we need to compute J^-1*y for some given y. This can be done straightforwardly in time O(n) using the above equation. I'll add more detail when I get the chance.
First approach (based on version2)
Let your row sum vector given by the matrix orig [r(1),r(2),...,r(n)].
Step 1. Take a random integer partition of the integer r(1) into exactly n-1 parts, say p(2), p(3), ..., p(n)
Step 2. Check if p(i)<=r(i) for all i=2...n. If not, go to Step 1.
Step 3. Fill out your random matrix first row and colum by the entries 0, p(2), ... , p(n), and consider the new row sum vector [r(2)-p(2),...,r(n)-p(n)].
Repeat these steps with a matrix of order n-1.
The point is, that you randomize one row at a time, and reduce the problem to searching for a matrix of size one less.
As pointed out by OP in the comment, this naive algorithm fails. The reason is that the matrices in question have a further necessary condition on their entries as follows:
FACT:
If A is an orig matrix with row sums [r(1), r(2), ..., r(n)] then necessarily for every i=1..n it holds that r(i)<=-r(i)+sum(r(j),j=1..n).
That is, any row sum, say the ith, r(i), is necessarily at most as big as the sum of the other row sums (not including r(i)).
In light of this, a revised algorithm is possible. Note that in Step 2b. we check if the new row sum vector has the property discussed above.
Step 1. Take a random integer partition of the integer r(1) into exactly n-1 parts, say p(2), p(3), ..., p(n)
Step 2a. Check if p(i)<=r(i) for all i=2...n. If not, go to Step 1.
Step 2b. Check if r(i)-p(i)<=-r(i)+p(i)+sum(r(j)-p(j),j=2..n) for all i=2..n. If not, go to Step 1.
Step 3. Fill out your random matrix first row and colum by the entries 0, p(2), ... , p(n), and consider the new row sum vector [r(2)-p(2),...,r(n)-p(n)].
Second approach (based on version1)
I am not sure if this approach gives you random matrices, but it certainly gives you different matrices.
The idea here is to change some parts of your orig matrix locally, in a way which maintains all of its properties.
You should look for a random 2x2 submatrix below the main diagonal which contains strictly positive entries, like [[a,b],[c,d]] and perturbe its contents by a random value r to [[a+r,b-r],[c-r,d+r]]. You make the same change above the main diagonal too, to keep your new matrix symmetric. Here the point is that the changes within the entries "cancel" each other out.
Of course, r should be chosen in a way such that b-r>=0 and c-r>=0.
You can pursue this idea to modify larger submatrices too. For example, you might choose 3 random row coordinates r1, r2, r2 and 3 random column coordinates c1, c2, and c3 and then make changes in your orig matrix at the 9 positions (ri,cj) as follows: you change your 3x3 submatrix [[a b c],[d e f], [g h i]] to [[a-r b+r c] [d+r e f-r], [g h-r i+r]]. You do the same at the transposed places. Again, the random value r must be chosen in a way so that a-r>=0 and f-r>=0 and h-r>=0. Moreover, c1 and r1, and c3 and r3 must be distinct as you can't change the 0 entries in the main diagonal of the matrix orig.
You can repeat such things over and over again, say 100 times, until you find something which looks random. Note that this idea uses the fact that you have existing knowledge of a solution, this is the matrix orig, while the first approach does not use such knowledge at all.
Considering I have 4 chromosomes (gi, i=1 to 4}) to represent 4 percentages of different things so that the sum of 4 percentages are equal to 100. How Do I represent this efficiently?
I know that it is possible by: g1/(g1+g2+g3+g4). However, This is not efficient. Consider all gi=0.2 or all gi=0.1 will represent 25% in these two cases. It is possible to generate many cases where different genes present same percentage. Is there any other efficient way, where unique set of combination of genes present unique set of percentages.
Thanks in advance.
I think you're confusing genes and chromosomes. A chromosome encodes a candidate solution to your problem. A gene is part of a chromosome.
Under this setting, why would you want that constraint on the chromosomes? it sounds like you want it on the genes of a chromosome.
In order to do this you can do a number of things: have each gene encode an integer in [0, 100]. If the genes do not add to 100 in the end, penalize the fitness of those chromosomes.
Another way, which might make crossover operators more natural to apply, is to have each gene store 100 bits. If x bits are set, that means the gene will encode x%.
Yet another way is to have the entire chromosome encode 100 set bits. Then each gene will hold a value x, which represents an interval. The number of set bits between two split points is the percentage associated to that gene. For example:
1 2 3 4 5 6 7 8 ... 100
1 1 1 1 1 1 1 1 ... 1
| | | | |
g1 g2 g3 g4
This can be done by generating 5 random numbers <= 100, sorting them and taking the differences between them.
One way to assign X units to N possibilities is to store X * (N-1) bits. Every unit is given (N-1) bits and if k of the (N-1) bits are set then the unit is assigned to k.
This is easy to work with as there are no invalid solutions and no penalties/repairs are necessary. This makes fitness evaluation, crossover and mutation easier to implement.
For example, the problem is to assign 5 units (X) to one of 4 (N) possibilities. Each individual is (4-1)x5=15 bits.
The bit string: 010 100 000 011 111 assigns the first 2 units to possibility 1 because both groups have 1 bit set. The third unit which has no bits set is assigned to 0. The fourth unit is assigned to 2 and the fifth to 3.
partition units
0 1
1 2
2 1
3 1
Lets say we have 3 people, Alice, Bob, and Charlie.
Lets say each of them have a resource, Aplles, Bannanas, and Coconuts.
Each of them have 3 of this resource.
The goal of the algorithm is to make 1-1 trades such that each of them end up with 1 of each of our 3 resources. A list of those trades is what I want to obtain.
Ideally I would like to know how to solve this. But I'm willing to settle for the name of this kind of problem, or a problem similar to it that I can research and get ideas from.
The problem I'm working on will have around 600 objects, with ~1000 people each with a random amount/type of starting resources, (with the assumption that there are enough resources to satisfy our end result) so Ideally any solution provided would be feasible for such a scale. But I'll take whatever I can get, I just need some kind of starting point.
The answers of ElKamina and Tyler Durden are decent, but they don't seem to take into account that Kuriso would like to perform 1-1 trades, that people may have multiple commodities, and multiple units of commodities. I have a naive solution that does.
I think the original example was a bit oversimplified, so let's take another one:
c1 c2 c3 c4
A 5 0 1 0
B 0 1 0 1
C 0 6 2 0
Where A,B,C are people and c1,c2,c3,c4 are the commodities.
First, let's calculate the ideal distribution, which is easily done: for each commodity, divide the sum of stuff by the number of people, rounded down, and everybody gets that:
c1 c2 c3 c4
A 1 2 1 0
B 1 2 1 0
C 1 2 1 0
Now let's define a WANT function denoting how much of a stuff c would person X need to get into the ideal position: WANT(X,c) = IDEAL(c) - Xc.
c1 c2 c3 c4 sum
A -4 2 0 0 -2
B 1 1 1 0 3
C 1 -4 -1 0 -4
Let's make a list of people ordered by the sum of their wants. Let's take the richest guy, the one with the lowest want, in this case C, and let's try to satisfy his wants by matching him up with people who has the most to offer of the commodity he wants most. If they can make a trade, great, if not, continue until we find a match (a match is guaranteed, eventually). In this example, C needs c1; the one offering the most c1 is A, iterating over the commodities, we find that A needs c2 and C does have surplus c2, so they exchange them. Update their position in the list, or remove them if they no longer have any needs. Iterate this until nobody has any wants. This won't produce properly equal distribution, but as equal as they can get to by 1 for 1 trading.
This is indeed a naive solution, with the heuristics that the richest guy has the most chance to offer stuff in return for the commodity he needs. The complexity is high, but with ordered lists it should be managable for the numbers you specified.
Assume you have a total number of x1 resources of kind 1,..., xn resources of kind n.
Assume you have k people and each of them have (or need to end up with y1, y2,..., yk resources respectively.
Now, pick a person i and assign him resources that are most prevalent. Once assignment is done, decrement the corresponding xj s (i.e. if resource j is assigned to i, decrement xj).
Keep repeating until all resources are assigned.
This is the way to assign stuff most evenly. It assumes that you dont care about sequences of trades, but the end result itself.
To restate this, let's say you have set of lists like this:
{ 1, 1, 1 }
{ 2, 2, 2 }
{ 3, 3, 3 }
and you want to swap elements from different sets until you have the sets like this:
{ 1, 2, 3 }
{ 1, 2, 3 }
{ 1, 2, 3 }
Now, you might notice that if we regard these lists as a single matrix then one matrix is the inverse of the other. You can perform this inversion by swapping across the 1-2-3 diagonal.
So item 2 in list 1 is swapped with item 2 in row 2, item 3 in list 1 is swapped with item 1 in list 3, and finally item 3 in list 2 is swapped with item 2 in list 3.
To sum up: do a matrix inversion by swapping across the diagonal.
Alice invents a key (s1, s2, s3, ... , sk). Bob makes a guess (g1, g2, g3, ... , gk).He is awarded one point for each si = gi.
Each s1 is an integer with the range of 0<=si<=11.
Given a q guesses with their scores bi
(g1, g2, g3, ... , gk) b1
(g1, g2, g3, ... , gk) b2
.
.
.
(g1, g2, g3, ... , gk) bq
Can you state if there is a key possible. Given 0<=si<=11, 1<=k<=11, 1<=q<=8.
For Example
2 2 1 1 2
1 1 2 2 1
For the guess 2 2 1 1 the score is 2
For the guess 1 1 2 2 the score is 1
Because there is a key possible let's say 2 1 1 3 which gives the desired scores.Hence the answer is yes
Another Example
1 2 3 4 4
4 3 2 1 1
For the guess 1 2 3 4 the score is 4
For the guess 4 3 2 1 the score is 1
This has no key which gives the desired scores hence answer is NO
I tried the brute force approach generating n^k such keys where n is the range of si.But it gave Time Limit exceeding error.
Its an interview puzzle. I have seen variants of this question but was not able to solve them.Can you tell me what should I read for such type of questions.
I don't know the best solution to this problem, but if you did a recursive search of the possible solution space, pruning branches which could not possibly lead to a solution, it would be much faster than trying all (n^k) keys.
Take your example:
1 2 3 4 4 -> 4
4 3 2 1 1 -> 1
The 3 possible values for g1 which could be significant are: 1, 4, and "neither 1 nor 4". Choose one of them, and then recursively look at the possible values for g2. Choose one, and recursively look at the possible values for g3, etc.
As you search, keep track of a cumulative score for each of the guesses from b1 to bq. Whenever you choose a value for a digit, increment the cumulative scores for all the guesses which have the same number in that position. Keep these cumulative scores on a stack (so you can back up).
When you reach a point where no solution is possible, back up and continue searching a different path. If you back all the way up to g1 and no more paths are left to search, then the answer is NO. If you find a solution, then the answer is YES.
When to stop searching a path and back up:
If the cumulative score of one of the guesses exceeds the given score
If the cumulative score of one of the guesses is less than the given score minus the number of levels left in the search tree (before you hit the bottom)
This approach could still be very slow, especially if "k" was large. But again, it will be far faster than generating (n^k) keys.