I'm implementing a genetic algorithm, and I'm uncertain on how to pick the breeders for the next generation:
I am holding a list of all the past individuals calculated,
Is it ok if I select the breeders from this list? or should I rather pick the best ones from the latest generation?
If you only select from the latest generation, it's possible for your population to evolve backwards. There's no guarantee that later generations are better than earlier ones. To prevent this, some algorithms maintain a pool of "elite" individuals that continually mix with the regular population. (The strategy is called "elitism".) A particularly successful version of this approach is Coello's micro-GA, which uses a very small population with elite preservation and frequent restarts to make progress.
It's usually preferable to select the ones that have the highest fitness value
Based on a certain function that you define, evaluate individuals in your population and choose the best N ones. For example, the lightest rocks in algorithm where you want to generate light rocks.
If computing the fitness value of all individuals in your population is costly operations, you should first select a sample based on some distribution. A good one is to select in a uniform fashion (all individuals have equal probabilities to be selected)
If you can't easily define a fitness function, a good technique is to run simulations. For example if your phenotype (criteria) is hard to define, like a shape of an irregular 3D object for example.
You can try one of the following methods to select breeders( parent)
- Roulette wheel selection
- Stochastic Universal Sampling
- Tournament Selection
- Random Selection
Reference :
https://www.tutorialspoint.com/genetic_algorithms/genetic_algorithms_parent_selection.htm
Related
I am implementing a roulette wheel selection algorithm for a genetic algorithm solution to the WHPP scheduling problem.
The problem I am running into is that the initial population (generated randomly) begins with very similar (very low) fitness values, resulting in even (at least very close to) probabilities between the parents and subsequently their children and therefore almost random selection right from the start of the execution.
The question is how would I go about this? Should I find another way of evaluating the population so that the best of them have a disproportionately higher chance to get selected? Or is the way that I generate the initial population not supposed to yield uniform fitness values?(meaning that I am doing something wrong right from the beginning)
This is for an AI assignment I have to turn over and I can't get a straight answer from the teaching staff for some reason. Thanks in advance, I know it's a very vague question but I can't get information anywhere.
Turns out that wheel selection was not a good option for my case. I ended up using a rank selection algorithm which does exactly what I was thinking to do with my wheel selection. That is, it assigns ranks based on fitness values and then you can calculate probabilities based on that.
This way fitter individuals get a better chance even with small advantage over the rest of the population. You can also control how much of an advantage they'll have using a bias multiplier variable. This also works in case some of the individuals have an extremely large fitness value compared to the others.
I am currently trying to coding a genetic algorithm which intended to find the optimal solution for timetable planning. I have successfully created the population and also able to calculate the fitness, what I confuse is that mating pool and the selection.
I am planning to do the tournament solution.
What I know So far is that I need to select a random number of candidate, and choose the "most fit" and the first parent. Repeat the step and find the second parent. the crossover each other. But How many crossovers I need to do? Until the same size of the population size that I set. then How about my original population?
Can Someone help me?
Your original population dies. If you want to keep the best solution(s) you can copy it to the new population (this is called elitism). You then generate offspring until your new population is full. Have a look at this Outline of the Basic Genetic Algorithm.
Cross-over is just one way to make children "different" from parents (in addition to mutation). This is independent from how you select good parents (e.g. tournaments). But note that there are so many GA variants that this may not be true for all of them.
I would consider to start with only mutation (no cross-over). This is much easier to implement, and sometimes good enough. You can always add cross-over later and see if you get an improvement.
Reading the book "Genetic Algorithms" by David E. Goldberg, he mentions fitness scaling in Genetic Algorithms.
My understanding of this function is to constrain the strongest candidates so that they don't flood the pool for reproduction.
Why would you want to constrain the best candidates? In my mind having as many of the best candidates as early as possible would help get to the optimal solution as fast as possible.
What if your early best candidates later on turn out to be evolutionary dead ends? Say, your early fittest candidates are big, strong agents that dominate smaller, weaker candidates. If all the weaker ones are eliminated, you're stuck with large beasts that maybe have a weakness to an aspect of the environment that hasn't been encountered yet that the weak ones can handle: think dinosaurs vs tiny mammals after an asteroid impact. Or, in a more deterministic setting that is more likely the case in a GA, the weaker candidates may be one or a small amount of evolutionary steps away from exploring a whole new fruitful part of the fitness landscape: imagine the weak small critters evolving flight, opening up a whole new world of possibilities that the big beasts most likely will never touch.
The underlying problem is that your early strongest candidates may actually be in or around a local maximum in fitness space, that may be difficult to come out of. It could be that the weaker candidates are actually closer to the global maximum.
In any case, by pruning your population aggressively, you reduce the genetic diversity of your population, which in general reduces the search space you are covering and limits how fast you can search this space. For instance, maybe your best candidates are relatively close to the global best solution, but just inbreeding that group may not move it much closer to it, and you may have to wait for enough random positive mutations to happen. However, perhaps one of the weak candidates that you wanted to cut out has some gene that on its own doesn't help much, but when crossed with the genes from your strong candidates in may cause a big evolutionary jump! Imagine, say, a human crossed with spider DNA.
#sgvd's answer makes valid points but I would like to elaborate more.
First of all, we need to define what fitness scaling actually means. If it means just multiplying the fitnesses by some factor then this does not change the relationships in the population - if the best individual had 10 times higher fitness than the worst one, after such multiplication this is still true (unless you multiply by zero which makes no real sense). So, a much more sensible fitness scaling is an affine transformation of the fitness values:
scaled(f) = a * f + b
i.e. the values are multiplied by some number and offset by another number, up or down.
Fitness scaling makes sense only with certain types of selection strategies, namely those where the selection probability is proportional to the fitness of the individuals1.
Fitness scaling plays, in fact, two roles. The first one is merely practical - if you want a probability to be proportional to the fitness, you need the fitness to be positive. So, if your raw fitness value can be negative (but is limited from below), you can adjust it so you can compute probabilities out of it. Example: if your fitness gives values from the range [-10, 10], you can just add 10 to the values to get all positive values.
The second role is, as you and #sgvd already mentioned, to limit the capability of the strongest solutions to overwhelm the weaker ones. The best illustration would be with an example.
Suppose that your raw fitness values gives values from the range [0, 100]. If you left it this way, the worst individuals would have zero probability of being selected, and the best ones would have up to 100x higher probability than the worst ones (excluding the really worst ones). However, let's set the scaling factors to a = 1/2, b = 50. Then, the range is transformed to [50, 100]. And right away, two things happen:
Even the worst individuals have non-zero probability of being selected.
The best individuals are now only 2x more likely to be selected than the worst ones.
Exploration vs. exploitation
By setting the scaling factors you can control whether the algorithm will do more exploration over exploitation and vice versa. The more "compressed"2 the values are going to be after the scaling, the more exploration is going to be done (because the likelihood of the best individuals being selected compared to the worst ones will be decreased). And vice versa, the more "expanded"2 are the values going to be, the more exploitation is going to be done (because the likelihood of the best individuals being selected compared to the worst ones will be increased).
Other selection strategies
As I have already written at the beginning, fitness scaling only makes sense with selection strategies which derive the selection probability proportionally from the fitness values. There are, however, other selection strategies that do not work like this.
Ranking selection
Ranking selection is identical to roulette wheel selection but the numbers the probabilities are derived from are not the raw fitness values. Instead, the whole population is sorted by the raw fitness values and the rank (i.e. the position in the sorted list) is the number you derive the selection probability from.
This totally erases the discrepancy when there is one or two "big" individuals and a lot of "small" ones. They will just be ranked.
Tournament selection
In this type of selection you don't even need to know the absolute fitness values at all, you just need to be able to compare two of them and tell which one is better. To select one individual using tournament selection, you randomly pick a number of individuals from the population (this number is a parameter) and you pick the best one of them. You repeat that as long as you have selected enough individuals.
Here you can also control the exploration vs. exploitation thing by the size of the tournament - the larger the tournament is the higher is the chance that the best individuals will take part in the tournaments.
1 An example of such selection strategy is the classical roulette wheel selection. In this selection strategy, each individual has its own section of the roulette wheel which is proportional in size to the particular individual's fitness.
2 Assuming the raw values are positive, the scaled values get compressed as a goes down to zero and as b goes up. Expansion goes the other way around.
I am working on a chess engine, and am using the Gene Expression Programming approach to evolve the evaluation function.
As there is no oracle for chess, my fitness function is only capable of finding the relative fitness of two individuals (by performing a chess match).
Since I do not have an absolute fitness measure, I might end up with some individuals which are better than each other in a circular fashion ( $R_A_B(A)>R_A_B(B), R_B_C(B)>R_B_C(C), R_A_C(C)>R_A_C(A)$ )
So, what are some ways of efficiently evolving the individuals in such a scenario, and how can I avoid ending up in such circular mess?
Thanks :)
The circular fashion ( R_A_B(A)>R_A_B(B), R_B_C(B)>R_B_C(C), R_A_C(C)>R_A_C(A) ) cannot be completely avoided, because chess game is a good example of a perfect NP problem and a detailed behavior of such systems cannot be evaluated with great accuracy.
lets consider a chess board position P. Lets represent variations generated by individual A form the position P in a chess board to belong the set Ap. Lets consider another set Bp where the evaluate function defined by B is used to obtain variations at that position. Lets define a function Q(x) that tests the quality of the variations provided by any individual as the game proceeds. So at P let Q(Ap) > Q(Bp) then for any other position p', Q(Bp') > Q(Ap').
Evolving an individual that generates best variations for all positions is impossible as you yourself stated that there is no oracle for chess;
but that's not going to be a problem. here I have suggested a method that might help;
Instead of trying to provided a unique rank to individuals in such a mess, why can't they be treated equally? If they are treated equally, One problem that could arise is that the first few generations could largely end up in such a circular mess (there would be too many individuals being equal in strength). So its not wrong to use a different fitness function for the first few generations to improve their efficiency enough to solve basic chess puzzles (by defining a fitness function with a simpler endpoint). Later generations would fall in this circular mess for lesser number of times and that would not be a problem. If the fitness function depends upon the result of a match played between two individuals in a particular population, then this would set the end point of this evolution as: attaining the playing strength of the hypothetical function G(x) (which is never possible). So the individuals generated would try to evolve in a broader scale, by trying to become perfect in all board situations (which would slow down the process if the individuals present in the initial generations are of weak playing strength).
The other method I would suggest would be to try to define a constant function N(x), that is made to work in a higher time complexity (i.e. evaluates longer than the individuals selected from a particular generation by checking more variations). Now we may compare the values R_A_N(A), R_B_N(B), R_C_N(C) separately to rank them in an order. It is not really necessary to create a unique constant evaluate function for this purpose, any random individual can be selected for this purpose.
Using a selective individual N (made to search for a greater depth) and then trying to find the R_A_N(A), is more like selecting a origin in a graph where only relative positions of certain points are known. Here the population is made to develop relative to the selected individual; N does not have to be a constant function with fixed parameters, it could also be one of the individuals present in this circular mess being made to run in an engine that tests more number of variations( higher search depth) . If individual A is chosen as N, and is made to play with itself run in a greater depth, then its obvious that the individual A which runs with a higher depth beats A run at a lower depth. here, we could define a fitness function that depends upon factors other than the end result of a chess match; for example,
so defining R_A_N(A), R_B_N(B), ... this way could avoid this circular mess.
I'm building my first Genetic Algorithm in javascript, using a collection of tutorials.
I'm building a somewhat simpler structure to this scheduling tutorial http://www.codeproject.com/KB/recipes/GaClassSchedule.aspx#Chromosome8, but I've run into a problem with breeding.
I get a population of 60 individuals, and now I'm picking the top two individuals to breed, and then selecting a few random other individuals to breed with the top two, am I not going to end up with a fairly small amount of parents rather quickly?
I figure I'm not going to be making much progress in the solution if I breed the top two results with each of the next 20.
Is that correct? Is there a generally accepted method for doing this?
I have a sample of genetic algorithms in Javascript here.
One problem with your approach is that you are killing diversity in the population by mating always the top 2 individuals. That will never work very well because it's too greedy, and you'll actually be defeating the purpose of having a genetic algorithm in the first place.
This is how I am implementing mating with elitism (which means I am retaining a percentage of unaltered best fit individuals and randomly mating all the rest), and I'll let the code do the talking:
// save best guys as elite population and shove into temp array for the new generation
for(var e = 0; e < ELITE; e++) {
tempGenerationHolder.push(fitnessScores[e].chromosome);
}
// randomly select a mate (including elite) for all of the remaining ones
// using double-point crossover should suffice for this silly problem
// note: this should create INITIAL_POP_SIZE - ELITE new individualz
for(var s = 0; s < INITIAL_POP_SIZE - ELITE; s++) {
// generate random number between 0 and INITIAL_POP_SIZE - ELITE - 1
var randInd = Math.floor(Math.random()*(INITIAL_POP_SIZE - ELITE));
// mate the individual at index s with indivudal at random index
var child = mate(fitnessScores[s].chromosome, fitnessScores[randInd].chromosome);
// push the result in the new generation holder
tempGenerationHolder.push(child);
}
It is fairly well commented but if you need any further pointers just ask (and here's the github repo, or you can just do a view source on the url above). I used this approach (elitism) a number of times, and for basic scenarios it usually works well.
Hope this helps.
When I've implemented genetic algorithms in the past, what I've done is to pick the parents always probabilistically - that is, you don't necessarily pick the winners, but you will pick the winners with a probability depending on how much better they are than everyone else (based on the fitness function).
I cannot remember the name of the paper to back it up, but there is a mathematical proof that "ranking" selection converges faster than "proportional" selection. If you try looking around for "genetic algorithm selection strategy" you may find something about this.
EDIT:
Just to be more specific, since pedalpete asked, there are two kinds of selection algorithms: one based on rank, one based on fitness proportion. Consider a population with 6 solutions and the following fitness values:
Solution Fitness Value
A 5
B 4
C 3
D 2
E 1
F 1
In ranking selection, you would take the top k (say, 2 or 4) and use those as the parents for your next generation. In proportional ranking, to form each "child", you randomly pick the parent with a probability based on fitness value:
Solution Probability
A 5/16
B 4/16
C 3/16
D 2/16
E 1/16
F 1/16
In this scheme, F may end up being a parent in the next generation. With a larger population size (100 for example - may be larger or smaller depending on the search space), this will mean that the bottom solutions will end up being a parent some of the time. This is OK, because even "bad" solutions have some "good" aspects.
Keeping the absolute fittest individuals is called elitism, and it does tend to lead to faster convergence, which, depending on the fitness landscape of the problem, may or may not be what you want. Faster convergence is good if it reduces the amount of effort taken to find an acceptable solution but it's bad if it means that you end up with a local optimum and ignore better solutions.
Picking the other parents completely at random isn't going to work very well. You need some mechanism whereby fitter candidates are more likely to be selected than weaker ones. There are several different selection strategies that you can use, each with different pros and cons. Some of the main ones are described here. Typically you will use roulette wheel selection or tournament selection.
As for combining the elite individuals with every single one of the other parents, that is a recipe for destroying variation in the population (as well as eliminating the previously preserved best candidates).
If you employ elitism, keep the elite individuals unchanged (that's the point of elitism) and then mate pairs of the other parents (which may or may not include some or all of the elite individuals, depending on whether they were also picked out as parents by the selection strategy). Each parent will only mate once unless it was picked out multiple times by the selection strategy.
Your approach is likely to suffer from premature convergence. There are lots of other selection techniques to pick from though. One of the more popular that you may wish to consider is Tournament selection.
Different selection strategies provide varying levels of 'selection pressure'. Selection pressure is how strongly the strategy insists on choosing the best programs. If the absolute best programs are chosen every time, then your algorithm effectively becomes a hill-climber; it will get trapped in local optimum with no way of navigating to other peaks in the fitness landscape. At the other end of the scale, no fitness pressure at all means the algorithm will blindly stumble around the fitness landscape at random. So, the challenge is to try to choose an operator with sufficient (but not excessive) selection pressure, for the problem you are tackling.
One of the advantages of the tournament selection operator is that by just modifying the size of the tournament, you can easily tweak the level of selection pressure. A larger tournament will give more pressure, a smaller tournament less.