I'm currently writing a paper on genetic algorithms. I'd like to have a small subsection on the termination condition which determines when the algorithm has to stop.
I found this nice site:
http://www.nd.com/genetic/termination.html
which lists some nice methods via which the termination condition in genetic algorithms is determined. However, the professor who's in charge of my paper is very reluctant to accept random internet sites as sources.
Can you give some recommendations on books (would be very good if I can get my hands on them in e-book form) which contain more information about these termination conditions and maybe expand on the various method's advantages or disadvantages?
Unfortunately, although this is a very interesting field of research, it has only received little attention until now.
Although the original question was originally requesting a book, you might be interested in this published article that discuss some termination criteria: On Stopping Criteria for Genetic Algorithms (Martín Safe, Jessica Carballido, Ignacio Ponzoni and Nélida Brignole) at this url: http://www.springerlink.com/content/cgt7635e3lhu3pdn/
You might also be interested in reading a few pages the PhD thesis "Analysis and Improvement of Genetic Algorithms using Concepts from Information Theory" from John Milton, section 3.6 (pages 127 and following), which discuss some termination conditions. Available at: http://epress.lib.uts.edu.au/scholarly-works/bitstream/handle/2100/1004/02Whole.pdf?sequence=2
Your best bet is probably any good general introductory text on genetic algorithms, which are likely to have a brief mention of possible termination criterion. The only such book I had to hand was A Field Guide to Genetic Programming (GP being very closely related to GA). In there, there is a short paragraph (section 3.5), which mention maximum number of generations, or "a problem-specific success predicate". In practice these two are the most used, with the latter just meaning stop whenever a complete solution is found.
Related
For my research project in biology for my final year I need to present a project in the field of Biotechnology. Being passionate about programming I immediately thought of Evolutionary Algorithms! However I am not sure if Evolutionary Algorithms would fall into the category of Biotechnology, hence I would rather confirm with the best and most passionate programming experts on the world.
Unfortunately no, a genetic algorithm (ga) is just an optimization technique that is inspired from various evolutionary processes like mutation or crossover. They belong to the area of evolutionary computing and artificial intelligence and not biotechnology.
Please follow this link for a brief introduction to genetic algorithms.
Biotechnology from the other hand has to do with actual organisms that are used in some way to make a product or an application. It sounds kind of broad but that is only because the particular field is in itself very broad. We use forms of biotechnology for thousands of years now in many common and not so common ways. This is not bad though as it gives you a lot of freedom regarding your project. Choose anything from food production to medicine and you will still be relevant to the subject.
Maybe the links provided will give you some inspiration.
Link one
Link two
Until you're implementing your evolutionary algorithms with organic material, no.
They are, of course, inspired from the way modern organisms have come to exist. But there's no biology in what you're doing.
No. It's just an example of a biological algorithm adapted for computational purposes.
Other examples include Ant-Colony Optimization, Flocking behavior, etc.
IIRC, Biotechnology requires the use of actual biology (i.e., living things or parts of them) adapted for technological purposes, not just an algorithmic emulation or modelling of their processes.
What is a collaborative algorithm? Is there a scientifically citable reference?
Details:
I found many articles about collaborative algorithms, but none (or other websites) with a definition.
I am actually looking for a term to describe distributed algorithms where each instance has all information at the beginning and can complete the whole task on its own, but the instances help each other whenever they have solved a sub-problem, so the other instances do not have to redo the work (hence "collaboration"). I picked up this terminology in A Collaborative Approach for Multi-Threaded SAT Solving. Do you think the term "collaborative algorithm" is suitable for this? If not, do you know of a better term?
No, there's no scientifically citable references.
All parallel/distributed programming is "collaborative" in a sense that several threads/nodes are collaborating on the same big task.
distributed algorithms where.. instances help each other whenever they have solved a sub-problem - even some web application clusters fit your description: individual cluster nodes "solve subproblems" and store the "solutions" in a distributed in-RAM storage (such as memcached or cassandra or many others) thus helping each other.
I think the term "collaborative algorithm" is not formal.
Actually the term "algorithm" itself is not really formal,
as far as I remember. I guess algorithm can be formalized as
"a program which runs on a Turing machine". I think I've seen
this definition somewhere.
So yes, I guess all in all the term you coined makes sense, but you
need to define it somehow yourself (either formally or informally).
Not sure what your background is but ... OK, in scientific papers
different authors sometimes use same terms/concepts for denoting
different things and sometimes they use different terms for
denoting the same thing.
Also, even though computer science papers are scientific not
all terms in them are formally defined. So I wouldn't draw
too many conclusions based on these papers unless I am familiar
to a decent extent with all of them or unless some of them
are considered really remarkable and widely accepted as
a de-facto standard in a particular sub-field or field.
I'm wondering if anyone knows of any software techniques taking advantage of biology? For example, in the robotics world, there are tons, but what about software?
Many concepts originally observed in biology have been used in software. For example Genetic Algorithm (GA).
Artificial life (AL) exposes/uses several principles of biology such as resilience to imperfect code snippets, addressing by content, imperfect reproduction (in some implementations, also sexual, i.e. multi-orginanisms-driven, reproduction) and a non-goal-driven utility function. An interesting result of AL, is the spontaneous production of macro phenomenons observed in domains such as ecology or epidemiology (domains largely influenced by biology), such as the emergence of parasites and even that of organisms which take advantage of parasites, or subtle predator-prey relationships.
Maybe software can be said to have gone "full circle" with some experiments in computing which involve real (carbon-based) DNA (or RNA) molecules! The original experiment in this area (PDF link) by Prof. Alderman (of RSA fame), who coded the various elements of a graph-related problem (an hamiltonian graph) with different DNA molecules and let the massive parallel computing power of bio-chemistry do the rest and solve the problem !
Back in the digital world, but with a strong inspiration from biology and indeed from anatomy of the cerebral cortex, and from many theoretical and clinical observations in the neuroscience field, we have Neural Networks (NN). In the area of NN, maybe worthy of a special notice, is Numenta's Hierarchical Temporal Memory model which, although it reproduces the [understanding we have of] the neo-cortex only very loosely, introduces the idea that the very same algorithm is applied in all areas and at all levels of the cognitive process powered by the brains, an idea largely supported by biological, anatomical and other forms of evidence.
If your question means "have biological ideas been used to optimize software?" then
Genetic programming (http://en.wikipedia.org/wiki/Genetic_programming) is one example. From the Wikipedia article:
In artificial intelligence, genetic programming (GP) is an evolutionary algorithm-based methodology inspired by biological evolution to find computer programs that perform a user-defined task. It is a specialization of genetic algorithms (GA) where each individual is a computer program. Therefore it is a machine learning technique used to optimize a population of computer programs according to a fitness landscape determined by a program's ability to perform a given computational task.
If your question means "what software techniques have been inspired by biology?" then
see more generally http://en.wikipedia.org/wiki/Bio-inspired_computing. I would expect that several other methods such as ant-swarms (http://en.wikipedia.org/wiki/Ant_colony_optimization) and Neural Networks (http://en.wikipedia.org/wiki/Neural_network_software) could also be used.
Artificial Neural Networks are another classic example. The software application tends to be pattern recognition and prediction of behaviour of complex systems.
Ant colony optimization, a search / optimization method, and Artificial Life like Conway's Game of Life
Most of the answers yet talk about AI. The title of your question hints towards software that hides itself in order not to be detected.
We got viruses.
We got virus-hunters...
Me myself, I even hid some bugs in my own programs ... :(
Alan Kay (the object technology pioneer) spoke at length about the influence of biology in the OOP paradigm. He's got a series of ideas about how objects are like "cells" and that OOP scales in a similar way to the way that cells can scale to produce massive architectures...
You can follow quite a bit of this in his Turing Award Speech:
http://video.google.com/videoplay?docid=-2950949730059754521# -- Skip to about the 30:55 mark
First of all, is this only possible on algorithms which have no side effects?
Secondly, where could I learn about this process, any good books, articles, etc?
COQ is a proof assistant that produces correct ocaml output. It's pretty complicated though. I never got around to looking at it, but my coworker started and then stopped using it after two months. It was mostly because he wanted to get things done quicker, but if you need to verify an algorithm this might be a good idea.
Here is a course that uses COQ and talks about proving algorithms.
And here is a tutorial about writing academic papers in COQ.
It's generally a lot easier to verify/prove correctness when no side effects are involved, but it's not an absolute requirement.
You might want to look at some of the documentation for a formal specification language like Z. A formal specification isn't a proof itself, but is often the basis for one.
I think that verifying the correctness of an algorithm would be validating its conformance with a specification. There is a branch of theoretical Computer Science called Formal Methods which may be what you are looking for if you need to get as close to proof as you can. From wikipedia,
Formal Methods are a particular kind
of mathematically-based techniques for
the specification, development and
verification of software and hardware
systems
You will be able to find many learning resources and tools from the multitude of links on the linked Wikipedia page and from the Formal Methods wiki.
Usually proofs of correctness are very specific to the algorithm at hand.
However, there are several well known tricks that are used and re-used again. For example, with recursive algorithms you can use loop invariants.
Another common trick is reducing the original problem to a problem for which your algorithm's proof of correctness is easier to show, then either generalizing the easier problem or showing that the easier problem can be translated to a solution to the original problem. Here is a description.
If you have a particular algorithm in mind, you may do better in asking how to construct a proof for that algorithm rather than a general answer.
Buy these books: http://www.amazon.com/Science-Programming-Monographs-Computer/dp/0387964800
The Gries book, Scientific Programming is great stuff. Patient, thorough, complete.
Logic in Computer Science, by Huth and Ryan, gives a reasonably readable overview of modern systems for verifying systems. Once upon a time people talked about proving programs correct - with programming languages which may or may not have side effects. The impression I get from this book and elsewhere is that real applications are different - for instance proving that a protocol is correct, or that a chip's floating point unit can divide correctly, or that a lock-free routine for manipulating linked lists is correct.
ACM Computing Surveys Vol 41 Issue 4 (October 2009) is a special issue on software verification. It looks like you can get to at least one of the papers without an ACM account by searching for "Formal Methods: Practice and Experience".
The tool Frama-C, for which Elazar suggests a demo video in the comments, gives you a specification language, ACSL, for writing function contracts and various analyzers for verifying that a C function satisfies its contract and safety properties such as the absence of run-time errors.
An extended tutorial, ACSL by example, shows examples of actual C algorithms being specified and verified, and separates the side-effect-free functions from the effectful ones (the side-effect-free ones are considered easier and come first in the tutorial). This document is also interesting in that it was not written by the designers of the tools it describe, so it gives a fresher and more didactic look at these techniques.
If you are familiar with LISP then you should definitely check out ACL2: http://www.cs.utexas.edu/~moore/acl2/acl2-doc.html
Dijkstra's Discipline of Programming and his EWDs lay the foundation for formal verification as a science in programming. A simpler work is Wirth's Systematic Programming, which begins with the simple approach to using verification. Wirth uses pre-ISO Pascal for the language; Dijkstra uses an Algol-68-like formalism called Guarded (GCL). Formal verification has matured since Dijkstra and Hoare, but these older texts may still be a good starting point.
PVS tool developed by Stanford guys is a specification and verification system. I worked on it and found it very useful for Theoram Proving.
WRT (1), you will probably have to create a model of the algorithm in a way that "captures" the side-effects of the algorithm in a program variable intended to model such state-based side-effects.
I've been reading things here and there for a while now about using an "ant colony" model as a heuristic approach to optimizing various types of algorithms. However, I have yet to find an article or book that discusses ant colony optimizations in an introductory manner, or even in a lot of detail. Can anyone point me at some resources where I can learn more about this idea?
On the off chance that you know German (yes, sorry …), a friend and I have written an introduction with code about this subject which I myself find quite passable. The text and code uses the example of TSP to introduce the concept.
Even if you don't know German, take a look at the code and the formulas in the text, this might still serve.
link Wikipedia actually got me started. I read the article and got to coding. I was solving a wicked variation of the traveling salesman problem. It's an amazing meta-heuristic. Basically, any type of search problem that can be put into a graph (nodes & edges, symmetric or not) can be solved with an ACO.
Look out for the difference between global and local pheromone trails. Local pheromones discourage one generation of ants from traversing the same path. They keep the model from converging. Global pheromones are attractors and should snag at least one ant per generation. They encourage optimum paths over several generations.
The best suggestion I have, is simply to play with the algorithm. Setup a basic TSP solver and some basic colony visualization. Then have some fun. Working with ants, conceptually, is way cool. You program their basic behaviors and then set them loose. I even grow fond of them. :)
ACOs are a greedier form of genetic algorithms. Play with them. Alter their communicative behaviors and pack behavior. You'll rapidly begin to see network / graph programming in an entirely different way. That's their biggest benefit, not the recipe that most people see it as.
You just gotta play with it to really understand it. Books & research papers only give a general sky-high understanding. Like a bike, you just gotta start riding. :)
ACOs, by far, are my favorite abstraction for graph problems.
National Geographic wrote an interesting article awhile back talking about some of the theories.
The best resource for these topics is Google scholar. Ive been working on Ant Colony Optimization algorithms for a while, here are some good papers:
Ant Colony Optimization - A New Metaheuristic
Ant Colony Optimization - Artificial Ants as a Computational Intelligence Technique
Just search for "Ant Colony" on google scholar.
Also, search for papers published by Marco Dorigo.
I am surprised nobody has mentioned the bible of ACO:
Marco Dorigo & Thomas Stützle: Ant Colony Optimization
This book is written by the author of ACO and it is highly readable. You can take it to the beach and have fun reading it. But it is also the most complete resource of all, great as a reference when implementing the thing.
You can read some excerpts on Google Books
Another great source of wisdom is the ACO Homepage
See for example this article on scholarpedia.
There is also discussion here in the What is the most efficient way of finding a path through a small world graph? question.
At first glance this seems to be closely related to (or prehaps a special case of) the Metropolis algorithm. So that's another possible direction for searching.
Addition: This PDF file includes a reference to the original Metropolis paper from 1953.
Well, i found the Homepage of Eric Rollins and his different Implementations (Haskell, Scala, Erlang,...) of a ACO Algorithm helpfull.
And also the Book from Enrique Alba, titled "Parallel Metaheuristics: A New Class of Algorithms" where you can find a whole chapter of explanation about ACO Algorithms and their different usages.
Hth