I am an undergrad student doing very basic research on the role genetic algorithms can/do play in video games. On Youtube, there are videos of people showing how they've used the algorithms to teach the computer player how to play.
http://www.youtube.com/watch?v=ofVKsxeYa6U&feature=related
I understand genetic algorithms to be a search algorithm that is best used when you know the general solution you wish to achieve but not exactly. Ex. In the TSP you know you want to find shortest possible route or in an exam scheduling problem, you want all the students to be able to take their exams with the least amount of "interrupts". In these, problems the algorithm's uses in clear. However I'm having trouble grasping the concept of "machine learning" with g.a.
When genetic algorithms are used to teach a computer how to play, how are they "learning"?
How have they learned to play to play the game?
What is the "optimization problem" they attempt to solve?
One use is Feature Selection.
Many times, especially in text problems - but not only - your feature space is huge, and many machine learning algorithms (KNN for example) are vulnerable for non-informative features, and get worse with large amount of features.
Using Feature selection algorithm, you can reduce the dimensions of your problem - but the question is - How to chose which features are redundant?
There are many ways to do it, but one of them is using Gentic Algorithm as a search function, and try to optimize the subset of the feature you want to keep.
This use is commonly used, and is even implemented in the open source ML library Weka in the AttributeSelection package as GeneticSearch
Related
The Min-Max search, as well as the more efficient Alpha-Beta search algorithm are well known and often used to implement an artificial intelligence (AI) player in games like tic-tac-toe, connect 4 and so on.
While AIs based on these search algorithms are basically unbeatable for humans if they can traverse the whole search tree, this gets infeasible when there are too many possiblities due to exponential growth (like in Go for example).
All those games mentioned so far are turn-based.
However, if we assume to have enough computational power, shouldn't it be possible to also apply these algorithms to real-time strategy (RTS) games? In theory, this should work by discretizing time into small enough frames and then simulating all possible actions at each time stamp.
Clearly, the search tree would quickly explode in size. However, I wonder if there exist any theoretical analyzes of such an approach for real time games? Or maybe even practical investigations which use a very reduced and simplified RTS?
Question: I am searching for references (if there exist any) on this topic.
This paper "Search in Real-Time Video Games", Cowling et al, 1998, asserts that A* was used widely in video game search.
There's Geisler's MS thesis 'An Empirical Study of Machine Learning Algorithms Applied to Model Player Behavior in a "First Person Shooter" Video Game' where he primarily uses ID3 and the boost algorithm to learn the behaviors of an expert Soldier of Fortune 2 FPS player and incorporate that in an agent playing the game.
There are several other similar papers online but it appears that most of them currently are using various machine learning algorithms to learn behaviors by observation and incorporate those into some kind of agent rather than primarily use optimized search.
"Learning Human Behavior from Observation for Gaming Applications", Moriarty and Gonzales, 2009, is an example for this.
I'd like to pose a few abstract questions about computer vision research. I haven't quite been able to answer these questions by searching the web and reading papers.
How does someone know whether a computer vision algorithm is correct?
How do we define "correct" in the context of computer vision?
Do formal proofs play a role in understanding the correctness of computer vision algorithms?
A bit of background: I'm about to start my PhD in Computer Science. I enjoy designing fast parallel algorithms and proving the correctness of these algorithms. I've also used OpenCV from some class projects, though I don't have much formal training in computer vision.
I've been approached by a potential thesis advisor who works on designing faster and more scalable algorithms for computer vision (e.g. fast image segmentation). I'm trying to understand the common practices in solving computer vision problems.
You just don't prove them.
Instead of a formal proof, which is often impossible to do, you can test your algorithm on a set of testcases and compare the output with previously known algorithms or correct answers (for example when you recognize the text, you can generate a set of images where you know what the text says).
In practice, computer vision is more like an empirical science: You gather data, think of simple hypotheses that could explain some aspect of your data, then test those hypotheses. You usually don't have a clear definition of "correct" for high-level CV tasks like face recognition, so you can't prove correctness.
Low-level algorithms are a different matter, though: You usually have a clear, mathematical definition of "correct" here. For example if you'd invent an algorithm that can calculate a median filter or a morphological operation more efficiently than known algorithms or that can be parallelized better, you would of course have to prove it's correctness, just like any other algorithm.
It's also common to have certain requirements to a computer vision algorithm that can be formalized: For example, you might want your algorithm to be invariant to rotation and translation - these are properties that can be proven formally. It's also sometimes possible to create mathematical models of signal and noise, and design a filter that has the best possible signal to noise-ratio (IIRC the Wiener filter or the Canny edge detector were designed that way).
Many image processing/computer vision algorithms have some kind of "repeat until convergence" loop (e.g. snakes or Navier-Stokes inpainting and other PDE-based methods). You would at least try to prove that the algorithm converges for any input.
This is my personal opinion, so take it for what it's worth.
You can't prove the correctness of most of the Computer Vision methods right now. I consider most of the current methods some kind of "recipe" where ingredients are thrown down until the "result" is good enough. Can you prove that a brownie cake is correct?
It is a bit similar in a way to how machine learning evolved. At first, people did neural networks, but it was just a big "soup" that happened to work more or less. It worked sometimes, didn't on other cases, and no one really knew why. Then statistical learning (through Vapnik among others) kicked in, with some real mathematical backup. You could prove that you had the unique hyperplane that minimized a particular loss function, PCA gives you the closest matrix of fixed rank to a given matrix (considering the Frobenius norm I believe), etc...
Now, there are still a few things that are "correct" in computer vision, but they are pretty limited. What comes to my mind is the wavelet : they are the sparsest representation in an orthogonal basis of function. (i.e : the most compressed way to represent an approximation of an image with minimal error)
Computer Vision algorithms are not like theorems which you can prove, they usually try to interpret the image data into the terms which are more understandable to us humans. Like face recognition, motion detection, video surveillance etc. The exact correctness is not calculable, like in the case of image compression algorithms where you can easily find the result by the size of the images.
The most common methods used to show the results in Computer Vision methods(especially classification problems) are the graphs of precision Vs recall, accuracy Vs false positives. These are measured on standard databases available on various sites. Usually the harsher you set the parameters for correct detection, the more false positives you generate. The typical practice is to choose the point from the graph according to your requirement of 'how many false positives are tolerable for the application'.
In industry, there is often a problem where you need to calculate the most efficient use of material, be it fabric, wood, metal etc. So the starting point is X amount of shapes of given dimensions, made out of polygons and/or curved lines, and target is another polygon of given dimensions.
I assume many of the current CAM suites implement this, but having no experience using them or of their internals, what kind of computational algorithm is used to find the most efficient use of space? Can someone point me to a book or other reference that discusses this topic?
After Andrew in his answer pointed me to the right direction and named the problem for me, I decided to dump my research results here in a separate answer.
This is indeed a packing problem, and to be more precise, it is a nesting problem. The problem is mathematically NP-hard, and thus the algorithms currently in use are heuristic approaches. There does not seem to be any solutions that would solve the problem in linear time, except for trivial problem sets. Solving complex problems takes from minutes to hours with current hardware, if you want to achieve a solution with good material utilization. There are tens of commercial software solutions that offer nesting of shapes, but I was not able to locate any open source solutions, so there are no real examples where one could see the algorithms actually implemented.
Excellent description of the nesting and strip nesting problem with historical solutions can be found in a paper written by Benny Kjær Nielsen of University of Copenhagen (Nielsen).
General approach seems to be to mix and use multiple known algorithms in order to find the best nesting solution. These algorithms include (Guided / Iterated) Local Search, Fast Neighborhood Search that is based on No-Fit Polygon, and Jostling Heuristics. I found a great paper on this subject with pictures of how the algorithms work. It also had benchmarks of the different software implementations so far. This paper was presented at the International Symposium on Scheduling 2006 by S. Umetani et al (Umetani).
A relatively new and possibly the best approach to date is based on Hybrid Genetic Algorithm (HGA), a hybrid consisting of simulated annealing and genetic algorithm that has been described by Wu Qingming et al of Wuhan University (Quanming). They have implemented this by using Visual Studio, SQL database and genetic algorithm optimization toolbox (GAOT) in MatLab.
You are referring to a well known computer science domain of packing, for which there are a variety of problems defined and research done, for both 2-dimnensional space as well as 3-dimensional space.
There is considerable material on the net available for the defined problems, but to find it you knid of have to know the name of the problem to search for.
Some packages might well adopt a heuristic appraoch (which I suspect they will) and some might go to the lengths of calculating all the possibilities to get the absolute right answer.
http://en.wikipedia.org/wiki/Packing_problem
I'm writing a scheduling program with a difficult programming problem. There are several events, each with multiple meeting times. I need to find an arrangement of meeting times such that each schedule contains any given event exactly once, using one of each event's multiple meeting times.
Obviously I could use brute force, but that's rarely the best solution. I'm guessing this is a relatively basic computer science problem, which I'll learn about once I am able to start taking computer science classes. In the meantime, I'd prefer any links where I could read up on this, or even just a name I could Google.
I think you should use genetic algorithm because:
It is best suited for large problem instances.
It yields reduced time complexity on the price of inaccurate answer(Not the ultimate best)
You can specify constraints & preferences easily by adjusting fitness punishments for not met ones.
You can specify time limit for program execution.
The quality of solution depends on how much time you intend to spend solving the program..
Genetic Algorithms Definition
Genetic Algorithms Tutorial
Class scheduling project with GA
There are several ways to do this
One approach is to do constraint programming. It is a special case of the dynamic programming suggested by feanor. It is helful to use a specialized library that can do the bounding and branching for you. (Google for "gecode" or "comet-online" to find libraries)
If you are mathematically inclined then you can also use integer programming to solve the problem. The basic idea here is to translate your problem in to a set of linear inequalities. (Google for "integer programming scheduling" to find many real life examples and google for "Abacus COIN-OR" for a useful library)
My guess is that constraint programming is the easiest approach, but integer programming is useful if you want to include real variables in you problem at some point.
Your problem description isn't entirely clear, but if all you're trying to do is find a schedule which has no overlapping events, then this is a straightforward bipartite matching problem.
You have two sets of nodes: events and times. Draw an edge from each event to each possible meeting time. You can then efficiently construct the matching (the largest possible set of edges between the nodes) using augmented paths. This works because you can always convert a bipartite graph into an equivalent flow graph.
An example of code that does this is BIM. Standard graphing libraries such as GOBLIN and NetworkX also have bipartite matching implementations.
This sounds like this could be a good candidate for a dynamic programming solution, specifically something similar to the interval scheduling problem.
There are some visuals here for the interval scheduling problem specifically, which may make the concept clearer. Here is a good tutorial on dynamic programming overall.
I'm looking for some papers on finding an infrastructure development strategy in games like Starcraft / Age of Empires. Basic facts characterising those games are:
continuous time (well - it could be split into 10s periods, or something like that)
many variables describing growth (many resources, buildings levels, etc.)
many variables influencing growth (technology upgrades, levels, etc.)
Most of what I could find is basically either:
tree search minimising time to get to a given condition (building/technology at level X)
tree search maximising value = each game variable*bias
genetic algorithms... obvious doing either of the above
Are there any better algorithms that can be tuned to look for a perfect solution of the early phase?
You might find some information on one or more of these books:
http://www.gamedev.net/columns/books/books.asp?CategoryID=7
I do not know of any specific algorithm but this does sound like a traveling salesman problem. It does look like you have your base rules so you are already on your way. If you know what end condition you want to reach then it shouldn't be to hard to build a heuristic algorithm for the above rules. Then you could just run a simulation of the build outs and then measure them against each other. Each time you do that you would have a better idea of how to get where you want. Check out this to learn about heuristic algorithms.
There is no "perfect solution" for the early phase (if your game is complex enough). If you've played these games online, you'll see players using various strategies and all of these working depending on the other player's strategy. Some try to attack very early, some are more defensive, some prefer developing economically rather than having lots of unprepared soldiers.
Given this, I believe you must try to figure out a good value function to be maximized.