I implemented a minimax algorithm for TTT. When I make the AI player make the first move, it evaluates all minimax values of the possible moves as 0. That means it can choose any square on the grid as the first move. However, any Tic Tac Toe guide will tell you that choosing a corner or center square when making the first move is the better choice since there is a higher chance of winning.
Why is my algorithm not reflecting this?
EDIT: To clarify, what I'm trying to ask is: is this a limitation of the minimax algorithm or is my implementation incorrect?
Your algorithm does not have to reflect this: if you try out all starting positions, you would find that the corners and the center give you more path to win than other cells.
With tic-tac-toe's lack of complexity, the minimax can look ahead all the way through the end of the game, starting with the very first move. The number of available moves goes down quickly as you progress through the game, so the full search finishes pretty fast.
With more complex games (othello, checkers, chess), the so-called "opening books" become more important. The number of available moves in the beginning of the game is enormous, so a traditional approach is to pick a move from the "book" of openings, and stay with pre-calculated "book moves" for the first three to six plays. Moves outside the book are not considered, saving a lot of CPU on moves that remain essentially the same.
Related
I am developing a software for a board game (2 player) which has 10x4 cells and both the players have 9 pieces each. Initially, the pieces for player 1 will be at the top of the board and for player 2 all the pieces will be at the bottom of the board (similar to chess but a lot less complex!).
I have used MiniMax algorithm to calculate the next best move. Now, the algorithm itself seems to work fine. The problem that I am facing is in Heuristic value calculation.
If there is no best move found in the depth till I am searching (4 currently), my code simply takes the first step which it finds from the move list. That is probably because the score of all the moves are same till the depth of 4!
So, it just keeps stalling. Eg., it will move piece 1 from position A to B in 1st turn and in the 2nd turn it will move the same piece from position B to A.
It keeps on doing that until the opponent moves closer to my pieces.
Now, what I want would love to know is how do I make sure that if the opponent is NOT closing in, I do that instead of stalling.
Currently, I am calculating the heuristic values based on the difference between my pieces and opp pieces.
How to calculate the values such that those moves which lead to position closer to the opponent's pieces are selected? Appreciate your help! Thanks!
I'm working on a simple tic-tac-toe problem, I'm and trying to understand how the Minimax algorithm works.
If I use the utility function 1 for X win, -1 for O win, and 0 for game in progress, then I don't understand how the algorithm prioritizes shorter solutions.
As I understand it goes to the deepest node first, and event if it's not the shortest path, but it leads to a possible win, then it'll pick it.
Let me explain in the example. Here's the state of the board and X turn (the notation is from https://www.hackerrank.com/challenges/tic-tac-toe):
OX_
_X_
__O
If we search from top left position to the right down, then the algorithm will find that if we put X in the position (0, 2) cause it leads to inevitable win next turn:
OXX
_X_
__O
However, a smarter choice would be (2, 1) and the immediate winning position:
OX_
_X_
_XO
I don't see how Minimax or Alpha-Beta pruning would prioritize such behavior.
So my question is if I understand it correctly, and how I could improve it.
Minimax for Tic Tac Toe has few possible numerical valuations: win, lose, draw. The Minimax algorithm assigns the maximum of all the ways the player's position could be minimized by the other player's subsequent moves. A win in the next move would be assigned Infinity, the clear choice. Otherwise, in Tic Tac Toe, it will prioritize a draw except for a few special cases, such as winning in the next move or being able to create a fork (leading to a certain win in the move after next).
Alpha Beta pruning is an optimization method that avoids exploring sections of the search tree when it can be shown that any part of the section would yield a worse result than one that has already been found; for example, say one full node exploration yielded a potential draw and another node's leaf showed a loss; there would be no sense in exploring other children of the latter node (unless you assumed the other player would not always play their best move). You might find this link helpful: https://www.ntu.edu.sg/home/ehchua/programming/java/JavaGame_TicTacToe_AI.html
I am trying to implement the Monte Carlo search tree method on a rather complicated game, but there is something I don't understand about it (maybe this applies to other search algorithms, I'm not sure). Sorry if my English isn't clear enough.
My question is: what do the tree nodes represent - possible game states or possible moves the player can make? I'll try to explain why both make no sense to me.
If the nodes are game states, why does choosing the move which may lead to the best one make sense? It is very possible that my opponent will choose a different play that will lead to a very bad game state for me.
If the nodes are possible moves, the same logic applies, just one level down - there is no point in choosing the best child of each move, because there is no grantee that my opponent will play in a way that lets me play that move. Instead, it makes sense to look at children more randomly, but that just contradicts the way this method is supposed to work.
So I guess the common implementation is some sort of mix between the two? I did do some research, and I know that sometimes search algorithms use a deterministic opponent which has a simple strategy, and use it for the search. This seems like it wouldn't work for me, because there is no intuitive strategy for the game I am implementing (this is why I chose the monte carlo method - no evaluation function needed). It feels like there is something really simple I'm missing.
I hope my question is clear enough, and I'll be grateful for any help.
Nodes in the game tree represent states - the edges are the moves that lead from one state to another.
In traditional MiniMax game search, which is what I am familiar with, the best move from a particular position is the move that denies the opponent her opportunity to play the best potential move at the next level, and so on, until the search depth is exhausted.
As an example, in a two-ply search (one full move look-ahead), if, when the initial players plays move X1, player two has the potential response moves Y1, Y2, and Y3 with respective scores (from her perspective) of 1, 2, and 3, then X1's score from the first player's perspective becomes -3. If player 1's X2 move comes back with a score of, say, -4 then X1 is the better initial move (it denies player two the opportunity to achieve the score of 4), but if the X2 derived score is -2 then X1 remains the best move at the top of the tree.
The way I understand it, Monte Carlo tree search means limiting the number of game nodes that are examined to some random sub-set. With Monte Carlo, you don't have to go all the way to the terminating node (for games where one can always be reached), though you can, but it can be used with an evaluation function as well, if desired.
I am new to game programming I currently am using unity's 2D power to remake an 2D game for sake of practice. So I am just little confuse about one thing, so far every enemy in my game was dumb(they move on predefined paths) but this enemy, alien-copter, flies and follow the player wherever the player goes. My question is should I implement any pathfinding algorithm? I had studied A* but I trying to figure out if A* is gonna be helpful in my envoirnment because player will be moving and the enemy have to keep on looking for shortest path. I tried to write code for this AI my code was working perfect when there were no obstacles but my game has obstacles so I want something efficient so what do you people think should I implement A* or any other algorithm or not?
As regards AI, if you have obstacles in your game, you will need to implement pathfinding of some sort. Note, that just taking the shortest block (node) is not an option because such algorithm is not complete, i.e. the path may not be found even if there is one. Imagine a going after b:
(wall)(wall)(wall)
(free)(free)a (wall) b
(wall)(wall)(wall)
The shortest (best) path is to the left since up, down and right are blocked. On the next move however, the best move is to go right because it's closer. That's how a will get trapped in a loop. So yes you are right, you do need to get the path from a to b. It will typically be in the form of a list of nodes. Then you just select head (aka element at index 0) of the list.
A* is the perfect option in your case. It is complete, efficient and it will even give you the shortest path. Path recomputation shouldn't be an issue for a small game. Furthermore, if obstacles in your game grid are static for the duration of the game you can precompute paths and cache them. For instance, path from (x,y) to (x1,y1) is [(a,b), (c,d) ...]. So you can store it in some sort of map data structure where key is the two points - start, target and value - the path. This will of course depend on whether you want to classify an enemy as an obstacle for other enemies and other gameplay factors
This is a question I have been toying with for a week or so, proposed by a colleague:
Imagine a game played on a
36x36 grid. The goal of the game is to create four corners of a square of any size (eg., 2x2, 3x3, 4x4, and so on). The first player places a game-piece anywhere except the center four grid spaces. After the first move, players can place their game-piece anywhere on the grid. Game-pieces cannot be moved after they are placed. And that's it; The game is simple and fun.
I have been trying to come up with an algorithm to win, or at least do well at this game. Any suggestions?
This is a game of perfect information where players take turns, like chess, so the same approach used in chess engines applies here. Use a minimax (with alpha-beta pruning probably) algorithm to search the tree of valid moves. You can use some evaluation function to guide your search, favoring positions that have the most almost-completed squares.
Like FogleBird wrote a Minimax Algorithm would work best. The problem is how to evaluate the score of the current board. The game is pretty complex there are over a thousand fields to begin with. In a small game like tic tac toe you can compute all possible moves till the end of the search tree in minimax then you give 1 point to the winning player and a -1 to the losing and backtrack the tree to find your best move. In this game you need some kind of heuristic to calculate a score for the board after descending the three 10 moves.
I don't have much information about the game so I can only guess good heuristics:
Points because of completed squares (if you can get more then one square) this would be the easiest way because your heuristic is directly related to the game points
minus points because of completed squares of your enemy
number of possible squares
number of owned fields at the sides of the board
number of owned fields in small neighbourhoods
There are plenty of heuristics possible and most of the time you would need a mix of some of them.
Do you need to fill the square or just place it in corners?
For instance, is the following a win?
.......................
.X..X..................
.......................
.......................
.X..X..................
.......................
or the following?
.......................
.XXXX..................
.X..X..................
.X..X..................
.XXXX..................
.......................
or the following?
.......................
.XXXX..................
.XXXX..................
.XXXX..................
.XXXX..................
.......................
ok I'm reading junk into the game.. as its ambiguous..
I'm assuming that this game is similar to "dots and lines" where the move space is to connect 2 adjacent dots with a line. so the 2x2 grid would have 9 verticies, 4 1x1 win positions and 1 2x2 win position. With the game ending with a win for the person who completes a square, and a tie once both players have exhausted winable solutions.
since your working squares some of the logic is nice. you can calculate membership of any line to all possible boxes. so in the 2x2 example a radial would have membership in 2 1x1 boxes and a side would have a membership in one 1x1 and one 2x2. This membership becomes important.
at the begining of the game you generate all memberships for all line segments. make 2 copies.. (like playing battleship) the ENEMY COPY will be initilized to the number of turns he has left to finish every box.. so on the 36x36 there will be a 144 moves left to finish the large box
4 sets of 140moves to finish the 4 35x35 boxes
During his moves you decrement all of the affected boxes on the ENEMY COPY
During your move any move you make invalidates ALL boxes containing your move. you set these to negative 1,, or infinite or 2 billion... just something to know that these squares are impossible.
You now create a ANTI-ENEMY COPY which is the reverse of the enemy copy.. this contains how many moves to complete a given square.
For a given move.. first check for a win situation. if you can move and win. (antienemy with a square at one)
Then check if a block is needed. (enemy board with any square at one)
now add a minimax type function if your so inclined..
or use search to find the positions that destroy as many low finish squares as possible, or create a line that reduces multiple low squares on the anti-enemy board. This should work reasonably as it isnt trying to finish any specific square but a minimax where your both shooting for the low score is going to be a better scenario.