game theory algorithms: how to proceed - algorithm

Take an example: Permutation Game(interviewstreet.com). I would like to know how do I proceed in such questions.
P.S.: Please don't post the full algorithm(as that would spoil the fun), just a few pointers.

I would setup a small game with a small N and a random permutation and then draw a complete Alpha-Beta Tree...
http://en.wikipedia.org/wiki/Alpha-beta_pruning
of all possible moves, and then work bottom-up making the optimal choice for each player at each point.
Then generalize from there once you see the pattern.
In game theory terminology you need to use Backward Induction to find the Subgame Perfect Equilibrium.

N is rather small. In each turn, there are two possibilities for each number: remove that number or don't remove that number. Try both possibilities, resulting in an O(N*2^N) algorithm for each test case. In practice this will be lower since the game usually ends before all numbers are removed and you can cut the search short quite often.
So you want a backtracking function that tries all possibilities of removing the numbers and returns 1 if Alice wins and 2 if Bob wins. At depth k (first depth is k = 0), if k % 2 = 0, then it's Alice's turn. She wins if all the immediate recursive calls (that have depth k + 1) return 1. If at least one of them returns 2, then she loses, because there is at least one way for Bob to win.
Example run on 1 3 2:
k = 0 - Alice removes 1:
k = 1 - Bob removes 3 => Bob wins because only 2 remains
- Bob removes 2 => Bob wins because only 3 remains (note: this step
is not needed, as Bob already proved he can win at this k = 1)
- Alice removes 3 => Alice wins because 1 2 is increasing
- Alice removes 2 => Alice wins because 1 3 is increasing
So Alice definitely has a winning strategy (when both play optimally) if she removes 3 or 2 in the first move, because the recursive branches of these two never give Bob as the winner.
Example run on 5 3 2 1 4 (partial run):
k = 0 - Alice removes 5
k = 1 - Bob removes 3
k = 2 - Alice removes 2 => 1 4 => Alice wins
k = 1 - Bob removes 2
k = 2 - Alice removes 3 => 1 4 => Alice wins
k = 1 - Bob removes 1
k = 2 - Alice removes 3 => 2 4 => Alice wins
k = 1 - Bob removes 4
k = 2 - Alice removes 3
k = 3 - Whatever Bob removes, he wins
k = 2 - Alice removes 2
k = 3 - Whatever Bob removes, he wins
k = 2 - Alice removes 1
k = 3 - Whatever Bob removes, he wins
...
As you can see, there is at least one way for Bob to end up winning if Alice starts by removing 5. If you do the same for her other possibilities, you will probably get the same result. Thus, it's Bob who will definitely win if he plays optimally.

Pen and Paper
Use pen and paper, make sure you understand the rules of the game exactly, then think of all possible strategies for the game.
For a relatively simple problem like this, think backwards from the point when game is won, unrolling one step at a time and making sure the player who made that step could not have made a better step, that's the optimality requirement.
For more complicated problems, read up Wikipedia on game theory.

Related

Particular Nim game variant (2 or 3 sticks)

You start with a stack of n sticks. Each player removes 2 or 3 sticks each turn. The player who removes the last stick wins. If their is only one stick left, the game is a draw.
I need to determine who will win a game of n stacks in general, expressed as a function of n. However, unless we start the game with 2, 3 or 5 sticks, it is always possible to direct the game so that it ends in a draw. I drew the game tree for 9 sticks and more and it is always possible to prevent a loss by making choices that lead to one stick remaining. How can I write a winning rule for this given problem?
However, unless we start the game with 2, 3 or 5 sticks, it is always possible to direct the game so that it ends in a draw.
I don't think this is true.
For example, suppose we start with 10 sticks. If you remove x sticks, I will always remove 5-x sticks. This will mean that after one turn each there are 5 sticks left, and after two turns each I have won.
The same will apply for any multiple of 5.
Now consider other possible values modulo 5.
Here's the rule:
Losing Position: n = 5k
Draw: n = 5k+1 or n = 5k+4
Winning Position: n = 5k+2 or n = 5k+3
You can observe the pattern by building a table like the one shown below:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 .....
L D W W D L D W W D L D W W D L D
Steps for Building the table:
Observe that you lose if n = 0, mark it as L.
When n = 1, it is draw, mark it as D.
When n = 2, you can only draw 2 sticks. So, your opponent happens to face n=0 which is losing, so you win, mark it as W.
For n = 3, you can take 2 or 3 sticks, so, your opponent can end up at 1 or 0 sticks. 1 is a draw and 0 is loss for him and therefore win for us. So, we will chose win, mark it as W.
For n=4, similarily, opponent can end up at 2 or 1. So, we can draw or lose. We will choose draw, mark it as D.
For n = 5, we can either make our opponent end up at 2 or 3. Both of them are win for him. So, we lose. Mark it as L.
Basically, to determine the state(L, W or D) of a number n, we need to look at states of already computed states n-2 and n-3.
Observe that this pattern (LDWWD) repeats after that.

Combinatorial Game. Who wins if both players play optimally

Players A and B play a game optimally and move alternately. They
start with 1. Each player in his turn multiplies the current number
with any integer from [2,9]. If after a player's turn, the number is
more than or equal to n, he wins.
A starts. Given n, who wins?
For example,
Numbers 2,3..,9 are winning numbers (Player A will win)
Numbers 10,11,..,18 are losing numbers (Player A will lose)
Numbers 19,20,..,162 are winning numbers
What would be the winning strategy? How can the Sprague-Grundy theorem be applied to solve this?
According to Sprague-Grundy theorem each state of an impartial game can be assigned a non-negative integer called Grundy number, such that the player who moves in this state will lose iff this number is 0, and win iff this number is non-zero.
If the Grundy numbers for the states are known, then the winning strategy is to always make a move to a state in which Grundy number is 0.
The algorithm for computing Grundy number for some state of a general game is as follows:
if current player can't make a valid move:
Grundy number := 0 (this player has lost)
else:
for each move in this state:
for each sub-game the game splits into after that move:
compute Grundy number of the sub-game
compute XOR of Grundy numbers of the sub-games
Grundy number := MEX of those XORs
MEX is minimum excludant function. MEX of a set of non-negative integers is equal to the smallest non-negative integer, that does not belong to this set.
For example:
MEX(0) = 1
MEX(0, 1) = 2
MEX(0, 2) = 1
MEX(0, 1, 2) = 3
MEX(0, 1, 3) = 2
MEX(1, 2, 3) = 0
MEX(10, 100, 1000) = 0
Naive implementation of this algorithm for this game in Python 3 may look like this:
import functools
from itertools import count
def mex(s):
for i in count():
if i not in s:
return i
#functools.lru_cache(10000)
def sprague_grundy(n, cur=1):
if cur >= n:
return 0
move_results = {sprague_grundy(n, cur*move) for move in range(2, 9+1)}
return mex(move_results)
for i in count(1):
print(sprague_grundy(i))
Often the easiest way to understand the general formula for the Grundy number is to just look at the sequence and try to notice the relationships.
In this game you can figure out the general formula by simply looking at n numbers for games in which player A wins in inital state, without actually calculating Grundy numbers.
But we can still look at the counts of Grundy numbers of the initial state of the game for consecutive n (0 means player A loses in the initial state, 1,2,3,4 mean player A wins):
$ python3 sprague_grundy.py | uniq -c
1 0
1 1
2 2
4 3
1 4
9 0
18 1
36 2
72 3
18 4
162 0
324 1
648 2
1296 3
324 4
2916 0
It is possible to notice that for player A all the losing initial states are for
Or in other words the initial state for player A is losing iff
Basically you make an array A[] where A[i] stores whether number i is a winning position or losing with respect to the player who starts the game.Let it be player A. Basic rule, from a losing position you can go only to a winning one and a winning position is such that there is always a losing position reachable from it.Following code is explanatory ( 1 means winning w.r.t to A and 0 means losing).
for each i from 1 to 9
A[i]=1
for each i from 10 to n
flag=0
A[i]=0
for each j from 2 to 9
if i is divisible j and A[i/j] is 0
flag=1
if flag is 1
A[i]=1
Now if A[n] is 1 it is winning for him else he loses.
This is an O(n) solution both in time and memory.You can reduce memory, but
time I can't come up with a better solution. There might be a O(1) solution but I am unaware of it.

Permutation Game - 2nd input case - explanation

Permutation Game (30 Points)
Alice and Bob play the following game:
1) They choose a permutation of the first N numbers to begin with.
2) They play alternately and Alice plays first.
3) In a turn, they can remove any one remaining number from the permutation.
4) The game ends when the remaining numbers form an increasing sequence. The person who played the last turn (after which the sequence becomes increasing) wins the game.
Assuming both play optimally, who wins the game?
Input:
The first line contains the number of test cases T. T test cases follow. Each case contains an integer N on the first line, followed by a permutation of the integers 1..N on the second line.
Output:
Output T lines, one for each test case, containing "Alice" if Alice wins the game and "Bob" otherwise.
Constraints:
1 <= T <= 100
2 <= N <= 15
The permutation will not be an increasing sequence initially.
Sample Input:
2
3
1 3 2
5
5 3 2 1 4
Sample Output:
Alice
Bob
Explanation: For the first example, Alice can remove the 3 or the 2 to make the sequence increasing and wins the game.
Can someone please help me out on the second input case: 5 3 2 1 4
The increasing sequences possible are:
1) 3 4 - Removing 5 , 2 , 1 in any sequence
2) 2 4 - Removing 5 , 3 , 1 in any sequence
3) 1 4 - Removing 5 , 3 , 2 in any sequence
So the output should be Alice?
Please do not share any code. Thanks
If Alice removes any of 5,3,2,1 then Bob removes 4. So, the increasing sequence can be of only one element, elements can be removed in any order. Hence, Bob wins.
If Alice removes 4, then also the increasing sequence has to be of one element. Bob wins.
So, Bob wins.
A possible case might be 4 or 5 is considered as increasing seq
As the input parameters are n>=2
But Alice would play optimally and remove 5 to win
NOTE: This isn't a programming problem and really doesn't belong on this site...
It sure looks like Alice should be the winner of the second test case.
Flow:
// Start state
5 3 2 1 4
// Alice remove 5
3 2 1 4
// Bob remove 3, 2, or 1
(2 1 4) or (3 1 4) or (3 2 4)
// Alice remove first number remaining
(1 4) or (2 4)
// Alice won!

How can I maximally partition a set?

I'm trying to solve one of the Project Euler problems. As a consequence, I need an algorithm that will help me find all possible partitions of a set, in any order.
For instance, given the set 2 3 3 5:
2 | 3 3 5
2 | 3 | 3 5
2 | 3 3 | 5
2 | 3 | 3 | 5
2 5 | 3 3
and so on. Pretty much every possible combination of the members of the set. I've searched the net of course, but haven't found much that's directly useful to me, since I speak programmer-ese not advanced-math-ese.
Can anyone help me out with this? I can read pretty much any programming language, from BASIC to Haskell, so post in whatever language you wish.
Have you considered a search tree? Each node would represent a choice of where to put an element and the leaf nodes are answers. I won't give you code because that's part of the fun of Project Euler ;)
Take a look at:
The Art of Computer Programming, Volume 4, Fascicle 3: Generating All Combinations and Partitions
7.2.1.5. Generating all set partitions
In general I would look at the structure of the recursion used to compute the number of configurations, and build a similar recursion for enumerating them. Best is to compute a one-to-one mapping between integers and configurations. This works well for permutations, combinations, etc. and ensures that each configuration is enumerated only once.
Now even the recursion for the number of partitions of some identical items is rather complicated.
For partitions of multisets the counting amounts to solving the generalization of Project Euler problem 181 to arbitrary multisets.
Well, the problem has two aspects.
Firsty, the items can be arranged in any order. So for N items, there are N! permutations (assuming the items are treated as unique).
Secondly, you can envision the grouping as a bit flag between each item indicating a divide. There would be N-1 of these flags, so for a given permutation there would be 2^(N-1) possible groupings.
This means that for N items, there would be a total of N!*(2^(N-1)) groupings/permutations, which gets big very very fast.
In your example, the top four items are groupings of one permutation. The last item is a grouping of another permutation. Your items can be viewed as :
2 on 3 off 3 off 5
2 on 3 on 3 off 5
2 on 3 off 3 on 5
2 on 3 on 3 on 5
2 off 5 on 3 off 3
The permutations (the order of display) can be derived by looking at them like a tree, as mentioned by the other two. This would almost certainly involve recursion, such as here.
The grouping is independent of them in many ways. Once you have all the permutations, you can link them with the groupings if needed.
Here is the code you need for this part of your problem:
def memoize(f):
memo={}
def helper(x):
if x not in memo:
memo[x]=f(x)
return memo[x]
return helper
#memoize
def A000041(n):
if n == 0: return 1
S = 0
J = n-1
k = 2
while 0 <= J:
T = A000041(J)
S = S+T if k//2%2!=0 else S-T
J -= k if k%2!=0 else k//2
k += 1
return S
print A000041(100) #the 100's number in this series, as an example
I quickly whipped up some code to do this. However, I left out separating every possible combination of the given list, because I wasn't sure it was actually needed, but it should be easy to add, if necessary.
Anyway, the code runs quite well for small amounts, but, as CodeByMoonlight already mentioned, the amount of possibilities gets really high really fast, so the runtime increases accordingly.
Anyway, here's the python code:
import time
def separate(toseparate):
"Find every possible way to separate a given list."
#The list of every possibility
possibilities = []
n = len(toseparate)
#We can distribute n-1 separations in the given list, so iterate from 0 to n
for i in xrange(n):
#Create a copy of the list to avoid modifying the already existing list
copy = list(toseparate)
#A boolean list indicating where a separator is put. 'True' indicates a separator
#and 'False', of course, no separator.
#The list will contain i separators, the rest is filled with 'False'
separators = [True]*i + [False]*(n-i-1)
for j in xrange(len(separators)):
#We insert the separators into our given list. The separators have to
#be between two elements. The index between two elements is always
#2*[index of the left element]+1.
copy.insert(2*j+1, separators[j])
#The first possibility is, of course, the one we just created
possibilities.append(list(copy))
#The following is a modification of the QuickPerm algorithm, which finds
#all possible permutations of a given list. It was modified to only permutate
#the spaces between two elements, so it finds every possibility to insert n
#separators in the given list.
m = len(separators)
hi, lo = 1, 0
p = [0]*m
while hi < m:
if p[hi] < hi:
lo = (hi%2)*p[hi]
copy[2*lo+1], copy[2*hi+1] = copy[2*hi+1], copy[2*lo+1]
#Since the items are non-unique, some possibilities will show up more than once, so we
#avoid this by checking first.
if not copy in possibilities:
possibilities.append(list(copy))
p[hi] += 1
hi = 1
else:
p[hi] = 0
hi += 1
return possibilities
t1 = time.time()
separations = separate([2, 3, 3, 5])
print time.time()-t1
sepmap = {True:"|", False:""}
for a in separations:
for b in a:
if sepmap.has_key(b):
print sepmap[b],
else:
print b,
print "\n",
It's based on the QuickPerm algorithm, which you can read more about here: QuickPerm
Basically, my code generates a list containing n separations, inserts them into the given list and then finds all possible permutations of the separations in the list.
So, if we use your example we would get:
2 3 3 5
2 | 3 3 5
2 3 | 3 5
2 3 3 | 5
2 | 3 | 3 5
2 3 | 3 | 5
2 | 3 3 | 5
2 | 3 | 3 | 5
In 0.000154972076416 seconds.
However, I read through the problem description of the problem you are doing and I see how you are trying to solve this, but seeing how quickly the runtime increases I don't think that it would work as fast you would expect. Remember that Project Euler's problems should solve in around a minute.

Why does this simple shuffle algorithm produce biased results?

It seems that this simple shuffle algorithm will produce biased results:
# suppose $arr is filled with 1 to 52
for ($i < 0; $i < 52; $i++) {
$j = rand(0, 51);
# swap the items
$tmp = $arr[j];
$arr[j] = $arr[i];
$arr[i] = $tmp;
}
You can try it... instead of using 52, use 3 (suppose only 3 cards are used), and run it 10,000 times and tally up the results, you will see that the results are skewed towards certain patterns...
The question is... what is a simple explanation for why it will happen?
The correct solution is to use something like
for ($i < 0; $i < 51; $i++) { # last card need not swap
$j = rand($i, 51); # don't touch the cards that already "settled"
# swap the items
$tmp = $arr[j];
$arr[j] = $arr[i];
$arr[i] = $tmp;
}
But the question is... why does the first method, seemingly also totally random, make the results biased?
Update 1: thanks for folks here pointing out that it needs to be rand($i, 51) for it to shuffle correctly.
See this:
The Danger of Naïveté (Coding Horror)
Let's look at your three card deck as an example. Using a 3 card deck, there are only 6 possible orders for the deck after a shuffle: 123, 132, 213, 231, 312, 321.
With your 1st algorithm there are 27 possible paths (outcomes) for the code, depending on the results of the rand() function at different points. Each of these outcomes are equally likely (unbiased). Each of these outcomes will map to the same single result from the list of 6 possible "real" shuffle results above. We now have 27 items and 6 buckets to put them in. Since 27 is not evenly divisible by 6, some of those 6 combinations must be over-represented.
With the 2nd algorithm there are 6 possible outcomes that map exactly to the 6 possible "real" shuffle results, and they should all be represented equally over time.
This is important because the buckets that are over-represented in the first algorithm are not random. The buckets selected for the bias are repeatable and predictable. So if you're building an online poker game and use the 1st algorithm a hacker could figure out you used the naive sort and from that work out that certain deck arrangements are much more likely to occur than others. Then they can place bets accordingly. They'll lose some, but they'll win much more than they lose and quickly put you out of business.
Here's the complete probability tree for these replacements.
Let's assume that you start with the sequence 123, and then we'll enumerate all the various ways to produce random results with the code in question.
123
+- 123 - swap 1 and 1 (these are positions,
| +- 213 - swap 2 and 1 not numbers)
| | +- 312 - swap 3 and 1
| | +- 231 - swap 3 and 2
| | +- 213 - swap 3 and 3
| +- 123 - swap 2 and 2
| | +- 321 - swap 3 and 1
| | +- 132 - swap 3 and 2
| | +- 123 - swap 3 and 3
| +- 132 - swap 2 and 3
| +- 231 - swap 3 and 1
| +- 123 - swap 3 and 2
| +- 132 - swap 3 and 3
+- 213 - swap 1 and 2
| +- 123 - swap 2 and 1
| | +- 321 - swap 3 and 1
| | +- 132 - swap 3 and 2
| | +- 123 - swap 3 and 3
| +- 213 - swap 2 and 2
| | +- 312 - swap 3 and 1
| | +- 231 - swap 3 and 2
| | +- 213 - swap 3 and 3
| +- 231 - swap 2 and 3
| +- 132 - swap 3 and 1
| +- 213 - swap 3 and 2
| +- 231 - swap 3 and 3
+- 321 - swap 1 and 3
+- 231 - swap 2 and 1
| +- 132 - swap 3 and 1
| +- 213 - swap 3 and 2
| +- 231 - swap 3 and 3
+- 321 - swap 2 and 2
| +- 123 - swap 3 and 1
| +- 312 - swap 3 and 2
| +- 321 - swap 3 and 3
+- 312 - swap 2 and 3
+- 213 - swap 3 and 1
+- 321 - swap 3 and 2
+- 312 - swap 3 and 3
Now, the fourth column of numbers, the one before the swap information, contains the final outcome, with 27 possible outcomes.
Let's count how many times each pattern occurs:
123 - 4 times
132 - 5 times
213 - 5 times
231 - 5 times
312 - 4 times
321 - 4 times
=============
27 times total
If you run the code that swaps at random for an infinite number of times, the patterns 132, 213 and 231 will occur more often than the patterns 123, 312, and 321, simply because the way the code swaps makes that more likely to occur.
Now, of course, you can say that if you run the code 30 times (27 + 3), you could end up with all the patterns occuring 5 times, but when dealing with statistics you have to look at the long term trend.
Here's C# code that explores the randomness for one of each possible pattern:
class Program
{
static void Main(string[] args)
{
Dictionary<String, Int32> occurances = new Dictionary<String, Int32>
{
{ "123", 0 },
{ "132", 0 },
{ "213", 0 },
{ "231", 0 },
{ "312", 0 },
{ "321", 0 }
};
Char[] digits = new[] { '1', '2', '3' };
Func<Char[], Int32, Int32, Char[]> swap = delegate(Char[] input, Int32 pos1, Int32 pos2)
{
Char[] result = new Char[] { input[0], input[1], input[2] };
Char temp = result[pos1];
result[pos1] = result[pos2];
result[pos2] = temp;
return result;
};
for (Int32 index1 = 0; index1 < 3; index1++)
{
Char[] level1 = swap(digits, 0, index1);
for (Int32 index2 = 0; index2 < 3; index2++)
{
Char[] level2 = swap(level1, 1, index2);
for (Int32 index3 = 0; index3 < 3; index3++)
{
Char[] level3 = swap(level2, 2, index3);
String output = new String(level3);
occurances[output]++;
}
}
}
foreach (var kvp in occurances)
{
Console.Out.WriteLine(kvp.Key + ": " + kvp.Value);
}
}
}
This outputs:
123: 4
132: 5
213: 5
231: 5
312: 4
321: 4
So while this answer does in fact count, it's not a purely mathematical answer, you just have to evaluate all possible ways the random function can go, and look at the final outputs.
From your comments on the other answers, it seems that you are looking not just for an explanation of why the distribution is not the uniform distribution (for which the divisibility answer is a simple one) but also an "intuitive" explanation of why it is actually far from uniform.
Here's one way of looking at it. Suppose you start with the initial array [1, 2, ..., n] (where n might be 3, or 52, or whatever) and apply one of the two algorithms. If all permutations are uniformly likely, then the probability that 1 remains in the first position should be 1/n. And indeed, in the second (correct) algorithm, it is 1/n, as 1 stays in its place if and only if it is not swapped the first time, i.e. iff the initial call to rand(0,n-1) returns 0.
However, in the first (wrong) algorithm, 1 remains untouched only if it is neither swapped the first time nor any other time — i.e., only if the first rand returns 0 and none of the other rands returns 0, the probability of which is (1/n) * (1-1/n)^(n-1) ≈ 1/(ne) ≈ 0.37/n, not 1/n.
And that's the "intuitive" explanation: in your first algorithm, earlier items are much more likely to be swapped out of place than later items, so the permutations you get are skewed towards patterns in which the early items are not in their original places.
(It's a bit more subtle than that, e.g. 1 can get swapped into a later position and still end up getting swapped back into place through a complicated series of swaps, but those probabilities are relatively less significant.)
The best explanation I've seen for this effect was from Jeff Atwood on his CodingHorror blog (The Danger of Naïveté).
Using this code to simulate a 3-card random shuffle...
for (int i = 0; i < cards.Length; i++)
{
int n = rand.Next(cards.Length);
Swap(ref cards[i], ref cards[n]);
}
...you get this distribution.
(source: typepad.com)
The shuffle code (above) results in 3^3 (27) possible deck combinations. But the mathematics tell us that there are really only 3! or 6 possible combinations of a 3 card deck. So some of the combinations are over-represented.
You would need to use a Fisher-Yates shuffle to properly (randomly) shuffle a deck of cards.
Here's another intuition: the single shuffle swap can't create symmetry in the probability of occupying a position unless at least 2-way symmetry already exists. Call the three positions A, B, and C. Now let a be the probability of card 2 being in position A, b be the probability of card 2 being in position B, and c be the probability of it being in position C, prior to a swap move. Assume that no two probabilities are the same: a!=b, b!=c, c!=a. Now compute the probabilities a', b', and c' of the card being in these three positions following a swap. Let's say that this swap move consists of position C being swapped with one of the three positions at random. Then:
a' = a*2/3 + c*1/3
b' = b*2/3 + c*1/3
c' = 1/3.
That is, the probability that the card winds up in position A is the probability it was already there times the 2/3 of the time position A isn't involved in the swap, plus the probability that it was in position C times the 1/3 probability that C swapped with A, etc. Now subtracting the first two equations, we get:
a' - b' = (a - b)*2/3
which means that because we assumed a!=b, then a'!=b' (though the difference will approach 0 over time, given enough swaps). But since a'+b'+c'=1, if a'!=b', then neither can be equal to c' either, which is 1/3. So if the three probabilities start off all different before a swap, they will also all be different after a swap. And this would hold no matter which position was swapped - we just interchange the roles of the variables in the above.
Now the very first swap started by swapping card 1 in position A with one of the others. In this case, there was two way symmetry before the swap, because the probability of card 1 in position B = probability of card 1 in position C = 0. So in fact, card 1 can wind up with symmetric probabilities and it does end up in each of the three positions with equal probability. This remains true for all subsequent swaps. But card 2 winds up in the three positions after the first swap with probability (1/3, 2/3, 0), and likewise card 3 winds up in the three positions with probability (1/3, 0, 2/3). So no matter how many subsequent swaps we do, we will never wind up with card 2 or 3 having exactly the same probability of occupying all three positions.
See the Coding Horror post The Danger of Naïveté.
Basically (suposing 3 cards):
The naive shuffle results in 33 (27)
possible deck combinations. That's
odd, because the mathematics tell us
that there are really only 3! or 6
possible combinations of a 3 card
deck. In the KFY shuffle, we start
with an initial order, swap from the
third position with any of the three
cards, then swap again from the second
position with the remaining two cards.
The simple answer is that there are 52^52 possible ways for this algorithm to run, but there are only 52! possible arrangements of 52 cards. For the algorithm to be fair, it needs to produce each of these arrangements equally likely. 52^52 is not an integer multiple of 52!. Therefore, some arrangements must be more likely than others.
an illustrative approach might be this:
1) consider only 3 cards.
2) for the algorithm to give evenly distributed results, the chance of "1" ending up as a[0] must be 1/3, and the chance of "2" ending up in a[1] must be 1/3 too, and so forth.
3) so if we look at the second algorithm:
probability that "1" ends up at a[0]:
when 0 is the random number generated,
so 1 case out of (0,1,2), therefore,
is 1 out of 3 = 1/3
probability that "2" ends up at a[1]:
when it didn't get swapped to a[0] the
first time, and it didn't get swapped
to a[2] the second time: 2/3 * 1/2 =
1/3
probability that "3" ends up at a[2]:
when it didn't get swapped to a[0] the
first time, and it didn't get swapped
to a[1] the second time: 2/3 * 1/2 =
1/3
they are all perfectly 1/3, and we
don't see any error here.
4) if we try to calculate the probability of of "1" ending up as a[0] in the first algorithm, the calculation will be a bit long, but as the illustration in lassevk's answer shows, it is 9/27 = 1/3, but "2" ending up as a[1] has a chance of 8/27, and "3" ending up as a[2] has a chance of 9/27 = 1/3.
as a result, "2" ending up as a[1] is not 1/3 and therefore the algorithm will produce pretty skewed result (about 3.7% error, as opposed to any negligible case such as 3/10000000000000 = 0.00000000003%)
5) the proof that Joel Coehoorn has, actually can prove that some cases will be over-represented. I think the explanation that why it is n^n is this: at each iteration, there are n possibility that the random number can be, so after n iterations, there can be n^n cases = 27. This number doesn't divid the number of permuations (n! = 3! = 6) evenly in the case of n = 3, so some results are over-represented. they are over-represented in a way that instead of showing up 4 times, it shows up 5 times, so if you shuffle the cards millions of times from the initial order of 1 to 52, the over-represented case will show up 5 million times as opposed to 4 million times, which is quite big a difference.
6) i think the over-representation is shown, but "why" will the over-representation happen?
7) an ultimate test for the algorithm to be correct is that any number has a 1/n probability to end up at any slot.
The Naive algorithm picks the values of n like so:
n = rand(3)
n = rand(3)
n = rand(3)
3^3 possible combinations of n
1,1,1, 1,1,2....3,3,2 3,3,3 (27 combinations) lassevk's answer shows the distribution among the cards of these combinations.
the better algorithm does:
n = rand(3)
n = rand(2)
n! possible combinations of n
1,1, 1,2, 2,1 2,2 3,1 3,2 (6 combinations, all of them giving a different result)
As mentioned in the other answers, if you take 27 attempts to get 6 results, you cannot possibly attain the 6 results with even distribution, since 27 is not divisible by 6. Put 27 marbles into 6 buckets and no matter what you do, some buckets will have more marbles than others, the best you can do is 4,4,4,5,5,5 marbles for buckets 1 through 6.
the fundamental problem with the naive shuffle is that swaps too many times, to shuffle 3 cards completely, you need only do 2 swaps, and the second swap need only be among the first two cards, since the 3rd card already had a 1/3 chance of being swapped. to continue to swap cards will impart more chances that a given card will be swapped, and these chances will only even out to 1/3, 1/3, 1/3 if your total swap combinations is divisible by 6.
Not that another answer is needed, but I found it worthwhile to try to work out exactly why Fisher-Yates is uniform.
If we are talking about a deck with N items, then this question is: how can we show that
Pr(Item i ends up in slot j) = 1/N?
Breaking it down with conditional probabilities, Pr(item i ends up at slot j) is equal to
Pr(item i ends up at slot j | item i was not chosen in the first j-1 draws)
* Pr(item i was not chosen in the first j-1 draws).
and from there it expands recursively back to the first draw.
Now, the probability that element i was not drawn on the first draw is N-1 / N. And the probability that it was not drawn on the second draw conditional on the fact that it was not drawn on the first draw is N-2 / N-1 and so on.
So, we get for the probability that element i was not drawn in the first j-1 draws:
(N-1 / N) * (N-2 / N-1) * ... * (N-j / N-j+1)
and of course we know that the probability that it is drawn at round j conditional on not having been drawn earlier is just 1 / N-j.
Notice that in the first term, the numerators all cancel the subsequent denominators (i.e. N-1 cancels, N-2 cancels, all the way to N-j+1 cancels, leaving just N-j / N).
So the overall probability of element i appearing in slot j is:
[(N-1 / N) * (N-2 / N-1) * ... * (N-j / N-j+1)] * (1 / N-j)
= 1/N
as expected.
To get more general about the "simple shuffle", the particular property that it is lacking is called exchangeability. Because of the "path dependence" of the way the shuffle is created (i.e. which of the 27 paths is followed to create the output), you are not able to treat the different component-wise random variables as though they can appear in any order. In fact, this is perhaps the motivating example for why exchangeability matters in random sampling.
The clearest answer to show the first algorithm fails is to view the algorithm in question as a Markov chain of n steps on the graph of n! vertices of all the permutation of n natural numbers. The algorithm hops from one vertex to another with a transition probability. The first algorithm gives the transition probability of 1/n for each hop. There are n^n paths the probability of each of which is 1/n^n. Suppose the final probability of landing on each vertex is 1/n! which is a reduced fraction. To achieve that there must be m paths with the same final vertex such that m/n^n=1/n! or n^n = mn! for some natural number m, or that n^n is divisible by n!. But that is impossible. Otherwise, n has to be divisible by n-1 which is only possible when n=2. We have contradiction.

Resources