Random indice of a boolean grid - algorithm

Question

Let's say I have a square boolean grid (2D array) of size N. Some of the values are true and some are false (the <true values> / <false values> ratio is unspecified). I want to randomly choose an indice (x, y) so that grid[x][y] is true. If I wanted a time-efficient solution, I'd do something like this (Python):
x, y = random.choice([(x, y) for x in range(N) for y in range(N) if grid[x][y]])
But this is O(N^2), which is more than sufficient for, say, a tic-tac-toe game implementation, but I'm guessing it would get much more memory-consuming for large N.
If I wanted something that's not memory consuming, I'd do:
x, y = 0, 0
t = N - 1
while True:
x = random.randint(0, t)
y = random.randint(0, t)
if grid[x][y]:
break
But the issue is, if I have a grid of size of order 10^4 and there is only one or two true values in it, it could take forever to "guess" which indice is the one I'm interested in. How should I go about making this algorithm optimal?

Answer 1

If the grid is static or doesn't change much, or you have time to do some preprocessing, you could store an array that holds the number of true values per row, the total number of true values, and a list of the non-zero rows (all of which you could keep updated if the grid changes):
grid per row
0 1 0 0 1 0 2
0 0 0 0 0 0 0
0 0 1 0 0 0 1
0 0 0 0 1 0 1
0 0 0 0 0 0 0
1 0 1 1 1 0 4
total = 8
non-zero rows: [0, 2, 3, 5]
To select a random index, choose a random value r up to the total number of true values, iterate over the array with the number of true values per non-zero row, adding them up until you know what row the r-th true value is in, and then iterate over that row to find the location of the r-th true value.
(You could simply pick a non-empty row first, and then pick a true value from that row, but that would create non-uniform probabilities.)
For an N×N-sized grid, the pre-processing would take N×N time and 2×N space, but the worst case look-up time would be N. In practice, using the JavaScript code example below, the pre-processing and look-up times (in ms) are in the order of:
grid size pre-processing look-up
10000 x 10000 5000 2.2
1000 x 1000 50 0.22
100 x 100 0.5 0.022
As you can see, look-up is more than 2000 times faster than pre-processing for a large grid, so if you need to randomly select several positions on the same (or slightly altered) grid, pre-processing makes a lot of sense.
function random2D(grid) {
this.grid = grid;
this.num = this.grid.map(function(elem) { // number of true values per row
return elem.reduce(function(sum, val) {
return sum + (val ? 1 : 0);
}, 0);
});
this.total = this.num.reduce(function(sum, val) { // total number of true values
return sum + val;
}, 0);
this.update = function(row, col, val) { // change value in grid
var prev = this.grid[row][col];
this.grid[row][col] = val;
if (prev ^ val) {
this.num[row] += val ? 1 : -1;
this.total += val ? 1 : -1;
}
}
this.select = function() { // select random index
var row = 0, col = 0;
var rnd = Math.floor(Math.random() * this.total) + 1;
while (rnd > this.num[row]) { // find row
rnd -= this.num[row++];
}
while (rnd) { // find column
if (this.grid[row][col]) --rnd;
if (rnd) ++col;
}
return {x: col, y: row};
}
}
var grid = [], size = 1000, prob = 0.01; // generate test data
for (var i = 0; i < size; i++) {
grid[i] = [];
for (var j = 0; j < size; j++) {
grid[i][j] = Math.random() < prob;
}
}
var rnd = new random2D(grid); // pre-process grid
document.write(JSON.stringify(rnd.select())); // get random index
Keeping a list of the rows which contain at least one true value only makes sense for very sparsely populated grids, where many rows contain no true values, so I haven't implemented it in the code example. If you do implement it, the look-up time for very sparse arrays is reduced to less than 1µs.

Answer 2

You can go with a dictionary implemented as a binary tree with logarithmic depth. This takes O(N^2) space and allows you to search/delete in O(log(N^2)) = O(logN) time. You can for example use Red-Black Tree.
The algorithm to find a random value might be:
t = tree.root
if (t == null)
throw Exception("No more values");
// logarithmic serach
while t.left != null or t.right != null
pick a random value k from range(0, 1, 2)
if (k == 0)
break;
if (k == 1)
if (t.left == null)
break
t = t.left
if (k == 2)
if (t.right == null)
break
t = t.right
result = t.value
// logarithmic delete
tree.delete(t)
return result
Of course, you can represent (i, j) indices as i * N + j.
Without additional memory you can't track changes to the state of cells. And in my opinion you can't get better than O(N^2) (iterating through the array).