Minizinc array sorting - sorting

Lets say I have an array declaration looking like this
array[1..5] of int: temp = [1,0,5,0,3];
Is there a way to initiate a new array looking the same as temp but without the 0's? The result would look like the following
[1,5,3]
or sort the array in such a way that the 0's would be either in the beginning or in the end of the array, which would be
[0,0,1,5,3]
or
[1,5,3,0,0]
Thanks

Even though Axel has answered this, I'll show another approach which - in my book is a little neater.
Case 1: the array ("temp") is a constant array.
Then one can simply write
array[int] of int: temp2 = [temp[i] | i in index_set(temp) where temp[i] != 0];
MiniZinc 2 (in contrast to version 1.*) don't need the size declaration if it can be calculated; it suffices to just use "array[int]". Also, "index_set" is used to be a little more general, e.g. to handle cases where the indices are from 0..4 (see the commented line).
Case 2: the array ("s") is an array of decision variables
If the array to handle is decision variables, we don't know (per definition) how many 0's there are and must rely on the alternative variant, namely to sort the array. One can then use the "sort" function, as shown in the model.
include "globals.mzn";
% constant
array[1..5] of int: temp = [1,0,5,0,3];
% array[0..4] of int: temp = array1d(0..4, [1,0,5,0,3]);
array[int] of int: temp2 = [temp[i] | i in index_set(temp) where temp[i] != 0];
% decision variables
array[1..5] of var int: s;
array[1..5] of var int: s2 = sort(s); % NOT CORRECT, see model below
solve satisfy;
constraint
s = [1,0,5,0,3]
;
% show our variables
output
[
"temp: \(temp)\n",
"temp2: \(temp2)\n",
"s: \(s)\n",
"s2: \(s2)\n",
];
Update
For the stable version of decision variables, this works what I can see. It calculating the position where to place this number depending on if "s[i]" is 0 or not. Not very pretty though.
int: n = 5;
array[1..n] of var 0..5: s;
array[1..n] of var lb_array(s)..ub_array(s): s2;
solve satisfy;
constraint
s = [1,0,5,0,3] /\
forall(i in 1..n) (
if s[i] != 0 then
s2[sum([s[j]!=0 | j in 1..i-1])+1] = s[i]
else
s2[sum([s[j]!=0 | j in 1..n]) + sum([s[j]=0 | j in 1..i-1])+1 ] = 0
endif
)
;
output
[
"s: \(s)\n",
"s2: \(s2)\n",
]
;
The output is
s: [1, 0, 5, 0, 3]
s2: [1, 5, 3, 0, 0]

Using MiniZinc 2, this can be done as follows:
array[1..5] of int: temp = [1,0,5,0,3];
% calculate upper bound of temp index
int: i_max = max(index_set(temp));
% use array comprehension to count non-zero elements
int: temp_non_zero = sum([1 | i in 1..i_max where temp[i] != 0]);
% copy non-zero elements to new array
array[1..temp_non_zero] of int: temp2 = [temp[i] | i in 1..i_max where temp[i] != 0];
% calculate upper bound for temp2 index
int: i2_max = max(index_set(temp2));
solve satisfy;
% show our variables
output
["\ni_max=" ++ show(i_max)]
++ ["\ni2_max=" ++ show(i2_max)]
++ ["\n" ++ show(temp2[i]) | i in 1..i2_max]
;

Another alternative:
A 1:1 mapping between the two arrays is established as an array of unique index values (= array positions). These indices are then sorted. The comparison weights elements higher if they point to a zero. Thus, the zero values are shifted to the back while leaving the order of the non-zero elements unchanged.
int: n = 5;
int: INF = 99999; % infinity
array[1..n] of var 0..5: s;
array[1..n] of var 1..n: map;
array[1..n] of var 0..5: s2;
solve satisfy;
% set s[]
constraint
s = [1,0,5,0,3]
;
% 1:1 mapping between s[] and s2[]
constraint
forall (i in 1..n) (
exists(j in 1..n) (
map[j] = i
)
)
;
constraint
forall(i in 1..n) (
s2[i] = s[map[i]]
)
;
% sort the map and move zero values to the back
constraint
forall(i in 1..n-1) (
(if s2[i] != 0 then map[i] else INF endif) <=
(if s2[i+1] != 0 then map[i+1] else INF endif)
)
;
output
[
"s: \(s)\n",
"map: \(map)\n",
"s2: \(s2)\n",
]
;
Output:
s: [1, 0, 5, 0, 3]
map: [1, 3, 5, 4, 2]
s2: [1, 5, 3, 0, 0]

Related

Dynamic Programming: max sum through a a 2 column list, left column skips a row

To preface, my English is not quite perfect so apologies for any mistakes.
The problem goes as follows:
Given a list of two choices in each column, determine the maximum sum possible from choosing only one of the two options in the column. The twist is: if the bottom of the two values is chosen, the next column is entirely skipped.
Example:
5
9 3 5 7 3
5 8 1 4 5
If you were to choose 5 initially from [9, 5], the column [3, 8] would be skipped. Whereas, if 9 was chosen, the next column would NOT be skipped and you could choose from [3, 8] (if 8 was chosen, the next column would be skipped and if 3 was chosen, it would not be, etc).
When you attempt to solve this using DP, the most important aspect of the problem is to define the right states of the DP.
Define D[i][j] = maximum sum until index i, if u choose element j || j in [0,1]
L = [
[9, 3, 5, 7, 3],
[5, 8, 1, 4, 5]
]
def find_max_sum(L):
D = [[0, 0] for _ in range(len(L[0]))]
D[0][0] = max(L[0][0], 0)
D[0][1] = max(L[1][0], 0)
for i in range(1, len(D)):
D[i][0] = L[0][i] + max(D[i-1])
if i > 1:
D[i][1] = max(L[1][i] + max(D[i-2]), max(D[i-1]))
else:
D[i][1] = max(L[1][i], max(D[i-1]))
return max(D[-1])
print(find_max_sum(L))
Let N be the number of columns, then we can store our choices in an array a(2 X N).
And lets define a function f(i,j) which gives the maximum possible sum from the first j+1 columns(0,1...j) where in j-th column we either pick the first option(i=0) or we pick the second option(i=1).
So the answer to this problem will be max(f(i,j)) for each i=[0,1] and j=[0,1,...N-1].
For each column j we need to handle 2 cases:
1- choosing the first option ---> f(0,j)
2- choosing the second option ---> f(1,j)
To calculate the values of column j ( f(0,j) , f(1,j) ) we need to consider these cases for column j-1:
1- we picked the first option from column j-1
2- we did not pick any option from column j-1(means we picked the second option from column j-2)
note: we can not pick the second option from column j-1 because we are picking an option from column j according to our function definition.
Here is the code written in C++
int OO = 1e6;// a big value
//reading the input
int a[2][100];
int N; cin >> N;
for (int i = 0; i < N; i++) cin >> a[0][i];
for (int i = 0; i < N; i++) cin >> a[1][i];
//initialization
int f[2][100];
f[0][0] = a[0][0];
f[1][0] = a[1][0];
int result = max(f[0][0],f[1][0]);
for (int i = 1; i < N; i++) {
int val1 = f[0][i - 1];
int val2 = -OO;
if (i - 2 >= 0)
val2 = f[1][i - 2];
f[0][i] = max(val1, val2) + a[0][i];
f[1][i] = max(val1,val2) + a[1][i];
result = max(result, max(f[0][i], f[1][i]));
}
cout << result << endl;

How many PR numbers exist in a given range?

It is not a homework problem. I am just curious about this problem. And my approach is simple brute-force :-)
My brute-force C++ code:
int main()
{
ll l,r;
cin>>l>>r;
ll f=0;
ll i=l;
while(i<=r)
{
ll j=0;
string s;
ll c=0;
s=to_string(i);
// cout<<s<<" ";
ll x=s.length();
if(x==1)
{
c=0;
}
else
{
j=0;
//whil
while(j<=x-2)
{
string b,g;
b="1";
g="1";
b=s[j];
g=s[j+1];
ll k1,k2;
k1=stoi(b);
k2=stoi(g);
if(__gcd(k1,k2)==1)
{
c=1;
break;
}
j++;
}
}
ll d=0;
j=0;
while(j<=x-1)
{
if( s[j]=='2' || s[j]=='3' || s[j]=='5' || s[j]=='7')
{
string b;
b="1";
b=s[j];
ll k1=stoi(b);
if(i%k1==0)
{
//d=0;
}
else
{
d=1;
break;
}
}
j++;
}
if(c==1 || d==1)
{
// cout<<"NO";
}
else
{
f++;
// cout<<"PR";
}
// cout<<"\n";
i++;
}
cout<<f;
return 0;
}
You are given 2 integers 'L' and 'R' . You are required to find the count of all the PR numbers in the range 'L' to 'R' inclusively. PR number are the numbers which satisfy following properties:
No pair of adjacent digits are co-prime i.e. adjacent digits in a PR number will not be co-prime to each other.
PR number is divisible by all the single digit prime numbers which occur as a digit in the PR number.
Note: Two numbers 'a' and 'b' are co-prime, if gcd(a,b)=1.
Also, gcd(0,a)=a;
Example:
Input: [2,5].
Output: '4'.
(Note: '1' is not a prime-number, though its very common)
(All the integers: '2','3','4','5') satisfy the condition of PR numbers :-)
Constraints on 'L','R': 1 <= L, R <= 10^18
What can be the the most efficient algorithm to solve this ?
Note: This will solve only part 1 which is No pair of adjacent digits are co-prime i.e. adjacent digits in a PR number will not be co-prime to each other.
Here is a constructive approach in python: instead of going throught all numbers in range and filtering by conditions, we will just construct all numbers that satisfy the condition. Note that if we have a valid sequence of digits, for it to continue being valid only the rightmost digit matters in order to decide what the next digit will be.
def ways(max_number, prev_digit, current_number):
if current_number > max_number:
return 0
count = 1
if prev_digit == 0:
if current_number != 0:
count += ways(max_number, 0, current_number * 10)
for i in range(2, 10):
count += ways(max_number, i, current_number * 10 + i)
if prev_digit == 2 or prev_digit == 4 or prev_digit == 8:
for i in [0, 2, 4, 6, 8]:
count += ways(max_number, i, current_number * 10 + i)
if prev_digit == 3 or prev_digit == 9:
for i in [0, 3, 6, 9]:
count += ways(max_number, i, current_number * 10 + i)
if prev_digit == 5 or prev_digit == 7:
count += ways(max_number, 0, current_number * 10)
count += ways(max_number, prev_digit, current_number * 10 + prev_digit)
if prev_digit == 6:
for i in [0, 2, 3, 4, 6, 8, 9]:
count += ways(max_number, i, current_number * 10 + i)
return count
As we are generating all valid numbers up to max_number without any repeats, the complexity of this function is O(amount of numbers between 0 and max_number that satisfy condition 1). To calculate the range a to b, we just need to do ways(b) - ways(a - 1).
Takes less than 1 second to caculate these numbers from 0 to 1 million, as there are only 42935 numbers that satisfy the result. As there are few numbers that satisfy the condition, we can then check if they are multiple of its prime digits to satisfy also condition 2. I leave this part up to the reader as there are multiple ways to do it.
TL;DR: This is more commonly called "digit dynamic programming with bitmask"
In more competitive-programming-familiar terms, you'd compute dp[n_digit][mod_2357][is_less_than_r][digit_appeared][last_digit] = number of numbers with n_digit digits (including leading zeroes), less than the number formed by first n_digit digits of R and with the other properties match. Do it twice with R and L-1 then take the difference. The number of operations required would be about 19 (number of digits) * 210 (mod) * 2 * 24 (it's only necessary to check for appearance of single-digit primes) * 10 * 10, which is obviously manageable by today computers.
Think about how you'd check whether a number is valid.
Not the normal way. Using a finite state automaton that take the input from left to right, digit by digit.
For simplicity, assume the input has a fixed number of digits (so that comparison with L/R is easier. This is possible because the number has at most as many digits as R).
It's necessary for each state to keep track of:
which digit appeared in the number (use a bit mask, there are 4 1-digit primes)
is the number in range [L..R] (either this is guaranteed to be true/false by the prefix, otherwise the prefix matches with that of L/R)
what is the value of the prefix mod each single digit prime
the most recent digit (to check whether all pairs of consecutive digits are coprime)
After the finite state automaton is constructed, the rest is simple. Just use dynamic programming to count the number of path to any accepted state from the starting state.
Remark: This method can be used to count the number of any type of object that can be verified using a finite state automaton (roughly speaking, you can check whether the property is satisfied using a program with constant memory usage, and takes the object piece-by-piece in some order)
We need a table where we can look up the count of suffixes that would match a prefix to construct valid numbers. Given a prefix's
right digit
prime combination
mod combination
and a suffix length, we'd like the count of suffixes that have searchable:
left digit
length
prime combination
mod combination
I started coding in Python, then switched to JavaScript to be able to offer a snippet. Comments in the code describe each lookup table. There are a few of them to allow for faster enumeration. There are samples of prefix-suffix calculations to illustrate how one can build an arbitrary upper-bound using the table, although at least some, maybe all of the prefix construction and aggregation could be made during the tabulation.
function gcd(a,b){
if (!b)
return a
else
return gcd(b, a % b)
}
// (Started writing in Python,
// then switched to JavaScript...
// 'xrange(4)' -> [0, 1, 2, 3]
// 'xrange(2, 4)' -> [2, 3]
function xrange(){
let l = 0
let r = arguments[1] || arguments[0]
if (arguments.length > 1)
l = arguments[0]
return new Array(r - l).fill(0).map((_, i) => i + l)
}
// A lookup table and its reverse,
// mapping each of the 210 mod combinations,
// [n % 2, n % 3, n % 5, n % 7], to a key
// from 0 to 209.
// 'mod_combs[0]' -> [0, 0, 0, 0]
// 'mod_combs[209]' -> [1, 2, 4, 6]
// 'mod_keys[[0,0,0,0]]' -> 0
// 'mod_keys[[1,2,4,6]]' -> 209
let mod_combs = {}
let mod_keys = {}
let mod_key_count = 0
for (let m2 of xrange(2)){
for (let m3 of xrange(3)){
for (let m5 of xrange(5)){
for (let m7 of xrange(7)){
mod_keys[[m2, m3, m5, m7]] = mod_key_count
mod_combs[mod_key_count] = [m2, m3, m5, m7]
mod_key_count += 1
}
}
}
}
// The main lookup table built using the
// dynamic program
// [mod_key 210][l_digit 10][suffix length 20][prime_comb 16]
let table = new Array(210)
for (let mk of xrange(210)){
table[mk] = new Array(10)
for (let l_digit of xrange(10)){
table[mk][l_digit] = new Array(20)
for (let sl of xrange(20)){
table[mk][l_digit][sl] = new Array(16).fill(0)
}
}
}
// We build prime combinations from 0 (no primes) to
// 15 (all four primes), using a bitmask of up to four bits.
let prime_set = [0, 0, 1<<0, 1<<1, 0, 1<<2, 0, 1<<3, 0, 0]
// The possible digits that could
// follow a digit
function get_valid_digits(digit){
if (digit == 0)
return [0, 2, 3, 4, 5, 6, 7, 8, 9]
else if ([2, 4, 8].includes(digit))
return [0, 2, 4, 6, 8]
else if ([3, 9].includes(digit))
return [0, 3, 6, 9]
else if (digit == 6)
return [0, 2, 3, 4, 6, 8, 9]
else if (digit == 5)
return [0, 5]
else if (digit == 7)
return [0, 7]
}
// Build the table bottom-up
// Single digits
for (let i of xrange(10)){
let mod_key = mod_keys[[i % 2, i % 3, i % 5, i % 7]]
let length = 1
let l_digit = i
let prime_comb = prime_set[i]
table[mod_key][l_digit][length][prime_comb] = 1
}
// Everything else
// For demonstration, we just table up to 6 digits
// since either JavaScript, this program, or both seem
// to be too slow for a full demo.
for (let length of xrange(2, 6)){
// We're appending a new left digit
for (let new_l_digit of xrange(0, 10)){
// The digit 1 is never valid
if (new_l_digit == 1)
continue
// The possible digits that could
// be to the right of our new left digit
let ds = get_valid_digits(new_l_digit)
// For each possible digit to the right
// of our new left digit, iterate over all
// the combinations of primes and remainder combinations.
// The ones that are populated are valid paths, the
// sum of which can be aggregated for each resulting
// new combination of primes and remainders.
for (let l_digit of ds){
for (let p_comb of xrange(16)){
for (let m_key of xrange(210)){
new_prime_comb = prime_set[new_l_digit] | p_comb
// suffix's remainder combination
let [m2, m3, m5, m7] = mod_combs[m_key]
// new remainder combination
let m = Math.pow(10, length - 1) * new_l_digit
let new_mod_key = mod_keys[[(m + m2) % 2, (m + m3) % 3, (m + m5) % 5, (m + m7) % 7]]
// Aggregate any populated entries into the new
// table entry
table[new_mod_key][new_l_digit][length][new_prime_comb] += table[m_key][l_digit][length - 1][p_comb]
}
}
}
}
}
// If we need only a subset of the mods set to
// zero, we need to check all instances where
// this subset is zero. For example,
// for the prime combination, [2, 3], we need to
// check all mod combinations where the first two
// are zero since we don't care about the remainders
// for 5 and 7: [0,0,0,0], [0,0,0,1],... [0,0,4,6]
// Return all needed combinations given some
// predetermined, indexed remainders.
function prime_comb_to_mod_keys(remainders){
let mod_map = [2, 3, 5, 7]
let mods = []
for (let i of xrange(4))
mods.push(!remainders.hasOwnProperty(i) ? mod_map[i] - 1 : 0)
function f(ms, i){
if (i == ms.length){
for (let idx in remainders)
ms[idx] = remainders[idx]
return [mod_keys[ms]]
}
let result = []
for (let m=ms[i] - 1; m>=0; m--){
let _ms = ms.slice()
_ms[i] = m
result = result.concat(f(_ms, i + 1))
}
return result.concat(f(ms, i + 1))
}
return f(mods, 0)
}
function get_matching_mods(prefix, len_suffix, prime_comb){
let ps = [2, 3, 5, 7]
let actual_prefix = Math.pow(10, len_suffix) * prefix
let remainders = {}
for (let i in xrange(4)){
if (prime_comb & (1 << i))
remainders[i] = (ps[i] - (actual_prefix % ps[i])) % ps[i]
}
return prime_comb_to_mod_keys(remainders)
}
// A brute-force function to check the
// table is working. Returns a list of
// valid numbers of 'length' digits
// given a prefix.
function confirm(prefix, length){
let result = [0, []]
let ps = [0, 0, 2, 3, 0, 5, 0, 7, 0, 0]
let p_len = String(prefix).length
function check(suffix){
let num = Math.pow(10, length - p_len) * prefix + suffix
let temp = num
prev = 0
while (temp){
let d = temp % 10
if (d == 1 || gcd(prev, d) == 1 || (ps[d] && num % d))
return [0, []]
prev = d
temp = ~~(temp / 10)
}
return [1, [num]]
}
for (suffix of xrange(Math.pow(10, length - p_len))){
let [a, b] = check(suffix)
result[0] += a
result[1] = result[1].concat(b)
}
return result
}
function get_prime_comb(prefix){
let prime_comb = 0
while (prefix){
let d = prefix % 10
prime_comb |= prime_set[d]
prefix = ~~(prefix / 10)
}
return prime_comb
}
// A function to test the table
// against the brute-force method.
// To match a prefix with the number
// of valid suffixes of a chosen length
// in the table, we want to aggregate all
// prime combinations for all valid digits,
// where the remainders for each combined
// prime combination (prefix with suffix)
// sum to zero (with the appropriate mod).
function test(prefix, length, show=false){
let r_digit = prefix % 10
let len_suffix = length - String(prefix).length
let prefix_prime_comb = get_prime_comb(prefix)
let ds = get_valid_digits(r_digit)
let count = 0
for (let l_digit of ds){
for (let prime_comb of xrange(16)){
for (let i of get_matching_mods(prefix, len_suffix, prefix_prime_comb | prime_comb)){
let v = table[i][l_digit][len_suffix][prime_comb]
count += v
}
}
}
let c = confirm(prefix, length)
return `${ count }, ${ c[0] }${ show ? ': ' + c[1] : '' }`
}
// Arbitrary prefixes
for (let length of [3, 4]){
for (let prefix of [2, 30]){
console.log(`prefix, length: ${ prefix }, ${ length }`)
console.log(`tabled, brute-force: ${ test(prefix, length, true) }\n\n`)
}
}
let length = 6
for (let l_digit=2; l_digit<10; l_digit++){
console.log(`prefix, length: ${ l_digit }, ${ length }`)
console.log(`tabled, brute-force: ${ test(l_digit, length) }\n\n`)
}

Reconstructing input to encoder from output

I would like to understand how to solve the Codility ArrayRecovery challenge, but I don't even know what branch of knowledge to consult. Is it combinatorics, optimization, computer science, set theory, or something else?
Edit:
The branch of knowledge to consult is constraint programming, particularly constraint propagation. You also need some combinatorics to know that if you take k numbers at a time from the range [1..n], with the restriction that no number can be bigger than the one before it, that works out to be
(n+k-1)!/k!(n-1)! possible combinations
which is the same as the number of combinations with replacements of n things taken k at a time, which has the mathematical notation . You can read about why it works out like that here.
Peter Norvig provides an excellent example of how to solve this kind of problem with his Sudoku solver.
You can read the full description of the ArrayRecovery problem via the link above. The short story is that there is an encoder that takes a sequence of integers in the range 1 up to some given limit (say 100 for our purposes) and for each element of the input sequence outputs the most recently seen integer that is smaller than the current input, or 0 if none exists.
input 1, 2, 3, 4 => output 0, 1, 2, 3
input 2, 4, 3 => output 0, 2, 2
The full task is, given the output (and the range of allowable input), figure out how many possible inputs could have generated it. But before I even get to that calculation, I'm not confident about how to even approach formulating the equation. That is what I am asking for help with. (Of course a full solution would be welcome, too, if it is explained.)
I just look at some possible outputs and wonder. Here are some sample encoder outputs and the inputs I can come up with, with * meaning any valid input and something like > 4 meaning any valid input greater than 4. If needed, inputs are referred to as A1, A2, A3, ... (1-based indexing)
Edit #2
Part of the problem I was having with this challenge is that I did not manually generate the exactly correct sets of possible inputs for an output. I believe the set below is correct now. Look at this answer's edit history if you want to see my earlier mistakes.
output #1: 0, 0, 0, 4
possible inputs: [>= 4, A1 >= * >= 4, 4, > 4]
output #2: 0, 0, 0, 2, 3, 4 # A5 ↴ See more in discussion below
possible inputs: [>= 2, A1 >= * >=2, 2, 3, 4, > 4]
output #3: 0, 0, 0, 4, 3, 1
possible inputs: none # [4, 3, 1, 1 >= * > 4, 4, > 1] but there is no number 1 >= * > 4
The second input sequence is very tightly constrained compared to the first just by adding 2 more outputs. The third sequence is so constrained as to be impossible.
But the set of constraints on A5 in example #2 is a bit harder to articulate. Of course A5 > O5, that is the basic constraint on all the inputs. But any output > A4 and after O5 has to appear in the input after A4, so A5 has to be an element of the set of numbers that comes after A5 that is also > A4. Since there is only 1 such number (A6 == 4), A5 has to be it, but it gets more complicated if there is a longer string of numbers that follow. (Editor's note: actually it doesn't.)
As the output set gets longer, I worry these constraints just get more complicated and harder to get right. I cannot think of any data structures for efficiently representing these in a way that leads to efficiently calculating the number of possible combinations. I also don't quite see how to algorithmically add constraint sets together.
Here are the constraints I see so far for any given An
An > On
An <= min(Set of other possible numbers from O1 to n-1 > On). How to define the set of possible numbers greater than On?
Numbers greater than On that came after the most recent occurrence of On in the input
An >= max(Set of other possible numbers from O1 to n-1 < On). How to define the set of possible numbers less than On?
Actually this set is empty because On is, by definition, the largest possible number from the previous input sequence. (Which it not to say it is strictly the largest number from the previous input sequence.)
Any number smaller than On that came before the last occurrence of it in the input would be ineligible because of the "nearest" rule. No numbers smaller that On could have occurred after the most recent occurrence because of the "nearest" rule and because of the transitive property: if Ai < On and Aj < Ai then Aj < On
Then there is the set theory:
An must be an element of the set of unaccounted-for elements of the set of On+1 to Om, where m is the smallest m > n such that Om < On. Any output after such Om and larger than Om (which An is) would have to appear as or after Am.
An element is unaccounted-for if it is seen in the output but does not appear in the input in a position that is consistent with the rest of the output. Obviously I need a better definition than this in order to code and algorithm to calculate it.
It seems like perhaps some kind of set theory and/or combinatorics or maybe linear algebra would help with figuring out the number of possible sequences that would account for all of the unaccounted-for outputs and fit the other constraints. (Editor's note: actually, things never get that complicated.)
The code below passes all of Codility's tests. The OP added a main function to use it on the command line.
The constraints are not as complex as the OP thinks. In particular, there is never a situation where you need to add a restriction that an input be an element of some set of specific integers seen elsewhere in the output. Every input position has a well-defined minimum and maximum.
The only complication to that rule is that sometimes the maximum is "the value of the previous input" and that input itself has a range. But even then, all the values like that are consecutive and have the same range, so the number of possibilities can be calculated with basic combinatorics, and those inputs as a group are independent of the other inputs (which only serve to set the range), so the possibilities of that group can be combined with the possibilities of other input positions by simple multiplication.
Algorithm overview
The algorithm makes a single pass through the output array updating the possible numbers of input arrays after every span, which is what I am calling repetitions of numbers in the output. (You might say maximal subsequences of the output where every element is identical.) For example, for output 0,1,1,2 we have three spans: 0, 1,1 and 2. When a new span begins, the number of possibilities for the previous span is calculated.
This decision was based on a few observations:
For spans longer than 1 in length, the minimum value of the input
allowed in the first position is whatever the value is of the input
in the second position. Calculating the number of possibilities of a
span is straightforward combinatorics, but the standard formula
requires knowing the range of the numbers and the length of the span.
Every time the value of the
output changes (and a new span beings), that strongly constrains the value of the previous span:
When the output goes up, the only possible reason is that the previous input was the value of the new, higher output and the input corresponding to the position of the new, higher output, was even higher.
When an output goes down, new constraints are established, but those are a bit harder to articulate. The algorithm stores stairs (see below) in order to quantify the constraints imposed when the output goes down
The aim here was to confine the range of possible values for every span. Once we do that accurately, calculating the number of combinations is straightforward.
Because the encoder backtracks looking to output a number that relates to the input in 2 ways, both smaller and closer, we know we can throw out numbers that are larger and farther away. After a small number appears in the output, no larger number from before that position can have any influence on what follows.
So to confine these ranges of input when the output sequence decreased, we need to store stairs - a list of increasingly larger possible values for the position in the original array. E.g for 0,2,5,7,2,4 stairs build up like this: 0, 0,2, 0,2,5, 0,2,5,7, 0,2, 0,2,4.
Using these bounds we can tell for sure that the number in the position of the second 2 (next to last position in the example) must be in (2,5], because 5 is the next stair. If the input were greater than 5, a 5 would have been output in that space instead of a 2. Observe, that if the last number in the encoded array was not 4, but 6, we would exit early returning 0, because we know that the previous number couldn't be bigger than 5.
The complexity is O(n*lg(min(n,m))).
Functions
CombinationsWithReplacement - counts number of combinations with replacements of size k from n numbers. E.g. for (3, 2) it counts 3,3, 3,2, 3,1, 2,2, 2,1, 1,1, so returns 6 It is the same as choose(n - 1 + k, n - 1).
nextBigger - finds next bigger element in a range. E.g. for 4 in sub-array 1,2,3,4,5 it returns 5, and in sub-array 1,3 it returns its parameter Max.
countSpan (lambda) - counts how many different combinations a span we have just passed can have. Consider span 2,2 for 0,2,5,7,2,2,7.
When curr gets to the final position, curr is 7 and prev is the final 2 of the 2,2 span.
It computes maximum and minimum possible values of the prev span. At this point stairs consist of 2,5,7 then maximum possible value is 5 (nextBigger after 2 in the stair 2,5,7). A value of greater than 5 in this span would have output a 5, not a 2.
It computes a minimum value for the span (which is the minimum value for every element in the span), which is prev at this point, (remember curr at this moment equals to 7 and prev to 2). We know for sure that in place of the final 2 output, the original input has to have 7, so the minimum is 7. (This is a consequence of the "output goes up" rule. If we had 7,7,2 and curr would be 2 then the minimum for the previous span (the 7,7) would be 8 which is prev + 1.
It adjusts the number of combinations. For a span of length L with a range of n possibilities (1+max-min), there are possibilities, with k being either L or L-1 depending on what follows the span.
For a span followed by a larger number, like 2,2,7, k = L - 1 because the last position of the 2,2 span has to be 7 (the value of the first number after the span).
For a span followed by a smaller number, like 7,7,2, k = L because
the last element of 7,7 has no special constraints.
Finally, it calls CombinationsWithReplacement to find out the number of branches (or possibilities), computes new res partial results value (remainder values in the modulo arithmetic we are doing), and returns new res value and max for further handling.
solution - iterates over the given Encoder Output array. In the main loop, while in a span it counts the span length, and at span boundaries it updates res by calling countSpan and possibly updates the stairs.
If the current span consists of a bigger number than the previous one, then:
Check validity of the next number. E.g 0,2,5,2,7 is invalid input, becuase there is can't be 7 in the next-to-last position, only 3, or 4, or 5.
It updates the stairs. When we have seen only 0,2, the stairs are 0,2, but after the next 5, the stairs become 0,2,5.
If the current span consists of a smaller number then the previous one, then:
It updates stairs. When we have seen only 0,2,5, our stairs are 0,2,5, but after we have seen 0,2,5,2 the stairs become 0,2.
After the main loop it accounts for the last span by calling countSpan with -1 which triggers the "output goes down" branch of calculations.
normalizeMod, extendedEuclidInternal, extendedEuclid, invMod - these auxiliary functions help to deal with modulo arithmetic.
For stairs I use storage for the encoded array, as the number of stairs never exceeds current position.
#include <algorithm>
#include <cassert>
#include <vector>
#include <tuple>
const int Modulus = 1'000'000'007;
int CombinationsWithReplacement(int n, int k);
template <class It>
auto nextBigger(It begin, It end, int value, int Max) {
auto maxIt = std::upper_bound(begin, end, value);
auto max = Max;
if (maxIt != end) {
max = *maxIt;
}
return max;
}
auto solution(std::vector<int> &B, const int Max) {
auto res = 1;
const auto size = (int)B.size();
auto spanLength = 1;
auto prev = 0;
// Stairs is the list of numbers which could be smaller than number in the next position
const auto stairsBegin = B.begin();
// This includes first entry (zero) into stairs
// We need to include 0 because we can meet another zero later in encoded array
// and we need to be able to find in stairs
auto stairsEnd = stairsBegin + 1;
auto countSpan = [&](int curr) {
const auto max = nextBigger(stairsBegin, stairsEnd, prev, Max);
// At the moment when we switch from the current span to the next span
// prev is the number from previous span and curr from current.
// E.g. 1,1,7, when we move to the third position cur = 7 and prev = 1.
// Observe that, in this case minimum value possible in place of any of 1's can be at least 2=1+1=prev+1.
// But if we consider 7, then we have even more stringent condition for numbers in place of 1, it is 7
const auto min = std::max(prev + 1, curr);
const bool countLast = prev > curr;
const auto branchesCount = CombinationsWithReplacement(max - min + 1, spanLength - (countLast ? 0 : 1));
return std::make_pair(res * (long long)branchesCount % Modulus, max);
};
for (int i = 1; i < size; ++i) {
const auto curr = B[i];
if (curr == prev) {
++spanLength;
}
else {
int max;
std::tie(res, max) = countSpan(curr);
if (prev < curr) {
if (curr > max) {
// 0,1,5,1,7 - invalid because number in the fourth position lies in [2,5]
// and so in the fifth encoded position we can't something bigger than 5
return 0;
}
// It is time to possibly shrink stairs.
// E.g if we had stairs 0,2,4,9,17 and current value is 5,
// then we no more interested in 9 and 17, and we change stairs to 0,2,4,5.
// That's because any number bigger than 9 or 17 also bigger than 5.
const auto s = std::lower_bound(stairsBegin, stairsEnd, curr);
stairsEnd = s;
*stairsEnd++ = curr;
}
else {
assert(curr < prev);
auto it = std::lower_bound(stairsBegin, stairsEnd, curr);
if (it == stairsEnd || *it != curr) {
// 0,5,1 is invalid sequence because original sequence lloks like this 5,>5,>1
// and there is no 1 in any of the two first positions, so
// it can't appear in the third position of the encoded array
return 0;
}
}
spanLength = 1;
}
prev = curr;
}
res = countSpan(-1).first;
return res;
}
template <class T> T normalizeMod(T a, T m) {
if (a < 0) return a + m;
return a;
}
template <class T> std::pair<T, std::pair<T, T>> extendedEuclidInternal(T a, T b) {
T old_x = 1;
T old_y = 0;
T x = 0;
T y = 1;
while (true) {
T q = a / b;
T t = a - b * q;
if (t == 0) {
break;
}
a = b;
b = t;
t = x; x = old_x - x * q; old_x = t;
t = y; y = old_y - y * q; old_y = t;
}
return std::make_pair(b, std::make_pair(x, y));
}
// Returns gcd and Bezout's coefficients
template <class T> std::pair<T, std::pair<T, T>> extendedEuclid(T a, T b) {
if (a > b) {
if (b == 0) return std::make_pair(a, std::make_pair(1, 0));
return extendedEuclidInternal(a, b);
}
else {
if (a == 0) return std::make_pair(b, std::make_pair(0, 1));
auto p = extendedEuclidInternal(b, a);
std::swap(p.second.first, p.second.second);
return p;
}
}
template <class T> T invMod(T a, T m) {
auto p = extendedEuclid(a, m);
assert(p.first == 1);
return normalizeMod(p.second.first, m);
}
int CombinationsWithReplacement(int n, int k) {
int res = 1;
for (long long i = n; i < n + k; ++i) {
res = res * i % Modulus;
}
int denom = 1;
for (long long i = k; i > 0; --i) {
denom = denom * i % Modulus;
}
res = res * (long long)invMod(denom, Modulus) % Modulus;
return res;
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Only the above is needed for the Codility challenge. Below is to run on the command line.
//
// Compile with: gcc -std=gnu++14 -lc++ -lstdc++ array_recovery.cpp
//
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include <string.h>
// Usage: 0 1 2,3, 4 M
// Last arg is M, the max value for an input.
// Remaining args are B (the output of the encoder) separated by commas and/or spaces
// Parentheses and brackets are ignored, so you can use the same input form as Codility's tests: ([1,2,3], M)
int main(int argc, char* argv[]) {
int Max;
std::vector<int> B;
const char* delim = " ,[]()";
if (argc < 2 ) {
printf("Usage: %s M 0 1 2,3, 4... \n", argv[0]);
return 1;
}
for (int i = 1; i < argc; i++) {
char* parse;
parse = strtok(argv[i], delim);
while (parse != NULL)
{
B.push_back(atoi(parse));
parse = strtok (NULL, delim);
}
}
Max = B.back();
B.pop_back();
printf("%d\n", solution(B, Max));
return 0;
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Only the above is needed for the Codility challenge. Below is to run on the command line.
//
// Compile with: gcc -std=gnu++14 -lc++ -lstdc++ array_recovery.cpp
//
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include <string.h>
// Usage: M 0 1 2,3, 4
// first arg is M, the max value for an input.
// remaining args are B (the output of the encoder) separated by commas and/or spaces
int main(int argc, char* argv[]) {
int Max;
std::vector<int> B;
const char* delim = " ,";
if (argc < 3 ) {
printf("Usage: %s M 0 1 2,3, 4... \n", argv[0]);
return 1;
}
Max = atoi(argv[1]);
for (int i = 2; i < argc; i++) {
char* parse;
parse = strtok(argv[i], delim);
while (parse != NULL)
{
B.push_back(atoi(parse));
parse = strtok (NULL, delim);
}
}
printf("%d\n", solution(B, Max));
return 0;
}
Let's see an example:
Max = 5
Array is
0 1 3 0 1 1 3
1
1 2..5
1 3 4..5
1 3 4..5 1
1 3 4..5 1 2..5
1 3 4..5 1 2..5 >=..2 (sorry, for a cumbersome way of writing)
1 3 4..5 1 3..5 >=..3 4..5
Now count:
1 1 2 1 3 2 which amounts to 12 total.
Here's an idea. One known method to construct the output is to use a stack. We pop it while the element is greater or equal, then output the smaller element if it exists, then push the greater element onto the stack. Now what if we attempted to do this backwards from the output?
First we'll demonstrate the stack method using the c∅dility example.
[2, 5, 3, 7, 9, 6]
2: output 0, stack [2]
5: output 2, stack [2,5]
3: pop 5, output, 2, stack [2,3]
7: output 3, stack [2,3,7]
... etc.
Final output: [0, 2, 2, 3, 7, 3]
Now let's try reconstruction! We'll use stack both as the imaginary stack and as the reconstituted input:
(Input: [2, 5, 3, 7, 9, 6])
Output: [0, 2, 2, 3, 7, 3]
* Something >3 that reached 3 in the stack
stack = [3, 3 < *]
* Something >7 that reached 7 in the stack
but both of those would've popped before 3
stack = [3, 7, 7 < x, 3 < * <= x]
* Something >3, 7 qualifies
stack = [3, 7, 7 < x, 3 < * <= x]
* Something >2, 3 qualifies
stack = [2, 3, 7, 7 < x, 3 < * <= x]
* Something >2 and >=3 since 3 reached 2
stack = [2, 2 < *, 3, 7, 7 < x, 3 < * <= x]
Let's attempt your examples:
Example 1:
[0, 0, 0, 2, 3, 4]
* Something >4
stack = [4, 4 < *]
* Something >3, 4 qualifies
stack = [3, 4, 4 < *]
* Something >2, 3 qualifies
stack = [2, 3, 4, 4 < *]
* The rest is non-increasing with lowerbound 2
stack = [y >= x, x >= 2, 2, 3, 4, >4]
Example 2:
[0, 0, 0, 4]
* Something >4
stack [4, 4 < *]
* Non-increasing
stack = [z >= y, y >= 4, 4, 4 < *]
Calculating the number of combinations is achieved by multiplying together the possibilities for all the sections. A section is either a bounded single cell; or a bound, non-increasing subarray of one or more cells. To calculate the latter we use the multi-choose binomial, (n + k - 1) choose (k - 1). Consider that we can express the differences between the cells of a bound, non-increasing sequence of 3 cells as:
(ub - cell_3) + (cell_3 - cell_2) + (cell_2 - cell_1) + (cell_1 - lb) = ub - lb
Then the number of ways to distribute ub - lb into (x + 1) cells is
(n + k - 1) choose (k - 1)
or
(ub - lb + x) choose x
For example, the number of non-increasing sequences between
(3,4) in two cells is (4 - 3 + 2) choose 2 = 3: [3,3] [4,3] [4,4]
And the number of non-increasing sequences between
(3,4) in three cells is (4 - 3 + 3) choose 3 = 4: [3,3,3] [4,3,3] [4,4,3] [4,4,4]
(Explanation attributed to Brian M. Scott.)
Rough JavaScript sketch (the code is unreliable; it's only meant to illustrate the encoding. The encoder lists [lower_bound, upper_bound], or a non-increasing sequence as [non_inc, length, lower_bound, upper_bound]):
function f(A, M){
console.log(JSON.stringify(A), M);
let i = A.length - 1;
let last = A[i];
let s = [[last,last]];
if (A[i-1] == last){
let d = 1;
s.splice(1,0,['non_inc',d++,last,M]);
while (i > 0 && A[i-1] == last){
s.splice(1,0,['non_inc',d++,last,M]);
i--
}
} else {
s.push([last+1,M]);
i--;
}
if (i == 0)
s.splice(0,1);
for (; i>0; i--){
let x = A[i];
if (x < s[0][0])
s = [[x,x]].concat(s);
if (x > s[0][0]){
let [l, _l] = s[0];
let [lb, ub] = s[1];
s[0] = [x+1, M];
s[1] = [lb, x];
s = [[l,_l], [x,x]].concat(s);
}
if (x == s[0][0]){
let [l,_l] = s[0];
let [lb, ub] = s[1];
let d = 1;
s.splice(0,1);
while (i > 0 && A[i-1] == x){
s =
[['non_inc', d++, lb, M]].concat(s);
i--;
}
if (i > 0)
s = [[l,_l]].concat(s);
}
}
// dirty fix
if (s[0][0] == 0)
s.splice(0,1);
return s;
}
var a = [2, 5, 3, 7, 9, 6]
var b = [0, 2, 2, 3, 7, 3]
console.log(JSON.stringify(a));
console.log(JSON.stringify(f(b,10)));
b = [0,0,0,4]
console.log(JSON.stringify(f(b,10)));
b = [0,2,0,0,0,4]
console.log(JSON.stringify(f(b,10)));
b = [0,0,0,2,3,4]
console.log(JSON.stringify(f(b,10)));
b = [0,2,2]
console.log(JSON.stringify(f(b,4)));
b = [0,3,5,6]
console.log(JSON.stringify(f(b,10)));
b = [0,0,3,0]
console.log(JSON.stringify(f(b,10)));

Algorithm for equiprobable random square binary matrices with two non-adjacent non-zeros in each row and column

It would be great if someone could point me towards an algorithm that would allow me to :
create a random square matrix, with entries 0 and 1, such that
every row and every column contain exactly two non-zero entries,
two non-zero entries cannot be adjacent,
all possible matrices are equiprobable.
Right now I manage to achieve points 1 and 2 doing the following : such a matrix can be transformed, using suitable permutations of rows and columns, into a diagonal block matrix with blocks of the form
1 1 0 0 ... 0
0 1 1 0 ... 0
0 0 1 1 ... 0
.............
1 0 0 0 ... 1
So I start from such a matrix using a partition of [0, ..., n-1] and scramble it by permuting rows and columns randomly. Unfortunately, I can't find a way to integrate the adjacency condition, and I am quite sure that my algorithm won't treat all the matrices equally.
Update
I have managed to achieve point 3. The answer was actually straight under my nose : the block matrix I am creating contains all the information needed to take into account the adjacency condition. First some properties and definitions:
a suitable matrix defines permutations of [1, ..., n] that can be build like so: select a 1 in row 1. The column containing this entry contains exactly one other entry equal to 1 on a row a different from 1. Again, row a contains another entry 1 in a column which contains a second entry 1 on a row b, and so on. This starts a permutation 1 -> a -> b ....
For instance, with the following matrix, starting with the marked entry
v
1 0 1 0 0 0 | 1
0 1 0 0 0 1 | 2
1 0 0 1 0 0 | 3
0 0 1 0 1 0 | 4
0 0 0 1 0 1 | 5
0 1 0 0 1 0 | 6
------------+--
1 2 3 4 5 6 |
we get permutation 1 -> 3 -> 5 -> 2 -> 6 -> 4 -> 1.
the cycles of such a permutation lead to the block matrix I mentioned earlier. I also mentioned scrambling the block matrix using arbitrary permutations on the rows and columns to rebuild a matrix compatible with the requirements.
But I was using any permutation, which led to some adjacent non-zero entries. To avoid that, I have to choose permutations that separate rows (and columns) that are adjacent in the block matrix. Actually, to be more precise, if two rows belong to a same block and are cyclically consecutive (the first and last rows of a block are considered consecutive too), then the permutation I want to apply has to move these rows into non-consecutive rows of the final matrix (I will call two rows incompatible in that case).
So the question becomes : How to build all such permutations ?
The simplest idea is to build a permutation progressively by randomly adding rows that are compatible with the previous one. As an example, consider the case n = 6 using partition 6 = 3 + 3 and the corresponding block matrix
1 1 0 0 0 0 | 1
0 1 1 0 0 0 | 2
1 0 1 0 0 0 | 3
0 0 0 1 1 0 | 4
0 0 0 0 1 1 | 5
0 0 0 1 0 1 | 6
------------+--
1 2 3 4 5 6 |
Here rows 1, 2 and 3 are mutually incompatible, as are 4, 5 and 6. Choose a random row, say 3.
We will write a permutation as an array: [2, 5, 6, 4, 3, 1] meaning 1 -> 2, 2 -> 5, 3 -> 6, ... This means that row 2 of the block matrix will become the first row of the final matrix, row 5 will become the second row, and so on.
Now let's build a suitable permutation by choosing randomly a row, say 3:
p = [3, ...]
The next row will then be chosen randomly among the remaining rows that are compatible with 3 : 4, 5and 6. Say we choose 4:
p = [3, 4, ...]
Next choice has to be made among 1 and 2, for instance 1:
p = [3, 4, 1, ...]
And so on: p = [3, 4, 1, 5, 2, 6].
Applying this permutation to the block matrix, we get:
1 0 1 0 0 0 | 3
0 0 0 1 1 0 | 4
1 1 0 0 0 0 | 1
0 0 0 0 1 1 | 5
0 1 1 0 0 0 | 2
0 0 0 1 0 1 | 6
------------+--
1 2 3 4 5 6 |
Doing so, we manage to vertically isolate all non-zero entries. Same has to be done with the columns, for instance by using permutation p' = [6, 3, 5, 1, 4, 2] to finally get
0 1 0 1 0 0 | 3
0 0 1 0 1 0 | 4
0 0 0 1 0 1 | 1
1 0 1 0 0 0 | 5
0 1 0 0 0 1 | 2
1 0 0 0 1 0 | 6
------------+--
6 3 5 1 4 2 |
So this seems to work quite efficiently, but building these permutations needs to be done with caution, because one can easily be stuck: for instance, with n=6 and partition 6 = 2 + 2 + 2, following the construction rules set up earlier can lead to p = [1, 3, 2, 4, ...]. Unfortunately, 5 and 6 are incompatible, so choosing one or the other makes the last choice impossible. I think I've found all situations that lead to a dead end. I will denote by r the set of remaining choices:
p = [..., x, ?], r = {y} with x and y incompatible
p = [..., x, ?, ?], r = {y, z} with y and z being both incompatible with x (no choice can be made)
p = [..., ?, ?], r = {x, y} with x and y incompatible (any choice would lead to situation 1)
p = [..., ?, ?, ?], r = {x, y, z} with x, y and z being cyclically consecutive (choosing x or z would lead to situation 2, choosing y to situation 3)
p = [..., w, ?, ?, ?], r = {x, y, z} with xwy being a 3-cycle (neither x nor y can be chosen, choosing z would lead to situation 3)
p = [..., ?, ?, ?, ?], r = {w, x, y, z} with wxyz being a 4-cycle (any choice would lead to situation 4)
p = [..., ?, ?, ?, ?], r = {w, x, y, z} with xyz being a 3-cycle (choosing w would lead to situation 4, choosing any other would lead to situation 4)
Now it seems that the following algorithm gives all suitable permutations:
As long as there are strictly more than 5 numbers to choose, choose randomly among the compatible ones.
If there are 5 numbers left to choose: if the remaining numbers contain a 3-cycle or a 4-cycle, break that cycle (i.e. choose a number belonging to that cycle).
If there are 4 numbers left to choose: if the remaining numbers contain three cyclically consecutive numbers, choose one of them.
If there are 3 numbers left to choose: if the remaining numbers contain two cyclically consecutive numbers, choose one of them.
I am quite sure that this allows me to generate all suitable permutations and, hence, all suitable matrices.
Unfortunately, every matrix will be obtained several times, depending on the partition that was chosen.
Intro
Here is some prototype-approach, trying to solve the more general task of
uniform combinatorial sampling, which for our approach here means: we can use this approach for everything which we can formulate as SAT-problem.
It's not exploiting your problem directly and takes a heavy detour. This detour to the SAT-problem can help in regards to theory (more powerful general theoretical results) and efficiency (SAT-solvers).
That being said, it's not an approach if you want to sample within seconds or less (in my experiments), at least while being concerned about uniformity.
Theory
The approach, based on results from complexity-theory, follows this work:
GOMES, Carla P.; SABHARWAL, Ashish; SELMAN, Bart. Near-uniform sampling of combinatorial spaces using XOR constraints. In: Advances In Neural Information Processing Systems. 2007. S. 481-488.
The basic idea:
formulate the problem as SAT-problem
add randomly generated xors to the problem (acting on the decision-variables only! that's important in practice)
this will reduce the number of solutions (some solutions will get impossible)
do that in a loop (with tuned parameters) until only one solution is left!
search for some solution is being done by SAT-solvers or #SAT-solvers (=model-counting)
if there is more than one solution: no xors will be added but a complete restart will be done: add random-xors to the start-problem!
The guarantees:
when tuning the parameters right, this approach achieves near-uniform sampling
this tuning can be costly, as it's based on approximating the number of possible solutions
empirically this can also be costly!
Ante's answer, mentioning the number sequence A001499 actually gives a nice upper bound on the solution-space (as it's just ignoring adjacency-constraints!)
The drawbacks:
inefficient for large problems (in general; not necessarily compared to the alternatives like MCMC and co.)
need to change / reduce parameters to produce samples
those reduced parameters lose the theoretical guarantees
but empirically: good results are still possible!
Parameters:
In practice, the parameters are:
N: number of xors added
L: minimum number of variables part of one xor-constraint
U: maximum number of variables part of one xor-constraint
N is important to reduce the number of possible solutions. Given N constant, the other variables of course also have some effect on that.
Theory says (if i interpret correctly), that we should use L = R = 0.5 * #dec-vars.
This is impossible in practice here, as xor-constraints hurt SAT-solvers a lot!
Here some more scientific slides about the impact of L and U.
They call xors of size 8-20 short-XORS, while we will need to use even shorter ones later!
Implementation
Final version
Here is a pretty hacky implementation in python, using the XorSample scripts from here.
The underlying SAT-solver in use is Cryptominisat.
The code basically boils down to:
Transform the problem to conjunctive normal-form
as DIMACS-CNF
Implement the sampling-approach:
Calls XorSample (pipe-based + file-based)
Call SAT-solver (file-based)
Add samples to some file for later analysis
Code: (i hope i did warn you already about the code-quality)
from itertools import count
from time import time
import subprocess
import numpy as np
import os
import shelve
import uuid
import pickle
from random import SystemRandom
cryptogen = SystemRandom()
""" Helper functions """
# K-ARY CONSTRAINT GENERATION
# ###########################
# SINZ, Carsten. Towards an optimal CNF encoding of boolean cardinality constraints.
# CP, 2005, 3709. Jg., S. 827-831.
def next_var_index(start):
next_var = start
while(True):
yield next_var
next_var += 1
class s_index():
def __init__(self, start_index):
self.firstEnvVar = start_index
def next(self,i,j,k):
return self.firstEnvVar + i*k +j
def gen_seq_circuit(k, input_indices, next_var_index_gen):
cnf_string = ''
s_index_gen = s_index(next_var_index_gen.next())
# write clauses of first partial sum (i.e. i=0)
cnf_string += (str(-input_indices[0]) + ' ' + str(s_index_gen.next(0,0,k)) + ' 0\n')
for i in range(1, k):
cnf_string += (str(-s_index_gen.next(0, i, k)) + ' 0\n')
# write clauses for general case (i.e. 0 < i < n-1)
for i in range(1, len(input_indices)-1):
cnf_string += (str(-input_indices[i]) + ' ' + str(s_index_gen.next(i, 0, k)) + ' 0\n')
cnf_string += (str(-s_index_gen.next(i-1, 0, k)) + ' ' + str(s_index_gen.next(i, 0, k)) + ' 0\n')
for u in range(1, k):
cnf_string += (str(-input_indices[i]) + ' ' + str(-s_index_gen.next(i-1, u-1, k)) + ' ' + str(s_index_gen.next(i, u, k)) + ' 0\n')
cnf_string += (str(-s_index_gen.next(i-1, u, k)) + ' ' + str(s_index_gen.next(i, u, k)) + ' 0\n')
cnf_string += (str(-input_indices[i]) + ' ' + str(-s_index_gen.next(i-1, k-1, k)) + ' 0\n')
# last clause for last variable
cnf_string += (str(-input_indices[-1]) + ' ' + str(-s_index_gen.next(len(input_indices)-2, k-1, k)) + ' 0\n')
return (cnf_string, (len(input_indices)-1)*k, 2*len(input_indices)*k + len(input_indices) - 3*k - 1)
# K=2 clause GENERATION
# #####################
def gen_at_most_2_constraints(vars, start_var):
constraint_string = ''
used_clauses = 0
used_vars = 0
index_gen = next_var_index(start_var)
circuit = gen_seq_circuit(2, vars, index_gen)
constraint_string += circuit[0]
used_clauses += circuit[2]
used_vars += circuit[1]
start_var += circuit[1]
return [constraint_string, used_clauses, used_vars, start_var]
def gen_at_least_2_constraints(vars, start_var):
k = len(vars) - 2
vars = [-var for var in vars]
constraint_string = ''
used_clauses = 0
used_vars = 0
index_gen = next_var_index(start_var)
circuit = gen_seq_circuit(k, vars, index_gen)
constraint_string += circuit[0]
used_clauses += circuit[2]
used_vars += circuit[1]
start_var += circuit[1]
return [constraint_string, used_clauses, used_vars, start_var]
# Adjacency conflicts
# ###################
def get_all_adjacency_conflicts_4_neighborhood(N, X):
conflicts = set()
for x in range(N):
for y in range(N):
if x < (N-1):
conflicts.add(((x,y),(x+1,y)))
if y < (N-1):
conflicts.add(((x,y),(x,y+1)))
cnf = '' # slow string appends
for (var_a, var_b) in conflicts:
var_a_ = X[var_a]
var_b_ = X[var_b]
cnf += '-' + var_a_ + ' ' + '-' + var_b_ + ' 0 \n'
return cnf, len(conflicts)
# Build SAT-CNF
#############
def build_cnf(N, verbose=False):
var_counter = count(1)
N_CLAUSES = 0
X = np.zeros((N, N), dtype=object)
for a in range(N):
for b in range(N):
X[a,b] = str(next(var_counter))
# Adjacency constraints
CNF, N_CLAUSES = get_all_adjacency_conflicts_4_neighborhood(N, X)
# k=2 constraints
NEXT_VAR = N*N+1
for row in range(N):
constraint_string, used_clauses, used_vars, NEXT_VAR = gen_at_most_2_constraints(X[row, :].astype(int).tolist(), NEXT_VAR)
N_CLAUSES += used_clauses
CNF += constraint_string
constraint_string, used_clauses, used_vars, NEXT_VAR = gen_at_least_2_constraints(X[row, :].astype(int).tolist(), NEXT_VAR)
N_CLAUSES += used_clauses
CNF += constraint_string
for col in range(N):
constraint_string, used_clauses, used_vars, NEXT_VAR = gen_at_most_2_constraints(X[:, col].astype(int).tolist(), NEXT_VAR)
N_CLAUSES += used_clauses
CNF += constraint_string
constraint_string, used_clauses, used_vars, NEXT_VAR = gen_at_least_2_constraints(X[:, col].astype(int).tolist(), NEXT_VAR)
N_CLAUSES += used_clauses
CNF += constraint_string
# build final cnf
CNF = 'p cnf ' + str(NEXT_VAR-1) + ' ' + str(N_CLAUSES) + '\n' + CNF
return X, CNF, NEXT_VAR-1
# External tools
# ##############
def get_random_xor_problem(CNF_IN_fp, N_DEC_VARS, N_ALL_VARS, s, min_l, max_l):
# .cnf not part of arg!
p = subprocess.Popen(['./gen-wff', CNF_IN_fp,
str(N_DEC_VARS), str(N_ALL_VARS),
str(s), str(min_l), str(max_l), 'xored'],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
result = p.communicate()
os.remove(CNF_IN_fp + '-str-xored.xor') # file not needed
return CNF_IN_fp + '-str-xored.cnf'
def solve(CNF_IN_fp, N_DEC_VARS):
seed = cryptogen.randint(0, 2147483647) # actually no reason to do it; but can't hurt either
p = subprocess.Popen(["./cryptominisat5", '-t', '4', '-r', str(seed), CNF_IN_fp], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
result = p.communicate()[0]
sat_line = result.find('s SATISFIABLE')
if sat_line != -1:
# solution found!
vars = parse_solution(result)[:N_DEC_VARS]
# forbid solution (DeMorgan)
negated_vars = list(map(lambda x: x*(-1), vars))
with open(CNF_IN_fp, 'a') as f:
f.write( (str(negated_vars)[1:-1] + ' 0\n').replace(',', ''))
# assume solve is treating last constraint despite not changing header!
# solve again
seed = cryptogen.randint(0, 2147483647)
p = subprocess.Popen(["./cryptominisat5", '-t', '4', '-r', str(seed), CNF_IN_fp], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
result = p.communicate()[0]
sat_line = result.find('s SATISFIABLE')
if sat_line != -1:
os.remove(CNF_IN_fp) # not needed anymore
return True, False, None
else:
return True, True, vars
else:
return False, False, None
def parse_solution(output):
# assumes there is one
vars = []
for line in output.split("\n"):
if line:
if line[0] == 'v':
line_vars = list(map(lambda x: int(x), line.split()[1:]))
vars.extend(line_vars)
return vars
# Core-algorithm
# ##############
def xorsample(X, CNF_IN_fp, N_DEC_VARS, N_VARS, s, min_l, max_l):
start_time = time()
while True:
# add s random XOR constraints to F
xored_cnf_fp = get_random_xor_problem(CNF_IN_fp, N_DEC_VARS, N_VARS, s, min_l, max_l)
state_lvl1, state_lvl2, var_sol = solve(xored_cnf_fp, N_DEC_VARS)
print('------------')
if state_lvl1 and state_lvl2:
print('FOUND')
d = shelve.open('N_15_70_4_6_TO_PLOT')
d[str(uuid.uuid4())] = (pickle.dumps(var_sol), time() - start_time)
d.close()
return True
else:
if state_lvl1:
print('sol not unique')
else:
print('no sol found')
print('------------')
""" Run """
N = 15
N_DEC_VARS = N*N
X, CNF, N_VARS = build_cnf(N)
with open('my_problem.cnf', 'w') as f:
f.write(CNF)
counter = 0
while True:
print('sample: ', counter)
xorsample(X, 'my_problem', N_DEC_VARS, N_VARS, 70, 4, 6)
counter += 1
Output will look like (removed some warnings):
------------
no sol found
------------
------------
no sol found
------------
------------
no sol found
------------
------------
sol not unique
------------
------------
FOUND
Core: CNF-formulation
We introduce one variable for every cell of the matrix. N=20 means 400 binary-variables.
Adjancency:
Precalculate all symmetry-reduced conflicts and add conflict-clauses.
Basic theory:
a -> !b
<->
!a v !b (propositional logic)
Row/Col-wise Cardinality:
This is tough to express in CNF and naive approaches need an exponential number
of constraints.
We use some adder-circuit based encoding (SINZ, Carsten. Towards an optimal CNF encoding of boolean cardinality constraints) which introduces new auxiliary-variables.
Remark:
sum(var_set) <= k
<->
sum(negated(var_set)) >= len(var_set) - k
These SAT-encodings can be put into exact model-counters (for small N; e.g. < 9). The number of solutions equals Ante's results, which is a strong indication for a correct transformation!
There are also interesting approximate model-counters (also heavily based on xor-constraints) like approxMC which shows one more thing we can do with the SAT-formulation. But in practice i have not been able to use these (approxMC = autoconf; no comment).
Other experiments
I did also build a version using pblib, to use more powerful cardinality-formulations
for the SAT-CNF formulation. I did not try to use the C++-based API, but only the reduced pbencoder, which automatically selects some best encoding, which was way worse than my encoding used above (which is best is still a research-problem; often even redundant-constraints can help).
Empirical analysis
For the sake of obtaining some sample-size (given my patience), i only computed samples for N=15. In this case we used:
N=70 xors
L,U = 4,6
I also computed some samples for N=20 with (100,3,6), but this takes a few mins and we reduced the lower bound!
Visualization
Here some animation (strengthening my love-hate relationship with matplotlib):
Edit: And a (reduced) comparison to brute-force uniform-sampling with N=5 (NXOR,L,U = 4, 10, 30):
(I have not yet decided on the addition of the plotting-code. It's as ugly as the above one and people might look too much into my statistical shambles; normalizations and co.)
Theory
Statistical analysis is probably hard to do as the underlying problem is of such combinatoric nature. It's even not entirely obvious how that final cell-PDF should look like. In the case of N=odd, it's probably non-uniform and looks like a chess-board (i did brute-force check N=5 to observe this).
One thing we can be sure about (imho): symmetry!
Given a cell-PDF matrix, we should expect, that the matrix is symmetric (A = A.T).
This is checked in the visualization and the euclidean-norm of differences over time is plotted.
We can do the same on some other observation: observed pairings.
For N=3, we can observe the following pairs:
0,1
0,2
1,2
Now we can do this per-row and per-column and should expect symmetry too!
Sadly, it's probably not easy to say something about the variance and therefore the needed samples to speak about confidence!
Observation
According to my simplified perception, current-samples and the cell-PDF look good, although convergence is not achieved yet (or we are far away from uniformity).
The more important aspect are probably the two norms, nicely decreasing towards 0.
(yes; one could tune some algorithm for that by transposing with prob=0.5; but this is not done here as it would defeat it's purpose).
Potential next steps
Tune parameters
Check out the approach using #SAT-solvers / Model-counters instead of SAT-solvers
Try different CNF-formulations, especially in regards to cardinality-encodings and xor-encodings
XorSample is by default using tseitin-like encoding to get around exponentially grow
for smaller xors (as used) it might be a good idea to use naive encoding (which propagates faster)
XorSample supports that in theory; but the script's work differently in practice
Cryptominisat is known for dedicated XOR-handling (as it was build for analyzing cryptography including many xors) and might gain something by naive encoding (as inferring xors from blown-up CNFs is much harder)
More statistical-analysis
Get rid of XorSample scripts (shell + perl...)
Summary
The approach is very general
This code produces feasible samples
It should be not hard to prove, that every feasible solution can be sampled
Others have proven theoretical guarantees for uniformity for some params
does not hold for our params
Others have empirically / theoretically analyzed smaller parameters (in use here)
(Updated test results, example run-through and code snippets below.)
You can use dynamic programming to calculate the number of solutions resulting from every state (in a much more efficient way than a brute-force algorithm), and use those (pre-calculated) values to create equiprobable random solutions.
Consider the example of a 7x7 matrix; at the start, the state is:
0,0,0,0,0,0,0
meaning that there are seven adjacent unused columns. After adding two ones to the first row, the state could be e.g.:
0,1,0,0,1,0,0
with two columns that now have a one in them. After adding ones to the second row, the state could be e.g.:
0,1,1,0,1,0,1
After three rows are filled, there is a possibility that a column will have its maximum of two ones; this effectively splits the matrix into two independent zones:
1,1,1,0,2,0,1 -> 1,1,1,0 + 0,1
These zones are independent in the sense that the no-adjacent-ones rule has no effect when adding ones to different zones, and the order of the zones has no effect on the number of solutions.
In order to use these states as signatures for types of solutions, we have to transform them into a canonical notation. First, we have to take into account the fact that columns with only 1 one in them may be unusable in the next row, because they contain a one in the current row. So instead of a binary notation, we have to use a ternary notation, e.g.:
2,1,1,0 + 0,1
where the 2 means that this column was used in the current row (and not that there are 2 ones in the column). At the next step, we should then convert the twos back into ones.
Additionally, we can also mirror the seperate groups to put them into their lexicographically smallest notation:
2,1,1,0 + 0,1 -> 0,1,1,2 + 0,1
Lastly, we sort the seperate groups from small to large, and then lexicographically, so that a state in a larger matrix may be e.g.:
0,0 + 0,1 + 0,0,2 + 0,1,0 + 0,1,0,1
Then, when calculating the number of solutions resulting from each state, we can use memoization using the canonical notation of each state as a key.
Creating a dictionary of the states and the number of solutions for each of them only needs to be done once, and a table for larger matrices can probably be used for smaller matrices too.
Practically, you'd generate a random number between 0 and the total number of solutions, and then for every row, you'd look at the different states you could create from the current state, look at the number of unique solutions each one would generate, and see which option leads to the solution that corresponds with your randomly generated number.
Note that every state and the corresponding key can only occur in a particular row, so you can store the keys in seperate dictionaries per row.
TEST RESULTS
A first test using unoptimized JavaScript gave very promising results. With dynamic programming, calculating the number of solutions for a 10x10 matrix now takes a second, where a brute-force algorithm took several hours (and this is the part of the algorithm that only needs to be done once). The size of the dictionary with the signatures and numbers of solutions grows with a diminishing factor approaching 2.5 for each step in size; the time to generate it grows with a factor of around 3.
These are the number of solutions, states, signatures (total size of the dictionaries), and maximum number of signatures per row (largest dictionary per row) that are created:
size unique solutions states signatures max/row
4x4 2 9 6 2
5x5 16 73 26 8
6x6 722 514 107 40
7x7 33,988 2,870 411 152
8x8 2,215,764 13,485 1,411 596
9x9 179,431,924 56,375 4,510 1,983
10x10 17,849,077,140 218,038 13,453 5,672
11x11 2,138,979,146,276 801,266 38,314 14,491
12x12 304,243,884,374,412 2,847,885 104,764 35,803
13x13 50,702,643,217,809,908 9,901,431 278,561 96,414
14x14 9,789,567,606,147,948,364 33,911,578 723,306 238,359
15x15 2,168,538,331,223,656,364,084 114,897,838 1,845,861 548,409
16x16 546,386,962,452,256,865,969,596 ... 4,952,501 1,444,487
17x17 155,420,047,516,794,379,573,558,433 12,837,870 3,754,040
18x18 48,614,566,676,379,251,956,711,945,475 31,452,747 8,992,972
19x19 17,139,174,923,928,277,182,879,888,254,495 74,818,773 20,929,008
20x20 6,688,262,914,418,168,812,086,412,204,858,650 175,678,000 50,094,203
(Additional results obtained with C++, using a simple 128-bit integer implementation. To count the states, the code had to be run using each state as a seperate signature, which I was unable to do for the largest sizes. )
EXAMPLE
The dictionary for a 5x5 matrix looks like this:
row 0: 00000 -> 16 row 3: 101 -> 0
1112 -> 1
row 1: 20002 -> 2 1121 -> 1
00202 -> 4 1+01 -> 0
02002 -> 2 11+12 -> 2
02020 -> 2 1+121 -> 1
0+1+1 -> 0
row 2: 10212 -> 1 1+112 -> 1
12012 -> 1
12021 -> 2 row 4: 0 -> 0
12102 -> 1 11 -> 0
21012 -> 0 12 -> 0
02121 -> 3 1+1 -> 1
01212 -> 1 1+2 -> 0
The total number of solutions is 16; if we randomly pick a number from 0 to 15, e.g. 13, we can find the corresponding (i.e. the 14th) solution like this:
state: 00000
options: 10100 10010 10001 01010 01001 00101
signature: 00202 02002 20002 02020 02002 00202
solutions: 4 2 2 2 2 4
This tells us that the 14th solution is the 2nd solution of option 00101. The next step is:
state: 00101
options: 10010 01010
signature: 12102 02121
solutions: 1 3
This tells us that the 2nd solution is the 1st solution of option 01010. The next step is:
state: 01111
options: 10100 10001 00101
signature: 11+12 1112 1+01
solutions: 2 1 0
This tells us that the 1st solution is the 1st solution of option 10100. The next step is:
state: 11211
options: 01010 01001
signature: 1+1 1+1
solutions: 1 1
This tells us that the 1st solutions is the 1st solution of option 01010. The last step is:
state: 12221
options: 10001
And the 5x5 matrix corresponding to randomly chosen number 13 is:
0 0 1 0 1
0 1 0 1 0
1 0 1 0 0
0 1 0 1 0
1 0 0 0 1
And here's a quick'n'dirty code example; run the snippet to generate the signature and solution count dictionary, and generate a random 10x10 matrix (it takes a second to generate the dictionary; once that is done, it generates random solutions in half a millisecond):
function signature(state, prev) {
var zones = [], zone = [];
for (var i = 0; i < state.length; i++) {
if (state[i] == 2) {
if (zone.length) zones.push(mirror(zone));
zone = [];
}
else if (prev[i]) zone.push(3);
else zone.push(state[i]);
}
if (zone.length) zones.push(mirror(zone));
zones.sort(function(a,b) {return a.length - b.length || a - b;});
return zones.length ? zones.join("2") : "2";
function mirror(zone) {
var ltr = zone.join('');
zone.reverse();
var rtl = zone.join('');
return (ltr < rtl) ? ltr : rtl;
}
}
function memoize(n) {
var memo = [], empty = [];
for (var i = 0; i <= n; i++) memo[i] = [];
for (var i = 0; i < n; i++) empty[i] = 0;
memo[0][signature(empty, empty)] = next_row(empty, empty, 1);
return memo;
function next_row(state, prev, row) {
if (row > n) return 1;
var solutions = 0;
for (var i = 0; i < n - 2; i++) {
if (state[i] == 2 || prev[i] == 1) continue;
for (var j = i + 2; j < n; j++) {
if (state[j] == 2 || prev[j] == 1) continue;
var s = state.slice(), p = empty.slice();
++s[i]; ++s[j]; ++p[i]; ++p[j];
var sig = signature(s, p);
var sol = memo[row][sig];
if (sol == undefined)
memo[row][sig] = sol = next_row(s, p, row + 1);
solutions += sol;
}
}
return solutions;
}
}
function random_matrix(n, memo) {
var matrix = [], empty = [], state = [], prev = [];
for (var i = 0; i < n; i++) empty[i] = state[i] = prev[i] = 0;
var total = memo[0][signature(empty, empty)];
var pick = Math.floor(Math.random() * total);
document.write("solution " + pick.toLocaleString('en-US') +
" from a total of " + total.toLocaleString('en-US') + "<br>");
for (var row = 1; row <= n; row++) {
var options = find_options(state, prev);
for (var i in options) {
var state_copy = state.slice();
for (var j in state_copy) state_copy[j] += options[i][j];
var sig = signature(state_copy, options[i]);
var solutions = memo[row][sig];
if (pick < solutions) {
matrix.push(options[i].slice());
prev = options[i].slice();
state = state_copy.slice();
break;
}
else pick -= solutions;
}
}
return matrix;
function find_options(state, prev) {
var options = [];
for (var i = 0; i < n - 2; i++) {
if (state[i] == 2 || prev[i] == 1) continue;
for (var j = i + 2; j < n; j++) {
if (state[j] == 2 || prev[j] == 1) continue;
var option = empty.slice();
++option[i]; ++option[j];
options.push(option);
}
}
return options;
}
}
var size = 10;
var memo = memoize(size);
var matrix = random_matrix(size, memo);
for (var row in matrix) document.write(matrix[row] + "<br>");
The code snippet below shows the dictionary of signatures and solution counts for a matrix of size 10x10. I've used a slightly different signature format from the explanation above: the zones are delimited by a '2' instead of a plus sign, and a column which has a one in the previous row is marked with a '3' instead of a '2'. This shows how the keys could be stored in a file as integers with 2×N bits (padded with 2's).
function signature(state, prev) {
var zones = [], zone = [];
for (var i = 0; i < state.length; i++) {
if (state[i] == 2) {
if (zone.length) zones.push(mirror(zone));
zone = [];
}
else if (prev[i]) zone.push(3);
else zone.push(state[i]);
}
if (zone.length) zones.push(mirror(zone));
zones.sort(function(a,b) {return a.length - b.length || a - b;});
return zones.length ? zones.join("2") : "2";
function mirror(zone) {
var ltr = zone.join('');
zone.reverse();
var rtl = zone.join('');
return (ltr < rtl) ? ltr : rtl;
}
}
function memoize(n) {
var memo = [], empty = [];
for (var i = 0; i <= n; i++) memo[i] = [];
for (var i = 0; i < n; i++) empty[i] = 0;
memo[0][signature(empty, empty)] = next_row(empty, empty, 1);
return memo;
function next_row(state, prev, row) {
if (row > n) return 1;
var solutions = 0;
for (var i = 0; i < n - 2; i++) {
if (state[i] == 2 || prev[i] == 1) continue;
for (var j = i + 2; j < n; j++) {
if (state[j] == 2 || prev[j] == 1) continue;
var s = state.slice(), p = empty.slice();
++s[i]; ++s[j]; ++p[i]; ++p[j];
var sig = signature(s, p);
var sol = memo[row][sig];
if (sol == undefined)
memo[row][sig] = sol = next_row(s, p, row + 1);
solutions += sol;
}
}
return solutions;
}
}
var memo = memoize(10);
for (var i in memo) {
document.write("row " + i + ":<br>");
for (var j in memo[i]) {
document.write(""" + j + "": " + memo[i][j] + "<br>");
}
}
Just few thoughts. Number of matrices satisfying conditions for n <= 10:
3 0
4 2
5 16
6 722
7 33988
8 2215764
9 179431924
10 17849077140
Unfortunatelly there is no sequence with these numbers in OEIS.
There is one similar (A001499), without condition for neighbouring one's. Number of nxn matrices in this case is 'of order' as A001499's number of (n-1)x(n-1) matrices. That is to be expected since number
of ways to fill one row in this case, position 2 one's in n places with at least one zero between them is ((n-1) choose 2). Same as to position 2 one's in (n-1) places without the restriction.
I don't think there is an easy connection between these matrix of order n and A001499 matrix of order n-1, meaning that if we have A001499 matrix than we can construct some of these matrices.
With this, for n=20, number of matrices is >10^30. Quite a lot :-/
This solution use recursion in order to set the cell of the matrix one by one. If the random walk finish with an impossible solution then we rollback one step in the tree and we continue the random walk.
The algorithm is efficient and i think that the generated data are highly equiprobable.
package rndsqmatrix;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.stream.IntStream;
public class RndSqMatrix {
/**
* Generate a random matrix
* #param size the size of the matrix
* #return the matrix encoded in 1d array i=(x+y*size)
*/
public static int[] generate(final int size) {
return generate(size, new int[size * size], new int[size],
new int[size]);
}
/**
* Build a matrix recursivly with a random walk
* #param size the size of the matrix
* #param matrix the matrix encoded in 1d array i=(x+y*size)
* #param rowSum
* #param colSum
* #return
*/
private static int[] generate(final int size, final int[] matrix,
final int[] rowSum, final int[] colSum) {
// generate list of valid positions
final List<Integer> positions = new ArrayList();
for (int y = 0; y < size; y++) {
if (rowSum[y] < 2) {
for (int x = 0; x < size; x++) {
if (colSum[x] < 2) {
final int p = x + y * size;
if (matrix[p] == 0
&& (x == 0 || matrix[p - 1] == 0)
&& (x == size - 1 || matrix[p + 1] == 0)
&& (y == 0 || matrix[p - size] == 0)
&& (y == size - 1 || matrix[p + size] == 0)) {
positions.add(p);
}
}
}
}
}
// no valid positions ?
if (positions.isEmpty()) {
// if the matrix is incomplete => return null
for (int i = 0; i < size; i++) {
if (rowSum[i] != 2 || colSum[i] != 2) {
return null;
}
}
// the matrix is complete => return it
return matrix;
}
// random walk
Collections.shuffle(positions);
for (int p : positions) {
// set '1' and continue recursivly the exploration
matrix[p] = 1;
rowSum[p / size]++;
colSum[p % size]++;
final int[] solMatrix = generate(size, matrix, rowSum, colSum);
if (solMatrix != null) {
return solMatrix;
}
// rollback
matrix[p] = 0;
rowSum[p / size]--;
colSum[p % size]--;
}
// we can't find a valid matrix from here => return null
return null;
}
public static void printMatrix(final int size, final int[] matrix) {
for (int y = 0; y < size; y++) {
for (int x = 0; x < size; x++) {
System.out.print(matrix[x + y * size]);
System.out.print(" ");
}
System.out.println();
}
}
public static void printStatistics(final int size, final int count) {
final int sumMatrix[] = new int[size * size];
for (int i = 0; i < count; i++) {
final int[] matrix = generate(size);
for (int j = 0; j < sumMatrix.length; j++) {
sumMatrix[j] += matrix[j];
}
}
printMatrix(size, sumMatrix);
}
public static void checkAlgorithm() {
final int size = 8;
final int count = 2215764;
final int divisor = 122;
final int sumMatrix[] = new int[size * size];
for (int i = 0; i < count/divisor ; i++) {
final int[] matrix = generate(size);
for (int j = 0; j < sumMatrix.length; j++) {
sumMatrix[j] += matrix[j];
}
}
int total = 0;
for(int i=0; i < sumMatrix.length; i++) {
total += sumMatrix[i];
}
final double factor = (double)total / (count/divisor);
System.out.println("Factor=" + factor + " (theory=16.0)");
}
public static void benchmark(final int size, final int count,
final boolean parallel) {
final long begin = System.currentTimeMillis();
if (!parallel) {
for (int i = 0; i < count; i++) {
generate(size);
}
} else {
IntStream.range(0, count).parallel().forEach(i -> generate(size));
}
final long end = System.currentTimeMillis();
System.out.println("rate="
+ (double) (end - begin) / count + "ms/matrix");
}
public static void main(String[] args) {
checkAlgorithm();
benchmark(8, 10000, true);
//printStatistics(8, 2215764/36);
printStatistics(8, 2215764);
}
}
The output is:
Factor=16.0 (theory=16.0)
rate=0.2835ms/matrix
552969 554643 552895 554632 555680 552753 554567 553389
554071 554847 553441 553315 553425 553883 554485 554061
554272 552633 555130 553699 553604 554298 553864 554028
554118 554299 553565 552986 553786 554473 553530 554771
554474 553604 554473 554231 553617 553556 553581 553992
554960 554572 552861 552732 553782 554039 553921 554661
553578 553253 555721 554235 554107 553676 553776 553182
553086 553677 553442 555698 553527 554850 553804 553444
Here is a very fast approach of generating the matrix row by row, written in Java:
public static void main(String[] args) throws Exception {
int n = 100;
Random rnd = new Random();
byte[] mat = new byte[n*n];
byte[] colCount = new byte[n];
//generate row by row
for (int x = 0; x < n; x++) {
//generate a random first bit
int b1 = rnd.nextInt(n);
while ( (x > 0 && mat[(x-1)*n + b1] == 1) || //not adjacent to the one above
(colCount[b1] == 2) //not in a column which has 2
) b1 = rnd.nextInt(n);
//generate a second bit, not equal to the first one
int b2 = rnd.nextInt(n);
while ( (b2 == b1) || //not the same as bit 1
(x > 0 && mat[(x-1)*n + b2] == 1) || //not adjacent to the one above
(colCount[b2] == 2) || //not in a column which has 2
(b2 == b1 - 1) || //not adjacent to b1
(b2 == b1 + 1)
) b2 = rnd.nextInt(n);
//fill the matrix values and increment column counts
mat[x*n + b1] = 1;
mat[x*n + b2] = 1;
colCount[b1]++;
colCount[b2]++;
}
String arr = Arrays.toString(mat).substring(1, n*n*3 - 1);
System.out.println(arr.replaceAll("(.{" + n*3 + "})", "$1\n"));
}
It essentially generates each a random row at a time. If the row will violate any of the conditions, it is generated again (again randomly). I believe this will satisfy condition 4 as well.
Adding a quick note that it will spin forever for N-s where there is no solutions (like N=3).

Find all possible combinations of a String representation of a number

Given a mapping:
A: 1
B: 2
C: 3
...
...
...
Z: 26
Find all possible ways a number can be represented. E.g. For an input: "121", we can represent it as:
ABA [using: 1 2 1]
LA [using: 12 1]
AU [using: 1 21]
I tried thinking about using some sort of a dynamic programming approach, but I am not sure how to proceed. I was asked this question in a technical interview.
Here is a solution I could think of, please let me know if this looks good:
A[i]: Total number of ways to represent the sub-array number[0..i-1] using the integer to alphabet mapping.
Solution [am I missing something?]:
A[0] = 1 // there is only 1 way to represent the subarray consisting of only 1 number
for(i = 1:A.size):
A[i] = A[i-1]
if(input[i-1]*10 + input[i] < 26):
A[i] += 1
end
end
print A[A.size-1]
To just get the count, the dynamic programming approach is pretty straight-forward:
A[0] = 1
for i = 1:n
A[i] = 0
if input[i-1] > 0 // avoid 0
A[i] += A[i-1];
if i > 1 && // avoid index-out-of-bounds on i = 1
10 <= (10*input[i-2] + input[i-1]) <= 26 // check that number is 10-26
A[i] += A[i-2];
If you instead want to list all representations, dynamic programming isn't particularly well-suited for this, you're better off with a simple recursive algorithm.
First off, we need to find an intuitive way to enumerate all the possibilities. My simple construction, is given below.
let us assume a simple way to represent your integer in string format.
a1 a2 a3 a4 ....an, for instance in 121 a1 -> 1 a2 -> 2, a3 -> 1
Now,
We need to find out number of possibilities of placing a + sign in between two characters. + is to mean characters concatenation here.
a1 - a2 - a3 - .... - an, - shows the places where '+' can be placed. So, number of positions is n - 1, where n is the string length.
Assume a position may or may not have a + symbol shall be represented as a bit.
So, this boils down to how many different bit strings are possible with the length of n-1, which is clearly 2^(n-1). Now in order to enumerate the possibilities go through every bit string and place right + signs in respective positions to get every representations,
For your example, 121
Four bit strings are possible 00 01 10 11
1 2 1
1 2 + 1
1 + 2 1
1 + 2 + 1
And if you see a character followed by a +, just add the next char with the current one and do it sequentially to get the representation,
x + y z a + b + c d
would be (x+y) z (a+b+c) d
Hope it helps.
And you will have to take care of edge cases where the size of some integer > 26, of course.
I think, recursive traverse through all possible combinations would do just fine:
mapping = {"1":"A", "2":"B", "3":"C", "4":"D", "5":"E", "6":"F", "7":"G",
"8":"H", "9":"I", "10":"J",
"11":"K", "12":"L", "13":"M", "14":"N", "15":"O", "16":"P",
"17":"Q", "18":"R", "19":"S", "20":"T", "21":"U", "22":"V", "23":"W",
"24":"A", "25":"Y", "26":"Z"}
def represent(A, B):
if A == B == '':
return [""]
ret = []
if A in mapping:
ret += [mapping[A] + r for r in represent(B, '')]
if len(A) > 1:
ret += represent(A[:-1], A[-1]+B)
return ret
print represent("121", "")
Assuming you only need to count the number of combinations.
Assuming 0 followed by an integer in [1,9] is not a valid concatenation, then a brute-force strategy would be:
Count(s,n)
x=0
if (s[n-1] is valid)
x=Count(s,n-1)
y=0
if (s[n-2] concat s[n-1] is valid)
y=Count(s,n-2)
return x+y
A better strategy would be to use divide-and-conquer:
Count(s,start,n)
if (len is even)
{
//split s into equal left and right part, total count is left count multiply right count
x=Count(s,start,n/2) + Count(s,start+n/2,n/2);
y=0;
if (s[start+len/2-1] concat s[start+len/2] is valid)
{
//if middle two charaters concatenation is valid
//count left of the middle two characters
//count right of the middle two characters
//multiply the two counts and add to existing count
y=Count(s,start,len/2-1)*Count(s,start+len/2+1,len/2-1);
}
return x+y;
}
else
{
//there are three cases here:
//case 1: if middle character is valid,
//then count everything to the left of the middle character,
//count everything to the right of the middle character,
//multiply the two, assign to x
x=...
//case 2: if middle character concatenates the one to the left is valid,
//then count everything to the left of these two characters
//count everything to the right of these two characters
//multiply the two, assign to y
y=...
//case 3: if middle character concatenates the one to the right is valid,
//then count everything to the left of these two characters
//count everything to the right of these two characters
//multiply the two, assign to z
z=...
return x+y+z;
}
The brute-force solution has time complexity of T(n)=T(n-1)+T(n-2)+O(1) which is exponential.
The divide-and-conquer solution has time complexity of T(n)=3T(n/2)+O(1) which is O(n**lg3).
Hope this is correct.
Something like this?
Haskell code:
import qualified Data.Map as M
import Data.Maybe (fromJust)
combs str = f str [] where
charMap = M.fromList $ zip (map show [1..]) ['A'..'Z']
f [] result = [reverse result]
f (x:xs) result
| null xs =
case M.lookup [x] charMap of
Nothing -> ["The character " ++ [x] ++ " is not in the map."]
Just a -> [reverse $ a:result]
| otherwise =
case M.lookup [x,head xs] charMap of
Just a -> f (tail xs) (a:result)
++ (f xs ((fromJust $ M.lookup [x] charMap):result))
Nothing -> case M.lookup [x] charMap of
Nothing -> ["The character " ++ [x]
++ " is not in the map."]
Just a -> f xs (a:result)
Output:
*Main> combs "121"
["LA","AU","ABA"]
Here is the solution based on my discussion here:
private static int decoder2(int[] input) {
int[] A = new int[input.length + 1];
A[0] = 1;
for(int i=1; i<input.length+1; i++) {
A[i] = 0;
if(input[i-1] > 0) {
A[i] += A[i-1];
}
if (i > 1 && (10*input[i-2] + input[i-1]) <= 26) {
A[i] += A[i-2];
}
System.out.println(A[i]);
}
return A[input.length];
}
Just us breadth-first search.
for instance 121
Start from the first integer,
consider 1 integer character first, map 1 to a, leave 21
then 2 integer character map 12 to L leave 1.
This problem can be done in o(fib(n+2)) time with a standard DP algorithm.
We have exactly n sub problems and button up we can solve each problem with size i in o(fib(i)) time.
Summing the series gives fib (n+2).
If you consider the question carefully you see that it is a Fibonacci series.
I took a standard Fibonacci code and just changed it to fit our conditions.
The space is obviously bound to the size of all solutions o(fib(n)).
Consider this pseudo code:
Map<Integer, String> mapping = new HashMap<Integer, String>();
List<String > iterative_fib_sequence(string input) {
int length = input.length;
if (length <= 1)
{
if (length==0)
{
return "";
}
else//input is a-j
{
return mapping.get(input);
}
}
List<String> b = new List<String>();
List<String> a = new List<String>(mapping.get(input.substring(0,0));
List<String> c = new List<String>();
for (int i = 1; i < length; ++i)
{
int dig2Prefix = input.substring(i-1, i); //Get a letter with 2 digit (k-z)
if (mapping.contains(dig2Prefix))
{
String word2Prefix = mapping.get(dig2Prefix);
foreach (String s in b)
{
c.Add(s.append(word2Prefix));
}
}
int dig1Prefix = input.substring(i, i); //Get a letter with 1 digit (a-j)
String word1Prefix = mapping.get(dig1Prefix);
foreach (String s in a)
{
c.Add(s.append(word1Prefix));
}
b = a;
a = c;
c = new List<String>();
}
return a;
}
old question but adding an answer so that one can find help
It took me some time to understand the solution to this problem – I refer accepted answer and #Karthikeyan's answer and the solution from geeksforgeeks and written my own code as below:
To understand my code first understand below examples:
we know, decodings([1, 2]) are "AB" or "L" and so decoding_counts([1, 2]) == 2
And, decodings([1, 2, 1]) are "ABA", "AU", "LA" and so decoding_counts([1, 2, 1]) == 3
using the above two examples let's evaluate decodings([1, 2, 1, 4]):
case:- "taking next digit as single digit"
taking 4 as single digit to decode to letter 'D', we get decodings([1, 2, 1, 4]) == decoding_counts([1, 2, 1]) because [1, 2, 1, 4] will be decode as "ABAD", "AUD", "LAD"
case:- "combining next digit with the previous digit"
combining 4 with previous 1 as 14 as a single to decode to letter N, we get decodings([1, 2, 1, 4]) == decoding_counts([1, 2]) because [1, 2, 1, 4] will be decode as "ABN" or "LN"
Below is my Python code, read comments
def decoding_counts(digits):
# defininig count as, counts[i] -> decoding_counts(digits[: i+1])
counts = [0] * len(digits)
counts[0] = 1
for i in xrange(1, len(digits)):
# case:- "taking next digit as single digit"
if digits[i] != 0: # `0` do not have mapping to any letter
counts[i] = counts[i -1]
# case:- "combining next digit with the previous digit"
combine = 10 * digits[i - 1] + digits[i]
if 10 <= combine <= 26: # two digits mappings
counts[i] += (1 if i < 2 else counts[i-2])
return counts[-1]
for digits in "13", "121", "1214", "1234121":
print digits, "-->", decoding_counts(map(int, digits))
outputs:
13 --> 2
121 --> 3
1214 --> 5
1234121 --> 9
note: I assumed that input digits do not start with 0 and only consists of 0-9 and have a sufficent length
For Swift, this is what I came up with. Basically, I converted the string into an array and goes through it, adding a space into different positions of this array, then appending them to another array for the second part, which should be easy after this is done.
//test case
let input = [1,2,2,1]
func combination(_ input: String) {
var arr = Array(input)
var possible = [String]()
//... means inclusive range
for i in 2...arr.count {
var temp = arr
//basically goes through it backwards so
// adding the space doesn't mess up the index
for j in (1..<i).reversed() {
temp.insert(" ", at: j)
possible.append(String(temp))
}
}
print(possible)
}
combination(input)
//prints:
//["1 221", "12 21", "1 2 21", "122 1", "12 2 1", "1 2 2 1"]
def stringCombinations(digits, i=0, s=''):
if i == len(digits):
print(s)
return
alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
total = 0
for j in range(i, min(i + 1, len(digits) - 1) + 1):
total = (total * 10) + digits[j]
if 0 < total <= 26:
stringCombinations(digits, j + 1, s + alphabet[total - 1])
if __name__ == '__main__':
digits = list()
n = input()
n.split()
d = list(n)
for i in d:
i = int(i)
digits.append(i)
print(digits)
stringCombinations(digits)

Resources