I have a probability problem, which I need to simulate in a reasonable amount of time. In simplified form, I have 30 unfair coins each with a different known probability. I then want to ask things like "what is the probability that exactly 12 will be heads?", or "what is the probability that AT LEAST 5 will be tails?".
I know basic probability theory, so I know I can enumerate all (30 choose x) possibilities, but that's not particularly scalable. The worst case (30 choose 15) has over 150 million combinations. Is there a better way to approach this problem from a computational standpoint?
Any help is greatly appreciated, thanks! :-)
You can use a dynamic programming approach.
For example, to calculate the probability of 12 heads out of 30 coins, let P(n, k) be the probability that there's k heads from the first n coins.
Then P(n, k) = p_n * P(n - 1, k - 1) + (1 - p_n) * P(n - 1, k)
(here p_i is the probability the i'th coin is heads).
You can now use this relation in a dynamic programming algorithm. Have a vector of 13 probabilities (that represent P(n - 1, i) for i in 0..12). Build a new vector of 13 for P(n, i) using the above recurrence relation. Repeat until n = 30. Of course, you start with the vector (1, 0, 0, 0, ...) for n=0 (since with no coins, you're sure to get no heads).
The worst case using this algorithm is O(n^2) rather than exponential.
This is actually an interesting problem. I was inspired to write a blog post about it covering in detail fair vs unfair coin tosses all the way to the OP's situation of having a different probability for each coin. You need a technique called dynamic programming to solve this problem in polynomial time.
General Problem: Given C, a series of n coins p1 to pn where pi represents the probability of the i-th coin coming up heads, what is the probability of k heads coming up from tossing all the coins?
This means solving the following recurrence relation:
P(n,k,C,i) = pi x P(n-1,k-1,C,i+1) + (1-pi) x P(n,k,C,i+1)
A Java code snippet that does this is:
private static void runDynamic() {
long start = System.nanoTime();
double[] probs = dynamic(0.2, 0.3, 0.4);
long end = System.nanoTime();
int total = 0;
for (int i = 0; i < probs.length; i++) {
System.out.printf("%d : %,.4f%n", i, probs[i]);
}
System.out.printf("%nDynamic ran for %d coinsin %,.3f ms%n%n",
coins.length, (end - start) / 1000000d);
}
private static double[] dynamic(double... coins) {
double[][] table = new double[coins.length + 2][];
for (int i = 0; i < table.length; i++) {
table[i] = new double[coins.length + 1];
}
table[1][coins.length] = 1.0d; // everything else is 0.0
for (int i = 0; i <= coins.length; i++) {
for (int j = coins.length - 1; j >= 0; j--) {
table[i + 1][j] = coins[j] * table[i][j + 1] +
(1 - coins[j]) * table[i + 1][j + 1];
}
}
double[] ret = new double[coins.length + 1];
for (int i = 0; i < ret.length; i++) {
ret[i] = table[i + 1][0];
}
return ret;
}
What this is doing is constructing a table that shows the probability that a sequence of coins from pi to pn contain k heads.
For a deeper introduction to binomial probability and a discussion on how to apply dynamic programming take a look at Coin Tosses, Binomials and Dynamic Programming.
Pseudocode:
procedure PROB(n,k,p)
/*
input: n - number of coins flipped
k - number of heads
p - list of probabilities for n-coins where p[i] is probability coin i will be heads
output: probability k-heads in n-flips
assumptions: 1 <= i <= n, i in [0,1], 0 <= k <= n, additions and multiplications of [0,1] numbers O(1)
*/
A = ()() //matrix
A[0][0] = 1 // probability no heads given no coins flipped = 100%
for i = 0 to k //O(k)
if i != 0 then A[i][i] = A[i-1][i-1] * p[i]
for j = i + 1 to n - k + i //O( n - k + 1 - (i + 1)) = O(n - k) = O(n)
if i != 0 then A[i][j] = p[j] * A[i-1][j-1] + (1-p[j]) * A[i][j-1]
otherwise A[i][j] = (1 - p[j]) * A[i][j-1]
return A[k][n] //probability k-heads given n-flips
Worst case = O(kn)
Related
Given two numbers n and k, find x, 1 <= x <= k that maximises the remainder n % x.
For example, n = 20 and k = 10 the solution is x = 7 because the remainder 20 % 7 = 6 is maximum.
My solution to this is :
int n, k;
cin >> n >> k;
int max = 0;
for(int i = 1; i <= k; ++i)
{
int xx = n - (n / i) * i; // or int xx = n % i;
if(max < xx)
max = xx;
}
cout << max << endl;
But my solution is O(k). Is there any more efficient solution to this?
Not asymptotically faster, but faster, simply by going backwards and stopping when you know that you cannot do better.
Assume k is less than n (otherwise just output k).
int max = 0;
for(int i = k; i > 0 ; --i)
{
int xx = n - (n / i) * i; // or int xx = n % i;
if(max < xx)
max = xx;
if (i < max)
break; // all remaining values will be smaller than max, so break out!
}
cout << max << endl;
(This can be further improved by doing the for loop as long as i > max, thus eliminating one conditional statement, but I wrote it this way to make it more obvious)
Also, check Garey and Johnson's Computers and Intractability book to make sure this is not NP-Complete (I am sure I remember some problem in that book that looks a lot like this). I'd do that before investing too much effort on trying to come up with better solutions.
This problem is equivalent to finding maximum of function f(x)=n%x in given range. Let's see how this function looks like:
It is obvious that we could get the maximum sooner if we start with x=k and then decrease x while it makes any sense (until x=max+1). Also this diagram shows that for x larger than sqrt(n) we don't need to decrease x sequentially. Instead we could jump immediately to preceding local maximum.
int maxmod(const int n, int k)
{
int max = 0;
while (k > max + 1 && k > 4.0 * std::sqrt(n))
{
max = std::max(max, n % k);
k = std::min(k - 1, 1 + n / (1 + n / k));
}
for (; k > max + 1; --k)
max = std::max(max, n % k);
return max;
}
Magic constant 4.0 allows to improve performance by decreasing number of iterations of the first (expensive) loop.
Worst case time complexity could be estimated as O(min(k, sqrt(n))). But for large enough k this estimation is probably too pessimistic: we could find maximum much sooner, and if k is significantly greater than sqrt(n) we need only 1 or 2 iterations to find it.
I did some tests to determine how many iterations are needed in the worst case for different values of n:
n max.iterations (both/loop1/loop2)
10^1..10^2 11 2 11
10^2..10^3 20 3 20
10^3..10^4 42 5 42
10^4..10^5 94 11 94
10^5..10^6 196 23 196
up to 10^7 379 43 379
up to 10^8 722 83 722
up to 10^9 1269 157 1269
Growth rate is noticeably better than O(sqrt(n)).
For k > n the problem is trivial (take x = n+1).
For k < n, think about the graph of remainders n % x. It looks the same for all n: the remainders fall to zero at the harmonics of n: n/2, n/3, n/4, after which they jump up, then smoothly decrease towards the next harmonic.
The solution is the rightmost local maximum below k. As a formula x = n//((n//k)+1)+1 (where // is integer division).
waves hands around
No value of x which is a factor of n can produce the maximum n%x, since if x is a factor of n then n%x=0.
Therefore, you would like a procedure which avoids considering any x that is a factor of n. But this means you want an easy way to know if x is a factor. If that were possible you would be able to do an easy prime factorization.
Since there is not a known easy way to do prime factorization there cannot be an "easy" way to solve your problem (I don't think you're going to find a single formula, some kind of search will be necessary).
That said, the prime factorization literature has cunning ways of getting factors quickly relative to a naive search, so perhaps it can be leveraged to answer your question.
Nice little puzzle!
Starting with the two trivial cases.
for n < k: any x s.t. n < x <= k solves.
for n = k: x = floor(k / 2) + 1 solves.
My attempts.
for n > k:
x = n
while (x > k) {
x = ceil(n / 2)
}
^---- Did not work.
x = floor(float(n) / (floor(float(n) / k) + 1)) + 1
x = ceil(float(n) / (floor(float(n) / k) + 1)) - 1
^---- "Close" (whatever that means), but did not work.
My pride inclines me to mention that I was first to utilize the greatest k-bounded harmonic, given by 1.
Solution.
Inline with other answers I simply check harmonics (term courtesy of #ColonelPanic) of n less than k, limiting by the present maximum value (courtesy of #TheGreatContini). This is the best of both worlds and I've tested with random integers between 0 and 10000000 with success.
int maximalModulus(int n, int k) {
if (n < k) {
return n;
}
else if (n == k) {
return n % (k / 2 + 1);
}
else {
int max = -1;
int i = (n / k) + 1;
int x = 1;
while (x > max + 1) {
x = (n / i) + 1;
if (n%x > max) {
max = n%x;
}
++i;
}
return max;
}
}
Performance tests:
http://cpp.sh/72q6
Sample output:
Average number of loops:
bruteForce: 516
theGreatContini: 242.8
evgenyKluev: 2.28
maximalModulus: 1.36 // My solution
I'm wrong for sure, but it looks to me that it depends on if n < k or not.
I mean, if n < k, n%(n+1) gives you the maximum, so x = (n+1).
Well, on the other hand, you can start from j = k and go back evaluating n%j until it's equal to n, thus x = j is what you are looking for and you'll get it in max k steps... Too much, is it?
Okay, we want to know divisor that gives maximum remainder;
let n be a number to be divided and i be the divisor.
we are interested to find the maximum remainder when n is divided by i, for all i<n.
we know that, remainder = n - (n/i) * i //equivalent to n%i
If we observe the above equation to get maximum remainder we have to minimize (n/i)*i
minimum of n/i for any i<n is 1.
Note that, n/i == 1, for i<n, if and only if i>n/2
now we have, i>n/2.
The least possible value greater than n/2 is n/2+1.
Therefore, the divisor that gives maximum remainder, i = n/2+1
Here is the code in C++
#include <iostream>
using namespace std;
int maxRemainderDivisor(int n){
n = n>>1;
return n+1;
}
int main(){
int n;
cin>>n;
cout<<maxRemainderDivisor(n)<<endl;
return 0;
}
Time complexity: O(1)
For the "rod cutting" problem:
Given a rod of length n inches and an array of prices that contains prices of all pieces of size smaller than n. Determine the maximum value obtainable by cutting up the rod and selling the pieces. [link]
Introduction to Algorithms (CLRS) page 366 gives this pseudocode for a bottom-up (dynamic programming) approach:
1. BOTTOM-UP-CUT-ROD(p, n)
2. let r[0 to n]be a new array .
3. r[0] = 0
4. for j = 1 to n
5. q = -infinity
6. for i = 1 to j
7. q = max(q, p[i] + r[j - i])
8. r[j] = q
9. return r[n]
Now, I'm having trouble understanding the logic behind line 6. Why are they doing max(q, p[i] + r[j - i]) instead of max(q, r[i] + r[j - i])? Since, this is a bottom up approach, we'll compute r[1] first and then r[2], r[3]... so on. This means while computing r[x] we are guaranteed to have r[x - 1].
r[x] denotes the max value we can get for a rod of length x (after cutting it up to maximize profit) whereas p[x] denotes the price of a single piece of rod of length x. Lines 3 - 8 are computing the value r[j] for j = 1 to n and lines 5 - 6 are computing the maximum price we can sell a rod of length j for by considering all the possible cuts. So, how does it ever make sense to use p[i] instead of r[i] in line 6. If trying to find the max price for a rod after we cut it at length = i, shouldn't we add the prices of r[i] and r[j - 1]?
I've used this logic to write a Java code and it seems to give the correct output for a number of test cases I've tried. Am I missing some cases in which where my code produces incorrect / inefficient solutions? Please help me out. Thanks!
class Solution {
private static int cost(int[] prices, int n) {
if (n == 0) {
return 0;
}
int[] maxPrice = new int[n];
for (int i = 0; i < n; i++) {
maxPrice[i] = -1;
}
for (int i = 1; i <= n; i++) {
int q = Integer.MIN_VALUE;
if (i <= prices.length) {
q = prices[i - 1];
}
for (int j = i - 1; j >= (n / 2); j--) {
q = Math.max(q, maxPrice[j - 1] + maxPrice[i - j - 1]);
}
maxPrice[i - 1] = q;
}
return maxPrice[n - 1];
}
public static void main(String[] args) {
int[] prices = {1, 5, 8, 9, 10, 17, 17, 20};
System.out.println(cost(prices, 8));
}
}
They should be equivalent.
The intuition behind the CLRS approach is that they are trying to find the single "last cut", assuming that the last piece of rod has length i and thus has value exactly p[i]. In this formulation, the "last piece" of length i is not cut further, but the remainder of length j-i is.
Your approach considers all splits of the rod into two pieces, where each of the two parts can be cut further. This considers a superset of cases compared to the CLRS approach.
Both approaches are correct and have the same asymptotic complexity. However, I would argue that the CLRS solution is more "canonical" because it more closely matches a common form of DP solution where you only consider the last "thing" (in this case, the last piece of uncut rod).
I guess both of the approach are correct.
before we prove both of them are correct lets define what exactly each approach does
p[i] + r[j - i] will give you the max value you can obtain from a rod of length j and of the piece is of size "i"(cannot divide that piece further)
r[i] + r[j-i] will give you the max value you can obtain from a rod of length i and the first cut was made at length "i"(can devide both the pieces further)
Now consider we have a rod of length X and the solution set will contain piece of length k
and since k is 0 < k < X you will find the max value at p[k] + r[X-k] in the first approach
and in the second approach you can find the same result with r[k] + r[X-k] since we know that r[k] will be >= p[k]
But in you approach you can get the result much faster(half of the time) since you are slicing the rod from both ends
so in you approach you can run the inner loop for half of the length should be good.
But I think in you code there is a bug in inner for loop
it should be j >= (i / 2) instead of j >= (n / 2)
Let m, n be integers such that 0<= m,n< N.
Define:
Algorithm A: Computes m + n in time O(A(N))
Algorithm B: Computes m*n in time O(B(N))
Algorithm C: Computes m mod n in time O(C(N))
Using any combination of algorithms A, B and C describe an algorithm for N X N matrix addition and matrix multiplication with entries in Z/NZ. Also indicate the algorithm's run time big-O notation.
Attempt at solution:
For N X N addition in Z/NZ:
Let A, and B be N X N matrices in Z/NZ with entries a_{ij} and b_{ij} such that i,j in {0,1,...,N} where i represents the row and j represents the column in the matrix. Also, let A + B = C
Step 1. Run Algorithm A to get a_{ij} + b_{ij} = c_{ij} in time O(A(N))
Step 2. Run Algorithm C to get c_{ij} mod N in time O(C(N))
Repeat Steps 1 and 2 for all i,j in {0,1,...,N}.
This means that we have to repeat the steps 1,2 N^2 times. So the total run time is estimated by
N^2[ O(A(N)) + O(C(N)) ] = O(N^2 A(N)) + O(N^2 C(N)) = O(|N^2 A(N)| + |(N^2 C(N)|).
For multiplication algorithm I just replaced step 1 by Algorithm B and got the total run time to beO(|N^2 B(N)| + |(N^2 C(N)|) just like above.
Please tell me if I am approaching this problem correctly, especially with the big-O notation.
Thanks.
Your algorithm for matrix multiplication is wrong, and will yield a wrong answer, since A*B_{i,j} != A_{i,j} * B_{i,j} (with exception for some unique cases like zero matrix)
I assume the goal of the question is not to implement an efficient matrix multiplication, since it's a hard and still studied problem, so I will answer for the naive implementation of matrix multiplication.
For any indices i,j:
(AB)_{i,j} = Sum( A_{i,k} * B_{k,j}) =
= A_{i,1} * B_{1,j} + A_{i,2} * B_{2,j} + ... + A_{i,k} * B_{k,j} + ... + A_{i,n} * B_{n,j}
As you can see, for each pair i,j there are n multiplications and n-1 additionsׄ Regarding the amount of invokations of C - it depends if you need to invoke it after each addition, or only once when you are done (it really depends on how many bits you have to represent each number), so for each pair of i,j - you might need it anywhere from once to 2n-1 invokations.
This gives us total complexity of (assuming 2n-1 modolus for each (i,j) pair, if less are needed as explained above - adjust accordingly):
O(n^3*A + n^3*B + n^3*C)
As a side note, a good sanity check that shows your algorithm is indeed incorrect - it is proven that matrix multiplication cannot be done better than Omega(n^2 logn) (Raz,2002), and best current implementation is ~O(n^2.3)
#include <stdio.h>
void main()
{
int i, j;
int a[3][3], b[3][3];
printf("enter elements for 1 matrix\n");
for (i = 0; i < 3; i++)
{
for (j = 0; j < 3; j++)
{
scanf("%d", &a[i][j]);
}
printf("\n");
}
printf("enter elements for 2 matrix\n");
for (i = 0; i < 3; i++)
{
for (j = 0; j < 3; j++)
{
scanf("%d", &b[i][j]);
}
printf("\n");
}
printf("the sum of matrix 1 and 2\n");
for (i = 0; i < 3; i++)
{
for (j = 0; j < 3; j++)
{
printf("%d\n", (a[i][j] + b[i][j]));
}
printf("\n");
}
}
Given an array arr of length n, find any elements within arr such that their sum is x and sum of their squares is least. I'm trying to find the algorithm with least complexity. So far, I've written a simple recursive algorithm that finds all the subset within array and put the sum check as base condition. I've written my code is javascript as below:
var arr = [3, 4, 2, 1];
var arr2 = arr.map(function(n) { return n*n; });
var max_sum = 5;
var most_min = -1;
function _rec(i, _sum, _square) {
if(_sum >= max_sum) {
if(most_min == -1 || _square < most_min) {
most_min = _square;
console.log("MIN: " + most_min);
}
console.log("END");
return;
}
if(i >= arr.length)
return;
console.log(i);
var n = arr[i];
// square of above number
var n2 = arr2[i];
_sum = _sum + n;
_square = _square + n2;
_rec(i+1, _sum, _square);
_sum = _sum - n;
_square = _square - n2;
_rec(i+1, _sum, _square);
}
_rec(0, 0, 0);
Visit http://jsfiddle.net/1dxgq6d5/6/ to see the output of above algorithm. Above algorithm is quite simple, it is finding all subsets by evaluating two choices at every recursive step; 1) choose the current number or 2) reject and then carry on with recursion.
I'm trying to find an algorithm which is more efficient then simple recursion above. Any suggestion or help would be appreciated.
One more hypothesis
I'm thinking that if I sort the array, and find the subset of element with least variance (separations between each other) such that their sum is x would fulfill my requirements. Not sure, if this is going to be very helpful, but I'm currently working this in hope to improve my current blind recursive approach.
First off, you're finding subsets, not permutations, because you don't care about the order of the elements in each set.
Secondly, even without trying to minimize the sum of the squares, just finding whether there's a subset that sums to a target number is NP-complete -- this is the subset sum problem. It's currently believed by most computer scientists that P != NP, so there's no efficient (polynomial-time) algorithm for this.
Subset sum is only weakly NP-hard, so it's possible to get an efficient solution with dynamic programming (assuming that the input array consists of integers having a relatively small sum). Switch from trying all possibilities recursively and depth-first to trying all possibilities iteratively and breadth-first by storing the possibilities for the first k elements in an array. Before considering element k + 1, filter this array by discarding all but the lowest sum of squares for each total that can be made.
I solved the problem in more efficient way than simple recursion. I'm using dynamic programming approach. Below is the python code I wrote:
_sum=7
_set=[1,1,2,3,4,6]
current_idx = 0
sum_mapping = [[-1 for i in range(len(_set) + 1)] for i in range(_sum)]
max_sum = _set[current_idx]
for i in range(0, _sum):
current_sum = i + 1
for j in [i for i in range(0, current_idx+1)][::-1] + \
[i for i in range(current_idx + 1, len(_set))]:
required_value = current_sum - _set[j]
if required_value < 0:
break
if required_value == 0 or sum_mapping[required_value - 1][j] != -1:
_j = j + 1
sum_mapping[i][_j:] = [j]*(len(_set) - j)
break
if max_sum == current_sum:
current_idx = current_idx + 1
max_sum = max_sum + _set[current_idx]
_cur = sum_mapping[_sum-1][len(_set)]
if _cur != -1:
_l_sum = _sum
while _l_sum != 0:
print(_set[_cur])
_l_sum = _l_sum - _set[_cur]
_cur = sum_mapping[_l_sum -1][len(_set)]
Here is ideone output: http://ideone.com/OgGN2f
Suppose I have a set of coins having denominations a1, a2, ... ak.
One of them is known to be equal to 1.
I want to make change for all integers 1 to n using the minimum number of coins.
Any ideas for the algorithm.
eg. 1, 3, 4 coin denominations
n = 11
optimal selection is 3, 0, 2 in the order of coin denominations.
n = 12
optimal selection is 2, 2, 1.
Note: not homework just a modification of this problem
This is a classic dynamic programming problem (note first that the greedy algorithm does not always work here!).
Assume the coins are ordered so that a_1 > a_2 > ... > a_k = 1. We define a new problem. We say that the (i, j) problem is to find the minimum number of coins making change for j using coins a_i > a_(i + 1) > ... > a_k. The problem we wish to solve is (1, j) for any j with 1 <= j <= n. Say that C(i, j) is the answer to the (i, j) problem.
Now, consider an instance (i, j). We have to decide whether or not we are using one of the a_i coins. If we are not, we are just solving a (i + 1, j) problem and the answer is C(i + 1, j). If we are, we complete the solution by making change for j - a_i. To do this using as few coins as possible, we want to solve the (i, j - a_i) problem. We arrange things so that these two problems are already solved for us and then:
C(i, j) = C(i + 1, j) if a_i > j
= min(C(i + 1, j), 1 + C(i, j - a_i)) if a_i <= j
Now figure out what the initial cases are and how to translate this to the language of your choice and you should be good to go.
If you want to try you hands at another interesting problem that requires dynamic programming, look at Project Euler Problem 67.
Here's a sample implementation of a dynamic programming algorithm in Python. It is simpler than the algorithm that Jason describes, because it only calculates 1 row of the 2D table he describes.
Please note that using this code to cheat on homework will make Zombie Dijkstra cry.
import sys
def get_best_coins(coins, target):
costs = [0]
coins_used = [None]
for i in range(1,target + 1):
if i % 1000 == 0:
print '...',
bestCost = sys.maxint
bestCoin = -1
for coin in coins:
if coin <= i:
cost = 1 + costs[i - coin]
if cost < bestCost:
bestCost = cost
bestCoin = coin
costs.append(bestCost)
coins_used.append(bestCoin)
ret = []
while target > 0:
ret.append(coins_used[target])
target -= coins_used[target]
return ret
coins = [1,10,25]
target = 100033
print get_best_coins(coins, target)
solution in C# code
public static long findPermutations(int n, List<long> c)
{
// The 2-dimension buffer will contain answers to this question:
// "how much permutations is there for an amount of `i` cents, and `j`
// remaining coins?" eg. `buffer[10][2]` will tell us how many permutations
// there are when giving back 10 cents using only the first two coin types
// [ 1, 2 ].
long[][] buffer = new long[n + 1][];
for (var i = 0; i <= n; ++i)
buffer[i] = new long[c.Count + 1];
// For all the cases where we need to give back 0 cents, there's exactly
// 1 permutation: the empty set. Note that buffer[0][0] won't ever be
// needed.
for (var j = 1; j <= c.Count; ++j)
buffer[0][j] = 1;
// We process each case: 1 cent, 2 cent, etc. up to `n` cents, included.
for (int i = 1; i <= n; ++i)
{
// No more coins? No permutation is possible to attain `i` cents.
buffer[i][0] = 0;
// Now we consider the cases when we have J coin types available.
for (int j = 1; j <= c.Count; ++j)
{
// First, we take into account all the known permutations possible
// _without_ using the J-th coin (actually computed at the previous
// loop step).
var value = buffer[i][j - 1];
// Then, we add all the permutations possible by consuming the J-th
// coin itself, if we can.
if (c[j - 1] <= i)
value += buffer[i - c[j - 1]][j];
// We now know the answer for this specific case.
buffer[i][j] = value;
}
}
// Return the bottom-right answer, the one we were looking for in the
// first place.
return buffer[n][c.Count];
}
Following is the bottom up approach of dynamic programming.
int[] dp = new int[amount+ 1];
Array.Fill(dp,amount+1);
dp[0] = 0;
for(int i=1;i<=amount;i++)
{
for(int j=0;j<coins.Length;j++)
{
if(coins[j]<=i) //if the amount is greater than or equal to the current coin
{
//refer the already calculated subproblem dp[i-coins[j]]
dp[i] = Math.Min(dp[i],dp[i-coins[j]]+1);
}
}
}
if(dp[amount]>amount)
return -1;
return dp[amount];
}