Related
I encountered the following algorithmic question which has strict constraints on runtime (<10s and no large memory footprint) and I am stumped. My approach fails half the test cases.
Question
A box contains a number of items that can only br removed 1 at a time or 3 at a time.
How many ways can the box be emptied? the answer can be very large so return it as modulo of 10^9+7.
for example,there are n=7 items initially.They can be removed nine ways,as follows:
1.(1,1,1,1,1,1,1)
2.(1.1.1.1.3)
3.(1,1,1,3,1)
4.(1,1,3,1,1)
5.(1,3,1,1,1)
6.(3,1,1,1,1)
7.(1,3,3)
8.(3,1,3)
9.(3,3,1)
So the function should return 9.
Function Description:
Your function must take in a parameter, n for the number of items, and return an integer which denotes the number of ways to empty the box.
Constraints: 1<=n<=10^8
Sample cases :
Input: 1
Sample OutPut: 1
Explanation: There is only 1 way to remove 1 item. Answer=(1%1000000007)=1
Input: 7
Sample OutPut: 9
There is only 9 ways to remove 7 items
My Approach
This leads to a standard recurrence relation where f(n) = f(n-3) + f(n-1) for n > 2, so i did it as follows
def memoized_number_of_ways(dic, n):
if n not in dic:
dic[n] = memoized_number_of_ways(dic, n-3) + memoized_number_of_ways(dic, n-1)
return dic[n]
def numberOfWays(n):
# Write your code here
memoize = {1:1,2:1,3:2}
import math
ans = memoized_number_of_ways(memoize,n)
return ans % (math.pow(10,9) + 7)
However this fails on any case where n > 10**2. How can you do this problem while accomodating n up to 10^8 and in less than 10s with not much memory?
Just write your recurrence using matrices (pardon my way of writing matrices, StackOverflow doesn't allow LaTeX).
[f(n) ] = [1 0 1] [f(n-1) ]
[f(n-1)] = [1 0 0] [f(n-2) ]
[f(n-2)] = [0 1 0] [f(n-3) ]
Now all you have to do is raise a 3x3 matrix (modulo fixed constant) to power n (or n-3 or something like that, depending on your "base case column vector", fill in the details), and then multiply it by a "base case column vector". This can be done in time O(logn).
PS: You may want to lookup matrix exponentiation.
Three solutions, fastest takes about 31 μs for n=108 on tio.run (which has medium-fast computers).
A matrix power solution like described by advocateofnone that takes about 1 millisecond (Try it online!):
import numpy as np
from time import time
class ModInt:
def __init__(self, x):
self.x = x % (10**9 + 7)
def __add__(a, b):
return ModInt(a.x + b.x)
def __mul__(a, b):
return ModInt(a.x * b.x)
def __str__(self):
return str(self.x)
def solve(n):
O = ModInt(0)
I = ModInt(1)
A = np.matrix([[O,I,O], [O,O,I], [I,O,I]])
return (A**n)[2,2]
for _ in range(3):
t0 = time()
print(solve(10**8), time() - t0)
Output (result and time in seconds for n=108, three attempts):
109786077 0.0010712146759033203
109786077 0.0010180473327636719
109786077 0.0009677410125732422
Another, taking about 0.5 milliseconds (Try it online!):
import numpy as np
from time import time
def solve(n):
A = np.matrix([[0,1,0], [0,0,1], [1,0,1]])
power = 1
mod = 10**9 + 7
while n:
if n % 2:
power = power * A % mod
A = A**2 % mod
n //= 2
return power[2,2]
for _ in range(3):
t0 = time()
print(solve(10**8), time() - t0)
One based on #rici's solution in the comments, takes about 31 μs (Try it online!):
from timeit import repeat
def solve(n):
m = 10**9 + 7
def abc(n):
if n == 0:
return 0, 1, 0
a, b, c = abc(n // 2)
d = a + c
e = b + d
A = 2*a*b + c*c
C = 2*b*c + d*d
E = 2*c*d + e*e
D = A + C
B = E - D
if n % 2:
A, B, C = B, C, D
return A%m, B%m, C%m
return sum(abc(n)) % m
n = 10**8
print(solve(n))
for _ in range(3):
t = min(repeat(lambda: solve(n), 'gc.enable()', number=1000)) / 1000
print('%.1f μs' % (t * 1e6))
Explanation: Looking at the matrix powers from my previous solutions, I noticed they only actually contain five different values, and they're consecutive result numbers from our desired sequence. For example, A**19 is:
[[277 189 406]
[406 277 595]
[595 406 872]]
I gave them names in increasing order:
| b a c |
| c b d |
| d c e |
Squaring that matrix results in a matrix for larger n, with entries A/B/C/D/E. And if you square the above matrix, you'll find the relationships A = 2*a*b + c*c etc.
My helper function abc(n) computes the entries a/b/c of the n-th matrix power. For n=0, that's the identity matrix, so my a/b/c are 0/1/0 there. And in the end I, return the e-value (computed as e=b+d=a+b+c).
Here's a simple iterative O(n) time / O(1) space solution whose optimized version takes 6 seconds on a medium-fast machine (unoptimized takes 15 seconds there).
Unoptimized (Try it online!):
def solve(n):
mod = 10**9 + 7
a = b = c = 1
for _ in range(n):
a, b, c = b, c, (a+c) % mod
return a
print(solve(7))
print(solve(10**8))
Optimized (Try it online!):
def solve(n):
mod = 10**9 + 7
a = b = c = 1
for _ in range(n // 300):
for _ in range(100):
a += c
b += a
c += b
a %= mod
b %= mod
c %= mod
for _ in range(n % 300):
a, b, c = b, c, (a+c) % mod
return a
Your solution is on the right track and the bug is not related to your algorithm (Yay).
The problem is when you are performing operations on some big numbers you lose precision. Notice that you can apply the mod 10 ** 9 + 7 along your code since addition is not affected by it. By doing so you keep all your numbers below a certain size and you will not have any floating point precision errors:
import math
def memoized_number_of_ways(dic, n):
if n not in dic:
dic[n] = (memoized_number_of_ways(dic, n-3) + memoized_number_of_ways(dic, n-1)) % (math.pow(10,9) + 7)
return dic[n]
def numberOfWays(n):
memoize = {1:1,2:1,3:2}
ans = memoized_number_of_ways(memoize,n)
return ans
Note that for you to be able to answer the question for n > 1000 you need to solve this recursion error problem.
Unfortunately even a very efficient solution (hint: you don't really need more than 3 items in your dict at any moment) will not solve the question for n ~ 10 ** 9 under a second. And you will need to find another way - a great option is the second answer here :)
One of my friends got this question in google coding contest. Here goes the question.
Find the number of N-digit numbers that are divisible by both X and Y.
Since the answer can be very large, print the answer modulo 10^9 + 7.
Note: 0 is not considered single-digit number.
Input: N, X, Y.
Constraints:
1 <= N <= 10000
1 <= X,Y <= 20
Eg-1 :
N = 2, X = 5, Y = 7
output : 2 (35 and 70 are the required numbers)
Eg-2 :
N = 1, X = 2, Y = 3
output : 1 (6 is the required number)
If the constraints on N were smaller, then it would be easy (ans = 10^N / LCM(X,Y) - 10^(N-1) / LCM(X,Y)).
But N is upto 1000, hence I am unable to solve it.
This question looks like it was intended to be more difficult, but I would do it pretty much the way you said:
ans = floor((10N-1)/LCM(X,Y)) - floor((10N-1-1)/LCM(X,Y))
The trick is to calculate the terms quickly.
Let M = LCM(X,Y), and say we have:
10a = Mqa + ra, and
10b = Mqb + rb
The we can easily calculate:
10a+b = M(Mqaqb + raqb + rbqa + floor(rarb/M)) + (rarb%M)
With that formula, we can calculate the quotient and remainder for 10N/M in just 2 log N steps using exponentiation by squaring: https://en.wikipedia.org/wiki/Exponentiation_by_squaring
Following python works for this question ,
import math
MOD = 1000000007
def sub(x,y):
return (x-y+MOD)%MOD
def mul(x,y):
return (x*y)%MOD
def power(x,y):
res = 1
x%=MOD
while y!=0:
if y&1 :
res = mul(res,x)
y>>=1
x = mul(x,x)
return res
def mod_inv(n):
return power(n,MOD-2)
x,y = [int(i) for i in input().split()]
m = math.lcm(x,y)
n = int(input())
a = -1
b = -1
total = 1
for i in range(n-1):
total = (total * 10)%m
b = total % m
total = (total*10)%m
a = total % m
l = power(10 , n-1)
r = power(10 , n)
ans = sub( sub(r , l) , sub(a,b) )
ans = mul(ans , mod_inv(m))
print(ans)
Approach for this question is pretty straight forward,
let, m = lcm(x,y)
let,
10^n -1 = m*x + a
10^(n-1) -1 = m*y + b
now from above two equations it is clear that our answer is equal to
(x - y)%MOD .
so,
(x-y) = ((10^n - 10^(n-1)) - (a-b)) / m
also , a = (10^n)%m and b = (10^(n-1))%m
using simple modular arithmetic rules we can easily calculate a and b in O(n) time.
also for subtraction and division performed in the formula we can use modular subtraction and division respectively.
Note: (a/b)%MOD = ( a * (mod_inverse(b, MOD)%MOD )%MOD
My objective is to find the sum of all numbers from 4 to 666554 which consists of 4,5,6 only.
SUM = 4+5+6+44+45+46+54+55+56+64+65+66+.....................+666554.
Simple method is to run a loop and add the numbers made of 4,5 and 6 only.
long long sum = 0;
for(int i=4;i <=666554;i++){
/*check if number contains only 4,5 and 6.
if condition is true then add the number to the sum*/
}
But it seems to be inefficient. Checking that the number is made up of 4,5 and 6 will take time. Is there any way to increase the efficiency. I have tried a lot but no new approach i have found.Please help.
For 1-digit numbers, note that
4 + 5 + 6 == 5 * 3
For 2-digits numbers:
(44 + 45 + 46) + (54 + 55 + 56) + (64 + 65 + 66)
== 45 * 3 + 55 * 3 + 65 * 3
== 55 * 9
and so on.
In general, for n-digits numbers, there are 3n of them consist of 4,5,6 only, their average value is exactly 5...5(n digits). Using code, the sum of them is ('5' * n).to_i * 3 ** n (Ruby), or int('5' * n) * 3 ** n (Python).
You calculate up to 6-digits numbers, then subtract the sum of 666555 to 666666.
P.S: for small numbers like 666554, using pattern matching is fast enough. (example)
Implement a counter in base 3 (number of digit values), e.g. 0,1,2,10,11,12,20,21,22,100.... and then translate the base-3 number into a decimal with the digits 4,5,6 (0->4, 1->5, 2->6), and add to running total. Repeat until the limit.
def compute_sum(digits, max_val):
def _next_val(cur_val):
for pos in range(len(cur_val)):
cur_val[pos]+=1
if cur_val[pos]<len(digits):
return
cur_val[pos]=0
cur_val.append(0)
def _get_val(cur_val):
digit_val=1
num_val=0
for x in cur_val:
num_val+=digits[x]*digit_val
digit_val*=10
return num_val
cur_val=[]
sum=0
while(True):
_next_val(cur_val)
num_val=_get_val(cur_val)
if num_val>max_val:
break
sum+=num_val
return sum
def main():
digits=[4,5,6]
max_val=666554
print(digits, max_val)
print(compute_sum(digits, max_val))
Mathematics are good, but not all problems are trivially "compressible", so knowing how to deal with them without mathematics can be worthwhile.
In this problem, the summation is trivial, the difficulty is efficiently enumerating the numbers that need be added, at first glance.
The "filter" route is a possibility: generate all possible numbers, incrementally, and filter out those which do not match; however it is also quite inefficient (in general):
the condition might not be trivial to match: in this case, the easier way is a conversion to string (fairly heavy on divisions and tests) followed by string-matching
the ratio of filtering is not too bad to start with at 30% per digit, but it scales very poorly as gen-y-s remarked: for a 4 digits number it is at 1%, or generating and checking 100 numbers to only get 1 out of them.
I would therefore advise a "generational" approach: only generate numbers that match the condition (and all of them).
I would note that generating all numbers composed of 4, 5 and 6 is like counting (in ternary):
starts from 4
45 becomes 46 (beware of carry-overs)
66 becomes 444 (extreme carry-over)
Let's go, in Python, as a generator:
def generator():
def convert(array):
i = 0
for e in array:
i *= 10
i += e
return i
def increment(array):
result = []
carry = True
for e in array[::-1]:
if carry:
e += 1
carry = False
if e > 6:
e = 4
carry = True
result = [e,] + result
if carry:
result = [4,] + result
return result
array = [4]
while True:
num = convert(array)
if num > 666554: break
yield num
array = increment(array)
Its result can be printed with sum(generator()):
$ time python example.py
409632209
python example.py 0.03s user 0.00s system 82% cpu 0.043 total
And here is the same in C++.
"Start with a simpler problem." —Polya
Sum the n-digit numbers which consist of the digits 4,5,6 only
As Yu Hao explains above, there are 3**n numbers and their average by symmetry is eg. 555555, so the sum is 3**n * (10**n-1)*5/9. But if you didn't spot that, here's how you might solve the problem another way.
The problem has a recursive construction, so let's try a recursive solution. Let g(n) be the sum of all 456-numbers of exactly n digits. Then we have the recurrence relation:
g(n) = (4+5+6)*10**(n-1)*3**(n-1) + 3*g(n-1)
To see this, separate the first digit of each number in the sum (eg. for n=3, the hundreds column). That gives the first term. The second term is sum of the remaining digits, one count of g(n-1) for each prefix of 4,5,6.
If that's still unclear, write out the n=2 sum and separate tens from units:
g(2) = 44+45+46 + 54+55+56 + 64+65+66
= (40+50+60)*3 + 3*(4+5+6)
= (4+5+6)*10*3 + 3*g(n-1)
Cool. At this point, the keen reader might like to check Yu Hao's formula for g(n) satisfies our recurrence relation.
To solve OP's problem, the sum of all 456-numbers from 4 to 666666 is g(1) + g(2) + g(3) + g(4) + g(5) + g(6). In Python, with dynamic programming:
def sum456(n):
"""Find the sum of all numbers at most n digits which consist of 4,5,6 only"""
g = [0] * (n+1)
for i in range(1,n+1):
g[i] = 15*10**(i-1)*3**(i-1) + 3*g[i-1]
print(g) # show the array of partial solutions
return sum(g)
For n=6
>>> sum456(6)
[0, 15, 495, 14985, 449955, 13499865, 404999595]
418964910
Edit: I note that OP truncated his sum at 666554 so it doesn't fit the general pattern. It will be less the last few terms
>>> sum456(6) - (666555 + 666556 + 666564 + 666565 + 666566 + 666644 + 666645 + 666646 + 666654 + 666655 + 666656 + + 666664 + 666665 + 666666)
409632209
The sum of 4 through 666666 is:
total = sum([15*(3**i)*int('1'*(i+1)) for i in range(6)])
>>> 418964910
The sum of the few numbers between 666554 and 666666 is:
rest = 666555+666556+666564+666565+666566+
666644+666645+666646+
666654+666655+666656+
666664+666665+666666
>>> 9332701
total - rest
>>> 409632209
Java implementation of question:-
This uses the modulo(10^9 +7) for the answer.
public static long compute_sum(long[] digits, long max_val, long count[]) {
List<Long> cur_val = new ArrayList<>();
long sum = 0;
long mod = ((long)Math.pow(10,9))+7;
long num_val = 0;
while (true) {
_next_val(cur_val, digits);
num_val = _get_val(cur_val, digits, count);
sum =(sum%mod + (num_val)%mod)%mod;
if (num_val == max_val) {
break;
}
}
return sum;
}
public static void _next_val(List<Long> cur_val, long[] digits) {
for (int pos = 0; pos < cur_val.size(); pos++) {
cur_val.set(pos, cur_val.get(pos) + 1);
if (cur_val.get(pos) < digits.length)
return;
cur_val.set(pos, 0L);
}
cur_val.add(0L);
}
public static long _get_val(List<Long> cur_val, long[] digits, long count[]) {
long digit_val = 1;
long num_val = 0;
long[] digitAppearanceCount = new long[]{0,0,0};
for (Long x : cur_val) {
digitAppearanceCount[x.intValue()] = digitAppearanceCount[x.intValue()]+1;
if (digitAppearanceCount[x.intValue()]>count[x.intValue()]){
num_val=0;
break;
}
num_val = num_val+(digits[x.intValue()] * digit_val);
digit_val *= 10;
}
return num_val;
}
public static void main(String[] args) {
long [] digits=new long[]{4,5,6};
long count[] = new long[]{1,1,1};
long max_val= 654;
System.out.println(compute_sum(digits, max_val, count));
}
The Answer by #gen-y-s (https://stackoverflow.com/a/31286947/8398943) is wrong (It includes 55,66,44 for x=y=z=1 which is exceeding the available 4s, 5s, 6s). It gives output as 12189 but it should be 3675 for x=y=z=1.
The logic by #Yu Hao (https://stackoverflow.com/a/31285816/8398943) has the same mistake as mentioned above. It gives output as 12189 but it should be 3675 for x=y=z=1.
Following is text from Data structure and algorithm analysis by Mark Allen Wessis.
Following x(i+1) should be read as x subscript of i+1, and x(i) should be
read as x subscript i.
x(i + 1) = (a*x(i))mod m.
It is also common to return a random real number in the open interval
(0, 1) (0 and 1 are not possible values); this can be done by
dividing by m. From this, a random number in any closed interval [a,
b] can be computed by normalizing.
The problem with this routine is that the multiplication could
overflow; although this is not an error, it affects the result and
thus the pseudo-randomness. Schrage gave a procedure in which all of
the calculations can be done on a 32-bit machine without overflow. We
compute the quotient and remainder of m/a and define these as q and
r, respectively.
In our case for M=2,147,483,647 A =48,271, q = 127,773, r = 2,836, and r < q.
We have
x(i + 1) = (a*x(i))mod m.---------------------------> Eq 1.
= ax(i) - m (floorof(ax(i)/m)).------------> Eq 2
Also author is mentioning about:
x(i) = q(floor of(x(i)/q)) + (x(i) mod Q).--->Eq 3
My question
what does author mean by random number is computed by normalizing?
How author came with Eq 2 from Eq 1?
How author came with Eq 3?
Normalizing means if you have X ∈ [0,1] and you need to get Y ∈ [a, b] you can compute
Y = a + X * (b - a)
EDIT:
2. Let's suppose
a = 3, x = 5, m = 9
Then we have
where [ax/m] means an integer part.
So we have 15 = [ax/m]*m + 6
We need to get 6. 15 - [ax/m]*m = 6 => ax - [ax/m]*m = 6 => x(i+1) = ax(i) - [ax(i)/m]*m
If you have a random number in the range [0,1], you can get a number in the range [2,5] (for example) by multiplying by 3 and adding 2.
What does the following loop do?
k = 0
while(b!=0):
a = a^b
b = (a & b) << 1
k = k + 1
where a, b and k integers.
Initially a = 2779 and b = 262 then what will be the value of k after the termination of the loop?
I'd be happy if you people try solving the problem manually rather than programmatically.
EDIT : Dropping the tags c and c++. How can I solve the problem manually?
After this chunk is executed:
a = a^b ;
b = (a & b) << 1;
b will take on the integer representation of whatever bits were both set in b and not set in a. Assuming that the input numbers will be of the form 2x, a will become a + b, and b will be itself multiplied by 2 (due to shifting). This means that the loop will terminate when the MSB of a and b are the same (which will be 780th bit in this example). Since b starts off at the 63th bit, there will eventually be 718 iterations: 780 - 63 + 1 (the last iteration) = 718.
You can see this when you step through this with a = 21 and b = 20:
a = 10
b = 01
k = 0
a = 11
b = 10
k = 1
a = 01 (a + b no longer holds here, but it is irrelevant as this is the termination case)
b = 00
k = 2
What does the following loop do?
It computes [power of a] - [power of b] + 1
If you look at the bit-patterns it becomes quite clear. For initial values of a = 210 and b = 25 it looks like this:
k = 0, a = 10000000000, b = 100000
k = 1, a = 10000100000, b = 1000000
k = 2, a = 10001100000, b = 10000000
k = 3, a = 10011100000, b = 100000000
k = 4, a = 10111100000, b = 1000000000
k = 5, a = 11111100000, b = 10000000000
Here is an ideone.com demo.
For the values you mention in your post I get k = 718.
In a language that supports numbers this big, the answer is 718, and the question is not at all interesting either way.
It looks like the loop is performing a series of bitwise operations while incrementing k by 1 at each iteration.
There's a decent-looking tutorial about bitwise operators here.
Hope that helps.
Since you have mentioned that a qnd b is an integer, i think the value 2779 is too big for a integer variable to accommodate this is same for the variable b 262 as well. So both will have there minimum value I think , so after executing a = a^b ;
b = (a & b) << 1; these statements the value of a and be will be 0. and k's value will be equal to 1. Since b is 0 , the while loop's condition will be false and it will exit the loop.
EDIT : The answer is applicable for a programming language like C/C++ or C#
It's not really C syntax.. If we consider that the code is sequential, I am agree with #Paul. But, considering that
a + b == (a^b) + ((a&b) << 1)
where a^b is the sum without applying carry from each bit, and (a&b)<<1 is carry from each bit, we could say that it performs a sum, IFF C code would be
int k = 0;
while(b){
int old_a = a;
a = a^b;
b = (old_a & b) << 1; // note that the need the original value of a here
k++; // number of recursion levels for carry
};
The difference in code could be because of "paralel" way of execution in the original code (or language).