Complexity class - algorithm

Assume that methods m1 and m2 are static void and compute the same result by processing an argument of type Object[]. Empirically we find that m1 -> T(N) = 100N and m2 -> T(N) = 10Nlog2N where the times are in microseconds. For what size inputs is it better to use m1 and m2?
So I would use m1 for big numbers while I would use m2 for small numbers right? just checking the answers.

You're looking for the value of N > 0 such that 100N > 10N log2 N, so that's just an algebra problem. Divide both sides by 10N and you get 10 > log2 N, i.e., N < 2**10, i.e. N < 1024. Not that hard!-)

Well, log2N is going to hit 10 when N is 1024, so according to the formulae you've given us, you would use the second one for N <= 1024 and the first for N >= 1024. (What you do "on the boundary" doesn't matter - they'll be equal.)
However, from a practical point of view you should quite possibly pick one and maintain just that one. Do you expect very large inputs, or lots of small inputs? Is this definitely a bottleneck in your code? Which is the simpler algorithm?
There's a lot more to deciding which algorithm is the best than just which is the fastest for a given input. I would rather maintain a simple algorithm which runs slightly slower than a fast but insanely complicated one.

Look at this: http://www.wolframalpha.com/input/?i=plot+100*n+and+plot+10*n*log2(n)
I haven't figured out yet how to modify the scale though. In any case, for future reference.

Related

Prime Factorization of numbers of form x^a + b where x is prime

I need to calculate prime factorization of large numbers, by large numbers I mean of range 10^100.
I get an input a[0] <= 10^5 (whose prime factors I have already calculated using the sieve and other optimizations). After that I get series of inputs of a[1], a[2], a[3] all in range 2 <= a[i] <= 10^5. I need to calculate the product and get the factors of the new product. I have the following maths
Let X be the data in memory and X can be represented as:
X = (p[0]^c1)(p[1]^c2)(p[2]^c[3]) .... where p[i] are its prime factors.
So I save this as,
A[p[0]] = c1, A[p[1]] = c2.... as p[i] <= 100000 this seems to work pretty well.
and as new number arrives, I just add the power of primes of the new number in A.
So this works really well and also is fast enough. Now I am thinking of optimizing space and compensating with reduction of time efficiency.
So If I can represent Any number P as x^a + b where x is a prime. Can I factorize it? P obviously doesn't fit in the memory but 2 <= x, a, b <= 100000? Or Is there any other method that is possible which will save me the space of A? I am okay with a slower algorithm than the above one.
I don't think representing a number as xa + b with prime x makes it any easier to factor.
Factoring hundred-digit numbers isn't all that hard these days. A good personal computer with lots of cores running a good quadratic sieve can factor most hundred-digit numbers in about a day, though you should know that hundred-digit numbers are about at the limit of what is reasonable to factor with a desktop computer. Look at Jason Papadopoulos' program msieve for a cutting edge factorization program.
First, you'll better do some math on paper (perhaps some simplifications are possible; I don't know...).
Then you need to use some arbitrary precision arithmetic (a.k.a. bignums or bigints) library. I recommend GMPlib but they are other ones.
See also this answer.

2^n mod (m) algorithm

In class, we were presented with an algorithm for 2^n mod(m).
to find 2^n mod(m){
if n=0 {return 1;}
r=2^(n-1)mod(m);
if 2r < m {return 2r;}
if 2r > =m {return 2r-m;}
}
We were told that the runtime is O(n*size(m)) where size of m is the number of bits in m.
I understand the n part, but I cannot explain the size(m) unless it is because of the subtraction involved. Can anyone shed some light on that?
Thanks in advance.
The n part is clear, as you have already understood yourself. The size(m) (which is the number of digits in m, which is basically log(m)) is because of mod. Even though your CPU does that for you in one instruction, it takes log(m) (let's say 32 bits) times. If m is very large, as is common with encryption keys, this can become considerable.
Why number of digits in m? Remember division:
abcdefghijk | xyz
|-----
alm | nrvd...
opq
stu
wabc
.......
The number of times you do the minus, is at most the number of digits in the dividend.
I believe this is used in cryptography (so called noninvertible function).
If we need to compute (2**n) mod m recursively, this would be the most obvious way to do it. Since the depth of recursion is n, the O(n) complexity is obvious.
However, if we would like to support arbitrary size of m (512 bit keys are possible in cryptography, and are much larger than any arithmetic register), we should also consider that (in most cases we don't have to use arbitrary precision arithmetics, so this term is usually 1).
EDIT #Mysticial: The function does not call the hardware mod operation explicitely, all it does is shift and substraction. shift is always O(1) while addition/substraction is O(ceil(m/sizeof_ALU_precision))

Showing that a recurrence relation is O(n log n)

T (n) = T (xn) + T ((1 − x)n) + n = O(n log n)
where x is a constant in the range 0 < x < 1. Is the asymptotic complexity the same when x = 0.5, 0.1 and 0.001?
What happens to the constant hidden in the O() notation. Use Substitution Method.
I'm trying to use the example on ~page 15 but I find it weird that in that example the logs change from default base to base 2.
I also do not really understand why it needed to be simplified so much just so as to remove cnlog2n from the left side, could that not be done in the first step and the left side would just have "stuff-cnlog2n<=0" then evaulated for any c and n like so?
From what I tried it could not prove that T(n)=O(n)
Well, if you break this up into a tree using Master's theorem, then this will have a constant "amount" to calculate each time. You know this because x + 1 - x = 1.
Thus, the time depends on the level of the tree, which is logarithmic since the pieces are reducing each time by some constant amount. Since you do O(n) calcs each level, your overall complexity is O( n log n ).
I expect this will be a little more complicated to "prove". Remember that it doesn't matter what base your logs are in, they're all just some constant factor. Refer to logarithmic relations for this.
PS: Looks like homework. Think harder yourself!
This seems to me exactly the recurrence equation for the average case in Quicksort.
You should look at CLRS explanation of "Balanced Partitioning".
but I find it weird that in that example the logs change from default base to base 2
That is indeed weird. Simple fact is, that it's easier to prove with base 2, than with base unknown x. For example, log(2n) = 1+log(n) in base 2, that is a bit easier. You don't have to use base 2, you can pick any base you want, or use base x. To be absolutely correct the Induction Hypothesis must have the base in it: T(K) <= c K log_2(K). You can't change the IH later on, so what is happening now is not correct in a strict sense. You're free to pick any IH you like, so, just pick one that makes the prove easier: in this case logs with base 2.
the left side would just have "stuff-cnlog2n<=0" then evaulated for any c and n
What do you mean with 'evaluated for any c and n'? stuff-cnlog2n<=0 is correct, but how do you prove that there is a c so that it holds for all n? Okay, c=2 is a good guess. To prove it like that in WolframAlpha, you need to do stuff <= 0 where c=2, n=1 OK!, stuff <=0 where c=2, n=2OK!, stuff <= 0 where c=2, n=3OK!, ..., etc taking n all the way to infinity. Hmm, it will take you an infinite amount of time to check all of them... The only practical way (I can think of right now) for solving this is to simplify stuff-cnlog2n<=0. Or maybe you prefer this argument: you don't have WolframAlpha at your exam, so you must simplify.

Most efficient algorithm to compute a common numerator of a sum of fractions

I'm pretty sure that this is the right site for this question, but feel free to move it to some other stackexchange site if it fits there better.
Suppose you have a sum of fractions a1/d1 + a2/d2 + … + an/dn. You want to compute a common numerator and denominator, i.e., rewrite it as p/q. We have the formula
p = a1*d2*…*dn + d1*a2*d3*…*dn + … + d1*d2*…d(n-1)*an
q = d1*d2*…*dn.
What is the most efficient way to compute these things, in particular, p? You can see that if you compute it naïvely, i.e., using the formula I gave above, you compute a lot of redundant things. For example, you will compute d1*d2 n-1 times.
My first thought was to iteratively compute d1*d2, d1*d2*d3, … and dn*d(n-1), dn*d(n-1)*d(n-2), … but even this is inefficient, because you will end up computing multiplications in the "middle" twice (e.g., if n is large enough, you will compute d3*d4 twice).
I'm sure this problem could be expressed somehow using maybe some graph theory or combinatorics, but I haven't studied enough of that stuff to have a good feel for it.
And one note: I don't care about cancelation, just the most efficient way to multiply things.
UPDATE:
I should have known that people on stackoverflow would be assuming that these were numbers, but I've been so used to my use case that I forgot to mention this.
We cannot just "divide" out an from each term. The use case here is a symbolic system. Actually, I am trying to fix a function called .as_numer_denom() in the SymPy computer algebra system which presently computes this the naïve way. See the corresponding SymPy issue.
Dividing out things has some problems, which I would like to avoid. First, there is no guarantee that things will cancel. This is because mathematically, (a*b)**n != a**n*b**n in general (if a and b are positive it holds, but e.g., if a == b ==-1 and n == 1/2, you get (a*b)**n == 1**(1/2) == 1 but (-1)**(1/2)*(-1)**(1/2) == I*I == -1). So I don't think it's a good idea to assume that dividing by an will cancel it in the expression (this may be actually be unfounded, I'd need to check what the code does).
Second, I'd like to also apply a this algorithm to computing the sum of rational functions. In this case, the terms would automatically be multiplied together into a single polynomial, and "dividing" out each an would involve applying the polynomial division algorithm. You can see in this case, you really do want to compute the most efficient multiplication in the first place.
UPDATE 2:
I think my fears for cancelation of symbolic terms may be unfounded. SymPy does not cancel things like x**n*x**(m - n) automatically, but I think that any exponents that would combine through multiplication would also combine through division, so powers should be canceling.
There is an issue with constants automatically distributing across additions, like:
In [13]: 2*(x + y)*z*(S(1)/2)
Out[13]:
z⋅(2⋅x + 2⋅y)
─────────────
2
But this is first a bug and second could never be a problem (I think) because 1/2 would be split into 1 and 2 by the algorithm that gets the numerator and denominator of each term.
Nonetheless, I still want to know how to do this without "dividing out" di from each term, so that I can have an efficient algorithm for summing rational functions.
Instead of adding up n quotients in one go I would use pairwise addition of quotients.
If things cancel out in partial sums then the numbers or polynomials stay smaller, which makes computation faster.
You avoid the problem of computing the same product multiple times.
You could try to order the additions in a certain way, to make canceling more likely (maybe add quotients with small denominators first?), but I don't know if this would be worthwhile.
If you start from scratch this is simpler to implement, though I'm not sure it fits as a replacement of the problematic routine in SymPy.
Edit: To make it more explicit, I propose to compute a1/d1 + a2/d2 + … + an/dn as (…(a1/d1 + a2/d2) + … ) + an/dn.
Compute two new arrays:
The first contains partial multiples to the left: l[0] = 1, l[i] = l[i-1] * d[i]
The second contains partial multiples to the right: r[n-1] = 1, r[i] = d[i] * r[i+1]
In both cases, 1 is the multiplicative identity of whatever ring you are working in.
Then each of your terms on the top, t[i] = l[i-1] * a[i] * r[i+1]
This assumes multiplication is associative, but it need not be commutative.
As a first optimization, you don't actually have to create r as an array: you can do a first pass to calculate all the l values, and accumulate the r values during a second (backward) pass to calculate the summands. No need to actually store the r values since you use each one once, in order.
In your question you say that this computes d3*d4 twice, but it doesn't. It does multiply two different values by d4 (one a right-multiplication and the other a left-multiplication), but that's not exactly a repeated operation. Anyway, the total number of multiplications is about 4*n, vs. 2*n multiplications and n divisions for the other approach that doesn't work in non-commutative multiplication or non-field rings.
If you want to compute p in the above expression, one way to do this would be to multiply together all of the denominators (in O(n), where n is the number of fractions), letting this value be D. Then, iterate across all of the fractions and for each fraction with numerator ai and denominator di, compute ai * D / di. This last term is equal to the product of the numerator of the fraction and all of the denominators other than its own. Each of these terms can be computed in O(1) time (assuming you're using hardware multiplication, otherwise it might take longer), and you can sum them all up in O(n) time.
This gives an O(n)-time algorithm for computing the numerator and denominator of the new fraction.
It was also pointed out to me that you could manually sift out common denominators and combine those trivially without multiplication.

Programming problem - Game of Blocks

maybe you would have an idea on how to solve the following problem.
John decided to buy his son Johnny some mathematical toys. One of his most favorite toy is blocks of different colors. John has decided to buy blocks of C different colors. For each color he will buy googol (10^100) blocks. All blocks of same color are of same length. But blocks of different color may vary in length.
Jhonny has decided to use these blocks to make a large 1 x n block. He wonders how many ways he can do this. Two ways are considered different if there is a position where the color differs. The example shows a red block of size 5, blue block of size 3 and green block of size 3. It shows there are 12 ways of making a large block of length 11.
Each test case starts with an integer 1 ≤ C ≤ 100. Next line consists c integers. ith integer 1 ≤ leni ≤ 750 denotes length of ith color. Next line is positive integer N ≤ 10^15.
This problem should be solved in 20 seconds for T <= 25 test cases. The answer should be calculated MOD 100000007 (prime number).
It can be deduced to matrix exponentiation problem, which can be solved relatively efficiently in O(N^2.376*log(max(leni))) using Coppersmith-Winograd algorithm and fast exponentiation. But it seems that a more efficient algorithm is required, as Coppersmith-Winograd implies a large constant factor. Do you have any other ideas? It can possibly be a Number Theory or Divide and Conquer problem
Firstly note the number of blocks of each colour you have is a complete red herring, since 10^100 > N always. So the number of blocks of each colour is practically infinite.
Now notice that at each position, p (if there is a valid configuration, that leaves no spaces, etc.) There must block of a color, c. There are len[c] ways for this block to lie, so that it still lies over this position, p.
My idea is to try all possible colors and positions at a fixed position (N/2 since it halves the range), and then for each case, there are b cells before this fixed coloured block and a after this fixed colour block. So if we define a function ways(i) that returns the number of ways to tile i cells (with ways(0)=1). Then the number of ways to tile a number of cells with a fixed colour block at a position is ways(b)*ways(a). Adding up all possible configurations yields the answer for ways(i).
Now I chose the fixed position to be N/2 since that halves the range and you can halve a range at most ceil(log(N)) times. Now since you are moving a block about N/2 you will have to calculate from N/2-750 to N/2-750, where 750 is the max length a block can have. So you will have to calculate about 750*ceil(log(N)) (a bit more because of the variance) lengths to get the final answer.
So in order to get good performance you have to through in memoisation, since this inherently a recursive algorithm.
So using Python(since I was lazy and didn't want to write a big number class):
T = int(raw_input())
for case in xrange(T):
#read in the data
C = int(raw_input())
lengths = map(int, raw_input().split())
minlength = min(lengths)
n = int(raw_input())
#setup memoisation, note all lengths less than the minimum length are
#set to 0 as the algorithm needs this
memoise = {}
memoise[0] = 1
for length in xrange(1, minlength):
memoise[length] = 0
def solve(n):
global memoise
if n in memoise:
return memoise[n]
ans = 0
for i in xrange(C):
if lengths[i] > n:
continue
if lengths[i] == n:
ans += 1
ans %= 100000007
continue
for j in xrange(0, lengths[i]):
b = n/2-lengths[i]+j
a = n-(n/2+j)
if b < 0 or a < 0:
continue
ans += solve(b)*solve(a)
ans %= 100000007
memoise[n] = ans
return memoise[n]
solve(n)
print "Case %d: %d" % (case+1, memoise[n])
Note I haven't exhaustively tested this, but I'm quite sure it will meet the 20 second time limit, if you translated this algorithm to C++ or somesuch.
EDIT: Running a test with N = 10^15 and a block with length 750 I get that memoise contains about 60000 elements which means non-lookup bit of solve(n) is called about the same number of time.
A word of caution: In the case c=2, len1=1, len2=2, the answer will be the N'th Fibonacci number, and the Fibonacci numbers grow (approximately) exponentially with a growth factor of the golden ratio, phi ~ 1.61803399. For the
huge value N=10^15, the answer will be about phi^(10^15), an enormous number. The answer will have storage
requirements on the order of (ln(phi^(10^15))/ln(2)) / (8 * 2^40) ~ 79 terabytes. Since you can't even access 79
terabytes in 20 seconds, it's unlikely you can meet the speed requirements in this special case.
Your best hope occurs when C is not too large, and leni is large for all i. In such cases, the answer will
still grow exponentially with N, but the growth factor may be much smaller.
I recommend that you first construct the integer matrix M which will compute the (i+1,..., i+k)
terms in your sequence based on the (i, ..., i+k-1) terms. (only row k+1 of this matrix is interesting).
Compute the first k entries "by hand", then calculate M^(10^15) based on the repeated squaring
trick, and apply it to terms (0...k-1).
The (integer) entries of the matrix will grow exponentially, perhaps too fast to handle. If this is the case, do the
very same calculation, but modulo p, for several moderate-sized prime numbers p. This will allow you to obtain
your answer modulo p, for various p, without using a matrix of bigints. After using enough primes so that you know their product
is larger than your answer, you can use the so-called "Chinese remainder theorem" to recover
your answer from your mod-p answers.
I'd like to build on the earlier #JPvdMerwe solution with some improvements. In his answer, #JPvdMerwe uses a Dynamic Programming / memoisation approach, which I agree is the way to go on this problem. Dividing the problem recursively into two smaller problems and remembering previously computed results is quite efficient.
I'd like to suggest several improvements that would speed things up even further:
Instead of going over all the ways the block in the middle can be positioned, you only need to go over the first half, and multiply the solution by 2. This is because the second half of the cases are symmetrical. For odd-length blocks you would still need to take the centered position as a seperate case.
In general, iterative implementations can be several magnitudes faster than recursive ones. This is because a recursive implementation incurs bookkeeping overhead for each function call. It can be a challenge to convert a solution to its iterative cousin, but it is usually possible. The #JPvdMerwe solution can be made iterative by using a stack to store intermediate values.
Modulo operations are expensive, as are multiplications to a lesser extent. The number of multiplications and modulos can be decreased by approximately a factor C=100 by switching the color-loop with the position-loop. This allows you to add the return values of several calls to solve() before doing a multiplication and modulo.
A good way to test the performance of a solution is with a pathological case. The following could be especially daunting: length 10^15, C=100, prime block sizes.
Hope this helps.
In the above answer
ans += 1
ans %= 100000007
could be much faster without general modulo :
ans += 1
if ans == 100000007 then ans = 0
Please see TopCoder thread for a solution. No one was close enough to find the answer in this thread.

Resources