Given the algorithm,
y := 3
for i = 1,...,n:
y := y * y * y
return y
I would like to determine its runtime. I think the algorithm simply calculates y^(3^n). But this doesn't have anything to do with the runtime, right? I think the runtime is determined by n, it would be O(n). Is that correct?
Assuming that a multiplication ( x * x ) is O(1), then the complexity for your code is O(n). That's what you will get if y in your code is represented using a machine integer.
But the problem is that x * x will overflow for large enough x if you use machine integers. In this case, it means that the code won't compute the correct value of y^(3^n) except for very small values of n and y.
To get a correct answer for larger values, you need to use a "big integer" (or arbitrary precision) representation. For such representations, x * x is NOT a O(1) operation any more. It is actually O(log(x).log(x)) ...
I will leave it to you to estimate the complexity of the entire computation if you use a "big integer" representation.
Related
I'm trying to understand the time complexity (Big-O) of the following algorithm which finds x such that g^x = y (mod p) (i.e. finding the discrete logarithm of y with base g modulo p).
Here's the pseudocode:
discreteLogarithm(y, g, p)
y := y mod p
a := g
x := 1
until a = y
a := (a * g) mod p
x++
return x
end
I know that the time complexity of this approach is exponential in the number of binary digits in p - but what does this mean and why does it depend on p?
I understand that the complexity is determined by the number of loops (until a = y), but where does p come into this, what's this about binary digits?
The run time depends upon the order of g mod p. The worst case is order (p-1)/2, which is O(p). The run time is thus O(p) modular multiplies. The key here is that p has log p bits, where I use 'log' to mean base 2 logarithm. Since p = 2^( log p ) -- mathematical identity -- we see the run time is exponential in the number of bits of p. To make it more clear, let's use b=log p to represent the number of bits. The worst case run time is O(2^b) modular multiplies. Modular multiplies take O(b^2) time, so the full run time is O(2^b * b^2) time. The 2^b is the dominant term.
Depending upon your particular p and g, the order could be much smaller than p. However, some heuristics in analytical number theory show that on average, it is order p.
EDIT: If you are not familiar with the concept of 'order' from group theory, here is brief explanation. If you keep multiplying g by itself mod p, it eventually comes to 1. The order is the number of multiplies before that happens.
Having a system of linear congruences, I'd like to determine if it has a solution. Using simple algorithms that solve such systems is impossible, as the answer may grow exponentially.
One hypothesis I have is that if a system of congruences has no solution, then there are two of them that contradict each other. I have no idea if this holds, if it did that would lead to an easy O(n^2 log n) algo, as checking if a pair of congruences has a solution requires O(log n) time. Nevertheless for this problem I'd rather see something closer to O(n).
We may assume that no moduli exceeds 10^6, especially we can quickly factor them all to begin with. We may even further assume that the sum of all moduli doesn't exceed 10^6 (but still, their product can be huge).
As you suspect, there's a fairly simple way to determine whether the set of congruences has a solution without actually needing to build that solution. You need to:
Reduce each congruence to the form x = a (mod n) if necessary; from the comments, it sounds as though you already have this.
Factorize each modulus n as a product of prime powers: n = p1^e1 * p2^e2 * ... * pk^ek.
Replace each congruence x = a (mod n) with a collection of congruences x = a (mod pi^ei), one for each of the k prime powers you found in step 2.
And now, by the Chinese Remainder Theorem it's enough to check compatibility for each prime independently: given any two congruences x = a (mod p^e) and x = b (mod p^f), they're compatible if and only if a = b (mod p^(min(e, f)). Having determined compatibility, you can throw out the congruence with smaller modulus without losing any information.
With the right data structures, you can do all this in a single pass through your congruences: for each prime p encountered, you'll need to keep track of the biggest exponent e found so far, together with the corresponding right-hand side (reduced modulo p^e for convenience). The running time will likely be dominated by the modulus factorizations, though if no modulus exceeds 10^6, then you can make that step very fast, too, by prebuilding a mapping from each integer in the range 1 .. 10^6 to its smallest prime factor.
EDIT: And since this is supposed to be a programming site, here's some (Python 3) code to illustrate the above. (For Python 2, replace the range call with xrange for better efficiency.)
def prime_power_factorisation(n):
"""Brain-dead factorisation routine, for illustration purposes only."""
# DO NOT USE FOR LARGE n!
while n > 1:
p, pe = next(d for d in range(2, n+1) if n % d == 0), 1
while n % p == 0:
n, pe = n // p, pe*p
yield p, pe
def compatible(old_ppc, new_ppc):
"""Determine whether two prime power congruences (with the same
prime) are compatible."""
m, a = old_ppc
n, b = new_ppc
return (a - b) % min(m, n) == 0
def are_congruences_solvable(moduli, right_hand_sides):
"""Determine whether the given congruences have a common solution."""
# prime_power_congruences is a dictionary mapping each prime encountered
# so far to a pair (prime power modulus, right-hand side).
prime_power_congruences = {}
for m, a in zip(moduli, right_hand_sides):
for p, pe in prime_power_factorisation(m):
# new prime-power congruence: modulus, rhs
new_ppc = pe, a % pe
if p in prime_power_congruences:
old_ppc = prime_power_congruences[p]
if not compatible(new_ppc, old_ppc):
return False
# Keep the one with bigger exponent.
prime_power_congruences[p] = max(new_ppc, old_ppc)
else:
prime_power_congruences[p] = new_ppc
# If we got this far, there are no incompatibilities, and
# the congruences have a mutual solution.
return True
One final note: in the above, we made use of the fact that the moduli were small, so that computing prime power factorisations wasn't a big deal. But if you do need to do this for much larger moduli (hundreds or thousands of digits), it's still feasible. You can skip the factorisation step, and instead find a "coprime base" for the collection of moduli: that is, a collection of pairwise relatively prime positive integers such that each modulus appearing in your congruences can be expressed as a product (possibly with repetitions) of elements of that collection. Now proceed as above, but with reference to that coprime base instead of the set of primes and prime powers. See this article by Daniel Bernstein for an efficient way to compute a coprime base for a set of positive integers. You'd likely end up making two passes through your list: one to compute the coprime base, and a second to check the consistency.
P(x,y,z){
print x
if(y!=x) print y
if(z!=x && z!=y) print z
}
Trivial Algorithm here, values x,y,z are chosen randomly from {1,...r} with r >= 1.
I'm trying to determine the average case complexity of this algorithm and I measure complexity based on the number of print statements.
The best case here is T(n) = 1 or O(1), when x=y=z and the probability of that is 1/3.
The worst case here is still T(n) = 3 or still O(1) when x!=y!=z and the probability is 2/3.
But when it comes to mathematically deriving the average case:
Sample Space is n possible inputs, Probability over Sample Space is 1/n chance
So, how do I calculate average case complexity? (This is where I draw a blank..)
Your algorithm has three cases:
All three numbers are equal. The probability of this is 1/r, since
once you choose x, there's only one choice for y and for z. The cost
for this case is 1.
x != y, but x == z or y == z. The probability of this is 1/r * (1/(r - 1))* 1/2,
since once you choose x, you only have r -1 choices left for y, and z can only be
one of these two choices. Cost = 2.
All three numbers are distinct. Probability that all three are distinct is
1/r * (1/(r - 1))*(1/(r - 2)). Cost = 3.
Thus, the average case can be computed as:
1/r + 1/r * (1/(r - 1)) + 1/r * (1/(r - 1))*(1/(r - 2)) * 3 == O(1)
Edit: The above expression is O(1), since the whole expression is made up of constants.
The average case will be somewhere between the best and worst cases; for this particular problem, that's all you need (at least as far as big-O).
1) Can you program the general case at least? Write the (pseudo)-code and analyze it, it might be readily apparent. You may actually program it suboptimally and there may exist a better solution. This is very typical and it's part of the puzzle-solving of the mathematics end of computer science, e.g. it's hard to discover quicksort on your own if you're just trying to code up a sort.
2) If you can, then run a monte carlo simulation and graph the results. i.e., for N = 1, 5, 10, 20, ..., 100, 1000, or whatever sample is realistic, run 10000 trials and plot the average time. If you're lucky X = sample size, Y = avg. time for 10000 runs at that sample size will graph out a nice line, or parabola, or some easy-to-model curve.
So I'm not sure if you need help on (1) finding or coding the algorithm or (2) analyzing it, you will probably want to revise your question to specify this.
P(x,y,z){
1.print x
2.if(y!=x)
3. print y
4.if(z!=x && z!=y)
5. print z
}
Line 1: takes a constant time c1 (c1:print x)
Line 2: takes a constant time c2 (c2:condition test)
Line 3 :takes a constant time c3 (c3 :print y)
Line 3: takes a constant time c4 (c4:condition test)
Line 4: takes a constant time c5 (c5:print z)
Analysis :
Unless your function P(x,y,z) does not depend on input size " r" the program will take a constant amount of time to run since Time Taken :T(c1)+T(c2+c3)+T(c4+c5) ..summing up the Big O of the function P(x,y,z) is O(1) where 1 is a constant and indicates constant amount of time since T(c1),T(c2),..T(c5) all take constant amount of time.. and say if the function P(x,y,z) iterates from 1 to r..then the complexity of your snippet would have changed and will be in terms of the input size i.e "r"
Best Case : O(1)
Average Case : O(1)
worst Case : O(1)
Pollard Rho factorization method uses a function generator f(x) = x^2-a(mod n) or f(x) = x^2+a(mod n) , is the choice of this function (parabolic) has got any significance or we may use any function (cubic , polynomial or even linear) as we have to identify or find the numbers belonging to same congruence class modulo n to find the non trivial divisor ?
In Knuth Vol II (The Art Of Computer Programming - Seminumerical Algorithms) section 4.5.4 Knuth says
Furthermore if f(y) mod p behaves as a random mapping from the set {0,
1, ... p-1} into itself, exercise 3.1-12 shows that the average value
of the least such m will be of order sqrt(p)... From the theory in
Chapter 3, we know that a linear polynomial f(x) = ax + c will not be
sufficiently random for our purpose. The next simplest case is
quadratic, say f(x) = x^2 + 1. We don't know that this function is
sufficiently random, but our lack of knowledge tends to support the
hypothesis of randomness, and empirical tests show that this f does
work essentially as predicted
The probability theory that says that f(x) has a cycle of length about sqrt(p) assumes in particular that there can be two values y and z such that f(y) = f(z) - since f is chosen at random. The rho in Pollard Rho contains such a junction, with the cycle containing multiple lines leading on to it. For a linear function f(x) = ax + b then for gcd(a, p) = 1 mod p (which is likely since p is prime) f(y) = f(z) means that y = z mod p, so there are no such junctions.
If you look at http://www.agner.org/random/theory/chaosran.pdf you will see that the expected cycle length of a random function is about the sqrt of the state size, but the expected cycle length of a random bijection is about the state size. If you think of generating the random function only as you evaluate it you can see that if the function is entirely random then every value seen so far is available to be chosen again at random to find a cycle, so the odds of closing the cycle increase with the cycle length, but if the function has to be invertible the only way to close the cycle is to generate the starting point, which is much less likely.
Mary got a magic ball for her birthday. The ball, when thrown from
some height, bounces for the double of this height. Mary's thrown the
ball from her balcony which is x above the ground. Help her
calculate how many bounces are there needed for the ball to reach whe
height w.
Input: One integer z (1 ≤ z ≤ 106) as the number of test cases. For
every test, integers x and w (1 ≤ x ≤ 109, 0 ≤ w ≤ 109).
Output: For every case one integer equal to the number of bounces
needed fot the ball to reach w should be printed.
OK, so, though it looks unspeakably easy, I can't find a more efficient way to solve it than a simple, dumb, brutal approach of a loop multiplying x by 2 till it's at least w. For a maximum test, it will get a horrific time, of course. Then, I thought of using previous cases which saves quite a bit time providing that we can get the closest yet smaller result from the previous cases in a short time (O(1)?) which, however, I can't (and don't know if it's possible..) implement. How should this be done?
You are essentially trying to solve the problem
2i x = w
and then finding the smallest integer greater than i. Solving, we get
2i = w / x
i = log2 (w / x)
So one approach would be to compute this value explicitly and then take the ceiling. Of course, you'd have to watch out for numerical instability when doing this. For example, if you are using floats to encode the values and then let w = 8,000,001 and x = 1,000,000, you will end up getting the wrong answer (3 instead of 4). If you use doubles to hold the value, you will also get the wrong answer when x = 1 and w = 536870912 (reporting 30 instead of 29, since 1 x 229 = 536870912, but due to inaccuracies in the double the answer is erroneously rounded up to 30). It looks like we'll have to switch to a different approach.
Let's revisit your initial solution of just doubling the value of x until it exceeds w should be perfectly fine here. The maximum number of times you can double x until it reaches w is given by log2 (w/x), and since w/x is at most one billion, this iterates at most log2 109 times, which is about thirty times each. Doing thirty iterations of a multiply by two is probably going to be extremely fast. More generally, if the upper bound of w / x is U, then this will take at most O(log U) time to complete. If you have k (x, w) pairs to check, this takes time O(k log U).
If you're not satisfied with doing this, though, there's another very fast algorithm you could try. Essentially, you want to compute log2 w/x. You could start off by creating a table that lists all powers of two along with their logarithms. For example, your table might look like
T[1] = 0
T[2] = 1
T[4] = 2
T[8] = 3
...
You could then compute w/x, then do a binary search to figure out where in which range the value lies. The upper bound of this range is then the number of times the ball must bounce. This means that if you have k different pairs to inspect, and if you know that the maximum ratio of w/x is U, creating this table takes O(log U) time and each query then takes time proportional to the log of the size of the table, which is O(log log U). The overall runtime is then O(log U + k log log U), which is extremely good. Given that you're dealing with at most one million problem instances and that U is one billion, k log log U is just under five million, and log U is about thirty.
Finally, if you're willing to do some perversely awful stuff with bitwise hackery, since you know for a fact that w/x fits into a 32-bit word, you can use this bitwise trickery with IEEE doubles to compute the logarithm in a very small number of machine operations. This would probably be faster than the above two approaches, though I can't necessarily guarantee it.
Hope this helps!
Use this formula to calculate the number of bounces for each test case.
ceil( log(w/x) / log(2) )
This is pseudo-code, but it should be pretty simple to convert it to any language. Just replace log with a function that finds the logarithm of a number in some specific base and replace ceil with a function that rounds up a given decimal value to the next int above it (for example, ceil(2.3) = 3).
See http://www.purplemath.com/modules/solvexpo2.htm for why this works (in your case, you're trying to solve the equation x * 2 ^ n = w for an integer n, and you should start by dividing both sides by x).
EDIT:
Before using this method, you should check that w > x and return 1 if it isn't. (The ball always has to bounce at least once).
Also, it has been pointed out that inaccuracies in floating point values may cause this method to sometimes fail. You can work around this by checking if 2 ^ (n-1) >= w, where n is the result of the equation above, and if so returning (n - 1) instead of n.