I'm trying to understand the time complexity (Big-O) of the following algorithm which finds x such that g^x = y (mod p) (i.e. finding the discrete logarithm of y with base g modulo p).
Here's the pseudocode:
discreteLogarithm(y, g, p)
y := y mod p
a := g
x := 1
until a = y
a := (a * g) mod p
x++
return x
end
I know that the time complexity of this approach is exponential in the number of binary digits in p - but what does this mean and why does it depend on p?
I understand that the complexity is determined by the number of loops (until a = y), but where does p come into this, what's this about binary digits?
The run time depends upon the order of g mod p. The worst case is order (p-1)/2, which is O(p). The run time is thus O(p) modular multiplies. The key here is that p has log p bits, where I use 'log' to mean base 2 logarithm. Since p = 2^( log p ) -- mathematical identity -- we see the run time is exponential in the number of bits of p. To make it more clear, let's use b=log p to represent the number of bits. The worst case run time is O(2^b) modular multiplies. Modular multiplies take O(b^2) time, so the full run time is O(2^b * b^2) time. The 2^b is the dominant term.
Depending upon your particular p and g, the order could be much smaller than p. However, some heuristics in analytical number theory show that on average, it is order p.
EDIT: If you are not familiar with the concept of 'order' from group theory, here is brief explanation. If you keep multiplying g by itself mod p, it eventually comes to 1. The order is the number of multiplies before that happens.
Related
I have this formula:
index = (a * k) % M
which maps a number 'k', from an input set K of distinct numbers, into it's position in a hashtable. I was wondering how to write a non-brute force program that finds such 'M' and 'a' so that 'M' is minimal, and there are no collisions for the given set K.
If, instead of a numeric multiplication you could perform a logic computation (and / or /not), I think that the optimal solution (minimum value of M) would be as small as card(K) if you could get a function that related each value of K (once ordered) with its position in the set.
Theoretically, it must be possible to write a truth table for such a relation (bit a bit), and then simplify the minterms through a Karnaugh Table with a proper program. Depending on the desired number of bits, the computational complexity would be affordable... or not.
If a is co-prime to M then a * k = a * k' mod M if, and only if, k = k' mod M, so you might as well use a = 1, which is always co-prime to M. This also covers all the cases in which M is prime, because all the numbers except 0 are then co-prime to M.
If a and M are not co-prime, then they share a common factor, say b, so a = x * b and M = y * b. In this case anything multiplied by a will also be divisible by b mod M, and you might as well by working mod y, not mod M, so there is nothing to be gained by using an a not co-prime to M.
So for the problem you state, you could save some time by leaving a=1 and trying all possible values of M.
If you are e.g. using 32-bit integers and really calculating not (a * k) mod M but ((a * k) mod 2^32) mod M you might be able to find cases where values of a other than 1 do better than a=1 because of what happens in (a * k) mod 2^32.
Having a system of linear congruences, I'd like to determine if it has a solution. Using simple algorithms that solve such systems is impossible, as the answer may grow exponentially.
One hypothesis I have is that if a system of congruences has no solution, then there are two of them that contradict each other. I have no idea if this holds, if it did that would lead to an easy O(n^2 log n) algo, as checking if a pair of congruences has a solution requires O(log n) time. Nevertheless for this problem I'd rather see something closer to O(n).
We may assume that no moduli exceeds 10^6, especially we can quickly factor them all to begin with. We may even further assume that the sum of all moduli doesn't exceed 10^6 (but still, their product can be huge).
As you suspect, there's a fairly simple way to determine whether the set of congruences has a solution without actually needing to build that solution. You need to:
Reduce each congruence to the form x = a (mod n) if necessary; from the comments, it sounds as though you already have this.
Factorize each modulus n as a product of prime powers: n = p1^e1 * p2^e2 * ... * pk^ek.
Replace each congruence x = a (mod n) with a collection of congruences x = a (mod pi^ei), one for each of the k prime powers you found in step 2.
And now, by the Chinese Remainder Theorem it's enough to check compatibility for each prime independently: given any two congruences x = a (mod p^e) and x = b (mod p^f), they're compatible if and only if a = b (mod p^(min(e, f)). Having determined compatibility, you can throw out the congruence with smaller modulus without losing any information.
With the right data structures, you can do all this in a single pass through your congruences: for each prime p encountered, you'll need to keep track of the biggest exponent e found so far, together with the corresponding right-hand side (reduced modulo p^e for convenience). The running time will likely be dominated by the modulus factorizations, though if no modulus exceeds 10^6, then you can make that step very fast, too, by prebuilding a mapping from each integer in the range 1 .. 10^6 to its smallest prime factor.
EDIT: And since this is supposed to be a programming site, here's some (Python 3) code to illustrate the above. (For Python 2, replace the range call with xrange for better efficiency.)
def prime_power_factorisation(n):
"""Brain-dead factorisation routine, for illustration purposes only."""
# DO NOT USE FOR LARGE n!
while n > 1:
p, pe = next(d for d in range(2, n+1) if n % d == 0), 1
while n % p == 0:
n, pe = n // p, pe*p
yield p, pe
def compatible(old_ppc, new_ppc):
"""Determine whether two prime power congruences (with the same
prime) are compatible."""
m, a = old_ppc
n, b = new_ppc
return (a - b) % min(m, n) == 0
def are_congruences_solvable(moduli, right_hand_sides):
"""Determine whether the given congruences have a common solution."""
# prime_power_congruences is a dictionary mapping each prime encountered
# so far to a pair (prime power modulus, right-hand side).
prime_power_congruences = {}
for m, a in zip(moduli, right_hand_sides):
for p, pe in prime_power_factorisation(m):
# new prime-power congruence: modulus, rhs
new_ppc = pe, a % pe
if p in prime_power_congruences:
old_ppc = prime_power_congruences[p]
if not compatible(new_ppc, old_ppc):
return False
# Keep the one with bigger exponent.
prime_power_congruences[p] = max(new_ppc, old_ppc)
else:
prime_power_congruences[p] = new_ppc
# If we got this far, there are no incompatibilities, and
# the congruences have a mutual solution.
return True
One final note: in the above, we made use of the fact that the moduli were small, so that computing prime power factorisations wasn't a big deal. But if you do need to do this for much larger moduli (hundreds or thousands of digits), it's still feasible. You can skip the factorisation step, and instead find a "coprime base" for the collection of moduli: that is, a collection of pairwise relatively prime positive integers such that each modulus appearing in your congruences can be expressed as a product (possibly with repetitions) of elements of that collection. Now proceed as above, but with reference to that coprime base instead of the set of primes and prime powers. See this article by Daniel Bernstein for an efficient way to compute a coprime base for a set of positive integers. You'd likely end up making two passes through your list: one to compute the coprime base, and a second to check the consistency.
I have simple algorithm, something like
h = SHA1(message)
r = a^b mod p
r = h * r mod p
l = Str1 || Str2
if ( l == r)
return success
else
return false
Now I want to compute its complexity, but I didn't konw how to do it. I don't know e.g. how the multiplication is done, so I don't understand how to do it. Assume worst case O(n^2) or best case or average case? Maybe I must look on it from other side?
Additionaly the numbers are keep as a byte arrays.
If you want to know the complexity of this algorithm, you just have to add the complexitys of the operations you use and sum it up.
sha1(message) has a complexity depending on the length m of the message, so lets say poly(m), since I dont know the complexity of sha1.
ab mod p can be done in O(log b) multiplications.
h * r mod p is exactly one multiplication
Str1 || Str2 Is this bitwise or? If yes it will take O(s) where s is the length of Str1
l == r will take as much comparisons as the length of the byte array is. This will also be s.
When numbers are realy big. The can not multiplicated in one processor step, so complexity of one multiplications will be in O(log p), since log p is the length of the numbers.
All together you get O(poly(m) + log(b) ⋅ log(p) + s).
Notice: If the length of the numbers (log(b) and log(p)) will never change, this part will be constant. This also holds for s.
You said the numbers are 256 Bit long, so the complexity is only O(poly(m)), which is the complexity of the Sha1-algorithm.
Notice: If you have an algorithm with any complexity, an you only use input of a fixed length, the complexity will always be constany. Complexity is a tool to see how the runtime will expand if the input is growing. If it is not growing, the runtime will also not.
If your input has always a fixed length, than you are more interested in the performance of the implementation of an algorithm.
P(x,y,z){
print x
if(y!=x) print y
if(z!=x && z!=y) print z
}
Trivial Algorithm here, values x,y,z are chosen randomly from {1,...r} with r >= 1.
I'm trying to determine the average case complexity of this algorithm and I measure complexity based on the number of print statements.
The best case here is T(n) = 1 or O(1), when x=y=z and the probability of that is 1/3.
The worst case here is still T(n) = 3 or still O(1) when x!=y!=z and the probability is 2/3.
But when it comes to mathematically deriving the average case:
Sample Space is n possible inputs, Probability over Sample Space is 1/n chance
So, how do I calculate average case complexity? (This is where I draw a blank..)
Your algorithm has three cases:
All three numbers are equal. The probability of this is 1/r, since
once you choose x, there's only one choice for y and for z. The cost
for this case is 1.
x != y, but x == z or y == z. The probability of this is 1/r * (1/(r - 1))* 1/2,
since once you choose x, you only have r -1 choices left for y, and z can only be
one of these two choices. Cost = 2.
All three numbers are distinct. Probability that all three are distinct is
1/r * (1/(r - 1))*(1/(r - 2)). Cost = 3.
Thus, the average case can be computed as:
1/r + 1/r * (1/(r - 1)) + 1/r * (1/(r - 1))*(1/(r - 2)) * 3 == O(1)
Edit: The above expression is O(1), since the whole expression is made up of constants.
The average case will be somewhere between the best and worst cases; for this particular problem, that's all you need (at least as far as big-O).
1) Can you program the general case at least? Write the (pseudo)-code and analyze it, it might be readily apparent. You may actually program it suboptimally and there may exist a better solution. This is very typical and it's part of the puzzle-solving of the mathematics end of computer science, e.g. it's hard to discover quicksort on your own if you're just trying to code up a sort.
2) If you can, then run a monte carlo simulation and graph the results. i.e., for N = 1, 5, 10, 20, ..., 100, 1000, or whatever sample is realistic, run 10000 trials and plot the average time. If you're lucky X = sample size, Y = avg. time for 10000 runs at that sample size will graph out a nice line, or parabola, or some easy-to-model curve.
So I'm not sure if you need help on (1) finding or coding the algorithm or (2) analyzing it, you will probably want to revise your question to specify this.
P(x,y,z){
1.print x
2.if(y!=x)
3. print y
4.if(z!=x && z!=y)
5. print z
}
Line 1: takes a constant time c1 (c1:print x)
Line 2: takes a constant time c2 (c2:condition test)
Line 3 :takes a constant time c3 (c3 :print y)
Line 3: takes a constant time c4 (c4:condition test)
Line 4: takes a constant time c5 (c5:print z)
Analysis :
Unless your function P(x,y,z) does not depend on input size " r" the program will take a constant amount of time to run since Time Taken :T(c1)+T(c2+c3)+T(c4+c5) ..summing up the Big O of the function P(x,y,z) is O(1) where 1 is a constant and indicates constant amount of time since T(c1),T(c2),..T(c5) all take constant amount of time.. and say if the function P(x,y,z) iterates from 1 to r..then the complexity of your snippet would have changed and will be in terms of the input size i.e "r"
Best Case : O(1)
Average Case : O(1)
worst Case : O(1)
Pollard Rho factorization method uses a function generator f(x) = x^2-a(mod n) or f(x) = x^2+a(mod n) , is the choice of this function (parabolic) has got any significance or we may use any function (cubic , polynomial or even linear) as we have to identify or find the numbers belonging to same congruence class modulo n to find the non trivial divisor ?
In Knuth Vol II (The Art Of Computer Programming - Seminumerical Algorithms) section 4.5.4 Knuth says
Furthermore if f(y) mod p behaves as a random mapping from the set {0,
1, ... p-1} into itself, exercise 3.1-12 shows that the average value
of the least such m will be of order sqrt(p)... From the theory in
Chapter 3, we know that a linear polynomial f(x) = ax + c will not be
sufficiently random for our purpose. The next simplest case is
quadratic, say f(x) = x^2 + 1. We don't know that this function is
sufficiently random, but our lack of knowledge tends to support the
hypothesis of randomness, and empirical tests show that this f does
work essentially as predicted
The probability theory that says that f(x) has a cycle of length about sqrt(p) assumes in particular that there can be two values y and z such that f(y) = f(z) - since f is chosen at random. The rho in Pollard Rho contains such a junction, with the cycle containing multiple lines leading on to it. For a linear function f(x) = ax + b then for gcd(a, p) = 1 mod p (which is likely since p is prime) f(y) = f(z) means that y = z mod p, so there are no such junctions.
If you look at http://www.agner.org/random/theory/chaosran.pdf you will see that the expected cycle length of a random function is about the sqrt of the state size, but the expected cycle length of a random bijection is about the state size. If you think of generating the random function only as you evaluate it you can see that if the function is entirely random then every value seen so far is available to be chosen again at random to find a cycle, so the odds of closing the cycle increase with the cycle length, but if the function has to be invertible the only way to close the cycle is to generate the starting point, which is much less likely.