I have two languages:
A = { <M, w> | M accepts w after running for at most 2^500 steps }
B = { <M, w, 1^t> | M accepts w after running for at most t steps }
I need to figure out if these languages are in the class P. I know a language is in the class P if it runs in poly-time. I'm pretty sure the language A runs in exponential time but I'm not quite sure if a constant like 2^500 makes it poly-time instead.
Any help appreciated, thanks!
Algorithmic time is expressed as a function of the input size. If, for any input, A takes 2^500 steps, then it is actually constant time (no matter what the input, the running time is constant), which is definitely in P.
B takes t steps, where t is presumably the size of the input, so it is linear time (the time increases linearly with the input size), which is also in P.
If you have a problem that requires, for example, 2^t steps or t! (factorial) steps, then it is NOT in P. Look up Big O notation
A language is in P, if there exist an algorithm, that can decide if a given input belongs to the language and runs in polynomial time. Polynomial time, means that you can find an upper bound by an polynomial function in the length of the input.
To address your example:
Define the algorithm AL₁ as:
For the element <M,w> run M(w).
Since M(w) takes at most a constant number c = 2⁵⁰⁰ steps, the complexity of AL₁ is bounded by a constant number, which is polynomial. AL₁ ∈ O(1). So A is in P.
Define algorithm AL₂ as:
For the element <M,w,1t> run M(w,1t).
Since M(w,1t) takes at most t steps, we have to bring the input length and t together. Notice, that the input consists of w and 1t. 1t means that the number t is input as unary (e.g. 1⁵ = 11111₁ = 5₁₀). This is important for the input length. E.g. the length of 1²⁵⁶ is 256, but the length of 256 = 2⁸ is only 8 = log₂(256) in binary.
So the input length is len(w) + t and AL₂ ∈ O(t) holds. So the complexity of AL₂ is also bounded by a polynomial and so the language B is in P.
Let me add an example to show haw important it is to differ between unary and other numeral systems.
C = { <M, w, t> | M accepts w after running for at most t steps }
C is basically like B, but t is not in unary numeral system, so the length of t is logc(t). The basis c of the logarithm does not matter, because it results only in a constant factor.
The input length is now len(w) + log(t), so O(t) is not necessary polynomial in the input length. Lets say len(w) is a constant number (for simplification) , then t = clogc(t) is exponential in the input length and thus C is not in P.
Related
My question isn't about some specific code but rather theoritical, so my apologies in advance if that's not the right place to post it.
Given a certain problem - for the sake of discussion, let it be a permutation problem - with an input of size N, and P the constant time needed to calculate 1 permutation; in this case, we need about T = ~ (P)*(N!) = O(N!) time to produce all results. If by any chance our algorithm takes much more than the expected time, it may be safe to assume it doesn't terminate.
For example, for P = 0.5 secs, N = 8, ⇒ T = 0.5*8! = 20.160 secs. Any value above T is 'suspicious' time.
My question is, how can one introduce a probability function which asymptotically reaches 1, while running time increases infinitely?
Our 'evaluation' shall depend on a constant time P, the size N of our input, and the time complexity Q of our current problem, thus it may have this form: f(P, N, Q) = ... , whereas 0 ≤ f ≤ 1, f increasing, and f indicates the probability of having 'fallen' in a non-terminating state.
If this approach isnt enough, how can we make sure that after a certain amount of time, our program is probably running endlessly?
I would like to understand why the following naive primality test algorithm is not polynomial.
IsPrime (n: an integer)
Begin
For i=2 to n-1 do
If (n % i == 0) then
return (no)
EndIf
EndFor
return (yes)
End
This algorithm is said to be exponential in the size of the input n. Why is it true? And why the following sorting test algorithm is said polynomial and not exponential?
IsSorted (T[n]: an array of n integer)
Begin
For i = 1 to n-1 do
If (T[i] > T[i+1]) then
return (no)
EndIf
EndFor
return (yes)
End
The input size is typically measured in bits. To represent the number n the input size would be log2(n). The primitive primality test is linear in n, but exponential in log2(n).
The naive primarily test is polynomial in the value of the input (that is, the actual number which the function receives), but exponential in the size (bits, bytes, etc.) of the input.
If you have a number n consisting of b bits, we have b = O(log n) and also n = O(2b).
The running time is thus O(n) or O(2b).
To second the answer given by Henry, the algorithm in the original question in fact has a polynomially bounded running time - if unary encoding is used for the input!
More precisely, the runtime bound does not only depend on the algorithm itself but also the used encoding scheme. Consider the following algorithm in C-like syntax.
INPUT: integer n
for (int i = 0; i < n; i++)
{
wait one second
}
Apparently, the algorithm takes n seconds to terminate; the time is linear in n. If the input is encoded using unary encoding, the amount of time scales linearly in the encoding length of n. However, if n is encoded using binary encoding the amount of time scales exponentially in the encoding length of n (as the encoding length of n scales logarithmically in the value of n).
To put it all in a nutshell, the statement that the algorithm in the question is not polynomial without any additional information is not correct. However, apparently it is a convention that binary encoding (or any other positional notation) is used unless otherwise stated.
That being said, I admit that the depenence of the runtime bound on the encoding scheme tends to be taught a bit imprecisely. The term pseudopolynomial is also floating around.
I'm trying to understand the time complexity (Big-O) of the following algorithm which finds x such that g^x = y (mod p) (i.e. finding the discrete logarithm of y with base g modulo p).
Here's the pseudocode:
discreteLogarithm(y, g, p)
y := y mod p
a := g
x := 1
until a = y
a := (a * g) mod p
x++
return x
end
I know that the time complexity of this approach is exponential in the number of binary digits in p - but what does this mean and why does it depend on p?
I understand that the complexity is determined by the number of loops (until a = y), but where does p come into this, what's this about binary digits?
The run time depends upon the order of g mod p. The worst case is order (p-1)/2, which is O(p). The run time is thus O(p) modular multiplies. The key here is that p has log p bits, where I use 'log' to mean base 2 logarithm. Since p = 2^( log p ) -- mathematical identity -- we see the run time is exponential in the number of bits of p. To make it more clear, let's use b=log p to represent the number of bits. The worst case run time is O(2^b) modular multiplies. Modular multiplies take O(b^2) time, so the full run time is O(2^b * b^2) time. The 2^b is the dominant term.
Depending upon your particular p and g, the order could be much smaller than p. However, some heuristics in analytical number theory show that on average, it is order p.
EDIT: If you are not familiar with the concept of 'order' from group theory, here is brief explanation. If you keep multiplying g by itself mod p, it eventually comes to 1. The order is the number of multiplies before that happens.
I have simple algorithm, something like
h = SHA1(message)
r = a^b mod p
r = h * r mod p
l = Str1 || Str2
if ( l == r)
return success
else
return false
Now I want to compute its complexity, but I didn't konw how to do it. I don't know e.g. how the multiplication is done, so I don't understand how to do it. Assume worst case O(n^2) or best case or average case? Maybe I must look on it from other side?
Additionaly the numbers are keep as a byte arrays.
If you want to know the complexity of this algorithm, you just have to add the complexitys of the operations you use and sum it up.
sha1(message) has a complexity depending on the length m of the message, so lets say poly(m), since I dont know the complexity of sha1.
ab mod p can be done in O(log b) multiplications.
h * r mod p is exactly one multiplication
Str1 || Str2 Is this bitwise or? If yes it will take O(s) where s is the length of Str1
l == r will take as much comparisons as the length of the byte array is. This will also be s.
When numbers are realy big. The can not multiplicated in one processor step, so complexity of one multiplications will be in O(log p), since log p is the length of the numbers.
All together you get O(poly(m) + log(b) ⋅ log(p) + s).
Notice: If the length of the numbers (log(b) and log(p)) will never change, this part will be constant. This also holds for s.
You said the numbers are 256 Bit long, so the complexity is only O(poly(m)), which is the complexity of the Sha1-algorithm.
Notice: If you have an algorithm with any complexity, an you only use input of a fixed length, the complexity will always be constany. Complexity is a tool to see how the runtime will expand if the input is growing. If it is not growing, the runtime will also not.
If your input has always a fixed length, than you are more interested in the performance of the implementation of an algorithm.
As part of a programming assignment I saw recently, students were asked to find the big O value of their function for solving a puzzle. I was bored, and decided to write the program myself. However, my solution uses a pattern I saw in the problem to skip large portions of the calculations.
Big O shows how the time increases based on a scaling n, but as n scales, once it reaches the resetting of the pattern, the time it takes resets back to low values as well. My thought was that it was O(nlogn % k) when k+1 is when it resets. Another thought is that as it has a hard limit, the value is O(1), since that is big O of any constant. Is one of those right, and if not, how should the limit be represented?
As an example of the reset, the k value is 31336.
At n=31336, it takes 31336 steps but at n=31337, it takes 1.
The code is:
def Entry(a1, q):
F = [a1]
lastnum = a1
q1 = q % 31336
rows = (q / 31336)
for i in range(1, q1):
lastnum = (lastnum * 31334) % 31337
F.append(lastnum)
F = MergeSort(F)
print lastnum * rows + F.index(lastnum) + 1
MergeSort is a standard merge sort with O(nlogn) complexity.
It's O(1) and you can derive this from big O's definition. If f(x) is the complexity of your solution, then:
with
and with any M > 470040 (it's nlogn for n = 31336) and x > 0. And this implies from the definition that:
Well, an easy way that I use to think about big-O problems is to think of n as so big it may as well be infinity. If you don't get particular about byte-level operations on very big numbers (because q % 31336 would scale up as q goes to infinity and is not actually constant), then your intuition is right about it being O(1).
Imagining q as close to infinity, you can see that q % 31336 is obviously between 0 and 31335, as you noted. This fact limits the number of array elements, which limits the sort time to be some constant amount (n * log(n) ==> 31335 * log(31335) * C, for some constant C). So it is constant time for the whole algorithm.
But, in the real world, multiplication, division, and modulus all do scale based on input size. You can look up Karatsuba algorithm if you are interested in figuring that out. I'll leave it as an exercise.
If there are a few different instances of this problem, each with its own k value, then the complexity of the method is not O(1), but instead O(k·ln k).