If I have an algorithm,where the input is just a number and the output is its set of divisors.So my input will always be one number and the number of iterations in the algorithm will depend on how big the number is.What will be the big-oh notation of such an algorithm?
Algorithm:
1: Set m := 2.
2: Set S := {} for S a multi-set.
3: while m <= n^0.5
do
4: if m divides N then
5: Set S U m
6: else
7: Set m = m + 1.
8: end if
9: end while
10: Return the set S of divisors found.
You can either let the variable in big-O notation for the asymptotic complexity be the input number itself, or it can be the number of bits needed to express that number. These two conventions give rise to dramatically different asymptotic classifications, so it is important to be clear about which one you use when you report the result.
In general people tend to use the number-of-bits convention when they are speaking about cases where the number is so large that you need bignums, and the meaning-of-the-number convention when the inputs are bounded by the size of a machine word. But that's not something you can rely on other than to get a first guess which you'll need to verify for yourself makes sense in your particular situation.
The choice tends to go hand-in-hand with the cost model you're using for arithmetic operations. When you're counting bits it's typical to assume that arithmetic on n-bit values take O(n) time, whereas when you're working with the meaning of the input number, you typically assume that arithmetic on numbers work in constant time.
In your case you would get something like O(2^n) or O(sqrt(m)) where n is the number of bits in the input and m is the input itself. (Details depend on how your multiset primitives perform).
See also pseudo-polynomial time.
Related
Problem Instance and Big-O Notation
Hello,
I am trying to understand a solution to a problem, but I am not understanding a part of the solution as to how the Problem instance was calculated.
Here are the questions and the solutions:
"An instance of a 'Perfect' decision problem is an integer n >= 1 and the problem is to decide if n is the sum of each of its proper divisors. For example, 6 is 'Perfect' because 1+2+3 = 6."
Since the input to 'Perfect' is a single integer n, an appropriate size parameter is m = log(n), since this is roughly the number of bits needed to represent n.
"Suppose an algorithm for deciding 'Perfect' requires O(n^2) steps, where n is the problem instance. Use the answer above and Big-O growth terminology to describe the growth of the algorithm's running time"
The algorithm has exponential running time since n^2 = (2^(logn))^2 = 4^(logn)
I can't seem to understand or figure out how the problem instance with the parameter size m = logn become 2^(logn)....
We are talking about m bits. m is equal to log(n), according to the statement.
Now every bit can be either 0 or 1.
Suppose you have to a representation of 2 bits : _ _. Each one of them can either be 0 or 1. And possible representations become 2^2=4.
Similaryly, the possible values for the above case become 2^m or 2^log(n).
As stated by #AbhinavMathur above, the text is clearly wrong. The time to solve the problem is exponential in the number of bits.
I've seen coding problems that are similar to this:
int doSomething(String s)
Where it says in the problem description that s will contain at most one of every character, so s cannot be more than length 26. I think in this case, iterating over s would be constant time.
But I've also seen problems where inputs are constrained to a random large number, like 10^5, just to avoid stack overflows and other weird edge cases. If we are going to consider inputs that are constrained by constants to be constant complexity, shouldn't these inputs also be considered constant complexity?
But it doesn't make sense to me to consider s to be of O(n) complexity either, because there are many problems were people allocate char[26] arrays to hold every letter of the alphabet. How does if make sense to consider an input that we know will be less than or equal to 26 to be of greater complexity than an array of size 26?
The point of analyzing the complexity of algorithms is to estimate how long it will take to run it. If the problem you're trying to solve limits the maximum value of n to a constant, you can consider n to be a constant and you wouldn't be wrong. But would that be useful if you wanted to predict whether an algorithm that does 2^n operations will run in a few seconds for n = 26? On the other hand, if you had an algorithm that does n*m operations and m is at most 3, how useful would it be to include m in the complexity analysis?
Calculating complexity has focus on what is the most critical variable related to the running time. If the running time is dominant by the length of s, it is our main focus of analyzing complexity and that should be in bigO notation. And in that case, of course it's not a constant.
If the input is constrained to a large number like 10^5.
And if the algorithm is getting slower proportional to that input.
for example,
int sort(string s); //length of s is less than 10^5
In this case, depending on what sorting algorithm you use,
the running time will be proportional to the length of s
like O(n^2) or O(nlogn) if n is the length of s
In this case you cannot say it's constant because running time is very different as the length of s is changing.
But if the algorithm inside has nothing to do with the length of s, like it has constant calculation time, then you can say 10^5 constraint is just a constant.
I am comparing two algorithms that determine whether a number is prime. I am looking at the upper bound for time complexity, but I can't understand the time complexity difference between the two, even though in practice one algorithm is faster than the other.
This pseudocode runs in exponential time, O(2^n):
Prime(n):
for i in range(2, n-1)
if n % i == 0
return False
return True
This pseudocode runs in half the time as the previous example, but I'm struggling to understand if the time complexity is still O(2^n) or not:
Prime(n):
for i in range(2, (n/2+1))
if n % i == 0
return False
return True
As a simple intuition of what big-O (big-O) and big-Θ (big-Theta) are about, they are about how changes the number of operations you need to do when you significantly increase the size of the problem (for example by a factor of 2).
The linear time complexity means that you increase the size by a factor of 2, the number of steps you need to perform also increases by about 2 times. This is what called Θ(n) and often interchangeably but not accurate O(n) (the difference between O and Θ is that O provides only an upper bound but Θ guarantees both upper and lower bounds).
The logarithmic time complexity (Θ(log(N))) means that when increase the size by a factor of 2, the number of steps you need to perform increases by some fixed amount of operations. For example, using binary search you can find given element in twice as long list using just one ore loop iterations.
Similarly the exponential time complexity (Θ(a^N) for some constant a > 1) means that if you increase that size of the problem just by 1, you need a times more operations. (Note that there is a subtle difference between Θ(2^N) and 2^Θ(N) and actually the second one is more generic, both lie inside the exponential time but neither of two covers it all, see wiki for some more details)
Note that those definition significantly depend on how you define "the size of the task"
As #DavidEisenstat correctly pointed out there are two possible context in which your algorithm can be seen:
Some fixed width numbers (for example 32-bit numbers). In such a context an obvious measure of the complexity of the prime-testing algorithm is the value being tested itself. In such case your algorithm is linear.
In practice there are many contexts where prime testing algorithm should work for really big numbers. For example many crypto-algorithms used today (such as Diffie–Hellman key exchange or RSA) rely on very big prime numbers like 512-bits, 1024-bits and so on. Also in those context the security is measured in the number of those bits rather than particular prime value. So in such contexts a natural way to measure the size of the task is the number of bits. And now the question arises: how many operations do we need to perform to check a value of known size in bits using your algorithm? Obviously if the value N has m bits it is about N ≈ 2^m. So your algorithm from linear Θ(N) converts into exponential 2^Θ(m). In other words to solve the problem for a value just 1 bit longer, you need to do about 2 times more work.
Exponential versus linear is a question of how the input is represented and the machine model. If the input is represented in unary (e.g., 7 is sent as 1111111) and the machine can do constant time division on numbers, then yes, the algorithm is linear time. A binary representation of n, however, uses about lg n bits, and the quantity n has an exponential relationship to lg n (n = 2^(lg n)).
Given that the number of loop iterations is within a constant factor for both solutions, they are in the same big O class, Theta(n). This is exponential if the input has lg n bits, and linear if it has n.
i hope this will explain you why they are in fact linear.
suppose you call function and see how many time they r executed
Prime(n): # 1 time
for i in range(2, n-1) #n-1-1 times
if n % i == 0 # 1 time
return False # 1 time
return True # 1 time
# overall -> n
Prime(n): # Time
for i in range(2, (n/2+1)) # n//(2+1) -1-1 time
if n % i == 0 # 1 time
return False # 1 time
return True # 1 time
# overall -> n/2 times -> n times
this show that prime is linear function
O(n^2) might be because of code block where this function is called.
Why does this algo have an exponential time complexity?
I understand that "Modulus" is a bitwise operator and operates on individual bits. Hence In the worst case, we need to perform sqrt(2^n) divisions. So this is an exp time algorithm.
If that is true, won't all algo become exponential time? Please explain.
Find-Factor(X)
1: if X is even then
2: return ”2 is a factor”
3: end if
4: for i = 3 to Sqrt(X) by +2 do
5: test if X%i = 0, if yes, output ”i is a factor”
6: end for
7: return ”X is a prime.”
Exponentiality comes from the number of iterations. In the worst case (which is the case number is prime) You have to do X / 2 iterations (in this algorithm and it is not a good one, for instance you can limit the loop with sqrt(X) instead of X). And this is exponential with the number of bits in your input = ln(X). Not with number X.
By the way, there are probabilistic checks which determines whether a given number is prime or not very quickly. And also there is a rather complicated algorithm which does the same job deterministically. You can google and find them.
They're exponential in the length of the number's representation because the representation itself is exponential, the next digit is 10n+1 or 2n+1 or whatever. The linear search just has that much more to search through, that's all.
When talking about time complexity we usually use n as input, which is not a precise measure of the actual input size. I am having trouble showing that, when using specific size for input (s) an algorithm remains in the same complexity class.
For instance, take a simple Sequential Search algorithm. In its worst case it takes W(n) time. If we apply specific input size (in base 2), the order should be W(lg L), where L is the largest integer.
How do I show that Sequential Search, or any algorithm, remains the same complexity class, in this case linear time? I understand that there is some sort of substitution that needs to take place, but I am shaky on how to come to the conclusion.
EDIT
I think I may have found what I was looking for, but I'm not entirely sure.
If you define worst case time complexity as W(s), the maximum number of steps done by an algorithm for an input size of s, then by definition of input size, s = lg n, where n is the input. Then, n = 2^s, leading to the conclusion that the time complexity is W(2^s), an exponential complexity. Therefore, the algorithm's performance with binary encoding is exponential, not linear as it is in terms of magnitude.
When talking about time complexity we usually use n as input, which is not a precise measure of the actual input size. I am having trouble showing that, when using specific size for input (s) an algorithm remains in the same complexity class.
For instance, take a simple Sequential Search algorithm. In its worst case it takes W(n) time. If we apply specific input size (in base 2), the order should be W(lg L), where L is the largest integer.
L is a variable that represents the largest integer.
n is a variable that represents the size of the input.
L is not a specific value anymore than n is.
When you apply a specific value, you aren't talking about a complexity class anymore, you are talking about an instance of that class.
Let's say you are searching a list of 500 integers. In other words, n = 500
The worst-case complexity class of Sequential Search is O(n)
The complexity is n
The specific instance of worst-case complexity is 500
Edit:
Your values will be uniform in the number of bits required to encode each value. If the input is a list of 32bit integers, then c = 32, the number of bits per integer. Complexity would be 32*n => O(n).
In terms of L, if L is the largest value, and lg L is the number of bits required to encode L, then lg L is the constant c. Your complexity in terms of bits is O(n) = c*n, where c = lg L is the constant specific input size.
What I know is that the maximum number
of steps done by Sequential Search is,
obviously, cn^2 + nlg L. cn^2 being
the number of steps to increment loops
and do branching.
That's not true at all. The maximum number of steps done by a sequential search is going to be c*n, where n is the number of items in the list and c is some constant. That's the worst case. There is no n^2 component or logarithmic component.
For example, a simple sequential search would be:
for (int i = 0; i < NumItems; ++i)
{
if (Items[i] == query)
return i;
}
return -1;
With that algorithm, if you search for each item, then half of the searches will require fewer than NumItems/2 iterations and half of the searches will require NumItems/2 or more iterations. If an item you search for isn't in the list, it will require NumItems iterations to determine that. The worst case running time is NumItems iterations. The average case is NumItems/2 iterations.
The actual number of operations performed is some constant, C, multiplied by the number of iterations. On average it's C*NumItems/2.
As Lucia Moura states: "Except for the unary encoding, all the other encodings for natural
numbers have lengths that are polynomially related"
Here is the source. Take a look at page 19.