The “APS” (All Possible Sorting) algorithm sorts an array A of size n by generating all possible sequences of elements of n, and for each sequence, checking to see if the elements are in sorted (ascending) order.
a) What is the worst-case time complexity of APS? Explain your logic / show your work.
My answer:
Worst case is O(n!) because it generates all possible sequences and then checks if sorted.
Preferably, I would like someone to tell me if I'm right or wrong and how to get to the answer. This big O stuff confuses me.
APS is generating all possible permutations of N elements, which gives you n! different possible sortings, so you are correct.
Proving that it is O(n!) just requires you to prove that the runtime is asymptotically upper-bounded by n!, which basically means you have to prove that:
f(n) = O(n!) if, for some m and c, |f(n)| < |m * n!| for all n > c.
Proving this is easier if you have the actual algorithm written out, but if you walk through your logic it should do the trick.
Related
I was reading about Big-O Notation
So, any algorithm that is O(N) is also an O(N^2).
It seems confusing to me, I know that Big-O gives upper bound only.
But how can an O(N) algorithm also be an O(N^2) algorithm.
Is there any examples where it is the case?
I can't think of any.
Can anyone explain it to me?
"Upper bound" means the algorithm takes no longer than (i.e. <=) that long (as the input size tends to infinity, with relevant constant factors considered).
It does not mean it will ever actually take that long.
Something that's O(n) is also O(n log n), O(n2), O(n3), O(2n) and also anything else that's asymptotically bigger than n.
If you're comfortable with the relevant mathematics, you can also see this from the formal definition.
O notation can be naively read as "less than".
In numbers if I tell you x < 4 well then obviously x<5 and x< 6 and so on.
O(n) means that, if the input size of an algorithm is n (n could be the number of elements, or the size of an element or anything else that mathematically describes the size of the input) then the algorithm runs "about n iterations".
More formally it means that the number of steps x in the algorithm satisfies that:
x < k*n + C where K and C are real positive numbers
In other words, for all possible inputs, if the size of the input is n, then the algorithm executes no more than k*n + C steps.
O(n^2) is similar except the bound is kn^2 + C. Since n is a natural number n^2 >= n so the definition still holds. It is true that, because x < kn + C then x < k*n^2 + C.
So an O(n) algorithm is an O(n^2) algorithm, and an O(N^3 algorithm) and an O(n^n) algorithm and so on.
For something to be O(N), it means that for large N, it is less than the function f(N)=k*N for some fixed k. But it's also less than k*N^2. So O(N) implies O(N^2), or more generally, O(N^m) for all m>1.
*I assumed that N>=1, which is indeed the case for large N.
Big-O notation describes the upper bound, but it is not wrong to say that O(n) is also O(n^2). O(n) alghoritms are subset of O(n^2) alghoritms. It's the same that squares are subsets of all rectangles, but not every rectangle is a square. So technically it is correct to say that O(n) alghoritm is O(n^2) alghoritm even if it is not precise.
Definition of big-O:
Some function f(x) is O(g(x)) iff |f(x)| <= M|g(x)| for all x >= x0.
Clearly if g1(x) <= g2(x) then |f(x)| <= M|g1(x)| <= M|g2(x)|.
For an algorithm with just a single Loop will get a O(n) and algorithm with a nested loop will get a O(n^2).
Now consider the Bubble sort algorithm it uses the nested loop in it,
If we give an already sort set of inputs to a bubble sort algorithm the inner loop will never get executed so for a scenario like this it gets O(n) and for the other cases it gets O(n^2).
I need to prove the correctness of Heap's algorithm for generating permutations. The pseudocode for it is as follows:
HeapPermute(n)
//Implements Heap’s algorithm for generating permutations
//Input: A positive integer n and a global array A[1..n]
//Output: All permutations of elements of A
if n = 1
write A
else
for i ←1 to n do
HeapPermute(n − 1)
if n is odd
swap A[1] and A[n]
else swap A[i] and A[n]
(taken from Introduction to the Design and Analysis of Algorithms by Levitin)
I know I need to use induction to prove its correctness, but I'm not sure exactly how to go about doing so. I've proved mathematical equations but never algorithms.
I was thinking the proof would look something like this...
1) For n = 1, heapPermute is obviously correct. {1} is printed.
2) Assume heapPermute() outputs a set of n! permutations for a given n. Then
??
I'm just not sure how to go about finishing the induction step. Am I even on the right track here? Any help would be greatly appreciated.
For n = 1, heapPermute is obviously correct. {1} is printed.
Assume heapPermute() outputs a set of n! permutations for a given n. Then
??
Now, given the first two assumptions, show that heapPermutate(n+1) returns all the (n+1)! permutations.
Yes, that sounds like a good approach. Think about how to recursively define a set of all permutations, i.e. how can be permutations of {1..n} be expressed in terms of permutations of {1.. n-1}. For this, recall the inductive proof that there are n! permutations. How does the inductive step proceed there?
A recursive approach is definitely the way to go. Given your first two steps, to prove that heapPermutate(n+1) returns all the $(n+1)!$ permutations, you may want to explain that each element is adjoined to each permutation of the rest of the elements.
If you would like to have a look at an explanation by example, this blog post provides one.
I recently had a friend challange me to solve a question that he saw on a job interview:
N elements are split into k arrays. Find an algorithm that returns whether any of the elements are identical in (k log N) time.
Please do not provide an answer, I'd really like to solve it myself.
The question I have is: Is there a website similar to regexpal to test the complexity of my algorithm? If not, does anyone have any suggestions on how to actually find the complexity?
I have a general idea, but it's been a while since I last tried to do a problem like this.
Edit: New question to add to this. how does K Log N compare to N log N. Obviously when K is 1 it's just log N, which is more efficient than O(n), but if K >= n it's even worse than N log N, correct?
You usually find the complexity of the algorithm by reasoning and mathematical proof. Running the algorithm on some data won't give you the true complexity (BigOh or Theta) it'll just give you some estimate on the data that was provided.
Quicksort for instance is average BigOh n log n but worst case BigOh n^2, the worst case happens if you select a bad pivot.
You need to figure out the worst cases yourself and do the analysis I'm afraid. If you need help (or a reminder) doing the analysis a good place to look is YouTube or iTunesU in the computer science category on major US universities (for instance the MIT Introduction to Algorithms).
Edit: New question to add to this. how does K Log N compare to N log N. Obviously when K is 1 it's just log N, which is more efficient than O(n), but if K >= n it's even worse than N log N, correct?
To answer your edit, you could use the klogn algorithm when k < n and the nlogn algo when n < k
So most everyone should know that max_element(unsorted_array) can be solved in O(n) time. I realized that while that it is easy to compute that, it seems it would be much harder to solve it in a less than optimal solution, such as n*log(log(n)) time. Now obviously an algorithm could simply be O(n + n*log(log(n)) ), where the more time consuming part of the algorithm has no real purpose. At the same time, you could just use the regular O(n) algorithm log(log(n)) times. Neither of these are very interesting.
So my question is, is there an algorithm that can find the max element in a set of numbers (stored in the container of your choice) in a way that there is no redundant loops or operations, but is Θ(n*log(log(n))) ?
Van Emde Boas Trees?
There is a basic misconception here:
O(n + n*log(log(n)) ) is exactly identical to O(n log(log(n)))
Please read the wiki page carefully: http://en.wikipedia.org/wiki/Big_O_notation
The Big-O notation is asymptotic. This means that O(f(n) + g(n)) = O(max(f(n), g(n))) for all functions f, g. This is not a trick, they are really equal.
Symbols like O(n^2), O(n), etc., are not functions, they are sets; specifically O(f(n)) means "the set of all functions which are asymptotically less than or equal to a constant times f(n)". If f(n) >= g(n), then O(f(n)) contains O(g(n)) and so adding g(n) into that equation changes nothing.
How about proof that it cannot be done.
Theory: It is possible to determine the maximum element of an unsorted array without examining every element.
Assume you have examined all but one element of the unsorted array of n (n>1) items.
There are two possibilities for largest element of the array.
The largest element you have yet seen (out of n-1).
The one element you have not seen
The unexamined element could be bigger (unless an examined element is the absolute maximum value representable); the array is unsorted.
Result: Contradiction. You must examine the nth element in order to determine the maximum (in a mathematics context; you can take a shortcut in computer science under one probably rare circumstance)
Since it doesn't matter what value n has for this, it should apply for all n except the degenerate case (n=1)
If this isn't a valid response, I may be unclear on the requirements... ?
our assignment is to implement a linked list class to do the following with time complexity of O(sqrt(n)):
insert an element at i th place
delete an element at i th place
retrive i th element of the list.
I found a few data structures like skip list. but the time complexity should be O(sqrt(n)).
anyone can help?
Are you having a little trouble understanding the Big O notation?
Remember, Big O means "Upper Bound".
Mathematically, f(x) = O( g(x) ) if f(x) <= C ( g(x) ).
That means, we can say that f(x) is O(x^2) and f(x) is O(x^1000000000), because O(x^2) and O(x^1000000000) are both upper bounds (though, not necessarily "good" upper bounds).
Now, Skip Lists are O(log n) time, as you researched.
That means because log(n) < sqrt(n) ,
we can say that Skip Lists are also O( sqrt(n) ).
Skip lists have complexity O(log n) for search, deletion and insertion, so they satisfy your requirement. Remember that any function that is O(log n) is also O(sqrt(N)), because sqrt grows asymptotically faster than log.
It seems like this should be doable with a simplified version of the skip list idea using only two linked lists. But you would need to be somewhat clever with the algorithms to make sure that the maximum number of elements in the shorter list is bounded by O(sqrt(N)) and the maximum distance in the longer list between adjacent elements of the shorter list is also bounded by O(sqrt(N)).