Skiena, in The Algorithm Design Manual, states that insertion into a sorted array is O(n). Yet searching for an item in a sorted array is O(log n), because you can do a binary search.
Couldn't insertion also be O(log n), if I did binary search comparisons to figure out where in the array it should go?
Finding the position is only half the battle. Show me how to place a 2 in its place into [1,3,4,5,6,7] using fewer than five move operations.
You can do O(log n) search on a sorted array but when you insert an item you need to shift data, so shift is O(n).
You can use binary search to figure out where the element should go.
However, inserting the element means that you have to make space for it. This is done by moving all elements after the new element to the right. That takes O(n) complexity.
Related
What is the best running time using theta notation for:
Find an element in a sorted array
Find an element in a sorted linked list
Inserting an element in a sorted array, once the position is found
Inserting an element in a sorted linked list, once the position is found
So far I have 2) is theta(n) and 4) is theta(1) only because I remembered my prof just said the answer in class, but is there an explanation on how to get these?
First of all reading one of your answers it seems like you might be asking for complexity in O[big o].Theta notation is used when the complexity is bound asymptotically both above and below. Big O notation is for when the complexity is bound asymptotically, only above.
1. Find an element in a sorted array:
Using binary search it can be O(logn). But in Best case Ω(1)
2. Find an element in a sorted linked list
You can't use binary search here. You have to traverse the entire list to find a particular number. No way of going to a particular position without traversing numbers before(or after) it. So in worst case, you traverse n(length) times. So O(n)
Ω(1) because in best case you can find it in the beginning.
3. Inserting an element in a sorted array, once the position is found
O(n) since you have to shift all the numbers to the right of the position of new insertion place.
Ω(1) because in the best case you might just add it at the end.
4. Inserting an element in a sorted linked list, once the position is found
Ɵ(1) O(1) Ω(1), because adding a new element in a particular position (after you know the position and you have a pointer to that position) is theta(1)
What would be the complexity of a search operation in an unsorted array that allows duplicates. My guesses were O(N) since it allows duplicates, the whole array needs to be searched. But I'm new to algorithm complexity and I cannot be sure about my answer, can you please confirm if I'm correct.
Since the array is unsorted, you have to look at half of the array on average to find the element you are searching for. Therefore, complexity is linear - O(N). Duplicates or no duplicates, the same complexity.
Searching for elements within an unordered array would indeed be O(N) as no heuristic can speed up the search.
It is O(n) because in the worst case you still need to look at every element.
I thought of a sorting algorithm but I am not sure if this already exists.
Say we have a container with n items:
We choose the 3rd element and do a binary search on the first 2, putting it in the correct position. The first 3 items in the container are sorted.
We choose the 4th element and do a binary search on the first 3 and put it in the correct position. Now the first 4 items are sorted.
We choose the 5th element and do a binary search on the first 4 items and put it in the correct position. Now 5 items are sorted.
.
.
.
We choose the nth element and do a binary search on the other n-1 elements putting it in the correct position. All the items are sorted.
Binary search takes logk for k elements and let's say that the insertion takes constant time. Shouldn't this take:
log2 to put the 3rd element in the correct spot.
log3 to put the 4th element in the correct spot.
log4 to put the 5th element in the correct spot.
.
.
.
log(n-1) to put the nth element in the correct spot.
log2 + log3 + log4 + ... log(n-1) = log((n-1)!) ?
I may be talking nonsense but this looked interesting.
EDIT:
I did not take the insertion time into consideration. What if the sorting was done in a sorting array with gaps between the elements? This would allow for fast inserting without having to shift many elements. After a number of inserts, we could redistribute the elements. Considering that the array is not sorted (we could use a shuffle to ensure this) I think that the results could be quite fast.
It sounds like insertion sort modified to use binary search. It's fairly well-known, but not particularly well-used (as far as I know), possibly because it doesn't affect the O(n²) worst case, but makes the O(n) best case take O(n log n) instead, and because insertion sort isn't commonly used on anything but really small arrays or those already sorted or nearly sorted.
The problem is that you can't really insert in O(1). Random-access insert into an array takes O(n), which is of course what the well-known O(n²) complexity of insertion sort assumes.
One could consider a data structure like a binary search tree, which has O(log n) insert - it's not O(1), but we still end up with an O(n log n) algorithm.
Oh O(log (n!)) = O(n log n), in case you were wondering about that.
Tree sort (generic binary search tree) and splaysort (splay tree) both use binary search trees to sort. Adding elements to a balanced binary search tree is equivalent to doing a binary search to find where to add the elements then some tree operations to keep the tree balanced. Without a tree of some type, this becomes insertion sort as others have mentioned.
In the worst case the tree can become highly unbalanced, resulting in O(N^2) for tree sort. Using a self-balancing binary search tree yields O(N log N), at least on average. Splay sort is an adaptive sort, making it rather efficient when the input is already nearly sorted.
I think by binary search, he meant that there is an insertion taking place placed on a searchable index of where we would expect to find the item we are inserting. in which case it would be called insertion sort... Either way it's still N*log(N)
I was reading binary search tree and was thinking that why do we need BST at all? All the things as far as I know can also be achieve using simple sorted arrays. For e.g. - In order to build a BST having n elements, we requires n*O(log n) time i.e. O(nlog n) and lookup time is O(log n). But this thing can also be achieve using array. We can have a sorted array(requires O(nlog n) time), and lookup time in that is also O(log n) i.e. binary search algo. Then why do we need another data structure at all? Are there any other use/application of BST which make them so special?
--Ravi
Arrays are great if you're talking about write once, read many times type of interactions. It's when you get down to inserting, swapping, and deletion in which BST really start to shine compared to an array. Since they're node based, rather than based on a contiguous chunk of memory, the cost of moving an element either into the collection or out of the collection is fast while still maintaining the sorted nature of the collection.
Think of it as you would the difference in insertion between linked lists versus arrays. This is an oversimplification but it highlights an aspect of the advantage I've noted above.
Imagine you have an array with a million elements.
You want to insert an element at location 5.
So you insert at the end of the array and then sort.
Let's say the array is full; that's O(nlog n), which is 1,000,000 * 6 = 6,000,000 operations.
Imagine you have a balanced tree.
That's O(log n), plus a bit for balancing = 6 + a bit, call it 10 operations.
So, you've just spent 6,000,000 ops sorting your array. You then want to find that element. What do you do? binary search - O(log n) - which is exactly the same as what you're going to do when you search in the tree!
Now imagine you want to allocate -another- element.
Your array is full! what do you do? re-allocate the array with n extra elements and memcpy the lot? you really want to memcpy 4mbytes?
In a tree, you just add another element...
How about sorted insertion time?
In graphics programming if you have extended object(i.e. which represent an interval in each dimension and not just a point) you can add them to the smallest level of a binary tree(typically an octree) where they fit in entirely.
And if you don't pre-calculate the tree/sortedlist the O(n) random insertion time in a list can be prohibitively slow. Insertion time in a tree on the other hand is only O(log(n)).
I just wrote this exam, in which there was question: Consider a array of size 2n, where the numbers in odd positions are sorted in ascending order and the the numbers in even positions in descending order. Now, if I have to search for a number in this array, which is a good way to do this? The options were:
Quick sort and then binary search
Merge the two sorted arrays and then binary search
Sequential search
In 1, Quick Sort takes O(n log n) and binary search, O(log n)
In 2, Merge takes O(n) and then O(log n) for binary search
In 3, it takes O(n).
So 3 turns out to be the way to go. Is that correct? Is there a better choice which might not have been given?
EDIT: I accepted Lukas's answer since he was the first. Sigh, that was the other option. I get a -1. :(
You can do two binary searches -- that's O(log n), because you ignore constants (2 in this case).
Would it not be easier if you just did a binary search on both sets? Thus O(log n) and O(log n)?
Would that not be easier, so if found in the first, the second step is not even required?
Check whether it's even or odd, and binary search on the part of the array you're interested in. O(log n).
I think second option fits your criteria.
Merge sort: We can easily get full array of sorted numbers from two half sorted arrays with time complexity of O(n)
And then binary search will have time complexity of O(logn)