The algorithm described to in this this MIT lecture and written out in this SO question for finding a peak in a 1d array makes sense.
So does its analysis of O(log n); we re dividing the array into halves
How can I update it to find all peaks in the array? What would that complexity be?
For finding all peaks, you can't do any better than just going through the whole array and comparing every element to its neighbors. There's no way to tell whether an element you didn't look at is or isn't a peak, so you have to look at all of them.
Thus, the time complexity is O(n) for n elements.
Related
I am prepping for interview leet-code type problems and I came across the k closest problem, but given a sorted array. This problem requires finding the k closest elements by value to an input value from the array. The answer to this problem was fairly straight forward and I did not have any issues determining a linear-time algorithm to solve it.
However, working on this problem got me thinking. Is it possible to solve this problem given an unsorted array in linear time? My first thought was to use a heap and that would give an O(nlogk) time complexity solution, but I am trying to determine if its possible to come up with an O(n) solution? I was thinking about possibly using something like quickselect, but the issue is that this has an expected time of O(n), not a worst case time of O(n).
Is this even possible?
The median-of-medians algorithm makes Quickselect take O(n) time in the worst case.
It is used to select a pivot:
Divide the array into groups of 5 (O(n))
Find the median of each group (O(n))
Use Quickselect to find the median of the n/5 medians (O(n))
The resulting pivot is guaranteed to be greater and less than 30% of the elements, so it guarantees linear time Quickselect.
After selecting the pivot, of course, you have to continue on with the rest of Quickselect, which includes a recursive call like the one we made to select the pivot.
The worst case total time is T(n) = O(n) + T(0.7n) + T(n/5), which is still linear. Compared to the expected time of normal Quickselect, though, it's pretty slow, which is why we don't often use this in practice.
Your heap solution would be very welcome at an interview, I'm sure.
If you really want to get rid of the logk, which in practical applications should seldom be a problem, then yes, using Quickselect would be another option. Something like this:
Partition your array in values smaller and larger than x. <- O(n).
For the lower half, run Quickselect to find the kth largest number, then take the right-side partition which are your k largest numbers. <- O(n)
Repeat step 2 for the higher half, but for the k smallest numbers. <- O(n)
Merge your k smallest and k largest numbers and extract the k closest numbers. <- O(k)
This gives you a total time complexity of O(n), as you said.
However, a few points about your worry about expected time vs worst-case time. I understand that if an interview question explicitly insists on worst-case O(n), then this solution might not be accepted, but otherwise, this can well be considered O(n) in practice.
The key here being that for randomized quickselect and random or well-behaved input, the probability that the time complexity goes beyond O(n) decreases exponentially as the input grows. Meaning that already at largeish inputs, the probability is as small as guessing at a specific atom in the known universe. The assumption on well-behaved input concerns being somewhat random in nature and not adversarial. See this discussion on a similar (not identical) problem.
This is my question I have got somewhere.
Given a list of numbers in random order write a linear time algorithm to find the ๐th smallest number in the list. Explain why your algorithm is linear.
I have searched almost half the web and what I got to know is a linear-time algorithm is whose time complexity must be O(n). (I may be wrong somewhere)
We can solve the above question by different algorithms eg.
Sort the array and select k-1 element [O(n log n)]
Using min-heap [O(n + klog n)]
etc.
Now the problem is I couldn't find any algorithm which has O(n) time complexity and satisfies that algorithm is linear.
What can be the solution for this problem?
This is std::nth_element
From cppreference:
Notes
The algorithm used is typically introselect although other selection algorithms with suitable average-case complexity are allowed.
Given a list of numbers
although it is not compatible with std::list, only std::vector, std::deque and std::array, as it requires RandomAccessIterator.
linear search remembering k smallest values is O(n*k) but if k is considered constant then its O(n) time.
However if k is not considered as constant then Using histogram leads to O(n+m.log(m)) time and O(m) space complexity where m is number of possible distinct values/range in your input data. The algo is like this:
create histogram counters for each possible value and set it to zero O(m)
process all data and count the values O(m)
sort the histogram O(m.log(m))
pick k-th element from histogram O(1)
in case we are talking about unsigned integers from 0 to m-1 then histogram is computed like this:
int data[n]={your data},cnt[m],i;
for (i=0;i<m;i++) cnt[i]=0;
for (i=0;i<n;i++) cnt[data[i]]++;
However if your input data values does not comply above condition you need to change the range by interpolation or hashing. However if m is huge (or contains huge gaps) is this a no go as such histogram is either using buckets (which is not usable for your problem) or need list of values which lead to no longer linear complexity.
So when put all this together is your problem solvable with linear complexity when:
n >= m.log(m)
This question already has answers here:
Finding the median of an unsorted array
(9 answers)
Closed 6 years ago.
I would like to know if there exists an algorithm to find the median of an array of odd length. Obviously one could just sort the array and take the middle but ideally by only being interested in the median one could make gains in terms of time complexity of the algorithm.
If no such algorithm exists, any suggestions regarding how to go about developing such an algorithm would be great.
Thanks
This is solved by a selection algorithm, and can be done in O(n) time. Quickselect, or its refinement introselect, are popular methods.
A very brief summary of quickselect is to run quicksort, but rather than sorting both halves at each step, you only sort the half that contains the element you're looking for, which can be determined by counting how many elements are in each partition.
C++, for example, actually has this as a standard library function: nth_element.
You can use the Selection algorithm that can find the kth smallest element of an array with k is the half of the size of the array.
For unstructured data, it's within O(n).
But always keep in mind, that theoretical complexity is not everything!
Read also this question.
Yes, an algorithm exists. The problem you are talking about is finding the kth largest element where k is the value of half+1 of the array length. Here is a link to a way to do it in O(n) time, Median of medians.
I'm currently learning complexity (or efficiency however you call it), and I read about it in a book I got.
There is written something which I find pretty senseless and I need an explanation. I've tried looking online but I didn't find an answer for this certain example that they're giving.
For an algorithm that gets the max number in a single-dimensional array the size of n the input length would be n.
"For an algorithm that gets the max number in a two-dimensional array the size of n*n the input length would still be n."
I don't understand why the input length would be 'n' in both cases even though for the two-dimensional you have to go through n*n numbers...
It says
input length = the amount of work done ...
doesn't make any sense to me.
Would anyone care to explain? They certainly don't explain this there.
It's a common misconception (much seen here on SO) that the complexity of a scan across a 2D array with n*n elements is O(n^2). It's not, it's O(n). A scan is a linear operation, one element after another.
The 2D array is a polite fiction, it is really just a convenience for accessing a 1D array. After all, in languages which implement arrays properly (i.e. none of this array of pointers to blocks of memory) a 2D array is just a set of adjacent memory locations. And even in languages which do implement 2D arrays as arrays of pointers they're just linear segments of memory with interruptions
If a scan across a 2D array were O(n^2) then you could magically transform it to O(n) by ignoring the 2d-ness and just scanning the underlying 1d block of memory.
O(n^2) describes a different complexity class of operations such as those in which each pair of elements in the input is operated upon.
Reading in the comments that this book is written in Hebrew I would assume that the issue is a translation error or some other error in proofreading. The definition given in the comments of input length "input length is the measurement that indicates the work load of an algorithm" doesn't match what you would assume the term means at all in English.
To answer the question about complexity, they are reusing the variable 'n' in multiple places which makes it slightly confusing. They use 'n' to describe the dimension of the array and to describe the complexity. O(n) simply means the complexity is linear to the input. O(n^2) would be an exponential complexity. In this case with an array of n*n elements the input is n*n or n^2, but the complexity of the algorithm is still O(n) (or linear). This is because the algorithm still only operates on each input element once, whether the input is n or n*n. It would still be linear if it operated one each element 2 or three times as 3n and n are both linear functions (any x*n would be linear).
I hope this helps.
Big-O notation is used to classify TYPES of algorithms (complexity classes), not necessarily how much time it will ACTUALLY take to run. For instance O(cn) is just O(n) where c is a constant.
n is the size of the input whether that input is an nxn matrix or just an 'n' length array. The big-O 'n' and the program variable name are not referring to the same thing.
Given an unsorted integer array, and without making any assumptions on
the numbers in the array:
Is it possible to find two numbers whose
difference is minimum in O(n) time?
Edit: Difference between two numbers a, b is defined as abs(a-b)
Find smallest and largest element in the list. The difference smallest-largest will be minimum.
If you're looking for nonnegative difference, then this is of course at least as hard as checking if the array has two same elements. This is called element uniqueness problem and without any additional assumptions (like limiting size of integers, allowing other operations than comparison) requires >= n log n time. It is the 1-dimensional case of finding the closest pair of points.
I don't think you can to it in O(n). The best I can come up with off the top of my head is to sort them (which is O(n * log n)) and find the minimum difference of adjacent pairs in the sorted list (which adds another O(n)).
I think it is possible. The secret is that you don't actually have to sort the list, you just need to create a tally of which numbers exist. This may count as "making an assumption" from an algorithmic perspective, but not from a practical perspective. We know the ints are bounded by a min and a max.
So, create an array of 2 bit elements, 1 pair for each int from INT_MIN to INT_MAX inclusive, set all of them to 00.
Iterate through the entire list of numbers. For each number in the list, if the corresponding 2 bits are 00 set them to 01. If they're 01 set them to 10. Otherwise ignore. This is obviously O(n).
Next, if any of the 2 bits is set to 10, that is your answer. The minimum distance is 0 because the list contains a repeated number. If not, scan through the list and find the minimum distance. Many people have already pointed out there are simple O(n) algorithms for this.
So O(n) + O(n) = O(n).
Edit: responding to comments.
Interesting points. I think you could achieve the same results without making any assumptions by finding the min/max of the list first and using a sparse array ranging from min to max to hold the data. Takes care of the INT_MIN/MAX assumption, the space complexity and the O(m) time complexity of scanning the array.
The best I can think of is to counting sort the array (possibly combining equal values) and then do the sorted comparisons -- bin sort is O(n + M) (M being the number of distinct values). This has a heavy memory requirement, however. Some form of bucket or radix sort would be intermediate in time and more efficient in space.
Sort the list with radixsort (which is O(n) for integers), then iterate and keep track of the smallest distance so far.
(I assume your integer is a fixed-bit type. If they can hold arbitrarily large mathematical integers, radixsort will be O(n log n) as well.)
It seems to be possible to sort unbounded set of integers in O(n*sqrt(log(log(n))) time. After sorting it is of course trivial to find the minimal difference in linear time.
But I can't think of any algorithm to make it faster than this.
No, not without making assumptions about the numbers/ordering.
It would be possible given a sorted list though.
I think the answer is no and the proof is similar to the proof that you can not sort faster than n lg n: you have to compare all of the elements, i.e create a comparison tree, which implies omega(n lg n) algorithm.
EDIT. OK, if you really want to argue, then the question does not say whether it should be a Turing machine or not. With quantum computers, you can do it in linear time :)