I was thinking about the fastest possible algorithm to return all combinations of unique 3 elements from an array of n elements. The obvious one is the O(n^3) solution, which takes under consideration all possible combinations, but this is bruteforce, and I intend to find something much quicker. Looking for an answer in C++
In the worst case (all the items of the array are different) you have
n ! / ((n - 3)! * 3!) == n * (n - 1) * (n - 2) / 6
distinct items to output and thus O(n**3) is all you can achieve.
If the array has many items, but few distinct ones, you can preprocess it:
remove all the item ocurrencies, but three:
[0, 1, 1, 1, 1, 0, 2, 1, 2, 2, 2, 1] -> [0, 0, 1, 1, 1, 2, 2, 2]
If you have a good hash function for the array's items the preprocess stage takes O(N). In the best case (all the items are same) the prepocess takes O(N)
and the only answer output is O(1) so you have O(N) for the entire
routine.
For the arbitrary array, you can't have complexity better than O(N) (since you have to scan the entire array). Finally, the complexity in case of preprocessing and good hash function for the array's items is in the range of
[O(N)..O(N**3)]
If you're lucky the process will be much quicker; if you have large data to output, well, you have but output the large collection...
There is no way to do that. Because no matter what you do you need to get those 3 different elements. which has size nC3 = n*(n-1)*(n-2)/6. You need to iterate this any times for sure. So you will have O(n^3) complexity minimum.
Related
Let's say I have an unsorted array Unsorted_Arr= [2, 8, 1, 3, 6, 7, 5, 4].
Right before the very last pass of merging, I would have two arrays, Arr_Left = [1, 2, 3, 8] and Arr_Right = [4, 5, 6, 7]. To merge them, I would need to iterate through all the n elements of Arr_Right fully, and iterate through n-1 elements of Arr_Left. In total, I would have traversed through n-1 elements of the original Unsorted_Arr. Drop the -1, and I have a time-complexity of O(n) for the merge.
While I understand why the recursive portion of merge sort is log n, since our code contains a portion of code that runs at O(n), shouldn't the worst case scenario of a merge sort be O(n)?
But those two arrays are themselves not necessarily already sorted, so you have to split them and then re-merge them as well. And you'd have to repeat that down until there was only one or zero elements in each array (because a 1 or 0 element array is always sorted) and then re-merge them all.
And at each level, that's at least 2*n operations (for all of the split arrays at that level), which is O(n) for each level.
So how many levels deep would you have to go with the splitting before every array was only 1 or 0 in length? That's Log(n).
Combining the number of levels that you have to split and then re-merge all of the sublists (O(Log(n))) with how many operations have to be performed at each level (O(n)) becomes: O(Log(n)) * O(n) which reduces to O(n*Log(n)).
I have been stuck in one question.
Given 1 billion numbers we need to find the largest 1 million numbers. One approach is to sort the
numbers and then take the first million numbers from that is in O(n log n). Propose an algorithm that has
expected O(n) time complexity.
Is it Heap sort which can do that with having O(n) complexity?
The general version of the problem you're trying to solve here seems to be the following:
Given n numbers, report the largest k of them in (possibly expected) time O(n).
If you just need to find the top k elements and the ordering doesn't matter, there's a clever O(n)-time algorithm for this problem based on using fast selection algorithms. As a refresher, a selection algorithm takes as input an array A and a number m, then reorders the array A so that the m smallest elements are in the first m slots and the remaining elements occupy the larger slots. The quickselect algorithm does this in (expected) time O(m) and is fast in practice; the median-of-medians algorithm does this in worst-case O(m) time but is slower in practice. While these algorithms typically are framed in terms of finding the smallest k elements, they work just as well finding the largest k elements.
Using this algorithm as a subroutine, here's how we can find the top k elements in time and space O(m):
Initialize a buffer of 2k elements.
Copy the first k elements of the array into the buffer.
While there are elements remaining in the array:
Copy the next k of them into the buffer.
Use a selection algorithm to place the k largest elements
of the buffer in the first k slots of the buffer.
Discard the remaining elements of the buffer.
Return the contents of the buffer.
To see why this works, notice that after each iteration of the loop, we maintain the invariant that the buffer holds the k largest elements of the ones that have been seen so far (though not necessarily in sorted order). Therefore, the algorithm will identify the top k elements of the input and return them in some order.
In terms of time complexity - there's O(k) work to create the buffer, and across all iterations of the loop we do O(n) work copying elements into the buffer. Each call to the selection algorithm takes (expected) time O(k), and there are O(n / k) calls to the algorithm for a net runtime of O(n + k). Under the assumption that k < n, this gives an overall runtime of O(n), with only O(k) total space required.
No general sorting algorithm can do this in O(n) time. Furthermore, without additional constraints (e.g., the billion numbers are taken from the numbers 1 to 1,000,000) there is no sorting algorithm at all that is going to work for this.
However, there is a simple O(n) algorithm to do this:
initialize a return buffer with 1,000,000 empty cells
for each item in your list of 1,000,000,000, do the following:
check each of the 1,000,000 cells in order
if the number from the input is bigger than the number in your buffer, swap them and keep going
if you're looking at a blank cell, put the number you're holding down
if you get to the end of the list and are holding a number, throw it out
Here's an example with a list of 10 things and we want the biggest 5:
Input: [6, 2, 4, 4, 8, 2, 4, 1, 9, 2]
Buffer: [-, -, -, -, -]
[6, -, -, -, -] … see a blank, drop the 6
[6, 2, -, -, -] … 2 < 6, skip, then see a blank, drop the 2
[6, 4, 2, -, -] … 4 < 6 but 4 > 2, swap out 2, see blank, drop 2
[6, 4, 4, 2, -] … 4 <= 4,6 but 4 > 2, swap out 2, see blank, drop 2
[8, 6, 4, 4, 2] … 8 > 6, swap, then swap 6 for 4, etc.
[8, 6, 4, 4, 2] … 2 <= everything, drop it on the floor
[8, 6, 4, 4, 4] … 4 <= everything but 2, swap, then drop 2 on floor
[8, 6, 4, 4, 4] … 1 <= everything, drop it on the floor
[9, 8, 6, 4, 4] … 9 > everything, swap with 8 then drop a 4 on the floor
[9, 8, 6, 4, 4] … 2 <= everything, drop it on the floor
You do 1,000,000 comparisons and potentially up to 1,000,000 swaps for every element in the input (consider input in sorted ascending order). That means you do work proportional to 1,000,000 * n, a linear amount of work in the size of the input n.
You can generally do better than sorting. Most people would solve this problem by using a heap as the structure. The time complexity to build the heap will be O(n). But you will then have to do a million "pop" operations, the time complexity for each pop being O(log n), but you are not doing the full n pops (only n/1000 pops in this case).
I say you can "generally" do better than sorting because most sorting algorithms in libraries are O(n log n). But there is the "distribution sort" that is actually O(n + k), where k is the number of possible values in the range you are sorting and depending on the value of k, you might do better sorting.
Update
To incorporate the suggestion made by #pjs, create a "minimum" heap with the first million values from the billion where a pop operation removes the minimum value from the heap. Then for the next 999,000,000 values, check to see if each one is greater than the current minimum value on the heap and if so, pop the current minimum value from the heap and push the new value. When you are done, you will be left with the 1,000,000 largest values.
I need a little help trying to figure out something:
Given a sequence of unordered numbers (less than 15.000) - A - I must answer Q queries (Q <= 100000) of the form i, j, x, y that translate as the following:
How many numbers in range [i,j] from A are bigger (or equal) than x but smaller than y with all numbers in the sequence smaller than 5000.
I am under the impression this requires something like O(logN) because of the big length of the sequence and this got me thinking about BIT (binary indexed trees - because of the queries) but a 2D BIT is too big and requires way to much time to run even on the update side. So the only solution I see here should be 1D BIT or Segment Trees but I can't figure how to work out a solution based on these data structures. I tried retaining the positions in the ordered set of numbers but I can't figure out how to make a BIT that responds to queries of the given form.
Also the algorithm should fit in like 500ms for the given limits.
Edit 1: 500ms for all of the operations on preprocessing and answering the queries
EDIT 2: Where i, j are positions of first and last element in the sequence A to look for elements bigger than x and smaller than y
EDIT 3: Example:
Let there be 1, 3, 2, 4, 6, 3 and query 1, 4, 3, 5 so between positions 1 and 4 (inclusive) there are 2 elements (3 and 4) bigger (or equal) than 3 and smaller than 5
Thank you in advance! P.S: Sorry for the poor English!
Implement 2D-range counting by making a BIT-organized array of sorted subarrays. For example, on the input
[1, 3, 2, 4, 6, 3]
the oracle would be
[[1]
,[1, 3]
,[2]
,[1, 2, 3, 4]
,[6]
,[3, 6]
].
The space usage is O(N log N) (hopefully fine). Construction takes O(N log N) time if you're careful, or O(N log^2 N) time if not (no reason to be for your application methinks).
To answer a query with maximums on sequence index and value (four of these can be used to answer the input queries), do the BIT read procedure for the maximum index, using binary search in the array to count the number of elements not exceeding the maximum value. The query time is O(log^2 N).
I have a university assignment that requires me to code the Counting Sort algorithm with n threads in Java. We haven't really been given more information than that. I thought that the best way would be to partition the array into n sections, then each thread sorts a section. The problem is that I am unsure of how to partition the array properly; I have only seen examples on how to partition into 2 sections, not n sections.
I would appreciate it if someone could provide me with the logic on how to partition it like I've explained, or give some pseudo-code. No source code please, this is an assignment I have to do.
I have no problem with the actual sorting, just the partitioning.
Thanks.
Definitions
Let's say that you have an array a[0..n-1] to sort and you want to do it using k threads.
For simplicity, let's assume that the smallest element has value 0 and the largest have a value m. If the smallest is not equal 0, then you can scale the values during assigning elements to threads.
Splitting into threads
Partition your array into k chunks each consisting of at most floor(m/k) + 1 different values of elements.
The i-th chunk consists of elements a[j] such that:
(i - 1) * (floor(m/k) + 1) <= a[j] < i * (floor(m/k) + 1)
For example, if you have an array with 10 elements:
a[0..9] = {1, 2, 5, 0, 3, 7, 2, 3 ,4, 6} and k = 3, then m = 7 and the 3 chunks are:
chunk_1: elements in range [0,3) -> [1, 2, 0, 2]
chunk_2: elements in range [3,6) -> [5, 3, 3, 4]
chunk_3: elements in range [6,9) -> [6, 7]
Next, assign each chunk to a separated thread. Each thread sorts one chunk and to get the whole array sorted, just concatenate the results from all threads in order:
thread_1 thread_2 ... thread_k
Complexity:
As you know, the complexity of the count sort is O(n + L) where n is the number of elements to sort, and L is the maximum value of element.
First, notice that you can scale down values in each thread in such a way, that L < floor(m/k) + 1 in that thread, so the complexity of count sort in each thread always depends on the number of elements in that thread.
If you assume that the distribution of the values is uniform, then the expected number of elements in each thread is also floor(m/k) so the total complexity of each thread is O(m/k).
The first idea popping into my mind is to partition the array recursively. That means if you can partition into 2, you can also partition into 4 , right?
A more advanced and modern approach is to partition into many more parts than you have threads or processes. Then assign these parts dynamically to the threads.
I'm working on a sorting/ranking algorithm that works with quite large number of items and I need to implement the following algorithm in an efficient way to make it work:
There are two lists of numbers. They are equally long, about 100-500 thousand items. From this I need to find the n-th biggest product between these lists, ie. if you create a matrix where on top you have one list, on the side you have the other one and each cell is the product of the number above and the number on the side.
Example: The lists are A=[1, 3, 4] and B=[2, 2, 5]. Then the products are [2, 2, 5, 6, 6, 15, 8, 8, 20]. If I wanted the 3rd biggest from that it would be 8.
The naive solution would be to simply generate those numbers, sort them and then select the n-th biggest. But that is O(m^2 * log m^2) where m is the number of elements in the small lists, and that is just not fast enough.
I think what I need is to first sort the two small lists. That is O(m * log m). Then I know for sure that the biggest one A[0]*B[0]. Second biggest one is either A[0]*B[1] or A[1]*B[0], ...
I feel like this could be done in O(f(n)) steps, independent of the size of the matrix. But I can't figure out an efficient way to do this part.
Edit: There was an answer that got deleted, which suggested to remember position in the two sorted sets and then look at A[a]*B[b+1] and A[a+1]*B[b], returning the bigger one and incrementing a/b. I was going to post this comment before it got deleted:
This won't work. Imagine two lists A=B=[3,2,1]. This will give you
matrix like [9,6,3 ; 6,4,2 ; 3,2,1]. So you start at (0,0)=9, go to
(0,1)=6 and then the choice is (0,2)=3 or (1,1)=4. However, this will
miss the (1,0)=6 which is bigger then both. So you can't just look to
the two neighbors but you have to backtrack.
I think it can be done in O(n log n + n log m). Here's a sketch of my algorithm, which I think will work. It's a little rough.
Sort A descending. (takes O(m log m))
Sort B descending. (takes O(m log m))
Let s be min(m, n). (takes O(1))
Create s lazy sequence iterators L[0] through L[s-1]. L[i] will iterate through the s values A[i]*B[0], A[i]*B[1], ..., A[i]*B[s-1]. (takes O(s))
Put the iterators in a priority queue q. The iterators will be prioritized according to their current value. (takes O(s) because initially they are already in order)
Pull n values from q. The last value pulled will be the desired result. When an iterator is pulled, it is re-inserted in q using its next value as the new priority. If the iterator has been exhausted, do not re-insert it. (takes O(n log s))
In all, this algorithm will take O(m log m + (s + n)log s), but s is equal to either m or n.
I don't think there is an algorithm of O(f(n)), which is independent of m.
But there is a relatively fast O(n*logm) algo:
At first, we sort the two arrays, we get A[0] > A[1] > ... > A[m-1] and B[0] > B[1] > ... > B[m-1]. (This is O(mlogm), of course.)
Then we build a max-heap, whose elements are A[0]*B[0], A[0]*B[1], ... A[0]*B[m-1]. And we maintain a "pointer array" P[0], P[1], ... P[m-1]. P[i]=x means that B[i]*A[x] is in the heap currently. All the P[i] are zero initially.
In each iteration, we pop the max element from the heap, which is the next largest product. Assuming it comes from B[i]*A[P[i]] (we can record the elements in the heap come from which B[i]), we then move the corresponding pointer forward: P[i] += 1, and push the new B[i] * A[P[i]] into the heap. (If P[i] is moved to out-of-range (>=m), we simply push a -inf into the heap.)
After the n-th iteration, we get the n-th largest product.
There are n iterations, and each one is O(logm).
Edit: add some details
You don't need to sort the the 500 000 elements to get the top 3.
Just take the first 3, put them in a SortedList, and iterate over the list, replacing the smallest of the 3 elements with the new value, if that is higher, and resort the resulting list.
Do this for both lists, and you'll end with a 3*3 matrix, where it should be easy to take the 3rd value.
Here is an implementation in scala.
If we assume n is smaller than m, and A=[1, 3, 4] and B=[2, 2, 5], n=2:
You would take (3, 4) => sort them (4,3)
Then take (2,5) => sort them (5, 2)
You could now do an zipped search. Of course the biggest product now is (5, 4). But the next one is either (4*2) or (5*3). For longer lists, you could keep in mind what the result of 4*2 was, compare it only with the next product, taken the other way. That way you would only calculate one product too much.