I am taking the Algorithms course in coursera. One of the assignments is the following:
Randomized queue. A randomized queue is similar to a stack or queue,
except that the item removed is chosen uniformly at random among items
in the data structure.
I am trying to find a way to implement dequeue (randomly removing an item) in a constant amount of time. I have thought of an idea to do this recquiring a deque (which supports removing and adding an item from the front and back in constant time). My idea is as follows:
Use a deque as an underlying data structure inside the randomized queue
Enqueue - use a library function to generate an integer between 0 and 1 inclusive. If the integer is 0, add the item to the front of the deque. Otherwise, add it to the back.
Dequeue - any direction would be fine.
The reason why the randomness happens in enqueue rather than in dequeue is because I find it to be not exactly random (E.G. n calls to enqueue will have dequeue only return either the first or nth item). So to be sure the items are being removed randomly, I decided to enqueue them randomly.
This idea seems good to me because I cannot find holes in it, but the problem is I cannot prove that my idea would really work. I don't know much about randomness. In fact, this is only the 5th time I am working with random data structures.
Is my idea correct? Will it generate a data structure that removes items at random?
Enqueueing only at the ends does not produce a uniformly random sequence. The last item to be enqueued is necessarily at either ends, and the first item to be enqueued is more likely to be somewhere in the middle than at either ends after enqueueing other items.
To illustrate, take the set of three items {1, 2, 3}, the smallest set that does not result in a uniform distribution. Enqueueing them in that order gives the following possible results (in parenthesis is where to enqueue the next item).
[1] -> (front) -> [1, 2] -> (front) -> [1, 2, 3]
[1] -> (front) -> [1, 2] -> (back) -> [3, 1, 2]
[1] -> (back) -> [2, 1] -> (front) -> [2, 1, 3]
[1] -> (back) -> [2, 1] -> (back) -> [3, 2, 1]
These four results are the only possibilities and are all equally likely. And as you can see, the last item is never in the middle while both the first and second items are in the middle twice.
What you want is to dequeue at a random place. But you don't need to preserve the order of other items, since they are uniformly distributed. That means you can just swap the last item with a random one, and then dequeue that one (which became the last item).
I don't think your proposed approach will work because of the uniformity requirement. Uniformity means that every item in the queue has an equal likelihood of being dequeued. Your proposal always adds elements to one of the ends, and dequeues from one end or the other. Consequently, on any given dequeue request the non-end elements have zero probability of being selected.
An alternative might be to use an array-based stack. Add elements at the end of the stack, but for dequeueing choose an element at random, swap it with the last element, and then pop it. That will have uniformity of selection, and all of the component operations are constant time.
Related
I have a ragged array represented as a contiguous block of memory along with its "shape", corresponding to the length of each row, and its "offsets", corresponding to the index of the first element in each row. To illustrate, I have an array conceptually like this:
[[0, 0, 0],
[1, 1, 1, 1],
[2],
[3, 3],
[4, 4]]
Represented in-memory as:
values: [0, 0, 0, 1, 1, 1, 1, 2, 3, 3, 4, 4]
shape: [3, 4, 1, 2, 2]
offsets: [0, 3, 7, 8, 10]
I may have in the hundreds of millions of rows with typically, say, 3-20 four-byte floats per row, though with no hard upper bound on the row length.
I wish to shuffle the rows of this array randomly. Since the array is ragged, I can't see how the Fisher-Yates algorithm can be applied in a straightforward manner. I see how I can carry out a shuffle by randomly permuting the array shape, pre-allocating a new array, and then copying over rows according to the permutation generating the new shape with some book-keeping on the indexes. However, I do not necessarily have the RAM required to duplicate the array for the purposes of this operation.
Therefore, my question is whether there is a good way to perform this shuffle in-place, or using only limited extra memory? Run-time is also a concern, but shuffling is unlikely to be the main bottleneck.
For illustration purposes, I wrote a quick toy-version in Rust here, which attempts to implement the shuffle sketched above with allocation of a new array.
shape is redundant since shape[i] is offset[i+1]-offset[i] (if you extend offset by one element containing the length of the values array). But since your data structure has both these fields, you could shuffle your array by just in-place shuffling the two descriptor vectors (in parallel), using F-Y. This would be slightly easier if shape and offset were combined into an array of pairs (offset, length), which also might improve locality of reference, but it's certainly not critical if you have some need for the separate arrays.
That doesn't preserve the contiguity of the rows in the values list, but if all your array accesses are through offset, it will not require any other code modification.
It is possible to implement an in-place swap of two variable-length subsequences using a variant of the classic rotate-with-three-reversals algorithm. Given P V Q, a sequence conceptually divided into three variable length parts, we first reverse P, V, and Q in-place independently producing PR VR QR. Then we reverse the entire sequence in place, yielding Q V P. (Afterwards, you'd need to fixup the offsets array.)
That's linear time in the length of the span from P to Q, but as a shuffle algorithm it will add up to quadratic time, which is impractical for "hundreds of millions" of rows.
As often happens, I started with a complex idea and then simplified it. Here is the simple version, with the complex one below.
What we're going to do is quicksort it into a random arrangement. The key operation is partitioning. That is we want to take a section of m blocks and randomly partition them into m_l blocks on the left and m_r on the right.
The idea here is this. We have a queue of temporarily copied blocks on the left, and a queue of temporarily copied blocks on the right. It is a queue of blocks, but the queue size is the number of elements in it. The partitioning logic looks like this:
while m_l + m_r < m:
pick the larger queue, breaking ties randomly
if the queue is empty:
read a block into it
get block from queue
if random() < m_l / (m_l + m_r):
# put the block on the left
until we have enough room to write the block:
copy blocks into the left queue
write block to the left
m_l--
else:
# put the block on the right
until we have enough room to write the block:
copy blocks into the right queue
write block to the right
m_r--
And now we need to recursively repeat until we've quicksorted it into a random order.
Note that, unlike with a regular quicksort, we are constructing our partitions to be exactly the size we expect. So we don't have to recurse. Instead we can:
# pass 1
partition whole array
# pass 2
partition 0..(m//2-1)
partition (m//2)..(m-1)
# pass 3
partition 0..(m//4-1)
partition (m//4)..(m//2-1)
partition (m//2)..((3*m)//4-1)
partition ((3*m)//4)..(m-1)
etc.
The result is time O(n * log(m)). And the queues will never get past 5k data where k is the largest block size.
Here is an approach that we can calculate in time O(n log(n)). The maximum space needed is O(k) where k is the maximum block size.
First note, shape and offsets are largely redundant. Because shape[i] = offset[i+1] - offset[i] for all i but the last. So with O(1) extra data (which we already have in values.len()) we can make shape redundant, then (ab)use it, however we want, and then calculate it at the end.
So let's start by picking a random permutation of 0..(shape.len()-1) and placing it in shape. This will be where each element will go, and can be found in time O(n) using Fisher-Yates.
Our idea now is to use quicksort to actually get them to the right places.
First, our pivot. For O(n) work in a single pass we can add up the lengths of all blocks which will come before the median block, and also find the length of said median block.
Now quicksort is dependent upon being able to swap things. But we can't do that directly (your whole problem). However the idea is that we'll partition from the middle out. And so the values, shape and offsets arrays will have beginning and ending sections that we haven't gotten to, and a piece in the middle that we've rewritten. Where those sections meet we'll also need to have queues of blocks copied off of the left and right and not yet written back. And, of course, we'll need to have a record of where the boundaries are.
So the idea is this.
set up the data structures.
copy a few blocks in the middle to one of the queues - enough to have a place for the median block to go.
record where the median will go
while have not finished partitioning:
pick the larger queue (break ties by farthest from its end, then randomly)
if it is empty:
read a block into it
figure out where its next block needs to be written
copy blocks in its way to the appropriate queue
write the block out
Where writing the block out means writing its elements to the right place, setting its offset, and setting its shape to the still final location for that block.
This operation will partition around the median block. Recursively repeat to sort each side into blocks being in their final position.
And, finally, fix the shape array back to what it was supposed to be.
The time complexity of this is O(n log(n)) for the same reason that quicksort is. As for space complexity, if k is the largest block size, any time the queues get past size 4k then the next block you extract must be able to be written, so they cannot grow any farther. This makes the maximum space used O(k).
I have several sets of pairs like:
a: ([1, 2], [4,5])
b: ([1, 3])
c: ([4, 7], [1, 8])
d: ([9, 7], [1, 5])
...
Where no two pairs are identical, and the elements of no pair are identical. Each set can contain many pairs. There is a smallish number of elements (around 200).
From each set I take one pair. Now, I want to take pairs in such a way, that the number of elements is the smallest possible.
The problem is too large to try every combination, is there any algorithm or heuristic that might help me find the optimal (or a close guess)?
The problem has a definite NP-complete feel about it. So here are two greedy approaches that may produce reasonable approximate answers. To figure which is better you should implement both and compare.
The first is bottom up. Give each set a value of 2 if it has a pair selected from it, and (n+1)/n if it has n pairs partially selected from it. At each round, give each element a value for being selected which is the sum of the amount by which adding it increases the value of all of the sets. In the round select the element with the highest value, then update the value of all of the sets, update the value of all remaining elements, and continue.
This will pick elements that look like they are making progress towards covering all sets.
The second is top down. Start with all elements selected, and give each set a value of 1/n where n is the number of selected pairs. Elements that are required for all pairs in a given set are put into the final set. Of the remaining elements, find the one who increases the value the least if it is removed, and remove it.
The idea is that we start with too big a cover and repetitively remove the one which seems least important for covering all the sets. What we are left with is hopefully minimal.
Let's say I have an array of ~20-100 integers, for example [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] (actually numbers more like [106511349 , 173316561, ...], all nonnegative 64-bit integers under 2^63, but for demonstration purposes let's use these).
And many (~50,000) smaller arrays of usually 1-20 terms to match or not match:
1=[2, 3, 8, 20]
2=[2, 3, NOT 8]
3=[2, 8, NOT 16]
4=[2, 8, NOT 16] (there will be duplicates with different list IDs)
I need to find which of these are subsets of the array being tested. A matching list must have all of the positive matches, and none of the negative ones. So for this small example, I would need to get back something like [3, 4]. List 1 fails to match because it requires 20, and list 2 fails to match because it has NOT 8. The NOT can easily be represented by using the high bit/making the number negative in those cases.
I need to do this quickly up to 10,000 times per second . The small arrays are "fixed" (they change infrequently, like once every few seconds), while the large array is done per data item to be scanned (so 10,000 different large arrays per second).
This has become a bit of a bottleneck, so I'm looking into ways to optimize it.
I'm not sure the best data structures or ways to represent this. One solution would be to turn it around and see what small lists we even need to consider:
2=[1, 2, 3, 4]
3=[1, 2]
8=[1, 2, 3, 4]
16=[3, 4]
20=[1]
Then we'd build up a list of lists to check, and do the full subset matching on these. However, certain terms (often the more frequent ones) are going to end up in many of the lists, so there's not much of an actual win here.
I was wondering if anyone is aware of a better algorithm for solving this sort of problem?
you could try to make a tree with the smaller arrays since they change less frequently, such that each subtree tries to halve the number of small arrays left.
For example, do frequency analysis on numbers in the smaller arrays. Find which number is found in closest to half of the smaller arrays. Make that the first check in the tree. In your example that would be '3' since it occurs in half the small arrays. Now that's the head node in the tree. Now put all the small lists that contain 3 to the left subtree and all the other lists to the right subtree. Now repeat this process recursively on each subtree. Then when a large array comes in, reverse index it, and then traverse the subtree to get the lists.
You did not state which of your arrays are sorted - if any.
Since your data is not that big, I would use a hash-map to store the entries of the source set (the one with ~20-100 integers). That would basically let you test if a integer is present in O(1).
Then, given that 50,000(arrays) * 20(terms each) * 8(bytes per term) = 8 megabytes + (hash map overhead), does not seem large either for most systems, I would use another hash-map to store tested arrays. This way you don't have to re-test duplicates.
I realize this may be less satisfying from a CS point of view, but if you're doing a huge number of tiny tasks that don't affect each other, you might want to consider parallelizing them (multithreading). 10,000 tasks per second, comparing a different array in each task, should fit the bill; you don't give any details about what else you're doing (e.g., where all these arrays are coming from), but it's conceivable that multithreading could improve your throughput by a large factor.
First, do what you were suggesting; make a hashmap from input integer to the IDs of the filter arrays it exists in. That lets you say "input #27 is in these 400 filters", and toss those 400 into a sorted set. You've then gotta do an intersection of the sorted sets for each one.
Optional: make a second hashmap from each input integer to it's frequency in the set of filters. When an input comes in, sort it using the second hashmap. Then take the least common input integer and start with it, so you have less overall work to do on each step. Also compute the frequencies for the "not" cases, so you basically get the most bang for your buck on each step.
Finally: this could be pretty easily made into a parallel programming problem; if it's not fast enough on one machine, it seems you could put more machines on it pretty easily, if whatever it's returning is useful enough.
I have a linked list as 1->2->3->4->5->6
I need to change it to 1->6->2->5->3->4
i.e.last element linked to first element, last second element linked to the second element and so on.
I used 2 pointers, one fast and one slow. Once I reach the center, I put all the elements in the second half in a stack. [4, 5, 6]
Now, using a third pointer, I traverse the original linked list and insert node from the stack i.e. pop [6, 5, 4]
Is there any better solution than this?
I think this is optimal.
I use 2 pointers. One slow, one jump at a time and other fast, 2 jumps at a time.
Hence, I find the center and also the mid count.
Now, from the center till the end I reverse the linked list.
I have 2 linked lists now, one from start to center and other reversed from center to end.
Simply take one element from list 1 and link the element from link 2 to this and increment both the lists.
No extra space required and time complexity is O(N)
I just got this question from a textbook exercise saying
"Stack and Queue ADT's can be implemented using array. Which one is simpler to implement using an array? Explain"
I think using an array is probably not the best way to implement both a stack and queue in the first place because of the fixed space in an array, unless it is resized after each overflow of item.
I do not have a perfect response to this but which one of them is simpler to implement using arrays?
The only difference that I can think of is that with a stack, you only have to keep track of the front of the stack in the array, while with a queue you will need of keep track of both the front and end of the queue.
"Keep track of" means "storing an array index/offset for".
Other than that, the standard operations on stacks and queues are fairly similar in number; push(), pop() for stacks, and enqueue(), dequeue() for queues, and neither data type is particularly complex or difficult to implement.
A stack would be better implemented as an array compared to a queue, mainly because of how the types of operations affect the array itself.
Queue
For a queue data structure, you need to be able to remove elements from one end and push elements into the other. When you have an array, adding or removing an element from the front of the array is relatively bad because it involves you having to shift every other element to accommodate the new one.
queue: [2, 3, 4, 5, 6]
enqueue: 1
queue: [1, 2, 3, 4, 5, 6] (every element had to shift to fit 1 in the front)
or if you oriented your queue the opposite way,
queue: [1, 2, 3, 4, 5, 6]
dequeue: 1
queue: [2, 3, 4, 5, 6] (every element had to shift when 1 was removed from the front)
So no matter which direction you orient your queue, you will always have some operation (enqueue or dequeue) which involves adding/removing an element from the front of the array, which in turn causes every other element to shift which is relatively inefficient (would be great to avoid, and is why most queues aren't implemented with an array).
Stack
With a stack data structure, you only need to add and remove elements from the same end. This allows us to avoid the problem we were having with adding/removing elements from the front of the array. We just need to orient our stack to add and remove elements from the back of the array, and we will not encounter the problem with having to shift all the elements when something is added or removed.
stack: [1, 2, 3, 4]
push: 5
stack: [1, 2, 3, 4, 5] (nothing had to be shifted)
pop:
stack: [1, 2, 3, 4] (nothing had to be shifted)
Yes, It is obvious that array is not the best to implement queue or stack in data structure for real life problems.
I think, Implementation of a stack is always easier than the implementation of a queue because in stack we just have to push the element on the highest index and pop the same element from the same index. And if we want to push another element than we will push it on the same index. Every operation performs on the same index.
But in the case of a queue, there are two indices to trace one from which we have to dequeue the element and another index for the operation enqueue.
We have to update the indices for their corresponding operations(i.e. front when deque and end when enqueue).