Related
If I have a list of 10K elements, and I want to randomly iterate through all of them, is there an algorithm that lets me access each element randomly, without just sorting them randomly first?
In other words, this would not be ideal:
const sorted = list
.map(v => [math.random(), v])
.sort((a,b) => a[0]- b[0]);
It would be nice to avoid the sort call and the mapping call.
My only idea would be to store everything in a hashmap and access the hash keys randomly somehow? Although that's just coming back to the same problem, afaict.
Just been having a play with this and realised that the Fisher-Yates shuffle works well "on-line". For example, if you've got a large list you don't need to spend the time to shuffle the whole thing before you start iterating over items, or, equivalently, you might only need a few items out of a large list.
I didn't see a language tag in the question, so I'll pick Python.
from random import randint
def iterrand(a):
"""Iterate over items of a list in a random order.
Additional items can be .append()ed arbitrarily at runtime."""
for i, ai in enumerate(a):
j = randint(i, len(a)-1)
a[i], a[j] = a[j], ai
yield a[i]
This is O(n) in the length of the list and by allowing .append()s (O(1) in Python) the list can be built in the background.
An example use would be:
l = [0, 1, 2]
for i, v in enumerate(iterrand(l)):
print(f"{i:3}: {v:<5} {l}")
if v < 4:
l.append(randint(1, 9))
which might produce output like:
0: 2 [2, 1, 0]
1: 3 [2, 3, 0, 1]
2: 1 [2, 3, 1, 1, 0]
3: 0 [2, 3, 1, 0, 1, 3]
4: 1 [2, 3, 1, 0, 1, 3, 7]
5: 7 [2, 3, 1, 0, 1, 7, 7, 3]
6: 7 [2, 3, 1, 0, 1, 7, 7, 3]
7: 3 [2, 3, 1, 0, 1, 7, 7, 3]
8: 2 [2, 3, 1, 0, 1, 7, 7, 3, 2]
9: 3 [2, 3, 1, 0, 1, 7, 7, 3, 2, 3]
10: 2 [2, 3, 1, 0, 1, 7, 7, 3, 2, 3, 2]
11: 7 [2, 3, 1, 0, 1, 7, 7, 3, 2, 3, 2, 7]
Update: To test correctness, I'd do something like:
# trivial tests
assert list(iterrand([])) == []
assert list(iterrand([1])) == [1]
# bigger uniformity test
from collections import Counter
# tally 1M draws
c = Counter()
for _ in range(10**6):
c[tuple(iterrand([1, 2, 3, 4, 5]))] += 1
# ensure it's uniform
assert all(7945 < v < 8728 for v in c.values())
# above constants calculated in R via:
# k<-120;p<-0.001/k;qbinom(c(p,1-p), 1e6, 1/k))
Fisher-Yates should do the trick as good as any, this article is really good:
https://medium.com/#oldwestaction/randomness-is-hard-e085decbcbb2
the relevant JS code is very short and sweet:
const fisherYatesShuffle = (deck) => {
for (let i = deck.length - 1; i >= 0; i--) {
const swapIndex = Math.floor(Math.random() * (i + 1));
[deck[i], deck[swapIndex]] = [deck[swapIndex], deck[i]];
}
return deck
}
to yield results as you go, so you don't have to iterate through the list twice, use generator function like so:
const fisherYatesShuffle = function* (deck) {
for (let i = deck.length - 1; i >= 0; i--) {
const swapIndex = Math.floor(Math.random() * (i + 1)); // * use ;
[deck[i], deck[swapIndex]] = [deck[swapIndex], deck[i]];
yield deck[i];
}
};
(note don't forget some of those semi-colons, when the next line is bracket notation).
I tried to de-duplicate the sent arrays and then merge them into arrays
import { from, BehaviorSubject, distinct, switchMap, toArray } from "rxjs";
let list$ = new BehaviorSubject([1, 2, 3, 2, 3, 5]);
list$.pipe(
switchMap((e) => from(e)),
distinct(),
toArray()
).subscribe(console.log);
expected result:
BehaviorSubject -> [1, 2, 3, 2, 3, 5]
switchMap -> 1 2 3 2 3 5
distinct -> 1 2 3 5
toArray -> [1, 2, 3, 5]
console.log -> [1, 2, 3, 5]
Actually did not receive any value in console.log, why is this and how can I work as expected
"rxjs": "^7.2.0"
toArray only emits once the source observable completes.
The following should work as expected.
list$.pipe(
take(1),
switchMap(e => e),
distinct(),
toArray()
).subscribe(console.log);
If what you really want to do is filter unique values of an array, then RxJS's unique operator might be overkill. I wouldn't bother turning your array into a stream. Just filter the array.
list$.pipe(
map(a => [...new Set(a)])
).subscribe(console.log);
So, if the source does not stop after the first notification, I assume that it will continue emit other arrays and that you want to filter the duplicates on each array emitted. In other words, if the list$ of your example emits first [1, 2, 3, 2, 3, 5] and then [3, 2, 1, 6, 6, 6,] what you want to log are 2 arrays, [1, 2, 3, 5] and [3, 2, 1, 6].
If my assumption is right, than the solution could be the following
list$.pipe(
concatMap((e) => from(e).pipe(
distinct(),
toArray()
)),
).subscribe(console.log);
The trick here is that each from(e) stream will complete when there are no more elements in the array. Therefore, since it completes, the toArray operator can actually work.
scan could do the trick.
list$.pipe(
switchMap((e) => from(e)),
distinct(),
scan((acc, curr) => [...acc, curr], []),
).subscribe(console.log);
// will print: [1], [1, 2], [1, 2, 3], [1, 2, 3, 5]
You could insert debounceTime in the pipe, if you need less emissions:
list$.pipe(
switchMap((e) => from(e)),
distinct(),
scan((acc, curr) => [...acc, curr], []),
debounceTime(0)
).subscribe(console.log); // will print [1, 2, 3, 5]
If the only requirement is to remove duplicates, you're better off handling it using vaniall JS. See here: https://stackoverflow.com/a/9229821/6513921
We'll take the shortest solution without any regards to performance: uniq = [...new Set(array)];
You could then write a custom RxJS operator to include it in the pipe with other operators.
const { BehaviorSubject, from } = rxjs;
const { map, switchMap } = rxjs.operators;
const uniqueArray = (obs$) => {
return (obs$) => {
return obs$.pipe(
map(arr => [...new Set(arr)])
);
};
};
const sub = new BehaviorSubject([1, 2, 3, 2, 3, 5]);
sub.asObservable().pipe(
uniqueArray()
).subscribe(console.log);
sub.next([6, 3, 1, 6, 7, 1, 1]);
.as-console-wrapper { max-height: 100% !important; top: 0; }
<script src="https://unpkg.com/rxjs#6.2.2/bundles/rxjs.umd.min.js"></script>
In a project of mine, I'm trying to filter newly gathered information that also contains all the data from the previous request. With this filtered data, I'd like to add it to the old data as a new array. New data comes in as an array, and the old data is kept stored in a hash of arrays.
I've tried a number of different methods to remove all past data points from the current data unsuccessfully. An important detail here is that the new data may contain duplicate values that match older ones, but are technically new and should be treated as unique.
Here's an example data set:
x = {
'a' => [],
'b' => [1],
'c' => [],
'd' => [2, 3, 1, 5, 6, 3]
}
y = [0, 2, 3, 5, 1, 5, 6, 3, 1, 10, 7]
z = [0, 5, 10, 7]
x is the old data and y is the new data. The desired output of the filtering would be z that would then be added to x giving us:
x = {
'a' => [],
'b' => [1],
'c' => [],
'd' => [2, 3, 1, 5, 6, 3]
'e' => [0, 5, 10, 7]
}
I would need to continue repeating this for a bit based on some other criteria.
The main hurdle here is getting the filtering done correctly and has been proving difficult for me. Here's a list of some of the things I've tried:
I've tried iterating across the hash's keys and then simply subtracting the arrays, but that doesn't work properly as it gets rid of duplicates too, unfortunately.
irb(main):024:0> d = [2, 3, 1, 5, 6, 3]
=> [2, 3, 1, 5, 6, 3]
irb(main):025:0> y = [0, 2, 3, 5, 1, 5, 6, 3, 1, 10, 7]
=> [0, 2, 3, 5, 1, 5, 6, 3, 1, 10, 7]
irb(main):026:0> y - d
=> [0, 10, 7]
I've tried unions
irb(main):029:0> y | d
=> [0, 2, 3, 5, 1, 6, 10, 7]
and intersections. (which are definitely wrong)
irb(main):030:0> y & d
=> [2, 3, 5, 1, 6]
I tried (unsuccessfully) implementing the following from the second comment here
class Array
def delete_elements_in(ary)
ary.each do |x|
if index = index(x)
delete_at(index)
end
end
end
I've also tried reject!
irb(main):057:0> x = { 'a' => [], 'b' => [1], 'c' => [], 'd' => [2, 3, 1, 5, 6, 3] }
=> {"a"=>[], "b"=>[1], "c"=>[], "d"=>[2, 3, 1, 5, 6, 3]}
irb(main):058:0> y = [0, 2, 3, 5, 1, 5, 6, 3, 1, 10, 7]
=> [0, 2, 3, 5, 1, 5, 6, 3, 1, 10, 7]
irb(main):059:0> x.each_key { |key| y.reject! { |v| a[key].index(v) } }
=> {"a"=>[], "b"=>[1], "c"=>[], "d"=>[2, 3, 1, 5, 6, 3]}
irb(main):060:0> y
=> [0, 10, 7]
A more recent attempt I tried creating a new array from all of x's values and then using that against y, also unsuccessfully. I had just recently thought of trying to keep an array of 'seen' numbers, but I'm still stuck for items that actually need to be removed even though duplicate.
Throughout all this, I've been unable to get [0, 5, 10, 7] as a result.
Halp!
Here's something that might work for you:
>> existing = x.values.flatten
#> [1, 2, 3, 1, 5, 6, 3]
>> z = y.dup # This avoids altering the original `y` array
>> existing.each { |e| z.delete_at(z.index(e)) if z.index(e) }
>> z
#> [0, 5, 10, 7] # z now contains the desired result
>> x['e'] = z
>> pp x
{"a"=>[],
"b"=>[1],
"c"=>[],
"d"=>[2, 3, 1, 5, 6, 3],
"e"=>[0, 5, 10, 7]}
Here's the whole thing in a single method:
def unique_array_filter(hash, new_array)
existing = hash.values.flatten
next_key = hash.keys.max.next
temp = new_array.dup
existing.each { |e| temp.delete_at(temp.index(e)) if temp.index(e) }
hash[next_key] = temp
hash
end
>> unique_array_filter(x, y)
#> {"a"=>[], "b"=>[1], "c"=>[], "d"=>[2, 3, 1, 5, 6, 3], "e"=>[0, 5, 10, 7]}
x.merge(x.keys.max.next => y.difference(x.values.flatten))
#=> {"a"=>[], "b"=>[1], "c"=>[], "d"=>[2, 3, 1, 5, 6, 3], "e"=>[0, 5, 10, 7]}
where Array#difference is defined as follows.
class Array
def difference(other)
h = other.each_with_object(Hash.new(0)) { |e,h| h[e] += 1 }
reject { |e| h[e] > 0 && h[e] -= 1 }
end
end
See the link for an explanation of Array#difference.
I'm weirdly stuck with the following:
I have an Observable that emits either an array with some items or an empty array, and this Observable emits only once - let's call it startingArray$
Then I have a hot Observable that emits individual items that I would like to push to startingArray once startingArray has been emitted - let's call it additions$
What code gives me the resulting observable startingArrayPlusAdditions$ that continuosly grows as more items are emitted?
startingArrayPlusAdditions$ should emit every time additions$ emits, but it should not emit when startingArray$ emits initially
Not sure that I understood all your problem but here's a proposal:
const { Observable } = Rx;
const startingArray$ = Observable.of([1, 2, 3]);
const additions$ = Observable.from([4, 5, 6, 7, 8]);
const startingArrayPlusAdditions$ = startingArray$
.combineLatest(additions$)
.scan((acc, current) => {
const [startingArray, addition] = current;
if (acc === null) {
return [...startingArray, addition];
} else {
acc.push(addition);
return acc;
}
}, null)
.do(console.log)
.subscribe();
The output is:
[1, 2, 3, 4]
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6, 7]
[1, 2, 3, 4, 5, 6, 7, 8]
So just as you asked:
startingArrayPlusAdditions$ should emit every time additions$ emits, but it should not emit when startingArray$ emits initially
Here's a working Plunkr: https://plnkr.co/edit/rKXLJrmA7mSzpQgoemlD?p=preview
I need to get all the possible number combinations from denom_arr which equal the amt.
denom_arr = [4,3,1]
amt = 10
This case would produce:
[4, 4, 1, 1]
[3, 3, 3, 1]
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[4, 3, 1, 1, 1]
[4, 3, 3]
. . . (other cases...)
Problem is the code I wrote is breaking after 1-3 and I'm not sure how to make it loop over the same index to get case 4-6+
set, sets = [], []
i = 0
loop do
i = 0 if denom_arr[i].nil?
loop do
set << denom_arr[i]
break if set.inject(:+) > amt
end
set.pop if set.inject(:+) > amt
if set.inject(:+) == amt
sets << set
set = []
denom_arr.shift
end
i += 1
sets
break if denom_arr.empty?
end
UPDATE
I know this can be done with recursion with memoization/dynamic programming techniques, but I am trying to do this strictly in a loop for the sake of testing a theory.
I would do this recursively
def possible_sums(arr, amt)
return [[]] if amt == 0
return [] if amt < 0
arr.reduce([]) do |sums, e|
sums.concat(
possible_sums(arr, amt-e)
.map { |sum| sum.unshift(e).sort }
)
end.uniq
end
p possible_sums([4,3,1], 10)
# => [
# [1, 1, 4, 4], [3, 3, 4], [1, 1, 1, 3, 4], [1, 1, 1, 1, 1, 1, 4],
# [1, 3, 3, 3], [1, 1, 1, 1, 3, 3], [1, 1, 1, 1, 1, 1, 1, 3],
# [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
# ]
Although this is potentially inefficient in that it repeats work, this can be alleviated by using dynamic programming (essentially, memoizing the results of the recursive function).
UPDATE Here is an iterative solution:
def possible_sums_it(arr, amt)
sums = Array.new(amt+1) { [] }
sums[0] << []
(1..amt).each do |i|
arr.each do |e|
if i-e >= 0
sums[i].concat(
sums[i-e].map { |s| [e, *s].sort }
)
end
end
sums[i].uniq!
end
sums[amt]
end
This is in fact the dynamic programming algorithm for the problem.
So if you squint at it just right, you'll see that essentially what it is doing, is calculating all the possible sums for 0 up to amt into the sums array, using what is basically the recursive algorithm, but instead of the recursive call, we lookup a value in sums that we have calculated beforehand.
This works because we know that we won't need sums[i] before sums[j] for j < i.