Let's say I have a set of rewriting rules that I want my Observable to implement:
A => B B
B => C
That is, whenever there's an A on the observable, output two Bs. Whenever there's a B (even when it comes from an A), output a C. This
seems like natural use case for expand, and it's pretty easy to implement. As expected, each A turns into ABCBC.
But what if I want to combine the expansion with another operator? Let's say my rules are:
A => B repeated N times, where N is the number of "A"s so far.
B => C, but only if the number of "B"s so far is divisible by 10.
Grouping values by whether they're As or Bs sounds like a job for groupBy, and counting the number of outputs for each sounds like scan, but how do I combine those operators with expand?
Two years late and not strictly an answer to your question, but another approach that yields the same result and might be easier to build upon.
Building streams recursively:
This approach builds out your stream recursively. Kind of the same idea you may use to build a nieve parser.
function recursiveOperator(){
return s => s.pipe(
mergeMap(val => concat(
of(val),
streamFor(val)
))
)
}
function streamFor(x){
let stream;
if (x == 'A') {
stream = from(['B','B']);
}else if (x == 'B') {
stream = of('C');
}else{
stream = EMPTY;
}
return stream.pipe(
recursiveOperator()
)
}
zip(
timer(0, 4000),
from(['A','B','A'])
).pipe(
map(([x,y]) => y),
recursiveOperator()
).subscribe(console.log);
output:
abcbc // Then wait 4 seconds
bc // Then wait 4 seconds
abcbc
Using a Subject:
I'm pretty sure that as-is, this introduces a race condition. They're resolved consistently since JS is single-threaded and the event loop is predictable, but you're having two streams each pushing values to the same subject and I haven't yet built in anything that ensures no unwanted interleaving.
With some tweaking, however, this can be much more performant for large inputs.
const valueSubject = new Subject<string>();
valueSubject.subscribe(console.log);
zip(
timer(0, 4000),
from(['A', 'B', 'A'])
).pipe(
map(([x,y]) => y)
).subscribe(val => valueSubject.next(val));
valueSubject.pipe(
tap(val => {
if (val == 'A') {
valueSubject.next('B');
valueSubject.next('B');
}else if (val == 'B') {
valueSubject.next('C');
}
})
).subscribe();
output:
abcbc // Then wait 4 seconds
bc // Then wait 4 seconds
abcbc
Related
I have a code which should give number 1 or 5 or 10 or 50.
rust compiler says "pattern 4_u8..=u8::MAX not covered"
the code:
use rand::Rng;
fn main() {
let rand_num: u8 = rand::thread_rng().gen_range(0, 4);
println!("{}", rand_num);
let coin: Coin;
match rand_num{
0 => coin = Coin::Penny,
1 => coin = Coin::Nickel,
2 => coin = Coin::Dime,
3 => coin = Coin::Quarter,
}
println!("{}", value_in_cents(coin));
}
enum Coin {
Penny,
Nickel,
Dime,
Quarter,
}
fn value_in_cents(coin: Coin) -> u8 {
match coin{
Coin::Penny => return 1,
Coin::Nickel => return 5,
Coin::Dime => return 10,
Coin::Quarter => return 25,
}
}
i have no idea how can i fix it
Matches in rust need to be exhaustive. In other words they need to cover every possible case.
When you are matching on the enum Coin you are matching every variant in the enum - so it is valid.
But when you are matching on an integer (in this case, a u8) the possible values can be anywhere from 0 to 255. Consider what happens if the number generated passed to match is 9 - rust would attempt to match and find no matching branch. From there coin would be undefined and the rest of the program would be faulty.
Logically we know that the match will always be exhausted, since rand::thread_rng().gen_range(0, 4); will generate a number within that range. But either way the rules applied to match must be followed. In this case, you can add a fallback branch which will cover any remaining case.
match rand_num{
0 => coin = Coin::Penny,
1 => coin = Coin::Nickel,
2 => coin = Coin::Dime,
3 => coin = Coin::Quarter,
_ => panic!("Invalid coin");
}
This would allow your code to compile - but in the case that an invalid number is generated the program would crash.
As an aside, you could simplify this code by assigning the value of the match directly to the coin variable:
let coin = match rand_num{
0 => Coin::Penny,
1 => Coin::Nickel,
2 => Coin::Dime,
3 => Coin::Quarter,
_ => panic!()
}
Other answers are valid, except for the fact that there is a dedicated instruction for this kind of situations: unreachable!().
match rand_num {
0 => coin = Coin::Penny,
1 => coin = Coin::Nickel,
2 => coin = Coin::Dime,
3 => coin = Coin::Quarter,
_ => unreachable!(),
}
It won't change much, as if unreachable!() is actually reached, it will just panic. However, this greatly improves readability.
Match requires you to list out all possible values that your variable rand_num could match to. In your case, that would be all possible u8 values. The compiler doesn't know your random range cannot generate other numbers than the ones you listed, so it throws out an error. To solve this, use the wildcard pattern:
_ => {panic!()}
I've modelled a navigation concept reactively, (a user navigating forward and backward pages) but I'm having trouble optimizing it using Reactive Extensions. I'm sure it's some kind of buffering or sharing operator, but I'm not quite sure which one it is or where to shove it.
This is a working, simplified example of what I've implemented, in TypeScript and with rxjs.
Each time the user clicks, we increment it by one step. (In reality, there is a network request in here which might return a new step or none, hence why this is not a simple + 1.)
describe('navigate', () => {
it('should increment', () => {
const initial = Observable.of(0);
const plusOne = (ns: Observable<number>) => ns.pipe(map(n => n + 1), tap(x => console.log("Computed", x)));
const clicks = Observable.from([plusOne, plusOne, plusOne]);
const expected = cold("(wxyz|)", { w: 0, x: 1, y: 2, z: 3 });
const result = clicks.pipe(
scan(
(s: Observable<number>, x: (ns: Observable<number>) => Observable<number>) => x(s),
initial),
startWith(initial),
switchAll(),
)
expect(result).toBeObservable(expected);
})
})
It produces the correct output, and the test passes.
But in terms of execution, in the console you'll see this printed:
Computed 1
Computed 1
Computed 2
Computed 1
Computed 2
Computed 3
which makes sense, but if the plusOne operation is expensive (e.g. a network request) it won't do to have the plusOnes computed from the start every time.
How do I memoize the result of the last plusOne so that subsequent plusOnes need not calculate the entire chain again?
Or if I'm looking at this the wrong way, how should I be approaching this problem?
Attempt
Since the top of this question, I did think of one solution.
const result = clicks.pipe(
scan(
(s: Observable<number>, x: (ns: Observable<number>) => Observable<number>) => x(s).pipe(shareReplay(1)),
initial),
startWith(initial),
switchAll(),
)
which executes like:
Computed 1
Computed 2
Computed 3
However I think this leads to a chain which would look like:
xs.map(x => x + 1).shareReplay(1).map(x => x + 1).shareReplay(1).map(x => x + 1).shareReplay(1)
and which I don't think would be terribly efficient (if each shareReplay(1) caches a value). Is there a better way to do this?
I am just beginning with Scala development and am trying to filter out unnecessary lines from an iterator using filter and collect. But the operation seems to be too slow.
val src = Source.fromFile("/home/Documents/1987.csv") // 1.2 Million
val iter = src.getLines().map(_.split(":"))
val iter250 = iter.take(250000) // Only interested in the first 250,000
val intrestedIndices = range(1, 100000, 3).toSeq // This could be any order
val slicedData = iter250.zipWithIndex
// Takes 3 minutes
val firstCase = slicedData.collect { case (x, i) if intrestedIndices.contains(i) => x }.size
// Takes 3 minutes
val secondCase = slicedData.filter(x => intrestedIndices.contains(x._2)).size
// Takes 1 second
val thirdCase = slicedData.collect { case (x,i ) if i % 3 == 0 => x}.size
It appears the intrestedIndices.contains(_) part is slowing down the program in the first and second case. Is there an alternative way to speed this process up.
This answer helped solve the problem.
You iterate over all interestedIndices in first two cases in linear time. Use Set instead of Seq to improve performance – Sergey Lagutin
For the record, here's a method to filter with an (ordered) Seq of indices, not necessarily equidistant, without scanning the indices at each step:
def filterInteresting[T](it: Iterator[T], indices: Seq[Int]): Iterator[T] =
it.zipWithIndex.scanLeft((indices, None: Option[T])) {
case ((indices, _), (elem, index)) => indices match {
case h :: t if h == index => (t, Some(elem))
case l => (l, None)
}
}.map(_._2).flatten
If I chain clauses such as
var results = elements
.Where(n => n > 3)
.Where(n => n % 2 == 0);
is this slower than just
var results = elements.Where(n => n > 3 && n % 2 == 0);
Explain why or why not?
EDIT: It seems that the consensus is that even POCO objects iterate twice. If this is the case can someone explain why Microsoft wouldn't combine these predicates. I stumbled across Enumerable.CombinePredicates that I thought did this. Can someone please explain what this does then.
Edit:
I looked a little closer. The WhereEnumerableIterator returned by the Where extension method actually overrides the Where method and combines the predicates into a single callback.
public override IEnumerable<TSource> Where(Func<TSource, bool> predicate) {
return new Enumerable.WhereEnumerableIterator<TSource>(
this.source,
Enumerable.CombinePredicates<TSource>(this.predicate, predicate));
}
private static Func<TSource, bool> CombinePredicates<TSource>(
Func<TSource, bool> predicate1, Func<TSource, bool> predicate2
) {
return (TSource x) => predicate1(x) && predicate2(x);
}
So, the speed difference I saw on my machine should probably be attributed to something else.
The first example will loop over the elements collection once to find items that satisfy the condition item > 3, and again to find items that satisfy the condition item % 2 == 0.
The second example will loop over the elements collection once to find items that satisfy the condition item > 3 && item % 2 == 0.
In the examples provided, the second will most likely always be faster than the first, because it only loops over elements once.
Here is an example of some pretty consistent results I get on my machine (.NET 3.5):
var stopwatch = new System.Diagnostics.Stopwatch();
var elements = Enumerable.Range(1, 100000000);
var results = default(List<int>);
stopwatch.Start();
results = elements.Where(n => n > 3).Where(n => n % 2 == 0).ToList();
stopwatch.Stop();
Console.WriteLine(stopwatch.Elapsed);
stopwatch.Reset();
stopwatch.Start();
results = elements.Where(n => n > 3 && n % 2 == 0).ToList();
stopwatch.Stop();
Console.WriteLine(stopwatch.Elapsed);
Console.WriteLine("Done");
Console.ReadLine();
Results:
00:00:03.0932811
00:00:02.3854886
Done
EDIT:
#Rawling is right in that my explanation only applies to LINQ as used on collections of POCO objects. When used as an interface to LINQ-to-SQL, NHibernate, EF, etc., your results will be more implementation-dependent.
If you're talking LINQ-to-objects, each Where involves setting up a new iterator state machine, which is expensive, so yes, it's slower than putting both conditions together.
If you're talking about LINQ-to-something else, well, it depends; an extra Where might just involve an extra string concatenation somewhere. It's still likely to be more expensive, but the exect difference depends on the LINQ provider.
Let's say we have an array of age groups and an array of the number of people in each age group
For example:
Ages = ("1-13", "14-20", "21-30", "31-40", "41-50", "51+")
People = (1, 10, 21, 3, 2, 1)
I want to have an algorithm that combines these age groups with the following logic if there are fewer than 5 people in each group. The algorithm that I have so far does the following:
Start from the last element (e.g., "51+") can you combine it with the next group? (here "41-50") if yes add the numbers 1+2 and combine their labels. So we get the following
Ages = ("1-13", "14-20", "21-30", "31-40", "41+")
People = (1, 10, 21, 3, 3)
Take the last one again (here is "41+"). Can you combine it with the next group (31-40)? the answer is yes so we get:
Ages = ("1-13", "14-20", "21-30", "31+")
People = (1, 10, 21, 6)
since the group 31+ now has 6 members we cannot collapse it into the next group.
we cannot collapse "21-30" into the next one "14-20" either
"14-20" also has 10 people (>5) so we don't do anything on this either
for the first one ("1-13") since we have only one person and it is the last group we combine it with the next group "14-20" and get the following
Ages = ("1-20", "21-30", "31+")
People = (11, 21, 6)
I have an implementation of this algorithm that uses many flags to keep track of whether or not any data is changed and it makes a number of passes on the two arrays to finish this task.
My question is if you know any efficient way of doing the same thing? any data structure that can help? any algorithm that can help me do the same thing without doing too much bookkeeping would be great.
Update:
A radical example would be (5,1,5)
in the first pass it becomes (5,6) [collapsing the one on the right into the one in the middle]
then we have (5,6). We cannot touch 6 since it is larger than our threshold:5. so we go to the next one (which is element on the very left 5) since it is less than or equal to 5 and since it is the last one on the left we group it with the one on its right. so we finally get (11)
Here is an OCaml solution of a left-to-right merge algorithm:
let close_group acc cur_count cur_names =
(List.rev cur_names, cur_count) :: acc
let merge_small_groups mini l =
let acc, cur_count, cur_names =
List.fold_left (
fun (acc, cur_count, cur_names) (name, count) ->
if cur_count <= mini || count <= mini then
(acc, cur_count + count, name :: cur_names)
else
(close_group acc cur_count cur_names, count, [name])
) ([], 0, []) l
in
List.rev (close_group acc cur_count cur_names)
let input = [
"1-13", 1;
"14-20", 10;
"21-30", 21;
"31-40", 3;
"41-50", 2;
"51+", 1
]
let output = merge_small_groups 5 input
(* output = [(["1-13"; "14-20"], 11); (["21-30"; "31-40"; "41-50"; "51+"], 27)] *)
As you can see, the result of merging from left to right may not be what you want.
Depending on the goal, it may make more sense to merge the pair of consecutive elements whose sum is smallest and iterate until all counts are above the minimum of 5.
Here is my scala approach.
We start with two lists:
val people = List (1, 10, 21, 3, 2, 1)
val ages = List ("1-13", "14-20", "21-30", "31-40", "41-50", "51+")
and combine them to a kind of mapping:
val agegroup = ages.zip (people)
define a method to merge two Strings, describing an (open ended) interval. The first parameter is, if any, the one with the + in "51+".
/**
combine age-strings
a+ b-c => b+
a-b c-d => c-b
*/
def merge (xs: String, ys: String) = {
val xab = xs.split ("[+-]")
val yab = ys.split ("-")
if (xs.contains ("+")) yab(0) + "+" else
yab (0) + "-" + xab (1)
}
Here is the real work:
/**
reverse the list, combine groups < threshold.
*/
def remap (map: List [(String, Int)], threshold : Int) = {
def remap (mappings: List [(String, Int)]) : List [(String, Int)] = mappings match {
case Nil => Nil
case x :: Nil => x :: Nil
case x :: y :: xs => if (x._2 > threshold) x :: remap (y :: xs) else
remap ((merge (x._1, y._1), x._2 + y._2) :: xs) }
val nearly = (remap (map.reverse)).reverse
// check for first element
if (! nearly.isEmpty && nearly.length > 1 && nearly (0)._2 < threshold) {
val a = nearly (0)
val b = nearly (1)
val rest = nearly.tail.tail
(merge (b._1, a._1), a._2 + b._2) :: rest
} else nearly
}
and invocation
println (remap (agegroup, 5))
with result:
scala> println (remap (agegroup, 5))
List((1-20,11), (21-30,21), (31+,6))
The result is a list of pairs, age-group and membercount.
I guess the main part is easy to understand: There are 3 basic cases: an empty list, which can't be grouped, a list of one group, which is the solution itself, and more than one element.
If the first element (I reverse the list in the beginning, to start with the end) is bigger than 5 (6, whatever), yield it, and procede with the rest - if not, combine it with the second, and take this combined element and call it with the rest in a recursive way.
If 2 elements get combined, the merge-method for the strings is called.
The map is remapped, after reverting it, and the result reverted again. Now the first element has to be inspected and eventually combined.
We're done.
I think a good data structure would be a linked list of pairs, where each pair contains the age span and the count. Using that, you can easily walk the list, and join two pairs in O(1).