Related
Here's RxJS code I'm trying to reproduce with RxPY.
const counter$ = interval(1000);
counter$
.pipe(
mapTo(-1),
scan((accumulator, current) => {
return accumulator + current;
}, 10),
takeWhile(value => value >= 0)
)
.subscribe(console.log);
9
8
7
6
5
4
3
2
1
0
-1
And here's what I through was equivalent but is not
counter = rx.interval(1)
composed = counter.pipe(
ops.map(lambda value: value - 1),
ops.scan(lambda acc, curr: acc + curr, 10),
ops.take_while(lambda value: value >= 0),
)
composed.subscribe(lambda value: print(value))
9
9
10
12
15
19
Could someone help me to understand what I'm missing here?
I don't know python at all, but I do notice one difference in your map between your js and python:
mapTo(-1) // always emits -1
-- vs --
ops.map(lambda value: value - 1) # emits interval index - 1
I think the solution is simple, just remove the "value":
ops.map(lambda value: -1)
However, if your "current value" is always -1 you can simplify by not using map at all and put -1 in your scan() function. Here's what it looks like in rxjs:
const countDown$ = interval(1000).pipe(
scan(accumulator => accumulator - 1, 10)
takeWhile(value => value >= 0)
);
How do I filter a publisher for the elements having the highest value without knowing the highest value beforehand?
Here is a little test to illustrate what I'm trying to achieve:
#Test
fun filterForHighestValuesTest() {
val numbers = Flux.just(1, 5, 7, 2, 8, 3, 8, 4, 3)
// what operators to apply to numbers to make the test pass?
StepVerifier.create(numbers)
.expectNext(8)
.expectNext(8)
.verifyComplete()
}
Ive started with the reduce operator:
#Test
fun filterForHighestValuesTestWithReduce() {
val numbers = Flux.just(1, 5, 7, 2, 8, 3, 8, 4, 3)
.reduce { a: Int, b: Int -> if( a > b) a else b }
StepVerifier.create(numbers)
.expectNext(8)
.verifyComplete()
}
and of course that test passes but that will only emit a single Mono whereas I would like to obtain a Flux containing all the elements having the highest values e.g. 8 and 8 in this simple example.
First of all, you'll need state for this so you need to be careful to have per-Subscription state. One way of ensuring that while combining operators is to use compose.
Proposed solution
Flux<Integer> allMatchingHighest = numbers.compose(f -> {
AtomicInteger highestSoFarState = new AtomicInteger(Integer.MIN_VALUE);
AtomicInteger windowState = new AtomicInteger(Integer.MIN_VALUE);
return f.filter(v -> {
int highestSoFar = highestSoFarState.get();
if (v > highestSoFar) {
highestSoFarState.set(v);
return true;
}
if (v == highestSoFar) {
return true;
}
return false;
})
.bufferUntil(i -> i != windowState.getAndSet(i), true)
.log()
.takeLast(1)
.flatMapIterable(Function.identity());
});
Note the whole compose lamdba can be extracted into a method, making the code use a method reference and be more readable.
Explaination
The solution is done in 4 steps, with the two first each having their own AtomicInteger state:
Incrementally find the new "highest" element (so far) and filter out elements that are smaller. This results in a Flux<Integer> of (monotically) increasing numbers, like 1 5 7 8 8.
buffer by chunks of equal number. We use bufferUntil instead of window* or groupBy because the most degenerative case were numbers are all different and already sorted would fail with these
skip all buffers but one (takeLast(1))
"replay" that last buffer, which represents the number of occurrences of our highest value (flatMapIterable)
This correctly pass your StepVerifier test by emitting 8 8. Note the intermediate buffers emitted are:
onNext([1])
onNext([5])
onNext([7, 7, 7])
onNext([8, 8])
More advanced testing, justifying bufferUntil
A far more complex source that would fail with groupBy but not this solution:
Random rng = new Random();
//generate 258 numbers, each randomly repeated 1 to 10 times
//also, shuffle the whole thing
Flux<Integer> numbers = Flux
.range(1, 258)
.flatMap(i -> Mono.just(i).repeat(rng.nextInt(10)))
.collectList()
.map(l -> {
Collections.shuffle(l);
System.out.println(l);
return l;
})
.flatMapIterable(Function.identity())
.hide();
This is one example of what sequence of buffers it could filter into (keep in mind only the last one gets replayed):
onNext([192])
onNext([245])
onNext([250])
onNext([256, 256])
onNext([257])
onNext([258, 258, 258, 258, 258, 258, 258, 258, 258])
onComplete()
Note: If you remove the map that shuffles, then you obtain the "degenerative case" where even windowUntil wouldn't work (the takeLast would result in too many open yet unconsumed windows).
This was a fun one to come up with!
One way to do it is to map the flux of ints to a flux of lists with one int in each, reduce the result, and end with flatMapMany, i.e.
final Flux<Integer> numbers = Flux.just(1, 5, 7, 2, 8, 3, 8, 4, 3);
final Flux<Integer> maxValues =
numbers
.map(
n -> {
List<Integer> list = new ArrayList<>();
list.add(n);
return list;
})
.reduce(
(l1, l2) -> {
if (l1.get(0).compareTo(l2.get(0)) > 0) {
return l1;
} else if (l1.get(0).equals(l2.get(0))) {
l1.addAll(l2);
return l1;
} else {
return l2;
}
})
.flatMapMany(Flux::fromIterable);
One simple solution that worked for me -
Flux<Integer> flux =
Flux.just(1, 5, 7, 2, 8, 3, 8, 4, 3).collectSortedList(Comparator.reverseOrder()).flatMapMany(Flux::fromIterable);
StepVerifier.create(flux).expectNext(8).expectNext(8).expectNext(7).expectNext(5);
One possible solution is to group the Flux prior to the reduction and flatmap the GroupedFlux afterwards like this:
#Test
fun filterForHighestValuesTest() {
val numbers = Flux.just(1, 5, 7, 2, 8, 3, 8, 4, 3)
.groupBy { it }
.reduce { t: GroupedFlux<Int, Int>, u: GroupedFlux<Int, Int> ->
if (t.key()!! > u.key()!!) t else u
}
.flatMapMany {
it
}
StepVerifier.create(numbers)
.expectNext(8)
.expectNext(8)
.verifyComplete()
}
sample dataset with single column:
5.1,
4.9,
4.7,
4.6,
5,3.
5.4,
4.6,
5,
4.4,
4.9,
5.4,
4.8,
4.8,
4.3,
5.8
I want it to be sorted in ascending order first and then select values at intervals and return it as an array.
For example, if the interval = 5, and the sorted dataset is
4.3,
4.4,
4.6,
4.6,
4.7,
4.8,
4.8,
4.9,
4.9,
5,
5.1,
5,3.
5.4,
5.4,
5.8
It should return Array(4.3, 4.7, 5, 5.8).
Is there any way to do this in an optimistic way?
Thanks in advance
Shakti
this is what i have tried, but not able to get the first value.
val interval = 5
val count = df.count() //15
val n = (count/interval).toInt //3
println(s"interval: $interval, count: $count, n: $n")
val window = Window.orderBy("col1")
val sorted = df.withColumn("rowId", functions.row_number().over(window))
sorted.show()
val sb = new StringBuilder
for (i <- 0 to n) {
val intervalPoint = interval * i
println(s"i: $i, intervalPoint: $intervalPoint")
sb.append(s"rowId == $intervalPoint or ")
}
sb.delete(sb.size - 3, sb.size - 1)
println(s"sb: ${sb.toString()}") //rowId == 0 or rowId == 5 or rowId == 10 or rowId == 15
val intervals = sorted.where(sb.toString()).select("col1").collectAsList()
println(s"intervals: $intervals") //[[4.7], [5.0], [5.8]]
as you can see, first it has to be sorted by col and append a row id. hopefully, these two would be done in one scan. and another scan of entire dataset to get the intervals, and that too, i was unable to get the first value. If this has to be applied on several columns, it has to be in a loop, and the no. of scans would increase.
one possible solutions (cleaner) :
lets say you have a dataframe 'df' :
df.createOrReplaceTempView("temp_table_1")
sparkSession.sql("select col1 from (select ROW_NUMBER() OVER (ORDER BY col1) AS id, col1 from temp_table_1) y where id%(SOME_INTERVAL) = 0 order by col1").show()
I'm new to RxJS. I know I could just .filter and .map an observable to get the change I'm looking for. But, is there any method which combines the two into one function?
Yes there is.
FlatMap.
Suppose you have an Observable of numbers (1, 2, 3, 4, 5, ...) and you want to filter for even numbers and map them to x*10.
var tenTimesEvenNumbers = numbers.flatMap(function (x) {
if (x % 2 === 0) {
return Rx.Observable.just(x * 10);
} else {
return Rx.Observable.empty();
}
});
As of rxjs v6.6.7, the solution becomes as following:
// Initialise observable with some numbers
const numbers = of(1, 2, 3, 4, 5, 6, 7, 8, 9, 10);
// Pipe the observable using mergeMap
const tenTimesEvenNumbers = numbers.pipe(
mergeMap((x: number) => {
// If the number is even, return an observable containing the number multiplied by ten
// Otherwise return an empty observable
return x % 2 === 0 ? of(x * 10) : EMPTY;
})
);
// Subscribe to the observable and print the values
tenTimesEvenNumbers.subscribe((value: number) =>
console.log('Value:', value)
);
The above will print:
Value: 20
Value: 40
Value: 60
Value: 80
Value: 100
Here is a working stackblitz as well.
I need to be able to process one word or words and verify that it has valid syllables. There are some syllabification rules that could be used:
V CV VC CVC CCV CCCV CVCC
where V is a vowel and C is a consonant. e.g.,
pronunciation (5 Pro-nun-ci-a-tion; CCV-CVC-CV-V-CVC)
Or is there a simple code that can be used, or a library in c++? In class we're talking about binary search trees, hash tables, etc, but i can't really see the relation. Any help would appreciated, thanks.
Whenever we have collected a full pattern-string, we can either discard it and begin collecting to a new pattern-string, or keep it and try to get a longer pattern-string. We don't know in advance (without examining the rest of the input-string), whether we should keep or discard the current string, so we need to keep both possibilities in mind.
We can build a state machine that can keep track of this for us. The base-states are identified by the sequence of characters we have examined so far:
State C V
"" {"C"} {"V",""}
"C" {"CC"} {"CV",""}
"CC" {"CCC"} {""}
"CCC" {} {""}
"CV" {"CVC",""} {}
"CVC" {""} {}
"V" {""} {}
Since we don't always know which action to take, we can be in several possible states at once. Those sets of possible states form super-states:
Index Super-state C V
0 {} 0 0 Fail
1 {""} 2 9 Accept
2 {"C"} 3 8
3 {"CC"} 4 1
4 {"CCC"} 0 1
5 {"","C"} 6 13 Accept
6 {"C","CC"} 7 8
7 {"CC","CCC"} 4 1
8 {"","CV"} 12 9 Accept
9 {"","V"} 5 9 Accept
10 {"","C","CC"} 11 13 Accept
11 {"C","CC","CCC"} 7 8
12 {"","C","CVC"} 10 13 Accept
13 {"","CV","V"} 12 9 Accept
The transitions are between super-states. Each member of the super-state is advanced with the same symbol. All members without such transition are discarded. If a member has two possible destinations, both are added to the new super-state.
You might notice that some rows are very similar. Super-state 3 and 7 have the same transitions. As are 6 and 11, and 8 and 13. You could collapse those into one state each, and update the indices. I'm not going to demonstrate that here.
This could easily be encoded into a programming language:
// index = 0 1 2 3 4 5 6 7 8 9 10 11 12 13
int[] consonant = new int[] { 0, 2, 3, 4, 0, 6, 7, 4, 12, 5, 11, 7, 10, 12 };
int[] vocal = new int[] { 0, 9, 8, 1, 1, 13, 8, 1, 9, 9, 13, 8, 13, 9 };
int[] accept = new int[] { 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1 };
int startState = 1;
int failState = 0;
bool CheckWord(string word)
{
int state = startState;
foreach (char c in word)
{
if (IsVocal(c))
{
state = vocal[state];
}
else if (IsConsonant(c))
{
state = consonant[state];
}
if (state == failState) return false;
}
return accept[state] != 0;
}
Example:
> CheckWord("pronunciation")
true
> CheckWord("pronunciationn")
false