Microsoft introduced the IObservable<T> interface to the BCL with .NET Framework 4, and I thought, "Great, finally, I must use it!" So I dug deep and read posts and documentation and even implemented the pattern.
After doing so I've realized that the basic implementation actually sends all the T events to all of its subscribers without any filtering on it; i.e. plain broadcast. I read somewhere that the Observable pattern is meant for plain broadcast. I feel that this is not true and that I am missing something.
My questions:
If I add a filtering mechanism, what is the difference between using the Observable pattern and just using plain CLR events?
When should one use this pattern, and when should one choose to use plain CLR events?
What are the main advantages of the Observable pattern?
Observable is the cornerstone of the Rx library. They provide pretty much all the implementations and operators you'll need. The idea behind IObservable<T> and Rx is not just the "handling" of events, but enabling "LINQ to Events." So you can easily compose "event streams," which gives you a lot of power compared to regular event handling.
Note that the sample MSDN implementation of IObservable<T> is incorrect; the doc team has been notified.
Related
I am designing a package where I want to provide an API based on the observer pattern: that is, there are points where I'd like to emit a signal that will trigger zero or more interested parties. Those interested parties shouldn't necessarily need to know about each other.
I know I can implement an API like this from scratch (e.g. using a collection of channels or callback functions), but was wondering if there was a preferred way of structuring such APIs.
In many of the languages or frameworks I've played with, there has been standard ways to build these APIs so that they behave the way users expect: e.g. the g_signal_* functions for glib based applications, events and addEventListener() for JavaScript DOM apps, or multicast delegates for .NET.
Is there anything similar for Go? If not, is there some other way of structuring this type of API that is more idiomatic in Go?
I would say that a goroutine receiving from a channel is an analogue of an observer to a certain extent. An idiomatic way to expose events in Go would be thus IMHO to return channels from a package (function). Another observation is that callbacks are not used too often in Go programs. One of the reasons is also the existence of the powerful select statement.
As a final note: some people (me too) consider GoF patterns as Go antipatterns.
Go gives you a lot of tools for designing a signal api.
First you have to decide a few things:
Do you want a push or a pull model? eg. Does the publisher push events to the subscribers or do the subscribers pull events from the publisher?
If you want a push system then having the subscribers give the publisher a channel to send messages on would work really well. If you want a pull method then just a message box guarded with a mutex would work. Other than that without knowing more about your requirements it's hard to give much more detail.
I needed an "observer pattern" type thing in a couple of projects. Here's a reusable example from a recent project.
It's got a corresponding test that shows how to use it.
The basic theory is that an event emitter calls Submit with some piece of data whenever something interesting occurs. Any client that wants to be aware of that event will Register a channel it reads the event data off of. This channel you registered can be used in a select loop, or you can read it directly (or buffer and poll it).
When you're done, you Unregister.
It's not perfect for all cases (e.g. I may want a force-unregister type of event for slow observers), but it works where I use it.
I would say there is no standard way of doing this because channels are built into the language. There is no channel library with standard ways of doing things with channels, there are simply channels. Having channels as built in first class objects frees you from having to work with standard techniques and lets you solve problems in the simplest most natural way.
There is a basic Golang version of Node EventEmitter at https://github.com/chuckpreslar/emission
See http://itjumpstart.wordpress.com/2014/11/21/eventemitter-in-go/
F# (at least in Visual Studio 2012) has both Control.Observable and Control.Event.
How are they related?
Which one should be used when?
Are there performance differences between the two?
I would also love to know what Haskell modules / packages / features the .NET IEnumerable / IObservable duality achieved with reactive extensions to .NET correspond to.
To answer the first part of your question, there is a number of differences between IEvent and IObservable. The reason why there are two similar types is that IEvent has been designed for F# (earlier and it is left there mostly for compatibility reasons) and the IObservable type was added later on to the .NET (and so F# added support for it too). Here are some differences:
IEvent does not support removing of event handlers, so when you create a processing pipeline (combining map, filter and others) and then call RemoveHandler on the resulting event, it leaves some handlers attached (yes, that's a leak and we wrote a more detailed paper about it)
On the other hand IObservable is able to remove handlers.
As a result of the previous point, IObservable behaves differently with respect to stateful combinators. For example, when you use Event.scan, you can attach multiple handlers to the resulting event and they will see the same state. IObservable creates a "new state" for every attached handler (unless you use subject explicitly).
In practical F# programming, this means:
You should generally prefer IObservable if you want to be able to remove event handlers (using RemoveHandler or when using AwaitObservable in F# async workflows).
If you want to declare events (usable from C#) then you need to create properties of type IEvent and so you need to use Event combinators.
As mentioned in comments, the F# model is heavily influenced by functional reactive programming (FRP) which is an idea that was first developed in Haskell, so you should find a plenty of similar libraries. The F# version is "less pure" in order to be more practical for .NET programming.
I have a problem where some objects change their properties based on some internal logic. For the sake of simplicity, let's imagine an object RandomSource that has a public Int field named Value. The object has its own thread and, sometimes, it updates the Value field.
Now, other objects in the system are interested in being notified that the Value was updated. In C#, I could define a companion delegate that objects subscribe, and which it is raised when the property is updated.
My question is thus the following: how do I do this in Scala? Which is the most "idiomatic" solution?
Observers are still pretty standard, also known as Publisher/Listener. You can roll your own, or you could use some actor-based solution if you want async notifications.
On the functional side, this stuff is more often done through functional reactive frameworks. There's Naftoli's Reactive, but Akka also provides a Dataflow, which amounts to very much the basic concepts.
Beyond these, this is a field of research on the evolution of Scala, so you can be sure you'll see more of them.
What is LINQ to events a.k.a RX Framework aka the Reactive Extensions in .NET 4.0 (but also available as backported versions)?
In other words, what is all the stuff in System.Reactive.dll for?
.NET Rx team (this is not an official name) found that any push sequence (events, callbacks) can be viewed as a pull sequence (as we normally do while accessing enumerables) as well – or they are Dual in nature. In short observer/observable pattern is the dual of enumeration pattern.
So what is cool about about this duality?
Anything you do with Pull sequences (read declarative style coding) is applicable to push sequences as well. Here are few aspects.
You can create Observables from existing events and then use them as first class citizens in .NET – i.e, you may create an observable from an event, and expose the same as a property.
As IObservable is the mathematical dual of IEnumerable, .NET Rx facilitates LINQ over push sequences like Events, much like LINQ over IEnumerables
It gives greater freedom to compose new events – you can create specific events out of general events.
.NET Rx introduces two interfaces, IObservable and IObserver that "provides an alternative to using input and output adapters as the producer and consumer of event sources and sinks" and this will soon become the de-facto for writing asynchronous code in a declarative manner. Here is a quick example.
//Create an observable for MouseLeftButtonDown
var mouseLeftDown=Observable.FromEvent<MouseButtonEventArgs>
(mycontrol,"MouseLeftButtonDown");
//Query the above observable just to select the points
var points = from ev in mouseEvents
select ev.EventArgs.GetPosition(this);
//Show points in the window's title, when ever user
//presses the left button of the mouse
points.Subscribe(p => this.Title = "Location ="
+ p.X + "," + p.Y);
You may go through these posts as well to get the head and tail in detail. Also have a look at the relates source code as well.
Part I - System.Reactive or the .NET Reactive Extensions (Rx) – Concepts and First Look
Part II - LINQ To Events - More on .NET Reactive Extensions (Rx)
Part III - LINQ To Events - Generating GetEventName() Wrapper Methods using T4 Text Templates
Check out this set of articles
You can read more about it here:
http://themechanicalbride.blogspot.com/2009/07/introducing-rx-linq-to-events.html
My first exposure was on this blog, as I am reading his book on F#:
http://tomasp.net/articles/reactive-ii-csevents.aspx
Basically my understanding of it is that when you write an event handler for mouse movements, for example, you may want to continuously capture these events, and perhaps be able to count how many movements or clicks, but, basically, you want to use that information in more ways than just as an event handler. So, just treat the events as a continuous stream.
Also, check out this Channel 9 video: Expert to Expert: Brian Beckman and Erik Meijer - Inside the .NET Reactive Framework (Rx)
From the show description:
The .NET Reactive Framework (Rx) is the mathematical dual of LINQ to
Objects. It consists of a pair of
interfaces IObserver/IObservable that
represent push-based, or observable,
collections, plus a library of
extension methods that implement the
LINQ Standard Query Operators and
other useful stream transformation
functions.
I think the essence of the RX is changing the angle with which we look at information. Naturally, every piece of information is a result of some process, which takes place in the time-space continuum. Traditional approach works with the projection of the process to the "current" time plane, losing the details of the time dimension as the result.
RX works with the projection to some other dimension, capturing the time component as well, so it is no wonder LINQ to RX is a superset of the regular LINQ.
The new IObservable/IObserver frameworks in the System.Reactive library coming in .NET 4.0 are very exciting (see this and this link).
It may be too early to speculate, but will there also be a (for lack of a better term) IQueryable-like framework built for these new interfaces as well?
One particular use case would be to assist in pre-processing events at the source, rather than in the chain of the receiving calls. For example, if you have a very 'chatty' event interface, using the Subscribe().Where(...) will receive all events through the pipeline and the client does the filtering.
What I am wondering is if there will be something akin to IQueryableObservable, whereby these LINQ methods will be 'compiled' into some 'smart' Subscribe implementation in a source. I can imagine certain network server architectures that could use such a framework. Or how about an add-on to SQL Server (or any RDBMS for that matter) that would allow .NET code to receive new data notifications (triggers in code) and would need those notifications filtered server-side.
Well, you got it in the latest release of Rx, in the form of an interface called IQbservable (pronounced as IQueryableObservable). Stay tuned for a Channel 9 video on the subject, coming up early next week.
To situate this feature a bit, one should realize there are conceptually three orthogonal axes to the Rx/Ix puzzle:
What the data model is you're targeting. Here we find pull-based versus push-based models. Their relationship is based on duality. Transformations exist between those worlds (e.g. ToEnumerable).
Where you execute operations that drive your queries (sensu lato). Certain operators need concurrency. This is where scheduling and the IScheduler interface come in. Operators exist to hop between concurrency domains (e.g. ObserveOn).
How a query expression needs to execute. Either verbatim (IL) or translatable (expression trees). Their relationship is based on homoiconicity. Conversions exist between both representations (e.g. AsQueryable).
All the IQbservable interface (which is the dual to IQueryable and the expression tree representation of an IObservable query) enables is the last point. Sometimes people confuse the act of query translation (the "how" to run) with remoting aspects (the "where" to run). While typically you do translate queries into some target language (such as WQL, PowerShell, DSQLs for cloud notification services, etc.) and remote them into some target system, both concerns can be decoupled. For example, you could use the expression tree representation to do local query optimization.
With regards to possible security concerns, this is no different from the IQueryable capabilities. Typically one will only remote the expression language and not any "truly side-effecting" operators (whatever that means for languages other than fundamentalist functional ones). In particular, the Subscribe and Run operations stay local and take you out of the queryable monad (therefore triggering translation, just as GetEnumerator does in the world of IQueryable). How you'd remote the act of subscribing is something I'll leave to the imagination of the reader.
Start playing with the latest bits today and let us know what you think. Also stay tuned for the upcoming Channel 9 video on this new feature, including a discussion of some of its design philosophy.
While this sounds like an interesting possibility, I would have several reservations about implementing this.
1) Just as you can't serialize non-trivial lambda expressions used by IQueryable, serializing these for Rx would be similarly difficult. You would likely want to be able to serialize multi-line and statement lambdas as part of this framework. To do that, you would likely need to implement something like Erik Meijer's other pet projects - Dryad and Volta.
2) Even if you could serialize these lambda expressions, I would be concerned about the possibility of running arbitrary code on the server sent from the client. This could easily pose a security concern far greater than cross-site scripting. I doubt that the potential benefit of allowing the client to send expressions to the server to execute outweighs the security vulnerability implications.
8 (now 10) years into the future: I stumbled over Qactive (former Rxx), a Rx.Net based queryable reactive tcp server provider
It is the answer to the "question in question"
Server
Observable
.Interval(TimeSpan.FromSeconds(1))
.ServeQbservableTcp(new IPEndPoint(IPAddress.Loopback, 3205));
Client
var datasourceAddress = new IPEndPoint(IPAddress.Loopback, 3205);
var datasource = new TcpQbservableClient<long>(datasourceAddress);
(
from value in datasource.Query()
//The code below is actually executed on the server
where value <= 5 || value >= 8
select value
)
.Subscribe(Console.WriteLine);
What´s mind blowing about this is that clients can say what and how frequently they want the data they receive and the server can still limit and control when, how frequent and how much data it returns.
For more info on this https://github.com/RxDave/Qactive
Another blog.sample
https://sachabarbs.wordpress.com/2016/12/23/rx-over-the-wire/
One problem I would love to see solved with the Reactive Framework, if it's possible, is enabling emission and subcription to change notifications for cached data from Web services and other pull-only services.
It appears, based on a new channel9 interview, that there will be LINQ support for IObserver/IObservable in the BCL of .NET 4.
However it will essentially be LINQ-to-Objects style queries, so at this stage, it doesn't look like a 'smart subscribe' as you put it. That's as far as the basic implementations go in .NET 4. (From my understanding from the above interview)
Having said that, the Reactive framework (Rx) may have more detailed implementations of IObserver/IObservable, or you may be able to write your own passing in Expression<Func...> for the Subscribe paramaters and then using the Expression Tree of the Func to subscribe in a smarter way that suits the event channel you are subscribing to.