I want to use Twisted to support my website. Sometimes documents should be editable, and sometimes must be only readable. I want to use the 'websocketsite' facilities to handle the situations when documents are editable, and a regular 'site' when the document are only readable. I don't see howto handle these two type of factories under the same reactor. Should I usage two ports and two reactors? Is there a way to handle the situation with just one reactor?
Here are 3 examples of mixing static Web, WSGI and WebSocket on top of Twisted:
Mixing Twisted Web and WebSocket
Mixing multiple WebSocket services with Twisted Web
Mixing WSGI (Flask) with WebSocket
Disclosure: I am original author of Autobahn and work for Tavendo.
Related
Let's say I wanted to create a collaborative web application where users can simultaneously work on e.g. a diagram. I have my interactive web site (HTML/CSS/JavaScript) on the client side, and let's say a Java application synchronizing a centralized model on the server side. How would client and server exchange messages? To get a better idea of how the system is supposed to work like, here are some details and a small illustration:
Requirements
The underlaying technology should support authorized communication to show/hide sensitive information.
The model of the diagram should only exist on the server (centralized model, no copies), whereby changes should be broadcasted to other users (other browsers who are currently editing the same document).
After receiving changes to the model, the server shall perform computationally higher demanding tasks such as consistency checks and inform the user if e.g. an action is invalid.
Constraints
The system must not poll or set flags (it should be a direct communication).
The website must not be reloaded during use.
Software running on the client must be restricted to JavaScript.
Technology I've found
Here are some of the solutions I've looked at, ranked from (imho) most suitable to least, whereby I'm not sure if the list is even complete and all my assumptions are correct.
Java Web Service using javax.jws - The service would offer an API described by WSDL and be capable of responding with SOAP messages using the HTTP protocol, but can JavaScript process these messages?
Servlets & Java Server Pages (JSPs) - Would, as far as I know, require a page reload to fetch new content (isn't this pretty much the same as PHP? Both are executed on the server to generate HTML pages..)
I've already found this question which, however, does not provide concrete technologies in the answer. Thank you for any hints and clarification, I dearly appreciate it!
As you have already found out, there are many approaches to this common problem.
SOAP (sometimes called "Web Services" or "WS*") is a standard that was developed for application to application communication. It's strengths are precise specifications (and therefore many good libraries), interface sharing for client and server (through WSDL), and discoverability.
However, SOAP has very limited -if any- support in Browsers.
The most common modern approach is probably RESTful web services based on XML or JSON. When used in a browser as client, JSON is preferred, as the unmarshalling is trivial in JavaScript. Many frameworks and libraries like Angular and jQuery take on the tedious cross-browser AJAX request crafting. Some boilerplate is required to wrap the data in HTML, however. This downside can also be an upside if you consider the cleaner architecture and maintainability.
Since you're using Java on your server, you might also want to look into JSFs AJAX capabilities. It doesn't really use REST (no well-defined resources), but loads server-side rendered chunks of your HTML page asynchronously (read: AJAX) and re-renders them in the DOM. This can be seen as a compromise between real REST and full page reloading. The good side of it is you don't have to define any resources but your backing beans. The downsides are: possible overhead, since you're downloading HTML instead of pure data; and bad testability since you cannot access the rendered chunks outside the JSF context.
Wholly different issues are 1. the synchronization of concurrent requests and 2. communication with clients from server. For the latter you can either look into Web Sockets (the next big thing) or the good-old polling.
If concurrent edits of the same data are a concern to you, you'll have to implement a sort of strategy to tell whether edits can be merged or whether one has to be declined. The client will need to handle such situations and offer the user reviewing, overriding or reverting etc.
EDIT: more on Java servlets and JSP
Servlets are nothing but Java classes that handle specific HTTP requests, usually bound to certain URL paths. They're comparable to PHP pages: In PHP the web server gets a request and searches for a corresponding PHP file. Then it parses the PHP script and returns the response. Servlets do the same, only the syntax is very different. JSP is mostly a try at bringing more PHP-ish syntax to Java. Behind the curtains, they're rendered to pure Java servlets.
Java servlets have a long history which is older than AJAX, but many frameworks have taken on the challenge of implementing flexible resource handling on the server. Most notably Jersey and RESTEasy, which are implementations of the Java EE standard JAX-RS; and Spring, which is a request-based framework that competes well against Java EE standards. Then there is also JSF, which is sometimes called a successor of JSP. To the user it's little different than JSP, but the syntax is nicer and IMHO it is more feature-rich than JSP. Especially considering extensions like PrimeFaces. In contrast to JSP, JSF embraces AJAX and many built-in components are capable of communicating with the server using AJAX, including calling Java functions and re-rendering parts of the page.
I would recommend you try websockets since connections will be persistent between clients and server and any of them can send a message at any time.
You could implement a websocket server in Java on the server and use Javascript's built in websockets functionallity on the clients.
Take a look at https://www.websocket.org/ to learn more about websockets.
Our requirements for a real-time web framework include:
lightweight framework
scala support on server side
flexible on communication mechanism : may be Ajax, Server Sent Event or WebSocket.
relatively little changes required to client html.
E.g. using the WebSockets js library is fine
introducing significant compile time/server side page processing is not. E.g. Play routing annotations are not acceptable
must have working examples for both:
web clients
server to server communications
fully functional build. Preferably sbt, but maven maybe acceptable
I have evaluated the following frameworks: and each one of them has one or more drawbacks that make usage within our application less than desirable.
Play: somewhat heavy, but more importantly it introduces custom annotations/processing into the html page. We need VANILLA html pages.
Spray: closer to the mark. But although I found a number of example applications, the actor-based communication is not working in those examples. The SimpleServer example has a built-in "cases" counter (from SimpleClient) that do not work as given: they could certainly be made to work .. eventually..
atmosphere: lacking examples
jetty, netty: lacked fully functional examples buildable within sbt or maven
socko : The markdown essentially stipulates using eclipse/scala-IDE for running tests/doing development. That is a non-starter for us (IJ shop). It was unclear how to run examples and/or start their servers from sbt / command line.
I ended up writing a fair amount of custom code wrapped around Netty. After it is in better shape I may drop it on GitHub.
http://xitrum-framework.github.io/ is actively developed and contains SocksJs support. It is rather lightweight, you can directly annotate routes on actors and they become exposed on the web.
Socket.IO seems to be the most popular and active WebSocket emulation library. Juggernaut uses it to create a complete pub/sub system.
Faye is also popular and active, and has its own javascript library, making its complete functionality comparable to Juggernaut. Juggernaut uses node for its server, and Faye can use either node or rack. Juggernaut uses Redis for persistence (correction: it uses Redis for pub/sub), and Faye only keeps state in memory.
Is everything above accurate?
Faye says it implements Bayeux -- i think Juggernaut does not do this -- is that because Juggernaut is lower level (IE, I can implement Bayeux using Juggernaut)
Could Faye switch to using the Socket.IO browser javascript library if it wanted to? Or do their javascript libraries do fundamentally different things?
Are there any other architectural/design/philosophy differences between the projects?
Disclosure: I am the author of Faye.
Regarding Faye, everything you've said is true.
Faye implements most of Bayeux, the only thing missing right now is service channels, which I've yet to be convinced of the usefulness of. In particular Faye is designed to be compatible with the CometD reference implementation of Bayeux, which has a large bearing on the following.
Conceptually, yes: Faye could use Socket.IO. In practise, there are some barriers to this:
I've no idea what kind of server-side support Socket.IO requires, and the requirement that the Faye client (there are server-side clients in Node and Ruby, remember) be able to talk to any Bayeux server (and the Faye server to any Bayeux client) may be deal-breaker.
Bayeux has specific requirements that servers and clients support certain transport types, and says how to negotiate which one to use. It also specifies how they are used, for example how the Content-Type of an XHR request affects how its content is interpreted.
For some types of error handling I need direct access to the transport, for example resending messages when a client reconnects after a Node WebSocket dies.
Please correct me if I've got any of this wrong - this is based on a cursory scan of the Socket.IO documentation.
Faye is just pub/sub, it's just based on a slightly more complex protocol and has a lot of niceties built in:
Server- and client-side extensions
Wildcard pattern-matching on channel routes
Automatic reconnection, e.g. when WebSockets die or the server goes offline
The client works in all browsers, on phones, and server-side on Node and Ruby
Faye probably looks a lot more complex compared to Juggernaut because Juggernaut delegates more, e.g. it delegates transport negotiation to Socket.IO and message routing to Redis. These are both fine decisions, but my decision to use Bayeux means I have to do more work myself.
As for design philosophy, Faye's overriding goal is that it should work everywhere the Web is available and should be absolutely trivial to get going with. I'ts really simple to get started with but its extensibility means it can be customized in quite powerful ways, for example you can turn it into a server-to-client push service (i.e. stop arbitrary clients pushing to it) by adding authentication extensions.
There is also work underway to make it more flexible on the server side. I'm looking at adding clustering support, and making the core pub-sub engine pluggable so you could use Faye as a stateless web frontend for another pub-sub system like Redis or AMQP.
I hope this has been helpful.
AFAIK, yes, apart from the fact Juggernaut only uses Redis for Pubsub, not persistence. Also means client libraries in most languages have already been written (since it just needs a Redis adapter).
Juggernaut doesn't implement Bayeux, but rather has a very simple custom JSON protocol
I don't know, but probably
Juggernaut is very simple, and designed to be that way. Although I haven't used Faye, from the docs it looks like it has a lot more features than just PubSub. Being built on top of Socket.IO has it advantages too, Juggernaut's supported in practically every browser, both desktop and mobile.
I'll be really interested in what Faye's author has to say. As I say, I haven't used it and it would be great to know how it compares to Juggernaut. It's probably the case of using the best tool for the job. If it's pubsub you need, Juggernaut does that very well.
Faye certainly could.
Another example of a similar project on top of Socket.IO:
https://github.com/aaronblohowiak/Push-It
EDIT: I forgot to include the prime candidate for web applications: JSON over HTTP/REST + Comet. It combines the best features of the others (below)
Persevere basically bundles everything I need in a server
The focus for Java and such is definitely on Comet servers, but it can't be too hard to use/write a client.
I'm embarking on an application with a server holding data, and clients executing operations which would affect this data, and thus require some sort of notification across all interested/subscribed clients.
The first client will probably be written in WPF, but we'll probably need to add clients written in other languages, e.g. a Java (Swing?) client, and possibly, a web client.
The actual question(s): What protocol should I use to implement this? How easy would it be to integrate with JS, Java and .NET (precisely, C#) clients?
I could use several interfaces/protocols, but it'd be easier overall to use one that is interoperable. Given that interoperability is important, I have researched a few options:
JSON-RPC
lightweight
supports notifications
The only .NET lib I could find, Jayrock doesn't support notifications
works well with JS
also true of XML-based stuff (and possibly, even binary protocols) BUT this would probably be more efficient, thanks to native support
Protobuf/Thrift
IDL makes it easy to spit out model classes in each language
doesn't seem to support notifications
Thrift comes with RPC out of the box, but protobufs don't
not sure about JS
XML-RPC
simple enough, but doesn't support notifications
SOAP: I'm not even sure about this one; I haven't grokked this yet.
seems rather complex
Message Queues/PubSub approach: Not strictly a protocol, but might be fitting
I hardly know anything about them, and got lost amongst the buzzwords`-- JMS? **MQ?
Perhaps combined with some RPC mechanism above, although that might not be strictly necessary, and possibly, overkill.
Other options are, of course, welcome.
I am partial to the pub/sub design you've suggested. I'd take a look at ZeroMQ. It has bindings to C#, Java, and many other platforms.
Bindings list: http://www.zeromq.org/bindings:clr
I also found this conversation on the ZeroMQ dev listing that may answer some questions you have about multiple clients and ZeroMQ: http://lists.zeromq.org/pipermail/zeromq-dev/2010-February/002146.html
As XMPP was mentioned, SIP has a similar functionality. This might be more accessible for you.
We use Servoy for this. It does automatic data broadcasting to web-clients and java-clients. I'm not sure if broadcasts can be sent to other platforms, you might be able to find an answer to that on their forum.
If you want to easily publish events to clients across networks, you may wish to look at a the XMPP standard. (Used by, amongst other things, Jabber and Google Talk.)
See the extension for publish-subscribe functionality.
There are a number of libraries in different languages including C#, Java and Javascript.
You can use SOAP over HTTP to modify the data on the server and SOAP over SMTP to notify the subscribed clients.
OR
The server doesn't know anything about the subscription and the clients call the server by timeout to track updates they are interested in, using XML-RPC, SOAP (generated using WSDL), or simply HTTP GET if there is no need to pass back complex data on tracking.
Can XMPP be applied in a similar way to the reverse Ajax pattern? Can it be used to implement http long wait like Comet? Is there an example of using such a technique with XMPP?
Yes. Take a look at XMPP over BOSH. It's a long-lived non-polling technique. From the BOSH specification:
This specification defines a transport protocol that emulates the semantics of a long-lived, bidirectional TCP connection between two entities (such as a client and a server) by efficiently using multiple synchronous HTTP request/response pairs without requiring the use of frequent polling or chunked responses.
jbosh is an open source Java implementation. Tigase.org has a number of projects, including BOSH support, and appears to be mostly Java/GWT. JAXMPP is a Java library in the framework, although it's not clear what feature set it supports. I've not tried any of these myself.
Like Ire told you can use bosh for that. On the client-side(javascript) you could use strophe.js.
Prosody is an easy to install XMPP-server which supports BOSH.
To learn how to create programs using strophe.js you could read Professional XMPP programming with javascript and Jquery