I am redesigning a web application which previously has been rendered server side to a Single Page Application and started to read about websockets . The web application will be using sockets to have new records and/or messages pushed to the client. I have been wondering why most pages which make use of sockets don't handle all their communication over the socket. Most of the times there is RESTful backend in addition to the websocket. Would it be a bad idea to have the client query for new resources over the socket? If so why - other than that a RESTful api might be easier to use with other devices?
I can imagine that using websockets would probably not be the best idea in case the network connection is kind of bad like on mobile devices, but that probably should work quite well with a reasonable connection to the web.
I found this related question, however it is from 2011 and seems a little outdated:
websocket api to replace rest api?
No, it won´t be a bad idea. Actually I work in an application that uses a WebSocket connection for all what is data interaction, the web server only handles requests for resources, views under different languages, dimensions .. etc..
The problem may be the lack of frameworks/tools based on a persistent connection. For many years most of frameworks, front and back end, have been designed and built around the request/response model. The approach shift may be no so easy to accept.
Coming back to this question a few years later, I would like to point out a few aspects to illustrate that having all your communication through websockets does have its drawbacks:
there is no common support for compression. You can easily configure your webserver to compress http requests and browsers have been known to happily accept compressed responses for years, however for web sockets it is still not that easy (even though the situation has improved)
client frameworks often are build upon commonly used standards like rest. The further away you move from frameworks expectations, the less addons or features will be available.
caching in the browser is not as easy. By now this goes a long way, reaching into the realm of offline availability and PWAs.
when using technology, that is only used by a subset of users, it is more likely to find new bugs, or bugs might take longer to fix. And if it's not bugs, there might be an edge case somewhere around the corner. This isn't an issue per se - but something to be aware of. If one runs into those things, they often easily take up quite some time to fix or work around.
We are going to develop rest server for our application (and all logic is on client javascript).
So we thought to use Phalcon php, but we also need to create realtime chat system, which is much more easy to do using node.js. This made us think about using node.js instead of phalcon
Unfortunatly, we are not good expirienced in node.js, we love phalcon for its performance and internal beauty.
The quiestion is, did anybody compare phalcon and node.js performance? May be it's better to use node.js only for long polling chat requests, but i dont like when project is connected with so different tools.
You are trying to compare two different things IMO.
node.js has a lot of power and flexibility but so does Phalcon. If you want to create a chat application with Phalcon, then you will need to implement some sort of polling mechanism in your browser that would refresh the chat window every X seconds. Getting/Inserting the data from the database will be Phalcon's job. Javascript will be used to do the polling i.e. refresh the chat page every X seconds.
The problem with this approach is that you might be hitting your web server every X seconds from every client that has the chat application open - and thus refreshing the chat contents, even when there are no messages. This can become very intensive very quickly.
node.js has the ability to send messages to the subscribed clients instantly. Web sockets can do the same thing I believe.
Check this video out, which will give you an idea of how this can be achieved easily:
https://www.youtube.com/watch?v=lW1vsKMUaKg
The idea is to use technologies that will not burden your hardware, rather collaborate with it. Having a "subscription" notification system (such as sockets or node.js) reduces the load on your application since only the subscribed clients receive the new messages and no full refresh is needed from the chat clients.
Phalcon is great for the back end with its speed and it can be used to construct the message which in turn will be passed to the transport layer and sent to the client. Depending on how you want to implement this, there are plenty of options around and you can easily mix and match technologies :)
as #Nikolaos Dimopoulos said, you're trying to compare two different things.
But here is my advice, while you're experienced with PhalconPHP framework, and you want to benefit from Phalcon speed and performance, you can implement the web app in Phalcon FW, and the chatting system in Node.JS as a service.
If your web application "The Phalcon app" needs to push messages from the backend, you can use http://elephant.io/ library for that, I have done this before with Yii framework and Node, and it's working perfectly.
My advice is to use what you already know, experimenting with nodejs just for the chat application.
Mainly because you said you do not have experience with it, so, because the chat app is something a lot of people made you'll find plenty of examples.
By doing so you will learn a lot from node and might even think about migrating from Phalcon if it suits your needs, using the features offered by expressjs for example.
I would not choose one over the other based on performance.
Hi I am trying to compare the performance of zk and gwt.
In my comparision I cant write any javascript by myself if the framework itself converts some code into js its ok(like gwt) but I cant write js by myself
On writing code in the above way for almost anything done on browser a request is sent to the server in ZK.
Hence eventually if you compare the no of request sent by zk to server is too high as compared to gwt.
i would like to ask the following.
whose performance is better zk or gwt while ignoring the above.
if we dont ignore the above then is my conclusion that gwts performance is better than zk right ?
I know that there might be other parameters when comparing performance... but if the difference between requests are so high i cant really see zk beating gwt which some people have said on some forums
pls help
thanks
GWT and ZK are very different architecturally. GWT is client-centric so it sends less requests to the server. With GWT you can basically control everything by yourself. This on the otherhand means, the developer is responsible for handling server requests and asynchornouse data transfer between server and client. If you are an experienced web developer then the result could be great. If you are not then it could be pretty challenging and complex as there are some network and security issues you need to take care.
On the other hand, ZK is server-centric, it handles these server-client async communication for you. With ZK developers can focus more on the business logic without taking care of client-server issues. As the framework takes care of client-server talk, there are more requests by default. If you wish to minimize the request, you can probably follow their developer's guide to do some tuning.
IMO ZK and GWT are both matured framework. You can probably think about your project requirements and your experiences in Web development then choose the solution.
The previous posts are basiclly, but there are some features in zk that,I think, must be talked about.
Remember, every request is a zk-event, cos zk is completely event-based.
Client-Side Event-Handling
Zk allows you to implement client-side event-handling just as well as server-side,
which includes preventing zk from firing an event to the server.
You can simple manipulate the client objects in js as you can do it in Java on the
server-side or just do some post processing and and then...
Use Client-Side Event-Firing
Zk got that usefull js method zAu.send(zk.Event,int) that let's you send custom
or standart events to the server.
Deferrable Event-Listeners
Deferrable-events are queued at client-side, so you can minimize the number of
requests.
Stubonly
Stubonly is a way to make a component client-side only. But by now this is only
available for zk EE.
So, you can see, if you like, you can limit the requests to maybe the same amount of requests of GWT.
If it make sense or not depandce on you/your applycation. But I think you can say that zk is more powerfull than GWT.
You maybe interested in iZUML and defer rendering too.
ZK is server side focused, since GWT is all in the browser, so ZK needs much more ajax hits to the server to get js fragments, screen portions, etc. whereas GWT just hit the server at the beginning to download the js app (which will be cached in the client for ever) and whenever the app needs some data from the backend.
Said that, IMHO GWT should perform better since you dont need any sort of view logic being executed in the server (lighter servers), and there are less requests in the wire.
GWT and ZK are kind of different.
I've done some ZK projects and also some GWT projects. I didn't have performance issues in either cases,but I feel I need to write more code when I use GWT.
My experience from real world projects with two teams, one writing using the zk framework and the second using GWT. Identical UI projects having data served from the same data service layer. The zk team completed the project in one third of the time the GWT team consumed. Both projects deployed in production. Users were happy using either app. Finally we kept the zk version in production. Why? Simply because the maintenance of the zk app was so much faster and simple compared to GWT! Sorry GWT... Nice technology backed up by a giant but we will keep going with zk.
I am fairly new to programming with the Play framework as well as Akka, although I've been reading about them for a while. I am now starting a proof-of-concept application on the default/basic Play environment. My question stems from the web service client api in Play (http://www.playframework.org/documentation/2.0.1/ScalaWS).
This application basically needs to mediate calls to a remote SOAP web service in as scalable and performant a way as possible. Browser makes ajax calls in JSON, Play app needs to transform them to SOAP/XML and vice versa on the response.
If I used the play web service client directly through the controller, these calls can be asynchronous, which is way better than what we do now (blocking). However, I'm not clear on how this exactly this would behave under heavy load. Will the concurrency/thread-management be largely left to the underlying Netty server? Do I have any way to tune it?
An alternative would be to use an Akka actor system from the controllers, where I can control the routing policy, pool size, fault-tolerance etc. If I take this approach, would it still make sense to use Play's async WS client? If so, would this approach ( of composing Futures?) be the recommended pattern?
Another factor that seems to make the Akka approach more attractive is that this application would eventually have several other responsibilities, so we could control/tune the resources allowed to this ActorSystem and reduce risk of the entire app getting dragged down by the SOAP service.
The two options you are detailing would work :
Use the play API for WS to handle requests/responses asynchronously
Use Akka to do the same thing and manage your WS call synchronously in your actor
First, there is no right or wrong solution.
The Play! WS API solution seams the easiest to implement and test. Many people in the community rely on it (I do).
On the other hand, even if the Akka solution seams heavier (not that much) to set up, it brings you more flexibility in the future. You can simply use Async play! blocks and work with promises for async computation. There is also implicit conversions between play promises and akka future. Finally, to monitor your actors you can have a look at Typesafe console.
If the big thing is performances, premature optimisation often leads to more (and unnecessary) complexity. As far as I am concern, I would begin with the API WS and if required in the future move to the Akka solution.
I thought about getting rid of all client-side Ajax calls (jQuery) and instead use a permanent socket connection (Socket.IO).
Therefore I would use event listeners/emitters client-side and server-side.
Ex. a click event is triggered by user in the browser, client-side emitter pushes the event through socket connection to server. Server-side listener reacts on incoming event, and pushes "done" event back to client. Client's listener reacts on incoming event by fading in DIV element.
Does that make sense at all?
Pros & cons?
There is a lot of common misinformation in this thread that is very inaccurate.
TL/DR;
WebSocket replaces HTTP for applications! It was designed by Google with the help of Microsoft and many other leading companies. All browsers support it. There are no cons.
SocketIO is built on top of the WebSocket protocol (RFC 6455). It was designed to replace AJAX entirely. It does not have scalability issues what-so-ever. It works faster than AJAX while consuming an order of magnitude fewer resources.
AJAX is 10 years old and is built on top of a single JavaScript XMLHTTPRequest function that was added to allow callbacks to servers without reloading the entire page.
In other words, AJAX is a document protocol (HTTP) with a single JavaScript function.
In contrast, WebSocket is a application protocol that was designed to replace HTTP entirely. When you upgrade an HTTP connection (by requesting WebSocket protocol), you enable two-way full duplex communication with the server and no protocol handshaking is involved what so ever. With AJAX, you either must enable keep-alive (which is the same as SocketIO, only older protocol) or, force new HTTP handshakes, which bog down the server, every time you make an AJAX request.
A SocketIO server running on top of Node can handle 100,000 concurrent connections in keep-alive mode using only 4gb of ram and a single CPU, and this limit is caused by the V8 garbage collection engine, not the protocol. You will never, ever achieve this with AJAX, even in your wildest dreams.
Why SocketIO so much faster and consumes so much fewer resources
The main reasons for this is again, WebSocket was designed for applications, and AJAX is a work-around to enable applications on top of a document protocol.
If you dive into the HTTP protocol, and use MVC frameworks, you'll see a single AJAX request will actually transmit 700-900 bytes of protocol load just to AJAX to a URL (without any of your own payload). In striking contrast, WebSocket uses about 10 bytes, or about 70x less data to talk with the server.
Since SocketIO maintains an open connection, there's no handshake, and server response time is limited to round-trip or ping time to the server itself.
There is misinformation that a socket connection is a port connection; it is not. A socket connection is just an entry in a table. Very few resources are consumed, and a single server can provide 1,000,000+ WebSocket connections. An AWS XXL server can and does host 1,000,000+ SocketIO connections.
An AJAX connection will gzip/deflate the entire HTTP headers, decode the headers, encode the headers, and spin up a HTTP server thread to process the request, again, because this is a document protocol; the server was designed to spit out documents a single time.
In contrast, WebSocket simply stores an entry in a table for a connection, approximately 40-80 bytes. That's literally it. No polling occurs, at all.
WebSocket was designed to scale.
As far as SocketIO being messy... This is not the case at all. AJAX is messy, you need promise/response.
With SocketIO, you simply have emitters and receivers; they don't even need to know about each-other; no promise system is needed:
To request a list of users you simply send the server a message...
socket.emit("giveMeTheUsers");
When the server is ready, it will send you back another message. Tada, you're done. So, to process a list of users you simply say what to do when you get a response you're looking for...
socket.on("HereAreTheUsers", showUsers(data) );
That's it. Where is the mess? Well, there is none :) Separation of concerns? Done for you. Locking the client so they know they have to wait? They don't have to wait :) You could get a new list of users whenever... The server could even play back any UI command this way... Clients can connect to each other without even using a server with WebRTC...
Chat system in SocketIO? 10 lines of code. Real-time video conferencing? 80 lines of code Yes... Luke... Join me. use the right protocol for the job... If you're writing an app... use an app protocol.
I think the problem and confusion here is coming from people that are used to using AJAX and thinking they need all the extra promise protocol on the client and a REST API on the back end... Well you don't. :) It's not needed anymore :)
yes, you read that right... a REST API is not needed anymore when you decide to switch to WebSocket. REST is actually outdated. if you write a desktop app, do you communicate with the dialog with REST? No :) That's pretty dumb.
SocketIO, utilizing WebSocket does the same thing for you... you can start to think of the client-side as simple the dialog for your app. You no longer need REST, at all.
In fact, if you try to use REST while using WebSocket, it's just as silly as using REST as the communication protocol for a desktop dialog... there is absolutely no point, at all.
What's that you say Timmy? What about other apps that want to use your app? You should give them access to REST? Timmy... WebSocket has been out for 4 years... Just have them connect to your app using WebSocket, and let them request the messages using that protocol... it will consume 50x fewer resources, be much faster, and 10x easier to develop... Why support the past when you're creating the future?
Sure, there are use cases for REST, but they are all for older and outdated systems... Most people just don't know it yet.
UPDATE:
A LOT of people have been asking me recently how can they start writing an app in 2018 (and now soon 2019) using WebSockets, that the barrier seems really high, that once they play with Socket.IO they don't know where else to turn or what to learn.
Fortunately the last 3 years have been very kind to WebSockets...
There are now 3 major frameworks that support BOTH REST and WebSocket, and even IoT protocols or other minimal/speedy protocols like ZeroMQ, and you don't have to worry about any of it; you just get support for it out of the box.
Note: Although Meteor is by far the most popular, I am leaving it out because although they are a very, very well-funded WebSocket framework, anyone who has coded with Meteor for a few years will tell you, it's an internal mess and a nightmare to scale. Sort of like WordPress is to PHP, it is there, it is popular, but it is not very well made. It's not well-thought out, and it will soon die. Sorry Meteor folks, but check out these 3 other projects compared to Meteor, and you will throw Meteor away the same day :)
With all of the below frameworks, you write your service once, and you get both REST and WebSocket support. What's more, it's a single line of config code to swap between almost any backend database.
Feathers Easiest to use, works the same on the front and backend, and supports most features, Feathers is a collection of light-weight wrappers for existing tools like express. Using awesome tools like feathers-vuex, you can create immutable services that are fully mockable, support REST, WebSocket and other protocols (using Primus), and get free full CRUD operations, including search and pagination, without a single line of code (just some config). Also works really great with generated data like json-schema-faker so you can not only fully mock things, you can mock it with random yet valid data. You can wire up an app to support type-ahead search, create, delete and edit, with no code (just config). As some of you may know, proper code-through-config is the biggest barrier to self-modifying code. Feathers does it right, and will push you towards the front of the pack in the future of app design.
Moleculer Moleculer is unfortunately an order of magnitude better at the backend than Feathers. While feathers will work, and let you scale to infinity, feathers simply doesn't even begin to think about things like production clustering, live server consoles, fault tolerance, piping logs out of the box, or API Gateways (while I've built a production API gateway out of Feathers, Moleculer does it way, way better). Moleculer is also the fastest growing, both in popularity and new features, than any WebSocket framework.
The winning strike with Moleculer is you can use a Feathers or ActionHero front-end with a Moleculer backend, and although you lose some generators, you gain a lot of production quality.
Because of this I recommend learning Feathers on the front and backend, and once you make your first app, try switching your backend to Moleculer. Moleculer is harder to get started with, but only because it solves all the scaling problems for you, and this information can confuse newer users.
ActionHero Listed here as a viable alternative, but Feathers and Moleculer are better implementations. If anything about ActionHero doesn't Jive with you, don't use it; there are two better ways above that give you more, faster.
NOTE: API Gateways are the future, and all 3 of the above support them, but Moleculer literally gives you it out of the box. An API gateway lets you massage your client interaction, allowing caching, memoization, client-to-client messaging, blacklisting, registration, fault tolerance and all other scaling issues to be handled by a single platform component. Coupling your API Gateway with Kubernetes will let you scale to infinity with the least amount of problems possible. It is the best design method available for scalable apps.
Update for 2021:
The industry has evolved so much that you don't even need to pay attention to the protocol. GraphQL now uses WebSockets by default! Just look up how to use subscriptions, and you're done. The fastest way to handle it will occur for you.
If you use Vue, React or Angular, you're in luck, because there is a native GraphQL implementation for you! Just call your data from the server using a GraphQL subscription, and that data object will stay up to date and reactive on it's own.
GraphQL will even fall-back to REST for you when you need to use legacy systems, and subscriptions will still update using sockets. Everything is solved when you move to GraphQL.
Yes, if you thought "WTH?!?" when you heard you can simply subscribe, like with FireBase, to a server object, and it will update itself for you. Yes. That's now true. Just use a GraphQL subscription. It will use WebSockets.
Chat system? 1 line of code.
Real time video system? 1 line of code.
Video game with 10mb of open world data shared across 1m real-time users? 1 line of code. The code is just your GQL query now.
As long as you build or use the right back-end, all this realtime stuff is now done for you with GQL subscriptions. Make the switch as soon as you can and stop worrying about protocols.
Socket.IO uses persistent connection between client and server, so you will reach a maximum limit of concurrent connections depending on the resources you have on server side, while more Ajax async requests can be served with the same resources.
Socket.IO is mainly designed for realtime and bi-directional connections between client and server and in some applications there is no need to keep permanent connections. On the other hand Ajax async connections should pass the HTTP connection setup phase and send header data and all cookies with every request.
Socket.IO has been designed as a single process server and may have scalability issues depending server resources that you are bound to.
Socket.IO in not well suited for applications when you are better to cache results of client requests.
Socket.IO applications face with difficulties with SEO optimization and search engine indexing.
Socket.IO is not a standard and not equivalent to W3C Web Socket API, It uses current Web Socket API if browser supports, socket.io created by a person to resolve cross browser compatibility in real time apps and is so young, about 1 year old. Its learning curve, less developers and community resources compared with ajax/jquery, long term maintenance and less need or better options in future may be important for developer teams to make their code based on socket.io or not.
Sending one way messages and invoking callbacks to them can get very messy.
$.get('/api', sendData, returnFunction); is cleaner than
socket.emit('sendApi', sendData); socket.on('receiveApi', returnFunction);
Which is why dnode and nowjs were built on top of socket.io to make things manageable. Still event driven but without giving up callbacks.