I have an high transaction application which uses LightStreamer to stream data. It does this over HTTP.
I am not sure how to do performance testing of this (Strategy). Can someone please help me on this?
Googling yield some result but they are not in detail of approach and mainly gives info about one tool.
Looking at the Lightstreamer website this seems more of a data push technology, using websockets and other stack components, rather than a pure "streaming" technology for live video or audio. Am I missing something?
http://en.wikipedia.org/wiki/Lightstreamer
Related
I have been given the task to see if it's feasible to convert an Ajax-based application to Websockets. Fundamentally, I'm clear on how to go about doing this. My question, is regarding best practices regarding scaling the codebase.
The application in question has about 80 different Ajax requests. Some of these are clearly user account related, some are related to the operation of the actual application and yet others are of a global nature (statistics, etc) and finally we have a chat service.
I'm wondering if there are any best practices on how to implement something like this. It's entirely possible to do all of this through the same WS channel, but I'm considering an approach where each functional group connects to a different WS channel/topic. I think this might help keep the codebase clean and separate the different functional areas, but on the other hand I have no idea about the performance implications of this.
I've looked around for tutorials on this subject but it seems that all Websocket tutorials are of a rather basic nature and don't focus on the issues of large scale websocket development from a point of keeping the codebase clean vs performance issues.
Thanks for any insights.
I'm working on the design of a web app which will be using AJAX to communicate with a server on an embedded device. But for one feature, the client will need to get very frequent updates (>10 per second), as close to real time as possible, for an extended period of time. Meanwhile typical AJAX requests will need to be handled from time to time.
Some considerations unique to this project:
This data will be very small, probably no more than a single numeric value.
There will only be 1 client connected to the server at a time, so scaling is not an issue.
The client and server will reside on the same local network, so the connection will be fast and reliable.
The app will be designed for Android devices, so we can take advantage of any platform-specific browser features.
The backend will most likely be implemented in Python using WSGI on Apache or lighttpd, but that is still open for discussion.
I'm looking into Comet techniques including XHL long polling and hidden iframe but I'm pretty new to web development and I don't know what kind of performance we can expect. The server shouldn't have any problem preparing the data, it's just a matter of pushing it out to the client as quickly as possible. Is 10 updates per second an unreasonable expectation for any of the Comet techniques, or even regular AJAX polling? Or is there another method you would suggest?
I realize this is ultimately going to take some prototyping, but if someone can give me a ball-park estimate or better yet specific technologies (client and server side) that would provide the best performance in this case, that would be a great help.
You may want to consider WebSockets. That way you wouldn't have to poll, you would receive data directly from your server. I'm not sure what server implementations are available at this point since it's still a pretty new technology, but I found a blog post about a library for WebSockets on Android:
http://anismiles.wordpress.com/2011/02/03/websocket-support-in-android%E2%80%99s-phonegap-apps/
For a Python back end, you might want to look into Twisted. I would also recommend the WebSocket approach, but failing that, and since you seem to be focused on a browser client, I would default to HTTP Streaming rather than polling or long-polls. This jQuery Plugin implements an http streaming Ajax client and claims specifically to support Twisted.
I am not sure if this would be helpful at all but you may want to try Comet style ajax
http://ajaxian.com/archives/comet-a-new-approach-to-ajax-applications
I bet WebSockets are much faster.
So if I want to use lots of ajax continuously, WebSockets are recommended as an alternative?
It's two different technology things.
With AJAX client just send request to server and wait to response.
Websocket is HTML5 implementation for Comet technology, the idea is push information from server to client.
Ajax is slower because the overhead.
But Ajax is more compatible, Websocket is currently a bit experimental and it is not widely supported by most browser.
I think is not a issue of speed, is a issue of comunication and real time process
If you're in regular need of asynchronous data transmission I'd say you should try out websockets, it's really easy to use. One problem is - because of the early stage - the websocket interface might change (which has happended already) and render your web application unusable until you update your code.
Definitely WebSockets!. During my last internship, we used WebSockets together with Erlang to build a chat app for feature phones. Not only was it resource-efficient, it was also faster,
and really realtime - connections are always open, until either end closes.
And oh, they are experimental but works well on Chrome, Firefox, Android and iOS.
The only problem was with parsing data with BB which was because of the library we were using, so we rewrote the library's data parsing algorithm and it worked.
We tested on only these platforms.
You can checkout Socket.IO which improves upon the raw implementation of WebSockets.
Visit this link for some quick demos: http://socket.io/#how-to-use
Can anyone comment on the performance implications of storing streaming media in a SharePoint 2007 document library? I’ve heard this can be detrimental to the performance of the farm due to the media being streamed from storage in a SQL DB.
Has anyone had any firsthand experience with this and if so, what alternatives have you used to provide users with the ability to publish and mange their own video content? Assume a secure internal environment so external services like YouTube are not viable in this scenario.
I have tried this on a test deploy and it had very poor performance. Not only did the SharePoint server struggle, but the video the client was trying to stream was very laggy. Granted, we did not have a state of the art server set up, but I was the only one accessing the server and it couldn't even handle that. Given my experience, I would advise against it.
I can stream FLV's for flash movies from a document library with reasonable performance. I still opted for deploying them to a separate non-sharepoint website because the video's where fairly static and do take up a lot of SQL space.
You might consider activating blobcaching to get around the streaming from the database, see: http://msdn.microsoft.com/en-us/library/aa604896.aspx
I want to learn more on how to build CEP based applications. So I looked around and found several products (overview found here: http://rulecore.com/CEPblog/?page_id=47).
But as there are quite a few at the moment, I don't know which is the best to start with. And overall I just would consider the one available for free. The rest is a bit to expensive for just private use ;)
Esper is for free, but without Esper studio it seems quite tedious to develop a cep app. Streambase offers a free trial, but I couldn't find out how long you can use this (if only for a month, no that helpful for longer research). Oracle CEP suite seems quite complete, but in the cep scene - as far as I can see - it is the least recognized compared to Esper or Streambase.
So do you have any hints on what is the best way to start with cep development? Is it worth to spent time on working through the oracle documenation or is it better to start with Esper or Streambase?
Cheers,
Andreas
Microsoft's CEP offering StreamInsight which closely resembles the reactive programming model of the Rx Framework and LINQ.
A Hitchhiker's Guide to StreamInsight Queries is a good place to start.
Some Code Examples
I would recommend using LINQPad which can connect to Stream Insight as a canvas for your queries.
The current CEP tools do not solve identical problems! So depending on what you like to do you'd like use different tools. In short, my personal choices would be:
For building data driven algorithms, coding in a type of SQL with extensions - The Coral8 engine from Aleri. Free for test and development (Was anyway before bought by Aleri)
For detecting event patterns (situations), no coding (declarative style) but configuration using XML - RuleCore, free test subscription to (Web)service
For a mix of both with low level control and coding in Java - Esper, GPL.
For creating data driven computation logic using graphical boxes-and-arrows style of GUI: StreamBase.
I think the best choice is to compare the solutions that are freely available and then make something with them.
I'm not sure what your end goals are, if it's to learn a technology that you use at work or just to play around with something cool, but for me on a project like this, the deciding factor would be which tool can I use to make something I could share with the world.
In this case, my options would probably be Esper or OpenESB. That way, I could put the project on a resume (especially if I was applying for a job that used CEP tools) and share it with the world.
You could read the blog of Curt Monash (http://www.dbms2.com) , he writes about things like CEP.
would there be any interest in a free subscription to the ruleCore (Cloud, SaaS or whatever these are called today) Service? It would be running on smaller and less reliable (no cluster) hardware and probably only usable for testing out small low performance kind of things. If support#rulecore.com gets a couple of requests of this kind I'm sure it's put up onto the todo list...
For detecting event patterns I found that rulecore is pretty easy to use. I have only tried to detect patterns of low and medium complexity and that did work fine. It takes some time to get used to the concepts but is it actually a very small system so it was not that bad. And you need to like XML as everything is done using XML.
If you are trying to create a trading application then StreamBase would be better. But for monitoring stuff rulecore feels better.
If you have continuous streams (market feeds, IoT sensors, Twitter, news, etc), then stream processing technology is the right choice for you. Stream processing / streaming analytics is only a part of different CEP solutions (streams, rules, patterns, etc.).
There are several open source options for stream processing in the meantime, e.g. Apache Storm, Apache Spark or Apache Samza, but also powerful proprietary products such as IBM InfoSphere Streams, TIBCO StreamBase or Software AG's Apama.
Take a look at my blog post respectively article for more details about different stream processing and streaming analytics solutions (open source and proprietary):
Comparison of Stream Processing and Streaming Analytics Alternatives (Apache Storm, Spark, IBM InfoSphere Streams, TIBCO StreamBase, Software AG Apama)
i would start with the free trial of Aleri Coral8 (currently Sybase)