Looking through 2 screen scraping frameworks and thought I'd ask here for any experience. I need to recommend a framework for screen scraping an old 7032 IBM mainframe system and I'm currently looking at Inventu's Flynet (impressive, but expensive) and Zephyr's Passport HOI (cheaper and less feature rich). The goal is to encapsulate the terminal app in a webservice (WCF) and let less than 50 concurrent user access the service. Any ideas or recommendations?
Thanks
If I remember well the Expect tool can be useful. Other resources are:
x3270 script
Jagacy 3270
Related
We are going to develop rest server for our application (and all logic is on client javascript).
So we thought to use Phalcon php, but we also need to create realtime chat system, which is much more easy to do using node.js. This made us think about using node.js instead of phalcon
Unfortunatly, we are not good expirienced in node.js, we love phalcon for its performance and internal beauty.
The quiestion is, did anybody compare phalcon and node.js performance? May be it's better to use node.js only for long polling chat requests, but i dont like when project is connected with so different tools.
You are trying to compare two different things IMO.
node.js has a lot of power and flexibility but so does Phalcon. If you want to create a chat application with Phalcon, then you will need to implement some sort of polling mechanism in your browser that would refresh the chat window every X seconds. Getting/Inserting the data from the database will be Phalcon's job. Javascript will be used to do the polling i.e. refresh the chat page every X seconds.
The problem with this approach is that you might be hitting your web server every X seconds from every client that has the chat application open - and thus refreshing the chat contents, even when there are no messages. This can become very intensive very quickly.
node.js has the ability to send messages to the subscribed clients instantly. Web sockets can do the same thing I believe.
Check this video out, which will give you an idea of how this can be achieved easily:
https://www.youtube.com/watch?v=lW1vsKMUaKg
The idea is to use technologies that will not burden your hardware, rather collaborate with it. Having a "subscription" notification system (such as sockets or node.js) reduces the load on your application since only the subscribed clients receive the new messages and no full refresh is needed from the chat clients.
Phalcon is great for the back end with its speed and it can be used to construct the message which in turn will be passed to the transport layer and sent to the client. Depending on how you want to implement this, there are plenty of options around and you can easily mix and match technologies :)
as #Nikolaos Dimopoulos said, you're trying to compare two different things.
But here is my advice, while you're experienced with PhalconPHP framework, and you want to benefit from Phalcon speed and performance, you can implement the web app in Phalcon FW, and the chatting system in Node.JS as a service.
If your web application "The Phalcon app" needs to push messages from the backend, you can use http://elephant.io/ library for that, I have done this before with Yii framework and Node, and it's working perfectly.
My advice is to use what you already know, experimenting with nodejs just for the chat application.
Mainly because you said you do not have experience with it, so, because the chat app is something a lot of people made you'll find plenty of examples.
By doing so you will learn a lot from node and might even think about migrating from Phalcon if it suits your needs, using the features offered by expressjs for example.
I would not choose one over the other based on performance.
I will try first to explain my web application goal.
It is dedicated to an intranet and the architecture will consist in a server connected to the web and less than 10 clients.
The application will be used to give aeronautical information. This will be achieved by retrieving with cURL requests (php scripts) launched every X minutes (CRON jobs) on remote sites (meteorology,airways and airport information) and saved in an XML file or a DB. The information gathered is then presented on a web page(a kind of well-organised synthesis) to air trafic controllers to enhance their situation awareness.
As data gathered must reach the client in real-time, I cannot rely on browser interaction : if an airport is closing due to bad weather,that piece of information has to be displayed as soon as possible without any user interaction.
The number of airport monitored will be around 30 (thus giving you an idea of the server load, knowing that meteorology reports are stored on the X website, airport data on the Y website etc..) .
After reading a lot on Reverse Ajax (Server-Push), I really need a professional experience to help me choose the best approach to develop this application.
The Server-Push technologies I discovered on the net are:
1) APE (Ajax Push Engine) -> This one makes me feel like trying to open a door with a bazooka (can handle thousands of connections).
2) Long polling (Comet) -> I fear this one could put to much stress on the server load.
3) Web sockets -> I must first wait for it to get mature and supported by firefox 6 (no more security issues)
As I am completely new to server-push, I hope you will help me find the appropriate way to achieve displaying these data in a close to real-time manner. It would certainly be a pity if I ended up setting "refresh" buttons to update the air pressure at the location of airport "A" using Ajax.
Thanks for reading.
If you speak C, there is an example of how to do pushing here. This 3-calls API is much simpler than others so it might match your relatively simple needs.
I want to be able to monitor the performance(load time of the entire page, load times of individually downloaded js/cs files , amount of memory used by the browser for the page,etc) of my web application from the perspective of the user(i.e the browser client).
Is there any tool/plugin , that can help me monitor all of these?
This list should serve as a good starting point for you. It is a complete list of end user monitoring tools.
In order to count as an End User Experience Monitoring tool it must be able to track the response times that real users experience when visiting the site – not a robot which is synthetically pinging the site. Specifically I am referring to tools that would enable IT operations to ensure that the real end users of an application or website are experiencing good performance. As I have alluded to in a previous post “speed solves a lot of problems” – claiming that even if your usability is not perfect – if it runs fast – people are less likely to notice.
We built a tool to solve this problem, take a look at Bucky.
Try FIDDLER or CHARLES
Try http://www.yottaa.com (disclaimer: i work here) - it runs real browsers in different locations to monitor your site and record asset level performance data. You can also try Gomez or Keynote Systems.
You can try out Atatus - https://www.atatus.com/ which offers performance monitoring and error tracking in one place for all your apps.
Disclaimer: I work at Atatus.
Just wondering: Would it be acceptable to start some simple Ajax instant messaging (web-based) for a large social network service (considering thousands of registered users)? I am new to this, so I'm just wondering. What if to check for a new message every two or three seconds?
Edited: Could a plain shared server handle so many requests every so often? And yes, I would roll my own program.
There're many web im client based on standard XMPP protocol. You could try iJab or JWChat.
It doesn't make sense to write your own, unless you have some unique requirements, but whether the server can handle this largely depends on the server language and webserver setup, as to how well it scales.
You will need to do some heavy load testing, to ensure that the high load that is expected will work, as your traffic will be very heavy. For example, if your social networking site is soccer related, then during the World Cup you may expect to see more traffic than Wed mornings.
If you asked the question with:
I want to use language X.
I want to use webserver Y.
I am using this framework for the
webservice.
I would like to accept voice
recordings and webcam recordings
over IM, as well as text.
How well will this scale on my one 1GHz server?
If you manage to make a peer-to-peer (P2P) browser-embedded chat, than even a shared hosting will do for dozens of thousands of simultaneous users :) :)
For a pet project, I have been looking for a web chat script capable of running potentially tens of thousands of users simultaneously. I don't want to use any kind of applet or browser extension, so on the client side, it should be simple Ajax. On the server side I'm pretty much open to anything.
I'm not looking for bells and whistles, a simple text-only chat is more than enough, as long as it supports a number of 'channels' or 'rooms' simultaneously, and a very large number of users.
When I first started researching the chat scripts out there, it seemed like the only viable option was to run an IRC server and just build a web interface on top of that. I know I could get good performance and stability with that setup, but could I get better performance by using something else?
Any ideas?
You might want to check cometd
I believe there are some chat scripts already using cometd.
I have no idea regarding stability tho.
You can have a look at Jabbify.
Not sure about the rooms and channels part, but it is built on the AJAX and MVC model.
I am going with Twitch.me, which is based on node.js