I was checking how does VMWare remote console work inside ESXI and I found out that you can not inspect the queries sent from that console.
Once the DOM is rendered I can comunicate with the server and send commands with my keyboard to the server and I receive an answer.
I can not see any updates in the DOM, this question is already answered in other places, the canvas just draws things inside without modifying the HTML element, and there seem to be some 3rd party tools to be able to follow those changes.
But how does it work on the network side? I do not see any query in the inspector, but I´m 100% sure it is communicating with the server, as I'm typing to the server and receiving the output of the commands.
If I use wireshark to monitor the network I just see some TLS queries.
Is there any way to monitor what is happening there inside the browser?
I'm not aware of the tool you are talking about.
HTML5 Canvases cannot be "Inspected"
You cannot see the separate elements in it. It is one single raster rendering. Just like you mentioned in your question. It is just a rendering, There are no separate elements in it to be inspected.
There are techniques to make it interactive, maybe even take in your keyboard input. But all of them involve redrawing the entire canvas on every tiny pixel update.
Now, To answer your main question:
Network Tab
If you wanna see the communication with the server, see the Network tab in Chrome dev tools. It should show every request sent from your web application.
Related
I've read that since user agent is the same between both, the recommend method is to use feature detection. That is fine and good for some situations, where you may want to display a Flash video/movie/app vs. a javascript slideshow, but my issue is to display a correct interface based on the user's input device.
The assumption I'm making is that if a user is in the "Metro" IE10 they are probably expecting to use their fingers instead of a mouse. That being the case, I'd like to give them an interface with large hit boxes.
My question: Is there a way to tell the difference and display an appropriate interface? Or am I stuck with making the user manually switch modes via links on my site that set a cookie?
Still there's no way to detect normal IE from the crippled Metro IE, but know you can know at the server if the user has a touch screen http://blogs.msdn.com/b/ie/archive/2012/07/12/ie10-user-agent-string-update.aspx
That post includes other comments about how to perform detection in javascript.
If you use the msPointerPoint interfaces, your client will receive the same messages whether they're using the mouse or touch. You can also use the gestures api - there was just a blog post on the IE blog which discusses how to use gestures from the mouse browser.
IE exposes a unified stack for messages so you can use the same input processing and your UI will work whether you're using touch/pen or mouse.
Think of a dictionary application that grabs words upon user's click from any other application (including browsers) and pops up it's meaning. Is it possible to twist this behavior based on the word's HTML attributes?
Edit: I noticed something cool. OSX's built-in dictionary app has a shortcut (cmd+control+d) that actually pops up a word's meaning (the word you're hovering your mouse above). It also highlights the word. The highlight area grows as you increase the font size. check out this fiddle and try various font sizes.
Yes/no
If your dictionary application embeds a browser instance (like a webkit browser) then it should be possible to access the DOM.
If you want your application to manipulate an already running browser instance it has no control over then it could probably done if
that browser has a browser plugin that interacts with your application
you intercept low level commands to mess directly with the kernel or the memory of the browser.
Neither of those are easy to do for all browsers.
I'm trying to grab web thumbnails by using X server to run Firefox in headless (Xvfb) X11.
I am looking for a way to know when Firefox FINISHED rendering and then I can use image programs to grab the windows.
Problem: I can NOT determine when Firefox finishes rendering. All I know is that the status bar should have an output of "Done" once it finishes rendering (IE, as shown in Firefox in GUI).
Can anyone know what can I use to determine whether if Firefox finishes rendering a page or not?
What I did is just sleep 40 seconds but this may not always work.
Environment:
Centos 5.7, Xvfb, Firefox 3.6.23
Shell commands:
Xvfb: 1 -screen 0 1024x768x24 &
export DISPLAY=localhost:1.0
DISPLAY=localhost:1.0 firefox http://www.example.com -width 1024 -height 768
sleep 40 **[ NEED some thing to tell me if Firefox finishes rendering ]**
DISPLAY=localhost:1.0 import -window root example.com.png
pkill firefox
Three methods off the top of my head.
Write your own extension that hooks to window.onload and fires the image grabber.
Write a greasemonkey script that hooks to window.onload and fires an alert. Wait for such alert, then run your image grabber.
Periodically grab the image of the Stop button (should be at known coordinates relative to the window) and analyze its color. If it's disabled, the rendering is finished.
Note that I don't think this can be done with FF 4.x and above easily. This is due to "features" of these versions. My recommendation to you is to use some different browser if you can, like Opera. I did not test it myself yet, but I think it does not have all those flaws noticed with FF. See below.
In your case following solution may be efficient:
Write some Greasemonkey script which adds an absolute positioned rectangle somewhere on the screen.
Let this rectangle flash in a 3 cycle when the page has finished loading: red, blue, transparent
Write a script which waits at least 2 such cycles to complete (so it sees red, blue, something, red, blue) and then do a screenshot with the correct timing in the transparent phase.
This should be doable with X11 and VNC and similar. Sorry that I don't have a better solution nor code I can present here.
I for my part would solve it using EasyRFB which I wrote to solve similar things. However I cannot recommend that to you, yet, except if you are very desparate Genius able to read and understand horrible undocumented code fragments with ease ;)
BTW, thanks for noting the idea of a package which is able to screenshot arbitrary web pages, which makes an excellent use case for EasyRFB. Never thought about this myself before.
Looking at GitHub I noticed, that there are some solutions for taking web screenshots based on Selenium or WebKit. These are certainly good for promotional shots, but they apparently cannot cover the general usecase on how users see web pages.
Perhaps it would be interesting to be able to automate screenshots from arbitrary pages as seen by a Windows 95 IE 4, FF 1 or Debian Potato with Chimera etc. I'll have a look into this ;)
Note that there is another posting on Stackoverflow from me.
There is something I call EasyRFB which I wrote in a mix of Python (core), PHP (small web helpers) AJAX and bash (control scripts). It was tested with XVnc but shall work with any VNC. It somehow is able to do what you want, but beware, it is development code, undocumented, complex and horribly written and may be faulty. You can find it there:
GitHub: https://github.com/hilbix/pyrfb/tree/easyrfb
My old development directory: http://hydra.geht.net/easyrfb/
It was written for following purposes:
Being able to control a remote GUI from a Mobile even in a situation where the Internet connection is extremely slow and extremely unreliable
Complete UI automation / replace mouse by commandline
Automated QA tests, based on what the user sees in contrast to what the browser or some windows spy program sees, in cases where the output needs not to be pixel-color-perfect as well
It was not written to be fast nor efficient nor easy to use nor for others (yet). It works, for me, somehow.
What it does:
is Web centric and gives a AJAX Web frontend to VNC servers, somehow.
keeps a highly compressed .jpg updated to the contents of the screen.
can take lossless screenshots
is able to fuzzy-match templates against the contents of the screen
templates can be Web-edited with edit.html - however this editor is basic, not self-explanatory and not documented at all
Shell scripts can be written which can wait for these templates to match and send commands like keypresses, mousemoves and clicks to VNC
There are some .sh scripts which could help to understand how this is done, but I am not able to publish the really helpful stuff yet, sorry
Nothing is documented and I am not able to explain how to use it. Either you find out yourself or you are lost. Sorry. Better don't even look at it, it might hurt you.
And now to the problems noticed with FF while doing all this:
Newer FF versions fail to properly update the screen. The last version known working correctly at my side was the FF 3.6 series, all later sometimes develop some strange behavior.
For example:
Newer versions now update the status before they update the screen contents.
Also they have such a high CPU and memory demand, that finishing the last step on updating the screen can take some time. On slow hardware (like the NSLU2, never tested) I think it might even take minutes after the Spinners etc. are already in the "finished" state.
So even looking at the spinner to become gray including(!) naive Greasemonkey solutions which notify about "onload()" both fail with FF 4.x and above, because parts of the screen may still not be updated when you detect that it is ready.
Maybe it has only to do with XVnc, perhaps there is a bug in my scripts which triggers all this, but with FF 3.x everything was right and I think, reverting to an old version would immediately solve all those problems seen. (But I already have workarounds in place for all those trouble. But I cannot publish that scripts, sorry.)
It gets worse.
FF 4.x and above apparently delay event processing a bit. So jumping the mouse out of the scrollbar often keeps it highlighted, I think the event does not reach the scrollbar.
On a normal desktop you do not jump your mouse, but my solution is able to do so. The fix is to generate additional, superfluous mouse movement events to have FF do the right thing.
Also I noticed, I really have no idea if it is due to FF or the web pages shown in FF, that sometimes things do not land where they are supposed to be. I never observed this before with FF 3.x though. The trick which solves this is to reload the page. It might have to do with incremental screen updates, but I did not dive into this yet as the retry (automated clicking on the reload button) works for me.
I'm trying to automate some tests on Internet Explorer. How can I find out if the state of the webpage being shown is changed? Is there any way to know when the page finished loading, if it changes, if it is scrolled, etc.?
I want to know how I can get this state information from another running process.
Selenium allows you to test for completion of a page load and many other events within Internet Explorer and many other web browsers. It's a tool for autmoated testing of web pages.
It allows you to write tests that perform actions and test the state of pages.
You could either look at the source for selenium and see what it's doing or just use it to do your testing.
I'm not sure how a web page would scroll by iteself in an automated test, but you could detect if the scroll Has happened by storing the window.scrollX value when the page is loaded and testing to see if the value has changed.
Basically, if you want to detect things like window scrolling in Internet explorer, you probably need your own piece of code running in the browser, like a BHO (Browser Helper Object).
There might be a way to do Event Sinking. Have a look here: http://asp.dotnetheaven.com/howto/doc/Interop/TestClient_2.aspx
I'm using ruby-libnotify in a Ruby GTK app, and it works great to create a bubble popup in Ubuntu. I'm on Hardy, and it all works great. Then I had others try the app on Jaunty, and instead of a bubble popup with the new Notify-OSD system, as I expected, the notification turned into a dialog box.
I looked into it, and found the Ubuntu wiki states that the problem is because I set a timeout of 0:
Some programs specify an expire_timeout of 0 to produce notifications that never close by themselves, assuming that they can be closed manually as they can in notification-daemon. Because this is usually done for a message that requires response or acknowledgement, Notify OSD presents it as an alert box rather than as a bubble.
Is there a way I can use libnotify in some way to have a normal bubble with a "never expire" timeout? I would actually prefer it if I could use the old notification system, even, since Notify-OSD doesn't seem to support permanent bubbles at all.
It is unacceptable to have the dialog for me, as it doesn't stay over all windows, so the user won't see the popup right away, necessarily (which is the whole point of using the bubble popup).
It looks like you are just trying to use Notify-OSD for something it was not designed for. Notify-OSD bubbles are informational and transient, meaning that no critical information should be put in them as they are made to be ignorable.
According to the Ubuntu Design Guidelines, it looks like you are trying to make a morphing alert box, which should suit your needs nicely.
Sort of a sideways answer, but perhaps if the notification API doesn't quite map onto what you want to do you should look into using a more general library that allows you to draw your own on-screen bubbles. xosd comes to mind, though I remember it to be quite limited, but perhaps there are other options...
I remember using some command line tool to display notifications. You could just call it using system or ``.