How to save all the resources that are sent to a browser? - debugging

I have a same site deployed on 2 different servers, The source files on the servers are identical, so I have to assume what is being sent by webservers on servers are different, as one has javascript that works in IE11 and the other server sends something that breaks the Javascript on the very same page and input (posted data).
How do I either save all the resources sent to a request or use IE11 F12 tools to save all the received resources?
Is there any tools for emulating browsers and saving all the sent responses to disk? (I tried saving the network traffic in IE11 to disk , but there are details about reponse times etc, that are making a compare extremely difficult).

You can save all the files delivered by the server to IE11 by clicking the Page button, then choosing Save As. Select Web Page, complete to save all the files.
The first thing to check would be to use the IE developer tools to watch for errors as the page loads.
Corrected due to comment - thank you

Related

Xhr request shows pending for a certain format in all the browsers on my machine

In my windows machine I run a simple fileserver that serves certain files from a folder. I access these files via chrome/firefox browsers
For a certain file format (In my case ".bin" file) the xhr request always stalls with a message saying "pending". But If I rename the file extension to ".cbin" and reload the page on browser again it works.
Why are the browsers preventing a certain file to be loaded ? All of this used to work a month back without issues (ie loading the bin files). I have disabled my antivirus too.
Any help would be invaluable. Thanks
After hours of searching on the web and experimenting I realized that the browsers have recently added rules by which they block downloads of certain files (exe,dmg,zip,gzip,bin) etc on a http connection for security reasons.
Hope this will help someone who faces the same issue.
You can read more about this issue here and here.

MVC 3 with IE, poor bundle performance

We have deployed an MVC 3 website on an IIS6 box.
Everything runs fine, but the performance is abysmal.
Can anyone help me understand
why am I getting 20 second response times to get a script bundle?
why bundled scripts are not cached by IE even if the Expires header is set?
The site is several times faster in Chrome (I have noticed the cache behaviour is correct), but we cannot force customers to use it.
Any help would be great. I'm kind of wondering if it's a server-side setting that's forcing the bundle recompilation each request, or if it's just IE acting like usual.
Edit: as per comments request, I'm including also the bundle request headers:
If you have different download times for a full reload between the two browsers it could be that you are doing intense computations with a client side framework like angularjs (I have seen big performance differences from highly complex angularjs apps between the two browsers).
If both your browsers show the same download time, it is either a network issue, or a server issue.
The IE caching could be a separate issue, break your problem into two parts - look for the cause of the slow downloads first.
All I can do now is suggest an approach to finding the issue.
Summary of what you know
It looks like you have:
Server sends an Expires header one year from now
When you reload the page (i.e. you don't force a full refresh using Ctrl+F5)
IE doesn't take any notice of the cache header, and when it sends it's new request it doesn't use If-Modified-Since or If-None-Match
Chrome behaves differently and respects the Expires and/or ETag response headers (it doesn't even make the request again for the bundle).
EDIT 1: You also seem to be saying (though it would be good to see a timeline from chrome) that Chrome downloads the files faster, implying it is not a server-side problem. Your latest comment states that Chrome's downloads are also slow. (end edit)
And you also seem to be saying that this behaviour is consistent (i.e. 100 requests in IE, and 100 requests in Chrome show the above behaviour with no deviations).
Approach
You should break this problem into two parts:
Why is the download so slow?
Is there a server-side performance problem? Look for common download times in IE and Chrome, and Firefox (it could be due to bundling/minification/compression on the server).
Is there a network connectivity issue (dropped packets, for instance)? Look for inconsistent download times, Start times, Request times, between requests in a given browser and the same behaviour across all browsers.
Is a script slowing down IE, but not Chrome (this is not uncommon, I maintain legacy sites where the scripts don't run well in IE but do in Chrome) - look at different profile results between browsers.
Why is the javascript not being cached in IE? Troubleshoot (1) first, then worry about this.
It is possible that the two are related, but approaching them separately will be a start. Number 1 is far easier to diagnose that 2, the top references to caching javascript in IE on the web are to prevent it in order to help with development.
Root cause diagnosis
EDIT 1 The first thing to do is try the site from a browser on the server, or very close to the server to see if you have a network issue. (end edit)
Tools like Fiddler, the browser developer tools, timeline and script profiler, and YSlow are your friend. Compare each of the following between Chrome and IE (and see what happens in Firefox as well) and spot the difference. Note: you may need to clear the browser cache between tests.
browser developer tools -> script profile: see if you have a slow running script in IE compared to Chrome
similar analysis in a tool like YSlow (look for comparisons between the two browsers, not script improvements)
request and response headers, and timeline from a normal (i.e. not full reload) page load
request and response headers, and timeline from a full page reload (Ctrl+F5)
Start and Request durations for every js file for a given browser, and between browsers (this may point to network issues)? I note that the Start and Request alone are taking 0.6s and 1s each in IE - that is very very poor performance.
5 requests, and 5 full reloads with cache clearing between (that is, don't chase a ghost - be consistent in your test methodology)
Download times should be no different between Chrome and IE with no scripts actually running so also add a control test. Assuming that your bundle files don't "do anything" (i.e. they contain functions that the page calls rather than kicking off long processes by themselves) then create a blank page in your site which references exactly the same javascript files - not just the bundle, but every single js reference.
With the control test you can compare pure download times and caching behaviour in IE to Chrome, without any client side javascript running (use the developer tools profiler to verify no scripts are running). If your bundle files do kick off long running things, just temporarily disable those things by putting return statements at the top of the script and concentrate only on the download into the browser.

Firefox not re-validating static files from child frames

I'm working with an application that has an iframe - both the outer html body and the frame require certain javascript and CSS files. To cut down on load times, all these static files have expiry set to a year from now and essentially should be loaded from cache for normal page hits - which is expected behavior in IE8 and FF3.6
However, once I reload/refresh (F5) the page, I expect the browsers to send an 'If-Updated-Since' requests to the server for these files. IE8 sends the requests for all the files used outside as well as within the iframe. But, FF3.6 only sends the requests for files used outside (not for the files used within the iframe, it just loads those from cache!).
The response headers are exactly the same for all files regardless of whether they are in the iframe or not. Is there a reason for this behavior of FF? Any way to avoid it?
Note: I can append version parameters to the source, or add a version folder in the path, etc. But, I want to know if this quirk can be avoided/has a good reason behind it?
Firefox behaves correctly - the server indicated that the scripts are good for a year so there is no reason to send pointless requests which waste time, bandwidth and server resources. For debugging purposes you can keep the Shift key pressed while clicking the Reload button, it will make sure that all data is refreshed. However, for end users adding the version information to the URL (e.g. http://example.com/.../script.js?version=1.2.3) is probably the best solution. This makes sure that the cached version can be used as long as it is valid and the new version is downloaded as soon as you update the script.

?_escaped_fragment_= - headless browser

what do I have to do to add a ?_escaped_fragment_= support to my server? I want google to be able to crawl through my ajax site. My hashes are already in #! form
But I have no idea how to tell my server that when I enter mywebsite.com/?_escaped_fragment_=section to my browser so the url mywebsite.com/section and it would be equal to mywebsite.com/#!
thanks
Simple answer - my method (soon to be used for a site with ca. 50,000 AJAX-generated URLs) is to have a node.js server using a headless environment (try zombie, phantomjs, or any other) to load the site, making sure it's able to execute javascript and read the DOM - then at runtime, if it's google requesting the fragment, fire a request to the node.js server, which loads the site, executes the javascript, waits for the response, and delivers back the HTML, which is output to the browser.
If that sounds like a lot of work - I'm about 90% finished on the code that does it all for you, where you'd simply drop one line of (PHP) code at the top of your site/app and it does the rest for you, using a remote node.js server.
The code will be open source so if you want to set it up yourself on a node server, you can - or if it's a PITA to set it up yourself, I'll probably have a live server up and running which your app/website would fire ?_escaped_fragment_ requests to, and get the html snapshot back. It also implements caching so that these are only requested once every X days.
Watch this space - just got a few kinks to work out, and it'll be on my site (josscrowcroft.com) and I'll put it in a github repo too.

Are there any diagnostics tools for troubleshooting content delivery with Opera Mini?

I have an application that I'm targeting a wide variety of devices and platforms. The application can render different HTML based upon the type of client. However due to the complexity of the application, it shares a considerable amount of JavaScript libraries that rely on a number of async and ajax method calls.
One of the targets for the application is Opera Mini. This "sort-of" works but it seems like sometimes when building up the specialized markup to send down to the Opera Mini JVM client it does not wait until the async calls are complete. Are there any techniques or tools to see what's going on with the Opera Server (not my application web server) Side processing of the page to determine what I can do to make this solid?
It would appear that after further investigation that the server side browser is fairly picky when it comes to CSS. I can't remember the exact problem, but as soon as I removed the stylesheet all content was displayed properly. At that point I slowly re-introduced the CSS and everythning came back online and worked as expected.
Your javascript will only be allowed a short time before it is aborted:
JavaScript running on the Mini server
will only run for a couple of seconds
before pausing, for resource
constraint reasons. This applies to
JavaScript run due to an event firing
e.g. onload, as well as code run
because of a user action.
~ http://dev.opera.com/articles/view/opera-mini-web-content-authoring-guidelines/#javascript
So the best would be to serve the least javascripty version of your site to the Opera Mini user-agent.
You can type server:source in the address bar once a page is loaded if you want to see the current DOM tree.
It's also possible to post that source to a script on your server using server:source?post=http://your.server.com/script. It will send three fields as a POST request: url, host and html. You can then make your script save it to a file.
(Answering an old question in case it helps someone.)

Resources