Morbo server works only after constant refresh - ajax

I am developing a web application with Mojolicious. The morbo development server is a wonderful thing that works great, but once I start returning complicated hashes on the stack and then rendering a webpage, the morbo server will start to act funny. In my browser, if I navigate to one of those webpages that use a complicated hash, the browser will tell me that the connection has been reset. I have to refresh about 10-12 times before the page will load.
For example:
The code below shows one of my app controllers. It simply gets a json object from an AJAX request, and then returns a different json object. It works fine, except that the browser demands to be refreshed a thousand times before it will load.
package MyApp::Controller::Library;
use Mojo::Base 'Mojolicious::Controller';
use Mojo::Asset::File;
use MyApp::Model::Generate;
use MyApp::Model::Database;
use MyApp::Model::IpDatabase;
use Mojo::JSON qw(decode_json);
# Receives a json object from an AJAX request and
# sends the necessary information back to be
# displayed in a table.
sub list_ajax_catch {
my $self = shift;
my $json = $self->param('data');
my $input = decode_json $json;
$self->render(
json => {
"Object A" => {
"name" => "Object A's Name",
"description" => "A Description for Object A",
"height" => "10",
"width" => "5",
}
}
);
}
1;
The problem is not limited to this instance. It seems that anytime there is a lot of processing on the server, the browser has troubles resetting. It doesn't matter what browser, I've tried Chrome, IE, Firefox, and others (on multiple computers). It doesn't matter if I'm not even sending or receiving data back and forth from the html to the app. All that seems to trigger it is if there is any amount of processing in my web app that is more than just rendering templates, BUT if I am running Hypnotoad, everything is fine.
This example is not one that requires a lot of processing, but it does cause the browser to reset, and as you can see, it shouldn't take long to run or freeze anything up. I thought the problem was a timeout issue, but by default, timeout doesn't happen until after 15 seconds, so it can't be that.

I have figured out the problem! This has been an issue for me for over a month now and I am so glad that it is working again. My problem was that when I started the morbo development server, I used the following command:
morbo -w ~/web_dev/my_app script/my_app
The -w allows me to watch a directory for changes so that I don't have to restart the app each time I changed some of my JavaScript files. My problem was that the directory I watched also contained my log files. So each time I went to my webpage, the logs would change and the server would restart.

Related

Http Post - long delay at state ExecuteRequestHandler

I have an asp.net mvc website hosted on Windows Server 2012r2 Standard which uses KnockoutJS to display data in a grid. This server is dedicated to the process that I'm having trouble with - it does not server any other requests.
An ajax call is made to a "GetRecords" action of a controller. This returns data for a couple of dozen records very quickly.
The user is able to make amendments to the data and submit for update. The knockout code makes another ajax call, this time posting the records. At this point the site "hangs" for a long time (over 10 minutes), but it does complete successfully and the updated date is persisted to a database. During the "hang time" the CPU for the IIS Worker Processes hovers around the 50% mark.
I'm trying to figure out what's causing the delay. It seems that the delay happens before the first line of code of the controller action is reached. I've added trace statements to the action and I can see that once the 1st line is executed, then the action completes within a couple of seconds.
From the IIS manager, I've drilled in to "Worker Processes"\"Current Requests" during the time the page is "hung", I can see that the State is listed as "ExecuteRequestHandler" and the Module Name is "ManagedPipelineHandler". There are no other "Current Requests" displayed.
Using the Chrome dev tools, I've captured the json being posted for the update, it is approx 4mb in size.
I've ruled out the problem being caused by bandwidth because I've tested from a browser running locally (on the web server), and I get the same delay.
Also, when I post the same number of records on the same site hosted on my dev VM then it works fine - completes end-to-end in under 3 seconds.
Any suggestion on steps I can take to improve performance of the post?
I have created a process dump of the IIS worker process when it is in the "hanging" state, this is available at: onedrive link
It seems that "Thread 28" is causing the issue, since this has a "Time spent in user mode" value of over 2 minutes. I requested the process dump about 2 minutes after making the http post request from the website. The post did eventually complete ok after about 20 minutes
Able to work around this problem bypassing the MVC model binding. The view model param (editBatchVm) that was passed into the controller method has been replaced. So, instead of:
public void ResubmitRejectedVouchersAsNewBatch(EditBatchViewModel editBatchVm)
{
I now have:
public void ResubmitRejectedVouchersAsNewBatch()
{
string requestData = "";
using (var reader = new StreamReader(Request.InputStream))
{
requestData = reader.ReadToEnd();
}
EditBatchViewModel editBatchVm = JsonConvert.DeserializeObject<EditBatchViewModel>(requestData);

How to force loading the current version of an offline-able web app?

I'm doing a tiny offline-able web app / PWA. It's meant to be opened from a home screen icon and mimic a regular app by loading entirely from a cache when offline.
The app is written using Vue and to accomplish the above I'm just using their PWA template and whatever it generates. To the best of my knowledge what this does is set up workbox using the GenerateSW plugin to precache everything in the Webpack build, and registers it using register-service-worker. That is, I have fairly little control out of the box over the fine details, it's meant to be a turnkey solution.
That said, I'm not sure how to actually load a new build of the application when it's available. The above can detect this - the generated SW registration file with my changes looks like this:
import debug from 'debug';
import { register } from 'register-service-worker';
if (process.env.NODE_ENV === 'production') {
register(`${process.env.BASE_URL}service-worker.js`, {
ready(...args) {
log('App is being served from cache by a service worker.\n', ...args);
},
cached(...args) {
log('Content has been cached for offline use.', ...args);
},
updated(...args) {
log('New content is available; please refresh.', ...args);
},
offline(...args) {
log('No internet connection found. App is running in offline mode.', ...args);
},
error(error, ...args) {
log('Error during service worker registration:', error, ...args);
}
});
}
When I make a new build of the application, and I refresh the app in a browser, the updated() callback is executed, but nothing else is done. When I tried adding:
window.location.reload(true);
which should be a forced refresh, I just get a refresh loop. I'm assuming this is because the service worker cache is completely independent from the browser cache and unaffected by things like the above or Ctrl+F5. (Which makes the "please refresh" rather misleading.)
Since this is going to mimic a native app, and it's supposed to be a simple line-of-business tool, I don't really need to do anything more complicated than immediately reload to the new version of the app when an update is available. How can I achieve this?
Okay so the behaviour I've observed is that the update does happen automatically, it's just not obvious as to what the exact sequence of events is. I'll try to describe my best understanding of how the generated service worker works in the installed PWA scenario. I'll speak in terms of "app versions" for simplicity, because the mental model behind this is closer to how apps, not webpages work:
You deploy v1 of your application to a server, install / precache it on a device, and run it for the first time.
When you suspend and resume your app, it does not hit your servers at all.
The app will check for an update when it's either cold-started, or you reload the page, i.e. using the pull down gesture on Android.
(Possibly also periodically as the cached version goes stale, but I haven't checked this.)
Say you've deployed v2 of your app in the meantime. Reloading an instance of v1 of the app will find this update, and precache it.
(One reason why prompting the user to reload doesn't seem to make sense. Whatever the reloading is meant to accomplish has already happened.)
Reloading an instance of v1 again does absolutely nothing. The app remains running between reloads, and you'll just get v1 afterwards.
(Reason number two why prompting the user to reload is pointless - it's not what causes a new version of an app to load.)
However, next time you cold start your app - e.g. nuke it from the task switcher and reopen - v2 of your app will be loaded and I'm guessing v1 will get cleaned out. That is, your app must fully shut down so an update will load.
In short, for an application to be updated from v1 to v2, the following steps need to occur:
Deploy v2 to server
Refresh instance of v1 on device, or shut down and reopen the app.
Shut down and reopen the app (again).

Log Javascript console output in Laravel Dusk

I am using Laravel 5.6 and Laravel Dusk 3.0.9.
Dusk is pretty handy, but when a test fails on a page where there is some Javascript functionality it can be pretty hard to work out what went wrong. Dusk generates a screenshot, which helps, but what I really need is to see the output from the Javascript console.
Apparently this is possible - Dusk creates a tests/Browser/console directory as part of its installation, and this PR suggests the JS console output is logged, or at least loggable.
There is no documentation (that I can find) about how to actually do that though. Browsing the diffs in that PR, I see a new method logConsole() (later renamed to storeConsoleLog() as #Jonas pointed out in the comments), but I can't seem to get it to do anything, eg:
$browser->visit('/somewhere')
->select('#foo', '2')
->value('#date', '2018-07-29')
->storeConsoleLog('bar')
->assertEnabled('button[type=submit]');
This test fails, and generates a nice screenshot, but there is no sign of any logfile. I've tried moving the position of ->storeConsoleLog('bar') around in the chain, eg first or last, and as a separate line before or after the chain:
$browser->visit('/somewhere')
->...full chain here;
$browser->storeConsoleLog('bar');
But none of them make any difference. My JS has a series of console.log()s which I use when testing myself in a browser, and which will tell me exactly what went wrong. I was expecting this functionality to log those messages.
Am I misunderstanding that PR, is this even possible? If so, how?
UPDATE
By debugging the storeConsoleLog() method in vendor/laravel/dusk/src/Browser.php I can see that the method is being called correctly, but there is no console content to log. If I manually repeat the steps the test is taking in Chrome, there are lines being written to Chrome devtools console, in fact 3 are written on page load. Why can't Dusk see those?
UPDATE 2
I found that if you remove '--headless' from the driver() method in DuskTestCase, the browser will be displayed during tests. You can then display the dev tools for that browser, and watch the console output live as the tests run. It is too fast to really be useful, and if there is an error the browser closes and you lose whatever was on the console anyway (unless there's a way to leave the browser open on failure?), but adding this here in case it is useful to someone!
There is apparently no way to get non-error console output from the browser.
There are other log types, but Chrome only supports browser (used by Dusk) and driver.
You'll probably have to use alerts. But screenshots don't contain alerts, so you have to get the text:
$browser->driver->switchTo()->alert()->getText()
You could also use something like document.write() and check the output on the screenshot.
It is possible. There's an answer here written in Python from 2014 that shows it's been possible since at least that long.
Override \Laravel\Dusk\TestCase::driver:
protected function driver()
{
$desired_capabilities = new DesiredCapabilities([
'browserName' => 'chrome',
'platform' => 'ANY',
]);
// the proper capability to set to get ALL logs stored
$desired_capabilities->setCapability('loggingPrefs', [
'browser' => 'ALL',
'driver' => 'ALL',
]);
// return the driver
return RemoteWebDriver::create(
'http://localhost:9515',
$desired_capabilities
);
}

How to handle every request in a Firefox extension?

I'm trying to capture and handle every single request a web page, or a plugin in it is about to make.
For example, if you open the console, and enable Net logging, when a HTTP request is about to be sent, console shows it there.
I want to capture every link and call my function even when a video is loaded by flash player (which is logged in console also, if it is http).
Can anyone guide me what I should do, or where I should get started?
Edit: I want to be able to cancel the request and handle it my way if needed.
You can use the Jetpack SDK to get most of what you need, I believe. If you register to system events and listen for http-on-modify-request, you can use the nsIHttpChannel methods to modify the response and request
let { Ci } = require('chrome');
let { on } = require('sdk/system/events');
let { newURI } = require('sdk/url/utils');
on('http-on-modify-request', function ({subject, type, data}) {
if (/google/.test(subject.URI.spec)) {
subject.QueryInterface(Ci.nsIHttpChannel);
subject.redirectTo(newURI('http://mozilla.org'));
}
});
Additional info, "Intercepting Page Loads"
non sdk version and with much much more control and detail:
this allows you too look at the flags so you can only watch LOAD_DOCUMENT_URI which is frames and main window. main window is always LOAD_INITIAL_DOCUMENT_URI
https://github.com/Noitidart/demo-on-http-examine
https://github.com/Noitidart/demo-nsITraceableChannel - in this one you can see the source before it is parsed by the browser
in these examples you see how to get the contentWindow and browserWindow from the subject as well, you can apply this to sdk example, just use the "subject"
also i prefer to use http-on-examine-response, even in sdk version. because otherwise you will see all the pages it redirects FROM, not the final redirect TO. say a url blah.com redirects you to blah.com/1 and then blah.com/2
only blah.com/2 has a document, so on modify you see blah.com and blah.com/1, they will have flags LOAD_REPLACE, typically they redirect right away so the document never shows, if it is a timed redirect you will see the document and will also see LOAD_INITIAL_DOCUMENT_URI flag, im guessing i havent experienced it myself

If I Rapidly Click the Browser Back Button Twice, The User is Logged Out in Our Cake App

This is a weird bug, and I'm not even sure how to begin figuring out what's going on.
We are using Cake 1.3.8 with our sessions in the database. I am not using ACL or any other access control. If we navigate into the application and click around a bit, and then rapidly click the browser back button twice (I've tried in Firefox and Chrome) the user is logged out more often than not and receives the error message 'You are not authorized to access that location'.
All of my searches thus far have involved people wanting to make the page inaccessible if a user logged out and then used the back button. I'm not seeing anything reported with regards to the issue I'm seeing.
Does anybody know if this is a Cake issue or have any thoughts on debugging what is going wrong?
Update: I found where the problem is. I have the security set to high, because we need the session to be closed whenever somebody closes the browser. I also have the timeout set very high because we do large binary uploads to S3, and don't want the user logged out while it's uploading or downloading. The specific block of code in cake_sessions.php that's causing the problem is:
$time = $this->read('Config.time');
$this->write('Config.time', $this->sessionTime);
if (Configure::read('Security.level') === 'high') {
$check = $this->read('Config.timeout');
$check -= 1;
$this->write('Config.timeout', $check);
if (time() > ($time - (Security::inactiveMins() * Configure::read('Session.timeout')) + 2) || $check < 1) {
$this->renew();
$this->write('Config.timeout', 10);
}
}
$this->valid = true;
I would guess this is because session IDs are regenerated between requests when security = high. Source:
http://book.cakephp.org/compare/44/CakePHP-Core-Configuration-Variables/cakephp/cakephp1x
You only need one out of sync request, say for a missing image and you will lose the session. I've generally found it unworkable because it's not possible to prevent users double-clicking on links and buttons and invalidating their session.
I would think about using medium security, setting the session timeout fairly short and using an AJAX script to refresh the session at regular intervals (eg every 60s). That way the user will be logged out quickly if the tab/window is closed.
If security is a priority I would suggest hacking the core to make sure the session cookies are set to http_only to help guard against session hijacking by XSS attacks. Cakephp 1.x supports PHP4 so probably isn't setting this by default.
http://php.net/manual/en/function.setcookie.php
It's possible that the session is erased and before it can be written again, the back button is clicked removing the auth from the session variables.
Page loads -> Back Button Clicks -> sessions is erased (but before session is rewritten) -> Back button clicks -> Session checks no existing session.
The only thing that I can think is happening is that when you're going back a page too quickly your code can't validate the person quickly enough (round trip from checking credentials) and throws an error that gets displayed on the next page that is loaded (second backed-to page).
Are you sure the person is actually logged out, or is it just the error being thrown?
Without seeing any code, it will be difficult to nail it down any further.

Resources