We had a H12 response timeout error page issue on our Heroku server.
When that happen, We estimate there was only about 20+ concurrent users.
We want to reproduce this in a stress test, and then update the server configuration to ensure it wont happen again.
But we couldn't reproduce the H12 application error page again regardless of how much stress we put on Heroku.
I used https://github.com/thomasdondorf/puppeteer-cluster to try to simulate 250 concurrent users.
The code is like this:
const { Cluster } = require('puppeteer-cluster');
var workerID = Math.floor(Math.random() * 10000);
(async () => {
// Create a cluster with 2 workers
const cluster = await Cluster.launch({
concurrency: Cluster.CONCURRENCY_CONTEXT,
maxConcurrency: 5,
puppeteerOptions: {
headless: true,
args:['--no-sandbox', '--disable-setuid-sandbox']
}
});
// Define a task (in this case: screenshot of page)
await cluster.task(async ({ page, data: url }) => {
await page.goto(url, {
waitUntil: 'networkidle2', timeout: 0
});
await page.waitForTimeout(10 * 1000)
const path = url.replace(/[^a-zA-Z]/g, '_') + '.png';
await page.screenshot({ path });
console.log(`${workerID}: Screenshot of ${url} saved: ${path}`);
});
// Add some pages to queue
for(var i=0; i<100; i++){
cluster.queue('https://server.com/page1');
cluster.queue('https://server.com/page2');
cluster.queue('https://server.com/page3');
}
// Shutdown after everything is done
await cluster.idle();
await cluster.close();
})();
I believe the code above simulates 5 concurrent users loading the pages. Then we created 50 dynos, effectively simulating 250 users.
But unfortunately, we didnt get the H12 error page at all. But we got a bunch of H27.
Any Heroku experts know why we didn't able to reproduce H12 in a stress test setup like this?
Related
I can't tell if this is a bug on my side or not, but I just can't figure out why this is not working.
Running a basic nodejs server and on a test page on nextjs, socket io is working perfectly fine. However, when I'm using it on my actual page, it does constant polling and never starts the ws connection, and the events don't work. The polling does have an appropriate response, with "upgrades":["websocket"] and all.
And, it seems that after the constant polling on that broken page starts, then the test page also starts doing it and doesn't work unless I restart my nodejs server. If I navigate to the broken page from the test page through nextjs, it also works, but a refresh then breaks it again. Any reason why this is so?
You can do this with this script. This is my personal working script for socket IO chat app.
Backend Server
require("dotenv").config();
const port = process.env.SOCKET_PORT || 3000;
const main_server_url = process.env.SERVER_URL;
var express = require("express");
var app = express();
var server = app.listen(port);
var connectionOptions = {
"force new connection": true,
"reconnection": true,
"reconnectionDelay": 2000, //starts with 2 secs delay, then 4, 6, 8, until 60 where it stays forever until it reconnects
"reconnectionDelayMax": 60000, //1 minute maximum delay between connections
"reconnectionAttempts": "Infinity", //to prevent dead clients, having the user to having to manually reconnect after a server restart.
"timeout": 10000, //before connect_error and connect_timeout are emitted.
"transports": ["websocket"] //forces the transport to be only websocket. Server needs to be setup as well/
}
var io = require("socket.io").listen(server, connectionOptions);
var axios = require("axios");
var users = [];
var connections = [];
console.log("Server connected done");
io.sockets.on("connection", function (socket) {
var server_url = main_server_url;
console.log(server_url);
console.log(people);
connections.push(socket);
console.log("Connected : total connections are " + connections.length);
// rest of events of socket
});
Front End JS for load IO for client
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.3.0/socket.io.js"></script>
<script type="text/javascript">
var base_url = YOUR_BASE_URL;
var port = YOUR_SOCKET_PORT;
var socket_port_url = base_url + ":" + port;
var socket = io(socket_port_url);
socket.on('done', (data) => {
console.log(data);
});
</script>
I am setting up a new VueJS-Project and implemented a workbox-backgroundSync in my progressive web application. To test this workbox plugin i need to disconnect my device (chrome) from the internet and fire a POST-request. Under chrome this request got a state of pending for couple of minutes, even with no internet connection. So my application is waiting to the request response.
My POST-request:
axios
.post('/createrecipe', recipe)
.then(response => {
commit('createRecipes', recipes);
commit('setSnackbar', { text: "Recipe was created. ( " + response.data.status + response.status + " )", color:'success', snack:true });
console.log("Response from Server for create Recipe: ", response);
})
.catch(error => {
commit('setSnackbar', { text: "Your Post will be stored and send by Internet-Connection", color: 'warning', snack: true });
console.log(error)
});
My ServiceWorker:
const matchCb = ({url, event}) => {
return (url.pathname === '/api/createrecipe');
};
const bgSyncPlugin = new workbox.backgroundSync.Plugin('createdRecipes-post-storage-offline', {
maxRetentionTime: 24 * 60, // Retry for max of 24 Hours
});
workbox.routing.registerRoute(
matchCb,
workbox.strategies.networkOnly({
plugins: [bgSyncPlugin]
}),
'POST'
);
I interrupted the network in Chrome-devtool under the network tab and the background process went through successfully. But this is not the best solution for the users, because they dont know how to control the devtool.
I expect that the request will be canceled, while chrome has no internet connection.
Here is a screenshot of the request in a state of pending instead of failed:
enter image description here
Side Note: My web application runs under localhost.
Not the best solution but my mate Ilja KO found a workaround. I just set a timeout with the instance of axios to abort the requests, even if chrome got a internet-connection or not.
axios.defaults.timeout = 5000;
I'm using service worker for push notifications, following this article. Everything is working with Chrome but with Firefox (v.44.0.2) I have a weird issue.
On successful login to my app, I register the service worker which does nothing but waiting for push events; I see that is correctly registered (from some logging and from about:serviceworkers). Now, if I refresh the page (CTRL+R) all my POST have CORS issues (missing Access-Control-Allow-Origin header) due to this service worker and the user is redirected to login page; from here on all POSTs do not work for the same reason.
Conversely, if I login, unregister the service worker and then refresh, there are no problems at all. Any idea of what's going on? Again my service worker just handles push events, no caching no other processing done and it perfectly works on Chrome.
Here's my service worker code ( SOME_API_URL points to a real API which is not needed for testing purpose cause the issue happens after the service worker registers, no push events needed)
self.addEventListener('push', function(event) {
// Since there is no payload data with the first version
// of push messages, we'll grab some data from
// an API and use it to populate a notification
event.waitUntil(
fetch(SOME_API_URL).then(function(response) {
if (response.status !== 200) {
// Either show a message to the user explaining the error
// or enter a generic message and handle the
// onnotificationclick event to direct the user to a web page
console.log('Looks like there was a problem. Status Code: ' + response.status);
throw new Error();
}
// Examine the text in the response
return response.json().then(function(data) {
if (data.error || !data.notification) {
console.error('The API returned an error.', data.error);
throw new Error();
}
var title = data.notification.title;
var message = data.notification.message;
var icon = data.notification.icon;
var notificationTag = data.notification.tag;
return self.registration.showNotification(title, {
body: message,
icon: icon,
tag: notificationTag
});
});
}).catch(function(err) {
console.error('Unable to retrieve data', err);
var title = 'An error occurred';
var message = 'We were unable to get the information for this push message';
var notificationTag = 'notification-error';
return self.registration.showNotification(title, {
body: message,
tag: notificationTag
});
})
);
});
self.addEventListener('notificationclick', function(event) {
console.log('On notification click: ', event.notification.tag);
// Android doesn't close the notification when you click on it
// See: http://crbug.com/463146
event.notification.close();
// This looks to see if the current is already open and
// focuses if it is
event.waitUntil(
clients.matchAll({
type: 'window'
})
.then(function(clientList) {
for (var i = 0; i < clientList.length; i++) {
var client = clientList[i];
if (client.url == '/' && 'focus' in client)
return client.focus();
}
if (clients.openWindow) {
return clients.openWindow('/');
}
})
);
});
Firefox 44 has bug 1243453, which causes the Origin header of cross-origin requests to get dropped if the service worker doesn't listen for fetch events.
The bug has been fixed in Firefox 45, which will be released the week of March 8, 2016 (next week, as of the time of this writing). In the meantime, and for users who don't immediately upgrade to the latest Firefox release, you can work around the problem by adding this code to the service worker:
addEventListener('fetch', function(evt) {
evt.respondWith(fetch(evt.request));
});
The way to start locomotive.js 0.3.x no longer works in 0.4.x, as the signature of app.boot is different. I have:
before(function(done) {
var self = this;
var lcm = new locomotive.Locomotive();
lcm.boot('test', function() {
self.app = lcm.express;
self.app.listen(4000, done);
});
});
It throws "Error: connect ECONNREFUSED" when I tried to connect with supertest:
request(this.app)
.post('problems/' + problemId + '/solution_ratings')
.set('Content-Type', 'application/json')
.send({access_token: playerId, solution_group_id: 1, rate: 4})
.expect(200, done);
What is the proper way to start a locomotive server for functional testing?
[Update]
I have to start the server in the same process as the test in order to use sinon.js to stub/spy calls to models.
I've answered this on Locomotive's github for an issue, however I wanted to share here to get some feedback, as in better ways, cleaner, or any other input you might have.
I require multiple servers in an OAuth2 environment (auth, resource, dashboard..) of which mostly are Locomotive framework based.
I like functional tests since they best represent OAuth2 based authentication as opposed to only focusing my tests on a resource service where the authentication token might not best represent the user.
Here is the setup I've come up with for starting up the locomotive servers:
For your tests, say like in test/util.severs.js
var fork = require('child_process').fork,
async = require("async");
var authApp;
var serverStatus = [];
var start = function(done) {
authApp = fork( __dirname + '/../../authorization/server.js');
serverStatus.push(function(callback){
authApp.on('message', function(m){
//console.log('[AUTHORIZATION] SENT MESSAGE: ', m);
if (m.status == 'listening') return callback();
});
});
// add others servers if you swing that way!
async.parallel(serverStatus, function(){
done();
});
}
var stop = function(done) {
authApp.kill();
done();
}
module.exports = {
start: start,
stop: stop
};
Note that I'm using async here since I'm working with multiple servers (Environment is in OAuth2 which requires many servers to startup (IE resource, dashboard...)), ignore async if all you'll ever have is one server.
require the above file in your Mocha tests and do
servers = require('./util/servers');
...
before(servers.start);
// tests away!!!
after(servers.stop);
Then in each of my locomotive-project/server.js I do the following...
// if being started as a parent process, send() won't exist, simply define
// it as console.log or what ever other thing you do with logging stuff.
if (process.send === undefined){
process.send = function(msg){
console.log("status: ", msg.status);
};
}
process.send({status: 'starting'});
...
app.boot(function(err) {
if (err) {
process.send({status: 'error', message: err});
return process.exit(-1);
} else {
// notify any parents (fork()) that we are ready to start processing requests
process.send({status: 'listening'});
}
});
Using Locomotive 0.4.x here, could be different based on your version.
Is this the best way to go? Ideally I would like to move this over to Grunt, that is test server initialization (which is quite hefty as we build lots of data into the test DB), and then functional tests could run. So wondering if anyone has delved into grunt to do this instead of a before() task in Mocha.
Heroku is not detecting when a client disconnects.
I cloned their starter app and added a couple lines for sse:
var sse = require('connect-sse')();
app.get('/testing', sse, function(req, res, next) {
var interval;
interval = setInterval(function() {
var date;
date = new Date();
res.json({
ping: date
});
return console.log(date);
}, 2000);
return req.on('close', function() {
clearInterval(interval);
return console.log("cleared");
});
});
When I go to my app I see the correct reponse:
id: 0
data: {"ping":"2014-01-05T23:53:49.835Z"}
id: 1
data: {"ping":"2014-01-05T23:53:51.839Z"}
But after I close the browser tab heroku does not register that I did.
With heroku logs -t I keep getting the date printed out and never cleared
This works locally with foreman start but not on heroku.
Anyone know what I am doing wrong?
Based on the documentation for the connect-sse node module, it looks like you should be referring to the sse middleware without parentheses, like so:
app.get('/testing', sse, function(req, res, next) {
// ...
});