I'd like to be able to extend the mocha test results and listen to them from the available mocha object. First, I'm looking at getting the "passes" results.
It looks like they might be subscribed to from suite but I'm not sure how...
I've tried the following which I thought would listen to the end of all of my tests:
var suite = mocha.suite.suites[0];
suite.on("end", function(e){ console.log(e, "mocha - heard the end of my test suite"); } );
My simple hack which works but isn't elegant at all - sad really:
setTimeout(function(){
var passes = $(".passes").find("em").text();
console.log("ui - heard the end of my test suite - passes: " + passes);
}, 500);
I did some more digging in mocha.js and finally discovered that mocha.run() actually returns the runner which emits all the events I was looking.
The original example I was using only had: mocha.run()
So if Mocha.run() returns a runner, then I realized that I could subscribe to it:
var runner = mocha.run();
var testsPassed = 0;
var onTestPassedHandler = function(e){
testsPassed++;
console.log("onTestPassedHandler - title: " + e.title + " - total:" + testsPassed);
};
runner.on("pass", onTestPassedHandler);
/**
* These are all the events you can subscribe to:
* - `start` execution started
* - `end` execution complete
* - `suite` (suite) test suite execution started
* - `suite end` (suite) all tests (and sub-suites) have finished
* - `test` (test) test execution started
* - `test end` (test) test completed
* - `hook` (hook) hook execution started
* - `hook end` (hook) hook complete
* - `pass` (test) test passed
* - `fail` (test, err) test failed
*/
much better!
You can also get similar events at
mocha.suite.beforeEach(function() {} )
mocha.suite.afterEach(function() {} )
mocha.suite.afterAll( function() {} )
Related
I am attempting to write a simulation that can read from a config file for a set of apis that each have a set of properties.
I read the config for n active scenarios and create requests from a CommonRequest class
Then those requests are built into scenarios from a CommonScenario
CommonScenarios have attributes that are using to create their injection profiles
That all seems to work no issue. But when I try to use the properties / CommonScenario request to build a set of Assertions it does not work as expected.
// get active scenarios from the config
val activeApiScenarios: List[String] = Utils.getStringListProperty("my.active_scenarios")
// build all active scenarios from config
var activeScenarios: Set[CommonScenario] = Set[CommonScenario]()
activeApiScenarios.foreach { scenario =>
activeScenarios += CommonScenarioBuilder()
.withRequestName(Utils.getProperty("my." + scenario + ".request_name"))
.withRegion(Utils.getProperty("my." + scenario + ".region"))
.withConstQps(Utils.getDoubleProperty("my." + scenario + ".const_qps"))
.withStartQps(Utils.getDoubleListProperty("my." + scenario + ".toth_qps").head)
.withPeakQps(Utils.getDoubleListProperty("my." + scenario + ".toth_qps")(1))
.withEndQps(Utils.getDoubleListProperty("my." + scenario + ".toth_qps")(2))
.withFeeder(Utils.getProperty("my." + scenario + ".feeder"))
.withAssertionP99(Utils.getDoubleProperty("my." + scenario + ".p99_lte_assertion"))
.build
}
// build population builder set by adding inject profile values to scenarios
var injectScenarios: Set[PopulationBuilder] = Set[PopulationBuilder]()
var assertions : Set[Assertion] = Set[Assertion]()
activeScenarios.foreach { scenario =>
// create injection profiles from CommonScenarios
injectScenarios += scenario.getCommonScenarioBuilder
.inject(nothingFor(5 seconds),
rampUsersPerSec(scenario.startQps).to(scenario.rampUpQps).during(rampOne seconds),
rampUsersPerSec(scenario.rampUpQps).to(scenario.peakQps).during(rampTwo seconds),
rampUsersPerSec(scenario.peakQps).to(scenario.rampDownQps) during (rampTwo seconds),
rampUsersPerSec(scenario.rampDownQps).to(scenario.endQps).during(rampOne seconds)).protocols(httpProtocol)
// create scenario assertions this does not work for some reason
assertions += Assertion(Details(List(scenario.requestName)), TimeTarget(ResponseTime, Percentiles(4)), Lte(scenario.assertionP99))
}
setUp(injectScenarios.toList)
.assertions(assertions)
Note scenario.requestName is straight from the build scenario
.feed(feederBuilder)
.exec(commonRequest)
I would expect the Assertions get built from their scenarios into an iterable and pass into setUp().
What I get:
When I print out everything the scenarios, injects all look good but then I print my "assertions" and get 4 assertions for the same scenario name with 4 different Lte() values. This is generalized but I configured 12 apis all with different names and Lte() values, etc.
Details(List(Request Name)) - TimeTarget(ResponseTime,Percentiles(4.0)) - Lte(500.0)
Details(List(Request Name)) - TimeTarget(ResponseTime,Percentiles(4.0)) - Lte(1500.0)
Details(List(Request Name)) - TimeTarget(ResponseTime,Percentiles(4.0)) - Lte(1000.0)
Details(List(Request Name)) - TimeTarget(ResponseTime,Percentiles(4.0)) - Lte(2000.0)
After the simulation the assertions all run like normal:
Request Name: 4th percentile of response time is less than or equal to 500.0 : false
Request Name: 4th percentile of response time is less than or equal to 1500.0 : false
Request Name: 4th percentile of response time is less than or equal to 1000.0 : false
Request Name: 4th percentile of response time is less than or equal to 2000.0 : false
Not sure what I am doing wrong when building my assertions. Is this even a valid approach? I wanted to ask for help before I abandon this for a different approach.
Disclaimer: Gatling creator here.
It should work.
Then, there are several things I'm super not found of.
assertions += Assertion(Details(List(scenario.requestName)), TimeTarget(ResponseTime, Percentiles(4)), Lte(scenario.assertionP99))
You shouldn't be using the internal AST here. You should use the DSL like you've done for the injection profile.
var assertions : Set[Assertion] = SetAssertion
activeScenarios.foreach { scenario =>
You should use map on activeScenarios (similar to Java's Stream API), not use a mutable accumulator.
val activeScenarios = activeApiScenarios.map(???)
val injectScenarios = activeScenarios.map(???)
val assertions = activeScenarios.map(???)
Also, as you seem to not be familiar with Scala, you should maybe switch to Java (supported for more than 1 year).
i tried to write a cake script for my ci. I'm new to cake.
As part of this script I wanted to execute MSpec tests.
Task("Run-Tests")
.IsDependentOn("Build")
.Does(() => {
var configurationIntoTests = configuration + "/*.Tests.dll";
MSpec("../src/ERP.BusniessLogic.Tests/bin" + configurationIntoTests);
MSpec("../src/ERP.DapperDataAccess.Tests/bin" + configurationIntoTests);
MSpec("../src/ERP.DomainModel.Tests/bin" + configurationIntoTests);
MSpec("../src/ERP.Shared.Tests/bin" + configurationIntoTests);
MSpec("../src/ERP.Web.Tests/bin" + configurationIntoTests);
});
I assumed that it would give a console output as MSBuild does, since it has no return value. See API
As you may expect there is no console output, which means I don't know what the result of the tests is.
How can i get this result to report it to my ci?
Using the MSpec(string, MSpecSettings) overload will let you set what kind of report, it's name and where to put it using the MSpecSettings class.
MSpec("../src/Progresso.ERP.BusniessLogic.Tests/bin/" + configurationIntoTests,
new MSpecSettings {
ReportName = "Progresso.ERP.BusniessLogic.Tests",
HtmlReport = true,
OutputDirectory = "./build"
});
Update
Studying your example code I notice a / is missing before configuration
var configurationIntoTests = configuration + "/*.Tests.dll";
should be
var configurationIntoTests = "/" + configuration + "/*.Tests.dll";
Otherwise i.e. bin/Debug/ becomes binDebug and the test globber will not find any assemblies and MSPec won't even be executed.
Here is my custom command:
exports.command = function (element, time, debug) {
let waitTime = time || 10000
if (debug) {
return this
.log('waiting ' + waitTime + 'ms for: ' + element)
.waitForElementVisible(element, waitTime)
}
return this
.waitForElementVisible(element, waitTime)
}
I have also set this variable in the globalModules: abortOnFailure: true.
When I call this in a pageObject though like this:
findElement() {
this.waitFor('#driversLicenseNumbers');
return this
}
The object isn't found (which is expected and intended since I'm upgrading to Nightwatch v1.0.14) and the error message is logged to the console, but the test doesn't fail.
× Timed out while waiting for element <#driversLicenseNumbers> to be
present for 10000 milliseconds. - expected "visible" but got: "not
found"
Does anyone know what I'm doing wrong here?
There is already an open issue on the Nightwatch issues board regarding this specific problem. Here it is!
This behavior affects custom_commands in nightwatch#1.0.15 & nightwatch#0.9.21 (according to the BUG report, yet I am running nightwatch#0.9.21 & this behavior is not reproducible for me).
Basically your test fails, but it does so silently, at the end of the test, where you get the timeout error.
Proposed fix: Install a different version (npm install --save-dev nightwatch#0.9.x), or a suitable version that hasn't introduced the defect yet.
Cheers!
I'm wondering if there is a way to get mocha to list all the tests that it will execute. I don't see any reasonable options when I list them using mocha --help; There are several reporters, but none appear designed to list the files that will be processed (or name the tests that will be run).
The way reporters work is by listening for events dispatched by mocha, these events are only dispatched when running a real test.
A suite contains a list of tests, so there is the info that you need. However, the suite is usually only initialized on run().
If you are ok with running mocha from nodejs instead of from the command line you can create this diy solution based on the code from Using-mocha-programmatically:
var Mocha = require('mocha'),
fs = require('fs'),
path = require('path');
// First, you need to instantiate a Mocha instance.
var mocha = new Mocha(),
testdir = 'test';
// Then, you need to use the method "addFile" on the mocha
// object for each file.
// Here is an example:
fs.readdirSync(testdir).filter(function(file){
// Only keep the .js files
return file.substr(-3) === '.js';
}).forEach(function(file){
// Use the method "addFile" to add the file to mocha
mocha.addFile(
path.join(testdir, file)
);
});
// Here is the code to list tests without running:
// call mocha to load the files (and scan them for tests)
mocha.loadFiles(function () {
// upon completion list the tests found
var count = 0;
mocha.suite.eachTest(function (t) {
count += 1;
console.log('found test (' + count + ') ' + t.title);
})
console.log('done');
});
I have something like a microtime() function at the very start of my node.js / express app.
function microtime (get_as_float) {
// Returns either a string or a float containing the current time in seconds and microseconds
//
// version: 1109.2015
// discuss at: http://phpjs.org/functions/microtime
// + original by: Paulo Freitas
// * example 1: timeStamp = microtime(true);
// * results 1: timeStamp > 1000000000 && timeStamp < 2000000000
var now = new Date().getTime() / 1000;
var s = parseInt(now, 10);
return (get_as_float) ? now : (Math.round((now - s) * 1000) / 1000) + ' ' + s;
}
The code of the actual app looks something like this:
application.post('/', function(request, response) {
t1 = microtime(true);
//code
//code
response.send(something);
console.log("Time elapsed: " + (microtime(true) - t1));
}
Time elapsed: 0.00599980354309082
My question is, does this mean that from the time a POST request hits the server to the time a response is sent out is give or take ~0.005s?
I've measured it client-side but my internet is pretty slow so I think there's some lag that has nothing to do with the application itself. What's a quick and easy way to check how quickly the requests are being processed?
Shameless plug here. I've written an agent that tracks the time usage for every Express request.
http://blog.notifymode.com/blog/2012/07/17/profiling-express-web-framwork-with-notifymode/
In fact when I first started writing the agent, I took the same approach. But I soon realized that it is not accurate. My implementation tracks the time difference between request and the response by substituting the Express router. That allowed me to add tracker functions. Feel free to give it a try.