I am seeking a code coverage tool for Mathematica to determine test coverage.
I know about the Testing Framework, only:
https://reference.wolfram.com/language/tutorial/UsingTheTestingFramework.html
You can install a paclet named "Coverage" the same way as the "Lint" paclet described here: https://mathematica.stackexchange.com/a/192690/12
Once done, restart the system and search the docs for "coverage".
Update: This paclet is now called "Instrumentation".
Related
I used to use Cypress 9 on previous projects.
By default, when running cypress open or cypress open --browser chrome used to run all tests for all React components.
However I installed Cypress 10 for the first time on a project that didn't have e2e tests yet. I added test specs, but I don't see any option to run all tests altogether.
It seems I have to run the tests one by one, clicking on each of them.
Can anyone please suggest how do I run all the tests automatically?
It's been removed in Cypress v10, here are the change notes related
During cypress open, the ability to "Run all specs" and "Run filtered specs" has been removed. Please leave feedback around the removal of this feature here. Your feedback will help us make product decisions around the future of this feature.
The feedback page to register your displeasure is here
You can create a "barrel" spec to run multiple imported specs.
I can't vouch for it working the same as v9 "Run all tests", but can't see any reason why not.
// all.spec.cy.js
import './test1.spec.cy.js' // relative paths
import './test2.spec.cy.js'
...
As #Constance says, reinstated in v11.20.
But still a very handy technique if you want to run a pre-defined subset of your tests.
In Cypress version 11.2.0 the Run All button has been reinstated.
You need to set experimentalRunAllSpecs to true in cypress.config.js.
Please see Configuration - End-to-End Testing
If Cypress Test Runner is not a must, I suggest to utilize the CLI/Node Cmd approach
You can trigger all the test(s) by npx cypress run(Still the video recording & screenshot on failed steps would be saved in the respective folders) to run all or with any other cypress flags to filter out specific spec files, or browser etc.
As per the feedback discussion there is a workaround the same as #Fody's answer that will achieve the same result as v9. Also worth noting though is the section on Continuous Integration and the Update 1 that includes a fix for preventing this workaround creating issues with the cypress run command.
Are there any current workarounds?
Yes. If you are impacted by the omission of this feature, it is possible to achieve the same level of parity as 9.x with a workaround Gleb Bahmutov explains here: https://glebbahmutov.com/blog/run-all-specs-cypress-v10/
This will still inherit the same problems as the previous implementation (which is why it was removed) but it will work in certain cases where the previous implementation was not problematic for your use case
https://github.com/cypress-io/cypress/discussions/21628#discussion-4098510
It was removed because people used it wrong.
The Test Runner is for debugging single tests. But by running all tests, then performance will quickly become a problem and crash the entire suite.
Running all tests should only be performed from the CLI.
Sources
https://github.com/cypress-io/cypress/issues/681
https://github.com/cypress-io/cypress/discussions/21628
I have a very special requirement in integration testing in go. In the past few days I have read through many blogs but I am unable to get the solution for this problem. Please share me if you have done anything below.
Infra details:
Having Go project up and running
Having my automation framework written in JAVA
My Exact requirement is:
I have to instrument the Go code and make the go code running
Execute my java automation code
Stop the Go code / get the code coverage report from Go
Thanks in advance!!
write your Go code as a test, in a file ending_test in the package directory
run the Go test as $ go test --coverprofile outfile . If it needs to run as a server then add some code to time it out
run your additional Java code
wait for the timeout
use a command like $ go tool cover -html=outfile -o cover.html to see the coverage analysis
The goc tool should meet your needs, it basically just need three steps:
launch a services registry center via goc server command
use goc build . to build your Go code and execute the generated binary which would register itself into the registry center above
Execute your automation codes, then you can use goc profile to get the code coverage profile at anytime
I found some solution which helps to analyze a single graph.
Free Open source solutions to analyze a single graph...
JMeter Plugins - look onto custom graphs in this package;
JMeter Result Analysis Plugin
JWeter tool for logs analyzing & visualization
Looking for a solution or open source tool which helps to analyze the JMeter dashboard report which having 23 graphs.
I am not sure to fully understand the question , but I'll try to answer.
Have you generated the HTML report as described here ?
https://jmeter.apache.org/usermanual/generating-dashboard.html
This reports provides the most complete information for analyzing a load test.
Is your question about generating it, in this case I pointed you to the manual.
If not, what is your issue with the HTML report , and why do you write "some of them provide better results reporting out-of-box than JMeter's original ones" ?
Since AFAIK, the HTML report provides the same information with even more graphs and dashboards.
Edit:
Regarding analysis of the report, this does not exist currently and this is usually the role of a performance/load test expert.
Maybe in the future through IA, that might exist. So for now, try to understand what each graph provides as information and correlate between them, or ask for assistance of a performance tester.
I used a tool that would help me improve test coverage by checking that the lines of the file that I am testing actually mattered to the spec.
It was done by disabling each line in the tested file and seeing if any tests failed. If no tests failed after a line was disabled, it was considered not covered.
I can't seem to find it today, maybe it was in a different language. Any ideas?
Found it, its called Mutation testing:
Mutation testing is not an alternative to line coverage. While line coverage asks “what percentage of our code is run by our tests,” mutation testing asks “what code can I change without breaking your tests?” Mutation testing tools answer this question by applying and testing small modifications to your application.
Article:
https://blog.blockscore.com/how-to-write-better-code-using-mutation-testing/
RailsConf talk: http://confreaks.tv/videos/railsconf2014-mutation-testing-with-mutant
I'm looking for a way to automate functional testing of vagrant provisioning scripts (using puppet and shell scripts). please notice: I'm asking about functional testing and not unittesting the puppet modules, the puppet catalog etc.
Is there a recommended way to do that? maybe something along python doctests: "with this input this should be the CLI output". Searched around but couldn't find a working example or a recommanded tool.
If there isn't any vagrant/puppet sanctioned solution for this, is there a simple CLI testing tool? where I can make asserts about certain outputs vs certain inputs?
I saw this related question but it didn't really get a good answer
Maybe serverspec would suite your needs. In particular, with the command resource type, which you can read about here, you should be able to do exactly what you've described.
Note that in addition you can use it to test other resources like processes or opened ports without having to deal with ad-hoc command line scripts.