To motivate ourselves to increase code coverage of unit tests, we have defined this rules for coverage:
overall code coverage must be > 80%
overall code coverage must not be less than the last time
code coverage on new code must be > 90%
To fulfill rules 1 and 2 I configured a Quality Gate with these conditions:
"Coverage - value - is less than - 80"
"Coverage - delta since previous analysis - is less than - 0"
For rule 3 I think that the following condition would meet it: "Coverage on new code - value - is less than - 90". But it is not possible to choose a value for this condition, only a delta. What is the meaning of a delta here? New code shouldn't have a delta, because it's new. How must i configure the condition to fulfil rule 3?
By default, SonarQube comes with a predefined Quality Gate that is designed to achieve exactly what you want: progressively make you increase your code coverage.
You can take a look at this "SonarQube way" quality gate on Nemo.
The important line is the one which is highlighted on the screenshot. It means: "The code introduced since the beginning of developments on the current version must be covered at least at 80%". If you admit that you are constantly refactoring and rewriting parts of your code, then ultimately your code will be covered at least at 80%.
Related
I am able to generate performance trend report using Jenkins Performance plugin. In performance report what is the significance/meaning of the exponent numbers highlighted in red color in below image. Example -2, +5698, -6657 etc. Not able to find any information related to this
The +value indicates that this build contributed to increase in the response time from the previous build by that 'value'.
Ex: In your case, for the first URI, the average response time increased by +5698 ms, similarly incase of -ve value, it means the current build is faster than the previous by that value. I was able to verify this on my performance trend report.
However i am not sure what the + and - equates to in the "samples" column. Let me know if this works!
Is it possible in SonarQube to calculate code coverage for a delta only?
For instance: a project had 1000 lines yesterday and its unit test coverage results are already in SonarQube. A new commit was pushed today with an extra 100 lines of code and additional test cases. These additional test cases cover 70 of the 100 new lines. Is there a way, possibly using TimeMachine, to retrieve/calculate the code coverage for the delta only? (in this case 70%)
You're looking for "Coverage on New Code", which is calculated against the "Leak period", i.e. the first listing in Administration > General > Differential Views.
Your problem is that differential values are calculated during analysis, so you can't update the leak period value and retroactively get exactly what you described. But narrow the leak period value down from the default 30 days (maybe previous_version?) and you'll get close going forward.
I'm using SonarQube 4.2.1 for analyzing maven projects and I'd like to fail the quality gate when the number of issues is higher than the last successful analysis (QG was green).
I'm aware of Fail SonarQube quality gate when coverage decreases but it does not cover all my needs, because the QG would be green also, when build #1 has 5 issues (with QG green), build #2 8 issues and build #3 has 7 issues, which is less than the last analysis but still more than last successful build #1.
Any idea how to do this?
You just have to add the following condition:
Metric: "New issues"
Delta since previous analysis
"Is greater than"
"0" in error
I had a couple questions about the output from a simple run of VW. I have read around the internet and the wiki sites but am still unsure about a couple of basic things.
I ran the following on the boston housing data:
vw -d housing.vm --progress 1
where the housing.vm file is set up as (partially):
and output is (partially):
Question 1:
1) Is it correct to think about the average loss column as the following steps:
a) predict zero, so the first average loss is the squared error of the first example (with the prediction as zero)
b) build a model on example 1 and predict example 2. Average the now 2 squared losses
c) build a model on example 1-2 and predict example 3. Average the now 3 squared losses
d) ...
Do this until you hit the end of the data (assuming a single pass)
2) What is the current features columns? It appears to be the number of non-zero features + an intercept. What is shown in the example, suggests that a feature is not counted if it is zero - is that true? For instance, the second record has a value of zero for 'ZN'. Does VW really look at that numeric feature as missing??
Your statements are basically correct. By default, VW does online learning, so in step c it takes the current model (weights) and updates it with the current example (rather than learning from all the previous examples again).
As you supposed, the current features column is the number of (non-zero) features for the current example. The intercept feature is included automatically, unless you specify --noconstant.
There is no difference between a missing feature and a feature with zero value. Both means that you won't update the corresponding weight.
I used 10-fold cross validation in Weka.
I know this usually means that the data is split in 10 parts, 90% training, 10% test and that this is alternated 10 times.
I am wondering on what Weka calculates the resulting AUC. Is it the average of all 10 test sets? Or (and I hope this is true), does it use a holdout test set? I can't seem to find a description of this in the weka book.
Weka averages the test results. And this is a better approach then the holdout set, I don't understand why you would hope for such approach. If you hold out the test set (of what size?) your test would not be statisticaly significant, It would only say, that for best chosen parameters on the training data you achieved some score on arbitrary small part of data. The whole point of cross validation (as the evaluation technique) is to use all the data as training and as testing in turns, so the resulting metric is approximation of the expected value of the true evaluation measure. If you use the hold out test it would not converge to expected value (at least not in a reasonable time) and what is even more important - you would have to choose another constant (how big hold out set and why?) and reduce the number of samples used for training (while cross validation has been developed due to the problem with to small datasets for both training and testing).
I performed cross validation on my own (made my own random folds and created 10 classifiers) and checked the average AUC. I also checked to see if the entire dataset was used to report the AUC (similar as to when Weka outputs a decision tree under 10-fold).
The AUC for the credit dataset with a naive Bayes classifier as found by...
10-fold weka = 0.89559
10-fold mine = 0.89509
original train = 0.90281
There is a slight discrepancy between my average AUC and Weka's, but this could be from a failure in replicating the folds (although I did try to control the seeds).