I want to use SonarQube on my project. The project is quite a big and scanning whole files take much time. Is it possible to scan only changed files in the last commit, and provide report based only on changed lines of code?
I want to check if added or modified lines make the project quality worst and I don't care about old code.
For example, if person A created a file with 9 bugs and then commited changes - the report and quality gate should show 9 bugs. Then person B edited the same file adding few lines containing 2 additional bugs, then commited changes - the report should show the 2 last bugs and quality gate should be executed on the last changes (so should consider the last 2 bugs)
I was able to narrow scan to only changed files in the last commit- but report is generated based on whole files. I had an idea about cutting only changed lines of code, paste them to new file and run sonar scan on the file - but I'm almost sure the SonarQube needs the whole context of file.
Is it possible to somehow achieve my usecase ?
No, it is impossible. I saw a lot of similar questions. These are answers to two of them:
New Code analysis only:
G Ann Campbell:
Analysis will always include all code. Why? Why take the time to
analyze all of it when only a file or two has been changed? Because
any given change can have far-reaching effects. I’ll give you two
examples:
I check in a change that deprecates a much-used method. Suddenly,
issues about the use of deprecated code should be raised all over the
project, but because I only analyzed that one file, no new issues were
raised.
I modify a much-used method to return null in some cases. Suddenly all
the methods that dereference the returned value without first
null-checking it are at risk of NullPointerExceptions. But only the
one file that I changed was analyzed, so none of those “Possible NPE”
issues are raised. Worse, they won’t be raised until after each
individual file happens to be touched.
And that’s why all files are included in each analysis.
I want sonar analysis on newly checkin code:
G Ann Campbell:
First, the SonarQube interface and default Quality Gate are designed to help you focus
on the New Code Period. You can’t keep analysis from picking up those
old issues, but you can decide to only pay attention to issues raised
on newly-changed code. That means you would essentially ignore the
issues on the left side of the project homepage with a white
background and focus instead on the New Code values over the yellow
background on the right. We call this Fixing the Leak, or
alternately Clean as You Code.
Second, if you have a commercial edition, then branch and PR analysis
are available to you. With Short-Lived Branch (SLB) and PR analysis
still covers all files, but all that’s reported in the UI is what’s
changed in the PR / SLB.
Ideally, you’ll combine both of these things to make sure your new
code stays clean.
The position in this matter has not changed over the last years, so don't expect it will be changed.
Related
That's basically the idea. I own a project and I want to break any new build on TeamCity based on a code coverage percentage. As simply as: this percentage can never go down. This way I ensure that new commits are covered.
TeamCity provides this out of the box. Simply go to the configuration for the project, and click 'Failure Conditions'. This gives you a place whwre you can add a failure condition on a metric change. One of the available metric changes is 'Percentage of line coverage'. You can set it so that the build fails if this is less the 0 difference from the last build.
Beware adding this though, especially if you have projects where the code coverage is not 100% already, as a refactoring which reduces the number of lines in the project and all of those lines happen to be covered by tests will result on the overall coverage going down, and a failing build despite not adding any new functionality.
Now that the Cutoff date plugin for Sonar is deprecated (I've tried it, and it doesn't seem to work at all), is there any way to exclude issues based on a set date?
For a large project, it is desirable to really start at fresh at given point.
Maintaining a low alert threshold ( < 100 alerts ) is much more manageable for the developers than cluttering the issues listings with old history/low priority issues. (1500++). Identifying the new and relevant ones are then much harder.
What we want to focus on are new issues.
Example:
when you have changed the quality rules for say 200.000+ lines of code, you really only are interested in what is produced from now on and changes to already existing code that breaks the new rules.
If my feeling is correct, you're not aware of the way to work with differential periods in SonarQube. Because out-of-the-box SonarQube is designed to cover such common use case. See :
http://www.sonarqube.org/differentials-four-ways-to-see-whats-changed/
http://www.sonarqube.org/using-differentials-to-move-the-team-in-the-right-direction/
http://www.sonarqube.org/differentials-but-wait-theres-more/
Is there a trick to localize syntax errors? I occasionally run into this situation when editing existing code. Here's a latest example, do you see an efficient way to find the error? (I found the error already, but not efficiently)
http://yaroslavvb.com/upload/save/syntax-error.nb
May be, I am missing something, but in your example it was quite easy: once I put your code into a notebook and tried to run, the usual orange bracket appeared, and when you expand the messages, it states very clearly that the problem was in an empty parentheses in your vertexFormula function. In my experience, most of the time, the orange box gives enough hints.
Another great way which I use on a daily basis it through the code highlighting in Workbench. It immediately highlights syntax errors, plus you have a very powerful Eclipse-based navigation both for a single package and multiple packages. It might seem as you lose some flexibility by going to Workbench from an interactive FrontEnd development, but I found the opposite (or may be this is the revenge of my enterprise Java background): you still can have your notebook(s) in a Workbench project, where you do the initial development, but then they get attached to the project and a number of packages that you already developed and use. The transition from notebook to a package could not be easier, since you can continue use that notebook after you transfer the code into a package, and you don't even have to worry about loading the package, as long as you do it just once inside a project. Overall, I find the Workbench - based development much more fun as soon as your project goes over some critical mass (I'd say, perhaps around 1000 loc, but ymmv). But, complete new and independent chunks of functionality I still prefer to prototype entirely in the FrontEnd.
If you adhere to some specific coding standards in your code, or work with code which does, you might be able to develop some simple partial parser which at least breaks the code into complete chunks (function defenitions, Module-s etc, CompoundExpressions). Then, you could use ToExpression (say, Map it on a list of string code chunks returned by your partial parser), to see which piece of code is problematic (it will return $Failed on it). But if you use Workbench, this is unnecessary altogether.
Do you intermingle refactoring changes with feature development/bug fixing changes, or do you keep them separate? Large scale refactorings or reformatting of code that can be performed with a tool like Resharper should be kept separate from feature work or bug fixes because it is difficult to do a diff between revisions and see the real changes to code in amongst the numerous refactoring changes. Is this a good idea?
When I remember, I like to check in after a refactoring in preparation for adding a feature. Generally it leaves the code in a better state but without a change in behaviour. If I decide to back out the feature, I can always keep the better structured code.
Keep it simple.
Every check in should be a distinct, single, incremental change to the codebase.
This makes it much easier to track changes and understand what happened to the code, especially when you discover that an obscure bug appeared somewhere on the 11th of last month. Trying to find a one-line change amidst a 300-file refactoring checkin really, really sucks.
Typically, I check in when I have done some unit of work, and the code is back to compiling/unit tests are green. That may include refactorings. I would say that the best practice would be to try to separate them out. I find that to be difficult to do with my workflow.
I agree with the earlier responses. When in doubt, split your changes up into multiple commits. If you don't want to clutter the change history with lots of little changes (and have your revisions appear as one atomic change), perform these changes in a side branch where you can split them up. It's a lot easier to read the diffs later (and be reassured that nothing was inadvertently broken) if each change is clear and understandable.
Don't change functionality at the same time you are fixing the formatting. If you change the meaning of a conditional so that a whole bunch of code can be outdented, change the logic in one change and perform the outdent in a subsequent change. And be explicit with your commit messages.
If the source code control system allows it..
(this does not work in my current job due to the source code control system not liking a single user checking out a single file to more than one location.)
I have two working folders
Both folders are checkout from the same branch
I use one folder to implement the new feature development/bug fixing changes
In the other folder I do the refactoring,
After each refactoring I check in the refactoring folder
Then update the new feature development folder that merges in my refactorings
Hence each refactoring is in own checkin and other developers get the refactoring quickly, so there are less merge problems.
I have two tables, pages and revisions. Revisions has a foreign key to a page. The contents of a page is the latest entry in the revisions table for that page. The revisions are full copies of the contents, no deltas.
As an experiment, I would like to visualize the revision state of the current revision. If text is new in the current revision, don't do anything. If it is from a recent revision, give it a green background color. If it's very old, give it a red background color. In between, orange. A heat map diff of the age of the content, so to speak.
My question is: How I can extract this data from the revisions of a page? Pointers to literature would be equally useful to actual code solving this problem.
Not really relevant, but just in case: It's for a Ruby project, Ruby on Rails in fact. Here's the project, on github.
Update: here's an example test case, written in Ruby. http://pastie.org/631604
Update:
[ long and slightly off-topic answer on longest-common-subsequence deleted ]
I've integrated my Hunt-McIlroy Algorithm subsequence finder with your test case, which now passes. I made various mods to your test case, see it here at pastie.org. Likewise, here is the rdiff module. Here is my svn log for why your test case was changed.
One quick way to do it is to get the successive versions of the page and run them through the diff utility to get deltas, so you know what to color how. You could of course reinvent the code that goes from two complete pages and finds which bits they have in common, but it's going to be faster to reuse existing code.
You can use svn blame command to gain similar results. Of course revisions and pages needs to be stored in svn. If migrating to svn is a roadblock, maybe examining svn sources how blame command is written might help.
Edit: #August
In order to visualize this, I need something that doesn't care about lines. Don't I?
Well, you need blame for rows and diff for contents of single row. The first one is performed by VCS, second one you can do by yourself or third party code. For every change store in database deltas of blame commends (only changed rows need to be saved). In sample results for one row we have:
Rev. num. Value
23 Hello worl
36 Hello cruel world
45 Hello wonderful world
The desired for you result I assume is (for clarity I've skiped white spaces)
Afer first diff:
(Hello)(23)(cruel)(36)(worl)(23)(d)(36)
After second diff:
(Hello)(23)(wonderful)(45)(worl)(23)(d)(36)
Unified diff doesn't help in this case, so diff need to be done otherwise. You can write algoritm for diff yourself or find appropiate code in merge tools. Below is example of how TortiseMerge does the stuff.
tortise merge http://img169.imageshack.us/img169/7871/merge.png
The problem isn't simple one but I think that my ideas might help you a little or give any clues.
One thing. Heat implies activity or energy, so I would flip your colors around so that
the most recent are red (hot) and the older text is blue/green (cooled off).
You can use any DVCS to achieve that. I'd recommend git. It will be even better than using db.