In the documentation of the SonarQube issue lifecycle (https://docs.sonarqube.org/display/SONAR/Issue+Lifecycle) one of the two possible resolutions is:
Removed - set automatically when either the related coding rule or the file is no longer available. The rule may not be available either because it has been removed from the profile or because the underlying plugin has been uninstalled. The file could be unavailable because it has been removed from the project, moved to a different location or renamed.
Even after analyzing dozens of commits of an open-source project, in which files were clearly renamed and moved, the Removed count is still zero.
Does anybody understand why this is? Shouldn't the counter increase?
SonarQube detects file move, so when E.G. A.java is moved to B.java, its issues will move with it.
Removed, as described in the docs, is used when the rule no longer applies. Try removing a high issue count rule from your profile and reanalyzing; you should see the Removed number increase.
Related
One of the brilliant aspects of Firely.Terminal is its ability to interoperate with the local FHIR package cache (~/.fhir) in a way that is fully compatible with HAPI tools using the cache. Sadly, that no longer seems to be the case.
Today I updated Firely.Terminal to version 2.4.2 and it seems that the new version walks all over the FHIR package cache, changing files without having been asked to.
It used to be that the only thing Firely.Terminal changed in existing packages was the generation of a missing .index.json. For newly installed packages, the only difference to a HAPI-installed package was some additional fields in .index.json (presence of some fields containing null which would normally be suppressed, and the addition of a fhirVersion field).
When the new Firely.Terminal is told to add a package to a scope (fhir install) it automatically 'bakes' it, which seems to involve things like snapshotting all StructureDefinition resources and expanding all ValueSet resources. Even resources whose content remains unscathed get their timestamps trashed. The same fate befalls all packages that are listed as dependencies in the manifest of the package being added to the scope.
There is an 'unbake' command (e.g. fhir unbake --package kbv.ita.for#1.0.1) but this does not operate recursively. What's more, when it says 'Bake successfully removed from KBV.ITA.FOR#1.0.1' (note the erroneous capitalisation) then that is an outright lie - the contents of the package directory are completely unchanged, except for the removal of the file .bake.json.
Hence the only way of restoring the package cache to working order is to identify all trashed packages, delete them all, and then reference them with some HAPI tool in order to get them re-cached.
I wouldn't mind so much if Firely.Terminal trashed its own cache. But what it destroys is the global HAPI package cache for the current user, and that is simply not acceptable.
Is there any way of suppressing the destructive behaviour of Firely.Terminal? Ideally globally (with machine-wide effect), but a secret command switch would do in a pinch. If that is not possible: does anyone know which of the older versions is the newest that still works, and where to get it?
Note: if the cached packages are write-protected then Firely.Terminal doesn't take the hint - it tries to clobber the files anyway and spews out oodles of 'access denied' messages. What's more, it doesn't even stop when an error occurs; instead it continues on its merry way and trashes everything that one might have forgotten to write-protect.
Background: one of the properties of the FHIR package cache that is important for our work is that the files in the cache are exactly the same as those in the (normative) published packages. In particular, we need profiles published without snapshots to not contain snapshots, value sets published without expansions to not contain expansions and so on. For one thing, this makes it possible to verify that the cached files are exactly the same as those contained in the published packages (or fixed versions thereof). For another, we need to control the context in which profiles are snapshotted, value sets expanded and so on because it may be necessary to supply dependencies that are different from those declared in the package manifest. The latter is sometimes necessary because the profile/package version management in the context of electronic prescriptions in Germany is a bit, erm, peculiar and can diverge from FHIR standards. For this to work at all the resources must be snapshotted/expanded dynamically (depending on the use context), not statically on disk. Things are moving in a more standards-compliant direction but we are not quite there yet.
Latest version without bake (on install)
From some quick testing of the latest versions of Firely Terminal, it seems 2.2.0 is the latest without bake functionality (and auto-bake on install). Installation instructions:
> dotnet tool uninstall --global Firely.Terminal
> dotnet tool install --global Firely.Terminal --version 2.2.0
Baking
The bake functionality has been introduced to provide packages with snapshots, because not all downstream tooling (most notably sushi) are able to generate these themselves.
Currently bake might be a little too aggressive by default, also recalculating snapshots for packages that already have them. In principle, this should not be a problem since snapshots are a just a cache for the calculation of all the layered differentials. Since snapshot logic still evolves it might even be desirable now and then to recalculate. But in newer versions we will look to:
Change the default to not recalculate when already provided
Provide a global setting to change that default to never calculate/always (re)calculate snapshots
This should prevent Firely Terminal from touching any files that don't need touching in the package cache. I'm not sure from your question if there was anything broken in the state of the shared FHIR cache after 'baking', given your use of 'thrashing' and 'destroying'?
Unbaking
The unbake command is intended to remove snapshots from a folder of packages. I see in my testing that it's not doing that, which I'll take as an issue to fix.
I've got a copy of sonar-scanner using a plugin that's still early in its development life. At the moment it seems to have some trouble with some particular files that prevent the scan from completing.
I'd like to build a list of files that should be skipped over as part of the scan process so that I can at least scan the rest of our code. Is there any way I can can have sonar.exclusions read from a file. I'd prefer this because:
The list is expected to be quite long, and I'm worried about exceeding a length limitation for sonar.exclusions
It would be ideal to be able to easily add to this list as needed (so a file I can just append to would be preferable).
(And before anyone asks, I'm already working with the vendor to fix the bug in the affected plugin - I just want a way to proceed with implementing SonarQube so I can proceed with getting the analysis pipeline setup while they're working on the fix).
Ok, so my "solution" is only a partial solution, I still have to add new files manually when I encounter an issue that breaks the plugin.
But it also turns out that sonar.exclusions doesn't appear to have a length limit (or, if it does, it's very generous).
Re the default quality gate, strangely, we are unclear of the definition “new code”!
To illustrate, let’s say we change a file by adding new code. Is default sonar quality gate analysis done on only the new lines of code or the whole file?
We are unclear but suspect it is the whole file! I’m being told by colleagues that projects are failing quality gate because files with pre-existing blockers etc. were touched/changed.
Any clarification would be much appreciated.
First, analysis will scan every line of every file.
Let's say
I'm using a recent version of SonarQube
I've set the leak period (this can be configured at the global and project levels) to 30 days
That means that any line of code added or updated within the last 30 days is considered "new" and thus, "in the leak period".
If I make a commit that adds a bug, it's marked as a bug in "new code".
If I change a line with an existing bug but don't fix the bug (Why???) then I have an "old" bug on "new" code. Since the assumption is that you'll "clean as you code" (including fixing the old issues in the code you're working on) no work has been put in to "properly" handling this case.
To define the New Code Period globally go to Administration -> Configuration -> General Settings -> New Code Period:
For project specific settings, go to Administration -> New Code Period on the project:
The SonarQube documentation explains the two modes Previous Version and Number of days.
I am using TFS 2012 express. When i try to use get latest version I get so many conflicts(more than 250). When I compare it i found that almost most of the conflicts are due to Case differences. Some times i do i get valid conflicts and which can be manipulated using merge tool. But comparing this lot number of conflicts(more than 250) is very difficult to me. For example in below image you can see changes like MsfgTemp and MSFgtemp(this kind of case differences) . I do get lot of this kind of differences(one more i got is Val and val) . No one has made any changes to those lines actually. But still conflicts are shown. Why this kind of conflicts I get during get latest version? How to solve this?
EDIT:
I have found some thing interesting after some research on it. I did several steps to find reason for it. When i was doing it no other user in my team was using TFS or changing anything.
The server has MsfgTemp(in so many lines of code).
I created new workspace and without changing anything compared with server version. As expected no difference found .
Now I changed some lines of code and saved the project.
Now when I see difference I found lot of differences. All changes are like MsfgTemp and MSFgtemp. Another was val and Val .
So it is sure that the reason is not some one changed or anything else. While saving project something causes these changes. My project is of VB6. I edit projects using Microsoft Visual basic 6.0 only.
So now what may be the reason for this kind of difference and how it can be solved?
You should check the TFS file history or use the annotate option to see who changed those lines and when they were changed.
Since TFS only stores the files it doesn't change the contents you should first find out who checked in the changes and that will lead you to the how and why it changed.
NOTE: from looking at the little bit of code you posted in the example the same variable name is used two lines above in both files without conflict.
TempVar = MSFgtemp.TextMatrix(i, TaxDeductedCol)
the conflict your are noticing just two lines later is for a very similarly named variable but I believe this one is different since it is declared inside the loop, so this might be a bug fix with a poorly named temp variable inside the loop...
TempVar = MsfgTemp.TextMatrix(i, ArrAddColNum(k))
I hope this qualifies as programming related since it involves how to structure a project.
Because I've always used the web site model with VS.net I never had solution and project files and putting everything into source control worked great. I knew that everything I had in my web site directory was all I needed for the web site.
Now I'm using asp.net MVC and it only has a project model so now I have these solution and project files. If I work on it alone it's fine but once other people start to add/delete files from the project our solution file gets messed up and people end up having to grab the latest solution file, see what got changed and then add back/remove their files and check in the solution file again. It's become sort of a problem because sometimes people don't realize the solution file was changed, they make other changes and then when they check in everything other people do an update on their files they find that their files are gone from the project (although still physically on disk).
Is this normal? Is there a way to structure a project so that we don't need to check in solution and project files?
Your developers are not using TFS correctly. You should have multiple check-outs turned on, and everyone needs to be careful to merge their changes correctly when checking in. TFS will prompt you to do this, and accepting the defaults is nearly always the right thing to do.
It's not uncommon to have one or two developers who never get it, and you might have to help them now and then. But every programmer who works on a team needs to learn how to use source control tools correctly. If they can't manage that, they shouldn't be writing software.
[edit] It occurs to me that you might run into these problems if you check in the *.sln file directly, rather than choosing to "Add Solution to Source Control".
I don't think it's normal - what are you using for source control? It sounds like developers aren't respecting changes that others a making - checking in without merging first.
I know that early on in a project, when lots of files are being added & deleted, it can be a problem to keep up - you need to check out the project file, add your files, then check in the new file & project so other developers can also update it. You'll probably have multiple project files in a solution - perhaps one interim solution would be to have one "holding" project for each developer, then clean them up periodically - though these types of temporary fixes do have a tendency to become permanent.
I don't know of a way to set up a project file that's not in source control, though I suppose you could create a script that would generate them.
Having been through this, the key is respect & good communication between the developers.
This tends to happen with TFS multiple check outs. It can be hard to grasp coming from VSS to TFS as VSS allowed one person to check a file out at one time. Auto-merge should work most of the time for you but a couple of rules should ease the pain:
Check in early and often (if you add remove or rename a file check it in straight away even if it is a blank holder)
Before you check in do a get latest, this will ask you to resolve conflicts locally
Try to get continuous integration set up so that developers always know the state of the buidl and whether it is OK to check in\out.
We had a bit fo pain at the start of our current project but it soon settled down when we followed the rules above.
Personally, I think making changes to project and solution files requires discipline and clear (well understood) rules throughout your development team. These files (.sln, .*proj) are the bottlenecks of your project, and any errors or inconsistencies can cost you in team downtime. Changes need to be well thought out, planned and then executed.
They must be secured by source control (which you're already using, excellent) and your team members should work on the basis of only making the changes they need, and not leaving project or solution files checked out for an extended period.
If you are allowing multiple (shared) checkouts, this could become problematic in terms of overwriting another user's changes. Depending on your source control mechanism, people may be required to manually merge changes. Personally, I'd ask people to negotiate their project/solution changes with each other over merging (this can't always be achieved).
A third option if you are using TFS is the shelve feature. If someone needs to make changes locally, they can shelve the changes and merge later.
Lastly, another strategy is to try to architect your solution to be as modularized as possible - so people are distributed, working on separate projects and do not (ideally) have to overlap on too many common areas.
I'm not sure if you are using TFS, as people have mentioned, but if you are (or if you are using source control with similar capabilities) you can set it such that sln and csproj files are exclusive lockouts and are not able to be merged.
We have done this with quite large teams and while it causes some initial issues as people get used to it in the long run it has resolved many issues that were previously causing problems. Essentially you trade longer term merge issues/complexity for short term compile/checkin issues which we have found to be a good trade off.
Once you have set it to forced exclusive checkout and no merge you then get your dev teams used to the fact they should keep locks on the sln and proj files for as shorter time as possible.
Always check them in.
Always check out latest (merge if possible), make sure your change is there, before checking in a new version.
If your source control doesn't require a special action to check in from an old version, GET A DIFFERENT SOURCE CONTROL.