I've haven't looked at SonarQube for some time. The latest version looks quite interesting.
The last time I checked out this product, they had a site where they had pushed various popular open source libraries (like Tomcat, ActiveMQ etc) through the SonarQube process. It was a great way of seeing how the product works.
Is this facility still available for the latest versions?
Have a look at http://sonarcloud.io (and click on "browse").
It is not a demo site - it is a globally available SonarQube server, managed by SonarSource (the company behind SonarQube). Many huge open source projects are managed there for free, but you can also host your own, private projects for closed source.
It gets updated every couple of weeks and for public releases.
Besides that, there is http://next.sonarqube.com , the publicly visible SonarQube server for SonarQube itself and related projects (yes: SonarQube for SonarQube).
This instance gets updated every couple of days. If you want to have a look at what features SonarQube's next version will look like: have a look there.
Related
I am looking for a tool to manage the collection of binary files (input components) that make up a software release. This is a software product and we have released multiple versions each year for the last 20 years. The details and types of files may vary, but this is something many software teams need to manage.
What's a Software Release made of?
A mixture of files go into our software releases, including:
Windows executables/binaries (40 DLLs and 30+ EXE files).
Scripts used by the installer to create a database
API assemblies for various platforms (.NET, ActiveX, and Java)
Documentation files (HTML, PDF, CHM)
Source code for example applications
The full collected files for a single version of the release are about 90MB. Most are built from source code, but some are 3rd party.
Manual Process
Long ago we managed this manually.
When starting each new release the files used to build the last release would be copied to a new folder on a shared drive.
The developers would manually add or update files in this folder (hoping nothing was lost or deleted accidentally).
The software installer script would be compiled using the files in this folder to produce a SETUP.EXE (output).
Iterate steps 2 and 3 during validation & testing until release.
Automatic Process
Some years ago we adopted CI (building our binaries nightly or on-demand).
We resorted to putting 3rd party binaries under version control since they usually don't change as often.
Then we automated the process of collecting & updating files for a release based on the CI build outputs. Finally we were able to automate the construction of our SETUP.EXE.
Remaining Gaps
Great so far, but this leaves us with two problems:
Rebuilding Assemblies The CI mostly builds projects when something has changed, but when forced it will re-compile a binary that doesn't have any code change. The output is a fresh build of a binary we've previously tested (hint: should we always trust these are equivalent?).
Latest vs Stable Mostly our CI machine builds the latest versions of each project. In some cases this is ok, but often we want to release an older tested or stable version. To do this we have separate CI projects for the latest and stable builds - this works but is clumsy.
Thanks for your patience if you've got this far :-)
I Still Haven't Found What I'm Looking For
After some time searching for solutions it seems it might be easier to build our own solution, but surely someone else has solved these problems before!?
What we want is a way to store and manage binary files (either outputs from CI, or 3rd party files) such that each is tagged with a version (v1.2.3.4) that allows:
The CI to publish new versions of each binary (but reject rebuilt versions that already exist).
The development team to make a recipe for a software release (kinda like NuGet packages.config) that specifies components to include:
package name
version
path/destination in the release folder
The Automatic package script to use the recipe collect the required files, and compile the install package (e.g. SETUP.EXE).
I am aware of past debates about storing binaries in a VCS. For now I am looking for a better solution. That approach does not appear ideal for long-term ongoing use (e.g. how to prune old binaries)... amongst other issues.
I have tried some artifact repositories currently available. From my investigation these provide a solution for component/artifact storage and version control. However they do not provide tools for managing a list of components/artifacts to include in a software release.
Does anybody out there know of tools for this?
Have you found a way to get your CI infrastructure to address these remaining issues?
If you're using an artifact repository to solve this problem, how do you manage and automate the process?
This is a very broad topic, but it sounds like you want a release management tool (e.g. BuildMaster, developed by my company Inedo), possibly in conjunction with a package management server like ProGet (which you tagged, and is how I discovered this question).
To address some specific questions you have, I'll associate it with a feature that would solve the problem:
A mixture of files go into our software releases, including...
This is handled in BuildMaster with artifacts. This video gives a basic overview of how they are manually added to releases and deployed to a file system: https://inedo.com/support/tutorials/buildmaster/deployments/deploying-a-simple-web-app-to-iis
Of course, once that works to satisfaction, you can automate the import of artifacts from your existing CI tool, create them from a BuildMaster deployment plan itself, pull them from your package server, whatever. Down the line you can also have your CI tool call the BuildMaster release management API to create a release and automatically have it include all the artifacts and components you want (this is what most of our customers do now, i.e. have a build step in TeamCity create a release from a template).
Rebuilding Assemblies ... The output is a fresh build of a binary we've previously tested (hint: should we always trust these are equivalent?)
You can mostly assume they are equivalent functionally, but it's only the times that they are not that problems arise. This is especially true with package managers that do not lock dependencies to specific version numbers (i.e. NuGet, npm). You should be releasing exactly the same binary that was tested in previous environments.
[we want] the development team to make a recipe for a software release (kinda like NuGet packages.config) that specifies components to include:
This is handled with releases. A developer can choose its name, dates, etc., and associate it with a pipeline (i.e. a set of testing stages that the artifacts are deployed to), then can "click the deploy button" and have the automation do all the work.
Releases are grouped by "application", similar to a project in TeamCity. As a more advanced use case, you can use deployables. Deployables are essentially individual components of an application you include in a release; in your case the "Documentation" could be a deployable, and maybe contain an artifact of the .pdf and .docx files. Deployables from other applications (maybe a different team is responsible for them, or whatever) can then be referenced and "included" in a release, or you can reference ones from a past release.
Hopefully that provides some overview and fits your needs. Getting into this space is a bit overwhelming because there are so many terms, technologies, and methodologies, but my advice is to start simple and then slowly build upon it, e.g.:
deploy a single, manually uploaded component through BuildMaster to a share drive, then manually deploy it from there
add a deployment plan that imports the component
add a second plan and associate it with the 2nd stage that takes the uploaded artifact and deploys it to the target, bypassing the need for the share drive
add more deployment plans and associate them with pipeline stages and promote through them all to "close out" a release
add an agent and deploy to that instead of the default localhost server
add more components and segregate their deployment with deployables
add event listeners to email team members at points in the process
start adding approvals if you require gated "sign-offs"
and so on.
In the old version, there was a dashboard for the whole project from different views, but in the latest version there isn't. Why was this dashboard removed?
The short answer is that rather than making you figure out which measures are most important, and making you figure out how to display them, recent versions of SonarQube handle the hard work for you with a standard, non-customizable project homepage, and the new Projects space.
I'm not able to use Notepad++'s update or plugin manager. I suspected it was a proxy problem, so I added my proxy to the plugin-manager settings but it still doesn't work.
Then I contacted my network admin and he said that he needs the server's URL to add it in the exception list. How can I find to what server it's trying to connect to?
What IP/URL does Notepad++'s plugin manager connect to for updates?
According to the project's creator, Notepad++ Plugin Manager's update URL is hosted on nppxml.bruderste.in
Hosting was moved from *.sourceforge.net to nppxml.bruderste.in in October 2016 due to a controversy where Sourceforge began bundling adware with installers for abandoned projects.
New Plugin Manager URL: https://nppxml.bruderste.in/pm/xml/plugins.zip
From the sponsor page for NP++ Plugin Manager
Since 2010, Plugin Manager has been hosted on Sourceforge.net. They've
been a great host for us, and generously supported the large amount of
traffic that the plugin manager generates. To put some numbers on the
traffic, whilst the plugin list is only 50KB, and is normally only
download when it changes, each change generates around 500GB of
traffic, with a steady 200GB of traffic each month.
In 2015 there was some controversy surrounding SourceForge bundling
adware with abandoned projects. This has since been cleared up, and we
applaud SourceForge for taking the right path in this regard. However,
this sparked some discussion and thinking around what we could do to
transfer the hosting of the list off SourceForge and onto another
provider, and what benefits and disadvantages that would bring. There
are things we aren't able to do easily with the sourceforge hosting
system, and relying on the goodwill of a website is not good in the
long run for the project. Using a more traditional hosting platform
would allow us to address some of the current issues we have around
automating the list updates, and serving a dynamic "bleeding edge"
list.
So, I'm very grateful to Nexinto that they are sponsoring the hosting and bandwidth requirements for Notepad++ Plugin Manager, and hope you can forgive the small logo and link in the plugin which means we are reliably hosted on VMs that we control, and hence can extend what we do, bringing automatic fortnightly releases of new plugins to the plugin manager.
For reference, the old url was https://downloads.sourceforge.net/project/npppluginmgr/xml/plugins.zip
If you are not sure if problem is proxy but plugin list is no displayed, try the next.
From "Plugin Manager, Settings", select checkbox "Use development plugin list (may contain untested, unvalidated or uninstallable plugins)
At the moment for v7.5 and v7.6:
Proxy setting are no displayed from Plugins Manager interface
Plugins server is http://nppxml.bruderste.in
Downloaded files by Plugin Manager:
http://nppxml.bruderste.in/pm/xml/plugins.zip
http://nppxml.bruderste.in/pm/xml/plugins2.md5.txt
TeamCity appears to store the definitions for builds, projects, templates etc as XML internally.
This is exposed in the "Administration > Audit" view where you can see diffs that people made to individual configurations, at URLs like http://teamcityserver/admin/settingsDiffView.html?id=project:project10&versionBefore=8&versionAfter=9&actionId=3151
I'd like to manage a TeamCity setup partially from outside the web interface - e.g. for example keep the build definitions in version control and perhaps programmatically generate them.
Is there any way I can directly upload definitions in this format (or any similar alternative)? I'm aware that there are various APIs and extension points to TeamCity but haven't managed to find any that gives direct access to anything like this.
I can live with the format changing with TeamCity versions if necessary - it would be a reasonable price to pay for the other benefits.
For TeamCity 9.x and newer
As reported by Ganesh in the comments to this answer, an option was added in 9.x that supports changes and versioning through Source Code Management (SCM) tools. Please see his answer for 9.x and beyond.
For TeamCity 8.x and older
It might not be the "approved" way, but you can edit the project files on disk, and those changes will appear in your build configs. I have successfully edited them outside of the Web UI after they were created.
So, you could probably open that folder up as a restricted network share or set up ssh.
You'll find it at $TeamCityData/config/projects/ and then they are stored in subfolders such as $projectName/buildTypes/$buildFile.xml
An example is:
E:\TeamCityData\config\projects\CSandbox\buildTypes\CSandbox_Project1TrunkBuildUnitTest.xml
TeamCity 9 adds a new "Versioned Settings" feature which keeps these XML files under version control and allows changes to be made via the VCS.
In TeamCity 9.0 this can be git or mercurial, and the upcoming TeamCity 9.1 will add support for Perforce and Subversion.
I've been using it with git for a few months and it works quite nicely in practice.
I sometimes have trouble persuading TeamCity to notice changes coming in from the VCS - particularly when deleting projects - but otherwise it's been really useful for standardising configuration and spinning up new job chains quickly.
Another slight annoyance is that you can't configure the location within the repository that the settings come from - it's always .teamcity in the root - so I've had to use multiple branches or repositories to manage multiple TeamCity servers.
We have something about 20 people in our team and we are using sonar for now to analyse new code before submiting it to the main stream. So each designer uses it's own Sonar installed on his machine.
What I'm trying to do is to create a one instance of the Sonar which each designer will be able to use. The only concern I have is what will happen if:
One designer will launch analysis on one revision of file and right after that the second designer will launch analysis on another revision of this file (in the worst case we can have a bunch of such a files). First designer won't be able to see his violations and won't be able to see code he wrote at all. Do we have some mechanism to overcome this?
What will happen if two designers will analyse the same project at the same time? AFAIK, Sonar won't allow them to do so. Any workaround for this?
Of course, we can, somaehow, create a project on the sonar side for each team member, but this has it's drawbacks, such as issues, marked as false positive in one proect won't appear as such an issues in another project and so on.
Any ideas on such an issues?
What you probably want to set up is:
a central Sonar instance that analyses the code base on a regular basis (for instance every day) based on the code located in the repository. This instance should be the reference and the project manager(s) will use it to monitor the project.
ask the developers to run local analyses before commiting their code:
either using Sonar Eclipse if you're coding in Java, C++ or Python. Everything is perfectly described in the documentation, more precisely the "Checking code prior to commit" section
or using the Issues Report plugin if your language is not supported yet in Sonar Eclipse.