Do you use source code analyzers? If so, which ones and for which language development?
Do you find them helpful in solving potential bugs in your code? Or are most of their warnings trivial?
After prolonged use, do you find your code quality to be higher than before?
I use a few static analysis tools in Java. FindBugs is the first line of defense, catching a lot of common errors and giving pretty useful feedback. It often spots the silly mistakes of tired programmers and doesn't place a high burden on the user.
PMD is good for a lot of other more niggly bugs, but requires a lot more configuration. You'll find that PMDs defaults are often over the top. There are too many rules that are probably beneficial on a tiny scale but ultimately don't help other programmers maintain your code. Some of the PMD rules often smack of premature optimisation.
Probably more useful is the CPD support in PMD. It attempts to find code that has been duplicated elsewhere, in order to make refactoring much easier. Run over an entire project, this really helps determine where the biggest priorities are for cleaning up code and stopping any DRY violations.
Checkstyle is also handy, making sure your coders conform to some coding style standard. it has a bit of overlap with PMD but is generally much more usable.
Finally, Cobertura is a great test coverage suite. Very handy for finding out where the unit tests are lacking, and where you should be prioritising the creation of new tests.
Oh, and I've also been testing out Jester. It seems to be pretty good for finding holes in tests, even where the code has some coverage. Not recommended yet, simply because I've not used it enough, but one to test out.
I run these tools both from within Eclipse and as part of an automated build suite.
For C, I use MEMWATCH. It's really easy to use and free.
I've used it to find many memory bugs in the past.
I used resharper and MS TS (basically FXCop) and both of them quite usefull especially in the following areas :
Identifying dead code
Wide Scope
Performance improvements (related with globalization etc.)
Recommendations are not always great but generally improved the quality of the code.
I'm a long term user of PC-Lint for C and C++ and find it very helpful. These tools are most useful when taking over a code base you are unfamilier with. Over time you hit a law of diminishing returns, where the number of new bugs you find tends to trail off.
I always still to a full project lint on a big release.
Edit: There is a nice list of relevent tools on Wikipedia here
I'm pretty happy with ReSharper. Not only does it give useful bits of information while coding (e.g. needless casts, apply readonly and so forth) but its refactoring features are excellent for rearranging the code very quickly.
It doesn't cover everything, so FxCop (or similar) is a decent addition to the toolbox. However, as Resharper gives immediate feedback, the turnaround time is really good. (I'm aware that FxCop can be run from VS, but its just not the same imo).
I find analyzers somewhat useful, i use the buildin to visual studio (ex. /analyze for c/c++ and the custom rules for .net), occasionally i use stylecop and codeitright for c# mostly for guidelines how things should be.
I don't think there is a perfect tool for everything, that finds every bug, but i think the tools help to find some bugs, not untraceable, but believe me you would spend a ton of time finding them.
Yes your code quality is SOMEWHAT better than before, but i also believe manual debugging is still needed alot. Source analyzers are not the ultimate cure they are a good medicine though. If there was a tool that you just execute it and find any kind of bugs and fixes it for you would cost millions.
Some programmers that i know swear that IBM Rational PurifyPlus is superb, but that is their opinion i just had 2-3 sessions with the tool.
But always remember one of the basic principles of programming logical errors are the hardest for find and fix, so long debugging hours are inevitable. A good code analyzer combined with unit testing may work miracles thought.
PS. i tend to produce far less errors in C# than in C++, someone may say i am wrong but although i use c++ more years than c# i find the "code it and i will take care of it" gc approach of C# far easier than c++ especially for projects you rush thing to finish at the time limit/deadline, which EVERY project is like this days...
I use StyleCop for C#. It's a great tool to keep consistent code style that leads to better code quality. Also ReSharper does some code analysis but it's pretty basic.
Related
I just finished (mostly) a major application that I've been working on for a little over a year (off and on). It is around 86k lines of code, 50k of those is from Visual Studio's auto-generated dataset. It's largely a GUI to interacting with the database, generating reports, etc. It deals with money and manages contracts so it is important for it to be as bug free as possible.
I've walked through the code, and ran the program myself. I, for the most part, cannot find more bugs. I am however, sure there are, I've just been working on the system so long I can't see them anymore. I know there are some, because of some intermittent issues I run across, but can never pinpoint.
How should I go about software testing in order to discover the remaining bugs?
I know this is a little late, but have you heard of Test-Driven Development?
There are lots of tests you could build to discover the "remaining" bugs:
1) Unit tests
2) Integration tests
3) Behavior/Business/Acceptance tests
You could always attend a Developer Testing Bootcamp to get more ideas.
You can involve some of your end users and do a beta test that way. The less experience they have with the application until now, the more likely they are to try things you didn't think of.
Since you didn't use TDD to write it. Your best bet now is to add as many automated tests as possible to cover common scenarios. That way, when you do find bugs, and there are ALWAYS bugs in programs, you can hopefully minimize the risk to the rest of the system when you fix them.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I recently started a career in software development after graduating a couple of years ago in CS. The current project I'm on is a large ongoing project that has it's origins in the 90s with a mix of C, C++, and Java. There are multiple platforms (UNIX, WIN, etc) being supported, older technologies in use like CVS, and some dated documentation in some areas.
The extent of my software development skills stem from going to university as I've had little real world experience. I felt like I had a decent foundation in CS but I cannot but help feel slightly overwhelmed by it all. I'm excited to be part of something so huge but at the same time I feel like it's a lot of information to absorb.
My coworkers have been great people and answer a lot of questions I. My employer hired me knowing that I am entry level.
I've tried poking around the source code and examining how everything gets built but it's on a scale I've never seen before.
How do more experienced people situate themselves when joining a large ongoing project? What are some common tasks you do when getting yourself up to speed?
Good question. I haven't had your exact experience, but in cases like this I like to think, "how do you eat a whale?" The answer is (predictably) "one bite at a time." Reasonable people won't expect you to grasp the whole thing immediately, but they will want to see progress. Perhaps there are some small areas of the larger project that are not too complex, without too many dependencies. Work toward understanding one of those and you're one 'bite' (and/or 'byte') closer to expertise on the whole project.
Being familiar with all existing documentation I would try to get the big picture. Literally.
generate a TreeMap of the source code
I would use GrandPerspective on Mac or WinDirStat on Windows. It will give you some insights about the structure of the project's files (sometimes it gives some hints about the code structure). Having this, you can ask your colleagues for some of the clusters, what they do, how they relate to each other.
learn how to build the project
This is important to have it compiling all the time if you are about to do any changes. Having tests executed at the build time is always a good thing, so ask for it also. Even better if there is some kind of continuous integration server in place. If there is, look at its configuration - figure out how the build is done. If there was no CI server, but you already got the knowledge how to build the project, create such a server on your local machine, and show it to your fellows - they should fell in love with it.
browse the source code with Structure101 or similar tool
This is useful especially for Java projects. This tool does great job. That will give you more details about the code structure, and sometimes about the system architecture. This experience may be sometimes hard, you may learn from this tool that a code is basically a Big Ball of Mud ;)
look for tests, and explore them
If you will be lucky there may be some JUnit, or CPPUnit tests. This is always good to try to understand what those tests are doing. It may be a good starting point to explore the code further.
My coworkers have been great people
and answer a lot of questions I. My
employer hired me knowing that I am
entry level.
You have little to worry about, you're employer knows what you are capable of and your co-workers seem eager to help you out - to be honest most developers love explaining things to others...
From what I've seen, it take truly 6+ years to become fully knowledgeable in a language, so don't expect to become a guru within a year... and even these so called gurus end up learning something new about their language everyday.
Learning a new system (large) will always take time.... the systems were usually not built in 2 weeks but over many years, so don't expect to understand it fully yet. You'll eventually discover what each part does piece by piece.
I know how you feel, because I felt like that once...
"I took a speed reading course and read 'War and Peace' in twenty minutes. It involves Russia." (Woody Allen)
I agree on what the others said before me. You need some tools that give you an overview on the code. I personally used inFusion (http://www.intooitus.com/inFusion) because it gives also other interesting data beside structure.
The method that has worked best for me is to grab a copy from source control, with the intention of throwing this version away...
Then try and refactor the code. It is even better if you can refactor the code that you know you will be working on at a later stage.
The reason this is effective is because:
refactoring gives you a goal for you to aim towards. Whereas "playing" an "breaking" the code is great - it is unfocused.
To refactor code you really have to understand the code.
Refactored code leaves code that has less concepts to retain in memory. If you don't understand a large codebase its not because you are a graduate - its because nobody can retain more than 7 (give or take a few) concepts at a time.
If you follow correct refactoring guidelines it means you will be writing tests. Although, make sure that you will be working on the modules that you are testing as writing tests can be very time consumning (although very rewarding)
Do invest in buying this book at some point:
http://www.amazon.co.uk/Refactoring-Improving-Design-Existing-Technology/dp/0201485672
But these links should get you started:
Signs that your code needs refactoring and what refacoring to use (From Refactoring - Martin Fowler)
http://industriallogic.com/papers/smellstorefactorings.pdf
A taxonomy of code smells:
http://www.soberit.hut.fi/mmantyla/BadCodeSmellsTaxonomy.htm
Good luck!!!
I agree to the first comment but I also Think that you have to learn and see the big picture in some way. You have to trace the main flow from code at least.
I was in the exact same situation several years ago when I joined a software project with 50+ ClearCase version control vobs, 5 million lines of code, and some of it dating back to the 1980's.
The first thing I did was look through every source controlled directory and made a quick summary of my best guess about what the software in that folder did and what language the code was. You can make a pretty good guess by looking at filenames and any comments or documents in those folders.
I then looked at the build scripts to see if they were readable enough to get an idea of dependencies between different parts of the code.
Finally - and I believe this was the most valuable - throw an IDE like Eclipse or NetBeans on top of the code and start reading through pieces of it. Having the ability to jump to the definition of any functions or classes using the IDE allows you to move around a massive software baseline with relative ease.
Overall, have some confidence - it is unlikely that anyone else on the project knows all of the code, so you don't need to either. Use what other people said to get a good idea of the overall project and interfaces and requirements (if they exist) and poke through the code to get an idea of the most commonly used classes and methods.
Microsoft has recently put a release of their Code Contracts framework on DevLabs with a commercial license. We're interested on using them in our project (mostly C#, some C++/CLI) to gradually replace all the custom validation code, but I'm keen to know about the experience other people have had with it before we commit to it, specifically:
Do you think the framework is sufficiently mature for large and complex commercial projects?
What problems have you run into while using it?
What benefits have you got from it?
Is it currently more pain than it's worth?
I realise that this is a somewhat subjective question as it requires opinion, but given that this framework is a very significant part of .NET 4.0 and will (potentially) change the way we all write validation code, I hope that this question will be left open to gather experience on the subject to help me make a decision to a specific, answerable question:
Should we be starting to use it next month?
Note that we do not ship a code API, only a web service one, so for the majority of code breaking compatibility in terms of the exception type thrown is not a concern. However, as I'm hoping more people than just me will benefit from this post and its answers, any detail around this area is more than welcome.
The last mature response to this was in 2009, and .NET 4 is out. I figure we're due for an update:
Code Contracts might well be mature enough for your Debug releases.
I realise this is somewhat of an upgrade from “Harmless” to “Mostly Harmless”.
The Code Contracts home page links to quite thorough documentation in PDF format. The documentation outlines usage guidelines in section 5. To summarize, you can pick how brave you feel about the Contract Tools re-writing your IL in your Release builds.
We're using the “don't rewrite my Release IL” mode.
So far, I'm most enjoying this unexpected benefit: there's less code, thus less code to test. All your guard clauses melt away.
if(arg != null) {
throw new ArgumentNullException("arg");
}
// Blank line here insisted upon by StyleCop
becomes:
Contract.Requires(arg != null);
Your functions are shorter. Your intent is clearer. And, you no longer have to write a test named ArgumentShouldNotBeNull just to reach 100% coverage.
So far, I've run into two problems:
I had a unit test which relied on a contract failure to succeed. You might argue the existence of the test was a blunder, but I wanted to document this particular prohibition in the form of a test. The test failed on my build server because I didn't have the tools installed. Solution: install the tools.
We're using two tools that rewrite IL: Code Contracts and PostSharp. They didn't get along too well. PostSharp's 2.0.8.1283 fixed the problem. I'd cautiously evaluate how any two IL-rewriting tools get along, though.
So far, the benefits are outweighing the hazards.
Addressing out-of-date concerns raised in other answers:
Code Contracts's documentation is quite thorough, though regrettably in PDF.
There's at least one Code Contract forum hosted by Microsoft.
Code Contracts Standard Edition is free if you have any VS2010 license.
.NET 4 is out. I've run into Microsoft's contracts when implementing generic collection interfaces.
I've been playing around with the code contracts some more myself on a small but moderately complex standalone project, which needs to inherit from some BCL classes and use other ones.
The contracts thing seems great when you're working in a completely isolated environment with just your own code and primitive types, but as soon as you start using BCL classes (which until .NET 4.0 do not have their own contracts) the verifier cannot check whether they will violate any of the requires/ensures/invariants and so you get a lot of warnings about potentially unsatisfied constraints.
On the other hand, it does find some invalid or potentially unsatisfied constraints which could be real bugs. But it's very hard to find these because there is so much noise that it's hard to find out which ones you can fix. It's possible to suppress the warnings from the BCL classes by using the assume mechanism, but this is somewhat self-defeating as these classes will have contracts in the future and assumptions will lessen their worth.
So my feeling is that for now, because in 3.5 we're trying to build on a framework that the verifier does not sufficiently understand, that it's probably worth waiting for 4.0.
Judging by this thread I would say it is not quite mature enough to use for an enterprise level project. I haven't used it myself, but people are still running into bugs that would bring your contract-critical project to a halt. It seems like a really great framework and the example videos they've provided have been exciting, but I'd wait for:
Existence of a community forum. You're going to want to be able to discuss inevitable problems you run into with other developers, and you want to know there is a decently strong base of developers out there to discuss solutions with.
A successful pilot project release. Generally, when Microsoft Research releases something that they think is mature enough to be used in a commercial project, they will work with an organization to pilot it, and then release that project open source to as a proof of concept and trial-by-fire of all of the major features. This would give a lot of confidence that most of the common contract scenarios are covered and working.
More complete documentation. Plain and simple, at some point you're going to want to do something with contracts that you can't do yet using Microsoft Code Contracts. You want to be able to quickly and clearly reason that your scenario is not yet supported. The current documentation is going to keep you guessing and trying different things, though, in my opinion, which will result in a lot of wasted time.
It's not mature enough.
It will be as soon as Microsoft releases it with the affordable editions of VS, but without the static code analysis it's not usable at all.
The editions of VS, that have it, are so insanely expensive that only a handful of people will ever be able to afford it.
It's a shame Microsoft killed this amazing idea with their pricing policy. I wish Code Contracts would become mainstream, but they won't.
Epic fail.
I'm interested in what tools and methods are used for diagnosing flaws in large scale functional programs. What tools are useful? My current understanding is that 'printf' debugging (e.g. add logging and redeploy) is what is typically used.
If you've done debugging of a functional system what was different about it then debugging a system built with an OO or procedural language?
Sadly, printf debugging seems to be the state of practice for Standard ML, Objective Caml, and Haskell. There's a little bit of debugging at the interactive read-eval-print loop, but once your app hits 25,000 or 50,000 lines that's less useful.
If you're lucky enough to be using Haskell, there's an exception: QuickCheck is indispensible for testing and deubgging. QuickCheck can be used even on combinations of Haskell and C code, as demonstrated by experience with the Xmonad window manager.
It's worth noting that around 1990 Andrew Tolmach built a very nice time-travel debugger for Standard ML of New Jersey, but that it was not considered worth maintaining. It's also worth noting that at one point the OCaml debugger (also a time-travel debugger) worked only on bytecode, which was inconvneient, and refused to violate abstraction barriers, which made it useless. This was around release 3.07 or so; perhaps things have improved.
Also in the early 1990s, Henrik Nilsson built an interesting debugger for Haskell, but mostly what it did was prevent the debugger from accidentally changing the evaluation behavior of the program. This was interesting, but only to lavzy-evaluation weenies.
As someone who has built or worked on large applications in all three of these languages, I find the state of play depressing.
The main tools we use at work (a Haskell shop) are:
QuickCheck
HPC: visual Haskell program coverage tool (we developed this in house)
Logging/printf/trace
Sometimes, the GHCi debugger
My current job is to implement new features and support a large system implemented in ocaml and C#. Most of the "logic" is implemented in caml and the GUI and data access is in C#. The debugging techniques are pretty much as you describe lots of logging and assert to work out what's gone wrong.
Additionally we have a large number of unit tests, which are just caml scripts for testing the logic and help to spot any regression errors.
We also use continuous integration to check the build and run nightly test scripts, including some automated testing of the GUI though our "automation" style scripting interface.
I quite often use the C# debugger for debugging the C# portion of the application, the ocaml debugger does yet work under windows so we don't use it. Although we hope one day we may fix this but it isn't top of our priority list. I have occasionally used windbg to investigate managed and unmanaged memory problems, though this turned out to be caused by a third party component implemented in C#.
So overall, nothing out of the ordinary but it seems to work okay, we don't see too many production problems.
Thanks,
Rob
F# has Visual Studio integration, so you can attach the debugger to your program and set breakpoints, watches, etc, just like with any other .NET language.
However, I prefer to avoid debugging as much as possible, by writing short functions that I can unit-test individually.
A couple of years ago when I did this I had to use a combination of printf debugging and QuickCheck. These days I would also use the ghci built-in debugger.
The biggest headache was actually laziness causing space-time leaks. There still doesn't seem to be a good answer to these: just do lots of profiling and keep trying to figure it out.
OCaml and F# both have excellent debuggers. OCaml's is time reversible. F#'s has excellent IDE and multithreading support.
Right now I'm not sure...
I'd say yes. I'm using it. I know for a fact that Universal are using it on some of their (thousands of) sites. I will add some caveats, however:
There are serious problems with setting it up, especially if you want to debug into the libraries.
The helper functions favour prototype, as opposed to the more modern jQuery. This is changing rapidly, however.
The documentation is a bit chaotic, again the Castle Team are working on that.
I'm not guaranteeing every last "out-there" feature works, but the point of the system is actually to keep it simple.
Compared to vanilla ASP.NET, it's an absolute joy. I assure you that you won't miss viewstate.
We have been building a fairly large application with it for the past year and a half. Its been nice not to have to deal with the old ASP/Page based model and use the better Model/View/Controller design pattern.
To get the new stuff you really need to work off the trunk of development because they don't do releases very often. We have a lot of tests that get the framework involved so when an update in the framework breaks something we depend on we know about it immediately.
If you have to work in .NET this beats the heck out of the alternatives.
There is an overview on the monorail forum: http://forum.castleproject.org/viewforum.php?f=6
I'm using it for an application and haven't had any big issues with it.
The biggest problem is indeed find good documentation and examples.
I've had no problems setting it up. Julian, I don't think it is constructive to say things like "serious problems" without any further clarification or example.
Debugging into the libraries is trivial. Because it's open source, you can debug into the whole thing.
I've been using MonoRail for production for ages on many projects, as an employee, as an indie contractor, and for non-work related sites.
I know I'm biased on that, however I can whole heartedly promise that my positive usage experience is what lured me into contributing to the project, not the other way around.