Can anybody suggest a one/two line explanation of the "five" SonarQube categories, in such a way that a non-developer can understand what the percentage figure means?
Efficiency
Maintainability
Portability
Reliability
Usability
One word "synonym" for non-developers (not exact synonym though, but enough to give a quick idea):
Efficiency : performance
Maintainability : evolution
Portability : reuse
Reliability : resilience
Usability : design
Most of those metrics are presented in this Wikipedia entry
Efficiency:
Efficiency IT metrics measure the performance of an IT system.
An effective IT metrics program should measure many aspects of performance including throughput, speed, and availability of the system.
Maintainability
.
is the ease with which a product can be maintained in order to:
correct defects
meet new requirements
make future maintenance easier, or
cope with a changed environment
.
Portability:
the software codebase feature to be able to reuse the existing code instead of creating new code when moving software from an environment to another.
Reliability:
The IEEE defines reliability as "The ability of a system or component to perform its required functions under stated conditions for a specified period of time."
Note from this paper:
To most project and software development managers, reliability is equated to correctness, that is, they look to testing and the number of "bugs" found and fixed.
While finding and fixing bugs discovered in testing is necessary to assure reliability, a better way is to develop a robust, high quality product through all of the stages of the software lifecycle.
That is, the reliability of the delivered code is related to the quality of all of the processes and products of software development; the requirements documentation, the code, test plans, and testing.
Usability
studies the elegance and clarity with which the interaction with a computer program or a web site (web usability) is designed.
Usability differs from user satisfaction insofar as the former also embraces usefulness (see Computer user satisfaction).
See for instance usabilitymetrics.com
This represents for each category the density of violations (non-respect) of rules in the source code.
Related
I was going through this wiki article on SQALE(Software Quality Assessment based on Lifecycle Expectations). The Software Quality assurance part of it is clear. But I am unable to understand the "based on LifeCycle expectations" part of the model. Can someone please explain in a clear to understand way the LifeCycle expectations part.
Ever since I first encountered SQALE, I've suspected that the creators were working backward from the acronym, and "lifecycle expectations" was the best that they could come up with that wasn't already taken. (There are other similarly acronymed guidelines or methodologies in the software quality space, such as Squale and SQuaRE).
All but the most rudimentary models of software quality incorporate the notion of quality over the full lifecycle of a software product (from "gleam in someone's eye" phase all the way through eventual decomissioning), not just its state at the time of initial shipping.
Such models also acknowledge that the desirable investment in any given aspect of software quality (e.g.: maintainability) is not equal for all software products. For example, one would tend to make very different decisions in maintainability investments for a boxed-style software product one hopes to sell (and upgrade) for years vs for a short-term marketing promo site that will be decomissioned one month after it goes live.
So... All that "lifecycle expectations" bit probably implies is something along the lines of covering software quality aspects that affect multiple part of the lifecycle of a product, as well as allowing adjustment/calibration to different expectations for the various quality characteristics.
If you're interested in how the SQALE authors might have meant this (assuming they weren't only trying to shoehorn the name into the acronym), there are a few hints in the SQALE manual that can be downloaded from http://www.sqale.org/download. They seem to feel that they've done something novel and useful in projecting the ISO/IEC 9126 quality characteristics over a product lifecycle in a chronological order, and perhaps this is what the name is meant to reflect.
BTW, if you're interested in software quality and want to understand this sort of quality modeling in more depth, I'd recommend taking a look at SQuaRE (incl. the parts of ISO/IEC 9126 for which there are no SQuaRE replacements yet). It's not necessarily the most exciting reading, but I find that it provides excellent background for evaluating the utility of various quality methodologies and tools.
I am looking for 3 examples of how I can use function-point measure in quality control or assurance over some other simpler measures i.e. 3 examples where function-points are preferable over some other simpler methods in quality control/assurance.
Function points are used to estimate the effort required to build an application. An alternative would be, e.g., COCOMO which is based on the number of lines of code. FP-based approaches are often considered to be preferrable to COCOMO at early stages of the project development when no clear estimate of the number of lines of code to be developed can yet be given.
Another advantage of the FP-based effort estimation is provided by availability of effort tables (e.g., in Capers Jones' books) for different kinds of applications and different kinds of activities involved in a software project. Hence, by using FPs project manager can obtain a more refined insight in the effort required.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed last year.
Improve this question
There has been a considerable amout of discussion about code metrics (e.g.: What is the fascination with code metrics?). I (as a software developer) am really interested in those metrics because I think that they can help one to write better code. At least they are helpful when it comes to finding areas of code that need some refactoring.
However, what I would like to know is the following. Are there some evaluations of those source code metrics that prove that they really do correlate with the bug-rate or the maintainability of a method. For example: Do methods with a very high cyclomatic-complexity really introduce more bugs than methods with a low complexity? Or do methods with a high difficulty level (Halstead) really need much more amount to maintain them than methods with a low one?
Maybe someone knows about some reliable research in this area.
Thanks a lot!
Good question, no straight answer.
There are research papers available that show relations between, for example, cyclomatic complexity and bugs. The problem is that most research papers are not freely available.
I have found the following: http://www.pitt.edu/~ckemerer/CK%20research%20papers/CyclomaticComplexityDensity_GillKemerer91.pdf. Though it shows a relation between cyclomatic complexity and productivity. It has a few references to other papers however, and it is worth trying to google them.
Here are some:
Object-oriented metrics that predict maintainability
A Quantitative Evaluation of Maintainability Enhancement by Refactoring
Predicting Maintainability with Object-Oriented Metrics - An Empirical Comparison
Investigating the Effect of Coupling Metrics on Fault Proneness in Object-Oriented Systems
The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics
Have a look at this article from Microsoft research. In general I'm dubious of development wisdom coming out of Microsoft, but they do have the resources to be able to do long-term studies of large products. The referenced article talks about the correlation they've found between various metrics and project defect rate.
Finally I did find some papers about the correlation between software metrics and the error-rate but none of them was really what I was looking for. Most of the papers are outdated (late 80s or early 90s).
I think that it would be quite a good idea to start an analysis of current software. In my opinion it should be possible to investigate some populare open source systems. The source code is available and (what I think is much more important) many projects use issue trackers and some kind of version control system. Probably it would be possible to find a strong link between the log of the versioning systems and the issue trackers. This would lead to a very interesting possibility of analyzing the relation between some software metrics and the bug rate.
Maybe there still is a project out there that does exactly what I've described above. Does anybody know about something like that?
We conducted an empirical study about the bug prediction capabilities of the well-known Chidamber and Kemerer object-oriented metrics. It turned out these metrics combined can predict bugs with an accuracy of above 80% when we applied proper machine learning models. If you are interested, you can ready the full study in the following paper:
"Empirical Validation of Object-Oriented Metrics on Open Source Software for Fault Prediction. In IEEE Transactions on Software Engineering, Vol. 31, No. 10, October 2005, pages 897-910."
I too was once fascinated with the promises of code metrics for measuring likely quality, and discovering how long it would take to write a particular piece of code given its design complexity. Sadly, the vast majority of claims for metrics were hype and never bore any fruit.
The largest problem is that the outputs we want to know (quality, time, $, etc.) depend on too many factors that cannot all be controlled for. Here is just a partial list:
Tool(s) Operating system
Type of code (embedded, back-end, GUI, web)
Developer experience level
Developer skill level
Developer background
Management environment
Quality focus
Coding standards
Software processes
Testing environment/practices
Requirements stability
Problem domain (accounting/telecom/military/etc.)
Company size/age
System architecture
Language(s)
See here for a blog that discusses many of these issues, giving sound reasons for why the things we have tried so far have not worked in practice. (Blog is not mine.)
https://shape-of-code.com
This link is good, as it deconstructs one of the most visible metrics, the Maintainability Index, found in Visual Studio:
https://avandeursen.com/2014/08/29/think-twice-before-using-the-maintainability-index/
See this paper for a good overview of quite a large number of metrics, showing that they do not correlate well with program understandability (which itself should correlate with maintainability): "Automatically Assessing Code Understandability: How Far Are We?", by Scalabrino et al.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I am looking after some ways to measure the performance of a software development team. Is it a good idea to use the build tool? We use Hudson as an automatic build tool. I wonder if I can take the information from Hudson reports and obtain from it the progress of each of the programmers.
The main problem with performance metrics like this, is that humans are VERY good at gaming any system that measures their own performance to maximize that exact performance metric - usually at the expense of something else that is valuable.
Lets say we do use the hudson build to gather stats on programmer output. What could you look for, and what would be the unintended side effects of measuring that once programmers are clued onto it?
Lines of code (developers just churn out mountains of boilerplate code, and other needless overengineering, or simply just inline every damn method)
Unit test failures (don't write any unit tests, then they won't fail)
Unit test coverage (write weak tests that exercise the code, but don't really test it properly)
Number of bugs found in their code (don't do any coding, then you won't get bugs)
Number of bugs fixed (choose the easy/trivial bugs to work on)
Actual time to finish a task based against their own estimate (estimate higher to give more room)
And it goes on.
The point is, no matter what you measure, humans (not just programmers) get very good at optimizing to meet exactly that thing.
So how should you look at the performance of your developers? Well, that's hard. And it involves human managers, who are good at understanding people (and the BS they pull), and can look at each person subjectively in the context of who/where/what they are to figure out if they are doing a good job or not.
What you do once you've figured out who is/isn't performing is a whole different question though.
(I can't take credit for this line of thinking. It's originally from Joel Spolsky. Here and here)
Do NOT measure the performance of each individual programmer simply using the build tool. You can measure the team as a whole, sure, or you can certainly measure the progress of each programmer, but you cannot measure their performance with such a tool. Some modules are more complicated than others, some programmers are tasked with other projects, etc. It's not a recommended way of doing this, and it will encourage programmers to write sloppy code so that it looks like they did the most work.
No.
Metrics like that are doomed to failure. Different people work on different parts of the code, on different classes of problem, and absolute measurements are misleading at best.
The way to measure developer performance is to have excellent managers that do their job well, have good specs that accurately reflect requirements, and track everyone's progress carefully against those specs.
It's hard to do right. A software solution won't work.
I think this needs a very careful approach when deciding the ways to measure developers performance as most the traditional methods such as line of codes, number of check ins, number of bugs fixed etc. are proven to be subjective with todays software engineering concepts. We need to value team performance approach rather measuring individual KPIs in a project. However working in commercial development environment its important to keep a track and a close look at following factors of individual developers;
Code review comments – Each project, we can decide the number of code reviews need to be conducted for a given period. Based on the code reviews individuals get remarks about their coding standard improvements. Recurring issues of code reviews of same individual’s code needs to be brought in to attention. You can use automated code reviews tools or manual code reviews.
Test coverage and completeness of tests. – The % covered needs to be decided upfront and if certain developer fails to attempt it often, it needs to be taken care of.
Willingness to sign in to complex tasks and deliver them without much struggle
Achieving what’s defined as “Done” in a user story
Mastery level of each technical area.
With agile approach in some of the projects, the measurements of the development team and the expected performance are decided based on the releases. At each release planning there are different ‘contracts’ negotiated with the team members for the expected performance. I find this approach is more successful as there is no reason of adhering to UI related measurements in a release where there is a complex algorithm to be released.
I would NOT recommend using build tool information as a way to measure the performance / progress of software developers. Some of the confounding problems: possibly one task is considerably harder than another; possibly one task is much more involved in "design space" than "implementation space"; possibly (probably) the more efficient solution is the better solution, but that better solution contributes less lines of code than a terribly inefficient one which provides many many more lines of code; etc.
Speaking of KPI in software developers. www.smartKPIs.com may be a good resource for you. It contains a user friendly library of well-documented performance measures. At the moment it lists over 3300 KPI examples, grouped in 73 functional areas, as well as 83 industries and sub-categories.
KPI examples for the software developers are available on this page www.smartKPIs.com - application development They include but not limited to:
Defects removal efficiency
Data redundancy
In addition to examples of performance measures, www.smartKPIs.com also contains a catalogue of performance reports that illustrate the use of KPIs in practice.
Examples of such reports for information technology are available on: www.smartKPIs.com - KPIs in practice - information technology
The website is updated daily with new content, so check it from time to time for additional content.
Please note that while examples of performance measures are useful to inform decisions, each performance measure needs to be selected and customized based on the objectives and priorities of each organisation.
You would probably do better measuring how well your team tracks to schedules. If a team member (or entire team) is consistantly late, you will need to work with them to improve performance.
Don't short-cut or look for quick and easy ways to measure performance/progress of developers. There are many many factors that affect the output of a developer. I've seen alot of people try various metrics ...
Lines of code produced - encourages developers to churn out inefficient garbage
Complexity measures - encourages over analysis and refactoring
Number of bugs produced - encourages people to seek out really simple tasks and to hate your testers
... the list goes on.
When reviewing a developer you really need to look at how good their work is and define "good" in the context of what the comany needs and what situations/positions the company has put that indivual in. Progress should be evaluated with equal consideration and thought.
There are many different ways of doing this. Entire books written on the subject. You could use reports from Hudson but I think that would lead to misinformation and provide crude results. Really you need to have task tracking methodology.
Check how many lines of the codes each has written.
Then fire the bottom 70%.. NO 90%!... EVERY DAY!
(for the folks that aren't sure, YES, I am joking. Serious answer here)
We get 360 feedback from everyone on the team. If all your team members think you are crap, then you probably are.
There is a common mistake that many businesses make when setting up their release management tool. The Salesforce release management toolkit is one of the best ones available in the market today, but if you do not follow the vital steps of setting it up, you will definitely have some very bad results. You will get to use it but not to its full capacity. Establishing release management processes in isolation from the business processes is one of the worst mistakes to make. Release management tools go hand in hand with the enterprise strategy, objectives, governance, change management plus some other aspects. The processes of release management need to be formed in such a way that everyone in the business is on the same page.
Goals of release management
The main goal of release management is to have a consistent set of reliable and repeatable processes that are resource independent. This enables the achievement of the most favorable business value while at the same time optimizing the utilization of resources available. Considering that most organizations focus on running short, high-yield business projects, it is essential for optimization of the delivery value chain of the application to make certain that there are no holdups in the delivery of the business value.
Take for instance the force.com migration toolkit, as this tool has proven to be great in governance. A release management tool should allow for optimal visibility and accountability in governance.
Processes and release cycles
The release management processes must be consistent for the whole business. It is necessary to have streamlined and standardized processes across the various tool users. This is because they will be using the same platform and resources that enable efficient completion of their tasks. Having different processes for different divisions of your business can lead to grievous failures in tool management. The different sets of users will need to have visibility into what the others are doing. As aforementioned, visibility is of great importance in any business process.
When it comes to the release cycles, it is also imperative to have one centralized system that will track all the requirements of the different sets of users. It is also necessary to have this system centralized so that software development teams get insight into the features and changes requested by the business. Requests have to become priorities to make sure that the business gets to enjoy maximum benefit. Having a steering team is important because it is involved in the reviewing of business requirements plus also prioritizing the most appropriate changes that the business needs to make.
The changes that should happen to the Salesforce system can be very tricky and therefore having a regular meet up between the business and IT is good. This will help to determine the best changes to make to the system that will benefit the business. By considering the cost and value of implementing a feature, the steering committee has the task of deciding on the most important feature changes to make.
Here also good research http://intersog.com/blog/tech-tips/how-to-manage-millennials-on-software-development-teams
This is an old question but still, something you can do is borrow Velocity from Agile Software Development, where you assign a weight to each task and then you calculate how much "weight" you solve in each sprint (or iteration or whatever DLC you use). Of course this comes in hand with the fact that, like a commenter mentioned before, you need to actively keep track yourself of whether your developers are working or chatting online.
If you know your developers are working responsively, then you can rely on that velocity to give you an estimate of how much work the team can do. If at any iteration this number drops (considerably), then either it was poorly estimated or the team worked less.
Ultimately, the use of KPIs together with velocity can give you per-developer (or per-team) insights on performance.
Typically, directly using metrics for performance measurement is considered a Bad Idea, and one of the easy ways to run a team into the ground.
Now, you can use metrics like % of projects completed on-time, % of churn as code goes toward completion, etc...it's a wide field.
Here's an example:
60% of mission-critical bugs were written by Joe. That's a simple, straightforward metric. Fire Joe, right?
But wait, there's more!
Joe is the Senior Developer. He's the only guy trusted to write ultra-reliable code, every time. He's written about 80% of the mission-critical software, because he's the best.
Metrics are a bad measurement of developers.
I would share my experience and how I learnt a very valuable process on measuring the team performance. I must admit, I have fallen on tracking KPI simply because most of the departments would do the same but not really for the insight until I had the responsibility to evaluate developers performance where after a number of reading I evolved with the following solution.
One every project, I would entertain the team in a discussion on the project requirements and involve them so everyone knows what is to be done. In the same discussion through collaboration we would break the projects in to tasks and weight those tasks. Now previously we would estimate the project completion as 100% where each task has a percentage contribution. Well this did work for a while but was not the best solution. Now we would based the task on weight or points to be exact and use relative measurements to compare the task and differentiate the weights for example. There is a requirement to develop a web form to gather user data.
Task would go about like
1. User Interface - 2 Points
2. Database CRUD - 5 Points
3. Validation - 4 Points
4. Design (css) - 3 Points
With this strategy We can pin point a weekly approximate on how much we have completed and what is pending on the task force. We can also be able to pin point who has performed best.
I must admit that I still face some challenges on this strategy such as not every developer is comfortable on every technology. Somehow some are excited to learn technologies simply because they find 2015 high % of points fall in that section some would do what they can.
Remember, do not track a KPI for their own sake, track it for it's insight.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Has anyone used the kanban method for software development management?
I am evaluating kanban as a technique and would be curious to hear from anyone who has actually applied it in practice as to how effective it is. I've seen questions like: is-anyone-using-kanban, kanban-vs-scrum, and apply-kanban-in-an-agile-team but they don't address my concerns.
What I'm interested in specifically is:
Does it actually offer the advantages is claims in terms of dynamically identifying bottlenecks?
Is it easy to execute in practice, or does it have logistical challenges that you need to manage?
Does it scale well to project teams with many parallel streams of work and many developers?
How does it compare to critical path analysis (as implemented in MS Project), how is it different?
What other benefits can be gained from applying kanban?
Thanks.
The Kanban method is foremost a catalyst for continuous process improvements. It’s not a quick fix or a fixed set of steps/practices. The method has a few foundational principles and core properties, as described in David J Andersons recent blog post, that can lead you the way to continuous process improvements.
To your questions:
The Kanban method in itself does not identify bottlenecks. When implementing work-in-progress limits to a process that creates stress on your process you will eventually create a pull system and then it becomes easier to identify bottlenecks in your process. Tools like a visual kanban board and Cumulative Flow Diagrams will help you identify the bottlenecks in the process.
If you apply the foundational principles and core properties and you have the stamina/patience/dedication it is not too hard. You need to manage the change process as with every organizational change but the Kanban Method is designed to make small and non-threatening changes.
Yes there are many documented cases of this.
The Kanban method does not identify a specific method for planning and projecting future deliveries. David J Anderson has a background in Theory of Constraints and uses TOC as a model in most of the writing I have read. I think the practical difference between using MS Project style big up front planning and the empirical based planning used in many kanban implementations is the big difference. When working with a project plan designed in MS Project in the beginning of a project you know very little of the actual problem domain and you make assumptions. Based on these assumptions you device a plan. A critical path is calculated based on these assumptions. With a stable kanban system and you use TOC as your model you plan “only” to have your constraint/bottleneck on the critical path. You take into account the historical variability of the work passing the constraint and you create buffers around you bottleneck with the appropriate risk you want to take. The thinking is that every hour lost at the bottleneck will be an hour lost for the whole system.
The main benefit of the Kanban Method is that it is a catalyst for continuous process improvements. It starts with what you got and makes non-threatening changes that sticks. Kanban is a method that is Made to Stick
In the article Applying Kanban to PC Deployment the Team has to account for the following equipment:
160 new PCs to be deployed
40 new laptops to be deployed
120 PCs and 10 laptops to be refreshed and redeployed
... we are exploring the use of Kanban to manage a short-term functional
project. This example focuses on using Kanban to create a transparent
process to track the flow of equipment through a number of complex
steps, without incurring additional costs for tracking software,
complex processes and training, or duplication of effort. Improved
uniformity or quality of the deployment process will also help improve
efficiencies in troubleshooting and repair times as well as ensure a
document-ably high level of conformance to software and licensing
standards.
The page above has also links to Kanban applied ...
to Tech Support
to PC Deployment (see Quote above)
to a Development Group
Challenges, Additional Concepts, and Wrapup
I also don't have a lot of experience with it, but I think I can offer some insight.
1 & 4: the main difference between Kanban boards and other techniques, like CPM, is that a Kanban board, in a correct implementation, forces you to impose work-in-progress limits. This creates a pull system, since new items are accepted by workers only when they have capacity. This differs from an MS project type project where tasks are assigned to workers before-hand (i.e. pushed).
It is much easier to identify a bottleneck in a pull system, because work items will be queuing up at some stage in the process. In a push system, work is pushed through the system (whether it is 'done done' or not), so its difficult to find bottlenecks.
Another advantage of a pull system is you can start to base work timelines on actual results (lead and cycle time), as opposed to prediction. Yes, the size and granularity of stories does affect this, but with techniques such as cumulative flow diagrams this becomes less important.
2: Most implementations are pretty simple, and therein lies some of the strength of the technique. I think if you're having problems with the logistics of the technique, you're doing it wrong. Have a look here for a nice 'kickstart example'.
Few definitions to focus on before jumping onto the differences:
Agile – A structured and iterative framework to track and manage projects. This approach is used in managing software development projects. It allows cross-functional teams to collaborate on users expectations.
Kanban – A framework which utilizes visualization technique, limiting the number of tasks to be taken in “Work in Progress” column. The segregation of a similar type of tasks can be done here. To simplify it, allocate colors to tasks using the swim lanes.
Scrum – The approach followed here is breaking down a complex task into simpler smaller manageable pieces which are easy to collaborate upon by the respective owners of the [scrum][1].
Similarities between Kanban and Scrum
Frameworks of agile methodologies
Used to track the progress of the project
Provide the team transparency in tracking the work progress
Make use of visualization
Differences between Kanban and Scrum
Roles – Scrum is dependent on the scrum owners and is worked upon by them respectively. Kanban is independent of cross-functional team members and parallel roles.
Release cycle – Scrum makes use of sprints whose duration varies from one week to two weeks. The user stories are then taken up for development, testing and bug fixes. Kanban does not follow any cycle and the process is continuous in nature.
Tracking parameters – Scrum makes use of velocity in planning upcoming sprints taking into account the complexity and number of user stories completed in the previous sprint. Kanban ensures limiting of user stories in “Work in Progress” column to avoid bottlenecks. It tracks the time taken to finish a task from the starting to the end.
The scope of improvement – Scrum does not encourage changes in ongoing sprints. Kanban is open to any changes before the completion of the project. It is flexible in nature.
Fit factor – Scrum is suitable for projects with clearly defined user stories. Acknowledgement on the same by the client for timely completion of the project makes it a fit. Kanban being flexible in nature allows variations in priorities on the basis of the current scenario.
Pick process – Scrum picks the entire batch of user stories from the product backlog for development. Kanban follows the maximum number of tasks allowed in the columns to maintain the sanity of the framework and to avoid bottlenecks.
Delivery – Scrum follows delivery based on sprint planning and prioritize based on the specifications given by the client.Kanban follows the continuous delivery model based on business needs.
The above points are easy to remember if you are able to visualize working on them. Ideally where the scrum follows a rather predefined set of principles. Kanban is backed up by the principle of flexibility. It allows you to track tasks that are of utmost importance for delivery.
I don't have specific experience with using Kanban in software but I am familiar with the practice from a manufacturing point of view, so I was curious as to the implementation. Reading your link, the thing that struck me as a possible hitch is what felt like an underlying assumption about the same-sized ness of the units of work (features, stories, whatever). While keeping things "story sized" is a good goal, there are often mixes of bigger and little stories floating around, and the small number constraints in their pipeline therefore seem artificial. If the goal is to highlight bottlenecks, I would think that stand ups and sprint planning and retrospectives will do that well enough. If the goal is to facilitate prioritization, I think that putting constraints on numbers of tasks by types wouldn't do that as well as simply ordering them top to bottom.
I guess I don't really see what value its adding; that being said, I don't see the harm in trying it and adopting whatever pieces work.
1. Does it actually offer the advantages is claims in terms of dynamically identifying bottlenecks?
This has been my experience. By setting your WIP limits to reflect available capacity, if there is work that needs to make use of that capacity but has to wait for it to become available then you will see it as the queue of work backs up ahead of the bottleneck. I have seen this happen when there is an overworked QA team with an upstream development team who keep producing even though there is no chance of it getting looked at by QA. The solution we took to this was to lend some of the developers to the QA team who then helped alleviate the bottleneck.
2. Is it easy to execute in practice, or does it have logistical challenges that you need to manage?
This will depend on many factors which will be specific to the context to which you are applying it. One of the great strengths of Kanban is it does not require an immediate 'all or nothing' change from how you are currently working. The chapter 'A Recipe for Success' in David Anderson's 'Kanban' book gives a great way of approaching the change, starting with 'Focus on Quality'
3. Does it scale well to project teams with many parallel streams of work and many developers?
In the project where I first used it we ended up with a team of seventeen developers and we had moved the QA team of four into our team too. We also had lots of external dependencies which we used Kanban to model.
4. How does it compare to critical path analysis (as implemented in MS Project), how is it different?
Pass as have never used
5. What other benefits can be gained from applying kanban?
There are many but the one I will call out is that it gives you metrics that are genuinely useful for discussing and steering the work, both with the team and with stakeholders and other people outside the team. Specifically the use of 'Throughput' and 'Cycle Time' help give you a probabilistic for cast of when work will get done.