I know there are metrics specifically to validate ontologies, but I need something to measure the system that communicates with them. I also know ISO 9126, but I need more specific metrics, which validates, for example, how extensible or maintainable the software is to increase or change with the ontology model.
If understand your question precisely, I think you should read these papers:
Coupling metrics for ontology-based system : http://goo.gl/mhVs2G
Semantic oriented ontology cohesion metrics for ontology-based
systems : http://dl.acm.org/citation.cfm?id=1663922
Metrics for ontologies : http://goo.gl/auhFQK
Evaluating Evaluation Metrics for Ontology-Based Applications:
Infinite Reflection.
Metrics for evaluation of ontology-based information extraction
Semantic oriented ontology cohesion metrics for ontology-based
systems
May be some of the work done by one of my colleagues, Filip Radulovic, might be relevant here.
Software quality model for the evaluation of semantic technologies
Extending Software Quality Models–A Sample in The Domain of Semantic Technologies
and his other papers.
Related
As http://www.healthintersections.com.au/?p=2487 claims, it is possible to model fitness data with FHIR by coding it in Observation resource. Also, Open mHealth is a standard specially designed for mobile health data, which usually include wellness and fitness data. What are their strengths and weaknesses in comparison with each other?
First, it should be possible to map between the two with no data loss. So no need to be too stressed about your decision.
Second, HL7 is an ANSI-accredited standards development organization. You can have a high level of confidence that FHIR is a safe and well-designed way to be storing and/or representing this information. I work at Redox - this is how we would integrate fitness data. Our goal at Redox, which I assume is the goal of an organization like mHealth is to offer a slightly more approachable version of something HL7 makes.
At the end of the day, it's worth investing your time into learning the why behind the FHIR design decisions. For example, coding is always an array in FHIR, but the code system in Open mHealth is implicit, it doesn't show up anywhere in the data. That's a huge problem if you're trying to integrate with software that doesn't support the one SNOMED code that Open mHealth has chosen.
Is there a way to calculate modularity metric from the preexisting metrics in SonarQube? I want to calculate modularity so that I can use it for my technical debt calculation.
The Modularity of software is a composite metric. Thus, first you need to define the modularity and participating metrics. The paper you mentioned (http://arxiv.org/ftp/arxiv/papers/1309/1309.5689.pdf) offers one of the ways.
An alternative way is to detect modularity design smells and form a modularity metric formula (that uses the number of detected instances of modularity design smells). The modularity design smells are: Insufficient Modularization, Broken Modularization, Cyclically-dependent Modularization, and Hub-like Modularization (refer "Refactoring for software design smells").
Does anyone know of any work (academic or otherwise) that thoroughly compares the ability of software complexity metrics to predict the existence of bugs?
Is something like crap4j what you're looking for? I don't know that it predicts the existence of bugs, but it does help to highlight the most likely places for bugs to occur.
Try looking at "Empirical studies of quality models in object-oriented systems" by Briand and Wust. "Predicting Maintainability with Object-Oriented Metrics - An Empirical Comparison" by Dagpinar and Jahnke, "Object-oriented metrics that predict maintainability" by Li and Henry.
Can anybody suggest a one/two line explanation of the "five" SonarQube categories, in such a way that a non-developer can understand what the percentage figure means?
Efficiency
Maintainability
Portability
Reliability
Usability
One word "synonym" for non-developers (not exact synonym though, but enough to give a quick idea):
Efficiency : performance
Maintainability : evolution
Portability : reuse
Reliability : resilience
Usability : design
Most of those metrics are presented in this Wikipedia entry
Efficiency:
Efficiency IT metrics measure the performance of an IT system.
An effective IT metrics program should measure many aspects of performance including throughput, speed, and availability of the system.
Maintainability
.
is the ease with which a product can be maintained in order to:
correct defects
meet new requirements
make future maintenance easier, or
cope with a changed environment
.
Portability:
the software codebase feature to be able to reuse the existing code instead of creating new code when moving software from an environment to another.
Reliability:
The IEEE defines reliability as "The ability of a system or component to perform its required functions under stated conditions for a specified period of time."
Note from this paper:
To most project and software development managers, reliability is equated to correctness, that is, they look to testing and the number of "bugs" found and fixed.
While finding and fixing bugs discovered in testing is necessary to assure reliability, a better way is to develop a robust, high quality product through all of the stages of the software lifecycle.
That is, the reliability of the delivered code is related to the quality of all of the processes and products of software development; the requirements documentation, the code, test plans, and testing.
Usability
studies the elegance and clarity with which the interaction with a computer program or a web site (web usability) is designed.
Usability differs from user satisfaction insofar as the former also embraces usefulness (see Computer user satisfaction).
See for instance usabilitymetrics.com
This represents for each category the density of violations (non-respect) of rules in the source code.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed last year.
Improve this question
There has been a considerable amout of discussion about code metrics (e.g.: What is the fascination with code metrics?). I (as a software developer) am really interested in those metrics because I think that they can help one to write better code. At least they are helpful when it comes to finding areas of code that need some refactoring.
However, what I would like to know is the following. Are there some evaluations of those source code metrics that prove that they really do correlate with the bug-rate or the maintainability of a method. For example: Do methods with a very high cyclomatic-complexity really introduce more bugs than methods with a low complexity? Or do methods with a high difficulty level (Halstead) really need much more amount to maintain them than methods with a low one?
Maybe someone knows about some reliable research in this area.
Thanks a lot!
Good question, no straight answer.
There are research papers available that show relations between, for example, cyclomatic complexity and bugs. The problem is that most research papers are not freely available.
I have found the following: http://www.pitt.edu/~ckemerer/CK%20research%20papers/CyclomaticComplexityDensity_GillKemerer91.pdf. Though it shows a relation between cyclomatic complexity and productivity. It has a few references to other papers however, and it is worth trying to google them.
Here are some:
Object-oriented metrics that predict maintainability
A Quantitative Evaluation of Maintainability Enhancement by Refactoring
Predicting Maintainability with Object-Oriented Metrics - An Empirical Comparison
Investigating the Effect of Coupling Metrics on Fault Proneness in Object-Oriented Systems
The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics
Have a look at this article from Microsoft research. In general I'm dubious of development wisdom coming out of Microsoft, but they do have the resources to be able to do long-term studies of large products. The referenced article talks about the correlation they've found between various metrics and project defect rate.
Finally I did find some papers about the correlation between software metrics and the error-rate but none of them was really what I was looking for. Most of the papers are outdated (late 80s or early 90s).
I think that it would be quite a good idea to start an analysis of current software. In my opinion it should be possible to investigate some populare open source systems. The source code is available and (what I think is much more important) many projects use issue trackers and some kind of version control system. Probably it would be possible to find a strong link between the log of the versioning systems and the issue trackers. This would lead to a very interesting possibility of analyzing the relation between some software metrics and the bug rate.
Maybe there still is a project out there that does exactly what I've described above. Does anybody know about something like that?
We conducted an empirical study about the bug prediction capabilities of the well-known Chidamber and Kemerer object-oriented metrics. It turned out these metrics combined can predict bugs with an accuracy of above 80% when we applied proper machine learning models. If you are interested, you can ready the full study in the following paper:
"Empirical Validation of Object-Oriented Metrics on Open Source Software for Fault Prediction. In IEEE Transactions on Software Engineering, Vol. 31, No. 10, October 2005, pages 897-910."
I too was once fascinated with the promises of code metrics for measuring likely quality, and discovering how long it would take to write a particular piece of code given its design complexity. Sadly, the vast majority of claims for metrics were hype and never bore any fruit.
The largest problem is that the outputs we want to know (quality, time, $, etc.) depend on too many factors that cannot all be controlled for. Here is just a partial list:
Tool(s) Operating system
Type of code (embedded, back-end, GUI, web)
Developer experience level
Developer skill level
Developer background
Management environment
Quality focus
Coding standards
Software processes
Testing environment/practices
Requirements stability
Problem domain (accounting/telecom/military/etc.)
Company size/age
System architecture
Language(s)
See here for a blog that discusses many of these issues, giving sound reasons for why the things we have tried so far have not worked in practice. (Blog is not mine.)
https://shape-of-code.com
This link is good, as it deconstructs one of the most visible metrics, the Maintainability Index, found in Visual Studio:
https://avandeursen.com/2014/08/29/think-twice-before-using-the-maintainability-index/
See this paper for a good overview of quite a large number of metrics, showing that they do not correlate well with program understandability (which itself should correlate with maintainability): "Automatically Assessing Code Understandability: How Far Are We?", by Scalabrino et al.