The Influence of Organizational Structure On Software Quality

One goal of software metrics is to estimate and forecast the quality of a software product. In recent years, a number of metrics for the software product as well as the process have been developed, which mostly deal with things such as code complexity, code churn or productivity of the process.

But while it is widely known that a software product is largely influenced by the structure of the developing organization, this fact has not been taken into account in existing quality examinations of software products. In their paper “The Influence of Organizational Structure On Software Quality: An Empirical Case Study“, Nagappan, Murphy (both Microsoft Research) and Basili develop a scheme of organizational metrics to evaluate the complexity of an organization.

They identified eight different metrics, among them:

Metric	Assertion
Number of Engineers (NOE)	The more people, who touch the code, the lower the quality
Number of Ex-Engineers (NOEE)	A large loss of team members affects the knowledge retention and thus quality
Edit Frequency (EF)	The more edits to components, the higher the instability and lower the quality
Depth of Master Ownership (DMO)	The lower level the ownership, the better the quality

They evaluated their metrics on the Codebase of Windows Vista (over 3400 Binaries, >50 Million Lines of Code, thousands of developers) with very astonishing results. They tried to estimate which binaries of Vista would have a fault in the field (are failure-prone) and which are not failure-prone. 86.2% of the predicted failure-prone binaries were actually failure-prone (precision) and 84% of all actual failure-prone binaries were correctly predicted (recall). They repeated their prediction with different metrics and compared the results:

Metric	Precision	Recall
Organizational Structure	86.2%	84.0%
Code Churn	78.6%	79.9%
Code Complexity	79.3%	66.0%
Dependencies	74.4%	69.9%
Code Coverage	83.8%	54.4%
Pre-Release Bugs	73.8%	62.9%

As it can be seen, the organizational structure metrics out-performed every other metric in terms of precision as well as recall!

One might argue that this might only be true for such big organizations, but the authors state that the results of a reduced study indicate that 30 developers and 3 levels of organizational depth are sufficient to draw reliable predictions.

The article is very interesting because organizational structure metrics are not widely known (the paper is of January 2008), not even to mention applied in quality estimations, yet. When I first heard about this, I was very suprised that they performed that well in comparison to other established metrics (some of them tens of years old). Even though it might of course be that the results of the authors are not generally applicable, the article is definitely worth reading and I’m sure that these metrics will in future be of great relevance in practice.

Links to the article:
The Influence of Organizational Structure On Software Quality: An Empirical Case Study. By Nachiappan Nagappan; Brendan Murphy; Victor Basili. January 2008