A Guide to Selecting Software Measures and Metrics

Capers Jones

• Publisher: Auerbach Publications
• Year: 2017
• ISBN:
  9781315314624
• Edition: 1
• Topic: Betriebswirtschaft
• Subtopic: Projektmanagement

Going where no book on software measurement and metrics has previously gone, this critique thoroughly examines a number of bad measurement practices, hazardous metrics, and huge gaps and omissions in the software literature that neglect important topics in measurement. The book covers the major gaps and omissions that need to be filled if data about software development is to be useful for comparisons or estimating future projects.

Among the more serious gaps are leaks in reporting about software development efforts that, if not corrected, can distort data and make benchmarks almost useless and possibly even harmful. One of the most common leaks is that of unpaid overtime. Software is a very labor-intensive occupation, and many practitioners work very long hours. However, few companies actually record unpaid overtime. This means that software effort is underreported by around 15%, which is too large a value to ignore. Other sources of leaks include the work of part-time specialists who come and go as needed. There are dozens of these specialists, and their combined effort can top 45% of total software effort on large projects.

The book helps software project managers and developers uncover errors in measurements so they can develop meaningful benchmarks to estimate software development efforts. It examines variations in a number of areas that include:

• Programming languages
• Development methodology
• Software reuse
• Functional and nonfunctional requirements
• Industry type
• Team size and experience

Filled with tables and charts, this book is a starting point for making measurements that reflect current software development practices and realities to arrive at meaningful benchmarks to guide successful software projects.

Chapter 1

Introduction

As the developer of a family of software cost-estimating tools, the author is often asked what seems to be a straightforward question: How accurate are the estimates compared to historical data?

The answer to this question is surprising. Usually the estimates from modern parametric estimation tools are far more accurate than the historical data used by clients for comparisons! This fact is surprising because much of what are called historical data are incomplete and omit most of the actual costs and work effort that were accrued.

In some cases historical data capture only 25% or less of the full amount of effort that was expended. Among the author’s IT clients, the average completeness of historical effort data is only about 37% of the true effort expended when calibrated by later team interviews that reconstruct the missing data elements such as unpaid overtime.

Quality data are incomplete too. Most companies do not even start measuring quality until after unit test, so all requirement and design defects are excluded, as are static analysis defects and unit test defects. The result is a defect count that understates the true numbers of bugs by more than 75%. In fact, some companies do not measure defects until after release of the software.

Thus when the outputs from an accurate parametric software cost-estimating tool such as Software Risk Master™ (SRM), COCOMO II, CostXpert, ExcelerPlan, KnowledgePlan, True-Price, SEER, or SLIM are compared to what are called historical data, the results tend to be alarming and are also confusing to clients and client executives.

The outputs from the estimating tools often indicate higher costs, more effort, and longer schedules than the historical d...