Unit Testing and Defect Rates

Steve McConnell's Code Complete points to an interesting study where the defect detection rates of many common-defect detection techniques used individually have a modal rate of about 30-50%. Unit testing under that study had a lowest defect detection rate of 15%, a modal rate of 30% and highest rate of 50%. Given the focus that different methodologies such as XP have had on unit testing that is quite low and a little disappointing. However the study noted;

When used individually, no method had a statistically significant advantage over any of the others. The variety of errors people found was so great, however, that any combination of two methods - including independent groups using the same method - increased the total number of defects found by a factor of almost two.

So unit testing used in conjunction with regression testing (for instance in continuous integration) improves the value of unit testing. From the list in that book of the value of the different removal methods, we have decided to use unit testing in conjunction with formal code reviews and formal design inspections. However, our design inspection are based on the stories (use cases) and are a couple of pages to make sure the starting point for coding is ok and agreed on.

Unit Testing has value beyond defect removal rates. It is amazing how developer confidence improves with unit testing. Kent Beck writes:

While there are many rational explanations for why programmers should automate tests, for me the compelling argument is emotional. When I write automated tests I feel more confident in my work than when I don't write automated tests.

Which I agree with. The unit testing adds a sense of completeness to code as well. With all unit testing libraries and the integration of unit testing into IDEs it has become very simple to do so as part of a project. But the study that McConnell points to suggests that used in combination with another defect removal methodology they are even more effective.
Permalink, Unit Testing and Defect Rates, Dec 2009, cam

More Reading on Software engineering

Most Popular on South Sea Republic

The articles that have been viewed the most:

Most Popular Restaurants in Phoenix

Phoenix Eats Out is the restaurant review site for Phoenix, Scottsdale and Old Town Scottsdale which lists the modernist and contemporary restaurants, taverns and bars in the greater Phoenix area. This is the list of the most popular restaurants pages from phoenixeatsout.com that have been viewed the most; My personal favourite restaurants in Phoenix are AZ88, Postinos, Bomberos with Grazie, Humble Pie, Orange Table, The Vig, Fez and others coming close behind. View the complete list with the photo-journalistic style images on phoenixeatsout.com

Most Popular Hikes in Arizona

Arizona is an outdoor state and has lots of hiking in the city and around the state. Phoenix is unusual for most cities in having several large mountains in the center of the city with great hiking. Anyone who comes to Phoenix has to do the Echo Canyon trail on Camelback and the Summit Hike on Squaw Peak or Piesta Peak. The views of the city, suburbs and surrounding mountains are wonderful from Camelback and Piesta Peak. For more experienced hikers there is the McDowell Mountains in North Scottsdale that has several difficult and strenuous hikes in Tom's Thumb and Bell Pass. Alternatively, you can hike the highest mountain in Arizona. At 12,600 feet Humphrey's Peak is a long and difficult hike.

Alternate Australian Constitutions

Between 2004 and 2009 this site, southsearepublic.org, was a constitutional blog based on scoop which focused on Australian and global constitutional issues. One of the strongest aspects of it was the development of constitutions by those involved in the blog. These constitutions are the outcome: The constitutions were built using principles from Montesquieu's separation of powers, the enlightnment's universal political rights and the ancient Athenian technology of sortition and choice by lot.

Archives For South Sea Republic

South Sea Republic started in 2004 as an Australian constitutional blog in 2004 based on scoop software. It was an immigrative outgrowth of Kuro5hin. The archives for each year since then; The articles are ordered by views.

Who Is Cam Riley

Cam Riley I am an Australian living in the United States as a permanent resident. I am a software developer by trade and mostly work in Java and jump between middleware and front end. I originally worked in the New York area of the United States in telecommunications before moving to Washington DC and working in a mix of telecommunications, energy and ITS. I started my own software company before heading out to Arizona and working with Shutterfly. Since then I have joined a startup in the Phoenix area and am thoroughly enjoying myself.

I do a lot of photography which I post on this website, but also on flickr. I have a photo-journalistic website which lists the modernist and contemporary restaurants in phoenix. I have a site on the Australian Flying Corps [AFC] which has been around since the 1990s and which I unfortunately lost the .org URL to during a life event; however, it is under the www.australianflyingcorps.com URL now. The AFC website has gone through several iterations since the 90s and the two most recent are Australian Flying Corps Archives(2004-2002) and Australian Flying Corps Archives(2002-1999) which are good places to start.

Websites Worth Reading

Websites of friends, colleagues and of interest;