Originally Published Wednesday, April 04, 2007
The value of testing is important and critical to the success of many projects. However, testing is NOT solely responsible for the quality of a project!
When we discuss the role of testing and quality in our internal training we emphasize the primary objectives of the tester as providing information and assessing (preferably via quantifiable measurements) quality (and specifically focusing on the quality traits or attributes that are most critical to the success of the project).
But, there are still some managers who would like to believe that the testing group within an organization is responsible for quality. And there are even some testers who will tell you they are responsible for quality. I won’t begin to speculate how this folktale came to be, or why it is perpetuated. But, it is a folktale (any belief or story passed on traditionally, esp. one considered to be false or based on superstition). Folktales usually persist in societies for a variety of reasons. There is no doubt that quality is important. But, quality is hard to define; quality is hard to measure, and it is hard to translate quality into value for the customer. There is no doubt that as a tester I can influence the quality; however, responsibility implies the ability to also control. How much control does a typical testing organization in a commercial software company actually have upon the quality of a project?
There are 3 simple tests to refute the conjecture that testing is responsible for quality.
- Can we test in quality?
- Can testing find all the defects?
- Will all the defects be fixed? (Here a defect implies deviation from explicit or implicit requirements or expectations, or unexpected behavior.)
The answer to these three questions is no! Unless you are an absolute novice to software engineering you will probably agree we cannot test in quality. Testing cannot find all the defects in any complex software project (and there is ample evidence of this). And anyone who has any experience on a project of significant size will admit that not all defects are fixed (including functional defects). So, as a tester I know I can’t test in quality, I will never find all the defects, and I know that some of the defects I find will not get fixed (because I can’t use electric cattle prods on developers and I am not going to go into the code and fix the bug myself), so how in the world can I be responsible for the quality of a product?
As a tester I am responsible for providing information regarding the perceived quality and/or risk of a project as assessed by a series of tests, observations, and experiments, and that information is provided to management (who hopefully use it) to determine whether a commercial software product is released to the public.
So, who is ultimately responsible for the quality of a product? Often times folks will rally around the “everyone is responsible for quality” mantra. But, W. Mark Manduke wrote an nice article in STQE magazine a few years ago stating “When quality is declared to be everyone’s responsibility, no one is truly designated to be responsible for it, and quality issues fade into the chaos of the crisis du jour.” He concluded “…when management truly commits to a quality culture, everyone will, indeed, be responsible for quality.”
Thus, ultimately the management team (the decision makers) is responsible and accountable for quality.
9 Comments
Good post BJ,
Unfortunately, many people (including management) think that it is testers responsibility to ensure quality. I have heard the arguments like “Testers are paid to do that job” and for every slipped bug – testers head is on the chopping block.
In my own experience of MS-GDCI (hyderabad), I have heard people making claims about “testing to ensure zero defect” or “PSP/TSP and six sigma together produce zero defect code hence testing is not required” What do you say?
This post of yours looks different from your previous ones … you seem to echo the views of Cem Kaner and others in Context driven school …
“Testing is an act of questioning a product in order to evaluate it”
“Testing is a service”
“Testing is an act of technical evaluation done for stakeholders to know about quality related information”
Are there any changes in your approach to testing off late?
If you were to re-write your “End the seggration of 4 schools of testing” blog — will there be any changes?
Shrini
Wednesday, April 11, 2007 2:43 AM by Shrini
Hi Shrini,
Yes, I have also encountered managers who seemingly blame the testing effort for bugs that slip into the released product. These managers are simply ignorant of the purpose and value of testing in an organization, and perhaps it is part of our jobs to enlighten these misguided folks.
Six sigma is similar to poka-yoke, a defect prevention scheme that solves problems in manufacturing that found its way into software development a few years ago. If software development was similar to building widgets then it might work. Fortunately, I think we are at the point of realizing six sigma does not adapt well to innovation companies.
TSP/PSP evolved from the same folks (SEI) that introduced CMM. I actually received training in TSP/PSP from Watts Humphrey about 4 years ago or so. The general tenet is to push quality upstream, defect prevention, make developer responsible for the work they produce, intensive unit testing, etc. All theoretically good stuff. Does it work? Maybe in some situations, but certainly not all.
But, projects delivered using six sigma and TSP/PSP approaches do not eliminate testing. They greatly reduce the number of testers; and the testers who remain on the project tend to be extremely technical and have stronger CS type backgrounds. The bugs these testers are finding aren’t your typical ‘cherries.’
There is no change in my approach or my view of the purpose of testing or the testing discipline. As you get to know me more, you will find out that I don’t waffle on my position or change my song just to make others happy. Nor I am a mindless droid who simply regurgitates the hype of charasmatic snake oil salesmen. I carefully analyze various viewpoints and assimilate the knowledge to form my own perspective. If I am wrong then I will admit it, eat a little humble pie, commit to learn more and improve my knowledge, and move on (hopefully in the right direction).
So, in response to your question about rewriting the post regarding ending the segregation of the ’so-called’ 4 schools…I would not make any changes.
I have a lot of respect for Bret Pettichord, and he and I exchanged email about my post. (I always enjoy chatting with Bret, and I appreciate his professionalism.) I also know that Bret has now included a 5th school which is the agile school. I respect his views and his insights. However, I maintain that those who subscribe to one type of school are limiting their outlook and their potential contribution to the discipline. Testers who understand that the different schools all have beneficial traits in various situations or contexts will ultimately be more successful than those who pigeon-hole themselves in one particular school of thought.
Wednesday, April 11, 2007 11:41 PM by I.M.Testy
Hi William,
This is a good post. The comments that followed are also interesting.
In a Banking and Finance vertical of an organization, not to name, the weekly target for a tester is not in terms of number of test cases or otherwise, rather in terms of number of bugs that he finds in a week. I always feel sad about the testing guys there. I guess the management there should read this post of yours.
Regards,
Rahul Verma.
Monday, April 23, 2007 6:23 AM by Rahul Verma
Hi Rahul,
Thanks for your comment. I previously posted about the misuse of bug count as ‘targets’ or as evaluation criteria for testers. The title of the post is Bug counts as key performance indicators (KPI) for testers (http://blogs.msdn.com/imtesty/archive/2006/06/26/647628.aspx). (for some reason the insert/edit link function on the community server seems to be broken today.) I think you will enjoy that post as well.
Tuesday, April 24, 2007 5:31 PM by I.M.Testy
Hi Bj,
Thanks for sharing the link.
I have read the contents. There was another link there to an external blog post at MSDN. I read that as well.
I was so carried away by all this, that I thought to write about it and share the information with more people. You can find my post “Using Bug Count for Performance Evaluation of Testers” at the below link-
http://testingperspective.blogspot.com/2007/04/using-bug-count-for-performance.html
As this is based on your post and thoughts, please have a look when you find time and share your views as well.
Thanks again for sharing useful information.
Regards,
Rahul Verma.
Wednesday, April 25, 2007 4:48 AM by Rahul Verma
Hi Bj,
The link is http://testingperspective.blogspot.com/2007/04/using-bug-count-for-performance_25.html
Due to some issues with blogspot, I had to delete and post it again.
Regards,
Rahul Verma.
Wednesday, April 25, 2007 9:04 AM by Rahul Verma
Good article on Testing vs. Quality.
My comment is, that your are right that WE – the people with the competences within testing, are the ones who should “teach” the managers what testings is about. If they do not study this themselves (and they probably don’t) – we really have to help them. We, as in companies, managers, testers, test managers etc. – do not improve in this area, these tasks of developing projects, if we do not try to teach “newcommers” – managers and actually also people from business, what testing is about, how it should be done – and not least, the outcome to expect from it. We have that responsibility.
cheers,
Anders
Tuesday, July 31, 2007 6:26 AM by BBLsolutions – Ensuring software quality
Hi Anders,
Considering that most CS programs teach very little about software testing, and many managers in the industry today seem to lack even a CS background, your statement “…the people with the competencies [sic] within testing, are the ones who should “teach” the managers what testing is about.”
- Bj -
上一篇談 Tester 的職責是什麼, 這一篇談另一個觀念, Tester 不應該被叫做 QA (quality assurance), 因為基本上, Tester 沒辦法真的確保 “Quality”.
Wednesday, June 11, 2008 4:50 AM by Eric Hu’s Weblog