“Your assertion that techniques, or engineering principles or concepts are “merely rote, mechanical task[s]” is simply fallacious”.

Yes, it would be, if that’s what I had said. I’d invite you to have a look at the sentence to which you refer and read what it says. Here it is again.

“I would (and do) teach them that testing is not merely a rote, mechanical task; that it’s not merely the application of engineering principles;…”

That’s not an assertion that engineering principles are rote mechanical tasks. HERE is an assertion that engineering principles are rote mechancal tasks:

“Engineering procedures are rote, mechanical tasks.”

See the difference between the two sentences? I do. Perhaps I can help. Note that I differentiate the two in the first sentence, and equate the two in the second. Note that I disagree with the second sentence, and stand by the first one. Let’s go on.

“…that it’s not simply comparing the product with the spec; or other such simplifications. Instead, testing is the process of investigating the product to reveal quality-related information about it, for people to whom that information might matter.”

You agree with that, don’t you?

You suggest…

“Software testing is a professional discipline in the field of computer science.”

I think that’s true to some degree. Alas it’s not very well supported by the certain professional organizations in the field itself. The content of the ACM/IEEE’s curriculum guide to this day is less than one full course in a four-year program. I wish those guidelines addressed testing more; as it is, the I think professional organizations trivialize it.

So to be clear, testing IS a professional discipline in the field of computer science, but it’s also much more than that. Testing involves asking questions about the way humans interact with the software. It involves asking questions about the way that humans can make mistakes as they’re developing software, and as they’re testing it. So in additional to technical skills, we need other kinds of skills too.

I agree when you say that “Techniques are not rote mechanical tasks”, but I fear that they can become that way if we teach testers that other skills aren’t important too. In your original post, you assert “I am not an expert in systems thinking, cognitive psychology, or epistemology.” I’m not either, but I have studied them a good deal in the effort to gain more expertise. The value that these studies provide to me is that they add to “great breadth of knowledge and the skills”, and help me to understand ways in which “there is not a single approach that works in all scenarios”, as you put it. Technical skills are important things, but they’re not the only important things.

My guess is that we agree on too. Don’t we? It’s weird; I see you objecting strenuously to my message, and then proceeding to agree violently with it on many points.
Thursday, September 07, 2006 10:17 AM by Michael Bolton

Your assertion that techniques, or engineering principles or concepts are “merely rote, mechanical task[s]” is simply fallacious, and trivializes the excellent work of world renowned industry professionals such as Copeland, Myers, Beizer, Binder, Whittaker, etc. I would expect such a juvenile statement from someone who does not understand formal testing theory or the judicious application of techniques to improve effectiveness and efficiency; not from another professional.

Techniques are not gimmicks or rote mechanical tasks. In fact they require a great deal of knowledge and skill to apply effectively. Techniques are not the only tools in the professional tester’s toolbox, and similar to tools techniques can be fallible if applied incorrectly or by untrained, unknowledgeable individuals. It is also no secret that all professional testers have used exploratory testing as a testing tool long before Kaner coined the term, and long before a handful of consultants exploited the phrase for monetary gain promoting it as the holy grail of software testing.

You and I agree that great testing starts with great thinking skills. The difference is I don’t try to teach people how to think, I teach them to unlock their intellectual horsepower and use their knowledge to solve complex problems in software testing effectively and efficiently. The testing courses I teach at Microsoft and the University of Washington, and the presentations at various conferences focus on fundamental discipline knowledge and technical skills necessary to succeed as professional software testers.

I can teach an intelligent person with great problem solving skills and who knows how to think analytically the fundamental techniques, methods, and engineering concepts and principles used in software testing. More importantly I also need to be able to assess whether or not the people I teach can effectively apply what I taught them via Human Performance Technology (HPT) models and other learning effectiveness measures.

Software testing is not a religion, it is not a political debate between “us” and “them,” everything has context because real life doesn’t occur in a vacuum, and there are not 4 distinct schools separating testers or their approaches to solving problems in software testing. Software testing is a professional discipline in the field of computer science. Professional testers require great breadth of knowledge and the skills because there is not a single approach that works in all scenarios. At the end of the day, I think we both want professional testers making smart decisions about what and how to test each unique problem space they encounter using the most appropriate technique, method, or approach, in order to present qualified information for accurate risk assessment.

- Bj –
Saturday, September 02, 2006 4:34 AM by I.M.Testy

I’m confused by your assertion that

“The idea that solutions to the problems in software testing practices and processes can be found by studying philosophy, psychology, and systems theory is twaddle.”

You seem to contradict yourself when you say, immediately after that,

“These are interesting topics and can certainly heighten a person’s ability to think abstractly and question assumptions.”

Are thinking and questioning assumptions not core to the testing mission? You seem to agree that they are when you ask

“But, are these subjects really unique to software testing or do they also apply as equally to development, or to any other job requiring a high degree of innovation and ingenuity?”

So I’m confused; if these skills are important to development (as I agree) and any other job, etc. (as I agree) are they not important to testing. Can you help me understand? Perhaps it would help for you to clarify your ideas about what the big problems of testing might be.

You point out that “Secondly, the problems of software testing are complex and they certainly aren’t already solved.” I don’t think that James is suggesting that problems that we’re having in testing have been solved; I think he’s suggesting that we might find solutions if we study these other disciplines.

I’m not sure how you (or a fictitious archaeologist spouting philosophy while disclaiming it) distinguish fact and truth. But thinking and reading about epistemology reminds me that issues related to testing and quality are subjective and fundamentally uncertain, just as you point out in your description of trying to decide whether something is a bug. Asking “wait–how do I know this?” is a question that expert testers ask themselves all the time, isn’t it? I’d argue that if there’s not some fundamental uncertainty lurking about, I’m not asking enough questions to be testing effectively.

You suggest (and I agree) that “Systems thinking is related to general systems theory which studies the organization and interdependent relationships of systems.” Isn’t software testing all about questioning the organization and interdependent relationships of systems?

You suggest (and I disagree) that “general systems theory is usually applied to natural sciences and systems thinking is typically used to model human organizational behavior.” I would have thought that general systems theory is applied to systems generally (else it ceases to become general), and that systems thinking is applied to whatever systems you care to think about. That is, I think your interpretation of general systems is specialized somehow.

I’d also suggest that in software testing we create models to help us design all of our tests, not just state transition tests. A model is a subset of a space; it’s a representation of a slice of some reality; it’s a map of some territory. We model users; we model the input space for functions; we model workflows; we model platforms; we model everything because we can’t cover everything.

Finally, you assert “the simple fact is that I don’t need to be an expert in social sciences to be successful as a professional tester.” Indeed. These days, one doesn’t have to be an expert in anything to be successful (i.e. employed) as a professional tester; one doesn’t even need to be expert in testing itself. The quality of the software that we ship and that we use is testament to that. James (and I, and many others) would like to change that by recognizing that testing is a deeply intellectual and challenging activity; that we provide value to management and to our customers when we think critically about software in ways that others don’t.

Perhaps your focus is on what you’d teach new testers. I would (and do) teach them that testing is not merely a rote, mechanical task; that it’s not merely the application of engineering principles; that it’s not simply comparing the product with the spec; or other such simplifications. Instead, testing is the process of investigating the product to reveal quality-related information about it, for people to whom that information might matter. I contend that this work starts with thinking skills. Where does it start for you?

—Michael B.
Thursday, August 31, 2006 6:43 AM by Michael Bolton