Originally Published Friday, August 11, 2006
Last year an ‘expert’ tester and consultant claimed test techniques were gimmicks. I don’t claim to be an expert tester, but I do consider myself to be a professional tester constantly striving to improve my skills and broaden my knowledge to make me a better tester and promote the discipline I have chosen as a career. So, I have learned that test techniques are not gimmicks, but are in fact effective systematic procedures to prove (or disprove) that specific attributes or capabilities of the software function as expected and/or conform to any applicable standards or guidelines. Test techniques also help focus testers towards designing highly effective test cases that target specific functional areas of the product, and also establish a solid foundation from which additional testing can progress.
Techniques are similar to tools in a professional’s toolbox. The professional tester knows what techniques to use in specific situations, and how to use them most effectively. Functional testing techniques such as boundary value analysis (boundary testing), equivalence class partitioning, combinatorial analysis (pair-wise testing), or state transition testing each serve very different purposes. These techniques systematically evaluate specific functional attributes of a software project, but they are not the only ‘tools’ a professional tester uses to ‘test’ software.
The greatest value of test techniques is their potential to instill a high level of confidence in testing coverage of specific functional areas with a fewer number of tests, and thus reduce the amount of redundant testing. However, the value or effectiveness of using any test technique ultimately depends on the individual tester’s ‘system’ and domain knowledge, and the ability to apply the applicable technique to the correct situation.
Over the next few weeks I plan to discuss how to use some of the various testing techniques and demonstrate why these are valuable tools for the professional tester.
2 Comments
I suspect that you are considering only one interpretation of “gimmick”. Merriam-Webster Online offers several:
[quote]
1 a : a mechanical device for secretly and dishonestly controlling gambling apparatus b : an ingenious or novel mechanical device : GADGET
2 a : an important feature that is not immediately apparent : CATCH b : an ingenious and usually new scheme or angle c : a trick or device used to attract business or attention
[/quote]
You assert that test techniques are not gimmicks. If you take interpretation 1A, then I agree with you. On the other hand, James could mean that a test technique might be “an ingenious or novel mechanical device”–mechanical in the sense of a set of steps, or a pattern.
But what’s more important, I think, is that you miss what I see as the important part of the article: test techniques (and the mastery of them) are little things compared to the much more important skills of thinking critically about the system that we’re testing and about testing itself. Another point of the article is that over the years, we’ve heard plenty about techniques and how important they are. What we haven’t heard about very often is that techniques are heuristic devices; they’re fallible methods for solving problems.
It’s interesting to me that you seem to object to something in the article, but that you agree with its thrust: that skill of the tester is crucial. If that’s what you’re intending to discuss, I’m interested in reading about it.
—Michael B.
Thursday, August 31, 2006 4:01 AM by Michael Bolton
Hi Michael
I think everyone understands the connotation of the word gimmick, and that Bach’s post intended to trivialize techniques as “tricks or devices used to attract business or attention.” You have to be a much better snake-oil salesman to convince anyone that Bach’s statement “Techniques are gimmicks” could be construed as ‘Techniques are ingenious or novel mechanical devices.’
BTW…if you are going to support your argument citing definitions, the definition (without removing the word ‘device’) cannot be twisted to suit your needs. Also, your cited references 1a, and 1b clearly imply a physical device (GADGET) and thus are not applicable in the context of this discussion.
Software testing is a field of computer science, so when I use the term techniques in the context of software testing I imply the denotative meanings:
The systematic procedure by which a complex or scientific task is accomplished.
the body of specialized procedures and methods used in any specific field, esp. in an area of applied science.
technical skill; ability to apply procedures or methods so as to effect a desired result.
Conversely, heuristic implies experimentation, trial and error, speculative formulation serving as a guide in investigation or problem solving, rules of thumb, or educated guesses.
“Unlike algorithms, heuristics do not guarantee optimal, or even feasible, solutions and are often used with no theoretical guarantee.” – Dictionary of computing.
Your assertion that techniques are heuristic devices is false simply based on a comparison of the definitions of the 2 words. But, your assertion that techniques can be fallible is true. They are fallible in large part because the technical skill or ability to apply the procedures or methods to effect a desired result is lacking.
It seems to me that Bach and others simple assume that techniques imply intellectually brain-dead, step-by-step, rigid, and rudimentary procedures that provide little value.
Correct application of testing techniques requires not only critical thinking of the problem space, but also in-depth knowledge of the system under test (down to the code level).
So, we are in absolutely agreement that the skill and knoweldge of a professional tester is paramount to the success of any software testing effort. However, I assert that the correct application of established testing techniques by a professional tester is more effective in targeted testing as compared to trial-and-error, experimentation, or simple wild ass guess (SWAG) methods.
Think of it in these terms. A Marine Corps sniper is more effective than an untrained person who picks up a gun and fires wildly from their hip or over their head. The untrained person may get lucky, but smart money is on the sniper.
- Bj -
Friday, September 01, 2006 11:36 PM by I.M.Testy