I.M. Testy

Originally Published Tuesday, September 30, 2008

Last week I attended the Software Testing and Performance conference in Boston. I presented a workshop on Systematic Testing Techniques, as well as a talk on random test data generation, and combinatorial analysis. One way I continue to learn about our profession and increase my own knowledge is by going to conferences to hear different points of view from practitioners from around the world. So, I also attended several talks during the conference, but there was one talk in particular that was particularly entertaining (and I don’t mean that in a good way).

When I listen to other testers sometimes I hear something that is new to me and I desire to learn more about it. Sometimes I hear something prophetic that makes me think, contemplate alternatives, or reflect more deeply on my own personal perspectives. Sometimes I hear something revolutionary that causes me to reevaluate my position. And, sometimes I hear something so irrational I almost barf up a lung!

In this case the speaker opened his talk with an attack on a quote from the ISTQB foundation syllabus used to describe boundary testing which states, "Behavior at the edge of each equivalence partition is more likely to be incorrect…" Now I know the speaker a bit, and I know he disdains the ISTQB and other certification organizations, but what surprised me was his initial rebuttal by emphatically stating, "equivalence class partitions are figments of our imaginations!"

These days I usually just try to shake off wild and baseless comments as bombastic bloviations used to generate controversy. But, in this case what caught my attention was when the speaker later said that he and another well-known person defined boundaries as "a dividing point between two otherwise contiguous regions of behavior; or a principle or mechanism by which things are classified into different sets." What!? I couldn’t believe what I heard, so I had to stop reading email and look up at the presentation. As I visually processed the words I thought my head was going to explode from the so seemingly obvious contradiction.

Now, I am not a linguistic expert, but I am pretty sure that "otherwise contiguous regions of behavior" and "classifying things into different sets" are just overly simplistic ways of describing equivalence class partitions. But, I could be wrong. So, I began thinking that since most people start learning about sets in elementary schools they probably understand the foundation of equivalence class partitioning is set theory which basically states "a set is an aggregate, class, or collection of objects," and the collection of objects or ‘classification of things’ in different sets is based on an equivalent relation between the elements in each set. The application of equivalence class partitions in our profession is elegantly explained by Lee Copeland in his excellent book A Practitioner’s Guide to Software Test Design by stating "An equivalence class consists of a set of data that is treated the same by the module or that should produce the same result." Equivalence class partitioning is also discussed in-depth in books by noted experts in the industry such as Beizer, Binder, Myers, Jorgensen, Perry, and Marick just to name a few.

In fact, the concept of sets and equivalence almost seems instinctive in most humans and is generally expressed at a young age. I remember my young daughter at age 2 or so separating beads by color into "different sets" on the carpet. The red beads in one group, blue in another, and so on. She was diligent to make sure the different sets of beads did not touch as she put them into the appropriate piles. If a pile of beads got to close to another pile she would run the edge of her hand between the "contiguous regions" to clearly delineate the "dividing point."  When I asked her to get me a red bead, she would randomly grab one from the pile, because all the red beads were…red, and there were no significant differences among the red beads (elements) in the set she created that were relevant in that context of that game.

Perhaps the majority of the industry’s experts are wrong and I wasted my time reading books on software testing principles and practices because this person is right and equivalence class partitioning is really only a figment of our imagination.

However, on the other hand, although I certainly have never claimed to be an expert, I am still pretty darn sure the underlying foundation of computers and computer software is somewhat influenced by mathematical principles, and as a tester I might be able to use those same principles to help me design effective tests that might help me better evaluate discrete functional capabilities and attributes of software components and more efficiently expose certain categories or patterns of errors.

But, why should we get mired down and confused with facts (especially all that boring math stuff) when it is much easier to appeal to some peoples’ emotions. So,forget everything you just read…and if anyone asks why testing is so hard just tell them testing is an art with no practical foundation in logic because software is…well, it’s just magic!

Originally Published Thursday, November 29, 2007

In the last post we decomposed the set of characters in the ANSI character set into valid and invalid class subsets for use in a base filename component on the Windows Xp operating system. The second part of the testing technique of equivalence class partitioning is to then use this information in tests that will adequately evaluate the functional capabilities of the base filename parameter used by COMDLG32.DLL in Windows Xp.

(Note: It is appropriate here to differentiate between a test case and a test. A test case is the hypothesis or assertion that I am either trying to prove or disprove. A test is the process or procedure I am using to prove or disprove my hypothesis. In other words, a test case in this example might be "Validate a base filename can be saved using valid characters in the ANSI 1252 character set." In order to prove or disprove this hypothesis, I will execute multiple tests using the various subsets outlined in the previous post, and as described below. If any test fails, the test case fails.)

There are basically 2 procedures for defining the tests that use the data from an equivalence class table. Paul Jorgensen describes a procedure for robust equivalence class testing in which each valid class subset is tested individually. Glenford Myers suggests a procedure in which valid class subsets are combined in a test until all valid class subsets have been covered. (Jorgensen refers to this as weak equivalence class testing.) Both authors agree that subsets in the invalid subsets must be tested in a way that only one parameter is invalid and any other parameters use valid variables. Using Myers’ approach (or what Jorgensen describes as weak equivalence class testing) the tests for the base filename parameter are illustrated in the table below.

[Edited 11/30] I apparently made too many assumptions about conducting these tests, so I must clarify some of those assumptions.

• Any time a key is pressed there is essentially a state change. The primary state change we are concerned with for this particular functional test is whether or not a file is saved to the file system. However, in cases of invalid input there are other state changes that may be interesting. In most well designed Windows applications an erroneous input in an input control will be highlighted (usually after an error message is displayed). This is indeed an expected state change. The only time no noticeable state change occurs (there is still a state change because the machine is processing WM_Key messages) is with Test #5. But, another noticeable state change (the one primarily identified in the table below) is the state change to the list view control.

• There are 3 ways to effectively affect the Save button. One way is to press enter, another is to press the Alt + s key mnemonic, and thirdly to left mouse click the button control (including simulating any of those actions via automation). In some cases the visible state changes may vary; however, the purpose of these tests it so verify the existence of a file name in the file system for valid cases, and the non-existence and no abhorrent side effects in the case of members of the invalid class subsets. So, regardless of the visible state changes, the tester can use any of these procedures to affect the Save button.

• Application of techniques (systematic procedures) at a granular level is very different than simply trying things to see what happens, exploratory testing, guessing, or wild speculation. I made the assumption that readers are familiar with designing and executing atomic functional tests at a granular level in which certain variables are controlled. For example, once we enter an invalid string in the file name text box and press the save button the Save As… dialog is now dirty; meaning that the next input could produce an error but I will not know if the error is a result of an error with the base filename parameter, or with the Save As… dialog state. Equivalence class partitioning is a low level functional test of a single parameter, so in order to test the base filename parameter the tester (or automated test) should minimally close and instantiate the Save As… dialog on each test.

Reducing tests while increasing breadth of coverage

The total number of possible tests of valid string combinations for just the base filename parameter using only characters within the ANSI 1252 character set is 214²⁵¹ + 214²⁵⁰ + 214²⁴⁹ + …214² + 214¹. This number of tests, of course, is a virtual impossibility, so by employing the equivalence class partitioning testing technique we are able to systematically produce a minimum baseline set of tests that has a high probability of both proving or disproving our hypothesis or test purpose, as well as providing great variability in the test data to increase breadth of data coverage. The minimum possible number of valid tests determined by combining at least one element from each valid class subset is only 4 tests. But, let’s look at each valid test a little more closely.

Test #3 is probably a red-herring! This is only an invalid filename on Windows NT 4.0 and below. So, if your application is ported from that time frame, and you are using a custom function for your file save functionality rather than Windows APIs then you might consider running this test once. If if passes, you can probably forget running this again ever again on that product. Test #1 evaluates the literal strings in valid subset V1 can be listed in an array or enumeration and one element can be selected at random throughout the development lifecycle, or each literal string can be tested once and the probability of failure in a later build is most likely less than .001%. Test #4 is also a test that probably doesn’t require a great deal of retesting of various combinations of elements from subsets V2, V3, V4, V5 , & V7. Elements from the valid class subsets described in Test #2 are most interesting and this is the test that we will probably want to automate and run repeated throughout the development lifecycle because it provides great breadth of coverage. Remember this is the minimum number of valid tests. What isn’t covered in this set is common or ‘real-world’ data sets which we would certainly want to include. Additionally, Test #2 relies on at least one element from each indicated subset. We might also want to consider additional tests that focus on subsets V4 and V5 only. Also, we might also consider testing valid class subset V6 as a special case if we suspected a private function excluded code point values that were not assigned character glyphs. However, if these 4 valid tests pass the probability of failure of any combination of these data sets used in this parameter is minimal. Random selection of elements for Test #2 (and possibly Test #4) may slightly increase the probability of exposing a defect in the base filename parsing routine. Tests #5 through #26 are tests for invalid filenames, or in the case of Test #19 and #20 where the expected result is to truncate the leading or trailing space character.

This of course only analyzes (or tests) the base filename parameter and assumes a nominal valid 3-letter extension, valid filenames do not preexist on the file system, and within the specific context described in the first post. If the context changes, (e.g. this example does not apply to Unix platforms, or Windows Vista, or other parameters) then this set of tests (assuming we would include at least Tests #2, and #5 through #26 as part of our regression suite ran on each new build) would provide a high level of confidence in the functional capability of this specific parameter.

Next, we would also decompose the data for the file extension parameter (which is different than the base filename parameter) because in the File name textbox we can enter either the base filename or the base filename and an extension, and once we verify the functionality of the base filename component, we can then proceed to the next step in analyzing the functionality of the arguments passed to the File name textbox parameter which we shall examine at a later date.

It is important to note the this and any other techniques are simply systematic procedures designed to help us wrap our minds around complex problems. They are not the only approach to testing (only a fool would advocate a single approach to testing), but when used in conjunction with various other techniques, methods, and approaches EQ can help to establish a strong foundation of confidence in low level operations. Of course, as has been previously stated, the limiting factor of this and other functional techniques is the knowledge of the professional tester to think critically and rationally analyze the "overall system" within the appropriate context.

Originally Published Thursday, November 15, 2007

Again, I am remiss in my postings…too many irons in the fire these days. Two weeks ago, I posted a challenge to decompose a set of character data (The ANSI Latin 1 Character Set) into valid and invalid equivalence class subsets in order to test the base filename parameter of a filename passed to COMDLG32.DLL on the Windows Xp platform from the user interface using the File Save As… dialog of Notepad.

As illustrated below the filename on a Windows platform is composed of two separate parameters. Although the file name parameter of the Save As… dialog will accept a base filename, a base filename with an extension, or a path with a filename with or without an extension, the purpose of the challenge was to decompose the limited set of characters into equivalence class subsets for the base filename component only (the part outlined with green). (Of course, complete testing will include testing with and without extensions, but let’s first focus on building a foundation of tests to adequately evaluate the base filename parameter first, then we can expand our tests from there to include extensions.)

As suggested in the earlier post, in order to adequately decompose this set of data within the defined, real world context (and not in alternate philosophical alternate universes) a professional tester would need to understand programming concepts, file naming conventions on a Windows platform, Windows Xp file system, basic default character encoding on the Windows Xp operating system (Unicode), some historical knowledge of the FAT file system, and even a bit of knowledge of the PC/AT architecture. The following is a table illustrating how I would decompose the data set into equivalence class subsets.

Valid Class Subsets

• V₁ – escape sequence literal strings
        (STX, SOT, ETX, EOT, ENQ, ACK, BEL,
        BS, HT, LF, VT, FF, CR, SO, SI, DLE,
        DC1, DC2, DC3, DC4, NAK, SYN, ETB,
        CAN, EM, SUB, ESC, FS, GS, RS, US,
        DEL)
• V₂ – space character (0×20) (but not as
         only, first, or last character in the 
         base file name)
• V₃ – period character (0x2E) (but not as
         only character in the base file name)
• V₄ – ASCII characters
         punctuation (0×21, 0×23 – 0×29, 0x2B – 0x2D, 0x3B, 0x3D, 0×40, 0x5B, 0x5D, - 
         0×60, 0x7B, 0x7D, 0x7E),
         numbers (0×30 – 0×39)
         alpha (0×41 – 0x5A, 0×61 – 0x7A)
• V₅ – Ox80 through 0xFF
• V₆ – 0×81, 0x8D, 0x8F, 0×90, 0x9D
• V₇ – Component length between 1 – 251
         characters (assuming a default 3-
         letter extension and a maximum path
         length of 260 characters)
• V₈ – Literal string CLOCK$ (NT 4.0 code 
         base)
• V₉ – a valid string with a reserved
           character 0×22 as the first and
           last character in the string

Invalid Class Subsets

• I₁ – control codes 
       (Ctrl + @, Ctrl + B, Ctrl + C, Ctrl + ], Ctrl + N, 
       etc.)
• I₂ – escape sequence literal string NUL
• I₃ – Tab character
• I₄ – reserved words
        (LPT1 – LPT4, COM1 – COM4, CON, PRN, AUX,
        etc.)
• I₅ – reserved words
         (LPT5 – LPT9, COM5 – COM9)
• I₆ – reserved characters (/ : < > | )
         (0x2F, 0x3A, 0x3C, 0x3E, 0x7C) by
        themselves or as part of a string of 
        characters
• I₇ – reserved character 0×22 as the only
        character or > 2 characters in the string
• I₈ – a string composed of > 1 reserved character
        0x5C
• I₉ – a string containing only 2 reserved
         characters 0×22
• I₁₀ – period character (0x2E) as only
         character in a string
• I₁₁ – two period characters (0x2E) as only
         characters in a string
• I₁₂ – > 2 period characters (0x2E) as only
         characters in a string
• I₁₃ – reserved character 0x5C as the only 
           character in the string
• I₁₄ – space character (0×20) as only character in 
         a string
• I₁₅ – space character (0×20) as first character in
          a string
• I₁₆ – space character (0×20) as last character in a
          string
• I₁₇ – reserved characters (* ?)  (0x2A, 0x3F)
• I₁₈ – a string of valid characters that contains at least one reserved characters (* ?)  (0x2A, 0x3F)
• I₁₉ – a string of valid characters that contains at
          least one reserved character 0x5C but not
          in the first position
• I₂₀ – string > 251 characters
• I₂₁ – character case sensitivity
• I₂₂ – empty

Discussion of valid equivalence class subsets

• Valid subset V₁ is composed of the literal strings for control characters (or escape sequences) between 0×01 and 0x1F, and including 0x7F. The literal strings for control characters may cause problems under various configurations or unique situations. The book How to Break Software: A Practical Guide to Testing goes into great detail explaining various fault models for these various character values. The literal strings in this subset should be tested as the base filename component and possibly in a separate test as an extension component. However, on the Windows platform the probability of one particular string in this subset behaving or being handled differently than any of the others is very low negating the need to test every string in this subset; although the overhead to test all would be minimal and once complete would not likely require repeated testing of all literal strings in this subset during a project cycle.
• Valid subset V₂ provides guidance on the use of the space character in valid filenames. On the Windows operating system a space character (0×20) is allowed in a base filename, but is not permitted as the only character as a file name. Typical behavior on the Windows platform also truncates the space character if it is used as the first character of a base filename or the last character of a base filename. However, if the extension is appended to the base filename in the Filename edit control on the Save or Save As… dialog a space character can be the last character in the base filename. Also note that a space character by itself or as the first character in a filename is acceptable on a UNIX based operating system. Also, although we can force the Windows platform to save a file name with only a space character by typing “ .txt” (including the quotes) in the Filename edit control on the Save/Save As… dialog this practice is not typical of reasonable Windows users’ expectations.
• Valid subset V₃ is the period character (0x2E) which is allowed in a base filename, but it is not a valid filename if it is the only character in the base filename (see Invalid subset for the period character).
• Valid subset V₄ is composed of ‘printable’ ASCII characters that are valid ASCII characters in a Windows filename. The subset includes punctuation characters, numeric characters, and alpha characters. We could also decompose this subset further into additional subsets including valid punctuation characters, numbers, upper case, and lower case characters if we wanted to ensure that we test at least one element from the superset at least once.
• Valid subset V₅ is the set of character code points between 0×80 and 0xFF.
• Valid subset V₆ is a superset of subset V₅ and are separated only because they are code points that do not have character glyphs assigned to those code point values. These would be interesting especially if we needed to test filenames for backwards compatibility on Windows 9x platforms.
• Valid subset V₇ is the minimum and maximum component length assuming the filename is saved in the root directory (C:\). 
• Valid subset V₈ is a probably a red-herring. On the NT 4 platform the string CLOCK$ was a reserved word. On an older application first created for the Windows NT 4 platform that does not use the system Save/Save As dialog we might want to test this string just to make sure the developer did not hard code the string in an error handling routine.
• Valid subset V₉ is an interesting case because this invalid reserved character (0×22) is handled differently when used in first and last character positions of a base filename. When used in the first and last positions of a base filename the characters are truncated and if the remaining string is valid the filename is saved. If only one 0×22 character is used, or if two or more 0×22 characters are used in a string other than the first and last character positions the result will be an error message.

Discussion of invalid equivalence class subsets

• Invalid subset I₁ consists of the control code inputs for escape sequences in the range of 0×01 through 0x1F, and also includes 0x7F. Pressing the control key (CTRL) and any of the control codes keys will cause a system beep.
• Invalid subset I₂ is the literal string “nul”. Nul is a reserved word but could be processed differently than other reserved words on the Windows platform because it is also used in many coding languages as a character for string termination.
• Invalid subset I₃ is the tab character which can be copied and pasted into the Filename textbox control. Pasting a tab into the and pressing the save button will generate an error message.
• The invalid subset I₄ includes literal strings for reserved device names on the PC/AT machine and the Windows platform. Using any string in this subset result in an error message indicating the filename is a reserved device name.
• Invalid subset I₅ also includes reserved device names for LPT5 – LPT9 and COM5 – COM9. However these must be separated into a unique subset because using these specific device names as the base filename on the Windows Xp operating system result in an error message indicating the filename is invalid.
• Invalid subsets I₆, I₇, and I₈, include reserved characters on a Windows platform. When characters in this subset are used by themselves or in any position in a string of characters the result is an error message indicating the above file name is invalid.
• Invalid subsets I₉, I₁₀, I₁₃, also include reserved characters and the space and period characters. When these subsets are tested as defined no error message displayed and focus is restored to the File name control on the Save/Save As… dialog.
• Invalid subsets I₁₁, I₁₂, also include the reserved character (0x2E) as 2 characters in the string and greater than 2 characters in a string. The state machine changes are different.
• Invalid subsets I₁₅ and I₁₆ define the space character when used in the first or last character position of a string. These are placed in the invalid class because Windows normal behavior is to truncate a leading or trailing space character in a file name. If the leading or trailing space character was not truncated and saved as part of the file name on a Windows platform that would constitute a defect.
• Invalid subset I₁₇ and I₁₈ contains two additional reserved characters; the asterisk and the question mark (0x2A and 0x3F respectively). If these characters are used by themselves or as a character in a string of other valid characters a file will not be saved, and no error message will occur. However, the state of the Save/Save As… dialog does change. If the default file type is .txt and there are text files displayed in the Folder View control on the Save As… dialog the files with the .txt extension will be removed after the Save button is depressed. If the default file type is All files then all files will be removed from the Folder View control on the Save As… dialog after the Save button is depressed.
• Invalid subset I₁₉ is a string of valid characters which contains at least backslash character except as the lead character in the string. (Of course, this assumes the string is random and the position of the backslash character in the string is not in a position which would resolve to a valid path.) The backslash character is a reserved character for use as a path delimiter in the file system. An error message will appear indicating the path is invalid.
• Invalid subset I₂₀ tests for extremely long base file name lengths of greater than 252 characters. Note that an interesting anomaly occurs with string lengths. A base file name string length which tests the boundaries of 252 or 253 valid characters will cause an error message to display indicating the file name is invalid. However, a base file name string length of 254 or 255 characters will actually get saved as file name but is not associated with any file type. Any base file name string longer than 255 characters again instantiates an error message.
• Invalid subset I₂₁ describes the tests for case sensitivity. The Windows platform does not consider character case of characters that have an upper case and a lower case representation. For example, a file name with a lower case Latin character ‘a’ is considered the same as a file name with the upper case Latin character ‘A’.
• Invalid subset I₂₂ is, of course, an empty string

Of course, this is a partial list of the complete data set since the filename on a Windows Xp operating system can be any valid Unicode value of which there are several thousand character code points, including surrogate pair characters.

The first and by far the most complex step in the application of the functional technique of equivalence class partitioning is data decomposition. This requires an incredible amount of knowledge about the system. Data decomposition is an exercise in modeling data. The less one understands the data set, or the system under test the greater the probability of missing something. Next week we will analyze the equivalence class subsets to define are baseline set of tests to evaluate the base filename component.

Originally Published Wednesday, October 31, 2007

Wow…where does the time go? I was remiss last week in posting, and it has been a month since I posted about equivalence class partitioning. So, let’s get back to it shall we?

Equivalence class partitioning (ECP) is a functional testing technique useful in either black box or white box test design. A technique is a systematic approach to help solve a complex problem. Techniques are not silver-bullets, but they are a logical and analytical approach to problem solving that heavily draws upon the tester’s cognitive abilities (having a basis in or reducible to empirical factual knowledge) as opposed to random guessing (or little men running about inside one’s head triggering turbid thoughts). Contrary to popular misconceptions the application of ECP is not a rote, brain-dead activity. The ECP technique requires in-depth knowledge of the data set (data type, encoding method, etc), the programming language used in the implementation, the algorithm structure, the operating environment, protocols, and even the hardware platform may impact how the data for a particular parameter might be decomposed. The effectiveness of the application of this technique solely lies in the testers ability to adequately decompose the data set for a given parameter into subsets in which any element from a specific subset would produce the same result as any other element from that subset.

Essentially the ECP technique analyzes input or output variable data for each specific parameter and decomposes the data into discrete valid and invalid class subsets. See my article in Software Testing & Performance magazine for a more in-depth understanding of the equivalence class partitioning technique. The article uses a simple next date program as an example because working with integer type data within a limited range is usually a good introduction to this complex technique.

A frequent reader of this blog suggested he often sees this technique applied to simple parameters limited to number type inputs, but would like to understand how this is applied to a more complex parameter that takes a string input. Interestingly enough, one of the exercises I designed in our internal training (and also used in the workshop I present regularly at the Software Testing and Performance conferences) deals specifically with string input. Let me outline the exercise.

Your goal is to adequately decompose the set of data (the ANSI Latin 1 character set) into valid and invalid equivalent class subsets to evaluate the behavior of the base filename parameter passed to COMDLG32.DLL’s file save functionality from a user’s perspective (GUI) on the Windows Xp operating system with an NTFS file system on a PC/AT platform.

NOTES:

• On the Windows environment the base filename parameter is separate and not interdependent on the extension parameter. (Some frequent readers may remember I explained this point previously to an individual who mistakenly assumed the base filename parameter and extension parameter were interdependent on the Windows environment.) So, remember, the purpose of this exercise is to focus initially on the base filename parameter (don’t worry about the extension parameter…yet).
• To expose the user perspective of the file save functionality of COMDLG32 we will use Notepad.exe, and select the File -> Save As… menu item to instantiate the Save As… dialog.
• Characters greater than 0x7F can be entered using the numeric input method by holding down the ALT key and pressing number keypad keys equivalent to the characters decimal value. For example to enter the € character press and hold the ALT key and press the 0 1 2 8 keys on the numeric keypad. Release the ALT key and the character is displayed. (You can also use my Babel tool to generate specific characters by inputting the Unicode values for the characters in the Custom range groupbox and copying them into the filename control.)
• Not all characters can be entered using the numeric input method. For example, control sequences cannot be directly entered from the keyboard using this method (or by holding down the control key).
• In order to succeed, one must understand
  • Common programming concepts (especially for the C family of languages)
  • Windows File naming conventions
  • Windows Xp, file I/O APIs
  • Windows Xp NTFS file system
  • Knowledge of basic character encoding
  • Historical knowledge of FAT file systems

Give it a try…next week I will offer a solution that you can compare your own results against.

Originally Published Sunday, September 30, 2007

I have been teaching formal testing techniques for several years at Microsoft and University of Washington Extension. Techniques are systematic procedures to help solve a complex problem. A technique does not find all types problems; techniques are generally very good at finding very specific classes of defects. But, the usefulness or effectiveness of any particular technique relies on the in-depth system and domain knowledge of the tester, and in the tester’s skill and ability to apply the correct technique to a problem space.

One of those techniques that I teach is equivalence class partitioning (ECP). On the surface ECP appears rather simple, but the effectiveness in the application of this technique relies on the tester’s ability to decompose input and output variables into discrete subsets of valid and invalid classes.

Over-generalization of the data reduces the number of subsets in a specific class and increases the likelihood of under-testing. Hyper-decomposition of the variables may increase the number of subsets in a specific class without providing additional benefit (although it may potentially increase the number of tests). Based on this observation I have proposed the following hypothesis or Bj’s Theory of Equivalence Class Partitioning Data Decomposition: Over-generalization of data reduces the baseline tests and increases the probability of missing errors or generating false positives. Hyper-analysis of data increases the probability of redundancy and reduces the overall effectiveness of each test.

ECP is not some rote activity, but in order for it to be most effective the tester requires a lot of in-depth knowledge about the overall system (hardware, operating system, environment, etc), the domain (client program, programming language used to develop the application, API interfaces, etc.), and the data sets. Limited knowledge or inability to adequately analyze the data based on the system and domain spaces in one or more of these areas is an impediment to the effective application of this technique. I suspect those who fail to perform an in-depth analysis of the system, domain, and data interaction leads some people disregard this technique, or become hyper-critical of its usefulness.

In this months issue of Software Test & Performance magazine I discuss the theory and application of the equivalence class partitioning technique as a viable and extremely useful tool used by professional testers. The magazine is free to download, and the article goes into great depth explaining when and how to apply the technique of equivalence class partitioning. Let me know if you have any questions, and I would appreciate your comments or feedback.