Technical Presentation
Combinatorial Analysis: Beyond Pair-wise Testing
Fault analysis reveals that interaction between the variables of dependent parameters is a source of failure in complex systems. Imagine you are assigned to test a feature with 20 parameters that are interdependent. There are 5 possible variable states for each parameter. The total number of possible combinations is greater than a half trillion; which means that at one test per millisecond it would take more than 3000 years to test all possible combinations. Which combinations do we test? Pair-wise testing is a systematic procedure to effectively reduce the total number of tests by selecting a set of tests that evaluates every pair combination because historical and root cause analysis shows the majority of errors caused by the interaction of variables occurs between 2 parameters rather than interaction between the variables for 3 or more parameters. This talk compares orthogonal arrays to pair wise analysis, and then provides a detailed example of how to use a powerful, highly configurable combinatorial analysis tool that is freely available to systematically test complex interdependent parameters. Attendees will learn:- The advantages of pair-wise analysis over orthogonal arrays
- How to analyze and logically decompose and model a feature set
- How to customize a model file for smart pair-wise analysis
- How to use custom features of the PICT tool such as weighting, conditional and unconditional constraining, negative testing, seeding and output randomization
- Measured effectiveness of pair-wise testing from real case studies
