I.M. Testy

For the past few weeks my test automation class at the University of Washington has been focused on API (application programming interface) testing, or component level testing. Boris Beizer defines component level testing as “an integrated aggregate of one or more units” and that a “component can be anything from a unit to an entire system.”

This seems a bit confusing at first but then we realize that a single method (or function) may be a unit, or a component, or may (although unlikely) be the ‘system.’ A collection of methods wrapped in a library or DLL that interact to meet a functional requirement is a component. Rather than a developer having to call each method individually to achieve some usually repetitive functional outcome from the library that functionality is usually exposed via a call to a single API.

In this situation the students are testing a public API in a single library (DLL) that calls several methods to produce a randomly generated string (outcome) based on parameterized property values. The interface is the single API call (and the property variables) in an automated test, so we might consider the DLL that contains this API as the ‘system’ under test based on Beizer’s definition of component. Also, since students don’t actually see the underlying code, API testing in this context is ‘black-box’ testing.

The debate of who is responsible for API or component testing is tangential to the practice. I promote API testing because even in today’s extreme programming and TDD development lifecycle models we testers are still finding way too many ‘functional’ bugs (as opposed to behavioral bugs) during the integration and system levels of testing.

Also, generally (not all) software is designed and developed in layers similar to this simplified illustration. In well-designed, more easily testable projects the business logic or logical functionality is (should be) contained in classes or libraries (DLLs), and the public methods or APIs in those libraries know nothing about the user interface. and visa versa. End user inputs at the GUI are marshaled to the business logic layer via event handlers and properties (get/set accessors in C#) in different classes.

So, it shouldn’t be a surprise to anyone that certain categories of functional issues are more easily exposed at the API level of testing as opposed to testing through the UI. In fact, sometimes the UI properties and event handler layers in one project may actually mask some bugs in the APIs which aren’t exposed until much later when someone else uses that API in a different application or feature. The value of API testing is that a lot of functional testing can be performed very early in the project cycle, and functional testing can progress while the UI layer is unstable or in flux.

I sometimes think we are stuck testing from the end-user’s perspective. It seems that we often approach testing by trying to expose both behavioral type bugs and functional type bugs by testing completely through the UI. But, if we think of testing in layers the same way many products are developed then I wonder if we could better focus our test designs to target specific categories of functional bugs earlier and concentrate on behavioral issues and end-2-end customer scenarios when we have a more stable UI?