For the past few weeks my test automation class at the University of Washington has been focused on API (application programming interface) testing, or component level testing. Boris Beizer defines component level testing as “an integrated aggregate of one or more units” and that a “component can be anything from a unit to an entire system.”
This seems a bit confusing at first but then we realize that a single method (or function) may be a unit, or a component, or may (although unlikely) be the ‘system.’ A collection of methods wrapped in a library or DLL that interact to meet a functional requirement is a component. Rather than a developer having to call each method individually to achieve some usually repetitive functional outcome from the library that functionality is usually exposed via a call to a single API.
In this situation the students are testing a public API in a single library (DLL) that calls several methods to produce a randomly generated string (outcome) based on parameterized property values. The interface is the single API call (and the property variables) in an automated test, so we might consider the DLL that contains this API as the ‘system’ under test based on Beizer’s definition of component. Also, since students don’t actually see the underlying code, API testing in this context is ‘black-box’ testing. 
The debate of who is responsible for API or component testing is tangential to the practice. I promote API testing because even in today’s extreme programming and TDD development lifecycle models we testers are still finding way too many ‘functional’ bugs (as opposed to behavioral bugs) during the integration and system levels of testing.
Also, generally (not all) software is designed and developed in layers similar to this simplified illustration. In well-designed, more easily testable projects the business logic or logical functionality is (should be) contained in classes or libraries (DLLs), and the public methods or APIs in those libraries know nothing about the user interface. and visa versa. End user inputs at the GUI are marshaled to the business logic layer via event handlers and properties (get/set accessors in C#) in different classes.
So, it shouldn’t be a surprise to anyone that certain categories of functional issues are more easily exposed at the API level of testing as opposed to testing through the UI. In fact, sometimes the UI properties and event handler layers in one project may actually mask some bugs in the APIs which aren’t exposed until much later when someone else uses that API in a different application or feature. The value of API testing is that a lot of functional testing can be performed very early in the project cycle, and functional testing can progress while the UI layer is unstable or in flux.
I sometimes think we are stuck testing from the end-user’s perspective. It seems that we often approach testing by trying to expose both behavioral type bugs and functional type bugs by testing completely through the UI. But, if we think of testing in layers the same way many products are developed then I wonder if we could better focus our test designs to target specific categories of functional bugs earlier and concentrate on behavioral issues and end-2-end customer scenarios when we have a more stable UI?
3 Comments
Hi! As a UW grad myself (‘06!)I was excited to hear that you teach there. I do a lot of testing work with APIs now, but have had trouble finding a good automation program. Are there any programs you recommend for API testing?
Great to see they are actually teaching this kind of testing at College these days.
Once everyone has a handle on this kind of functional testing it would be well worth people’s while to look into an API that will be more similar to that they are likely to find in the real-world. e.g. Hitting a database, communicating with some other external third party API, performing IO, Integrating with some other system/service.
Teaching students the value and benefit of being able to test their own code makes such a fundamental change to the way they can learn, and also write software.
Hi!
I’m an API testing engineer at SAP and i’ve encoutered with some interesting (open source!) UI test application about 2 months ago called – “Selenium” – which has .NET and Java api’s – give it a try… enjoy.
2 Trackbacks/Pingbacks
[...] API Testing: Testing in Layers – Bj Rollison presents the case for functional testing at the API level as a key way of identifying functional problems in the application, and how we don’t have to just test at the end user level. [...]
[...] parsing errors, calculation problems, etc.) while testing through the UI. But, as indicated in my previous post, well-designed software is usually built in layers and a good many of the ‘functional’ issues [...]