Skip to content

API Testing–How can it help?

After a rather wet and soggy weekend, I woke up this morning to a beautiful sunny day in Seattle. Despite it being a bit cool, I do enjoy the sunshine so much more than the dreary gray days of a Seattle winter. Most of the leaves have fallen from the trees which makes good mulch for the gardens, but just adds more work to my stack. The good news is that there is snow in the mountains and the ski resorts in the area have opened early this year, so I hope to get in some good ski days.

In the previous post I attempted to explain the subtle differences between unit testing and API testing. It should also be noted that testing at the API layer is different than testing through the GUI. API testing is primarily focused on the functionality of the business logic of the software, and not necessarily the behavior or the “look and feel” from the end user customer perspective. In fact, the 1st tier customer of the API tester is the developer who designed and develops the API. The second tier customers are the developers who utilize the APIs in building the user interface or applications that sit on top of the underlying infrastructure of a program. And finally, API testers must also consider the various end-to-end scenarios of the end user customers at the integration level of testing (without a GUI).

This post will discuss why API testing is an important activity in the complete software development lifecycle (SDLC). Teams that have multiple developers and a continuous integration (CI) build process can greatly benefit from API testing. Key benefits of API testing include:

  • Reduced testing costs
  • Improved productivity
  • Higher functional (business logic) quality   

Reduced Testing Costs

It shouldn’t be a surprise to anyone that finding most types of functional bugs early in the SDLC is more cost effective. The primary goal of unit testing, and component/integration levels of testing (API testing) is to flush out as many functional issues in the business logic layer of a software program as early as possible in the SDLC. Driving functional quality upstream not only reduces production costs, but can also reduce testing costs.

API testing can reduce the overall cost of test automation.  Automated API tests are based on the API’s interface. So, once the API interface is defined testers can begin to design and develop automated tests. Having a battery of automated tests ready as the functional APIs come on-line pushes testing upstream in parallel with development rather than later in the SDLC. This also enables earlier tester engagement and closer collaboration between testers and developers.

Also, since API interfaces are generally very stable, so automated API tests are less impacted by changes as compared to GUI based automated tests. Many testers are familiar with the constant upkeep and maintenance typically associated with GUI based automated tests. The constant massaging of GUI automation is often a huge cost in a test automation effort and a contributing factor to why so many automation projects fail. Automated API tests in general require a lot less maintenance unless there is a fundamental change in the underlying infrastructure or design of the program.

Another significant way that API testing can reduce testing costs is by refocusing testing. Many test strategies rely  heavily on finding functional bugs typically using exploratory type testing through the GUI. But, most software produced today is developed in “layers” (see Testing in Layers). A more robust test strategy should focus the bulk of functional testing at the API layer that contains the “business logic” of the program. Of course some functional issues will still be found while testing through the GUI, but the focus of testing at the GUI layer should be on  behavioral testing. A test strategy that provides a multi-tiered approach is more effective than the typical approach of throwing a bunch of bodies to bang on the GUI in an attempt to beat out the bugs. A multi-tiered test strategy may even reduce the total testing time by reducing the need to spend long cycles trying to uncover a lot of functional bugs through the GUI.

Improved Productivity

There are different ways to evaluate productivity, but certainly one way is to ensure production keeps moving forward. Continuous integration is a keystone of Agile development projects, and at Microsoft this means daily builds. If the build breaks, production grinds to a virtual halt and forward momentum is blocked until the issue is fixed. A build break negatively impacts the productivity of the entire team. A suite of low level integration tests can help identify potential build breaks especially involving dependent modules before new fixes or features are merged into a higher level branch.

API testing can also improve productivity of testing. For example, structural testing is a white box test approach intended to test the structure or flow of a program. If increased levels of code coverage is an important goal, then the most efficient way to improve structural coverage is to identify untested code paths and design and develop API level tests that will tactically target untested code.

Perhaps the most significant improvement to productivity is gained through teamwork. Building and releasing great software products require a team effort. A team of people working closely together. Gone are the days of the adversarial relationship between developers and testers. Changes in technologies, changes in customer demands, and changes in how we build software require close collaboration between developers and testers, and testers being actively engaged throughout the SDLC and not just at the beginning (picking apart a spec), or at the end (banging out  bugs via the GUI pretending to mimic a ‘customer’). A team focused on delivering high quality can greatly add to a team’s overall productivity.

Higher Functional Quality

One of the advantages and also disadvantages of testing at the API layer is that you can test the API in ways that are different then how the GUI interacts with the API. For example, the Morse Code Trainer has an interface for the methods that parse the dots and dashes and plays a system beep of 1 unit duration for each dot in the stream, and a system beep for 3 units of duration for each dash in the stream. The duration of a unit is based on the WordsPerMinute property value.

   1:    interface ISoundGenerator
   2:    {
   3:      void PlayMorseCode(string morseCodeString);
   4:   
   5:      int WordsPerMinute { get; set; }
   6:    }
 

Testing this property at the API level we could “set” a negative integer value to make sure nothing really bad happens. But, a well-designed GUI would never accept an integer value less than 1 (which is painfully slow) nor above 150 (which is ridiculously high). A better design might be to use a drop-down list of values ranging from 5 (the minimum requirement for a basic license) to 20 words per minute (required for the highest level amateur radio operator license). Of course, it may be possible to find functional anomalies while API testing that could not be found via testing through the GUI. But, the important thing an API tester must consider is how a bug found at the component or integration levels of testing adversely affects a scenario, or the customer.

API testers work alongside of the developers. An API tester may also provide input into the initial API design, engage in code reviews before check-ins, and of course write automated tests to test the API (component level) and APIs in end-to-end scenarios (integration level). Having testers engage with developers early and throughout the SDLC helps ensure team work and instills the idea that quality is a collaborative effort.

2 Comments

  1. Johan Hoberg wrote:

    Another very good and valuable post. Thank you for providing great reference material! :)

    Tuesday, December 6, 2011 at 1:35 AM | Permalink
  2. Great post, Bj. Automating API tests also goes with the key principle of “Test Early. Test Often.” and indeed helps improve productivity through tester/developer collaboration. This emphasizes the need for testers to be familiar with programming so they can write and maintain automated API tests.

    Tuesday, December 6, 2011 at 11:35 AM | Permalink

3 Trackbacks/Pingbacks

  1. API Design for C++ on Tuesday, December 6, 2011 at 11:12 PM

    [...] http://www.testingmentor.com/imtesty/2011/12/01/api-testinghow-can-it-help/ [...]

  2. Weekly Article Links #3 « Practical Tips on Software Testing on Tuesday, January 3, 2012 at 11:27 PM

    [...] plan sample Webform Best Practices Anatomy of a good bug report API Testing–How can it help? Why Checking Is Not [...]

  3. [...] • I.M.Testy прогуливается в Сиэтле, наслаждается солнечным днем и размышляет о пользе тестирования API. [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *
*
*

ivanoff.shaunte@mailxu.com pestadarcie@mailxu.com