I.M. Testy

When someone compares agile with “traditional’ development I have no clue what they are talking about. I have never worked in a software company that produced software as if it were a widget on a factory assembly line. I have never worked in an organization where people didn’t talk to each other constantly. I have never worked for a company that didn’t periodically reflect on its own processes and/or procedures to identify areas of improvement. I have never worked for a company that wasn’t capable of adapting to changing to ‘requirements’ when necessary. And, I would say that the majority of people that I have worked with are highly motivated individuals who strive to do the best possible work, and who are capable of adapting, improvising, and over-coming obstacles in order to ship complex world class software.

This is the way it has been for me during my almost 2 decades in this industry. So, when I read books and papers that compare agile to “traditional” approaches I ask myself, “What in the hell do they mean by “traditional” approach?

For example, when I joined the Windows 95 team we had weekly builds, and at least once per week the build had to satisfy the requirements for “self-host” status. Self-host meant that anyone in the company could use that build for their day to day work. Also, many of the self-host builds are released to key strategic industry partners and 3rd party vendors so they could take advantages of new features in their software.

After the PDC release of Windows 95 we made several key changes in the operating system based on feedback from our customers and partners. Also, late in the cycle after beta 2 we had to make a major change in how Windows started because of a quirk in a relatively popular MS DOS based game. Throughout a typical lifecycle features are added, removed, and tweaked based on several factors including customer feedback.

Instead of sprints we generally have milestones. At the end of each milestone many teams have a post-mortem or retrospective to review key objectives, discuss things that went well, things that could have been improved, and  “tune and adjust its behavior” to make sure everyone is aligned and find ways to increase effectiveness.

Most developers were also doing unit testing. (TDD is simply another way of restating Dave Gelperin’s and Bill Hetzel’s famous quote from around 1987 “Test then code.”). Testers and developers talked frequently, and on many occasions the developer I worked closely with would come to my office to debug a hard to reproduce issue. And the majority of testers who were hired in the early 90’s were typically required to have an in-depth technical and often coding background in order to be able to engage in all aspects of the software development lifecycle and perform all the various job roles that might be required from a testing professional.

By 1998 we were doing daily builds. We have well established unit test libraries that not only assist developers in refactoring, but also help prevent build breaks and down-stream regressions. We still do a lot of API (component) level testing, as well as integration and system level testing.

Of course some of our products have longer release lifecycles than others. However, as indicated earlier we deliver “working software” at least once a week internally, and even to some premier customers (especially after the third milestone leading up to the final release of that version. But, some of our teams release every month.

Stepping back and looking at the debates, the books, and conference presentations I realized that I can relate well to the Agile principles. We often use models, principles and other abstractions to describe things. And, when all is said and done Agile principles are simply another way to present concepts that are intended to enable a team of highly motivated people to work together to ship a high quality product that satisfies their customers.

We have different models to generalize our processes to help us describe our typical workflow to others. But in truth there is no single way to build software, and models are simply our abstract expression of how we view the process. In fact, each team chooses processes and procedures that work for them and their particular product in their unique context.

(BTW, I am also really tired of Agile pundits always comparing Agile to waterfall models. Why don’t they compare their understanding of Agile to the V-model, W-model, or especially to spiral or other iterative models of software lifecycles? These are models; how your team chooses to implement a development lifecycle may resemble one of these models or you may adopt things from different models to implement your processes and approaches to software development.

Remember, George Box stated, “All models are wrong, some models are useful.” We should implement the concepts embodied in the model;  we should not implement the model.)

So, to me, Agile is the ability of a team to interact with each other, to adapt to changes, to periodically reassess its productivity in order to strive to become more effective and efficient in delivering software to its customers. Agile (and every other lifecycle models) is not a process; Agile is a mindset.

This week I am in Bad Homburg, Germany where I gave a keynote at the Testing & Finance conference. It was really great to see some very dear friends and meet new colleagues from around the world. Bad Homburg is about 30 minutes by train from Frankfurt, and it is a quaint little town with beautiful parks and Schloss Homburg which was built around 1680 and was the summer residence of the German Kaiser.

Also this week the newest edition of the Automated Software Testing magazine published an article I wrote entitled UI Automation Beneath the Presentation Layer. It discusses some tips on how to use reflection for automating applications developed using managed code. If you have questions or comments about my article please let me know. Your feedback is appreciated.

I have always respected the work of Dion Johnson, and I was quite honored when he asked me to write an article for the magazine. The magazine and the Automated Testing Institute website provide a wonderful resource to people interested in software test automation. I really look forward to working with Dion and the Automated Testing Institute more closely in the future to provide whatever help I can to this monumental task he has undertaken. I also hope there are others out there who will share some of their ideas as well. I think the days of Not Invented Here (NIH) and I’ve Got A Secret (IGAS) syndrome are becoming issues of the past. We still have too much reinvention and not enough reuse of test code. I think that we have well established there is no single approach to testing that is effective in all contexts. We might approach problems from different perspectives, but I also think that all testers are passionate and have very similar goals of personal and professional improvement. To mature our discipline, and increase our effectiveness in our roles we should collaborate more and pool our resources, and the Automated Testing Institute is a growing community of professionals.

Technorati Tags: Automation,Reflection
Windows Live Tags: Automation,Reflection
WordPress Tags: Automation,Reflection

I started my career at Microsoft in 1994 working on the Windows 95 International Test team. Globalization testing is a unique specialty in software testing just like performance, security, and other specific areas of testing. Globalization testing doesn’t necessarily require a tester to be bi-lingual, or be from a country other than the United States. A good globalization tester has an in-depth understanding of such things as character encoding types and issues associated with the different types, character mapping and conversion issues, data manipulation by the application, operating system, and network protocols.

Many people might also say that globalization testers also need to know that different locales (places) around the world use different formats for date and time (national conventions). For example, in the United States the default long date format is Thursday, June 03, 2010 but in Germany it is Donnerstag, 3. Juni 2010. A tester doesn’t have to ‘read’ German to see the abstract date format has changed from dddd, MMMM dd, yyyy to dddd, d. MMMM yyyy.

Testing for support of these different national conventions used around the world is referred to as basic international sufficiency testing. I suspect the reason why some people might assume basic international sufficiency testing these different national conventions is the domain of the globalization tester is because the national conventions are set by default on the different localized versions of a software product so that’s when they are tested. But, this reasoning is absurd!

First, not all products are “localized” into all languages or ‘locales.” So, who tests the Canadian long date format of MMMM-dd-yy, or the Georgian (Georgia) long date format of yyyy ‘წლის’ dd MM, dddd? Also, Vista and later versions of Windows allow the user to ‘customize’ the date and time “format pictures” to use different separator symbols and orderings.

Secondly, way too many bugs such as hard-coded date formats are found way too late in the testing cycle (because localized versions tend to lag US English language version). And of course, we all know the cost of finding bugs later in the lifecycle are more costly to correct.

So, we must ask if there is a way for basic international sufficiency testing to be ‘pushed upstream?’ And of course the answer is yes. The easiest way is to host a “globalization bug bash” early in the cycle. (A “bug bash” is a day where testers are given some basic training on attack patterns, fault models, etc., in a general focus area and then spend a day exploring different areas of the product trying to flush out bugs in a competition style format.) Another way is to assign each tester a different locale (preferably one that is not associated with a localized language version) and have them set their test and self-host environments to that locale during their testing.

This is easily accomplished on Windows test environments by having testers launch the Regional and Language control panel applet (the short cut is Start –> Run, then type “intl.cpl” without the quotes, and press the OK button).

This just tests for a basic level of international sufficiency, and any good tester would want to explore their project’s capability to support the more than 150 different locale national conventions at a deeper level. This is especially true if your product is going to be used by customers around the world (including Canada). But, of course, we don’t want to run the same tests on all 150+ locales supported by the operating system.

The national convention settings for a particular locale are stored in a data type called the LCID, and when we change our locale (Format on the latest Regional and Language control panel applet) through the user interface we are actually calling various National Language Support (NLS) APIs. A “world-wide” application should use the universal NLS APIs and data available via the operating system.

One way to test our application’s ability to correctly use the national convention data supplied by the operating system is to set customized conventions. For example, did you know the Windows 7 operating system allows a digit grouping symbol to be a string of up to 3 characters? Or the Negative sign symbol can be a string of up to 4 characters.

Although having testers change their default locale (Format) on their test environment and self-host machines is a good first step in basic international sufficiency testing, we also want to see if our application can process a negative value of “!NEG7” instead of just “–7,” and any textboxes correctly display the customized negative sign symbol (especially at the upper extreme boundary of the textbox size property.

To customize the national convention settings we simply click the Advanced settings… button on the Formats property sheet of the Region and Language control panel applet which instantiates a new dialog with 4 property sheets for Numbers, Currency, Time, and Date.

Solution for Test Automation

That’s all well and fine for basic testing, or testing a “few” customized values, but if we wanted to test the permutations for each convention, or the combination of different conventions on numbers, currency, time, or date formats the number of tests is astronomical. Typically, testers writing an automated test would try to navigate the user interface of the Regional and Language control panel applet and the Customize Format property sheets in order to set custom conventions.

In the past I provided some code snippets for changing the convention settings on the Customize Format property sheets on versions of Windows pre-Vista. Earlier this year I also provided code snippets for customizing the date format picture and the time format picture.

That’s all well and good, but I recently released a new test automation library called GlobalTester for test developers to use in their automated test scripts. The GlobalTester library provides testers methods to set custom national conventions for the current user without having to navigate the user interface of the Region and Language options control panel applet. These national conventions include number formats, currency formats, date formats, time formats, and also current location.

The following example illustrates how we might design a test script to customize the date format for a test and reset the date format to its original setting (restoring the test environment to pre-test conditions). (Usage documentation for the GlobalTester library is on the Testing Mentor website.)

   1: namespace CustomizeDateSettingsExampleScript

   2: {

   3:   using System;

   4:   using System.Globalization;

   5:   using TestingMentor.TestTool.GlobalTester;

   6:  

   7:   class MyTest

   8:   {

   9:     static void Main()

  10:     {

  11:       try

  12:       {

  13:         CustomDateFormat time = new CustomDateFormat();

  14:  

  15:         string defaultLongDateFormat = 

  16:           CultureInfo.CurrentCulture.DateTimeFormat.LongDatePattern;

  17:         string newLongDateFormat = "MMM - d (yyyy) gg";

  18:  

  19:         if (time.ChangeLongDateFormat(newLongDateFormat))

  20:         {

  21:           // Launch AUT

  22:           // Execute test - (e.g. AUT implements long date string)

  23:           // Oracle - (e.g. compare long date format against customized pattern)

  24:  

  25:           // Reset test platform to original configuration

  26:           time.ChangeLongDateFormat(defaultLongDateFormat);

  27:         }

  28:         else

  29:         {

  30:           // Date format not changed; test not executed (e.g. invalid 

  31:           // day, month, year, and era format pictures)

  32:         }

  33:       }

  34:  

  35:       catch(ArgumentOutOfRange e)

  36:       {

  37:         // Test script failure - (e.g. long date format string argument out of range)

  38:       }

  39:  

  40:       finally

  41:       { 

  42:         // Log test results

  43:       }

  44:     } 

  45:   }

  46: }