Many people might also say that globalization testers also need to know that different locales (places) around the world use different formats for date and time (national conventions). For example, in the United States the default long date format is Thursday, June 03, 2010 but in Germany it is Donnerstag, 3. Juni 2010. A tester doesn’t have to ‘read’ German to see the abstract date format has changed from dddd, MMMM dd, yyyy to dddd, d. MMMM yyyy.

Testing for support of these different national conventions used around the world is referred to as basic international sufficiency testing. I suspect the reason why some people might assume basic international sufficiency testing these different national conventions is the domain of the globalization tester is because the national conventions are set by default on the different localized versions of a software product so that’s when they are tested. But, this reasoning is absurd!

First, not all products are “localized” into all languages or ‘locales.” So, who tests the Canadian long date format of MMMM-dd-yy, or the Georgian (Georgia) long date format of yyyy ‘წლის’ dd MM, dddd? Also, Vista and later versions of Windows allow the user to ‘customize’ the date and time “format pictures” to use different separator symbols and orderings.

Secondly, way too many bugs such as hard-coded date formats are found way too late in the testing cycle (because localized versions tend to lag US English language version). And of course, we all know the cost of finding bugs later in the lifecycle are more costly to correct.

So, we must ask if there is a way for basic international sufficiency testing to be ‘pushed upstream?’ And of course the answer is yes. The easiest way is to host a “globalization bug bash” early in the cycle. (A “bug bash” is a day where testers are given some basic training on attack patterns, fault models, etc., in a general focus area and then spend a day exploring different areas of the product trying to flush out bugs in a competition style format.) Another way is to assign each tester a different locale (preferably one that is not associated with a localized language version) and have them set their test and self-host environments to that locale during their testing.

This is easily accomplished on Windows test environments by having testers launch the Regional and Language control panel applet (the short cut is Start –> Run, then type “intl.cpl” without the quotes, and press the OK button).

This just tests for a basic level of international sufficiency, and any good tester would want to explore their project’s capability to support the more than 150 different locale national conventions at a deeper level. This is especially true if your product is going to be used by customers around the world (including Canada). But, of course, we don’t want to run the same tests on all 150+ locales supported by the operating system.

The national convention settings for a particular locale are stored in a data type called the LCID, and when we change our locale (Format on the latest Regional and Language control panel applet) through the user interface we are actually calling various National Language Support (NLS) APIs. A “world-wide” application should use the universal NLS APIs and data available via the operating system.

One way to test our application’s ability to correctly use the national convention data supplied by the operating system is to set customized conventions. For example, did you know the Windows 7 operating system allows a digit grouping symbol to be a string of up to 3 characters? Or the Negative sign symbol can be a string of up to 4 characters.

Although having testers change their default locale (Format) on their test environment and self-host machines is a good first step in basic international sufficiency testing, we also want to see if our application can process a negative value of “!NEG7” instead of just “–7,” and any textboxes correctly display the customized negative sign symbol (especially at the upper extreme boundary of the textbox size property.

To customize the national convention settings we simply click the Advanced settings… button on the Formats property sheet of the Region and Language control panel applet which instantiates a new dialog with 4 property sheets for Numbers, Currency, Time, and Date.

Solution for Test Automation

That’s all well and fine for basic testing, or testing a “few” customized values, but if we wanted to test the permutations for each convention, or the combination of different conventions on numbers, currency, time, or date formats the number of tests is astronomical. Typically, testers writing an automated test would try to navigate the user interface of the Regional and Language control panel applet and the Customize Format property sheets in order to set custom conventions.

In the past I provided some code snippets for changing the convention settings on the Customize Format property sheets on versions of Windows pre-Vista. Earlier this year I also provided code snippets for customizing the date format picture and the time format picture.

That’s all well and good, but I recently released a new test automation library called GlobalTester for test developers to use in their automated test scripts. The GlobalTester library provides testers methods to set custom national conventions for the current user without having to navigate the user interface of the Region and Language options control panel applet. These national conventions include number formats, currency formats, date formats, time formats, and also current location.

The following example illustrates how we might design a test script to customize the date format for a test and reset the date format to its original setting (restoring the test environment to pre-test conditions). (Usage documentation for the GlobalTester library is on the Testing Mentor website.)

   1: namespace CustomizeDateSettingsExampleScript

   2: {

   3:   using System;

   4:   using System.Globalization;

   5:   using TestingMentor.TestTool.GlobalTester;

   6:  

   7:   class MyTest

   8:   {

   9:     static void Main()

  10:     {

  11:       try

  12:       {

  13:         CustomDateFormat time = new CustomDateFormat();

  14:  

  15:         string defaultLongDateFormat = 

  16:           CultureInfo.CurrentCulture.DateTimeFormat.LongDatePattern;

  17:         string newLongDateFormat = "MMM - d (yyyy) gg";

  18:  

  19:         if (time.ChangeLongDateFormat(newLongDateFormat))

  20:         {

  21:           // Launch AUT

  22:           // Execute test - (e.g. AUT implements long date string)

  23:           // Oracle - (e.g. compare long date format against customized pattern)

  24:  

  25:           // Reset test platform to original configuration

  26:           time.ChangeLongDateFormat(defaultLongDateFormat);

  27:         }

  28:         else

  29:         {

  30:           // Date format not changed; test not executed (e.g. invalid 

  31:           // day, month, year, and era format pictures)

  32:         }

  33:       }

  34:  

  35:       catch(ArgumentOutOfRange e)

  36:       {

  37:         // Test script failure - (e.g. long date format string argument out of range)

  38:       }

  39:  

  40:       finally

  41:       { 

  42:         // Log test results

  43:       }

  44:     } 

  45:   }

  46: }

Time and date information is commonly pulled from the operating system by many developers for use in headers or footers on documents, default file names, printing, and other places time/date stamps are useful or important. To ensure our products are “world-ready” we should modify the formats to validate whether our product supports various national conventions used in different regions (locales) around the world. In the previous post I illustrated how to programmatically customize the date formats on a Windows environment for including some basic globalization tests in your test automation. This week let’s look at how we can programmatically change both the short time and long time formats.

We will again need the 2 Win32 API functions SetLocaleInfo() and PostMessage() that we marshaled over into the NativeMethods class. Since that code doesn’t change I won’t repeat it here you can simply refer to the code snippet in the previous post. In this situation we need to set the lcType in SetLocaleInfo() to the LOCALE_STIMEFORMAT constant. Then we can pass a null-terminated string to the lcData variable in the SetLocaleInfo() function. MSDN explains “The maximum number of characters allowed for this string is 80, including a terminating null character. The string can consist of a combination of hour, minute, and second format pictures.”

Once again, to simplify that a bit I wrote some more wrapper methods to change the time format. Also, since we will be calling SetLocaleInfo()  and PostMessage() a lot for customizing date, time, and other national conventions I created a wrapper method called UpdateLocaleInformation() to remove redundancy.

   1: namespace TestingMentor.TestTool.GlobalTester

   2: {

   3:   using System;

   4:  

   5:   public enum TimeFormatType

   6:   {

   7:     LongTimeFormat = 0x00001003,

   8:     ShortTimeFormat = 0x00000079

   9:   }

  10:  

  11:   public class CustomTimeFormat

  12:   {

  13:     private int timeFormatType = (int)TimeFormatType.ShortTimeFormat;

  14:     private string timeFormatPicture = string.Empty;

  15:  

  16:     public int SetTimeFormatType

  17:     {

  18:       set

  19:       {

  20:         if (value == (int)TimeFormatType.ShortTimeFormat ||

  21:           value == (int)TimeFormatType.LongTimeFormat)

  22:         {

  23:           this.timeFormatType = value;

  24:         }

  25:         else

  26:         {

  27:           throw new ArgumentOutOfRangeException("TimeFormatType invalid");

  28:         }

  29:       }

  30:     }

  31:   

  32:     public string SetTimeFormatPicture

  33:     {

  34:       set { this.timeFormatPicture = value; }

  35:     }

  36:  

  37:     public bool ChangeTimeFormat()

  38:     {

  39:       return UpdateLocaleInformation(

  40:         this.timeFormatType,

  41:         this.timeFormatPicture);

  42:     }

  43:  

  44:     private bool UpdateLocaleInformation(int localeType, string localeData)

  45:     {

  46:       bool success = false;

  47:       if (NativeMethods.SetLocaleInfo(

  48:         NativeMethods.SystemDefaultLocale,

  49:         localeType,

  50:         localeData))

  51:       {

  52:         NativeMethods.PostMessage(

  53:           NativeMethods.BroadcastMessage,

  54:           NativeMethods.SettingChangeMessage,

  55:           IntPtr.Zero,

  56:           IntPtr.Zero);

  57:         success = true;

  58:       }

  59:  

  60:       return success;

  61:     }

  62:   }

  63: }

Once again, we simply have to set the SetTimeFormatType property to either the Short time or Long time format, provide the format picture by setting the SetTimeFormatPicture property, and then call ChangeTimeFormat(). The sample below illustrates how to change the short time format with different time separators and a reverse order.

   1: static void Main(string[] args)

   2: {

   3:   CustomTimeFormat time = new CustomTimeFormat();

   4:   time.SetTimeFormatType = (int)TimeFormatType.ShortTimeFormat;

   5:   time.SetTimeFormatPicture = "ss'mm,hh - tt";

   6:   if (time.ChangeTimeFormat())

   7:   {

   8:     Console.WriteLine("Success");

   9:   }

  10: }

Now, we can also customize the AM/PM designator as well. To change the AM/PM designator we need to add a few more properties and another wrapper method. In this case, I’ve added the SetAmPmDesignator property, the SetAmPmString property, and the ChangeAmPmDesignator() method.

   1: public enum AmPmDesignator

   2: {

   3:   AM = 0x00000028,

   4:   PM = 0x00000029

   5: }

   6:  

   7: public class CustomTimeFormat

   8: {

   9:   public int SetAmPmDesignator

  10:   {

  11:     set

  12:     {

  13:       if (value == (int)AmPmDesignator.AM || value == (int)AmPmDesignator.PM)

  14:       {

  15:         this.designatorForAmPm = value;

  16:       }

  17:       else

  18:       {

  19:         throw new ArgumentOutOfRangeException("AmPmDesignator invalid.");

  20:       }

  21:     }

  22:   }

  23: }

  24: public string SetAmPmString

  25: {

  26:   set { this.timeDesignator = value; }

  27: }

  28:  

  29: public bool ChangeAmPmDesignator()

  30: {

  31:   return UpdateLocaleInformation(

  32:     this.designatorForAmPm,

  33:     this.timeDesignator);

  34: }

The code snippet below illustrates how to change the AM designator from “AM” to “In the morning.”

   1: static void Main(string[] args)

   2: {

   3:   CustomTimeFormat time = new CustomTimeFormat();

   4:   time.SetAmPmDesignator = AmPmDesignator.AM;

   5:   time.SetAmPmString = "In the morning.";

   6:   if (time.ChangeAmPmDesignator())

   7:   {

   8:     Console.WriteLine("Success");

   9:   }

  10: }

Modifying national conventions is one way to test for globalization support upstream and should be done early in the testing cycle rather than relying on a separate globalization testing cycle. Time and date are perhaps the most visible national conventions used in many different ways in our applications. We should test the common (equivalent) conventions used in various regions around the world, and customizing these settings helps ensure the developer is properly calling NLS APIs and not using custom functions.

Also, check out the beta release of the GlobalTester automation library that has this functionality and more, and let me know what you think.

There are several settings that we can modify and customize for more robust globalization testing such as number, currency, time and date formats. Modifying these settings can help us test that our application is globalized to use National Language System (NLS) APIs provided by the system.Although a user would change these settings using the Regional Options user interface property sheets, if the purpose of our test is not to emulate user interaction, then modifying the custom regional settings for globalization testing programmatically is more efficient.

Last year I talked about how to programmatically make changes to the settings in the Region and Language control panel applet when doing globalization testing. Unfortunately, the code sample provided in the previous post was appropriate for versions of Windows XP and earlier. For versions of Windows Vista and later things have changed a bit. Also, the previous sample tried to be a one-size fits all and relied on the test developer to set the appropriate lcType constants and lcData argument variables required by the Win32 function SetLocaleInfo().

This time, I decided to simplify things a bit and wrapped some methods to call the appropriate Win32 API functions and properties to set lcType and lcData values to make it easier to incorporate into automated tests. I also separated the various advanced custom formats for Region and Language options into separate classes. Of course, I have a beta version of an automation library (DLL) called GlobalTest.DLL on my website that testers can use in their automated test cases, but this week let’s look at the class for setting custom date formats.

Making these changes programmatically still requires the Win32 SetLocaleInfo() function. MSDN also states this function modifies the specified values for all applications, so to prevent potential issues in other applications running on the system we should also broadcast the WM_SETTINGCHANGE message. To broadcast the WM_SETTINGCHANGE message we will also need the Win32 PostMessage() function. Since we are Process Invocation (PInvoke) to call these unmanaged functions we should put them in a separate class that I’ve called NativeMethods. I also included all necessary constant values required by these methods in the NativeMethods class also as illustrated below.

   1: namespace TestingMentor.TestTool.GlobalTester

   2: {

   3:   using System;

   4:   using System.Runtime.InteropServices;

   5:  

   6:   internal sealed class NativeMethods

   7:   {

   8:     internal const int SystemDefaultLocale = (int)0x00000800;

   9:     internal const int BroadcastMessage = (int)0x0000FFFF;

  10:     internal const int SettingChangeMessage = (int)0x0000001A;

  11:  

  12:     private NativeMethods() { }

  13:  

  14:     [DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]

  15:     [return: MarshalAs(UnmanagedType.Bool)]

  16:     internal static extern bool SetLocaleInfo(

  17:       int locale,

  18:       int localeType,

  19:       string localeData);

  20:  

  21:     [DllImport("user32.dll", SetLastError = true)]

  22:     [return: MarshalAs(UnmanagedType.Bool)]

  23:     internal static extern bool PostMessage(

  24:       int handle,

  25:       int message,

  26:       IntPtr wParam,

  27:       IntPtr lParam);

  28:   }

  29: }

The class for the custom wrapper method is TestingMentor.TestTool.GlobalTester.SetDateFormat. There is a public enumeration for the short date and long date constants. One of these values must be assigned to the SetDateType property. The other property that must be set is the SetDateFormatPicture. The big change in the SetLocaleInfo() function is that the lcData type is a null-terminated string that MSDN refers to as a format picture. Current versions of Windows allow users to customize the order of the month, day and year, the format for each, and even allow different separators between the date elements. The format picture enables the user to select various format types in different orders for either the short date or the long date. See MSDN’s Month, Day, Year and Era Format Pictures for the various supported format types.

   1: namespace TestingMentor.TestTool.GlobalTester

   2: {

   3:   using System;

   4:  

   5:   public enum DateFormatType

   6:   {

   7:     ShortDate = 0x0000001F,

   8:     LongDate = 0x00000020

   9:   }

  10:  

  11:   public class CustomDateFormat

  12:   {

  13:     private string dateFormatPicture = string.Empty;

  14:     private int dateType = (int)DateFormatType.ShortDate;

  15:  

  16:     public string SetDateFormatPicture

  17:     {

  18:       set { this.dateFormatPicture = value; }

  19:     }

  20:  

  21:     public int SetDateType

  22:     {

  23:       set

  24:       {

  25:         if (value == (int)DateFormatType.ShortDate ||

  26:           value == (int)DateFormatType.LongDate)

  27:         {

  28:           this.dateType = value;

  29:         }

  30:         else

  31:         {

  32:           throw new ArgumentOutOfRangeException("Invalid DateType");

  33:         }

  34:       }

  35:     }

  36:  

  37:     public bool ChangeDateFormat()

  38:     {

  39:       bool success = false;

  40:       if (NativeMethods.SetLocaleInof(

  41:         NativeMethods.SystemDefaultLocale,

  42:         this.dateType,

  43:         this.dateFormatPicture))

  44:       {

  45:         NativeMethods.PostMessage(

  46:           NativeMethods.BroadcastMessage,

  47:           NativeMethods.SettingChangeMessage,

  48:           IntPtr.Zero,

  49:           IntPtr.Zero);

  50:       }

  51:  

  52:       return success;

  53:     }

  54:   }

  55: }

Once the SetDateType and SetDateFormatPicture properties are assigned we simply have to call ChangeDateFormat() method to change the settings and broadcast the message to the system. The code snippet below illustrates how a tester would change the default long date format in an automated test to determine globalization support in the application under test. Customizing the date format is useful if the application under test uses a date string in any way. For example, if the application includes a function to insert a date string in an edit control, or if the date is printed as a header or footer in a document, or if a date string is appended to a record.

   1: using TestingMentor.TestTool.GlobalTester;

   2: ...

   3:     static void Main(string[] args)

   4:     {

   5:       CustomDateFormat date = new CustomDateFormat();

   6:       date.SetDateType = (int)DateType.LongDate;

   7:       date.SetDateFormatPicture = "[ dd % MM | yyyy ]";

   8:       if (date.ChangeDateFormat())

   9:       {

  10:         Console.WriteLine("Long date was changed");

  11:       }

  12:     }

Programmatically changing the date format is an easy way testers can customize date formats in their automated tests without having to manipulate the controls on Region and Language property sheet. Also note, that since the format picture is a string the order of the supported date format types is now controlled by the arrangement in the string, and the separator characters can be different between the day and month and the month and year as illustrated in the example above.

Modifying national conventions is one way to test for globalization support upstream and should be done early in the testing cycle rather than relying on a separate globalization testing cycle.

Next week I will discuss customizing the time format. Also, check out the beta release of the GlobalTester automation library that has this functionality and more and let me know what you think.

The past series of posts have focused on one of localization testing which describes the largest category of localization class issues reported by testers performing localization testing, and what we categorize as usability/behavioral type issues because they adversely impact the usability of the software or how end users interact with the product. This is the last post in this series, but I do intend to publish a more complete paper covering localization testing in the near future….stay tuned. This final post in this series will discuss issues that affect the layout of controls on a dialog or window and are generally referred to as clipping or truncation.

Clipping

Clipping occurs when the top or bottom portion of a control (including label controls that contain static text) is cut off and does not display the control or the control’s contents completely as illustrated below. Clipping and truncation is quite common on East Asian language versions because the default font size used in Japanese, Korean, and Chinese language versions is a 9 point font instead of the 8 point font used in English and other language versions. Clipping often occurs because developers fail to size controls adequately for larger fonts (especially common in East Asian language versions), or for display resolutions set to custom font sizes. Clipping also occurs because many localization tools are incapable of displaying a true WYSIWYG or runtime view of dialogs, requiring localizers to ‘guess’ when resizing control on dialog layouts.

It is possible to test for potential clipping and truncation problem areas without a localized application. English language version should function and display properly on all localized language versions of the Windows operating system. So, one way to check for potential clipping or truncation issues is to install the English language version of the application under test on an East Asian language version of the Windows operating system. Another testing method to test for potential clipping and truncation issues is to change the Windows display appearance or the custom font size via the Display Properties control panel applet.

However, due to the limitations of most current localization tools inability to dynamically resized controls and dialogs, and inability to display dialogs at runtime or present a true WYSIWYG view during the localization process, the localized language versions must also be tested for clipping and truncations caused by improper sizing and layout of controls.

Truncation

Truncation is similar to clipping, but typically occurs when the right side of controls are cut off (or the left side of the controls in bi-directional displays used in Hebrew and Arabic languages) and do not completely display the entire control or the control’s contents.

Other Layout Issues

Because some localization tools may not provide a true ‘WYSIWYG’ display of what a dialog or property sheet will look like at runtime, occasionally resizing may cause several controls to overlap. This is especially true when dialogs contain dynamic controls that are dependent on certain configurations or machine states.

In East Asian cultures it is common for an individual’s surname to precede the given (first) name. (It is also uncommon to have a middle name, so this field should never be required.) Therefore, the controls for name type data may need to be repositioned on dialogs in East Asian language versions. The localization team will reposition the last name label and textbox controls and the given name controls. This means that the logical tab order be reset. Also, the surname textbox control should have focus when the dialog is displayed instead of the first given name field.

The tab order of controls should allow for easy, intuitive navigation of a dialog. Design guidelines suggest a tab order that changes the focus of controls from left to right and top to bottom. Focus should change between each control in a logical order, and dialogs should never have loss of tab focus’ where no control on the dialog appears to have focus.

Tab order is typically problematic even in English language versions in the early lifecycle of many projects when the user interface is in flux. There is also a high probability of introducing tab order problems any time the controls on a dialog change.

All localization testing doesn’t have to be manual

In the past much of the localization testing has been repetitive manual testing. Testers would manually step through every menu item and other link to instantiate every dialog and property sheet in the program and inspect it visually and test the behavior of such things as tab order, access keys, etc. for errors. This painstaking process would be repeated multiple times during the project lifecycle on every localized language version. Unfortunately, not only was this boringly repetitive, but because the manual testers were looking at so many dialogs during the workday their eyes simply tired out leading to missed bugs. So, there must be a better way.

We know that each dialog has a 2-dimensional size usually measured in pixels. Once we know the height and width of the dialog or property sheet we can measure the distance from the left most edge of the dialog to the leading edge of the first control. Using control properties such as size and location that are stored in the form’s resource file we can measure the size and position of each control on a dialog or property sheet. Once all controls are identified the distance and position of the controls can then be measured in relation to the dialog or property sheet and other controls.

Using a simple example, let’s consider 1 dimension of a dialog as 250 pixels wide. The dialog contains a label control that is 15 pixels to the right of the left most edge of the dialog, and that label is 45 pixels in length. The textbox control next to the label starts at position 70, so there are 10 pixels between the right edge of the label control and the left edge of the textbox control. Now, let’s say that textbox control is 150 pixels wide. By calculating the width of the 2 controls plus the distance between the controls we can see that truncation will occur on this dialog. Similarly, we can also evaluate the relative position of controls on a dialog and detect alignment both horizontally and vertically more accurately than the human eye.

Of course this is not a simple solution, but if you have thousands of dialogs and property sheets, and multiple language versions investing in an automated solution may be invaluable. One internal case study testing efficiency increased and significantly reduced manual testing and overall direct costs, and the effectiveness/accuracy of reported issues also increased. Perhaps not for everyone, but it is possible!

Part 1 provided an overview of localization class issues, and Part II discussed issues with non-translated strings in a localized product and gave some helpful hints to manage that problem during the software development lifecycle. In Part III I will cover various issues with access key mnemonics. An access key mnemonic is the underlined letter on a menu or control that corresponds to a key on the native keyboard layout for a particular language.

Missing & duplicate access key mnemonics

Interestingly enough, most localization tools have built in tools to test for duplicate key mnemonics; however, missing or duplicate access key mnemonics is another significant issue in localization testing, and also affects the English language version as well. Duplicate or missing key mnemonics can adversely affect the usability of software because it impacts the ability of the user to easily access or invoke commonly repeated functions using the keyboard. Duplicate or missing duplicate key mnemonics can also negatively impact the software’s ability to meet certain accessibility requirements.

Although missing or duplicate access key mnemonics are sometimes caused by poorly designed dialogs with an overabundance of controls, there are other factors that can cause duplicate key mnemonics. For example, some controls may dynamically appear in some dialogs in specific machine states. These dynamically generated controls may also come from a file that is different than the file which generated the dialog. Another reason for duplicate key mnemonics could also be dynamically generated key mnemonic assignments, which are especially problematic in situations where a dialog contains a mixture of statically assigned key mnemonics and dynamically generated key mnemonics.

Manual testing for missing or duplicate key mnemonics is especially labor intensive, and finding ways to automate this testing will save countless hours of time sitting in front of a computer checking menus and dialogs. There is also a large probability of missing duplicate key mnemonic assignment problems using manual testing methods because eyes get tired, people get bored, and some keys are grayed out (as in the illustration below) or may not be present in certain machine states. Fortunately, there are several automation tools that detect duplicate key mnemonic problems and automated detection is more effective than manual test approaches. For example, Automation.Element.AccessKeyProperty in the UIAutomation class library in C# is one approach to more efficiently test access key mnemonics.

Access key mnemonic assignments

As a general rule of thumb (heuristic), key mnemonics should be assigned to characters mapped to the default state of the keyboard for each particular language. For the Latin 1 family of languages, access key mnemonics should generally not be assigned to non-ASCII characters; even if that particular character is accessible on the default state keyboard layout for a particular language. Certain, access key mnemonics should never be assigned to a character glyph that is formed through combining characters used in languages such as Thai and Arabic. Also, punctuation characters should never be assigned as access key mnemonics.

Of course, the default keyboard layout for many non-Latin 1 languages only contain characters in the native script for that language, and assigning non-ASCII characters as access key mnemonics may be the only choice. However,

Japanese hiragana and katakana glyphs, Korean Hangeul glyphs, and all East Asian ideographic glyphs are invalid character assignments for access key mnemonics. The default keyboard layout for most East Asian languages (Japanese, Korean, Simplified Chinese, and Traditional Chinese) is the standard keyboard layout similar to the US English keyboard. In the above example, their is no way for a Japanese user to access the ‘My Computer’ (マイ コンピユータ）menu item because it is using a katakana character as an access key mnemonic (which violates several guidelines for access key mnemonic assignment). Also, the standard key mnemonic guidelines described below should be used for all East Asian language versions for consistency and backwards compatibility.

Another general guideline to follow for access key mnemonic assignments is to avoid the lower case Latin letters ‘g’ ‘y’ ‘p’ ‘q’ and ‘j’ because there is a high probability of confusion especially with high display resolutions especially with the letters i and l, and q and g. These letters could also be hard to discern on high resolution desktop settings as well. If the number of controls on a single dialog or in a menu list require usage of inappropriate key mnemonics, then perhaps the problem is the design of the dialog.

East Asian language versions should use the identical key mnemonics as the English language version. The characters assigned as key mnemonics in the East Asian language are capitalized, enclosed within parenthesis, and positioned at the end of the translated string. Even when a key mnemonic appears within words or acronyms which are not translated or transliterated into the target the key mnemonic should be relocated to the end of the string and enclosed within parenthesis for consistency.

The character assigned as the key mnemonic should be capitalized because many East Asian computer users use an English key keyboard, and for users whose native language does not frequently employ Latin characters it is much easier for those users to visually identify key mnemonics which are capitalized with keys on the keyboard which are also capital case characters.

Accelerator Keys

Accelerator keys are commonly refered to as shortcut keys.  Accelerator keys are keys (such as F1 – F12 and Esc) or key combinations (Ctrl + Shift + B, or Ctrl C) that allow users to evoke certain functions without navigating the software menus via access keys or using the mouse to click button controls on a toolbar. Here is good source for common Windows accelerator keys, and here is one for common accelerator keys for Office products.

Shortcut key combinations are common throughout all language versions. Contrary to the Wikipedia entry on the subject some language versions localize the letter key (not a mnemonic…it is not underlined). For example,  in German the Ctrl key is localized as "Strg" and, dispite it is generally frowned upon to change the ASCII upper case letter assigned to an accelerator key combination the Spanish versions of Office use Ctrl+G (Guardar) for Save instead of Ctrl+S, and Ctrl+N (Negrita) for Bold instead of Ctrl+B. Also, letter keys used as part of an accelerator key combination are capitalized. East Asian language versions use Ctrl to designate the Control key. Also, accelerator key combinations do include an elipsis after the letter.

In Part IV I will discuss common layout issues such as clipping and truncation.

I should be of no surprise to anyone that localization testing generally focuses on changes in the user interface, although as mentioned in the previous post these are not the only changes necessary to adapt a product to a specific target market. But, the most common category of localization class bug are usability or behavioral type issues that do involve the user interface. Bugs in this category generally include un-localized or un-translated text, key mnemonics, clipped or truncated text labels and other user interface controls, incorrect tab order, and layout issues. Fortunately, the majority of problems in this category do not require a fix in the software’s functional or business layer. Also, the majority of problems in this category do not require any special linguistic skills in order to identify, and in some cases, an automated approach can be even more effective than the human eye (more on that later).

Perhaps the most commonly reported issue in this category is “un-localized” or un-translated textual string. Unfortunately, in many cases un-translated strings is also an over-reported problem that only serves to flood the the defect tracking database with unnecessary bugs. Translating textual strings is a demanding task, and made even more difficult when there are constant changes in the user interface or contextual meaning of messages early in the product life cycle. Over-reporting of un-translated text too early in the product cycle only serves to artificially inflate the bug count, and causes undue pressure and creates extra work for the localization team.

Identifying this type of bugs is actually pretty easy. Here’s a simple heuristic; if you are testing a non-English language version in a language you are not familiar with and you can clearly read the textual string in English it is probably not localized or translated into the target language. The illustration below provides a pretty good example of this general rule of thumb. A tester doesn’t have to read German to realize that the text in the label control under the first radio button is not German.

There are several causes of un-localized text strings to appear in dialogs and other areas of the user interface. For example:

• Worse case scenario is that the string is hard-coded into the source files
• Perhaps localizers did not have enough time to completely process all strings in a particular file
• Perhaps this is a new string in a file localizers thought was 100% localized
• Strings displayed in some dialogs come from files other than the file that generates the dialog, and the localization team has not process that file
• And, sometimes (usually not often), a string may simply be overlooked during the localization process

Testing for un-localized text is often a manually intensive process of enumerating menus, dialogs, and other user interface dialogs, message boxes and form, and form elements.  But, if the textual strings are located in a separate resource file (as they should be), a quick scan of resource files might more quickly reveal un-translated textual strings. Of course, there is little context in the resource file, and I also hope the localization team is reviewing their own work as well prior to handing it over to test.

Also, here are a few suggestions that might help focus localization testing efforts early in the project milestone and reduce the number of ‘known’ or false-positive un-translated text bugs being reported:

• Ask the localization team to report the percentage of translation completion by file or module for each test build. Early in the development lifecycle only modules that are reported to be 100% complete which appear to have un-translated text should be reported as valid bugs. Of course, sometimes some strings are used in multiple modules, or may be coming from external resources. But, especially early in the development lifecycle reporting a gaggle of un-translated text bugs is simply “make work.” As the life cycle starts winding down…all strings are fair game for bug hunters!
• Testers should use tools such a Spy++ or Reflector to help identify the module or other resources, and the unique resource ID for the problematic string or resource. This is much better then than simply attaching an image of the offending dialog to a defect report. Identifying the module and the specific resource ID number allows the localization team to affect a quick fix instead of having to search for the dialog through repro steps and track down the problem.
• Also remember that not all textual strings are translated into a specific target language. Registered or trademarked product names are often not translated into different languages. In case of doubt, ask the localization team if a string that appears un-localized is a ‘true’ problem or not.

Unlocalized strings usually due to hard coded strings also tend to occur in menu items. This is especially true in the Windows Start menu or sub-menu items hard-coded in the INF or other installation/setup files. For example, the image on the right shows a common problem on European versions of Windows. Many European language versions  localize the name of the Program Files folder, and the menu item in the start menu. But, often times when we install an English language version of software to Windows it creates a new "Programs" menu item (and even a new Program Files directory, rather than detecting the default folder to install to. In the example on the left, the string Accessories is a hard-coded folder name. But, there is another issue as well. This illustrates not only a problem with the non-translated string "Accessories," but also shows one full-width Katakana string for ‘Accessories’ and another half-width string.

In part 3 I will discuss another often problematic area in localization….key mnemonics.

When I first joined Microsoft 15 years ago I was on the Windows 95 International team. Our team was responsible for reducing the delta between the release of the English version and the Japanese version to 90 days, and I am very proud to say that we achieved that goal and Windows 95 took Japan by storm. It was so amazing that even people without computers were lined up outside of sales outlets waiting to purchase a copy of Windows 95. The growth of personal computers in Japan shot through the roof over the next few years. Today the Chinese market is exploding, and eastern European nations are experiencing unprecedented growth as well.  While the demand for the English language versions of our software still remains high, many of our customers are demanding software that is ‘localized’ to accommodate the customers national conventions, language, and even locally available hardware. Although much of the Internets content is in English, non-English sites on the web are growing, and even ICANN is considering allowing international domain names that contain non-ASCII characters this week in Seoul, Korea.

But, a lot has changed in how we develop software to support international markets. International versions of Windows 95 were developed on a forked code base. Basically, this means the source code contained #ifdefs to instruct the compiler to compile different parts of the source code depending on the language family. From a testing perspective this is a nightmare, because if the underlying code base of a localized version is fundamentally different than the base (US English) version then the testing problem is magnified because there is a lot of functionality that must be retested. Fortunately today, much software being produced is based on a single-worldwide binary model. (I briefly explained the single world wide binary concept at a talk in 1991, and Michael Kaplan talks about the advantages here.) In a nutshell, a single worldwide binary model is a development approach in which any functionality any user anywhere in the world might need is codified in the core source code so we don’t need to modify the core code once it is compiled to include some language/locale specific functionality.  For example, it was impossible to input Japanese text into Notepad on an English version of Windows 95 using an Input Method Editor (IME); I needed the localized Japanese version. But, on the English version of Windows Xp, Vista, or Windows 7 all I have to do is install the appropriate keyboard drivers and font files and expose the IME functionality. In fact, these days I can map my keyboard to over 150 different layouts and install fonts for all defined Unicode characters on any language version of the Windows operating system.

The big advantage of the single worldwide binary development model is that it allows us to differentiate between globalization testing and localization testing.  At Microsoft we define globalization as “the process of designing and implementing a product and/or content (including text and non-text elements) so that it can accommodate any locale market (locale).” And, we define localization as “the process of adapting a product and/or content to meet the language, cultural and political expectations and/or requirements of a specific target market.” This means we can better focus on the specific types of issues that each testing approach is most effective at identifying. For localization testing, this means we can focus on the specific things that change in the software during the “adaptation processes” to localize a product for each specific target market.

The most obvious adaptation process is the ‘localization’ or actually the translation of the user interface textual elements such as menu labels, text in label controls, and other string resources that are commonly exposed to the end user. However, the translation of string resources is not the only thing that occurs during the localization process. The localization processes that are required to adapt a software product to a specific local may also include other changes such as font files and drivers installed by default, registry keys set differently, drivers to support locale specific hardware devices, etc.

3 Categories of Localization Class Bugs

I am a big fan of developing a bug taxonomic hierarchy as part of my defect tracking database as a best practice because it better enables me to analyze bug data more efficiently. If I see a particular category of bug or a type of bug in a category that is being reported a lot, then perhaps we should find ways to prevent or at least minimize the problem from occurring later in the development lifecycle. After years of analyzing different bugs, I classified all localization class bugs into 3 categories; functionality, behavioral/usability, and linguistic quality.

Functionality type bugs exposed in localized software affect the functionality of the software and require a fix in the core source code. Fortunately, with a single worldwide binary development model where the core functionality is separated from the user interface the number of bugs in this category of localization class bugs is relatively small.  Checking the appropriate registry keys are set and files are installed in a new build is reasonably straight forward and should be built into the build verification test (BVT) suite. Other types of things that should be checked include application and hardware compatibility. It is important to identify these types of problems early because they are quite costly to correct, and can have a pretty large ripple effect on the testing effort.

Behavioral and usability issues primarily impact the usefulness or aesthetic quality of the user interface elements. Many of the problems in this category do not require a fix in the core functional layer of the source code. The types of bugs in this category include layout issues, un-translated text,  key mnemonic issues, and other problems that are often fixed in the user interface form design, form class, or form element properties. This category of problems often accounts for more than 90% of all localization class bugs. Fortunately, the majority of problems in this category do not require any special linguistic skills; a careful eye for detail is all that is required to expose even the most discrete bugs in this category.

The final category of localization class bug is linguistic quality. This category of bugs are primarily mis-translations. Obviously, the ability to read the language being tested is required to identify most problems in this category of errors. We found testers spent a lot of time looking for this type of bug, but later found the majority of linguistic quality type issues reported were resolved as won’t fix. There are many reasons for this, but here is my position on this type of testing….Stop wasting the time of your test team to validate the linguistic quality of the product. If this is a problem then hire a new localizer, hire a team of ‘editors’ to review the work of the localizer, or hire a professional linguistic specialist from the target locale as an auditor. Certainly, if testers see an obvious gaff in a translation then we must report it; however, testers are generally not linguistic experts (even in their native tongue), and I would never advocate hiring testers simply based on linguistic skills nor as a manager would I want to dedicate my valuable testing resources on validating linguistic quality…that’s usually not where their expertise lies, and it probably shouldn’t be.

What’s Next

Since behavioral /usability category issues are the most prevalent type of localization class bug this series of posts will focus on localization testing designed to evaluate user interface elements and resources. The next post will expose the often single most reported bug in this category.

Years ago life was pretty simple with regard to data input. Most computer programs were limited to ASCII characters and a set of character glyphs mapped into the code points between 0×80 and 0xFF (high or extended ASCII) depending on the language. The set of characters was limited to 256 code points (0×00 through 0xFF) primarily due to the CPU architecture. Multiple languages were made available via ANSI code pages. Modifying the glyphs in the upper 127 character code points between 0×80 and 0xFF worked pretty well expect for East Asian language versions. So, someone came up with the brilliant idea of encoding a character glyph with 2 bytes instead of just one. This double byte encoding worked quite well except that many developers were unaware that a lead byte could be an 0xE5 character and a trail byte could be a reserved character such as 0×5C (backslash). So, an unknowledgeable developer who stepped incrementally though a string byte by byte would often encounter all sorts of defects in their code. Fortunately today, most of us no longer have to deal with ANSI based character streams on a daily basis. Today most operating system platforms, the Internet, and many of our applications implement Unicode for data input, manipulation, data interchange, and data storage.

Unicode was designed to solve a lot of the problems with data interchange between computers, especially between computer systems using different language version platforms. For example, using a Windows 95 operating system there was virtually no way to view a file containing double byte encoded Chinese ideographic characters using Notepad on an English version of Windows 95. But, on Windows Xp or Vista not only can we view the correct character glyph we can also enter Chinese characters by simply installing the appropriate keyboard drivers and fonts. No special language version or language pack necessary! So, if we created a Unicode document using Russian characters those same character glyphs would appear no matter what language version operating system or application I used as long as the OS and application were 100% Unicode compliant.

However, Unicode of course has its own unique problems. Unicode was originally based on the UCS-2 Universal Multiple Octet Coded Character Set defined by ISO/IEC 10646. Essentially, UCS-2 provided an encoding schema in which each character glyph is encoded with 16-bits (or 32-bits for UCS-4). A pure 16-bit or 32-bit encoding format didn’t really appeal to a lot of people due to various problems that would arise in string parsing. Most data around the world up to that point (with the exception of East Asian language files) were encoded with 8-bit characters. So, some really creative folks came up with ingenious ways to encode characters that more or less captured the essence of UCS (i.e., one code point == one character) using UCS transformation formats (UTF).

Another problem with UCS-2 and a pure 16-bit encoding was the limitation of 65,635 character code points. It wasn’t very long before most people realized this set of code points was not adequate for our data needs. But, instead of adopting a UCS-4 encoding schema the Unicode Consortium redefined a range of character code points in the private use area as surrogates. These surrogate pairs would reference 16-bit character code points in different UCS-4 planes.

A while back I designed a tool called Str2Val to help developers and testers troubleshoot problematic strings. For example, lets assume the following string ṙｭϑӈɅ䩲Ẩլ｡ḩ»ﾓNJĬջḰǝĦ涃ᾬよㇳლȝỄ caused an error in a text box control that accepted a string of Unicode characters. A professional tester would isolate the problematic character or combination of characters causing the error and reference the exact character code point(s) by encoding format in the defect report. I recently upgraded the Str2Val tool to show the same string by various encoding formats such as UTF 16 (big and little endian), UTF-8, UTF-7, and decimal. Not only is this a good tool for trouble shooting problematic strings, it is also a useful training tool to explain the differences in the various common UCS Transformation Formats or UTF encoding methods.

Why is this important as a tester? Well, if you think you represent your customers yet the only characters you use in your testing are the ones labeled on the keyboard that is currently staring you in the face then you are only dealing with a small fraction of the data used by customers around the world (assuming that your software is used outside the country where it is developed, and most English language versions of software are used around the world if they are available on the open market.) If you don’t know how the characters are encoded or which types of problems can arise from the various encoding methods then do you really know how to devise good tests, or are you just guessing? Do you know how to design robust tests with stochastic test data, or are you stuck with stale static data strings in flat files that you simply use over and over again? When a defect occurs in a string of characters (since string data is quite common in testing) can you troubleshoot the cause or isolate the code point, or do you simply just say "yea!…I found another bug!" and throw it back at the developer to figure out?

Well, for those of you living outside the Pacific Northwest you are probably unaware of the recent wind storm with winds gusting to 60+ miles per hour that left more than 1 million people on the eastern side of the state without power. The damage was pretty extensive, and since I live in a fairly remote area I was without power for more than 7 days and without the Internet for almost 9 days. I do have a generator, but it hadn’t been used in almost 4 years. Sure, I started it every 6 months for about 15 minutes each time, but after the first full day of operation the generator started doing wierd things. So, during the past week I have become pretty good at fixing generators (mine and my neighbors), tracing electrical systems, troubleshooting furnace problems, splitting a lot of firewood, cutting up fallen trees, and repairing fences.

After the sun set (which is quite early) I had little else to do (other than making sure nobody stole my generator), so between stoking the fire I started developing a DLL for Unicode string generation in automated tests based on the GString utility. While reviewing the data tables I created for the GString utility with the Unicode Handbook I noticed some holes (OK…defects). Some of the boundaries for code ranges that are not assigned to any Unicode script group were incorrect. (That will teach me to use a web page with the listing put together by a web developer rather than using the Unicode handbook.) But, I also found a problem that prevented unassigned code points from being generated even if the Only use assigned code points check box was unchecked.So, the (hopefully final) update to GString is complete, including the GString.DLL! So, along with the massive overhaul of the Unicode data tables, the new GString package available from my personal website also includes a new DLL for anyone needing to generate strings of random Unicode characters in test automation. The GString zip file also includes detailed documentation on the utility and the dll usage. Let me know if you have any questions about the tool or using random string generation in your testing.

Well, now back to (mostly) normal life.

After a week in Boston presenting at the 3rd Software Testing and Performance Conference I am relaxing in Baltimore (where I grew up) visiting family and friends. For the second year in a row I presented a workshop on functional and structural testing techniques, and also presented a double-track session on GUI test automation using C#. One speaker cancelled at the last moment, so I volunteered to present the globalization testing basics talk I presented at STAR West a few weeks before. At both conferences I promised the attendees a tool to generate strings of random Unicode characters, and while relaxing along the waterfront of Baltimore’s inner harbor (the weather was quite beautiful this weekend) I managed to finish the tool (at least I am meeting the functional requirements I wanted to achieve).

So, without further ado, on my site Software Testing Mentor is a new section for tools and utilities where you will find the tool I have named "GString." GString will generate random strings of Unicode characters between the ranges of U+0020 and U+FFFF up to 65,535 characters in length either as a fixed length string or a random length string. The ranges of Unicode code points that are not assigned to a language script, and special areas such as Private use and surrogate areas are excluded from the generated strings. The resultant string can be copied to the clipboard and pasted into the edit control you are testing. (I am already thinking that a 2.0 version will populate the edit control that has focus automatically.)

GString is written in C# and requires the 2.0 .NET runtime available from Microsoft if you don’t already have it installed on your computer.

Well, back up to Boston for a few days before heading home. If you have any comments about the tool (or find any defects) please let me know.