Archive for April, 2010
Globalization Testing: Customizing Time Formats
Time is a commodity in short supply. I have been juggling a lot lately and there never seems to be enough time to do everything I need to do, and even less time to do the things I want to do. (Blogging falls under the want to do category.) I wish sometimes I could slow down the hands of time, but that is beyond my control. What is within my control is changing the time format displayed on the computer. And if I need to do that in an automated test to increase the robustness of my test to include globalization, then I can programmatically change the time format without having to manipulate the Region and Language settings control panel applet.
Time and date information is commonly pulled from the operating system by many developers for use in headers or footers on documents, default file names, printing, and other places time/date stamps are useful or important. To ensure our products are “world-ready” we should modify the formats to validate whether our product supports various national conventions used in different regions (locales) around the world. In the previous post I illustrated how to programmatically customize the date formats on a Windows environment for including some basic globalization tests in your test automation. This week let’s look at how we can programmatically change both the short time and long time formats.
We will again need the 2 Win32 API functions SetLocaleInfo() and PostMessage() that we marshaled over into the NativeMethods class. Since that code doesn’t change I won’t repeat it here you can simply refer to the code snippet in the previous post. In this situation we need to set the lcType in SetLocaleInfo() to the LOCALE_STIMEFORMAT constant. Then we can pass a null-terminated string to the lcData variable in the SetLocaleInfo() function. MSDN explains “The maximum number of characters allowed for this string is 80, including a terminating null character. The string can consist of a combination of hour, minute, and second format pictures.”
Once again, to simplify that a bit I wrote some more wrapper methods to change the time format. Also, since we will be calling SetLocaleInfo() and PostMessage() a lot for customizing date, time, and other national conventions I created a wrapper method called UpdateLocaleInformation() to remove redundancy.
1: namespace TestingMentor.TestTool.GlobalTester
2: {
3: using System;
4:
5: public enum TimeFormatType
6: {
7: LongTimeFormat = 0x00001003,
8: ShortTimeFormat = 0x00000079
9: }
10:
11: public class CustomTimeFormat
12: {
13: private int timeFormatType = (int)TimeFormatType.ShortTimeFormat;
14: private string timeFormatPicture = string.Empty;
15:
16: public int SetTimeFormatType
17: {
18: set
19: {
20: if (value == (int)TimeFormatType.ShortTimeFormat ||
21: value == (int)TimeFormatType.LongTimeFormat)
22: {
23: this.timeFormatType = value;
24: }
25: else
26: {
27: throw new ArgumentOutOfRangeException("TimeFormatType invalid");
28: }
29: }
30: }
31:
32: public string SetTimeFormatPicture
33: {
34: set { this.timeFormatPicture = value; }
35: }
36:
37: public bool ChangeTimeFormat()
38: {
39: return UpdateLocaleInformation(
40: this.timeFormatType,
41: this.timeFormatPicture);
42: }
43:
44: private bool UpdateLocaleInformation(int localeType, string localeData)
45: {
46: bool success = false;
47: if (NativeMethods.SetLocaleInfo(
48: NativeMethods.SystemDefaultLocale,
49: localeType,
50: localeData))
51: {
52: NativeMethods.PostMessage(
53: NativeMethods.BroadcastMessage,
54: NativeMethods.SettingChangeMessage,
55: IntPtr.Zero,
56: IntPtr.Zero);
57: success = true;
58: }
59:
60: return success;
61: }
62: }
63: }
Once again, we simply have to set the SetTimeFormatType property to either the Short time or Long time format, provide the format picture by setting the SetTimeFormatPicture property, and then call ChangeTimeFormat(). The sample below illustrates how to change the short time format with different time separators and a reverse order.
1: static void Main(string[] args)
2: {
3: CustomTimeFormat time = new CustomTimeFormat();
4: time.SetTimeFormatType = (int)TimeFormatType.ShortTimeFormat;
5: time.SetTimeFormatPicture = "ss'mm,hh - tt";
6: if (time.ChangeTimeFormat())
7: {
8: Console.WriteLine("Success");
9: }
10: }
Now, we can also customize the AM/PM designator as well. To change the AM/PM designator we need to add a few more properties and another wrapper method. In this case, I’ve added the SetAmPmDesignator property, the SetAmPmString property, and the ChangeAmPmDesignator() method.
1: public enum AmPmDesignator
2: {
3: AM = 0x00000028,
4: PM = 0x00000029
5: }
6:
7: public class CustomTimeFormat
8: {
9: public int SetAmPmDesignator
10: {
11: set
12: {
13: if (value == (int)AmPmDesignator.AM || value == (int)AmPmDesignator.PM)
14: {
15: this.designatorForAmPm = value;
16: }
17: else
18: {
19: throw new ArgumentOutOfRangeException("AmPmDesignator invalid.");
20: }
21: }
22: }
23: }
24: public string SetAmPmString
25: {
26: set { this.timeDesignator = value; }
27: }
28:
29: public bool ChangeAmPmDesignator()
30: {
31: return UpdateLocaleInformation(
32: this.designatorForAmPm,
33: this.timeDesignator);
34: }
The code snippet below illustrates how to change the AM designator from “AM” to “In the morning.”
1: static void Main(string[] args)
2: {
3: CustomTimeFormat time = new CustomTimeFormat();
4: time.SetAmPmDesignator = AmPmDesignator.AM;
5: time.SetAmPmString = "In the morning.";
6: if (time.ChangeAmPmDesignator())
7: {
8: Console.WriteLine("Success");
9: }
10: }
Modifying national conventions is one way to test for globalization support upstream and should be done early in the testing cycle rather than relying on a separate globalization testing cycle. Time and date are perhaps the most visible national conventions used in many different ways in our applications. We should test the common (equivalent) conventions used in various regions around the world, and customizing these settings helps ensure the developer is properly calling NLS APIs and not using custom functions.
Also, check out the beta release of the GlobalTester automation library that has this functionality and more, and let me know what you think.
Globalization Testing: Customizing the Date Format
The ability of our software products to function correctly in a global environment is becoming more and more important. Our software should support national conventions used by the various locales around the globe. For example, in some regions of the world the period character is used as the number group separator and the comma is used as the decimal symbol (radix). European calendars generally start on Monday rather than Sunday which is customary in the United States. Era based calendars are still in common use in Japan and Korea, date formats and order, and time formats also vary by region or locale. As testers we need to test our software to ensure our customers around the world can use the national conventions they are accustomed to, and not force them down a US-centric, one-size-fits-all format or standard.
There are several settings that we can modify and customize for more robust globalization testing such as number, currency, time and date formats. Modifying these settings can help us test that our application is globalized to use National Language System (NLS) APIs provided by the system.Although a user would change these settings using the Regional Options user interface property sheets, if the purpose of our test is not to emulate user interaction, then modifying the custom regional settings for globalization testing programmatically is more efficient.
Last year I talked about how to programmatically make changes to the settings in the Region and Language control panel applet when doing globalization testing. Unfortunately, the code sample provided in the previous post was appropriate for versions of Windows XP and earlier. For versions of Windows Vista and later things have changed a bit. Also, the previous sample tried to be a one-size fits all and relied on the test developer to set the appropriate lcType constants and lcData argument variables required by the Win32 function SetLocaleInfo().
This time, I decided to simplify things a bit and wrapped some methods to call the appropriate Win32 API functions and properties to set lcType and lcData values to make it easier to incorporate into automated tests. I also separated the various advanced custom formats for Region and Language options into separate classes. Of course, I have a beta version of an automation library (DLL) called GlobalTest.DLL on my website that testers can use in their automated test cases, but this week let’s look at the class for setting custom date formats.
Making these changes programmatically still requires the Win32 SetLocaleInfo() function. MSDN also states this function modifies the specified values for all applications, so to prevent potential issues in other applications running on the system we should also broadcast the WM_SETTINGCHANGE message. To broadcast the WM_SETTINGCHANGE message we will also need the Win32 PostMessage() function. Since we are Process Invocation (PInvoke) to call these unmanaged functions we should put them in a separate class that I’ve called NativeMethods. I also included all necessary constant values required by these methods in the NativeMethods class also as illustrated below.
1: namespace TestingMentor.TestTool.GlobalTester
2: {
3: using System;
4: using System.Runtime.InteropServices;
5:
6: internal sealed class NativeMethods
7: {
8: internal const int SystemDefaultLocale = (int)0x00000800;
9: internal const int BroadcastMessage = (int)0x0000FFFF;
10: internal const int SettingChangeMessage = (int)0x0000001A;
11:
12: private NativeMethods() { }
13:
14: [DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
15: [return: MarshalAs(UnmanagedType.Bool)]
16: internal static extern bool SetLocaleInfo(
17: int locale,
18: int localeType,
19: string localeData);
20:
21: [DllImport("user32.dll", SetLastError = true)]
22: [return: MarshalAs(UnmanagedType.Bool)]
23: internal static extern bool PostMessage(
24: int handle,
25: int message,
26: IntPtr wParam,
27: IntPtr lParam);
28: }
29: }
The class for the custom wrapper method is TestingMentor.TestTool.GlobalTester.SetDateFormat. There is a public enumeration for the short date and long date constants. One of these values must be assigned to the SetDateType property. The other property that must be set is the SetDateFormatPicture. The big change in the SetLocaleInfo() function is that the lcData type is a null-terminated string that MSDN refers to as a format picture. Current versions of Windows allow users to customize the order of the month, day and year, the format for each, and even allow different separators between the date elements. The format picture enables the user to select various format types in different orders for either the short date or the long date. See MSDN’s Month, Day, Year and Era Format Pictures for the various supported format types.
1: namespace TestingMentor.TestTool.GlobalTester
2: {
3: using System;
4:
5: public enum DateFormatType
6: {
7: ShortDate = 0x0000001F,
8: LongDate = 0x00000020
9: }
10:
11: public class CustomDateFormat
12: {
13: private string dateFormatPicture = string.Empty;
14: private int dateType = (int)DateFormatType.ShortDate;
15:
16: public string SetDateFormatPicture
17: {
18: set { this.dateFormatPicture = value; }
19: }
20:
21: public int SetDateType
22: {
23: set
24: {
25: if (value == (int)DateFormatType.ShortDate ||
26: value == (int)DateFormatType.LongDate)
27: {
28: this.dateType = value;
29: }
30: else
31: {
32: throw new ArgumentOutOfRangeException("Invalid DateType");
33: }
34: }
35: }
36:
37: public bool ChangeDateFormat()
38: {
39: bool success = false;
40: if (NativeMethods.SetLocaleInof(
41: NativeMethods.SystemDefaultLocale,
42: this.dateType,
43: this.dateFormatPicture))
44: {
45: NativeMethods.PostMessage(
46: NativeMethods.BroadcastMessage,
47: NativeMethods.SettingChangeMessage,
48: IntPtr.Zero,
49: IntPtr.Zero);
50: }
51:
52: return success;
53: }
54: }
55: }
Once the SetDateType and SetDateFormatPicture properties are assigned we simply have to call ChangeDateFormat() method to change the settings and broadcast the message to the system. The code snippet below illustrates how a tester would change the default long date format in an automated test to determine globalization support in the application under test. Customizing the date format is useful if the application under test uses a date string in any way. For example, if the application includes a function to insert a date string in an edit control, or if the date is printed as a header or footer in a document, or if a date string is appended to a record.
1: using TestingMentor.TestTool.GlobalTester;
2: ...
3: static void Main(string[] args)
4: {
5: CustomDateFormat date = new CustomDateFormat();
6: date.SetDateType = (int)DateType.LongDate;
7: date.SetDateFormatPicture = "[ dd % MM | yyyy ]";
8: if (date.ChangeDateFormat())
9: {
10: Console.WriteLine("Long date was changed");
11: }
12: }
Programmatically changing the date format is an easy way testers can customize date formats in their automated tests without having to manipulate the controls on Region and Language property sheet. Also note, that since the format picture is a string the order of the supported date format types is now controlled by the arrangement in the string, and the separator characters can be different between the day and month and the month and year as illustrated in the example above.
Modifying national conventions is one way to test for globalization support upstream and should be done early in the testing cycle rather than relying on a separate globalization testing cycle.
Next week I will discuss customizing the time format. Also, check out the beta release of the GlobalTester automation library that has this functionality and more and let me know what you think.
Is this as good as it gets?
Last month I was travelling throughout Europe. A few days in Switzerland to speak at Swiss Testing Days and visit our communication server team in Zurich. About a week and a half in Denmark teaching at our offices there, a few days in Dublin teaching, and finally a full day meeting with one of our teams in Reading, UK. The only good thing about being on the road that much is getting in more reading time. Two of the books I read while travelling were Agile Testing by Lisa Crispin and Janet Gregory. The other was Clean Code by Robert Martin. I found both books thought provoking and informative. Of course I didn’t agree with everything, but even the statements I didn’t agree with made me think about that topic a bit more. Martin’s book inspired me to improve my testing and coding craft. If nothing else, testers should read the first chapter of Clean Code, those who design and develop automated tests should read the whole book.
But, one statement that has always troubled me that I saw repeated in the Agile Testing book is in regards to exploratory testing as the most effective way to expose important bugs. Every time I hear/read this or one of its many derivatives it makes me ponder. If this is really true then why do we invest so much time and resources in other approaches to software testing. Why do we spend so much time in static and dynamic analysis, reviews, and coverage analysis? Why do we try to prevent bugs when they can (and presumably are) found easier and ‘quicker’ downstream via exploratory testing? Why don’t we simply continue to hire hoards of diverse people to rapidly explore the product just before release and beat out all those important/critical bugs?
I wonder why it seems that we can ‘think critically’ when we have the product to play with, but we can’t seem to apply critical thinking while reviewing documented requirements, or a model? Personally, I am not sure that ‘trying’ something and then deciding what to do based on the outcome or state of the computer, or deciding whether something is a bug after the fact constitutes critical thinking. Perhaps sometimes it is, but I suspect that sometimes it is just guessing. To me, critical thinking involves the ability to foresee potential issues; not simply stumble upon them, or apply an attack from my bag of exploratory tricks to reveal them downstream in the development lifecycle when I have the product to play with.
I sometimes wonder if exploratory testing really is the holy grail of software testing, or if we have just given up on other approaches because they are too hard. Could it be that our testers are not capable of ‘testing’ without also engaging their fingers? Are our testers untrained or under-trained in our profession? Is the reason why our pre-defined test cases are seemingly ineffective due to the realization that we suck at test design, and learning how to design more robust and ineffective tests is just too hard? Is the reason we can’t think of tests until we get the product in hand perhaps due to our inability to model or is it because we really don’t know that much about the systems we are testing? (You can’t model what you don’t understand and you don’t understand what you can’t model.) Besides, why should we burden testers with the overhead of learning about programming concepts, computer science, and math. You don’t need all that stuff to find bugs.
Now many of you might think that I despise exploratory testing. I am not anti-exploratory testing (whichever form that takes); because we all do it! Show me a tester who doesn’t include some form of exploratory testing in his or her repertoire and I’ll show you a thoughtless simpleton who needs explicit instructions for determining which side of the heal of a loaf of bread to butter. But, I have always thought that we could actually improve the craft of testing. I thought we might play a much bigger role in an organization. I thought we could help reduce overall production costs by being involved earlier and throughout the lifecycle. I thought we could help reduce risk through defect prevention and in-depth analysis. And, I thought we could learn to use tools and techniques to help us design better tests up front, and rely on a team of testers who are capable of thinking while executing a test case.
Yes, it is true that exploratory testing is fun, and we don’t really need to understand all that computer science stuff to find bugs using an exploratory approach. Regardless of what empirical studies on the topic reveal, exploratory testing ‘feels’ so much more productive. Why should we base our profession on evidence when we can get by so easily with emotion? Why should we have to think critically during the design or implementation of software when we can simply muddle our way through some graphical user interface to find bugs? If our approaches to unit testing and component level testing are so shoddy as to simply invoke happy path” tests then of course finding bugs by exploration seems rather logical. But, as developers become more adept at unit can component level testing in flushing out functional bugs earlier we may only need ‘testers’ to explore the product before release for behavioral issues, compatibility problems, and the few remaining obscure functional bugs. Or possibly exploratory testing providing a ‘good enough’ service to our team by finding the bugs that matter to the majority of customers, and perhaps that is ‘good enough.’
Personally, I think we can improve our testing processes. I think we will need to improve the practice of software testing. I think that our job is not simply to find bugs, but a more important objective is to prevent defects (which can help reduce overall product costs). I also don’t view testing as destructive, and I think those who do are doomed to become obsolete. Successful businesses don’t hire folks who just want to ‘destroy’ the products they build, I suspect they want to hire people who can help the company grow and improve. Finding bugs helps me ship a product, but it doesn’t necessarily help me grow the business, reduce production costs, or improve quality long-term.
But maybe we don’t need to change. Maybe our businesses are satisfied with the status quo. Maybe we don’t need to think about cost savings or helping our teams mature their processes, reduce overall risk, and potentially improve quality earlier. Maybe I am too idealistic thinking that testers can provide greater value to an organization beyond finding bugs at the end of the development cycle. Maybe our craft has reached the the apex of of our profession and this is as good as it gets.