Archive for December, 2010
Looking back…
This has been a rather eventful year; both positive and negative. I have done quite a bit this past year professionally and personally, but certainly not as much as I had hoped to accomplish. As the year is winding down, my attention is focused on fulfilling my daughter’s Christmas dreams and wishes. Christmas is still a magical time for her and I both.
I don’t know if she still truly believes in Santa Claus, but at least she does a good job of pretending. Tonight we will perform our yearly ritual of putting milk and cookies on the fireplace hearth and toss some carrots out onto the back lawn for the reindeer. We won’t build a fire in that fireplace on Christmas eve because she doesn’t want Santa Claus to get burnt as he comes down the chimney. Of course, after she is tucked soundly in bed, I will eat the cookies, drink the milk, and throw the carrots into the greenbelt behind my house for the critters to munch on.
She is delighted by finding the present she asks for from Santa Claus under the tree. She only asks for one thing from Santa so “he” feels obliged to satisfy her wish. There are some smaller gifts as well; puzzles, books, American Girl doll accessories, etc.). But, it is not the presents that make this a special time for her and I. It is all the things we do leading up to Christmas day that make this a wonderful time of year for our family.
Well, I am won’t blather on about Christmas, or what a special time it has become for me since my daughter was born. Instead, I would actually like to thank Michael Larsen for taking the time to write several detailed reviews of How We Test Software at Microsoft. As I read his reviews of chapter 5 and chapter 6 (the ones I wrote) he actually brought out several key points that I think were a little obscure such as, “Bj champions the use of Exploratory Testing (ET). ET is a great way to get a handle on an application, especially when a tester is new to it.” Michael also caused me to reflect on the following point in the book, “Bj argues that Exploratory Testing can be sufficient for small software projects, or software with limited distribution, or software with a limited shelf life. But that it doesn’t scale well for large-scale, complex or mission-critical applications.” In retrospect I wish I would have expanded on that statement by saying that in our experiences at Microsoft ET does not scale well as a primary approach to testing, and the teams that tried to use ET as a primary approach have changed their strategy. However, ET is used throughout MS; it always has been and always will be a valued tool in any professional tester’s toolbox. Fortunately, Michael was able to read between the lines and wrote, “I get what he’s trying to say, in that Exploratory Testing techniques will not be the be all and the end all with testing (nor should it be; all tools have their right time and their right place).” Right on!
So, whether you have read the book or not, I highly recommend that you visit Michael’s blog to read the reviews for insightful thoughts on the book. I also recommend his other thoughtful posts on the topic of software testing.
Basic International Sufficiency Testing in Action
A person who speaks 3 languages is trilingual; a person who speaks 2 languages is bilingual; and a person who speaks one language is an American (or Brit…depending on which side of the pond you are on).
I work at a company with tremendous cultural diversity embodied in very smart people from around the globe. Yet, it seems that when people come to Redmond their cultural uniqueness seems to disappear. Maybe it is an engineering thing, maybe it is an assimilation thing, but whatever it is, it is not good thing especially considering that a growing number of our customers come from non-US markets and the way they interact with software is often quite different compared to the US centric scenarios and personas I so often seen used to design and develop our software and services.
Monday afternoon I taught a class on globalization testing basics geared towards SDETs who are not experts in globalization. These testers usually come to the training mostly because they’ve been tagged by their manager to help with globalization testing efforts on their team. But, what they learn is that with a little understanding of some basic concepts and with the aid of a few tools they can incorporate international sufficiency testing concepts into their test designs (both exploratory and automated) and drive quality upstream (e.g. find some bugs sooner).
One of the topics I discuss in the class is customizing the current user locale settings. I also discussed this in an earlier post this year. Customizing the national conventions settings for the current user locale include things like custom date and time pictures, as well as number and currency formats. Of course to really understand custom locale settings we should have a good technical understanding of the national language support (NLS) API functions and specifically the LCType parameter Locale Information Constants.
<tangent alert> When I and other people talk about encouraging testers to develop their technical skills or knowledge there are some people who simply assume that we mean “technical” equates to coding. This is really a shallow view of how they interpret ‘technical.’ When I refer to technical knowledge and skills I am primarily referring to an understanding of how the system (or parts of the system) work and how to use that knowledge and their skills to design more effective tests.</tangent alert>
In this case, our technical understanding of the variable arguments that can be passed to the lpLCData parameter of an NLS API function such as SetLocaleInfo() for the various LCType constants can help is more fully explore the functionality of features that use the operating system’s NLS settings. For example, the constant value for the decimal symbol for number formats is LOCALE_SDECIMAL. When this LCType is specified the value we can pass to the lpLCData parameter is a string of up to 3 characters in length (“maximum number of characters allowed for this string is four, including a terminating null character.”)
In class I often use a custom decimal symbol as an example of customizing national convention settings by manually changing the number format decimal symbol from a period character (.) to random Unicode characters. From previous examples I knew of a minor anomaly in the calculator (calc.exe) in which the decimal button changed to show “abc” (or some other random string of 1 to 3 characters), but the decimal symbol in the result window only displayed the first character (“a”).
But, in this week’s class I gave an example of changing the decimal symbol from a period character to the literal string “dot.” Then, Andreas Schiffler, one of the SDETs in the class used the example “dot” string in the calculator and quickly discovered an anomaly.
It seems that the calculator does not like the lower case letter ‘d’ as a decimal point. If you customize the decimal symbol for the number format to the Unicode lower case letter d, then launch the calculator (calc.exe) and press the decimal symbol key (or perform any calculation in which the result includes a decimal value) the calculator result window will show “Overflow.”
Andreas initially thought this might be caused due to the fact that the letter ‘d’ is a formatting character. However, we quickly discovered the upper case letter ‘D’ is also a formatting character, but the upper case ‘D’ and other formatting characters do not cause a similar incorrect condition. (I went home and automated this test looking for clues using the GlobalTester library and pumped in over 10000 random Unicode characters, and none of them have caused an overflow.)
Now this particular problem might seem out of context, or have no ‘real-world’ scenario because the commonly used characters for the decimal symbol are the period (.), the comma (,) and the space ( ) characters. And for the life of me I can’t think of any national convention in this world that uses the letter ‘d’ as a decimal symbol in their number formats. But, this application is using the NLS settings (at least to some level of implementation), and since we allow the user to customize these settings, then I shouldn’t expect that functionality to break. The bottom line is that something out of the ordinary is going on that probably needs investigating.
Sometimes simply changing the current user default locale settings might reveal basic internationalization issues. And sometimes customizing the national conventions and using randomized data for those conventions might reveal problems with how the developer implements national language support in the product. You don’t have to be an expert in globalization to discover these issues! You just have to know a little bit about the technicalities of NLS, and have a desire to potentially find some pretty cool, or at least weird anomalies earlier in your testing.
Generating Random Dates
Last week was mostly a blur for me. Three hockey games (1 game with my regular team, 1 substituting on another team, and 1 in the over 40 league). By Thursday I was pretty wiped out, and with all the holiday stuff going on I completely blew off updating my blog.
Since my daughter was about 6 months old her and I have always had a daddy & daughter day at least once per week where I focus 100% on her and we do what she wants to do. About every 2 weeks she wants to go ice skating, and she completely surprised me by saying that she wanted to go to a free hockey clinic for kids for our our daddy & daughter day on Sunday. Castle Ice hosts a free hockey clinic for kids once per month. Although my daughter understands all the rules and enjoys watching games (she has become quite a Sid the Kid fan), I know that the odds of her ever taking up the sport are somewhere between 0 and .00001. But, she said she had a great time pushing the puck around on the ice and doing some of the drills with the other kids, and I was sure proud of her for giving it a try.
Ok…enough about my excuses for not posting last week, and time to move on from combinatorial testing and get back to something else I am interested in…random test data generation.
Every once in awhile a discussion comes up about how to generate random dates. There are several examples on the web to generate random dates. But many of them hard code locale specific formats such as month/day/year or day/month/year. Some allow you to specify a separator character such as a dash (-) or a forward slash (/) or other character. Many of these generators are limited to out-dated formatting options compared to the formats that are available (see MSDN Day, Month, Year and Era format picture).
In an earlier post I discussed how to we can programmatically change the date format to incorporate globalization testing or international sufficiency testing in our automated tests. I also created an automation library called GlobalTester to help testers change other settings to push international sufficiency testing upstream, and I talk about that in a previous post as well.
So, instead of generating dates based on some hard-coded format (that may or may not be the same format used by the current user locale settings), or using some hard-coded separator character (that may or may not be the character for the current user locale) I decided to create a method that focused on generating random dates and allow the operating system to decide the proper formatting.
An easy way to generate a single random date string within a specified range that is formatted to the current user settings for the date format is illustrated below.
1: public static string RandomDate(DateTime startDate, DateTime endDate)
2: {
3: try
4: {
5: Random prng = new Random();
6: int range = ((TimeSpan)(endDate - startDate)).Days;
7:
8: // Returns a string that follows the current short date formatting options
9: return startDate.AddDays(prng.Next(range)).ToShortDateString();
10:
11: // Returns a string that follows the current long date formatting options
12: // return startDate.AddDays(prng.Next(range)).ToLongDateString();
13: }
14: catch (ArgumentOutOfRangeException)
15: {
16: throw new ArgumentOutOfRangeException(
17: "The end date must be later than the start date");
18: }
19: }
Now, all we have to do is define our start date range and our end date range and call the method. To generate a random date string using the long date format settings for the current user simply change the RandomDate method above to return the value in line 12.
1: static void Main(string[] args)
2: {
3: DateTime startDate = new DateTime(
4: DateTime.Now.Year - 100, DateTime.Now.Month, DateTime.Now.Day);
5: DateTime endDate = new DateTime(
6: DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day);
7:
8: Console.WriteLine(RandomDate(endDate, startDate));
9: }
Because we can use a constructor to pass int values for the year, month and day this approach allows us to set a date range in the past, the future, or we can specifically define a range of dates.
The above method works well if we only need a single random date, but if we need several random dates this method will not produce reasonably random dates. To produce multiple random dates at one time we should use a slightly different approach. A better approach to generate multiple random dates at one time is to use an IEnumerable interface as illustrated.
1: public static IEnumerable<DateTime> RandomDate(
2: DateTime startDate,
3: DateTime endDate)
4: {
5: Random prng = new Random();
6: int range = ((TimeSpan)(endDate - startDate)).Days;
7: while (true)
8: {
9: yield return startDate.AddDays(prng.Next(range));
10: }
11: }
1: static void Main(string[] args)
2: {
3: DateTime startDate = new DateTime(
4: DateTime.Now.Year - 100, DateTime.Now.Month, DateTime.Now.Day);
5: DateTime endDate = new DateTime(
6: DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day);
7:
8: int i = 0;
9: foreach (DateTime date in RandomDay(startDate, endDate))
10: {
11: Console.WriteLine(date.ToLongDateString());
12: Console.WriteLine(date.ToShortDateString());
13: if (++i == 10)
14: {
15: break;
16: }
17: }
18: }
Compared to all the random date generators out there, this is about the simplest implementation available. We don’t have to use a generator tool or library, we can simply include these methods in our libraries or code. If you know of an easier solution, please let us know.
Some additional information can be found at