I.M. Testy

Originally Published Thursday, September 10, 2009

I am really happy to announce that our book has been released in China and available on the Chinese Amazon site! This was really a monumental effort driven by my friend and colleague Kelly Zhang.

We look forward to the feedback from the Chinese testing community, and we hope this provides our Chinese friends  with some additional perspectives on software testing (or at least some interesting stories).

Originally Published Wednesday, December 24, 2008

I have a library of books and white papers on software testing, engineering processes and management, and software development that I have read and reference quite often. For new testers I generally recommend A Practitioner’s Guide to Software Test Design by Lee Copeland, and How to Break Software: A Practical Guide to Testing by James Whittaker. There are 5 books I highly recommend (not including How We Test Software at Microsoft which I co-authored and also highly recommend).

In my current role as a teacher, trainer, and mentor of new testers the 2 books that are constantly on my desktop are Testing Object-Oriented Systems: Models, Patterns, and Tools, by Robert V. Binder, and Software Testing Techniques, 2nd edition by Boris Beizer. Not that I don’t frequently reference other books, but to me these are the quintessential books on the foundational knowledge of software testing techniques and methodologies for intermediate to advanced testers with a strong technical background.

But, the booklet that I would keep in my shirt pocket if I tested products on a day-to-day basis would be Josh Poley’s Black Book. Josh’s Black Book is the ultimate desktop reference for software testers (and developers). While this book is primarily intended to aid those who work on projects developed in C/C++, it has loads of information that is valuable to any tester working on just about any technology. From decimal and named entities of ISO characters to error codes for DOS, VB, JScript, HTTP, and of course Windows Errors this book is jammed packed with great information and quick reminders for both developers and testers.

Originally Published Saturday, December 06, 2008

This past year has been quite busy for me. Too busy. Besides trying to keep up with my busy teaching schedule, driving some key initiatives and collaborating on others, planning new course development for SDETS, I presented at 11 conferences around the world, wrote a few magazine articles, and developed a new software test automation program at the University of Washington. Somewhere in the midst of all that I co-authored a book with Alan Page and Ken Johnston that is now available to order, and should be on bookstore shelves within a week.

Collectively we have more than 3 decades of experience in various roles and business groups around the company. Coupled with insights and experiences from the many other testers (past and present) at the company the book is filled with great ideas and examples of some of the testing processes and procedures used around the company.

But it is not just another book of how to test software. This book provides a lot of insight into Microsoft, illustrates some of our best practices (and also reveals some of our faux pas’), and answers the question (albeit indirectly) we get all the time; “How do you test software at Microsoft?”

Originally Published Thursday, May 10, 2007

In retrospect, my father had the incredible foresight to introduce me to electronics at a young age. When I was in junior high we built a vacuum tube AM radio kit from Heathkit. That thing put out enough heat to warm a small apartment, but there was something satisfying about sitting by the window at night fine tuning in the local stations on a radio I built. Then my parents suggested that I apply to a new ‘blue ribbon’ high school in our area, and I was fortunately accepted at Eastern Technical High School‘s engineering/electronics program. Somewhere along the way I became more interested in social sciences. But, obviously my long and winding path eventually led me to a career in the tech sector and I have a job that I love. I only bring up my past to provide some context for readers to understand why I take some of the strong positions that I take. For example, I strongly advocate that testers should have an in-depth knowledge of the ‘system’ under test.

In my last post I mentioned computer system internals is one particular area of study that is especially useful for professional testers. Similar to how medical doctors study human biology and anatomy, testers who understand the inner workings of a computer are more capable of designing tests to expose errors in complex systems, decomposing test data into discrete equivalence class subsets, and troubleshooting the cause of errors rather than simply reporting the various symptoms. Since I teach software testing at 2 local colleges (and also advise Microsoft STEs who lack a strong technical background) I am always looking for books I think will benefit a tester’s technical knowledge. Just before leaving Ireland, I came across a book entitled Code: The Hidden Language of Computer Hardware and Software by Charles Petzold. I am a big fan of Petzold’s books on Windows programming, but I somehow overlooked this particular book in the past.

Although the book was a great refresher for me, I enjoy Charles Petzold’s style of writing and I did learn a few new things. Charles Petzold is amazingly capable of taking a rather complex subject and explaining it in a fashion that is easy to comprehend. Code: The Hidden Language of Hardware and Software starts with an explanation of basic codes such as Morse code and Braille, then uses the basics of electrical switches leading to an understanding of logic gates to explain a simple ‘computer’ (switches and lights) that adds and subtracts in binary. There is also a brief discussion of Boolean algebra and its relation to computer science. The book progresses to explain how computer memory works, how computers encode characters (ASCII only but it is a good intro to character encoding basics), basic computer architecture, and ends with a high level discussion of some programming languages and the impact of the graphical user interface.

One of the best things I liked about this book is the way it really walks the reader through an understanding of assembly language. The Assembly language is good to know for debugging.  Debugging is an important skill for testers because it is often required for root cause analysis of certain types of defects, it is also important to isolate the cause of a defect that manifests itself with diverse symptoms, allows testers to participate in resolving complex problems such as isolating memory leaks, and of course to more effectively troubleshoot  problems with test automation. At Microsoft, many groups now require testers to debug some defects to line of code. This doesn’t mean that a developer’s time is more valuable than a tester’s time, and not all defects require debugging, and there is always a trade-off for time a tester spends debugging (it generally depends on the type of defect). But, in general testers engaging in debugging is a good thing because it brings testers and developers closer together and increases a tester’s credibility.

So, for testers wanting to increase their ‘system’ knowledge I would highly recommend

Code: The Hidden Language of Computer Hardware and Software

Originally Published Tuesday, October 24, 2006

I recently went to Portland, and when I am there I make it a point to always stop by Powells Book Store. They have a whole building about the size of a typical Barnes & Noble dedicated to technical books. It had been several years since I had read Peopleware: Productive Projects and Teams by Tom DeMarco and Timothy Lister, so when I saw the second edition sitting on the shelf with 8 new chapters I just had to get it.

DeMarco and Lister’s have a witty and enjoyable style of writing that makes their books fun to read. I simply couldn’t put the book down on a flight from Seattle to London. Not only is the book engaging, it provides strong evidence to support their arguments and dispells many common myths, misconceptions, and misinformation around efficiency and effectiveness of people, projects, and the workplace. It provides managers with better information and keen insight on how to manage technical people and technical projects.

As Joel Spolsky wrote, "I can’t recommend this book highly enough. It is the one thing every software manager needs to read…not just once, but once a year."

• working with data types
• using branches and loops
• basic exception handling

It now astounds me that some individuals can spend years in a testing role and never read a single book on the subject. I say ‘now’ because when I first started in software testing I hadn’t read a book on the subject  either. In fact, it wasn’t until I had to start teaching software testing to new testers at Microsoft that I began reading books on the subject in earnest instead of just skimming through chapters in Kaner’s book, Testing Computer Software.

When I thought the purpose of testing was simply to find bugs, I didn’t need books on software testing to help me find bugs. I was very good at finding errors in Microsoft products (it really isn’t that hard). But, as I matured in this profession, books on software testing provided me new perspectives on the application of time-proven techniques to help me become more effective and more efficient in my testing endeavors. Books on software testing presented diverse insight into various methodologies and approaches to solve problems improve processes. Books on software testing portrayed a breadth of information from a myriad of outlooks that enabled me to draw logical conclusions based on a wealth of industry experiences rather than from a limited (Microsoft-ish) view of testing based on personal trail and error in large market computer software.

Now, I read a lot of books on software testing. Some books are less than I’d expect, most are as dry as English humor, and there are a mere handful I’d consider great treatises on the subject of software testing. But, the important point is that I learn something from each new book I read. The good news is that today there is a plethora of books on software testing. (This is great compared to 10 years ago when there were but a few books that broached the subject.) But, for the folks who have chosen to enter into the professional discipline of software testing it is sometimes hard to know where to begin. To complicate matters a bit more, the books I recommend differ according to one’s primary role in the organization. But, since most new testers (including those with CS degrees) receive very little formal education in software testing, I think it is important for all new testers (and developers) to become familiar with the basic techniques used in the practice of software testing.

This list of software testing books was compiled based on input from Test Managers and Test Architects at Microsoft as recommended reading for new testers at the company, and are books I constantly reference in our internal training.

1. A Practitioner’s Guide to Software Test Design, Lee Copeland, 2003

2. The Art of Software Testing, 2nd edition, Glenford Myers, et. el., 2004

3. Software Testing Techniques, 2nd edition, Boris Beizer, 1990

4. How to Break Software: A Practical Guide to Testing, James Whittaker, 2002

5. Testing Object-Oriented Systems: Models, Patterns, and Tools, Robert V. Binder, 1999