I remember as a young lad working construction for my uncle one summer. The hours were long, it was hot, and I would much rather have been somewhere else. But, I was saving up for my first motorcycle, so I did whatever jobs I could find. Perhaps it was because my mind wasn’t really engaged in what I was doing, but as I was hammering nails into drywall the hammer slipped off the head of the nail and mashed my thumbnail. Man that hurt. Of course after yelling out a few expletives I screamed,  “stupid [expletive deleted] hammer!” Years later, I was working an engine and the wrench slipped and the back of my hand slammed into the engine block. Covered with grease (and now a bit of blood) I examined my hand to assess the damage in order to decide whether to keep working or tend to my wounds, and the first thing I thought of as I looked at my somewhat mangled hand was “stupid wrench.” Do you see the pattern here? Isn’t it a bit funny that often when we use a tool incorrectly, misuse a tool to do something the tool was not designed to do, or do not really know how to use the tool to begin with and a problem ensues our initial reaction is to blame the tool. Of course, it couldn’t be our fault; it has to be the tool!

I sometimes see this deflection of responsibility repeated by testers who attempt to apply various techniques or methodologies to a software testing problem and later discover they missed or overlooked a bug. Their immediate reaction is “such and such” technique sucks because it didn’t find “this” bug. Of course it could never be our own fault! It could never be that we don’t sufficiently understand the principles of a technique or approach in order to apply it correctly. It could never be that we don’t fully understand the ‘system’ in depth that we are testing. And it could never be that we are using a particular technique or approach in the wrong context. The problem could never be the fault of the person wielding the tool…it must be the tool.

A few years ago I was rebuilding a sailing dinghy. I was ready to mount the rub strake and started drilling holes through the fiberglass hull and the teak strip. For some bizarre reason, I decided to simply hold the rub strake against the gunwale as I moved forward drilling the holes rather than use C-clamps. My grip was sufficiently tight to hold the rub strake tight against the hull, and I had used an electric drill for years and knew well how to use the tool. But, after drilling several holes I suddenly experienced an excruciating pain in my left index finger. Yep…the drill bit went right into the proximal interphalangeal joint on my left index finger. After screaming a few profanities (which by the way helps us deal with pain) I flung the drill across the garage only to put a hole in the drywall and hear it crash to the floor in pieces. With that I hurled a few more obscenities, wrapped my finger with ice, and headed off to the doctor and thought to myself…”man, that was stupid of me not to use clamps.” To add insult to injury, the doctor asked, “Why didn’t you use a clamp?” I replied, “Well, I was in a bit of a hurry.” He shook his head, smiled, and asked the redundant question, “In retrospect that wasn’t too smart was it?”

When we misuse tools, apply them in the wrong context, or if we really don’t understand how to use tools appropriately bad things can happen. (And sometimes the scars are permanent!)

While I don’t think that misapplication of a testing technique or approach would require a trip to the hospital, it might cost us a missed bug or added redundant tests. This also doesn’t mean that all techniques or approaches to testing are 100% effective in finding all categories of anomalies. But, we have to remember for the most successful application of any tool, technique, or approach used in testing depends heavily on the person wielding the tool in the appropriate situation and we must learn to use them smartly!

I am an avid fly-fisherman, and I am spending a few of these last winter evenings tying flies in preparation for the new year. The lakes are still too cold so the trout are deep and lethargic, and many of the rivers are closed and too damn cold and swollen anyway. So, now is a good time restock my fly boxes, reflect on past years, and dream of the up-coming season. While fly fishing is an enjoyable escape from the day to day torrent of technology, when I can’t stand in a river and wave a stick I can still relax with a good book that conjures memories of years past or engulfs my mind in an adventurous narrative as if I were there. I don’t really enjoy reading most fiction books, but I do enjoy reading the memoirs of people such as John Gierach. (Perhaps it is the sailor in me that loves a good yarn, or the fact that I can relate personally to the stories.) Anyway, this weekend I acquired a book entitled Fishless Days, Angling Nights by Sparse Gray Hackle that introduced a legend in American fly-fishing by the name of Theodore Gordon. I had never heard of Mr. Gordon before this (perhaps that is because I tend to fish a lot more soft hackle flies rather than dry flies), but I found a quote by him quite interesting. He said, “The great charm of fly fishing is that we are always learning.”

This morning I thought how apropos this statement is to software testing. But, there of course, as it pertains to the practice of software testing I would rephrase Mr. Gordon’s statement to state, “The great demand of software testing is that we must always learn.”

There are different types of people who fly fish. There are those who buy a fly rod and a pocket full of store bought flies and head to the water just to catch fish. This group of people don’t seem to care much about the techniques of casting, learning how to read water, or understand patterns of fish or aquatic entomology; they are simply there to try to catch fish using whatever slop-shod mooching approach seems to work. Yes, they still catch fish…usually the small, dumb farm raised trout stocked into regional lakes by the state’s department of wildlife who will bite at anything. On the other end of the spectrum are true purists who fish with cane poles, tie their own flies to match the hatch (sometimes right beside the river), and study trout, regional entomological lifecycles, and the geological formations of a river bed to better pin-point where the big, smart trout are hiding. Then there are the group in between these extremes with varying degrees of skill and knowledge. Depending on how much time a person devotes to both practice and learn (and their capacity to learn) about the sport will often make a huge difference in both their enjoyment and their effectiveness in catching large, persnickety trout.

From my observations of the testing community over the past few decades, I can see a similar pattern regarding the spectrum of skills and knowledge of people who participate in the practice. In the past, it was not uncommon for some companies to hire ‘clever’ people who were simply good at finding bugs into testing roles. Some companies hired developers who would (often times begrudgingly) take the job as a stepping stone into a developer position. Unfortunately, some people at both extremes of this spectrum often stagnated because they did not learn more about software testing or the technological advances that were happening around them. At one extreme, I suspect that some people thought as long as they were finding behavioral type issues they were providing a benefit to the company because they were ‘good at representing the customer.’ At the other end of the spectrum the developer’s in testing roles who failed to realize the challenges in software testing. In both extremes complacency and stagnation usually occurs. Of course, there are many other testers between these extremes; some who will go on to become professional testers and have significant impact, and others who will simply belly-ache and whine about how unfair it is or claim how wrong any change is and why change won’t work.

As professional testers we must constantly strive to improve our knowledge of testing, technology, and the systems we are working on. We must also increase our skills and abilities as the demands of the role expand beyond the traditional comfort zone of behavioral testing and ‘playing customer advocate’ by executing ‘tests’ to find bugs at the end of a cycle. The challenges of testing complex systems built around advancing technologies significantly raises the aptitude bar for testers. Emerging practices such as TDD and agile lifecycles designed to drive engineering quality upstream and form closer customer connections is also impacting the role of testing and how testing adds value in the lifecycle (and I don’t think the role of testing in an agile lifecycle is trying to wedge testing between the end of a sprint cycle and the release to customers in order to provide a pseudo-proxy customer buffer…that’s a bottleneck.) Reinstituting best practices such as design and code reviews and inspections (when warranted), or developing new approaches or tools to help increase testing effectiveness and reduce costs also require greater skills and knowledge among testers.

The formidable challenges of testing software that lie ahead will require highly intelligent critical thinkers who also have an in-depth understanding of the systems they are working on, and who possess the technical aptitude to provide valued input in throughout the product cycle. Indeed we work in a very dynamic industry filled with diverse challenges that demand continued learning and greater proficiency of the skills used in our profession.

Last week I was at the Test2008 conference in India. The organizers from PureTesting planned a grand event with workshops in Hyderabad, Delhi, Bangalore, and Pune. Then the main conference was then held outside of New Delhi. When I arrived in Delhi at the conference I was told I would be on a discussion panel. Surprise!

Although the conference organizers thought the topic would be controversial, in retrospect it turned out to be a non-issue to the majority of the audience. But, during the discussion one person asked the most important question of the session. He essentially said that new people coming into the industry and specifically the testing discipline are sometimes confused because there is sometimes contradictory information. "So," he asked, "how do new people know who the leaders in testing are?"

Rather than drone on forever, here is a list of traits of leaders whom I respect, and attributes I try to follow when I lead a team or mentor people.

• Leaders are able to foresee  technological changes and changes in business practices on the horizon and predict how those changes will influence the careers of the people they manage or mentor.
• Leaders don’t let the people they are managing or mentoring to become stagnant.
• Leaders constantly seek opportunities for the people the manage or mentor to flourish.
• Leaders constantly help the people they manage or mentor develop their careers even if that means moving to a different role or team.
• Leaders connect with the people they manage or mentor and develop a nurturing bond.
• Leaders delegate responsibility because they explicitly trust in the people they manage or mentor to do the best they can do. Similarly, people respect leaders who they know grew to become a leader and were not merely placed in some position of management.
• Leaders understand the challenges in the industry and they unleash the potential of the people they manage or mentor to take on and tackle those challenges. If the team fails the leaders accept the responsibility and support their people for giving it their best effort. Then, they rethink the problem, and try again.
• Leaders don’t say that something can’t be done, or you can’t do such and such; they continuously search for alternative solutions to problems.
• Leaders identify hard problems and point the people they manage or mentor in the right direction and say, let’s figure out how to solve this together as a team!
• Leaders don’t whine about changes in the industry. We work in one of the most dynamic industries in the world, and leaders can successfully lead their teams to face new challenges head on.
• Leaders don’t shamelessly ridicule other people or hurl personal insults.
• Leaders challenge the ideas and statements of others, but they do so in a professional manner.
• Leaders present compelling points of view based on rational logic and empirical analysis. Not everyone may agree with a point of view, but they comprehend the results, and may sometimes present conflicting data which is repeatable in unbiased studies.
• Leaders must also occasionally make hard decisions that may be unpopular with the people they manage or mentor or even their own managers.
• Leaders don’t attempt to segregate the discipline or mislead neophytes with reckless statements based on emotional or philosophical ideals.
• Leaders have a strong personal constitution and are not swayed by emotional opinions or baseless peer-pressure.
• Leaders not only strive to improve the people around them, but they also continually strive to be the best they can be.
• Leaders never become apathetic or dispassionate. (If a person is apathetic or dispassionate then it is way past time for them to leave and pursue other directions.)
• Leaders are often recognized as technical experts in their fields.
• Leaders are respected by other leaders in their field.
• Leaders don’t refute challenges to ideas or statements with hypothetical or philosophical multi-syllabic hyperbole, they present substantiated facts or logical and rational points of view within the context of the discussion.
• Leaders know to criticize in private and promote in public.
• Leaders are also competent contributors.
• Leaders know the difference between ‘big-bang’ one-time dog-and-pony shows, and achievements that provide lasting results, and they reward accordingly.
• Leaders figure out how to permanently fix small problems so they can tackle larger and larger issues.
• Leaders drive themselves and others around them to be the best they can be because they know that being good enough is simply not good enough in the long run.
• Most importantly, leaders provide strategic direction and help guide and grow the people they manage or mentor to face new and exciting challenges.

I rattled off some of these traits and in the end I told the person that I have come across a lot managers in this industry (people who have people reporting to them in some capacity), but in my opinion there are too few real leaders. Fortunately I personally see many new highly knowledgeable and technically skilled people coming into the discipline again along with many experienced people who are reemerging. These people represent the potential for the type of leaders we need to help drive the discipline forward. Are you one of those people?

As a young lad growing up on the shores of the Chesapeake Bay I would often spend part of my summer vacation from grade school helping my grandfather work the crab pots on the north shore. Now, don’t think “Dangerous Catch,” crabbing in the Chesapeake is much different than crabbing in Alaskan waters, although we did have our share of squalls and nor’easters that will humble any man. Crabbing is hard work, especially for a young boy. On one particular day working the pots with my grandfather I recall something he said which has stuck with me my entire life. The water was rough that day and we were being tossed around a bit. It was raining…not like the wimpy Seattle rain, I mean hard East coast rain that comes down in sheets with drops so large they hurt when they hit you. Even in the driving rain the smell of dead fish, rotten chicken pieces and turkey necks that we used for bait permeated everything, the slime of fish guts coated the decks making footing somewhat precarious, and the brine of the Chesapeake seeped into every little cut and nick on my hands. I wasn’t having a lot of fun that day, and although my grandfather knew it he didn’t say anything to me until I complained aloud. Now my grandfather was a tall, barrel-chested man with a stern face, and he never put up with any senseless, petty whining. But, instead of scolding me he simply said, “Son, we are not rich and there is no inheritance coming your way, so you better find a job you like doing.” At first, I was a bit puzzled. What did that have to do with how I was feeling at the moment? But, I soon realized what he meant, and my grandfather’s sage advice has stuck with me ever since and guided me in my career pursuits.

My father, who was also every bit as practical and pragmatic as my grandfather taught me the value of working hard, and the virtues of responsibility. He taught me to take pride in the things I did well, and take responsibility for things that didn’t work out so well. My father also taught me never to quit, to always look at alternative options, and that the path of least resistance or the easy path through life is not always the best path to take. He also said if something was worth doing, do the very best you can do, and never sit back and rest on my laurels. My father was not the type of man to do something half-way, or ‘good-enough.’ One summer we built a new barn on our property. This wasn’t the typical barn, instead my father got some telephone poles and even got his friend at the telephone company to bring out a truck with an auger to drill the holes 10 feet deep! Next we used oak planks between the stalls. Now hammering nails through oak is not easy business and the entire project took about 2 weeks. But, my father was adamant about not having to rebuild the walls of the stall if a horse tried to kick it down.

Practicality in practice

Now perhaps it was my upbringing, or perhaps it is my Myers-Briggs ISTP personality traits, but it is hard for me to understand how some people embark on a career, or pursue a job opportunity without knowing (or perhaps worse..completely ignoring the fact) that they will be challenged to continually improve their knowledge and skills everyday. This is especially true of many of our chosen profession of software testing. We work in a highly dynamic industry and we must continue to learn and develop new skills to meet the growing diversity and increasingly complex challenges that will potential advance the discipline as well as provide benefit to our employers.

Sure, we can whine and complain about the changes. We can point fingers, and otherwise express negative emotions and utter baseless, counter-productive propaganda slogans such as, “automation can’t do such and such,” or “I know non-coding testers who find more bugs than (fill in the blank here)” or the virtually inverse argument “tester’s  who write code are biased and don’t know how to think like an end-user.” I learned a long time ago that although misery loves company, it is usually not those who whine the most or put forth pointless and petty gripes, or who only see the world through bi-polar, black and white lenses who drive meaningful change. Sure, it is easy to empathize with some people in some situations; but empathy without practical solutions to resolve the situation effectively is simply a pathetic play of the irresponsible victim card.

For example, we unfortunately often get mired down in a this vs. that, exploratory vs. scripted, or STE vs SDET debate that is generally too myopic (on my project… blah, blah, blah) and often counter-productive (meaningless comparisons, ridiculous measures such as raw bug count, and baseless assertions that lack empirical evidence to substantiate the claim, etc). There seems to be an opinion that if someone writes automation, or comes with a coding background then they are not good ‘exploratory’ or behavioral, or (and I hate using the phrase) “black-box” testers.  (All testers are “black-box testers!). Yet, some of the most talented testers I know in the industry began with very little in-depth knowledge of the ‘system’ or an understanding of programming concepts and applied themselves to  grow their knowledge and skills about the overall system and technologies to advance their careers, their teams, and the industry by taking on greater challenges. I also know highly talented testers with great industry experience and very strong coding backgrounds who are masters at explorative, behavioral,  or “black-box” testing.

So, for those of you who have been living in a cave, or have your head permanently implanted in the posterior end of your alimentary canal it is obvious the industry and the profession of software testing is changing and there is an increasing demand for greater system knowledge, experience, and technical skills for many testing positions in the industry. There are many reasons for these changes, and the simple fact is that the industry’s needs and strategic direction primarily dictate what skills and knowledge are required to remain competitive in the industry and advance the business.

Are ya’ gonna lay there and bleed, or ya’ gonna’ cowboy up!?

Now, many people mistakenly assume that I only promote technical knowledge or advancement though increased technical skills. (Of course, many of these people are so narrow minded and biased they only equate ‘technical knowledge and skill with ‘coding’ skills and programming knowledge.) I do talk about these subjects a lot because…well, they are directly related to our professional growth, and because I studied and interviewed thousands of testers to perform a skills gap analysis (delta between the disciplinary needs of the business in relation to the actual skills and knowledge of the existing workforce at the time). The skills gap analysis indicated that many people in the testing industry at the time were well-educated, intelligent people who were very capable of thinking critically about problems within the appropriate context. But, it also illustrated that a significant population in the testing profession at that time lacked in-depth knowledge of the system they were working and the technical skills to expand their own productivity or become more influential in driving product quality. So, as a manager and mentor it was (and is) my responsibility to help people understand changes based on the maturation of the industry and the profession, and to help them grow professionally, and that is the role I took on with great passion. I decided to focus on growing people’s technical skills and knowledge, and their understanding of established, and contextually effective software testing processes.

One of my first public talks revolved around the idea that good-enough was simply not good enough any longer based on changing customer views and increasing maintenance costs. That was in 2003! In 2005, I presented empirical evidence suggesting that exploratory testing by ‘presumable power-users’ and ‘domain experts’ is not as effective as some claims, and I advised testers to begin thinking about the future and the increasing demand for testers with more in-depth system knowledge and technical skills to not only remain effective and productive in the industry, but to help drive it forward. Some people jeered me, and one respondent from the StarWest 2005 conference wrote “Bj is a fool, and his talk shows why Microsoft products suck and why Microsoft will fail” on the feedback forms. A few others said they agreed with the message, but they really didn’t like hearing it out loud. One person told me “Yes, we know we need to improve, but your talk was like hitting us in the face with a frying pan full of hot grease.”

I mention this not to “toot my own horn;” that is simply not my style, for I am yet a simple man who is still learning to improve myself. I only mention this because here we are 5 years later still arguing over petty things that we sometimes do not have the knowledge, skill, experience, business insight, scope of influence, or credibility to change.  (And in case you don’t know, whining. griping, and emotional conjecture is simply not viewed as being very credible in trying to influence change in a room full of decision makers or business leaders.) Right now, the industry, and the testers in the industry need leaders to help them adjust to changes, provide strategic vision, and guide career growth that enables each person to be successful in their professional development and compete with their peers. Responsible leaders are those who face challenges head on, who put forth logical and rational arguments based on analysis and empirical evidence, but who continue to search for ways to solve not only the immediate problems, but also have the ability to envision potential side-effects and deal with those as well before they become problems.

Our profession is multi-dimensional, and as testers we must continually strive to increase our knowledge and our skills in order to take on new and exciting challenges in this dynamic and agile world of software testing. The path is not always easy, but it does provide many rewards to those who work in a job they truly enjoy.

It has been quite some time since I have posted. Part of that is due to personal distractions (getting my garden planted and my sailboat ready for the upcoming season), and part of that is being ‘in the zone’ working on some special projects at work. DeMarco and Lister began talking about being "in the zone" in their famous book Peopleware written in 1987. (In my opinion, this is a must read for anyone in the software business, especially for managers who should reread it yearly.) The Croatian psychologist Mihaly Csikszentmihalyi also discusses a similar concept he calls "flow," and identifies 9 characteristics of ‘flow.’

Being in the "zone" for me is a myopic mental state where we are so focused on completing a task the world whizzes by and time becomes irrelevant. For many it is sort of a magical place; a momentary escape from reality. For example, when I sit down to write code in the evening the time passes so quickly that I soon discover it is 1 am in the morning. But, I want to complete one more class, and before I realize it is 4 am. (I am a slow coder.) People who are addicted to computer games know  all too well about the ‘zone.’

This past month I had to come out of my zone and fly down to San Mateo, California to present a workshop and 2 talks at the Software Testing and Performance conference. It was a welcome break, and was nice running into old friends and meeting new acquaintances. I had the pleasure of meeting Karen Johnson and reconnecting with Doug Hoffman (who I had only met once previously) for lunch one day, and interestingly enough the conversation found its way to being in the zone. Karen brought up the fact that email is a constant distraction that often times impedes productivity.

Often times when we are in our zone our productivity increases. But, every time something changes our focus, such as responding to an email on a completely unrelated or tangential topic, or answering a phone call we are sucked out of the zone, and according to Lister and DeMarco it takes approximately 30 minutes to get back into the zone or back to our peak point of productivity.

For example, when I sit down to write, or code, or meditate I don’t want to be disturbed and will usually retreat to my boat or some quite place where I know I won’t be disturbed. I turn off my cell phone, no radios, no newspapers, magazines, no instant messaging (which I abhor) , and certainly no email. If I must stay at the office, then I block of my calendar for at least 4 hours and will sometimes disappear into some nook or cranny on campus.

I know that email is the life blood of many tech-companies. Unfortunately, I suspect that many of us could use a good bloodletting and need to relearn the art of verbal communication. I also suspect that we could better optimize our time by not being tethered to some email client during every single minute our waking hours. Let’s face reality. For most of us 80% of the email we get is noise that we will forget about within the next 4 hours or so, 15% is good to know stuff (but not necessarily critical to our success), and I suspect that approximately 5% is really important stuff that we must respond to immediately. Of course, these percentages depend on our primary job. For example a manager or a consultant probably gets a larger percentage of email the requires immediate response.

So, I wonder if we managed our own use of email (and instant messaging) a bit more effectively how our own personal productivity might increase?

The debate over whether testers need to at least understand programming concepts is still raging within the discipline. To me this debate is puzzling because it seems to suggest that as a professional, I don’t have to really understand or be completely proficient in critical aspects of my trade. Even Cem Kaner noted, “I think that the next generation of testers will have to have programming skills.” Actually, there was a time not so long ago when testers had to have programming skills, so it is nice that Cem now acknowledges that skill as useful in testing.

Unfortunately, occasionally even within Microsoft a few people still want to differentiate between STE and SDET by blindly assuming that STE meant non-programming testers. The fact is, that the old STE ladder level guidelines clearly stated skills such as debugging production code, and design and develop effective automation as required skills for Microsoft testers. Unfortunately, some managers chose to selectively ignore these skill requirements and some groups chose to differentiate between GUI testers and  any tester who could write code by labeling them as STE and SDET respectively.  (This was a horrible abomination of job titles in my opinion.) The new SDET competencies at Microsoft are designed, and supposed to be implemented in a manner, to reinforce the essential skills we expect from our testers so a tester at a certain level in their career stage in one business unit essentially has equitable skills of any other tester at the same level in their career stage in any group in the company.

But, people are often resistant to change, and as I wrote in my last post some people choose to wallow in self-pity, pretend they are a victim of some evil plot, hypercriticize change with dogmatic arrogance, and incessantly bemoan dubiously negative aspects of change from an often overly emotional narrow-minded perspective. A person who moved from a testing role to program management stated, “I was a tester because I understand how users think and how they use products and I wanted to use that knowledge to make our software better.” Really? We make software better by beating quality into it? Does this demonstrate a good understanding of software processes and good business logic? I only ask these questions because it is pretty well-known that it is much cheaper to prevent defects, and that many defects can be found in the design process. So, I am asking myself why in the world didn’t this person start as a Program Manager (responsible for interpreting marketing analysis and customer feedback into requirements and product design) or become one before now? What is even more amazing about this statement is that it doesn’t even acknowledge the fact that as a program manager this person is now in a role that should have a direct connection to the customer and a greater impact on making our software better. A development strategy or process that emphasizes customer advocacy primarily in the testing phases is ridiculously immature and a gross waste of resources since it is widely known through empirical studies that it is cheaper to prevent defects by better designs and clear requirements as opposed to finding them during a testing cycle.

The same person stated, “I wanted to keep breaking software in the incredibly fun, very effective way I had been doing.” (Personally, I find API testing (which can also use a black-box approach), and white box test design extremely fun and intellectually challenging, and is also very effective when used appropriately.) Unfortunately, this comment seems to perpetuate a myth that testers make software better by finding bugs, and it also demonstrates an extremely limited view of the role and overall potential value of software testing to an organization. This is indeed a very narrow, antiquated (in technology time), and immature view of software testing that emphasizes testing as primarily a bug finding endeavor. However, Beizer wrote that black box testing exercises approximately 35 – 65% of the product, and Marne Hutcheson and I have empirical data that demonstrates that GUI testing (which is one type of black box testing most people are familiar with, and the type of testing most non-technical testers are limited to perform) is not as effective as most people want to believe, and it is often more costly as compared to using a variety of approaches to software testing. Again, even Kaner notes, “Programmer productivity has grown dramatically over the years, a result of paradigmatic shifts in software development practice. Testing practice has evolved less dramatically and our productivity has grown less spectacularly. This divergence in productivity has profound implications—every year, testers impact less of the product. If we continue on this trajectory, our work will become irrelevant because its impact will be insignificant.” (I highly suspect the ‘testers’ Kaner is referring to in this context are primarily non-technical, GUI testers since that is the type of testing emphasized in his BBST course.)

There is no doubt that a person who does not at least understand programming concepts or have an in-depth technical understanding of the system they are testing is unable to perform various activities that may be required in the role of a professional software tester. That person cannot perform code reviews (which have been proven to find certain classes of defects more effectively than any other type of testing); they cannot analyze code to determine which areas of the code have not been tested and design tests from a white-box approach to increase testing effectiveness and reduce risk, they cannot debug errors and identify root causes of defects, they cannot automate tests to free up their time or reduce costs during the maintenance phase of the product lifecycle, they may not be able to adequately analyze and decompose test data, etc. While some companies don’t rely on their testers to do this type of work, these are certainly tasks that any professional tester should be able to perform.

I guess there are some software companies that are not interested in actually maturing their processes, reducing long term costs, have no interest in the intellectual property value of testing artifacts, or simply want to continue to rely primarily on GUI testing to get a ‘gut-feel’ of their product before releasing it. However, many large companies that produce software (Microsoft, Cisco, Google, Siemens, etc.) understand the value add proposition that professional testers provide to the organization health and specifically hire people into testing roles who have both broad technical skills as well as the common traits we tend to associate with good testers.

This post is not to question the need for non-technical people who have in-depth and current domain or business knowledge of the application space, or who understand the market expectations and customer demands/needs in the software engineering process. The question I ask is whether the value these individuals bring to the software development process is misplaced, or would their contribution be more cost effective and provide greater overall value to the customer if they were in a role (other than testing) that better utilized their knowledge by contributing to defining requirements and designing high quality software rather than trying beat in quality through bug finding?

"If a man is called to be a streetsweeper, he should sweep streets even as Michelangelo painted, or Beethoven composed music or Shakespeare wrote poetry. He should sweep streets so well that the hosts of heaven and earth will pause to say, here lived a great streetsweeper who did his job well."- Rev. Martin Luther King Jr.

As I was growing up my parents taught me the value of working for things I wanted, and to how to bear the burdens associated with responsibility. When I was a teen we moved to the country because my mother had become very active in horseback riding and my father wanted to stop boarding the horses. So, we moved out of the suburbs and away from the upper Chesapeake. I was not really happy about this because I love the water. As a young boy I spent a great deal of time with my friends hanging out in the marinas where I learned a lot about boats, hard work (from the bright work to the bilge), and life in general. As I young lad I remember listening intently to the trials and tribulations of old blue crabbers, fishermen, and sailors. I loved swimming in the tributaries of the upper Chesapeake, fishing for yellow perch, sunfish, and going crabbing and walking the riverbeds at low tide for soft-shell crabs. I also was not really looking forward to the change because it meant getting up very early in the morning to feed horses, and cleaning stalls after coming home from school. But, since I also opted to get a horse, that is burden of responsibility I had to bear.

Throughout life our environment changes. That’s just the way it is. New ‘doors’ open, and some ‘doors’ close. I often recall my first manager at Microsoft telling me that it is up to me to control my own destiny at the company. But, I had been around long enough to realize that life also sometime throws us curves or unexpectedly changes direction, and Microsoft is certainly a dynamic company. It is often during times of change where new and exciting opportunities present themselves. Even during times of change I believe we still control our own destiny (at least somewhat). When our environment changes (as it always does) we generally have many choices. For example, often we can embrace the change and dynamically adapt and/or rise up to new challenges that life presents. Or, we can choose to wallow in self-pity, pretend we are a victim of some evil plot, hypercriticize the change with dogmatic arrogance, and incessantly bemoan the dubiously negative aspects of the change from an often overly emotional narrow-minded perspective. This latter choice is usually not an especially productive one (personally or professionally), and generally only demonstrates one’s extremely biased, and limited view and their incapacity to grasp the "big picture."

Let’s face it; we have chosen to work in one of the most dynamic industries in the world. Change is all around us, although some things remain relatively the same. For example, the techniques medical doctors use in the initial screening of certain maladies have remained relatively constant for decades, but at the same time advancements medical imagery has made tremendous technological leaps forward. Likewise, the practice of exploratory testing has been used extensively in software testing for decades, but the effective application of combinatorial analysis (pair-wise, triplet, n-wise) of interdependent or semi-coupled parameters has only recently become a mainstream best practice.

The testing discipline is undergoing tremendous change these days. There are many people around the world who are serious about maturing and advancing our profession. Some ideas are great, some still need to be refined, but at least they are seriously investigating at ways to advance the profession. As a tester working in a rapidly changing industry we must constantly re-evaluate our skills, and increase our professional knowledge of software testing and computer systems in general in order to provide the best possible service to our employer.

I think if someone chooses testing as a profession, then they should strive to become a professional in the discipline, and develop and refine the skills and knowledge that entails.

The easiest thing to criticize is that which one does not fully comprehend.

There has been a lot of discussion lately about Jerome Groopman’s book How Doctor’s Think and a correlation between doctors in the medical profession and software testers. The book is an excellent read, and provides readers with valuable insights not only with medicine, but we can certainly abstract many of the fundamental lessons to testing software. Indeed there are many similarities between doctors practicing medicine, and software testers testing software. I won’t belabor this point since many others have discussed the similarities, and one only has to read the book to discover the obvious.

However, there are several aspects that some people seemingly blindly overlook when comparing medical doctors or others in the medical profession to software testers. For example, medical doctors are highly educated in biology, chemistry, human anatomy, physiology, etc. In other words, doctors spend a great deal of time being formally educated in subjects directly related to their profession. Compare this with many testers working in the field of software testing. This is not to imply that in order to be a professional tester that one must study computer science at a university; however, I know a great number of testers who have worked in the industry for several years and still do not understand basic programming concepts, lack the ability to perform an in-depth analysis of test data, and are incapable of adequately decomposing or modeling features below the user interface. In other words, testers who lack the basic knowledge of computer systems and software are utterly blind to the internal workings of the system on which they are working. Doctors also spend a great deal of time post graduation reading medical journals and attending medical conferences and workshops to continue their education and build their knowledge and skills. Dorothy Graham, Marne Hutcheson, Steve McConnell, and I have collected a lot of empirical evidence to suggest that very few testers have ever received any formal training in software testing methods, and less than 1% of testers polled have read more than one book on software testing. Lee Copeland presented an excellent talk at a conference entitled "The Nine Forgettings" in which he asks, "How can we call ourselves professional testers if we are ignorant of the fundamental techniques of our craft?"

Doctors also realize there are practical techniques (systematic procedures) commonly used in their profession that are extremely useful in the correct situation and valuable in either diagnosing an ailment or identifying contraindications, and competent doctors know when and how to apply those techniques. Conversely, I often hear testers whom I suspect have a superficial understanding of the overall system or fail to adequately perform an in-depth investigation of the system on which they are working refer to testing techniques as folklore. Who needs techniques…if we pound on the keyboard or touch screen long enough we are surely bound to find a bug! (The only folklore I’ve ran across in the industry is a belief that only a handful of people really understand, or are even capable of understanding exploratory testing.)

Medical science also has an common body of knowledge and professional jargon that is universally accepted throughout the trade. When doctors refer to exploratory surgery there is a common understanding of what is being discussed. Doctor’s don’t pointlessly argue how one person’s practice of exploratory surgery is different than another person’s approach and therefore it is not really the ‘exploratory’ surgery that "I" preach. Reputable doctors also don’t needlessly make up words to make themselves sound like they offer something new or revolutionary. When they discuss medical conditions they discuss things within rational contexts. For example, when someone complains about numbness in the fingers they may perform tests to check for adequate blood flow to the hand, or they may ask the patient about neck injuries or pain in the neck or upper shoulder region to see if the problem might be caused by a pinched nerve. I don’t suspect that too many doctors would ask the patient what they had for breakfast, or if they recently stubbed their toe to diagnose numbness in the fingers (but, I am not a medical doctor so that is just guessing on my part). Basically doctor’s understand the head bone is connected to the neck bone and don’t waste a lot of time hypothesizing "what if the head bone was connected to the knee bone?"

Finally, a really big difference about doctors and many testers is in how they fundamentally approach their job. Doctors are constantly striving to help people stay healthy. In other words, they are trying to prevent the spread of illnesses, or searching for ways to help people from becoming sick to begin with. Of course, this means that doctors must have an in-depth understanding of the ‘system,’ the types of ‘bugs’ a system might be exposed to, and how those ‘bugs’ can act on the ‘system’ or how the ‘system’ reacts to those foreign agents.

As Groopman discussed in his book, "There are primary care physicians in every hospital who speak with great sensitivity and concern, and their longtime patients love them, but clinically they are incompetent…" Likewise, I have met many people, some at MS and some in the industry, who still think software testing is simply about finding bugs, and that additional skills or knowledge of the profession is unnecessary if one is a good bug finder. I often listen with amusement at talks or when I read articles by people who profess that testing is about providing information, but the only ‘information’ they seem to provide is how to expose yet another obscure bug, or denigrate practices by incorrectly applying techniques out of context (such as attempting to apply pairwise analysis on a function requiring sequential inputs then claiming pairwise analysis is not a best practice because it is not good at finding bugs while failing to mention where and when the technique is best applied and other types of information that the technique provides such as increased code coverage). As I said in the opening, it is easy to criticize those things we least understand (unless of course we are purposefully obfuscating facts for personal motives). And the uselessness of any critique is only exacerbated by irrational arguments, illogical alternatives, or personal attacks.

So, how does all this relate to Microsoft’s hiring practices and why we primarily hire people with computer science backgrounds?

Easy, graduate coming from university with a computer science background have a strong understanding of the ‘system’ and the ‘system internals’ much like the doctor understands physiology and human anatomy. This doesn’t mean that we only hire testers with a computer science degree. Myself, and many other testers at Microsoft do not have a computer science degree; however, we have also not stagnated or simply rested on our ability to find bugs. We have constantly strived to improve our technical skills and overall understanding of computer systems, software testing practices, and testing knowledge. Interestingly enough, Microsoft career job guidelines have always required testers to be able to automate tests, debug production code, and design tests from both a white box and black box perspective, etc. The difference now is that those skill requirements are universally applied across the company.

Also, much of our internal training can now be based on a common baseline of expectations. For example, when we teach testers how to design tests from a white box perspective, or when I talk about the best practices of code reviews I expect that the testers already know how to read code. Our automation courses require that testers already know how to write code using procedural programming and object oriented programming approaches. Our training simply exposes testers to namespaces, classes, and methods commonly used in test automation (not in application development), and mentor them on how to design tests that will provide value to the organization from an automated testing perspective. (Also, I can attest that our testers are very good critical thinkers and not simply mindless droids who bang on keyboards or pump out simple rote, prescriptive scripts (similar to what is described here) and label them automated tests.

Finally, one of the things we reinforce in our training and throughout a person’s career growth at Microsoft is driving quality upstream and defect prevention. We have senior testers who are mentoring developers in how to design and write better unit tests. Our testers are engaged in design and code reviews. Many teams now rely on testers to debug problems to the line of code in order to identify patterns of problems. We have testers who are doing root cause analysis of specific classes or types of defects and building tools or enabling processes to identify those types of defects earlier in the project or development cycle. And yes, our testers still design and execute a lot of manual tests. They use the product in development everyday by dog-fooding. And most testers even participate in newsgroups and sit with or listen in on product support calls to indirectly hear from end users.

So, much like you probably seek out the best possible medical doctor as a primary care provider or when struck with an illness, at Microsoft we want highly competent individuals in our software testing roles who understand that testing is a huge challenge and requires a great breadth of skills and knowledge in order to test software from various perspectives and approaches, and who possess a great deal of in-depth knowledge of the system they are working on. We seek professionals who realize that constantly bemoaning the inadequacies or drawbacks of an approach or perspective solve nothing, and instead actively engage in finding great, scalable solutions to problems in order to reduce testing costs and help drive overall quality upstream.

In a dynamic industry that changes rapidly as professional testers we must constantly reevaluate our skills and knowledge to stay abreast of changes in technology, and learn more about our chosen profession. We must be vigilant and strive unceasingly to improve our skills and knowledge of computers, computer software, and our profession in order to remain competitive with our peers and remain a valuable team member in our organization or to our employer.

It never ceases to amaze me that every time we see a calamity involving software the immediate reaction of the sensationalist media types and other people who are generally misinformed is to blame inadequate testing. Recently, it seems that Joel Spolsky not only fell victim to rumors and misinformation, but also to the lunacy of blaming the testing organization for the rather lack luster adoption by individual consumers to move to Windows Vista.

Sales of Windows Vista are slow for a myriad of reasons. The UI is a radical departure (which most design experts applaud), security features are annoying (which most security experts applaud, yet many individual consumers dislike), machine requirements (means many would have to purchase new hardware), performance issues, etc. But inadequate testing or too much automation? Pah…leaze! I usually like reading Spolsky’s blog, but must admit this is the first time I can recall in which he has used unsubstantiated rumors and innuendo to reach such a faulty conclusion as suggested in his post. Too bad…

Joel’s attack was not about testing per se, but more about his assumption that Windows Vista suffered because of too much test automation. He based his conclusion on the information from a blog noted for its petty bitching about things inside of Microsoft. (Let me say this…there are a lot of things about MS and corporate policy that I don’t like. Bitching about those things is easy and usually non-productive. But, working with the diversity of people who make up one of the most influential software companies of our time to effect change is difficult and challenging. I mostly choose the later route.) And the non-productive bitch-fest blog based its disinformation on the observations of Chris Pirillo’s blog post written in May of 2006 exposing several critical issues such as:

• Shouldn’t the phrases used during the setup state (Copying Windows files, Expanding files, etc) be intercapped? i.e., Copying Windows Files, Expanding Files, Installing Features, etc.?
  [Uh....no. Check with the terminologist and documentation experts]
• The "You’re Ready to Start" graphic flickered and jumped when I pressed the Windows button.
  [Does it happen every time, or was it an anomaly, or did was it perhaps an electrical or wind storm...because here in Washington when we get wind storms my lights flicker all the time.]
• Control Panel>Additional Options displays…get this…"There are no items to display." Well, if there are no items to display, then why did you show me the link in the first place?
  [Perhaps because when user's install Additional options such as the Java Control Panel, they have a common place to access them]
• The Undo and Redo button for the Photo Gallery Viewer are at the bottom of the window, whereas I would’ve expected them to be at the top. Thought they weren’t there until I looked hard.
  [ Is this inconsistent with other Task Panes? Or is this based on your personal preference? BTW...I happen to like them at the bottom no cluttering up other stuff.]
• What if I don’t like the red "X" close button? Why can’t I change that to…purple?
  [Great suggestion that I am sure millions of user's will appreciate. I think they have that scheduled as a hotfix somewhere around the turn of the next century]
• Microsoft Windows Mail splash screen needs an overhaul
  [Great...I am sure the customers who really want less functional defects or better performance will be more than happy to know we instead decided to spend time overhauling the splash screen based on your valuable input.]

This is not to say that Chris’ feedback was ignored or not valued (I also enjoy reading Chris’ blog from time to time). Personally, I would like to think we investigate a lot of the feedback we get from our customers (we simply don’t act on all of it because…we’ll we have bigger problems to solve other than allowing users to change the color of the close button.)

Spolsky stated, " The old testers at Microsoft checked lots of things: they checked if fonts were consistent and legible, they checked that the location of controls on dialog boxes was reasonable and neatly aligned, they checked whether the screen flickered when you did things, they looked at how the UI flowed, they considered how easy the software was to use, how consistent the wording was, they worried about performance, they checked the spelling and grammar of all the error messages, and they spent a lot of time making sure that the user interface was consistent from one part of the product to another, because a consistent user interface is easier to use than an inconsistent one. And, None of those things could be checked by automated scripts."

Unfortunately, I guess Joel hasn’t remained abreast of advances in test automation and perhaps assumes that automation is still primarily record/playback or simple rote scripts in some proprietary or limited scripting language. As software becomes more complex, and as the capabilities of test automation improve the fact is that quite a lot can be done with automation.

For example, checking the consistency of fonts in each dialog throughout a product is more effectively and efficiently accomplished via automation than by human eyes. Automation can also detect differences in the alignment of controls on dialog boxes and also detect clipping and truncation of controls or text in controls more efficiently than humans (and we actually have empirical data to prove it). Spelling and politically sensitive words are more efficiently detected via automation. Most testers and developers that I know are not experts in English grammar and I would never hire an English language major as a tester just to test grammar. We made our documentation/content experts responsible for messages and other ‘string’ content thus pushing quality upstream!

(Also, as a side note…the majority of the testers that worked on Windows Vista were the "old testers at Microsoft" doing they same work they did shipping Windows Xp.)

Isn’t if funny that when certain things don’t meet our personal expectations (Vista sucks), or we are fearful of change (test automation is evil and we should continue to hire hoards of non-technical people to bang on keyboards) it is so easy to jump to faulty conclusions, base assumptions on unfounded hearsay and rumor, constantly utter petty unproductive complaints, and generally play the victim. Of course, being a victim provides you with attention from others, and useless bitching and wild speculation is way more fun than facts and research…besides…it makes people laugh, and when people laugh they are happy, and the role of the tester is to make people happy…right?

When some people talk about “expert testers” they often refer to the personal traits such as curiosity, passion, dynamic, detail oriented, strong constitution, honesty, integrity, etc. We can describe the characteristics of these traits that we admire and we certainly look for when hiring new testers. People can strive to emulate the characteristics of these traits, but it is virtually impossible to teach someone to be passionate or curious. So, are there skills and knowledge we can teach to increase expertise and enable people to become more effective and more efficient in a increasingly complex world?

There are people on our teams whom we respect because they seem to have a knack for testing. But, testing is not an art or a knack that you simply acquire during your journey down the road of life. The testers most respected by other testers and developers alike seem to consistently demonstrate tacit knowledge of the discipline. So, in order to better prepare all testers at Microsoft to excel in their roles I was interested in the tacit knowledge of our star performers that could then be converted into explicit knowledge (which is something that can be more easily learned). I am wrapping up a case study on this work, and I can already see that many conclusions map to an article Rob Foshay wrote regarding expertise and the types of knowledge that are common in experts in various fields. The article summarized three types of knowledge; how things work, context knowledge, and ill-structured problem solving. I found it especially interesting how these categories mapped so well to the skills and knowledge of software testers in general.

How things work implies in-depth system knowledge. (This is not general systems thinking.) Forshay describes this as follows; “…the expert knows the parts of the system and how they interact, both when they work as intended, and when they don’t work as intended. Furthermore, the expert can predict or explain how the system is behaving, or how it will behave if he or she makes a change.” So, what are the parts of the system for testers? The system includes the domain or customer space, different supported hardware platforms, operating systems, interfaces between applications and the operating system and/or hardware, the application itself, algorithms in the application, and even the programming language used are all important parts of the system and provide valuable context for designing and developing tests.

Context knowledge is the understanding of “…the particular requirements and design problems of a particular type of system, and its users, and how various designs have worked in the past.” It is always important to identify clear goals and define a course of action to achieve those target goals rather than guess and simply try different stuff.

Software by nature is an ill-structured problem. An ill-structured problem has a known starting point, a general idea of where the project is going, but the solution may not be well understood, or there may be several equally acceptable alternative solutions, and of course, it could all change. Forshay discusses strategies for solving ill-structured problems. Star performers are extremely efficient at solving complex problems using general rules, heuristics, techniques, and experience. But, as Forshay states, “…don’t confuse these strategies with the content-free problem-solving strategies advocated in the 1960’s, and still sometimes taught today: research has shown that such general strategies are rarely used by experts.”