I.M. Testy

You need testers because…

with 9 comments

9 comments on “You need testers because…”

1. Pingback: OpenQuality.ru | Качество программного обеспечения

2. Pingback: Bugs That Automated Tests Aren’t Good at Finding

3. For me the important thing to add here is that you want this level of testing expertise *inside* your development team.

   The problems stem from where the dev team is not properly cross functional. People fall into the trap of thinking “It’s a dev team, therefore it should be made up of pure techies”. Maybe to these techies a mixed case state listing is fine – they’re good coders and use case insensitive comparisons everywhere…

   I would hate to see this article used to justify a separate stage to the development process. It sounds attractive to pass developed code over to a dedicated testing team who will try and find bugs, and report them back to the developers to fix later. But what you get there is a reliance on manual procedures, unecessary hand-off delays, and the bugs are still discovered too late – just this time your release is delayed rather than timely but buggy.

   Separation of testing from development causes qualtiy problems that feed back on themselves – a tendency to build a QA empire and not notice that this is reducing your efficiency at developing working software. Testing – unit, integration, sceanrio, regression, usability, percieved quality, whatever – is /all/ part of development.

   [Bj's Reply] Hi Andy, I mostly agree with you. Testing is/should be part of the development process. But, as Joe noted testing software and developing software are 2 separate roles with distinct perspectives, and these roles should coexist in symbiotic relationship to produce a higher quality product.

   I would also say that if your perception of testing is “a reliance on manual procedures, unecessary hand-off delays, and the bugs are still discovered too late” then the test processes you are familiar with are probably antiquated, or the testers you’ve worked with may not have had adequate skills beyond manual testing to work along side the dev team.

   Also, there is a difference between a buggy release and release that is completely unusable by some segment of your potential customers (e.g. those whose credit cards are Jan, Feb, Mar). Testing in Production (TiP) by testers (or developers) would have easily found this (among the many) issue.

   So, I would not promote a separate distinct testing stage where bits are thrown over the wall as some unattached test team. But, I do promote a team of dedicated testers that work alongside the development team. The scope of testing is enormous, and by including people on the team whose specific role is focused on various aspects of testing is (in my experience) a better business strategy than relying on testing done by developers only.

   Reply ↓

4. Pingback: The Morning Brew - Chris Alcock » The Morning Brew #812

5. I think the bug on the months combobox above is likely caused by a developer trying to be too helpful, by removing months already passed in the current year. You could probably book in Jan/Feb/March if you first changed the Year field to 2012 (which would be an unnatural flow of input).
   I saw exactly the same bug yesterday (think it was on a Center Parcs website if I remember correctly), So maybe it’s related to a commonly used set of date controls. You’re right though, this kind of thing is exactly the type of bug you don’t find in your own code

   [Bj's Reply] Hi Simon, I would agree with your assessment of the possible cause. Code reuse has potential side effects that we sometimes can’t predict and requires additional testing. There are other examples of bugs we typically don’t find in our own code which is why we will always need testing as a separate activity.

   Reply ↓

6. Reminds me of those day when my team and I would test a product in the middle of night only to find unnecessary, annoying confirmation pop up windows with some text which could only be understood by developer. Same is true with shoddy testers also. I have come across testers who blindly close their eyes even the wake of most horrible defects. I feel it is your ingrained habits which make you a good developer or tester. And a good developer could be a good tester too and vice varsa.

   [Bj's Reply]Hi Tarun, ahhh…I remember those long nights…wait…those long nights are happening again with my new job!

   I agree there are great developers who are awesome at testing, and there are awesome testers who are great at coding. These skills are not necessarily mutually exclusive or contradictory.

   And unfortunately yes, there can be shoddy professional in all trades.

   Reply ↓

7. BJ, when I was a programmer (during the freelance stage), I built a travel reservation web-site. My site didn’t had these bugs… In fact my site didn’t had ANY bug at all! Right?

   Well, no, not actually. I can think today of many bugs I was not aware then…
   As Ron Avitsur says, testing your own code is akin to seeing your own blind-spots.

   [Bj's Reply]I hear ya Shmuel! How dare me to call the kettle black! My own code is fraught with (minor) bugs . Fortunately, nobody is using my code to try to make online reservations.

   This wasn’t so much about developers not introducing bugs into a product (we know and they know it happens). And, I think most developers do try to do their best to at least validate their code to some extent.

   More than anything, from the outside looking in this looks like bad project management.

   Reply ↓

8. Nice, Bj!

   “But, testers have a different mindset as compared to developers, and we have different perspectives.”

   Yes, exactly! When two groups have distinct roles and motivations, they approach a given task differently.

   We need to remember that developers tend to test their code. That’s good! But, pretty much by definition, they are testing *what is*, rather than *what should be*.

   http://strazzere.blogspot.com/2010/05/slideshow-optimistic-developers.html

   [Bj's Reply] Hi Joe, yes…that is exactly why Beizer, Myers, and others strongly supported independent testing.

   However, I will also say that I agree with Bob Martin when he wrote “Write [unit] tests that have the potential to expose problems and then run them frequently…” in his book “Clean Code.” I think Agile teams are finding that their best developers are doing just this to help drive some aspects of quality upstream and reduce downstream regressions. The less capable developers will continue to write unit tests that only check for their idea of “correctness” and continue to check in code the repeatly breaks the build or drives up production costs.

   I also admire Bob Martin’s quote, “Proviing that code is correct is impractical. Tseting does not guarantee correctness. However, good testing can minimize risk.”

   Reply ↓

9. Hi BJ,

   Do you believe, this bug of “missing” three months in a drop down could not have or could have been found by developers unit test or API test?

   or is just a case of a developer of PM in a hurry to take something “live” and a mistake somehow slipped through the cracks.

   You should probably do a post on types of bugs that unit testing (or developer testing at any level) would’nt catch.

   That would be a fitting reply to all those who swear by automated unit testing/API Testing.

   Shrini

   [Bj's Reply]Hi Shrini, good to hear from you again.
   Since I don’t know the root cause of the bug, then I can’t say exactly at what level of testing or what approach would have worked to find that specific issue. Even if the approach were TiP (Testing In Production) the key element missing was the “testing” part since this site had been live for more than a week, and the problem was still there the day after. (Maybe they have a really slow testing or bug fix process:-)).

   WRT to other bugs I mentioned those certainly could and should have been found pre-post to live services. IMHO it lacks professionalism to release test code/data into production; not to mention potential risks in some cases.

   I am an ardent supporter of unit level testing and API level testing. In fact, my current team is primarily responsible for testing the APIs that manage the “social networking” aspects of the Windows phone. There is a sister team that is primarily responsible for the user experience aspect, but I encourage my team to self-hosting and eat the dog-food as well to make sure we have customer empathy and can better explain the customer impact of bugs we find during API testing.

   But as you know, unit and API tests won’t find behavioral or usability issues. There are some other types of issues that those approaches are ill-suited for. But, there are some types of issues that can be found earlier using these approaches that can add tremendous business value and help reduce production costs (a different kind of quality). But, you have a great suggestion in discussing the types of issues and which testing approaches might be better sutied to help expose them (which is why we need to rely on several different approaches and test software in production at different stages).

   Reply ↓