Well said. As such, I wish we could move on, and stop having Agile frame every conversation about software development. It feels like it is holding us back at this point.

[Bj's Reply]Hi Chris, I think the Agile agenda has done great work in the sense of reviving some industry ‘best practices’ such as better upfront design before coding via TDD, improved unit testing to better enable refactoring, teamwork versus dev/test animosity, and of course customer focus.

But, I also agree with you that there are some pundits out there who have made a religion out of it. So, while Agile has some pretty solid principles…it is also not a religion.

Regardless, I was at Microsoft and was in charge of the nightly build process you describe in your post. There are several ways agile differs from what you describe, which you rightly call the v-model.

First, the test cases you describe were in my group almost to a test written after the code was authored as a way to validate the work. TDD dictates writing the code to make the test pass, not writing a test to see if the code works. The distinction was generally “fudged” by only counting it if the code was checked into the source tree.

Second, there was still a lot of energy and effort expended on generating documentation up front. Developers would require detailed requirement specifications, the acceptance of which was the entrance criteria for the design phase. Testers would require detailed design and technical specifications, the acceptance of which was the entrance criteria for the testing phase. Everyone wanted to review the detailed testing specifications, the acceptance of which was the entrance criteria for the actual testing work to commence.

Third, the builds were “stand alone” as you rightly pointed out but they were often amalgamations of half finished feature sets with no way to easily install or uninstall the compiled code. The bar for “stand alone” was frightfully small and usually only meant it could be compiled successfully and somehow place on a machine where testing could be performed. Even later in the product lifecycle when minimum marketable features were available, the “working product” model was usually only enforced later in the milestone. Early milestone builds were often treated as throwaways. I know this from personal experience because a large portion of my time spent managing these builds was spent figuring out ways to completely wipe an entire drive space, install a brand new OS, and then somehow get the required bits onto the machine for the white box tests to run. We had to do this because there were no installers, no way to safely uninstall, and often no GUI to monitor.

You’re correct that agile is a philosophy or process more than a methodology, but like any philosophy there are key principles with which you must at least tacitly adhere or else you really aren’t a follower so much as making the claim.

[Bj's Reply]Hi Curtis, I also can’t quite fathom the argument given by some people in the industry in opposition to testers having a richer technical depth as well as a overall systems view. Over the last few years I have promoted getting back to our roots so to speak (back to “traditional” software testing approaches) only to be met with ridicule and personal attacks. It seems that convincing some people in testing that they should understand something about the system’s they are working on at a deeper level seems more akin to a religious war, but it is good controversial blog post fodder.

Good point about TDD. Even today testers write tests after the implementation. But, TDD (which I believe is revived from the earlier “Test, then code” mantra) is about developers writing unit tests prior to writing code. Personally I think this is a great idea because it forces developers to really think through the design before cobbling together code.

Funny how different groups at MS can be. The only documentation I saw on Windows 95 was the original ‘Chicago’ vision, and on IE 4 we had a few 1 page feature specs. But, I get your point. I still know groups that require devs and testers to produce lengthy documents, get a ’sign off,’ prop it on a server to bit rot. With builds again I guess I was lucky working on legacy systems where the setup worked (most of the time).

I am not suggesting that Microsoft is Agile (although some might), I am saying that I can see many Agile tenets in the groups I worked in. At the same time, I can also see practices from other models as well.

This is great, and has been my experience as well. I am relatively new to the industry, and learning to test in an agile has both encouraged me to change my processes, and also to see similarities to other methods.

I am glad you brought up the W- model, because waterfall is not the only thing out there. Great points.

[Bj's Reply]Hi Devon, I am sometimes surprised by people who seem to only view the world from opposite ends of a spectrum. But, of course, I still see people on forums looking for assembly line solutions, which is why I embrace (most) Agile principles as sage wisdom by experienced professionals.

As of now, I look at exploratory testing as the option to do acceptance testing in a customer – contractor -type of setting. What seems to happen in this area, is that without emphasis on exploratory testing, we look at the requirements, and work on ignoring a lot of the things that are relevant for use and quality.

If you have 30 days for acceptance testing a delivery, and you have the option to either predescribe with detailed plans the questions you’d be asking within your limited time or have a structure in which it’s easy to change your mind as the picture of what state the software is in evolves, I’d go for the latter.

[Bj's Reply] Hi Maaret, acceptance testing of a delivery (esp. in the customer role) is typically based on contractual obligations (e.g. does the deliverable satisfy specified standards). In the customer/contractor acceptance testing scenario the customer typically runs tests to determine whether to ‘accept’ the deliverable or reject it because it doesn’t meet specified criteria.

Yes, in Agile there is usually heavy emphasis on pair programming, [extensive] unit testing, TDD, etc. The fundamental idea is to essentially produce better quality code that is more thoroughly tested upfront and rely less on a iterative type cycle where developers hack together code and testers whack on daily builds to find as many bugs as possible, repeat and rinse until some marketing date comes along. Ideally (assuming robust unit testing as suggested by Robert Martin in Clean Code) many of the ‘important’ or ‘critical’ functional issues would be found earlier, and exploratory tesing would be valuable in helping identify behavioral issues and some overlooked logic problems.

[Bj's Reply]Great list of ideas Rikard.