Logo Elisabeth Hendrickson’s Thoughts on Testing, Agile, and Agile Testing

The WordCount Simulation

November 13th, 2008
Filed under Agile, Running the Business

I’ve mentioned my WordCount simulation here before, and some folks have expressed curiosity about it. I started writing a blog post about it, and quickly realized that it would take a whole lot of blog posts to tell all the stories I want to tell. So I’ll start by explaining the simulation in more detail, and in subsequent blog posts, I’ll tell stories. Some will be amusing little anecdotes. Others will be tragic tales of woe. You’ll laugh, you’ll cry…oh, never mind. Let me just describe the simulation.

So this is the simulation that I use in my Agile Testing class, as well as in other contexts where I want to teach lessons about increasing Agility. The mechanics of the simulation itself are very general: the simulation models the organization of a software company. It just happens to work really well for making Agile concepts very visible, and visceral.

At this point WordCount is a mature simulation: I have honed and refined it over the last several years, and have run it countless times. The simulation requires a lot of moving parts. Just printing the supplies to run the simulation can be a chore. It involves 26 files whose contents are printed on 5 different colors of index cards, 5 different colors of paper and cardstock, and 2 different sizes of stickers. Fortunately for my sanity, I now outsource most of the printing work to a local printer. (Though I swear I can hear the manager groaning when he sees me walk in the door.)

All those moving parts support a relatively simple organization: WordCount, Inc. is a fictional company that makes word counting software. Each participant chooses a role within the company from the following descriptions:

The Product Managers interact with the Customer (played by me or a co-instructor/co-facilitator), define the product, and write requirements on blue index cards.

The Developers turn the requirements from the Product Managers into executable instructions (“code”) on green index cards. The code is then installed on the Computer.

The Testers design test cases on yellow index cards and execute them against the code installed on the Computer by submitting input on white index cards, and receiving output, also on white index cards. Of course, not all input can be processed. If the Computer cannot process the input, it throws a catastrophic error on a purple index card. This usually prompts the Testers to write a bug report on a red index card.

The Computer follows the instructions that the Developers wrote on the green index cards (the “code”) to take the input on the white index cards and process it to generate output, either word counts on a white index card or an error on a purple catastrophic error card. Multiple participants can play the role of the Computer, but the Computer team must coordinate their efforts internally. As you can imagine, sometimes different participants playing the role of Computer interpret the code differently, and give different output for the same input. This inevitably leads to tremendous confusion among the Developers and Testers.

The lone Interoffice Mail Courier (there can be only one) delivers messages and Project Artifacts between groups.

The Observers watch the interactions and share their observations with the group during debriefs. They’re not really part of the company, but they are active participants in the simulation. I usually hand one of the Observers a camera so we have photographic evidence that we can reflect on later as we distill lessons learned from the simulation to apply in the real world.

Throughout the simulation, we work in rounds. So we work on the simulation for 15 minutes, then pause to hold a mini-retrospective in which we reflect and adapt the process to increase Agility.

When we start, there is an existing process in place that resembles a very traditional organization with groups working in silos and very constrained communication. The process explicitly defines responsibilities and communication paths by specifying things like “Communication between Developers, Testers, and Product Managers occurs only through Interoffice Mail,” and “Only Developers may create or modify Code.”

Each group starts with an initial set of artifacts. So the Product Managers have notes from their fictional predecessor’s conversations with the customer, the Developers and Testers have an initial set of requirements, the Developers and the Computer have version 0 of the code, and the Testers have an existing set of tests.

As we begin the simulation, I explain that: as the Customer I desperately need their word counting software to run my business; I have been promised a delivery of the basic word counting system Real Soon Now; so far nothing has been delivered to me. I also make it clear that the more features they ship, the more money they make, and that their goal should be to maximize revenue.

We then work for 15 minutes.

In a typical first round, Product Managers work furiously to produce requirements on blue cards while Developers frantically write code on green cards and Testers crank out test cases on yellow cards. Pens go flying as participants scribble madly on colored 3×5 index cards.

The Interoffice Mail Courier stands by, ready to deliver messages, but typically only has to deliver a handful. One Interoffice Mail Courier was so concerned about the potential volume of mail that he recruited an assistant. They both stood idle for most of the first round, bored. Similarly, the Computer is also usually idle for the first round, and often starts throwing errors just to amuse itself. The Product Managers, Developers, and Testers are so busy producing artifacts that they don’t have time to communicate or even execute the software that already exists.

When we debrief the first round, the Product Managers, Developers, and Testers often report feeling stressed, pressured, and frustrated while the Interoffice Mail Courier and Computers report being under-utilized, idle, and bored.

But all that stress and self-imposed pressure doesn’t yield success. As of today, no group has ever made money in the first round. No group has even come close. In fact, out of all the times I have run this simulation, only one group has ever managed to demonstrate the software to the customer during the first round.

And yet some managers look at the efforts of the Product Managers and Developers and Testers in the first round and see focused productive work happening. In a couple of cases, managers have walked up to me during the first round and said, “Look at how focused each of the teams is! And notice how quiet it is in here! Everyone is working so hard! This is great!” One of the managers was kidding: she knew that the lack of communication would be a problem. But the other manager? She was not kidding. She was dead serious. That’s what she thought productive teams looked like.

As you can probably imagine, I have lots of stories about this simulation. There’s the story of the Hero who single-handedly almost caused the failure of the whole company. There’s the story of the group that actually did fail entirely, and then blamed me for their failure, as though I had played some trick on them. There’s the story of the Micromanaging Manager. There’s a story about a Project Manager who quit in disgust after 15 minutes. And there’s a story about a Product Manager Who Always Said No.

There are stories about Developers arguing with the Computer about the correct interpretation of the code. And there are numerous stories of Product Managers who got so wrapped up writing requirements they forgot to talk to the customer, and Developers who wrote Code that was so complex even they had no idea how to interpret it, and Testers who spent so much time documenting test cases that they forgot to execute them, and myriad other forms of organizational dysfunction that just happen to look a whole lot like real life. That’s why simulations are so much fun: they give us the opportunity to experience all the complexities of the real world distilled into a microcosm.

Along the way I’ve learned numerous lessons like “Never Use a Communication Solution to Solve a Visibility Problem,” and “When Someone Says They Want Control, They Might Just Need Visibility,” and “If the Customer Offers You an Example, Take It.” So I’ll tell stories about those lessons too.

But all those stories will have to wait for another day. This entry is long enough as it is.

[ANN] A Public Offering of my Agile Testing Class

October 30th, 2008
Filed under Agile, Announcements, Running the Business

And now for a blatant commercial announcement: I’m hosting a public offering of my Agile Testing class with Dale Emery on December 10 and 11 in Pleasanton, CA.

I predict that this is going to be a great class.

First, it features my WordCount simulation, and that’s always fun. I’ve run that simulation over 100 times and I learn something new every time. If you’re looking for a way to get people in your organization to understand, viscerally, why silos and Agile do not mix, this simulation is the best way I’ve discovered so far. It’s also a great way to see the impact that an integrated value-focused test effort that yields fast, visible feedback can have on Agility. (Note to self: I should blog more about lessons I’ve learned from that simulation.)

Second, the class features my Acceptance Test Driven Development (ATDD) demo. ATDD is a powerful way to involve testers early and make testing part of the definition of “Done.” And it’s also a powerful way to integrate test automation efforts into the development cycle instead of succumbing to test-last automation.

Third, Dale will be there. Dale and I make a great team.

When I ran a 1-day variation of this class at the STAREast and STARWest conferences as the “Adapting to Agile” tutorial, it sold out. I am (of course) hoping that this class will do the same.

I have had some people ask if I will hold future public offerings of this class, or other classes. The answer is yes - but only if I can fill this class. So if you want to see me offer more public classes, please come to this class and/or pass on the word to your friends and colleagues. Please help me fill this class…

Unintentionally Hard Tests

July 7th, 2008
Filed under Running the Business, Thinking Like a Tester

I finally decided that too many of the things on my To Do list are things that I probably should not be handling personally, like dealing with getting business cards and routine invoicing. And too many of the things that I should be handling personally, and quickly, are being left undone too long.

So I posted an ad for a part time assistant on Craig’s List. And being test obsessed, I devised a test to give me a good indication of whether a given candidate had enough of the skills I needed to bring in for an in-person interview.

I asked the candidates who wrote an articulate reply to the job posting (and there were a lot of them!) to do a little research and find out what conference I’m speaking at in August, where the conference is, and what the titles of my presentations are.

It seemed to me like this was a pretty easy test that would weed out anyone who couldn’t do basic research on the web. But I made the test harder than I intended by asking about conferences in August. I forgot that there are two conferences listed on my web site for August: Agile2008 (where I am not speaking), and STANZ (where I am giving two talks).

When I realized my mistake, I thought “No problem. Good candidates will search for my name in the Agile2008 program and realize I am not speaking there.” Then I tried it myself and discovered that searching for a specific speaker is apparently a use case that the Agile conference organizers didn’t think about when they published the program this year. Even I found it difficult to determine absolutely that I am not on the program. There are too many places to look, and even the PDF program is a little difficult to search for a given speaker’s name.

Sometimes what seems like a simple test - whether of software or of a person - turns out to be much more difficult than we intended or imagined. Although I suppose that as long as none of the job candidates crash as readily as software, we’ll all be OK.

And if any of them manage the task without giving up, I will definitely know something about their ability to find information on the web!