Logo Elisabeth Hendrickson’s Thoughts on Testing, Agile, and Agile Testing

AYE Conference Next Week!

October 31st, 2007
Filed under Ruminations

The AYE Conference is next week and I’m very, very excited about it! I will be there, doing several sessions. Dave Smith and I are doing a session on Renegotiating where participants will be playing the “Renegotiate” game - a game that Dave and I created. I will also be running my WordCount simulation in my session titled Reflect and Adapt. And I’ll be running a fun session on using our worst nightmares to improve our testing, titled “What Could Possibly Go Wrong?”

And I’m really excited to participate in the sessions that other presenters are doing. Dale Emery will be doing sessions on Resistance as a Resource and Putting Your Power to Work. Diana Larsen (co-author with Esther Derby, of Agile Retrospectives) will be doing a session on Cultivating Trust in Teams.

The hosts for the conference are all doing sessions as well. And that includes Esther Derby, Johanna Rothman (co-author, also with Esther, of Behind Closed Doors: Secrets of Great Management), and also Jerry Weinberg, industry icon and early pioneer, and author of many books including The Psychology of Computer Programming, Becoming a Technical Leader, and Secrets of Consulting.

It’s designed to be a small conference, and the organizers cap participation at 99 people. That’s why I was surprised to learn that they still have spaces left. If you aren’t doing anything next week, and you feel like coming to Phoenix, AZ, you should come join us. It’s a tremendous opportunity to spend 3 days of intense interaction with some amazingly cool people.

From the Mailbox: TDD and Automated Testing

October 18th, 2007
Filed under Ruminations

A reader asked: “I am hoping that you can answer my question to clarify something for me and another co-worker. Could you tell me the difference between TDD and Automated Testing? He says that TDD is for unit testing and Automated Testing is for System test. He also says Scrum doesn’t include anything about TDD so we don’t have to do it.”

Let’s start with definitions.

TDD stands for Test Driven Development, a specific technique introduced with Extreme Programming (XP). TDD tests are unit and sometimes code-level integration tests. But TDD itself is a development approach rather than a testing approach. However, a key result of doing TDD is that you end up with a fully automated set of code-facing tests. The Wikipedia article on TDD provides a good overview.

So the characterization of TDD being about unit testing is partly correct, but it is a misleading oversimplification.

Now, about automated testing. An automated test is one that can be executed by the computer rather than by a human. The term applies to any kind of automated test, and is not limited to system tests.

However, the distinction between developer tests and acceptance or system tests is useful. Developer tests are code-facing, and a result of TDD. Acceptance tests are business-facing, and on Agile projects, their automation is the result of a collaboration between the business stakeholders and the developers. Both are important. One of the mistakes that Agile teams tend to make is attempting to substitute unit tests for acceptance tests, or vice versa.

Now back to the assertion that “Scrum doesn’t include anything about TDD so we don’t have to do it.” It’s true that in Agile Software Development with Scrum, Ken Schwaber and Mike Beedle say: “During a Scrum, only the team can define its work.”

But that doesn’t mean it’s responsible to say: “Scrum doesn’t say we have to do it. So we don’t have to if we don’t wanna. Neener neener neener thbbbbpppt.”

Scrum also says that “Although a team has the authority to decide how to do its work, the team is responsible for using and conforming to any existing charters, standards, conventions, architectures, and technology. These ensure that the products of the project and the Scrum Team fit in with other organizational products and can be understood by others.” (See Agile Development with Scrum, pages 38 - 39)

Furthermore, as the Agile Project Leadership Network’s Declaration of Interdependence states, being Agile means delivering a continuous flow of value. To achieve that continuous flow of value, we have to use development practices that provide the visibility and feedback needed to make frequent, incremental changes.

Put another way, Mary Poppendieck says that speed requires discipline. I extend that to agility in general: if you want your team to be Agile, it must also be disciplined. Agile doesn’t mean “do whatever feels good.”

That need for disciplined development practices is why many teams have found that embedding XP-like practices (TDD, continuous integration, collective code ownership, pairing and/or frequent code walkthroughs, etc.) within Scrum is an incredibly powerful combination.

The bottom line: automated testing at all levels can be difficult, but the alternatives are worse. Without automated testing, both code-facing and business-facing, the manual testing burden becomes too large, the team begins incurring technical debt, and velocity slows.

So I don’t buy the argument that “Scrum doesn’t say we have to…” I think the real question is not what Scrum says or doesn’t say, but rather what practices are needed to ensure continued Agility.

Oh, the irony…

October 16th, 2007
Filed under Lessons Learned, Ruminations, Running the Business

So I’m working on course materials for a class I’ll be teaching next week on Acceptance Test Driven Development.

I decided to try using Keynote on my trusty no-longer-all-that-new Mac instead of PowerPoint. I find PowerPoint on the Mac incredibly annoying. Whenever I want to edit a presentation on the Mac that I created on the PC (like, all of the presentations I have), I get Converting Metadata messages all over the place. When editing a 300 slide file, making even the smallest change can take forever. So, Keynote seemed like a better alternative.

Now I should explain that although most of my slide decks look reasonably simple, I tend to use a lot of animations for certain types of things - like bringing a workflow to life.

And I also use the presenter notes area as a place to put the meat of the materials for participants so that when I print books with both the slide and the presenter notes, there’s a good amount of text. The result are course notes that read kind of like a book with a whole lot of illustrations.So in other words, I am not your typical lightweight presentation software user. And I’ve crashed PowerPoint a lot as a result.

But I had high hopes for Keynote. For starters, it’s Mac software. Built for Macs. None of this ported over from Windows crap.

Alas, I was doomed to disappointment.

First, when I attempted to do some fiddly edits on the text under a page with a ton of animations, Keynote died a horrible death. It didn’t exactly freeze entirely, but I could no longer edit the content. I could attempt to save/export my file, but every time I did - no matter what format - it told me that the file was corrupt. Sadly, I’ve gotten out of the habit of saving every 30 seconds, so I lost a fair amount of work.

When I finally recovered everything an hour later, I vowed not to make the same mistake again. I not only started saving rabidly every 30 seconds, but I also checked everything into my subversion repository. However, when a subsequent commit failed with the message “Directory …/.svn containing working copy admin area is missing” I learned that there’s an interaction bug between Keynote and Subversion. I used the workaround kindly published by Daniel Sadilek (bless you, dude). And I was back in business. Annoyed, but back in business.

But here’s the thing. I’m already annoyed, and now I’m looking at my .key file for a 27 slide file, and it’s 3.8Mb. What??? I know I’m abusive to presentation software. I understand that my usage is a tad on the abnormal side. But 3.8Mb for 27 slides? Slides with graphic drawings and text, no photos, no movies, no sounds? That’s nuts. With a little experimentation I discovered that more than half that is due to two slides that make extensive use of animations. And I’m betting that those same animations had something to do with Keynote dying a horrible death when I tweaked the text underneath. Someday when I have more time I’ll see if I can reproduce the bug.

Let’s just say I’m re-evaluating my decision to use Keynote. PowerPoint is looking pretty good right now. Sigh.

And this is also a lesson re-learned about software testing: there’s nothing like a real user to find real problems. For Agile projects, that means automated unit and acceptance tests are necessary but not sufficient. Make sure real humans try the software under real world conditions. Given the topic of the course materials I was working on, this was a timely reminder…though a far more painful one than I really wanted.