Originally published on stickyminds.com

Groove. Groovy. In the groove. The word “groove” has positive connotations. We’re in the groove when we’re focused and productive. When I’m finding lots of bugs in new software and simultaneously learning a lot about the system, I’m in a groove.

However, being “in a rut” has negative connotations. When I’m doing the same thing over and over again, I’m no longer in the groove, I’m in a rut.

So how do you know when a groove becomes a rut, and what can you do about it? It’s a subtle change that slowly sneaks up on you. Comparing activities day to day reveals little difference. Yet the more we wear the groove, the deeper it gets.

Imagine you’re a tester assigned to find bugs in a project. With the first build of the system into test, you’re just getting started. You’re not yet in the groove. By the next build into test, you’ve got a handle on things. You’re beginning to get an idea of where the weak spots are and you’re having fun exploiting them to find serious issues. A few more days into the project, somewhere around build 3, you become a bug-finding machine.

But by build 10 or 11, you’re beginning to dread new builds coming into test because it means rerunning tests to ensure nothing else broke as a result of recent changes. Your energy flags. You begin to wonder if the developers put the bugs there on purpose just to test your bug-finding skills. Within a few more builds you’re spending most of your time running tests you’ve already run. You’re using the same test data as before because it would take too much time and energy to come up with new test data.

Now you’re in a rut. The longer the project goes on, the more deeply entrenched you become. If subsequent releases of the software have only minor changes, you may even extend your rut from one project to the next. I know because I’ve done just that, only to wake up one day and realize that I haven’t done anything new for months.

Being in a rut is a dangerous state for a tester. Given that there are an infinite number of tests we could conceivably run on any given piece of software, we need as much creativity, originality, and energy as we can muster.

Certainly, there are times when we need to rerun tests. However, even when rechecking areas of the software we’ve already tested, we can vary the tests. Varying tests and test data ensures that we’re looking at the same functionality, just in a different way.

Taking the “Groove Test”
When working on a long project, I monitor the depth of my groove to ensure it hasn’t become a rut. Whenever I’ve been working on a project long enough to get in the groove, I begin asking myself these questions at the end of every day:

1. Do I still feel like I’m in a groove?
2. Have I done anything innovative today?
3. How could I update my test data?
4. Can I think of at least three different ways to run each of the tests I just ran?

When I notice internal resistance to change, I can tell that my nice little groove is about to turn into a perilous rut. That’s when I know I need to make changes. Maybe I can get a snapshot of some different production data to test against. Maybe I can add some complexity to my tests. (After all, by now the software should be mature enough to handle harsher tests.) Perhaps it’s time to look at the system from another perspective. Or maybe I can work with another tester to think up some truly devious ways of stressing the system. Now we’re talking!

I have a fifth question I also ask myself:

5. Am I still having fun?

When it stops being fun, I know I’m no longer being as observant as I need to be. That’s when the nasty ones sneak by me. I fail to notice signs that all is not well. I declare a set of features to have “Passed” when I didn’t set any real checkpoints at which the bugs might have been caught.

Failing to spot bugs because we’re too deep in a rut can happen to any of us, whether we pre-plan tests or use an exploratory approach. That’s why two different testers can take the same set of tests with the same version of the software and find vastly different bugs. The difference in the quality of their efforts is in their level of observation. One of my early managers declared that our test group achieved “Quality through Vigilance.” It’s hard to be vigilant if you’re asleep at the keyboard.

So what do you do if you’ve just discovered you’re in a rut?

• Throw away the test documentation for a day, pretending you’re new to the project and the software.
• Analyze the software under test in a different way. If you’ve already done boundary analysis and flowcharting, try looking at the architecture of the system to see the big picture. If you’ve looked at the relationship between the pieces and parts, try designing real-world use cases.
• Change your data. If you’re using test data, get real-world data. If you’re already using a snapshot of production data, get production data from a different system. If you’ve exhausted all your sources for production data, combine the data you have from varying sources into a gigantic set of diverse test data.
• Change your timing. Time is an oft-forgotten dimension of testing. Try doing the same thing but at a different pace or with a different rhythm.

So take the five-question “Groove Test” to gauge your ability to see new possibilities and act on them. Whenever you start resisting the idea that your tests might need to change, it’s time to change them!

published in STQE Magazine, November/December 2001
Perhaps you’ve just changed careers and are looking for a leg up in your new chosen field. Perhaps you’re an old pro wondering how to take your career to the next level. No matter how long you’ve been doing what you do, it’s always good to have someone by your side to help move your career forward: a mentor.  See article (pdf).

Originally published on stickyminds.com

Some time ago, I was acting as a QA manager in a troubled organization. We were experiencing quality problems: the software was shipping with more defects than we wanted and was becoming less rather than more stable with time. I felt quite certain I knew what the problem was. “We don’t have any coding standards,” I fumed. “We don’t have any guidelines to help developers make good decisions about the tradeoff between improving performance and bullet-proofing. We don’t have any documentation on how to use our own APIs” (Application Programming Interface).

I took the opportunity to bend the ear of a friend in the industry, Brian Lawrence, over a friendly lunch one day. He cocked his eyebrows at me quizzically (a Brian Lawrence trademark) and asked a simple question: “Are you sure that’s the problem?”

At first, I was defensive. “Of course that’s the problem!” I grumbled. But Brian was right. The lack of coding standards was a problem, but it wasn’t The Problem. There were a number of reasons for our quality problems. The most important of those problems was a lack of requirements and design analysis, rather than a lack of coding standards.

It’s Not the Problem?
I often fall into the trap of thinking that the first problem I see must be The Problem that needs to be solved. Perhaps the problem I spotted is indeed worth correcting, but I almost never manage to spot the true critical issue at first glance.

At one company, I thought that The Problem was a lack of unit tests. I started beating the drum for better unit tests. Some of the developers agreed with me and encouraged me. Others politely ignored me until I went away. Still others rolled their eyes whenever I started my “Unit Testing Is Your Friend” stump speech.

Then I attended a meeting in which we were discussing bugs in a particular area of the code. The developer responsible commented, “Oh, good. I was wondering when you guys were going to start finding those bugs.” I was shocked. The developer already knew about the bugs. He didn’t need better unit tests; he needed time to fix the bugs. He knew he wouldn’t get that time until the test group found the bugs themselves.

I had found a problem: in some cases the unit tests were insufficient. However, that wasn’t The Problem. In this case, the deeper problem was that developers didn’t get time to fix the bugs they found. As far as management was concerned, until a tester found it, it didn’t exist.

No, It’s a Symptom
In a meeting toward the end of another project, a developer began talking about some changes he’d made to the way a particular feature worked. “Wait,” I interrupted. “I thought we were in code lockdown phase. Now you’re telling me that you’ve added functionality. What’s going on here?” Three weeks to ship and we were still changing the way the software worked. “Poor change control will sink us!” I thought.

However, it turns out that the changes the developer was making were part of the original plan. The problem wasn’t feature creep or poor change control, it was that the schedule said we were supposed to be done with the feature, so development claimed to be “done” while still implementing functionality outside the process. What we really needed was a more honest schedule, not a better change control process.

My Problem, Your Problem, The Problem
On yet another project, we were in code lockdown phase. Only Sharon, the development manager, was supposed to have access to check in code, thus forcing all changes to go through her. Because we were very late in the process, Sharon was to review all changes before checking them in—enforced code inspection.

I heard a rumor that one of the developers was checking in code on his own, without going through Sharon. That developer’s area was also currently the most bug-ridden. “Aha!” I thought. “Here’s the problem!” “Sharon, I heard that Bob is checking in changes without going through you. Is that true?” I asked in a meeting. Sharon looked at me, then looked away guiltily. “Well, I’m pretty overloaded and I don’t understand what that code does anyway. So I gave Bob permission to check in his stuff without going through me.”

I’d found a problem all right. Bob’s code was so complex that the manager of the department, a fine developer in her own right, didn’t understand it. The problem wasn’t unmonitored changes; it was the complexity of the design.

Solve Symptoms; Keep Looking for Problems
In each of these stories, I really had found a problem. However, I hadn’t found the real problem. I’d found symptoms. So if problems hide behind problems, how can you spot The Problem?

1. Open up your mind to other possibilities. Ask yourself, “What if the problem I see didn’t exist? What other problems might be at work here?”
2. Talk with others about what you’re seeing and hearing that’s telling you there’s a problem. Listen carefully to their responses. Often people who are not part of the problem have the greatest insight into what might be going on.
3. If the problem you noticed is simple enough to address, fix it and see what else crops up.

You might even find out that The Problem is a lack of definition of the problem(s).  For more information on defining the problem, see Are Your Lights On? by Gerald M. Weinberg and Donald C. Gause.