Checking Alignment, Redux

I’ve been writing a lot lately. Writing for long stretches leaves me mentally drained, nearly useless. The words dry up. I stop making sense. I find it increasingly difficult to form coherent sentences that concisely convey my meaning. Eventually I can’t even talk intelligibly.

I recall attending a party after a week of solid writing a few years ago.

“How are you?” my host asked when I arrived.

“Unh.” I muttered. “Good.”

“What have you been up to?” she inquired.

“Um. Writing.” I stopped talking and stared back at her expectantly.

I wanted to be social, but no more words would come. I stood there just staring at her. It didn’t even occur to me to ask how she was doing or what she was up to.

My host looked at me sideways, unsure how to respond to my blank stare. It wasn’t a Halloween party, and yet I was doing a passable impression of a zombie. How does one respond to zombified guests?

Anyway, my point is that I’m in one of those states now. And thus I may have great difficulty making myself understood. Producing words that fit together to express ideas is becoming increasingly difficult.

I’m guessing this is why I failed to explain myself well in my last post. Or at least I am inferring from the response to that last post that there is a gap between what I intended to say and what most people understood me to be saying.

I had three points that I wanted to make in my last post:

  1. It’s easy to speculate about the connection between actual needs, intentions, and implementation.
  2. Empirical evidence trumps speculation. Every single time.
  3. Testers are NOT the only people who gather that empirical evidence.

Given that’s what I meant to say, I certainly didn’t expect UTest, a testing services company, to like the post so much that they would tweet:

We couldn’t agree more! It’s all about the testing!

Yes, it is all about the testing. But—and this is a crucial BUT—it is not all about the testers.

In fact, much of the kind of testing that goes into ensuring alignment between intentions/implementation and actual need is something that testers have very little to do with, and it’s something that cannot ever be outsourced to a testing services company.

Let’s look at the sides of the triangle of alignment again:

Actual Need: the value our users and/or customers want.

Intentions: the solution we intend to deliver in order to serve the Actual Need. The product owner, product manager, business analyst, or designer is the one who typically sets the intentions. It’s their job to listen to the cacophony of conflicting requests and demands and suggestions in order to distill a clear product vision. For now let’s just call this person the product owner. They own the product vision and decide what gets built.

Implementation: the solution the team actually delivers.

So who makes sure that the intentions and implementation match the actual needs?

The best person to do this is usually the person who set the intentions in the first place: the product owner. They’re supposed to be steering the project.

If the product owner has no way of verifying that they asked for the right thing and can’t tell whether or not the resulting software delivers the expected value, the project is doomed.

Seriously, I’ve lived through this as a team member and also seen it from the sidelines. The person responsible for setting the intentions needs a way to tell whether the actual needs are being met. They need feedback on the extent to which the intentions they set for the team pointed us in the right direction. Otherwise we end up in a painful cycle of requirements churn that can ultimately end in organizational implosion if we hit the end of the runway before we deliver real value.

Michael Bolton’s story of getting out of the building and picking up sample checks on his lunch hour is fabulous. But to me, it’s not a story about testing. Rather it’s a great story about how having multiple examples are key to truly understanding requirements.

Further, I’ll suggest that in this story Michael was acting as a Team Member rather than a Tester. The fact that Michael is a world class tester is not the most salient part of the story. The important thing is that he noticed the team needed something and he went out of his way to get it.

It is important not to confuse Michael’s initiative as a team member with an exclusive job responsibility of testers. Michael took the initiative. That’s one of the reasons why he is a world class tester. But picking up that sample check is something that a programmer could have done. Or the product owner. Everyone on a project can contribute to establishing a shared understanding of the full scope of the requirements. And everyone has a hand in gathering empirical evidence, not just testers.

Testers happen to be really good at gathering information. Teams need testers. But teams also need the testing mindset to be baked into the culture. Team members need to ask these key questions before taking action:

  • How will I know my efforts had the effect I intended?
  • How will I know my intentions were correct?
  • How will I know my results are delivering real value?

These questions are at the core of the test-first mindset. And the answer to these questions is never, “I’ll just ask the testers.”

Checking Alignment

Let’s start at the beginning. Somebody, somewhere, needs some software.

Maybe we’re serving an internal “customer” who needs a simple bailing-wire-and-duct-tape app to connect system A with completely unrelated (except that they need to be able to share data) system B. Or maybe we’re in a startup that’s trying to Change the World with a grand vision, or perhaps a modest vision, to give people software that makes their lives better.

Either way, we build software because there are people who need it. Let’s call what users need the Actual Need. We want to serve that Actual Need by building a truly kick butt solution.

On Agile teams we use user stories in an attempt to capture actual needs. For example:

As a Banking Customer I want to use my ATM card to withdraw money from an automated banking machine while I’m in Kiev so that I can buy a cup of fabulous local coffee with the local currency, Hryvnia.

This is way better than “The system shall accept a rectangular piece of plastic made to conform with standard …” It humanizes the problem space and puts the user at the forefront.

But user stories aren’t typically written by real users. They’re written by surrogates: Business Analysts or Product Managers or the like. These internal people go out and find the Actual Need and then set Intentions for what we need to build to meet those needs.

And then the Software Development team brings the intentions to life with the Implementation.

That’s software development in a nutshell from gathering requirements through deployment. It’s all about finding the happy place where we are addressing real needs with awesome solutions.

So here’s the big question:

How do we know that our Intentions matched the Actual Need, that the Implementation matched our Intentions, and ultimately that the Implementation matched the Actual Need?

Three Sides of Alignment

If software development projects exhibited mathematical properties then we could count on the relationship between each of these three things being symmetrical. That is, if A = B, and B = C, then mathematically speaking, A must also equal C.

But that doesn’t work with software.

We can set Intentions that, if implemented, would match the Actual Need. And we can communicate those Intentions effectively so that we end up with an Implementation that does what we intended it to do. But that does not mean that users won’t experience any problems with the delivered solution.

As an aside, this is fundamentally why waterfall does not work. Even if we could build the perfect requirements document that perfectly captured the actual needs of the business or our user base, there is no way to ensure that the resulting implementation will be a success. And by the time we release it’s way too late.

Back to my assertion that alignment is not symmetrical in software systems. Consider my story of trying to get Hryvnias in Kiev.

So there I was in Kiev with only a few Hryvnia in my pocket. I needed cash. So I took my trusty ATM card to an AutoBank machine. Actually, I walked by any number of AutoBank machines looking for one that had the right symbols on the front so I could be sure my card would work, and that was built into the wall of a bank so that it wasn’t a fake ATM machines designed to skim info. Yes, I can be paranoid sometimes. So anyway, I think I marched Sarah halfway across the city before I found one I would stick my card into.

Having finally found what I deemed to be a trustworthy ATM, I put in my card and entered my pin. And then I got a scary looking message: “You entered your PIN number incorrectly 3 times.” And the machine ate my card.

Here we see an example of how alignment is not symmetric. The Implementation of the ATM software no doubt matched the Intentions of those who built it. ATM machines are a mature and robust technology after all. And the Intentions addressed the Actual Need. Again, ATM machines are a known quantity at this point. But the Implementation spat in the face of my Actual Need. I was thwarted. Not only was my need for cash not met, but now my card was gone. And I had not entered my PIN number 3 times; I just entered it once. (My guess is that the real issue had to do with whether or not the ATM machine supported my card and not with the actual PIN number.)

But I digress. Back to the point.

We have Actual Need, Intentions, and Implementation. How do we know that all three of these things are in alignment?

The product owner can speculate that the Intentions accurately describe the Actual Need. If the product owner is hands off we often see development teams unilaterally asserting that the Implementation matches the Intentions. Worse, some product owners remain hands off because they want plausible deniability when things go wrong. That’s just…ew. Throughout all this, the business stakeholders can assume that by doing what we set out to do, the Implementation will meet the Actual Need and we’ll all be rich.

And we will be fooling ourselves. Such guesses and speculation allow us to become wrapped up in the illusion of progress.

If we want to know whether our Intentions are in alignment with the Actual Need, Steve Blank would say that we have to get out of our cubes and talk to potential users or customers.

If we want be sure our Implementation matches our Intentions, we have to state those Intentions concretely, with examples and explicit expectations. As long as we’re doing that we might as well go whole hog and do ATDD. It’s the best way I know to drive out ambiguity, clarify assumptions, and provide an automated safety net to alert us any time the Implementation strays from the Intentions. But automated checks aren’t enough. We also have to explore to discover risks and vulnerabilities that would jeopardize the spirit of the Intentions even if not the letter.

Finally, if we want to be sure our Implementation matches the Actual Need we have to watch customer behavior carefully. That means monitoring usage statistics, watching conversions, and generally listening to what the Lean Startup guys have to say on validated learnings.

All of these activities are aspects of testing. And while testers are still important, not everything that involves some aspect of testing should be done by people with QA or Test in their title.

Too often software teams take a narrow view of “testing.” They think (to paraphrase Stuart Taylor) that it’s about checking or breaking. They relegate it to the people with “QA” or “Test” in their title. Typically we only test whether the Implementation meets the Intentions. We speculate about the rest.

And then we’re surprised by failures in production, angry customers, and declining revenue.

The harsh fact is that empirical evidence trumps speculation. Every. Single. Time. And testing, real testing, is all about getting that empirical evidence. It’s not something testers can do alone. There are too many kinds of testing involved in ensuring that all three things are in alignment: Actual Need, Intentions, and Implementation.

And ultimately that’s why testing is a whole team responsibility.