From the Mailbox: Sources for Metrics?

A question from my email archives:

“I’ve been tasked with defining metrics for software quality. Do you have a recommended book or reference to point me in the right direction?”

The classic book on software metrics is Kan’s Metrics and Models in Software Quality Engineering. It’s not for the casual reader, however.

More approachable is Putnam and Myer’s more recently published Five Core Metrics: The Intelligence Behind Successful Software Management.

I also suggest reading Cem Kaner’s work on metrics. Kaner points out that most of measures we take on software projects don’t actually measure what we think we’re measuring. His paper “Software Engineering Metrics: What Do TheyMeasure and How Do We Know?” is a good summary. On a related note, Doug Hoffman provides examples of how measurement can be fraught with peril in his paper “The Darker Side of Metrics.”

Since the variations of possible metrics you could take are almost infinite, I recommend Vic Basili’s Goal-Question-Metric approach to ensure you choose the metrics that are best suited to your particular context.

For using metrics to communicate, I suggest my own article (co-authored with Kathy Iberle), “Show and Tell.”

We were heavily influenced by Tufte’s Visual Display of Quantitative Information. While not specifically about software, I highly recommend his book. I also recommend the classic How to Lie with Statistics by Huff.

Easily Distracted

Jan Chong is a doctoral candidate at Stanford, studying Organizational Behavior. Jan has two CS degrees, so it’s natural that she would combines her interest in software development and organizational behavior by studying XP teams. She presented some of her preliminary findings at Agile 2005.

One of the things that Jan observed while doing her research is that programmers working in pairs are interrupted regularly. That makes sense. The highly collaborative nature of XP increases work-related interruptions. Moreover, pairing means there’s twice the probability of being interrupted. Fortunately, working in pairs also makes it easier to refocus.

But the big surprise for me in Jan’s report was the observation that programmers working alone created interruptions for themselves. Here’s how Jan described one non-XP, non-pairing programmer:

“…Ian has few external interruptions, which gives him the opportunity to devote long stretches of uninterrupted attention to his primary work task. Instead, however, we find that Ian’s session is filled with points at which he turns his attention away from his primary work task without any external impetus. Ian’s session alternates between engagement in periods of focused work and sustained periods of social conversation or other activities unrelated to work.”

Be honest. Do you ever create distractions for yourself? I do. All the time. And apparently I’m not alone.

My distractions are usually work-related. “I should check my mail,” I think. Or “I’d better look up how to…,” But these short diversions often devolve into, well, goofing off. The next thing I know one quick query on Google has turned into two hours of romping through interesting technology-related articles that have nothing to do with the problem at hand.

Reflecting on Jan’s observation helped me recognize a pattern: I’m particularly prone to these self-generated distractions when working on large, complex tasks. And Jan’s observations of Ian increases my suspicion that creating distractions is a common coping mechanism. (That would certainly help explain the wild success of You Tube.)

So if self-generated distractions provide a way of coping with big, boring, or complex tasks, the cure would be to make tasks small, interesting, and straightforward.

Sure enough, I notice that I’m better at sticking to the task at hand when I take small steps with clear completion criteria. For example, if working on my website, I’ll work on one image, text for one page, or one style at a time. The more I try to do at once, the more tempted I am to create a distraction. Distractions provide a mental break. I feel less overwhelmed, and thus less easily distracted, when working in small, well-defined increments. (You might note a similarity here with TDD, yet another way in which XP supports a high degree of productivity.)

Aha! That means being easily distracted isn’t a sign of personal weakness; it’s an indication that I need to redefine or reorganize my work, take smaller steps, and begin with the end in mind.

I can do that. Right after I check out what’s new on Reddit.com.

The Twelve Project Meetings

An Ode to W. Edwards Deming at Holiday Time

The holidays have begun taking their toll on my sanity.

My eldest daughter has become inordinately fond of the mind-numbingly repetitive 12 Days of Christmas song and has been forcing me to sing it every night. If she were 5 or 6 years old, I would think her obsession with the song is a cute and mostly harmless manifestation of her excitement about the holidays.

However, because she is a precocious 11 year old, I am beginning to suspect my daughter is conducting a subtle form of psychological warfare. I think she is intentionally ensuring that I go to sleep with visions of partridges in fruit trees, geese and swans a-swimming, lords and ladies dancing, and golden rings (f-i-v-e of them) prancing about in my head. Perhaps she thinks this will brainwash me into buying more presents. Perhaps she’s right.

Whatever my daughter’s true intentions, it’s her fault that I had the annoying tune stuck in my head as I was, for reasons completely unrelated to the holidays, revisiting Deming’s 14 points for management. I thought I’d share the results with you.

The Twelve Project Meetings
sung to the tune of The Twelve Days of Christmas
with apologies to W. Edwards Deming

At the project kickoff meeting, my manager gave to me: a vision of constancy.

At the second project meeting, my manager gave to me: true leadership and a vision of constancy.

At the third project meeting, my manager gave to me: free quality*, true leadership, and a vision of constancy.

At the fourth project meeting, my manager gave to me: more vendor trust, free quality, true leadership, and a vision of constancy.

At the fifth project meeting, my manager said to me: “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

At the sixth project meeting, my manager gave to me: constant improvement, “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

At the seventh project meeting, my manager gave to me: seven weeks of training, constant improvement, “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

At the eighth project meeting, my manager gave to me: aid doing better, seven weeks of training, constant improvement, “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

At the ninth project meeting, my manager gave to me: lines between groups blurred, aid doing better, seven weeks of training, constant improvement, “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

At the tenth project meeting, my manager gave to me: end to empty slogans, lines between groups blurred, aid doing better, seven weeks of training, constant improvement, “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

At the eleventh project meeting, my manager said to me: elevate pride in work, end to empty slogans, lines between groups blurred, aid doing better, seven weeks of training, constant improvement, “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

At the twelfth project meeting, my manager said to me: “We’ll do this together!”, elevate pride in work, end to empty slogans, lines between groups blurred, aid doing better, seven weeks of training, constant improvement, “DRIVE OUT FEAR!”, more vendor trust, free quality, true leadership, and a vision of constancy.

* Deming’s third point says build quality in; this has been widely popularized by Crosby’s “Quality is Free.” Besides, I needed something to rhyme with “three.”

Inside the Secret Fears of Agilists

Now with Agile

Scary isn’t it.

Agile, the edgy counterpoint to high ceremony, heavyweight, quality-of-result-depends-on-quality-of-process methodologies, could become just another vapid marketing buzzword.

How does this happen, you ask?  It’s an old pattern:

  1. A group of Really Smart People notice a Problem and create a Solution.  They share their Solution with the Industry.
  2. Others recognize the coolness, usefulness, righteousness, niftyness, or downright practicality of the Solution.  They adopt the Solution.
  3. The Solution works, at least for some of the people, some of the time.  Those who succeed with the Solution tell others, prompting the widespread response: “Hey!  I gotta get me some of that!”  The increased demand prompts more books/classes/conferences/services/products, and the hype from these new offerings in turn fuels increased demand, resulting in a self-perpetuating cycle of publicity.
  4. Executives hear about the Solution and mandate it, top-down.
  5. The Solution has now taken on a life of its own, and will be exploited as a Key Value Proposition by Mega-Vendors that sell to Executives.

This has happened before and will happen again.  It’s not just Agile.  We’ve seen this happen with specific programming languages (now with Java!), general technology (AJAX is Web 2.0!), open source (Free is Good!), other process models (Now Level 5!), and even the Internet itself (e-Everything!).

I’m fatalistic about the pattern.  It’s inevitable.  Even if Agile hasn’t reached Step 5 quite yet, it will get there.

But that’s OK because the real value of Agile transcends buzzword compliance.  Achieving true agility means finding the right balance of discipline and flexibility to systematically remove impediments to productivity within a given context.  That’s easy to say, hard to do, and impossible to package up with a neat little bow.

Those hoping to cash in on Agile Mania will discover that Agile is a little more difficult to grasp than the designed-for-auditability Capability Maturity Model.  They’ll find that creating a One-Size-Fits-All Template-Driven Standardized Unified Agile Process ends up looking way too much like the high ceremony processes Agile is supposed to replace.  They’ll learn that Agile does not lend itself to using statistical process control techniques to run huge projects on auto-pilot.  And they’ll eventually discover that you cannot assess Agile Compliance with an overly simplified checklist of audit success criteria.

It is possible that somewhere along the line, the word “Agile” will lose all meaning.  But if that happens it won’t be the end of Agile.  It will just be the end of a buzzword.  The Agile community may eventually decide to come up with a new, more meaningful label around which we can gather.  Or not.  Either way, Agility by any other name would retain the same balance of discipline and flexibility, and that’s the truly important thing.

Dig First

Late last night, I decided to change my domain record to use a different DNS server. I had good reasons, reasons that I will explain in a moment. But the thing about making DNS changes is that they take hours to propagate. Knowing I wouldn’t see the effects of my change for several hours, I made the changes then went to bed.

This morning, cradling a cup of coffee, I thought I’d do a little blogging. Imagine my dismay when I discovered that testobsessed.com couldn’t be found. “Oh crud, what did I do?” I wondered. Continue reading

Time to Re-negotiate?

On every project, we make commitments based on negotiated agreements, even when we don’t think we’re negotiating. We agree to accomplish certain tasks by a given deadline. We agree to follow a particular process. We agree to work late one day so we can leave early another. Or we agree to work over a weekend because we want to do whatever it takes to make the project succeed.

But sometimes we discover that the negotiated agreement no longer fits for us for some reason. We need to revisit the agreement. We need to re-negotiate. Continue reading

Who Carries the Risk?

I attended Ken Schwaber’s Certified Scrum Master training in June 2005. Yup, I’m a Certified Scrum Master. But I didn’t do the training for the certification. I did it for the learning. And for the chance to meet Ken.

I got more than I bargained for. I learned a lesson that has influenced my actions deeply and that forced me to re-evaluate choices I made in the past. I learned that for years I’d been accepting risk that was not mine to accept. Continue reading