Agile Certifications

Certification of software professionals has been a hot topic for quite a while. At least 15 years. Maybe longer.

I keep hoping that the whole thing will blow over.

But it hasn’t. And it’s not going to. Too many people have too much of a financial stake in the success of certifications. Certification customers, including individuals and their employers, want certifications to have value. Certification providers want to continue making money.

But while I’ve been able to ignore most development and test certification initiatives up until now, I don’t think I am going to be able to ignore Agile certifications for much longer.

So I guess it’s time for me to talk about this publicly. I’ll start with tester certifications.

General certification programs, like the ISTQB tester certification, focus on knowledge of “best practices” and definitions. I have nothing against learning the material in the ISTQB Syllabus. There’s good stuff in there (even if the most recent books in the Foundations bibliography were published in 2004). However, I do have a problem with charging a whopping huge amount of money for test preparation classes, and testing people on their ability to memorize the contents on a Body of Knowledge, then slapping a Certified Tester stamp on their forehead.

The classes surely have some value. The trainers I know who teach certification prep classes certainly have much to offer. I see no harm in learning what these people have to teach.

But the cost of these classes is high. Certification preparation class can cost hundreds of dollars more than comparable non-certification classes, and the ISTQB test fee is another couple of hundred dollars.

And for what?

It is not clear to me that there is any evidence demonstrating a positive correlation between competence at software testing and possession of an ISTQB certification. (Some wags have argued that there is a negative correlation. I’m not going there.)

Rather, I suspect there is no correlation. I do not believe that certified software testers are any better at testing, on average, than uncertified testers.

And because I do not think there is a correlation between tester certification and competence, I see no value in software testing certifications. I think they’re a marketing scheme concocted to increase training revenues.

But people buy into this stuff, and classes leading to certification outsell classes that don’t lead to certification.

It’s important for me to note that I don’t have any problem with certifications in a specific technology. When Microsoft certifies someone as an MCSE, it means that Microsoft, the creator of a technology, is certifying that the candidate has met minimum competency requirements in that technology. Microsoft is not pretending to certify someone as a developer; they’re certifying that the candidate knows some specific fiddly details about a specific technology related to development.

It may seem like I’m splitting hairs. But it’s an important difference. There is a right and a wrong answer for specific technical questions like how to change a Windows Registry setting (hint: it’s not the form you submit to Microsoft to register your copy of Windows). General topics, like Software Testing, are not so clear cut. What’s right in one context could be dead wrong in another.

So, enough on tester certifications. I’ve successfully ignored them up until now, and plan to continue ignoring them in the future.

What I’m really concerned about are general Agile certifications.

I started hearing the rumblings around Agile certification some years back. In response, the Agile Alliance published a statement about certifications. It’s a good statement. I’m delighted the AA published it. I was in the room when Brian Marick and a small group decided to write the statement and I think they did a fabulous job on it. That can’t have been an easy thing to write.

And I was delighted when Laurent Bossavit and Brian Marick started WeVouchFor, a different kind of certification involving endorsements of competence rather than tests of knowledge.

Sure, there’s always a risk with endorsements that it becomes a kind of you-scratch-my-back-I-scratch-yours mutual love-fest. But I think that even such reciprocal endorsement arrangements say far more than commercial exam-based certifications that have a pass rate in the high 90%s. I’m not alone in that perspective, but endorsements alone are not enough for a lot of people. They want the certification.

I do understand the desire to have Agile certifications.

Agile is relatively new. There are a lot of people, and companies, sporting big ol’ “Now with Agile!” stickers slapped on top of their old RUP/CMM/CMMi/current-hot-thing stickers. So it can be difficult to tell those with deep Agile understanding from those who think they’ll make more money by adopting the hot buzzword-of-the-month.

And so employers look for objective evidence, like certifications, that someone who claims to know Agile actually knows what they’re talking about. And individuals want those certifications as a form of evidence.

The only real Agile certifications that I am aware of right now are the various Scrum certifications. Since Scrum is now the most popular Agile process, it’s no surprise that the Certified Scrum Master (CSM) is the most commonly-held Agile-related certification available today.

As an aside, it’s my understanding that the CSM designation started as a kind of an in-joke. I got my CSM by taking the CSM class. In it, Ken Schwaber said that the certification meant that we probably knew a little more at the end of class than we did at the beginning. But he wasn’t guaranteeing it. And then he taught us all the “secret handshake” (woof) so we could prove to other CSMs that we were in the club. (For the record, I took the CSM class so I could meet Ken Schwaber and learn about Scrum from one of the originators. The certification was a side effect of taking the class. The AHA! moments and resulting deep learnings are far more valuable to me than the certification.)

Then the Scrum Alliance decided to take the CSM, and other Scrum certifications, seriously. They put teeth in the certification.

And that’s fine. It seems to me that the Scrum certifications are like technology certifications. The Scrum Alliance is certifying knowledge of Scrum, a specific process. They’re not trying to certify general knowledge across all things Agile. They’re not saying that being a CSM means you’re generally competent. They’re just saying that being a CSM means you know what a Scrum Master does within the Scrum process. You know the mechanics.

Further, the CSM class is fabulous with or without certification. It’s experience-based, participative, interactive and leads to deep learning. I recommend it.

But this week, I am again hearing the rumblings for general Agile certifications, not just Scrum certifications.

People are asking me how to become Certified Agile Testers. The very thought makes me queasy. Agile Testing isn’t a process or a technology. It’s testing in an Agile context. And that’s not something I know how to certify someone in.

And just today I ran across a site run by the self-proclaimed World Agile Qualifications Board. And that made me angry. Really angry. Angry enough to write this post and not to link to their site. You can search them out if you want to, but I won’t drive traffic their way.

Alan Page suggested on Twitter that perhaps it’s an April Fool’s Joke, like Waterfall2006.com. I’m hoping he’s right. I’d rather feel stupid for falling for the joke than outraged at the reality.

However, even if it turns out to be a practical joke, the strength of my anger surprised me.

On reflection, and being brutally honest, I realized that it’s an anger borne of fear.

I fear that the quest for certification and availability of general Agile certifications, no matter how dodgy, will lead people away from the non-certification classes and services that I, and others like me, offer.

In this economy, that could be disastrous for my business.

I would like to believe that people in this industry are sufficiently discerning that they will come to my Agile Testing classes because my classes are valuable. The kinds of things I teach are not the kinds of things that are certifiable.

How do you certify someone on the realization that they’ve been playing the hero all too often? Or on the bone-deep visceral understanding around the effects of changes in feedback latency. These are the kinds of lessons that I believe participants in my Agile Transformation simulation learn. And they are not things that translate to a certification exam.

And so I am afraid, even though I know that fear is a lousy compass.

Being afraid makes me feel pressure to offer something certification related. But doing so goes against my principles for all the reasons I’ve already explained.

Maybe the best thing I could do would be to offer a self-certification. Want a certification? Declare yourself certifiably Test Obsessed. Here’s the certificate:

Certifiably Test Obsessed

Subscribe

Subscribe to our e-mail newsletter to receive updates.

21 Responses to Agile Certifications

  1. Danny Faught March 18, 2009 at 2:30 pm #

    I’ll pick a nit with one point. I think that testers with a certification are likely to be better at testing on average than the general population of testers, simply because someone who holds a certification is much more likely to have attended a training course than the average tester.

    Elisabeth responds:

    When I was a hiring manager and doing lots of tester interviews, I found no correlation whatsoever between certified testers and competence in testing. As a trainer, I similarly find no correlation across a large population of testers who end up in my classes. This is all anecdotal; I have no statistical data to back this up, but I stand by my original statement.

    Pretty scary that you’re feeling competitive pressure from certifications, when the associated courses and exam probably cost more than your classes.

    Elisabeth responds:

    I’m not feeling direct competitive pressure. Yet. But I know that there is great demand for certification. And it’s a demand I cannot, in good faith, meet.

  2. Lisa Crispin March 18, 2009 at 2:32 pm #

    Some people behind this latest certification had contacted me to try to involve me. They seemed sincere, but I have always been anti-tester-certification, never mind agile. I hesitated because I thought, maybe I could give them some input that would make their certification less damaging – but I can see that’s a futile effort. Thank you so much for this valuable post.

  3. Matthew Heusser March 18, 2009 at 2:47 pm #

    I wonder if I am “someone on twitter” or if a bunch of us asked that april fool’s question at the same time! 🙂

    Elisabeth Responds:

    There may have been a few folks calling it a joke at the same time.

    Sadly, as I dug further into this mess, what I’m finding is that WAQB seems to be brought to us by people who are deeply involved in ISTQB. I just wish it had been an April Fool’s joke. Sigh.

    Either way, great post, and some solid points.

    Just today, I was thinking of making this my new email signature line. I believe it holds true:

    “The main argument for certification, as I see it, is so that companies can outsource discernment in staffing to a objective evaluator. The problem is that if certifications were in finance, Bernie Madoff would have been a PMP/CSM/Six Sigma who was CMMI-5.

    I’m sorry, but I’m afraid that discernment in staffing just isn’t something you can outsource – at least no successfully. You’re gonna have to bite the bullet and learn to interview.”

  4. Jonathan House March 18, 2009 at 2:50 pm #

    Elisabeth,

    Thanks for posting this – Lisa pointed the Salt Lake Agile roundtable group at this earlier today and we’ve been having our own private rant about the WAQB.

    I’m happy to echo your concerns about certifications – both specifically for testing, and for software development in general. Many people have discussed the difference between skill-based certification where you have to prove your competence with the material, and test based certification where you have to prove you can take a test.

    Danny above points out that a certification implies that someone who is certified is more likely to have taken a training class on the subject. That’s a good point, although in my experience the people that are the best at their areas (e.g. “Agile” testing) don’t have certifications, nor would they pursue them unless they felt a professional pressure to have that certification. In my world the listing of a certification is a claim that the person claiming the certification is “trained”, and thus deserving of a thorough grilling on the subject.

  5. James Bach March 18, 2009 at 2:54 pm #

    Danny, taking a class doesn’t make someone a better tester, especially with all the bad classes out there. Far more than good classes. I know of only three classes besides mine that I admire (Elisabeth’s is one of them). Do you REALLY think that ANY testing class is better than not taking a class? Do you think a class that emphasizes the importance of paperwork and fuzzy-headed terminology like “equivalence class partitioning” makes ANYONE a better tester?

    Think about it. If a tester came to you and said, “I haven’t yet taken a class on testing and I have no certification, but give me something to test right now and I will show you what I can do” wouldn’t that impress you? Demonstration is MUCH more powerful than passing a test of verbal doctrine or certifiably sitting-in-a-room while someone lectures.

    A tester’s enthusiasm is certification enough for me to let him take my time to demonstrate his skills. I’m really surprised that there are people in our industry who don’t get that.

  6. Matthew Heusser March 18, 2009 at 3:23 pm #

    On commentor wrote:
    “That’s a good point, although in my experience the people that are the best at their areas (e.g. “Agile” testing) don’t have certifications, nor would they pursue them unless they felt a professional pressure to have that certification”

    At one time, I felt professional pressure to pursue ISTQB certification. I turned to Dr. Cem Kaner, a pillar of the test community, about it. He looked me straight in the eye and said these exact words: “Matt, if you do that, they win.”

    I stopped pursuing the certification, and I’d say my career has done just fine, thank you.

    I’ve also gotten into a number of some-what public discussions about some of these certs on mailing lists. On two occasions, people emailed me back (one a PMP and CSM, another an ISTQB instructor) and admitted privatly that they saw little value in the certs. The PMP/CMS said he felt like a fraud, and the ISQTB-instructor said he “had to do what he had to do to pay the bills, you know what I mean?”

    So my turnaround question to anyone feeling “pressure” to be certified is simply this: Do you want to be one of those guys?

  7. Mark Crowther March 18, 2009 at 3:47 pm #

    “It is not clear to me that there is any evidence demonstrating a positive correlation between competence at software testing and possession of an ISTQB certification.”

    As a hiring manager my view is there isn’t but the expectation is there is. Time and again I’ve had CVs and interviews because a candidate is “ISEB’d up” and that’s meant to excite me, it only means they’re a bit more interested in testing than perhaps others.

    I would rather see a Hendrickson/Bach/Lyndsay course on someone’s CV and know the candidate will have learned something not just studied.

    Yes to your statement James, I hired one of the best testers I’ve ever hired becuase she said “I’ll show you”, that means way more to me than a certificate.

    I hope agile doesn’t become ISEBd!

  8. Dave Nicolette March 18, 2009 at 4:39 pm #

    Very well said.

  9. Oliver Erlewein March 18, 2009 at 5:23 pm #

    @James B & Elisabeth:
    Maybe we should pick up James’s great idea of the anti-qualification badge again. Organised insensibility can only be met with organised revolt.

    I know so many people that absolutely detest ISTQB that I might say that I haven’t met one which defended ISTQB. So why is ISTQB still being done? Where is the revolt?

    I do think yourselves and others in your loose group should start thinking about propagating the “right stuff”. Otherwise we WILL loose. And it seems as if slowly the consequences of a win for certification will cost you money. That makes it valid to get organised and do something about it.

    What would be nice is a good and informative largely unbiased website discussing the pro’s and cons that can be presented to stakeholders for consideration. Additionally a colleague of mine had the idea of a de-certification web site, where frustrated ISTQB (or other) certification holders can denounce their certification.

    …just an idea.

    Does this post mean I will this mean I get a visit from Homeland Security?

  10. J. B. Rainsberger March 19, 2009 at 1:34 am #

    Thank you for writing this, Elisabeth. I hesitated tweeting about the WAQB because I didn’t want to over-react. I figured I’d let other people over-react for me, knowing that many of them would react well. Thank you for acting as one of those people.

  11. J. B. Rainsberger March 19, 2009 at 1:35 am #

    I almost forgot… I’d already launched the site http://www.agilecertificationnow.com on April 1, 2006 anyway, and beat them to the punch.

  12. Joao Rodrigues March 19, 2009 at 9:42 am #

    Regarding Danny’s reply, “testers with a certification are likely to be better at testing on average than the general population of testers”, there can be a number of reasons to think the opposite. We could find a correlation between professionals that are certified on /certain/ courses and unability to do the job (generically speaking):
    -If an employee has atended a certification programme, probably the enterprise paid for it because no one there knew anything about the topic, including the newcomer. Someone that learned through experience and communication with experienced professionals would probably be much better doing the job.
    -A paid certification is somehow a substitute for competence. An alternative to being good at something, is holding a marketing competence label, and showing it around, we all have seen this…

    So these are just 2 reasons why there could be negative correlation as well.

  13. Nikki Sedgwick March 20, 2009 at 5:13 am #

    The only thing going for ISTQB qualifications and the like is that it’s the only way I’ve found of proving (in a commercial/corporate/non-specialist market) that I have the skills I’ve always had.

  14. Michael Hunter March 20, 2009 at 2:57 pm #

    Elisabeth, Anyone who chooses not to take one of your courses simply because it does not lead to a * Tester certification is certifiable!

  15. Arjun Rathore March 20, 2009 at 3:15 pm #

    I have been a performance engineer for almost a decade now and I find this certification business little troubling. This is being driven by the industry. It does have some value but should not be the main criteria for employment. I believe there is no short cut for experience. When I interview people I can tell within 15 minutes the capabilty of a person, just by asking broad questions and not even getting too technical. Also what if I get a certified guy and suddenly the technology changes. How well this guy will adapt? I normally look for people with potential and it is ok if the person does not have full knowledge of the current environment. If that person is quick on the feet and pick up things quickly I am fine. I personally do not give much weightage to certifications.

  16. Jeff Anderson March 22, 2009 at 12:15 pm #

    as always great post.

    I hold absolutely no certifications in any methodology, software discipline, project management approach and yet I’m clearly regarded as the firm expert on SDLC.

    I agree with you, technical certifications are good, because they at least test whether you have basic working knowledge with some specific product.

    Certificates the tester knowledge on a best practice are pretty meaningless, the devil is in the details, and nothing can replace real experience.

    Regards

  17. Paul Denize March 22, 2009 at 12:59 pm #

    http://www.agilecertificationnow.com
    This website just cracked me up. I think we need some printable posters for the workplace to spread the message. I’d love to have just one customer embarrased to ask for certification because they are scared of looking stupid/ignorant which I’m sure they are not. They are just sold the hupe about certification, the sales pitch from tarining establishments that make money for certicication and courses – and in contrast a lack of alternative view being shoved at them.

  18. Hillary Johnson March 23, 2009 at 11:48 am #

    Thinking about certification in general, why does your barber need to be certified, whereas chefs and fry cooks do not? The emergence of certification regimes is pretty arbitrary.

    Certifications in any field are almost a form of unionization, meant to provide centralized control and oversight over who gets in. It’s always meant to protect the members of a profession by creating barriers to entry that are designed to keep demand higher than supply which in turn keeps salaries high. However, the inevitable result is that anyone with a touch of innovation in their soul is turned off from jumping through those hoops, and you end up with a legion of plodders.

    Agile being based on a small, salient set of values and practices, it can only be made certifiable by crufting it up with extraneous nonsense, which certainly isn’t very agile. In my native field of publishing and journalism, there are no certifications, because it’s acknowledged that talent and convictioon count so much more than any BOK that the skillset is unquantifiable. Imagine looking at a book you pick up for a “certified Novelist” label–well, I submit that certifying Agile practitioners is no less ridiculous.

  19. M P March 23, 2009 at 2:07 pm #

    I just grill the candidates (with certifications) a bit more than a candidate without the certifications 🙂 It becomes very interesting when you start talking about the imprecision in the testing glossary and the set-in-stone answers that you get from “certified” candidates.

    I have seen that certification is almost a pre-requisite in the European market compared to the US market. I don’t know what drives the psychology of the decision makers there.

    My experience has been that once you end up with a manager who is metrics driven, they buy into the snazzy marketing literature from ISTQB and such and even companies marketing testing software so well, that they turn into kool-aid fueled management hacks that insist that its the certified-way or the highway.

    In my book, a PMP is a pretty mediocre person and CSM is Certified **** Master.

    Good candid post Elizabeth and sorry for the rant.

  20. Scott Seltzer March 24, 2009 at 1:29 am #

    From my experience, many people I’ve interviewed that took software testing courses (not necessarily for certification) didn’t seem to have any practical skills at all. Sure, they spouted off some terminologies including acronyms I’ve never heard of (I’m QA manager for 15 years and I read a lot on our subject), but I felt like they never even turned on a computer; many couldn’t even do the most basic computer skills.

    In defense of Danny’s comment, I’d say that a tester with experience plus training might be better than someone with the same years of experience and no training/certifcation.

    On the other hand, I’m wondering about the relationship between testing certifications and university degrees. I personally have very little value for my diploma, though I’m certainly not going to keep mention of my degree off of my resume. But when I hire, I’m not a stickler for degrees and my current test lead is successful without his having one. Obviously James Bach, the Buccaneer-Scholar (http://www.buccaneerscholar.com/), is a great example of an expert in our field without any formal training.

    After writing the above, I looked for articles online that compare having a certification to having a degree and this one is the most relevant I could find: http://www.prototest.com/protoblog/2008/10/7/are-testing-certifications-bogus.html. Why have most of us accepted the value of degrees but get so defensive about certifications? Why differentiate between skill-based and knowledge-based in software testing but not in university degrees?

    -Scott

  21. Anthony Cairns July 4, 2009 at 11:12 am #

    The entire discussion around Test (and indeed other) certification is quite interesting really. As a Test Management Consultant with over 11 years experience I have found that by having some kind of certification is something to help you get your foot in the door for companies who swear by them. The structured testing methodologies are basically great to enable testing personnel to be able to draw out a “pencil” outline of what is to be achieved, and then it is down to professional themselves to use their understanding and knowledge as to how well they “colour” the plan in with real content and scope.