2nd Annual QA/Test Job Posting Study

This is a guest blog post by Daniel Frank, my assistant. Daniel took on the challenge of updating the QA/Test job study for 2011, just in time for making New Year’s resolutions. Enjoy! Elisabeth

It’s been a little over a year since Elisabeth published “Do Testers Have to Write Code,” the results of an in-depth survey of job ads that she and Melinda conducted to see if employers expect testers to program. The resounding conclusion, with 80% of tester job ads requesting some kind of programming skill, was “Yes.”

This year we wanted to see if things have changed, so I conducted the same study again. I also wanted to add a bit more granularity to the study, to see if there were any trends that were missed last time.

I screened the lists with the same basic guidelines as our previous study. That means I restricted my search to the US only. I only counted a job if it was described as a testing/QA position in the job title. I did not include recruiter listings in order to avoid the risk of including duplicate jobs or even fake jobs used to gather pools of applicants.

Our final sample size this year is 164 jobs. That’s a little less than last year. Why?

The lists were sparse. There just aren’t that many job ads out there. Many of the job ads I found were from recruiters or were repeats, with the same company listing the same position several weeks in a row.

The simple fact that I had a hard time finding the same number of ads as last year is interesting information all on its own. From an overall economic standpoint, the country is in no more of a slump than we were in 2010. So why are there fewer listings for testers? Could it be that Alberto Savoia, who recently declared testing dead, is correct? We’ll come back to that question later.

Back to the study…

Like last year, the majority of our jobs came from Craigslist (90) and LinkedIn (64). The rest of them came from a smattering of other sites.

The data includes an even higher proportion of jobs in California than last year: 102 of the listings were in CA, with the remainder divided in small chunks between 28 other states. Unsurprisingly,Texas, Massachusetts, and Washington are the three runners up.

Last year there was some question of whether or not the sample was biased simply because we’re located in California. However, I took extra steps to try and get equal representation. The simple fact is that a search that might find 70 jobs when I filter the location for CA will result in 30 jobs or fewer if I filter for another area. If anything, I’d estimate that California is actually under represented.

I kept track of the job titles. By far the most popular title is “QA Engineer” (99 of the listings). 136 of the titles contained “QA” compared with only 32 containing the word “Test.”

An interesting side note: when I searched for the word “test” in the body of job ads, I found far more developer positions than similar searches for “qa” did. It would seem that at the same time QA/Test positions are requiring more coding skills, developer positions are requiring more testing skills. That might be another interesting job ad survey project.

So how much coding are testers expected to do?

Of the 164 listings, 102 jobs say they require knowledge of at least one programming language, and 38 jobs indicate coding is a nice to have. That’s 140 out of 164, or 85.37% of the sample. That’s an even higher percentage than last year. It’s difficult to say if the 5% uptick represents a real increase in demand, but at the very least it’s fair to say that demand for testers who code remains high.

I used the same criteria that Elisabeth and Melinda used last year. That means that I counted a job as requiring programming if the job required experience in or knowledge of a specific language, or if the job duties mentioned a language. There were 7 jobs which listed broad experience requirements like “must be able to script in multiple languages,” which also counted as requiring programming.

There were some judgment calls to be made about what may or may not count as a programming language. For the purpose of the results here, I counted SQL or other relational database knowledge as a programming language in order to be consistent with last year. However, unlike last year, I tracked proficiency in relational databases separately. This will let me track specific trends more easily in future studies.

One of the questions Elisabeth wanted to answer last year was whether jobs with self-identified Agile organizations required testers to code more than other jobs. This year 46 of of the 58 Agile job ads list programming skills as required or nice to have. That’s 79.31%, which is actually a lot less than last year’s 90%. However, this is one of those places where the small sample size has to be taken into consideration. In 2010, 49 out 55 agile jobs mentioned programming. Today, 46 out of 58 jobs mention it. Just a few jobs result in a 10% variation.

An enduring question about any kind of job is how much it pays. I saw even less mentions of pay this time around. Only 7 jobs even listed it, and 5 of those were button-pushing game testing positions in the $10-$20/hour range. The other two ran around $85,000-$105,000. Most positions simply don’t provide up front salary estimations, so we cannot draw any real conclusions from these data points.

Just for fun, I also noted whenever a job requested a certification. In 164 jobs I found exactly 4 mentions of certification, and not a single one was required. 3 of them were vendor or technology certifications that had nothing to do with testing. And even in the single instance where a testing certification was nice to have, it was the CSTE offered by QAI, rather than the much more hyped ISTQB. So it would seem that testing certifications are not much in demand. The bottom line is that someone looking to improve their marketability would be much better served by upskilling to a new proficiency rather than picking up an irrelevant certification.

And that’s about it for our study. If you’d like to dig through the raw data to look for any trends I may have missed, I’ll be happy to send it to you. Drop me a line.

Now back to the question about the number of QA/Test jobs out there. Could it be that there are fewer QA/Test positions? Was this just a matter of luck and timing, or is there a trend?

Alberto Savoia gave a talk titled “Test is Dead” at GTAC (dressed as the Grim Reaper). He may have used intentionally inflammatory hyperbole to make his point, but that doesn’t change the fact that he had interesting points to make.

Alberto points out that especially in web development, speed is paramount. Further, the biggest challenge isn’t in building “it” right, but in building the right “it.” So the goal is to get a minimum viable product out as quickly as possible, and get fast feedback from real users and customers. Traditional black box testing ends up taking a back seat in this type of development, and these projects often rely heavily on user feedback instead.

At STARWest 2011, James Whittaker of Google gave a talk titled “All That Testing is Getting in the Way of Quality” where he talked about the closest thing to a traditional testing role they have at Google. It’s called the “Test Engineer,” and they spend anywhere from 20%-80% of their time writing code. He also explains how Google utilizes their user bases to do almost all of their exploratory tests. As he puts it, “Users are better at being users than testers are, by definition.”

With James and Alberto’s talks firmly in mind, I can’t help but wonder if the difficulty I experienced in finding job ads that met my criteria is indicative of a sea-change in the industry rather than an anomaly. Could it be that we’re seeing a reduction in the number of QA/Test positions?

What do you think? Are you seeing fewer QA/Test positions in your organization or (if you’re looking) in your job search?


Subscribe to our e-mail newsletter to receive updates.

8 Responses to 2nd Annual QA/Test Job Posting Study

  1. Nahuel December 23, 2011 at 9:25 pm #


    Very interesting data, I’m currently seeing an increment in automation and coding background testers, I’m from Argentina (LatinAmerica) where the testing is not so diffused (as it is in India, Germany or the Netherlands) and most of the companies outsourcing testing services are requesting some kind of knowledge regarding this activities.
    I believe that intern manual testers are dying and it is just matter of time to enter (if we are not already in) in a new testing phase, we have to keep learning and be up to the future expectations.

  2. Stian Danielsen December 23, 2011 at 10:42 pm #

    Heya Daniel and thanks for your very interesting follow-up to last years study!

    Here in Norway the market for QA/Testers is very good at the moment – we are usually a bit behind US trends so the possible downslide you might have identified in the US is not happening yet here in Norway.

    In general I would say it’s a good idea to pick up some programming skills if you don’t have any already. Ideally, I would become proficient in SQL, a scripting language and a proper programming language like Java, C# or C++.

    Personally I enjoy being involved all the way from the role of test leader/planner to implementing automation tests. Being able to cover the whole spectrum gives me a more secure feeling about my job position.

  3. Marlena December 24, 2011 at 1:32 am #

    What an interesting read! Thanks for putting this together for a 2nd year in a row. Regarding certification, if you did this study for Europe or New Zealand, you’d likely find many more jobs that require certification.

  4. Ben Kelly December 24, 2011 at 7:20 am #

    I wonder if this is indicative of a move by employers to find testers through other channels. It would be interesting to see whether hiring for other IT roles were similarly down. Personally I much prefer to find testers through word of mouth and back-channels before trying to resort to the open market. It’s much easier to avoid having to slog through hundreds of resumes from the testing dead before coming across that diamond in the rough. I have no evidence either way, but perhaps there are other avenues that are not so visible that have received an uptick of tester hiring activity.

  5. Anna Baik December 25, 2011 at 2:09 pm #

    “Of the 164 listings, 102 jobs say they require knowledge of at least one programming language, and 38 jobs indicate coding is a nice to have.”

    Given that this year you tracked requests for SQL knowledge separately, I’d be really interested to hear what number of the jobs above required SQL only. I’m in the UK, so it’s likely that there will be strong differences, but my experience is that it’s not uncommon for manual testing roles to ask for SQL knowledge, and that testers who have that experience but no other programming experience don’t tend to describe themselves as able to program. (Even if they’re extremely proficient!)

  6. testobsessed December 26, 2011 at 6:33 pm #

    Hi Anna,

    Just 19 of the 140 jobs that we counted as requesting programming skills (either required or nice to have) only counted because they mentioned relational database/SQL skills. Removing those ads shows that nearly 74% of jobs want programming skill in something other than (or in addition to) SQL.


  7. Lisa Crispin December 28, 2011 at 10:41 pm #

    Interesting. I know of several companies in the Denver/Boulder area who are practically desperate for good testers who would fit well into an agile team. And they’re not all looking for testers who can code, though they do want technical type backgrounds. I would have expected that in other areas too.

  8. dfrank January 5, 2012 at 3:42 am #

    Hey Ben,

    That’s not the first time I’ve heard that theory. It’s one of those theories that sounds sensible, but there’s really no way to test it. Companies that aren’t announcing to the public that they’re hiring, by necessity, will not show up on any public searches.

    I agree that your suggestion is just as likely a possibility as the idea that “test is dead.” Probably more likely, though I suspect Elisabeth would chide us for speculating too much. 😉

    Either way, it’s fascinating to see any downticks within an industry that is generally known to be one of the few still rapidly growing.

Leave a Reply