A reader wrote to ask me, “My team and I have been discussing the definition of ‘test’ and ‘testing.’ What’s your definition?”
There are almost as many definitions of software testing as there are books on the topic.
- In his 1979 book The Art of Software Testing, Glenford Myers said, “Testing is the process of executing a program with the intent of finding errors.”
- Kaner, Falk and Nguyen agreed in their 1999 book, Testing Computer Software (2nd Ed): “The purpose of testing a program is to find problems in it.”
- In his 1983 book The Complete Guide to Software Testing, Bill Hetzel said, “Testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results.”
- The 1990 IEEE Standard Glossary of Software Engineering Terminology (Std 610.12-1990) says testing is, “The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component.”
But whatever definition of testing you subscribe to, the result of testing is information: information about bugs, discrepancies, misunderstandings, current status, and software behavior under a variety of conditions.
Dale Emery and I discussed this at length and decided that we agree that: Testing is a process of gathering information by making observations and comparing them to expectations.
(Please note that this definition is intentionally vague about where those expectations come from.)
That leaves the question about the definition of the word “test.”
There are numerous definitions including the one in the IEEE glossary (Standard 610.12) that defines “test” as “An activity in which a system or component is executed under specified conditions, the results are observed or recorded, and an evaluation is made of some aspect of the system or component.”
However, Dale and I find a slight reframe helpful: A test is an experiment designed to reveal information, or answer a specific question, about the software or system.
Anyone have definitions for “Testing” and “Test” that you like better?