The 21st century definition of TEST

What is the difference between a test and an experiment?


I propose that in the 21st century and the realm of software development that these definitions must morph to our needs.  There is little difference in the general definition. Yet many people in quality control or quality assurance departments appear to dislike the word experiment.   Defining actions a person takes to perform a 'test-case' as an experiment appears to rankle feathers.   I find this interesting.

Test - (verb) take measures to check the quality, performance, or reliability of (something), esp. before putting it into widespread use or practice.

Experiment - (verb) perform a scientific procedure, esp. in a laboratory, to determine something.

I would like to define that within the modern software world that the word test have a more specific meaning.  I propose:

Test - (verb) a highly repeatable measure to check the quality, performance or reliability of (something), esp. before (something) is created and then put into use or practice.

This definition would distinguish testing from experimenting within the domain of software engineering.  First, it separates testing from experimenting by the aspect of 'highly repeatable' measures.  In todays world of software development if we are not using the power of computers to make our measurements repeatable (which computers happen to be extremely good at) then we are not using the exponential leverage of our own industry.

Second, it suggest that a distinguishing feature of a test is that it can and should be conceived before the thing being tested is created. Well this is just good scientific practice in the first place.  One creates an experiment with a belief they know what will happen and the open mind to experiment and measure the actual results (true or false).  Therefore one must have a hypothesis first.  It may be proven false - at which point the scientist has learned something very valuable.  This aspect of experiment is understood in scientific circles; but in the software industry it needs to be explicitly stated.

These additional aspects of the definition of test when used within the software industry would imply that we could distinguish between a person running the software under development and seeing if the system had the expected behavior via an experiment (probe - sense - respond; Cynefin (video) Complex topology) versus a test in which the person executed a highly repeatable measure to check if the previously predicted behavior actually happened (sense - categorize - respond; Cynefin Simple topology).

Expanding our understanding of the terms we use within a technical field is part and parcel of our industry.  This is the Ubiquous Language activity of the Domain-Driven Design practice.

A video that cannot be unseen. If you want to truly understand the difference between a unit test and integration tests.

Watch what happens when you write plenty of unit test - but skip the integration test.

Skipped the Integration Test didn't you?