Skip to main content

The 21st century definition of TEST

What is the difference between a test and an experiment?


I propose that in the 21st century and the realm of software development that these definitions must morph to our needs.  There is little difference in the general definition. Yet many people in quality control or quality assurance departments appear to dislike the word experiment.   Defining actions a person takes to perform a 'test-case' as an experiment appears to rankle feathers.   I find this interesting.

Test - (verb) take measures to check the quality, performance, or reliability of (something), esp. before putting it into widespread use or practice.

Experiment - (verb) perform a scientific procedure, esp. in a laboratory, to determine something.
I would like to define that within the modern software world that the word test have a more specific meaning.  I propose:
Test - (verb) a highly repeatable measure to check the quality, performance or reliability of (something), esp. before (something) is created and then put into use or practice.
This definition would distinguish testing from experimenting within the domain of software engineering.  First, it separates testing from experimenting by the aspect of 'highly repeatable' measures.  In todays world of software development if we are not using the power of computers to make our measurements repeatable (which computers happen to be extremely good at) then we are not using the exponential leverage of our own industry.

Second, it suggest that a distinguishing feature of a test is that it can and should be conceived before the thing being tested is created. Well this is just good scientific practice in the first place.  One creates an experiment with a belief they know what will happen and the open mind to experiment and measure the actual results (true or false).  Therefore one must have a hypothesis first.  It may be proven false - at which point the scientist has learned something very valuable.  This aspect of experiment is understood in scientific circles; but in the software industry it needs to be explicitly stated.

These additional aspects of the definition of test when used within the software industry would imply that we could distinguish between a person running the software under development and seeing if the system had the expected behavior via an experiment (probe - sense - respond; Cynefin (video) Complex topology) versus a test in which the person executed a highly repeatable measure to check if the previously predicted behavior actually happened (sense - categorize - respond; Cynefin Simple topology).

Expanding our understanding of the terms we use within a technical field is part and parcel of our industry.  This is the Ubiquous Language activity of the Domain-Driven Design practice.


Post a Comment

Most Popular on Agile Complexification Inverter

Software Development terms applied to Home Construction

Let's Invert the typically wrong headed view of Software Development project management as a construction project.  We can map it the other way just to see if it works... to have some fun, to explore the meaning of phrases we toss around quite frequently.


Normally Project Management terms come from a construction domain.  We are going to apply the lexicon of modern software to the construction of a home.  We will follow the construction project and meet some of the people doing the work.

This is a very small (8 homes from $600,000 skyward) program in my 30-40 year old neighborhood.

About 6 months ago I saw the programs landing page go up.  It gives casual observers and some of the stakeholders a general idea of the intent of the program.  And most importantly who to contact for additional information if you happen to be interested in their products.

The Refuge program has 8 product projects and has them running independently.  Yet much of their DevOps infrastructure has already b…

Where is Shakespeare When We Need Him?

We are desperately searching for a term for people that connotes the best of human kind.  The creative, sensing, combinatorial synergistic, empathic solutioning persons that have yet to been labeled with a role name that works.

Some of the old terms:
Staff, Workforce, Human Resource, My Team, Army, Company

Shakespeare created 1700 words in his time.  He mutated verbs to nouns, and vice-a-versa, transformed verbs into adjectives, and formed words from whole cloth never before heard.  This skill is rare, but there is a poet that can create the term we need in the twenty-first century.

What should this term define?

21st Century Human Resource; the generalizing specialist.

Yes, but what more?  What less?

Suggest your poetry in the comments, let us see if we cannot do 1/1700 as well as The Bard.

By-the-way; who create the phrase "coin a word"?



A TED Play List - How do you create new words
6:52
Erin McKeanGo ahead, make up new words! In this fun, short talk from TEDYouth, lexicographer Er…

a little feedback please...

some feedback please...
How do you like the new look and feel of our site?
  ___)  nah (I like the old one better - bring it back)
  _X_)  yeah (much cleaner and easy to navigate)



powered by Typeform
See Also:

One Dark and Stormy during a Hurricane

I'm from the Carolina's where legend has it that our family commonly just hunkered down in the home on the coast and waterways than to head for inland shelter. Now that's from the old school days of barely improved (read paved) roads. They counted a storms severity by how high on the back porch steps (about 15 - top to ground) the water reached.  I don't recommend this action in todays world of long range forecast and transportation options.

I do recommend a drink or two in a hotel bar, far far away.

This is the week that Harvey came ashore in Texas.  I live on a hill in the little old town of Grapevine outside Dallas and Fort Worth.  And thank you all for letting me know that a storm is coming... I didn't get out and walk Malibu before the rain hit, so I grabbed a hat and we went anyway.  Much nicer walk with the drizzle, I'd say.

I'll raise a glass to you - if you were not smart enough to do the responsible thing, at the last responsible moment.

I do re…

Innovation in the Automobile Industry

In the 1900s the automobile industry was the most important and innovation industry in the USA.  But one could question if this was good for our society in the long run.  And one could question if they actually innovated.

In the early 1900s there were few automobiles, very little infrastructure created to support the industry.  For example the road system was still designed for horse drawn wagons and the wagon wheel (remember a steal rim and wooden compression spoke wheel).  The future US Highways, or the 1950s Interstate Highway System at the cost of $425 billion were decades and many innovations away. There was no gas service station, there were however horse stables, farriers, and blacksmiths in each town along the roads.  There was no real "road map", there was no road naming system, like was created in 1926 - the United States Numbered Highway System.

The industry employees millions of people, and was a large factor in the economy of the USA.  It created or was created b…