Skip to main content

The 21st century definition of TEST

What is the difference between a test and an experiment?


I propose that in the 21st century and the realm of software development that these definitions must morph to our needs.  There is little difference in the general definition. Yet many people in quality control or quality assurance departments appear to dislike the word experiment.   Defining actions a person takes to perform a 'test-case' as an experiment appears to rankle feathers.   I find this interesting.

Test - (verb) take measures to check the quality, performance, or reliability of (something), esp. before putting it into widespread use or practice.

Experiment - (verb) perform a scientific procedure, esp. in a laboratory, to determine something.
I would like to define that within the modern software world that the word test have a more specific meaning.  I propose:
Test - (verb) a highly repeatable measure to check the quality, performance or reliability of (something), esp. before (something) is created and then put into use or practice.
This definition would distinguish testing from experimenting within the domain of software engineering.  First, it separates testing from experimenting by the aspect of 'highly repeatable' measures.  In todays world of software development if we are not using the power of computers to make our measurements repeatable (which computers happen to be extremely good at) then we are not using the exponential leverage of our own industry.

Second, it suggest that a distinguishing feature of a test is that it can and should be conceived before the thing being tested is created. Well this is just good scientific practice in the first place.  One creates an experiment with a belief they know what will happen and the open mind to experiment and measure the actual results (true or false).  Therefore one must have a hypothesis first.  It may be proven false - at which point the scientist has learned something very valuable.  This aspect of experiment is understood in scientific circles; but in the software industry it needs to be explicitly stated.

These additional aspects of the definition of test when used within the software industry would imply that we could distinguish between a person running the software under development and seeing if the system had the expected behavior via an experiment (probe - sense - respond; Cynefin (video) Complex topology) versus a test in which the person executed a highly repeatable measure to check if the previously predicted behavior actually happened (sense - categorize - respond; Cynefin Simple topology).

Expanding our understanding of the terms we use within a technical field is part and parcel of our industry.  This is the Ubiquous Language activity of the Domain-Driven Design practice.




A video that cannot be unseen. If you want to truly understand the difference between a unit test and integration tests.

Watch what happens when you write plenty of unit test - but skip the integration test.

Skipped the Integration Test didn't you?

Post a Comment

Most Popular on Agile Complexification Inverter

David's notes on "Drive"

- "The Surprising Truth about what Motivates Us" by Dan Pink.

Amazon book order
What I notice first and really like is the subtle implication in the shadow of the "i" in Drive is a person taking one step in a running motion.  This brings to mind the old saying - "there is no I in TEAM".  There is however a ME in TEAM, and there is an I in DRIVE.  And when one talks about motivating a team or an individual - it all starts with - what's in it for me.

Introduction

Pink starts with an early experiment with monkeys on problem solving.  Seems the monkeys were much better problem solver's than the scientist thought they should be.  This 1949 experiment is explained as the early understanding of motivation.  At the time there were two main drivers of motivation:  biological & external influences.  Harry F. Harlow defines the third drive in a novel theory:  "The performance of the task provided intrinsic reward" (p 3).  This is Dan Pink's M…

Exercise:: Definition of Ready & Done

Assuming you are on a Scrum/Agile software development team, then one of the first 'working agreements' you have created with your team is a 'Definition of Done' - right?



Oh - you don't have a definition of what aspects a user story that is done will exhibit. Well then, you need to create a list of attributes of a done story. One way to do this would be to Google 'definition of done' ... here let me do that for you: http://tinyurl.com/3br9o6n. Then you could just use someone else's definition - there DONE!

But that would be cheating -- right? It is not the artifact - the list of done criteria, that is important for your team - it is the act of doing it for themselves, it is that shared understanding of having a debate over some of the gray areas that create a true working agreement. If some of the team believes that a story being done means that there can be no bugs found in the code - but some believe that there can be some minor issues - well, …

What belongs on the Task Board?

I wonder about these questions a lot - what types of task belong on the task board?  Does every task have to belong to a Story?  Are some tasks just too small?  Are some tasks too obvious?  Obviously some task are too larger, but when should it be decomposed?  How will we know a task is too large?

I answer these questions with a question.  What about a task board motivates us to get work done?  The answer is: T.A.S.K.S. to DONE!



Inherent in the acronym TASKS is the point of all tasks, to get to done.  That is the measure of if the task is the right size.  Does it motivate us to get the work done?  (see notes on Dan Pink's book: Drive - The surprising Truth about what motivates us) If we are forgetting to do some class of task then putting it on the board will help us remember.  If we think some small task is being done by someone else, then putting it on the board will validate that someone else is actually doing it.  If a task is obvious, then putting it on the board will take vi…

Elements of an Effective Scrum Task Board

What are the individual elements that make a Scrum task board effective for the team and the leadership of the team?  There are a few basic elements that are quite obvious when you have seen a few good Scrum boards... but there are some other elements that appear to elude even the most servant of leaders of Scrum teams.









In general I'm referring to a physical Scrum board.  Although software applications will replicated may of the elements of a good Scrum board there will be affordances that are not easily replicated.  And software applications offer features not easily implemented in the physical domain also.





Scrum Info Radiator Checklist (PDF) Basic Elements
Board Framework - columns and rows laid out in bold colors (blue tape works well)
Attributes:  space for the total number of stickies that will need to belong in each cell of the matrix;  lines that are not easy eroded, but are also easy to replace;  see Orientation.

Columns (or Rows) - labeled
    Stories
    To Do
    Work In P…

What is your Engagement Model?

What must an Agile Transformation initiative have to be reasonably assured of success?

We "change agents" or Agilist, or Organizational Development peeps, or Trouble Makers, or Agile Coaches have been at this for nearly two decades now... one would think we have some idea of the prerequisites for one of these Transformations to actually occur.  Wonder if eight Agile Coaches in a group could come up with ONE list of necessary and sufficient conditions - an interesting experiment.  Will that list contain an "engagement model"?  I venture to assert that it will not.  When asked very few Agile Coaches, thought leaders, and change agents mention much about employee engagement in their plans, models, and "frameworks".  Stop and ask yourselves ... why?

Now good Organizational Development peeps know this is crucial, so I purposely omitted them from that list to query.

One, central very important aspect of your Agile Transformation will be your Engagement model.