Two students test the same plant fertiliser and get opposite results. The difference usually isn't the fertiliser — it's how carefully the experiment was designed. This article covers what actually separates a fair test from a flawed one.
Running an experiment is the easy part. Running one whose result can actually be trusted is much harder — and the difference almost always comes down to design choices made before a single measurement is taken. This article brings together variables, hidden threats to fairness, and repeat trials into one practical guide to building an experiment worth believing.
Every fair experiment is built around three types of variable working together.
| Variable | Definition | Example (Fertiliser Experiment) |
|---|---|---|
| Independent | The one factor deliberately changed | Amount of fertiliser applied |
| Dependent | The outcome being measured | Plant height after two weeks |
| Controlled | Factors kept identical across all groups | Soil type, water, sunlight, pot size, plant species |
A confounding variable is a factor that was not controlled and could have influenced the result — making it impossible to know whether the independent variable actually caused the outcome, or something else did.
If the fertilised plants are placed on a sunny windowsill and the unfertilised plants are placed in a dim corner, sunlight becomes a confounding variable. Even if the fertilised plants grow taller, there is no way to know whether the fertiliser worked, the extra sunlight worked, or both — the test has stopped being fair.
A single result could simply be chance — a slightly warmer room, a slightly heavier seed, an ordinary bit of random variation. Repeating an experiment several times and averaging the results (using the mean, covered elsewhere in this series) makes it far less likely that a fluke result gets mistaken for a real effect.
These two words are often confused, but they check different things.
| Term | Question It Answers | Example |
|---|---|---|
| Reliability | Do repeated trials give consistent results? | A stopwatch that gives the same time for the same event, every time |
| Validity | Does the test actually measure what it claims to measure? | A "memory test" that accidentally tests reading speed instead |
A test can be reliable without being valid — consistently measuring the wrong thing is still consistent. Genuinely trustworthy results need both.
Technology companies commonly use a method called A/B testing to decide between two versions of a website — for example, testing two different button colours to see which leads to more people signing up. Visitors are randomly split into two groups (the independent variable), sign-up rate is measured (the dependent variable), and every other part of the page is kept identical (the controlled variables). Crucially, this is run across thousands of visitors rather than just a handful, for exactly the same reason a school experiment needs repeat trials: a small sample can produce a result that looks meaningful but is really just chance.
The actual measuring is often the smallest part of good experimental work. Deciding what to control, how many times to repeat a test, and whether the results can be trusted is where a fair experiment is really won or lost — long before any data is collected.
10 questions. Select an answer for each, then submit to see your score instantly.