Test Design Techniques

Which tests to write, and how many

Writing a test is easy. Deciding which tests to write — and knowing when you've written enough — is the senior skill. Six formal techniques, with worked examples and Playwright snippets, that turn “test everything” into “test exactly what matters”.

The coverage question: “is 10 tests enough?” has no answer. “Do 10 tests cover every equivalence class and every boundary value?” has a yes or no answer. These techniques give you the vocabulary.

1. Equivalence Partitioning

Group inputs into classes where any member behaves identically. Test one per class — not ten.

When to reach for it

Input validation, range checks, categorical dropdowns, state-dependent behaviour.

How many tests

One representative per class. 3 classes → 3 tests. Resist the urge to 'just add a few more'.

Worked example

Sign-up age field (accepts 18-100):
  Invalid below  — age < 18   → reject
  Valid          — 18 ≤ age ≤ 100 → accept
  Invalid above  — age > 100   → reject
  Malformed      — non-numeric → reject

4 partitions → 4 tests (one per class).
NOT 40 tests with ages 1, 2, 3… 100.

Playwright snippet

// One test per equivalence class
test.each([
  { age: 10,  expected: 'reject', class: 'invalid-below' },
  { age: 25,  expected: 'accept', class: 'valid' },
  { age: 150, expected: 'reject', class: 'invalid-above' },
  { age: 'x', expected: 'reject', class: 'malformed' },
])('age=$age ($class) → $expected', async ({ age, expected }) => {
  await page.getByLabel(/age/i).fill(String(age));
  await page.getByRole('button', { name: /submit/i }).click();
  if (expected === 'accept') {
    await expect(page.getByText(/welcome/i)).toBeVisible();
  } else {
    await expect(page.getByRole('alert')).toBeVisible();
  }
});

Common pitfall: Equivalence partitioning assumes homogeneity WITHIN a class. Always pair with BVA — the boundary is where partitions lie, and classes rarely behave the same at edges.

2. Boundary Value Analysis (BVA)

Bugs cluster at boundaries. Test min, min+1, max, max-1 — and one value on each side of the boundary.

When to reach for it

Any numeric range, string-length validator, collection size, pagination, timeout thresholds.

How many tests

2 values per boundary (just-inside, just-outside). Two boundaries (min, max) = 4 BVA tests on top of equivalence.

Worked example

Password length: must be 8-20 characters.
  Boundary: 8    (just inside, min)     → accept
  Boundary: 7    (just outside, min-1)  → reject
  Boundary: 20   (just inside, max)     → accept
  Boundary: 21   (just outside, max+1)  → reject
  Also worth: 0 (empty), 9999 (extreme)

Playwright snippet

const boundaries = [
  { len: 7,   expected: 'reject', edge: 'min-1' },
  { len: 8,   expected: 'accept', edge: 'min' },
  { len: 20,  expected: 'accept', edge: 'max' },
  { len: 21,  expected: 'reject', edge: 'max+1' },
];

test.each(boundaries)('password length=$len ($edge)', async ({ len, expected }) => {
  const pwd = 'x'.repeat(len);
  await page.getByLabel(/password/i).fill(pwd);
  await page.getByRole('button', { name: /register/i }).click();
  if (expected === 'accept') {
    await expect(page.getByText(/account created/i)).toBeVisible();
  } else {
    await expect(page.getByText(/password length/i)).toBeVisible();
  }
});

Common pitfall: Inclusive vs exclusive boundaries. 'Age ≥ 18' and 'Age > 17' describe the same rule — but the bug often lives in which one the code actually uses. Always test with the documented inclusive value.

3. Decision Tables

Enumerate every combination of conditions → actions. Builds comprehensive coverage for rule-based logic.

When to reach for it

Discount rules, authorisation matrices, feature flags, pricing, eligibility checks — anywhere the outcome depends on multiple independent conditions.

How many tests

2^n rows for n boolean conditions. Collapse 'don't care' cells to keep it tractable (n=4 → 16 rows; n=6 → 64, usually too many — partition further).

Worked example

Free shipping rules:
┌─────────────────┬──┬──┬──┬──┬──┬──┬──┬──┐
│ Cart > $50      │ F│ F│ F│ F│ T│ T│ T│ T│
│ Member          │ F│ F│ T│ T│ F│ F│ T│ T│
│ Coupon applied  │ F│ T│ F│ T│ F│ T│ F│ T│
├─────────────────┼──┼──┼──┼──┼──┼──┼──┼──┤
│ Free shipping?  │ F│ F│ F│ T│ T│ T│ T│ T│
└─────────────────┴──┴──┴──┴──┴──┴──┴──┴──┘

8 rows = 8 tests covering every combination.

Playwright snippet

const rules: Array<[boolean, boolean, boolean, boolean]> = [
  // [cartOver50, member, coupon, expectFreeShipping]
  [false, false, false, false],
  [false, false, true,  false],
  [false, true,  false, false],
  [false, true,  true,  true ],   // member + coupon = free ship
  [true,  false, false, true ],
  [true,  false, true,  true ],
  [true,  true,  false, true ],
  [true,  true,  true,  true ],
];

test.each(rules)(
  'cartOver50=%s member=%s coupon=%s → freeShip=%s',
  async (cartOver50, member, coupon, expect_) => {
    await setupCart({ over50: cartOver50, member, coupon });
    const shipping = await page.getByTestId('shipping-cost').textContent();
    if (expect_) expect(shipping).toBe('FREE');
    else expect(shipping).not.toBe('FREE');
  },
);

Common pitfall: Conditions that look independent often aren't. 'Coupon' and 'Member' may share a backend field in production, so row 4 above might give unexpected results. Verify your conditions are truly orthogonal before collapsing rows.

4. State Transition Testing

Model the system as a finite state machine. Test every valid edge AND every invalid one you want to block.

When to reach for it

Order lifecycle, auth flows, workflow approvals, subscription states, document review — anywhere entities move through defined states.

How many tests

Minimum: every edge once (minimum coverage). Better: every pair of edges (0-switch / 1-switch), catching chained-transition bugs.

Worked example

Order states: pending → paid → shipped → delivered
                                        ↓
                                      refunded

Valid transitions (cover each once):
  pending  → paid
  paid     → shipped
  shipped  → delivered
  paid     → refunded
  shipped  → refunded

Invalid transitions (must all return 400/409):
  pending  → shipped      (skipped payment)
  delivered → paid        (already delivered)
  refunded → shipped      (cannot reship refunded)

Playwright snippet

// Cover every VALID edge as a positive test
const validTransitions = [
  ['pending',  'paid'],
  ['paid',     'shipped'],
  ['shipped',  'delivered'],
  ['paid',     'refunded'],
  ['shipped',  'refunded'],
] as const;

test.each(validTransitions)('valid: %s → %s', async (from, to) => {
  const orderId = await createOrderInState(from);
  const res = await request.post(`/api/orders/${orderId}/transition`, { data: { to } });
  expect(res.status()).toBe(200);
});

// Cover every INVALID edge as a negative test (one per illegal combination)
const invalidTransitions = [
  ['pending',   'shipped'],
  ['delivered', 'paid'],
  ['refunded',  'shipped'],
];

test.each(invalidTransitions)('invalid: %s → %s rejected', async (from, to) => {
  const orderId = await createOrderInState(from);
  const res = await request.post(`/api/orders/${orderId}/transition`, { data: { to } });
  expect([400, 409]).toContain(res.status());
});

Common pitfall: Missing the 'from any state' transitions — e.g. cancellation from any non-terminal state. A state machine drawn in a meeting almost always misses 20 % of real edges. Reconstruct it from the code, not from memory.

5. Pairwise / Combinatorial Testing

For N parameters with M values each, generate the smallest set of tests that covers every PAIR of values — not every triple.

When to reach for it

Browser × OS × locale × feature-flag matrices. When full cross-product is impractical but every pair must be validated.

How many tests

Orders of magnitude fewer than the full cross-product. 4 params × 4 values full = 256 combos; pairwise = typically 16–20.

Worked example

Full cross-product:
  browsers=4 × os=3 × locales=5 × flags=2 = 120 combos

Pairwise covers every browser×os pair, every browser×locale pair,
every os×flag pair, etc — in ~20 rows.

Tools: PICT (Microsoft), AllPairs (Python), ACTS (NIST).
Example pairwise output (abbreviated):
  chrome  macOS  en-US  on
  chrome  linux  pt-BR  off
  firefox win11  en-US  off
  firefox macOS  de-DE  on
  safari  macOS  pt-BR  off
  safari  linux  fr-FR  on
  ...

Playwright snippet

// Use a library (e.g. @fast-check/pairwise) or a generated CSV
import pairwise from './pairwise-generator';

const matrix = pairwise({
  browser: ['chrome', 'firefox', 'safari'],
  viewport: ['mobile', 'tablet', 'desktop'],
  locale: ['en-US', 'de-DE', 'ja-JP'],
  theme: ['light', 'dark'],
});

test.each(matrix)(
  '$browser × $viewport × $locale × $theme',
  async ({ browser, viewport, locale, theme }) => {
    await setupEnv({ browser, viewport, locale, theme });
    await page.goto('/');
    await expect(page.getByRole('heading', { level: 1 })).toBeVisible();
  },
);

// Full matrix would be 3×3×3×2 = 54 tests.
// Pairwise covers the same bug-finding in ~9 tests.

Common pitfall: Pairwise catches interactions between 2 parameters. Bugs caused by 3-way interactions (e.g. only fails on safari + mobile + rtl-locale) slip through — if you suspect them, use 3-wise (t-way with t=3) or add targeted scenarios.

6. Exploratory Charters

Time-boxed undirected testing guided by a mission statement. Not scripted — but structured enough to reproduce.

When to reach for it

New feature just shipped. You don't know the bugs yet — scripted tests can only find what you already suspect.

How many tests

30–90 min per session. 3–5 sessions cover a release. Keep a log: mission, observations, questions, bugs found, tests to scriptify.

Worked example

Charter format:
┌──────────────────────────────────────────────────┐
│ MISSION: Investigate the checkout flow           │
│ WITH: Chrome DevTools throttling                 │
│ TO DISCOVER: race conditions or UI glitches      │
│ TIME-BOX: 60 minutes                             │
├──────────────────────────────────────────────────┤
│ OBSERVATIONS                                     │
│ - Clicking Pay twice sometimes creates 2 orders  │
│ - Address form loses state on back button        │
│ - Success page shows $0 total when throttled     │
├──────────────────────────────────────────────────┤
│ BUGS FILED: PROD-1042, PROD-1043                 │
│ TESTS TO AUTOMATE: double-submit guard,          │
│                     form restoration             │
└──────────────────────────────────────────────────┘

Playwright snippet

// Exploratory sessions produce CANDIDATES for automation.
// After the session, convert observations into scripted tests:

test('double-click Pay does not create two orders (found in exploratory)', async ({ page, request }) => {
  await page.goto('/checkout');
  await fillCheckoutForm(page);

  const payButton = page.getByRole('button', { name: /pay/i });
  await Promise.all([payButton.click(), payButton.click()]);

  // Assert only one order was created
  const orders = await request.get('/api/orders/me').then((r) => r.json());
  expect(orders.data.filter((o) => o.status === 'pending')).toHaveLength(1);
});

Common pitfall: Exploratory is not 'ad hoc poking'. Without a charter, it becomes unreproducible and un-summarisable. Always write down the mission BEFORE the session and notes DURING — reconstruction from memory misses half the findings.

← Back to all Labs

Which tests to write, and how many

Sign-up age field (accepts 18-100): Invalid below — age < 18 → reject Valid — 18 ≤ age ≤ 100 → accept Invalid above — age > 100 → reject Malformed — non-numeric → reject 4 partitions → 4 tests (one per class). NOT 40 tests with ages 1, 2, 3… 100.

// One test per equivalence class test.each([ { age: 10, expected: 'reject', class: 'invalid-below' }, { age: 25, expected: 'accept', class: 'valid' }, { age: 150, expected: 'reject', class: 'invalid-above' }, { age: 'x', expected: 'reject', class: 'malformed' }, ])('age=$age ($class) → $expected', async ({ age, expected }) => { await page.getByLabel(/age/i).fill(String(age)); await page.getByRole('button', { name: /submit/i }).click(); if (expected === 'accept') { await expect(page.getByText(/welcome/i)).toBeVisible(); } else { await expect(page.getByRole('alert')).toBeVisible(); } });

Password length: must be 8-20 characters. Boundary: 8 (just inside, min) → accept Boundary: 7 (just outside, min-1) → reject Boundary: 20 (just inside, max) → accept Boundary: 21 (just outside, max+1) → reject Also worth: 0 (empty), 9999 (extreme)

Free shipping rules: ┌─────────────────┬──┬──┬──┬──┬──┬──┬──┬──┐ │ Cart > $50 │ F│ F│ F│ F│ T│ T│ T│ T│ │ Member │ F│ F│ T│ T│ F│ F│ T│ T│ │ Coupon applied │ F│ T│ F│ T│ F│ T│ F│ T│ ├─────────────────┼──┼──┼──┼──┼──┼──┼──┼──┤ │ Free shipping? │ F│ F│ F│ T│ T│ T│ T│ T│ └─────────────────┴──┴──┴──┴──┴──┴──┴──┴──┘ 8 rows = 8 tests covering every combination.

const rules: Array<[boolean, boolean, boolean, boolean]> = [ // [cartOver50, member, coupon, expectFreeShipping] [false, false, false, false], [false, false, true, false], [false, true, false, false], [false, true, true, true ], // member + coupon = free ship [true, false, false, true ], [true, false, true, true ], [true, true, false, true ], [true, true, true, true ], ]; test.each(rules)( 'cartOver50=%s member=%s coupon=%s → freeShip=%s', async (cartOver50, member, coupon, expect_) => { await setupCart({ over50: cartOver50, member, coupon }); const shipping = await page.getByTestId('shipping-cost').textContent(); if (expect_) expect(shipping).toBe('FREE'); else expect(shipping).not.toBe('FREE'); }, );

Order states: pending → paid → shipped → delivered ↓ refunded Valid transitions (cover each once): pending → paid paid → shipped shipped → delivered paid → refunded shipped → refunded Invalid transitions (must all return 400/409): pending → shipped (skipped payment) delivered → paid (already delivered) refunded → shipped (cannot reship refunded)

// Cover every VALID edge as a positive test const validTransitions = [ ['pending', 'paid'], ['paid', 'shipped'], ['shipped', 'delivered'], ['paid', 'refunded'], ['shipped', 'refunded'], ] as const; test.each(validTransitions)('valid: %s → %s', async (from, to) => { const orderId = await createOrderInState(from); const res = await request.post(`/api/orders/${orderId}/transition`, { data: { to } }); expect(res.status()).toBe(200); }); // Cover every INVALID edge as a negative test (one per illegal combination) const invalidTransitions = [ ['pending', 'shipped'], ['delivered', 'paid'], ['refunded', 'shipped'], ]; test.each(invalidTransitions)('invalid: %s → %s rejected', async (from, to) => { const orderId = await createOrderInState(from); const res = await request.post(`/api/orders/${orderId}/transition`, { data: { to } }); expect([400, 409]).toContain(res.status()); });

Full cross-product: browsers=4 × os=3 × locales=5 × flags=2 = 120 combos Pairwise covers every browser×os pair, every browser×locale pair, every os×flag pair, etc — in ~20 rows. Tools: PICT (Microsoft), AllPairs (Python), ACTS (NIST). Example pairwise output (abbreviated): chrome macOS en-US on chrome linux pt-BR off firefox win11 en-US off firefox macOS de-DE on safari macOS pt-BR off safari linux fr-FR on ...

// Use a library (e.g. @fast-check/pairwise) or a generated CSV import pairwise from './pairwise-generator'; const matrix = pairwise({ browser: ['chrome', 'firefox', 'safari'], viewport: ['mobile', 'tablet', 'desktop'], locale: ['en-US', 'de-DE', 'ja-JP'], theme: ['light', 'dark'], }); test.each(matrix)( '$browser × $viewport × $locale × $theme', async ({ browser, viewport, locale, theme }) => { await setupEnv({ browser, viewport, locale, theme }); await page.goto('/'); await expect(page.getByRole('heading', { level: 1 })).toBeVisible(); }, ); // Full matrix would be 3×3×3×2 = 54 tests. // Pairwise covers the same bug-finding in ~9 tests.

Charter format: ┌──────────────────────────────────────────────────┐ │ MISSION: Investigate the checkout flow │ │ WITH: Chrome DevTools throttling │ │ TO DISCOVER: race conditions or UI glitches │ │ TIME-BOX: 60 minutes │ ├──────────────────────────────────────────────────┤ │ OBSERVATIONS │ │ - Clicking Pay twice sometimes creates 2 orders │ │ - Address form loses state on back button │ │ - Success page shows $0 total when throttled │ ├──────────────────────────────────────────────────┤ │ BUGS FILED: PROD-1042, PROD-1043 │ │ TESTS TO AUTOMATE: double-submit guard, │ │ form restoration │ └──────────────────────────────────────────────────┘

// Exploratory sessions produce CANDIDATES for automation. // After the session, convert observations into scripted tests: test('double-click Pay does not create two orders (found in exploratory)', async ({ page, request }) => { await page.goto('/checkout'); await fillCheckoutForm(page); const payButton = page.getByRole('button', { name: /pay/i }); await Promise.all([payButton.click(), payButton.click()]); // Assert only one order was created const orders = await request.get('/api/orders/me').then((r) => r.json()); expect(orders.data.filter((o) => o.status === 'pending')).toHaveLength(1); });