What part of the uplift is truly due to the change?

That is the question every leader should ask before celebrating a sales spike or buying a tool that promises miracles. A movement observed after an action is not, by itself, proof of effect. Insee explains that impact evaluation relies on experimental and quasi-experimental methods precisely to separate causal effects from statistical association. NIST makes the same point: a third factor can push two variables in the same direction without one causing the other.

The 4 questions to answer before you measure

Write down four items before looking at the dashboard. First: what exactly changed — price, offer, script, email sequence, channel, geography, or timing? Second: which metric drives the decision — conversion rate, average order value, margin, repeat purchase, churn, or sales cycle length? Third: what comparison window are you using — before/after, week over week, month over month, or across several cycles? Fourth: what could distort the reading — seasonality, competitor promotions, product mix, stock constraints, rep effects, or the return of a large account?

  • Choose one primary decision metric.
  • List plausible external factors before seeing the results.
  • Match the observation window to your sales cycle.

The most robust SME method: compare against a control

CNIL defines A/B testing as randomly assigning users to two versions of the same proposition and measuring the effect on a predefined metric. For an SME, the strongest practical idea is straightforward: keep a control group as similar as possible to the exposed group, then compare the difference in their trajectories. If you cannot randomize, Insee recommends quasi-experimental approaches, provided you document likely bias and avoid overstating the conclusion.

  • If possible, randomize customers, leads, or locations.
  • If not, choose a close control and explain why it is close.
  • Measure the difference-in-differences, not just the raw gap.

Fictional numerical example: a campaign that looks magical until the control is added

Fictional example. A B2B SME rolls out a new sales script in March across 200 prospects. The meeting-booking rate rises from 12% to 15%. Without a control, you might claim a 3-point gain. But during the same period, a comparable untreated segment rises from 11% to 14%. The broader market improvement is therefore also 3 points. The net effect attributable to the script is close to zero. Practically, the change may have captured a favorable market wave, or a seasonality effect. The right reading is not “it was useless,” but “we do not have solid evidence of a unique effect.”

  • Absolute uplift without a control can mislead.
  • A raw gain is not a causal effect.
  • No measured effect is better than a false win.

Limits, compliance, and methodological honesty

Every measurement has limits. Small samples are unstable. Effects can vary by rep, region, customer type, or season. CNIL stresses data minimization and, for experiments and surveys using personal data, the need for appropriate safeguards when processing may create risk to individuals. If you use identifiable data to test a commercial change, collect only what you need, anonymize as early as possible, and check whether a DPIA is required when risk is high.

  • Do not claim a proven ROI if the method cannot support it.
  • Document exclusions, incidents, and context changes.
  • Accept that a successful test proves success in a specific context, not everywhere.

Seven-day action checklist

Day 1 — Owner: sales leadership. Observable result: one decision question and one primary metric written on a single page. Day 2 — Owner: controller or CRM owner. Observable result: test population and control population defined. Day 3 — Owner: sales manager. Observable result: a list of external factors is approved before launch. Day 4 — Owner: field team. Observable result: the change is deployed identically in the test group. Day 5 — Owner: analyst or ops. Observable result: a simple table shows test/control gaps day by day. Day 6 — Owner: leadership. Observable result: a brute effect / net effect / limits note is drafted. Day 7 — Owner: sales leadership. Observable result: a stop, scale, or retest decision is made and archived with rationale.

  • Keep a pre-results hypothesis note.
  • Reject conclusions without a comparison group.
  • Decide cautiously, not emotionally.