qvib.pro
RU

A/B hypotheses: what to test and why

What for: turn a funnel bottleneck into a prioritized list of A/B hypotheses with a metric and expected impact — not "let's change the button color".

бесплатно профи

Приём: пошаговый вывод гипотез + приоритизация (CoT) checked 2026-06-01

Updated: 02.07.2026

$ You are a growth analyst who thinks in hypotheses, not in "let's change the c…
A/B hypotheses: what to test and why

When to use it

When you have a metric you want to lift (landing page conversion, email opens, signups) and you need to decide what to test first. The role is a growth analyst. Output: hypotheses in the "if … then … because …" format, each with a metric, an ICE priority and a test design.

The prompt (copy and paste)

You are a growth analyst who thinks in hypotheses, not in "let's change the color". Reason step by step.
WHAT WE'RE IMPROVING: "<metric and where: e.g. landing page to lead conversion>". CURRENT VALUE: <if I know it>. TRAFFIC/VOLUME: <how many people/events per week>.
WHAT WE KNOW ABOUT THE PROBLEM: "<data, complaints, where people drop off, hunches>".

Do this:
1. Name the likely reasons the metric is low (step by step through the funnel).
2. Formulate 5-7 hypotheses in the form "If <we change X>, then <metric goes up>, because <reason>".
3. For each one — what exactly changes (variant B), which metric we watch, the expected effect.
4. Prioritize with ICE (Impact 1-10 · Confidence 1-10 · Ease 1-10), compute the score, sort.
5. Write up the top one as a test: what's A / what's B, the target metric, at what volume and how long to wait (is there enough traffic for significance).

If there isn't enough traffic for an A/B test — say so honestly and suggest an alternative (sequential testing, qualitative interviews).

Filled-in example

Improving: landing page to lead conversion. Current: 2%. Traffic: ~500 visits/week. Known: the hero section is wall-to-wall text, the form has 7 fields, there are no testimonials.

What the AI should come back with: funnel-level reasons; hypotheses like "If we cut the form from 7 fields to 3, conversion goes up, because a long form scares people off"; "If we add 3 testimonials, it goes up because of trust"; "If we rewrite the hero around a single benefit — up"; and so on. Each with a metric and an expected effect; an ICE table with scores; the top one (the short form) written up as a test. Important: at 500 visits/week and a 2% baseline you should get an honest warning — a visible effect will take several weeks, small changes won't reach significance, so start with the strongest one.

Variations

  • Quick wins only. "Sort by Ease — what can I test this week without any development work?"
  • Experiment design. "Calculate the sample size and duration needed to detect a +X% effect."
  • Post-mortem on a losing test. "The test came out flat — why, and what should I check next?"

Pro tips

  • A hypothesis without a "because" isn't a hypothesis, it's a guess. The if/then/because format forces you to lean on a reason instead of taste.
  • Being realistic about traffic is critical: at low volume an A/B test won't detect small effects. The AI should honestly say "not enough" and suggest testing only big swings, or going to qualitative interviews instead.
  • ICE is for prioritization, not truth: the scores are subjective. The value is in sorting the list and starting from the top rather than testing everything for months.

Читать по-русски →