Skip to main content
Disclosure: Unircretraites is reader-supported. If you sign up through our links, we may earn a commission at no extra cost to you. It never changes our recommendations, and we say when a tool is not right for you. How we make money

How to Run Statistically Significant A/B Tests

Decide the goal and the change before you start, split visitors randomly between the two versions, run the test to a length you fixed in advance, and read the result only at the end. Small sites should test bigger changes, because small ones need more visitors than they have.

Free plan, no card needed to open it. Affiliate link: we may earn a commission at no cost to you. How we make money.

U

By Tune (AS_Tune)

Publisher

Illustration of a web page: turn visitors into enquiries
  • Fix it step by step

    7 clear steps, a few hours to set up; two to four weeks to run for a typical site.. Jump to the steps.

  • Free to read

    No sign-up to read this guide or use the free resource at the end.

  • A no-cost place to start

    Our pick is Systeme.io: a free plan with email, funnels and automation. No card needed to open it. See why.

Steps at a glance1Choose one goalChoose one goal2Choose one changeChoose one change3Estimate the visitors you needEstimate the visitors you need4Split traffic randomlySplit traffic randomly5Fix the end dateFix the end date6Read the result at the endRead the result at the end7Record and actRecord and act

The problem

Stopping a test the moment one version leads is the most common mistake. Early differences are often chance, and the result flips if you wait.

A trustworthy test needs enough visitors and a rule for when to stop. Many small sites do not have the traffic for small changes, and that is useful to know.

This guide is for you if

  • You can show two versions of a page to different visitors
  • You have a steady flow of visitors and a clear goal
  • You are willing to wait for the test to finish

It is not for you if

  • You get very few visitors
  • You change the page during the test
  • You want a quick answer from a handful of visits

The steps

A few hours to set up; two to four weeks to run for a typical site.

  1. 1

    Choose one goal

    One number such as form submissions per visitor. Decide it before you look at any data.

  2. 2

    Choose one change

    A headline, a button, a form length. If you change several things you will not know which one worked.

  3. 3

    Estimate the visitors you need

    Use a sample size calculator with your current conversion rate and the smallest improvement that would matter. If the number is more than you can get in a month, test a bigger change.

  4. 4

    Split traffic randomly

    Half see the current version and half see the new one at the same time. Use your testing tool's split, not alternate days.

  5. 5

    Fix the end date

    Choose it from the sample size and run whole weeks. Do not stop early because one version looks ahead.

  6. 6

    Read the result at the end

    Most tools show a significance level; 95% is a common convention. If the result is not clear, treat it as no difference.

  7. 7

    Record and act

    Keep the winner, note what you learned, and test the next idea. A test that shows no difference still teaches you that the change does not matter.

Finished the steps? Put them to work.

Free plan, no card needed to open it.

Common mistakes

  • Stopping when one version leads.
  • Testing several changes at once.
  • Running tests with too few visitors.
  • Changing the page mid-test.

Our recommendation

Systeme.io: our pick for solo creators and small businesses starting out

Why

  • Free plan with funnels, email, a course and a blog; no credit card needed.
  • Paid plans start at $17/month when you outgrow 2,000 contacts.
  • Funnels, email automation and checkout live in one account.

Not right for you if

  • You need two-way SMS, call tracking or a shared team inbox.
  • You manage several client or location accounts.
  • You need advanced CRM pipelines and reporting.

Price: Free plan (2,000 contacts); Startup $17/month, Webinar $47/month, Unlimited $97/month. Source, checked 27 Oct 2026.

Alternative: If you need SMS, calls, booking and reviews for a local service business or agency, HighLevel fits better (14-day trial).

Disclosure: Unircretraites is reader-supported. If you sign up through our links, we may earn a commission at no extra cost to you. It never changes our recommendations, and we say when a tool is not right for you. How we make money

A free resource for this

Landing page test plan. Free to use, no sign-up.

How to Run Statistically Significant A/B Tests: questions

That the difference is unlikely to be chance, by a threshold you chose in advance. It does not mean the difference is large or important.
Long enough to reach the sample size you calculated, in whole weeks. Decide before you start.
It depends on your conversion rate and the size of difference you want to detect. Use a sample size calculator.
Yes, with bigger changes and longer tests. Or use customer interviews when traffic is too low.