Squoosh AI
  • Blog
  • Partners
Sign inBook a demo→
Squoosh AI
  • Blog→
  • Partners→
  • Sign in→
Book a demo→
← BlogInsightsNovember 3, 2026

Why bundling changes into one test backfires

Combining five changes into one test saves a slot and hides what worked. Here is why, and how to test changes apart without waiting for more traffic.

ST
Squoosh Team
Insights
2 min read
Why bundling changes into one test backfires

When the queue is long, the tempting move is to combine. Redo the headline, shorten the form, move the reviews, change the button, and call it one test. It uses one slice of traffic, and it feels efficient.

It is the least efficient way to learn.

A bundle hides what happened

A test of five changes gives one answer. If it wins, you do not know which change won, or whether one of them held it back. If it loses, one good change may be buried under a bad one.

Here is a plain example. One change lifts conversion by 6%. Another lowers it by 5%. Together they move it by less than 1%, a result most live tests cannot tell from no change at all. The team files it under "didn't work" and drops both ideas, including the good one.

You still cannot test them one by one

The reason teams bundle is the arithmetic in You have more test ideas than traffic: every live test costs weeks of visitors, and testing five changes separately takes five places in the queue.

Try them apart, then test what matters

Give the village each change as its own version. Every version meets the same villagers, so the ranking shows which changes do well on their own. You can try the bundle too, and see whether it beats its best part.

Then spend live traffic where it pays. If one change leads clearly, test that one live. The others wait their turn, and the changes that did poorly never use a place at all.

One live test then answers one question, and the ranking tells you which question to ask first.

Keep exploring

  • What is synthetic A/B testing?
  • See the Squoosh AI benchmark
  • You have more test ideas than traffic
  • Squoosh Bench: UX-1 on 318 real A/B tests

The Squoosh Dispatch

New research,
straight to your inbox.

Deep dives, release notes, and benchmarks. No promos, no tracking beyond the clicks.

Squoosh AISquoosh AI

We simulate how humans interact with digital experiences, so you can see what works before it goes live.

Product

  • Overview
  • How it works
  • Benchmark
  • Your data

Company

  • Partners
  • Careers
  • Press

Resources

  • Blog
  • Documentation
  • Support

Legal

  • Terms
  • Privacy
  • Trust center
© 2026 Squoosh, Inc.