For years, A/B testing on Shopify meant one thing: pay for an app. $50 to $500+ a month for tools that injected JavaScript into your storefront, swapped content in the visitor's browser, and, irony of ironies, slowed down the very store they were supposed to optimize.
That era is over for a whole class of tests. Shopify's Rollouts feature, announced in the Winter '26 Edition and expanded through the Spring '26 Edition in June 2026, builds scheduling, gradual publishing, and A/B testing directly into the admin. For theme-level split tests, the third-party tool is now optional.
Here's what it actually does, what it can't do, and how to run your first test this week.
What Rollouts is
A rollout is a scheduled set of changes to your published theme, and on eligible plans, your checkout and customer account configuration. Three capabilities in one feature:
- Scheduling. Prepare a seasonal update or campaign theme in advance and publish it at an exact date and time. BFCM banners going live at midnight without a human clicking publish.
- Gradual publishing. Ship a change to 10% of traffic first. If conversion holds, ramp to 25%, 50%, 100%. If it tanks, pull it back before most visitors ever saw it.
- A/B testing. Send a percentage of traffic to a modified copy of your theme and compare conversion rate and revenue against the original.
The mechanics are clean. When you create a rollout, Shopify copies your published theme. You edit the copy in the normal theme editor; your live store is untouched, and you can keep working on it while the test runs. Experiments are mutually exclusive, so parallel tests don't contaminate each other's visitors.
The important architectural difference: this runs server-side. Traditional testing apps work by injecting scripts that rewrite the page after it loads. That's where the flicker comes from: visitors briefly see version A before it flips to version B. It's also why testing tools consistently show up in speed audits as some of the heaviest scripts on a store. Rollouts adds no client-side weight. The visitor is simply served one version or the other.
What it costs, honestly
There's no add-on fee. Rollouts is part of your subscription, but it's not fully available on every plan. Here's the actual breakdown:
- Scheduling and gradual publishing work on Basic and up.
- Traffic-split experiments with analytics (the actual A/B testing) need the Grow plan or higher.
- Market-specific targeting needs Advanced or Plus.
A note if you've read conflicting guides: Shopify renamed its plan ladder in 2026, and "Grow" is the new name for the old mid-tier plan. Older write-ups saying you need Advanced to run experiments are working from the outdated names. Shopify's Help Center is the source of truth: experiments are available on Grow and up.
How to run your first test
- Create the rollout. In your admin, find Rollouts next to your published theme (you'll also see it surfaced under Markets). Create a rollout, name it, set a traffic percentage.
- Change one thing. Click through to the theme editor and modify the rollout copy. One meaningful change: a new hero, a restructured product page, simplified navigation. If you change five things and conversion moves, you've learned that something worked, which is the same as learning nothing.
- Start the split small. 10% is a sensible opening allocation to confirm nothing is broken, then move toward an even split. An uneven split takes longer to reach a verdict on the variant side.
- Wait for real data. This is where most first tests die. A rough rule of thumb is at least 1,000 visitors per variant before a result means anything. For many stores that's a week or more. Ending a test after two good days is reading tea leaves. And if your store doesn't have the traffic to fill a sample, don't test at all: apply proven patterns to low-traffic pages and save experiments for the pages where visitors and money actually are.
- Ship the winner, kill the loser, repeat. Shopify tracks conversion rate and revenue per version. When you have a verdict, publish the winner and start the next test.
What Rollouts can't do
Be clear-eyed about the limits, because they're real:
- No price or discount testing. Prices aren't theme data, so they're outside what Rollouts splits.
- No audience segmentation. You can't target mobile vs. desktop, new vs. returning, or specific geographies. The split is random.
- No custom conversion goals. You get Shopify's standard metrics.
- Global theme settings apply to both versions. Typography and color-scheme changes at the settings level can't be isolated to one variant.
- The unit of a test is a whole theme version. It's not an element-level visual editor.
If your testing program lives on price elasticity experiments, granular segmentation, or custom event tracking, your paid tool still earns its subscription. Keep it. Run Rollouts alongside it for structural theme tests. Just never run an app test and a rollout on the same page at the same time, or the treatments will collide and muddy both results.
The honest take
Most Shopify stores have never run a single A/B test. Not because merchants don't care about conversion, but because the tooling cost $50 to $500 a month plus setup time, and that math never penciled for a store doing $20k/month.
That excuse is gone. The single biggest reason stores didn't test has been deleted from the equation. You don't need an agency to run your first theme test. The steps above are genuinely all of it.
Where it gets harder: knowing what to test first, reading results without fooling yourself, and building a testing cadence that compounds instead of a one-off experiment that gets forgotten. That's judgment, not tooling.
Frequently asked questions
Which Shopify plan do you need for Rollouts experiments?
Experiments (the actual A/B testing with analytics) require the Grow plan or higher. Scheduling and gradual publishing work on Basic and up. Market-specific targeting requires Advanced or Plus. If you've read that experiments need the Advanced plan, that guide predates Shopify's 2026 plan rename: Grow is the new name for the old mid-tier plan.
Is Shopify Rollouts free?
There's no separate fee or per-test charge. Rollouts is included in your Shopify subscription, with features unlocking by plan tier as described above.
Can Shopify Rollouts test prices or discounts?
No. Prices aren't theme data, so they sit outside what Rollouts can split. If price testing is core to your program, a dedicated third-party tool still earns its subscription.
Does Rollouts slow down my store like testing apps do?
No. Rollouts runs server-side: each visitor is simply served one theme version or the other. There's no injected JavaScript, no added page weight, and no flicker, which is the main speed complaint against traditional testing apps.
How long should I run my first A/B test?
Until each variant has seen roughly 1,000 visitors at minimum. For most stores that's at least a week. Ending a test after two good days gives you noise, not a result.
Can I run a Rollouts experiment and a testing app at the same time?
Not on the same pages. The two treatments will collide and contaminate both results. If you keep a paid tool for price or segmentation tests, keep its experiments on different pages than your active rollout.
Facing this on your store and not sure where the money is leaking? Reach out. We'll take a look and tell you honestly what's worth testing first, and whether you even have the traffic to test at all.