While the terms “A/B testing” and “split testing” are often used interchangeably, they can refer to two distinct methods. More specifically, “split testing” usually means split URL testing (also called URL redirect testing).
Both methods involve showing different versions of content to visitors and measuring which performs better. However, the key difference lies in how those versions are delivered — and it all comes down to the URL.
As mentioned above, A/B testing and split testing (also called split URL testing or URL re-direct testing) are similar practices, but they’re not quite the same.
A/B testing involves experimenting with just a single change to something on your site that keeps the original URL in tact. Split testing, on the other hand, involves directing visitors to entirely different URLs and often involve larger-scale changes.
That means there are two big differences when it comes to A/B testing vs. split testing:
From that fundamental difference, there can be uses for each method that all come down to whether or not a URL change is required to deliver the variant(s) in the test.
Reasons for needing the URL to change for test delivery can include:
In short, any situation that would require a separate URL to be used is a great use case for a split test.
To demonstrate the difference between A/B testing and split testing even further, let’s take a look at a few hypothetical examples to see which technique could be the best option:
| Example | A/B Test or Split Test? |
| Your CTA buttons have always been bright red to help them stand out from the rest of the page. But the color red has a rather negative connotation, so you’re considering switching them to green to see if that encourages more people to click. | A/B test |
| You’re running Meta ads and want to test whether sending shoppers to a product page or a custom landing page leads to more conversions. Each page lives at a different URL. | Split test |
| You want to test out a multi-step checkout process to see if you can increase your average order value, but it’s going to require large scale changes to your theme code. | Either (depends on testing tool) |
| You’ve never given any thought to the order of the products on your collection pages before, but now you’re wondering if listing them from most to least popular would generate more sales than simply going from A to Z. | A/B test |
| If you run an online clothing store, you’ll need to offer sizing information for most visitors to feel comfortable placing an order. But all these details can be somewhat unwieldy, and you don’t know whether you should add this information directly to the page so that it’s especially easy to find or include it as an optional popup in order to help save space. | A/B test |
| You’ve got a redesign of your site navigation that you want to test against your standard nav menu. | Either (depends on testing tool) |
| You’re using a third-party page builder to create custom landing pages. You want to test which variant performs best, but they aren’t native Shopify templates. | Split test |
Ready to get started with an A/B or split URL test of your own?
With the right tools, it’s remarkably easy. For example, Shopify merchants can use Shogun A/B Testing to set up, manage, and measure the results of these experiments with minimal time investment and absolutely no technical expertise required.
First of all, Shogun A/B Testing is available on the Shopify App Store, which means you can install it with just one click (many other A/B testing solutions require you to manually edit your site’s code in order to complete the integration).
Once you’ve downloaded the app, you’ll find that you can create both A/B tests and split URL tests with Shogun A/B Testing.
Whichever type of experiment you choose, Shogun A/B Testing offers advanced features that give you a high degree of control over who sees your tests and how they’re evaluated.
This includes the ability to determine exactly how much traffic goes to the original version of the page and how much goes to the new variant. You should strongly consider sending over more traffic to the original version — that way, if your intuition was wrong and the changes you made to the new variant are actually having a negative effect on user experience, fewer visitors will be affected.
Indeed, there’s always risk involved when you make changes to your online storefront. But by controlling the flow of traffic, you can at least minimize this risk.
And instead of showing your test to everyone who just happens to drop by your store, Shogun A/B Testing allows you to target specific audiences.
For example, if you want to experiment with some potential improvements to your store’s mobile experience, then it would make sense to only run your test for visitors who are using a smartphone or tablet. With Shogun A/B Testing, you can do just that.
Shogun A/B Testing allows you to target your tests based on a variety of other conditions as well, including which site referred the visitor to your store, whether the visitor is new or returning, and the visitor’s physical location. You can even add multiple conditions to the same audience segment if you want — in that case, you can determine whether every condition or only one condition must be met in order for a visitor to be included.
It’s also worth mentioning that Shogun A/B Testing keeps track of many different metrics for your tests. For both the original version and the new variant, you can view the conversion rate, average order value, total sales, and much more. This enables you to look at performance from a variety of angles and pick a winner according to your own unique needs and preferences.
With Shogun A/B Testing, you’ll be able to access the insights you need to make your store as perfect as possible.
