According to Harvard Business Review, ecommerce brands that implement price testing see an average increase of 6% in their gross profits.
How many extra dollars would that translate into for your store?
This comprehensive guide covers everything that you must consider in order to run accurate price tests and access these incredible benefits. Specifically, you’ll learn:
Shopify price testing is the process of showing different prices to different customers in order to see which options will produce the best results for your bottom line.
Shopify does not offer a built-in feature for A/B testing prices, but merchants can still run these experiments by using workarounds such as discounting, sequential price testing, or apps like Shogun.
Shopify doesn’t offer native price testing support. In your store settings, you’ll find that you can only add one price at a time for each product/variant in your catalog.
Given this fact, how are Shopify merchants to access the benefits of price testing?
The best way is to use a third-party Shopify app designed specifically for this purpose. But if you would like to run your own price tests without using an app, there are a couple workarounds available: you could use product duplication or a pre/post test instead.
To run a price test on Shopify without using an app, one option is to set up duplicate products or product variants on your storefront, with the only difference between these duplicates being the price.
The first step is to create your product duplicate:
There are two edits you’ll need to make to this duplicate product page. The first is to change the price, of course. The second is that you must hide this duplicate product from your main catalog — otherwise, it will be easy for customers to stumble across the fact that the same product is available at two different prices. This would cause confusion, and some customers might even begin to lose trust in your brand.
You can change the price in the “Price” section of the settings for your duplicate product. And in the “Status” section of the settings, open the dropdown menu and select “Unlisted”. The duplicate product will then be published but hidden from your main catalog, accessible only by direct link.
Since your duplicate product is unlisted, you’ll need to direct traffic to it yourself. For example, if you’re using an ad platform like Facebook, you could run a campaign that sends prospects straight to the URL for the duplicate product page. For comparison, you should also run a separate campaign that sends prospects to the URL for the original product page as well.
Then, you can analyze the conversion rates of each campaign on your ad platform to see which price performs best — this is how you price test with product duplication.
Summary of Steps for Product Duplication Method
Another price testing workaround that’s available within the Shopify platform is the compare-at price feature, which will show a crossed-out “original” price next to the current price of a product, highlighting that the item is now available at a discount.
With this tool, you could use the “original” price as a baseline and then try out different compare-at prices to get a sense of how much demand you may be able to generate by lowering the price to certain levels.
Of course, discounts aren’t necessary for this kind of testing — you could also just change the base price of the product and see what happens. This is known as a pre/post test, as it involves analyzing the sales performance of a product before and after you change the price.
As mentioned above, you can change the price of a product through the product’s settings page on your Shopify account:
You can then use Shopify’s built-in analytics to review the sales performance for the product both before and after you changed its price in order to determine what kind of effect the price change may have had.
Before you run a price test using the product duplication or pre/post method, you should be aware that both of these techniques come with some significant risks.
We already mentioned that product duplication can create a lot of confusion. While hiding duplicate products from your main catalog and making them only accessible through a direct link will help mitigate this risk, it’s not a perfect solution.
For example, if someone clicks on the direct link to the duplicate product, they might then browse over to your main catalog and see that the product is listed at a different price there.
Customers won’t be able to see the direct link from your main catalog, but they will be able to go to your main catalog from the direct link — this fix works one way but not the other.
Duplicate product listings will also make it more difficult for you to keep track of your inventory and analytics.
And worst of all, creating duplicate products often results in caching issues, where the price that a customer sees listed on the product page doesn’t always match up with the price that’s shown in their cart when they’re finally ready to check out. If the checkout price happens to be higher, it will seem like you’re trying to scam them, and the customer may get so upset that they vow to never shop at your store again.
With pre/post tests, you can ensure that only one price is displayed at a time, so you’ll avoid all the chaos that comes with publishing duplicate products.
But the problem with pre/post tests is that there is no control in these experiments. 100% of your traffic immediately switches over from one price to the other, so even if you manage to address every other potential variable, you’re still always going to be comparing two separate periods of time.
For example, one of your competitors might happen to start running a huge sale right after you begin your experiment, drawing business away from your store and making the post portion of your pre/post test appear worse than it would be under normal conditions.
You may then erroneously conclude that your new prices are hurting your sales, when if you had tested them at any other time you would have found that they actually would have helped.
To avoid these issues, you’ll need to go beyond the default options available on the Shopify platform and install a third-party A/B testing app like Shogun.
Using an A/B testing app will enable you to show some customers the price you want to test while showing everyone else the original price.
For any given customer, it’ll be like there’s only one price available, so there won’t be any of the confusion or frustration that comes with publishing duplicate products. And both of these prices will be published during the exact same period of time, fixing the biggest flaw of pre/post tests.
To summarize, the following table breaks down the advantages and disadvantages of all the different methods for running price tests on Shopify:
| Method | Pros | Cons |
|---|---|---|
| Duplicate products | Natively supported by Shopify | Risk of damaging customer relationships and losing track of inventory and analytics |
| Pre/post tests | Natively supported by Shopify | Often produces misleading results |
| A/B testing app | Most accurate option available | Some additional cost |
To minimize the risks and maximize the potential benefits of your price testing efforts, you should first analyze your competitors and survey your customers.
This will allow you to determine which price points are most worth testing out, so you won’t waste any of your valuable time. And these measures will also help ensure that you can run price tests without damaging your reputation.
Let’s go over exactly what each of these steps entail.
Before you run any tests at all, you’ll need to come up with some kind of hypothesis to start with.
Do you think raising or lowering the price of a given product would be more beneficial? And once you pick a direction to go in, how far should you try moving the price exactly?
You can begin to answer these questions by looking around at what your competitors are doing.
It’s not like the old days when customers would need to get a clerk on the phone or even drive over to a physical brick-and-mortar store in order to see what one of your competitors is charging. There’s much less friction in the online marketplace – consumers can now find this information almost instantly with just a quick search query.
Whether you’re trying to position yourself as a high-end brand, a budget-friendly option, or something in between, you can be confident that most customers are aware of the prices that are available for similar products elsewhere. If your own prices don’t make sense in this context, you’re likely leaving a lot of money on the table.
Identify your top competitors and get a spreadsheet going for keeping track of their prices. This simple exercise can produce all kinds of potentially fruitful hypotheses, such as:
Keep in mind that your hypothesis is just a starting point — the great thing about price testing is that it often leads to unexpected places. By following actual customer behavior rather than just guessing at what your prices should be, you’ll come across ways to make your store more successful that you never would have considered otherwise.
For example, you might assume that raising prices will always lead to a lower number of sales. But in some cases, a higher price will successfully signal to customers that your products are of a higher quality, resulting in not just more profit per sale, but more sales overall as well. The results can be counterintuitive. This is why you test.
In addition to analyzing your competitors, you should also survey your customers before you actually start running any price tests.
This is because it’s important to not go overboard with the prices that you’re testing.
While your price test may be just an experiment, you must always keep in mind that it involves real interactions with your customers. If you go too far, you might alienate them — for example, if an experiment includes a drastic increase to the price of a product, some customers might be so shocked at the difference that they decide to check out your competitors instead. And in that case, who knows if they will ever return?
To prevent such scenarios, you should use a survey to establish a range of prices that most of your customers will find acceptable.
But don’t just straight-up ask your customers what they think the price of a product should be. Customers will often send back “lowball” responses to simple surveys like these, as many will think they can manipulate the situation into some savings for themselves down the line.
Instead, we recommend using the Van Westendorp pricing model. This involves asking customers the following four questions:
This is less direct than simply asking customers what they would pay, which improves the odds that you’ll receive genuine responses rather than people making artificially low suggestions out of self-intertest.
You can use software like Microsoft Excel to chart these responses. The x-axis should include all the prices that came up in the responses, while the y-axis should represent the percentage of respondents who gave each answer. Plot four lines across the graph to represent the percentage of Too Cheap, Acceptably Cheap, Acceptably Expensive, and Too Expensive responses as the price increases.
The Too Cheap and Acceptably Cheap lines will go down as the price increases, while the Acceptably Expensive and Too Expensive lines will go up.
Where do your Too Cheap and Acceptably Expensive lines meet? This is the Point of Marginal Cheapness, representing the absolute lowest point where you should set your price. If your price was any lower than this, then the second-most common response from your customers would be “this is too cheap”.
And where do your Acceptably Cheap and Too Expensive lines meet? This is the Point of Marginal Expensiveness, representing the absolute highest point where you should set your price. If it was any higher than this, then the second-most common response would be “this is too expensive”.
Between the Point of Marginal Cheapness and the Point of Marginal Expensiveness, the most common responses from your customers are “this is acceptably cheap” and “this is acceptably expensive”. Within this range, you will be able to experiment with any single price point without worrying about alienating your customers (or the vast majority of them, anyway).
It should be noted that the Van Westendorp pricing model doesn’t account for how much it costs you to stock the product that you’re testing. You need to turn a profit, of course — if the Point of Marginal Cheapness is below this breakeven point, then you’ll have to adjust and raise the lower bound of your own pricing experiment as high as it needs to be for you to avoid losing money.
Once you’ve analyzed your competitors and surveyed your customers, you’ll finally be ready to run a live price test on your store.
But you must be careful. There are many factors you should consider while setting up and running your price tests, or else you may get misleading results that guide you into making decisions which end up hurting your bottom line rather than helping it.
The following best practices will help you stay on the right track:
The price should be the only thing you change about your product during these tests.
If you were to adjust other product details (product descriptions, product photos, etc.), you won’t know whether it’s the change in price or any of these other changes that is truly driving any difference in performance.
Limiting variables is the way to increase test reliability and confidence for most brands out there.
When setting up your test, avoid times of the year that typically involve special sales or promotions, such as the holiday season, as this could make the results misleading (e.g., if you lower the price of a product in the weeks just before Christmas, you wouldn’t know if a spike in sales should be attributed to the new lower price or the usual holiday bump).
Also, you should ensure that your traffic and marketing mix are consistent and normalized throughout your test duration.
Just as running your experiment during the holiday season would skew the results, so would a big marketing push or new ad campaign that may be driving traffic that is different from your normal audience. Plan tests accordingly.
Another important consideration is that you need to run your test for long enough to achieve a statistically significant result.
The more samples you collect, the more likely it is that the results of your test accurately reflect how your entire customer base will behave.
This effect can actually be measured through a concept known as statistical significance.
Without getting too deep into the math involved, statistical significance represents the likelihood that there is a true causal relationship between two variables (e.g., a product’s price and its conversion rate).
A high level of statistical significance means it is very unlikely that the results of your experiment are just due to chance. Conversely, a low level of statistical significance means that the results may still be somewhat random at this point, so you shouldn’t yet take them into account when making decisions about where to set your prices.
To prevent misleading results, you must wait until you have reached statistical significance before concluding your test.
Most testing experts aim for a statistical significance level of 95%, which indicates there’s a 95% chance that the results of your test truly show what you’re trying to test for and just a 5% chance that they don’t.
The table below goes through a few scenarios of different traffic volumes and conversion rate lifts to give you a better sense of how this all works.
For reference, we’re looking at a 95% statistical significance level here, and “conversion rate lift” refers to a hypothetical raise in conversion rate from the original price to the price that you’re testing (e.g., a conversion rate lift from 3% to 6% means that the original price has a conversion rate of 3% and the new price has a conversion rate of 6%).
| Conversion rate lift | 1,000 visits per variant | 5,000 visits per variant | 10,000 visits per variant |
|---|---|---|---|
| 3% to 6% | Significant | Significant | Significant |
| 3% to 4% | Not significant | Significant | Significant |
| 3% to 3.5% | Not significant | Not significant | Significant |
As you can see, the more dramatic the results are, the less traffic that is needed to conclude your test. But even so, it will usually take thousands of visits to generate a statistically significant result, even in relatively dramatic cases such as the conversion rate doubling from 3% to 6%.
The average Shopify store receives somewhere between 2,000 to 3,000 visits per month. That’s for an entire month, not just one week, and for an entire site, not just one product page — so, looking at the chart above, it’s clear that a lot of stores are going to struggle to generate enough traffic to get statistically significant results in a reasonable amount of time.
Even for stores that get an above-average amount of traffic, in the vast majority of cases you can expect it to take at least three or four weeks to run a price test.
And if your traffic volume is only average, or worse? Well, it should be said that for these types of stores, price testing is probably not the best lever for you to pull right now.
In these cases, merchants should consider limiting their focus to competitor analysis and customer surveys to get more qualitative insight into where their pricing should be. And then, as your business grows, you may be able to return to price testing later.
If you’d like to learn more about test duration, watch our video on the right amount of time to run an A/B test:
As mentioned above, there are several apps available, including Shogun, Shoplift, Mida, and Intelligems and more, that enable you to try out different prices for a product over the same period of time.
That means you can isolate price as the only variable that’s being tested, producing more meaningful results and allowing you to quickly discover which prices will generate the most revenue in the long run.
Apps have various methods for setup when it comes to running price tests.
For example, some price testing apps require code implementations on the theme level, which means you would need to actually go in and manually edit the files that make up your Shopify theme.
And for some apps, the process might be advanced enough to require a developer to perform the implementation.
There are also price testing apps out there that end up embracing the duplicate product paradigm. When a price testing app requires the duplication of products in order to work properly, this can make your catalog quite messy on the backend, and then you’ll need to go through the trouble of undoing all the damage once you’re done testing.
All of this is worth researching before you get started with any app on the Shopify platform.
If you decide to go with Shogun A/B Testing, we have an AI-driven implementation that takes care of the theme duplication and code changes required to the theme in the background.
This takes the complexity and the risk out of the equation when setting up your storefront for price testing.
Below are some other app options on the platform:
| Shopify Apps that Support Price Testing |
|---|
| Shogun |
| Intelligems |
| Shoplift |
| Mida |
| FigPii |
| Eraya |
To demonstrate what it’s like to run a price test using an app, let’s take this step-by-step, using Shogun A/B Testing as an example:
It’s worth noting that Shogun A/B Testing provides two ways for you to depict lowering the price of a product: you can set it up so that the original price is simply replaced by the lower one, or you can use “compare-at” pricing to show the new price next to the old price, with the old price crossed out.
Once your Shogun price test is live, some of your visitors will still see the original price, while others will be shown the new price (depending on the traffic split you set within the test configuration page).
And once a visitor is assigned to a price, they’ll continue to see that same price for as long as the test is running to help prevent any unnecessary confusion.
They will also see the price anywhere the product is displayed: product recommendations, homepage mentions, anywhere. It’s price testing for that product across the entire theme — not just the product page.
You’ll be able to review how the original version and your new variant are performing in real-time according to metrics such as conversion rate, sales volume, etc. Shogun keeps track of statistical significance for you, so you’ll know exactly when you’ve collected enough data to declare a winner for your test.
Also, Shogun stores your test price data separately from your Shopify product data. No products are duplicated — there won’t be any mess to clean up in the backend of your site. And each visitor will see the correct price throughout the storefront for a seamless shopping experience.
You can watch this video for more information about price testing with Shogun:
We’ve now covered everything you need to know in order to run your own price tests.
To help you get started, let’s review a few specific price testing ideas that you could try out for your store:
| Strategy | Pros | Cons |
|---|---|---|
| Localizing your prices by offering higher rates in some areas and lower rates in others | Allows you to target your pricing according to the different income levels and consumer preferences of each area you serve, helping you generate more revenue overall | If customers discover that your pricing isn’t consistent, they may lose trust in your brand |
| Offering a subscription option | Convenient for customers who are likely going to keep ordering the same item anyway, and increases the odds that they will continue to do so | Subscriptions tend to have a lower conversion rate than one-time purchase options, as even if they are easy to cancel they require somewhat more of a commitment |
| Bundling items together at a discounted price | May interest customers looking for a good bargain and increase average order value for your store | Bundles often have a lower conversion rate than offering items separately, as some customers prefer to only buy what they really need |
| Rounding to the nearest whole number | Makes prices easier to process, potentially leading to more impulse buys | Can make items seem much more expensive, even if the price is only going up one cent (e.g., from $9.99 to $10) |
| Ending the price in .99 | Also known as “charm pricing”, this technique can make items seem much more affordable, even if the price is only going down one cent (e.g., from $10 to $9.99) | This strategy is associated with discounts and can make customers perceive your product to be of lesser quality |
To determine whether the advantages will outweigh the disadvantages for any of these tactics, you’ll need to run an A/B test to see how your own particular customers respond.
For more ideas, check out our video on ecommerce price optimization:
If you’re testing out higher prices, it might lead to lower conversion rates, though the increase in your profit margin might offset this effect. While testing out lower prices can usually improve conversion rates, this isn’t always the case — it may make your products appear to be of lesser quality, leading to fewer conversions. So, you’ll need to run your price test first before you can know whether it will hurt your conversion rates. Overall, you can reduce this risk by testing only a portion of your visitors and limiting your test to relatively small changes in price. Keep in mind conversion rates aren’t the only factor to look at. Be sure to look at overall profit to determine the success of a price test.
This will depend on how much traffic your store receives — the faster you’re able to collect samples for your test, the less time it will take to produce statistically significant results. But even stores that generate a fairly high amount of traffic should expect it to take at least three or four weeks to collect enough data.
Profit per visitor is perhaps the most important metric for a price test, as it gets to the heart of what you’re trying to answer with these efforts — where do your prices need to be in order to most help your bottom line? Looking at your profit per visitor, rather than just the total profit, will help adjust for the fact that one price variation may have received more visits than the other. Conversion rate, add-to-cart rate, and average order value are also particularly useful metrics for understanding how your prices affect customer behavior.
Customers should never be able to see two prices listed for the same product at the same time, as this will seem confusing and unfair. By using a price testing app like Intelligems, FigPii, Shogun, or Shoplift, you’ll be able to assign one group of customers to the price you want to test out while everyone else still sees the original price — each customer will only be able to view the single price that they’ve been assigned to. Also, it’s best to stick to incremental changes, as a big jump in price might shock customers into abandoning your store altogether.
It tends to take at least 1,000 sessions per variant to get a statistically significant result from a price test. In many cases, it may take 10,000+ sessions per variant. The more drastic the change in performance, the less traffic you’ll need — and vice versa (e.g., a price change that doubles your conversion rate may only need a couple thousand visits to confirm that it’s the best option, while one that leads to a relatively small improvement will need many more visits than that).
When done properly, price testing is absolutely safe for live traffic. But there are certainly some techniques that you’ll want to avoid — for example, some merchants try to price test by creating duplicate products in their catalog, which can lead to the price on the product page not matching up to the price that’s listed in the customer’s cart when they go to check out. Glitches like these are disastrous for your brand and can lose you customers for good.
Shopify doesn’t offer native support for price testing, so to price test without creating duplicate products you’ll need to install an app like Shogun. Your only other option would be to just change the price of a product and see what happens, but this wouldn’t be a true price test, as you would be comparing prices that were available during separate periods of time — with Shogun, you’ll be able to isolate price as the only variable that changes in your experiments, producing the most accurate and actionable results.
