Running A/B tests is one thing. Extracting valuable insights from these experiments — not to mention actually acting on them — is quite another.
The old, familiar way is to look at your data through a dashboard.
Dashboards are designed for one test in one store. That’s it.
When you’re dealing with a large portfolio of tests and storefronts, the single view dashboard starts to break down.
In these cases, you would need to hop around between many different dashboards, download the raw results data, and try to centralize everything in a spreadsheet somewhere. Then you’d have to try to make sense of it, and figure out what to do.
That’s hours of work right there.
Interpreting it and then acting on the results adds even more work on your plate (which is already full).
All of that changes now.
With Shogun A/B Testing’s new MCP connector, you can both analyze and manage your experiments through your favorite AI tool, such as Claude, Gemini, or ChatGPT. Instead of spending hours scrutinizing dashboards and spreadsheets, you’ll be able to find the answers you’re looking for by simply asking.
Synthesize results across an entire portfolio of stores, prove the revenue impact of a testing program, and help you hand specific insights to other stakeholders. You can be the hero your organization needs to fully take advantage of all the insights that your optimization program is generating.
Below, we’ll go over everything you need to know about this powerful new feature, including:
MCP stands for Model Context Protocol — this is an open standard created by Anthropic that allows AI assistants to securely connect to external tools and data sources.
The Shogun MCP server uses this protocol to provide AI assistants with controlled access to your A/B test details and analytics.
How much time have you wasted exporting CSVs, rebuilding pivots, and tab-switching between different stores just to analyze your A/B testing data?
With the Shogun MCP, all these tedious tasks are no longer necessary.
Now, whenever you have a question about this data, you’ll be able to get all the answers you need instantly through an AI client.
This goes far beyond just pulling up the results of your tests.
The Shogun MCP can also explain what happened, why, and what you should do about it. It’s like having a seasoned CRO expert on your team who has perfectly memorized all your testing data, can work at a lightning-fast pace, and is always available at a moment’s notice.
The Shogun MCP is particularly valuable for the following three types of users: agencies, portfolio companies, and high-velocity single-brand teams.
Who they are: CRO agencies work with brands — often dozens at a time — and use testing to figure out how to convert more of their web visitors into paying customers.
How the MCP adds value:
Who they are: Ecommerce companies that are managing a large portfolio of brands
How the MCP adds value:
Who they are: In-house optimization teams for a single brand that have made testing a core part of their growth strategy.
How the MCP adds value:
There’s almost no limit to how you can use the MCP Connector. Your creativity is the only limitation. Here are a few ideas to help you get the wheels spinning:
Feed results into the rest of the business: “Take the headline from this winning test and give me 10 Meta ad options.
Synthesize your whole portfolio in one question: “List all PDP tests started in the last four weeks that haven’t reached a useful conclusion.”
Prove impact in dollars, even across currencies: “Show me the net revenue testing generated across all storefronts this quarter, as one USD number.”
We’ve taken a number of steps to ensure that your data always stays secure with the Shogun MCP, including:
The Shogun MCP connector works with any MCP-compatible AI client, which includes Claude, ChatGPT, and Gemini.
Regardless of which client you use, it only takes a couple minutes to set up.
Step-by-step instructions for establishing your connection can be found on this support page
The Shogun MCP is available on Shogun A/B Testing’s Advanced and Unlimited plans, as well as custom contracts.
Prompt library
Copy any prompt straight into your AI client with the Shogun MCP connected.
Mine past results and store data for your next winner.
Are there any notable patterns across all my previous winning tests? And how about losing tests?
Which PDPs on my storefront have high cart interest but don’t generate a lot of revenue?
Given the average daily traffic to this page, how many simultaneous variations can we test and still reach 95% statistical significance within two weeks?
Has a similar test to this idea been run already — and if so, what were the results?
Given what we have already tested and what the results were, what should we test next?
Find the story behind the numbers, segment by segment.
Help me understand the likely reasons this test worked on my US storefront but performed terribly on my UK storefront.
Which tests have been running for eight weeks or more without a decision?
Flag any winning tests where a major segment, such as a device or channel split, is negative.
Flag any test that looks flat or negative overall but is clearly winning in at least one segment.
Was the improvement in this test front-loaded?
This test won — in which markets would I most likely see similar results from replicating this test without modification?
This test lost, so walk me through what likely went wrong, and what to test next.
Ship, pause, end and launch tests without opening a dashboard.
Ship all winning tests that have reached statistical significance.
End all tests that have been flat or losing for over eight weeks.
Pause any test currently running that has led to a significant drop in revenue per visitor.
That test won, so now spin up a follow-up test and note the hypothesis.
Clone the winning test onto the two storefronts still without a winner.
Prove the value of testing to clients and leadership.
Rank every running test by projected revenue impact so that I know where to focus.
Compare the revenue our testing generated this quarter against our retainer.
At 12 AM every Monday, create a summary of what changed across all my clients’ tests in the previous week.
Scheduling the recurring run is a feature of your AI client, not the Shogun MCP itself.
Assemble the quarterly review for this client, including tests run, winners shipped, and what’s queued across all of their storefronts.
In one sentence, make the case for renewing this testing program by describing its revenue impact this quarter.
These examples should give you more than enough inspiration to get started — it’s time to see what A/B testing can really do for your organization.
Connect the Shogun MCP
