Industry
Achieving Payments Optimization Part VI: A/B Testing Enhancements to Your Payment Processes
Welcome to the final installment of our blog series on achieving payments optimization! Thus far, we’ve defined and outlined strategies for pursuing the following payments optimization cornerstones:
Today, we’ll discuss the last key objective in any payments optimization journey: A/B testing new payment processing solutions. In reality, A/B testing is less an independent cornerstone and more an extension of all our other optimization goals. To hone in on the best strategies for maximizing revenue or combatting fraud, for example, you need to continuously run some tests! Let’s get into why and how Pagos can help.
Staying Competitive Through Innovation
First things first, let’s discuss why you should even A/B test in the first place. You work in commerce, so you don’t need to be reminded of how competitive the market is. But in a world where customers have seemingly endless options, it’s worth repeating how vital it is to present the best and most efficient shopping experience possible. This can mean constantly evolving your business and the goods or services you sell, but it can also mean regularly adjusting your payment processes to meet the ever-changing needs and preferences of customers, changing network rules, and even the risk appetite of issuers. A/B testing offers a strategic approach to innovating and removing friction; with a limited test, you can experiment with new processors, payment methods, markets, and even checkout experiences, all while measuring performance side-by-side with more established strategies.
Ultimately, there’s no one right answer on how to process payments or how to do better than your competitors—it’s specific to your business model. As such, you need to test changes with your product and your customer base to determine what moves the needle. We’ve provided some payments optimization north stars, and nothing will get you closer to them than well-tested and trusted solutions developed in-house by the people who know your business best.
Testing in the Complex World of Payments
When identifying the results of an A/B test, always keep in mind exactly how complex payment processing is. Because of these complexities, the introduction of one new or different variable will never shift only a single metric within your payments data; instead, you’re likely to see a rippling effect throughout your entire setup, across markets and even sales channels! This is important, because determining the success of any A/B test requires you to weigh any improvements against negative impacts.
For example, launching a new payment processor can present a myriad of challenges. Doing so could mean you’ve expanded into new markets, can now accept local payment methods, or have a lower cost option for processing specific transaction types. Regardless, this one change presents a plethora of variables that can impact customer experience and your bottom line. Questions arise: Does the new processor create more checkout friction? Does it result in higher approval rates for specific transaction types? Do you see more transactions coming in, but with payment methods that cost more to process? Are refunds more difficult to process or take a longer time to go through?
A/B testing provides a structured framework to systematically evaluate these variables and determine optimal solutions. You can introduce the new payment processor by first processing only a portion of your transaction volume through it. Then, after a sufficient length of time, review the data in its entirety: approval rate, transaction volume, decline code breakdown, total fees, chargeback rates, refund rate, and more. Only by comparing these metrics across your historical data for the new processor against the control group (your pre-existing processors) can you identify the comprehensive impact of your test.
Leveraging Pagos Products for A/B Testing
Pagos products offer invaluable tools for conducting A/B tests seamlessly. Here are just some ways you can leverage the Pagos suite of services to evaluate the results of any changes you make to your payment processing setup:
Visualizing Payments Data Before and During a Test
Peacock, our payments data visualization service, pulls in all your payments data from all your payment service providers (PSPs) and aggregates it into a single data feed. We then present this harmonized data back to you in organized dashboards. The charts in Peacock allow you to visualize changes to your payment metrics over time and across different segments of your customer base or lines of business. With such easy access to your clean data, you can identify the most important segments of your business or places where you have room for growth—areas where A/B tests might have the most impact.
Peacock is also then the ultimate tool for measuring the impact of an A/B test. Here are just a few ways you can use it:
The Processors dashboard in Peacock provides you with a single place to evaluate the performance of new processors or modifications to volume allocation among different payment service providers (PSPs). Compare the transaction volume, approval rate, and decline code breakdown for old and new processors side-by-side, and even see which payment methods perform better through each.
The Payment Methods dashboard facilitates in-depth analysis of transaction volume, average order value, and approval rates for newly introduced payment methods.
Using the data filter options in Peacock, you can look at only the specific segments of your transaction volume you're most concerned about. If you start accepting a new currency, you can filter the Fees dashboard—for example—to only show fees associated with transactions made in that currency, and identify the exact cost of accepting it. If you pass custom data with your transactions (e.g. soft descriptors or metadata), you can even filter Peacock charts and dashboards by those custom parameters.
Create a custom dashboard of the data visualizations most relevant to your A/B test. You can even filter each chart in your dashboard by a different parameter, allowing you to see the same data for different data segments side by side. For example, build a custom dashboard for monitoring chargeback rates and volume for each of your individual MIDs; see our Pagos Product Documentation for a full guide on how to do this!
Monitoring Test Results in Real Time
The last thing you want is for an A/B test to result in massive losses for your company. If a small change to your payment processing setup actually directly hurts your business, you’d want to know immediately. That’s where Canary can help! Canary is our automated real-time data monitoring and anomaly detection services. With this tool, you can set acceptable thresholds for specific payment metrics, and tell Canary to alert you whenever your data measures outside of those bounds. You can even tell Canary where to alert you should unexpected shifts occur—via email, slack message, or through a generic webhook.
Consider a situation where you attempt to increase approval rates by testing less restrictive CVV/AVS rules on a subset of your transaction volume. You hope to see more approved transactions, but accept the risk of potentially seeing an increase in fraud. An indicator that fraudsters are taking advantage of your experiment would be a sudden increase in your chargebacks or a change in the distribution of declines. To stay on top of this, you configure a Canary trigger that monitors chargeback count and alerts you if it increases above what has been typical for your business historically. In addition you can configure a trigger monitoring the share of fraud decline codes to further build awareness around how changes are impacting your key metrics and business. Should your data ever trip the trigger, you can stop the experiment immediately.
Identifying Testing Opportunities with Benchmark Data
At the top of this post, we mentioned the importance of A/B testing as a way to evolve your business and stay competitive. But how do you actually identify where your business might not be matching up with industry peers when it comes to payment processing capability and functionality?
Flamingo, our payments benchmarking service, allows you to anonymously compare your payments data to that of your industry peers—making it possible for you to know in the moment when your business isn’t keeping up with the market. How does your approval rate compare? Or your refund rate? Are other businesses experiencing the same declines you are, or is there something with your setup that’s turning away a disproportionate number of potential customers? Find all the answers in Flamingo, then design a strategy for A/B testing solutions.
Digging Deep Into Your Test Results
We’ve talked about how in the complicated world of payments, you expect A/B tests to have complex and rippling results throughout your payments setup. To really dig deep into the ways a test impacts your data, you’ll need a single, accessible, clean source of all your payments data across all your PSPs. With Puffin, our payments data warehouse, you can download reports of your aggregated and harmonized payments data to integrate into a custom workflow or for use in complex data analyses.
Payments Optimization Unlocked
And with that, we’ve reached the end of our payment optimization blog series. Thank you for joining us on this journey! Head over to pagos.ai to learn more about our services or contact our Sales team for a demo of what Pagos can do to get your business one step closer to optimized payments.
Like what you learned here? Check out our wealth of content in the Pagos Blog and our very own product documentation.
If you’re new to this series, check out the rest of our payments optimization posts:
By submitting, you are providing your consent for future communication in accordance with the Pagos Privacy Policy.