Your Payments Data Isn't Ready for AI (Yet)

Two members of the Pagos flock, Robert Clewley and Andy Barker, presented a webinar for PaymentsEd this week, making a point that a lot of merchants don't want to hear: your AI probably can't help you optimize your payments data in its current form.

Payments teams want to use AI to increase their analytic capability, identify issues in their setup, and find optimization opportunities. Of course they do! That's what AI is for, right? But unfortunately, too many companies are starting the AI conversation in the wrong place; they ask what the model can do before they ask what their data can support.

The Easy Data Illusion

Say you're optimizing subscription retry logic on a single payment service provider. You're managing one set of decline codes, one scheme fee and descriptor structure, one dispute workflow, one PSP's reporting format, and one time zone. It seems like an easy dataset to hand off to an AI agent for analysis. Which it may be at first, but what happens when things change?

A couple months in, BIN ranges have shifted, product code names have changed, your PSP rolled out some new services or small formatting changes, and the card networks introduced some interchange and scheme fee updates (as they do several times a year).  Suddenly your original AI doesn't recognize some of the ingested data or know how to harmonize it with similar data from before these changes hit your account. Your one-time build now requires constant maintenance to keep up.

On top of this, what happens when your business successfully expands into a new market and you set up a second PSP? Now you're not only normalizing one set of codes against previous versions of itself, you're managing two and attempting to keep them consistent with each other. Then, the changes start compounding. Maybe one PSP drops the order ID from its metadata, while another replaces it with an internal ID of its own. Or one reports timestamps in eastern time, another in UTC. Even worse, you may have built your whole payments strategy around how the first PSP behaves, so if the second one does things differently, your whole model breaks.

That's just the reality of data ingested from a multi-PSP setup. It's fragmented and messy, and the true reason why most teams aren't ready for AI, no matter how good their model is.

The Data Work is Hard, But You Can't Skip It

Data work is hard and often considered boring, so teams want to hand it off to automation. That doesn't end well. A 2021 Google Research paper on “data cascades in high-stakes AI” found that 92% of projects have at least one small data problem that surfaces early and compounds into an expensive failure later; 45% had two or more. Google's explanation for why this keeps happening is just that everyone wants to skip to the fun part of building an AI model, bypassing the necessary data work.

Think of it against the DIKW pyramid—the standard progression from Data to Information to Knowledge to Wisdom. Every layer above data, including normalization, reconciliation, reporting, forecasting, and strategy, depends on the layer below it being solid. Clean, usable data is a bottleneck the whole pyramid sits on, and skipping it moves failures downstream, to a point where they’re harder to trace. And by then, the problem could have been going on silently for months or even quarters.

Picture a team with a data warehouse, a business problem, and an LLM subscription. They hand the model a three-sentence prompt and let it run autonomously. Whatever latent issues already existed in the data get amplified and small errors compound. Then, the AI (in true AI fashion) will confidently provide answers based on incorrect data and out-of-date specs, leading you to inaccurate conclusions.

Setting AI Up for Success

AI does have a place in your data work; it just needs a human-guided ontology and a semantic layer before it can be particularly helpful. Your domain experts must define your setup, and work regularly alongside the LLM to update mappings and review anything a model pattern-matched on its own. Set milestones where a human defines and refines next steps, monitor for bad assumptions, and expect to spend a lot of time maintaining your data specs. AI readiness is only possible once you have data readiness.

That same discipline applies to how you manage the model day to day. Treat it like a junior analyst you directly manage: enthusiastic and well-educated in theory, but thin on practical experience. Keep the tone professional and concise, with no emotional or judgmental content, and don't waste time criticizing its mistakes. Provide onboarding, context, coaching, and feedback the way you would for any new hire. Expect it to show its work and cite its sources, and remember that you're always the one accountable for the final output, not the model.

The future of data analytics isn't fewer analysts. It's analysts who lead with good judgment. Harmonized, well-structured payments data comes first. Everything useful AI can do for you comes after.