I’ve been thinking a lot recently about how people operating in fraud mitigation spaces think about, quantify, and behave in response to fraud risk.

I have to say that I am largely disappointed by the profession as a whole. I see it dominated by people that are largely obsessed with detection. Last week , I was having a lunch with an old colleague, and we were reminiscing about the company we both worked at. He was telling me about conversations that he has with another person we worked with still at the company. Apparently, there is a sense that nothing that they have done has worked to reduce fraud losses since we left the company. And this is despite expending a good amount of time and resources. They increased friction, same level of losses. The increased friction has even, allegedly I am not there anymore, so I don’t know for sure, led to expanding the call center to handle the increased operational load due to a futile attempt to bring losses down.

That conversation got me thinking about how fraud professionals should be thinking about how they make progress and whether or not they are “winning”. At first glance, fraud mitigation appears reducible to one question: did losses go down? However, as I thought more and more about this, I had an epiphany. That isn’t the job at all. We were measuring the wrong thing, and the parallels with Moneyball became undeniable.

Playing Moneyball

Let me take a step back to Moneyball for a moment.

In baseball, players were valued (when they were quantitatively valued) by metrics like batting average and runs batted in. They were focused on measuring outcomes, not on what produced the outcomes.

Along came Billy Beane, and in 2002, he reshaped the way that Major League Baseball was played. How? By focusing on players that were undervalued by paying attention to the metrics that cause outcomes, not the outcomes themselves. So he started to focus on metrics like on-base percentage instead of batting average and runs batted-in. Think about this, having a high batting average is great, but it means nothing if they guys after you get thrown out and the inning ends before you can make it to home. Furthermore, you can’t bat in a run, if nobody from your team is on base.

My realization was that the fraud management organizations are out there doing a lot of the same things as the MLB was doing pre-2002 season. We focus on outcomes: dollars saved, false positives, recoveries. These metrics matter, but they are downstream effects, not the fundamental unit that drives results. In baseball, you can’t score if you don’t get on base, similarly, you can’t reduce fraud losses if you don’t reduce risk.

Billy Beane changed the way baseball is played by focusing on the underlying state that drives outcomes, namely, getting on base. Analogously, fraud management is that we as a profession are not optimizing the underlying conditions that produce fraud outcomes either.

Risk Reduction is the Game

Saying, “Risk reduction is the entire game of fraud mitigation” feels obvious. What is a control, a model, an alert, or [insert your method of reduction of choice] for if not to reduce risk?

My point is not that such a silly statement is obvious. It is to make you understand that the primary metric by which a fraud mitigation group should be judged is not by how much money left the organization, nor by how efficiently they handle operational volume, nor by how much money they prevent from walking out the door. No, it should be how much risk did they mitigate.

Even now as I write this, that feels obvious, so let me ask you a question that I think that most fraud professionals will struggle with.

What is risk?

If the goal is to mitigate risk, you must surely understand it well enough to answer such a simple question. Sure you could pull out a dictionary definition for risk, but that won’t help you to mitigate risk.

So here is a functional definition of risk, given to us by the economics profession:

Risk is the distribution of possible loss outcomes conditioned on the current system state. In terms of Fraud Risk, it is the probability distribution over possible fraud losses.

Take the time to read that again and again until it really sinks in. Fraud risk is a probability distribution! Your fraud loss for a given period of time is a realization of a draw from that probability distribution. There is a fundamental difference between a realization of the outcome loss from the underlying exposure to the probability distribution.

Without this understanding an organization can think that it is winning while losing. You can have a quarter with low losses relative to your historical losses, but be in a situation where the probability distribution has shifted unfavorably. Friction can alter your visibility into fraud without materially altering the loss distribution. Stable fraud losses during periods of increased attack pressure may actually indicate reduced underlying risk.

Fraud losses are observations drawn from the underlying risk distribution. Sure, declining fraud losses are a proxy for having fundamentally shifted the distribution, but not always.

An immediate implication of this framework is that realized fraud losses are noisy by construction. They are samples drawn from an underlying distribution, not the distribution itself. This means the metrics that the profession obsesses over are inherently limited: they are lagging indicators, statistically noisy, and partially endogenous to the very control systems they are attempting to measure.

This realization ultimately led me toward what I now think of as the AQFM framework: a probabilistic approach to fraud management centered on shaping loss distributions rather than merely reacting to observed losses.

The Fix

The first change that needs to happen is linguistic and conceptual. “Risk reduction” is too vague a phrase to anchor a serious fraud program, and is often interpreted too loosely to be operationally meaningful. Likewise, monthly or quarterly fraud losses are insufficient on their own, because they are noisy observations sampled from an underlying distribution rather than direct measurements of exposure.

Fraud management organizations need to become more precise in how they describe progress. Instead of speaking only in terms of dollars saved or losses avoided, they should think in terms of distributional shifts: reductions in tail exposure, changes in expected loss, attacker cost inflation, exploit scalability constraints, and the evolution of attack pressure over time.

I shall decline to define each of these concepts in detail for now in favor of making a broader point, though they do have precise operational meanings. Fraud management organizations need to measure, report on, and socialize concepts that extend beyond realized losses. These concepts are more difficult to measure, and often even more difficult to socialize internally, but they provide something critically important: visibility into the generative process of fraud loss itself.

Earlier this week, for example, I created an alert that will almost never fire. It may trigger once every three or four months, if that. In fact, I do not believe we have ever taken a realized loss from the exact scenario it monitors. Yet the scenario is plausible, and if it were to occur, the resulting loss could be meaningful.

From a traditional operational perspective, this alert appears almost useless. It produces negligible case volume, likely generates false positives when it does fire, and may never directly prevent a realized fraud loss. But from a distributional perspective, the alert materially truncates tail exposure for that scenario. In expected value terms, the control reduced estimated exposure to low-probability, high-severity losses by roughly $10,000.

That is the power of this mode of thinking. A control does not need to generate large operational statistics to meaningfully reshape the underlying loss distribution. This is the type of reasoning that underpins what I think of as the AQFM framework.

The ideas is to shape the causal mechanism of loss. Going back to Moneyball, Billy Beane was optimizing his team to get on base, because getting on base was the causal mechanism for winning games. For a fraud organization, reducing losses sustainably requires changing the structural conditions that generate the loss distribution in the first place.