Barnes Analytics

Contact

Blog

  • US Economic Outlook 2025

    With the 2024 election in the rear view mirror, we have a better picture of what policy for the next few years is going to look like. As such, we have a better idea on what to expect from the US economy over the next few years. That has largely colored my analysis this year.…

  • Validating Models with Limited Data

    Why validating with limited data is a real problem for fintechs Early-stage fintechs routinely face the same tension: the product and decision problem are real, but labeled examples are scarce. That scarcity creates two statistical headaches: Those are not opinions — they’re why resampling and uncertainty quantification exist in the first place. The bootstrap and…

  • 3 Ways to Detect Model Drift — Before It’s Too Late

    You launched a model that beat the benchmarks. Then approval rates shift, manual review queues spike, or fraud slips through. That’s model drift — and in fintech it’s not just an ML problem; it’s a business, customer, and regulatory problem. Below I quickly define the types of drift, then give three practical, complementary ways to…

  • Validating Machine Learning Models in Fintech

    Machine learning (ML) is rapidly moving from R&D notebooks into production decision systems across fintech — underwriting, fraud detection, credit scoring, pricing, customer onboarding, and more. That’s great: ML can unlock better risk segmentation, speed, and personalization. But with that upside comes concentrated operational, legal, and reputational risk: a model that breaks, drifts, or discriminates…

  • How We Stress Test Credit Models

    If you build credit decisioning models for a fintech, you know the scorecard moment: a bank partner, compliance officer, or auditor asks for evidence, and suddenly your model has to survive not only production traffic but regulatory scrutiny. Independent model validation is not just a compliance checkbox. Done well, it turns risk controls into a…

  • The Role of Independent Model Validation for Fintechs Dealing with TPRM Teams

    Introduction In the dynamic world of fintech, partnerships with banks and financial institutions are essential for scaling innovative solutions. However, these collaborations introduce complexities, especially concerning third-party risk management (TPRM). Banks are increasingly scrutinizing the models provided by fintech vendors, necessitating robust independent model validation to ensure compliance, mitigate risks, and build trust. Understanding Third-Party…

  • How to Survive a Bank Partner’s Model Review

    If your fintech depends on bank partners to offer deposit rails, underwriting capital, or other regulated plumbing, a model review from that partner is not a performance review — it is a credibility and business-continuity test. Banks are accountable to regulators for models that materially affect safety and soundness, so when they review a third-party…

  • Independent Validation vs. Internal QA: What’s the Difference?

    Independent Model Validation (IMV) is an objective, external (or organizationally independent) assessment focused on whether a model is conceptually sound, implemented correctly, and fit for its intended use. Internal QA is an essential, day-to-day quality and engineering practice that keeps models working and reproducible. One ensures correctness and code hygiene; the other is the required…

  • ROC Curves Suck For Fraud Detection

    So I have been thinking of ways to communicate the value of a new fraud model to stakeholders. A data scientist working in fraud detection might produce an ROC curve. Perhaps plot the ROC curve against some benchmark model’s and tell stakeholders, “Hey, look this model dominates the output from the benchmark. We should implement…

    ROC Curves Suck For Fraud Detection

  • SR 11-7 Explained for Fintechs

    In the evolving landscape of financial technology, fintech companies increasingly rely on quantitative models to drive decisions in areas such as credit scoring, fraud detection, and regulatory compliance. The Federal Reserve’s Supervisory Letter SR 11-7, issued in 2011, provides essential guidance on managing model risk—defined as the potential for adverse consequences arising from decisions based…

  • The Cost of Skipping Independent Model Validation

    Introduction In financial services and fintech, models drive critical decisions every day. From credit risk assessments to fraud detection, pricing algorithms, and marketing analytics, organizations rely on these models to make choices that can affect millions of customers and billions of dollars in assets. Despite the centrality of models, some firms treat independent model validation…