AI in Regulated Industries: What Boards in Financial Services Need to Get Right

The Financial Conduct Authority has been clear that its existing regulatory requirements apply fully to AI-enabled processes. This means several things in practice.

Consumer Duty requires that firms using AI in customer-facing contexts can demonstrate that AI-assisted processes deliver good outcomes for retail customers. An AI system used in loan decisions, insurance pricing, or investment advice must be assessed for compliance with Consumer Duty principles, including fairness, transparency, and avoidance of foreseeable harm. Firms cannot rely on 'the AI did it' as a defence against Consumer Duty breaches.

The Senior Managers and Certification Regime assigns personal accountability to designated senior managers for regulated activities. Where AI systems are used in activities covered by SMCR, the responsible senior manager must be able to demonstrate adequate oversight and governance of those AI systems. A head of retail banking who cannot explain the governance framework around their AI credit decisioning system is potentially in breach of their SMCR obligations.

Market abuse regulations apply to AI systems used in trading. Firms using AI for order generation, execution, or market-making must ensure those systems comply with MAR requirements and that appropriate surveillance is in place.

Financial services firms have long-established model risk management frameworks developed in response to regulatory requirements around quantitative models. The good news is that these frameworks provide a ready foundation for AI governance. The challenge is that AI models have characteristics that traditional model risk management frameworks were not designed for.

Traditional models are rule-based: you can read the code and understand how they make decisions. Large language models are not. The decision-making process in a neural network is not directly interpretable, which creates specific challenges for model validation, audit, and explainability requirements.

The SS1/23 guidance from the PRA on model risk management provides a framework for managing model risk that applies to AI models, and requires firms to maintain an inventory of models in use, conduct validation of models before production deployment, monitor model performance on an ongoing basis, and manage model change through a defined change control process. Applying this framework to AI requires some adaptation, particularly around explainability and validation methodology, but firms that already have strong MRM capability are significantly better placed than those starting from scratch.

Financial services firms handle some of the most sensitive personal and commercial data of any sector. Using this data in AI systems creates specific data governance obligations.

Fair lending requirements impose constraints on the data that can be used in credit decisioning AI. Demographic data that creates proxy discrimination against protected characteristics cannot be used in credit models, but AI systems trained on historical data may learn to use correlated variables that have the same discriminatory effect. Firms must test AI credit models for disparate impact and take remedial action where it is found.

Data minimisation requirements under UK GDPR mean that AI systems should only use personal data that is necessary for the specified purpose. Many AI systems can perform adequately with less data than they are provided; data minimisation in AI design reduces privacy risk without necessarily compromising performance.

Data residency requirements, particularly for firms with EU operations, mean that personal data processing by AI systems must comply with cross-border transfer restrictions. Firms using cloud-based AI services must confirm that the geographic processing locations meet their data residency obligations.

The PRA and FCA's operational resilience framework requires firms to identify their important business services and ensure they can deliver them within defined impact tolerances even in severe but plausible disruption scenarios.

As AI systems become embedded in important business services, including loan origination, claims processing, and trade execution, they become part of the operational resilience framework. Firms must map AI system dependencies into their important business service definitions, identify how AI system failures would affect their ability to deliver within impact tolerances, and develop recovery strategies for AI system failures that may be different from conventional IT system failures.

AI system failures can be particularly complex to manage because they may manifest as degraded performance rather than binary failure. An AI system that begins producing biased or inaccurate outputs at scale is a different failure mode from a system that is unavailable, and it may be harder to detect and remediate quickly.