Legacy model risk frameworks built for a pre-generative era are no longer adequate for the complexity of modern artificial intelligence deployments, and the Bank of England is now saying so explicitly. Newly published minutes from the joint BoE and Financial Conduct Authority Artificial Intelligence Consortium (AIC) have outlined sweeping recommendations that would fundamentally restructure how financial firms in the United Kingdom and the United States validate, govern, and contain risk across generative artificial intelligence (GenAI) systems. The intervention marks one of the most substantive regulatory signals yet that the supervisory machinery assembled around conventional predictive models is structurally misaligned with the architectures now being deployed at scale.
The Problem With SS1/23 in a GenAI World
The central tension is this: the Prudential Regulation Authority's SS1/23 Supervisory Statement on Model Risk Management was designed around the assumption that a model is a discrete, static, and auditable algorithm. GenAI systems are none of those things. According to the AIC minutes, GenAI applications are routinely classified by default as high-risk systems under SS1/23, triggering rigid control regimes that, in practice, slow or obstruct deployment without necessarily improving safety outcomes. The framework applies controls built for single-model logic to architectures that are, in reality, complex ecosystems layering multi-modal foundation models, retrieval-augmented generation (RAG) pipelines, third-party application programming interfaces (APIs), and autonomous orchestration agents into a single operational chain. Governing each component in isolation, the consortium argues, misses the systemic risk that emerges only at the level of the whole system.
A System-Level Governance Architecture
The AIC's proposed response is a shift from model-centric to system-level governance, built around four core principles. Whole-system testing would require institutions to evaluate individual components alongside the complete end-to-end AI system, explicitly accounting for third-party model components that may update independently of internal deployment cycles. Outcome-based explainability would redefine transparency not as the ability to interrogate black-box model weights — an increasingly futile exercise with frontier models — but as demonstrable evidence that the system behaves as intended, supported by auditable decision logs. Differentiated playbooks would create separate operational scorecards distinguishing the distinct risk profiles of AI developers from those of AI deployers. Finally, Human-in-the-Loop (HiTL) testing would embed structured human oversight and continuous red-teaming directly into critical financial workflows rather than treating oversight as a post-deployment audit function.
The consortium also proposed a four-step failure containment framework designed to intercept systemic disruptions before they propagate into broader operational resilience failures. The framework instructs institutions to first identify failure types by root cause — whether data corruption, prompt injection, model hallucination, or API latency — then detect signals through live operational telemetry tracking real-time anomalies. The third step requires gathering diagnostics through baseline audit trails sufficient to enable rapid root-cause analysis. The fourth is to deploy circuit breakers: automated controls capable of triggering fallback to deterministic systems or handing control to human operators when model outputs diverge from expected bounds. To develop this framework, the AIC evaluated six distinct AI edge-case scenarios, stress-testing the points at which traditional risk management breaks down when model outputs diverge from intended behaviour.
Three Systemic Threats the Minutes Do Not Understate
Beyond validation methodology, the AIC minutes identify three broader systemic concerns that extend across North American and European markets. The first is the pace of agentic AI development. Autonomous agentic systems — particularly in payments — are evolving fast enough that their capabilities risk outrunning the internal governance frameworks firms have in place to supervise them. The consortium recommends stress-testing scenarios where AI capabilities advance faster than current policy allows, essentially future-proofing governance against the model's own trajectory.
The second concern is third-party concentration. The financial sector's dependence on a narrow group of cloud and frontier model providers creates visibility gaps into the core architectures powering production systems. When those providers update their models — as they do regularly and often without granular change documentation — firms may have little insight into how their deployed systems have effectively changed. The AIC calls for stronger vendor oversight disciplines and mandatory auditable documentation from critical technology providers.
The third issue is a talent deficit that could undermine every other governance initiative. Managing GenAI systems at the standard the AIC envisions requires specialised skills in LLM Operations (LLMOps), model governance, and risk oversight — competencies that remain scarce across the industry. The consortium recommends targeted accelerator programmes to build this capacity, acknowledging that regulatory ambition without a qualified workforce to execute it amounts to policy aspiration rather than operational reality.
A Standardised Incident Reporting Regime on the Horizon
Perhaps the most consequential long-term signal in the minutes concerns incident reporting. The AIC stated directly that "greater standardisation of AI incident reporting could support cross-firm learning and improve visibility of failures, recognising that incidents may continue to occur despite the presence of safety mechanisms." The acknowledgement that safety mechanisms will not eliminate failures — and that cross-firm visibility is therefore essential — points toward a future where AI incident disclosures function more like operational risk event reporting under current prudential frameworks. For Chief Information Security Officers (CISOs) and Chief Risk Officers (CROs), the practical implications are immediate: audit AI supply chains for visibility and contingency fallback options, update internal policies to evaluate the complete AI chain from inputs through orchestration layers to outputs, implement live telemetry with automated circuit breakers, and align governance structures with outcome-based validation ahead of what the AIC characterises as incoming stricter enforcement mandates for critical technology providers in both the UK and US markets.
What This Means for Financial Institutions
The AIC's recommendations represent a regulatory inflection point, not a distant policy aspiration. The consortium — convened by the two most powerful financial supervisors in the United Kingdom — is signalling that the current default posture of classifying GenAI deployments as high-risk under frameworks designed for spreadsheet-era models is both commercially counterproductive and supervisorily inadequate. Institutions that move first to adopt system-level governance, embed HiTL testing, and build auditable AI supply chains will be better positioned when enforcement mandates arrive. Those that wait risk regulatory intervention on top of the operational failures the AIC's own edge-case scenarios suggest are already probable. The message from Threadneedle Street is unambiguous: the governance of GenAI must match the complexity of the systems being governed.
Written by the editorial team — independent journalism powered by Codego Press.
