top of page

The Kill Switch Goes Mainstream: What India's Draft Model Risk Rules Signal for AI Governance Everywhere

  • 16 hours ago
  • 7 min read

A banking regulator just wrote much of the responsible-AI playbook into draft rules. The mechanisms — board accountability, model inventories, lifecycle validation, human-in-command, and yes, kill switches — will look familiar to anyone who has built an AI governance program. That's exactly the point.

On June 24, 2026, the Reserve Bank of India released its draft Guidance on Regulatory Principles for Model Risk Management, 2026. Most of the coverage led with the dramatic part — banks would have to build a "kill switch" into every AI system they deploy — and that's a fair headline. But for governance teams, the kill switch is the least surprising thing in the document. The more important story is what surrounds it: a sector-wide, board-level accountability structure for every model an institution uses, with AI and machine learning singled out for a dedicated set of controls.



If you work in AI governance, you should read this draft even if you've never touched a balance sheet. Not because Indian banking rules will govern your organization, but because the RBI has essentially codified the operating model that responsible-AI practitioners have been converging on — and it's worth seeing what a regulator chooses to make mandatory when it puts pen to paper.


What the draft actually requires

The Guidance applies to eleven categories of RBI-regulated entities — commercial banks, small finance and payments banks, regional rural and co-operative banks, NBFCs, all-India financial institutions, asset reconstruction companies, and credit information companies. It builds on the RBI's August 2024 draft on model risk in credit and the August 2025 report of the Committee on a Framework for Responsible and Ethical Enablement of AI (FREE-AI). Once finalized, it will replace the credit-risk-model chapter of a guidance note that dates to 2002.

A few structural choices stand out.

It defines "model" so broadly that a spreadsheet counts. The draft deliberately refuses to let institutions escape scope on a technicality. Its own illustration is a spreadsheet-based loan-pricing calculator: harmless on its own, but the moment it takes borrower inputs, applies pricing logic, and produces a lending rate that drives a real decision, it becomes a model subject to the framework — whether or not the institution ever called it one. This is the same lesson many AI inventories learn the hard way: the risk lives in what the output decides, not in how sophisticated the tool looks.

Accountability sits with the board, not the tech team. Every regulated entity must maintain a board-approved Model Risk Management Framework (MRMF). The board owns the framework, the risk appetite, and the tiering policy. The Risk Management Committee of the Board reviews validation reports for high-risk models and approves their deployment, reviews model tiering at least annually, and keeps standing oversight of third-party and AI models. High-risk models can no longer be waved through by a risk or technology function in isolation. For the first time, AI model risk is explicitly a board agenda item across an entire financial sector.

It runs on three lines of defense. Model owners are the first line, an independent model-risk-management and validation function is the second, and internal audit is the third. This is the same control architecture that mature governance programs already use, now made a baseline expectation.

Risk tiering can't be gamed. Models are classified by a blend of materiality, complexity, and other factors — but the draft includes an explicit anti-dilution rule: one factor can't be used to offset another, so a low complexity score can't quietly drag down the tier of a highly material model. The tier has to reflect the model's composite risk profile. Anyone who has watched a high-stakes system get rated "low risk" because it was technically simple will appreciate why that sentence is in there.

Models live in an inventory, cradle to grave. Nothing may be used or relied upon unless it's in the inventory, and decommissioned models stay on the books for at least ten years. Validation reports for high-risk models go to the board's risk committee within three months of completion. The inventory isn't a compliance artifact — it's the substrate that makes enterprise-wide risk visible and lets you trace dependencies between models.

Consumer harm is an explicit red line. The draft states plainly that an institution should not use any model that harms consumers, and that grievance-redressal mechanisms must cover complaints arising from consumer-facing models. It's a short clause, but it reframes the whole exercise: model risk management isn't only about protecting the institution's balance sheet — it's about the people on the other side of the decision.

Third-party models stay your problem. This is the beat governance teams should sit with. An institution that acquires, uses, or relies on a third-party model is accountable for its outcomes, full stop. Vendor certification doesn't discharge the duty: the institution must still validate the model independently, and third-party models get enhanced board-committee oversight regardless of their risk tier. Contracts are expected to guarantee access to technical documentation, audit rights for the institution and its regulator, and continuity and exit arrangements. If your governance program treats AI as a supply chain, this chapter is the regulator agreeing with you — and refusing to let "but the vendor said it was fine" count as a control.


The AI-specific chapter is where it gets interesting

The draft reserves a dedicated chapter for models employing AI and ML, and this is where the regulator's thinking about modern systems shows through. A few provisions are worth flagging for anyone building governance controls today.

Autonomy is treated as a first-class risk axis. When assigning a risk tier to an AI model, institutions must consider not just materiality and complexity but the extent of reliance and the level of autonomy placed on the model's outputs for decision-making. As agentic systems move from pilots to production, "how much is this thing deciding on its own?" becomes a governance question with real weight — and the RBI has named it explicitly.

It names frontier AI. The Guidance asks institutions to define the scope of their AI models "including for foundational AI models and frontier AI models" and to add controls commensurate with potential impact. It's notable to see a central bank reach for that vocabulary in a regulatory instrument at all.

Explainability becomes a threshold, not an aspiration. Institutions must set explainability and transparency thresholds for every AI model, with higher bars for anything driving material decisions or significantly affecting customers. Where full explainability can't be achieved, the answer isn't to ban the model — it's to wrap it in compensating controls: enhanced validation, output corroboration before use, more frequent monitoring, and usage restrictions. That's a pragmatic posture, and a usable template.

Red-teaming, hallucination boundaries, and bias testing are in the text. The draft calls for structured challenge processes including red-teaming, particularly for customer-facing or generative systems; control boundaries to contain hallucinations; fairness assessments with mitigation (including recalibration or redesign) where models could unfairly treat customer groups; out-of-sample testing against overfitting; and management of output variability through confidence scores and probability outputs. Read as a list, it's a fairly complete AI assurance checklist.

Self-updating models get extra scrutiny. Where a model updates dynamically or automatically, the draft asks institutions to define exactly what is allowed to change on its own, justify why automatic updates are enabled at all, tighten data-quality checks, and monitor more frequently. It also expects ongoing monitoring for data drift and concept drift. For anyone running models that learn or refresh in production, this is the regulator drawing a bright line around "set it and forget it."

Customers get disclosure and an exit. Any system that interfaces with customers — generative AI included — has to tell users they're interacting with an AI, surface its limitations, and offer a path to a human. On the security side, the draft specifically calls out prompt injection, adversarial inputs, and limits on session and context persistence.

And then, the human-in-command requirement. Institutions must establish robust human oversight for AI, including override, suspension, and deactivation mechanisms — the "kill-switch arrangements" that grabbed the headlines. But the draft is careful to pair the switch with the human judgment that makes it useful. Oversight has to account for automation bias, over-reliance on model outputs, and decision fatigue, and the people doing the overseeing must have enough expertise to actually challenge, override, or escalate — not just rubber-stamp. A kill switch nobody is equipped or willing to pull is theater. The RBI seems to know this.


Why a healthcare-AI audience should care about a banking rule

Here's the throughline. Strip away the sector, and the RBI's draft is built from the same primitives that responsible-AI governance has been standardizing on across industries: a complete model inventory, risk-based tiering, independent validation across the lifecycle, change management with material-change triggers, business continuity and decommissioning planning, board-level accountability, and continuous monitoring. Swap "Risk Management Committee of the Board" for a clinical AI governance committee, and "regulated entity" for a health system, and most of this draft would read as sound practice in a hospital.

That convergence is the signal. We now have frameworks pointing the same direction from very different vantage points — the NIST AI Risk Management Framework, the EU AI Act, sector bodies like CHAI in healthcare, and now a major central bank's model-risk regime. They differ in jurisdiction and enforcement, but they agree on the shape of the work: govern the full lifecycle, tier by risk, keep a defensible inventory, validate independently, keep a human meaningfully in command, and make the board own the outcome.

For governance leaders, three takeaways travel well beyond banking:

  1. Scope by impact, not by sophistication. If an output materially drives a decision, it's in scope — spreadsheet, classifier, or frontier model. Inventory accordingly.

  2. Tier on autonomy, not just accuracy. As systems take on more independent action, the level of reliance you place on them belongs at the center of your risk rating.

  3. Operationalize the override. Kill switches, human-in-the-loop, and escalation paths only count if the people behind them have the expertise, authority, and freedom from automation bias to use them. Build for that, and review your overrides and near-misses as data.

  4. Own the models you didn't build. Vendor assurances aren't a control. If a third-party or foundation model drives your decisions, the accountability — and the validation burden — is yours.


The RBI draft is open for public comment until July 24, 2026, and it's still draft — the language will move before it's final. But the direction is unmistakable. The mechanisms that used to live in responsible-AI white papers are migrating into the regulatory floor, one jurisdiction and one sector at a time. The organizations that already treat lifecycle governance, model inventory, and human oversight as infrastructure won't be scrambling when the floor reaches them. They'll just be in compliance.


Primary sources

This post is commentary on a draft regulatory instrument and is not legal or compliance advice.

 
 
 

Comments

bottom of page