When AI accountability becomes personal: senior liability under the EU AI Act and UK Consumer Duty

Most discussions about AI governance begin with the technology. They explore how models are trained, how decisions are made, or how organisations should manage emerging risks. Increasingly, however, regulators are asking a different question altogether. They are not starting with the technology. They are starting with the people responsible for it.

Across Europe and the UK, AI regulation is reshaping expectations of executive accountability. The EU AI Act introduces detailed obligations for organisations that develop or deploy certain AI systems. In the UK, regulators continue to emphasise that existing governance frameworks, including the Financial Conduct Authority’s Consumer Duty and the Senior Managers and Certification Regime (SMCR), remain fully applicable when AI influences business decisions. The Digital Operational Resilience Act (DORA) reinforces similar principles across the financial sector by placing responsibility for operational resilience firmly on management bodies. Meanwhile, the Information Commissioner’s Office (ICO) continues to stress that organisations using AI must be able to demonstrate accountability, transparency and effective governance.

AI governance is becoming executive governance

Taken individually, these frameworks serve different purposes. Viewed together, however, they point in the same direction. AI governance is becoming inseparable from executive governance. Responsibility no longer sits solely with technical specialists, compliance teams or data scientists. It increasingly rests with named individuals who are expected to demonstrate that appropriate oversight existed before decisions were delegated to AI systems.

This represents an important shift in regulatory thinking. Historically, organisations often approached governance by documenting policies, establishing committees and assigning broad responsibilities. Those measures remain important, but regulators are now looking beyond organisational structures to understand how decisions were actually made. The question is no longer whether an AI governance framework exists. It is whether senior leaders can demonstrate that they exercised informed judgement, understood the associated risks and maintained effective oversight throughout the lifecycle of the system.

That distinction matters because accountability is not established after something goes wrong. It is demonstrated through the quality of governance before, during and after decisions are made.

For many boards, this represents unfamiliar territory. Artificial intelligence evolves rapidly, often crossing traditional organisational boundaries. Procurement teams acquire AI-enabled services. Business units deploy new capabilities. Technology teams integrate models into existing processes. Legal, compliance and risk functions provide oversight. Without clearly defined decision rights, responsibility can become fragmented, making it difficult to identify who ultimately owns the governance of a particular system.

Regulators are increasingly uncomfortable with that ambiguity. Where AI affects customers, employees, financial decisions or regulated activities, they expect organisations to know who is accountable, how decisions are reviewed and what evidence supports those decisions. Simply stating that responsibility sits with “the business” or “the AI team” is unlikely to satisfy that expectation.

What personal accountability looks like in practice

The practical consequence is that accountability is becoming increasingly personal. Chief executives, senior managers, general counsel, chief risk officers, chief information security officers, data protection officers and board members may all find themselves expected to explain how governance operated within their respective areas of responsibility. The precise obligations differ between regulatory regimes, but the underlying expectation remains remarkably consistent: senior leaders should be able to demonstrate control rather than merely assert that controls existed.

What does that accountability look like in practice? Contrary to some assumptions, regulators are rarely looking for evidence that an AI system never produced an error. Complex systems inevitably require judgement, adaptation and continuous improvement. Instead, they want to understand whether appropriate governance surrounded the technology throughout its lifecycle.

That begins with clarity over ownership. Every AI system capable of influencing regulated activities, customer outcomes or significant operational decisions should have an identifiable business owner. That individual does not need to understand every technical detail of the underlying model, but they do need to understand why the system is being used, the risks it introduces, the controls in place and the circumstances under which human intervention is required. Responsibility cannot disappear into a project team or become diluted across multiple functions.

The EU AI Act reinforces this principle through its requirements for governance, human oversight, technical documentation and record keeping. Organisations deploying high-risk AI systems are expected to ensure that appropriate oversight mechanisms exist throughout the system’s operation, supported by documentation capable of demonstrating compliance. These obligations are designed not simply to encourage good governance but to make that governance visible.

The same principle appears in different forms across UK regulation. Under the FCA’s Consumer Duty, firms remain responsible for delivering good customer outcomes regardless of whether decisions are supported by AI. Delegating part of a decision-making process to technology does not delegate regulatory responsibility. Similarly, under SMCR, named senior managers remain accountable for ensuring that the areas under their responsibility are appropriately governed, with clear responsibilities, effective controls and appropriate oversight.

DORA reaches a similar conclusion from the perspective of operational resilience. Boards and senior management are expected to oversee ICT risk, third-party dependencies and operational resilience arrangements. As organisations increasingly integrate AI into critical business processes, governance of those systems naturally becomes part of the wider resilience framework rather than a separate technical exercise.

Closing the gap between policy and evidence

Across these frameworks, a consistent expectation emerges. Organisations should be able to demonstrate not only that governance exists but how it operates in practice.

This is where many organisations discover a gap between policy and evidence.

An AI policy may describe governance principles, but a regulator is more likely to ask how those principles were applied to a specific system. A governance committee may meet regularly, but what decisions did it make? Were risks challenged? Were concerns escalated? Were conditions attached to deployment? Was implementation reviewed after launch? Documentation that answers those questions carries far greater evidential value than broad statements of intent.

For senior leaders, this changes the nature of governance conversations. Instead of asking whether policies have been approved, they should increasingly ask whether the organisation could demonstrate informed decision-making if challenged tomorrow.

In practice, that evidence is likely to include clearly documented governance decisions, defined ownership, records of risk assessments, validation and testing activities, monitoring arrangements, human oversight mechanisms, incident reporting procedures and periodic reviews. None of these documents exists in isolation. Together, they create an auditable record showing that AI deployment was subject to informed oversight rather than assumed to be operating safely.

This distinction is particularly important because regulators increasingly assess governance through the evidence available at the time decisions were made, not through explanations prepared after an incident has occurred. Retrospective justifications rarely carry the same weight as contemporaneous records demonstrating that appropriate questions were asked, relevant risks were considered and proportionate controls were implemented before deployment.

The difference between believing and proving

The organisations that respond most confidently to regulatory scrutiny are therefore not necessarily those with the most sophisticated AI capabilities. They are often those with the clearest governance disciplines. Their boards understand where AI is being used, senior leaders understand their respective responsibilities, and decision-making is supported by evidence that can withstand independent examination.

That is the difference between believing AI is governed and being able to prove it.

The DPG helps organisations translate regulatory obligations into practical AI governance frameworks that senior leaders can confidently stand behind. The focus is not simply on well-designed oversight, but on creating the evidence needed to demonstrate effective governance and withstand regulatory scrutiny.

 

---

 

Sources

1. European Commission – AI Act Regulatory Framework
   Provides the European Commission’s official overview of the EU AI Act, including its objectives, implementation timetable and the regulatory framework governing AI systems across the European Union.
   https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
2. 
   EUR-Lex – Regulation (EU) 2024/1689 (EU AI Act)
   The official legal text of the EU AI Act. This is the primary legislative source defining obligations for providers and deployers of AI systems, including governance, human oversight, documentation, transparency and post-market monitoring. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
3. 
   Financial Conduct Authority – Consumer Duty
   Explains the FCA’s expectations that regulated firms deliver good customer outcomes, including where AI is used to support products, services or customer decision-making. Reinforces the accountability of senior management for governance and oversight. https://www.fca.org.uk/firms/consumer-duty
4. 
   Financial Conduct Authority – Senior Managers & Certification Regime (SMCR)
   Sets out the UK’s framework for individual accountability within regulated financial services firms, defining the responsibilities of named senior managers and the evidence expected to demonstrate effective oversight. https://www.fca.org.uk/firms/senior-managers-certification-regime
5. 
   Information Commissioner’s Office – Guidance on AI and Data Protection
   The ICO’s comprehensive guidance on using AI in compliance with UK GDPR and the Data Protection Act 2018. Covers accountability, governance, fairness, transparency, explainability and Data Protection Impact Assessments (DPIAs). https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/guidance-on-ai-and-data-protection/
6. 
   Digital Operational Resilience Act (DORA)
   The official EU Regulation establishing operational resilience requirements for financial entities, including governance, ICT risk management, third-party oversight and executive accountability.
   https://eur-lex.europa.eu/eli/reg/2022/2554/oj
7. 
   FCA and ICO Joint Statement on AI in Financial Services
   Explains how the FCA and ICO are working together to support responsible AI innovation while reinforcing that existing regulatory obligations remain fully applicable when firms deploy AI.
   https://www.fca.org.uk/publications/corporate-documents/joint-fca-and-ico-statement-regulatory-expectations-regarding-firms
8. 
   OECD AI Principles
   Internationally recognised principles promoting trustworthy AI through accountability, transparency, robustness and responsible governance. Frequently referenced when discussing organisational AI governance frameworks. https://oecd.ai/en/ai-principles