Part III: When AI Starts Doing the Work, Who Owns the Outcome?

Human oversight and decision rights in AI governance
By Valerie Chan

As organizations move from experimenting with AI to embedding it into actual workflows, a more difficult question is emerging: Who owns the outcome?

In Part I, I wrote about why AI adoption and AI transformation are proving to be very different things. In Part II, I looked at what happens when AI agents begin participating in the work itself.

The next challenge is accountability.

If AI drafts client advice and a lawyer reviews it, who is ultimately responsible for the accuracy? If an AI system recommends a course of action, who is expected to validate it? If an agent can take the next step on its own, where does human responsibility begin and end?

These are no longer just technology questions. They are operating-model questions. And this is where many organizations may find that traditional AI governance is not enough.

Policies, approved tools, security controls and data protections all matter. But accountability cannot live only in a policy document. It has to become part of how people are managed, trained, measured and communicated with.

Accountability has to be built into the work.

If an employee is responsible for the quality of a client deliverable, that responsibility does not disappear because AI helped produce it. If a manager owns a business decision, accountability does not shift to the model because AI generated the recommendation.

And if an employee is responsible for protecting confidential information, that obligation remains whether the work is done manually or with an approved AI platform. Those expectations need to become explicit.

That means incorporating AI accountability into employee guidelines, performance expectations and management practices.

IBM has noted that AI is already making traditional performance evaluation more difficult. That is not surprising.  If AI reduces a four-hour task to 20 minutes, productivity alone becomes a less useful measure. The more important questions may be whether the employee knows what to verify, where judgment is still required, when to escalate and whether the use of AI actually improves the outcome.

That has implications for KPIs.

Organizations cannot tell employees to use AI responsibly while measuring them only on speed and volume. If careful review is expected, performance systems need to reinforce it.

The better measures may be less about frequency of AI use and more about quality of judgment: identifying opportunities to improve a workflow, validating output, recognizing risk, escalating appropriately and sharing useful practices across teams. That is where accountability starts to move from policy into culture.

Training has to move beyond tool use

Most AI training still focuses on functionality. How to prompt. Which tools are approved. What data cannot be entered. What security rules apply. Those remain necessary.

But employees and managers increasingly need training around judgment. What happens when the AI answer appears mostly correct, but not entirely? What if the employee is under significant time pressure? What if the AI recommendation conflicts with the employee’s own expertise? What if the output is client-facing? What if an AI agent can take an action rather than simply suggest one and takes it?

These are the kinds of scenarios organizations need to start practicing.

JPMorganChase has taken an interesting approach to this broader challenge through peer-led learning communities where employees share practical experience and work through real problems. That matters because AI capability is developing too quickly for periodic training alone to keep pace.

Employees need ongoing communication around what is working, where mistakes occurred, what the organization has learned and where human judgment remains essential. Managers need guidance on how to reinforce those expectations.

Leadership needs to consistently communicate what responsible AI use looks like.

That is why communications is becoming part of AI governance. A policy on an intranet cannot create the behavior by itself.

The practical question for organizations now is whether they could answer these questions today. Who owns the decision when AI is involved? Who can override it? Where does autonomy stop? What triggers escalation? And which decisions should remain human-only?

If those answers are unclear, the governance model probably is too.

Try mapping one real AI-enabled workflow against the decision-rights framework above and you may find the gaps are not in the technology at all, but in the operating model around it.

I’d be interested to hear where organizations are finding the hardest line to draw.

Share This Story With Friends & Colleagues.