Council Post: When AI Writes Your Code, Who Owns The ‘Why’?

Naman Kher, VP & Global Head of Solutions at Dexian | AI & Enterprise Technology Leader.

getty

Not long ago, I was speaking with a group of engineering leaders about how AI is changing software delivery. The conversation quickly moved beyond productivity gains and coding assistants to a more fundamental question: As AI begins writing a growing share of enterprise software, who is responsible for explaining why that software behaves the way it does?

It sounds like a simple question until an auditor, regulator, customer or executive asks it.

Increasingly, one answer is appearing inside engineering organizations: “The model suggested it.” That answer may explain how code was created. It does not explain why it belongs in production or who should maintain accountability for it. ​

AI is not eliminating human responsibility from software development. It is relocating it.

Historically, engineers spent much of their time translating requirements into code. As AI becomes increasingly capable of generating implementation details, human responsibility moves higher in the stack. Defining intent, establishing governance, validating outcomes and preserving the decision trail behind every change.

In many organizations, the biggest challenge becomes proving why the software was written the way it was.

Stack Overflow’s "2025 Developer Survey" found that "51% of professional developers use AI tools daily." Research from DX found that approximately 27% of merged code is now AI-generated. Clearly, AI-assisted development is becoming part of everyday software delivery.

The risk is not that AI is inherently unsafe. The risk is that AI-assisted development without governance becomes difficult to audit, explain and defend. Software that cannot be explained eventually becomes software that cannot be trusted.

​From Prompt-First To Specification-Driven Development

With every major technology shift, adoption tends to move faster than governance. We saw it with cloud computing, mobile platforms and low-code development. We are seeing it again with generative AI.

Many teams today operate in a prompt-first model. An engineer enters a prompt, reviews the generated output, makes minor adjustments and commits the code. The code may work perfectly, but there is often little documentation connecting the implementation to the original business intent.

Requirements may live in scattered tickets, chat conversations or individual memory. The reasoning behind a decision becomes increasingly difficult to reconstruct over time.

A specification-driven approach addresses this challenge without slowing delivery. Before code is generated, the intended outcome is documented in a concise specification. After the code is produced, reviewers validate that the implementation aligns with the approved intent.

The goal is traceability. When someone asks six months later why a function exists, the answer should be a documented decision rather than a collective guess.

Governance Requires Defense In Depth

No single control is sufficient for AI-assisted development. The volume and speed of generated code make it unrealistic to rely on one review stage or one individual reviewer.

Effective governance works as a layered system:

• Generation policies define what AI tools are permitted to create.
• Local developer checks identify issues before code leaves the workstation.
• AI-assisted reviews help detect patterns and vulnerabilities at scale.
• CI/CD pipeline gates enforce security, compliance and quality standards automatically.
• Repository-level controls provide a final layer of protection before code reaches production.

Any single control can fail, but well-designed systems assume that eventually one will.

So, what happens when a control fails? What catches the issue next? Governance is most effective when it is treated as an engineered system rather than a compliance exercise.

Governance Is Becoming A Competitive Advantage

For years, governance has often been viewed as a necessary cost of doing business. AI is changing that equation.

Organizations that can rapidly trace requirements to implementation, demonstrate compliance, explain design decisions and produce audit evidence gain a meaningful advantage. They move through customer reviews faster, respond to regulatory inquiries more effectively and reduce risk during acquisitions, security assessments and due diligence exercises.

In an environment where software is increasingly generated at machine speed, trust becomes a differentiator.

The companies that scale AI most successfully may not be the ones generating the most code. They may be the ones that can explain and defend their software most effectively.

The Test To Run This Week

If you want to understand the maturity of your AI-assisted development practices, select a recent pull request from a business-critical application and ask three questions:

1. What requirement or specification does this change implement?

2. Where is that requirement documented?

3. What review confirmed that the implementation aligns with the intended outcome?

If those answers are immediate and clear, you likely have a system. But if the answers are buried in messages, scattered across tickets or dependent on individual memory, you have identified a governance gap that AI will only amplify as adoption grows.

The Bottom Line​

For decades, software engineering was primarily about writing code. AI is changing that equation.

As machines assume more of the implementation work, the highest-value human contribution shifts toward intent, governance, oversight and accountability.

The organizations that succeed in the next phase of AI adoption will not necessarily be the ones that generate the most code. They will be the ones that can explain it, defend it and trust it.

In the age of AI-assisted development, code may increasingly be written by machines, but accountability remains a human responsibility.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?