Writing code versus shipping code: Productivity effects across generations of AI coding tools - CEPR

Writing code versus shipping code: Productivity effects across generations of AI coding tools - CEPR

Introduction

The promise of AI coding assistants has shifted dramatically from merely accelerating keystrokes to fundamentally altering how software reaches production. As organisations adopt tools ranging from GitHub Copilot to autonomous agents like Devin, the critical question is no longer how fast developers can write functions, but whether those functions actually ship, scale, and survive real-world constraints.

From autocomplete to agents: three generations of AI coding assistants

The first generation, epitomised by TabNine and early Copilot, operated as sophisticated autocomplete — predicting the next token within a single file. Developers reported 20–30% faster boilerplate creation, yet integration bugs often negated gains because the model lacked repository context. The second generation, led by ChatGPT-4 and Copilot Chat, introduced conversational refactoring and test generation across multiple files. A 2023 JetBrains survey showed 55% of UK professionals using these tools daily, but median pull-request cycle time barely improved because reviewers still spent hours verifying AI-generated edge cases. The third generation — agentic systems such as Cursor Composer, Windsurf, and Factory — can plan, execute, and iterate on multi-step tasks: scaffolding a new microservice, writing migration scripts, updating CI pipelines, and opening a draft PR. Early adopters at Monzo and Revolut report 40% reduction in ticket-to-deploy latency, provided the agent runs inside a sandboxed staging environment with automated contract tests.

Writing code faster does not equal shipping code faster

Productivity metrics that count lines of code or completed Jira tickets miss the hidden costs of AI-generated complexity. A 2024 CEPR working paper analysing 12,000 PRs across five European fintechs found that while AI-assisted commits were 35% larger, they introduced 18% more post-merge incidents related to observability gaps and database migration rollbacks. The bottleneck has moved from authoring to validation: security scanning, contract testing, and compliance checks now dominate lead time. Teams that pair agentic tools with policy-as-code gates — for example, requiring OPA rules to pass before merge — reclaim the speed advantage. Conversely, organisations that treat AI output as “senior-developer quality” without additional guardrails see mean-time-to-recovery increase because subtle logic errors propagate further before detection.

The human factor: experience, trust, and the changing role of the developer

Generational tool effects differ sharply by engineer seniority. Junior developers using second-generation chat interfaces often accept hallucinated APIs, inflating rework; however, when paired with third-generation agents that surface diff summaries and run tests automatically, their onboarding time to first production deploy drops from three months to six weeks at firms like Graphcore. Senior engineers, meanwhile, shift from syntax decisions to architectural guardrails: defining the contracts an agent must satisfy, reviewing generated threat models, and curating the retrieval-augmented context the agent consumes. Trust calibration becomes a skill — knowing when to let the agent refactor a legacy module versus when to intervene manually. UK hiring managers now list “AI-assisted delivery judgement” alongside Kubernetes and Terraform in job specs, signalling that the productivity lever has moved from typing speed to verification discipline.

Conclusion

Across three generations of AI coding tools, the productivity frontier has migrated from writing code to shipping code — and the winners are teams that embed agents inside rigorous, automated validation pipelines rather than treating generated output as finished work. As the UK tech sector navigates tighter regulatory scrutiny and cloud-cost pressures, the organisations that measure success by deployed, observable, and compliant increments — not by keystrokes saved — will capture the real gains from this technological wave.

Comments

Popular posts from this blog

Why IBM's Multi-Agent AI is a Turning Point for Enterprise Dev Teams

How Notion AI Is Reshaping Team Project Management

AI Tools Compared: ChatGPT, Deepseek, and CoPilot