Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford

What people said

Ianjit · 2026-01 · Reference

This Stanford study on developer productivity found 0 correlation between developers assessment of their own productivity and independent measures of their productivity. Any anecdotal evidence from developers on how AI has made them more or less productive is worthless. [VIDEO]

Read the original HN comment ↗

keeda · 2025-12 · Reference

As mentioned in the thread I linked, they acknowlege the productivity puzzle and try to control for it in their studies. It's worth reading them in detail, I feel like many of them did a decent job controlling for many factors. For instance, when measure the number of PRs they ensure that each one goes through the same review process whether AI-assisted or not, ensuring these PRs meet the same quality standards as humans. Furthermore, they did this as a randomly controlled trial comparing engineers without AI to those with AI (in most cases, the same ones over time!) which does control for a lot of the issues with using PRs in isolation as a holistic view of productivity. >... whose participants are working for organizations that are heavily invested in future success of AI. That seems pretty ad hom, unless you want to claim they are faking the data. Along with co-authors who are from premier institutes like NBER, MIT, UPenn, Princeton, etc. And here's the kicker: they all converge on a similar range of productivity boost, such as the Stanford study: > [VIDEO] (from Stanford, not an RCT, but the largest scale with actual commits from 100K developers across 600+ companies, and tries to account for reworking AI output. Same guys behind the "ghost engineers" story. The preponderence of evidence paints a very clear picture. The alternative hypothesis is that ALL these institutes and companies are colluding. Occam's razor and all that.

Read the original HN comment ↗

keeda · 2025-12 · Reference

The folks at Stanford in this video have a somewhat similar dataset, and they account for "code churn" i.e. reworking AI output: [VIDEO] -- I think they do so by tracking if the same lines of code are changed in subsequent commits. Maybe something to consider.

Read the original HN comment ↗