Experienced software developers using AI tools saw a 19% increase in task completion time

A recent study carried out by AI research nonprofit METR has started a lot of conversation after it revealed a counterintuitive detail — when experienced software developers had to work with AI tools while working in familiar codebases, they completed their tasks 19% slower. 

The study was carried out earlier this year on a group of developers with years of experience. The group predominantly used Cursor as the AI tool to help them complete tasks in open-source projects they knew well.

Prior to the study, the open-source developers believed that working with an AI would help them get things done faster, with many estimating it would decrease task completion time by 24%.

Even after the tasks were completed with AI, the developers were convinced they had decreased the time it took to finish by 20%.

However, the study reported opposite outcomes; collaborating with an AI increased task completion time by 19%.

There’s a lot we still don’t know about AI

The results have elicited surprise among developers on X, but the study’s lead authors, Joel Becker and Nate Rush, were shocked by them. Rush had expected “a 2x speed up, somewhat obviously” and had written that down before the experiment began.

Before now, there was widespread belief that AI makes human engineers far more productive, and several companies have invested substantially in companies selling AI products to aid software development based on those projections.

However, while the study has highlighted a false assumption, it has not changed the common fear of AI stealing human jobs. In fact, the shift has already begun, even in programming careers, with AI expected to replace entry-level coding positions.

Dario Amodei, CEO of Anthropic, is one of those who shares this sentiment and believes it could happen in the next one to five years.

Unlike previous studies that encouraged the belief that AI would quicken coders and software developers because they reported significant gains, the METR study has shown that they don’t apply to all software development scenarios.

In particular, the study was able to prove that experienced developers who had intimate knowledge of the quirks and requirements of large, established open source codebases were the most affected.

The study’s authors have also pointed out that other studies are known to rely on software development benchmarks for AI, and they sometimes misrepresent real-world tasks.

As for what caused the coders to work more slowly, things like needing to spend time going over and correcting what the AI models suggested were noted.

The authors do not expect this slowdown to apply in other scenarios, and despite the results,  the majority of the study’s participants, as well as the study’s authors, continue to use Cursor today.

Why? The authors say it is because AI makes the experience easier, which in turn, makes it feel less like a chore.

Software developers are not worried about AI, even with recent tech layoffs

Given how enthusiastic the METR study’s authors and the study’s participants are to forgive Claude for slowing them down, and their continued use of the tool, it is clear that more tech professionals see AI more as a collaborator rather than as a threat.

Unfortunately, there may be trouble on the horizon as the tech industry continues to witness layoffs explicitly linked to AI adoption at major companies.

This year, Microsoft has already dismissed up to 9,000 employees, with claims that 40% of the recent layoffs targeted software engineers because code-writing tasks were outsourced to AI tools.

The company has justified the layoffs with record-breaking profits that have kept its market cap hovering just behind Nvidia’s and ahead of Apple’s.

Google is another major tech company that has laid off hundreds of workers following its investment in AI startup Anthropic, though not all the dismissed workers were replaced with AI.

Other companies, like Salesforce and Intel, have also cited a shift towards AI in layoff announcements. Salesforce has cut 1,000 jobs this year to focus on AI roles, Intel has reduced its workforce by 15,000, and IBM plans to replace 30% of its back-office roles with AI by 2030.

Cryptopolitan Academy: Coming Soon - A New Way to Earn Passive Income with DeFi in 2025. Learn More