The question behind the productivity promise

You ask AI for a draft, an explanation or a piece of code. The answer arrives quickly, and the work feels easier. But did you finish sooner, or did you simply spend less energy getting there?

That distinction matters for workers deciding which tasks to delegate and for employers deciding whether AI investments are paying off. Today's brief examines existing research, not a study released today.

Known: easier can feel faster

A preregistered study by Sunny Yu and colleagues, posted in May 2026 and listed for the Cognitive Science Society proceedings, involved 1,237 participants: 401 predicted task times and 836 completed tasks. Researchers tested short activities such as information seeking and content creation, using an embedded GPT-4o interface. [1]

Participants expected AI assistance to save time. The study found no statistically significant difference in actual completion time between assisted and independent work overall, while AI users reported less subjective effort. Analyses excluded incorrect or inadequate responses. [1]

The finding concerns average expectations across separate groups; it does not show that every individual misjudged their own performance. Nor does a nonsignificant difference prove the two workflows are exactly equivalent. It does show why perceived ease is an unreliable substitute for timing a task. [1]

Known: the same mismatch appeared in coding

In METR's early-2025 randomized trial, 16 experienced open-source developers completed 246 real tasks in familiar repositories. They expected AI to reduce completion time by 24%. Afterward, they still estimated a 20% reduction. Measured time instead increased by 19%. The trial largely used Cursor with Claude 3.5 and 3.7 Sonnet. [2]

This was a small, specialized sample using older tools. It establishes a mismatch in that setting, not that AI makes all developers slower today.

Three questions, three different measurements

Speed

How long until an acceptable result, including checking and corrections?

Quality

How accurate, useful and complete is the final result?

Effort

How demanding does the work feel?

Uncertain: current tools may change the result

METR's February 2026 update complicates the older headline. Later experiments produced raw estimates suggesting shorter task times, but confidence intervals included no speedup. Researchers warned that selection effects and difficulties measuring work with concurrent agents made the results unreliable. Some developers declined to participate because they did not want to work without AI. [3]

METR considers greater speedups with newer tools likely, but says these data provide only weak evidence about their size. We should preserve both facts: the earlier slowdown was measured, and it is not a reliable estimate of the effect of current tools. [3]

The short-task study also cannot settle productivity in a whole job. Its model, interface, tasks and participant experience differ from prolonged professional workflows. [1]

Speculation: comfort could distort how organizations measure value

A plausible risk is that teams use satisfaction or output volume as a proxy for completed work. AI could make drafting more pleasant while shifting effort into review, correction and coordination. That is our interpretation, not a demonstrated economy-wide effect.

Reduced effort can still be valuable. A tool that helps someone finish demanding work with less exhaustion has a benefit worth measuring, even when it saves little time.

Preparation: run a small reality check

Choose a recurring, low-stakes task. Define what a good result looks like before starting. Compare several similar tasks with and without AI, alternating the order to reduce practice effects.

  1. Time the whole workflow: setup, prompting, waiting, reading, verification and corrections.
  2. Check quality consistently: use the same acceptance criteria for both methods.
  3. Rate effort separately: a simple 1–5 score can help you distinguish energy savings from time savings.
  4. Record the tool and date: rerun the comparison when your workflow or model changes substantially.

This is a practical personal comparison, not a controlled scientific experiment. Its purpose is to improve your own decisions.

Our read

Use AI where it improves the outcome you actually care about. Faster delivery, better work and less exhaustion are all useful. Measure them separately, and keep the benefits the evidence supports.

Primary sources

Evidence note · October 6, 2026: This is a synthesis of research published in 2025–2026. Results are specific to the tested tasks and tools. Speculation and the practical exercise are editorial interpretation, not findings reported by the studies.
Back to The Neural Tide