AI work needs a recovery test for the human skill it replaces
By OnTheRice · Sources and limitations · Report a correction
Research reference · source time Not applicable: no market-price call. · not a current quote or execution price · Reference source
Source confidenceStrong for the institutional report of the experiments; moderate for workplace transfer. No independent workplace replication was checked.
What’s changing?
We connect Berkeley's short-task experiments with the OTR data on faster, cheaper agent work. The study recruited 1,222 people across three experiments and found weaker persistence or accuracy after AI help was withdrawn in the tested tasks.
Key points
- We would ask analysts to explain one evidence chain without AI before showing agent output. Later, we would remove assistance for a small matching task and compare accuracy, completion and skipped questions. The practical target is retained judgement, not less AI use for its own sake. We would use 20 matched questions per participant, compare answer-only and explanation-first groups, and define performance as correct unaided answers after assistance ends. This proposed test has not been run.
What could change this view?
- A trial using guided explanations and independent checking may preserve human performance. The Berkeley tasks may not transfer to experienced analysts or longer workflows.
Bottom line
We would include an unaided recovery exercise in AI training. A fast process is more dependable when people can still detect a wrong answer and continue during an outage.
Its source facts and prices still need independent checks.
How likely is this call? · Three frameworks
This signal has no investible bullish or bearish call. A market-direction percentage does not apply.
Evidence behind this view
Checks whether separate pieces of evidence support the same view.
The OTR technology section retains the controlled-task result and proposes independent reasoning before agent output. Another OTR report focuses on small-model efficiency, creating the workflow connection.
What past price moves show
Compares today’s price moves with recorded past patterns.
Not applicable: this is a technology synthesis, not an investible or price forecast.
Evidence against this view
Shows evidence and risks that could challenge this view.
AI can provide practice and feedback. Short-term withdrawal effects do not establish a lasting loss of skill.
Why the direction changed
Explains recorded changes from expecting a rise to expecting a fall, or the reverse.
Learning review before publication
The supplied direction was retained: No investible directional call.
The supplied synthesis was retained.
Reason: Retain the recovery-test recommendation while keeping the Berkeley result bounded to its tested participants and short tasks. It does not establish permanent skill loss or transfer to every workplace.
Lessons considered: Keep the clocks separate; Look for independent evidence; Keep incomplete tests visible; Separate direction from explanation; Use lessons in the next call; Compare matching quantities.
This review belongs to the new draft before publication. Earlier published calls and their original deadlines remain unchanged.
What people say vs the facts
Compares recorded public discussion with checked market facts.
We separate an attention-grabbing claim about AI use from the narrower evidence: specific tasks, specific participants and short-term behaviour after help was removed.
How past forecasts performed
Shows recorded forecast results and what they can teach us.
We have not verified completed, matching technology forecasts in the OTR records inspected for this signal. The percentage is an uncalibrated judgement and is not a historical hit rate.
The pre-publication learning review is recorded under “Why the direction changed”, including cases where the supplied direction was retained.
Sources, original wording and price checks
Author-submitted research. The underlying facts and market prices have not been independently verified.
Original research record
We connect Berkeley's short-task experiments with the OTR data on faster, cheaper agent work. The study recruited 1,222 people across three experiments and found weaker persistence or accuracy after AI help was withdrawn in the tested tasks. This does not prove permanent harm. Our connection is that teams should test their ability to recover when an AI service fails, just as they test software recovery.
We would ask analysts to explain one evidence chain without AI before showing agent output. Later, we would remove assistance for a small matching task and compare accuracy, completion and skipped questions. The practical target is retained judgement, not less AI use for its own sake. We would use 20 matched questions per participant, compare answer-only and explanation-first groups, and define performance as correct unaided answers after assistance ends. This proposed test has not been run.
A trial using guided explanations and independent checking may preserve human performance. The Berkeley tasks may not transfer to experienced analysts or longer workflows.
We would include an unaided recovery exercise in AI training. A fast process is more dependable when people can still detect a wrong answer and continue during an outage.
Bayesian · what changed the estimate? We estimate a subjective 55% chance that a one-week task test will show lower unaided performance after answer-only assistance than after explanation-first assistance. This is an untested local experiment. we have not fitted a prior or evidence likelihoods, so this is not a computed posterior. de Finetti · are the comparisons fair? we treat repeated OTR editions as one underlying source when they reuse a report; our judgement must not gain confidence just from repetition. Kolmogorov · do the numbers agree? for the stated test event, the probability of it occurring and not occurring sums to 100%. None of these checks establishes historical calibration. This estimate is conditional on running the described pilot with its records and comparison method fixed before the test. No pilot outcome is claimed here.
The OTR technology section retains the controlled-task result and proposes independent reasoning before agent output. Another OTR report focuses on small-model efficiency, creating the workflow connection.
Not applicable: this is a technology synthesis, not an investible or price forecast.
AI can provide practice and feedback. Short-term withdrawal effects do not establish a lasting loss of skill.
Not applicable: no earlier market direction is assigned. We state the evidence dates and the condition that would change the technology view.
We separate an attention-grabbing claim about AI use from the narrower evidence: specific tasks, specific participants and short-term behaviour after help was removed.
We have not verified completed, matching technology forecasts in the OTR records inspected for this signal. The percentage is an uncalibrated judgement and is not a historical hit rate.
How likely is this call? · Three frameworks
Bayesian · what changed the estimate? We estimate a subjective 55% chance that a one-week task test will show lower unaided performance after answer-only assistance than after explanation-first assistance. This is an untested local experiment. we have not fitted a prior or evidence likelihoods, so this is not a computed posterior. de Finetti · are the comparisons fair? we treat repeated OTR editions as one underlying source when they reuse a report; our judgement must not gain confidence just from repetition. Kolmogorov · do the numbers agree? for the stated test event, the probability of it occurring and not occurring sums to 100%. None of these checks establishes historical calibration. This estimate is conditional on running the described pilot with its records and comparison method fixed before the test. No pilot outcome is claimed here.
Bayesian · what changed the estimate?
We estimate a subjective 55% chance that a one-week task test will show lower unaided performance after answer-only assistance than after explanation-first assistance. This is an untested local experiment. we have not fitted a prior or evidence likelihoods, so this is not a computed posterior.
de Finetti · are the comparisons fair?
we treat repeated OTR editions as one underlying source when they reuse a report; our judgement must not gain confidence just from repetition.
Kolmogorov · do the numbers agree?
for the stated test event, the probability of it occurring and not occurring sums to 100%. None of these checks establishes historical calibration. This estimate is conditional on running the described pilot with its records and comparison method fixed before the test. No pilot outcome is claimed here.
Evidence behind this view
The OTR technology section retains the controlled-task result and proposes independent reasoning before agent output. Another OTR report focuses on small-model efficiency, creating the workflow connection.
What past price moves show
Not applicable: this is a technology synthesis, not an investible or price forecast.
Evidence against this view
AI can provide practice and feedback. Short-term withdrawal effects do not establish a lasting loss of skill.
Why the direction changed
Not applicable: no earlier market direction is assigned. We state the evidence dates and the condition that would change the technology view.
What people say vs the facts
We separate an attention-grabbing claim about AI use from the narrower evidence: specific tasks, specific participants and short-term behaviour after help was removed.
How past forecasts performed
We have not verified completed, matching technology forecasts in the OTR records inspected for this signal. The percentage is an uncalibrated judgement and is not a historical hit rate.
Forecast prepared Publication time unverified
No explicit investible BULLISH or BEARISH call on this card.
What changed
Recorded public historyWe connect Berkeley's short-task experiments with the OTR data on faster, cheaper agent work. The study recruited 1,222 people across…
This is the first recorded public view. Further public updates will appear here.
- AI work needs a recovery test for the human skill it replaces
We connect Berkeley's short-task experiments with the OTR data on faster, cheaper agent work. The study recruited 1,222 people across three experiments and found weaker persistence or accuracy after AI help was withdrawn in the tested tasks.
Forecast prepared: 11 Oct, 02:53 SGTPublished by OTR: 11 Oct 2026, 03:26 SGT
Why the view changed
This is the first recorded view. Future source-backed updates will appear here.
Evidence, memory & review 2 linked sources
Directional takeaway: N/A
OTR reasoning: We connect Berkeley's short-task experiments with the OTR data on faster, cheaper agent work. The study recruited 1,222 people across three experiments and found weaker persistence or accuracy after AI help was withdrawn in the tested tasks.
Counterpoint: A trial using guided explanations and independent checking may preserve human performance. The Berkeley tasks may not transfer to experienced analysts or longer workflows.
Source trail
- news.berkeley.edu ↗The research record cites this source; its publication time has not been independently verified.
- platform.claude.com ↗The research record cites this source; its publication time has not been independently verified.
Review status: LEARNING REVIEWED WITH DISCLOSED LIMITS
No score or probability is presented.
Signals sharing sources
These records are in the same category and cite at least one of the same source documents. Shared evidence is not independent confirmation; each record keeps its own date and conclusion.