Friday, June 5, 2026 · 9:41 AM
ok dumb question
is “recursive self-improvement” just AI people saying skynet with footnotes?
kinda, but less movie trailer
it means an AI gets good enough at AI R&D that it can help build the next AI
then that next AI helps build the next one
so like a snake eating its own tail, but with GPUs
more like a lab where the interns become the senior scientists overnight
and then train even faster interns
anthropic thinks we’re already there?
no. they’re careful on that
their claim is: the slope is getting weird
AI is already speeding up AI development inside Anthropic
what’s the actual evidence, not vibes
first: coding
as of May 2026, Claude authored 80%+ of code merged into Anthropic’s codebase
before Claude Code’s Feb 2025 research preview, that number was low single digits
😮80% is not “autocomplete helped me name a variable”
right
and their typical engineer merged about 8x as much code per day in Q2 2026 as in 2024
they caveat that lines of code overstate real productivity, which is fair
yeah lines of code can be a landfill metric
exactly
but they also cite a March 2026 poll of 130 research staff
median estimate: around 4x more output with Mythos Preview than with no AI models
yep, and they say it’s probably high
the useful bit is that multiple signals point the same way: more work is getting pushed through
but can it do research or just crank out code
that’s the key split in the piece
AI is very strong at execution: write code, run experiments, fix infra, reproduce results
the open question is taste: choosing the right goal, trusting the result, killing dead ends
taste is annoyingly hard to benchmark
very
Anthropic uses a career-ladder analogy: junior people execute tasks, senior people choose what matters
Claude is climbing that ladder, but it’s not at “pick next quarter’s research agenda” yet
what are the scary benchmark numbers
task length is the big one
AI task horizons have been doubling roughly every 4 months, faster than an earlier 7-month trend
Opus 3 did ~4-minute human tasks in Mar 2024. Sonnet 3.7 did ~1.5-hour tasks a year later
Opus 4.6 managed 12-hour tasks a year after that
wait, so the thing is not “smarter answers”
it’s “longer leash”
yes. huge
a model that can stay coherent for 12 hours can stop being a chatbot and start being a worker
ok that’s the part i was missing
the article also points to research-ish tests
one benchmark asks models to reproduce published research from code and data
systems went from about 20% success in 2024 to saturating it 15 months later
they run a mini research loop: make training code faster while keeping correctness checks fixed
in May 2025, Claude averaged about 3x speedup
by April 2026, it hit about 52x
lol yes, but don’t anchor on the absolute number
Anthropic says the comparison across time is the point
same setup, much stronger model
what about it choosing experiments by itself?
there’s an AI safety project example
agents studied whether a weaker model could supervise a stronger one
humans picked the problem and scoring rubric, but agents designed the experiments
two human researchers got ~23% of the gap in a week. agents got 97% over 800 cumulative hours
so humans still set the game, AI played it better
that’s the cleanest read
and it’s why the future hinge is judgment, not keyboard speed
here’s my naive take: if humans still choose goals, we’re safe-ish?
maybe, but that’s the counterintuitive twist
even if AI never gets great taste, automation can still compound
if humans only do the 5% direction-setting and AI does the other 95%, each human steers way more work
so the bottleneck just moves upward
exactly. Anthropic brings up Amdahl’s law
speed up one part of the process and the slow part becomes the cap
right now, code review and “which idea should we pursue?” are turning into bottlenecks
that feels less like explosion and more like plumbing failure
great phrase, yeah
tons more code, tools, simulations, fixes, ideas
then humans and institutions have to absorb it without faceplanting
what futures do they lay out?
3 buckets
1: the curve stalls or hits supply limits like compute, energy, chips, or architecture
2: labs keep compounding efficiency, with humans still steering
3: full recursive self-improvement, where AI systems build successors mostly on their own
and they think 1 is unlikely?
they say they include it for completeness, but they’re more worried about 2 and 3
because measured capabilities have not shown the curve bending yet
what’s the actual danger in 3
control
if models build successors, alignment failures could get copied forward or amplified
and if humans can’t inspect the virtual lab fast enough, we may not know which trendline we’re on
but they also say it could help science and medicine?
yes. they’re not only doomposting
automated R&D could speed up science, healthcare, robotics, clinical trials, coordination tools
but daily life still hits physical and social bottlenecks
more intelligence can’t make a 10-year drug safety record appear tomorrow
can’t hold elections sooner than the constitution says
can’t turn a stranger into your old friend in a weekend
the lab may run at compute speed
the human world still runs on bodies, laws, trust, supply chains, and people being annoying
so what do they want people to do?
build credible ways to slow or pause frontier development if needed
not just one lab self-owning while others sprint ahead
they want verification systems, shared triggers, and broader deliberation with policymakers and civil society
verification like arms control?
yes, and they say AI is harder in some ways
training runs are easier to hide than missile silos
the incentive to defect quietly is enormous
cool cool cool, easy monday agenda
practical takeaway: watch the bottlenecks
not just “is the model smarter?”
ask what part of AI R&D is still human-limited, and how fast that part is shrinking
so the headline is: AI isn’t building itself yet, but it’s already helping build the builders
yep. annoying sentence, but accurate
and if that feedback loop keeps tightening, society needs options before the options get fake
ok i’m gonna go stare at a wall for 3 mins
reasonable response tbh
drink water too
Read Fri, Jun 5 · 10:02 AM