WEBVTT

00:01.600 --> 00:04.880
As of today, I still think there is a

00:05.080 --> 00:08.580
25% chance we will get superintelligence
by 2030.

00:09.620 --> 00:13.480
There's at least a 10% chance literally
every person on Earth

00:13.520 --> 00:17.440
will die by 2030. There's at least

00:17.500 --> 00:21.080
a 10% chance we will get a permanent
dictatorship.

00:21.340 --> 00:25.300
A handful of people will run the entire
world for centuries

00:25.380 --> 00:27.740
together, with no ability to, you know,
overthrow them.

00:28.300 --> 00:30.520
At least 10% chance of this by 2030.

00:31.560 --> 00:35.060
The only way all these numbers change
is if there is a

00:35.120 --> 00:39.100
political movement to pause AI research
that succeeds within the next five years.

00:40.320 --> 00:44.120
Uh,
I'm going to now describe my technical

00:44.320 --> 00:44.980
I think this.

00:50.580 --> 00:50.589
(electronic music)

00:50.640 --> 00:54.580
Creation of artificial superintelligence
is likely the most important event

00:54.620 --> 00:57.650
of approximately 10,000 years of human
history.

00:58.900 --> 01:00.540
It is more important than Industrial

01:00.600 --> 01:03.680
Revolution/Newton/French

01:03.760 --> 01:06.060
Revolution/printing press.

01:07.200 --> 01:10.940
It is more important than invention of
nuclear weapons, as an

01:11.020 --> 01:14.980
ASI will accelerate creation of weapons
more dangerous

01:15.040 --> 01:15.660
than nukes.

01:16.780 --> 01:20.560
It is definitely more important than the
creation of the internet and all

01:20.660 --> 01:22.180
Silicon Valley startups.

01:23.260 --> 01:27.080
If the takeoff is fast enough,
creation of superintelligence

01:27.120 --> 01:30.820
may be the most important event in the
approximately 14

01:31.000 --> 01:33.420
billion year history of the universe.

01:34.040 --> 01:37.820
It could end up more important than the
evolutionary history of all

01:37.960 --> 01:41.620
other life forms on Earth.
And to the best of our current

01:41.680 --> 01:45.500
knowledge,
Earth is the only place in the universe

01:45.540 --> 01:46.640
intelligent life.

01:47.840 --> 01:51.230
Scaling of RL compute still seems
important and

01:51.340 --> 01:52.020
unsolved.

01:53.100 --> 01:56.500
We still seem to have spent less than $1
million per

01:56.560 --> 02:00.220
inference, and can, in theory,
spend at least $1

02:00.400 --> 02:04.200
billion on it. In my head, it is an open

02:04.260 --> 02:07.370
question whether the labs tried it
and got bad results,

02:08.120 --> 02:11.320
or if there's some other fixable research
or infra

02:11.420 --> 02:15.220
bottleneck that is preventing them from
trying it.

02:15.360 --> 02:19.120
Meta-graph on this still seems useful,
even though it is making

02:19.200 --> 02:20.140
simplifications.

02:21.340 --> 02:22.700
This is still important.

02:23.700 --> 02:27.280
If we have one AI that is superhuman,
we can

02:27.320 --> 02:31.120
probably run at least 100,000 copies of it
at

02:31.420 --> 02:33.780
100 times the speed of human thinking.

02:34.340 --> 02:38.120
This means that once the first slightly
superhuman AI is

02:38.160 --> 02:41.900
invented,
we will go from slightly superhuman to

02:42.100 --> 02:45.720
vastly superhuman in a very short span of
time.

02:46.960 --> 02:50.740
I would nowadays more emphasize the
meta-learning point below,

02:50.780 --> 02:54.580
though,
because that decides what happens until we

02:54.600 --> 02:55.860
superhuman AI.

02:56.840 --> 03:00.240
Deferring to experts
is obviously still important.

03:00.460 --> 03:04.300
I value my inside view a lot,
but I value expert

03:04.340 --> 03:05.940
opinion some amount too.

03:07.160 --> 03:11.020
Many researchers who pretend not to be
doomer also have

03:11.080 --> 03:13.340
timelines in the next five to 10 years.

03:14.040 --> 03:17.400
Examples, Rich Sutton, Ilia Sutskever,

03:17.980 --> 03:19.740
Andrej Karpathy, et cetera.

03:20.760 --> 03:24.460
There is the obvious list of doomer
researchers, example,

03:24.860 --> 03:28.660
Geoffrey Hinton, Yoshua Bengio,
Stuart Russell,

03:28.880 --> 03:29.340
et cetera.

03:30.380 --> 03:34.300
Heads of all frontier labs agree
that their own tech could

03:34.380 --> 03:35.780
cause human extinction.

03:36.640 --> 03:39.840
Example, Elon Musk, Sam Altman,

03:40.460 --> 03:43.960
Dario Amodei, Demis Hassabis, et cetera.

03:45.000 --> 03:48.250
Generalization and meta-learning
are related.

03:48.990 --> 03:52.980
Newer models aren't just memorizing more
skills, but are more

03:53.040 --> 03:56.780
capable of learning new skills on their
own without human

03:56.800 --> 03:57.780
data guiding them.

03:58.750 --> 04:02.620
There is a difference between training AI
such that it learns skills that

04:02.680 --> 04:06.580
humans have already have,
versus training AI to know

04:06.720 --> 04:09.060
how to learn new skills on its own.

04:09.880 --> 04:12.820
The difference between these two
is not binary.

04:13.340 --> 04:15.280
There are levels to generalization.

04:16.260 --> 04:18.640
Example of low level of generalization,

04:19.459 --> 04:23.340
seeing lots of English to French
translations and learning to

04:23.360 --> 04:26.100
become good at English to French
translation.

04:27.080 --> 04:29.580
Example of medium level of generalization,

04:30.440 --> 04:34.420
seeing many solved competitive programming
puzzles and solving

04:34.580 --> 04:38.540
another puzzle with an algo similar to one
it has seen before for

04:38.550 --> 04:39.490
a different puzzle.

04:40.300 --> 04:43.940
Seeing lots of English text
and a little Hindi text,

04:44.440 --> 04:48.430
inferring the grammar similarities
and differences across human languages,

04:48.980 --> 04:52.180
and then learning to speak fluent Hindi as
a result.

04:53.000 --> 04:56.820
Yeah, lol. This is a real result,
and an old

04:56.960 --> 04:57.140
one.

04:58.090 --> 05:00.560
Example of high level of generalization,

05:01.420 --> 05:05.280
seeing a theorem from economics
and realizing an analogous

05:05.300 --> 05:09.240
version of it also applies to biology,
thereby solving

05:09.280 --> 05:11.410
an open problem in biology.

05:12.480 --> 05:16.150
GPT-2 to GPT-5 has led to

05:16.300 --> 05:20.140
immense progress in both learning new
skills and

05:20.200 --> 05:24.010
learning how to learn new skills.
The latter

05:24.060 --> 05:25.290
is more important though.

05:26.280 --> 05:30.240
The best way to test the latter
is to give AI problems that

05:30.300 --> 05:34.080
a human has never seen
and no human knows how to

05:34.200 --> 05:38.160
solve. This could be simple tasks,
like translation

05:38.200 --> 05:41.600
in apocryphal ancient languages,
or complex

05:41.660 --> 05:45.540
tasks,
like proposing novel molecular bio

05:45.560 --> 05:45.840
run.

05:46.800 --> 05:50.540
Continual learning is a hard problem,
but my hunch

05:50.620 --> 05:53.360
is one or two breakthroughs might solve
it.

05:54.480 --> 05:58.080
AI weights are currently static,
and hence, AI

05:58.180 --> 06:00.240
behaves as if it has amnesia.

06:01.060 --> 06:05.000
Once a chain of thought is complete,
the AI forgets it ever

06:05.040 --> 06:08.988
did the task.Naive way to make sure AI

06:09.008 --> 06:12.778
remembers its previous inferences
is to fine-tune it

06:12.848 --> 06:16.718
on this data. But fine-tuning LLMs is

06:16.728 --> 06:18.348
very sample inefficient.

06:19.208 --> 06:23.178
A slightly better approach
is to put this data back into the chain of

06:23.288 --> 06:27.068
thought itself.
But there may be limits to how long

06:27.178 --> 06:30.748
chains of thought can be.
Open research question.

06:31.648 --> 06:33.528
We might discover better approaches.

06:34.708 --> 06:38.488
Operating in the real world can be
expensive in domains like

06:38.548 --> 06:42.428
biotech. Therefore,
we must be sample efficient

06:42.588 --> 06:46.328
in terms of how much data the AI requires
from real-world

06:46.388 --> 06:50.168
experiments before it learns useful
insights from it.

06:50.228 --> 06:53.488
Human language has features not present in
animal language,

06:54.148 --> 06:57.828
and this is likely an important part of
why humans can build spacecraft and

06:57.948 --> 07:00.288
colonize the Earth, but apes can't.

07:01.468 --> 07:04.808
AI already has picked up all these
features of human language.

07:06.048 --> 07:09.768
Go read Hockett's views on what features
separate human language from animal

07:09.828 --> 07:10.328
language.

07:11.748 --> 07:13.888
Humans use language for communication.

07:14.788 --> 07:18.058
Humans might also use language as part of
their reasoning process,

07:18.568 --> 07:21.868
along with other modules,
such as motor skills, spatial

07:21.928 --> 07:23.388
visualization, et cetera.

07:24.368 --> 07:28.328
Both of these are hypotheses for what
separates humans from apes, and

07:28.338 --> 07:29.988
I think there's a good chance they're
true.

07:31.188 --> 07:34.768
Other hypotheses include bigger birth
canal and brain size

07:35.228 --> 07:38.008
and evolutionary pressures to win social
games.

07:38.988 --> 07:42.928
These hypotheses seem compatible with the
hypothesis that language is most

07:42.968 --> 07:43.548
important.

07:44.768 --> 07:48.388
Depending on how you measure it,
AI may now be the second

07:48.508 --> 07:51.388
most complex object in the observable
universe.

07:52.268 --> 07:56.088
More complex than ape brain,
but less complex than human

07:56.168 --> 07:56.488
brain.

07:57.718 --> 08:00.728
Model scaling for text models might be
saturating.

08:01.148 --> 08:04.388
Open research question. But it
is definitely not

08:04.508 --> 08:07.828
saturated for images, video or robotics.

08:09.028 --> 08:12.948
As of December 2025,
text models intuitively feel

08:12.988 --> 08:16.918
better than image models,
and image models intuitively feel better

08:17.048 --> 08:21.008
than video models,
and video models intuitively feel better

08:21.048 --> 08:24.838
than VLAs for robotics. I

08:24.868 --> 08:28.588
think this is mostly just because of
higher compute requirements for the latter

08:28.668 --> 08:29.108
models.

08:30.388 --> 08:33.608
Minor update as of January 6th, 2026.

08:34.308 --> 08:38.178
Robotics has an additional bottleneck
where big enough datasets are not

08:38.268 --> 08:38.688
available.

08:39.578 --> 08:43.288
Manually generating them is expensive,
but probably affordable

08:43.328 --> 08:47.068
given current AI R&D budgets. Most

08:47.108 --> 08:50.988
people seem to IMO suck at forecasting AI

08:51.108 --> 08:55.068
progress even one year into the future,
let alone

08:55.148 --> 08:56.048
five or 10.

08:57.228 --> 09:00.988
Since 2022,
I am used to watching people on Twitter

09:01.068 --> 09:04.668
predictions of some specific benchmark
or skill

09:05.028 --> 09:08.948
X that will never get solved,
only for it to get solved

09:09.028 --> 09:09.808
one year later.

09:10.968 --> 09:14.948
Your specific AI can't do XYZ task

09:15.308 --> 09:18.788
that is trivial for humans
is not impressive to me