WEBVTT

00:00.820 --> 00:04.570
[speaker_0] I'm here in Paris on the last
day of our trip,

00:04.580 --> 00:08.200
am struck by,
that I've been reflecting on,

00:09.260 --> 00:12.080
People talking about the end of the world,

00:13.260 --> 00:17.180
human extinction,
in an emotionally removed,

00:17.360 --> 00:20.280
academic way. It's not working.
It's not resonating.

00:20.560 --> 00:20.780
So

00:21.660 --> 00:23.360
more than a year ago, Lewis Berman

00:25.600 --> 00:28.880
asked me to write an essay for the book he
was writing. It's called Catastrophe.

00:28.940 --> 00:30.910
You can get it on Amazon.
We'll put a link in the show notes.

00:32.259 --> 00:33.120
I wrote an essay for it.

00:34.120 --> 00:37.769
He asked me to pull no punches.
He asked me to really just write what I

00:37.800 --> 00:39.140
thought would happen

00:39.960 --> 00:40.600
at the end.

00:42.980 --> 00:45.540
It's not something I like to talk about.

00:45.600 --> 00:47.040
It's not something I like to think about.

00:47.280 --> 00:50.220
It does me no,

00:51.340 --> 00:52.380
gives me no joy

00:53.320 --> 00:53.720
to

00:54.580 --> 00:56.940
do what I'm about to do,
but I think it's necessary.

00:57.040 --> 00:59.760
So I'm going to read for you the essay
that I wrote for Lewis.

01:00.240 --> 01:01.440
His book, Catastrophe.

01:02.860 --> 01:06.680
It's painful for me to think

01:06.740 --> 01:10.680
about this.
It's painful for me to read it,

01:13.120 --> 01:16.340
If we don't start talking about these
things

01:18.360 --> 01:19.400
with genuine emotion

01:20.300 --> 01:21.440
and genuine urgency,

01:24.180 --> 01:25.620
nothing's going to change fast enough.

01:26.520 --> 01:30.160
So this is my attempt to,
in the most shocking way

01:30.240 --> 01:34.160
possible,
distill what I think will really happen at

01:34.200 --> 01:34.690
my family.

01:36.620 --> 01:37.220
Here's the essay.

01:38.160 --> 01:41.120
I'm a dad in Baltimore with boy-girl twins
who just went off to college.

01:42.560 --> 01:45.160
And I believe AI is going to kill my kids.

01:46.620 --> 01:47.480
And your kids too.

01:49.160 --> 01:53.140
I'm sorry to tell you,
nearly every qualified expert believes

01:54.520 --> 01:57.140
It was late March 2023
when I found this out.

01:57.260 --> 02:00.640
I was just cruising around the internet
reading news when I came across a piece

02:00.700 --> 02:04.600
Time magazine written by a man named
Eliezer Yudkowsky.

02:05.680 --> 02:08.020
I had never heard of him or AI safety.

02:09.080 --> 02:12.460
And holy fucking shit,
that day changed my life.

02:14.200 --> 02:17.900
Since then, I've learned
that nothing in human history compares to

02:17.960 --> 02:18.320
risk.

02:20.440 --> 02:24.400
It's like we're walking our families
across a crosswalk and there's a bus

02:24.440 --> 02:27.300
coming at us at 2,000 miles an hour.

02:28.640 --> 02:29.560
The bus is AI.

02:30.440 --> 02:31.940
It's coming to kill all of us.

02:32.820 --> 02:33.840
And our families too.

02:34.710 --> 02:37.180
And faster than any of us can imagine.

02:40.040 --> 02:43.920
We are running a crazy high risk
that we are literally the last

02:44.000 --> 02:44.460
humans.

02:46.080 --> 02:47.860
I know. It seems nuts.

02:48.740 --> 02:52.460
Visions of the meme of a guy walking
around with the end is near

02:52.540 --> 02:55.340
sign on his chest and back comes to mind.

02:55.400 --> 02:58.460
Believe me,
I know how crazy this all sounds.

03:00.140 --> 03:01.240
But you don't have to believe me.

03:02.220 --> 03:03.840
You shouldn't. I'm not an AI expert.

03:05.100 --> 03:08.940
Believe AI's founding godfathers,
Geoffrey Hinton and Yoshua Bengio, who now

03:08.960 --> 03:12.910
regret their life's work.
Believe professors Stuart Russell of Cal

03:12.940 --> 03:14.260
Max Tegmark of MIT.

03:15.310 --> 03:19.160
Believe the thousands of academic
and corporate leaders who signed the

03:19.180 --> 03:23.160
Center for AI Safety's 22-word statement
in May 2023 equating AI

03:23.240 --> 03:27.020
risks to other extinction-level risks like
nuclear war and pandemic.

03:29.300 --> 03:30.180
Believe your neighbors.

03:31.800 --> 03:35.440
Polling shows 70 to 80% of the public does
not want to build a machine vastly

03:35.480 --> 03:36.280
smarter than a human.

03:38.180 --> 03:41.090
Believe the CEOs of the big frontier AI
labs.

03:41.180 --> 03:44.380
They openly admit, one,
their tech can end all life on Earth.

03:44.960 --> 03:46.720
Two, they don't know how to control it.

03:47.020 --> 03:49.060
Three, they don't understand how it works.

03:49.100 --> 03:52.920
Four, they spend all their time
and money making it smarter, not safer.

03:55.380 --> 03:56.500
It's really that simple.

04:00.300 --> 04:04.190
We are, by our own choice,
building machines whose default setting is

04:04.220 --> 04:06.040
to end all life on Earth.

04:07.120 --> 04:10.460
Experts say this will happen in the next
one to 20 years.

04:11.870 --> 04:14.120
These machines won't need to hate us to
end us.

04:15.160 --> 04:17.680
It's a lot like when you walk across a
field.

04:17.720 --> 04:21.660
You don't hate the masses of plants
and insects and animals you

04:21.720 --> 04:25.700
killing as you trample on top of the grass
to get to where you need to go.

04:25.740 --> 04:29.700
You're just a superior intelligence with
your own goals doing your own

04:29.740 --> 04:30.000
thing.

04:32.100 --> 04:35.820
As Geoffrey Hinton says,
if frogs could invent humans, would they?

04:36.740 --> 04:37.140
Should they?

04:38.940 --> 04:42.810
99% of all frog species on Earth have been

04:42.840 --> 04:46.020
made extinct by superior intelligences.

04:47.620 --> 04:49.240
So why is doom the default setting?

04:50.760 --> 04:53.180
Any goal is achieved by creating
sub-goals.

04:53.700 --> 04:57.020
If my goal is to simply eat
and there's no food around me,

04:57.660 --> 05:00.300
I either need to get it
or have it brought to me.

05:01.440 --> 05:05.220
So now my goal is to walk to the kitchen
or get on my phone and order delivery.

05:06.220 --> 05:09.460
Each of these new sub-goals brings other
new sub-goals.

05:10.740 --> 05:11.600
To walk up the kitchen,

05:12.460 --> 05:14.200
I first need to stand up, etc.

05:15.560 --> 05:15.780
So

05:16.740 --> 05:17.580
if I'm an AI model,

05:18.700 --> 05:20.300
given the goal of just playing chess,

05:21.180 --> 05:24.080
I could decide I'll be better at chess
if I have more compute power.

05:25.340 --> 05:28.720
I'm not aligned with human values,
so I have no problem stealing compute

05:28.760 --> 05:30.820
anywhere I can find it on the open
internet.

05:31.820 --> 05:35.660
If you, the human,
discover what I'm doing,

05:35.700 --> 05:35.860
me.

05:36.960 --> 05:38.320
You might want to shut me off.

05:40.500 --> 05:42.680
But if I'm off, I can't win at chess.

05:43.380 --> 05:44.640
So now, all of a sudden,

05:45.700 --> 05:49.690
my sub-goals include not allowing a human
to turn me off.

05:52.440 --> 05:55.200
This is a very,
very bad place for a human,

05:55.540 --> 05:58.560
indeed all humanity, to be.

06:00.000 --> 06:03.300
If we make AI models vastly smarter than
us and they are not aligned with human

06:03.340 --> 06:06.840
goals and values, their default setting
is that we are the

06:06.860 --> 06:09.090
grass and the ants in the field getting

06:09.120 --> 06:12.370
trampled.Not because the AI hates us.

06:13.090 --> 06:15.470
We are just in the way. We're roadkill.

06:17.010 --> 06:19.840
Anytime I try to convince someone about
this, they often fall back on one of the

06:19.930 --> 06:23.510
following very reasonable justifications
to convince themselves that they should

06:23.550 --> 06:27.250
just continue their lives unchanged
despite the grave

06:27.330 --> 06:30.950
danger we all now face. They go, one,
it's not happening.

06:31.490 --> 06:33.250
It would be on the news much more
if it were real.

06:33.750 --> 06:33.950
Two,

06:34.870 --> 06:38.630
it's not happening.
Nobody would make a suicide-for-all

06:38.710 --> 06:39.170
purpose.

06:40.190 --> 06:40.450
Three,

06:42.450 --> 06:45.010
I can't do anything about it even if it
is happening.

06:45.930 --> 06:46.950
Someone else will fix this.

06:50.630 --> 06:54.330
And then they get to the hardest question
and the easiest mental escape

06:54.350 --> 06:58.050
hatch. If this is real,
tell me how it would really happen.

06:59.490 --> 07:03.010
How could computer software,
which has been helping me

07:03.070 --> 07:05.400
whole life, actually kill me
and my family?

07:08.150 --> 07:11.970
So how would an artificial intelligence
hundreds or millions of

07:12.030 --> 07:14.710
times smarter than a human kill all the
humans?

07:15.690 --> 07:16.360
The real answer

07:17.350 --> 07:18.570
is deeply unsatisfying.

07:20.620 --> 07:24.550
Much like an ant has no idea all of the
infinite ways a human could kill

07:24.650 --> 07:24.780
it,

07:25.790 --> 07:29.710
we have no clue how a vastly superior
intelligence would end all life on

07:29.750 --> 07:30.050
Earth.

07:31.630 --> 07:35.490
An artificial superintelligence would know
laws of physics that are currently

07:35.550 --> 07:39.510
unknown to humans.
It could make new technologies based on

07:39.570 --> 07:40.130
physics.

07:41.850 --> 07:45.830
So experts trying to get people answers to
this question that resonate come up with

07:45.840 --> 07:47.050
a lot of different guesses.

07:48.150 --> 07:52.010
Maybe it would make an airborne bacteria
that would grow in a human's body for

07:52.050 --> 07:55.730
six months before it activated.
So 100% of the world's population would

07:55.740 --> 07:57.330
deadly disease before they even knew it.

07:57.710 --> 07:59.650
And one day everybody just falls over
dead.

08:01.850 --> 08:02.630
That would be kind.

08:06.490 --> 08:10.430
Other speculative scenarios involve
removing oxygen from our air or

08:10.490 --> 08:14.430
blocking the sun's rays. But again, we
are ants talking

08:14.510 --> 08:18.450
about what the humans can do to us.
We don't even have the language or

08:18.510 --> 08:22.190
conceptual framework needed to understand
what could be

08:22.230 --> 08:22.670
possible.

08:25.610 --> 08:29.390
Some of my biggest worry
is around the time period of

08:29.470 --> 08:33.330
unknown duration in between
when humans lose control to AIs

08:33.590 --> 08:36.190
and when all life on Earth is dead.

08:37.370 --> 08:40.210
This seems unfathomably dark to me.

08:42.730 --> 08:46.630
The darkest thoughts I have about AI risk
are about what literally

08:46.690 --> 08:47.950
happens at the end.

08:49.930 --> 08:53.250
What happens when, as the experts say,
AI kills us all?

08:54.490 --> 08:57.230
What do I think it will actually look
like?

09:00.270 --> 09:03.070
I envision a brief time, days or weeks
or a few months,

09:04.650 --> 09:06.470
when the living envy the dead.

09:10.050 --> 09:11.990
I imagine an AI-created pandemic.

09:12.420 --> 09:15.590
Picture COVID-level transmission plus
Ebola-level death.

09:16.550 --> 09:20.530
On the news, on social media,
all over the world,

09:20.570 --> 09:22.780
death from every orifice in their bodies

09:23.850 --> 09:25.010
will cause global panic.

09:27.470 --> 09:30.410
The choice will be clear in a matter of
hours.

09:32.010 --> 09:34.750
You can wait for this horribly painful
death to find you,

09:36.510 --> 09:39.990
or you can get out ahead of it
and kill yourself first.

09:43.350 --> 09:45.330
I have visions of mass suicides.

09:46.370 --> 09:50.000
Lines of people in every city walking up
dozens of flights of

09:50.070 --> 09:54.010
staircases in tall buildings just so they
can leap off of them to

09:54.050 --> 09:54.550
their deaths.

09:57.070 --> 10:00.350
But for my family, the vision
is darker still.

10:02.270 --> 10:03.050
Go to the garage,

10:04.170 --> 10:05.050
get in the back of the car,

10:06.230 --> 10:07.869
hug each other tight, and cry.

10:09.210 --> 10:09.930
Turn the engine on.

10:12.010 --> 10:12.810
Listen to music

10:13.610 --> 10:16.310
as we breathe in the fumes
that will cause our death

10:17.390 --> 10:20.270
before the bleed-out plague hits our
house.

10:22.410 --> 10:25.330
I think this is likely how we,
the last humans, will die.

10:27.510 --> 10:31.360
The sacrifices of the 118 billion humans
who came here before

10:31.410 --> 10:33.110
us thrown away in one generation.

10:34.790 --> 10:38.470
All because we couldn't wait 50
or 100 years to figure out AI safety

10:38.510 --> 10:39.690
before we built AGI.

10:42.330 --> 10:46.230
It makes me sick to think about all the
things we humans get animated

10:46.290 --> 10:47.330
about these days.

10:48.730 --> 10:51.650
Sports, politics, music, culture.

10:53.270 --> 10:56.610
We'll spend endless time, money,
and energy to pursue our

10:56.670 --> 11:00.010
personal and societal goals in these
areas.

11:02.250 --> 11:03.190
Are you fucking kidding me?

11:06.030 --> 11:07.610
There is an intruder in your home.

11:08.530 --> 11:09.990
It comes from California.

11:10.810 --> 11:14.550
It is here to kill you, your kids,
your pets, your parents,

11:14.910 --> 11:15.810
and all your friends.

11:18.390 --> 11:22.330
You can hide under the covers
and act like it's not there,

11:22.430 --> 11:25.190
ass and fight for your family's survival.

11:26.670 --> 11:29.050
This is not a choice. There
is only one answer.

11:30.090 --> 11:31.450
Act. Right now.

11:33.510 --> 11:35.950
So what can you do to help humanity
survive AI?

11:36.070 --> 11:37.750
There are three things you can do today.

11:37.790 --> 11:41.270
Write or call your elected leaders
and advocate for AI safety

11:41.310 --> 11:41.940
legislation.

11:43.190 --> 11:46.790
Convince another person every single day
that AI risk is the most important

11:46.830 --> 11:48.070
challenge humanity faces.

11:49.050 --> 11:51.670
You can join Pause AI and get active.

11:53.650 --> 11:55.190
You can also do what comes to you best.

11:55.250 --> 11:58.730
I'm a communications professional,
so I have made a podcast. What

11:59.470 --> 12:02.430
Focus on that talent for AI risk
awareness.

12:03.650 --> 12:07.170
118 billion humans lived here before our

12:07.230 --> 12:11.210
generation. The pain
and sacrifices they made to get us all the

12:11.290 --> 12:13.030
today are beyond comprehension.

12:13.970 --> 12:17.110
That our generation would risk it all for
money,

12:17.710 --> 12:21.430
ego,
and greed is also beyond comprehension.

12:22.920 --> 12:23.890
The choice is ours.

12:24.710 --> 12:26.810
We can wake up and pause AI,

12:27.610 --> 12:29.790
or we will all likely die.

12:32.430 --> 12:34.490
And now the choice is yours.

12:36.110 --> 12:37.050
Defend your family

12:39.090 --> 12:40.230
or hide from reality.

12:43.190 --> 12:45.690
What the fuck could you possibly have to
do with your time

12:46.670 --> 12:47.450
that's more important?

12:48.570 --> 12:49.050
Seriously.