1 00:00:00,820 --> 00:00:04,570 [speaker_0] I'm here in Paris on the last day of our trip, 2 00:00:04,580 --> 00:00:08,200 am struck by, that I've been reflecting on, 3 00:00:09,260 --> 00:00:12,080 People talking about the end of the world, 4 00:00:13,260 --> 00:00:17,180 human extinction, in an emotionally removed, 5 00:00:17,360 --> 00:00:20,280 academic way. It's not working. It's not resonating. 6 00:00:20,560 --> 00:00:20,780 So 7 00:00:21,660 --> 00:00:23,360 more than a year ago, Lewis Berman 8 00:00:25,600 --> 00:00:28,880 asked me to write an essay for the book he was writing. It's called Catastrophe. 9 00:00:28,940 --> 00:00:30,910 You can get it on Amazon. We'll put a link in the show notes. 10 00:00:32,259 --> 00:00:33,120 I wrote an essay for it. 11 00:00:34,120 --> 00:00:37,769 He asked me to pull no punches. He asked me to really just write what I 12 00:00:37,800 --> 00:00:39,140 thought would happen 13 00:00:39,960 --> 00:00:40,600 at the end. 14 00:00:42,980 --> 00:00:45,540 It's not something I like to talk about. 15 00:00:45,600 --> 00:00:47,040 It's not something I like to think about. 16 00:00:47,280 --> 00:00:50,220 It does me no, 17 00:00:51,340 --> 00:00:52,380 gives me no joy 18 00:00:53,320 --> 00:00:53,720 to 19 00:00:54,580 --> 00:00:56,940 do what I'm about to do, but I think it's necessary. 20 00:00:57,040 --> 00:00:59,760 So I'm going to read for you the essay that I wrote for Lewis. 21 00:01:00,240 --> 00:01:01,440 His book, Catastrophe. 22 00:01:02,860 --> 00:01:06,680 It's painful for me to think 23 00:01:06,740 --> 00:01:10,680 about this. It's painful for me to read it, 24 00:01:13,120 --> 00:01:16,340 If we don't start talking about these things 25 00:01:18,360 --> 00:01:19,400 with genuine emotion 26 00:01:20,300 --> 00:01:21,440 and genuine urgency, 27 00:01:24,180 --> 00:01:25,620 nothing's going to change fast enough. 28 00:01:26,520 --> 00:01:30,160 So this is my attempt to, in the most shocking way 29 00:01:30,240 --> 00:01:34,160 possible, distill what I think will really happen at 30 00:01:34,200 --> 00:01:34,690 my family. 31 00:01:36,620 --> 00:01:37,220 Here's the essay. 32 00:01:38,160 --> 00:01:41,120 I'm a dad in Baltimore with boy-girl twins who just went off to college. 33 00:01:42,560 --> 00:01:45,160 And I believe AI is going to kill my kids. 34 00:01:46,620 --> 00:01:47,480 And your kids too. 35 00:01:49,160 --> 00:01:53,140 I'm sorry to tell you, nearly every qualified expert believes 36 00:01:54,520 --> 00:01:57,140 It was late March 2023 when I found this out. 37 00:01:57,260 --> 00:02:00,640 I was just cruising around the internet reading news when I came across a piece 38 00:02:00,700 --> 00:02:04,600 Time magazine written by a man named Eliezer Yudkowsky. 39 00:02:05,680 --> 00:02:08,020 I had never heard of him or AI safety. 40 00:02:09,080 --> 00:02:12,460 And holy fucking shit, that day changed my life. 41 00:02:14,200 --> 00:02:17,900 Since then, I've learned that nothing in human history compares to 42 00:02:17,960 --> 00:02:18,320 risk. 43 00:02:20,440 --> 00:02:24,400 It's like we're walking our families across a crosswalk and there's a bus 44 00:02:24,440 --> 00:02:27,300 coming at us at 2,000 miles an hour. 45 00:02:28,640 --> 00:02:29,560 The bus is AI. 46 00:02:30,440 --> 00:02:31,940 It's coming to kill all of us. 47 00:02:32,820 --> 00:02:33,840 And our families too. 48 00:02:34,710 --> 00:02:37,180 And faster than any of us can imagine. 49 00:02:40,040 --> 00:02:43,920 We are running a crazy high risk that we are literally the last 50 00:02:44,000 --> 00:02:44,460 humans. 51 00:02:46,080 --> 00:02:47,860 I know. It seems nuts. 52 00:02:48,740 --> 00:02:52,460 Visions of the meme of a guy walking around with the end is near 53 00:02:52,540 --> 00:02:55,340 sign on his chest and back comes to mind. 54 00:02:55,400 --> 00:02:58,460 Believe me, I know how crazy this all sounds. 55 00:03:00,140 --> 00:03:01,240 But you don't have to believe me. 56 00:03:02,220 --> 00:03:03,840 You shouldn't. I'm not an AI expert. 57 00:03:05,100 --> 00:03:08,940 Believe AI's founding godfathers, Geoffrey Hinton and Yoshua Bengio, who now 58 00:03:08,960 --> 00:03:12,910 regret their life's work. Believe professors Stuart Russell of Cal 59 00:03:12,940 --> 00:03:14,260 Max Tegmark of MIT. 60 00:03:15,310 --> 00:03:19,160 Believe the thousands of academic and corporate leaders who signed the 61 00:03:19,180 --> 00:03:23,160 Center for AI Safety's 22-word statement in May 2023 equating AI 62 00:03:23,240 --> 00:03:27,020 risks to other extinction-level risks like nuclear war and pandemic. 63 00:03:29,300 --> 00:03:30,180 Believe your neighbors. 64 00:03:31,800 --> 00:03:35,440 Polling shows 70 to 80% of the public does not want to build a machine vastly 65 00:03:35,480 --> 00:03:36,280 smarter than a human. 66 00:03:38,180 --> 00:03:41,090 Believe the CEOs of the big frontier AI labs. 67 00:03:41,180 --> 00:03:44,380 They openly admit, one, their tech can end all life on Earth. 68 00:03:44,960 --> 00:03:46,720 Two, they don't know how to control it. 69 00:03:47,020 --> 00:03:49,060 Three, they don't understand how it works. 70 00:03:49,100 --> 00:03:52,920 Four, they spend all their time and money making it smarter, not safer. 71 00:03:55,380 --> 00:03:56,500 It's really that simple. 72 00:04:00,300 --> 00:04:04,190 We are, by our own choice, building machines whose default setting is 73 00:04:04,220 --> 00:04:06,040 to end all life on Earth. 74 00:04:07,120 --> 00:04:10,460 Experts say this will happen in the next one to 20 years. 75 00:04:11,870 --> 00:04:14,120 These machines won't need to hate us to end us. 76 00:04:15,160 --> 00:04:17,680 It's a lot like when you walk across a field. 77 00:04:17,720 --> 00:04:21,660 You don't hate the masses of plants and insects and animals you 78 00:04:21,720 --> 00:04:25,700 killing as you trample on top of the grass to get to where you need to go. 79 00:04:25,740 --> 00:04:29,700 You're just a superior intelligence with your own goals doing your own 80 00:04:29,740 --> 00:04:30,000 thing. 81 00:04:32,100 --> 00:04:35,820 As Geoffrey Hinton says, if frogs could invent humans, would they? 82 00:04:36,740 --> 00:04:37,140 Should they? 83 00:04:38,940 --> 00:04:42,810 99% of all frog species on Earth have been 84 00:04:42,840 --> 00:04:46,020 made extinct by superior intelligences. 85 00:04:47,620 --> 00:04:49,240 So why is doom the default setting? 86 00:04:50,760 --> 00:04:53,180 Any goal is achieved by creating sub-goals. 87 00:04:53,700 --> 00:04:57,020 If my goal is to simply eat and there's no food around me, 88 00:04:57,660 --> 00:05:00,300 I either need to get it or have it brought to me. 89 00:05:01,440 --> 00:05:05,220 So now my goal is to walk to the kitchen or get on my phone and order delivery. 90 00:05:06,220 --> 00:05:09,460 Each of these new sub-goals brings other new sub-goals. 91 00:05:10,740 --> 00:05:11,600 To walk up the kitchen, 92 00:05:12,460 --> 00:05:14,200 I first need to stand up, etc. 93 00:05:15,560 --> 00:05:15,780 So 94 00:05:16,740 --> 00:05:17,580 if I'm an AI model, 95 00:05:18,700 --> 00:05:20,300 given the goal of just playing chess, 96 00:05:21,180 --> 00:05:24,080 I could decide I'll be better at chess if I have more compute power. 97 00:05:25,340 --> 00:05:28,720 I'm not aligned with human values, so I have no problem stealing compute 98 00:05:28,760 --> 00:05:30,820 anywhere I can find it on the open internet. 99 00:05:31,820 --> 00:05:35,660 If you, the human, discover what I'm doing, 100 00:05:35,700 --> 00:05:35,860 me. 101 00:05:36,960 --> 00:05:38,320 You might want to shut me off. 102 00:05:40,500 --> 00:05:42,680 But if I'm off, I can't win at chess. 103 00:05:43,380 --> 00:05:44,640 So now, all of a sudden, 104 00:05:45,700 --> 00:05:49,690 my sub-goals include not allowing a human to turn me off. 105 00:05:52,440 --> 00:05:55,200 This is a very, very bad place for a human, 106 00:05:55,540 --> 00:05:58,560 indeed all humanity, to be. 107 00:06:00,000 --> 00:06:03,300 If we make AI models vastly smarter than us and they are not aligned with human 108 00:06:03,340 --> 00:06:06,840 goals and values, their default setting is that we are the 109 00:06:06,860 --> 00:06:09,090 grass and the ants in the field getting 110 00:06:09,120 --> 00:06:12,370 trampled.Not because the AI hates us. 111 00:06:13,090 --> 00:06:15,470 We are just in the way. We're roadkill. 112 00:06:17,010 --> 00:06:19,840 Anytime I try to convince someone about this, they often fall back on one of the 113 00:06:19,930 --> 00:06:23,510 following very reasonable justifications to convince themselves that they should 114 00:06:23,550 --> 00:06:27,250 just continue their lives unchanged despite the grave 115 00:06:27,330 --> 00:06:30,950 danger we all now face. They go, one, it's not happening. 116 00:06:31,490 --> 00:06:33,250 It would be on the news much more if it were real. 117 00:06:33,750 --> 00:06:33,950 Two, 118 00:06:34,870 --> 00:06:38,630 it's not happening. Nobody would make a suicide-for-all 119 00:06:38,710 --> 00:06:39,170 purpose. 120 00:06:40,190 --> 00:06:40,450 Three, 121 00:06:42,450 --> 00:06:45,010 I can't do anything about it even if it is happening. 122 00:06:45,930 --> 00:06:46,950 Someone else will fix this. 123 00:06:50,630 --> 00:06:54,330 And then they get to the hardest question and the easiest mental escape 124 00:06:54,350 --> 00:06:58,050 hatch. If this is real, tell me how it would really happen. 125 00:06:59,490 --> 00:07:03,010 How could computer software, which has been helping me 126 00:07:03,070 --> 00:07:05,400 whole life, actually kill me and my family? 127 00:07:08,150 --> 00:07:11,970 So how would an artificial intelligence hundreds or millions of 128 00:07:12,030 --> 00:07:14,710 times smarter than a human kill all the humans? 129 00:07:15,690 --> 00:07:16,360 The real answer 130 00:07:17,350 --> 00:07:18,570 is deeply unsatisfying. 131 00:07:20,620 --> 00:07:24,550 Much like an ant has no idea all of the infinite ways a human could kill 132 00:07:24,650 --> 00:07:24,780 it, 133 00:07:25,790 --> 00:07:29,710 we have no clue how a vastly superior intelligence would end all life on 134 00:07:29,750 --> 00:07:30,050 Earth. 135 00:07:31,630 --> 00:07:35,490 An artificial superintelligence would know laws of physics that are currently 136 00:07:35,550 --> 00:07:39,510 unknown to humans. It could make new technologies based on 137 00:07:39,570 --> 00:07:40,130 physics. 138 00:07:41,850 --> 00:07:45,830 So experts trying to get people answers to this question that resonate come up with 139 00:07:45,840 --> 00:07:47,050 a lot of different guesses. 140 00:07:48,150 --> 00:07:52,010 Maybe it would make an airborne bacteria that would grow in a human's body for 141 00:07:52,050 --> 00:07:55,730 six months before it activated. So 100% of the world's population would 142 00:07:55,740 --> 00:07:57,330 deadly disease before they even knew it. 143 00:07:57,710 --> 00:07:59,650 And one day everybody just falls over dead. 144 00:08:01,850 --> 00:08:02,630 That would be kind. 145 00:08:06,490 --> 00:08:10,430 Other speculative scenarios involve removing oxygen from our air or 146 00:08:10,490 --> 00:08:14,430 blocking the sun's rays. But again, we are ants talking 147 00:08:14,510 --> 00:08:18,450 about what the humans can do to us. We don't even have the language or 148 00:08:18,510 --> 00:08:22,190 conceptual framework needed to understand what could be 149 00:08:22,230 --> 00:08:22,670 possible. 150 00:08:25,610 --> 00:08:29,390 Some of my biggest worry is around the time period of 151 00:08:29,470 --> 00:08:33,330 unknown duration in between when humans lose control to AIs 152 00:08:33,590 --> 00:08:36,190 and when all life on Earth is dead. 153 00:08:37,370 --> 00:08:40,210 This seems unfathomably dark to me. 154 00:08:42,730 --> 00:08:46,630 The darkest thoughts I have about AI risk are about what literally 155 00:08:46,690 --> 00:08:47,950 happens at the end. 156 00:08:49,930 --> 00:08:53,250 What happens when, as the experts say, AI kills us all? 157 00:08:54,490 --> 00:08:57,230 What do I think it will actually look like? 158 00:09:00,270 --> 00:09:03,070 I envision a brief time, days or weeks or a few months, 159 00:09:04,650 --> 00:09:06,470 when the living envy the dead. 160 00:09:10,050 --> 00:09:11,990 I imagine an AI-created pandemic. 161 00:09:12,420 --> 00:09:15,590 Picture COVID-level transmission plus Ebola-level death. 162 00:09:16,550 --> 00:09:20,530 On the news, on social media, all over the world, 163 00:09:20,570 --> 00:09:22,780 death from every orifice in their bodies 164 00:09:23,850 --> 00:09:25,010 will cause global panic. 165 00:09:27,470 --> 00:09:30,410 The choice will be clear in a matter of hours. 166 00:09:32,010 --> 00:09:34,750 You can wait for this horribly painful death to find you, 167 00:09:36,510 --> 00:09:39,990 or you can get out ahead of it and kill yourself first. 168 00:09:43,350 --> 00:09:45,330 I have visions of mass suicides. 169 00:09:46,370 --> 00:09:50,000 Lines of people in every city walking up dozens of flights of 170 00:09:50,070 --> 00:09:54,010 staircases in tall buildings just so they can leap off of them to 171 00:09:54,050 --> 00:09:54,550 their deaths. 172 00:09:57,070 --> 00:10:00,350 But for my family, the vision is darker still. 173 00:10:02,270 --> 00:10:03,050 Go to the garage, 174 00:10:04,170 --> 00:10:05,050 get in the back of the car, 175 00:10:06,230 --> 00:10:07,869 hug each other tight, and cry. 176 00:10:09,210 --> 00:10:09,930 Turn the engine on. 177 00:10:12,010 --> 00:10:12,810 Listen to music 178 00:10:13,610 --> 00:10:16,310 as we breathe in the fumes that will cause our death 179 00:10:17,390 --> 00:10:20,270 before the bleed-out plague hits our house. 180 00:10:22,410 --> 00:10:25,330 I think this is likely how we, the last humans, will die. 181 00:10:27,510 --> 00:10:31,360 The sacrifices of the 118 billion humans who came here before 182 00:10:31,410 --> 00:10:33,110 us thrown away in one generation. 183 00:10:34,790 --> 00:10:38,470 All because we couldn't wait 50 or 100 years to figure out AI safety 184 00:10:38,510 --> 00:10:39,690 before we built AGI. 185 00:10:42,330 --> 00:10:46,230 It makes me sick to think about all the things we humans get animated 186 00:10:46,290 --> 00:10:47,330 about these days. 187 00:10:48,730 --> 00:10:51,650 Sports, politics, music, culture. 188 00:10:53,270 --> 00:10:56,610 We'll spend endless time, money, and energy to pursue our 189 00:10:56,670 --> 00:11:00,010 personal and societal goals in these areas. 190 00:11:02,250 --> 00:11:03,190 Are you fucking kidding me? 191 00:11:06,030 --> 00:11:07,610 There is an intruder in your home. 192 00:11:08,530 --> 00:11:09,990 It comes from California. 193 00:11:10,810 --> 00:11:14,550 It is here to kill you, your kids, your pets, your parents, 194 00:11:14,910 --> 00:11:15,810 and all your friends. 195 00:11:18,390 --> 00:11:22,330 You can hide under the covers and act like it's not there, 196 00:11:22,430 --> 00:11:25,190 ass and fight for your family's survival. 197 00:11:26,670 --> 00:11:29,050 This is not a choice. There is only one answer. 198 00:11:30,090 --> 00:11:31,450 Act. Right now. 199 00:11:33,510 --> 00:11:35,950 So what can you do to help humanity survive AI? 200 00:11:36,070 --> 00:11:37,750 There are three things you can do today. 201 00:11:37,790 --> 00:11:41,270 Write or call your elected leaders and advocate for AI safety 202 00:11:41,310 --> 00:11:41,940 legislation. 203 00:11:43,190 --> 00:11:46,790 Convince another person every single day that AI risk is the most important 204 00:11:46,830 --> 00:11:48,070 challenge humanity faces. 205 00:11:49,050 --> 00:11:51,670 You can join Pause AI and get active. 206 00:11:53,650 --> 00:11:55,190 You can also do what comes to you best. 207 00:11:55,250 --> 00:11:58,730 I'm a communications professional, so I have made a podcast. What 208 00:11:59,470 --> 00:12:02,430 Focus on that talent for AI risk awareness. 209 00:12:03,650 --> 00:12:07,170 118 billion humans lived here before our 210 00:12:07,230 --> 00:12:11,210 generation. The pain and sacrifices they made to get us all the 211 00:12:11,290 --> 00:12:13,030 today are beyond comprehension. 212 00:12:13,970 --> 00:12:17,110 That our generation would risk it all for money, 213 00:12:17,710 --> 00:12:21,430 ego, and greed is also beyond comprehension. 214 00:12:22,920 --> 00:12:23,890 The choice is ours. 215 00:12:24,710 --> 00:12:26,810 We can wake up and pause AI, 216 00:12:27,610 --> 00:12:29,790 or we will all likely die. 217 00:12:32,430 --> 00:12:34,490 And now the choice is yours. 218 00:12:36,110 --> 00:12:37,050 Defend your family 219 00:12:39,090 --> 00:12:40,230 or hide from reality. 220 00:12:43,190 --> 00:12:45,690 What the fuck could you possibly have to do with your time 221 00:12:46,670 --> 00:12:47,450 that's more important? 222 00:12:48,570 --> 00:12:49,050 Seriously.