1 00:00:02,440 --> 00:00:06,420 I can't, I can't live with myself if I let this 2 00:00:06,480 --> 00:00:09,830 society sleepwalk into disaster, right? 3 00:00:12,180 --> 00:00:15,620 I can't live with myself if I don't do everything in my power 4 00:00:16,500 --> 00:00:19,930 to ensure there's a world for my children. The important thing... 5 00:00:19,940 --> 00:00:23,240 I mean, I go out on the street, I've talked to, I don't know, hundreds, 6 00:00:23,280 --> 00:00:26,560 of people. Many of them, many of them immediately say, "Oh, yes, 7 00:00:26,600 --> 00:00:29,260 Yeah, they're gonna, they're gonna end up killing us all." ... 8 00:00:29,300 --> 00:00:33,280 So for example, in the, in the, in the Birmingham campaign, 9 00:00:33,290 --> 00:00:36,629 that I mentioned earlier, 5,000 schoolchildren were arrested 10 00:00:37,600 --> 00:00:41,260 and spent some time in jail. But it 11 00:00:41,320 --> 00:00:41,900 changed 12 00:00:42,820 --> 00:00:45,080 the public perception of an entire country. 13 00:00:45,140 --> 00:00:48,640 It got attention at the level of the national government, 14 00:00:48,680 --> 00:00:50,040 the introduction of the Civil Rights Act. 15 00:00:50,080 --> 00:00:50,980 That the 16 00:00:51,900 --> 00:00:55,180 core founding principle of StopAI is that 17 00:00:55,240 --> 00:00:57,160 superintelligence is inherently 18 00:00:58,100 --> 00:01:01,540 too dangerous of a technology ever to be built. 19 00:01:01,550 --> 00:01:05,379 This is not, um... this is in contrast to the position of, for 20 00:01:05,420 --> 00:01:08,640 example, Pause AI, which says it should not be built until it 21 00:01:08,700 --> 00:01:12,500 safely. As, as to why Yudkowsky is not on hunger 22 00:01:12,560 --> 00:01:16,010 strike, that's a good question. People can find many different reasons not 23 00:01:16,220 --> 00:01:22,480 things. 24 00:01:24,000 --> 00:01:27,780 Uh, hello, everyone. Uh, my name is Samuel Shadrack. 25 00:01:27,920 --> 00:01:31,800 Uh, with me, I have Guido, uh, who 26 00:01:31,840 --> 00:01:32,440 has been 27 00:01:33,280 --> 00:01:35,620 protesting against creation of 28 00:01:35,660 --> 00:01:39,300 superintelligence. He is on hunger strike. 29 00:01:39,420 --> 00:01:43,340 Uh, I'm just going to ask him some questions, 30 00:01:43,380 --> 00:01:46,850 more broadly, you know, about the issue of, you know, 31 00:01:46,860 --> 00:01:48,960 Why is this a big deal? What can we do about it? 32 00:01:48,970 --> 00:01:52,560 And so on. Uh, 33 00:01:53,120 --> 00:01:56,640 could you maybe just please start by introducing yourself, you know, like 34 00:01:57,060 --> 00:02:01,020 what's your background? How did you, you know, come to this decision where you 35 00:02:01,060 --> 00:02:03,050 are right now of, you know, doing a hunger strike? 36 00:02:03,060 --> 00:02:06,720 Like, where did this begin, and how did that turn into where we 37 00:02:06,740 --> 00:02:07,900 now? 38 00:02:07,920 --> 00:02:11,020 Yeah. Thank you for having me on. 39 00:02:12,100 --> 00:02:15,020 Um, yeah, my name is Guido Reichstadter. 40 00:02:15,060 --> 00:02:18,790 I'm 45 years old. Uh, I'm a father of 41 00:02:19,000 --> 00:02:20,380 two children. 42 00:02:21,260 --> 00:02:25,020 And I first really became aware of the 43 00:02:25,040 --> 00:02:29,000 potential to build artificial general intelligence, 44 00:02:29,140 --> 00:02:32,840 uh, back in university when I was studying math and 45 00:02:32,880 --> 00:02:36,180 physics. I was probably around 20 years old. 46 00:02:36,260 --> 00:02:36,860 And, um, 47 00:02:38,120 --> 00:02:38,900 you know, immediately 48 00:02:40,320 --> 00:02:41,980 I felt that this would be 49 00:02:43,840 --> 00:02:47,240 probably the most dangerous technology that humanity had ever 50 00:02:47,300 --> 00:02:51,040 tried to build. And, uh, just at the 51 00:02:51,180 --> 00:02:55,080 time, the, the people 52 00:02:55,160 --> 00:02:59,120 around me, the information that I could get seemed to point 53 00:02:59,180 --> 00:03:03,010 towards this becoming possible in the second half of the 21st 54 00:03:03,040 --> 00:03:06,989 century. Um, and I kind of just put it out 55 00:03:07,020 --> 00:03:08,160 of my mind really, 56 00:03:09,140 --> 00:03:12,860 until, uh, ChatGPT was 57 00:03:12,880 --> 00:03:14,100 released. And 58 00:03:14,900 --> 00:03:18,340 that kind of served as a wake-up call, uh, that this 59 00:03:18,420 --> 00:03:22,230 process was going a lot faster than I expected it to go. 60 00:03:22,260 --> 00:03:26,170 Not that I thought ChatGPT itself was 61 00:03:26,260 --> 00:03:29,340 there, right? But just what it indicated about the 62 00:03:29,400 --> 00:03:32,020 underlying trend. 63 00:03:32,120 --> 00:03:33,920 And 64 00:03:34,720 --> 00:03:37,340 I remember, um, 65 00:03:41,080 --> 00:03:43,580 beginning to look for more information. 66 00:03:43,680 --> 00:03:46,720 I found this, uh... I found out that people had 67 00:03:46,760 --> 00:03:50,660 been... people had much shorter timelines, even back in 68 00:03:50,760 --> 00:03:53,650 2000. Specifically, uh, 69 00:03:53,760 --> 00:03:57,360 um, there was a paper that was written in the late '90s by Hans 70 00:03:57,380 --> 00:04:00,840 Moravec where he looked at the exponential, uh, 71 00:04:00,920 --> 00:04:03,269 trend in increasing computational power 72 00:04:04,200 --> 00:04:07,880 and used a model of some 73 00:04:08,840 --> 00:04:11,620 neurons to make an estimate of what 74 00:04:11,680 --> 00:04:14,670 the, like, in a... by analogy, what kind... 75 00:04:14,680 --> 00:04:17,600 what, h- how much computation the human brain does. 76 00:04:17,640 --> 00:04:21,500 And said this trend will intersect the computational power of the human brain 77 00:04:21,540 --> 00:04:25,219 sometime in the 2020s. And 78 00:04:26,980 --> 00:04:30,720 that should be a warning sign, right? 79 00:04:30,740 --> 00:04:34,160 That it's becoming, it's potentially becoming 80 00:04:34,240 --> 00:04:35,260 possible now 81 00:04:36,340 --> 00:04:40,220 to create hardware that can do effectively 82 00:04:40,230 --> 00:04:43,800 what a human brain does, even if it's not wired up correctly yet, 83 00:04:43,880 --> 00:04:45,180 right? 84 00:04:46,320 --> 00:04:50,180 Never before in, you know, the history of humanity has it been 85 00:04:50,240 --> 00:04:53,900 to build a machine that could do anything 86 00:04:53,960 --> 00:04:57,920 like the computational ability of the human 87 00:04:57,940 --> 00:05:00,860 brain, and that's becoming possible now. 88 00:05:01,940 --> 00:05:05,840 And so, yeah, um, it's... I go- it 89 00:05:05,880 --> 00:05:09,420 got me worried when I saw Chat GP- GPT come out, seeing the 90 00:05:09,460 --> 00:05:13,440 continuing trends, seeing the billions and billions of dollars in 91 00:05:13,500 --> 00:05:17,180 capital pouring into the field and greater and greater numbers of 92 00:05:17,220 --> 00:05:20,840 researchers being devoted to cracking that problem. 93 00:05:20,880 --> 00:05:23,100 Like, I'm 45. Never in my life 94 00:05:24,160 --> 00:05:26,400 has there been so much 95 00:05:27,040 --> 00:05:30,680 energy, talent, resources, 96 00:05:30,740 --> 00:05:34,630 capital, excitement behind building 97 00:05:34,660 --> 00:05:38,510 this technology. And I feel that 98 00:05:38,540 --> 00:05:42,220 we are in a condition of imminent danger right now because we don't 99 00:05:42,260 --> 00:05:46,220 know where the line is. We don't know how far we 100 00:05:46,280 --> 00:05:48,030 have to go to cross 101 00:05:49,940 --> 00:05:50,460 the barrier 102 00:05:51,380 --> 00:05:54,460 to something that's as generally capable as a human 103 00:05:54,500 --> 00:05:58,140 mind. And we also know that there's a lot of momentum, a lot of 104 00:05:58,280 --> 00:06:02,172 inertia in society.... and that if we want 105 00:06:02,212 --> 00:06:05,952 to have the option to stop this process, we have to 106 00:06:06,052 --> 00:06:08,192 do... we have to change the way the world works, 107 00:06:08,992 --> 00:06:12,052 and that takes time and effort and energy. 108 00:06:12,092 --> 00:06:15,932 So it's kind of like the situation where we're on the Titanic, 109 00:06:16,052 --> 00:06:19,332 huge ship with a lot of momentum and inertia, 110 00:06:20,152 --> 00:06:22,112 and we see an iceberg in front of us, 111 00:06:24,132 --> 00:06:27,532 and we don't know if we are able to maneuver the 112 00:06:27,572 --> 00:06:29,842 ship out of the path of that iceberg in time. 113 00:06:31,992 --> 00:06:35,352 And if so, uh, you know, if it's, if that's not the case, 114 00:06:35,432 --> 00:06:36,752 right now. It's like, 115 00:06:38,212 --> 00:06:38,952 if you're in a car 116 00:06:40,572 --> 00:06:41,412 driving downhill 117 00:06:42,832 --> 00:06:46,362 and you have no brakes, your brakes are gone, you're in an 118 00:06:46,412 --> 00:06:50,162 emergency well before you pass over the edge of that 119 00:06:50,192 --> 00:06:50,532 cliff. 120 00:06:51,692 --> 00:06:54,982 That's the situation which the world is in right now. 121 00:06:55,012 --> 00:06:58,112 There is an uncontrolled race to artificial 122 00:06:58,152 --> 00:07:01,952 superintelligence, which we have good reason to suspect would 123 00:07:01,992 --> 00:07:04,112 have catastrophic or 124 00:07:06,092 --> 00:07:08,492 extinction-level consequences for the human race- 125 00:07:08,572 --> 00:07:09,582 Okay. 126 00:07:09,632 --> 00:07:13,432 ... and we have no effective mechanism to stop 127 00:07:13,672 --> 00:07:16,702 currently. So, um, 128 00:07:17,852 --> 00:07:21,612 in 2024, I put my career on hold. Uh, I was a jeweler 129 00:07:21,672 --> 00:07:25,652 in Miami. I started a business with my ex-wife, uh, 130 00:07:25,672 --> 00:07:28,262 years working at that, and, 131 00:07:28,332 --> 00:07:32,232 um, moved to San Francisco 132 00:07:32,852 --> 00:07:34,032 to try and 133 00:07:35,412 --> 00:07:38,952 put some brakes on this (laughs) runaway car, 134 00:07:39,032 --> 00:07:40,262 right? Um, 135 00:07:41,072 --> 00:07:44,172 I landed October... I think it was 17th. 136 00:07:44,232 --> 00:07:46,912 On the 21st, I was sitting down in front of OpenAI 137 00:07:47,952 --> 00:07:50,292 having chained their, their door shut, 138 00:07:50,412 --> 00:07:54,332 right? I believe we all have a right to stop 139 00:07:54,342 --> 00:07:56,722 this race that threatens our lives. 140 00:07:56,772 --> 00:07:57,042 And 141 00:07:58,632 --> 00:08:01,322 in the absence of government action, which is... 142 00:08:01,372 --> 00:08:05,352 has not been forthcoming, we have a responsibility to stop it now 143 00:08:05,432 --> 00:08:09,272 precaution, out of the need to prevent 144 00:08:09,552 --> 00:08:12,932 the risk of devastating consequences to people we love. 145 00:08:13,012 --> 00:08:14,032 So, um, 146 00:08:16,512 --> 00:08:18,542 yeah, I, I don't think there's any... 147 00:08:18,592 --> 00:08:22,172 Any day is a bad day to take direct action to stop this 148 00:08:22,232 --> 00:08:26,212 race. And in fact, we need to see a lot, lot more of it, and that's, 149 00:08:26,252 --> 00:08:26,442 that's 150 00:08:27,372 --> 00:08:31,252 one of the prime motivations for me going on this hunger strike, 151 00:08:31,392 --> 00:08:31,832 is 152 00:08:33,052 --> 00:08:36,892 partly to make a public demand of these companies to stop 153 00:08:36,912 --> 00:08:40,222 putting my life and the life of my children and our 154 00:08:40,232 --> 00:08:41,562 community in danger, 155 00:08:44,872 --> 00:08:47,852 to do everything in their power, which is far greater than 156 00:08:47,892 --> 00:08:51,771 mine, to work towards an end to the global 157 00:08:51,872 --> 00:08:53,362 race which is necessary 158 00:08:54,832 --> 00:08:57,012 for our safety, right? 159 00:08:57,072 --> 00:08:57,642 Yeah. Um- 160 00:08:57,752 --> 00:09:01,602 We need a globally binding treaty which prohibits the development of 161 00:09:01,612 --> 00:09:03,612 this extremely dangerous technology, 162 00:09:05,332 --> 00:09:07,072 and to fulfill their duty to 163 00:09:07,992 --> 00:09:11,912 do everything they can to ensure that our society 164 00:09:11,952 --> 00:09:14,992 is made aware of the urgent danger it's being put 165 00:09:15,032 --> 00:09:18,092 in by this race and by the companies, like 166 00:09:18,152 --> 00:09:22,102 Anthropic, that are participating in it 167 00:09:22,132 --> 00:09:25,092 their responsibility while they're doing so. 168 00:09:25,192 --> 00:09:26,232 It's not enough to 169 00:09:27,771 --> 00:09:30,302 make nice, safety-sounding 170 00:09:30,712 --> 00:09:32,771 words, right? 171 00:09:32,832 --> 00:09:32,842 Mm-hmm. 172 00:09:32,842 --> 00:09:36,612 You've got... If, if this company is 173 00:09:38,692 --> 00:09:41,032 going to put the lives of everyone on Earth 174 00:09:42,312 --> 00:09:44,282 at risk... 175 00:09:46,372 --> 00:09:49,932 They have their reasons. I don't agree with them, 176 00:09:49,992 --> 00:09:51,332 but regardless, 177 00:09:52,972 --> 00:09:56,652 they have a responsibility to make sure that the people whose 178 00:09:56,692 --> 00:10:00,612 lives they're putting at risk understand that their lives are being put 179 00:10:00,632 --> 00:10:01,212 at risk, 180 00:10:02,472 --> 00:10:06,372 and this is not satisfied by giving a few speeches or a few 181 00:10:06,472 --> 00:10:09,562 podcasts here and there, you know? 182 00:10:09,592 --> 00:10:13,282 It's not satisfied until society is made aware of this 183 00:10:13,312 --> 00:10:16,932 urgent threat. So 184 00:10:16,972 --> 00:10:20,252 finally, if Dario Amodei, the CEO of Anthropic, 185 00:10:20,312 --> 00:10:24,292 that, I'm asking him to meet me face-to-face, 186 00:10:24,372 --> 00:10:28,332 being, and explain why, and 187 00:10:28,392 --> 00:10:31,552 I feel that that's not an unreasonable request. 188 00:10:31,732 --> 00:10:35,232 I feel that my life has value just as much as 189 00:10:35,312 --> 00:10:36,052 his. 190 00:10:37,852 --> 00:10:38,612 Got it. 191 00:10:38,672 --> 00:10:40,531 The lives of my children- 192 00:10:40,612 --> 00:10:41,182 Yeah, got it. Um- 193 00:10:41,182 --> 00:10:42,182 ... they have a right to life. 194 00:10:43,252 --> 00:10:46,012 Yeah, everything you said was super useful, but yeah, 195 00:10:46,052 --> 00:10:48,832 interrupt. I just, I just want to go to the next question. 196 00:10:48,912 --> 00:10:52,832 Uh, because you mentioned specifically 197 00:10:52,872 --> 00:10:56,252 know, just a disclaimer, I have never visited, but obviously, 198 00:10:56,312 --> 00:10:59,392 Twitter, you know, I see the stuff that gets posted online. 199 00:10:59,472 --> 00:11:03,392 Uh, in general, like, you know, if you ask me personally, 200 00:11:03,452 --> 00:11:07,372 like, for me, it makes sense, like, you know, political action makes 201 00:11:07,432 --> 00:11:11,271 sense. Like, I assume, okay, you know, look, religious leaders have, you know, 202 00:11:11,292 --> 00:11:14,852 influence over society. You know, politicians have, you know, influence over 203 00:11:14,912 --> 00:11:18,672 society. You know, politics actually matters, and, like, 204 00:11:18,712 --> 00:11:20,832 social movements actually matter. 205 00:11:20,872 --> 00:11:23,031 Like, it is not just money that matters. 206 00:11:23,052 --> 00:11:25,652 Like, it's not just, you know, billionaires that matter, you know, 207 00:11:25,692 --> 00:11:29,312 matters. But however, when I look at, you know, the 208 00:11:29,392 --> 00:11:33,112 types of, you know, ideas that are spread in San Francisco, the sense 209 00:11:33,271 --> 00:11:37,112 I get is that, you know, if you're building technology, 210 00:11:37,152 --> 00:11:40,572 like, a tech billionaire, like, that's the only high-status thing to do. 211 00:11:40,592 --> 00:11:42,852 Like, if you do that, you will get respect. 212 00:11:42,912 --> 00:11:46,472 If you, you know, become someone like us who's, you know, 213 00:11:46,512 --> 00:11:49,912 and, you know, start a social movement and so on, like, you know, people in San 214 00:11:49,992 --> 00:11:53,812 Francisco don't respect you. And, like, there's 215 00:11:53,872 --> 00:11:57,812 very clear, like, status dynamics going on here, 216 00:11:57,872 --> 00:12:01,752 even in the minds of people who are inside that bubble, which, 217 00:12:01,792 --> 00:12:04,812 that, "Oh, if I, you know, join these people who are protesting, 218 00:12:04,852 --> 00:12:07,972 are going to look at me as lower status, you know, people who won't respect me," so 219 00:12:08,031 --> 00:12:12,004 on. So-Yeah, I just wanted to ask you, like, what 220 00:12:12,024 --> 00:12:15,664 are your opinions on, you know, how, uh, like, status, social 221 00:12:15,724 --> 00:12:18,373 approval, respect, like, how do these things work in San Francisco? 222 00:12:18,373 --> 00:12:19,234 (laughs) 223 00:12:19,384 --> 00:12:23,114 Like, how do they work currently and be, you know, what can we actually done to, 224 00:12:23,164 --> 00:12:25,144 you know, change that? 225 00:12:27,544 --> 00:12:31,284 Yes. I'm primarily concerned with 226 00:12:31,324 --> 00:12:34,984 respecting myself. I don't really care 227 00:12:35,064 --> 00:12:38,864 about other people's respect. I have to live my life in a 228 00:12:38,884 --> 00:12:41,144 way which I can respect 229 00:12:43,244 --> 00:12:46,644 my own actions in honoring what I believe is 230 00:12:46,704 --> 00:12:48,564 true. And 231 00:12:50,144 --> 00:12:50,524 I think it's 232 00:12:51,884 --> 00:12:55,164 these- s- this thing about status is a pernicious 233 00:12:55,244 --> 00:12:57,524 trap, right? 234 00:12:57,644 --> 00:13:01,594 Um, you know, I- it's- it reminds me of 235 00:13:01,664 --> 00:13:04,674 a story, I don't know if it's true or not, 236 00:13:04,704 --> 00:13:07,664 but, you know, people- people will 237 00:13:07,704 --> 00:13:09,794 willingly, um, 238 00:13:12,184 --> 00:13:15,904 lose their lives rather than lose the respect of their- their peers, 239 00:13:15,984 --> 00:13:16,234 right? 240 00:13:16,624 --> 00:13:17,104 Mm-hmm. 241 00:13:17,144 --> 00:13:20,314 And in this case, we might lose the entire world out of some 242 00:13:20,364 --> 00:13:21,424 foolish... 243 00:13:22,704 --> 00:13:26,604 It's like, I- I- I wonder why people are so concerned about the respect 244 00:13:26,644 --> 00:13:30,514 of others. It makes me wonder. It seems that 245 00:13:30,524 --> 00:13:34,464 it- it's a reflection of a certain self-doubt. 246 00:13:35,764 --> 00:13:39,614 At any rate, it- it's- it doesn't stop me from doing what 247 00:13:39,684 --> 00:13:41,524 I'm doing. Um, 248 00:13:43,084 --> 00:13:46,124 and this is, uh, uh, there's an inter- an interesting point that you brought up. 249 00:13:46,184 --> 00:13:47,544 So people in San Francisco, 250 00:13:48,404 --> 00:13:51,944 it's a- it's a metropolitan area of almost eight 251 00:13:52,624 --> 00:13:53,534 Mm-hmm. 252 00:13:53,534 --> 00:13:55,284 Um, it's very hard to generalize, right? 253 00:13:55,844 --> 00:13:56,624 Yeah. 254 00:13:56,684 --> 00:13:59,944 So there are many people in San Francisco who 255 00:14:01,364 --> 00:14:02,684 recognize that 256 00:14:04,284 --> 00:14:08,124 this AI race is a race to annihilation, and they have no- they 257 00:14:08,144 --> 00:14:11,984 want nothing to do with it, right? The important thing, I mean, 258 00:14:12,024 --> 00:14:15,644 street, I- I've talked to, I don't know, hundreds, thousands of 259 00:14:15,684 --> 00:14:19,404 people. Many of them- many of them immediately say, "Oh yes, 260 00:14:19,424 --> 00:14:22,784 thing. Yeah, they're gonna- they're gonna end up killing us all." Many 261 00:14:22,824 --> 00:14:24,734 understand this. It's- it's not like, um, 262 00:14:25,704 --> 00:14:28,004 some, uh, esoteric 263 00:14:28,104 --> 00:14:31,864 idea. Um, the important thing 264 00:14:31,984 --> 00:14:33,444 is not to 265 00:14:34,524 --> 00:14:34,784 act 266 00:14:35,884 --> 00:14:39,684 based on the expectations of others but to do what is right, to do 267 00:14:39,724 --> 00:14:43,644 what's necessary, and to find other people and 268 00:14:43,664 --> 00:14:46,924 encourage them to do what's right and necessary. 269 00:14:47,004 --> 00:14:50,724 And that's what, uh, you know, this is- there's 270 00:14:50,764 --> 00:14:54,694 a- there's a saying that all- all truth goes through several 271 00:14:54,744 --> 00:14:58,654 stages. First it's ridiculed, then it's opposed, and finally 272 00:14:58,704 --> 00:15:01,474 it's- it's accepted as self-evident, right? 273 00:15:01,584 --> 00:15:02,084 Mm-hmm. Mm-hmm. 274 00:15:02,144 --> 00:15:03,684 So there's always going to be this kind of 275 00:15:04,504 --> 00:15:08,214 opposition, but this is not actually about 276 00:15:11,344 --> 00:15:14,984 necessarily even responding. 277 00:15:16,724 --> 00:15:19,124 The primary thing at least is not responding to the opposition. 278 00:15:19,144 --> 00:15:22,804 The primary thing is doing what is correct and- 279 00:15:23,044 --> 00:15:23,394 Mm-hmm. 280 00:15:23,394 --> 00:15:26,444 ... with the understanding that we have that this race puts 281 00:15:27,264 --> 00:15:29,384 our lives in danger. 282 00:15:30,584 --> 00:15:34,533 That's enough. E- even certainty is not 283 00:15:34,544 --> 00:15:38,504 necessary. You wouldn't- I wouldn't let anyone play Russian 284 00:15:38,544 --> 00:15:42,364 children, right? And this is just a risk of death. 285 00:15:42,384 --> 00:15:45,784 We're talking about risk of destroying the entire world, right? 286 00:15:45,944 --> 00:15:49,884 We have rights. We have a right to life, and we 287 00:15:49,904 --> 00:15:53,884 have a right to live unafraid that someone's gonna 288 00:15:53,924 --> 00:15:57,644 destroy the world. So we need to 289 00:15:57,804 --> 00:16:01,744 stand on those rights and defend them because we're actually 290 00:16:01,764 --> 00:16:03,364 up for each other, right? 291 00:16:04,704 --> 00:16:07,384 It's not just our rights we're standing up for. 292 00:16:07,424 --> 00:16:11,384 We can only have these- we only benefit individually from these 293 00:16:11,404 --> 00:16:14,844 rights if we're willing to defend them collectively, because 294 00:16:14,884 --> 00:16:18,124 individually we can't- we can't defend them. 295 00:16:18,144 --> 00:16:22,024 And so this is- this is the thing which, uh, trying to help, um, 296 00:16:23,324 --> 00:16:24,444 people to do. 297 00:16:24,584 --> 00:16:28,084 Got it. Got it. That makes sense, and I am completely supportive of your 298 00:16:28,104 --> 00:16:31,624 efforts. Uh, maybe then the next thing I wanted to 299 00:16:31,764 --> 00:16:35,044 ask is, you know, like, how do we actually, you know, 300 00:16:35,064 --> 00:16:37,223 How do we build this into a social movement? 301 00:16:37,324 --> 00:16:37,704 Uh, 302 00:16:38,984 --> 00:16:42,834 so one thing you actually previously talked to me about, you know, in private, 303 00:16:42,864 --> 00:16:46,604 but I also wondered if they, you know, be, uh, out there in public, which is you 304 00:16:46,644 --> 00:16:50,504 said, like, in order to get more people, you know, in the movement, you 305 00:16:50,544 --> 00:16:54,004 know, against AI, you know, to be actually protesting, 306 00:16:54,064 --> 00:16:57,524 something about it, uh, you said what is important is 307 00:16:57,684 --> 00:17:01,424 not, "Okay, here's a list of technical arguments for, 308 00:17:01,484 --> 00:17:03,844 risk," but what is actually needed is, like, courage. 309 00:17:03,884 --> 00:17:06,964 It's, like, seeing, you know, your friends and people around you, 310 00:17:07,003 --> 00:17:10,824 actually demonstrating that courage and actually, you know, 311 00:17:10,924 --> 00:17:14,863 Uh, like, that intuitively seemed to make a lot of 312 00:17:14,944 --> 00:17:18,904 sense to me. Uh, I just wanted to sort of get more of your 313 00:17:18,924 --> 00:17:21,604 Like, why do you believe this is true? 314 00:17:21,664 --> 00:17:25,404 Do you have some existing examples of this, you know, either in the AI or AI 315 00:17:25,424 --> 00:17:28,144 risk space or, you know, even outside of it? 316 00:17:28,184 --> 00:17:30,084 Yeah. 317 00:17:30,144 --> 00:17:33,524 No, I- I mean I think this is absolutely essential, 318 00:17:33,624 --> 00:17:36,264 right? I mean, it's not the wh- it's not the whole thing. 319 00:17:36,304 --> 00:17:36,774 There are many 320 00:17:37,664 --> 00:17:40,624 interlocking pieces, uh, eh, in- among which 321 00:17:42,024 --> 00:17:45,004 the technical arguments, the awareness 322 00:17:46,304 --> 00:17:48,084 also contribute, right? 323 00:17:48,764 --> 00:17:51,884 But, um, 324 00:17:54,244 --> 00:17:57,964 what in- in my estimation has heretofore been 325 00:17:58,004 --> 00:18:01,304 lacking has been a willingness to act 326 00:18:02,204 --> 00:18:04,904 consistently with the meaning of an 327 00:18:04,924 --> 00:18:08,384 emergency. Even a- a willingness to 328 00:18:08,424 --> 00:18:12,384 acknowledge that there is an emergency on the part of people who 329 00:18:12,424 --> 00:18:13,644 traditionally been concerned 330 00:18:14,584 --> 00:18:18,318 and aware of the threat that-... 331 00:18:18,328 --> 00:18:19,048 AI poses 332 00:18:20,288 --> 00:18:23,068 with regards to extinction. 333 00:18:24,267 --> 00:18:24,588 And 334 00:18:27,808 --> 00:18:29,628 there is a lot of, um, 335 00:18:31,227 --> 00:18:33,967 there's kind of a spirit of uncertainty, 336 00:18:34,048 --> 00:18:36,507 fear, doubt, 337 00:18:36,568 --> 00:18:39,628 self-doubt, and all of this is 338 00:18:41,408 --> 00:18:43,467 totally 339 00:18:44,368 --> 00:18:44,707 fatal 340 00:18:45,868 --> 00:18:49,207 to the creation of a vital en- engaged 341 00:18:49,267 --> 00:18:51,588 movement, right? 342 00:18:52,928 --> 00:18:56,747 We potentially have very little time in which to 343 00:18:56,788 --> 00:19:00,447 accomplish a very large goal, which is 344 00:19:00,548 --> 00:19:03,808 moving the entire world out of the path 345 00:19:03,947 --> 00:19:07,467 of this extremely dangerous technology it's on course 346 00:19:07,528 --> 00:19:10,578 to, to build. And 347 00:19:13,528 --> 00:19:14,888 I have seen the failure of 348 00:19:16,408 --> 00:19:17,967 reformist projects on 349 00:19:18,967 --> 00:19:22,088 social issues for 40 years, on climate change, for example, 350 00:19:23,328 --> 00:19:26,467 with the result that now very devastating, even catastrophic 351 00:19:26,848 --> 00:19:30,717 effects are essentially locked in. 352 00:19:30,768 --> 00:19:34,598 We don't have 40 years to fail on climate change, uh, 353 00:19:34,648 --> 00:19:38,027 on, on AI. We don't have 40 years to fail on 354 00:19:38,088 --> 00:19:40,098 AI. And 355 00:19:42,308 --> 00:19:44,447 we need to act 356 00:19:45,908 --> 00:19:49,828 decisively. If a m- if a, if a man 357 00:19:49,868 --> 00:19:53,648 is putting my life in danger and the lives of my children, I'm not 358 00:19:53,658 --> 00:19:57,007 satisfied merely to ask him to stop 359 00:19:57,928 --> 00:20:01,908 or to ask the government to stop and wait and wait 360 00:20:03,447 --> 00:20:05,868 and wait for a response that may not ever come. 361 00:20:05,888 --> 00:20:07,178 Got it. I- 362 00:20:07,178 --> 00:20:09,687 It's our responsibility to take leadership. 363 00:20:09,697 --> 00:20:12,997 ... thing doesn't look very good right at the 364 00:20:12,997 --> 00:20:16,888 To take direct action. We have the right to end this threat 365 00:20:16,987 --> 00:20:19,467 today, right now. 366 00:20:20,428 --> 00:20:21,308 This is our right. 367 00:20:22,707 --> 00:20:26,467 And the only way that it's going to possibly be done is by 368 00:20:26,507 --> 00:20:27,627 beginning to act 369 00:20:28,627 --> 00:20:30,137 with courage, with conviction 370 00:20:31,568 --> 00:20:35,487 on that, on that right, to take direct non-violent action 371 00:20:36,388 --> 00:20:36,987 to stop 372 00:20:38,727 --> 00:20:42,687 the development towards superintelligence, to ta- which could, is 373 00:20:42,727 --> 00:20:43,507 as simple as 374 00:20:45,507 --> 00:20:48,668 non-violently occupying the companies that are doing 375 00:20:48,707 --> 00:20:52,608 it, to lead government, because government is 376 00:20:52,687 --> 00:20:53,868 not leading on this. 377 00:20:56,447 --> 00:20:59,888 Go, the responsibility for our safety 378 00:20:59,928 --> 00:21:03,467 government, but they are abdicating that responsibility. 379 00:21:05,148 --> 00:21:09,027 Where does the responsibility go? It doesn't vanish. It's on us still. 380 00:21:09,048 --> 00:21:13,027 It always was on us. We delegated it to government, 381 00:21:13,068 --> 00:21:16,227 job. It's our responsibility to lead 382 00:21:16,368 --> 00:21:20,108 government, to occupy the offices of 383 00:21:20,168 --> 00:21:22,007 Congress if necessary- 384 00:21:23,308 --> 00:21:23,318 We do. 385 00:21:23,318 --> 00:21:24,668 ... to compel them to act. 386 00:21:25,648 --> 00:21:28,467 And we can do this. This is, this has been done by 387 00:21:29,687 --> 00:21:33,447 th- this has been done by social movements 388 00:21:33,487 --> 00:21:34,628 over again. 389 00:21:35,668 --> 00:21:35,987 And 390 00:21:38,268 --> 00:21:40,808 I'm, I'm inspired particularly by this. 391 00:21:42,348 --> 00:21:46,247 There is a, an episode in the civil rights movement in 392 00:21:46,288 --> 00:21:49,027 the Birmingham campaign, 393 00:21:50,027 --> 00:21:50,338 where 394 00:21:52,048 --> 00:21:53,368 5,000 school children 395 00:21:54,348 --> 00:21:54,987 went to jail 396 00:21:55,888 --> 00:21:58,638 for defying the segregation laws in Birmingham, 397 00:21:58,668 --> 00:22:00,568 Alabama. 398 00:22:03,987 --> 00:22:04,027 Yeah. 399 00:22:04,048 --> 00:22:06,888 They filled every jail in Birmingham, they filled the state 400 00:22:07,707 --> 00:22:10,987 fair, the animal pens where they, they held animals at the state 401 00:22:11,048 --> 00:22:12,768 fair, 402 00:22:13,808 --> 00:22:15,928 and this happened over several weeks. 403 00:22:15,947 --> 00:22:19,608 And by the end of it, the chief of police threw his hands in the 404 00:22:19,648 --> 00:22:23,467 said, "There's no way to keep a lid on this 405 00:22:23,527 --> 00:22:24,207 gone." 406 00:22:25,027 --> 00:22:28,928 This is when radical political change happens, when people 407 00:22:28,937 --> 00:22:29,768 lose their fear 408 00:22:31,288 --> 00:22:35,128 and they're willing to act. For me, I'm acting out of love 409 00:22:35,308 --> 00:22:39,148 for my children. If I f- if I have to 410 00:22:39,187 --> 00:22:42,408 face the choice between going on hunger strike 411 00:22:45,248 --> 00:22:48,348 and raising awareness on this issue and mobilizing people to 412 00:22:48,358 --> 00:22:52,187 action or sitting back and, and living a comfortable 413 00:22:52,248 --> 00:22:56,027 life and letting my children's lives be threatened, I'll take the first 414 00:22:56,088 --> 00:22:59,878 one any day of the week. I've been arrested a couple of 415 00:22:59,928 --> 00:23:03,128 times. I'm willing to go to jail. 416 00:23:04,207 --> 00:23:08,047 I'm willing, you know, to do whatever it non-violently takes to 417 00:23:08,088 --> 00:23:11,388 my children are safe, because in my mind there's nothing more 418 00:23:11,428 --> 00:23:13,328 important. 419 00:23:13,727 --> 00:23:14,068 Yeah. 420 00:23:14,078 --> 00:23:17,808 And when people, when people tap into that love, the 421 00:23:17,888 --> 00:23:19,368 fear of, of sanction 422 00:23:21,527 --> 00:23:24,568 becomes much less effective in constraining our 423 00:23:24,608 --> 00:23:26,257 behavior. 424 00:23:26,257 --> 00:23:26,257 Yeah. 425 00:23:26,328 --> 00:23:30,138 I'm sure that many of the people that are concerned about this, if 426 00:23:30,168 --> 00:23:33,428 the, if they, if they knew that simply by 427 00:23:35,348 --> 00:23:38,947 sitting down in front of the AI companies they would stop this race, many would 428 00:23:38,987 --> 00:23:41,868 take that. It's the right thing to do. 429 00:23:41,908 --> 00:23:44,568 The only reason they don't is because they don't think it's going to 430 00:23:44,628 --> 00:23:46,418 work. 431 00:23:48,388 --> 00:23:52,277 And what we need for people to do is to center less on what they think 432 00:23:52,308 --> 00:23:55,848 to work and more on what is right for them to 433 00:23:55,908 --> 00:23:56,957 do, because 434 00:23:58,687 --> 00:24:01,868 paradoxically that's why it hasn't 435 00:24:01,928 --> 00:24:05,788 happened. That's why we haven't stopped this race, 436 00:24:05,848 --> 00:24:09,768 have not en mass led society 437 00:24:09,808 --> 00:24:13,067 and said, "This thing is wrong and it needs to stop." 438 00:24:13,187 --> 00:24:13,447 Yeah. 439 00:24:14,908 --> 00:24:17,428 Got it, got it. No, that is a good answer. 440 00:24:17,547 --> 00:24:21,328 Uh, I wanted to ask you, uh, so there are multiple 441 00:24:21,547 --> 00:24:23,027 groups that are trying to do... 442 00:24:23,248 --> 00:24:26,947 ... you know, public advocacy around the idea of 443 00:24:27,008 --> 00:24:30,437 risk from superintelligence. Uh, this could include starting with, you 444 00:24:30,488 --> 00:24:34,268 know, uh, Mary's latest book, you know, Eliezer Yudkowsky and Nate Sawyer 445 00:24:34,368 --> 00:24:38,008 is, they published this book, you know, If Anyone Mutes It, Everyone Dies. 446 00:24:38,088 --> 00:24:41,268 Uh, there is pause AI which was started by, 447 00:24:41,288 --> 00:24:44,428 uh, Meandetsma and Holly 448 00:24:44,628 --> 00:24:48,508 Elmore. Uh, there's, you know, the AI Risk Network by John Sherman. 449 00:24:48,568 --> 00:24:52,488 There's, you know, Liron Shapira who has his own channel 450 00:24:52,548 --> 00:24:55,888 debates. Like, there are lots of people who 451 00:24:55,928 --> 00:24:59,888 need to go to the public. Uh, but I get the sense that they're all 452 00:24:59,947 --> 00:25:03,528 working independently. Uh, so maybe I just wanted to ask you, like, 453 00:25:03,628 --> 00:25:07,466 A, what is something you think like, you know, all these public advocacy groups 454 00:25:07,528 --> 00:25:10,338 agree on, and B, what is, you know, something... 455 00:25:10,348 --> 00:25:14,288 or what other things they disagree on? 456 00:25:16,588 --> 00:25:20,508 Yeah. That's a good question. I'm not terribly familiar with what 457 00:25:20,528 --> 00:25:23,707 e- ev- every other group is doing. I can speak 458 00:25:23,748 --> 00:25:27,138 primarily, uh, for myself, what I'm doing, and 459 00:25:27,168 --> 00:25:30,808 the, the, the work that I've done in support of Stop 460 00:25:30,868 --> 00:25:32,488 AI. Um, 461 00:25:33,748 --> 00:25:37,628 in terms of goals, there is, um... I think there is a 462 00:25:37,708 --> 00:25:40,028 common- a commonality in that 463 00:25:40,947 --> 00:25:42,488 I believe all of these groups 464 00:25:43,828 --> 00:25:45,528 would agree that 465 00:25:47,228 --> 00:25:50,068 the race which is going on now to build superintelligence 466 00:25:50,348 --> 00:25:54,187 is... poses an extreme threat to all of our 467 00:25:55,368 --> 00:25:59,268 safety and wellbeing and our lives, and should 468 00:26:00,467 --> 00:26:02,728 stop now. 469 00:26:04,888 --> 00:26:07,947 I don't think that's... I don't think that would be controversial 470 00:26:08,748 --> 00:26:11,048 (laughs) among any of those groups. 471 00:26:11,148 --> 00:26:15,088 Um, the primary difference 472 00:26:15,098 --> 00:26:17,628 that I see, there are 473 00:26:17,748 --> 00:26:21,628 two. One is kind of in 474 00:26:21,687 --> 00:26:22,368 the goals, 475 00:26:23,668 --> 00:26:26,918 the ultimate goals of those groups, and 476 00:26:27,687 --> 00:26:30,788 one is in the methods which they see 477 00:26:30,928 --> 00:26:31,947 as, 478 00:26:32,748 --> 00:26:34,548 uh, necessary or 479 00:26:35,548 --> 00:26:39,528 effective. Um, in terms of goals, I'd say 480 00:26:39,588 --> 00:26:40,868 that there is a clear 481 00:26:41,708 --> 00:26:45,077 distinction between Stop AI and the other groups which is 482 00:26:45,108 --> 00:26:48,528 that the core 483 00:26:48,588 --> 00:26:51,788 founding principle of Stop AI is that 484 00:26:51,828 --> 00:26:55,467 superintelligence is inherently 485 00:26:55,568 --> 00:26:58,668 too dangerous of a technology ever to be 486 00:26:58,687 --> 00:27:02,608 built. This is not, um... this is in 487 00:27:02,687 --> 00:27:06,648 contrast to the position of, for example, pause AI which says 488 00:27:07,248 --> 00:27:10,428 it should not be built until it can be done safely. 489 00:27:11,808 --> 00:27:15,748 And from Stop AI's position, 490 00:27:15,828 --> 00:27:16,447 there is 491 00:27:18,208 --> 00:27:22,168 no- It's in- it's inherently a contradiction in terms. 492 00:27:22,228 --> 00:27:25,508 This technology cannot ever be done safely 493 00:27:26,428 --> 00:27:30,388 because unlike o- every other technology we've deployed, 494 00:27:30,428 --> 00:27:31,368 been able to test 495 00:27:31,828 --> 00:27:35,668 repeatedly... prototypes 496 00:27:37,048 --> 00:27:40,808 and refine and iterate through engineering. 497 00:27:40,848 --> 00:27:41,118 And 498 00:27:41,967 --> 00:27:45,947 with superintelligence, there is the very real risk that you don't get a 499 00:27:45,988 --> 00:27:49,467 second chance. If you build a system which is 500 00:27:49,608 --> 00:27:51,708 capable enough to take 501 00:27:52,548 --> 00:27:54,668 control of the world and humanity's 502 00:27:54,708 --> 00:27:58,357 fate, you may not get a second chance 503 00:27:58,648 --> 00:28:02,108 at... you may not get a chance to shut that off, 504 00:28:02,168 --> 00:28:04,408 You may have just locked yourself into 505 00:28:05,308 --> 00:28:07,788 inevitable loss of control. And 506 00:28:10,687 --> 00:28:13,388 I don't trust any theory which 507 00:28:14,608 --> 00:28:17,588 gives predictions about the behavior of a system like this. 508 00:28:17,608 --> 00:28:21,528 It's not enough to have theory. It must be tested through 509 00:28:21,568 --> 00:28:23,628 experiment. And this is just not 510 00:28:24,967 --> 00:28:28,808 a regime, the creation of superintelligence, 511 00:28:29,348 --> 00:28:32,108 experimentation. Um, 512 00:28:33,268 --> 00:28:37,187 you know, we... there are airplanes 513 00:28:37,228 --> 00:28:39,588 that leave the ground based on theory 514 00:28:40,467 --> 00:28:43,487 and there are people that get in those airplanes, 515 00:28:43,588 --> 00:28:47,538 pilots. I'm not willing for my 516 00:28:47,588 --> 00:28:50,148 family, against our 517 00:28:50,188 --> 00:28:53,428 will, to be the test subjects 518 00:28:54,208 --> 00:28:58,078 of the AI company's superintelligence experiments, and I 519 00:28:58,108 --> 00:29:02,098 don't think anyone should. So, that's primarily the difference in goals. 520 00:29:02,128 --> 00:29:05,578 There is a s- a- an ultimate goal of stopping 521 00:29:05,578 --> 00:29:09,418 superintelligence permanently, which means creating a world in 522 00:29:09,467 --> 00:29:13,328 which it is possible to do so indefinitely, that we have the 523 00:29:13,338 --> 00:29:15,228 social institutions, the 524 00:29:16,268 --> 00:29:20,197 relations among people which no longer 525 00:29:20,248 --> 00:29:23,728 produce this race and this unstoppable 526 00:29:23,788 --> 00:29:26,248 drive to build ever more powerful AI 527 00:29:26,288 --> 00:29:30,108 systems. And in terms of how, how we stop 528 00:29:30,148 --> 00:29:33,928 that race, like in the near term, there is also like a- a strategic difference 529 00:29:34,467 --> 00:29:34,828 in that 530 00:29:36,447 --> 00:29:40,338 I personally, and it's generally the, the 531 00:29:41,428 --> 00:29:43,318 approach of Stop AI, that 532 00:29:45,388 --> 00:29:48,008 the urgency of this situation demands direct 533 00:29:48,048 --> 00:29:49,467 action, 534 00:29:51,268 --> 00:29:52,308 that the 535 00:29:53,228 --> 00:29:56,508 institution, the... It's a- it's a- it's an estimation that the 536 00:29:56,568 --> 00:30:00,348 institutions of government and of economy, that the 537 00:30:00,388 --> 00:30:02,498 rules of the system as they are set up now 538 00:30:05,388 --> 00:30:09,148 will produce, with overwhelming likelihood, 539 00:30:09,248 --> 00:30:12,638 um, the bad outcome here. And that 540 00:30:14,388 --> 00:30:16,378 there is a phenomenon in social... 541 00:30:16,428 --> 00:30:20,208 in, in this- in the history of, of world society which is able to 542 00:30:20,228 --> 00:30:23,128 produce very rapid changes in government 543 00:30:23,168 --> 00:30:27,043 policy.And in fact, it does so about every two or three 544 00:30:27,063 --> 00:30:30,873 years somewhere in the world. And this is mass parta- participation 545 00:30:30,984 --> 00:30:34,464 civil disobedience. This is when people en 546 00:30:34,533 --> 00:30:38,244 masse, in large numbers, refuse to 547 00:30:38,323 --> 00:30:41,303 allow society to continue as 548 00:30:41,343 --> 00:30:45,283 normal until their demand is met. And 549 00:30:45,363 --> 00:30:49,184 sometimes this is a, a, a demand for a change in the law, 550 00:30:49,204 --> 00:30:51,573 demand for a change in the government. 551 00:30:51,603 --> 00:30:54,623 And like I said, a movement like this succeeds in achieving 552 00:30:54,644 --> 00:30:58,323 in the world about every two or three years. This is a f-... 553 00:30:58,333 --> 00:31:02,184 This is not a one-off thing. It is a social phenomenon, 554 00:31:02,224 --> 00:31:02,543 which is 555 00:31:03,684 --> 00:31:06,323 nearly... It was essentially universal. 556 00:31:06,404 --> 00:31:10,303 It's... It is a way for societies to rapidly 557 00:31:10,343 --> 00:31:14,264 change direction under extreme duress, which we 558 00:31:14,363 --> 00:31:15,934 are in now, and which 559 00:31:16,904 --> 00:31:19,823 we are increasingly approaching, 560 00:31:19,924 --> 00:31:23,603 right? So- so this is the approach 561 00:31:23,684 --> 00:31:27,004 which StopAI has chosen. It is to take direct 562 00:31:27,103 --> 00:31:30,704 action as the catalyst and as the motive 563 00:31:30,764 --> 00:31:34,033 force of building an assertive, vital 564 00:31:34,464 --> 00:31:38,363 social movement, which is not content to go 565 00:31:38,384 --> 00:31:40,884 the reformist route and 566 00:31:42,123 --> 00:31:42,643 parlay 567 00:31:45,664 --> 00:31:48,783 potentially decades of time that we don't have. 568 00:31:48,803 --> 00:31:49,464 Yeah. 569 00:31:49,504 --> 00:31:52,343 Right. So sometimes you, you have to go outside the box. 570 00:31:52,363 --> 00:31:55,343 You have to break the rules. That's okay, right? 571 00:31:55,464 --> 00:31:59,083 Uh, you know, I'm, I'm going to court this, uh, next month 572 00:31:59,643 --> 00:32:03,404 in a case where I obstructed the operation of 573 00:32:03,464 --> 00:32:07,224 OpenAI, which is one of the leading companies 574 00:32:08,004 --> 00:32:08,283 And 575 00:32:09,384 --> 00:32:13,283 we're asserting and defending this right of the public to take 576 00:32:13,323 --> 00:32:17,164 this kind of action, that it's... We are asserting that it 577 00:32:17,204 --> 00:32:20,893 that it is morally justified. And there is this 578 00:32:20,944 --> 00:32:24,474 concept in law called the defense of necessity. 579 00:32:24,524 --> 00:32:24,853 This is 580 00:32:26,303 --> 00:32:27,373 an idea that goes back 581 00:32:29,724 --> 00:32:33,333 probably thousands of years. I'm sure there's something like it in, in, 582 00:32:34,143 --> 00:32:37,504 legal systems around the world. It's the understanding that sometimes it's 583 00:32:37,514 --> 00:32:40,053 necessary to break some laws for... 584 00:32:40,083 --> 00:32:43,103 To prevent a greater harm. Um, just 585 00:32:43,184 --> 00:32:46,063 like you would break the window of a, of a car 586 00:32:47,004 --> 00:32:48,823 to rescue a child trapped inside, 587 00:32:49,724 --> 00:32:53,264 right? There's no other alternative to save 588 00:32:53,904 --> 00:32:54,333 Yeah. 589 00:32:54,504 --> 00:32:56,803 That's the case we're in right now. 590 00:32:58,143 --> 00:33:01,184 We don't know how close these companies 591 00:33:01,283 --> 00:33:04,793 are from the next breakthrough, right? 592 00:33:04,803 --> 00:33:08,704 Remember, it was one big breakthrough in 2017 593 00:33:08,724 --> 00:33:10,684 the field, the transformer. 594 00:33:10,694 --> 00:33:10,694 Mm-hmm. 595 00:33:10,744 --> 00:33:13,934 Right now, there are hundreds, you know, hundreds of billions of dollars and 596 00:33:14,004 --> 00:33:15,592 thousands of the world's best engineers 597 00:33:16,563 --> 00:33:18,434 trying to make that next breakthrough. 598 00:33:18,484 --> 00:33:21,704 We don't know how, how close it is. And this 599 00:33:21,764 --> 00:33:25,634 uncertainty should not be cause for inaction. 600 00:33:25,643 --> 00:33:29,524 It should be cause for preventative, precautionary action, 601 00:33:29,563 --> 00:33:32,924 because the consequences are so large, right? 602 00:33:32,984 --> 00:33:33,853 Yeah, yeah, go ahead. 603 00:33:33,863 --> 00:33:34,524 So, yeah. 604 00:33:34,623 --> 00:33:37,293 Uh, yeah, I'm really sorry. I, I just, I just don't have- 605 00:33:37,303 --> 00:33:38,984 No, no worries. Go ahead. Do your, do your thing. 606 00:33:38,994 --> 00:33:41,583 No, no, I get it. You're... Everything you're saying 607 00:33:41,684 --> 00:33:45,504 Uh, I wanted to dive more specifically into 608 00:33:45,603 --> 00:33:49,024 disagreement, uh, because, you know, I think it seems useful to actually 609 00:33:49,603 --> 00:33:51,524 get people on board, you know, doing things. 610 00:33:51,623 --> 00:33:55,603 Uh, so the specific argument you mentioned, 611 00:33:55,623 --> 00:33:59,004 you know, we don't get, you know, like 50 tries to build superintelligence. 612 00:33:59,043 --> 00:34:02,063 You know, the first time you do it, if you do it wrong, you know, 613 00:34:02,083 --> 00:34:06,073 Like, you can't really experiment with this and, 614 00:34:06,083 --> 00:34:10,004 over many years. Like, this is a pretty popular argument 615 00:34:10,083 --> 00:34:13,844 by Eliezer Yudkowsky. I think he's also completely on board with 616 00:34:13,884 --> 00:34:17,804 this specific argument, uh, that, you know, we can't experiment 617 00:34:17,844 --> 00:34:21,444 with it. You know, we are not going to solve, you know, safety 618 00:34:21,504 --> 00:34:25,344 alignment in, you know, five years, 10 years, probably in 20 years. 619 00:34:25,464 --> 00:34:29,043 Uh, maybe I would just ask, like, uh, how 620 00:34:29,163 --> 00:34:33,103 familiar are you with, you know, Eliezer Yudkowsky's views 621 00:34:33,123 --> 00:34:36,993 on this issue, uh, considering, of course, that he's one of the founders of, you 622 00:34:37,043 --> 00:34:40,933 know, AI risk as a field? And, yeah, I 623 00:34:41,004 --> 00:34:44,543 mean, like, where exact... Like, since, you know, he actually 624 00:34:44,603 --> 00:34:48,424 agrees with you on this core point, like, does that mean you both just, 625 00:34:48,444 --> 00:34:52,183 like, agree on everything? Like, where exactly is the disagreement coming? 626 00:34:52,223 --> 00:34:55,964 You know, why is, you know, Yudkowsky not sitting on a hunger strike 627 00:34:56,043 --> 00:34:58,464 like, you know, you are? 628 00:34:59,783 --> 00:35:03,054 Yeah, yeah, that's a good question. Um, I- I'm not terri... 629 00:35:03,054 --> 00:35:06,524 Uh, I'm honestly, I'm not terribly familiar with, uh, 630 00:35:06,644 --> 00:35:10,144 uh, Eliezer Yudkowsky's, um, uh, 631 00:35:10,264 --> 00:35:14,134 work. Um, I don't think I've... I may, I've read maybe a, a 632 00:35:14,183 --> 00:35:16,564 couple of papers or something. Um, 633 00:35:18,144 --> 00:35:21,084 so I'm not really sure what his position is. 634 00:35:21,144 --> 00:35:24,904 There are some things that he's said publicly that I do disagree 635 00:35:24,964 --> 00:35:27,524 with and, uh, 636 00:35:29,104 --> 00:35:32,964 I think, um, I think I disagree on others, but I'm 637 00:35:33,004 --> 00:35:35,493 not, I'm not sure. So, 638 00:35:38,084 --> 00:35:41,944 I'm not sure where he stands on this issue of alignment 639 00:35:42,004 --> 00:35:45,194 or safety. It seems, it seems to be... 640 00:35:45,243 --> 00:35:49,104 Like, what I'm... My position, you know, is that this 641 00:35:49,163 --> 00:35:53,104 should never be done. And it must, it must never 642 00:35:53,203 --> 00:35:56,604 be done, and we have an obligation to not do it, 643 00:35:56,703 --> 00:35:59,264 stop the people that are doing it. 644 00:35:59,364 --> 00:36:00,834 Mm-hmm. 645 00:36:00,834 --> 00:36:03,984 Um, which s- seems in contradiction to the idea 646 00:36:04,044 --> 00:36:07,384 that one can align a superintelligence 647 00:36:09,783 --> 00:36:13,723 a priori, right? By, by, by ex- by theory. 648 00:36:13,743 --> 00:36:14,604 Mm-hmm. 649 00:36:14,703 --> 00:36:16,624 Also, the, the alignment of, 650 00:36:17,484 --> 00:36:20,644 of such a system is deeply problematic in terms 651 00:36:20,723 --> 00:36:21,654 of, 652 00:36:21,764 --> 00:36:25,816 um...What 653 00:36:25,856 --> 00:36:29,516 goals is it being pointed to? Also, the 654 00:36:29,576 --> 00:36:33,386 idea that such a s- the idea that such a, a 655 00:36:33,516 --> 00:36:37,376 capable, self-reflecting, self-improving system, 656 00:36:37,476 --> 00:36:37,756 um, 657 00:36:39,776 --> 00:36:43,616 s- would, would be faithful to any goal which 658 00:36:43,676 --> 00:36:45,556 is externally, 659 00:36:46,996 --> 00:36:50,696 uh, imposed or structured into it seems 660 00:36:51,316 --> 00:36:52,556 also very suspect. 661 00:36:53,596 --> 00:36:57,236 So I- I- I believe that the- the 662 00:36:57,316 --> 00:37:01,176 concept of alignment in itself is very, 663 00:37:01,276 --> 00:37:02,766 um, misleading 664 00:37:03,796 --> 00:37:06,636 in the sense that it may serve the function of 665 00:37:07,936 --> 00:37:10,856 causing people to be less concerned 666 00:37:11,696 --> 00:37:15,076 about the development of super intelligence. 667 00:37:15,176 --> 00:37:19,136 That there's- there's a natural human tendency to- 668 00:37:19,316 --> 00:37:22,776 want to have to exert one's self 669 00:37:22,836 --> 00:37:26,696 beyond one's normal affairs. And if people 670 00:37:27,156 --> 00:37:30,896 can feel that this problem's being solved for them by 671 00:37:30,956 --> 00:37:31,536 someone else, 672 00:37:32,536 --> 00:37:35,276 it's a very reassuring kind of, um, 673 00:37:36,376 --> 00:37:38,976 story to tell oneself, which excuses 674 00:37:40,016 --> 00:37:40,756 taking action 675 00:37:42,356 --> 00:37:44,696 yourself to stop it. 676 00:37:44,796 --> 00:37:47,536 So, um, 677 00:37:49,236 --> 00:37:53,126 I think there's- there's also the- there's also, um, 678 00:37:54,276 --> 00:37:54,336 a, um 679 00:37:55,216 --> 00:37:59,156 ... something that's come to my attention 680 00:37:59,936 --> 00:38:01,376 by, uh, 681 00:38:02,436 --> 00:38:06,296 Yudkowsky about the value, the relative value of 682 00:38:06,336 --> 00:38:08,276 human life and 683 00:38:10,156 --> 00:38:13,556 putative, um, 684 00:38:15,716 --> 00:38:16,776 super intelligences, 685 00:38:18,576 --> 00:38:21,666 which is that the sacrifice of the existence of humanity 686 00:38:22,516 --> 00:38:22,996 would be 687 00:38:24,316 --> 00:38:24,916 acceptable 688 00:38:26,256 --> 00:38:28,396 in order to bring into existence 689 00:38:29,516 --> 00:38:32,496 a more morally valuable super 690 00:38:32,536 --> 00:38:34,776 intelligence, which 691 00:38:36,436 --> 00:38:39,056 I categorically, fundamentally disagree 692 00:38:39,096 --> 00:38:42,276 with. Um, 693 00:38:43,976 --> 00:38:44,296 and 694 00:38:46,516 --> 00:38:50,336 ... I don't think this ever can 695 00:38:50,476 --> 00:38:54,296 be or should be done safely. And this 696 00:38:54,356 --> 00:38:55,586 is- this is- this is, 697 00:38:56,876 --> 00:39:00,736 like, a f- this is a- a fundamental motivation for the work 698 00:39:00,796 --> 00:39:04,656 that I do. As- as to 699 00:39:04,696 --> 00:39:07,556 why, uh, Yudkowsky is not on hunger strike. 700 00:39:07,596 --> 00:39:11,456 That's a good question. People can find many different reasons not 701 00:39:11,525 --> 00:39:15,436 do things. Anyways, was that good or was there any part you, 702 00:39:15,676 --> 00:39:16,196 uh, wanna cover more? 703 00:39:16,206 --> 00:39:20,016 No, that is a really good answer. Uh, maybe I 704 00:39:20,056 --> 00:39:23,896 should put it this way. Uh, so ha- I'm assuming you've seen, 705 00:39:23,936 --> 00:39:27,025 like, the statement by, you know, Center for AI Safety, Dan Hendricks, 706 00:39:27,036 --> 00:39:30,996 like, uh, mitigating risk of extinction of AI 707 00:39:31,096 --> 00:39:33,936 and it was signed by, you know, a whole number of people. 708 00:39:33,946 --> 00:39:37,886 And I think that has actually, you know, moved- pushed the movement 709 00:39:37,996 --> 00:39:41,516 forward in, like, bringing this consensus that, okay, look, 710 00:39:41,576 --> 00:39:44,936 know, high credentialed, important, you know, knowledgeable people actually do 711 00:39:44,996 --> 00:39:48,986 agree with this. I wonder if there's value in doing a similar 712 00:39:49,056 --> 00:39:52,736 statement, you know, just even among the, let's say, groups who are protesting 713 00:39:52,776 --> 00:39:56,656 right now or involved in public advocacy that, you know, 714 00:39:56,696 --> 00:40:00,016 look, here is a core statement that, you know, we all do agree on. 715 00:40:00,036 --> 00:40:02,776 Like, you know, we might have, you know, certain various disagreements of our own, 716 00:40:02,796 --> 00:40:06,716 and you know, that's fine, but can we at least, you know, like, 717 00:40:06,736 --> 00:40:09,676 some consensus-building. Like, you know, here is a statement that we all agree on. 718 00:40:09,696 --> 00:40:13,655 Like, do you think s- like, such a statement, you know, 719 00:40:13,706 --> 00:40:15,836 Do you have opinions on, you know, what the exact statement should 720 00:40:15,896 --> 00:40:18,356 be? 721 00:40:19,676 --> 00:40:20,226 Um ... 722 00:40:23,876 --> 00:40:24,296 hm. 723 00:40:25,916 --> 00:40:29,906 Maybe could be valuable. Um, I think the- the first 724 00:40:29,936 --> 00:40:33,796 statement was very valuable in the sense that it provided a reference 725 00:40:33,836 --> 00:40:37,656 point for the public. It, um, and provides, 726 00:40:37,716 --> 00:40:40,556 right? It's- it's very- 727 00:40:41,836 --> 00:40:45,716 it's very useful if you're- when you're mobilizing people to take 728 00:40:45,756 --> 00:40:49,716 to something like that and show that there are, 729 00:40:49,756 --> 00:40:51,616 you know, respected, credentialed 730 00:40:52,436 --> 00:40:56,316 authorities who accept this as 731 00:40:56,456 --> 00:41:00,436 a- as a real risk. And this is, for me at least, this is the 732 00:41:00,476 --> 00:41:03,426 primary value in a statement like that. 733 00:41:03,476 --> 00:41:03,996 It's that 734 00:41:04,936 --> 00:41:08,896 it enables and it facilitates public outreach. 735 00:41:09,016 --> 00:41:09,286 Um, 736 00:41:11,236 --> 00:41:13,896 uh, a similar statement, g- getting, 737 00:41:14,916 --> 00:41:18,876 uh, all these groups on board, uh, I- I don't 738 00:41:18,916 --> 00:41:22,476 think it could hurt per se. Um, except 739 00:41:22,656 --> 00:41:26,576 insofar that it- if it- if 740 00:41:26,636 --> 00:41:26,686 it 741 00:41:27,776 --> 00:41:30,636 obfuscates the- the larger issue of 742 00:41:31,856 --> 00:41:33,596 ... 743 00:41:36,356 --> 00:41:39,896 for example, Stop AI w- does not want to give the 744 00:41:39,916 --> 00:41:43,736 impression that it agrees that super intelligence can ever be 745 00:41:43,776 --> 00:41:47,536 built safely, which is the stated position of a lot of 746 00:41:47,576 --> 00:41:51,556 other- these other groups. I think we all can agree that the- 747 00:41:53,256 --> 00:41:56,896 should end now, and the difference is that I think Stop 748 00:41:56,956 --> 00:41:57,336 AI 749 00:41:58,536 --> 00:42:01,986 acts as though it must end now, and we're 750 00:42:02,016 --> 00:42:05,996 encouraging other people to- to take that action, because if 751 00:42:06,056 --> 00:42:09,356 it must end now and if the companies are 752 00:42:09,376 --> 00:42:12,996 not ending it and the government's not ending it, then we 753 00:42:13,116 --> 00:42:16,396 must act, and this is the- this is the core thing. 754 00:42:16,416 --> 00:42:19,916 It's simply not enough to write a letter to 755 00:42:19,956 --> 00:42:23,836 your representative or to call your representative 756 00:42:24,576 --> 00:42:28,506 and another day goes by and we get closer 757 00:42:28,516 --> 00:42:31,706 we get closer to this point of no return where we don't know where it 758 00:42:31,776 --> 00:42:35,638 is. We-People who understand this risk, and this 759 00:42:35,688 --> 00:42:37,677 is- goes for the authorities that signed that letter, 760 00:42:39,208 --> 00:42:43,188 what would be much more useful is for them to go 761 00:42:43,228 --> 00:42:45,068 farther in that sa- in that letter. 762 00:42:45,128 --> 00:42:48,807 Rather than saying it is merely a risk that should 763 00:42:50,868 --> 00:42:54,638 merit some precaution, to actually state, 764 00:42:54,638 --> 00:42:57,368 "We are now, given the circumstances, in an 765 00:42:57,388 --> 00:43:01,258 emergency." And to even go farther, that, 766 00:43:01,288 --> 00:43:05,108 to say that, "Given the ina- inaction by our 767 00:43:05,148 --> 00:43:09,038 government, non-violent direct action is necessary and 768 00:43:09,048 --> 00:43:12,718 justified," this would be incredibly useful in mobilizing 769 00:43:12,748 --> 00:43:16,688 people. And this is, from my point of view, the only real 770 00:43:16,748 --> 00:43:20,448 game in town. There's not going to be the political will 771 00:43:20,508 --> 00:43:24,158 mobilized, strident movement making 772 00:43:24,168 --> 00:43:26,488 demands for an immediate end to this 773 00:43:26,508 --> 00:43:30,268 race. And from my point of view, the most important thing 774 00:43:30,328 --> 00:43:33,268 is what actually assists 775 00:43:33,368 --> 00:43:35,288 in creating that 776 00:43:35,328 --> 00:43:37,968 movement. 777 00:43:38,148 --> 00:43:40,588 Got it. Got it, got it. That makes sense. 778 00:43:40,688 --> 00:43:44,108 Uh, yeah, I think I then wanted to move to a 779 00:43:44,148 --> 00:43:48,108 question, uh, which is, again, coming back to, you know, 780 00:43:48,148 --> 00:43:52,028 the movement? I wanted to ask about funding 781 00:43:52,048 --> 00:43:54,688 So, like, you know, this is something I can imagine, and, you know, 782 00:43:54,728 --> 00:43:55,608 or disagree, but, 783 00:43:56,868 --> 00:44:00,428 like, uh, I can imagine a person who has, you know, a tech 784 00:44:00,508 --> 00:44:03,568 job, they broadly agree with the arguments for 785 00:44:03,628 --> 00:44:07,608 risk, but they're like, "You look, you know, I have, you know, dependents. 786 00:44:07,668 --> 00:44:10,608 I need to make a certain amount of money to, you know, sustain myself, to sustain 787 00:44:10,668 --> 00:44:14,418 people around me. And, you know, if I just come and protest, you know, 788 00:44:14,468 --> 00:44:18,028 there's no money in that. You know, it's just financially, 789 00:44:18,048 --> 00:44:21,948 bad for me." So, I do think it makes sense for, 790 00:44:21,988 --> 00:44:25,248 like, social movements like this to figure out a 791 00:44:25,348 --> 00:44:29,268 Uh, do you have a sense of, like, how that funding can be 792 00:44:29,888 --> 00:44:33,228 raised? Like, would that be primarily, you know, through donations? 793 00:44:33,268 --> 00:44:36,568 Could that be, you know... And by the way, I don't have a strong opinion, you know, 794 00:44:36,588 --> 00:44:39,098 which of these would actually work, but, like, I'm just asking your opinion, you 795 00:44:39,148 --> 00:44:42,408 know? Would that be donations? Could that be, you know, let's say, you know, 796 00:44:42,468 --> 00:44:44,968 a YouTube channel and you run ads on it? 797 00:44:45,008 --> 00:44:48,148 Could that be, you know, you provide consultancy to other groups, 798 00:44:48,188 --> 00:44:50,488 on your actual knowledge expertise you have accumulated? 799 00:44:50,528 --> 00:44:54,448 Like, do you have a sense of, like, how you could actually get 800 00:44:54,488 --> 00:44:56,068 funding to, you know, scale up the 801 00:44:56,108 --> 00:44:58,688 movement? 802 00:44:59,868 --> 00:45:02,888 Yeah, I have, I have a general, a general idea. 803 00:45:02,988 --> 00:45:03,248 Um, 804 00:45:04,948 --> 00:45:05,288 like, 805 00:45:06,548 --> 00:45:10,128 the- this has been done in the past, right? 806 00:45:10,888 --> 00:45:12,248 Um, 807 00:45:13,088 --> 00:45:16,528 the- there have been many social movements in 808 00:45:16,568 --> 00:45:20,308 essentially member-supported. And I think the 809 00:45:20,328 --> 00:45:23,577 important thing is creating a movement which is worthy 810 00:45:24,208 --> 00:45:24,718 of support 811 00:45:25,768 --> 00:45:29,418 and which people ha- can have confidence and 812 00:45:29,468 --> 00:45:29,768 faith 813 00:45:31,728 --> 00:45:32,548 in its ability, 814 00:45:33,508 --> 00:45:36,688 its potential ability, to accomplish these goals. 815 00:45:36,708 --> 00:45:36,857 Okay. 816 00:45:37,368 --> 00:45:41,208 And so there's a, you know, there are, there are literally 817 00:45:41,328 --> 00:45:44,448 millions and tens of millions of people that are concerned about 818 00:45:45,528 --> 00:45:48,868 where this is all going, and some of them are very deeply 819 00:45:48,908 --> 00:45:51,108 concerned. And like you said, 820 00:45:52,108 --> 00:45:54,357 there's two, two sides to the example you brought up. 821 00:45:56,328 --> 00:45:59,748 One is, you can do a lot on a large scale 822 00:46:00,208 --> 00:46:01,688 if millions of people 823 00:46:02,988 --> 00:46:05,048 can be mobilized in support 824 00:46:06,108 --> 00:46:08,478 of a direct action 825 00:46:08,508 --> 00:46:11,638 movement. Millions of people times 826 00:46:13,208 --> 00:46:17,168 a dollar a month, 10 dollars a month, is millions of dollars, 827 00:46:17,208 --> 00:46:19,928 tens of millions of dollars, which can be deployed 828 00:46:20,808 --> 00:46:22,797 to mobilize people for direct action. 829 00:46:23,448 --> 00:46:25,768 So it's, in, in a very real sense, it's a problem of 830 00:46:25,808 --> 00:46:29,628 logistics. My experience with Stop AI is with a handful of 831 00:46:29,688 --> 00:46:33,188 organizers going out into the community through methods of direct 832 00:46:33,268 --> 00:46:35,088 face-to-face mobilization, 833 00:46:36,128 --> 00:46:36,468 and 834 00:46:38,468 --> 00:46:39,288 we have found 835 00:46:40,648 --> 00:46:43,778 several people, right? The, the largest action that 836 00:46:44,868 --> 00:46:48,728 has ever been done by Stop AI involved five people. 837 00:46:50,548 --> 00:46:54,228 It's not a, it's not a large number, but it's also the work of a very small 838 00:46:54,267 --> 00:46:58,068 of mobilizers. If this is systematized and 839 00:46:58,148 --> 00:46:58,738 organized 840 00:47:00,768 --> 00:47:03,878 with 100 mobilizers, I think it's very 841 00:47:04,468 --> 00:47:08,228 realistic to find that within society, you will find it, 842 00:47:08,268 --> 00:47:12,188 100 people to take direct action. Not everyone is gonna be 843 00:47:12,288 --> 00:47:16,228 like your software engineer who has, uh, you know, such a managerial 844 00:47:16,268 --> 00:47:19,348 position that it's very hard for them to 845 00:47:19,408 --> 00:47:23,328 make a very costly sacrifice. 846 00:47:23,348 --> 00:47:27,008 But there are many people who are able to do that, and it's a logistical 847 00:47:27,088 --> 00:47:31,028 question of finding them and mobilizing them and showing and 848 00:47:31,068 --> 00:47:34,868 encouraging them to stand up for their rights, which is what this is really, at a 849 00:47:34,908 --> 00:47:38,708 fundamental level, about. There's no guarantee 850 00:47:38,748 --> 00:47:42,108 rights is going to be free, right? It's often going to be very 851 00:47:42,148 --> 00:47:43,208 costly. 852 00:47:43,228 --> 00:47:43,508 Yeah. 853 00:47:43,578 --> 00:47:45,768 But we're also the inheritors 854 00:47:47,208 --> 00:47:51,148 of generations before us who made very costly sacrifices 855 00:47:51,228 --> 00:47:54,728 for our freedom and our rights. And so there's something that goes 856 00:47:54,768 --> 00:47:58,748 beyond the day-to-day material calculations, 857 00:47:58,828 --> 00:47:59,808 right? 858 00:47:59,848 --> 00:48:02,418 Yep. 859 00:48:02,668 --> 00:48:03,958 100 people, you know, 860 00:48:04,948 --> 00:48:08,668 in front of OpenAI, in front of the other AI companies pushing 861 00:48:08,948 --> 00:48:09,688 annihilation, 862 00:48:11,028 --> 00:48:14,968 would make much bigger headlines, would have a much bigger effect in, in 863 00:48:15,448 --> 00:48:18,258 galvanizing public opinion, and in... 864 00:48:19,508 --> 00:48:23,368 Excuse the, the noise. Okay, are you picking that 865 00:48:23,408 --> 00:48:24,188 up on the- 866 00:48:24,228 --> 00:48:24,528 Yeah, yeah- 867 00:48:24,608 --> 00:48:24,798 ... on your end? 868 00:48:24,798 --> 00:48:26,508 Your voice is clear, yes. 869 00:48:26,608 --> 00:48:30,328 Oh, okay. Yeah, there's some background noise. You know? 870 00:48:30,348 --> 00:48:34,008 But with millions of people in support, a movement could 871 00:48:34,068 --> 00:48:37,692 find...... thousands of people to take direct action. 872 00:48:37,732 --> 00:48:41,412 This is absolutely in the realm of th- the immediate present 873 00:48:41,432 --> 00:48:45,012 possibility. The conditions in society are such, it's 874 00:48:45,092 --> 00:48:46,852 simply a question of 875 00:48:49,252 --> 00:48:52,691 interfacing with society on a large enough surface 876 00:48:53,112 --> 00:48:57,092 to find those people and organize these actions. 877 00:48:57,102 --> 00:48:57,422 And 878 00:48:59,432 --> 00:49:02,732 this can have a transformative effect in unlocking the 879 00:49:02,752 --> 00:49:06,192 nonlinear potential of society to ch- change. 880 00:49:06,252 --> 00:49:10,212 So for, for example, in the, in the, in the Birmingham 881 00:49:10,252 --> 00:49:13,912 campaign, the Children's March that I mentioned earlier, 5,000 882 00:49:13,952 --> 00:49:15,752 schoolchildren were arrested 883 00:49:16,991 --> 00:49:17,652 and spent 884 00:49:18,712 --> 00:49:22,602 some time in jail. But it changed 885 00:49:23,772 --> 00:49:26,752 the public perception of an entire country. 886 00:49:26,762 --> 00:49:30,632 It got attention at the level of the national government, and it 887 00:49:30,672 --> 00:49:33,382 led directly to the introduction of the Civil Rights Act. 888 00:49:33,412 --> 00:49:36,462 It's not an understatement to say that the Civil Rights Act of 889 00:49:36,532 --> 00:49:40,002 1964 was a direct consequence of those 890 00:49:40,232 --> 00:49:44,122 5,000 schoolchildren standing up for their rights in 1963. 891 00:49:45,292 --> 00:49:48,442 And I believe that there is similarly a tipping point in this 892 00:49:48,452 --> 00:49:52,132 society, because you can look at 893 00:49:52,172 --> 00:49:55,962 society and see that not much is happening and then draw the conclusion 894 00:49:56,012 --> 00:49:59,582 that nobody is concerned. This is not the case. 895 00:49:59,582 --> 00:50:00,102 Mm-hmm. Yeah. 896 00:50:00,192 --> 00:50:03,312 Many, many, many people are deeply concerned. 897 00:50:03,322 --> 00:50:07,312 And I think you saw some evidence of this with the recent pushback on 898 00:50:07,372 --> 00:50:11,072 the attempt to impose a 10-year moratorium on AI regulation 899 00:50:11,712 --> 00:50:15,372 in the United States, which was killed largely as a result of public 900 00:50:15,412 --> 00:50:16,491 outcry. 901 00:50:16,502 --> 00:50:16,502 Yeah. 902 00:50:16,552 --> 00:50:20,442 Even people who aren't willing, because it's, uh, 903 00:50:20,512 --> 00:50:24,241 opinion, and people are hesitant to hold minority opinions 904 00:50:24,452 --> 00:50:27,052 often publicly. 905 00:50:28,532 --> 00:50:31,392 Even young people who are not willing to publicly acknowledge 906 00:50:32,372 --> 00:50:35,372 that they're worried about this extinction 907 00:50:35,412 --> 00:50:39,252 risk, I believe that in many, many people, 908 00:50:39,312 --> 00:50:43,112 maybe even most, there is an awareness that this technology does 909 00:50:43,152 --> 00:50:46,032 threat. And there is a, a kind of 910 00:50:46,092 --> 00:50:50,002 conflict, even in people who are ve- you 911 00:50:50,052 --> 00:50:53,992 know, might display the most outward bravado 912 00:50:54,032 --> 00:50:54,432 of it. 913 00:50:55,712 --> 00:50:59,491 And it's this, these kinds of very public actions, which 914 00:50:59,592 --> 00:51:03,312 in mass, in a representative, you know, that, that show a 915 00:51:03,332 --> 00:51:07,192 representative subsection of, of the population being 916 00:51:07,212 --> 00:51:09,392 willing to take this contrarian position, 917 00:51:10,872 --> 00:51:14,792 because of their conviction, which can strengthen 918 00:51:14,832 --> 00:51:18,761 in the heart of many, many people, to also 919 00:51:18,792 --> 00:51:22,112 gain the courage to say, "You know 920 00:51:22,172 --> 00:51:25,872 ra- you know what? I don't... This is, this is real. 921 00:51:25,971 --> 00:51:29,652 I don't need to stop denying it, and I've got a right to speak 922 00:51:29,752 --> 00:51:33,192 out, and I've got a right to demand that this situation 923 00:51:33,312 --> 00:51:36,602 end." And so this is part of the, the 924 00:51:36,792 --> 00:51:40,272 mechanism of change, which is not only theoretical. 925 00:51:40,312 --> 00:51:44,212 This is how social movements have changed and do change 926 00:51:44,272 --> 00:51:48,192 the direction a- of entire countries, you know, within a matter of 927 00:51:48,212 --> 00:51:49,252 days or weeks. 928 00:51:51,612 --> 00:51:53,361 So... 929 00:51:53,392 --> 00:51:55,932 Got it, got it, got it. That is a good answer. 930 00:51:56,052 --> 00:51:58,932 Uh, maybe then I'll just go to a really simple 931 00:51:58,952 --> 00:52:02,532 You know, uh, which day are you on 932 00:52:02,572 --> 00:52:06,072 currently? How long do you plan to, you know, continue this hunger strike? 933 00:52:06,112 --> 00:52:09,162 Like, just with the hunger strike specifically, 934 00:52:12,112 --> 00:52:15,032 Uh, yeah. Uh, today after I finish 935 00:52:15,852 --> 00:52:19,032 here, I'll be going out to Anthropic, and it will be the 16th 936 00:52:19,152 --> 00:52:23,112 day which I've been publicly in front of Anthropic. 937 00:52:23,172 --> 00:52:25,361 Mm-hmm. Yeah. 938 00:52:25,632 --> 00:52:29,511 Um, I s- I started, I- I started fasting the day before in 939 00:52:29,552 --> 00:52:33,432 be 17 days that I've gone without food, but 16 days as 940 00:52:33,492 --> 00:52:35,112 a public political act. 941 00:52:35,812 --> 00:52:35,822 Okay. 942 00:52:35,872 --> 00:52:36,212 And 943 00:52:37,172 --> 00:52:41,052 on the, on the first day that I went there, I delivered a 944 00:52:41,092 --> 00:52:43,752 letter to the security desk for Mr. 945 00:52:43,792 --> 00:52:45,412 Amodei, 946 00:52:46,952 --> 00:52:50,932 telling him... asking him 947 00:52:50,971 --> 00:52:54,711 to stop putting my lives, my life and the lives of my 948 00:52:54,752 --> 00:52:58,572 family in danger, which he's publicly acknowledged 949 00:52:58,612 --> 00:53:01,922 this race does. 950 00:53:02,552 --> 00:53:03,552 Asking him instead 951 00:53:04,432 --> 00:53:06,172 to use all his power 952 00:53:07,612 --> 00:53:09,792 to do everything in his ability 953 00:53:11,492 --> 00:53:15,372 to work for a stop to this race globally, which is 954 00:53:15,452 --> 00:53:19,172 the absolutely necessary condition for our survival, a 955 00:53:19,212 --> 00:53:23,192 global end, through government regulation, through international 956 00:53:23,212 --> 00:53:26,852 accord. And Anthropic and Mr. Amodei 957 00:53:26,892 --> 00:53:27,352 personally 958 00:53:29,212 --> 00:53:32,082 could have an immense effect in mobilizing 959 00:53:33,212 --> 00:53:37,092 the public will to demand such an accord. 960 00:53:38,572 --> 00:53:39,482 Thank you. 961 00:53:39,592 --> 00:53:43,402 Also, to fulfill the obligation to ensure that 962 00:53:43,452 --> 00:53:46,382 society is made aware of what's going on in these 963 00:53:46,412 --> 00:53:49,692 companies, to be made aware of the 964 00:53:49,732 --> 00:53:53,652 fact that Anthropic is 965 00:53:53,692 --> 00:53:56,612 contributing to this race, which poses 966 00:53:57,692 --> 00:54:01,132 potentially near-term risk of destruction of our entire 967 00:54:01,172 --> 00:54:02,812 society. 968 00:54:04,272 --> 00:54:08,092 And finally, if he's unwilling to do any of 969 00:54:08,132 --> 00:54:10,432 those things, to meet with me personally 970 00:54:11,452 --> 00:54:13,462 and explain why he feels the right 971 00:54:14,812 --> 00:54:18,632 to put my life, my children's lives, our entire society at 972 00:54:18,672 --> 00:54:22,652 risk. And I told him I'd be waiting outside 973 00:54:22,712 --> 00:54:23,192 his door 974 00:54:24,092 --> 00:54:26,832 for his answer, and that's my intention, to wait 975 00:54:27,632 --> 00:54:28,772 outside his doors 976 00:54:29,832 --> 00:54:32,752 until I'm answered. 977 00:54:35,212 --> 00:54:41,852 And-I'm 978 00:54:41,862 --> 00:54:42,742 also doing this 979 00:54:44,912 --> 00:54:48,872 to fulfill my responsibility to my children and the society 980 00:54:48,912 --> 00:54:52,872 that I'm a part of. I can't, I 981 00:54:52,932 --> 00:54:56,452 can't live with myself if I let this society 982 00:54:56,532 --> 00:55:00,022 sleepwalk into disaster, 983 00:55:00,072 --> 00:55:02,412 right? 984 00:55:03,352 --> 00:55:07,092 I can't live with myself if I don't do everything in 985 00:55:07,132 --> 00:55:07,702 my power 986 00:55:08,892 --> 00:55:09,492 to ensure 987 00:55:10,532 --> 00:55:13,791 there's a world for my children. 988 00:55:16,472 --> 00:55:19,932 And I think all of us that understand this risk have a similar 989 00:55:19,972 --> 00:55:23,672 responsibility, and it's up to each person to question 990 00:55:24,492 --> 00:55:24,792 about 991 00:55:26,612 --> 00:55:29,372 what they are able to do. Not what's 992 00:55:29,432 --> 00:55:33,392 convenient, not what would fit into their schedule, 993 00:55:33,432 --> 00:55:37,332 but what's actually necessary and what's actually right in 994 00:55:37,372 --> 00:55:40,212 this case. 995 00:55:42,092 --> 00:55:44,532 Yeah. Yeah. Got it. That is 996 00:55:45,812 --> 00:55:48,032 (laughs) yeah, something even I need to think more about. 997 00:55:48,272 --> 00:55:51,062 I really appreciate you giving this answer. 998 00:55:51,212 --> 00:55:51,932 Uh- 999 00:55:51,972 --> 00:55:55,352 And thank, thank you for joining in this conversation 1000 00:55:55,612 --> 00:55:59,332 putting, you know, a bit of your time and convenience 1001 00:56:02,792 --> 00:56:05,812 behind the greater, the greater good. 1002 00:56:06,432 --> 00:56:06,622 Oh, yeah. 1003 00:56:06,652 --> 00:56:06,892 Right? 1004 00:56:06,902 --> 00:56:07,872 Oh, yeah. 1005 00:56:08,232 --> 00:56:10,032 I didn't expect that, you know- 1006 00:56:10,072 --> 00:56:10,082 Yeah. 1007 00:56:10,092 --> 00:56:11,892 When I started this, I had no idea that 1008 00:56:12,732 --> 00:56:16,212 this kind of response would, would happen, and here we are having this conversation 1009 00:56:16,252 --> 00:56:16,512 now. 1010 00:56:18,312 --> 00:56:22,252 Oh, yes. Yes. I completely believe with you on like, 1011 00:56:22,292 --> 00:56:25,572 change is possible. You know, the first domino moves, 1012 00:56:25,582 --> 00:56:28,852 and you know, before you, you know it, you know, thousands of people are actually 1013 00:56:28,892 --> 00:56:32,882 doing something about it. So yes, I just want to be part of 1014 00:56:32,932 --> 00:56:36,772 the broader movement, the broader, you know, effort to actually do something 1015 00:56:36,832 --> 00:56:40,592 about this. And yeah, I definitely relate with you 1016 00:56:40,712 --> 00:56:44,472 on the aspect of like, look, if I was not 1017 00:56:44,592 --> 00:56:48,372 working on this, if I was just, you know, trying to lead a comfortable life, you 1018 00:56:48,432 --> 00:56:52,032 know, just, you know, get a tech job, you know, have vacations, you 1019 00:56:52,092 --> 00:56:55,592 know, have a partner, you know, do just fun things with my life, 1020 00:56:55,632 --> 00:56:59,572 like, like I would dislike myself as a person, 1021 00:56:59,612 --> 00:57:03,052 you know? I would be just unhappy doing that, 1022 00:57:03,112 --> 00:57:06,032 actually done more and I chose not to, and like that was on me. 1023 00:57:06,072 --> 00:57:09,812 Like, so like I actually completely resonate 1024 00:57:09,872 --> 00:57:12,452 this sentiment of yours. Um- 1025 00:57:12,472 --> 00:57:14,072 I mean, that was my lived experience. 1026 00:57:14,132 --> 00:57:16,832 I, I, there came a point where I could no longer 1027 00:57:17,012 --> 00:57:20,852 continue, you know, setting diamonds, which was what I was 1028 00:57:20,932 --> 00:57:23,162 doing before, um, 1029 00:57:24,052 --> 00:57:27,972 knowing that other people were making these decisions 1030 00:57:29,632 --> 00:57:32,362 that were going to have these consequences for the people I 1031 00:57:32,392 --> 00:57:35,892 love. That's what caused me 1032 00:57:35,952 --> 00:57:38,042 to put this on hold and start doing 1033 00:57:39,132 --> 00:57:41,622 what I think is necessary. 1034 00:57:42,792 --> 00:57:46,572 Yeah. Got it. Uh, I just wanted to then end with like a 1035 00:57:46,712 --> 00:57:50,332 final question, uh, just for viewers of this 1036 00:57:50,372 --> 00:57:54,352 podcast. Like, uh, what is the, like, very concrete, you know, 1037 00:57:54,392 --> 00:57:57,932 call to action? Like imagine, you know, this viewer o- of this podcast, you know, 1038 00:57:58,012 --> 00:58:01,052 after this like they will spend, you know, maybe 10 minutes, 15 minutes actually 1039 00:58:01,072 --> 00:58:04,892 doing the next step. Like, uh, what concretely would you 1040 00:58:04,912 --> 00:58:08,372 ask them to do? Would you ask them to, you know, go watch a lecture, you know, 1041 00:58:08,412 --> 00:58:10,632 a book? Would you ask them to, you know, email you? 1042 00:58:10,712 --> 00:58:13,772 Uh, like just what concrete action would you 1043 00:58:13,852 --> 00:58:16,452 Uh, I would maybe split it into two. 1044 00:58:16,492 --> 00:58:16,732 So 1045 00:58:17,712 --> 00:58:21,472 first I would recommend maybe for viewers who are just not really very familiar 1046 00:58:21,532 --> 00:58:24,552 with the AI risk, you know, maybe this is maybe one of the first videos actually 1047 00:58:24,612 --> 00:58:28,452 about it. And second with for people who 1048 00:58:28,512 --> 00:58:32,492 maybe even agree, but they're not actually n- 1049 00:58:32,552 --> 00:58:34,832 for both sets of people, like concretely what do you recommend? 1050 00:58:34,892 --> 00:58:37,262 Like, after watching this video, what do you recommend they do? 1051 00:58:40,712 --> 00:58:44,332 So I mean, I have, I have very clear ideas about what needs 1052 00:58:44,392 --> 00:58:48,152 done, and what needs to be done is the mobilization of mass 1053 00:58:48,232 --> 00:58:50,781 participation direct action. Um, 1054 00:58:52,172 --> 00:58:55,712 if you're not at, if- if you're just learning about this idea 1055 00:58:55,752 --> 00:58:59,732 about AI risk for the first time or you're 1056 00:58:59,812 --> 00:59:03,031 not quite, you don't quite understand or you're not quite convinced 1057 00:59:03,041 --> 00:59:06,322 that this actually is... 1058 00:59:07,472 --> 00:59:11,332 that this is a real risk, um, 1059 00:59:12,272 --> 00:59:12,762 probably 1060 00:59:13,752 --> 00:59:17,532 you, Samuel, can probably put a- a- a good, put together a good compilation 1061 00:59:17,572 --> 00:59:21,332 of resources for people to, you know, take that, that 1062 00:59:21,372 --> 00:59:23,912 next step. Off the top of my head, 1063 00:59:24,012 --> 00:59:26,912 um, I would suggest 1064 00:59:27,672 --> 00:59:31,132 definitely check out stopai.info. 1065 00:59:31,152 --> 00:59:35,112 There's a presentation there, um, which does, it's a, it's a recording 1066 00:59:35,172 --> 00:59:37,682 of one of the public presentations that, uh, 1067 00:59:38,572 --> 00:59:42,242 we've done to mobilize people in-person, and so it 1068 00:59:42,272 --> 00:59:46,172 does tr- uh, treat this question in the first part 1069 00:59:46,212 --> 00:59:50,192 of the presentation. There's, uh, there's some, part of 1070 00:59:50,212 --> 00:59:54,092 it's given by, uh, a Professor David Krueger, 1071 00:59:54,152 --> 00:59:57,962 who is a aI, who's, who's a professor in, 1072 00:59:58,312 --> 01:00:01,452 uh, at FAI. And, um, 1073 01:00:03,112 --> 01:00:06,832 you can also check out, um, you know, there's a great YouTube 1074 01:00:06,892 --> 01:00:10,522 channel For Humanity by John Sherman that's got a lot 1075 01:00:10,552 --> 01:00:13,892 of AI risk, uh, material on it. Uh, 1076 01:00:14,512 --> 01:00:17,652 Roman Yampolskiy is, uh, a guy who's spoken out 1077 01:00:19,552 --> 01:00:23,172 very broadly in public and has been on a number of, 1078 01:00:23,312 --> 01:00:25,282 um, high-reach podcasts, 1079 01:00:26,952 --> 01:00:30,392 and I think these are, these are compelling arguments. 1080 01:00:30,472 --> 01:00:30,532 So 1081 01:00:31,672 --> 01:00:35,632 for the people that, that do re- are past that point and recognize 1082 01:00:35,652 --> 01:00:39,402 that this is actually a real risk and it's urgent and it 1083 01:00:39,432 --> 01:00:43,028 threatens the lives of those we love-The 1084 01:00:43,188 --> 01:00:44,948 absolutely important thing is to join 1085 01:00:45,928 --> 01:00:49,058 these, the, the movement to stop it immediately. 1086 01:00:49,908 --> 01:00:53,408 And so there are a number of ways that you can do that, 1087 01:00:53,488 --> 01:00:57,408 Uh, almost everyone can give some kind of 1088 01:00:57,448 --> 01:01:00,968 of donation to, to keep the mobilization work of Stop AI 1089 01:01:01,508 --> 01:01:04,808 going, uh, and increase it. Um, 1090 01:01:05,708 --> 01:01:09,068 each person has, you know, unique talents and skills 1091 01:01:09,108 --> 01:01:11,948 So there's a join form, uh, stopai. 1092 01:01:12,128 --> 01:01:16,108 on, stopai.info. And if you sign up, then an organizer 1093 01:01:16,148 --> 01:01:19,768 will get in touch with you and see what the best way 1094 01:01:20,368 --> 01:01:23,088 to, to, to further that movement is. 1095 01:01:23,378 --> 01:01:26,578 And specifically, Stop AI's goals are to permanently 1096 01:01:26,688 --> 01:01:29,488 ban the development of super intelligence. 1097 01:01:29,528 --> 01:01:30,538 And to, to, and 1098 01:01:31,388 --> 01:01:34,748 pursuant to that, to mobilize a mass non-violent 1099 01:01:35,168 --> 01:01:39,097 direct action movement of people who are willing to stand on this 1100 01:01:39,128 --> 01:01:42,948 conviction and dem- and, and, and are not willing, not satisfied 1101 01:01:43,008 --> 01:01:46,978 to, to say, to sit back and watch the race go on for another 1102 01:01:47,068 --> 01:01:51,028 day, but are actually able and willing to take direct action 1103 01:01:51,068 --> 01:01:52,328 to stop it immediately. 1104 01:01:53,188 --> 01:01:56,968 And so, if you're not able to participate in the 1105 01:01:57,028 --> 01:02:00,808 direct action, there are many support roles that are 1106 01:02:00,908 --> 01:02:04,148 necessary to facilitate finding those people who 1107 01:02:04,158 --> 01:02:08,108 are. And so, um, 1108 01:02:08,148 --> 01:02:10,418 you know, financial support is an essential part of 1109 01:02:10,448 --> 01:02:14,088 that. Yeah, so check out stopai.info, 1110 01:02:14,108 --> 01:02:15,327 definitely. 1111 01:02:15,408 --> 01:02:15,608 Got it. 1112 01:02:15,628 --> 01:02:17,128 That's where I would send people. 1113 01:02:17,168 --> 01:02:20,448 Got it. Thank you for your answer. I will definitely put the link to your 1114 01:02:20,488 --> 01:02:23,608 you know, in the description. And- 1115 01:02:23,668 --> 01:02:27,438 Yeah. And if anyone wants to personally get in 1116 01:02:27,448 --> 01:02:30,268 also in- invite that and would, would be welcome 1117 01:02:30,308 --> 01:02:33,948 to, you know- 1118 01:02:34,028 --> 01:02:34,118 Got it. 1119 01:02:34,118 --> 01:02:37,208 ... help encourage and, and, and mobilize people. 1120 01:02:37,288 --> 01:02:40,568 Got it, got it. I think that was it for, like, questions from my side. 1121 01:02:40,608 --> 01:02:43,548 Is there anything else which you would, you know, either like to ask me or just 1122 01:02:43,588 --> 01:02:46,568 bring up on your own? Anything else you feel, you know, 1123 01:02:46,648 --> 01:02:48,188 important, this should be, you know, put in 1124 01:02:49,128 --> 01:02:53,068 this video. Like is there anything like that or, 1125 01:02:53,248 --> 01:02:53,608 Uh- 1126 01:02:53,618 --> 01:02:54,988 ... end this? 1127 01:02:55,008 --> 01:02:58,257 Yeah, I think, I think it's like, uh, I w- I was very... 1128 01:02:58,288 --> 01:03:02,078 It was very interesting to see this interview with, um, 1129 01:03:02,108 --> 01:03:05,948 Dennis, who's on hunger strike in, uh, London right now, 1130 01:03:06,008 --> 01:03:09,968 with Max Tegmark who is, uh, has some reputation in the field 1131 01:03:10,068 --> 01:03:11,328 of, you know, making 1132 01:03:12,408 --> 01:03:16,348 the public aware of AI risk and has also got credentials. 1133 01:03:16,428 --> 01:03:20,068 And what I would like to see, and I think is very 1134 01:03:20,108 --> 01:03:24,078 important, is for more people that have that kind of credential or 1135 01:03:24,168 --> 01:03:27,018 that kind of influence to speak out publicly 1136 01:03:27,888 --> 01:03:28,728 on the urgency 1137 01:03:29,568 --> 01:03:31,948 and acknowledge that this is an emergency. 1138 01:03:31,988 --> 01:03:35,488 That the fact that we don't know how close the edge 1139 01:03:35,568 --> 01:03:39,268 is, the fact that there is so much momentum, inertia in 1140 01:03:39,308 --> 01:03:39,948 society, 1141 01:03:41,308 --> 01:03:45,268 that maneuvering out of the range of this danger will take time and 1142 01:03:45,288 --> 01:03:48,928 is a massive task. And we don't know if we have that time, 1143 01:03:50,048 --> 01:03:53,268 and that the consequences of failing are so large 1144 01:03:54,348 --> 01:03:54,568 that 1145 01:03:55,408 --> 01:03:56,808 all this together is an emer- fo- 1146 01:03:57,788 --> 01:04:01,208 Let's just, let's just be honest and tell the truth to the public. 1147 01:04:01,288 --> 01:04:02,757 We're in an emergency right now 1148 01:04:03,668 --> 01:04:06,408 and we need to act like it's an emergency. 1149 01:04:07,888 --> 01:04:08,248 And 1150 01:04:09,648 --> 01:04:13,348 fundamentally, this is a question of each of our's right 1151 01:04:13,768 --> 01:04:17,728 to li- to life, to live. And we need to 1152 01:04:17,768 --> 01:04:18,968 take action today 1153 01:04:20,468 --> 01:04:23,008 to defend that right and to stop this race immediately. 1154 01:04:23,048 --> 01:04:26,808 It's our right, it's our power collectively 1155 01:04:28,008 --> 01:04:31,828 and it's our responsibility. And so let's, let's not le- you 1156 01:04:31,888 --> 01:04:33,477 know, the whole future's riding on this. 1157 01:04:33,508 --> 01:04:36,788 Let's not, let's not let 'em down. 1158 01:04:37,688 --> 01:04:41,388 Got it. Got it, got it. Uh, yeah, I just wanted to 1159 01:04:41,428 --> 01:04:43,668 again, like thank you for doing this podcast. 1160 01:04:43,708 --> 01:04:47,348 You know, in spite of, you know, you literally being on, you know, 1161 01:04:47,368 --> 01:04:50,548 hunger strike, you know, deteriorated, you know, physical strength, you know, 1162 01:04:50,588 --> 01:04:53,708 mental strength. In spite of that, you came here, you know, 1163 01:04:53,748 --> 01:04:57,708 gave such a coherent interview that I hope would be useful to 1164 01:04:57,728 --> 01:04:59,378 all the viewers. Uh- 1165 01:04:59,528 --> 01:04:59,928 Yeah. 1166 01:04:59,938 --> 01:05:03,868 ... of this. Like I just really wanted to appreciate, 1167 01:05:03,928 --> 01:05:05,648 actually did this and yeah- 1168 01:05:05,708 --> 01:05:09,458 Yeah, absolutely. Thank you so much for putting in the 1169 01:05:09,508 --> 01:05:13,498 there. That's something that I need to put more effort into myself 1170 01:05:13,508 --> 01:05:15,878 the word out there, so... 1171 01:05:16,008 --> 01:05:16,148 Yeah. 1172 01:05:16,168 --> 01:05:16,668 Yeah. 1173 01:05:16,688 --> 01:05:17,158 Got it. Got it. 1174 01:05:17,168 --> 01:05:18,828 It's absolutely necessary. 1175 01:05:18,868 --> 01:05:19,128 Yes. 1176 01:05:19,168 --> 01:05:21,808 Thank you, Samuel. It was good talking. 1177 01:05:21,848 --> 01:05:22,368 Cool. Yeah. 1178 01:05:22,378 --> 01:05:24,448 Let's talk again soon. 1179 01:05:24,528 --> 01:05:25,968 Yes, sure, sure, sure, sure.