1 00:00:01,980 --> 00:00:05,720 And I came back and I read LessWrong, and I looked, okay, I looked, two 2 00:00:05,800 --> 00:00:09,440 years have passed and absolutely nothing has changed. 3 00:00:09,480 --> 00:00:12,300 (laughs) Like, you know, the capabilities have increased a lot, 4 00:00:12,340 --> 00:00:15,900 from 2022, 2023, to, you know, 2025, like AI 5 00:00:15,980 --> 00:00:19,800 capabilities have gone further a lot, but, you know, 6 00:00:19,820 --> 00:00:23,700 all these people talking about, you know, safety, risk, governance, like 7 00:00:23,780 --> 00:00:27,600 it's the same people, the same boring speeches, the same, 8 00:00:27,680 --> 00:00:30,730 arguments that happen on LessWrong, like actually nothing has changed. 9 00:00:30,740 --> 00:00:34,710 There's the same funders, you know, uh, Muskowitz and 00:40 John Talion doing 10 00:00:34,740 --> 00:00:37,230 the same sort of work, which I don't really believe in, 11 00:00:37,340 --> 00:00:41,250 uh... Yeah. And I, I am now starting 12 00:00:41,300 --> 00:00:44,720 to realize, look, if I don't personally do something in this 13 00:00:45,600 --> 00:00:49,540 fate of the entire species might be at stake (laughs) , and that is kind 14 00:00:49,560 --> 00:00:52,880 of an almost overwhelming sort of responsibility to be dealing with. 15 00:00:52,960 --> 00:00:56,280 Uh, even by 2035, you're looking at, what, an at least 16 00:00:56,520 --> 00:00:58,720 40%, 50% chance that we will build 17 00:00:58,760 --> 00:01:02,600 superintelligence. So, at that point you realize, look, 18 00:01:02,660 --> 00:01:06,600 if you're making a life plan that does not take into account 19 00:01:06,640 --> 00:01:10,000 the fact that, you know, superintelligence might be created on this 20 00:01:10,060 --> 00:01:13,960 like what are you doing with your life? Like, that's just a bad move. 21 00:01:13,980 --> 00:01:16,450 01:23 Less, like the world is on... 22 00:01:16,480 --> 00:01:20,340 Like, there is a difference between, look, the world is on fire and I'm upset about 23 00:01:20,360 --> 00:01:24,320 this, but I won't, you know, allow this to overwhelm me, 24 00:01:24,380 --> 00:01:27,200 to the public. I will be the calm voice in the face of 25 00:01:27,210 --> 00:01:31,020 And there is another way of processing it, which is, 26 00:01:31,080 --> 00:01:34,360 look, the world is on fire, and look, I don't care about this at all 27 00:01:34,380 --> 00:01:36,760 going to do my thing, because I never cared about this in the 28 00:01:36,800 --> 00:01:40,320 place. 29 00:01:44,840 --> 00:01:48,510 Hi everyone. My name is Guido Reichstagter, and I have the 30 00:01:48,580 --> 00:01:51,980 pleasure of interviewing Samuel Shadrack today. 31 00:01:52,060 --> 00:01:55,160 It's Sunday, September 21st, and 32 00:01:56,300 --> 00:02:00,260 I first met Samuel online when I heard 33 00:02:00,300 --> 00:02:02,280 that he had started a hunger strike 34 00:02:03,740 --> 00:02:07,240 in India on the issue of AI 35 00:02:07,300 --> 00:02:10,919 risk, because there are companies which are right now 36 00:02:10,940 --> 00:02:14,340 racing to build superintelligent AI systems that they 37 00:02:14,460 --> 00:02:17,520 have no reasonable plan for 38 00:02:17,560 --> 00:02:20,360 controlling, and this project puts 39 00:02:21,480 --> 00:02:24,140 all of us at risk. So, 40 00:02:25,600 --> 00:02:28,780 Samuel, I'm glad to be talking with you today. 41 00:02:28,900 --> 00:02:32,280 Oh, yes, nice to be here. Glad to speak with 42 00:02:32,340 --> 00:02:33,300 you. 43 00:02:33,440 --> 00:02:37,350 Awesome. Could you, uh, start out by telling us a little bit about 44 00:02:37,360 --> 00:02:41,060 yourself, your personal history and how you came to be 45 00:02:41,120 --> 00:02:41,910 aware of, uh, 46 00:02:42,920 --> 00:02:45,420 uh, what- what's going on with AI? 47 00:02:46,920 --> 00:02:50,620 Uh, yeah, sure. Uh, so I mean, I'm, like, currently 48 00:02:50,920 --> 00:02:54,540 24 years old. Uh, I've 49 00:02:55,000 --> 00:02:58,960 grown up in India my whole life. Uh, I 50 00:02:59,240 --> 00:03:03,180 graduated from IIT Delhi in 2023, so that's 51 00:03:03,240 --> 00:03:05,450 two years back, with like a BTech... 52 00:03:05,460 --> 00:03:08,260 like a Bachelor's and a Master's in biochemical 53 00:03:08,320 --> 00:03:11,829 engineering. Uh, I've always been generally 54 00:03:11,860 --> 00:03:15,400 interested in tech though, generally interested in, like, 55 00:03:15,420 --> 00:03:19,040 I've been, you know, the... even among, you know, the people 56 00:03:19,140 --> 00:03:22,940 at, you know, such a college, I was still, you know, the outlier of, you know, just 57 00:03:22,980 --> 00:03:26,200 a tech nerd who would just, you know, spend all my time reading online about 58 00:03:26,240 --> 00:03:30,020 various technologies, you know, that could be, uh, space technology, 59 00:03:30,160 --> 00:03:33,820 could be anti-aging, could be solar, could be climate change. 60 00:03:33,860 --> 00:03:37,540 Like, just anything really. You know, I would just, on my own go up and, 61 00:03:37,600 --> 00:03:41,360 read things. So, naturally at some point or the other, 62 00:03:41,400 --> 00:03:44,940 also got recommended some books around AI risk. 63 00:03:44,980 --> 00:03:48,020 I think the first time I got introduced to AI risk, 64 00:03:48,240 --> 00:03:49,820 uh, 65 00:03:50,680 --> 00:03:54,460 was this, uh, recommendation in, I think, 66 00:03:54,500 --> 00:03:57,660 Bill Gates' reading list. Uh, he had a couple of 67 00:03:57,740 --> 00:04:00,460 recommendations, uh, one of them was, I think, Factfulness. 68 00:04:00,480 --> 00:04:03,920 There was another one which was Superintelligence by Nick Bostrom. 69 00:04:04,040 --> 00:04:07,840 Uh, so when I read, you know, that particular book, that 70 00:04:08,100 --> 00:04:11,630 took AI risk seriously. I realized, okay, look, okay... 71 00:04:11,660 --> 00:04:14,720 That was the first time where I started realizing, okay, look, this actually makes 72 00:04:14,760 --> 00:04:18,420 sense and, like, this could actually be a big deal. 73 00:04:18,440 --> 00:04:21,360 Whereas until then, it was just kind of like, you know, 74 00:04:21,380 --> 00:04:25,060 fiction thing in my head. Like... Uh, maybe 75 00:04:25,100 --> 00:04:28,750 another point of background would be like, uh, 76 00:04:29,280 --> 00:04:33,200 so during my schooling, uh, I used to use the internet a fair bit. 77 00:04:33,240 --> 00:04:36,960 I was active, like, on Stack Overflow, Stack Exchange, like those sites were more 78 00:04:37,000 --> 00:04:40,980 popular back then. I think now, like, AI has kind of, like, reduced the 79 00:04:41,000 --> 00:04:44,620 readership of these sites, but back then they were pretty popular, 80 00:04:44,780 --> 00:04:48,580 uh, among academia, among even, like, people who are new to 81 00:04:48,600 --> 00:04:50,860 any, like, scientific field but want to learn more. 82 00:04:50,940 --> 00:04:54,480 So, my username there was like, ghostsinthecode. 83 00:04:54,580 --> 00:04:58,420 And this was based on this movie, I, Robot, 84 00:04:58,440 --> 00:05:00,840 which was fairly popular. It's like a Hollywood movie. 85 00:05:00,960 --> 00:05:04,520 Uh, Will Smith was an actor. And it is 86 00:05:04,560 --> 00:05:06,600 basically about an AI took- takeover. 87 00:05:06,660 --> 00:05:08,840 Like, uh, there is 88 00:05:08,920 --> 00:05:12,790 this, you know, huge tech company that builds these, you 89 00:05:12,820 --> 00:05:16,740 know, armies of robots with AI, and eventually at some point the AIs plan 90 00:05:16,780 --> 00:05:20,620 takeover and, you know, in very cinematic Hollywood fashion, 91 00:05:20,660 --> 00:05:24,140 at the last moment, you know, the hero, you know, Will Smith comes and just 92 00:05:24,180 --> 00:05:27,830 physically shoots down a bunch of robots and, you know, hacks into the center and, 93 00:05:27,840 --> 00:05:31,390 you know, takes it down and stuff. Uh, I think that was the first 94 00:05:31,420 --> 00:05:34,730 part, even just as like a fun curiosity, I thought about it through 95 00:05:34,800 --> 00:05:38,740 like, like just some of the philosophical 96 00:05:38,750 --> 00:05:42,680 them even way back then. Uh, but yeah, back then 97 00:05:42,820 --> 00:05:45,420 I just thought of it as, you know, fiction, thought experiment. 98 00:05:45,540 --> 00:05:49,000 I didn't think of it as, oh, literally, you know, 10 years later we're going to be 99 00:05:49,100 --> 00:05:52,580 having these AIs and these are going to be actual questions and, 100 00:05:52,640 --> 00:05:54,969 of the world is actually going to depend on these types of things. 101 00:05:56,100 --> 00:05:59,060 Uh, so maybe that is maybe the- my first introduction. 102 00:05:59,080 --> 00:06:02,840 My first serious introduction was, yeah, in college, uh, 103 00:06:02,900 --> 00:06:06,768 reading Superintelligence by Nick Bostrom.Then I, you know, read 104 00:06:06,808 --> 00:06:09,468 through the reviews of, uh, superintelligence. 105 00:06:09,568 --> 00:06:09,888 Uh, 106 00:06:11,348 --> 00:06:14,728 there were people saying, "Look, you know, half of this content is copied from, you 107 00:06:14,748 --> 00:06:18,108 know, Eliezer Yudkowsky. Just go read what Eliezer Yudkowsky has 108 00:06:18,128 --> 00:06:22,008 instead." And then I was like, "Hmm, interesting." And then I went and read a 109 00:06:22,228 --> 00:06:26,208 lot of work of, you know, Eliezer Yudkowsky. I read The Sequences. 110 00:06:26,228 --> 00:06:29,808 That gave me a lot of things to think about. 111 00:06:29,928 --> 00:06:30,448 Uh, 112 00:06:31,368 --> 00:06:34,148 I think Eliezer Yudkowsky is, like, a brilliant philosopher. 113 00:06:34,168 --> 00:06:37,198 Like, I disagree with him on a lot of points, 114 00:06:37,228 --> 00:06:40,928 brilliant, like, this surface area of, you know, novel topics he has covered 115 00:06:40,968 --> 00:06:44,748 is large, and he has interesting thoughts 116 00:06:44,888 --> 00:06:48,288 on a lot of them, some of which I agree with and some of 117 00:06:48,408 --> 00:06:48,968 Uh, 118 00:06:50,328 --> 00:06:51,868 so there's that. 119 00:06:52,688 --> 00:06:56,068 So I think that was the point where I was first sure, okay, look, 120 00:06:56,128 --> 00:06:59,468 some chance, you know, our species is going extinct as a result of this 121 00:06:59,488 --> 00:07:03,428 technology. Back then, I, this, when I say 122 00:07:03,488 --> 00:07:07,157 back then, I mean, like, you know, maybe fourth, fourth year of college. 123 00:07:07,188 --> 00:07:09,947 I think back then I assigned like, you know, a 15% 124 00:07:10,668 --> 00:07:13,498 probability we'll be getting, you know, superintelligence soon, like, you know, in 125 00:07:13,508 --> 00:07:17,148 the next five years, 10 years. Now that number has gone up. 126 00:07:17,188 --> 00:07:19,428 Now I'm like, at least 25%, maybe more. 127 00:07:19,468 --> 00:07:22,768 We can talk about, you know, technicalities of these numbers 128 00:07:22,788 --> 00:07:25,748 basic idea is, yeah. Uh, a couple years 129 00:07:25,788 --> 00:07:29,588 ago, reading Eliezer Yudkowsky's work, that was the first time I 130 00:07:29,628 --> 00:07:33,568 started taking this stuff like se- seriously and being like, "Okay, look, 131 00:07:33,628 --> 00:07:37,588 there is some actual non-negligible chance that, you know, the species may 132 00:07:37,628 --> 00:07:41,278 not survive the creation of this technology." 133 00:07:41,328 --> 00:07:44,868 After that it's just me trying various things 134 00:07:45,368 --> 00:07:48,988 around this. I started the Effective Altruism Club 135 00:07:49,248 --> 00:07:52,908 in my college, IIT Delhi. It did 136 00:07:52,968 --> 00:07:56,928 not go super well, to be quite blunt, but I learned useful 137 00:07:57,028 --> 00:08:00,308 things from the experience of trying to start that club over a period of six 138 00:08:00,348 --> 00:08:01,328 months. 139 00:08:02,728 --> 00:08:06,348 What else? I completed this, uh, two-month, like, 140 00:08:06,408 --> 00:08:10,328 full-time, you know, whatever, 50, 60-hour week intensive computer 141 00:08:10,388 --> 00:08:14,068 science course called ML Safety Scholars by 142 00:08:14,187 --> 00:08:18,028 Dan Hendrycks who's at UC- UC Berkeley, and like one of the famous 143 00:08:18,068 --> 00:08:22,048 names in the AI risk space. That did give me some 144 00:08:22,128 --> 00:08:25,988 technical background, and that actually scared me a little bit more 145 00:08:26,068 --> 00:08:29,948 it made me understand to the extent to which, like, AIs are just 146 00:08:30,008 --> 00:08:33,148 black boxes and, like, the people who are training these things have actually no 147 00:08:33,208 --> 00:08:37,158 idea what's going on inside them. Like, I had a surface level of 148 00:08:37,188 --> 00:08:40,368 understanding of this, but then actually, you know, you know like writing down the 149 00:08:40,408 --> 00:08:43,608 equations for a transformer, okay, obviously, okay, attention is like this. 150 00:08:43,628 --> 00:08:46,648 These are the key val- query value matrices. 151 00:08:46,688 --> 00:08:50,288 And look, this is, you know, what a scaling law is, this is how 152 00:08:50,868 --> 00:08:54,368 for the entire field of interpretability re- research looks like. 153 00:08:54,388 --> 00:08:58,328 You know, this is what people when they talk about, you know, 154 00:08:58,368 --> 00:09:01,488 ali- other AIs, you know, at a technical level, this 155 00:09:01,508 --> 00:09:05,218 And just reading a bunch of these things from a technical level made me 156 00:09:05,268 --> 00:09:08,298 understand the extent to which, like, AI researchers have no idea what they're 157 00:09:08,308 --> 00:09:12,148 doing. And, like, some of these fields are so nascent and so much 158 00:09:12,188 --> 00:09:15,278 of what actually works is just like black box approaches, 159 00:09:15,328 --> 00:09:18,268 like try a thing without understanding why it works. 160 00:09:18,308 --> 00:09:21,288 It's just, "Look, you know, the last six months it worked, 161 00:09:21,328 --> 00:09:25,148 again." And that was an intuition which I really got 162 00:09:25,168 --> 00:09:28,948 through that course and just studying this, the topic 163 00:09:29,028 --> 00:09:32,268 of AI and, you know, AI alignment, AI safety at a more technical 164 00:09:32,288 --> 00:09:35,928 level. After that what did I 165 00:09:35,988 --> 00:09:36,968 do? Uh, 166 00:09:38,168 --> 00:09:39,368 after that to be 167 00:09:40,188 --> 00:09:44,088 quite honest I did take like a break of like at least a year from 168 00:09:44,148 --> 00:09:48,048 these things. So, you know, in my fifth year, my last year of college, 169 00:09:48,108 --> 00:09:51,458 did not do much about all this. I was just focused on like graduating. 170 00:09:51,468 --> 00:09:52,828 I finished my thesis, 171 00:09:53,648 --> 00:09:57,588 I, uh, traveled for a few months and at 172 00:09:57,628 --> 00:10:00,128 the end of the travel I was actually low-key depressed. 173 00:10:00,148 --> 00:10:04,088 I was like, "What am I doing with my life?" It gave me 174 00:10:04,148 --> 00:10:07,348 of like, "Look, I actually need to do something useful 175 00:10:08,028 --> 00:10:11,968 this sort of just traveling and having fun and stuff 176 00:10:12,008 --> 00:10:15,308 to work out for me. Like, you know, in moderation it's good, 177 00:10:15,328 --> 00:10:16,848 the focus of my life. 178 00:10:18,548 --> 00:10:22,357 And I came back and I read LessWrong and I looked, okay, I 179 00:10:22,368 --> 00:10:24,568 looked, two years have passed and absolutely nothing has 180 00:10:24,588 --> 00:10:28,548 changed. (laughs) Like, you know, the capabilities have increased 181 00:10:28,588 --> 00:10:32,208 a lot, like, you know, from 2022 to 2023 to, you know, 2025, like 182 00:10:32,648 --> 00:10:36,588 AI capabilities have gone further a lot, but, you 183 00:10:36,648 --> 00:10:40,088 know, all these people talking about, you know, safety, risk, governance, 184 00:10:40,128 --> 00:10:44,048 like it's the same people, the same boring speeches, the same, you 185 00:10:44,128 --> 00:10:47,738 know, arguments that happen on LessWrong. Like actually nothing has changed. 186 00:10:47,748 --> 00:10:51,688 It's the same funders, you know, um, Amos Goertz and Jan Talon doing 187 00:10:51,708 --> 00:10:54,228 the same sort of work which I don't really believe in. 188 00:10:54,348 --> 00:10:54,568 Uh, 189 00:10:56,288 --> 00:11:00,068 yeah, and I- I am now starting to realize, look, if I don't personally do 190 00:11:00,108 --> 00:11:03,968 something in this space, you know, the fate of the entire species might 191 00:11:04,068 --> 00:11:07,968 be at stake. And that is kind of an almost overwhelming sort of 192 00:11:08,008 --> 00:11:11,828 responsibility to be dealing with, uh, 193 00:11:11,888 --> 00:11:14,968 but yeah, I mean, at some level that is the truth. 194 00:11:14,988 --> 00:11:16,988 Like someone needs to try out new approaches. 195 00:11:17,028 --> 00:11:20,118 Someone needs to figure out ways this field can actually grow and like make an 196 00:11:20,118 --> 00:11:24,088 actual impact, the field of AI risk specifically, 197 00:11:24,128 --> 00:11:27,538 be one of the people to help figure that out. Yeah. 198 00:11:27,568 --> 00:11:31,508 Mm-hmm Wow. And so you decide to go 199 00:11:31,628 --> 00:11:34,728 on hunger strike. 200 00:11:34,738 --> 00:11:34,738 Uh, yes. 201 00:11:34,788 --> 00:11:38,468 What was your... How did you- how did you- how did you come to 202 00:11:38,508 --> 00:11:41,468 this decision? What was your thought process around 203 00:11:41,508 --> 00:11:45,228 this? 204 00:11:47,068 --> 00:11:50,648 That's a good question. Uh, 205 00:11:52,228 --> 00:11:56,088 so I can give you a bunch of reasons from like political theory and I can 206 00:11:56,128 --> 00:11:59,148 give you a bunch of reasons which are just, okay, look, I did hit 207 00:11:59,168 --> 00:12:03,008 was like, "Let's see what works." So the hit 208 00:12:03,088 --> 00:12:07,048 reasons I, look, I started a YouTube channel, 209 00:12:07,088 --> 00:12:10,548 videos about AI risk, I gave a bunch of, you know, talks at various 210 00:12:10,568 --> 00:12:14,028 places, I-... wrote this, you 211 00:12:14,088 --> 00:12:17,748 know, database for whistleblowers, which is kind of, you know, 212 00:12:17,828 --> 00:12:21,048 politically sensitive, so maybe we don't have to discuss it. 213 00:12:21,068 --> 00:12:24,048 But that is another thing, you know, how do you support a whistleblower at an 214 00:12:24,128 --> 00:12:28,098 company? I was doing that. And at some point, yeah, this 215 00:12:28,168 --> 00:12:32,148 idea came to me, like, "Look here, hunger strike actually makes sense." Uh, 216 00:12:33,048 --> 00:12:35,088 so it's just, you know, try a bunch of things. 217 00:12:35,128 --> 00:12:38,368 You know, so far what I was trying was not giving me very concrete, you know, 218 00:12:38,408 --> 00:12:41,788 success. I couldn't look at any of the things I'd 219 00:12:41,808 --> 00:12:45,708 this is a big success and it actually made a difference." So I 220 00:12:45,748 --> 00:12:47,828 was like, "Okay, look, let's just try one more thing 221 00:12:49,028 --> 00:12:52,828 that may have a chance of actually working." Uh, the reason from political 222 00:12:52,928 --> 00:12:56,588 theory is basically, uh, making 223 00:12:56,648 --> 00:13:00,438 costly public signals. Like if you signal to the public that, "Look, 224 00:13:00,468 --> 00:13:03,588 willing to make significant sacrifices for this," like, 225 00:13:04,248 --> 00:13:07,968 that speaks to the public more than just a few, you know, you go write a blog post 226 00:13:08,028 --> 00:13:11,978 or you make a video. Uh, like words are cheap, you know, actions are 227 00:13:12,028 --> 00:13:16,008 costly. Uh, they also have certain thoughts around, 228 00:13:16,028 --> 00:13:19,588 you know, common knowledge, like, uh... 229 00:13:19,688 --> 00:13:23,228 So when... So, like, in general and... 230 00:13:23,248 --> 00:13:27,188 Okay, I'll make a disclaimer here that, you know, I don't want to try and, you 231 00:13:27,228 --> 00:13:31,208 know, topple over any government when I say this, but in 232 00:13:31,248 --> 00:13:35,188 the past, when people have tried to topple over 233 00:13:35,248 --> 00:13:37,798 they have noticed is, uh... 234 00:13:39,888 --> 00:13:41,708 A key factor here is common knowledge. 235 00:13:41,728 --> 00:13:45,508 Like it's not enough for a lot of people, like if 10% of the population are 236 00:13:45,568 --> 00:13:49,048 like, you know, "We are unhappy with this government 237 00:13:49,068 --> 00:13:52,448 to be in charge," that is not enough to actually topple the 238 00:13:52,488 --> 00:13:56,308 It's like, I know that I am unhappy, and I know the other person is 239 00:13:56,367 --> 00:14:00,288 unhappy, and the other person knows that I know that they are unhappy, 240 00:14:00,308 --> 00:14:03,958 that sort of common knowledge. And only when you do that among, you know, 241 00:14:03,968 --> 00:14:06,688 significant fraction of the population, that's when you can actually topple the 242 00:14:06,708 --> 00:14:10,488 government. And ways to establish this common 243 00:14:10,528 --> 00:14:14,448 very public things. So this could be starting with, yes, uh, 244 00:14:14,548 --> 00:14:17,758 let's say, a YouTube channel, you know, a charismatic speaker has, you know, 245 00:14:17,788 --> 00:14:21,488 millions of likes. That establishes common knowledge because 246 00:14:21,497 --> 00:14:23,048 "Oh, it's not just me liking this video. 247 00:14:23,068 --> 00:14:26,668 There's also, you know, 10 million other people who 248 00:14:26,728 --> 00:14:30,668 This could be in public demonstrations, public protests if, you know, if there's 249 00:14:30,888 --> 00:14:34,028 like, you know, there's a city and, you know, like, 250 00:14:34,068 --> 00:14:36,888 physically on the street and they all look at each other 251 00:14:36,968 --> 00:14:39,208 Oh, okay, so I'm not alone in believing this. 252 00:14:39,248 --> 00:14:42,268 We all believe the same thing." That establishes common 253 00:14:42,328 --> 00:14:46,248 knowledge. The final stage is, you know, the point at which, 254 00:14:46,288 --> 00:14:50,068 you know, these mass movements can, you know, actually occupy, you know, 255 00:14:50,148 --> 00:14:54,108 buildings. Uh... 256 00:14:54,188 --> 00:14:58,108 Yeah, and that again establishes common 257 00:14:58,688 --> 00:15:00,867 buildings are sym- like, symbols. 258 00:15:00,908 --> 00:15:01,398 They're, you know, 259 00:15:02,568 --> 00:15:05,068 like... yeah, they're just symbolically important. 260 00:15:05,148 --> 00:15:08,928 And when everyone sees, okay, like the entire country will see, says, 261 00:15:08,988 --> 00:15:12,248 look, this movement has actually gone and occupied this building 262 00:15:12,308 --> 00:15:15,388 to, you know, actually stop them," then they realize, 263 00:15:15,408 --> 00:15:18,288 are going to form the new government. That's just it. 264 00:15:18,408 --> 00:15:18,848 So 265 00:15:20,148 --> 00:15:23,528 I think, yeah, hunger strikes are one of those costly signals. 266 00:15:23,608 --> 00:15:24,108 And 267 00:15:24,948 --> 00:15:28,208 if a lot of people can come in, you know, even just like, share, subscribe on 268 00:15:28,228 --> 00:15:31,388 hunger strike, and if hunger strike gets, you know, a million likes, that can even 269 00:15:31,428 --> 00:15:35,128 establish the common knowledge required to actually bring about change. 270 00:15:36,308 --> 00:15:38,288 Yeah, I was really surprised that, 271 00:15:38,408 --> 00:15:42,348 um, there was the kind 272 00:15:42,388 --> 00:15:46,218 of reception or the kind of impact that even this 273 00:15:46,228 --> 00:15:49,928 kind of limited action made. I think the, 274 00:15:49,948 --> 00:15:53,828 the, the post, uh, that Michael Trauzzi made announcing 275 00:15:53,888 --> 00:15:57,768 his strike was viewed over- maybe over one and a half million 276 00:15:57,828 --> 00:16:01,708 times. And the one that I made was viewed also 277 00:16:01,728 --> 00:16:05,648 over a million times, and it's generated a number of 278 00:16:05,728 --> 00:16:08,808 articles in, you know, press around the 279 00:16:08,848 --> 00:16:11,268 world. So, 280 00:16:12,908 --> 00:16:16,528 definitely was surprising to me the impact of even this kind of 281 00:16:17,208 --> 00:16:18,188 limited action. 282 00:16:19,008 --> 00:16:20,788 And I think, you know, if this is what 283 00:16:21,988 --> 00:16:24,068 five people, 284 00:16:25,188 --> 00:16:25,828 four people 285 00:16:27,028 --> 00:16:29,988 going, making this kind of costly 286 00:16:30,067 --> 00:16:33,998 action can do, imagine what 100 people could 287 00:16:34,048 --> 00:16:37,328 do, or 1,000 people. 288 00:16:37,387 --> 00:16:41,308 So I think this is really a crucial part of building 289 00:16:41,318 --> 00:16:43,468 this common public 290 00:16:43,548 --> 00:16:47,058 awareness. People have to see people 291 00:16:47,928 --> 00:16:51,028 that are acting consistently with what it means to be in an 292 00:16:51,048 --> 00:16:52,588 emergency. 293 00:16:53,568 --> 00:16:56,387 And, uh, I believe we are in an emergency, and, um, 294 00:16:57,808 --> 00:17:00,168 I'd be interested in hear- hearing from you. 295 00:17:00,188 --> 00:17:03,328 What, what is it about this time that we're in that makes it an 296 00:17:03,348 --> 00:17:05,648 emergency? 297 00:17:07,728 --> 00:17:11,688 Oh, uh, that's a good question. Uh, (laughs) 298 00:17:11,708 --> 00:17:15,068 I think when used the word emergency, like there's a rational aspect to it 299 00:17:15,108 --> 00:17:18,988 emotional aspect to it. And, you know, I'm the type of person who tries to 300 00:17:19,028 --> 00:17:22,938 the two. Uh, I still have complicated thoughts on, 301 00:17:22,988 --> 00:17:26,677 well... Like, should these two be separate? 302 00:17:26,768 --> 00:17:30,458 Uh, so let me just first talk about the 303 00:17:30,568 --> 00:17:32,828 Uh, like rationally, you know, what do I believe? 304 00:17:32,848 --> 00:17:34,268 What is the state of the world right now? 305 00:17:34,308 --> 00:17:36,648 What could the state of the world be in the next few years? 306 00:17:36,788 --> 00:17:40,958 Uh... 307 00:17:41,608 --> 00:17:45,468 I think assuming, uh, there is no such social 308 00:17:45,548 --> 00:17:49,408 movement, no one actually tries to change the course 309 00:17:49,468 --> 00:17:52,498 labs are trying to take the course of history, uh, 310 00:17:53,548 --> 00:17:57,248 even in the next five years, there's, you know, 311 00:17:57,288 --> 00:17:58,808 going to build superintelligence. 312 00:17:58,888 --> 00:18:02,088 Uh, when I say superintelligence, I mean an AI that is better than all 313 00:18:02,128 --> 00:18:04,888 all tasks. And this is just the next five years. 314 00:18:04,908 --> 00:18:07,208 You know, just because they don't do it in the next 315 00:18:07,248 --> 00:18:09,858 won't do it afterwards. If you take, you know, another five years, so that's, 316 00:18:09,888 --> 00:18:13,868 know, 2035, you can easily put another, you know, 15 to 20% ta- 317 00:18:13,928 --> 00:18:16,308 like... Again these numbers are not very accurate. 318 00:18:16,328 --> 00:18:18,068 There's a lot of high uncertainty on these things. 319 00:18:18,128 --> 00:18:22,068 So-Even by 2035, you're looking at, what, an at 320 00:18:22,108 --> 00:18:25,688 least 40%, 50% chance that we will build superintelligence. 321 00:18:26,688 --> 00:18:30,148 So at that point you realize, look, if you're making a life plan 322 00:18:30,808 --> 00:18:34,388 that does not take into account the fact that, you know, superintelligence might 323 00:18:34,428 --> 00:18:38,168 be created on this earth, like, what are you doing with your life? 324 00:18:38,188 --> 00:18:41,948 Like, that's just a bad move. Uh, if superintelligence is 325 00:18:41,968 --> 00:18:43,968 created, what are the outcomes? Uh, 326 00:18:44,848 --> 00:18:46,908 I think multiple outcomes are possible. 327 00:18:46,928 --> 00:18:49,348 This is the thing I have more uncertainty over. 328 00:18:49,368 --> 00:18:52,728 Like, I have less uncertainty on like, okay, 329 00:18:52,768 --> 00:18:56,608 build soon. I have more uncertainty on, okay, if we build superintelligence, 330 00:18:56,648 --> 00:19:00,588 what happens next? I think there's a good chance 331 00:19:00,688 --> 00:19:02,888 just has its own thing it cares about. 332 00:19:02,928 --> 00:19:06,868 It can, you know, make plans over long time horizons 333 00:19:06,908 --> 00:19:10,668 the thing it wants to do, which probably has, like, 334 00:19:10,728 --> 00:19:14,148 with humans or what humans want and whatever. 335 00:19:14,228 --> 00:19:16,768 And it'll just take over the earth. 336 00:19:16,868 --> 00:19:17,228 Uh, 337 00:19:18,408 --> 00:19:22,328 it will basically ignore us, and as a side effect, like, we 338 00:19:22,448 --> 00:19:26,328 might just get wiped out because, 339 00:19:26,448 --> 00:19:29,428 uh, it will have things it wants to do. 340 00:19:29,448 --> 00:19:31,388 You know, there are resources on the earth. 341 00:19:31,448 --> 00:19:34,258 There are specific, you know, temperature and pressure conditions on the 342 00:19:34,288 --> 00:19:35,768 which are required for human survival. 343 00:19:35,808 --> 00:19:38,588 There are specific, you know, food resources, water resources which 344 00:19:38,608 --> 00:19:42,558 for human survival. If the AI decides that even one of these resources, 345 00:19:42,608 --> 00:19:46,388 it wants, it will just take those resources 346 00:19:46,428 --> 00:19:50,088 uninterested in what, you know, humans want or need to live. 347 00:19:50,848 --> 00:19:54,608 Uh, that is a possibility. This is the other separate possibility 348 00:19:54,648 --> 00:19:58,398 in, is even if you manage to, with certain techniques, keep the 349 00:19:58,448 --> 00:20:02,008 superintelligence under control, like, you know, you apply boxing techniques, 350 00:20:02,108 --> 00:20:05,948 uh, for all the, you know, alignment safety nerds out there, 351 00:20:06,208 --> 00:20:09,868 what AI boxing is. But it's... Yeah, literally keep the AI in a 352 00:20:09,928 --> 00:20:12,888 box so that even though it's smarter than you, it just, um, harder to then break 353 00:20:12,908 --> 00:20:16,668 out. Which is, to be clear, an extremely risky thing to be 354 00:20:16,708 --> 00:20:20,608 trying to do. But even if you actually managed to pull this off, 355 00:20:20,628 --> 00:20:23,828 I think the outcome with that sort of thing 356 00:20:23,868 --> 00:20:25,468 take over the world. Uh, 357 00:20:26,568 --> 00:20:30,228 you can, at this point, just, uh, topple over various governments. 358 00:20:30,268 --> 00:20:33,088 You know, this is a threat to, you know, the US government, the Chinese government, 359 00:20:33,108 --> 00:20:34,388 the Indian, Russian governments. 360 00:20:34,488 --> 00:20:37,998 If a small group of people have an actual superintelligence that's in their 361 00:20:38,028 --> 00:20:41,948 control, and they can do novel things with this, 362 00:20:42,028 --> 00:20:45,818 just build parallel industries. They can build, you know... 363 00:20:45,828 --> 00:20:46,018 (laughs) 364 00:20:46,948 --> 00:20:50,568 Uh, I mean, Yudkowsky famously talks about 365 00:20:50,968 --> 00:20:54,708 that is also possible. We can even just look at, you know, 366 00:20:54,788 --> 00:20:58,648 life, like, we are already not that far from, away from 367 00:20:58,808 --> 00:21:02,328 being a- able to, you know, genetically engineer life forms so you can 368 00:21:02,348 --> 00:21:04,868 populate the earth with, you know, engineered life forms. 369 00:21:04,988 --> 00:21:05,288 Uh, 370 00:21:06,508 --> 00:21:09,988 you can invent new energy sources with a 371 00:21:10,048 --> 00:21:14,037 sufficiently intelligent mind. Uh, like, we are already doing these 372 00:21:14,088 --> 00:21:15,488 things. Like, forget superintelligence. 373 00:21:15,508 --> 00:21:17,928 Like, humans might be able to do these things in 374 00:21:17,948 --> 00:21:19,968 Like, this is not science fiction in my head. 375 00:21:20,037 --> 00:21:23,688 Like, if humans make, you know, solar energy really cheap or they crack, 376 00:21:23,768 --> 00:21:27,648 fusion energy or something, like, these things seem totally possible to me, 377 00:21:27,688 --> 00:21:30,108 and the superintelligence is just going to accelerate these things. 378 00:21:30,128 --> 00:21:33,788 So if a small group of people can just build things like this, they can 379 00:21:33,868 --> 00:21:37,468 build, you know, their own power grid, they can be extremely, 380 00:21:37,488 --> 00:21:41,368 extremely persuasive. Like, just imagine someone like Hitler 381 00:21:41,428 --> 00:21:44,848 way more persuasive, and you have, like, direct access to millions of people. 382 00:21:44,888 --> 00:21:48,588 You can disperse very people at scale, you can get really good at cyberattacks. 383 00:21:48,648 --> 00:21:52,028 You know, AI is already, as of 2025, pretty good at 384 00:21:52,048 --> 00:21:56,008 cyberattacks. You know, Mark Zuckerberg has openly on the record 385 00:21:56,028 --> 00:21:56,388 you know, 386 00:21:57,328 --> 00:22:00,948 like, AI is already a threat to the 387 00:22:01,068 --> 00:22:05,058 Uh, so just a fi- you know, five years more of progress on that, 388 00:22:05,088 --> 00:22:07,968 just basically hack into any system, any government database and so 389 00:22:08,108 --> 00:22:11,908 on. Like, there's so many different capability just 390 00:22:11,988 --> 00:22:15,908 open up all at the same time. Yeah, it seems quite plausible to me a small 391 00:22:15,948 --> 00:22:19,248 of people will just take over the world and establish their own, you know, like, 392 00:22:19,288 --> 00:22:22,628 their permanent dictatorship of sorts and just do what they want. 393 00:22:22,648 --> 00:22:26,598 That seems possible to me. There's also a possibility that 394 00:22:26,628 --> 00:22:29,968 there's just some unknown unknown sort of thing happens which, you know, I can't 395 00:22:29,988 --> 00:22:32,298 predict. Like, these things are just really hard to predict. 396 00:22:32,348 --> 00:22:34,728 There are just so many intersecting things going on. 397 00:22:34,748 --> 00:22:38,178 There's AI capabilities going on, there's, you know, specific 398 00:22:38,228 --> 00:22:42,208 questions like, okay, if I had cyberattacking capability, if 399 00:22:42,268 --> 00:22:46,228 I had a fully autonomous military, if I had XYZ technology, 400 00:22:46,268 --> 00:22:48,508 like, how much power can I project on the world? 401 00:22:48,518 --> 00:22:51,128 Like, some of these questions are just honestly open questions. 402 00:22:51,148 --> 00:22:54,138 Like, me sitting in my bedroom is not... 403 00:22:54,148 --> 00:22:57,348 I'm not necessarily going to be able to, you know, forecast all of these questions 404 00:22:57,408 --> 00:23:00,388 correctly. So we might just end up with some unknown 405 00:23:00,428 --> 00:23:03,588 unknown, uh, possibility which, 406 00:23:03,668 --> 00:23:06,748 again, m- doesn't necessarily need to be good. 407 00:23:06,768 --> 00:23:08,248 You know, it could be good, could be bad. 408 00:23:08,318 --> 00:23:08,568 So 409 00:23:09,608 --> 00:23:12,928 it's just, yeah, it's just going to dramatically change the 410 00:23:12,968 --> 00:23:16,888 And just from a rational point of view, I think that constitutes an emergency when 411 00:23:16,928 --> 00:23:19,828 you look at these sort of extremely dramatic 412 00:23:19,888 --> 00:23:23,328 outcomes. Now, coming to an emotional point of view, 413 00:23:23,568 --> 00:23:27,108 uh, 414 00:23:29,428 --> 00:23:31,648 yeah, I don't know. I think I'm still working my way through 415 00:23:31,688 --> 00:23:35,468 So there's one set of philosophies which are kind of popular in 416 00:23:35,508 --> 00:23:39,188 Silicon Valley, and I have kind of a love-hate 417 00:23:39,208 --> 00:23:43,108 ideas which are popular in Silicon Valley, which is, yeah, basically a 418 00:23:43,168 --> 00:23:43,928 lot of, like, 419 00:23:44,788 --> 00:23:48,668 stoicism and, like, you know, if 420 00:23:48,708 --> 00:23:51,988 there is a crisis and you are the leader and you are calm, you know, 421 00:23:52,028 --> 00:23:55,888 good decisions. Like, there's a lot of emphasis on this sort of 422 00:23:55,988 --> 00:23:59,548 even if the world is on fire, you need to be calm, 423 00:23:59,588 --> 00:24:03,528 the good decisions here. Like, this is the thing that's popular in 424 00:24:03,548 --> 00:24:07,488 Silicon Valley. There's- they- there's a general emphasis on, like, 425 00:24:07,528 --> 00:24:10,708 this is what great people do, and 426 00:24:12,768 --> 00:24:15,748 to part... for part of me, that makes sense. 427 00:24:15,788 --> 00:24:19,688 Like... but, uh, there is 428 00:24:19,708 --> 00:24:23,488 also another part of me that is like, okay, if you take that ideology to 429 00:24:23,528 --> 00:24:27,468 its logical extreme, you really do end up with a person who 430 00:24:27,548 --> 00:24:29,568 not care about these things in the first place. 431 00:24:29,608 --> 00:24:33,472 Like, the world is on-Like, there's a difference between, "Look, 432 00:24:33,512 --> 00:24:37,472 is on fire and I'm upset about this, but I won't, you know, allow this 433 00:24:37,532 --> 00:24:39,992 to overwhelm me," or, "I won't show this to the public. 434 00:24:40,052 --> 00:24:43,831 I will be the calm voice in the face of the storm." And there is another 435 00:24:44,532 --> 00:24:47,312 way of processing it, which is, "Look, the world is on fire. 436 00:24:47,331 --> 00:24:50,492 And look, I don't care about this at all 437 00:24:50,552 --> 00:24:53,892 I never cared about this in the first place." And like, I think there's a 438 00:24:54,831 --> 00:24:58,292 dividing line between those. So, you know, 439 00:24:58,372 --> 00:25:02,331 emotionally, am I looking at it like I'm in an 440 00:25:02,372 --> 00:25:06,132 level? Yes. But at some level, I'm also kind of leading a 441 00:25:06,172 --> 00:25:10,072 relatively normal life. Like, okay, I have devoted my career, you know, full 442 00:25:10,132 --> 00:25:12,972 time to working on this, but apart from my career, what am I doing? 443 00:25:12,992 --> 00:25:16,562 You know, I have a relatively normal soc- a normal social life. 444 00:25:16,572 --> 00:25:20,392 You know, I have friends, you know, I have fun activities I do with them, 445 00:25:20,432 --> 00:25:21,872 I travel places. Like, 446 00:25:23,272 --> 00:25:26,372 if you look at, you know, other aspects of my life, 447 00:25:26,412 --> 00:25:30,232 and be like, "Oh, Samuel is, you know, in an emergency or a crisis right now." 448 00:25:32,492 --> 00:25:33,072 so 449 00:25:35,652 --> 00:25:38,452 yeah, all in all, I don't know. I think that just gives you a lot of 450 00:25:38,472 --> 00:25:41,612 on, you know, to what extent am I looking at this as an 451 00:25:41,652 --> 00:25:44,572 Yeah. 452 00:25:46,912 --> 00:25:49,732 Yeah. For, for me there's a couple of aspects to it. 453 00:25:49,812 --> 00:25:53,302 One is definitely that there is 454 00:25:54,831 --> 00:25:55,851 an immense risk 455 00:25:56,932 --> 00:26:00,622 in front of us. And 456 00:26:01,952 --> 00:26:05,802 it threatens the lives of everyone I care about, and really everyone 457 00:26:05,851 --> 00:26:09,792 on this planet. And like you said, I look at 458 00:26:09,831 --> 00:26:12,952 the trajectory. If there isn't a movement or if there 459 00:26:12,972 --> 00:26:15,432 effective, um, political 460 00:26:17,152 --> 00:26:17,652 project 461 00:26:18,992 --> 00:26:22,572 to prevent this risk, um, 462 00:26:22,591 --> 00:26:25,572 we're, we're gonna suffer those consequences, 463 00:26:25,632 --> 00:26:28,972 are. And because there's not 464 00:26:29,032 --> 00:26:32,292 currently any signs that we are on track 465 00:26:33,192 --> 00:26:36,952 to effectively manage it, um, like 466 00:26:37,452 --> 00:26:41,181 to me that's an emergency. It's kind of like, you 467 00:26:41,211 --> 00:26:44,972 know, we're in a car headed downhill, gaining 468 00:26:45,092 --> 00:26:47,932 speed and there's no brakes on the car. 469 00:26:47,952 --> 00:26:51,572 It's like, that's an emergency. 470 00:26:52,392 --> 00:26:52,592 So 471 00:26:53,912 --> 00:26:57,792 yeah, it is, it is hard sometimes to separate the 472 00:26:57,851 --> 00:26:59,831 rational aspect. 473 00:26:59,952 --> 00:27:08,341 Um... 474 00:27:08,432 --> 00:27:12,402 I think for me it, it boils down to rationally understanding 475 00:27:12,412 --> 00:27:14,402 we're in danger, and 476 00:27:15,492 --> 00:27:18,792 emotionally, the most important thing that I feel is 477 00:27:18,912 --> 00:27:22,831 not turning away or trying to, 478 00:27:22,952 --> 00:27:26,132 um, deny that fact, and 479 00:27:26,212 --> 00:27:29,952 then emotionally face it, face what it means, and then 480 00:27:30,072 --> 00:27:33,712 act rationally, appropriately to try and stop 481 00:27:33,752 --> 00:27:36,252 it. So, 482 00:27:37,032 --> 00:27:37,812 yeah. 483 00:27:38,831 --> 00:27:42,152 H- how are you? We should have, we should have got this, uh, done in the, 484 00:27:42,192 --> 00:27:45,242 beginning, but you've been on hunger strike now, 485 00:27:45,272 --> 00:27:49,212 fasting. How did you... Do you prefer to call it a fast or a 486 00:27:49,232 --> 00:27:52,501 hunger strike? And you've been, but you've been on it for eight days 487 00:27:52,572 --> 00:27:55,851 now, so I'm curious how your experience has 488 00:27:56,512 --> 00:27:59,152 personally. 489 00:27:59,212 --> 00:28:02,812 Yeah. Uh, I think I was initially a bit hesitant to call it a 490 00:28:02,892 --> 00:28:06,032 because, yeah, there were a lot of, you know, random Twitter trolls who 491 00:28:06,072 --> 00:28:08,061 "Look, you're not actually putting your life in 492 00:28:08,112 --> 00:28:12,012 Is that a hunger strike or not?" But at this point I'm like, "Look, I don't 493 00:28:12,092 --> 00:28:14,882 care about that." I do still consider this a hunger strike. 494 00:28:14,892 --> 00:28:18,632 Like, I am actually making a costly sacrifice 495 00:28:18,692 --> 00:28:20,672 actually worse as a result. And 496 00:28:21,851 --> 00:28:25,252 like, if I was just doing what's good for me, 497 00:28:25,272 --> 00:28:29,232 I'm doing this because I believe there's a bigger, you know, cause that needs to 498 00:28:29,272 --> 00:28:29,612 be, 499 00:28:30,492 --> 00:28:32,371 uh, advanced here. Uh, 500 00:28:33,312 --> 00:28:35,592 so yeah, I would call it a hunger strike. 501 00:28:35,692 --> 00:28:39,312 Uh, how am I doing? Ah, that's a good 502 00:28:39,432 --> 00:28:43,322 question. I think it's pretty much manageable, 503 00:28:43,371 --> 00:28:46,662 of, you know, today, which is day number eight of the hunger strike. 504 00:28:46,692 --> 00:28:49,652 I've publicly committed that I'll be doing this for at least 18 505 00:28:49,712 --> 00:28:51,092 days. 506 00:28:51,191 --> 00:28:52,112 Oh wow. 507 00:28:52,251 --> 00:28:54,581 Yes. Yeah. Uh... 508 00:28:57,752 --> 00:29:01,472 I mean, it is not great, but it is manageable. 509 00:29:01,512 --> 00:29:05,272 Like, I am definitely extremely, like, physically weak, like just 510 00:29:05,732 --> 00:29:09,472 less energy in my body. Like when I walk around, 511 00:29:09,512 --> 00:29:13,092 look, don't bump yourself into things, don't get yourself injured, you know, 512 00:29:13,152 --> 00:29:17,052 fall over. Uh, I have, I 513 00:29:17,092 --> 00:29:20,632 randomly have nausea at points. I have randomly, you know, 514 00:29:20,652 --> 00:29:22,492 points. Uh, 515 00:29:23,652 --> 00:29:26,262 like by itself, you know, none of these things is a big deal. 516 00:29:26,272 --> 00:29:27,952 All of this is very much manageable. 517 00:29:27,992 --> 00:29:28,592 It's just 518 00:29:30,192 --> 00:29:34,092 or, you know, I drink salt water, it tastes, you know, absolutely horrible. 519 00:29:34,192 --> 00:29:37,612 Uh, all these things by itself, they're small, but when they 520 00:29:37,652 --> 00:29:40,952 accumulate, you know, day after day after day just dealing with 521 00:29:41,892 --> 00:29:45,292 shit, it does kind of take a toll on you 522 00:29:45,331 --> 00:29:47,672 mentally. Also, I'm just kind of bored. 523 00:29:47,812 --> 00:29:51,632 I- I'm not doing anything productive and my mind is not clear 524 00:29:51,672 --> 00:29:53,892 enough to do any, like very focused work. 525 00:29:53,972 --> 00:29:54,132 So 526 00:29:55,892 --> 00:29:59,652 I end up spending a lot of time doing absolutely nothing, and then 527 00:29:59,712 --> 00:30:01,492 that feels not great. So 528 00:30:04,152 --> 00:30:06,312 yeah, it's just like a m- like a 529 00:30:07,132 --> 00:30:11,052 subpar quality of life. When you extend that, you know, over many, 530 00:30:11,132 --> 00:30:13,792 it does start taking a bit of a toll on you. 531 00:30:13,812 --> 00:30:16,992 But all in all, I think it's pretty much manageable. 532 00:30:17,032 --> 00:30:21,022 You know, I am fine doing this for, doing this for, you know, another week, 533 00:30:21,052 --> 00:30:24,081 another two weeks if it comes to it. Yeah. 534 00:30:24,112 --> 00:30:27,672 So to be clear, you're just taking, um, basically 535 00:30:27,812 --> 00:30:31,772 salts for electrolytes, but otherwise nothing, no 536 00:30:31,851 --> 00:30:34,672 other... no calories, no vitamins, nothing like that? 537 00:30:35,660 --> 00:30:39,150 Uh, no. Yeah, I'm just drinking water and I'm 538 00:30:39,200 --> 00:30:42,460 having, you know, table salt, you know, sodium 539 00:30:42,600 --> 00:30:46,470 chloride, uh, mag- uh, potassium chloride, which is again, 540 00:30:46,480 --> 00:30:50,340 another salt. I'm having, you know, magnesium capsules and that's about 541 00:30:50,380 --> 00:30:50,600 it. 542 00:30:50,610 --> 00:30:50,610 Yeah. 543 00:30:50,680 --> 00:30:53,900 I'm not having any calories, no sugar, no 544 00:30:53,940 --> 00:30:55,640 vitamins, so on. 545 00:30:55,650 --> 00:30:55,650 Mm. 546 00:30:55,700 --> 00:30:56,060 Yeah. 547 00:30:56,480 --> 00:30:59,380 Mm. Yeah, I- I wonder if that's a big difference in our 548 00:31:00,140 --> 00:31:03,920 experiences 'cause I am taking, um, a multivitamin 549 00:31:04,120 --> 00:31:08,020 and other vitamin supplements, but no c- no calories, and also 550 00:31:08,080 --> 00:31:12,000 electrolytes. And like, subjectively, 551 00:31:12,060 --> 00:31:15,960 of course, I feel tired and I have the same nausea 552 00:31:16,080 --> 00:31:19,600 and, um, you know, some stomach pain. 553 00:31:19,640 --> 00:31:23,600 But, um, mentally, I feel like very 554 00:31:23,640 --> 00:31:27,500 there, like very, um, you know, 555 00:31:27,560 --> 00:31:31,360 aware and in full possession of my faculties and everything, 556 00:31:32,600 --> 00:31:36,460 so. Or maybe I'm just (laughs) maybe I'm just 557 00:31:36,540 --> 00:31:40,360 (laughs) being there doesn't mean all that much (laughs) as it may- as it 558 00:31:40,420 --> 00:31:40,940 may to you. 559 00:31:42,120 --> 00:31:42,560 But, um, 560 00:31:43,520 --> 00:31:44,730 yeah, that's amazing. I didn't... 561 00:31:44,780 --> 00:31:48,560 I- I- I missed that part where you had- had committed to- to going for 18 562 00:31:48,620 --> 00:31:51,960 days, so that's... Yeah, that's impressive. 563 00:31:52,060 --> 00:31:55,860 Um, I'm curious, how is your... How are your family and friends, 564 00:31:55,960 --> 00:31:58,630 uh, responding? Do they know about this or what do they think? 565 00:31:58,660 --> 00:32:00,160 Have you talked to them about it? 566 00:32:01,320 --> 00:32:05,060 Uh, yeah, the response are mixed. I've definitely got just 567 00:32:05,120 --> 00:32:08,500 support, solidarity from some of my friends who 568 00:32:08,560 --> 00:32:11,580 thing you're doing and I wish you all the best," and so 569 00:32:11,700 --> 00:32:15,600 on. There's certain people who don't really 570 00:32:15,620 --> 00:32:18,900 they don't really buy the case for AI risk to the extent that I do. 571 00:32:18,920 --> 00:32:22,460 But despite that, they're like, "Look, you know, you believe in this, you want to 572 00:32:22,500 --> 00:32:24,240 do this, you know, good for you. All the best. 573 00:32:24,280 --> 00:32:25,940 I hope you, you know, I hope you stay safe. 574 00:32:25,980 --> 00:32:29,760 I hope your health stays fine, you know, don't get hospitalized and all that 575 00:32:29,840 --> 00:32:33,400 stuff." Uh, I have gotten a couple of 576 00:32:33,440 --> 00:32:36,650 responses I'm not happy about where I did realize, okay, look, 577 00:32:37,480 --> 00:32:41,180 these people, you know, out of a misguided sense of, you know, 578 00:32:41,220 --> 00:32:44,199 me might actually try to, you know, stop me from continuing on 579 00:32:44,240 --> 00:32:48,000 this. And to them, I have made very clear, like, "Look, this is 580 00:32:48,060 --> 00:32:50,580 important to me, you know, don't get in my way. 581 00:32:50,620 --> 00:32:54,420 Like, I know you have, you know, sort of certain, you know, care 582 00:32:54,460 --> 00:32:58,180 me, but if you actually truly cared about me, 583 00:32:58,220 --> 00:33:01,960 this is to me. So, you know, don't- don't get in my way." 584 00:33:02,000 --> 00:33:02,900 Yes, that 585 00:33:04,000 --> 00:33:04,160 I 586 00:33:05,900 --> 00:33:08,980 have gotten again, you know, even more the responses, they're like, 587 00:33:09,020 --> 00:33:11,960 they're not super happy with me doing this, but 588 00:33:12,960 --> 00:33:16,440 also end of the day, look, the choice is not in their hands, the choice is in my 589 00:33:16,450 --> 00:33:20,420 hands and this is what I'm doing. So yeah, the responses have been pretty mixed all 590 00:33:20,460 --> 00:33:23,350 around. I think I'm generally happy with the 591 00:33:27,040 --> 00:33:29,360 And, uh, I guess just as a final question, 592 00:33:30,160 --> 00:33:33,970 I'm wondering how do you see yourself, uh, 593 00:33:34,100 --> 00:33:37,460 going forward in the future? What's, um, 594 00:33:38,139 --> 00:33:39,420 you know, what's on your mind about 595 00:33:40,600 --> 00:33:42,420 what part you can play in this whole 596 00:33:43,440 --> 00:33:45,540 situation that we're in? 597 00:33:46,800 --> 00:33:50,720 Uh, that's a good question. Yeah, I mean 598 00:33:50,800 --> 00:33:54,160 if you ask me concretely, do I have like, a role I want to play and just 599 00:33:54,200 --> 00:33:55,980 consistently play that for five years? 600 00:33:56,000 --> 00:33:59,520 Like, I don't have conviction on what that role is. 601 00:33:59,580 --> 00:34:02,920 I'm still in a phase of, you know, hit and trial, try a bunch of things and see 602 00:34:02,960 --> 00:34:04,920 what works, uh- 603 00:34:04,960 --> 00:34:07,800 What's the goal that you're working towards? 604 00:34:07,900 --> 00:34:11,560 I mean, the high level goal is definitely, you know, get a pause 605 00:34:11,739 --> 00:34:15,440 on further research development of AI 606 00:34:15,540 --> 00:34:19,239 worldwide across all countries. This will require 607 00:34:19,280 --> 00:34:23,150 necessarily a treaty between the government of the United States and the 608 00:34:23,199 --> 00:34:27,080 government of China. This is going to require a certain 609 00:34:27,120 --> 00:34:29,280 number of en- enforcement mechanisms. 610 00:34:29,290 --> 00:34:32,500 This is going to require a lot of people actually convinced that their lives are in 611 00:34:32,560 --> 00:34:36,199 danger. Like, you cannot pull off these sort of large 612 00:34:36,260 --> 00:34:37,670 actually convinced a lot of people. 613 00:34:37,670 --> 00:34:41,520 Like, that is the high level goal. Uh, I could also 614 00:34:41,580 --> 00:34:44,900 go into like, what are the more smaller things 615 00:34:44,920 --> 00:34:48,239 that I'm working on, if you'd like to kn- know about those. 616 00:34:48,260 --> 00:34:51,920 Well, what's your... Yeah, what's your- what's your kind of, um, big 617 00:34:52,040 --> 00:34:55,940 picture of the path? How do- how- how generally 618 00:34:56,279 --> 00:34:59,340 do you think we... What do we need to do to get from here to 619 00:34:59,420 --> 00:35:01,960 there? 620 00:35:03,760 --> 00:35:07,580 Uh, good question. So I think in general, if you 621 00:35:07,620 --> 00:35:10,500 want to get a person to do something, there are two ways to do it. 622 00:35:10,520 --> 00:35:13,760 You know, A, you can convince them let- in their heart that, "Look, this is 623 00:35:13,820 --> 00:35:17,180 actually no- important." Like, you actually change their values 624 00:35:17,240 --> 00:35:20,080 And B, as you apply incentives on them, you apply pressure. 625 00:35:20,120 --> 00:35:23,060 Now, pressure could be actual physical 626 00:35:23,120 --> 00:35:26,160 It could be, you know, monetary incentives. 627 00:35:26,180 --> 00:35:29,540 You tell them, "Look, if you do this, you'll be respected," et cetera. 628 00:35:29,720 --> 00:35:33,600 I think this... Solving this problem is going to require a mix 629 00:35:33,640 --> 00:35:37,060 of both. Like, you can't use just force 630 00:35:37,100 --> 00:35:40,060 persuasion. There are going to be people doing a bi- bit of 631 00:35:40,120 --> 00:35:41,920 both. Uh, 632 00:35:43,480 --> 00:35:43,650 but- 633 00:35:43,680 --> 00:35:47,480 You mentioned earlier about, um, a mass movement. 634 00:35:48,690 --> 00:35:48,690 Uh- 635 00:35:48,720 --> 00:35:52,120 What do you think the... What do you see the role of a- 636 00:35:52,160 --> 00:35:56,070 process of moving society from where it is now 637 00:35:56,100 --> 00:35:59,640 to where it needs to be to actually implement that kind 638 00:35:59,720 --> 00:36:03,480 of a global treaty? Or even starting with the 639 00:36:03,520 --> 00:36:05,230 United States, right? 640 00:36:05,230 --> 00:36:05,230 Mm-hmm. 641 00:36:05,240 --> 00:36:09,060 Because the United States right now is nowhere in the ballpark of politically 642 00:36:09,760 --> 00:36:10,020 of 643 00:36:11,800 --> 00:36:15,640 seriously considering that even as an option. 644 00:36:15,700 --> 00:36:17,790 Yeah, that is fair. Okay, so the... 645 00:36:17,800 --> 00:36:21,120 I think historically, the way that, you know, effective altruist movement has 646 00:36:21,160 --> 00:36:25,060 approached this is a very softball kind of approach, which 647 00:36:25,140 --> 00:36:29,060 is, again, I'm not saying like, it is bad 648 00:36:29,120 --> 00:36:31,800 to try this. I'm just saying, you know, maybe they're failing. 649 00:36:31,860 --> 00:36:35,680 What they tried is essentially, "Look, we will get people in all the 650 00:36:35,740 --> 00:36:39,531 elite circles of government, you know, we will..."Uh, establish think 651 00:36:39,631 --> 00:36:42,491 tanks. We'll establish policy. Policy institutions. 652 00:36:42,531 --> 00:36:45,591 We'll get just people in government and, you know, just convince people in the US 653 00:36:45,611 --> 00:36:48,732 government this is important and just make a- action work that way. 654 00:36:48,752 --> 00:36:50,832 So they just went through the persuasion approach. 655 00:36:51,631 --> 00:36:54,131 And I'm not optimistic on this working out. 656 00:36:54,172 --> 00:36:58,131 I think the geopolitical power you get from developing AI is too 657 00:36:58,172 --> 00:37:01,971 large for, uh, people at the high levels of 658 00:37:02,031 --> 00:37:05,332 US government to be just listening toward this effective and trust people have to 659 00:37:05,392 --> 00:37:09,192 say. Uh, so just 100% persuasion is not g- 660 00:37:09,232 --> 00:37:12,682 really going to work in my view, uh, which comes to, you know, 661 00:37:12,692 --> 00:37:16,392 establishing a mass movement and actually applying pressure on the US 662 00:37:16,412 --> 00:37:18,372 Like, you can't do just persuasion. 663 00:37:18,392 --> 00:37:21,892 You need to actually pressure that, look, you know, if you don't actually do the 664 00:37:21,912 --> 00:37:25,761 things that we want you to do, like if we, the public, say we want this and the 665 00:37:25,812 --> 00:37:28,852 government is, like, not willing to do it, you need to actually apply 666 00:37:29,652 --> 00:37:33,352 pressure and be like, "Look, we are willing to vote new leaders 667 00:37:33,372 --> 00:37:36,392 We're willing to, you know, bi- boycott your products if required. 668 00:37:36,412 --> 00:37:39,611 You know, we are willing to protest on the street 669 00:37:39,712 --> 00:37:43,611 Uh, we are going to disrupt your way of doing 670 00:37:43,712 --> 00:37:46,531 think a mass movement is a good idea. 671 00:37:46,591 --> 00:37:50,392 Establishing a mass movement, again, requires persuasion, 672 00:37:50,412 --> 00:37:54,071 different audience. And instead of convincing, you know, 673 00:37:54,111 --> 00:37:57,772 government, you're now going to be convincing, 674 00:37:57,812 --> 00:38:00,352 public. Now, even in the public, who do you mean exactly? 675 00:38:00,392 --> 00:38:03,281 So one is there are people who are very technically skilled. 676 00:38:03,312 --> 00:38:05,741 This is, again, you know, the software developers essentially, 677 00:38:05,812 --> 00:38:09,241 Silicon Valley. So I think convince Silicon Valley 678 00:38:09,292 --> 00:38:13,272 I think convince, you know, literally every person in the United 679 00:38:13,292 --> 00:38:17,131 is another approach. Uh, I think multiple approaches need to be 680 00:38:17,152 --> 00:38:20,912 out and we need to see what works. Even on the, you know, 681 00:38:21,111 --> 00:38:25,031 broader geopolitical scale, like, okay, look, you convinced US, 682 00:38:25,111 --> 00:38:26,332 like, "You know, what about China? 683 00:38:26,352 --> 00:38:29,872 What about Israel?" You know, uh, there will be people who will need to 684 00:38:30,131 --> 00:38:34,091 attempt similar things even in China and I don't 685 00:38:34,131 --> 00:38:38,111 know, for political reasons, maybe I'm not super comfortable commenting 686 00:38:38,172 --> 00:38:41,832 that, but at some point, through some m- mix of either 687 00:38:41,852 --> 00:38:45,792 persuasion or- or force, you do need to also get the Chinese 688 00:38:45,832 --> 00:38:49,392 board with the idea that, you know, superintelligence is an extremely 689 00:38:49,430 --> 00:38:53,412 technology and you need to respond, you know, 690 00:38:53,471 --> 00:38:54,672 that it poses. 691 00:39:00,611 --> 00:39:04,511 All right. Well, thanks for sharing your time. 692 00:39:04,591 --> 00:39:05,161 Yeah. 693 00:39:05,912 --> 00:39:09,652 So there anything else, uh, you want to say or think is, um, 694 00:39:10,172 --> 00:39:12,591 good- important for our- our audience to 695 00:39:12,631 --> 00:39:15,152 hear? 696 00:39:15,372 --> 00:39:19,131 Uh, no, not much in particular. Yeah, 697 00:39:19,172 --> 00:39:22,812 just in general, look, if you like the work I'm doing, you know, 698 00:39:22,872 --> 00:39:25,692 like, share, subscribe to spread this to a wider 699 00:39:25,702 --> 00:39:29,551 I'd especially be interested in being connected to people, you know, 700 00:39:29,752 --> 00:39:33,721 space, in the legal space, even just other people who are really full 701 00:39:33,812 --> 00:39:37,582 time committed to this. Uh, there are 702 00:39:38,752 --> 00:39:42,652 ways you can actually, you know, contribute 703 00:39:42,712 --> 00:39:46,652 to this cause that are, uh, that A, won't require a lot of your time 704 00:39:46,692 --> 00:39:50,372 and even will require a lot of your time if you don't have a lot of time or energy 705 00:39:50,491 --> 00:39:54,131 to spare. You know, even just, you know, sharing, uh, my channel to 706 00:39:54,232 --> 00:39:56,832 others helps, even just sharing your own views in 707 00:39:56,852 --> 00:39:59,562 Like, don't be like, "You know, Samuel said this," 708 00:39:59,631 --> 00:40:03,372 actually believe and you put that on social media. Even 709 00:40:03,452 --> 00:40:07,372 If you do have a lot of time and energy to spare, uh, there 710 00:40:07,392 --> 00:40:09,352 are more important things you can do. 711 00:40:09,392 --> 00:40:13,232 You can start a channel of your own, you know, 712 00:40:13,352 --> 00:40:13,631 Uh, 713 00:40:15,031 --> 00:40:18,852 you can obviously donate money. Uh, there are things you can do. 714 00:40:18,912 --> 00:40:22,471 So in general, feel free to reach out to me 715 00:40:22,491 --> 00:40:26,091 you can contribute to the space. You can also obviously reach out to, 716 00:40:26,131 --> 00:40:29,531 Guido here. There are a number of people in the space who 717 00:40:29,591 --> 00:40:33,511 if you just openly come and ask, "Look, you know, I want to contribute. 718 00:40:33,551 --> 00:40:36,241 What can I do?" There are lots of people who will tell you, 719 00:40:36,252 --> 00:40:37,952 you could be doing. 720 00:40:37,971 --> 00:40:41,582 Sure. Thanks so much for coming on and, uh, and 721 00:40:41,652 --> 00:40:43,500 speaking, because this is, you know, this is 722 00:40:45,551 --> 00:40:46,781 probably the most important 723 00:40:47,952 --> 00:40:50,852 issue facing humanity in the near 724 00:40:50,952 --> 00:40:52,752 term. 725 00:40:53,772 --> 00:40:55,571 And thanks for doing your part. 726 00:40:55,652 --> 00:40:58,971 Oh, yeah. You're welcome.