WEBVTT 00:01.980 --> 00:05.720 And I came back and I read LessWrong, and I looked, okay, I looked, two 00:05.800 --> 00:09.440 years have passed and absolutely nothing has changed. 00:09.480 --> 00:12.300 (laughs) Like, you know, the capabilities have increased a lot, 00:12.340 --> 00:15.900 from 2022, 2023, to, you know, 2025, like AI 00:15.980 --> 00:19.800 capabilities have gone further a lot, but, you know, 00:19.820 --> 00:23.700 all these people talking about, you know, safety, risk, governance, like 00:23.780 --> 00:27.600 it's the same people, the same boring speeches, the same, 00:27.680 --> 00:30.730 arguments that happen on LessWrong, like actually nothing has changed. 00:30.740 --> 00:34.710 There's the same funders, you know, uh, Muskowitz and 00:40 John Talion doing 00:34.740 --> 00:37.230 the same sort of work, which I don't really believe in, 00:37.340 --> 00:41.250 uh... Yeah. And I, I am now starting 00:41.300 --> 00:44.720 to realize, look, if I don't personally do something in this 00:45.600 --> 00:49.540 fate of the entire species might be at stake (laughs) , and that is kind 00:49.560 --> 00:52.880 of an almost overwhelming sort of responsibility to be dealing with. 00:52.960 --> 00:56.280 Uh, even by 2035, you're looking at, what, an at least 00:56.520 --> 00:58.720 40%, 50% chance that we will build 00:58.760 --> 01:02.600 superintelligence. So, at that point you realize, look, 01:02.660 --> 01:06.600 if you're making a life plan that does not take into account 01:06.640 --> 01:10.000 the fact that, you know, superintelligence might be created on this 01:10.060 --> 01:13.960 like what are you doing with your life? Like, that's just a bad move. 01:13.980 --> 01:16.450 01:23 Less, like the world is on... 01:16.480 --> 01:20.340 Like, there is a difference between, look, the world is on fire and I'm upset about 01:20.360 --> 01:24.320 this, but I won't, you know, allow this to overwhelm me, 01:24.380 --> 01:27.200 to the public. I will be the calm voice in the face of 01:27.210 --> 01:31.020 And there is another way of processing it, which is, 01:31.080 --> 01:34.360 look, the world is on fire, and look, I don't care about this at all 01:34.380 --> 01:36.760 going to do my thing, because I never cared about this in the 01:36.800 --> 01:40.320 place. 01:44.840 --> 01:48.510 Hi everyone. My name is Guido Reichstagter, and I have the 01:48.580 --> 01:51.980 pleasure of interviewing Samuel Shadrack today. 01:52.060 --> 01:55.160 It's Sunday, September 21st, and 01:56.300 --> 02:00.260 I first met Samuel online when I heard 02:00.300 --> 02:02.280 that he had started a hunger strike 02:03.740 --> 02:07.240 in India on the issue of AI 02:07.300 --> 02:10.919 risk, because there are companies which are right now 02:10.940 --> 02:14.340 racing to build superintelligent AI systems that they 02:14.460 --> 02:17.520 have no reasonable plan for 02:17.560 --> 02:20.360 controlling, and this project puts 02:21.480 --> 02:24.140 all of us at risk. So, 02:25.600 --> 02:28.780 Samuel, I'm glad to be talking with you today. 02:28.900 --> 02:32.280 Oh, yes, nice to be here. Glad to speak with 02:32.340 --> 02:33.300 you. 02:33.440 --> 02:37.350 Awesome. Could you, uh, start out by telling us a little bit about 02:37.360 --> 02:41.060 yourself, your personal history and how you came to be 02:41.120 --> 02:41.910 aware of, uh, 02:42.920 --> 02:45.420 uh, what- what's going on with AI? 02:46.920 --> 02:50.620 Uh, yeah, sure. Uh, so I mean, I'm, like, currently 02:50.920 --> 02:54.540 24 years old. Uh, I've 02:55.000 --> 02:58.960 grown up in India my whole life. Uh, I 02:59.240 --> 03:03.180 graduated from IIT Delhi in 2023, so that's 03:03.240 --> 03:05.450 two years back, with like a BTech... 03:05.460 --> 03:08.260 like a Bachelor's and a Master's in biochemical 03:08.320 --> 03:11.829 engineering. Uh, I've always been generally 03:11.860 --> 03:15.400 interested in tech though, generally interested in, like, 03:15.420 --> 03:19.040 I've been, you know, the... even among, you know, the people 03:19.140 --> 03:22.940 at, you know, such a college, I was still, you know, the outlier of, you know, just 03:22.980 --> 03:26.200 a tech nerd who would just, you know, spend all my time reading online about 03:26.240 --> 03:30.020 various technologies, you know, that could be, uh, space technology, 03:30.160 --> 03:33.820 could be anti-aging, could be solar, could be climate change. 03:33.860 --> 03:37.540 Like, just anything really. You know, I would just, on my own go up and, 03:37.600 --> 03:41.360 read things. So, naturally at some point or the other, 03:41.400 --> 03:44.940 also got recommended some books around AI risk. 03:44.980 --> 03:48.020 I think the first time I got introduced to AI risk, 03:48.240 --> 03:49.820 uh, 03:50.680 --> 03:54.460 was this, uh, recommendation in, I think, 03:54.500 --> 03:57.660 Bill Gates' reading list. Uh, he had a couple of 03:57.740 --> 04:00.460 recommendations, uh, one of them was, I think, Factfulness. 04:00.480 --> 04:03.920 There was another one which was Superintelligence by Nick Bostrom. 04:04.040 --> 04:07.840 Uh, so when I read, you know, that particular book, that 04:08.100 --> 04:11.630 took AI risk seriously. I realized, okay, look, okay... 04:11.660 --> 04:14.720 That was the first time where I started realizing, okay, look, this actually makes 04:14.760 --> 04:18.420 sense and, like, this could actually be a big deal. 04:18.440 --> 04:21.360 Whereas until then, it was just kind of like, you know, 04:21.380 --> 04:25.060 fiction thing in my head. Like... Uh, maybe 04:25.100 --> 04:28.750 another point of background would be like, uh, 04:29.280 --> 04:33.200 so during my schooling, uh, I used to use the internet a fair bit. 04:33.240 --> 04:36.960 I was active, like, on Stack Overflow, Stack Exchange, like those sites were more 04:37.000 --> 04:40.980 popular back then. I think now, like, AI has kind of, like, reduced the 04:41.000 --> 04:44.620 readership of these sites, but back then they were pretty popular, 04:44.780 --> 04:48.580 uh, among academia, among even, like, people who are new to 04:48.600 --> 04:50.860 any, like, scientific field but want to learn more. 04:50.940 --> 04:54.480 So, my username there was like, ghostsinthecode. 04:54.580 --> 04:58.420 And this was based on this movie, I, Robot, 04:58.440 --> 05:00.840 which was fairly popular. It's like a Hollywood movie. 05:00.960 --> 05:04.520 Uh, Will Smith was an actor. And it is 05:04.560 --> 05:06.600 basically about an AI took- takeover. 05:06.660 --> 05:08.840 Like, uh, there is 05:08.920 --> 05:12.790 this, you know, huge tech company that builds these, you 05:12.820 --> 05:16.740 know, armies of robots with AI, and eventually at some point the AIs plan 05:16.780 --> 05:20.620 takeover and, you know, in very cinematic Hollywood fashion, 05:20.660 --> 05:24.140 at the last moment, you know, the hero, you know, Will Smith comes and just 05:24.180 --> 05:27.830 physically shoots down a bunch of robots and, you know, hacks into the center and, 05:27.840 --> 05:31.390 you know, takes it down and stuff. Uh, I think that was the first 05:31.420 --> 05:34.730 part, even just as like a fun curiosity, I thought about it through 05:34.800 --> 05:38.740 like, like just some of the philosophical 05:38.750 --> 05:42.680 them even way back then. Uh, but yeah, back then 05:42.820 --> 05:45.420 I just thought of it as, you know, fiction, thought experiment. 05:45.540 --> 05:49.000 I didn't think of it as, oh, literally, you know, 10 years later we're going to be 05:49.100 --> 05:52.580 having these AIs and these are going to be actual questions and, 05:52.640 --> 05:54.969 of the world is actually going to depend on these types of things. 05:56.100 --> 05:59.060 Uh, so maybe that is maybe the- my first introduction. 05:59.080 --> 06:02.840 My first serious introduction was, yeah, in college, uh, 06:02.900 --> 06:06.768 reading Superintelligence by Nick Bostrom.Then I, you know, read 06:06.808 --> 06:09.468 through the reviews of, uh, superintelligence. 06:09.568 --> 06:09.888 Uh, 06:11.348 --> 06:14.728 there were people saying, "Look, you know, half of this content is copied from, you 06:14.748 --> 06:18.108 know, Eliezer Yudkowsky. Just go read what Eliezer Yudkowsky has 06:18.128 --> 06:22.008 instead." And then I was like, "Hmm, interesting." And then I went and read a 06:22.228 --> 06:26.208 lot of work of, you know, Eliezer Yudkowsky. I read The Sequences. 06:26.228 --> 06:29.808 That gave me a lot of things to think about. 06:29.928 --> 06:30.448 Uh, 06:31.368 --> 06:34.148 I think Eliezer Yudkowsky is, like, a brilliant philosopher. 06:34.168 --> 06:37.198 Like, I disagree with him on a lot of points, 06:37.228 --> 06:40.928 brilliant, like, this surface area of, you know, novel topics he has covered 06:40.968 --> 06:44.748 is large, and he has interesting thoughts 06:44.888 --> 06:48.288 on a lot of them, some of which I agree with and some of 06:48.408 --> 06:48.968 Uh, 06:50.328 --> 06:51.868 so there's that. 06:52.688 --> 06:56.068 So I think that was the point where I was first sure, okay, look, 06:56.128 --> 06:59.468 some chance, you know, our species is going extinct as a result of this 06:59.488 --> 07:03.428 technology. Back then, I, this, when I say 07:03.488 --> 07:07.157 back then, I mean, like, you know, maybe fourth, fourth year of college. 07:07.188 --> 07:09.947 I think back then I assigned like, you know, a 15% 07:10.668 --> 07:13.498 probability we'll be getting, you know, superintelligence soon, like, you know, in 07:13.508 --> 07:17.148 the next five years, 10 years. Now that number has gone up. 07:17.188 --> 07:19.428 Now I'm like, at least 25%, maybe more. 07:19.468 --> 07:22.768 We can talk about, you know, technicalities of these numbers 07:22.788 --> 07:25.748 basic idea is, yeah. Uh, a couple years 07:25.788 --> 07:29.588 ago, reading Eliezer Yudkowsky's work, that was the first time I 07:29.628 --> 07:33.568 started taking this stuff like se- seriously and being like, "Okay, look, 07:33.628 --> 07:37.588 there is some actual non-negligible chance that, you know, the species may 07:37.628 --> 07:41.278 not survive the creation of this technology." 07:41.328 --> 07:44.868 After that it's just me trying various things 07:45.368 --> 07:48.988 around this. I started the Effective Altruism Club 07:49.248 --> 07:52.908 in my college, IIT Delhi. It did 07:52.968 --> 07:56.928 not go super well, to be quite blunt, but I learned useful 07:57.028 --> 08:00.308 things from the experience of trying to start that club over a period of six 08:00.348 --> 08:01.328 months. 08:02.728 --> 08:06.348 What else? I completed this, uh, two-month, like, 08:06.408 --> 08:10.328 full-time, you know, whatever, 50, 60-hour week intensive computer 08:10.388 --> 08:14.068 science course called ML Safety Scholars by 08:14.187 --> 08:18.028 Dan Hendrycks who's at UC- UC Berkeley, and like one of the famous 08:18.068 --> 08:22.048 names in the AI risk space. That did give me some 08:22.128 --> 08:25.988 technical background, and that actually scared me a little bit more 08:26.068 --> 08:29.948 it made me understand to the extent to which, like, AIs are just 08:30.008 --> 08:33.148 black boxes and, like, the people who are training these things have actually no 08:33.208 --> 08:37.158 idea what's going on inside them. Like, I had a surface level of 08:37.188 --> 08:40.368 understanding of this, but then actually, you know, you know like writing down the 08:40.408 --> 08:43.608 equations for a transformer, okay, obviously, okay, attention is like this. 08:43.628 --> 08:46.648 These are the key val- query value matrices. 08:46.688 --> 08:50.288 And look, this is, you know, what a scaling law is, this is how 08:50.868 --> 08:54.368 for the entire field of interpretability re- research looks like. 08:54.388 --> 08:58.328 You know, this is what people when they talk about, you know, 08:58.368 --> 09:01.488 ali- other AIs, you know, at a technical level, this 09:01.508 --> 09:05.218 And just reading a bunch of these things from a technical level made me 09:05.268 --> 09:08.298 understand the extent to which, like, AI researchers have no idea what they're 09:08.308 --> 09:12.148 doing. And, like, some of these fields are so nascent and so much 09:12.188 --> 09:15.278 of what actually works is just like black box approaches, 09:15.328 --> 09:18.268 like try a thing without understanding why it works. 09:18.308 --> 09:21.288 It's just, "Look, you know, the last six months it worked, 09:21.328 --> 09:25.148 again." And that was an intuition which I really got 09:25.168 --> 09:28.948 through that course and just studying this, the topic 09:29.028 --> 09:32.268 of AI and, you know, AI alignment, AI safety at a more technical 09:32.288 --> 09:35.928 level. After that what did I 09:35.988 --> 09:36.968 do? Uh, 09:38.168 --> 09:39.368 after that to be 09:40.188 --> 09:44.088 quite honest I did take like a break of like at least a year from 09:44.148 --> 09:48.048 these things. So, you know, in my fifth year, my last year of college, 09:48.108 --> 09:51.458 did not do much about all this. I was just focused on like graduating. 09:51.468 --> 09:52.828 I finished my thesis, 09:53.648 --> 09:57.588 I, uh, traveled for a few months and at 09:57.628 --> 10:00.128 the end of the travel I was actually low-key depressed. 10:00.148 --> 10:04.088 I was like, "What am I doing with my life?" It gave me 10:04.148 --> 10:07.348 of like, "Look, I actually need to do something useful 10:08.028 --> 10:11.968 this sort of just traveling and having fun and stuff 10:12.008 --> 10:15.308 to work out for me. Like, you know, in moderation it's good, 10:15.328 --> 10:16.848 the focus of my life. 10:18.548 --> 10:22.357 And I came back and I read LessWrong and I looked, okay, I 10:22.368 --> 10:24.568 looked, two years have passed and absolutely nothing has 10:24.588 --> 10:28.548 changed. (laughs) Like, you know, the capabilities have increased 10:28.588 --> 10:32.208 a lot, like, you know, from 2022 to 2023 to, you know, 2025, like 10:32.648 --> 10:36.588 AI capabilities have gone further a lot, but, you 10:36.648 --> 10:40.088 know, all these people talking about, you know, safety, risk, governance, 10:40.128 --> 10:44.048 like it's the same people, the same boring speeches, the same, you 10:44.128 --> 10:47.738 know, arguments that happen on LessWrong. Like actually nothing has changed. 10:47.748 --> 10:51.688 It's the same funders, you know, um, Amos Goertz and Jan Talon doing 10:51.708 --> 10:54.228 the same sort of work which I don't really believe in. 10:54.348 --> 10:54.568 Uh, 10:56.288 --> 11:00.068 yeah, and I- I am now starting to realize, look, if I don't personally do 11:00.108 --> 11:03.968 something in this space, you know, the fate of the entire species might 11:04.068 --> 11:07.968 be at stake. And that is kind of an almost overwhelming sort of 11:08.008 --> 11:11.828 responsibility to be dealing with, uh, 11:11.888 --> 11:14.968 but yeah, I mean, at some level that is the truth. 11:14.988 --> 11:16.988 Like someone needs to try out new approaches. 11:17.028 --> 11:20.118 Someone needs to figure out ways this field can actually grow and like make an 11:20.118 --> 11:24.088 actual impact, the field of AI risk specifically, 11:24.128 --> 11:27.538 be one of the people to help figure that out. Yeah. 11:27.568 --> 11:31.508 Mm-hmm Wow. And so you decide to go 11:31.628 --> 11:34.728 on hunger strike. 11:34.738 --> 11:34.738 Uh, yes. 11:34.788 --> 11:38.468 What was your... How did you- how did you- how did you come to 11:38.508 --> 11:41.468 this decision? What was your thought process around 11:41.508 --> 11:45.228 this? 11:47.068 --> 11:50.648 That's a good question. Uh, 11:52.228 --> 11:56.088 so I can give you a bunch of reasons from like political theory and I can 11:56.128 --> 11:59.148 give you a bunch of reasons which are just, okay, look, I did hit 11:59.168 --> 12:03.008 was like, "Let's see what works." So the hit 12:03.088 --> 12:07.048 reasons I, look, I started a YouTube channel, 12:07.088 --> 12:10.548 videos about AI risk, I gave a bunch of, you know, talks at various 12:10.568 --> 12:14.028 places, I-... wrote this, you 12:14.088 --> 12:17.748 know, database for whistleblowers, which is kind of, you know, 12:17.828 --> 12:21.048 politically sensitive, so maybe we don't have to discuss it. 12:21.068 --> 12:24.048 But that is another thing, you know, how do you support a whistleblower at an 12:24.128 --> 12:28.098 company? I was doing that. And at some point, yeah, this 12:28.168 --> 12:32.148 idea came to me, like, "Look here, hunger strike actually makes sense." Uh, 12:33.048 --> 12:35.088 so it's just, you know, try a bunch of things. 12:35.128 --> 12:38.368 You know, so far what I was trying was not giving me very concrete, you know, 12:38.408 --> 12:41.788 success. I couldn't look at any of the things I'd 12:41.808 --> 12:45.708 this is a big success and it actually made a difference." So I 12:45.748 --> 12:47.828 was like, "Okay, look, let's just try one more thing 12:49.028 --> 12:52.828 that may have a chance of actually working." Uh, the reason from political 12:52.928 --> 12:56.588 theory is basically, uh, making 12:56.648 --> 13:00.438 costly public signals. Like if you signal to the public that, "Look, 13:00.468 --> 13:03.588 willing to make significant sacrifices for this," like, 13:04.248 --> 13:07.968 that speaks to the public more than just a few, you know, you go write a blog post 13:08.028 --> 13:11.978 or you make a video. Uh, like words are cheap, you know, actions are 13:12.028 --> 13:16.008 costly. Uh, they also have certain thoughts around, 13:16.028 --> 13:19.588 you know, common knowledge, like, uh... 13:19.688 --> 13:23.228 So when... So, like, in general and... 13:23.248 --> 13:27.188 Okay, I'll make a disclaimer here that, you know, I don't want to try and, you 13:27.228 --> 13:31.208 know, topple over any government when I say this, but in 13:31.248 --> 13:35.188 the past, when people have tried to topple over 13:35.248 --> 13:37.798 they have noticed is, uh... 13:39.888 --> 13:41.708 A key factor here is common knowledge. 13:41.728 --> 13:45.508 Like it's not enough for a lot of people, like if 10% of the population are 13:45.568 --> 13:49.048 like, you know, "We are unhappy with this government 13:49.068 --> 13:52.448 to be in charge," that is not enough to actually topple the 13:52.488 --> 13:56.308 It's like, I know that I am unhappy, and I know the other person is 13:56.367 --> 14:00.288 unhappy, and the other person knows that I know that they are unhappy, 14:00.308 --> 14:03.958 that sort of common knowledge. And only when you do that among, you know, 14:03.968 --> 14:06.688 significant fraction of the population, that's when you can actually topple the 14:06.708 --> 14:10.488 government. And ways to establish this common 14:10.528 --> 14:14.448 very public things. So this could be starting with, yes, uh, 14:14.548 --> 14:17.758 let's say, a YouTube channel, you know, a charismatic speaker has, you know, 14:17.788 --> 14:21.488 millions of likes. That establishes common knowledge because 14:21.497 --> 14:23.048 "Oh, it's not just me liking this video. 14:23.068 --> 14:26.668 There's also, you know, 10 million other people who 14:26.728 --> 14:30.668 This could be in public demonstrations, public protests if, you know, if there's 14:30.888 --> 14:34.028 like, you know, there's a city and, you know, like, 14:34.068 --> 14:36.888 physically on the street and they all look at each other 14:36.968 --> 14:39.208 Oh, okay, so I'm not alone in believing this. 14:39.248 --> 14:42.268 We all believe the same thing." That establishes common 14:42.328 --> 14:46.248 knowledge. The final stage is, you know, the point at which, 14:46.288 --> 14:50.068 you know, these mass movements can, you know, actually occupy, you know, 14:50.148 --> 14:54.108 buildings. Uh... 14:54.188 --> 14:58.108 Yeah, and that again establishes common 14:58.688 --> 15:00.867 buildings are sym- like, symbols. 15:00.908 --> 15:01.398 They're, you know, 15:02.568 --> 15:05.068 like... yeah, they're just symbolically important. 15:05.148 --> 15:08.928 And when everyone sees, okay, like the entire country will see, says, 15:08.988 --> 15:12.248 look, this movement has actually gone and occupied this building 15:12.308 --> 15:15.388 to, you know, actually stop them," then they realize, 15:15.408 --> 15:18.288 are going to form the new government. That's just it. 15:18.408 --> 15:18.848 So 15:20.148 --> 15:23.528 I think, yeah, hunger strikes are one of those costly signals. 15:23.608 --> 15:24.108 And 15:24.948 --> 15:28.208 if a lot of people can come in, you know, even just like, share, subscribe on 15:28.228 --> 15:31.388 hunger strike, and if hunger strike gets, you know, a million likes, that can even 15:31.428 --> 15:35.128 establish the common knowledge required to actually bring about change. 15:36.308 --> 15:38.288 Yeah, I was really surprised that, 15:38.408 --> 15:42.348 um, there was the kind 15:42.388 --> 15:46.218 of reception or the kind of impact that even this 15:46.228 --> 15:49.928 kind of limited action made. I think the, 15:49.948 --> 15:53.828 the, the post, uh, that Michael Trauzzi made announcing 15:53.888 --> 15:57.768 his strike was viewed over- maybe over one and a half million 15:57.828 --> 16:01.708 times. And the one that I made was viewed also 16:01.728 --> 16:05.648 over a million times, and it's generated a number of 16:05.728 --> 16:08.808 articles in, you know, press around the 16:08.848 --> 16:11.268 world. So, 16:12.908 --> 16:16.528 definitely was surprising to me the impact of even this kind of 16:17.208 --> 16:18.188 limited action. 16:19.008 --> 16:20.788 And I think, you know, if this is what 16:21.988 --> 16:24.068 five people, 16:25.188 --> 16:25.828 four people 16:27.028 --> 16:29.988 going, making this kind of costly 16:30.067 --> 16:33.998 action can do, imagine what 100 people could 16:34.048 --> 16:37.328 do, or 1,000 people. 16:37.387 --> 16:41.308 So I think this is really a crucial part of building 16:41.318 --> 16:43.468 this common public 16:43.548 --> 16:47.058 awareness. People have to see people 16:47.928 --> 16:51.028 that are acting consistently with what it means to be in an 16:51.048 --> 16:52.588 emergency. 16:53.568 --> 16:56.387 And, uh, I believe we are in an emergency, and, um, 16:57.808 --> 17:00.168 I'd be interested in hear- hearing from you. 17:00.188 --> 17:03.328 What, what is it about this time that we're in that makes it an 17:03.348 --> 17:05.648 emergency? 17:07.728 --> 17:11.688 Oh, uh, that's a good question. Uh, (laughs) 17:11.708 --> 17:15.068 I think when used the word emergency, like there's a rational aspect to it 17:15.108 --> 17:18.988 emotional aspect to it. And, you know, I'm the type of person who tries to 17:19.028 --> 17:22.938 the two. Uh, I still have complicated thoughts on, 17:22.988 --> 17:26.677 well... Like, should these two be separate? 17:26.768 --> 17:30.458 Uh, so let me just first talk about the 17:30.568 --> 17:32.828 Uh, like rationally, you know, what do I believe? 17:32.848 --> 17:34.268 What is the state of the world right now? 17:34.308 --> 17:36.648 What could the state of the world be in the next few years? 17:36.788 --> 17:40.958 Uh... 17:41.608 --> 17:45.468 I think assuming, uh, there is no such social 17:45.548 --> 17:49.408 movement, no one actually tries to change the course 17:49.468 --> 17:52.498 labs are trying to take the course of history, uh, 17:53.548 --> 17:57.248 even in the next five years, there's, you know, 17:57.288 --> 17:58.808 going to build superintelligence. 17:58.888 --> 18:02.088 Uh, when I say superintelligence, I mean an AI that is better than all 18:02.128 --> 18:04.888 all tasks. And this is just the next five years. 18:04.908 --> 18:07.208 You know, just because they don't do it in the next 18:07.248 --> 18:09.858 won't do it afterwards. If you take, you know, another five years, so that's, 18:09.888 --> 18:13.868 know, 2035, you can easily put another, you know, 15 to 20% ta- 18:13.928 --> 18:16.308 like... Again these numbers are not very accurate. 18:16.328 --> 18:18.068 There's a lot of high uncertainty on these things. 18:18.128 --> 18:22.068 So-Even by 2035, you're looking at, what, an at 18:22.108 --> 18:25.688 least 40%, 50% chance that we will build superintelligence. 18:26.688 --> 18:30.148 So at that point you realize, look, if you're making a life plan 18:30.808 --> 18:34.388 that does not take into account the fact that, you know, superintelligence might 18:34.428 --> 18:38.168 be created on this earth, like, what are you doing with your life? 18:38.188 --> 18:41.948 Like, that's just a bad move. Uh, if superintelligence is 18:41.968 --> 18:43.968 created, what are the outcomes? Uh, 18:44.848 --> 18:46.908 I think multiple outcomes are possible. 18:46.928 --> 18:49.348 This is the thing I have more uncertainty over. 18:49.368 --> 18:52.728 Like, I have less uncertainty on like, okay, 18:52.768 --> 18:56.608 build soon. I have more uncertainty on, okay, if we build superintelligence, 18:56.648 --> 19:00.588 what happens next? I think there's a good chance 19:00.688 --> 19:02.888 just has its own thing it cares about. 19:02.928 --> 19:06.868 It can, you know, make plans over long time horizons 19:06.908 --> 19:10.668 the thing it wants to do, which probably has, like, 19:10.728 --> 19:14.148 with humans or what humans want and whatever. 19:14.228 --> 19:16.768 And it'll just take over the earth. 19:16.868 --> 19:17.228 Uh, 19:18.408 --> 19:22.328 it will basically ignore us, and as a side effect, like, we 19:22.448 --> 19:26.328 might just get wiped out because, 19:26.448 --> 19:29.428 uh, it will have things it wants to do. 19:29.448 --> 19:31.388 You know, there are resources on the earth. 19:31.448 --> 19:34.258 There are specific, you know, temperature and pressure conditions on the 19:34.288 --> 19:35.768 which are required for human survival. 19:35.808 --> 19:38.588 There are specific, you know, food resources, water resources which 19:38.608 --> 19:42.558 for human survival. If the AI decides that even one of these resources, 19:42.608 --> 19:46.388 it wants, it will just take those resources 19:46.428 --> 19:50.088 uninterested in what, you know, humans want or need to live. 19:50.848 --> 19:54.608 Uh, that is a possibility. This is the other separate possibility 19:54.648 --> 19:58.398 in, is even if you manage to, with certain techniques, keep the 19:58.448 --> 20:02.008 superintelligence under control, like, you know, you apply boxing techniques, 20:02.108 --> 20:05.948 uh, for all the, you know, alignment safety nerds out there, 20:06.208 --> 20:09.868 what AI boxing is. But it's... Yeah, literally keep the AI in a 20:09.928 --> 20:12.888 box so that even though it's smarter than you, it just, um, harder to then break 20:12.908 --> 20:16.668 out. Which is, to be clear, an extremely risky thing to be 20:16.708 --> 20:20.608 trying to do. But even if you actually managed to pull this off, 20:20.628 --> 20:23.828 I think the outcome with that sort of thing 20:23.868 --> 20:25.468 take over the world. Uh, 20:26.568 --> 20:30.228 you can, at this point, just, uh, topple over various governments. 20:30.268 --> 20:33.088 You know, this is a threat to, you know, the US government, the Chinese government, 20:33.108 --> 20:34.388 the Indian, Russian governments. 20:34.488 --> 20:37.998 If a small group of people have an actual superintelligence that's in their 20:38.028 --> 20:41.948 control, and they can do novel things with this, 20:42.028 --> 20:45.818 just build parallel industries. They can build, you know... 20:45.828 --> 20:46.018 (laughs) 20:46.948 --> 20:50.568 Uh, I mean, Yudkowsky famously talks about 20:50.968 --> 20:54.708 that is also possible. We can even just look at, you know, 20:54.788 --> 20:58.648 life, like, we are already not that far from, away from 20:58.808 --> 21:02.328 being a- able to, you know, genetically engineer life forms so you can 21:02.348 --> 21:04.868 populate the earth with, you know, engineered life forms. 21:04.988 --> 21:05.288 Uh, 21:06.508 --> 21:09.988 you can invent new energy sources with a 21:10.048 --> 21:14.037 sufficiently intelligent mind. Uh, like, we are already doing these 21:14.088 --> 21:15.488 things. Like, forget superintelligence. 21:15.508 --> 21:17.928 Like, humans might be able to do these things in 21:17.948 --> 21:19.968 Like, this is not science fiction in my head. 21:20.037 --> 21:23.688 Like, if humans make, you know, solar energy really cheap or they crack, 21:23.768 --> 21:27.648 fusion energy or something, like, these things seem totally possible to me, 21:27.688 --> 21:30.108 and the superintelligence is just going to accelerate these things. 21:30.128 --> 21:33.788 So if a small group of people can just build things like this, they can 21:33.868 --> 21:37.468 build, you know, their own power grid, they can be extremely, 21:37.488 --> 21:41.368 extremely persuasive. Like, just imagine someone like Hitler 21:41.428 --> 21:44.848 way more persuasive, and you have, like, direct access to millions of people. 21:44.888 --> 21:48.588 You can disperse very people at scale, you can get really good at cyberattacks. 21:48.648 --> 21:52.028 You know, AI is already, as of 2025, pretty good at 21:52.048 --> 21:56.008 cyberattacks. You know, Mark Zuckerberg has openly on the record 21:56.028 --> 21:56.388 you know, 21:57.328 --> 22:00.948 like, AI is already a threat to the 22:01.068 --> 22:05.058 Uh, so just a fi- you know, five years more of progress on that, 22:05.088 --> 22:07.968 just basically hack into any system, any government database and so 22:08.108 --> 22:11.908 on. Like, there's so many different capability just 22:11.988 --> 22:15.908 open up all at the same time. Yeah, it seems quite plausible to me a small 22:15.948 --> 22:19.248 of people will just take over the world and establish their own, you know, like, 22:19.288 --> 22:22.628 their permanent dictatorship of sorts and just do what they want. 22:22.648 --> 22:26.598 That seems possible to me. There's also a possibility that 22:26.628 --> 22:29.968 there's just some unknown unknown sort of thing happens which, you know, I can't 22:29.988 --> 22:32.298 predict. Like, these things are just really hard to predict. 22:32.348 --> 22:34.728 There are just so many intersecting things going on. 22:34.748 --> 22:38.178 There's AI capabilities going on, there's, you know, specific 22:38.228 --> 22:42.208 questions like, okay, if I had cyberattacking capability, if 22:42.268 --> 22:46.228 I had a fully autonomous military, if I had XYZ technology, 22:46.268 --> 22:48.508 like, how much power can I project on the world? 22:48.518 --> 22:51.128 Like, some of these questions are just honestly open questions. 22:51.148 --> 22:54.138 Like, me sitting in my bedroom is not... 22:54.148 --> 22:57.348 I'm not necessarily going to be able to, you know, forecast all of these questions 22:57.408 --> 23:00.388 correctly. So we might just end up with some unknown 23:00.428 --> 23:03.588 unknown, uh, possibility which, 23:03.668 --> 23:06.748 again, m- doesn't necessarily need to be good. 23:06.768 --> 23:08.248 You know, it could be good, could be bad. 23:08.318 --> 23:08.568 So 23:09.608 --> 23:12.928 it's just, yeah, it's just going to dramatically change the 23:12.968 --> 23:16.888 And just from a rational point of view, I think that constitutes an emergency when 23:16.928 --> 23:19.828 you look at these sort of extremely dramatic 23:19.888 --> 23:23.328 outcomes. Now, coming to an emotional point of view, 23:23.568 --> 23:27.108 uh, 23:29.428 --> 23:31.648 yeah, I don't know. I think I'm still working my way through 23:31.688 --> 23:35.468 So there's one set of philosophies which are kind of popular in 23:35.508 --> 23:39.188 Silicon Valley, and I have kind of a love-hate 23:39.208 --> 23:43.108 ideas which are popular in Silicon Valley, which is, yeah, basically a 23:43.168 --> 23:43.928 lot of, like, 23:44.788 --> 23:48.668 stoicism and, like, you know, if 23:48.708 --> 23:51.988 there is a crisis and you are the leader and you are calm, you know, 23:52.028 --> 23:55.888 good decisions. Like, there's a lot of emphasis on this sort of 23:55.988 --> 23:59.548 even if the world is on fire, you need to be calm, 23:59.588 --> 24:03.528 the good decisions here. Like, this is the thing that's popular in 24:03.548 --> 24:07.488 Silicon Valley. There's- they- there's a general emphasis on, like, 24:07.528 --> 24:10.708 this is what great people do, and 24:12.768 --> 24:15.748 to part... for part of me, that makes sense. 24:15.788 --> 24:19.688 Like... but, uh, there is 24:19.708 --> 24:23.488 also another part of me that is like, okay, if you take that ideology to 24:23.528 --> 24:27.468 its logical extreme, you really do end up with a person who 24:27.548 --> 24:29.568 not care about these things in the first place. 24:29.608 --> 24:33.472 Like, the world is on-Like, there's a difference between, "Look, 24:33.512 --> 24:37.472 is on fire and I'm upset about this, but I won't, you know, allow this 24:37.532 --> 24:39.992 to overwhelm me," or, "I won't show this to the public. 24:40.052 --> 24:43.831 I will be the calm voice in the face of the storm." And there is another 24:44.532 --> 24:47.312 way of processing it, which is, "Look, the world is on fire. 24:47.331 --> 24:50.492 And look, I don't care about this at all 24:50.552 --> 24:53.892 I never cared about this in the first place." And like, I think there's a 24:54.831 --> 24:58.292 dividing line between those. So, you know, 24:58.372 --> 25:02.331 emotionally, am I looking at it like I'm in an 25:02.372 --> 25:06.132 level? Yes. But at some level, I'm also kind of leading a 25:06.172 --> 25:10.072 relatively normal life. Like, okay, I have devoted my career, you know, full 25:10.132 --> 25:12.972 time to working on this, but apart from my career, what am I doing? 25:12.992 --> 25:16.562 You know, I have a relatively normal soc- a normal social life. 25:16.572 --> 25:20.392 You know, I have friends, you know, I have fun activities I do with them, 25:20.432 --> 25:21.872 I travel places. Like, 25:23.272 --> 25:26.372 if you look at, you know, other aspects of my life, 25:26.412 --> 25:30.232 and be like, "Oh, Samuel is, you know, in an emergency or a crisis right now." 25:32.492 --> 25:33.072 so 25:35.652 --> 25:38.452 yeah, all in all, I don't know. I think that just gives you a lot of 25:38.472 --> 25:41.612 on, you know, to what extent am I looking at this as an 25:41.652 --> 25:44.572 Yeah. 25:46.912 --> 25:49.732 Yeah. For, for me there's a couple of aspects to it. 25:49.812 --> 25:53.302 One is definitely that there is 25:54.831 --> 25:55.851 an immense risk 25:56.932 --> 26:00.622 in front of us. And 26:01.952 --> 26:05.802 it threatens the lives of everyone I care about, and really everyone 26:05.851 --> 26:09.792 on this planet. And like you said, I look at 26:09.831 --> 26:12.952 the trajectory. If there isn't a movement or if there 26:12.972 --> 26:15.432 effective, um, political 26:17.152 --> 26:17.652 project 26:18.992 --> 26:22.572 to prevent this risk, um, 26:22.591 --> 26:25.572 we're, we're gonna suffer those consequences, 26:25.632 --> 26:28.972 are. And because there's not 26:29.032 --> 26:32.292 currently any signs that we are on track 26:33.192 --> 26:36.952 to effectively manage it, um, like 26:37.452 --> 26:41.181 to me that's an emergency. It's kind of like, you 26:41.211 --> 26:44.972 know, we're in a car headed downhill, gaining 26:45.092 --> 26:47.932 speed and there's no brakes on the car. 26:47.952 --> 26:51.572 It's like, that's an emergency. 26:52.392 --> 26:52.592 So 26:53.912 --> 26:57.792 yeah, it is, it is hard sometimes to separate the 26:57.851 --> 26:59.831 rational aspect. 26:59.952 --> 27:08.341 Um... 27:08.432 --> 27:12.402 I think for me it, it boils down to rationally understanding 27:12.412 --> 27:14.402 we're in danger, and 27:15.492 --> 27:18.792 emotionally, the most important thing that I feel is 27:18.912 --> 27:22.831 not turning away or trying to, 27:22.952 --> 27:26.132 um, deny that fact, and 27:26.212 --> 27:29.952 then emotionally face it, face what it means, and then 27:30.072 --> 27:33.712 act rationally, appropriately to try and stop 27:33.752 --> 27:36.252 it. So, 27:37.032 --> 27:37.812 yeah. 27:38.831 --> 27:42.152 H- how are you? We should have, we should have got this, uh, done in the, 27:42.192 --> 27:45.242 beginning, but you've been on hunger strike now, 27:45.272 --> 27:49.212 fasting. How did you... Do you prefer to call it a fast or a 27:49.232 --> 27:52.501 hunger strike? And you've been, but you've been on it for eight days 27:52.572 --> 27:55.851 now, so I'm curious how your experience has 27:56.512 --> 27:59.152 personally. 27:59.212 --> 28:02.812 Yeah. Uh, I think I was initially a bit hesitant to call it a 28:02.892 --> 28:06.032 because, yeah, there were a lot of, you know, random Twitter trolls who 28:06.072 --> 28:08.061 "Look, you're not actually putting your life in 28:08.112 --> 28:12.012 Is that a hunger strike or not?" But at this point I'm like, "Look, I don't 28:12.092 --> 28:14.882 care about that." I do still consider this a hunger strike. 28:14.892 --> 28:18.632 Like, I am actually making a costly sacrifice 28:18.692 --> 28:20.672 actually worse as a result. And 28:21.851 --> 28:25.252 like, if I was just doing what's good for me, 28:25.272 --> 28:29.232 I'm doing this because I believe there's a bigger, you know, cause that needs to 28:29.272 --> 28:29.612 be, 28:30.492 --> 28:32.371 uh, advanced here. Uh, 28:33.312 --> 28:35.592 so yeah, I would call it a hunger strike. 28:35.692 --> 28:39.312 Uh, how am I doing? Ah, that's a good 28:39.432 --> 28:43.322 question. I think it's pretty much manageable, 28:43.371 --> 28:46.662 of, you know, today, which is day number eight of the hunger strike. 28:46.692 --> 28:49.652 I've publicly committed that I'll be doing this for at least 18 28:49.712 --> 28:51.092 days. 28:51.191 --> 28:52.112 Oh wow. 28:52.251 --> 28:54.581 Yes. Yeah. Uh... 28:57.752 --> 29:01.472 I mean, it is not great, but it is manageable. 29:01.512 --> 29:05.272 Like, I am definitely extremely, like, physically weak, like just 29:05.732 --> 29:09.472 less energy in my body. Like when I walk around, 29:09.512 --> 29:13.092 look, don't bump yourself into things, don't get yourself injured, you know, 29:13.152 --> 29:17.052 fall over. Uh, I have, I 29:17.092 --> 29:20.632 randomly have nausea at points. I have randomly, you know, 29:20.652 --> 29:22.492 points. Uh, 29:23.652 --> 29:26.262 like by itself, you know, none of these things is a big deal. 29:26.272 --> 29:27.952 All of this is very much manageable. 29:27.992 --> 29:28.592 It's just 29:30.192 --> 29:34.092 or, you know, I drink salt water, it tastes, you know, absolutely horrible. 29:34.192 --> 29:37.612 Uh, all these things by itself, they're small, but when they 29:37.652 --> 29:40.952 accumulate, you know, day after day after day just dealing with 29:41.892 --> 29:45.292 shit, it does kind of take a toll on you 29:45.331 --> 29:47.672 mentally. Also, I'm just kind of bored. 29:47.812 --> 29:51.632 I- I'm not doing anything productive and my mind is not clear 29:51.672 --> 29:53.892 enough to do any, like very focused work. 29:53.972 --> 29:54.132 So 29:55.892 --> 29:59.652 I end up spending a lot of time doing absolutely nothing, and then 29:59.712 --> 30:01.492 that feels not great. So 30:04.152 --> 30:06.312 yeah, it's just like a m- like a 30:07.132 --> 30:11.052 subpar quality of life. When you extend that, you know, over many, 30:11.132 --> 30:13.792 it does start taking a bit of a toll on you. 30:13.812 --> 30:16.992 But all in all, I think it's pretty much manageable. 30:17.032 --> 30:21.022 You know, I am fine doing this for, doing this for, you know, another week, 30:21.052 --> 30:24.081 another two weeks if it comes to it. Yeah. 30:24.112 --> 30:27.672 So to be clear, you're just taking, um, basically 30:27.812 --> 30:31.772 salts for electrolytes, but otherwise nothing, no 30:31.851 --> 30:34.672 other... no calories, no vitamins, nothing like that? 30:35.660 --> 30:39.150 Uh, no. Yeah, I'm just drinking water and I'm 30:39.200 --> 30:42.460 having, you know, table salt, you know, sodium 30:42.600 --> 30:46.470 chloride, uh, mag- uh, potassium chloride, which is again, 30:46.480 --> 30:50.340 another salt. I'm having, you know, magnesium capsules and that's about 30:50.380 --> 30:50.600 it. 30:50.610 --> 30:50.610 Yeah. 30:50.680 --> 30:53.900 I'm not having any calories, no sugar, no 30:53.940 --> 30:55.640 vitamins, so on. 30:55.650 --> 30:55.650 Mm. 30:55.700 --> 30:56.060 Yeah. 30:56.480 --> 30:59.380 Mm. Yeah, I- I wonder if that's a big difference in our 31:00.140 --> 31:03.920 experiences 'cause I am taking, um, a multivitamin 31:04.120 --> 31:08.020 and other vitamin supplements, but no c- no calories, and also 31:08.080 --> 31:12.000 electrolytes. And like, subjectively, 31:12.060 --> 31:15.960 of course, I feel tired and I have the same nausea 31:16.080 --> 31:19.600 and, um, you know, some stomach pain. 31:19.640 --> 31:23.600 But, um, mentally, I feel like very 31:23.640 --> 31:27.500 there, like very, um, you know, 31:27.560 --> 31:31.360 aware and in full possession of my faculties and everything, 31:32.600 --> 31:36.460 so. Or maybe I'm just (laughs) maybe I'm just 31:36.540 --> 31:40.360 (laughs) being there doesn't mean all that much (laughs) as it may- as it 31:40.420 --> 31:40.940 may to you. 31:42.120 --> 31:42.560 But, um, 31:43.520 --> 31:44.730 yeah, that's amazing. I didn't... 31:44.780 --> 31:48.560 I- I- I missed that part where you had- had committed to- to going for 18 31:48.620 --> 31:51.960 days, so that's... Yeah, that's impressive. 31:52.060 --> 31:55.860 Um, I'm curious, how is your... How are your family and friends, 31:55.960 --> 31:58.630 uh, responding? Do they know about this or what do they think? 31:58.660 --> 32:00.160 Have you talked to them about it? 32:01.320 --> 32:05.060 Uh, yeah, the response are mixed. I've definitely got just 32:05.120 --> 32:08.500 support, solidarity from some of my friends who 32:08.560 --> 32:11.580 thing you're doing and I wish you all the best," and so 32:11.700 --> 32:15.600 on. There's certain people who don't really 32:15.620 --> 32:18.900 they don't really buy the case for AI risk to the extent that I do. 32:18.920 --> 32:22.460 But despite that, they're like, "Look, you know, you believe in this, you want to 32:22.500 --> 32:24.240 do this, you know, good for you. All the best. 32:24.280 --> 32:25.940 I hope you, you know, I hope you stay safe. 32:25.980 --> 32:29.760 I hope your health stays fine, you know, don't get hospitalized and all that 32:29.840 --> 32:33.400 stuff." Uh, I have gotten a couple of 32:33.440 --> 32:36.650 responses I'm not happy about where I did realize, okay, look, 32:37.480 --> 32:41.180 these people, you know, out of a misguided sense of, you know, 32:41.220 --> 32:44.199 me might actually try to, you know, stop me from continuing on 32:44.240 --> 32:48.000 this. And to them, I have made very clear, like, "Look, this is 32:48.060 --> 32:50.580 important to me, you know, don't get in my way. 32:50.620 --> 32:54.420 Like, I know you have, you know, sort of certain, you know, care 32:54.460 --> 32:58.180 me, but if you actually truly cared about me, 32:58.220 --> 33:01.960 this is to me. So, you know, don't- don't get in my way." 33:02.000 --> 33:02.900 Yes, that 33:04.000 --> 33:04.160 I 33:05.900 --> 33:08.980 have gotten again, you know, even more the responses, they're like, 33:09.020 --> 33:11.960 they're not super happy with me doing this, but 33:12.960 --> 33:16.440 also end of the day, look, the choice is not in their hands, the choice is in my 33:16.450 --> 33:20.420 hands and this is what I'm doing. So yeah, the responses have been pretty mixed all 33:20.460 --> 33:23.350 around. I think I'm generally happy with the 33:27.040 --> 33:29.360 And, uh, I guess just as a final question, 33:30.160 --> 33:33.970 I'm wondering how do you see yourself, uh, 33:34.100 --> 33:37.460 going forward in the future? What's, um, 33:38.139 --> 33:39.420 you know, what's on your mind about 33:40.600 --> 33:42.420 what part you can play in this whole 33:43.440 --> 33:45.540 situation that we're in? 33:46.800 --> 33:50.720 Uh, that's a good question. Yeah, I mean 33:50.800 --> 33:54.160 if you ask me concretely, do I have like, a role I want to play and just 33:54.200 --> 33:55.980 consistently play that for five years? 33:56.000 --> 33:59.520 Like, I don't have conviction on what that role is. 33:59.580 --> 34:02.920 I'm still in a phase of, you know, hit and trial, try a bunch of things and see 34:02.960 --> 34:04.920 what works, uh- 34:04.960 --> 34:07.800 What's the goal that you're working towards? 34:07.900 --> 34:11.560 I mean, the high level goal is definitely, you know, get a pause 34:11.739 --> 34:15.440 on further research development of AI 34:15.540 --> 34:19.239 worldwide across all countries. This will require 34:19.280 --> 34:23.150 necessarily a treaty between the government of the United States and the 34:23.199 --> 34:27.080 government of China. This is going to require a certain 34:27.120 --> 34:29.280 number of en- enforcement mechanisms. 34:29.290 --> 34:32.500 This is going to require a lot of people actually convinced that their lives are in 34:32.560 --> 34:36.199 danger. Like, you cannot pull off these sort of large 34:36.260 --> 34:37.670 actually convinced a lot of people. 34:37.670 --> 34:41.520 Like, that is the high level goal. Uh, I could also 34:41.580 --> 34:44.900 go into like, what are the more smaller things 34:44.920 --> 34:48.239 that I'm working on, if you'd like to kn- know about those. 34:48.260 --> 34:51.920 Well, what's your... Yeah, what's your- what's your kind of, um, big 34:52.040 --> 34:55.940 picture of the path? How do- how- how generally 34:56.279 --> 34:59.340 do you think we... What do we need to do to get from here to 34:59.420 --> 35:01.960 there? 35:03.760 --> 35:07.580 Uh, good question. So I think in general, if you 35:07.620 --> 35:10.500 want to get a person to do something, there are two ways to do it. 35:10.520 --> 35:13.760 You know, A, you can convince them let- in their heart that, "Look, this is 35:13.820 --> 35:17.180 actually no- important." Like, you actually change their values 35:17.240 --> 35:20.080 And B, as you apply incentives on them, you apply pressure. 35:20.120 --> 35:23.060 Now, pressure could be actual physical 35:23.120 --> 35:26.160 It could be, you know, monetary incentives. 35:26.180 --> 35:29.540 You tell them, "Look, if you do this, you'll be respected," et cetera. 35:29.720 --> 35:33.600 I think this... Solving this problem is going to require a mix 35:33.640 --> 35:37.060 of both. Like, you can't use just force 35:37.100 --> 35:40.060 persuasion. There are going to be people doing a bi- bit of 35:40.120 --> 35:41.920 both. Uh, 35:43.480 --> 35:43.650 but- 35:43.680 --> 35:47.480 You mentioned earlier about, um, a mass movement. 35:48.690 --> 35:48.690 Uh- 35:48.720 --> 35:52.120 What do you think the... What do you see the role of a- 35:52.160 --> 35:56.070 process of moving society from where it is now 35:56.100 --> 35:59.640 to where it needs to be to actually implement that kind 35:59.720 --> 36:03.480 of a global treaty? Or even starting with the 36:03.520 --> 36:05.230 United States, right? 36:05.230 --> 36:05.230 Mm-hmm. 36:05.240 --> 36:09.060 Because the United States right now is nowhere in the ballpark of politically 36:09.760 --> 36:10.020 of 36:11.800 --> 36:15.640 seriously considering that even as an option. 36:15.700 --> 36:17.790 Yeah, that is fair. Okay, so the... 36:17.800 --> 36:21.120 I think historically, the way that, you know, effective altruist movement has 36:21.160 --> 36:25.060 approached this is a very softball kind of approach, which 36:25.140 --> 36:29.060 is, again, I'm not saying like, it is bad 36:29.120 --> 36:31.800 to try this. I'm just saying, you know, maybe they're failing. 36:31.860 --> 36:35.680 What they tried is essentially, "Look, we will get people in all the 36:35.740 --> 36:39.531 elite circles of government, you know, we will..."Uh, establish think 36:39.631 --> 36:42.491 tanks. We'll establish policy. Policy institutions. 36:42.531 --> 36:45.591 We'll get just people in government and, you know, just convince people in the US 36:45.611 --> 36:48.732 government this is important and just make a- action work that way. 36:48.752 --> 36:50.832 So they just went through the persuasion approach. 36:51.631 --> 36:54.131 And I'm not optimistic on this working out. 36:54.172 --> 36:58.131 I think the geopolitical power you get from developing AI is too 36:58.172 --> 37:01.971 large for, uh, people at the high levels of 37:02.031 --> 37:05.332 US government to be just listening toward this effective and trust people have to 37:05.392 --> 37:09.192 say. Uh, so just 100% persuasion is not g- 37:09.232 --> 37:12.682 really going to work in my view, uh, which comes to, you know, 37:12.692 --> 37:16.392 establishing a mass movement and actually applying pressure on the US 37:16.412 --> 37:18.372 Like, you can't do just persuasion. 37:18.392 --> 37:21.892 You need to actually pressure that, look, you know, if you don't actually do the 37:21.912 --> 37:25.761 things that we want you to do, like if we, the public, say we want this and the 37:25.812 --> 37:28.852 government is, like, not willing to do it, you need to actually apply 37:29.652 --> 37:33.352 pressure and be like, "Look, we are willing to vote new leaders 37:33.372 --> 37:36.392 We're willing to, you know, bi- boycott your products if required. 37:36.412 --> 37:39.611 You know, we are willing to protest on the street 37:39.712 --> 37:43.611 Uh, we are going to disrupt your way of doing 37:43.712 --> 37:46.531 think a mass movement is a good idea. 37:46.591 --> 37:50.392 Establishing a mass movement, again, requires persuasion, 37:50.412 --> 37:54.071 different audience. And instead of convincing, you know, 37:54.111 --> 37:57.772 government, you're now going to be convincing, 37:57.812 --> 38:00.352 public. Now, even in the public, who do you mean exactly? 38:00.392 --> 38:03.281 So one is there are people who are very technically skilled. 38:03.312 --> 38:05.741 This is, again, you know, the software developers essentially, 38:05.812 --> 38:09.241 Silicon Valley. So I think convince Silicon Valley 38:09.292 --> 38:13.272 I think convince, you know, literally every person in the United 38:13.292 --> 38:17.131 is another approach. Uh, I think multiple approaches need to be 38:17.152 --> 38:20.912 out and we need to see what works. Even on the, you know, 38:21.111 --> 38:25.031 broader geopolitical scale, like, okay, look, you convinced US, 38:25.111 --> 38:26.332 like, "You know, what about China? 38:26.352 --> 38:29.872 What about Israel?" You know, uh, there will be people who will need to 38:30.131 --> 38:34.091 attempt similar things even in China and I don't 38:34.131 --> 38:38.111 know, for political reasons, maybe I'm not super comfortable commenting 38:38.172 --> 38:41.832 that, but at some point, through some m- mix of either 38:41.852 --> 38:45.792 persuasion or- or force, you do need to also get the Chinese 38:45.832 --> 38:49.392 board with the idea that, you know, superintelligence is an extremely 38:49.430 --> 38:53.412 technology and you need to respond, you know, 38:53.471 --> 38:54.672 that it poses. 39:00.611 --> 39:04.511 All right. Well, thanks for sharing your time. 39:04.591 --> 39:05.161 Yeah. 39:05.912 --> 39:09.652 So there anything else, uh, you want to say or think is, um, 39:10.172 --> 39:12.591 good- important for our- our audience to 39:12.631 --> 39:15.152 hear? 39:15.372 --> 39:19.131 Uh, no, not much in particular. Yeah, 39:19.172 --> 39:22.812 just in general, look, if you like the work I'm doing, you know, 39:22.872 --> 39:25.692 like, share, subscribe to spread this to a wider 39:25.702 --> 39:29.551 I'd especially be interested in being connected to people, you know, 39:29.752 --> 39:33.721 space, in the legal space, even just other people who are really full 39:33.812 --> 39:37.582 time committed to this. Uh, there are 39:38.752 --> 39:42.652 ways you can actually, you know, contribute 39:42.712 --> 39:46.652 to this cause that are, uh, that A, won't require a lot of your time 39:46.692 --> 39:50.372 and even will require a lot of your time if you don't have a lot of time or energy 39:50.491 --> 39:54.131 to spare. You know, even just, you know, sharing, uh, my channel to 39:54.232 --> 39:56.832 others helps, even just sharing your own views in 39:56.852 --> 39:59.562 Like, don't be like, "You know, Samuel said this," 39:59.631 --> 40:03.372 actually believe and you put that on social media. Even 40:03.452 --> 40:07.372 If you do have a lot of time and energy to spare, uh, there 40:07.392 --> 40:09.352 are more important things you can do. 40:09.392 --> 40:13.232 You can start a channel of your own, you know, 40:13.352 --> 40:13.631 Uh, 40:15.031 --> 40:18.852 you can obviously donate money. Uh, there are things you can do. 40:18.912 --> 40:22.471 So in general, feel free to reach out to me 40:22.491 --> 40:26.091 you can contribute to the space. You can also obviously reach out to, 40:26.131 --> 40:29.531 Guido here. There are a number of people in the space who 40:29.591 --> 40:33.511 if you just openly come and ask, "Look, you know, I want to contribute. 40:33.551 --> 40:36.241 What can I do?" There are lots of people who will tell you, 40:36.252 --> 40:37.952 you could be doing. 40:37.971 --> 40:41.582 Sure. Thanks so much for coming on and, uh, and 40:41.652 --> 40:43.500 speaking, because this is, you know, this is 40:45.551 --> 40:46.781 probably the most important 40:47.952 --> 40:50.852 issue facing humanity in the near 40:50.952 --> 40:52.752 term. 40:53.772 --> 40:55.571 And thanks for doing your part. 40:55.652 --> 40:58.971 Oh, yeah. You're welcome.