WEBVTT 00:00.100 --> 00:04.000 [speaker_0] Pre-training scaling 100% does not work or here is an argument 00:04.160 --> 00:07.830 RL scaling 100% does not work. It's just intuitions that 00:07.940 --> 00:11.830 pre-training scale probably doesn't get there, RL scaling probably doesn't 00:11.880 --> 00:12.160 get there 00:12.180 --> 00:16.020 [speaker_1] ... essentially just by scaling RL 00:16.100 --> 00:19.920 just by scaling RL and pre-training, uh, getting to ASI just in the 00:20.040 --> 00:23.830 current paradigm by 2030, uh, in your definition of intelligence 00:23.920 --> 00:24.920 seems very, very unlikely. 00:24.960 --> 00:28.480 [speaker_0] But yeah, researcher, you can right now take a pen and paper, 00:28.540 --> 00:32.500 and come up with, okay, if I had huge amount of GPUs, here 00:32.540 --> 00:34.710 would try, and you can write them down. 00:34.730 --> 00:34.730 [speaker_1] Yeah. 00:34.730 --> 00:36.880 [speaker_0] And then we will ask, okay, well, why haven't these... 00:37.380 --> 00:41.160 Like, did somebody at, you know, OpenAI, Anthropic try them and they failed, or did 00:41.220 --> 00:42.400 nobody try them? 00:42.540 --> 00:44.980 [speaker_1] Uh, maybe you're right and we're on the vertical part of the S 00:45.380 --> 00:49.130 I think if you just plot it from AlexNet, it's mostly the same 00:49.140 --> 00:52.740 paradigm. Uh, and I would count AlexNet 00:53.460 --> 00:57.400 as one breakthrough, Transformer as another breakthrough, 00:57.460 --> 00:57.820 breakthrough. 00:58.900 --> 01:01.990 [speaker_0] Hello, everyone. Uh, my name is Samuel Shadrach. 01:02.700 --> 01:06.320 I graduated from IIT Delhi. I've been following the whole 01:06.440 --> 01:09.780 AI timelines debate for s- a while now. 01:09.920 --> 01:13.400 By when will humanity get super intelligence? 01:13.500 --> 01:16.560 Is this a good thing or a bad thing? What can we... 01:16.620 --> 01:19.940 And now I have a fairly strong opinion it's a bad thing. We should stop it. 01:20.000 --> 01:22.720 That's a whole separate discussion that we can have elsewhere. 01:23.100 --> 01:27.090 Today, I have with me, uh, Raghav. Uh, we are 01:27.140 --> 01:30.140 going to specifically just discuss, uh, AI timelines. 01:30.380 --> 01:33.480 Uh, when do we think, you know, super intelligence will come? 01:33.680 --> 01:37.060 Uh, you know, assuming the research, current development continues. 01:37.220 --> 01:38.980 Raghav, would you like to introduce yourself? 01:39.140 --> 01:42.980 [speaker_1] I'm been a friend of Sam. I've been following the AI safety debate 01:43.040 --> 01:46.880 for the last five, six years now, uh, and very 01:46.920 --> 01:50.680 interested in the subject. I have some strong opinions as well, 01:50.820 --> 01:54.340 as Samuel, but, uh, I have slightly differing opinions from 01:54.400 --> 01:56.260 That's why I'm keen to talk about this. 01:56.800 --> 02:00.680 [speaker_0] Yeah. First of all is, yeah, uh, the whole pre-training 02:00.800 --> 02:03.260 scaling, which right now everyone says is dead. 02:03.420 --> 02:06.180 Even now I'm not yet convinced pre-training scaling is dead. 02:06.540 --> 02:10.100 Yeah, I think there is still a small chance 02:10.259 --> 02:14.160 and you just extrapolate the pre-training, whatever chinchilla, 02:14.240 --> 02:18.210 whichever scaling curve that has been running for the past few 02:18.240 --> 02:20.980 we do that a few more years with new GPUs coming. 02:21.000 --> 02:24.960 I still think there is at least a little bit chance 02:25.040 --> 02:26.960 ASI, which I'm claiming. Uh- 02:27.420 --> 02:27.520 [speaker_1] Okay 02:27.980 --> 02:31.720 [speaker_0] ... a lot more realistic to me 02:31.940 --> 02:35.420 uh, some way of scaling up RL. Now, 02:35.820 --> 02:37.300 that may not be, like, 02:38.200 --> 02:41.769 just again, you just do the current, you know, RL scaling thing 02:41.840 --> 02:44.180 compute. Maybe one breakthrough is required there. 02:44.800 --> 02:47.600 I have, like, more probability mass than, okay, we need at least one breakthrough 02:47.660 --> 02:51.100 on how to do the RL thing better or some other breakthrough. 02:51.440 --> 02:54.780 I have a little bit less probability mass than, okay, you just blindly scale up RL 02:54.800 --> 02:57.500 and it works. But yeah, that's the thing is, like, we don't know. 02:57.820 --> 03:01.240 I have not h- seen like, okay, here is an argument that tells me that 03:01.580 --> 03:05.460 pre-training scaling 100% does not work, or here is an argument that tells me 03:05.600 --> 03:08.890 RL scaling 100% does not work. It's just intuitions that 03:09.380 --> 03:13.260 pre-training scale probably doesn't get there, RL scaling probably doesn't 03:13.320 --> 03:17.040 get there. We probably need another breakthrough, 03:17.080 --> 03:20.820 really knows. Uh, that's broadly the path, 03:21.180 --> 03:24.980 uh, of, you know, what technical capabilities will be 03:25.180 --> 03:28.700 Uh, then there is, okay, the most probably we'll need one more 03:28.780 --> 03:31.920 breakthrough. Why do I think, okay, even if we need one more breakthrough, we 03:31.980 --> 03:34.640 will... There's a good chance we get it in the 03:35.460 --> 03:39.320 For that, you have to extrapolate, well, in the last 10 years, how 03:39.360 --> 03:42.950 many major breakthroughs have happened in machine learning, deep learning, and, 03:42.960 --> 03:46.080 well, actually three or, like, three or four major breakthroughs have happened. 03:46.100 --> 03:49.960 So based on that extrapolate, okay, it doesn't seem surprising to me 03:50.100 --> 03:53.810 if in the next four years another, you know, smart researcher figures out yet 03:53.820 --> 03:57.620 another breakthrough. So there is that kind of thing. 03:58.520 --> 04:01.560 Then I have some specific heuristics and thing. 04:02.120 --> 04:05.760 If I were an AI capability researcher, if I had a huge number of 04:05.820 --> 04:09.420 GPUs, what are, you know, my crazy research hypothesis I might 04:09.500 --> 04:13.340 try for how to speed up RL? And to be clear, I think this is 04:13.360 --> 04:16.880 extremely dangerous thing to do. I think this is bad for the world to do. 04:17.019 --> 04:20.690 But, you know, if I wanted to do this, like, I think, okay, there are some 04:20.700 --> 04:23.960 hypothesis you could try. Uh, what else? 04:24.040 --> 04:27.640 Yeah, this is all, like, very specific to AI research capabilities, 04:27.680 --> 04:31.660 trajectory. And then I have, like, more very high level big picture kind of 04:31.760 --> 04:35.660 Like, you know, why is super intelligence important? 04:36.020 --> 04:39.200 Uh, you know, why the very fact that GPT-2 04:39.280 --> 04:43.060 exists should update your entire models of how the world works. 04:43.440 --> 04:44.080 The fact that 04:44.880 --> 04:48.560 bunch of matrix multiplication can do, you know, like, uh, like 04:48.680 --> 04:52.560 speaking human language rather than animal language, why this is a big deal? 04:52.640 --> 04:56.460 Like, you know, mat- why did matrix multiplication beat the 04:56.500 --> 04:59.690 literal billions of years of evolution that goes between, you know, 05:00.540 --> 05:01.740 uh- 05:01.760 --> 05:05.460 [speaker_1] I'm sorry, I, this, I don't think I'm, 05:05.520 --> 05:09.300 only. I mean, like, maybe we've not discussed this earlier. 05:09.330 --> 05:09.330 [speaker_0] Yeah. 05:09.340 --> 05:10.480 [speaker_1] But maybe we can get to it. 05:10.860 --> 05:11.159 [speaker_0] Yeah, sure. 05:11.380 --> 05:14.460 [speaker_1] The other one I know you're just saying, but this one seems new to me. 05:14.520 --> 05:17.740 Maybe you're framing it differently or something. Yeah. It's fine. 05:17.780 --> 05:21.040 We, we'll probably get to it in the order, but I'm just flagging it that I probably 05:21.080 --> 05:23.320 need you to double-click on this instinct. 05:23.480 --> 05:27.430 [speaker_0] Sure. Uh, I, I just mean like, uh, human language has features that 05:27.480 --> 05:28.940 are not present in animal language. 05:29.020 --> 05:32.720 There are some linguists that have studied that, 05:32.800 --> 05:36.700 like new evolutionary adaptation compared to most animals who don't really even 05:36.780 --> 05:40.180 use language. They kind of just use sounds 05:40.300 --> 05:44.040 And all of this has taken, like, literal billions of years in evolutionary 05:44.100 --> 05:47.420 history to build, and on the other side you have a bunch of 05:47.500 --> 05:51.340 bunch of GPUs in 50 whatever years of AI research history, and 05:51.380 --> 05:53.260 they have been able to crack human language. 05:53.320 --> 05:57.260 Like, that itself tells me, "All right, so intelligence is 05:57.280 --> 06:01.060 likely easier to build than I thought." So that is like-Yeah. 06:01.120 --> 06:04.780 So even just looking at GPT-2 tells me like, oh, okay, so maybe now the 06:04.820 --> 06:06.359 singularity could happen in my lifetime. 06:06.400 --> 06:09.900 Like, you know, until now I was thinking this is some extremely far 06:09.980 --> 06:12.820 Now it looks like, okay, this could actually happen. Uh, what else? 06:12.980 --> 06:16.810 Yeah, I mean, and I have outside level, outside view kind of stuff like, 06:16.880 --> 06:19.750 okay, which experts are actually correctly predicting this? 06:19.780 --> 06:21.600 Which experts are badly predicting this? 06:21.660 --> 06:25.480 I think a lot of people have been consistently badly predicting, 06:25.560 --> 06:29.540 uh, AI trajectory. Yeah, like the people who 06:30.020 --> 06:32.900 their predictions are coming more correct, and the people who keep making the 06:32.919 --> 06:35.640 pessimistic predictions, their predictions keep coming wrong. 06:36.140 --> 06:39.770 So there's that kind of stuff. I think that could summarize my whole 06:40.240 --> 06:43.540 [speaker_1] Fair enough. We'll start taking it one after the other. 06:44.040 --> 06:47.990 Uh, cool. So starting with, uh, you said pre-training, scaling, 06:48.000 --> 06:50.260 Pre-training scaling is work, and pre-training scaling is not dead. 06:50.660 --> 06:54.280 I think when people say pre-training scaling is dead, they don't mean 06:54.340 --> 06:58.040 more parameters and adding more data and adding more compute doesn't lead to 06:58.080 --> 07:00.420 better loss functions that leads to more capabilities. 07:00.480 --> 07:02.180 I don't think anybody denies that. 07:02.280 --> 07:05.820 Uh, people are uncertain that, uh, people are 07:05.859 --> 07:08.980 say-saying that it could stop, but there is no evidence to believe 07:09.060 --> 07:11.340 stop, and I don't think any serious researcher 07:11.620 --> 07:15.500 The argument against pre-training mostly comes from the fact that, um, 07:16.000 --> 07:19.760 it is economically unfeasible to scale because the, uh, amount 07:19.800 --> 07:20.100 of 07:21.100 --> 07:23.400 resources required to do the scaling gives you log. 07:23.440 --> 07:26.180 It does not linearly increase, it gives you log of the intelligence. 07:26.740 --> 07:30.680 Uh, so like increasing, uh, your compute by ten x et cetera gives you 07:30.700 --> 07:34.280 the amount of intelligence or capabilities and double the amount of 07:34.340 --> 07:34.600 goes. 07:34.610 --> 07:37.800 [speaker_0] Exactly. Double the amount of loss, 07:37.940 --> 07:40.140 loss number leads to this, uh, capability. 07:40.500 --> 07:44.200 [speaker_1] Sure. And so which also means that, uh, so th-then the 07:44.240 --> 07:48.140 debate essentially shifts to not that pre-training is dead, 07:48.180 --> 07:51.900 many ten x's can we do and how many doubling of intelligence or, 07:52.000 --> 07:54.020 slightly higher intelligence will essentially do it. 07:54.440 --> 07:56.820 Uh, I think you'd mentioned this in the OOTD 07:56.980 --> 08:00.800 GPT 4.5 was a big update for me, saying that if you just go 08:00.860 --> 08:04.500 on cranking pre-training, you might get a nicer model with not a 08:04.540 --> 08:08.160 capabilities. There was nothing 4.5 could do that was so 08:08.340 --> 08:12.240 far ahead of 4, uh, that essentially I think that 08:12.280 --> 08:13.200 this would essentially 08:14.300 --> 08:17.780 defeat, uh, like th-th-there would be new 08:18.180 --> 08:21.120 Uh, o1 was in a different update because o1 08:21.370 --> 08:23.060 And essentially, this is in the pa-current paradigm. 08:23.120 --> 08:27.000 So my submission for the pre-training argument is that not that pre-training is 08:27.060 --> 08:30.600 dead in the sense that you can't technically, uh, 08:30.880 --> 08:34.820 burn all, all the GDP in the world and like get a 08:34.900 --> 08:38.840 few doublings, etc., out of it. Uh, whether essentially A, there 08:38.860 --> 08:42.520 is a-- it's economically feasible to do it and B, 08:42.720 --> 08:46.560 even if it was economically feasible to do it, uh, is, uh, like 08:46.820 --> 08:50.700 essentially maybe the returns of on, on, on it that again is probably 08:50.740 --> 08:54.500 not worth it. Maybe spending hundred x the amount of 08:54.580 --> 08:58.500 times better model, uh, probably starts breaking a lot of other 08:58.620 --> 09:02.340 Uh, having said that, we are also almost on the edge 09:02.500 --> 09:05.940 of how much compute we can build. Everything is stretched out to its limit, 09:06.420 --> 09:07.660 uh, especially by twenty-thirty. 09:07.760 --> 09:11.680 The year you said, uh, fabs are already built 09:11.740 --> 09:15.580 out, etc. We have nowhere close to, uh, let's say 09:15.640 --> 09:19.280 four x-- four, four increasing our, uh, pre-training. 09:19.320 --> 09:22.420 We probably can do one or two more scale-ups in the next two 09:22.700 --> 09:26.520 Sorry, in, in the next four years. Uh, and that's maximum that we can do. 09:26.980 --> 09:30.180 Uh, even like we do not have enough fabs, we do not have enough electricity, we do 09:30.200 --> 09:34.180 not have enough... Some insane amount of physical bottlenecks 09:34.220 --> 09:35.680 the amount of compute in the world. 09:36.060 --> 09:40.040 Um, currently we are doing, uh, I think forty or 09:40.080 --> 09:41.510 fifty gigawatts of... 09:42.720 --> 09:46.690 Fifty gigawatts? No, thirty gigawatts of, uh, compute capacity is 09:46.720 --> 09:48.550 what twenty twenty-seven will get us. 09:48.620 --> 09:51.880 And, uh, if everything goes according to plan 09:51.960 --> 09:55.780 like all the supply chains get stretched exactly to the right limit, 09:56.240 --> 09:59.420 our best bet is to get to one fifty to two hundred gigawatts a year, which is again 09:59.440 --> 10:03.400 like six times more compute, not ten x more compute, um, from what 10:03.440 --> 10:05.960 we have right now. And like that's two hundred gigawatts per 10:06.440 --> 10:10.220 Uh, and that's assuming all that compute goes into training 10:10.260 --> 10:14.140 etc. So, so there are like lots of physical limitations 10:14.160 --> 10:17.920 till twenty-thirty that do not allow for arbitrarily amount 10:18.300 --> 10:20.780 Uh, we've already kind of pressed to the 10:20.880 --> 10:24.440 Uh, buying a laptop costs like three hundred, 10:24.600 --> 10:25.600 Costs three hundred, four hundred more. 10:25.760 --> 10:29.580 I don't think that the world can essentially take four orders of magnitude 10:29.620 --> 10:32.740 compute scaling that readily now. I think we, 10:33.140 --> 10:37.100 Coming to RL, I agree that essentially RL scaling 10:37.160 --> 10:39.070 There are two points that I want to make about RL. 10:39.070 --> 10:41.390 RL scaling, uh, is also extremely expensive. 10:41.840 --> 10:44.180 Uh, it's more expensive than pre-training 10:44.260 --> 10:48.170 Uh, the other point that I want to make about RL is, uh, 10:48.200 --> 10:51.940 gives us general capability increases, RL gets us very 10:52.120 --> 10:55.960 jagged increases in capability. You only get increase in capability in 10:56.000 --> 10:59.820 domains like coding and math, uh, which essentially kind of 10:59.860 --> 11:03.740 defeats the, the specific model of intelligence 11:03.800 --> 11:06.720 that it will be better at everything by twenty-thirty. 11:07.080 --> 11:10.780 So if we, if we cannot find good RL candidates 11:10.800 --> 11:14.740 loops for different kinds of event, we might not even solve, uh, forget 11:14.820 --> 11:17.120 solving for like robotics and like other things. 11:17.180 --> 11:21.140 Even in just like atoms world, we might not be able to solve all of it 11:21.280 --> 11:24.600 to human level because we will not just find enough RL data, etc., 11:25.320 --> 11:28.979 or enough closed loops, etc., uh, uh, to essentially do it. 11:29.080 --> 11:32.360 Uh, so, so I don't think that RL also scales 11:32.600 --> 11:36.540 arbitrarily that you can just thousand x the RL compute and actually get 11:36.580 --> 11:40.320 away with it. Uh, there is a limit to how much RL compute you can 11:40.400 --> 11:43.040 RL also just gets you increases in capability. 11:43.380 --> 11:47.220 Yes, economically viable, uh, the economic, the killer use 11:47.260 --> 11:51.140 case for LMS currently is coding, and coding is economically viable, etc. 11:51.760 --> 11:53.900 There's a small, uh, probability gap I have. 11:53.940 --> 11:57.460 This leads to some sort of recursive self-improvement or some sort of, uh, 11:57.480 --> 11:59.840 breakthrough, etc., that happens because of this. 12:00.200 --> 12:04.020 Uh, but aside from that gap, uh, which sure, we can talk about 12:04.400 --> 12:08.360 Uh, but aside from that gap, uh, the path essentially just by scaling 12:08.500 --> 12:12.370 RL or just by scaling pre-training or just by scaling RL and pre-training, 12:12.780 --> 12:16.320 uh, getting to ASI just in the current 12:16.740 --> 12:19.200 uh, in your definition of intelligence seems 12:19.240 --> 12:23.020 unlikely.Um, to me, like forget 25%, I would give 12:23.360 --> 12:26.360 sub 1% chance in the current paradigm, uh, in these specific 12:26.400 --> 12:30.260 circumstances. Not to say that, uh, I have a much higher probability was 12:30.340 --> 12:33.660 might get to superintelligence by 2030, but, uh, that's 12:33.740 --> 12:37.140 essentially assuming technology that haven't been invented yet come into 12:37.200 --> 12:40.460 being. Uh, so that was the second point about RL. 12:40.600 --> 12:43.940 Uh, your third point was tied into this, is like, okay, we might need one more 12:43.960 --> 12:45.960 breakthrough. Uh, current RL might not... 12:46.000 --> 12:47.640 You think there's a chance that might not be enough. 12:47.680 --> 12:49.360 Maybe it's enough, but we might, maybe it's not enough. 12:49.400 --> 12:50.920 Maybe we need one more breakthrough. 12:51.040 --> 12:54.469 And, uh, uh, your submission there was that, hey, 12:54.860 --> 12:58.590 breakthrough will happen, uh, because look at the last 10 years, 12:58.780 --> 13:02.740 we've gotten transformers and we've gotten pre-training scaling, 13:02.840 --> 13:06.280 RL. So looks like breakthroughs are coming very, very quickly. 13:06.700 --> 13:10.380 Um, I think is, uh, this, I don't think that, 13:11.000 --> 13:14.680 uh, there is enough data points for you to 13:15.040 --> 13:19.020 Uh, even sta- in the start of GBD, despite essentially almost 13:19.080 --> 13:22.800 all of world's intelligence and attention going to this problem, 13:22.940 --> 13:26.520 RL scaling, which we've cracked, uh, there's not been another scaling 13:26.580 --> 13:30.140 paradigm that we've cracked. So apart from pre-training and RL scaling, 13:30.260 --> 13:33.380 people are ready to throw computes at other scaling things, and that's not 13:33.420 --> 13:35.740 something that we've cracked. I'll take your rebuttal. 13:36.040 --> 13:36.280 [speaker_0] Oh, rebuttal. 13:36.300 --> 13:40.239 [speaker_1] Like what other scaling paradigm have, 13:40.280 --> 13:40.560 RL- 13:40.870 --> 13:40.890 [speaker_0] No, no. 13:40.900 --> 13:42.330 [speaker_1] Despite throwing insane- 13:42.330 --> 13:45.880 [speaker_0] Okay, it has a lot of attention, 13:45.920 --> 13:47.180 You're saying this mean that other breakthroughs- 13:47.220 --> 13:49.320 [speaker_1] No, no other breakthroughs. I'm saying other breakthroughs... 13:49.840 --> 13:53.020 I'm saying, uh, other breakthroughs are possible in the sense they're not 13:53.160 --> 13:57.080 physically impossible. But A, do you disagree that there are other, 13:57.200 --> 13:58.200 things that we can just throw, 13:59.220 --> 14:03.120 s- other, other scalable things that we can throw compute money, 14:03.140 --> 14:06.980 more intelligence aside from, uh, like just regular pre-training 14:07.200 --> 14:08.160 and RL? 14:08.560 --> 14:12.460 [speaker_0] I think there is a huge backlog of research hypothesis 14:12.540 --> 14:13.540 at all these AI companies. 14:13.600 --> 14:14.520 [speaker_1] I doubt it. 14:14.660 --> 14:15.460 [speaker_0] Like lot of very- 14:15.540 --> 14:15.980 [speaker_1] I doubt it 14:15.990 --> 14:19.830 [speaker_0] ... obvious things to try, but because compute is scarce, 14:19.920 --> 14:20.060 decide- 14:20.080 --> 14:20.480 [speaker_1] I don't know 14:20.500 --> 14:21.450 [speaker_0] ... okay, which ones to, uh- 14:21.480 --> 14:25.300 [speaker_1] I get it, but essentially, I, I, I, I, I hear that argument saying 14:25.320 --> 14:28.980 that obviously we can improve our models in X sector, but if there were such such 14:29.140 --> 14:32.740 obvious scaling paradigms that essentially could have been done 14:32.800 --> 14:36.780 than the current paradigms, uh, I think you would see some evidence of it. 14:37.080 --> 14:39.809 You would see there's, there's enough time essentially going into 14:39.809 --> 14:39.999 that- 14:40.260 --> 14:41.540 [speaker_0] No, I, I hear all excuses 14:41.800 --> 14:41.809 [speaker_1] ... that- 14:41.840 --> 14:44.360 [speaker_0] I'm saying like you right now, you are not an expert AI researcher. 14:44.460 --> 14:48.140 You can right now take a pen and paper, sit for half a day and come up with, okay, 14:48.380 --> 14:52.320 if I had huge amount of GPUs, here are 10 hypothesis I would try, and 14:52.380 --> 14:53.420 you can write them down. 14:53.430 --> 14:53.430 [speaker_1] Yeah. 14:53.430 --> 14:57.340 [speaker_0] And then we will ask, okay, well, why haven't these, like, 14:57.400 --> 15:01.100 know, Open-- Anthropic try them and they failed, or did nobody try them? 15:01.240 --> 15:01.400 [speaker_1] I- 15:01.469 --> 15:01.689 [speaker_0] If nobody tried- 15:01.710 --> 15:05.380 [speaker_1] ... doubt it's that easy. Can you name, 15:05.420 --> 15:09.370 that hasn't been tried that you think has a higher chance of it 15:09.740 --> 15:11.860 an RL level breakthrough if tried? 15:12.200 --> 15:14.880 [speaker_0] I'm not saying any one idea if I try that will definitely work. 15:14.940 --> 15:15.230 I'm saying- 15:16.040 --> 15:19.480 [speaker_1] No, any example of an idea 15:19.540 --> 15:23.460 If nobody's tried it, I didn't find any research papers on it, 15:23.520 --> 15:25.320 we'll probably get ASI. 15:25.420 --> 15:29.020 [speaker_0] Uh, if you just want random ideas, 15:29.200 --> 15:29.840 Uh- 15:29.920 --> 15:30.060 [speaker_1] Sure. 15:31.040 --> 15:34.670 Just to understand, like, what kind of ideas do you think 15:34.700 --> 15:38.220 these guys are so compute constrained that if there is an RL level breakthrough 15:38.300 --> 15:42.060 just sitting on like some researcher's notepad and they've not had the time. 15:42.160 --> 15:45.460 Because even for RL, they didn't actually have to use 15:45.500 --> 15:48.340 the idea. Uh, the idea, the idea, the, the- 15:48.960 --> 15:52.840 [speaker_0] Yeah. Uh, one example, most of the training still happens in very 15:52.940 --> 15:56.380 code, like, you know, PyTorch, you know, like four-bit, eight-bit floating point 15:56.460 --> 15:59.760 numbers. If you really wanted, you could optimize all this way down. 15:59.800 --> 16:03.300 You could literally run training inside an ASIC. You could optimize the code. 16:03.340 --> 16:06.540 [speaker_1] Yeah. So, so for example, that's like a very bad idea. No, no, no. 16:06.580 --> 16:10.400 So for example, running ASIC, so, so that essentially assumes 16:10.440 --> 16:12.220 like a lot of other things need to move in the world. 16:12.260 --> 16:15.990 One, the amount of GPU capacity that already exists allocated in the world 16:16.020 --> 16:19.480 needs to go away. Secondly, there's a reason why even of co... 16:19.520 --> 16:22.510 Like doing this does not get you that much compute because essentially 16:22.520 --> 16:25.570 compute constrained, you're memory constrained, uh, 16:25.570 --> 16:28.080 training and you're basically memory bandwidth constrained, not even memory 16:28.100 --> 16:31.380 constrained. And, uh, yes, sure, like we have like silicon 16:31.440 --> 16:34.200 photonics, and we have other breakthroughs 16:34.260 --> 16:38.159 engineering problem. Uh, I don't think it's as easy as like, hey, 16:38.200 --> 16:41.860 on ASIC and we get like 100X speed up and nobody's had the time 16:42.279 --> 16:45.200 Uh, there's so much incentive for any smart 16:45.240 --> 16:48.770 If you think it's that easy to go replace the entire compute 16:48.840 --> 16:52.430 ASIC and people have just like not tried it because of some other 16:52.460 --> 16:54.000 crunch, I, I think you're mistaken. 16:54.030 --> 16:54.110 [speaker_0] No, no, I- 16:54.150 --> 16:57.460 [speaker_1] Like there's insane amount of like capitalist in-incentive to do it. 16:57.660 --> 17:01.440 Like, th-which would be like, "Hey." I'm saying if there's any other 17:01.980 --> 17:04.950 uh, the amount of compute you need to quote 17:05.000 --> 17:08.680 small. Uh, and therefore if, if there were these insane... 17:08.720 --> 17:12.210 o1, for example, didn't require too much compute on forward 17:12.240 --> 17:15.940 could work. Uh, if you had such an idea, you could show that, hey, this 17:16.020 --> 17:19.870 works and we want to use this to scale our models and it's sitting in the labs and 17:19.900 --> 17:23.260 we just think, but looks like there's n-none, 17:23.580 --> 17:27.360 [speaker_0] Uh, oh, okay. Yeah. I'm saying there are a lot of research 17:27.460 --> 17:31.420 require a lot of compute to prove even as a proof of concept, okay, this is worth 17:31.460 --> 17:31.919 exploring. 17:32.040 --> 17:33.990 [speaker_1] I, I doubt it. 17:34.070 --> 17:34.070 [speaker_0] Yeah. 17:34.120 --> 17:34.900 [speaker_1] I think they all- 17:35.300 --> 17:35.570 [speaker_0] The idea is- 17:35.570 --> 17:39.300 [speaker_1] They all show signs of life. All of these research ideas show signs of 17:39.380 --> 17:42.580 life much before you have to scale them to get gains out of it. 17:43.220 --> 17:47.100 There are very few research ideas that quote unquote "only get unlocked at," 17:47.380 --> 17:50.850 if you only train them for, like only if you spend billion dollars of compute is 17:50.880 --> 17:54.220 the first sign of life you get from that research idea, uh, 17:54.280 --> 17:57.230 research idea saying that, "Hey, if I just keep throwing compute at it"- 17:57.330 --> 18:00.280 [speaker_0] In research, almost every breakthrough 18:00.380 --> 18:04.240 Like only after you scaled it to literal billions of dollars you saw that it was 18:04.260 --> 18:04.560 working. 18:05.060 --> 18:07.740 [speaker_1] No. Give me one research idea that's gonna 18:08.100 --> 18:09.940 GPT-1 was like a very small model. 18:10.480 --> 18:10.550 [speaker_0] RL or- 18:10.650 --> 18:11.669 [speaker_1] RNN was a very small model. 18:11.740 --> 18:13.160 [speaker_0] Uh, GPT- 18:13.280 --> 18:14.820 [speaker_1] RL, for example, didn't require billions of dollars. 18:14.980 --> 18:16.780 So o1, for example, was a very cheap model. 18:17.140 --> 18:21.040 Once GPT-4 was trained, training on chain of, 18:21.080 --> 18:24.964 of thought and doing RL was like a very, very small experimentSo, 18:25.174 --> 18:25.174 uh- 18:25.184 --> 18:27.854 [speaker_0] Can, uh, do you have numbers on that? 18:27.864 --> 18:29.584 [speaker_1] Yeah, yeah. So, so there's, A, there's ... 18:29.774 --> 18:33.424 I, I'll find out where I read about this, but basically the first version of the O1 18:33.444 --> 18:35.264 model was just get tried in a lab, et cetera. 18:35.604 --> 18:38.564 For example, uh, you can take Llama 3 right now, 18:39.164 --> 18:40.434 and, uh, you can take- 18:40.464 --> 18:43.323 [speaker_0] No, no, not, not about right now. 18:43.484 --> 18:46.334 Back then when O1 was first tried, like my guess without having- 18:46.334 --> 18:48.344 [speaker_1] Like I'm saying, you can make it more efficient. 18:48.384 --> 18:48.504 [speaker_0] S- sorry. 18:48.544 --> 18:49.364 [speaker_1] You can make it more 18:50.304 --> 18:50.523 efficient. Yeah. 18:50.744 --> 18:54.104 [speaker_0] Yeah, like without having read it, my guess 18:54.144 --> 18:57.524 required at least $10 million on top of the 18:57.564 --> 19:01.324 GPT-4 training cost, and like I have not actually checked the 19:01.384 --> 19:01.804 but yeah. 19:02.344 --> 19:05.724 [speaker_1] You needed the GPT-4 to get trained first, 19:05.764 --> 19:08.664 and then you found a new scaling paradigm on that scaling paradigm. 19:09.104 --> 19:09.193 [speaker_0] Yeah, yeah. 19:09.224 --> 19:09.634 [speaker_1] That I agree. 19:09.653 --> 19:10.944 [speaker_0] I'm saying first you had the whole- 19:11.004 --> 19:11.564 [speaker_1] But go from four to- 19:11.614 --> 19:13.513 [speaker_0] ... GPT-4 training cost, then GPT-4 trained. 19:13.524 --> 19:13.614 [speaker_1] Right. 19:13.984 --> 19:14.704 [speaker_0] Then you had- 19:15.164 --> 19:15.434 [speaker_1] But O1, when you look at- 19:15.434 --> 19:19.384 [speaker_0] ... to research hypothesis. Each of those hypothesis took at least $10 19:19.804 --> 19:21.124 to test, and 10 million is a random number. 19:21.164 --> 19:21.264 [speaker_1] No. 19:21.344 --> 19:22.824 [speaker_0] I think it's actually important. Uh- 19:22.904 --> 19:24.384 [speaker_1] I don't think it takes $10 million of test. 19:24.444 --> 19:27.164 I think it takes like a few hundred thousand dollars, 19:27.224 --> 19:30.924 dollars to test a research idea to see if that it has any 19:31.024 --> 19:34.564 sign or any chance. Yes, like you might see like different gains 19:34.664 --> 19:35.374 losses, et cetera. 19:35.394 --> 19:37.124 [speaker_0] That is then possibly 100 million is what I'm claiming. 19:38.004 --> 19:41.324 [speaker_1] I doubt it. I don't know if any of all these ideas take 10 million. 19:41.384 --> 19:44.463 For example, O1 didn't take 10 million post GPT-4 19:44.864 --> 19:47.324 [speaker_0] We can actually go and check that maybe. 19:47.404 --> 19:48.184 Like I know this is- 19:48.224 --> 19:48.274 [speaker_1] No, no 19:48.274 --> 19:51.044 [speaker_0] ... not public information, but we can go and see like- 19:51.244 --> 19:53.644 [speaker_1] No, no, I think we can check because the amount of 19:53.684 --> 19:54.924 Yeah, sure, because the actual amount ... 19:54.934 --> 19:57.954 So the idea, the way research works is get an idea, 19:58.164 --> 20:01.304 this thing. If y- if you see any sort on it or anything 20:01.314 --> 20:04.584 "Cool. You know what? This warrants more investigation, 20:04.744 --> 20:07.804 actually go improve something in the form." And then you can do optimizations 20:07.844 --> 20:09.224 it, and then you can make it better, et cetera. 20:09.524 --> 20:13.244 But the first amount of like this thing, the sign of life saying that this has, 20:13.304 --> 20:16.394 this is probably has some legs can come from not that much money. 20:17.024 --> 20:17.474 [speaker_0] Yeah. This is- 20:17.504 --> 20:19.144 [speaker_1] And then obviously getting actual real- 20:19.174 --> 20:21.064 [speaker_0] ... GPT works, however, ML is not like this. 20:21.724 --> 20:23.264 [laughs] I think that's my actual method. 20:23.304 --> 20:26.364 [speaker_1] All of ML has been like this. Like pre-training, for example, 20:26.404 --> 20:28.664 was a scary small model. GPT-2 was a very small model. 20:29.184 --> 20:32.864 Uh, it took like, uh, probably 150 or 2, like the, the initial 20:32.924 --> 20:35.284 GPT cost like a million dollars, I think. Not that much. 20:35.784 --> 20:39.704 Uh, GPT-3 again, like took like slightly larger amount of 20:40.024 --> 20:41.424 but nothing compared to right now. 20:41.824 --> 20:45.604 And the amount of, uh, the only reason you go from GPT-2 to 3 to 20:45.724 --> 20:49.484 4 to 5 or whatever subsequent models is because you see gains 20:49.524 --> 20:53.484 from scaling all the time. Uh, RNN, for example, you can see that, okay, 20:53.544 --> 20:55.884 scale the RNN paradigm, you get gains from it. 20:56.424 --> 20:59.673 Uh, and then, then you stop seeing gains, so that's why RNN didn't scale or 20:59.684 --> 21:01.214 whatever. Or, uh, then they went to LSTM. 21:01.284 --> 21:02.694 LSTM didn't scale, and then they went to transformer. 21:02.784 --> 21:05.754 Transformer scaled fairly much, and like, cool, we found a scalable paradigm. 21:06.204 --> 21:09.244 The idea that you can see gains and then you try to scale it 21:09.744 --> 21:10.064 So- 21:10.084 --> 21:10.524 [speaker_0] Yeah 21:10.534 --> 21:14.144 [speaker_1] ... uh, so it's not that, oh, I have to like literally roll dice of $10 21:14.224 --> 21:16.584 each time to get one idea. It's not that random. 21:16.944 --> 21:20.884 [speaker_0] Okay, I'll make a tighter claim. After GPT-2 21:20.924 --> 21:24.444 had come out, if you wanted to try out any ML research 21:24.484 --> 21:28.304 hypothesis, you probably needed at least $10 million 21:28.324 --> 21:31.924 life or not. Like for, for most of the research hypothesis- 21:31.984 --> 21:32.134 [speaker_1] No, you could use the, uh- 21:32.134 --> 21:34.064 [speaker_0] ... you needed at least $10 million since GPT- 21:34.104 --> 21:38.084 [speaker_1] No, no. So the id- the, the idea is that you train GPT-1, 21:38.104 --> 21:41.284 saw that there were gains on it. The obvious thing to do is that, "Hey, 21:41.324 --> 21:44.224 found a scalable paradigm. I think I can scale it further to see 21:44.324 --> 21:44.644 gains." 21:45.024 --> 21:45.194 [speaker_0] Sure. 21:45.384 --> 21:48.484 [speaker_1] And then you train GPT-3, and there are still gains from it, 21:48.524 --> 21:51.704 res- dead research ideas that might show signs of life, but o- 21:51.724 --> 21:53.084 it, they start breaking. 21:53.124 --> 21:56.944 [speaker_0] Yeah, I'm saying to try any other idea, 21:56.964 --> 22:00.604 pre-training scaling. If you have any other idea apart from 22:00.824 --> 22:04.584 GPT-2 came out 2018, right? So after 2018, if you had any other 22:04.644 --> 22:08.404 idea besides the whole pre-training scaling thing, you wanted to test it out, 22:08.424 --> 22:12.304 would need at least $10 million for most ideas to even test out and see if this 22:12.324 --> 22:13.164 has any life or not. 22:13.584 --> 22:15.604 [speaker_1] And that's also trivial amounts of money if... 22:15.704 --> 22:18.424 I think if people who know about how these things work, it's not completely 22:18.464 --> 22:20.764 unintuitive. They have, they have a big idea of test. 22:21.044 --> 22:24.964 I, I'm not saying is that, uh, there might be a breakthrough that 22:25.004 --> 22:28.004 in some old research paper or in scribbled in all the diaries 22:28.064 --> 22:32.024 overlooked, uh, but it is not as low-lying a fruit as you 22:32.564 --> 22:36.384 if, if only we had just spent more compute, 22:36.824 --> 22:40.054 thousands of thousands of scalable paradigms." Finding scalable paradigms is 22:40.104 --> 22:43.484 really, really hard. We've only done like two ti- 22:43.504 --> 22:46.213 last 10 years and two times in the last 50 years and two times in the last thousand 22:46.224 --> 22:50.114 years, and, uh, essentially, uh, uh, just because we've gotten 22:50.364 --> 22:53.764 lucky, just because we got lucky doesn't mean 22:53.804 --> 22:56.064 happening in the next two, three, four, five years. 22:56.084 --> 22:59.524 It's just like there'll just be like scalable paradigms after scalable 22:59.784 --> 23:02.464 that will just keep showing up because it's shown up last two times. 23:02.824 --> 23:05.744 [speaker_0] Okay, so we have identified some disagreement 23:05.864 --> 23:07.344 Uh, how do you think we can resolve this? 23:07.384 --> 23:11.014 Like what data points will work or what arguments about how do you think 23:11.014 --> 23:14.884 [speaker_1] One data point that would work, one data point 23:14.944 --> 23:18.904 move you is that if you look other, uh, if you look at other fields, you look 23:18.944 --> 23:22.884 at biology, et cetera, uh, just because like there's one breakthrough 23:22.924 --> 23:26.884 that essentially leads to a different class of 23:26.924 --> 23:28.414 discoveries or drug discovery happening. 23:28.464 --> 23:30.544 For example, you get, uh, let's say 23:31.704 --> 23:35.414 discovery through, like, for example, RNA delivery of drugs, 23:36.044 --> 23:39.044 uh, like the mRNA vaccine, et cetera, which you can like modify RNA and you can 23:39.064 --> 23:42.964 inject in people, et cetera. That means that, yes, a lot of, a lot 23:43.044 --> 23:43.624 of, uh... 23:44.544 --> 23:47.644 That, that was a big deal, like get a new branch of medicine, 23:47.664 --> 23:51.384 doesn't automatically mean that the amount of new breakthroughs will 23:51.424 --> 23:55.224 increase. If anything, what we've seen is that there is actually a slowdown in 23:55.284 --> 23:59.074 amount of new ideas and new researches in every mature field 23:59.104 --> 24:02.264 more, more attention goes into it because all 24:02.584 --> 24:05.844 Uh, so finding the next breakthrough is not a linear process. 24:05.924 --> 24:08.524 It's actually a super linear process. Not like it's a log process. 24:08.564 --> 24:12.004 You have to like spend 10X, 100X more resources to get more ideas, 24:12.464 --> 24:16.244 and, uh, you pluck the low-hanging fruits very, 24:16.284 --> 24:18.304 is not that hard. It's not that easy. 24:18.524 --> 24:21.244 So-Um, so other field at least do it. 24:21.284 --> 24:23.424 You can argue that M-ML is different for some reason. 24:23.544 --> 24:26.604 I don't know why scientific idea is, like, inherently be different in, uh, ML 24:26.624 --> 24:27.804 because like, oh, ML is different. 24:27.844 --> 24:31.704 Because mostly what happens is if enough eyeballs look at a problem, uh, 24:31.744 --> 24:34.044 they look at all the low-hanging fruits, then they go to the second level of 24:34.084 --> 24:36.804 low-hanging fruit, and they keep doing this, and they, 24:36.814 --> 24:38.184 hypothesis. And like, okay, cool. 24:38.664 --> 24:42.474 Uh, this has been already done in physics, for example, uh, or 24:42.504 --> 24:45.904 like chemistry, et cetera. We don't expect like crazy amount of math 24:45.924 --> 24:49.684 come out, uh, by a mathematician, uh, or like a 24:49.724 --> 24:52.864 great, like, new, new lines of math schools, 24:53.384 --> 24:56.424 Uh, and I feel like with more attention on a 24:56.474 --> 24:58.504 It becomes harder to e-curve. S-curve is not easier. 24:58.824 --> 24:59.364 [speaker_0] Okay, uh- 24:59.414 --> 25:01.724 [speaker_1] It depends on what part of the S-curve you're on, I think. 25:02.324 --> 25:03.924 [speaker_0] Yeah, I think that S-curve analogy is good. 25:04.004 --> 25:07.924 So yeah, if we are comparing to other scientific 25:07.944 --> 25:09.644 field in which experiments are expensive. 25:09.704 --> 25:13.304 So, like, we should not compare this to, like, theoretical math where you just need 25:13.384 --> 25:14.724 person sitting with pen and paper. 25:14.804 --> 25:17.884 Like, something like drug discovery is a better analogy for this. 25:18.304 --> 25:21.844 And by the way, I do think ML is a bit different, 25:21.944 --> 25:25.274 analogizing with other fields, uh, in drug discovery. 25:25.324 --> 25:27.004 [speaker_1] Or like theoretical physics, for example. 25:27.344 --> 25:30.244 [speaker_0] Sorry, y-you mean theoretical physics is cheap 25:30.744 --> 25:34.404 [speaker_1] Is also expensive. It was cheap at some point, 25:34.424 --> 25:35.964 like, breakthrough with pen and paper. 25:36.384 --> 25:40.004 But now if you want to, like, uh, experimental physics, sorry, 25:40.244 --> 25:40.314 physics- 25:40.314 --> 25:40.674 [speaker_0] Mm. Yeah 25:40.674 --> 25:44.524 [speaker_1] ... uh, you could make like a... Yeah, 25:44.544 --> 25:46.174 to do any new physics. 25:46.624 --> 25:50.224 [speaker_0] Sure. Yeah. Okay, fine. Experimental physics would work. 25:50.504 --> 25:54.254 Uh, yeah. Now to actually argue about experimental 25:54.284 --> 25:57.624 physics or drug discovery, I will have to actually read more about 25:58.144 --> 25:58.924 physics or drug discovery. [chuckles] 25:59.064 --> 26:01.564 [speaker_1] Uh, but do you agree that, that, like- 26:01.744 --> 26:02.534 [speaker_0] Also, we have to- 26:02.804 --> 26:02.814 [speaker_1] With the- 26:02.824 --> 26:03.084 [speaker_0] Yeah, no 26:03.204 --> 26:04.094 [speaker_1] ... enhanced attention and- 26:04.204 --> 26:05.604 [speaker_0] Also, we have to pick a time period. Sorry. 26:05.864 --> 26:09.684 Uh, also, like, if we take, you know, drug discovery or experimental physics as 26:09.724 --> 26:13.574 example, we have to take a time period in the 26:13.604 --> 26:17.584 was known that, okay, this thing is, you know, in like boom phase and like, you 26:17.624 --> 26:19.504 know, lots of new capabilities are coming out. 26:19.544 --> 26:22.364 Like, like in ML right now, we know we are in that sort of phase. 26:22.424 --> 26:24.064 Like, there may be new things we could try. 26:24.144 --> 26:26.524 Like, it's not like, okay, it's like a dead field- 26:26.564 --> 26:26.574 [speaker_1] Sure 26:26.574 --> 26:27.204 [speaker_0] ... mature field. 26:28.024 --> 26:28.564 We have not reached- 26:28.584 --> 26:32.424 [speaker_1] Sure, sure. Yeah, I agree. And like example, 26:32.464 --> 26:35.184 that. I don't know about experimental physics, 26:35.244 --> 26:39.224 also, where a bunch of like new physics 26:39.264 --> 26:41.144 1920s. There was a activity that came out. 26:41.184 --> 26:44.364 There's like, like, a bunch of these new... 26:44.434 --> 26:46.924 All of them essentially got started in 1920s. 26:47.144 --> 26:49.124 There was one period in the 1600s that happened. 26:49.244 --> 26:51.254 Uh, but, uh, I agree that we might- 26:51.604 --> 26:54.604 [speaker_0] Studying that particular time period to study, you know, 26:54.664 --> 26:57.334 that happened there? And after a few breakthroughs- 26:57.334 --> 26:57.334 [speaker_1] Yeah 26:57.384 --> 27:00.224 [speaker_0] ... came out, now to extrapolate, okay, 27:00.264 --> 27:02.244 breakthroughs come out, and how expensive will it be to run- 27:02.254 --> 27:02.554 [speaker_1] Sure, sure 27:02.584 --> 27:03.884 [speaker_0] ... experiments? I think that's the kind of- 27:03.924 --> 27:04.424 [speaker_1] And I agree 27:04.544 --> 27:05.344 [speaker_0] ... study. 27:05.384 --> 27:09.204 [speaker_1] And I agree. And, and, and then essentially, right, 27:09.304 --> 27:12.904 part of the S-curve, that then there, then you should 27:12.944 --> 27:14.344 expect more breakthroughs to come out. 27:14.724 --> 27:18.274 If you're on the horizontal part of the S-curve, then you should think, then you 27:18.284 --> 27:20.044 should expect less discoveries to come out. 27:20.104 --> 27:23.264 Which part of the S-curve you are on, I don't think either of us know. 27:23.384 --> 27:25.464 Uh, but I think- 27:25.604 --> 27:25.984 [speaker_0] I'm claiming- 27:26.074 --> 27:26.724 [speaker_1] ... the more time- 27:26.884 --> 27:30.504 [speaker_0] That's my claim. And also, sure, some decent probability they're not, 27:30.944 --> 27:33.384 [speaker_1] Okay. Uh, what will be your evidence to saying 27:33.444 --> 27:37.144 Like, how are you so sure? I have zero base to say that this thing, 27:37.184 --> 27:40.504 can, on hindsight be like, "Oh, looks like we were on the vertical part 27:40.784 --> 27:44.674 [speaker_0] Okay. And for me, it's just extrapolate last five to 10 27:44.724 --> 27:46.624 data points. Okay, which year did, uh... 27:46.804 --> 27:50.104 Well, actually you can go back to , you know, which year did AlexNet come out? 27:50.154 --> 27:50.154 [speaker_1] Uh- 27:50.184 --> 27:53.144 [speaker_0] Then which year did, you know, transformer come out? 27:53.184 --> 27:53.644 Which year did- 27:53.654 --> 27:53.654 [speaker_1] That- 27:53.654 --> 27:57.544 [speaker_0] ... GPT-2 come out? And just put these on like a year 27:57.584 --> 27:58.254 versus, you know- 27:58.304 --> 27:58.684 [speaker_1] That is- 27:58.733 --> 28:02.544 [speaker_0] ... new breakthrough kind of graph, 28:02.584 --> 28:04.344 this look like your S-curve is saturated? 28:04.404 --> 28:06.184 Yes or no?" And no, it doesn't look like there's- 28:06.304 --> 28:08.004 [speaker_1] And this is like an outside perspective. 28:08.044 --> 28:11.584 You don't have to trust it, but Ilya Sutskever, who 28:11.624 --> 28:15.564 was responsible for GPT, was responsible for RL, uh, comes 28:15.584 --> 28:18.764 and says essentially all the good ideas are down and we need to spend some time 28:18.804 --> 28:21.484 doing new research. And this is the time for... 28:21.524 --> 28:24.594 I don't know if you saw that episode or The R Kesh, but he's like, 28:24.684 --> 28:26.624 scaling is over. Now is the time for new research." 28:27.524 --> 28:29.384 [speaker_0] What are Ilya Sutskever's timelines? 28:29.464 --> 28:32.804 Are they less bullish than me when I'm saying 25% ASI 2030? 28:32.864 --> 28:34.994 Like, does Ilya have like less bullish timelines 28:35.914 --> 28:37.593 [speaker_1] I don't know, but I think that's a relevant. 28:38.024 --> 28:38.263 [speaker_0] Sorry? 28:38.984 --> 28:40.994 [speaker_1] I think that's irrelevant. I feel- 28:41.124 --> 28:42.624 [speaker_0] No, no, you brought up Ilya- 28:42.674 --> 28:42.674 [speaker_1] Yeah. No 28:42.674 --> 28:46.454 [speaker_0] ... then I was like, okay, like does Ilya agree with me already, 28:46.464 --> 28:46.784 thing. 28:47.464 --> 28:48.824 [speaker_1] It doesn't matter if Ilya agrees with you. 28:48.884 --> 28:52.424 What Ilya does agree, disagree with you on is 28:52.484 --> 28:56.164 that the low-hanging scaling fruits have been plucked and we need to go find new 28:56.204 --> 28:59.884 scaling breakthroughs, uh, which he's confident that he will find, 29:00.004 --> 29:03.734 economic incentive to say that. Uh, but he's saying that, "Okay, 29:03.764 --> 29:05.704 are down, and now we need to find something new to 29:06.164 --> 29:10.104 [speaker_0] Okay. No, but if you strongly defer to Ilya on 29:10.184 --> 29:12.234 this question, then Ilya's actual timelines are- 29:12.264 --> 29:13.284 [speaker_1] No, I don't. I don't, 29:14.164 --> 29:17.064 I don't, I don't defer strongly to Ilya on this 29:17.124 --> 29:20.434 All I'm saying is there's one extra evidence point saying that the guy who was 29:20.464 --> 29:23.164 involved with all these three breakthroughs comes and says 29:23.184 --> 29:26.604 no low-hanging fruits anymore, and we have to go find more, uh, 29:26.864 --> 29:30.364 should update you that we are probably not on the vertical part of 29:30.424 --> 29:33.424 [speaker_0] No, no, no. Uh, I took a different lesson from this. 29:33.584 --> 29:37.204 Uh, like Ilya has, again, as bullish as me timelines, 29:37.244 --> 29:40.764 low-hanging fruit is picked. What he means is, okay, the 29:40.804 --> 29:43.403 low-hanging fruit is not like one month low-hanging fruit. 29:43.464 --> 29:45.304 It's like two years, three years low-hanging fruit. 29:45.724 --> 29:48.604 [speaker_1] No, I think his time, his timelines are significantly longer. 29:48.684 --> 29:52.564 I think he's like in the next eight to 10 years will probably be some areas 29:52.624 --> 29:56.144 of research that we need to find to scale, and yeah. 29:56.524 --> 29:56.534 [speaker_0] Okay. 29:56.564 --> 29:58.244 [speaker_1] I, I think his is probably longer. 29:58.304 --> 30:02.124 [speaker_0] So, so again, I'm like, yeah, if you want to debate specifically Ilya's 30:02.204 --> 30:04.064 I think actually we have to go and find his timeline. 30:04.124 --> 30:05.244 [speaker_1] I don't want to debate Ilya's worldview. 30:05.324 --> 30:08.964 I'm saying that this, the, it's not a question of whether Ilya 30:09.004 --> 30:12.704 The question of, uh, Ilya has an evidence point 30:12.964 --> 30:15.153 may or may not update you on which part of the S-curve we are on. 30:15.284 --> 30:18.784 [speaker_0] Yeah. So for that, I need to first even understand does Ilya 30:18.824 --> 30:19.934 or does he have some major disagreement? 30:19.984 --> 30:23.924 [speaker_1] No, he doesn't. He doesn't. He, his timelines are, uh, 30:24.880 --> 30:27.429 [speaker_0] So then want to know what his timelines are. 30:27.900 --> 30:31.630 [speaker_1] I think he said the next... I might have to check on this, but 30:31.740 --> 30:35.640 Darkish episode, he says that like six to eight years, uh, 30:35.680 --> 30:37.180 and then we'll find something that we can scale. 30:38.100 --> 30:41.740 And, uh, there are no good scaling candidates in 30:42.200 --> 30:45.840 [speaker_0] Yeah, no, to continue this, like, I think, like, 30:45.960 --> 30:47.800 more... Like, are you sure about this? You know. 30:47.840 --> 30:48.120 [speaker_1] Yeah. 30:48.180 --> 30:49.120 [speaker_0] Give me more context and- 30:49.520 --> 30:52.860 [speaker_1] No, no, I'm saying, I'm saying, I'm saying that, 30:52.980 --> 30:55.180 S-curve we are on, uh, there's uncertainty on it. 30:55.740 --> 30:58.180 Uh, maybe you're right and we are on the vertical part of the S-curve. 30:58.580 --> 31:02.300 I think if you just plot it from AlexNet, it's mostly the same 31:02.340 --> 31:05.940 paradigm. Uh, and I would count AlexNet 31:06.660 --> 31:10.600 as one breakthrough, Transformers as another breakthrough, 31:10.640 --> 31:11.000 breakthrough. 31:11.980 --> 31:12.930 But apart from that, 31:13.760 --> 31:14.790 I don't see why- 31:15.160 --> 31:17.910 [speaker_0] Yes, once, uh, no, but there are also like minor ones 31:18.120 --> 31:21.800 [speaker_1] ... like, I don't know any other paradigm that 31:21.840 --> 31:25.220 Okay, I won't even count AlexNet 'cause AlexNet 31:25.280 --> 31:29.180 works. Uh, I would just count Transformers 31:29.240 --> 31:31.640 that, uh, show that scale is all you need. 31:32.160 --> 31:35.910 Uh, and then there's n- there's not been a third candidate for 31:35.960 --> 31:39.180 need, or a third S-curve that you can stack on top of these things. 31:39.560 --> 31:41.550 Pre-training was one S-curve. We exhausted it. 31:41.600 --> 31:45.520 [speaker_0] There are multiple points. One is like proof that Transformers 31:45.600 --> 31:49.520 all, and then there's a second data point 31:49.640 --> 31:50.570 Or even with, like- 31:50.620 --> 31:53.010 [speaker_1] Transformers are useful at all is... 31:53.050 --> 31:55.360 Transformers are only useful because they can scale. 31:55.740 --> 31:58.110 [speaker_0] No. Uh, GPT-2 was- 31:58.120 --> 31:58.420 [speaker_1] Because- 31:58.640 --> 32:02.060 [speaker_0] ... useful. Like, it was a breakthrough by itself, even 32:02.100 --> 32:04.370 anything about whether GPT-2 would scale or not. 32:04.420 --> 32:08.140 [speaker_1] If it doesn't, then... No, what I'm talking about is that if you 32:08.200 --> 32:11.800 assume that scaling is what gets you smarter models, uh, 32:12.120 --> 32:15.540 subscribe to that worldview, then you need paradigms that can scale. 32:15.790 --> 32:17.330 Pre-training is one paradigm that can scale. 32:17.560 --> 32:21.520 [speaker_0] Right now. At the time GPT-2 was invented, 32:21.700 --> 32:23.360 ML research community didn't believe it. 32:23.420 --> 32:26.790 [speaker_1] Sure. And I, like, I'm saying that it's, that's, that's 32:27.320 --> 32:28.880 irrespective, like, that's immaterial. 32:28.940 --> 32:32.769 What I'm saying right now is if you believe that intelligence comes 32:32.780 --> 32:33.570 scaling things up- 32:34.180 --> 32:34.460 [speaker_0] Sure 32:34.540 --> 32:38.220 [speaker_1] ... then scaling a paradigm up, 32:38.280 --> 32:38.800 scaling up. 32:39.460 --> 32:39.640 [speaker_0] Yeah. 32:39.700 --> 32:43.620 [speaker_1] In doing so, uh, we have found only two paradigms that have 32:43.700 --> 32:44.060 scaled up. 32:44.180 --> 32:47.300 [speaker_0] We have found only two paradigms that have scaled up. 32:47.360 --> 32:51.120 No, I mean, why does, uh, the whole scaling fully connected 32:51.180 --> 32:54.440 networks back in twenty twelve, twenty thirteen, why does that not count? 32:54.480 --> 32:57.420 Why does scaling LSTM not count? I, I'm not clear. 32:57.620 --> 33:00.000 [speaker_1] Because LSTM didn't scale. CNNs didn't scale. 33:00.820 --> 33:02.920 [speaker_0] What do you mean? CNNs do scale. 33:03.220 --> 33:06.540 [speaker_1] As in, like, you get diminishing returns from scaling 33:07.060 --> 33:08.950 CNNs, for example, cannot- 33:09.320 --> 33:09.330 [speaker_0] Yeah. 33:09.340 --> 33:12.020 [speaker_1] People try to do language experiments on CNN, 33:12.080 --> 33:15.490 People try to do language experiments on LSTM, they perform to a certain level, but 33:15.520 --> 33:17.500 essentially then they start degrading. 33:18.020 --> 33:21.420 So th- these, these paradigms have lots of 33:21.580 --> 33:22.900 limitations on how much they can scale. 33:23.540 --> 33:27.520 [speaker_0] Sure. So they, they did scale up to some amount, 33:27.560 --> 33:29.980 saturated, I guess. Same. Like you had some estimate, I guess. 33:30.020 --> 33:33.440 [speaker_1] So then, then it's a failed candidate. Yeah, 33:33.500 --> 33:37.440 Like, like a good candidate is that, hey, it doesn't matter how much compute 33:37.460 --> 33:39.220 we throw at it, just keep scaling. 33:39.720 --> 33:41.880 [speaker_0] No, no, it means that back then it succeeded, right? 33:42.140 --> 33:45.640 CNNs, we did scale up to some amount, then we realized we're getting diminishing 33:45.680 --> 33:46.200 returns. 33:46.280 --> 33:47.300 [speaker_1] No, but that's what I'm saying, like, 33:48.320 --> 33:50.640 that way, like... No, no, that didn't... 33:50.740 --> 33:54.180 I don't know what your definition of success is, 33:54.220 --> 33:58.060 to superintelligence. Candidates that can get us to superintelligence are, 33:58.140 --> 33:59.940 are candidates that you can throw. 33:59.980 --> 34:03.770 There is no limit to how much, uh, or there's no visible limit to 34:03.880 --> 34:05.880 how much compute you can throw at it. 34:05.920 --> 34:09.580 For example, if tomorrow we find out that pre-training has stopped scaling, 34:09.600 --> 34:11.089 call pre-training a failed candidate. 34:11.400 --> 34:15.240 Currently, we have two candidates that can absorb insane amounts of compute 34:15.320 --> 34:17.800 et cetera, and can keep expecting gains from it. 34:18.400 --> 34:22.010 [speaker_0] No, even the paradigm that gets us to superintelligence might 34:22.040 --> 34:24.240 saturate somewhere. E-e-every candidate- 34:24.250 --> 34:24.560 [speaker_1] Sure 34:24.580 --> 34:25.080 [speaker_0] ... can saturate. 34:25.220 --> 34:27.620 [speaker_1] They would, can saturate, but I'm saying... 34:27.820 --> 34:31.400 A-and at that point, essentially, uh, you can either draw a line and say 34:31.500 --> 34:35.060 intelligence is good enough. Assuming that, okay, so you're in a, 34:35.140 --> 34:37.680 where you do not have enough, uh... 34:37.700 --> 34:40.500 You've not achieved the level of, or like, not achieved, that you don't want to 34:40.540 --> 34:44.400 achieve. But I'm saying like, uh, you think that you can still throw more 34:44.439 --> 34:46.800 at this and get more intelligence out of this. 34:47.380 --> 34:51.180 Uh, and, uh, in that paradigm, there are only two things that can 34:51.240 --> 34:53.760 absorb seemingly infinite amounts of compute. 34:54.540 --> 34:55.670 Uh, and there's not been a third. 34:55.670 --> 34:57.680 [speaker_0] I have a problem with your infinite thing. 34:57.820 --> 34:58.160 It's- 34:59.500 --> 35:02.560 [speaker_1] Seemingly infinite in the sense that, sure, 35:02.600 --> 35:04.380 how much you can do it, or there might be like some... 35:04.560 --> 35:07.240 There are obviously practical limits to it, but there might be a theoretical limit 35:07.300 --> 35:10.860 as well. And I'm saying that maybe before we get there, uh, 35:11.140 --> 35:14.060 superintelligence comes and then you, you've achieved your goal 35:14.100 --> 35:14.700 to scale further. 35:15.720 --> 35:18.240 But until now, there's no evidence to see that there 35:18.640 --> 35:21.580 [speaker_0] Sorry, I'm still not super clear what your claim 35:21.620 --> 35:23.500 Can you like summarize this entire argument? 35:23.580 --> 35:26.540 [speaker_1] Maybe I'll, maybe I'll try to explain it in a 35:27.280 --> 35:31.200 A successful paradigm in my book is one that can 35:31.260 --> 35:35.009 absorb all the compute capacity that you can reasonably throw at it at 35:35.120 --> 35:35.680 point in time. 35:36.180 --> 35:39.790 [speaker_0] That point in time means what? In that year, how many GPUs had 35:39.860 --> 35:41.420 humanity manufactured? 35:41.440 --> 35:45.140 [speaker_1] So it's also... Yeah, in that point in time, 35:45.200 --> 35:49.180 So, so in twenty twenty-six, uh, there are two successful paradigms that can 35:49.260 --> 35:51.380 absorb all the compute and still keep on giving gain. 35:51.700 --> 35:53.420 We have not exhausted these two paradigms. 35:53.870 --> 35:53.870 [speaker_0] Right. 35:53.880 --> 35:57.500 [speaker_1] And we still have like returns to get from there, 35:57.540 --> 36:01.340 essentially get us to smarter models. There are just two of them. And twenty... 36:01.680 --> 36:05.600 And so I'm saying like you keep extrapolating it, our, our, our lim-- our, 36:05.760 --> 36:09.460 thing. So in twenty twenty-- by the time we get to twenty thirty, 36:09.500 --> 36:12.540 that these two paradigms will not get to superintelligence because there are 36:12.580 --> 36:15.900 practical limits to scaling them, if not theoretical limits. 36:16.020 --> 36:19.940 Uh, and, uh, we need either a third paradigm 36:19.960 --> 36:23.860 seventh paradigm to actually keep stacking these S-curves to get us 36:23.920 --> 36:26.639 to the world that you are claiming we'll be in, say, 36:27.000 --> 36:29.820 [speaker_0] Uh, okay. Sure. My probability 36:29.940 --> 36:33.772 ofPre-training scaling plus RL scaling gets us 36:33.832 --> 36:37.592 to ASI by 2030 is less, let's say, less than 36:37.632 --> 36:38.092 10%. 36:38.232 --> 36:41.372 [speaker_1] Yeah, like probably we do need a better, uh, 36:41.852 --> 36:44.292 breakthrough. Yeah. Probably we do need another breakthrough. 36:44.512 --> 36:45.292 I am saying that 36:46.691 --> 36:50.112 these breakthroughs are not super easy to come by. 36:50.172 --> 36:52.992 Dep- I think we d- just did the S curves debate. But yeah, 36:53.012 --> 36:56.672 The one crux we have is that I am very uncertain of 36:57.092 --> 37:00.051 you have somehow. You are somehow more certain in 37:00.092 --> 37:03.512 have, and this is just a prior belief thing. 37:03.632 --> 37:06.212 I don't know if there's, like, any evidence that you can show me. 37:06.532 --> 37:07.122 [speaker_0] Yeah, I think there are- 37:07.132 --> 37:07.901 [speaker_1] And maybe I don't know 37:07.901 --> 37:09.112 [speaker_0] ... S curves we are tracking. One is like, 37:10.092 --> 37:13.972 uh, curves for individual paradigms, and one is like some bigger curve 37:14.012 --> 37:17.012 of, you know, like humanity's ML research as a whole. 37:17.572 --> 37:21.212 So there, one is like the curve of pre-training 37:21.312 --> 37:25.292 started saturating. There's a curve for 37:25.392 --> 37:26.892 scale, when did they start saturating. 37:27.492 --> 37:31.042 There's a curve for, you know, when did RL scaling start, 37:31.092 --> 37:34.232 might saturate someday. And then there are multiple of these curves, but then 37:34.252 --> 37:38.032 there's a bigger overall trajectory of how fast is humanity's ML 37:38.092 --> 37:38.792 capability growing. 37:39.192 --> 37:42.852 [speaker_1] Uh, that I don't agree with because 37:42.932 --> 37:46.112 it. For example, if you start tracking it, there have been multiple AI winters and 37:46.152 --> 37:49.932 AI summers. There were times where people thought 37:50.412 --> 37:54.292 in AI, and if we just do work on GOFAI or we just do work on, 37:54.332 --> 37:57.082 like, some other paradigm, we'll be able to get to ASI from here. 37:57.112 --> 37:59.472 [speaker_0] Why aren't we including all of that as data points? 37:59.992 --> 38:03.752 [speaker_1] So if you keep including them, then essentially, uh, this could 38:03.812 --> 38:07.752 either be like a, a 1990s rush in 38:07.832 --> 38:10.142 AI of, like saying that, "Hey, we have super intelligence. 38:10.272 --> 38:14.162 We have chess playing AI, and we are, like, a few research ideas away from 38:14.252 --> 38:18.242 having, like, super intelligence AI because we have Deep 38:18.432 --> 38:21.312 we'll be like, "No, that, that S curve actually flatlined 38:21.352 --> 38:24.742 super intelligence." And then you have a larger S curve of, 38:25.072 --> 38:28.232 um, and, uh, transformers and R- RL. 38:28.692 --> 38:32.312 And that S curve, depending on where we are on the S curve, 38:32.372 --> 38:36.172 still saturate before we get to, uh, super intelligence. 38:36.912 --> 38:40.602 Uh, and, uh, either that could happen, that is one world that could happen, 38:40.772 --> 38:44.732 or another S curve gets stacked on top of it, and it keeps going till 38:44.772 --> 38:48.732 we reach super intelligence. Uh, so, uh, or maybe, like, how close we 38:48.752 --> 38:52.152 are to super intelligence, essentially whatever that bar is, 38:52.192 --> 38:55.662 there or we need to stack more S curves to basically get us there 38:55.692 --> 38:59.352 faster. Uh, I do agree that on infinite human 38:59.372 --> 39:01.432 timescale, we'll get to super intelligence at some 39:01.932 --> 39:04.772 Uh, but yeah, 2030 is the timelines that we're dealing with. 39:04.792 --> 39:06.832 [speaker_0] Again, I didn't understand your argument. 39:06.972 --> 39:10.952 If I track, you know, human- humanity's AI research progress since, 39:11.032 --> 39:14.812 1970, yes, there have been multiple spans of few 39:14.852 --> 39:17.912 years where we did get, you know, some one breakthrough, something happened, 39:17.952 --> 39:19.952 then we had, like, you know, 20 years of nothing happening. 39:20.032 --> 39:21.542 Yes, there have been multiple of these. 39:22.312 --> 39:24.912 Uh, right now we could be in either one of 39:24.952 --> 39:28.852 We might be about to reach an AI winter or we might be 39:28.912 --> 39:30.792 about... Yeah, we might get a few more breakthroughs. 39:30.812 --> 39:33.972 Those again might not, might or might not get to super intelligence. 39:34.252 --> 39:38.052 Sure, uh, where are we in this? So far I'm agreeing it's now a question 39:38.132 --> 39:40.252 how do you put the numbers on these things and... 39:40.961 --> 39:43.712 [speaker_1] The, the quest- the disagreement comes from the fact that, 39:44.192 --> 39:48.042 It's not a disagreement in the argument, it's a disagreement on how many 39:48.192 --> 39:51.792 S, if a new S curve is needed or will these S curves scale to super 39:51.832 --> 39:54.452 intelligence and how easy are these S curves to come by. 39:54.892 --> 39:58.352 [speaker_0] So what would be a data point that would actually change 39:58.752 --> 40:02.592 your mind? Like, for me, it's, like, fairly, yeah, almost obvious 40:02.692 --> 40:06.482 that, okay, yeah, there is a b- huge backlog of, backlog of research 40:06.572 --> 40:09.372 ideas that need to be, that will definitely try- 40:09.432 --> 40:09.852 [speaker_1] That I think- 40:10.332 --> 40:11.152 [speaker_0] And all of them require- 40:11.222 --> 40:11.222 [speaker_1] Yeah, but- 40:11.222 --> 40:13.392 [speaker_0] ... a lot of people to try and, yeah. 40:13.992 --> 40:17.472 [speaker_1] Sure. I think sure there might be some research 40:17.532 --> 40:21.472 overhang, but, uh, the probability of us 40:21.512 --> 40:24.692 finding a breakthrough in the research ideas might be below, uh... 40:24.772 --> 40:27.632 I think the ML research community is very, very smart. 40:28.092 --> 40:32.052 Uh, they figure out all the best candidates 40:32.102 --> 40:35.392 a daily basis, et cetera. And in the last five, six years- 40:35.512 --> 40:38.232 [speaker_0] Yeah, I think that's where, yeah, like, 40:38.292 --> 40:41.702 smart. It comes down to try, hit and trial random shit until it works. 40:42.042 --> 40:45.842 [laughs] Like, I don't think people had, like, some, uh, brilliant insight, 40:45.862 --> 40:48.661 "Okay, this is why this thing is definitely going to work," 40:48.672 --> 40:50.982 tried it a ton of time. I think people just, like- 40:51.031 --> 40:51.362 [speaker_1] No, they did 40:51.452 --> 40:53.672 [speaker_0] ... tried the random 20 random things to try 40:53.982 --> 40:56.672 [speaker_1] If you look to-- No, but I don't think it 40:56.732 --> 41:00.512 I think the people who try it have some intuition of, uh, why 41:00.572 --> 41:03.852 this could work or why this wouldn't work, and they might be wrong or right and, 41:03.892 --> 41:05.731 and the outcomes might look random, et cetera. 41:06.032 --> 41:09.732 But the selection of which experiments to try definitely has 41:09.752 --> 41:13.352 A good AI researcher is one that tries more successful experiments 41:13.932 --> 41:17.732 And a bad researcher is they keep making bad bets on research 41:17.772 --> 41:18.692 keep failing at it. 41:19.492 --> 41:23.372 And, uh, my, the, my, my, the data point that 41:23.452 --> 41:25.902 I want to look at is that despite so many, 41:26.812 --> 41:30.692 so much money flowing into, like, finding research ideas, I'm sure 41:30.752 --> 41:32.612 we'll be able to scale this further. 41:33.012 --> 41:36.992 But is there an RL level or a pre-training level idea, uh, 41:37.092 --> 41:41.012 already out there that has not been tried yet because of 41:41.052 --> 41:44.952 compute? Because people, like researchers are literally drawing 41:45.032 --> 41:48.912 whatever, um, a hat and implementing the idea as opposed to 41:48.952 --> 41:51.672 reading the idea, understanding the viability of it working 41:52.212 --> 41:54.632 Uh, I think, like, like I believe in the second one. 41:54.992 --> 41:58.692 Uh, and if that is true, then, then where is the idea is my question. 41:59.172 --> 42:00.732 Uh, and just because you got two ideas- 42:00.762 --> 42:03.832 [speaker_0] Yeah, that question is definitely a crux like... Like yeah, 42:03.892 --> 42:06.932 On one side you have like researchers understand nothing about the problem and 42:06.952 --> 42:10.572 they're just brute forcing. On the other end of the spectrum you have, 42:10.632 --> 42:14.432 researcher deeply understands the thing and they have an hypothesis 42:14.512 --> 42:17.372 actually running the training run, they already know with confidence this is 42:17.412 --> 42:21.212 definitely going to work. And you are saying, okay, researchers 42:21.272 --> 42:24.162 end of understanding things. I'm saying they're far closer to the end 42:24.212 --> 42:25.152 randomly brute forcing. 42:25.612 --> 42:28.752 [speaker_1] I don't know. If you look, like, if you look at, like Noam, 42:28.832 --> 42:32.702 heard Noam, what's his name? Yeah, Noam Shazeer talk about, 42:32.792 --> 42:36.402 uh, when they were getting into transformers, 42:36.402 --> 42:39.692 transformers out, when they were essentially scaling language models, 42:39.752 --> 42:41.252 wrote the attention paper, et cetera. 42:41.732 --> 42:43.852 Uh, it wasn't like a random idea that they had come. 42:44.172 --> 42:48.132 Uh, Noam Shazeer has like this history of 42:48.252 --> 42:51.992 ideas to try out. He's called like a, a magical researcher 42:52.012 --> 42:55.532 because he can seemingly look at like 100 ideas and figure out, 42:55.572 --> 42:59.072 like these are the one, two. He has like crazy intuition of these 42:59.132 --> 43:02.092 ideas that could work because he understands these things much more deeply 43:02.172 --> 43:02.872 average researcher. 43:03.062 --> 43:03.212 [speaker_0] Okay. Oh. 43:03.292 --> 43:05.932 [speaker_1] And there are the superstar researchers that can look at ideas 43:05.972 --> 43:09.944 like breakthrough ideas much more quickly.And I don't think it's as 43:10.004 --> 43:12.684 random as that, "Hey, let me just pick one and do it, and 43:12.744 --> 43:14.204 Otherwise, I'll go pick another one tomorrow." 43:14.664 --> 43:18.564 [speaker_0] For me, I see it more as, yeah, like, yes, Noam Shazeer probably did 43:18.604 --> 43:21.984 have some intuitions, but also it was random that he 43:22.044 --> 43:24.244 How would I put it? There have been three 43:24.304 --> 43:26.584 Noam Shazeer directly contributed to one of them. 43:27.084 --> 43:30.644 If you take any of the other research breakthroughs which Noam Shazeer did not 43:30.704 --> 43:34.584 make, and you put him in one year before 43:34.624 --> 43:37.824 happened and told him, "Look, here are all these hypothesis the different 43:37.844 --> 43:41.444 researchers are making. Which one do you think will work?" I don't 43:41.484 --> 43:44.144 have made that good a guess and told you, "Oh, this one will work." 43:44.724 --> 43:48.424 [speaker_1] And I saw a Noam Shazeer, uh, and Jeff Dean 43:48.744 --> 43:49.024 talk 43:49.824 --> 43:53.544 about this exact thing, and from the story that they 43:54.064 --> 43:56.864 their meeting perspective and understand, et cetera. 43:57.224 --> 43:58.524 [speaker_0] Yeah, if what you are saying is correct- 43:58.614 --> 43:58.614 [speaker_1] [laughs] 43:58.614 --> 44:02.564 [speaker_0] ... there should be, like, the same researcher who's consistently 44:02.604 --> 44:05.414 where the field is heading multiple times and should be multiple times- 44:05.744 --> 44:05.954 [speaker_1] That's true 44:05.964 --> 44:09.644 [speaker_0] ... not 100% co- correct, but, like, roughly able to see, okay, 44:09.684 --> 44:12.464 are probably going to work and then actually roughly ends up correct. 44:13.064 --> 44:13.614 Whereas I'm saying no actually- 44:13.624 --> 44:14.844 [speaker_1] That's been true 44:14.944 --> 44:15.904 [speaker_0] ... I'm saying no actually- 44:15.964 --> 44:18.744 [speaker_1] Because the amount of, the amount of breakthroughs have come 44:18.944 --> 44:21.854 No, the amount of breakthrough that have come from these superstar 44:21.884 --> 44:25.334 like, very, very high. Why is Ilya Sutskever around all big 44:25.384 --> 44:27.884 'Cause he has a crazy sense of which research ideas can work. 44:28.284 --> 44:30.304 Why is Noam Shazeer around all these big breakthroughs? 44:30.334 --> 44:32.364 'Cause he has a crazy idea of all these things to do. 44:32.404 --> 44:36.184 There's a random reason why a random ML PhD you've never heard of comes up with a 44:36.304 --> 44:38.964 crazy idea. It's mostly because you literally have to 44:39.784 --> 44:43.663 uh, what's that guy's name, who's the GPT-2 main, Alec 44:43.724 --> 44:47.524 Radford type level researcher. Apparently, Alec Radford has such a great 44:47.704 --> 44:51.464 sense of what could work. He's like, he literally used to 44:51.544 --> 44:55.434 do small experiments on Jupyter Notebooks, and 44:55.524 --> 44:59.184 he then once he got convinced that this could work, 44:59.244 --> 45:02.184 engineering to Greg Brockman or somebody who's like, "Yeah, 45:02.284 --> 45:05.964 Just keep scaling it up. No, I'm sure it will work." Uh, 45:06.104 --> 45:08.914 built so much intuition about, like, and what couldn't work 45:09.024 --> 45:12.804 this crazy ML whisperer guy who can just, like, look at the 45:12.884 --> 45:16.644 shape of the model and figure out, like, these are ideas worth pursuing, not worth 45:16.664 --> 45:19.813 pursuing. And if you look at, like, Thinking Machines Lab, which is, like, 45:19.884 --> 45:23.684 lab filled, filled with all these guys, John Schulman, uh, what's his name, Alec 45:23.724 --> 45:27.584 Radford, all the OG co-founder guys, they have essentially had, like, 45:27.764 --> 45:31.384 free rein to do whatever. And the best they came up with 45:31.764 --> 45:35.664 that updated me towards that, oh, okay, there's, like, a lot of things to do here, 45:35.704 --> 45:39.634 but there is no crazy research breakthrough paradigm that, that 45:39.764 --> 45:43.444 oh, we got, like, a pre-training level paradigm 45:43.524 --> 45:46.364 stack on pre-training and get, like, insane results. 45:46.404 --> 45:50.304 [speaker_0] Yeah, I think I've identified a data point 45:50.384 --> 45:54.284 on this. If you, uh, again, from these, you know, three or four superstar 45:54.304 --> 45:58.104 researchers, if you're able to document a public track 45:58.264 --> 46:02.164 of, well, uh, yeah, since maybe 2018 46:02.244 --> 46:04.844 till 2026, some at least 10 years or no. 46:05.864 --> 46:09.084 Well, yeah, at least, yeah, like more than five, at least seven, 46:09.664 --> 46:13.624 consistent track record. Like, okay, here they made these predictions in 2018. 46:13.684 --> 46:15.644 They made these prediction 2020. They made these- 46:15.664 --> 46:16.604 [speaker_1] Dario is one guy 46:16.684 --> 46:18.813 [speaker_0] ... in 2022, and they made these in 2024. 46:19.004 --> 46:22.564 And, like, they didn't single-handedly do all the 46:22.604 --> 46:25.224 roughly able to see where the next breakthroughs are going to come from. 46:25.544 --> 46:29.124 If you can show me, okay, just the same guy 46:29.144 --> 46:32.264 trend, then that would actually shift my 46:32.944 --> 46:36.304 [speaker_1] I don't know. I think I 100% believe what you're saying 46:36.324 --> 46:40.164 true. Uh, I don't know how... Like, I'm, I'm trying to think of what are ways to 46:40.204 --> 46:44.084 show you this is happening. Uh, one way to do this would be that if you 46:44.124 --> 46:48.104 look at the large breakthroughs, and you look at who's responsible 46:48.124 --> 46:51.124 or who's close to those breakthroughs, it will seem like it's the same people. 46:51.684 --> 46:55.304 And that should update you towards that, hey, how come, uh, Ilya 46:55.313 --> 46:57.424 Sutskever is involved with all the big breakthroughs? 46:57.484 --> 47:01.024 How come it came from the same guy who did AlexNet, is the same guy who did 47:01.424 --> 47:05.144 GPT-2, is the same guy who did RL? Why, why is it the same guy who's doing all 47:05.164 --> 47:08.424 these other things? Why is Noam Shazeer building all of these 47:08.504 --> 47:10.474 Uh, why is Alec Radford building all these 47:10.844 --> 47:13.834 It's because they have figured out or, or they have impeccable... 47:14.124 --> 47:18.064 So they, they talk about impeccable research taste, 47:18.074 --> 47:21.714 taste is what is really hard. And research taste is this intuition that 47:21.724 --> 47:25.624 researchers have that can figure out from a pile of, like, 47:25.664 --> 47:28.374 one to worth trying from the compute we have to get the breakthrough. 47:28.784 --> 47:32.624 [speaker_0] Yeah, so literally what you said, 47:32.684 --> 47:36.664 here is Alec Radford's track record of research hypothesis going back 47:36.684 --> 47:37.404 entire eight years- 47:37.464 --> 47:41.064 [speaker_1] But why won't you just buy Ilya's track record? 47:41.204 --> 47:44.324 [speaker_0] Uh, I literally don't know enough about this 47:44.414 --> 47:47.204 What did Ilya say in 2018? What did he say in '19? 47:47.224 --> 47:48.134 [speaker_1] No, he doesn't say anything. 47:48.134 --> 47:48.474 [speaker_0] I'm literally not saying- 47:48.524 --> 47:52.084 [speaker_1] But basically the fact that he doesn't, 47:52.164 --> 47:56.104 Basically, if he doesn't have to publicly make any of 47:56.164 --> 47:59.924 there's a reason why all big breakthroughs are around one person, it 48:00.024 --> 48:03.184 stands to reason that this person picks better research 48:04.084 --> 48:06.294 person who's picking research ideas at a random, at random. 48:06.864 --> 48:07.294 [speaker_0] No, no, but- 48:07.484 --> 48:07.614 [speaker_1] Like- 48:07.844 --> 48:11.764 [speaker_0] Was he personally the one who did the breakthrough, 48:11.804 --> 48:14.024 to be at the lab where somebody else did the breakthrough? 48:14.424 --> 48:16.384 [speaker_1] No, no. He personally was overseeing research. 48:16.424 --> 48:19.224 He personally was green-lighting the experiments that he thinks would work. 48:19.584 --> 48:22.984 [speaker_0] Okay. Uh, okay, then we can take literally Ilya Sutskever 48:23.044 --> 48:26.944 example. Uh, which breakthroughs would you say, okay, 48:26.964 --> 48:30.694 is significantly responsible for versus which ones you think he just happened 48:30.744 --> 48:31.264 to be there? 48:31.274 --> 48:31.274 [speaker_1] Deep learning. 48:31.304 --> 48:31.484 [speaker_0] Sorry? 48:31.544 --> 48:34.664 [speaker_1] Deep learning, he was-- Deep learning, he 48:36.004 --> 48:38.124 [speaker_0] No, when you say deep learning, 48:39.024 --> 48:42.084 Okay, I'll explain it. Cool. Uh, yeah, I agree with you. Fine. 48:42.184 --> 48:45.824 Ilya was significantly responsible, a-along with other people, significantly 48:45.884 --> 48:47.104 responsible for AlexNet, sure. 48:47.164 --> 48:50.534 [speaker_1] GPT, the GPT ideas he was significantly responsible for. 48:51.944 --> 48:54.804 Just training transformers. GPT-1 also. 48:55.144 --> 48:58.724 The idea that we can essentially scale transformers or we can, we can find a 48:58.784 --> 49:01.944 scalable transformer paradigm and build language models from it. 49:02.704 --> 49:06.234 [speaker_0] Uh, okay. So which model was this? Was this GPT-1? 49:06.504 --> 49:08.504 [speaker_1] This was the generative pre-transformer paper. 49:08.844 --> 49:11.024 He is directly responsible there. 49:11.084 --> 49:13.894 I think if we look at the GPT-1 paper- 49:13.894 --> 49:17.064 [speaker_0] One second please. I know this, I know it's annoying to, 49:17.124 --> 49:20.544 things middle of video, but, like, I actually want to now go read 49:20.554 --> 49:20.664 is. 49:21.044 --> 49:23.504 [speaker_1] Sure. It's this paper. I will send it to you. 49:23.936 --> 49:27.046 [speaker_0] Oh, okay. It's in the chat. See, uh, when was this published? 49:27.416 --> 49:28.736 [speaker_1] Five years ago. Twenty twenty... 49:29.965 --> 49:31.956 No, earlier than that. June 2018. 49:32.436 --> 49:33.636 [speaker_0] Is it June? You sure? 49:34.176 --> 49:37.266 [speaker_1] This is the AI summary. Yeah. Yeah, June 2018. 49:37.616 --> 49:41.215 [speaker_0] Okay, fine. Take care. I will buy that. Okay, fine. 49:41.276 --> 49:44.596 Ilya has been there at two major breakthroughs. Fine. 49:44.946 --> 49:48.896 [speaker_1] Then he's also been there at, uh, this thing, uh, for 49:48.976 --> 49:52.696 the o1 breakthrough as well. Uh, Ilya didn't 49:52.736 --> 49:56.576 green-light the experiment. Ilya was heading research that time and 49:56.596 --> 49:59.536 green-lighted the o1 experiment. RL, basically RL scaling. 49:59.896 --> 50:02.446 And if you look at Dario-- Sorry. Sorry. 50:02.456 --> 50:03.216 [speaker_0] Dario, for this- 50:03.776 --> 50:07.036 [speaker_1] Uh, he was at OpenAI at that time. He was head of research. 50:07.416 --> 50:11.066 He was personally seeing all the AI research that was happening at OpenAI, and 50:11.176 --> 50:11.956 OpenAI came out- 50:12.066 --> 50:15.716 [speaker_0] No, but if he's head of research, 50:15.756 --> 50:19.616 works in his lab, even if he does not want to, you know, do that, 50:19.656 --> 50:21.676 hypothesis or, you know, prioritize it. 50:21.686 --> 50:22.946 [speaker_1] You don't have to suggest the hypothesis. 50:23.016 --> 50:26.356 I'm saying researchers are mostly the same 50:26.396 --> 50:28.716 experiments, the best researchers. 50:29.476 --> 50:31.686 The best researchers know which, which can control- 50:31.696 --> 50:35.076 [speaker_0] But like OpenAI tried 100 things. Whichever of, 50:35.116 --> 50:37.836 worked, he could take credit for it simply because 50:38.276 --> 50:42.146 [speaker_1] Sure, but if, if there are 10,000 things that OpenAI didn't try, 50:42.236 --> 50:45.916 if there are three big paradigm, three big breakthroughs 50:46.256 --> 50:50.036 AI, and the same guy has been around for all 50:50.196 --> 50:53.946 the set of the things he tried. It's the, the, the set is all the things that are 50:53.946 --> 50:57.796 out there that he didn't try, and out of which he was able to freak out 50:57.896 --> 50:59.316 or be around all three things. 50:59.676 --> 51:03.546 [speaker_0] No, there are ways to be around the guy who 51:03.596 --> 51:04.346 gives the correct hypothesis. 51:04.406 --> 51:08.136 [speaker_1] But he's not like Sam Altman, who was probably around it, but he 51:08.176 --> 51:09.455 committed to the research direction. 51:09.956 --> 51:10.236 Anyway- 51:10.376 --> 51:10.455 [speaker_0] Okay 51:10.476 --> 51:13.576 [speaker_1] ... this is by the way, like even if you don't strongly believe 51:14.076 --> 51:15.676 [speaker_0] No, no, this is very important. Like this part 51:15.716 --> 51:19.536 Like, uh, was he the one who personally selected the 51:19.566 --> 51:21.146 "Okay, this one is worth trying, we should try it"? 51:21.196 --> 51:21.916 [speaker_1] Yes. 51:21.996 --> 51:22.216 [speaker_0] Or- 51:22.296 --> 51:25.816 [speaker_1] Yes, I, I did the... I'll tell, 51:25.856 --> 51:29.516 But there is an interview of Dario Amodei 51:29.956 --> 51:33.636 when he was working with Ilya Sutskever, who I think it's Dario 51:33.676 --> 51:37.076 Amodei, who basically, uh, I don't know if it's that. 51:37.116 --> 51:40.006 Anyway, it was, I think, one of these interviews 51:40.036 --> 51:43.616 about Ilya Sutskever, and he's saying then he came, 51:43.716 --> 51:47.526 research direction, saying that we need to do X, 51:47.596 --> 51:51.216 we'll do this, and we'll do that. And Ilya Sutskever drew two circles, 51:51.856 --> 51:55.496 two concentric circles. Inside he-- In one he-- And this was pre o1. 51:55.856 --> 51:59.776 He wrote pre-training, and outside he wrote RL, and he said, 52:00.056 --> 52:02.566 [speaker_0] Okay. Uh, if you can send me this, that will help. 52:02.626 --> 52:05.516 [speaker_1] I'll find, to find that clip. I'll try to find that clip. 52:05.536 --> 52:08.706 And this was like much before o1, when I think Dario or whoever 52:08.816 --> 52:12.696 and, and the guy was like, "Okay, uh, it 52:12.736 --> 52:13.696 makes sense." Like, 52:14.616 --> 52:16.816 why am I complicating the research agenda that long? 52:17.156 --> 52:19.956 [speaker_0] Okay, sure. If you send me this, that will again, 52:19.996 --> 52:23.246 Like now you have given me three different data points, and Ilya was involved 52:23.296 --> 52:26.696 directly, like not just like, okay, researcher overseeing, but he 52:26.716 --> 52:29.186 picking the hypothesis and saying this will work. Yeah. 52:29.216 --> 52:32.196 [speaker_1] Yeah. Yeah. I think I can to find... One second. 52:32.616 --> 52:36.016 Let me just do a random cloud search to see if they can find the things. 52:36.436 --> 52:39.986 Last time I remembered, maybe we'll find, but I, I've seen it and, 52:40.576 --> 52:44.336 uh, provided this is true, uh, would you agree that research is 52:44.396 --> 52:46.436 not as random as picking ideas from a hat? 52:46.736 --> 52:50.016 [speaker_0] Uh, yeah. If you show me that, yeah, now three different 52:50.056 --> 52:53.766 breakthroughs, uh, Ilya Sutskever personally was helping 52:53.876 --> 52:57.696 pick the hypothesis rather than just happen to be in the same room or 52:57.736 --> 53:01.416 overseeing the same lab. If you show this across three 53:01.556 --> 53:05.545 that would tell me that there is something spec- some specific way 53:05.616 --> 53:08.736 Ilya Sutskever personally looks at this problem, which almost nobody else in the 53:08.776 --> 53:10.156 world has. Yeah. 53:10.196 --> 53:13.646 [speaker_1] It's time to find out which interview. I think it 53:14.116 --> 53:18.036 Anyway, cool. Uh, I think are there a couple of other things 53:18.076 --> 53:21.976 that I think I disagreed with. Uh, so one was that research direction 53:22.116 --> 53:25.596 there might not be as many low-hanging fruits as you think there are. 53:25.716 --> 53:29.316 Uh, so 2030 might be in this thing. That was one crux he identified. 53:29.376 --> 53:33.336 The other one was that, uh, intelligence is easier to build than 53:33.476 --> 53:37.336 I thought. This is something that I think I've changed my mind on since 53:37.396 --> 53:41.206 Ooty, uh, like since he last spoke, uh, about this. 53:41.266 --> 53:45.076 It's basically the, if you've, if you saw the Richard Sutton 53:45.516 --> 53:46.656 Dwarkesh interview, 53:47.516 --> 53:48.146 uh, TCS- 53:48.296 --> 53:49.236 [speaker_0] Yes. 53:49.576 --> 53:51.476 [speaker_1] Do you remember what they spoke about? 53:51.556 --> 53:55.256 [speaker_0] I think Richard Sutton's timelines were also something 25% by 2030, 53:55.316 --> 53:56.296 remember correctly. 53:56.476 --> 53:58.896 [speaker_1] No. He thinks it's more LL by LL- 53:59.876 --> 54:03.436 [speaker_0] I am pretty confident Sutton had something like next five to 10 years 54:03.456 --> 54:05.266 chance ASI. But yeah- 54:05.516 --> 54:05.956 [speaker_1] But he thinks- 54:05.996 --> 54:08.426 [speaker_0] I can't quite remember. Actually, you know why- 54:08.536 --> 54:10.886 [speaker_1] But, but I also think that this whole idea that- 54:11.536 --> 54:14.256 [speaker_0] Ilya's timelines, what are his... 54:14.676 --> 54:15.366 Give me a minute. 54:15.936 --> 54:19.656 [speaker_1] Yeah, and I also don't think this whole idea 54:19.716 --> 54:23.136 agrees with you on timelines, it doesn't matter. Any- 54:23.236 --> 54:27.066 Like it does matter what shape of beliefs he has and why he agrees to those 54:27.136 --> 54:29.596 timelines and what shape of beliefs you have and why you agree to the timelines. 54:29.676 --> 54:31.476 Uh, there might be a fundamental disagreement 54:31.536 --> 54:34.516 You could update from some of his beliefs, not all of his beliefs, even though his 54:34.536 --> 54:35.436 conclusion are the same. 54:35.996 --> 54:39.796 [speaker_0] No, uh, like I initially started from a worldview 54:39.876 --> 54:40.096 of 54:41.116 --> 54:44.936 like pick even among the genius researchers, picking which research 54:44.976 --> 54:48.916 hypothesis works is kind of random, and it requires just a lot of hit 54:48.936 --> 54:51.166 and trial, and none of these people really know. 54:51.216 --> 54:55.146 You are trying to update me more towards a worldview of, no, there are a few 54:55.156 --> 54:59.036 genius researchers here who consistently seem to get all of the 54:59.076 --> 55:03.056 predictions right. And I'm like, let's say I did update to your 55:03.076 --> 55:06.616 worldview that all the more means I want to know, okay, what are these people's 55:06.656 --> 55:09.536 timelines? If now you're saying, okay, I should defer to these people now. 55:09.576 --> 55:10.346 [speaker_1] No, I'm sure. Go, go find out- 55:10.346 --> 55:13.386 [speaker_0] I literally need to know what does Ilya Sutskever 55:13.556 --> 55:14.156 Yeah, like- 55:14.316 --> 55:16.396 [speaker_1] Go find their timelines. That's not the point I was making. 55:16.456 --> 55:19.316 I was trying to make a point that you were saying that just because Ilya has 55:19.356 --> 55:23.096 timelines and you have short timelines, it doesn't matter, uh, 55:23.176 --> 55:27.116 what Ilya's arguments on research direction is or Ilya's time, like 55:27.156 --> 55:29.976 Ilya's stance is on why and how we get this breakthrough. 55:30.176 --> 55:31.116 [speaker_0] Those also matter. I agree. 55:31.416 --> 55:33.826 [speaker_1] Okay. So it-- So cool. Find Sutton's timelines. 55:33.916 --> 55:37.656 But the point Sutton made was that, uh, evolution actually gave us 55:37.716 --> 55:41.436 language very, very lateUh, and most of evolution was 55:41.476 --> 55:45.306 trying to optimize for things that we take for granted, which 55:45.916 --> 55:49.396 uh, whatever, like being physical dexterity, et cetera. 55:49.416 --> 55:53.216 And, uh, those things essentially, which according to you are part of 55:53.256 --> 55:57.056 intelligence, those things actually took a, like, 55:57.136 --> 56:00.356 time. Uh, and, uh, those would be very hard to do. 56:00.796 --> 56:04.676 Uh, the second thing that he says that, uh, he thinks that if you can get 56:05.416 --> 56:08.596 good at doing those parts, then everything else falls into place. 56:09.156 --> 56:13.076 Uh, but, uh, the argument is if you can get to language, uh, 56:13.176 --> 56:15.076 then getting to the physical stuff is easy. 56:15.236 --> 56:19.096 And he's like, "No, most of it is getting to the physical stuff." And 56:19.156 --> 56:22.516 then this language part and all of these part is something 56:22.696 --> 56:26.356 [speaker_0] Sorry, I want to interrupt you because I feel 56:26.436 --> 56:29.976 points we can both go check, and if we check, 56:30.026 --> 56:33.696 our debate more productive. Data point number one of, you know, Ilya 56:33.736 --> 56:36.726 Sutskever being personally present at all these three breakthroughs, I think you've 56:36.996 --> 56:40.856 proved it for AlexNet, I agree. For the GPT-1 thing, 56:40.896 --> 56:43.276 also kind of agree. More data would help, but I kind of agree. 56:43.336 --> 56:46.116 The o1 thing, I'm not yet convinced Ilya 56:46.156 --> 56:49.416 If you show me data, I can be convinced. Uh, that's one data point. 56:49.536 --> 56:49.686 [speaker_1] Yeah. I think I found it. 56:49.686 --> 56:53.426 [speaker_0] The other data point I want is, uh, yeah, literally what 56:53.476 --> 56:57.396 What are Sutskever's timelines? Uh, and I'm saying, like, let's 56:57.416 --> 57:00.056 first get these data points and then let's continue the discussion. 57:00.256 --> 57:00.646 I think that- 57:00.716 --> 57:02.375 [speaker_1] Found it. I think I found it. 57:02.396 --> 57:06.156 [speaker_2] The people who are most responsible for that 57:06.196 --> 57:10.116 and Jakub Pachocki. I think even like, uh, like, 57:10.336 --> 57:11.746 uh, Dota was kind of- 57:11.746 --> 57:15.056 [speaker_1] We can go like a few seconds before because he's talking about o1, 57:15.416 --> 57:16.756 verify that he's talking about o1. 57:17.116 --> 57:17.396 [speaker_0] And 57:18.216 --> 57:22.036 okay, uh, I might agree with you by the way, 57:22.116 --> 57:25.756 note, I'm like, why has this not been documented 57:25.896 --> 57:29.296 there for all the three breakthroughs? That seems like a very big deal. 57:29.896 --> 57:30.976 [speaker_1] What do you mean it's not documented? 57:31.656 --> 57:35.516 [speaker_0] Why isn't there like either a Hacker News post or a Lesswrong post 57:35.576 --> 57:37.936 saying here is the evidence Ilya was there at all three breakthroughs? 57:38.276 --> 57:41.976 [speaker_1] But that seems, I think it's common knowledge. 57:42.256 --> 57:45.596 I'm surprised you didn't, and I'm surprised you didn't 57:45.996 --> 57:48.876 see this, but basically it's been talked about in a 57:49.356 --> 57:52.965 Everyone knows that Alex Radford, Ilya Sutskever, Noam Shazeer are these 57:53.036 --> 57:56.676 like insane superstar researchers who whatever they touch, whatever 57:56.756 --> 57:59.206 ideas they pick turn out to be the right candidates always. 57:59.616 --> 58:00.805 There's another one where Jeff Dean and Noam Shazeer- 58:00.805 --> 58:03.836 [speaker_0] Okay. It's definitely not common knowledge, 58:03.856 --> 58:05.856 to actually write this up and update a bunch of people. 58:05.916 --> 58:09.766 Like, I think I'm not the only one for whom this 58:09.766 --> 58:11.076 from you this. Yeah. 58:11.786 --> 58:15.626 [speaker_1] Yeah. But, uh, okay, cool. Uh, I think I've-- The clip 58:15.636 --> 58:19.046 something Claude found out. Uh, this is not the clip that I was talking about 58:19.056 --> 58:22.926 originally, but I think this is even tighter evidence than what I 58:23.896 --> 58:27.196 because he's directly saying that the reasoning breakthrough came from 58:27.216 --> 58:28.886 Pachocki and Ilya Sutskever. 58:29.176 --> 58:33.116 [speaker_0] Okay. Yeah. I mean, I will need a few minutes to properly go 58:33.176 --> 58:34.056 okay, fine. 58:34.196 --> 58:34.676 [speaker_1] No stress, no stress. 58:34.716 --> 58:38.496 [speaker_0] Like for now I can maybe buy it. Okay. Let's, uh, buy this. 58:38.556 --> 58:42.076 Okay, cool. Yeah. Also then I want to know, yeah, what 58:42.136 --> 58:45.446 timelines then if you know that. Uh, any of the people you mentioned- 58:45.446 --> 58:45.886 [speaker_1] We can just ask chat 58:45.896 --> 58:48.666 [speaker_0] ... Alex Radford or Noam Shazeer or Ilya Sutskever. Yeah. 58:48.685 --> 58:52.546 [speaker_1] Yeah. This is one post I found on EA forum, uh, 58:52.596 --> 58:56.166 run by Lessig, uh, where they're talking about Sutskever's-- 58:56.216 --> 58:59.366 criticizing Sutskever for not having transparency on his timelines and 59:00.076 --> 59:00.976 not saying why. 59:02.596 --> 59:06.456 [speaker_0] I think you have successfully updated me a bit, well, 59:07.056 --> 59:10.986 towards the idea that Ilya Sutskever personally, like not 59:11.016 --> 59:14.376 even any of the other AI researchers in this space, but Ilya Sutskever 59:14.516 --> 59:18.376 specifically has like great research taste and he's consistently able 59:18.456 --> 59:20.076 to pick good research hypothesis. 59:20.116 --> 59:20.656 [speaker_1] Good. 59:20.756 --> 59:24.556 [speaker_0] Not yet shifted my timelines by a lot. I think that's where I'm at. 59:26.736 --> 59:26.745 [laughs] 59:26.776 --> 59:26.846 [speaker_1] Yeah. I have given you- 59:26.876 --> 59:30.776 [speaker_0] Because yeah, again, like it's not obvious to me, okay, 59:30.836 --> 59:34.696 if you told me, "Okay, update towards Ilya 59:34.736 --> 59:38.206 than you," then sure. But are Ilya's timelines 59:38.236 --> 59:38.965 [speaker_1] Ilya's timelines are-- 59:40.676 --> 59:43.856 His minimum is still longer than your maximum. 59:44.576 --> 59:47.976 Anyway, maybe not. I don't know. And I think his definition of 59:47.996 --> 59:48.776 might be very different. 59:49.876 --> 59:53.726 [speaker_0] Fair. So yeah, now we will have to go more into 59:54.356 --> 59:54.796 Uh, okay. 59:54.856 --> 59:55.876 [speaker_1] We can end the recording there. 59:56.356 --> 59:57.356 [speaker_0] Okay. Uh. 59:57.396 --> 01:00:00.876 [outro music]