Why Vlad Tenev and Tudor Achim of Harmonic Think AI Is About to Change Math—and Why It Matters

[00:00] Last I checked, there's a 43% chance that the next Millennium Prize will be solved by AI or like human with significant AI assist. I think that's an underestimate. [00:11] I mean, we could be lucky and Larry Guth might be on the path to the Riemann hypothesis, which would be amazing. But I think that... [00:20] you know, [00:21] If the next one... [00:23] is solved by a human, it would probably have to be in the very near future. And for sure, the the next next one will will probably be significantly solved by AI. [00:35] *music* [00:50] *Bell rings* [00:52] We're excited to welcome Vlad and Tudor to the show. [00:55] We've had the pleasure of knowing Vlad for many years at Sequoia, but what many of you may not know about Vlad is that in addition to being founder and CEO of Robinhood, [01:02] He's also an enormously talented mathematician. [01:06] Vlad and Tudor have teamed up to create Harmonic, an AI research lab. [01:10] with the goal of pushing the frontiers of human knowledge. [01:12] Specifically, they hope to create mathematical superintelligence, [01:16] with the thesis that understanding math allows you to better understand and reason about much of the broader world. [01:22] We're excited to talk to Tudor and Vlad about Harmonic, [01:25] about the ingredients that go into creating mathematical superintelligence, including synthetic data,

[01:30] reinforcement learning and self-play. [01:33] And when AI will win the IMO or a Millennium Prize, or even solve the Riemann hypothesis. [01:38] All right, Vlad Tudor, welcome to the show. [01:42] Well, thanks for having us. All right. So you guys, you have this core belief. [01:46] that math is reasoning. And you have what might be a contrarian belief that the best way to train a model [01:54] to perform well in math is to directly teach it math versus allowing math to emerge as a property of scale, which is what a lot of the other foundation model companies are doing. Can you talk a bit about that core belief? [02:08] Why do you need to teach the model directly math? And also maybe what does it mean that math is reasoning? [02:14] uh, when we started the company, we, you know, had a really big focus on math and maybe we can get to that later, but, you know, if you look around at all fields of science and engineering, uh, [02:24] well, almost all fields. Math is really at their foundation. And math has essentially become the way that people understand the universe. It's the way you model phenomena from black holes to atoms. It's the way you design things in engineering. And [02:38] There's a couple of reasons for that. First of all, it just happens to be the case that the universe is explainable by math. So you can write down a fairly compact set of symbols that explain things. But the other really important thing is that [02:49] It's a way to build these, you know, shared cognitive theories that are very objective and clear and transparent. And if you and I are discussing something that's, you know, rigorous, we can write down a set of deductive rules, we can agree on what, you know, the ground rules are of whatever we're modeling, and then from there, we can derive a

[03:06] you know, conclusions. And so, [03:09] With humans, what you see is that when people become very good at math, they tend to be good at other quantitative areas in science and engineering. And so our bet is that, you know, if you make a system that's really good at math, you're probably going to see the same phenomenon where it's true. It might not immediately write the world's best history essays, but when you ask it to do something scientific or something in engineering, it's just going to be really, really good at that. [03:30] That's why we started with math. And where is that boundary? Between, you know, help me with my math homework and write a history essay, there is some boundary that it's hard for math to cross. What do you think the outer edges of what's possible with math? [03:45] a model with sort of math at its core, where are those outer edges? [03:49] I'll give you sort of the non-AI perspective. So I studied math and, you know, I was really good at math from when I was a little kid. And I remember there were always like the seventh graders in math class that would raise their hands whenever the teacher would... [04:06] come up with something and it was always like a abstract thing. Like, you know, the, [04:11] side angle side of triangles. There'd be the annoying kid that was like, [04:16] when are we ever going to use this? And, you know, the teacher would kind of mumble a little bit and they'd be like, well, you know, math, just like, you're probably not going to use it soon, but it'll make you really good at, at other things. And, you know, [04:30] The other kids were always skeptical of that. And I bought into it. Um, and, [04:35] And so I just kept taking more and more advanced math. You know, I went to Stanford and I majored in it. Then I went to grad school to do a math PhD. And my belief was that.

[04:47] "Okay, if I just focus on math, then I'm going to get really good at problem solving and business problems." [04:54] other problems will be easy compared to, [04:57] you know, [04:58] Solving these like really tough, like abstract algebra problem sets that I was banging my head against the wall for 10 hours every week trying to do. And I think it basically ended up being correct. [05:12] It's like I didn't really pay attention to anything else. I took maybe like... [05:17] one computer science class intro to, to computer programming at Stanford. And, um, I, [05:25] you know, five years later, uh, when I became an entrepreneur, uh, [05:29] Um, [05:30] I found it really easy to pick up code. I found it really easy to like... [05:34] pick up contracts and, [05:36] You know, of course, I'm no lawyer, but... [05:39] you could understand that stuff. I mean, the logical underpinnings are... [05:43] relatively simple compared to abstract algebra and analysis. So I think [05:50] For humans, at least, I consider myself an example of, like, math transferring to other very monetizable things. And I think for AI... [06:00] - Yeah. [06:01] my intuition seems to suggest that it should be the same. [06:05] And you already see a little bit of evidence of this. At this point, it's an open secret in the industry that code training on a lot of code data leads to much better performance on reasoning benchmarks. So you can imagine what that'll look like when you have incredible math data sets that encompass a lot more.

[06:19] general types of reasoning. Yeah. [06:21] Yeah, that resonates. The idea that math teaches a human how to think critically, how to think logically, and that skill can be ported to a bunch of other domains. It stands to reason that that would work in AI also. Vlad, you casually mentioned that you studied a little bit of math. And just for anybody who's not quite familiar with your background in math, I believe you studied briefly under Terry Tao, who is [06:47] perhaps the world's greatest [06:48] living mathematician. Yeah. And then one of the things you mentioned to us was that you, you still catch up with him every now and then you have lunch when you're in LA, that sort of thing. So I'm curious, when you have lunch with Terry Tao, what do you guys talk about? [07:03] Um, do you give me any, do you give me a stock tips? [07:07] No, no, I'm not allowed to do that. Yeah. One, one of the, uh, yeah. One of the unfortunate things of being a public company CEO in the financial spaces. I have lots of stock tips, but I can't share any of them. I have to like keep them in turn and I can't even use them. Uh, I basically can't trade anymore. Um, [07:27] which is unfortunate because I love trading. But, yeah, I – [07:33] So, um... [07:36] to backtrack how I got to UCLA, um, [07:39] So Terry Tao is a professor at UCLA, and I think what's really amazing about him is [07:47] the breadth of the work. So most mathematicians get like very deep into a pretty narrow domain. And Terry,

[07:55] can get very deep across... [07:58] like dozens of domains. You know, he's made contributions to number theory, combinatorics, harmonic analysis, applied math. He's one of the leading lean contributors at this point, I'm sure. He's like formalizing his papers in lean and actually hopping on the... [08:17] the community Zulip and, [08:19] Uh, [08:20] like engaging with students. Um, and then he has a very popular blog. And I think the way that he's been able to do this is he, [08:28] He's just smarter than [08:30] 99.999% of people, probably even more than that. So from a very early age, it was very clear that, that he was like on an, on another plane. And I studied, uh, I did my math honors thesis in undergrad, uh, under this professor, Larry Guth, who's also really amazing. I mean, the, um, he actually had a, [08:53] a recent result that came out that was a groundbreaking result. Um, uh, [09:00] in, I want to say, number theory or something about the Riemann hypothesis. But yeah, this result in non-AI math really was quite something. And... [09:15] You know, he... [09:17] he kind of suggested I look at UCLA. I was like really interested in, [09:22] in his field, and I ended up...

[09:25] I ended up going there and being fortunate to study under Professor Tao. But I should be clear, I am a dropout. [09:34] And it's amazing that I can claim that after grad school, but I will claim dropout status. So I only did one year of... [09:41] of UCLA, so it was a intro to graduate level analysis, [09:47] Terry Tao taught my first year. [09:50] which was pretty amazing. [09:52] And one thing I remember was I was doing some reading with Professor Tao and [10:01] He gave me this book and he signed it. And I think he signed it because he wanted to make sure I would give it back to him when I was done reading it. [10:11] Little did he know that by signing it, he guaranteed he would never get that book back. And I bring it up every time I see him. I'm like, hey, you're not getting that book back. It's on my shelf next to all my other autographed first editions. [10:25] What does the math community think of AI math? Are people split? Or do people think it's, you know, the Path to the Promised Land and the way that we're going to solve Riemann and everything else? I think it's split. I think it seems to be split. Okay. And... [10:40] There's sort of like the younger mathematicians, I think, are very like pro- [10:46] AI and pro verification and tools like lean and [10:51] I think the older folks are a little bit more skeptical. So not surprising. I think you see that in pretty much every field.

[10:59] I think that... [11:01] My... [11:03] My guess would be that this will evolve. My mental model is something like chess. [11:09] where [11:10] You know, at first, [11:12] there will be perhaps a lengthy period of... [11:15] you know, humans plus AI assist. And that will lead to a lot of really good results. But, [11:24] Over time, I think the AI will... [11:27] get better and better. And you look at chess right now, [11:30] and [11:31] It's sort of like... [11:33] if there's a human assisting the AI, the AI would be annoyed of it. They would just want to just delete all of the input because it would only make the results worse. So I'm not sure if we're going to get to that point. I think humans will... [11:48] at some point... [11:50] I mean, they'll need to guide-- [11:54] the algorithms, but I think the, [11:57] the kind of definition of what a mathematician will do will, will fundamentally change. I was talking to one of my friends who's a mathematician at MIT and I asked him, [12:08] when we were kind of first starting this. [12:12] What do you think? And this is a young professor, like, very excited about the field. I was like, are you worried that you're kind of in a field that is going to fundamentally change? He's like, the field of math has always changed. Back in the 1800s, mathematicians used to be kind of like in the royal court, and they would be glorified calculators. They would solve quadratic equations by hand. And, of course, they were worried that,

[12:41] you know, when computers and calculators came out, the job would no longer exist. But, uh, [12:47] mathematicians get to define what math is. So I'm sure at some point it'll be [12:53] prompting and kind of guiding these AIs to solve problems. And I think, um, [12:59] Yeah, I think that's going to be very huge. Even if an AI solves the Riemann hypothesis, a human will always be in the loop because... [13:09] the humans kind of posed the Riemann hypothesis to begin with. [13:13] Um, [13:14] Yeah, just to hop on that, I think there's like, like in the future, you're going to have a lot of compute resources dedicated to math. And the question will be like a very human one, which is like, by which procedure do humans decide where to like direct all that, like, reasoning firepower. And I think that's going to be like the job of mathematicians, they have to choose, what do we work on? How do we interpret the results? How do we interpret failures to find answers, that kind of thing. [13:37] Do you think an AI math system can solve Riemann? Or where is the ceiling, do you think? [13:44] I think that it should be able to solve it or prove that it's undecidable. [13:51] that would also be an interesting result. Um, [13:55] Yeah, I think the... [13:58] If we think about like what a great mathematician like a Terry Tao, for instance, is capable of doing, you know, they're able to like synthesize lots of papers, lots of frontier results and learn from them in a way that.

[14:14] the other top human mathematicians can and kind of find connections between these things and um [14:20] sort of use them to create [14:22] new and more complex theories, uh, [14:26] I mean, that's exactly kind of how the system we're engineering works. And that's what computers are great at and AI models are great at synthesizing large amounts of information, finding patterns. [14:37] recursively self-improving. I think now on Metaculous... [14:42] Last I checked, there's a 43% chance that the next Millennium Prize will be solved by AI or like human with significant AI assist. [14:52] And [14:53] Um, [14:55] I think that's an underestimate. I mean, we could be lucky, and Larry Guth might be on the path to the Riemann hypothesis, which would be amazing. But I think that... [15:06] you know, [15:07] if the next one is solved by a human, it would probably have to be in the very near future. And for sure, the... [15:16] The next next one will probably be significantly solved by AI. [15:21] One of the things that you said I want to hit on, which is this idea of recursive self-improvement, because... [15:26] It seems like in the world of AI, there are... [15:30] If you were to draw a spectrum of human-only... [15:35] to ai only and then human in the loop is sort of the spectrum in the middle from lots of human little bit of ai to lots of ai a little bit of human one of the things that is interesting about harmonic at least the way i understand it is

[15:48] Because of lean, [15:49] You can encapsulate math in code. [15:53] Thank you. [15:53] Because of formal verification, you can objectively determine whether things are right or wrong, which means that you have an objective reward function that you can use with self-play, [16:07] to have very fast cycle times with reinforcement learning, which means that the progress of your model [16:14] has a chance to be extremely fast because there are no humans in the loop with that with that recursive self-improvement like objective function is clearly defined you can do self-play to just make the model better and better and better and better and better which is not something that we see in a lot of domains of ai most domains of ai it's a lot messier to kind of get the cycle time on improvement um [16:35] Can you just talk through... [16:38] the system a bit, a little bit of what I described, like what feeds into your model, what governs the rate at which it can get better, because it seems like something that will be able to get better at a pretty quick rate. [16:50] Yeah, happy to cover that. One point before going to that is just that I think the most interesting part about this is that, you know, there are other areas where recursive self improvement can work, for example, again, in those board games like AlphaGo. Yep. But I think what people don't realize is that [17:05] Well, what a lot of people don't realize is that [17:07] in these, let's say, [17:08] you know, [17:09] perfectly observed zero-sum games, you improve recursively just by playing against yourself, but you hit an optimal strategy. Yes. So at that point, it doesn't matter what system you have, it won't do better. The most exciting part about math is that there is no upper bound.

[17:25] So you're just going to keep putting compute in and it's going to keep getting better and [17:28] There's no end to it. And so when we talk about, do we think AI can solve a Riemann hypothesis or get a Millennium Prize? Like, those are very human milestones. And I think the real question is, like, will it ever stop? I mean, because it clearly will get there. And I think we're going to end up solving problems that are much, much harder than the Riemann hypothesis, which we haven't even conceived of yet, because it's almost like beyond us to write down such a hard problem. [17:50] Have you guys ever seen that Minecraft video of the AI... [17:55] beating Minecraft in 20 seconds. [17:58] No. I think that's a good analogy. It's like... [18:01] You know what Minecraft is, how a human would play it, and then the AI beating it in 20 seconds is just like... [18:08] incomprehensible yeah like you can't even kind of grok what's going on in the video feed hmm [18:14] Yeah, but I think if we just talk about how harmonic works, you can just think of it as... [18:20] There's a collection of agents that are essentially trying to solve problems. And it's true because we use lean, we're able to check whether our answers are correct and thereby derive... [18:31] a variety of training signals that we use to improve the system. But just to be clear, you know, the use of lean just lets you verify things. Lean doesn't itself tell you whether you're getting closer to the answer or whether you're getting smarter or not. It's just telling you whether it's correct or not. So there's a lot of open scientific challenges to making it get better quickly. And can you just say a word about what lean is just in case people aren't familiar? Yeah, totally. Lean is just another programming language, a really great one created by Leo

[19:01] be writing lean or the ais might just be writing lean in the future um but the idea is that the mathematical statements are encoded in the type system of the language so just very simply you know you have functions in lean [19:13] and the input types correspond to the assumptions of the mathematical theorem and the output type is the conclusion. And the point of Lean is that if you write a program that implements that function and it compiles, that means you can derive the output type from the input type, which in turn implies that you can conclude the conclusion from the assumptions. [19:31] So that's really the the fundament that's that's how you use lean for math. And I think one thing that's super interesting about lean is if you look at Leo DeMora, the creator who's at Amazon AWS now, he's not a mathematician, and he wrote this as a software verification tool. So he has, you know, the belief that [19:51] you know, [19:52] in the future software will be verified and the existing tools, things like COC and Isabel, which are kind of, [20:01] multi-decade old software verification tools are just not good, you know, and, and, [20:07] They're frankly unusable. The experience for developers is poor. And so he wanted to create a better software verification tool and, you know, [20:16] in the hopes of if you build something better, more people will use it and we'll have better software, which is a super noble [20:23] goal in its own right. But then what he didn't realize was [20:28] software verification. [20:31] - Uh,

[20:32] All it is is just proving that software has certain properties. [20:36] this thing became very popular in the math community and you had, [20:42] like thousands of mathematicians and math students building up an organic library called math lib, which you can think of as kind of like the largest, um, [20:53] math textbook. [20:54] open source, it's on GitHub, and it's just like growing at a pretty fast clip. And I [21:02] like the usage of lean for math... [21:05] I think. [21:06] to some degree has surpassed anyone's expectations. It might be more than the usage for [21:11] verified software at this point. And that might change as time [21:16] goes forward and with AI. One of the questions we always ask is why now? Because reinforcement learning has existed for a long time, math has existed for even longer, and it seems like math has really hidden an inflection point. You guys have chosen to start Harmonic at this point in time. [21:31] Why now? [21:34] Oh, I mean, there's two really good reasons why now that we're excited about. So first, [21:40] you know, [21:41] One is just that the AI systems have gotten better in an interesting way. So I, [21:46] I was actually talking with a friend, a close friend about, you know, RL for theorem proving back in 2015, in 2016. And, [21:56] One issue back then was that there wasn't even a great notion of an AI system that could predict something in an infinite action space.

[22:04] So in Go, [22:05] you know, you can place a, [22:06] piece somewhere, right? And it's either a black or white piece. But in math, you can really do anything like you can just [22:11] generate any like next step and so we didn't have great systems to do that so auto aggressive language models have gotten pretty good so that's one thing that makes it possible [22:19] I'm talking on the timescale of like a decade here, but that's really important. And the other thing that's kind of crazy is that lean has gotten really good. So if you had told a mathematician 20 years ago that a large fraction of the field would be excited about formal methods in math. [22:34] they might've thought you were crazy because back then formal methods were really isolated to formal logic or certain types of, you know, graph theory. Like if you guys have heard the four color theorem, that was one big success for formal math. But what's changed is that lean is so flexible and so exciting for people that they've contributed this thing called mathlib. So now there's a lot of body of knowledge that you can build on to prove things. And so the combination of AI and [22:57] starting to like even be a [22:59] possible fit for this problem. Plus lean working really well. And lean for was only released officially in September 2023. So those two things happening together really made it the right time to attack this. [23:09] Can you say a word about data and specifically synthetic data and what it is that fuels the [23:17] the model that you guys are building? [23:20] Yeah, so synthetic data is the [23:22] fuel for the model. There's a [23:25] There's an amazing resource called Mathlib, it's that open source repository. So that's a lot of human written lean. And it's written in a way that's very general and compact. So they're really proving advanced theorems, right? It's not necessarily the best fit for problem solving. And so

[23:42] As a result, [23:43] almost [23:44] the only data you can use for this is synthetic data you generate yourself because that original data is not very applicable. So it's kind of a data poor regime compared to most areas of AI. And so that process that I described where the agents themselves are trying to [23:58] solve problems. [23:59] and thereby generate training signals. That's the primary way in which you can get data. [24:04] And the other issue is that, you know, [24:06] You have to progress through levels of intelligence. So you're not necessarily proving the Riemann hypothesis up front. You're proving really simple things, but then you kind of amplify yourself recursively throughout the process. Turns out there's not as much math data on the Internet as cat videos. Unfortunately, no. Unfortunately, no. Well, yeah, it's interesting, though, because, you know, there's the data wall that the foundation model... [24:28] you know, the general purpose foundation model companies are running into and it's [24:33] At this point, they've exhausted, you know, what's available on the internet. And if you can generate most of the data that's required to train, that's kind of another advantage of having math at the core versus math. [24:45] hoping for math as an emergent property of scale. [24:49] Yeah, and I think the data wall kind of manifests itself in two ways. One is just, like you said, we're out of [24:56] internet data. Yeah. The other is... [24:59] - Uh, [24:59] the actual... [25:02] internet data quality that's out there. [25:05] You can think of that as providing kind of a ceiling to how smart these models can get, because if you train on the cat videos and all the nice Wikipedia content and,

[25:15] you know, the internet content. It's an open problem how to get something that's significantly smarter than that. And so you do need to get into some kind of [25:25] self-reinforcing, self-play regime, in our opinion, to... [25:30] All right. [25:30] get to a point where [25:32] you can surpass the... [25:34] the ability of [25:35] human mathematicians and researchers at [25:38] multiple tasks. And so I think [25:42] - Uh, [25:44] in many ways, like the path, it's inevitable that... [25:48] it takes kind of the alpha go to alpha zero approach. And we learn how to make these models create the vast majority of their data and have the data actually increase in complexity as these models continue to iterate. [26:05] I think the great thing about math is there's a simple path to doing this. You can basically measure the complexity [26:13] of a math problem and how difficult it is by how many lines of lean it takes to solve. [26:22] So you can actually look at the complexity of a system. [26:28] a lot of like... [26:29] problems are solved by breaking it down into smaller chunks. [26:33] and actually solving those chunks. And if you kind of think about how that works, [26:38] the smaller chunks [26:40] are then more manageable because they're sort of like, [26:43] fewer lines to solve than the big one. So if you get really good at that,

[26:48] and then you get good at solving the chunks [26:51] then you can kind of like, [26:52] train your model to do better. And as, as you kind of like keep, keep, [26:56] Uh... [26:57] turning the gears on that, the model gets better at solving incrementally harder and harder and more complex problems. I think that works very well in math because it mirrors how we solve math. [27:10] like on pen and paper. Yeah. [27:12] And we've been able to start with simple axioms and build up [27:17] like just... [27:18] giant complex structures. Maybe the Riemann hypothesis would be [27:24] hundreds of pages, if not more, to solve. Fermat's last theorem was, I think, [27:30] 200 pages, very, very complex. [27:34] So, uh... [27:36] Yeah, I do think eventually you'll get to a level where you'll be able to solve these things and... [27:41] the math is... [27:43] to some extent, like the original synthetic data. [27:46] Yeah. What do, [27:48] What determines the rate at which your model can get better? [27:55] uh, [27:56] the rate at which you can get better. Um, [27:59] Well, I think the [28:01] the, [28:03] Highest level one is energy. [28:04] so the more energy you can put into it the more attempts can happen in parallel [28:09] which means you generate data faster. [28:12] So if... [28:14] I mean, there's no rate-limiting step [28:16] I mean, sorry, there's a bunch of rate limiting steps, but there's no fundamental constraint on how fast it gets better. So it's really just about how much compute you put in.

[28:23] I think it's also, I mean, there's still a lot of [28:27] unsolved problems in this field, right? [28:31] uh, [28:33] eat [28:33] We benefit a lot from... [28:36] core theorems that have been proved in the past. And, you know, there's, if you think about like, competition math context, there's theorems that every student would just learn and use like, you know, AMGM inequality. [28:51] things of that nature. And, and so to some degree, um, [28:56] like math lib is incomplete. [28:58] there's very little, uh, content about geometry for instance. And it's like very theoretical and abstract. Um, [29:06] And so... [29:07] a limiting step is like what's in Mathlib, and of course, [29:11] at some point, [29:12] the models have to solve the problem of creating new theories and new structures and [29:17] kind of like [29:18] expanding to other domains and getting really, really good at, [29:22] formalizing things that haven't been formalized by humans. I think that'll be a big unlock, and that'll certainly happen within the next... [29:31] several, several years, you'll be able to say, Hey, here's just like this situation. It could be as simple as like, [29:39] a baseball team and they're like throwing balls back and forth to each other and you know [29:44] your systems would be able to kind of like, [29:47] auto formalize that and turn that into lean code on the fly. And I don't think we're quite there yet to the point where that's reliable, but when it does get reliable,

[29:59] um, [30:00] I think that'll be a really big unlock. [30:04] If everything goes right, what do you think Harmonic becomes? [30:10] Well, our mission statement is to explore the frontiers of human knowledge. So it's very important to us that, you know, the things we produce are correct and useful to humans. So I think in the best case, you know, we're able to build a tool that a lot of mathematicians use to... [30:25] close all the Millennium Prize problems and to go far beyond that. I think that'd be a great, you know, service to humanity. There's also other areas of commercial application for the software. So [30:35] you know the dream for software engineering is to be able to just check that code is correct to do that you need to have a very good model of how code works seem to be able to understand how the libraries work what they promise that kind of thing and so you can imagine a future where safety critical software is proven correct general software is proven correct and the way software engineering done is done can change as well so there's just like a lot of applications if you can make a system that's very good at math reasoning and very good at checking its reasoning [31:03] Yeah, really, we think there's a lot of applications. [31:06] Yeah, and I think the, I mean, math and software are two fairly obvious ones. I think software engineering... [31:16] Uh, [31:16] as a discipline is changing really quickly. I'm sure you guys are seeing all the reports of people doing crazy things with cursor and, [31:24] you know, [31:25] Claude 3.5, um, [31:28] I think in the future software engineering

[31:31] will be less about [31:33] like, [31:34] reviewing and collaborating on code as an artifact and will be more about collaborating on specs. What do we want the code to do? Can we be more rigorous about that? And I think that's where verification will will become a bigger thing, because as the cost of [31:53] Software goes to zero. [31:55] The cost of verified software will also... [31:58] go to zero and suddenly this thing that was like very impractical and expensive because you need [32:04] specialist humans to do it. [32:07] will just accelerate dramatically with AI. So I think you look out five, 10 years from now, [32:13] the vast majority, I mean, if we progress along the capability curve as we have been, the vast majority of software written will be verified and provably correct in [32:23] And I think that's a really exciting future. [32:26] I also think like on the more theoretical side, [32:29] It's not just math, but like... [32:31] Physics is... [32:33] Um, [32:34] essentially... [32:35] Math theoretical physics is... [32:38] you know, one of the main ways the frontier of math gets implemented. And, um, [32:45] I think it would be... [32:48] amazing to me personally to accelerate some of the [32:52] fundamental physics research at the frontier and really just develop an understanding for [32:58] why the universe is the way it is, why the laws of physics exist, and, um, also help figure out experiments to test those, um,

[33:08] So I would be very proud if we contributed to that effort. [33:12] And do you think you'll mostly be contributing to math and math adjacent areas like physics and software engineering? Or do you think, you know, anything that involves reasoning is in spec for harmonic? [33:26] Yeah, I mean, I think we try to be focused on the things that were still a small company over the... [33:33] over the long term, I think [33:36] if you believe math is reasoning, and we do, then yeah, if we get really, really good at math and... [33:44] computer science is a very natural analog [33:48] then, um, [33:50] Yeah, I mean, anything is in scope for those models. [33:54] Even the history essay, I think. [33:56] We'll see. [33:59] Tudor doesn't want to touch his three essays. [34:02] Yeah. [34:02] I really enjoyed writing history essays, even though my parents were like, you know, humanities, language arts, just ignore all that stuff. [34:12] I think my math skills led me to write great history essays too. [34:16] So hopefully Aristotle will be no different. One day. Aristotle wrote some great... [34:22] historical, uh, [34:23] Commentary. [34:26] You are truly a polymath. [34:27] Yeah, I mean, Poetics is, if you've read Poetics, it's... [34:33] Should we wrap it up with some rapid fire? Let's do it. Lightning round. [34:37] Okay. [34:38] In what year will you win the IMO?

[34:41] *laughs* [34:43] What do you think, Tudor? [34:45] soon. [34:46] All right. 2025. [34:48] Soon. Maybe 2024. All right. We'll sign you up for 2024. [34:53] All right, how about the Millennium Prize? [34:57] Ooh, that's a tough one. I would guess 2029. [35:04] 2029? Yeah. Okay, I heard 2028. [35:07] Is that what they're... Yeah. I guess it's a fully AI unassisted Millennium Prize or AI human hybrid. [35:16] Well, what do you think for hybrid? [35:19] Hybrid I could see 2028 are we talking on easy Millennium prize or hard? Yeah, is it like not easy easy? Stokes might be 2026 Riemann hypothesis. I'll give you 2029. All right. All right, there we go good [35:34] Given we can't even do arithmetic today with LLMs, that's pretty amazing. When do you think we'll have human or superhuman level reasoning more broadly defined? [35:47] I think to some degree, if you define it as something that can reason and solve math problems in excess of any human, like... [35:57] Something that Terry Tao would, you know, would give Terry Tao a run for... [36:01] Um, [36:03] for his money [36:04] I think we're a couple of years away, but I think the models within the next year will get to probably like 99.99th percentile.

[36:12] Would Terry agree with that? [36:16] I think so. Yeah. I don't know. You'd have to ask him, but I think he would agree with that. [36:22] One of our favorite questions is, [36:24] Who in the world of AI do you admire most? And we'll modify it slightly for you guys. Who in the world of AI or math do you admire most? [36:36] I like von Neumann. We were just talking before about von Neumann's biography. [36:45] I think, I think what I find really interesting about him was he started as a mathematician and he was discouraged, um, [36:52] I think his father, who was like a Hungarian businessman... [36:55] was trying to discourage him from... [36:58] doing math because it wasn't very like monetizable. Um, [37:02] And so he got his friend who was a great mathematician to try to talk him out of it. [37:09] The friend came back and he's like, I can't do it. This guy's too good. It would be just a disservice to society if he didn't use his talents for for math. And then, you know, he pioneered computer science. [37:22] and the von Neumann machine was like the blueprint for all modern computers. Um, [37:28] He contributed to the Manhattan Project, which, you know, is a little controversial, but very, very practical and impactful. Um... [37:37] And you know, [37:38] created probably the canonical text in game theory as well. So, um,

[37:45] Yeah, I... [37:46] I think [37:48] I think it's pretty amazing. Also a fellow Eastern European. Well, some people debate whether Hungary is in Eastern Europe. [37:55] Yeah, it's an interesting question. I think I definitely admire... [37:58] almost all scientists and mathematicians um but i think that like [38:02] You know, if you've heard of the mathematician Leibniz, what was kind of shocking to learn during the course of working on this company is that, you know, so Leibniz was competing with Newton to create calculus. [38:12] and you know newton's formulation went out but leibniz was basically there but one thing people don't know is that leibniz also had a lot of other work and one piece of work that is just [38:21] incredibly prescient keep in mind this is the late 1600s um he created this thing called an idea called the universal characteristic which is essentially the notion of having [38:32] a deductive language, [38:34] uh, [38:34] automated procedure to deduce things using that language and a body, an encyclopedia of work in that language that you can build on to derive things. And so the amazing thing to me is that this thinker [38:45] hundreds of years ago, essentially predicted what would be happening in 2024. And it seems that the only thing that was required was having like AI get a little better, [38:54] and having like computers that can do something like lean right and i think it's just incredible to have a human being predict that with no concept whatsoever what's going to come later but to understand that that's like such a fundamental thing that we're going to end up working on that hundreds of years later awesome thank you guys [39:12] Thanks for having us. Thanks for having us.

[39:38] Thank you.