What is Real? (Lee Smolin) | AI Podcast Clips

What is real? Let's start with an easy question. Put another way, how do we know what is real and what is merely a creation of our human perception and imagination? We don't know. We don't know. This is science. I presume we're talking about science. And we believe, or I believe, that there is a world that is independent of my existence and my experience about it and my knowledge of it. And this I call the real world. So you said science, but even bigger than science. Sure, sure. I need not have said this is science. I just was warming up. Warming up. Okay, now that we're warmed up, let's take a brief step outside of science. Is it completely a crazy idea to you that everything that exists is merely a creation of our mind? So there's a few, not many, this is outside of science now, people who believe perception is fundamentally what's in our human perception. The visual cortex and so on, the cognitive constructs that's being formed there is the reality. And then anything outside is something that we can never really grasp. Is that a crazy idea to you? There's a version of that that is not crazy at all. What we experience is constructed by our brains and by our brains in an active mode. So we don't see the raw world, we see a very processed world. We feel something that's very processed through our brains and our brains are incredible. But I still believe that behind that experience, that mirror or veil or whatever you wanna call it, there is a real world and I'm curious about it. Can we truly, how do we get a sense of that real world? Is it through the tools of physics from theory to the experiments? Or can we actually grasp it in some intuitive way that's more connected to our ape ancestors? Or is it still fundamentally the tools of math and physics that really allow us to grasp it? Well, let's talk about what tools they are. What you say are the tools of math and physics. I mean, I think we're in the same position as our ancestors in the caves or before the caves or whatever. We find ourselves in this world and we're curious. We also, it's important to be able to explain what happens when there are fires, when there are not fires, what animals and plants are good to eat and all that stuff. But we're also just curious. We look up in the sky and we see the sun and the moon and the stars and we see some of those move and we're very curious about that. And I think we're just naturally curious. So we make, this is my version of how we work. We make up stories and explanations. And there are two things which I think are just true of being human. We make judgments fast because we have to. We're to survive, is that a tiger or is that not a tiger? And we go. Act. We have to act fast on incomplete information. So we judge quickly and we're often wrong, or at least sometimes wrong, which is all I need for this. We're often wrong. So we fool ourselves and we fool other people readily. And so there's lots of stories that get told and some of them result in a concrete benefit and some of them don't. And so you said we're often wrong, but what does it mean to be right? Right, that's an excellent question. To be right, well, since I believe that there is a real world, I believe that to be, you can challenge me on this if you're not a realist. A realist is somebody who believes in this real objective world which is independent of our perception. If I'm a realist, I think that to be right is to come closer. I think, first of all, there's a relative scale. There's not right and wrong. There's right or more right and less right. And you're more right if you come closer to an exact, true description of that real world. Now, can we know that for sure? No. And the scientific method is ultimately what allows us to get a sense of how close we're getting to that real world? No on two counts. First of all, I don't believe there's a scientific method. I was very influenced when I was in graduate school by the writings of Paul Feyerabend who was an important philosopher of science who argued that there isn't a scientific method. There is or there isn't? There is not. There's not. Can you elaborate? I'm sorry if you were going to, but can you elaborate on the, what does it mean for there not to be a scientific method, this notion that I think a lot of people believe in in this day and age? Sure. Paul Feyerabend, he was a student of Popper who taught Karl Popper. And Feyerabend argued both by logic and by historical example that you name anything that should be part of the practice of science, say you should always make sure that your theories agree with all the data that's already been taken. And he'll prove to you that there have to be times when science contradicts, when some scientist contradicts that advice for science to progress overall. So it's not a simple matter. I think that, I think of science as a community. Of people. Of people, and as a community of people bound by certain ethical precepts, percepts, whatever that is. So in that community, a set of ideas they operate under, meaning ethically, of kind of the rules of the game they operate under. Don't lie, report all your results, whether they agree or don't agree with your hypothesis. Check the training of a scientist. Mostly consists of methods of checking, because again, we make lots of mistakes. We're very error prone. But there are tools, both on the mathematics side and the experimental side, to check and double check and triple check. And a scientist goes through a training, and I think this is part of it. You can't just walk off the street and say, yo, I'm a scientist. You have to go through the training, and the training, the test that lets you be done with the training is can you form a convincing case for something that your colleagues will not be able to shout down, because they'll ask, did you check this, and did you check that, and did you check this, and what about a seeming contradiction with this? And you've gotta have answers to all those things or you don't get taken seriously. And when you get to the point where you can produce that kind of defense and argument, then they give you a PhD. And you're kind of licensed. You're still gonna be questioned, and you still may propose or publish mistakes, but the community is gonna have to waste less time fixing your mistakes. Yes, but if you can maybe linger on it a little longer, what's the gap between the thing that that community does and the ideal of the scientific method? The scientific method is you should be able to repeat an experiment. There's a lot of elements to what construes the scientific method, but the final result, the hope of it is that you should be able to say with some confidence that a particular thing is close to the truth. Right, but there's not a simple relationship between experiment and hypothesis or theory. For example, Galileo did this experiment of dropping a ball from the top of a tower, and it falls right at the base of the tower. And Aristotelian would say, wow, of course it falls right to the base of the tower. That shows that the Earth isn't moving while the ball is falling. And Galileo says no weight is a principle of inertia and has an inertia in the direction with the Earth isn't moving, and the tower and the ball and the Earth all move together. When the principle of inertia tells you it hits the bottom, it does look, therefore my principle of inertia is right. And Aristotelian says no, Aristotle's science is right. The Earth is stationary. And so you've gotta get an interconnected bunch of cases and work hard to line up, it took centuries to make the transition from Aristotelian physics to the new physics. It wasn't done till Newton in 1680-something, 1687. So what do you think is the nature of the process that seems to lead to progress, if we at least look at the long arc of science, of all the community of scientists, they seem to do a better job of coming up with ideas that engineers can then take on and build rockets with or build computers with or build cool stuff with? I don't know, a better job than what? Than this previous century. So century by century, we'll talk about string theory and so on and kind of possible, what you might think of as dead ends and so on. We did not do anything with string theory. We'll straighten it out. We'll get on with string theory. But there is, nevertheless, in science, very often at least temporary dead ends. But if you look through centuries, the century before Newton and the century after Newton, it seems like a lot of ideas came closer to the truth that then could be usable by our civilization to build the iPhone, right? To build cool things that improve our quality of life. That's the progress I'm kind of referring to. Let me, can I say that more precisely? Yes. It's a low bar. I think it's important to get the time places right. There was a scientific revolution that partly succeeded between about 1900 or late 1890s and into the 1930s, 1940s and so on. And maybe some, if you stretched it, into the 1970s. And the technology, this was the discovery of relativity and that included a lot of developments of electromagnetism. The confirmation, which wasn't really well confirmed into the 20th century that matter was made of atoms and the whole picture of nuclei with electrons going around. This is early 20th century. And then quantum mechanics was from 1905, took a long time to develop, to the late 1920s and then it was basically in final form. And the basis of this partial revolution, we can come back to why it's only a partial revolution, is the basis of the technologies you mentioned. All of, I mean, electrical technology was being developed slowly with this. And in fact, there's a close relation between the development of electricity and the electrification of cities in the United States and Europe and so forth. And the development of the science. The fundamental physics since the early 1970s doesn't have a story like that so far. There's not a series of triumphs and progresses and there's not any practical application. So just to linger briefly on the early 20th century and the revolutions in science that happened there, what was the method by which the scientific community kept each other in check about when you get something right, when you get something wrong? Is experimental validation ultimately the final test? It's absolutely necessary. And the key things were all validated. The key predictions of quantum mechanics and of the theory of electricity and magnetism. So before we talk about Einstein, your new book before string theory, quantum mechanics, let's take a step back at a higher level question. What is, that you mentioned, what is realism? What is anti-realism? And maybe why do you find realism, as you mentioned, so compelling? Realism is the belief in an external world independent of our existence, our perception, our belief, our knowledge. A realist, as a physicist, is somebody who believes that there should be possible some completely objective description of each and every process at the fundamental level, which describes and explains exactly what happens and why it happens. That kind of implies that that system, in a realist view, is deterministic. Meaning there's no fuzzy magic going on that you can never get to the bottom of. You can get to the bottom of anything and perfectly describe it. Some people would say that I'm not that interested in determinism, but I could live with the fundamental world, which had some chance in it. So do you, you said you could live with it, but do you think God plays dice in our universe? I think it's probably much worse than that. In which direction? I think that theories can change and theories can change without warning. I think the future is open. You mean the fundamental laws of physics can change? Yeah. Okay, we'll get there. I thought we would be able to find some solid ground, but apparently the entirety of it, temporarily so. Okay, so realism is the idea that while the ground is solid, you can describe it. What's the role of the human being, our beautiful, complex human mind, in realism? Do we have a, are we just another set of molecules connected together in a clever way? Or the observer, does the observer, our human mind, consciousness, have a role in this realism view of the physical universe? There's two ways, there's two questions you could be asking. Does our conscious mind, do our perceptions play a role in making things become, in making things real or things becoming? That's question one. Question two is, does this, we can call it a naturalist view of the world, that is based on realism, allow a place to understand the existence of and the nature of perceptions and consciousness in mind? And that's question two. Question two I do think a lot about, and my answer, which is not an answer, is I hope so, but it certainly doesn't yet. Question one, I don't think so. But of course, the answer to question one depends on question two. So I'm not up to question one yet. So question two is the thing that you can kind of struggle with at this time. What about the anti-realists? So what flavor, what are the different camps of anti-realists that you've talked about? I think it would be nice if you can articulate for the people for whom there is not a very concrete real world, or there's divisions, or there's a, it's messier than the realist view of the universe. What are the different camps? What are the different views? I'm not sure I'm a good scholar and can talk about the different camps and analyze it. But some, many of the inventors of quantum physics were not realists, were anti-realists. And there are scholars, they lived in a very perilous time between the two world wars. And there were a lot of trends in culture which were going that way. But in any case, they said things like the purpose of science is not to give an objective, realist description of nature as it would be in our absence. This might be saying Niels Bohr. The purpose of science is as an extension of our conversations with each other to describe our interactions with nature. And we're free to invent and use terms like particle, or wave, or causality, or time, or space if they're useful to us and they carry some intuitive implication. But we shouldn't believe that they actually have to do with what nature would be like in our absence, which we have nothing to say about. Do you find any aspect of that, because you kind of said that we human beings tell stories. Do you find aspects of that kind of anti-realist view of Niels Bohr compelling? That we're fundamentally are storytellers and then we create tools of space and time and causality and whatever this fun quantum mechanics stuff is to help us tell the story of our world. Sure, I just would like to believe that there's an aspiration for the other thing. The other thing being what? The realist point of view. Do you hope that the stories will eventually lead us to discovering the real world as it is? Yeah. Is perfection possible, by the way? No. You mean will we ever get there and know that we're there? Yeah, exactly. That's for people 5,000 years in the future. We're certainly nowhere near there yet. Do you think reality that exists outside of our mind, do you think there's a limit to our cognitive abilities as, again, descendants of apes who are just biological systems? Is there a limit to our mind's capability to actually understand reality? Sort of there comes a point, even with the help of the tools of physics, that we just cannot grasp some fundamental aspects of that reality. Again, I think that's a question for 5,000 years in the future. I think there is a universality. Here, I don't agree with David Deutsch about everything, but I admire the way he put things in his last book. And he talked about the role of explanation. And he talked about the universality of certain languages or the universality of mathematics or of computing and so forth. And he believed that universality, which is something real, which somehow comes out of the fact that a symbolic system or a mathematical system can refer to itself and can, I forget what that's called, can reference back to itself and build, in which he argued for a universality of possibility for our understanding, whatever is out there. But I admire that argument. But it seems to me we're doing okay so far, but we'll have to see. Whether there is a limit or not. For now, we've got plenty to play with. Yeah.


Moore's Law is Not Dead (Jim Keller) | AI Podcast Clips

For over 50 years now, Moore's Law has served for me and millions of others as an inspiring beacon of what kind of amazing future brilliant engineers can build. I'm just making your kids laugh all of today. That's great. So, first, in your eyes, what is Moore's Law, if you could define for people who don't know? Well, the simple statement was, from Gordon Moore, was double the number of transistors every two years. Something like that. And then my operational model is we increase the performance of computers by 2x every two or three years. And it's wiggled around substantially over time. And also, in how we deliver performance has changed. But the foundational idea was 2x the transistors every two years. The current cadence is something like, they call it a shrink factor, like 0.6 every two years, which is not 0.5. But that's referring strictly, again, to the original definition of just... Yeah, of transistor count. A shrink factor, just getting them smaller and smaller and smaller. Because for a constant chip area, if you make the transistors smaller by 0.6, then you get one over 0.6 more transistors. So, can you linger on it a little longer? What's the broader... What do you think should be the broader definition of Moore's Law? When you mentioned how you think of performance, just broadly, what's a good way to think about Moore's Law? Well, first of all, I've been aware of Moore's Law for 30 years. In which sense? Well, I've been designing computers for 40. You're just watching it before your eyes, kind of thing. Well, and somewhere where I became aware of it, I was also informed that Moore's Law was going to die in 10 to 15 years. And I thought that was true at first, but then after 10 years, it was going to die in 10 to 15 years. And then at one point, it was going to die in five years. And then it went back up to 10 years, and at some point, I decided not to worry about that particular prognostication for the rest of my life. Which is fun. And then I joined Intel, and everybody said Moore's Law is dead. And I thought, that's sad, because it's the Moore's Law company. And it's not dead, and it's always been going to die. And humans like these apocryphal kind of statements like, we'll run out of food, or we'll run out of air, or we'll run out of room, or we'll run out of something. Right, but it's still incredible that it's lived for as long as it has. And yes, there's many people who believe now that Moore's Law is dead. You know, they can join the last 50 years of people who had the same idea. Yeah, there's a long tradition. But why do you think, if you can try to understand it, why do you think it's not dead currently? Well, first, let's just think, people think Moore's Law is one thing, transistors get smaller. But actually, under the sheet, there's literally thousands of innovations. And almost all those innovations have their own diminishing return curves. So if you graph it, it looks like a cascade of diminishing return curves. I don't know what to call that. But the result is an exponential curve. At least it has been. And we keep inventing new things. So if you're an expert in one of the things on a diminishing return curve, right, and you can see its plateau, you will probably tell people, well, this is done. Meanwhile, some other pile of people are doing something different. So that's just normal. So then there's the observation of how small could a switching device be? So a modern transistor is something like a thousand by a thousand by a thousand atoms, right? And you get quantum effects down around two to ten atoms. So you can imagine a transistor as small as ten by ten by ten. So that's a million times smaller. And then the quantum computational people are working away at how to use quantum effects. So a thousand by a thousand by a thousand atoms. That's a really clean way of putting it. Well, a fan, like a modern transistor, if you look at the fan, it's like 120 atoms wide. But we can make that thinner. And then there's a gate wrapped around it. And then there's spacing. There's a whole bunch of geometry. And a competent transistor designer could count both atoms in every single direction. Like there's techniques now to already put down atoms in a single atomic layer. And you can place atoms if you want to. It's just, you know, from a manufacturing process, if placing an atom takes ten minutes and you need to put ten to the 23rd atoms together to make a computer, it would take a long time. So the methods are both shrinking things and then coming up with effective ways to control what's happening. Manufacture stably and cheaply. Yeah. So the innovation stack is pretty broad. There's equipment, there's optics, there's chemistry, there's physics, there's material science, there's metallurgy. There's lots of ideas about when you put different materials together, how do they interact? Are they stable? Are they stable over temperature? You know, like are they repeatable? You know, there's like literally thousands of technologies involved. But just for the shrinking, you don't think we're quite yet close to the fundamental limits of physics. I did a talk on Moore's Law and I asked for a road map to a path of 100. And after two weeks, they said we only got to 50. 100 what, sorry? 100x shrink. 100x shrink? We only got to 50? And I said, why don't you give it another two weeks? Well, here's the thing about Moore's Law, right? So I believe that the next 10 or 20 years of shrinking is going to happen, right? Now, as a computer designer, you have two stances. You think it's going to shrink, in which case you're designing and thinking about architecture in a way that you'll use more transistors. Or conversely, not be swamped by the complexity of all the transistors you get. Right? You have to have a strategy. So you're open to the possibility and waiting for the possibility of a whole new army of transistors ready to work. I'm expecting more transistors every two or three years by a number large enough that how you think about design, how you think about architecture has to change. Like imagine you build buildings out of bricks, and every year the bricks are half the size, or every two years. Well, if you kept building bricks the same way, so many bricks per person per day, the amount of time to build a building would go up exponentially. Right. Right. But if you said, I know that's coming, so now I'm going to design equipment that moves bricks faster, uses them better, because maybe you're getting something out of the smaller bricks, more strength, thinner walls, less material efficiency out of that. So once you have a roadmap with what's going to happen, transistors, we're going to get more of them, then you design all this collateral around it to take advantage of it and also to cope with it. That's the thing people don't understand. It's like if I didn't believe in Moore's Law and then Moore's Law transistors showed up, my design teams were all drowned. So what's the hardest part of this influx of new transistors? I mean, even if you just look historically throughout your career, what's the thing, what fundamentally changes when you add more transistors in the task of designing an architecture? Well, there's two constants, right? One is people don't get smarter. By the way, there's some science showing that we do get smarter because of nutrition, whatever. Sorry to bring that up. Yeah, I'm familiar with it. Nobody understands it. Nobody knows if it's still going on. Or whether it's real or not. I sort of... But not exponentially. I would believe for the most part people aren't getting much smarter. The evidence doesn't support it. That's right. And then teams can't grow that much. Right. So human beings, we're really good in teams of 10, up to teams of 100, they can know each other. And beyond that, you have to have organizational boundaries. So you're kind of... Those are pretty hard constraints. So then you have to divide and conquer. As the designs get bigger, you have to divide it into pieces. The power of abstraction layers is really high. We used to build computers out of transistors. Now we have a team that turns transistors into logic cells, and another team that turns them into functional units, and another one that turns them into computers. So we have abstraction layers in there. And you have to think about when do you shift gears on that. We also use faster computers to build faster computers. So some algorithms run twice as fast on new computers. But a lot of algorithms are N-squared. So a computer with twice as many transistors in it might take four times as long to run. So you have to refactor the software. Simply using faster computers to build bigger computers doesn't work. So you have to think about all these things. So in terms of computing performance and the exciting possibility that more powerful computers bring, is shrinking the thing we've just been talking about, for you, one of the biggest exciting possibilities of advancement in performance? Or is there other directions that you're interested in, like in the direction of sort of enforcing given parallelism, or like doing massive parallelism in terms of many, many CPUs, stacking CPUs on top of each other, that kind of parallelism, or any kind of parallelism? Well, think about it a different way. So old computers, slow computers, you said A equal B plus C times D. Pretty simple, right? And then we made faster computers with vector units, and you can do proper equations and matrices. And then modern like AI computations, or like convolutional neural networks, where you convolve one large data set against another. And so there's sort of this hierarchy of mathematics, from simple equation to linear equations to matrix equations, to deeper kind of computation. And the data sets are getting so big that people are thinking of data as a topology problem. Data is organized in some immense shape. And then the computation, which sort of wants to be get data from immense shape and do some computation on it. So what computers have allowed people to do is have algorithms go much, much further. So that paper you referenced, the Sutton paper, they talked about, you know, like when AI started, it was apply rule sets to something. That's a very simple computational situation. And then when they did first chess thing, they solved deep searches. So have a huge database of moves and results, deep search, but it's still just a search. Right? Now we take large numbers of images, and we use it to train these weight sets that we convolve across. It's a completely different kind of phenomena. We call that AI. Now they're doing the next generation, and if you look at it, they're going up this mathematical graph. Right? And then computations, both computation and data sets support going up that graph. Yeah, the kind of computation that might, I mean, I would argue that all of it is still a search. Right? Just like you said, a topology problem of data sets, you're searching the data sets for valuable data, and also the actual optimization of neural networks is a kind of search for the... I don't know. If you had looked at the inner layers of finding a cat, it's not a search. It's a set of endless projections. So, you know, a projection, here's a shadow of this phone. Yeah. Right? And then you can have a shadow of that on something, a shadow on that on something. If you look in the layers, you'll see this layer actually describes pointy ears and round eyedness and fuzziness. But the computation to tease out the attributes is not search. Right. I mean, well... Like the inference part might be search, but the training is not search. Okay. And then in deep networks, they look at layers and they don't even know it's represented. And yet, if you take the layers out, it doesn't work. Okay. So, I don't think it's search. All right. Well... But you have to talk to a mathematician about what that actually is. We could disagree, but it's just semantics, I think. It's not... But it's certainly not... I would say it's absolutely not semantics, but... Okay. All right. Well, if you want to go there. So, optimization to me is search. And we're trying to optimize the ability of a neural network to detect cat ears. And the difference between chess and the space, the incredibly multi-dimensional, 100,000 dimensional space that neural networks are trying to optimize over, is nothing like the chessboard database. So, it's a totally different kind of thing. Okay. In that sense, you can say that it loses the meaning. I can see how you might say. The funny thing is, it's the difference between given search space and found search space. Right. Exactly. Yeah, maybe that's a different way to describe it. That's a beautiful way to put it. Okay. But you're saying, what's your sense in terms of the basic mathematical operations and the architectures, hardware that enables those operations? Do you see the CPUs of today still being a really core part of executing those mathematical operations? Yes. Well, the operations continue to be add, subtract, load, store, compare, and branch. It's remarkable. So, it's interesting that the building blocks of computers are transistors, and under that, atoms. So, you've got atoms, transistors, logic gates, computers, functional units of computers. The building blocks of mathematics at some level are things like adds and subtracts and multiplies. But the space mathematics can describe is, I think, essentially infinite. But the computers that run the algorithms are still doing the same things. Now, a given algorithm might say, I need sparse data, or I need 32-bit data, or I need a convolution operation that naturally takes 8-bit data, multiplies it, and sums it up a certain way. So, the data types in TensorFlow imply an optimization set. But when you go right down and look at the computers, it's and and or gates, do and adds, and multiplies. That hasn't changed much. Now, the quantum researchers think they're going to change that radically, and then there's people who think about analog computing, because you look in the brain, and it seems to be more analog-ish. That maybe there's a way to do that more efficiently. But we have a million X on computation, and I don't know the relationship between computational, let's say, intensity, and ability to hit mathematical abstractions. I don't know anybody who's described that. But just like you saw in AI, you went from rule sets to simple search to complex search to, say, found search. Those are orders of magnitude more computation to do. And as we get the next two orders of magnitude, like a friend, Roger Godori, said, every order of magnitude changes the computation. Fundamentally changes what the computation is doing. Oh, you know the expression, the difference in quantity is the difference in kind. The difference between ant and anthill, right? Between neuron and brain. There's this indefinable place where the quantity changed the quality. And we've seen that happen in mathematics multiple times, and my guess is it's going to keep happening. So your sense is, yeah, if you focus head down and shrinking the transistor... Well, it's not just head down. We're aware of the software stacks that are running the computational loads, and we're kind of pondering what do you do with a petabyte of memory that wants to be accessed in a sparse way and have the kind of calculations AI programmers want. So there's a dialogue and interaction, but when you go in the computer chip, you find adders and subtractors and multipliers. So if you zoom out then with, as you mentioned, Rich Sutton, the idea that most of the development in the last many decades in AI research came from just leveraging computation and just simple algorithms waiting for the computation to improve. Well, software guys have a thing that they call the problem of early optimization. So you write a big software stack, and if you start optimizing the first thing you write, the odds of that being the performance limiter is low. But when you get the whole thing working, can you make it 2x faster by optimizing the right things? Sure. While you're optimizing that, could you have written a new software stack, which would have been a better choice? Maybe. Now you have creative tension. But the whole time as you're doing the writing, that's the software we're talking about. The hardware underneath gets faster and faster. Well, this goes back to the Moore's Law. If Moore's Law is going to continue, then your AI research should expect that to show up. And then you make a slightly different set of choices. We've hit the wall. Nothing's going to happen. And from here, it's just us rewriting algorithms. That seems like a failed strategy for the last 30 years of Moore's Law's death. So can you just linger on it? I think you've answered it, but I'll just ask the same dumb question over and over. Why do you think Moore's Law is not going to die? Which is the most promising, exciting possibility of why it won't die in the next 5-10 years? Is it the continued shrinking of the transistor, or is it another S-curve that steps in? The shrinking of the transistor is literally thousands of innovations. Right. So there's stacks of S-curves in there. There's a whole bunch of S-curves just kind of running their course and being reinvented and new things. The semiconductor fabricators and technologists have all announced what's called nanowires. So they took a fin which had a gate around it and turned that into little wires, so you have better control of that, and they're smaller. And then from there, there are some obvious steps about how to shrink that. So the metallurgy around wire stacks and stuff has very obvious abilities to shrink. And there's a whole combination of things there to do. Your sense is that we're going to get a lot if this innovation from just that shrinking. Yeah, like a factor of 100 is a lot. Yeah, I would say. That's incredible. It's only 10 or 15 years. Now, you're smart and you might know, but to me it's totally unpredictable of what that 100x would bring in terms of the nature of the computation that people would be doing. Yeah, are you familiar with Bell's Law? So for a long time, it was mainframes, minis, workstation, PC, mobile. Moore's Law drove faster, smaller computers. And then when we were thinking about Moore's Law, Rajagirdari said every 10x generates a new computation. So scalar, vector, matrix, topological computation. And if you go look at the industry trends, there was mainframes and minicomputers and PCs. And then the internet took off and then we got mobile devices. And now we're building 5G wireless with one millisecond latency. And people are starting to think about the smart world where everything knows you, recognizes you. Like the transformations are going to be unpredictable. How does it make you feel that you're one of the key architects of this kind of future? So we're not talking about the architects of the high level people who build Angry Bird apps and Snapchat. Angry Bird apps, who knows? Maybe that's the whole point of the universe. I'm going to take a stand at that and the attention distracting nature of mobile phones. I'll take a stand. But anyway, in terms of the side effects of smartphones or the attention distraction, which part? Well, who knows where this is all leading? It's changing so fast. My parents used to yell at my sisters for hiding in the closet with a wired phone with a dial on it. Stop talking to your friends all day. My wife yells at my kids for talking to their friends all day on text. It looks the same to me. It's always echoes of the same thing. But you are one of the key people architecting the hardware of this future. How does that make you feel? Do you feel responsible? Do you feel excited? So we're in a social context. So there's billions of people on this planet. There are literally millions of people working on technology. I feel lucky to be doing what I do and getting paid for it. There's an interest in it. But there are so many things going on in parallel. The actions are so unpredictable. If I wasn't here, somebody else would do it. The vectors of all these different things are happening all the time. There's, I'm sure, some philosopher or meta-philosopher wondering about how we transform our world. So you can't deny the fact that these tools are changing our world. That's right. Do you think it's changing for the better? I read this thing recently that said the two disciplines with the highest GRE scores in college are physics and philosophy. And they're both sort of trying to answer the question, why is there anything? And the philosophers are on the theological side and the physicists are obviously on the material side. And there's a hundred billion galaxies with a hundred billion stars. It seems, well, repetitive at best. So, you know, there's on our way to 10 billion people. I mean, it's hard to say what it's all for, if that's what you're asking. Yeah, I guess I am. Things do tend to significantly increase in complexity. And I'm curious about how computation, like our world, our physical world inherently generates mathematics. It's kind of obvious, right? So we have XYZ coordinates. You take a sphere, you make it bigger, you get a surface that grows by R squared. Like it generally generates mathematics. And the mathematicians and the physicists have been having a lot of fun talking to each other for years. And computation has been, let's say, relatively pedestrian. Like computation in terms of mathematics has been doing binary algebra, and all those guys have been gallivanting through the other realms of possibility. Now, recently, the computation lets you do mathematical computations that are sophisticated enough that nobody understands how the answers came out. Machine learning. Machine learning. It used to be you get data set, you guess at a function. Computation is considered physics if it's predictive of new functions, new data sets. Modern, you can take a large data set with no intuition about what it is and use machine learning to find a pattern that has no function. And it can arrive at results that I don't know if they're completely mathematically describable. So computation has kind of done something interesting compared to A equal B plus C.


Neural networks learning spirals

Let's use TensorFlow Playground to see what kind of neural network can learn to partition the space for the binary classification problem between the blue and the orange dots. First is an easier binary classification problem with a circle and a ring distribution around it. Second is a more difficult binary classification problem of two dueling spirals. This little visualization tool on playground.tensorflow.org is really useful for getting an intuition about how the size of the network and the various hyperparameters affect what kind of representations that network is able to learn. The input to the network is the position of the point in the 2D plane and the output of the network is the classification of whether it's an orange or a blue dot. We'll hold all the hyperparameters constant for this little experiment and just vary the number of neurons and hidden layers. The hyperparameters are batch size of 1, learning rate of 0.03, the activation function is ReLU, and L1 regularization with a rate of 0.001. So let's start with one hidden layer and one neuron and gradually increase the size of the network to see what kind of representation it's able to learn. Keep your eye on the right side of the screen that shows the test loss and the training loss and the plot that shows sample points from the two distributions. And then the shading in the background of the plot shows the partitioning function that the neural network is learning. So a successful function is able to separate the orange and the blue dots. One hidden layer with one neuron. Two neurons. Three neurons. Four neurons. Eight neurons. Now let's take a look at the trickier spiral dataset, keeping most of the hyperparameters the same but decreasing the learning rate to 0.01 and adding to the input to the neural network extra features than just the coordinate of the point but also the squares of the coordinates, the multiplication, and the sign of each coordinate. Let's start with one hidden layer, one neuron. Two neurons. Four neurons. Six neurons. Eight neurons. Two hidden layers, two neurons in the second layer. Four neurons. Six neurons. Eight neurons. There you go. That's a basic illustration with the playground.tensorflow.org that I recommend you try that shows the connection between neural network architecture, dataset characteristics, and different training hyperparameters. It's important to note that the initialization of the neural network has a big impact in many of the cases, but the purpose of this video is not to show the minimal neural network architecture that's able to represent the spiral dataset but rather to provide a visual intuition about which kind of networks are able to learn which kinds of datasets. There you go. I hope you enjoyed these quick little videos, whether they make you think, give you a new kind of insights, or just fun and inspiring. See you next time, and remember, try to challenge yourself and learn something new every day.


2+2=5 in Java

This video is about how we can hack Java by using reflection of its own source code to make this 2 plus 2 statement, output 5. Just as George Orwell, one of my favorite writers, warned us about in 1984, about propaganda machines that sublimate the nature of truth. This video is not about politics, philosophy, nor is it about the apparent, as I have just learned, woke Twitter madness around 2 plus 2 equals 5. Though perhaps if we're living in the simulation and it's written in Java, this might be a way to make the simulation just a bit more dystopian. So here's what the full source code looks like and it uses Java's ability to do reflection, which is the ability of a programming language to inspect itself. So if we look at the code, it actually dives into the implementation of the integer class, pulls out the integer cache class from that implementation, makes it accessible and writable, pulls it into an array of integer object of size 256, and modifies that array. Now what does this array contain? So interestingly, if we look at the integer cache class inside the integer object implementation in Java, it defines a hard-coded low of negative 128 and a high that's passed in as a parameter, that's 127 as a default. And what that does is create a cache of integer objects from negative 128 to 127 and then reuses this cache every time an integer object with a value in this range is used. Now this is exactly the cache with reflection that we pull out and modify. It so happens that the 132nd element in the cache is where the 4 resides and so by way of obfuscation, it takes the 133rd element, which has the number 5 in it and it sizes it to 132nd, but you can just assign value 5 here. And then the result and your else in the code, if you use the integer objects and the number 4 comes up, it will instead output the number 5. There you go, 2 plus 2 equals 5. Check out the link in the description that points to the Stack Exchange Code Golf has a bunch of interesting discussions around this, including the possibility of taking the entire 256 element array and shuffling it, thereby not only making 2 plus 2 equals 5, but messing with the entirety of low value arithmetic in Java. So there you go, that's how you hack the simulation. Let me quickly thank the sponsors that somehow amazingly support the podcast and the videos I make. This time it's Asleep Mattress. Click the link to get a discount in the description. And by the way, I have a conversation with James Gosling, the creator of Java, coming up on the podcast, so check that out. And remember, try to learn something new every day.


Eric Weinstein's Harvard Story - The System Breaks Down in Novel Situations | AI Podcast Clips

Without naming names, can you tell the story of your struggle during your time at Harvard? Maybe in a way that tells the bigger story of the struggle of young, bright minds that are trying to come up with big, bold ideas within the institutions that we're talking about. You can start. I mean, in part, it starts with coffee with a couple of Croatians in the math department at MIT. And we used to talk about music and dance and math and physics and love and all this kind of stuff, as Eastern Europeans love to, and I ate it up. And my friend, Gordana, who was an instructor in the MIT math department when I was a graduate student at Harvard, said to me, and I'm probably going to do a bad version of her accent. Here we go. Eric, will I see you tomorrow at the secret seminar? And I said, what secret seminar? Eric, don't joke. I said, I'm not used to this style of humor, Gordana. The secret seminar that your advisor is running. I said, what are you talking about? Ha ha ha ha. You know, your advisor is running a secret seminar on this aspect. I think it was like the Chern-Simons invariant. Not sure what the topic was again, but she gave me the room number and the time and she was like not cracking a smile. I've never known her to make this kind of a joke. And I thought this was crazy. And I was trying to have an advisor. I didn't want an advisor, but people said you have to have one, so I took one. And I went to this room at like 15 minutes early and there was not a soul inside it. It was outside of the math department. And it was still in the same building, the science center at Harvard. And I sat there and I let five minutes go by, I let seven minutes go by, 10 minutes go by, there's nobody. I thought, okay, so this was all an elaborate joke. And then like three minutes to the hour, this graduate student walks in and like sees me and does a double take. And then I start to see the professors in geometry and topology start to file in. And everybody's like very disconcerted that I'm in this room. And finally the person who was supposed to be my advisor walks in to the seminar and sees me and goes white as a ghost. And I realized that the secret seminar is true, that the department is conducting a secret seminar on the exact topic that I'm interested in, not telling me about it. And that these are the reindeer games that the Rudolphs of the department are not invited to. And so then I realized, okay, I did not understand it. There's a parallel department. And that became the beginning of an incredible odyssey in which I came to understand that the game that I had been sold about publication, about blind refereeing, about openness and scientific transmission of information was all a lie. I came to understand that at the very top, there's a second system that's about closed meetings and private communications and agreements about citation and publication that the rest of us don't understand. And that in large measure, that is the thing that I won't submit to. And so when you ask me questions like, well, why wouldn't you feel good about talking to your critics or why wouldn't you fit? The answer is, oh, you don't know. Like if you stay in a nice hotel, you don't realize that there is an entire second structure inside of that hotel where like there's usually a worker's cafe in a resort complex that isn't available to the people who are staying in the hotel. And then there are private hallways inside the same hotel that are parallel structures. So that's what I found, which was in essence, just the way you can stay hotels your whole life and not realize that inside of every hotel is a second structure that you're not supposed to see as the guest. There is a second structure inside of academics that behaves totally differently with respect to how people get dinged, how people get their grants taken away, how this person comes to have that thing named after them. And by pretending that we're not running a parallel structure, I have no patience for that anymore. So I got a chance to see how the game, how hardball is really played at Harvard. And I'm now eager to play hardball back with the same people who played hardball with me. Let me ask two questions on this. So one, do you think it's possible, so I call those people assholes, that's the technical term. Do you think it's possible that that's just not the entire system, but a part of the system? You can navigate, you can swim in the waters and find the groups of people who do aspire to open this. The guy who rescued my PhD was one of the people who filed in to the secret seminar. Right, but are there people who are outside of this? Is he an asshole? Well, yes, I was a bad- No, but I'm trying to make this point, which is, this isn't my failure to correctly map these people. It's yours. You have a simplification that isn't going to work. I think, okay, asshole is the wrong term. I would say lacking of character. What would you have had these people do? Why did they do this? Why have a secret seminar? I don't understand the exact dynamics of a secret seminar, but I think the right thing to do is to see individuals like you. There might be a reason to have a secret seminar, but they should detect that an individual like you, a brilliant mind, who's thinking about certain ideas, could be damaged by this. I don't think that they see it that way. The idea is we're going to sneak food to the children we want to survive. Yeah, so that's highly problematic, and there should be people within that room. I'm trying to say, this is the thing, the ball is thrown but it won't be caught. The problem is they know that most of their children won't survive, and they can't say that. I see. Sorry to interrupt. You mean that the fact that the whole system is underfunded, that they naturally have to pick favorites. They live in a world which reached steady state at some level, let's say, in the early 70s. In that world, before that time, you have a professor like Norman Steenrod, and you'd have 20 children that is graduate students, and all of them would go on to be professors, and all of them would want to have 20 children. You start taking higher and higher powers of 20, and you see that the system could not, it's not just about money, the system couldn't survive. The way it's supposed to work now is that we should shut down the vast majority of PhD programs, and we should let the small number of truly top places populate mostly teaching and research departments that aren't PhD producing. We don't want to do that because we use PhD students as a labor force. The whole thing has to do with growth, resources, dishonesty. In that world, you see all of these adaptations to a ruthless world where the key question is where are we going to bury this huge number of bodies of people who don't work out? My problem was I wasn't interested in dying. You clearly highlight that there's aspects of the system that are broken, but as an individual, is your role to exit the system or just acknowledge that it's a game and win it? My role is to survive and thrive in the public eye. In other words, when you have an escapee of the system. Like yourself. Such as. And that person says, you know, I wasn't exactly finished. Let me show you a bunch of stuff. Let me show you that the theory of telomeres never got reported properly. Let me show you that all of marginal economics is supposed to be redone with a different version of the differential calculus. Let me show you that you didn't understand the self dual Yang-Mills equations correctly in topology and physics because they're in fact much more broadly found and it's only the mutations that happen in special dimensions. There are lots of things to say, but this particular group of people, like if you just take where are all the Gen X and millennial university presidents. Right. Okay. They're all, they're all in a holding pattern. Now where, why in this story, you know, was it a telomeres, was it an older professor and a younger graduate student? It's this issue of what would be called interference competition. So for example, orcas try to drown minky whales by covering their blow holes so that they suffocate because the needed resource is air. Okay. Well, what are the universities do? They try to make sure that you can't be viable, that you need them, that you need their grants. You need to be zinged with overhead charges or fringe rates or all of the games that the locals love to play. Well, my point is, okay, what's the cost of this? How many people died as a result of these interference competition games? You know, when you take somebody like Douglas Prasher who did green fluorescent protein and he drives a shuttle bus, right? Cause his grant runs out and he has to give away all of his research and all of that research gets a Nobel prize and he gets to drive a shuttle bus for $35,000 a year. What do you mean by died? Do you mean their career, their dreams? Yeah, the whole, as an academic, Doug Prasher was dead for a long period of time. Okay. So as a person who's escaped the system, can't you at this, cause you also have in your mind a powerful theory that may turn out to be useful, maybe not. Let's hope. Can't you also play the game enough like with the children, so like publish and, but also- If you told me that this would work, really what I want to do, you see, is I would love to revolutionize a field with an H index of zero. Like we have these proxies that count how many papers you've written, how cited are the papers you've written. All this is nonsense. That's interesting, sorry, what do you mean by a field with an H index of zero? So a totally new field. H index counts somehow how many papers have you gotten that get so many citations. Let's say H index undefined. Like for example, I don't have an advisor for my PhD, but I have to have an advisor as far as something called the math genealogy project that tracks who advised who, who advised whom down the line. So I am my own advisor, which sets up a loop, right? How many students do I have? An infinite number. Or descendants. They don't want to have that story. So I have to be, I have to have formal advisor, Raoul Bott. And my Wikipedia entry, for example, says that I was advised by Raoul Bott, which is not true. So you get fit into a system that says, well, we have to know what your H index is. We have to know, you know, where are you a professor if you want to apply for a grant? It makes all of these assumptions. What I'm trying to do is to impart, to show all of this is nonsense. This is proxy BS that came up in the institutional setting. And right now it's important for those of us who are still vital like Elon, it would be great to have Elon as a professor of physics and engineering. Yeah. Right? It seems ridiculous to say, but. Just as a shot in the arm. You know, like, it'd be great to have Elon at Caltech. Even one day a week. One day a month. Okay, well, why can't we be in there? It's the same reason. Well, why can't you be on The View? Why can't you be on Bill Maher? We need to know what you're going to do before we take you on the show. Well, I don't want to tell you what I'm going to do. Do you think you need to be able to dance the dance a little bit? I can dance the dance fine. To be on The View. Oh, come on. So you can't. I can do that fine. Here's where it's, the place that it goes south is, there's like a set of questions that get you into this more adversarial stuff. And you've in fact asked some of those more adversarial questions, this setting. And they're not things that are necessarily aggressive, but they're things that are making assumptions. Right. Right? So when you make, I have a question, it's like, you know, Lex, are you avoiding your critics? You know, it's just like, okay, well, why did you frame that that way? Or the next question would be, it's like, do you think that you should have a special exemption and that you should have the right to break rules and everyone else should have to follow them? Like that question I find innervating. It doesn't really come out of anything meaningful. It's just like, we feel we're supposed to ask that of the other person to show that we're not captured by their madness. That's not the real question you want to ask me. If you want to get really excited about this, you want to ask, do you think this thing is right? Yeah, weirdly I do. Do you think that it's going to be immediately seen to be right? I don't. I think it's going to, it's going to have an interesting fight and it's going to have an interesting evolution. And well, what do you hope to do with it in non-physical terms? I gosh, I hope it revolutionizes our relationship of well with people outside of the institutional framework and it re-inflicts us into the institutional framework where we can do the most good to bring the institutions back to health. You know, it's like, these are positive uplifting questions. And if you had Frank Wilczek, you wouldn't say, Frank, let's be honest, you have done very little with your life after the original huge show that you used to break under the physics scene. Like we weirdly ask people different questions based upon how they sit down. Yeah, that's very strange. Right. But you have to understand that. So here's the thing. I get these days a large number of emails from people with the equivalent of a theory of everything for AGI. And I use my own radar, BS radar to detect unfairly perhaps whether they're full of shit or not. Right. I love where you're going with this, by the way. My concern I often think about is there's elements of brilliance in what people write to me. And I'm trying to right now, as you made it clear, the kind of judgments and assumptions we make, how am I supposed to deal with you who are an outsider of the system and think about what you're doing? Because my radar is saying you're not full of shit. But I'm also not completely outside of the system. That's right. You've danced beautifully. You've actually got all the credibility that you're supposed to get, all the nice little stamps of approval. Not all, but a large enough amount. I mean, it's hard to put into words exactly why you sound, whether your theory turns out to be good or not, you sound like a special human being. I appreciate that. In a good way. But what am I supposed to do with that flood of emails from AGI folks? Why do I sound different? I don't know. And I would like to systemize that, I don't know. Look, when you're talking to people, you very quickly can surmise. Am I claiming to be a physicist? No, I say it every turn, I'm not a physicist. When you say something about bundles, you say, well, can you explain it differently? I'm pushing around on this area, that lever over there. I'm trying to find something that we can play with and engage. And another thing is that I'll say something at scale. So if I was saying completely wrong things about bundles on the Joe Rogan program, you don't think that we wouldn't hear a crushing chorus? Yes. Absolutely. And the same thing with geometric unity. So I put up this video from this Oxford lecture. I understand that it's not a standard lecture. But you haven't heard the most brilliant people in the field say, well, this is obviously nonsense. They don't know what to make of it. And they're going to hide behind, well, he hasn't said enough details. Where's the paper? Where's the paper? I've seen the criticism. I've gotten the same kind of criticism. I've published a few things, especially stuff related to Tesla. We did studies on Tesla vehicles. And the kind of criticism I've gotten showed that they're completely... Oh, right. Like the guy who had Elon Musk on his program twice is going to give us an accurate assessment. Yeah, exactly. Exactly. It's just very low level. Without actually ever addressing the content. You know, Lex, I think that in part you're trying to solve a puzzle that isn't really your puzzle. I think you know that I'm sincere. You don't know whether the theory is going to work or not. And you know that it's not coming out of somebody who's coming out of left field. The story makes sense. There's enough that's new and creative and different in other aspects where you can check me. That your real concern is, are you really telling me that when you start breaking the rules, you see the system for what it is and it's become really vicious and aggressive? And the answer is yes. And I had to break the rules in part because of learning issues. Because I came into this field with a totally different set of attributes. My profile just doesn't look like anybody else's remotely. But as a result, what that did is it showed me, is the system true to its own ideals? Or does it just follow these weird procedures? And then when you take it off the rails, it behaves terribly. And that's really what my story I think does, is it just says, well, he completely takes the system into new territory where it's not expecting to have to deal with somebody with these confusing sets of attributes. And I think what he's telling us is he believes it behaves terribly. Now if you take somebody with perfect standardized tests and a winner of math competitions and you put them in a PhD program, they're probably gonna be okay. I'm not saying that the system breaks down for everybody under all circumstances. I'm saying when you present the system with a novel situation, at the moment it will almost certainly break down with probability approaching 100%. But to me the painful and the tragic thing is it, sorry to bring out my motherly instinct, but it feels like it's too much, it could be too much of a burden to exist outside the system. Maybe, but- Psychologically. First of all, I've got a podcast that I kind of like. I've got amazing friends. I have a life which has more interesting people passing through it than I know what to do with. And they haven't managed to kill me off yet. So so far so good.


Vsauce: Elon Musk and the Responsibility of a Large Following | AI Podcast Clip with Michael Stevens

So you're a curious lover of science. What do you think of the efforts that Elon Musk is doing with space exploration, with electric vehicles, with autopilot sort of getting into the space of autonomous vehicles, with boring under L.A., and Neuralink trying to communicate brain-machine interfaces, communicate between machines and human brains? Well, it's really inspiring. I mean, look at the fandom that he's amassed. It's not common for someone like that to have such a following. And so it's- Engineering nerd. Yeah. So it's really exciting, but I also think that a lot of responsibility comes with that kind of power. So if I met him, I would love to hear how he feels about the responsibility he has when there are people who are such a fan of your ideas and your dreams and share them so closely with you. You have a lot of power. And he didn't always have that. He wasn't born as Elon Musk. Well, he was, but- Well, he was named that later. But the point is that I want to know the psychology of becoming a figure like him. Well, I don't even know how to phrase the question right, but it's a question about what do you do when you're following, your fans become so large that it's almost bigger than you? And how do you responsibly manage that? And maybe it doesn't worry him at all, and that's fine too, but I'd be really curious. And I think there are a lot of people that go through this when they realize, whoa, there are a lot of eyes on me. There are a lot of people who really take what I say very earnestly and take it to heart and will defend me. And that can be dangerous and you have to be responsible with it. Both in terms of impact on society and psychologically for the individual, just the burden psychologically on Elon? Yeah, yeah. How does he think about that part of his persona? Well, let me throw that right back at you, because in some ways you're just a funny guy that's gotten a humongous following, a funny guy with a curiosity. You've got a huge following. How do you psychologically deal with the responsibility? In many ways, you have a reach in many ways bigger than Elon Musk. What is the burden that you feel in educating, being one of the biggest educators in the world where everybody's listening to you? And actually, most of the world that uses YouTube for educational material trust you as a source of good, strong scientific thinking. It's a burden and I try to approach it with a lot of humility and sharing. I'm not out there doing a lot of scientific experiments. I am sharing the work of real scientists and I'm celebrating their work and the way that they think and the power of curiosity. But I want to make it clear at all times that, look, we don't know all the answers and I don't think we're ever going to reach a point where we're like, wow, and there you go. That's the universe. It's this equation, you plug in some conditions or whatever, and you do the math and you know what's going to happen tomorrow. I don't think we're ever going to reach that point. But I think that there is a tendency to sometimes believe in science and become elitist and become, I don't know, hard, when in reality it should humble you and make you feel smaller. I think there's something very beautiful about feeling very, very small and very weak and to feel that you need other people. So I try to keep that in mind and say, look, thanks for watching. Vsauce is not, I'm not Vsauce, you are. When I start the episodes, I say, hey, Vsauce, Michael here. Vsauce and Michael are actually a different thing in my mind. I don't know if that's always clear, but yeah, I have to approach it that way because it's not about me. Yeah, so it's not even, you're not feeling responsibility. You're just sort of plugging into this big thing that is scientific exploration of our reality and you're a voice that represents a bunch, but you're just plugging into this big Vsauce ball that others, millions of others are plugged into. Yeah. So I encourage curiosity and responsible thinking and an embracement of doubt and being okay with that.


How to Build AGI? (Ilya Sutskever) | AI Podcast Clips

What do you think it takes to, let's talk about AGI a little bit. What do you think it takes to build a system of human level intelligence? We talked about reasoning, we talked about long-term memory, but in general, what does it take, do you think? Well, I can't be sure, but I think that deep learning, plus maybe another small idea. Do you think self-play will be involved? So like you've spoken about the powerful mechanism of self-play where systems learn by sort of exploring the world in a competitive setting against other entities that are similarly skilled as them, and so incrementally improve in this way. Do you think self-play will be a component of building an AGI system? Yeah, so what I would say to build AGI, I think is going to be deep learning plus some ideas, and I think self-play will be one of those ideas. I think that that is a very, self-play has this amazing property that it can surprise us in truly novel ways. For example, like we, I mean, pretty much every self-play system, both our Dota bot, I don't know if OpenAI had a release about multi-agent where you had two little agents who were playing hide and seek, and of course also AlphaZero, they will all produce surprising behaviors. They all produce behaviors that we didn't expect. They are creative solutions to problems. And that seems like an important part of AGI that our systems don't exhibit routinely right now. And so that's why I like this area, I like this direction because of its ability to surprise us. To surprise us, and an AGI system would surprise us fundamentally. Yes, and to be precise, not just a random surprise, but to find a surprising solution to a problem is also useful. Right. Now, a lot of the self-play mechanisms have been used in the game context, or at least in the simulation context. How much, how far along the path to AGI do you think will be done in simulation? How much faith, promise do you have in simulation versus having to have a system that operates in the real world, whether it's the real world of digital real-world data, or real-world, like actual physical world of robotics? I don't think it's an either or. I think simulation is a tool, and it helps. It has certain strengths and certain weaknesses, and we should use it. Yeah, but, okay, I understand that. That's true, but one of the criticisms of self-play, one of the criticisms of reinforcement learning is one of the, its current power, its current results, while amazing, have been demonstrated in its simulated environments, or very constrained physical environments. Do you think it's possible to escape them, escape the simulated environments, and be able to learn in non-simulated environments, or do you think it's possible to also just simulate in a photorealistic and physics-realistic way the real world in a way that we can solve real problems with self-play in simulation? So, I think that transfer from simulation to the real world is definitely possible, and has been exhibited many times by many different groups. It's been especially successful in vision. Also, OpenAI in the summer has demonstrated a robot hand which was trained entirely in simulation in a certain way that allowed for seem-to-real transfer to occur. Is this for the Rubik's Cube? Yes, right. I wasn't aware that was trained in simulation. It was trained in simulation entirely. Really, so it wasn't in the physical, the hand wasn't trained? No, 100% of the training was done in simulation, and the policy that was learned in simulation was trained to be very adaptive, so adaptive that when you transfer it, it could very quickly adapt to the physical world. So the kind of perturbations with the giraffe or whatever the heck it was, were those part of the simulation? Well, the simulation was generally, so the simulation was trained to be robust to many different things, but not the kind of perturbations we've had in the video. So it's never been trained with a glove, it's never been trained with a stuffed giraffe. So in theory, these are novel perturbations. Correct, it's not in theory, in practice. That those are novel perturbations? Well, that's okay. That's a clean, small-scale but clean example of a transfer from the simulated world to the physical world. Yeah, and I will also say that I expect the transfer capabilities of deep learning to increase in general, and the better the transfer capabilities are, the more useful simulation will become. Because then you could take, you could experience something in simulation and then learn a moral of the story, which you could then carry with you to the real world. As humans do all the time when they play computer games. So let me ask sort of a embodied question, staying on AGI for a sec. Do you think AGI says that we need to have a body? We need to have some of those human elements of self-awareness, consciousness, sort of fear of mortality, sort of self-preservation in the physical space, which comes with having a body. I think having a body will be useful. I don't think it's necessary. But I think it's very useful to have a body for sure, because you can learn a whole new, you can learn things which cannot be learned without a body. But at the same time, I think that you can, if you don't have a body, you could compensate for it and still succeed. You think so? Yes. Well, there is evidence for this. For example, there are many people who were born deaf and blind, and they were able to compensate for the lack of modalities. I'm thinking about Helen Keller specifically. So even if you're not able to physically interact with the world, and if you're not able to, I mean, I actually was getting at, maybe let me ask on the more particular, I'm not sure if it's connected to having a body or not, but the idea of consciousness, and a more constrained version of that is self-awareness. Do you think an AGI system should have consciousness? We can't define consciousness, whatever the heck you think consciousness is. Yeah, hard question to answer, given how hard it is to define it. Do you think it's useful to think about? I mean, it's definitely interesting. It's fascinating. I think it's definitely possible that our systems will be conscious. Do you think that's an emergent thing that just comes from, do you think consciousness could emerge from the representation that's stored within your networks? So like that, it naturally just emerges when you become more and more, you're able to represent more and more of the world? Well, I'd say, I'd make the following argument, which is, humans are conscious, and if you believe that artificial neural nets are sufficiently similar to the brain, then there should at least exist artificial neural nets we should be conscious to. You're leaning on that existence proof pretty heavily. Okay. But that's the best answer I can give. No, I know, I know, I know. There's still an open question if there's not some magic in the brain that we're not, I mean, I don't mean a non-materialistic magic, but that the brain might be a lot more complicated and interesting than we give it credit for. If that's the case, then it should show up, and at some point, we will find out that we can't continue to make progress. But I think it's unlikely. So we talk about consciousness, but let me talk about another poorly defined concept of intelligence. Again, we've talked about reasoning, we've talked about memory. What do you think is a good test of intelligence for you? Are you impressed by the test that Alan Turing formulated with the imitation game with natural language? Is there something in your mind that you would be deeply impressed by if a system was able to do? I mean, lots of things. There's a certain frontier of capabilities today. Yes. And there exist things outside of that frontier, and I would be impressed by any such thing. For example, I would be impressed by a deep learning system which solves a very pedestrian task, like machine translation or computer vision task or something which never makes mistake a human wouldn't make under any circumstances. I think that is something which have not yet been demonstrated, and I would find it very impressive. Yeah, so right now they make mistakes in different, they might be more accurate than human beings, but they still, they make a different set of mistakes. So my, I would guess that a lot of the skepticism that some people have about deep learning is when they look at their mistakes and they say, well, those mistakes, they make no sense. Like if you understood the concept, you wouldn't make that mistake. And I think that changing that would be, that would inspire me. That would be, yes, this is progress. Yeah, that's a really nice way to put it. But I also just don't like that human instinct to criticize a model as not intelligent. That's the same instinct as we do when we criticize any group of creatures as the other. Because it's very possible that GPT-2 is much smarter than human beings at many things. That's definitely true. It is a lot more breadth of knowledge. Yes, breadth of knowledge, and even perhaps depth on certain topics. It's kind of hard to judge what depth means, but there's definitely a sense in which humans don't make mistakes, these models do. Yes, the same is applied to autonomous vehicles. The same is probably gonna continue being applied to a lot of artificial intelligence systems. We find, this is the annoying, this is the process of, in the 21st century, the process of analyzing the progress of AI is the search for one case where the system fails in a big way where humans would not, and then many people writing articles about it, and then broadly as the public generally gets convinced that the system is not intelligent. And we pacify ourselves by thinking it's not intelligent because of this one anecdotal case. And this seems to continue happening. Yeah, I mean, there is truth to that. Although I'm sure that plenty of people are also extremely impressed by the system that exists today. But I think this connects to the earlier point we discussed that it's just confusing to judge progress in AI. Yeah. And you have a new robot demonstrating something. How impressed should you be? And I think that people will start to be impressed once AI starts to really move the needle on the GDP. So you're one of the people that might be able to create an AGI system here, not you, but you and OpenAI. If you do create an AGI system and you get to spend sort of the evening with it, him, her, what would you talk about, do you think? The very first time? First time. Well, the first time I would just, I would just ask all kinds of questions and try to make it, to get it to make a mistake. And I would be amazed that it doesn't make mistakes. And I just keep asking broad. What kind of questions do you think, would they be factual or would they be personal, emotional, psychological? What do you think? All of the above. Would you ask for advice? Definitely. I mean, why would I limit myself talking to a system like this? Now, again, let me emphasize the fact that you truly are one of the people that might be in the room where this happens. So let me ask sort of a profound question about, I just talked to a Stalin historian. I've been talking to a lot of people who are studying power. Abraham Lincoln said, nearly all men can stand adversity, but if you want to test a man's character, give him power. I would say the power of the 21st century, maybe the 22nd, but hopefully the 21st, would be the creation of an AGI system and the people who have control, direct possession and control of the AGI system. So what do you think, after spending that evening having a discussion with the AGI system, what do you think you would do? Well, the ideal world I'd like to imagine is one where humanity, I like the board members of a company where the AGI is the CEO. So it would be, I would like, the picture which I would imagine is you have some kind of different entities, different countries or cities, and the people that leave their vote for what the AGI that represents them should do, and then AGI that represents them goes and does it. I think a picture like that, I find very appealing. You could have multiple, you would have an AGI for a city, for a country, and it would be trying to, in effect, take the democratic process to the next level. And the board can always fire the CEO. Essentially, press the reset button, say. Press the reset. Rerandomize the parameters. But let me sort of, that's actually, okay, that's a beautiful vision, I think, as long as it's possible to press the reset button. Do you think it will always be possible to press the reset button? So I think that it's definitely will be possible to build. So you're talking, so the question that I really understand from you is, will humans or, humans, people have control over the AI systems that they build? Yes, and my answer is, it's definitely possible to build AI systems which will want to be controlled by their humans. Wow, that's part of their, so it's not that just they can't help but be controlled, but that's, they exist, that one of the objectives of their existence is to be controlled. In the same way that human parents generally want to help their children, they want their children to succeed. It's not a burden for them. They are excited to help the children and to feed them and to dress them and to take care of them. And I believe with high conviction that the same will be possible for an AGI. It will be possible to program an AGI, to design it in such a way that it will have a similar deep drive, that it will be delighted to fulfill, and the drive will be to help humans flourish. But let me take a step back to that moment where you create the AGI system. I think this is a really crucial moment. And between that moment and the Democratic board members with the AGI at the head, there has to be a relinquishing of power. So as George Washington, despite all the bad things he did, one of the big things he did is he relinquished power. He, first of all, didn't want to be president. And even when he became president, he didn't keep just serving as most dictators do for indefinitely. Do you see yourself being able to relinquish control over an AGI system, given how much power you can have over the world? At first, financial, just make a lot of money, right? And then control by having possession of this AGI system. I'd find it trivial to do that. I'd find it trivial to relinquish this kind of power. I mean, you know, the kind of scenario you are describing sounds terrifying to me. That's all. I would absolutely not want to be in that position. Do you think you represent the majority or the minority of people in the AI community? Well, I mean. It's an open question and an important one. Are most people good is another way to ask it. So I don't know if most people are good, but I think that when it really counts, people can be better than we think. That's beautifully put, yeah. Are there specific mechanism you can think of of aligning AI gene values to human values? Is that, do you think about these problems of continued alignment as we develop the AI systems? Yeah, definitely. In some sense, the kind of question which you are asking is, so if I were to translate the question to today's terms, it would be a question about how to get an RL agent that's optimizing a value function which itself is learned. And if you look at humans, humans are like that because the reward function, the value function of humans is not external, it is internal. That's right. And there are definite ideas of how to train a value function. Basically an objective, and as objective as possible perception system that will be trained separately to recognize, to internalize human judgments on different situations. And then that component would then be integrated as the base value function for some more capable RL system. You could imagine a process like this. I'm not saying this is the process, I'm saying this is an example of the kind of thing you could do.


Yann LeCun: Human-Level Artificial Intelligence | AI Podcast Clips

What do you think it takes to build a system with human level intelligence? You talked about the AI system in the movie, Her, being way out of reach, our current reach. This might be outdated as well, but- It's still way out of reach. It's still way out of reach. What would it take to build Her, do you think? So I can tell you the first two obstacles that we have to clear, but I don't know how many obstacles there are after this. So the image I usually use is that there is a bunch of mountains that we have to climb and we can see the first one, but we don't know if there are 50 mountains behind it or not. And this might be a good sort of metaphor for why AI researchers in the past have been overly optimistic about the result of AI. You know, for example, Newell and Simon, right, wrote the general problem solver and they called it a general problem solver. General problem solver. And of course, the first thing you realize is that all the problems you want to solve are exponential. And so you can't actually use it for anything useful. But, you know. Yeah, so yeah, all you see is the first peak. So what are the first couple of peaks for Her? So the first peak, which is precisely what I'm working on, is self-supervised learning. How do we get machines to learn models of the world by observation, kind of like babies and like young animals? So we've been working with, you know, cognitive scientists. So this Emmanuelle Dupou, who's at FAIR in Paris, is half-time, is also a researcher in French University. And he has this chart that shows how many months of life baby humans can learn different concepts. And you can measure this in sort of various ways. So things like distinguishing animate objects from inanimate inanimate objects, you can tell the difference at age two, three months. Whether an object is going to stay stable, is going to fall, you know, about four months, you can tell. You know, there are various things like this. And then things like gravity, the fact that objects are not supposed to float in the air, but are supposed to fall, you run this around the age of eight or nine months. If you look at a lot of, you know, eight-month-old babies, you give them a bunch of toys on their high chair. First thing they do is they throw them on the ground when you look at them. It's because, you know, they're learning about, actively learning about gravity. Gravity, yeah. Okay, so they're not trying to annoy you, but they, you know, they need to do the experiment, right? So, you know, how do we get machines to learn like babies? Mostly by observation with a little bit of interaction and learning those models of the world, because I think that's really a crucial piece of an intelligent autonomous system. So if you think about the architecture of an intelligent autonomous system, it needs to have a predictive model of the world. So something that says, here is a world at time T, here is a state of the world at time T plus one if I take this action. And it's not a single answer, it can be a- Yeah, it can be a distribution, yeah. Well, but we don't know how to represent distributions in high-dimensional continuous spaces, so it's gotta be something weaker than that, okay? But with some representation of uncertainty. If you have that, then you can do what optimal control theorists call model predictive control, which means that you can run your model with a hypothesis for a sequence of action and then see the result. Now, what you need, the other thing you need is some sort of objective that you want to optimize. Am I reaching the goal of grabbing this object? Am I minimizing energy? Am I, whatever, right? So there is some sort of objective that you have to minimize. And so in your head, if you have this model, you can figure out the sequence of action that will optimize your objective. That objective is something that ultimately is rooted in your basal ganglia, at least in the human brain, that's what it is, basal ganglia, computes your level of contentment, or miscontentment, I don't know if that's a word, unhappiness, okay? Yeah, yeah. Discontentment. Discontentment, maybe. And so your entire behavior is driven towards kind of minimizing that objective, which is maximizing your contentment, computed by your basal ganglia. And what you have is an objective function, which is basically a predictor of what your basal ganglia is gonna tell you. So you're not gonna put your hand on fire because you know it's gonna burn, and you're gonna get hurt, and you're predicting this because of your model of the world and you're sort of predictor of this objective, right? So if you have those three components, you have four components, you have the hardwired contentment objective computer, if you want, calculator, and then you have the three components, one is the objective predictor, which basically predicts your level of contentment, one is the model of the world, and there's a third module I didn't mention, which is the module that will figure out the best course of action to optimize an objective given your model. Okay? Yeah. Callista Policy Network, or something like that, right? Now, you need those three components to act autonomously intelligently, and you can be stupid in three different ways. You can be stupid because your model of the world is wrong, you can be stupid because your objective is not aligned with what you actually want to achieve. Okay? In humans, that would be a psychopath. Right. And then the third way you can be stupid is that you have the right model, you have the right objective, but you're unable to figure out a course of action to optimize your objective given your model.


Garry Kasparov: Magnus Carlsen is a Lethal Combination of Fischer and Karpov | Lex Fridman Podcast

Where do you put Magnus Carlsen, the current world champion, in the list of all-time greats? In terms of style, moments of brilliance, consistency? It's a tricky question. You know, the moment you start ranking world champions, I think it's not fair because any new generation knows much more about the game than the previous one. So when people say, oh, Gary was the greatest, Fischer was the greatest, Magnus was the greatest, it disregards the fact that the great players of the past, the last year, Cabo Blanc, López, I mean, they knew so little about chess by today's standards. I mean, today, just any kid, you know, that spent a few years with his or her chess computer knows much more about the game simply just because you have access to this information. And it has been discovered generation after generation. We added more and more knowledge to the game of chess. It's about the gap between the world champion and the rest of the field. So it's the... Now, if you look at the gap, then probably Fischer, you know, could be on top, but very short period of time. Then you should also add a time factor. I was on top not as big as Fischer, but much longer. So and also, unlike Fischer, I succeeded in beating next generation. Here's the question. Yeah. Let's see if you still got the fire. Speaking of the next generation, because you did succeed beating the next generation. It's close. Okay. Anand, short Anand, the sheer of... Kramnik is already 12 years younger. So that's a neck. But still yet, I competed with them and I just beat most of them. And I was still dominant when I left at age 41. So back to Magnus. Magnus, I mean, consistency is phenomenal. The reason Magnus is on top and it seems unbeatable today, Magnus is a lethal combination of Fischer and Karpov, which is very unusual because Fischer's style was very dynamic, just fighting to the last point, just using every resource available. Karpov was very different. It's just he had an unparalleled ability to use every piece with a maximum effect, just means minimal resources always produce maximum effect. So now imagine that you merge these two styles. So it's like squeezing every stone for a drop of water, but doing it just for 50, 60, 70, 80 moves. I mean, Magnus could go on as long as Fischer with all his passion and energy, and at the same time being as meticulous and deadly as Karpov by just using every little advantage. So, and he has good, very good health. It's important. I mean, physical conditions are, by the way, very important. So a lot of people don't recognize it. There are latest studies shows that chess players burn thousands of calories during the game. So that puts him on the top of this field of the world champions. But again, it's the discussion that I saw recently in internet, whether Garry Kasparov of his peak, let's say late 80s, could beat Magnus Carlsen today, I mean, it's totally irrelevant because Garry Kasparov in 1989, okay, he's played great chess, but still, I knew very little about chess compared to Magnus Carlsen in 2019, who by the way, learned from me as well. So that's why, yeah. I'm extremely cautious in making any judgment that involves time gaps. You ask soccer fans, so who is your favorite? Pelé, Maradona or Messi? Yeah. Who's your favorite? Messi. Messi. Why? Because- Maybe Maradona, maybe. No, because you're younger, but that's simple. Your instinctive answer is correct because you didn't see Maradona in action. I saw all of them in action, so that's why. But since when I was just following it, it's Pelé and Maradona, they were big stars and it's, Messi's already just, I was gradually losing interest in just other things. So I remember Pelé in 1970, the final match Brazil-Italy. So that's the first World Cup soccer I watched. So that's the, and actually my answer when I just, because I was asked this question as well. So I say that it's just, while it's impossible to make a choice, I would still probably go with Maradona for a simple reason. The Brazilian team in 1970 could have won without Pelé. It was absolutely great. Still could have won. But Argentina team in 1986 without Maradona would not be unified. So this is, and Messi, he still hasn't won a title. That's- Could argue for that for an hour, but you could say if you ask Maradona, if you look in his eyes, especially let's say Garry Kasparov in 1989, he would have said, I was sure as hell would beat Magnus Carlsen. Just simply because- The confidence, the fire. Simply because, simply because again, it's just, they saw me in action. So this, again, it's the age factor that's important. Therefore with the passion and energy and being equipped with all modern ideas. But again, then you make a very just important assumption that you could empower Garry Kasparov in 89 with all ideas that have been accumulated over 30 years. That would not be Garry Kasparov. That would be someone else. Because again, I belong to 1989. I was way ahead of the field and I beat Karpov several times in the world championship matches and I crossed 2,800, which by the way, if you look at the rating, which is just, it's even today, so this is the rating that I retired. So it's still, it's just, it's a top two, two, three. So that's, it's Caruana and Deag. It's about the same rating now. And I crossed 2,800 in 1990. Well, just you look at the inflation. When I crossed 2,800 in 1990, there was only one player in 2,700 category, Anatoly Karpov. Now we had more than 50. So just when you see this, so if you add inflation, so I think my 2,851, it could probably, could be more valuable as Magnus 2,882, which was his highest rating. But anyway, again, too many hypotheticals.