Gary Marcus: Nature vs Nurture is a False Dichotomy | AI Podcast Clips

You've talked about this, you've written about it, you've thought about it, nature versus nurture. So what innate knowledge do you think we're born with and what do we learn along the way in those early months and years? Can I just say how much I like that question? You phrased it just right and almost nobody ever does, which is what is the innate knowledge and what's learned along the way? So many people dichotomize it and they think it's nature versus nurture when it is obviously has to be nature and nurture. They have to work together. You can't learn the stuff along the way unless you have some innate stuff. But just because you have the innate stuff doesn't mean you don't learn anything. And so many people get that wrong, including in the field. Like people think if I work in machine learning, the learning side, I must not be allowed to work on the innate side or that will be cheating. People have said that to me and it's just absurd. So thank you. But you could break that apart more. I've talked to folks who studied the development of the brain and the growth of the brain in the first few days, in the first few months in the womb, all of that, is that innate? So that process of development from a stem cell to the growth of the central nervous system and so on to the information that's encoded through the long arc of evolution. So all of that comes into play and it's unclear. It's not just whether it's a dichotomy or not. It's where most or where the knowledge is encoded. So what's your intuition about the innate knowledge, the power of it, what's contained in it, what can we learn from it? One of my earlier books was actually trying to understand the biology of this. The book was called The Birth of the Mind. How is it the genes even build innate knowledge? And from the perspective of the conversation we're having today, there's actually two questions. One is what innate knowledge or mechanisms or what have you, people or other animals might be endowed with. I always like showing this video of a baby ibex climbing down a mountain. That baby ibex, a few hours after its birth, knows how to climb down a mountain. That means that it knows not consciously something about its own body and physics and 3D geometry and all of this kind of stuff. So there's one question about what does biology give its creatures? What has evolved in our brains? How is that represented in our brains? The question I thought about in the book The Birth of the Mind. And then there's a question of what AI should have. And they don't have to be the same. But I would say that it's a pretty interesting set of things that we are equipped with that allows us to do a lot of interesting things. So I would argue or guess based on my reading of the developmental psychology literature, which I've also participated in, that children are born with a notion of space, time, other agents, places, and also this kind of mental algebra that I was describing before. You know, certain causation if I didn't just say that. So at least those kinds of things. They're like frameworks for learning the other things. So- Are they disjoint in your view or is it just somehow all connected? You've talked a lot about language. Is it all kind of connected in some mesh that's language-like of understanding concepts altogether? I don't think we know for people how they're represented and machines just don't really do this yet. So I think it's an interesting open question, both for science and for engineering. Some of it has to be at least interrelated in the way that like the interfaces of a software package have to be able to talk to one another. So the systems that represent space and time can't be totally disjoint because a lot of the things that we reason about are relations between space and time and cause. So I put this on and I have expectations about what's going to happen with the bottle cap. I put the bottle cap on top of the bottle and those span space and time. If the cap is over here, I get a different outcome. If the timing is different, if I put this here, after I move that, then I get a different outcome that relates to causality. So obviously these mechanisms, whatever they are, can certainly communicate with each other. So I think evolution had a significant role to play in the development of this whole colluge, right? How efficient do you think is evolution? Oh, it's terribly inefficient except that. Well, can we do better? I'll come to that in a second. It's inefficient except that once it gets a good idea, it runs with it. So it took, I guess a billion years, roughly a billion years to evolve to a vertebrate brain plan. And once that vertebrate brain plan evolved, it spread everywhere. So fish have it and dogs have it and we have it. We have adaptations of it and specializations of it. But and the same thing with a primate brain plan. So monkeys have it and apes have it and we have it. So there are additional innovations like color vision and those spread really rapidly. So it takes evolution a long time to get a good idea, being anthropomorphic and not literal here. But once it has that idea, so to speak, which cashes out into one set of genes or in the genome, those genes spread very rapidly. And they're like subroutines or libraries, I guess the word people might use nowadays or be more familiar with. They're libraries that get used over and over again. So once you have the library for building something with multiple digits, you can use it for a hand, but you can also use it for a foot. You just kind of reuse the library with slightly different parameters. Evolution does a lot of that, which means that the speed over time picks up. So evolution can happen faster because you have bigger and bigger libraries. And what I think has happened in attempts at evolutionary computation is that people start with libraries that are very, very minimal, like almost nothing. And then, you know, progress is slow and it's hard for someone to get a good PhD thesis out of it and then give up. If we had richer libraries to begin with, if you were evolving from systems that had an rich innate structure to begin with, then things might speed up. Or more PhD students, if the evolutionary process is indeed, in a meta way, runs away with good ideas, you need to have a lot of ideas, pool of ideas in order for it to discover one that you can run away with. And PhD students representing individual ideas as well. Yeah, I mean, you could throw a billion PhD students at it. The monkeys at typewriters with Shakespeare, yeah. Well, I mean, those aren't cumulative, right? That's just random. And part of the point that I'm making is that evolution is cumulative. So if you have a billion monkeys independently, you don't really get anywhere. But if you have a billion monkeys, and I think Dawkins made this point originally, or probably other people, Dawkins made it very nice in either a selfish teen or blind watchmaker. If there's some sort of fitness function that can drive you towards something, I guess that's Dawkins point. And my point, which is a variation on that, is that if the evolution is cumulative, the related points, then you can start going faster. Do you think something like the process of evolution is required to build intelligent systems? Not logically. So all the stuff that evolution did, a good engineer might be able to do. So for example, evolution made quadrupeds, which distribute the load across a horizontal surface. I don't think Eric come up with that idea. I mean, sometimes good engineers come up with ideas by looking at biology. There's lots of ways to get your ideas. Part of what I'm suggesting is we should look at biology a lot more. We should look at the biology of thought and understanding, and the biology by which creatures intuitively reason about physics or other agents, or how do dogs reason about people? They're actually pretty good at it. If we could understand, at my college we joked dog-nition, if we could understand dog-nition well and how it was implemented, that might help us with our AI. So do you think it's possible that the kind of timescale that evolution took is the kind of timescale that will be needed to build intelligent systems? Or can we significantly accelerate that process inside a computer? I mean, I think the way that we accelerate that process is we borrow from biology. Not slavishly, but I think we look at how biology has solved problems and we say, does that inspire any engineering solutions here? Try to mimic biological systems and then therefore have a shortcut. Yeah, I mean, there's a field called biomimicry, and people do that for material science all the time. We should be doing the analog of that for AI. And the analog for that for AI is to look at cognitive science or the cognitive sciences, which is psychology, maybe neuroscience, linguistics, and so forth. Look to those for insight.


Competing with SpaceX: America Was Built on Competition - Lockheed Martin CTO | AI Podcast Clips

What are your thoughts on the efforts of the new folks, SpaceX and Elon Musk? What are your thoughts about what Elon is doing? Do you see him as competition? Do you enjoy competition? What are your thoughts? First of all, certainly Elon, I'd say SpaceX and some of his other ventures are definitely a competitive force in the space industry. And do we like competition? Yeah, we do. And we think we're very strong competitors. I think it's, you know, competition is what the US is founded on in a lot of ways and always coming up with a better way. And I think it's really important to continue to have fresh eyes coming in, new innovation. I do think it's important to have level playing fields. And so you want to make sure that you're not giving different requirements to different players. But you know, I tell people, you know, I spent a lot of time at places like MIT. I'm going to be at the MIT Beaverworks Summer Institute over the weekend here. And I tell people this is the most exciting time to be in the space business in my entire life. And it is this explosion of new capabilities that have been driven by things like the, you know, the massive increase in computing power, things like the massive increase in comms capabilities, advanced and additive manufacturing are really bringing down the barriers to entry in this field. And it's driving just incredible innovation. It's happening at startups, but it's also happening at Lockheed Martin. I talked about some of these like, you know, Maya and Orion, but we're in the middle of what we call smart sets and software defined satellites that can essentially restructure and re map their purpose, their mission on orbit to give you almost unlimited flexibility for these satellites over their lifetimes. So those are just a couple of examples. But yeah, this is a great time to be in space.


Make Disadvantage Your Superpower | Lex Fridman | AMA #6

Zakaria asks, Dr. Lex, I'm a high school student in Libya. I am very interested in new technology and artificial intelligence, and I've taken the online course CS50x from Harvard. What do you advise people like me to learn? I'm at a disadvantage because of my country's educational system. I one day want to become a great engineer like you, Elon Musk, and many more that may not be as famous. I'm very grateful for the mere chance of you replying to this. What a world we're in, that a kid from nowhere can be asking questions of an MIT researcher. Zakaria, thank you very much for the question. You're too kind. I probably have a lot to say about this topic, but let me try to be brief and to the point. I think it's wonderful that you reached out and that you have a clarity to your passion. Don't lose track of that in your college years and the years to come. Don't forget that you have an interest in artificial intelligence. Don't forget that that fire was there when you were in high school and keep that fire burning. I think the question you've asked is about the disadvantage. I think we live in a world where there's a lot of injustice, where we don't all start from the same place. And that disadvantage doesn't always look the same for everyone. And how to make a more just world is, in many ways, what we're struggling as a society to understand now, but that's a much bigger topic than either you or I. I think what you should be focused on is what you can do for your life. Your circumstance is your circumstance. I've always believed that my disadvantages are my superpowers. And I think your disadvantage is your superpower. Use it as fuel. If you look at stories of great men, of great women in history, that story always has a lot of disadvantages that were overcome. There is something in the process of overcoming hardship that creates greatness. Most people quit through that process. If you can survive it, then your story will be a great one. So the education system in Libya may not be optimal for your passion of artificial intelligence. Perhaps you want to become a professor or a top researcher in the world. It's difficult to go from Libya to one of the best universities in America, for example. But it's not impossible. And once again, the story of great people starts very much like your own story. So practically speaking, what does that mean? I would take your education online and not only to study, but to build stuff. I think the best way you can overcome the physical constraints of your upbringing is by using the internet to build stuff and show it to the world. Build it, complete it, and show it to the world. So that means learning how to program. That means building something that you're really excited about and something that's new. And the best, if it's something that's new and a bunch of people use and it brings them joy or it brings them value, just even the smallest possible thing. So that might start at first by re-implementing things that other people have done, but then finding something that you're curious about, something that kind of gives you a little spark of joy imagining that you might be able to figure out that puzzle. Don't lose track of that joy. Like the moment you detect that this might be something that you might be excited about or you already are excited about, hold onto that. Hold onto that fire and go with it because there's going to be dark periods of time when you lose hope, when you're not so interested in that thing. Keep going and remember that that fire was there. And the thing is, at least for me, I think, when there's a passion, it can wane, it can go up and down, but it always returns. So once you have a passion, take those early few steps, which are always the hardest, and then just keep going. Just keep going one step at a time, keep grinding it out and believe it's gonna work out. So one of the most important thing is to finish, to finish building something cool. Not too ambitious, something small, something cool, something hopefully new. And then show it off to people, show it off to me, and I'll share it with the world. It might not be me, it might be others, and just putting yourself out there in the world, and you never know what might happen. I think in many ways, I feel highly undeserving of the things I've gotten. I have worked extremely hard, but I feel that there's a lot of people that also work hard and are better than me at a lot of the things that I do. So I've been just really fortunate. But I've also seen the magic of just being kind to people, working hard, and putting things you love out there. And the beauty of the internet, as it is, is that when you put stuff you love out there, something good will happen, something magical will happen. It might not be the kind of success you imagine, but it will be a kind of success that will be really fulfilling if you hear it, if you allow your sort of, if you open your heart to it. So like, don't be constrained at thinking that, you know, education at MIT or Stanford is a definition of success. Success might be your own business. Success might be working for a small startup that brings you joy. Success might be using your software engineering skills in your 20s to help out a farmer in agriculture, or a lawyer in a legal profession, or a doctor in medical. Like, the journey might take you all kinds of places, but as long as you allow yourself to be carried by your passion, you'll discover something magical. Again, without jealousy, with honesty, with hard work, with kindness, just putting yourself out there, I think magical things happen. But let me tell you, once again, as you mentioned disadvantage, I think disadvantage on an individual level, I believe you have to see it as a gift for yourself. You have to see it as a superpower. Because when your story is written, and I'm confident it will be written, it'll be a great story. And chapter two and three will include a bunch of disadvantages, a bunch of hardships you have to overcome. In fact, chapter four, five, and six will be hardship too. It's only the later chapters you get to have a little bit more fun. But really, I think overcoming hardship is kind of the fun part in life. So call me crazy, but I would recommend not to listen to anybody who tells you to be safe, to find a stable job. Listen to your heart, listen to your passion. Take bold, big risks when you're young. That is the time to do it. Work hard, build stuff, put it on GitHub, share it with the world, give joy and or value to others, and I think however this crazy world works with love and karma, it will repay you in the end. So good luck, I believe in you. Can't wait to listen to the book about your life on audiobook when I run in my 80s. So Zachariah, I'm cheering for you, and so is the rest of the world. So good luck.