Daniel Kahneman: Deep Learning (System 1 and System 2) | AI Podcast Clips

So we're not talking about humans, but if we think about building artificial intelligence systems, robots, do you think all the features and bugs that you have highlighted in human beings are useful for constructing AI systems? So both systems are useful for perhaps instilling in robots? What is happening these days is that actually what is happening in deep learning is more like a system one product than like a system two product. I mean, deep learning matches patterns and anticipate what's going to happen. So it's highly predictive. But what deep learning doesn't have, and many people think that this is the critical, it doesn't have the ability to reason, so there is no system two there. But I think very importantly, it doesn't have any causality or any way to represent meaning and to represent real interaction. So until that is solved, what can be accomplished is marvelous and very exciting, but limited. That's actually really nice to think of current advances in machine learning as essentially system one advances. So how far can we get with just system one? If we think of deep learning and artificial intelligence systems? It's very clear that deep mind has already gone way beyond what people thought was possible. I think the thing that has impressed me most about the developments in AI is the speed. It's that things, at least in the context of deep learning, and maybe this is about to slow down, but things moved a lot faster than anticipated. The transition from solving chess to solving Go, that's bewildering how quickly it went. The move from alpha Go to alpha zero is sort of bewildering the speed at which they accomplished that. Now, clearly, there are many problems that you can solve that way, but there are some problems for which you need something else. Something like reasoning. Well, reasoning and also, you know, one of the real mysteries, psychologist Gary Marcus, who is also a critic of AI, I mean, what he points out, and I think he has a point, is that humans learn quickly. Human don't need a million examples, they need two or three examples. So clearly, there is a fundamental difference. And what enables a machine to learn quickly, what you have to build into the machine, because it's clear that you have to build some expectations or something in the machine to make it ready to learn quickly, that at the moment seems to be unsolved. I'm pretty sure that DeepMind is working on it, but if they have solved it, I haven't heard yet. They're trying to actually, them and OpenAI are trying to start to get to use neural networks to reason. So assemble knowledge, of course, causality is, temporal causality is out of reach to most everybody. You mentioned the benefits of System 1 is essentially that it's fast, allows us to function in the world. Fast and skilled, yeah. It's skilled. And it has a model of the world. You know, in a sense, I mean, there was the earlier phase of AI attempted to model reasoning, and they were moderately successful, but, you know, reasoning by itself doesn't get you much. Deep learning has been much more successful in terms of, you know, what they can do. But now, it's an interesting question, whether it's approaching its limits. What do you think? I think absolutely. So I just talked to Gian Lacune, you mentioned, you know, so he thinks that the limits, we're not going to hit the limits with neural networks, that ultimately this kind of System 1 pattern matching will start to start to look like System 2 without significant transformation of the architecture. So I'm more with the majority of the people who think that yes, neural networks will hit a limit in their capability. He, on the one hand, I have heard him tell the Mises-Sabis essentially that, you know, what they have accomplished is not a big deal, that they have just touched, that basically, you know, they can't do unsupervised learning in an effective way. But you're telling me that he thinks that the current, within the current architecture, you can do causality and reasoning? So he's very much a pragmatist in a sense that's saying that we're very far away, that there's still, I think there's this idea that he says is we can only see one or two mountain peaks ahead, and there might be either a few more after or thousands more after. So that kind of idea. I heard that metaphor. Right, but nevertheless, it doesn't see a, the final answer not fundamentally looking like one that we currently have. So neural networks being a huge part of that. Yeah, I mean, that's very likely because, because pattern matching is so much of what's going on. And you can think of neural networks as processing information sequentially. Yeah, I mean, you know, there is, there is an important aspect to, for example, you get systems that translate and they do a very good job, but they really don't know what they're talking about. And for that, I'm really quite surprised. For that, you would need, you would need an AI that has sensation, an AI that is in touch with the world. Yes, self-awareness, and maybe even something that resembles consciousness kind of ideas. Certainly awareness of, you know, awareness of what's going on so that the words have meaning or can get, are in touch with some perception or some action. Yeah, so that's a big thing for Jan, and what he refers to as grounding to the physical space. So that's what, we're talking about the same thing. Yeah, so, but so how, how you ground? I mean the grounding, without grounding, then you get, you get a machine that doesn't know what it's talking about, because it is talking about the world, ultimately. The question, the open question is what it means to ground. I mean, we're very human-centric in our thinking, but what does it mean for a machine to understand what it means to be in this world? Does it need to have a body? Does it need to have a finiteness like we humans have? All of these elements, it's a very, it's an open question. You know, I'm not sure about having a body, but having a perceptual system, having a body would be very helpful too. I mean, if, if you think about human, mimicking human, but having a perception, that seems to be essential, so that you can build, you can accumulate knowledge about the world. So if, you can imagine a human completely paralyzed, and there is a lot that the human brain could learn, you know, with a paralyzed body. So if we got a machine that could do that, that would be a big deal. And then the flip side of that, something you see in children, and something in machine learning world is called active learning, maybe it is also, is being able to play with the world. How important for developing system one or system two, do you think it is to play with the world? To be able to interact with the world? Well, certainly a lot, a lot of what you learn is you learn to anticipate the outcomes of your actions. I mean, you can see that how babies learn it, you know, with their hands, how they, how they learn, you know, to connect, you know, the movements of their hands with something that clearly is something that happens in the brain, and the ability of the brain to learn new patterns. So you know, it's the kind of thing that you get with artificial limbs, that you connect it and then people learn to operate the artificial limb, you know, really impressively quickly, at least from what I hear. So we have a system that is ready to learn the world through action. At the risk of going into way too mysterious of land, what do you think it takes to build a system like that? Obviously, we're very far from understanding how the brain works, but how difficult is it to build this mind of ours? You know, I mean, I think that Jan LeCun's answer that we don't know how many mountains there are, I think that's a very good answer. I think that, you know, if you look at what Ray Kurzweil is saying, that strikes me as off the wall. But I think people are much more realistic than that, where actually Demis Hassabis is and Jan is, and so the people are actually doing the work fairly realistic, I think. To maybe phrase it another way, from a perspective not of building it, but from understanding it, how complicated are human beings in the following sense? You know, I work with autonomous vehicles and pedestrians, so we try to model pedestrians. How difficult is it to model a human being, their perception of the world, the two systems they operate under, sufficiently to be able to predict whether the pedestrian's going to cross the road or not? I'm fairly optimistic about that, actually, because what we're talking about is a huge amount of information that every vehicle has and that feeds into one system, into one gigantic system. And so anything that any vehicle learns becomes part of what the whole system knows. And with a system multiplier like that, there is a lot that you can do. So human beings are very complicated, and the system is going to make mistakes, but humans make mistakes. I think that they'll be able to, I think they are able to anticipate pedestrians, otherwise a lot would happen. They're able to get into a roundabout and into traffic, so they must know both to expect or to anticipate how people will react when they're sneaking in. And there's a lot of learning that's involved in that. Basically the pedestrians are treated as things that cannot be hit, and they're not treated as agents with whom you interact in a game-theoretic way. So I mean, it's a totally open problem, and every time somebody tries to solve it, it seems to be harder than we think. And nobody's really tried to seriously solve the problem of that dance, because I'm not sure if you've thought about the problem of pedestrians, but you're really putting your life in the hands of the driver. You know, there is a dance, there's part of the dance that would be quite complicated, but for example, when I cross the street and there is a vehicle approaching, I look the driver in the eye, and I think many people do that. And you know, that's a signal that I'm sending, and I would be sending that machine to an autonomous vehicle, and it had better understand it, because it means I'm crossing. So and there's another thing you do, that actually, so I'll tell you what you do, because I've watched hundreds of hours of video on this, is when you step in the street, you do that before you step in the street. And when you step in the street, you actually look away. Look away. Yeah. What is that, what that saying is, I mean, you're trusting that the car, who hasn't slown down yet, will slow down. Yeah. And you're telling him, I'm committed. I mean, this is like in a game of chicken. So I'm committed. And if I'm committed, I'm looking away. So there is, you just have to stop. So the question is whether a machine that observes that needs to understand mortality. Here I'm not sure that it's got to understand so much as it's got to anticipate. So and here, but you know, you're surprising me, because here I would think that maybe you can anticipate without understanding, because I think this is clearly what's happening in playing go or in playing chess. There's a lot of anticipation, and there is zero understanding. So I thought that you didn't need a model of the human and the model of the human mind to avoid hitting pedestrians. But you are suggesting that actually you do. And then it's a lot harder. So this is. Just to clarify, this is not entirely based on just the model of the human being, rather


I fasted for 3 days | Lex Fridman

I'm doing a 72 hour fast, three days, only water, coffee, and electrolytes, sodium, potassium, magnesium. The video is brought to you by Cash App. Thanks a lot to them for being a long-time supporter of this channel. Also, Athletic Greens for being a long-time supporter of my diet. Electrolytes are coming from a few sources, including Element, one packet of which has one gram of sodium, 200 milligrams of potassium, and 60 milligrams of magnesium. I also have these in pill form, one gram of sodium, and 120 milligrams of magnesium. That's good, especially when I'm doing a lot of running. I'm currently 25 hours into the fast. I'm tracking it with the Xero app. They used to be a sponsor of the podcast. I'm not sure if they're a sponsor anymore. Anyway, I love using them. I don't care if they're a sponsor or not. That's actually true for all the sponsors. I'm still doing light exercise. I just did a slow pace run for about four miles. I also did about 100 pushups and about 50 or 60 pull-ups. I'm hoping to do the same tomorrow and the day after. Why am I doing the fast? That's an important question. I think first and foremost is to reflect on life. I know perhaps you don't need it, but I personally like to create forcing functions for myself to pause and reflect on all the things that I'm grateful for in this life. Really, I'm talking about just simple things, the fact that I'm alive at all, the fact that I have food, the fact that I have shelter, the fact that I have people who I love and that bring joy to my life, and the fact that I love basically everything about this life, all the things I get to do, I really enjoy. In fact, I enjoy basically anything I do. That I'm deeply grateful for. I'm not exactly sure why, but there's something about fasting that forces you to really appreciate how short this ride is. Like the fact that you need food to run this thing somehow is a reminder that this is a thing with an expiration date, a quickly approaching one. I'm just a hamster briefly stepping out of the hamster wheel to reflect on how awesome it is to be in this one particular cage we call Earth. I intermittent fast 16, 8, or 24 hours very often. It always makes me feel great. The second reason I'm doing the fast is for the health benefits. You can look into the science. There's a lot of writing out there about it, but I tend to prefer to be a scientist of N of 1, of myself. Every time I do the 16, 8 intermittent or the one meal a day 24 hour fast, I just feel great. The way my mind feels, the way my body feels. I want to try out what 72 hours feels like because I heard great things. Great things. People throw all kinds of medical terms around like autophagy. From a robotics perspective, I tend to think of it as a hard drive defrag, maybe like a software update. So for the hardware layer, it's in the cells, it's like a defrag. And the software layer, it's like a software update. Let's think of it that way. Finally, the third reason I'm doing the fast is to reset the body and mind. I've relaxed for the last couple of months or two. A little bit too much brisket, a little bit too much whiskey with some really kind, beautiful Texan people. But I have a lot of really difficult tasks coming up. They've been piling up and I need to get the stuff done. So it's time to get back to work, time to get focused again, time to make the best of my days, my hours and minutes. I should probably say that I currently feel great. My mind is focused. My body feels great after the run. The run felt great. Everything feels great. But this is charted territory. I am now going into uncharted territory, 48 hours towards the 72 hour mark. So let's see how that goes. But so far, it's a beautiful thing. Let's see how I feel tomorrow. Quick check-in at the end of day two, 49 hours and 57 minutes in. It says 69% of the way in, 69. I got a pretty good run in, actually about six miles and I'm now going to do about 100 push-ups and 50 pull-ups. I'm feeling pretty good. It's been a crazy busy day. I got a chance to do a four-hour podcast today. I did my usual mumbling and incohesive conversational skills, but not worse than usual. So it's at the same level and my mind is actually sharper than ever. It's really able to focus on the main task I have to do. And there's a liberating aspect to not having to think about food. Even though I eat rarely throughout the day and I eat the same thing, there's still a kind of anticipation towards what you're going to eat later in the day. As I mentioned in the podcast, actually, it feels like thinking about food is kind of like when you talk about the weather. It's like the easy thought to have. And when you don't have a chance to have the easy thoughts, you can now focus on the more difficult thoughts. Like I can anticipate the exciting thing I'm doing later tonight. I'm actually going to read The Hunger Artist again by Kafka, as was mentioned in the podcast. I used to love that book and I used to read it a lot before I even discovered even intermittent fasting. This was many, many years ago. I was captivated by the madness and the artistry of what the main character was doing. I recommend that. I think it's a short story, if I remember correctly, The Hunger Artist. Anyway, I feel great. I got a chance to kind of breathe in and appreciate the hell out of life, especially the conversation I had today was on the self-destruction of human civilization via the man-made engineering of viruses. So the fasting combined with pondering all the ways in which human civilization will destroy itself made me really appreciate the hell out of being alive. After the past couple of months of eating a whole lot of brisket and drinking a whole lot of whiskey with a lot of amazing people, I definitely have some reserves to basically fast for a very long time, so I'll be okay. I'm feeling really good. It's actually kind of exciting how this particular biological system is able to store fat for when you actually need to fast. Not this like first world, I'm going to choose to fast, but more like I have to fast because there's no food. So it's kind of amazing how biology works like that. But I definitely have a bunch of fat to lose so I need to harden the body, harden the mind, and this is a good reset for that. In the summer, I'm doing some difficult projects in my work and going to have some fun with some difficult projects in the grappling world and going to do some running with a man, Mr. David Goggins. So I can't be full of brisket and whiskey for that, or at least if I am, I need to make sure I'm doing some running and exercise. Anyway, that's way too much talking for the day. I'm going to go do some push-ups and pull-ups, take a shower, and get back to work. I'll see you tomorrow. I'm 69% in. I have 31% left and I'm looking forward to breaking the fast tomorrow, responsibly, probably with some bone broth and just a little bit of chicken breast. I'm not going to do the crazy break the fast with a giant steak, although I'm very tempted to. Even the thought of that is making me hungry. My stomach just actually woke up. It's like, wait, are we going to have some steak? No. Sometimes it feels like there's not enough hours in the day, but at the same time, when you're running and there's a little bit of hunger and you haven't eaten in a long time and you breathe in, it feels like there's so many seconds in the day. It's just, the day is so long and so full of amazing experiences. That duality is so real when you're fasting that I have so much shit to do. I don't know how I'm going to get it done. At the same time, every second is full of an eternity that I'm just grateful for. See you tomorrow. All right, here we are at the end of this fun little journey. Day three, 75 hours and 44 minutes in. I just weighed myself and I'm 170 on the dot. Actually, the first time I stepped on the scale, it said 169.9, and then I couldn't believe it. So I stepped on again and it was 170. That's why I took the picture. And it was never 169.9 again. I also weighed myself right after eating three days ago and I was just a little bit over 180. So it's about 10 pounds of water weight that I lost. Maybe I would estimate it about like one pound of actual fat. That's not why I did it. I did it to challenge myself. I did it to reset the mind. I did it to take pause and reflect on all the things I'm grateful for. Maybe if I can give a few takeaways. Usually when I do a 16 hour fast or a 24 hour fast, one of the biggest benefits is the laser focus that I'm able to attain and maintain. That rhymes. Clearly, I have not lost my sense of humor, but it hasn't improved at all either. So it's at that low bar of dad jokes, plus a little extra loopy, but it is what it is. I think for day two and day three, there was a bit of an up and down on the focus, but mostly I've been able to get even an extra level of focus that I really enjoyed. The little bit of a downside, there's a bit of a darkness to the way I saw the world. Like I was a little bit sadder, I would say, or maybe I was just acknowledging the suffering that is life. Maybe I was more honest. I'm not sure what kind of experience people usually have with sleep, but I had excellent sleep. I slept over eight hours on all the nights. I woke up refreshed. I was feeling great. Plus I was doing the running. So I ran today again, four miles. I did pushups and pull-ups, feeling great. I was taking the element potassium, magnesium, sodium, plus the salt pills. Every time I took that stuff, maybe it's placebo, but it made me feel really great. So a lot of water. I did drink tea on Sweden, obviously just green tea with mint. And actually two things helped with hunger. It was a hot tea or really cold water. I don't know how that works. I'm sure there's an explanation for it, but I did feel hungry in unpleasant ways, like a hunger ache earlier on in the process, but I kind of reframed that as an opportunity to sort of pause and think, and then the hunger kind of faded away. Because when you realize it's not that big of a deal, I have a lot of fat, I'm going to be okay, the body's naturally going to go to the fat stores. I'm talking way too much. I may have mentioned this before, but one of the big things I noticed is just the experience of being free of having to think about what I'm going to eat today. Even though I eat the same exact thing, still the experience was that I felt free to sort of focus and think about the things I have going on outside of food, like the problems I'm working on, the things I have on my schedule, even just like small stuff. As I'm saying it now, it doesn't really make sense, but I really did feel like I gained a lot of hours in the day by not having to think about food at all. One of the tougher experiences actually was about an hour ago when I went to the grocery store to get the chicken breast and the chicken bone broth, which is what I'm going to break the fast with. I don't know if it was difficult or just like meditative and beautiful of an experience of just walking through, especially the fresh produce part of the store, and just everything looked amazing. The reason I'm breaking the fast with just a little bit of chicken breast and a little bit of bone broth is because I hear bad stories when you just overeat, which is exactly what I want to do right now. I want to eat a giant steak, but I'm going to be smart about this. I'm going to have a little chicken broth, a little bit of chicken breast, and then just wait three or four hours and then eat again about a thousand calories, just make a steak, a thousand, fifteen hundred calories at night. That's it. That'd be it for the day. Then return back to normal diet, normal strict diet tomorrow. Then continuing the running, the pushups and pull-ups. Again, harden the body, harden the mind. I'm not sure if I fully conveyed this, but it was actually more challenging than I expected, but it's not so hard that I can't do it regularly. I like the Georges St-Pierre model of doing it like four times a year, especially after you fall off the wagon of dieting. I think I'll do that. Maybe I'll try like five, seven day fast at some point. Okay, way too much talking. Time to go eat. I've been looking forward to this all day. Maybe I'll get the first couple bites on camera. Let's see how good this chicken and bone broth tastes. All right, here we go. No reason to wait. We're at 76 hours and 24 minutes, and I have looked forward to this very plain, but very incredibly delicious looking chicken for quite a few hours now. So here we go. This is some good chicken. I taste the caramelized part, but most of all I taste the coconut oil actually, and the salt. The oil and the salt. I think that's what my body was craving the most. Given my keto diet, I think the fat is what I was craving. Even though I have plenty of fat, the body still, I want more. Again, it's the salt. This tastes delicious. I can't. It would be hard to just eat this, but that's all I'm doing for the next, let's say, four hours about. It's just the bone broth and the chicken. This is a really memorable moment. Oh man. Life is made up of moments like this. It doesn't take much. This is the best tasting chicken breast I've ever had in my life. Just chicken breast, some spices, salt, and 76 hours of fasting. This is incredible. Damn. Damn, it's good to be alive.


Sterling Anderson, Co-Founder, Aurora - MIT Self-Driving Cars

Today we have Sterling Anderson. He's the co-founder of Aurora, an exciting new self-driving car company. Previously, he was the head of the Tesla Autopilot team that brought both the first and second generation autopilot to life. Before that, he did his PhD at MIT working on shared human machine control of ground vehicles, the very thing I've been harping on over and over in this class. And now he's back at MIT to talk with us. Please give him a warm welcome. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. It's good to be here. I was telling Lex just before, I think it's been a little while since I've been back to the Institute, and it's great to be here. I want to apologize in advance. I've just landed this afternoon from Korea via Germany where I've been spending the last week. And so I may speak a little slower than normal. Please bear with me. If I become incoherent or slur my speech, somebody flag it to me and we'll try to make corrections. So tonight I thought I'd chat with you a little bit about my journey over the last decade. It's been just over 10 years since I was at MIT. A lot has changed. A lot has changed for the better in the self-driving community. And I've been privileged to be a part of many of those changes. And so I wanted to talk with you a little bit about some of the things that I've learned, some of the things that I've experienced. And then maybe end by talking about sort of where we go from here and what the next steps are, both for the industry at large, but also for the company that we're building that as Lex mentioned is called Aurora. To start out with, there are a few sort of key phases or transitions in my journey over the last 10 years. As Lex mentioned, when I started at MIT, I worked with Carl Ianniema, Emilio Fazzoli, John Leonard, a few others on some of these sort of shared, adaptive automation approaches. I'll talk a little bit about those. From there, I spent some time at Tesla where I first led the Model X program as we both finished the development and ultimately launched it. I took over the autopilot program where we introduced a number of new, both active safety, but also sort of, you know, enhanced convenience features from auto steer to adaptive cruise control that we're able to refine in a few unique ways. And we'll talk a little bit about that. And then from there in December of last year, of 2016, I guess now, we started a new company called Aurora. And I'll tell you a little bit about that. So to start out with, when I came to MIT, it was 2007. The DARPA urban challenges were well underway at that stage. And one of the things that we wanted to do is find a way to address some of the safety issues in human driving earlier than potentially full self-driving could do. And so we developed what became known as the Intelligent Co-Pilot. What you see here is a simulation of that operating. I'll tell you a little bit more about that in just a second. But to explain a little bit about the methodology, the innovation, the key approach that we took that was slightly different from what traditional planning control theory we're doing was instead of designing in path space for the robot, we instead found a way to identify, plan, optimize, and design a controller subject to a set of constraints rather than paths. And so what we were doing is looking for homotopies through an environment. So imagine for a moment an environment that's pockmarked by objects, by their vehicles, by pedestrians, et cetera. If you were to create the Voronoi diagram through that environment, you would have a set of each unique set of paths or homotopies, continuously deformable paths that will take you from one location to another through it. If you then turn that into its dual, which is the Delaunay triangulation of said environment, presuming that you've got convex obstacles, you can then tile those together rather trivially to create a set of homotopies and transitions across which those paths can stake out sort of a given set of options for the human. It turns out humans tend to, this tends to be a more intuitive way of imposing certain constraints on human operation rather than enforcing that the ego vehicle stick to some arbitrary position within some distance of a safe path. You instead look to enforce only that the state of the vehicle remain within a constraint-bounded n-dimensional tube in state space. Those constraints being spatial, you imagine for a moment edges of the roadway or circumventing various objects in the roadway. Imagine them also being dynamic, right? So limits of tire friction imposed limits on side-slip angles. And so using that, what we did is found a way to create those homotopies, forward simulate the trajectory of the vehicle given its current state and some optimal set of control inputs that would optimize its stability through that. We use model predictive control in that work. And then taking that forward simulated trajectory, computing some metric of threat. For instance, if the objective function for that minimize the or maximize stability or minimize some of these parameters like wheel side slip, then wheel side slip is a fairly good indication of how threatening that optimal maneuver is becoming. And so what we did is then use that in a modulation of control between the human and the car, such that should the car ever find itself in a state where that forward simulated optimal trajectory is very near the limits of what the vehicle and it can actually handle, we will have transitioned control fully to the vehicle, to the automated system so that it can avoid an accident. And then it transitions back in some manner. And we played with a number of different methods of transitioning this control to ensure that we didn't throw off the human mental model, which was one of the key concerns. We also wanted to make sure that we were able to arrest accidents before they happen. What you see here is a simulation that was fairly faithful to the behavior we saw in test drivers up in Dearborn, Michigan. Ford provided us with a Jaguar S-type to test this on. And what we did, so what you see here is there's a blue vehicle and a gray vehicle. Both, in both cases, we have a poorly tuned driver model, in this case, a pure pursuit controller with a fairly short look ahead, shorter than would be appropriate given this scenario and these dynamics. The gray vehicle is without the intelligent copilot in the loop. You'll notice that, obviously, the driver becomes unstable, loses control, and leaves the safe roadway. The copilot, remember, is interested not in following any given path. It doesn't care where the vehicle lands on this roadway, provided it remains inside the road. In the blue vehicle's case, it's the exact same human driver model, now with the copilot in the loop. You'll notice that as this scenario continues, what you see here on the left is in this green bar is the portion of available control authority that's being taken by the automated system. You'll notice that it never exceeds half of the available control, which is to say that the steering inputs received by the vehicle end up being a blend of what the human and what the automation are providing. And what results is a path for the blue vehicle that actually better tracks the human's intended trajectory than even the copilot understood. Again, the copilot is keeping the vehicle stable, is keeping it on the road. The human is hewing to the center line of that roadway. So there were some very interesting things that came out of this. We did a lot of work in understanding what kind of feedback was most natural to provide to a human. Our biggest concern was if you throw off a human's mental model by causing the vehicle's behaviors to deviate from what they expect it to do in response to various control inputs, that that could be a problem. So we tried various things from adjusting, for instance, one of the key questions that we had early on was, if we couple the computer control and the human control via planetary gear and allow the human to feel actually a backwards torque to what the vehicle is doing, so the car starts to turn right, human will feel the wheel turn left. They'll see it start to turn left. Is that more confusing or less confusing to a human? And it turns out it depends on how experienced a human is. Some drivers will modulate their inputs based on the torque feedback that they feel through the wheel. And for instance, a very experienced driver expects to feel the wheel pull left when they're turning right. However, less experienced drivers, in response to seeing the wheel turning opposite to what the car is supposed to be doing, that's a rather confusing experience. So there were a lot of really interesting human interface challenges that we were dealing with here. We ended up working through a lot of that, developing a number of micro applications for it. One of those, at the time, Gil Pratt was leading a DARPA program focused on what they called at the time maximum mobility manipulation. We decided to see what this system could do in application to unmanned ground vehicles. So in this case, what you see is a human driver sitting at a remote console, as one would when operating an unmanned vehicle, for instance, in the military. What you see on the top left is the top-down view of what the vehicle sees. I should have played this in repeat mode. With bounding boxes bounding various cones. And what we did is we set up about 20 drivers, 20 test subjects, looking at this control screen and operating the vehicle through this track. And we set this up as a race with prizes for the winners, as one would expect, and penalized them for every barrel they hit. If they knocked over the barrel, I think they got a five-second penalty. If they brushed a barrel, they got a one-second penalty. And they were to cross the field as fast as possible. And they had no line of sight connection to the vehicle. And we played with some things on their interface. We caused it to drop out occasionally. We delayed it, as one would realistically expect in the field. And then we either engaged or didn't engage the co-pilot to try to understand what effect that had on their performance and their experience. And what we found was not surprisingly, the incidence of collisions declined. It declined by about 72% when the co-pilot was engaged versus when it was not. We also found that even with that 72% decline in collisions, the speed increased by, I'm blanking on the amount, but it was 20% to 30%-ish. Finally, and perhaps most interesting to me, after every run, I would ask the driver, and again, these were blind tests. They didn't know if the co-pilot was active or not. And I would ask them, how much control did you feel like you had over the vehicle? And I found that there was a statistically significant increase of about 12% when the co-pilot was engaged. And that is to say, drivers reported feeling more control of the vehicle 12% more of the time when the co-pilot was engaged than when it wasn't. And then I looked at the statistics, it turns out they actually, the average level of control that the co-pilot was taking was 43%. So they were reporting that they felt more in control when in fact they were 43% less in control, which was interesting and I think bears a little bit on sort of the human psyche in terms of, they were reporting the vehicle was doing what I wanted it to do, maybe not what I told it to do, which was kind of fun observation. And fun to, I think the most enjoyable part of this was getting together with the whole group at the end of the study and presenting some of this and seeing some of the reactions. So from there, we looked at a few other areas. My, Carl Yanim and I looked at a few different opportunities to commercialize this. Again, this was years ago and the industry was in a very different place than it is today. We started a company first called Gimlet, then another called Ride. This is the logo, it may look familiar to you. We turned that into, at the time it intended to roll this out across various automakers in their operations. At the time, very few saw self-driving as a technology that was really gonna impact their business going forward. They were, in fact, even ride sharing at the time was a fairly new concept that was, I think, to a large degree viewed as unproven. So, as I mentioned, December of last year, I co-founded Aurora with a couple of folks who have been making significant progress in this space for many years. Chris Hermsen, who formerly led Google's self-driving car group. Drew Bagnell is a professor at Carnegie Mellon University, exceptional machine learning and applied machine learning, was one of the founding members of Uber's self-driving car team and led autonomy and perception there. We felt like we had a unique opportunity at the convergence of a few things. One, the automotive world has really come into the full-on realization that self-driving, and particularly self-driving and ride sharing, and vehicle electrification are three vectors that will change the industry. That was something that didn't exist 10 years ago. Two, significant advances have been made in some of these machine learning techniques, in particular deep learning and other neural network approaches, in the computers that run them, and the availability of low-power GPU and TPU options to really do that well, in sensing technologies, in high-resolution radar, and a lot of the LIDAR development. So it's really a unique time in the self-driving world. A lot of these things are really coming together now. And we felt like by bringing together an experienced team, we had an interesting opportunity to build, from a clean sheet, a new platform, a new self-driving architecture, that leveraged the latest advances in applied machine learning, together with our experience of where some of the pitfalls tend to be down the road as you develop these systems, because you don't tend to see them early on. They tend to express themselves as you get into the long tail of corner cases that you end up needing to resolve. So we've built that team. We have offices in Palo Alto, California, and Pittsburgh, Pennsylvania. We've got fleets of vehicles operating in both Palo Alto and Pennsylvania. A couple of weeks ago, we announced that Volkswagen Group, one of the largest automakers in the world, Hyundai Motor Company, also one of the largest automakers in the world, have both partnered with Aurora. We will be developing, and are developing with them, a set of platforms, and ultimately will scale that, our technology, on their vehicles across the world. And one of the important elements of building Lex, I asked Lex before coming out here what this group would be most interested in hearing. One of the things that he mentioned was, what does it take to build a self-driving, build a new company in a space like this? One of the things that we found very important was a business model that was non-threatening to others. We recognize that our strengths and our experience over the last, in my case, a decade, in Chris's case, almost two, really lies in the development of the self-driving systems. Not in building vehicles, though I have had some experience there, but in developing the self-driving. And so our feeling was, if our mission is to get a technology to market as quickly, as broadly, and as safely as possible, that mission is best served by playing our position and working well with others who can play theirs, which is why you see the model that we've adopted, and is now, you'll start to see some of the fruits of that through these partnerships with some of these automakers. So at the end of the day, our aspiration and our hope is that this technology that is so important in the world in increasing safety, in improving access to transportation, in improving efficiency, in the utilization of our roadways and our cities. This is maybe the first talk I've ever given where I didn't start by rattling off statistics about safety and all these other things. If you haven't heard them yet, you should look them up. They're stark, right? The fact that most vehicles in the United States today have an average, on average, three parking spaces allocated to them. The amount of land that's taken up across the world in housing vehicles that are used less than 5% of the time. The number of people, I think in the United States, the estimate has been somewhere between six and 15 million people don't have access to the transportation they need, because they're elderly or disabled or one of many other factors. And so this technology is potentially one of the most impactful for our society in the coming years. It's a tremendously exciting technological challenge. And at the confluence of those two things, I think is a really unique opportunity for engineers and others who are not engineers who really want to get involved to play a role in changing our world going forward. So with that, maybe I'll stop with this and we can go to questions. Let's give Daryl in the warm hand. Hi, I'm Wayne, thanks for coming. I have a question. A lot of self-driving car companies are making extensive use of LiDAR, but you don't see a lot of that with Tesla. I wanted to know if you had any thoughts about that. Yeah, I don't want to talk about Tesla too much in terms of our specific, anything that wasn't public information I'm not gonna get into. I will say that for Aurora, we believe that the right approach is getting to market quickly and you get to market and doing so safely. And you get to market most quickly and safely if you leverage multiple modalities, including LiDAR. These are all just to clarify what's running in the background. These are all just Aurora videos of our cars driving on various test routes. Hi, I'm Luke from the Sloan School. A lot of customers have visceral-type connections to their automobile. I was wondering how you see that market, the car enthusiast market, being affected by AVs and then vice versa, how the AVs will be designed around those type of customers. Yeah, that's a good question. Thanks for asking, Luke. I am one of those enthusiasts. I very much appreciate being able to drive a car in certain settings. I very much don't appreciate driving in others. I remember distinctly several evenings, almost literally pounding my steering wheel, sitting in Boston traffic, on my way to somewhere. I do the same in San Francisco. I think the opportunity really is to turn sort of personal vehicle ownership and driving into more of a sport and something you do for leisure. I see it, a gentleman some time ago asked me to talk, hey, don't you think this is a problem for the country, I think you meant the world, if people don't learn how to drive? That's just something a human should know how to do. My perspective is it's as much of a problem as people not intrinsically knowing how to ride a horse today. If you wanna know how to ride a horse, go ride a horse. If you wanna race a car, go to a racetrack or go out to a mountain road that's been allocated for it. Ultimately, I think there is an important place for that because I certainly agree with you. I'm very much a vehicle enthusiast myself, but I think there is so much opportunity here in alleviating some of these other problems, particularly in places where it's not fun to drive, that I think there's a place for both. Yeah. All right. Can you hear or do I need to get? Yeah. Congratulations on the partnership that was announced recently, I think. So I have a two-part question. The first one is, so we heard last week from, I think there was a gentleman from Waymo, talking about how long they've been working on this autonomous car technology. And you seem to have ramped up extremely fast. So is there a licensing model that you've taken? I mean, how are you able to commercialize the technology in one year? So just to be clear, we're not actually commercializing. Just to distinguish, we are partnering and developing vehicles and we'll ultimately be running pilots, as we announced a week or two ago with the Moya shuttles. We are, however, I will distinguish that from broad commercialization of the technology. And I don't want to get too much into the nuances of that business model. I will say that it is one that's done in very close partnership with our automotive partners. Because at the end of the day, they understand their cars, they understand their customers, they have distribution networks. They are, our automotive partners are fairly well positioned provided they have the right support in developing the self-driving technology, they're fairly well positioned to roll it out at scale. So the second part of my question is, again, looking at this pace of adoption and the maturity of technology, do you see an open source model for autonomous cars as they become more and more? Unclear. I'm not convinced that an open source model is what gets to market most quickly. In the long run, it's not clear to me what will happen. I think there will be a handful of successful self-driving stacks that will make it. Nowhere near the number of self-driving companies today, but a handful, I think. Two questions, one is, in variably new product development, there's typically two types of bottlenecks. There's a technological bottleneck and an economic bottleneck, right? So technological bottleneck might be, hey, the sensors aren't good enough or the machine learning algorithms aren't good enough and so on, I'd be interested to hear, and it'll shift obviously over time. So I'd be interested to know what you would say is the current thing that if, hey, if this part of the architecture was 10 times better, we would, and then on the economic side, I'd be interested to know, gee, if sensors were 100 times cheaper, then so I'd be interested to hear your perspective on both. That's a great question. Let me start with the economic side of it and just to get that out of the way, because it's a little bit quicker answer. The economics of operating a self-driving vehicle on a shared network today would close, that business case closes even with high costs of sensors. That is not what's stopping us. And that's part of why the gentleman earlier who asked, should you use LiDAR or not? If your target is to initially deploy these in fleets, you would be wise to start at the top end of the market, develop and deploy a system that's as capable as possible, as quickly as possible, and then cost it down over time. And you can do that as computer vision, precision recall increase. Today, they're not good enough. And so economically, depending on your model of going to market, and we believe that the right model is through mobility services, you can cost down, you'll cost down the center. Inevitably, there's no unobtainium in LiDAR units today. There's no reason fundamentally that a should cost of a LiDAR unit will lead you to a $70,000 price point. However, if you build anything in low enough volumes, it's gonna be expensive. Many of these things will work their way into the standard automotive process. They'll work their way into tier one suppliers, and when they do, the automotive community has shown themselves to be exceptional at driving those costs down. And so I expect them to come way down. To your other question, technological bottlenecks and challenges. One of the key challenges of self-driving is and remains that of forecasting the intent and future behaviors of other actors, both in response to one another, but also in response to your own decisions in motion. That's a perception problem, but it's something more than a perception problem. It's also a prediction, and there are a number of different things that have to come together to solve this. We're excited about some of the tools that we're using in interleaving various modern machine learning techniques throughout the system to do things like project our own behaviors that were learned for the ego vehicle on others, and assume that they'll behave as we would had we been in that situation. Like an expert system kind of approach, right? Yeah, yeah, you assume nominal behavior and you guard against off nominal, right? But it's very much, it's not a solved problem, I wouldn't say. It's very much as you get into that really long tail of development, when you're no longer putting out demonstration videos, but you're instead just putting your head down and eking out those final nines, that's the kind of problem you tend to deal with. Yeah, thank you. Hi, so this question isn't necessarily about the development of self-driving cars, but more of like an ethics question. When you're putting human lives into like the hands of software, isn't there always the possibility for like outside agents with malicious intent to use it for their own gain? And how do you guys, if you do have a plan, how do you intend to protect against attacks like that? So security is a very real aspect of this that has to be solved. It's a constant game of cat and mouse. And so I think it just requires a very good team and a concerted effort over time. I don't think you solve it once, and I certainly wouldn't pretend to have a plan that solves it and is done with it. We try to leverage best practices where we can in the fundamental architecture of the system to make it less exposed, in particular key parts of the system, less exposed to nefarious actions of others. But at the end of the day, it's just a constant, just a constant development effort. Thank you for being here. So I had a question about what opportunities self-driving cars open up. Since driving has kind of been designed around like a human being at the center since the beginning, if you put a computer at the center, what society-wide differences, and maybe even within individual car differences that open up, like could cars go 150 miles an hour on the highway and get places much faster? Would cars look differently when a human doesn't need to be paying attention and stuff like that? Yeah, I think the answer is yes. And that's something that's very exciting. So one of the, I think one of the unique opportunities that automakers in particular have when self-driving technology gets incorporated into their vehicles is they can do things like play, like differentiate the user experience. They can provide services, augmented reality services or location services, many other sort of, it opens a new window into an entirely new market that automakers haven't historically played in. And it allows them to change the very vehicles themselves. As you mentioned, the interior can change as we validate some of these self-driving systems and confirm that they do in fact reduce the collision, the rate of collisions as we hope they will. You can start to pull out a lot of the extra mass and other things that we've added to vehicles to make them more passively safe, right? Roll cages, crumple zones, airbags, a lot of these things, presumably in a world where we don't crash, there is much less need for passive safety systems. So yes. Hi, I have a question about the go or no-go test that you conduct for certain features, like you mentioned the throttle control where you slow down the throttle, assuming that the driver has pressed the wrong pedal. When you test, when do you decide to launch that feature? How do you know it's definitely going to work in all scenarios because your data set might not be tested? It's a statistical evaluation in every case, right? You're right. This is part of the art of self-driving vehicle development is you will never have comprehensively captured every case, every scenario. That is, some of you may want to correct me on this. I think that's an unbounded set. It may in fact be bounded at some point, but I think it's un. And so you'll never actually have characterized everything. What you will have done, hopefully, if you do it right, is you will have established with a reasonable degree of confidence that you can perform at a level of safety that's better than the average human driver. And once you've reached that threshold and you're confident that you've reached that threshold, I think the opportunity to launch is real and you should seriously consider it. So thank you for your talk today first. And my question is, self-driving seems to be able to ultimately take over the world to some extent. But just like other technologies today that open up new opportunities, but also bring in adverse effects. So how do you respond to fear and negative effects that may come in one day? And specifically, what do you see as the positive and negative implications of future day self-driving? Positive and negative implications. So the positive ones I kind of listed and go find your favorite press article and they'll list them as well. The negative ones in the near term, I do worry a little bit about the displacement of jobs. Not a little bit, this will happen. It happens with every technology like this. I think it's incumbent on us to find a good way of transitioning those who are employed in some of the transportation sectors that will be affected into better work. There are a few opportunities that are interesting in that regard, but I think it's an important thing to start discussing now, because it's gonna take a few years. And by the time we've got these self-driving systems on the roads really starting to place that labor, I'd really like to have a new home for it. Hi, I'm Kasia from the Sloan School. My question was more about your business model, again, with partnering with both VW and Hyundai, and your just perspective on how you were able to effectively do that. Did not one of them wanna go sort of exclusive with you? And what was your sort of thought process about that? Yeah, so our mission, as I mentioned, is to get the technology to market broadly, and quickly and safely. We are, have been and remain convinced that the right way to do that is by providing it to as much of the industry as possible. To every automaker who shares our vision and our approach, we were pleased to see that both Volkswagen Group, and I'm assuming you all know the scope of Volkswagen, right? This is a massive automaker. Hyundai Motor, also very large, across Hyundai, Kia, and Genesis. They both shared our vision of how we should do this, which was important to us. They both shared a keen interest in making a difference at scale through their platforms. Volkswagen has, I think, a very admirable set of initiatives around vehicle electrification, a few other things. Hyundai is doing similar things. And so, for us, it was important that we enable everyone, and that was kind of what Aurora started to do. Hi, I had a question. Now, that I see a lot of companies are coming up with self-driving cars, right? So, most of the cars are pretty much, all the technology is bound only to the car. So, would we see something like an open network where car communicate with each other, regardless of which company they come from? And would this, in any way, increase the safety or the performance of vehicles and stuff like that? Yeah, I think you're getting it vehicle-to-vehicle, vehicle-to-infrastructure type communication. There are efforts ongoing in that, and it's certainly, it's only positive, right? Having that information available to you can only make things better. The challenge has historically been with vehicle-to-vehicle, and in particular, vehicle-to-infrastructure, or vice versa, that it doesn't scale well, one. And two, it's been slow. It's been much slower in coming than our development. And so, when we develop these systems, we develop them without the expectation that those communication protocol are available to us. We'll certainly protect for them, and it will certainly be a benefit once they're here. But until then, many of the hard problems that I would have welcomed 10 years ago, to have a beacon on every traffic light that just told me it's state, rather than having to perceive it, I would have certainly used those 10 years ago. Now, they're less significant, because we've kind of worked our way through a lot of the problems they would have solved. Thank you for your talk. My question is, what's your opinion about the cooperation of self-driving vehicles? So maybe, I think, if you can control a group of self-driving vehicles at the same time, you can achieve a lot of benefits to the traffic. Yes, that is where a lot of the benefits come from in infrastructure utilization, right, is in ride-sharing with autonomous vehicles. And specifically, the better we understand demand patterns, people movement, goods movement, the better we can sort of optimally allocate these vehicles at locations where they're needed. So yes, certainly that coordination, this is where, as I mentioned, these three vectors of vehicle electrification, ride-sharing and autonomy, or mobility as a service and autonomy, really come together with a unique value proposition. Okay, thank you. Yeah. Thank you so much for a great talk and being here. Thank you. Thank you. Thank you.