The most controversial Python feature | Walrus operator

This is the walrus operator, and this is the assignment expression that it enables. It's been added as a feature to Python 3.8 after a whole lot of drama. Majority of Python core developers, as far as I can tell, were against it, and the drama and toxicity over it drove Guido van Rossum, creator of Python, to step down from his leadership role of benevolent dictator for life after he accepted the walrus operator as part of the PEP 572 proposal. So what is it? First, let's talk about the assignment statement, the equal sign in Python. We can thank Fortran for this, I think, not very good design decision of using the equal sign for assignment. So signing, in this case, 42 to the variable x. Most programming languages use it, with a few exceptions. Pascal uses the walrus operator for assignment, and R uses a different operator. This goes against the notation of mathematics, where the equal sign is used to designate equality. But there's a lesson here, I think, that even bad design decisions, once accepted, once we get used to them, aren't so bad. It's all about consistency. So the assignment expression does a little bit more. It also signs 42 to x, performs the assignment operation, but as an expression, it returns the value that it assigned as well. This can be useful in a lot of contexts. Let me talk about the cases where I find them particularly useful. So I use regular expressions a lot as part of data science to clean up and organize data. So a common piece of code you might see is you perform a regular expression match, returning it to a variable like res here, that contains the regular expression object if a match was found, and contains none if no match was found. And then you have some kind of conditional, like an if statement, that checks whether a match was found. And if it has been found, then you do something with the regular expression object. In Python 3.8, all that gets combined into a single line that performs the match operation, assigns the result of the match to res, and then check if res contains a regular expression object. A similar use case is in reading files. There's a lot of ways to read a file, but a lower level version, there's usually a while loop, then there's a read operation that's stored in the chunk variable, or something like that. And then there's a conditional that checks whether the chunk contains anything, or if an end of file was reached, at which point you break out of the while loop. If it has not been reached, then you do something with the data that was fetched. In Python 3.8, this gets compressed into a single line that performs the read operation, the assignment of the fetched data to the variable chunk, and then the conditional check in the while loop, whether chunk returned any data, or if the end of the file was reached. To me, this is elegant. Other use cases in list comprehensions, you can share sub-expressions. Like in the filter, you can assign f of x to y, and then reuse that y in the output of the comprehension. You can also, in a list, reuse a value, for example, if it's expensive to compute. So you can compute f of x once, assign it to y, and then reuse y in the rest of the list definition. As a side note, I don't like to use chained assignments, but chain assignments have the same feature that the function f is only computed once. So this particular statement here is equivalent to computing f once, assigning it to a temp variable, and then assigning that temp variable to both x and y. There are quite a few criticisms that are both objective and subjective that talk about the complexities of programming language design and human nature in general. So first is the idea that both the equal sign and the walrus operator perform in assignments, so it might be confusing to beginners. I'm not sure about the history of PEP 572, but my guess is that this criticism was more prevalent before the exception was added, that assignment expressions can't be on a line, stand-alone, by themselves, without using parentheses. I think this clarifies to beginners that assignment expressions should not be used as an assignment statement. There's a set of principles defined in PEP 20, otherwise called Design of Python, that has some ideas to aspire to in the design of Python. Just like Guido said, some of these are subjective, and I think they are ideas to aspire to, as opposed to perfectly implement, because they're overlapping and there's a natural tension between them. So the first principle that's been brought up is there should be only one obvious way to do it, and some people argue that the walrus operator performs an operation that already had another way to do the same exact thing. I've also heard a funny kind of criticism that the equals operator worked like an assignment expression in C, and it kind of sucked in C, or at least was error-prone. So there you go, you have a case study in the real world where this kind of operator being used to designate an assignment expression was error-prone. Another Design of Python principle that people brought up is that simple is better than complex, and while it seems that the assignment expression makes the code simpler, in fact, it was argued that it was only reducing white space and was in fact adding complexity, or at least moving the complexity. Finally, the criticism that applies often in programming language design is that not enough testing was done on how actual developers will use it. I think that's probably the biggest challenge of programming language design, and design in general, is you don't know how people, how thousands or millions of developers are going to use this feature once it's in the wild, and how other features will interact with it when it's in the wild. So this criticism is certainly true, but I think it is always true. My own feelings about the walrus operators, I think when used properly, it's quite elegant and even beautiful. But it also represents more than that. To me, it represents the importance of leadership in a community of smart people that disagree. So this is the feature that led Guido to resign, and I spoke with him on the podcast that I host, and I'll speak to him again. I think he's one of the most brilliant language designers and programmers we have, also a great leader in the software engineering community. So this is a little excerpt from a post he made. It reads, Now that PEP 572 is done, I don't ever want to have to fight so hard for a PEP and find that so many people despise my decisions. I would like to remove myself entirely from the decision process. I'm basically giving myself a permanent vacation from being BDFL, and you all will be on your own. So what are you all going to do? Create a democracy? Anarchy? A dictatorship? A federation? So to me, the walrus operator represents more than just assignment expressions. It represents the power of leadership to break through a toxic stalemate. I think leaders have to make difficult decisions, sometimes unpopular decisions, and sometimes ones, if you look at the long arc of history, prove out to be bad decisions. But without leadership, I think we can't make progress. So the messiness, the chaos of democracy is that the divisiveness can be paralyzing. And we need leaders to inspire us, to guide us, and to make difficult, risky decisions. So to me, the walrus operator would be useful for regular expressions, but it will also be a reminder of the importance of leadership in the programming world and in our world in general. If you enjoy these short little videos, subscribe, and remember, try to learn something new every day.


The Ego is the Source of Fear - Steven Pressfield | AI Podcast Clips

In your book, The War of Art, you talk about resistance, with a capital R, as the invisible force in this universe of ours that finds a way to prevent you from starting or doing the work. Where do you think resistance comes from? Why is there a force in our mind that's constantly trying to jeopardize our efforts, with laziness, excuses, and so on? That's another great question. I mean, in Jewish mysticism, in Kabbalistic thinking, it's called the Yetzir Ha-Rah, right? And it's a force that if this up here is your soul, or Neshamah, trying to talk to you, us down here, the Yetzir Ha-Rah is this negative force in the middle. So I'm not the only one that ever thought about this. But, and I don't know if anybody really knows the answer, but here's my answer. I think that there are two places where we, as human beings, can seat our identity. One is the ego, the conscious ego, and the other is the greater self. And the self, in the Jungian sense, the self in the Jungian sense includes the unconscious, and butts up against what Jung called the divine ground, which what I would call the muse, the goddess, or whatever. And I think, and the ego is just this little dot inside this bigger self. And the ego has a completely different view of life as from the self. The ego believes, I'm gonna give you a long answer here. No, perfect. The ego believes that death is real. The ego believes that time and space are real. The ego believes that each one of us is separate from the other. I'm separate from you. I could punch you in the face and it wouldn't hurt me. It would only hurt you. And in the ego's world, the dominant emotion is fear, because we are all made of flesh. We can all die. We can all be hurt. We can all be ruined, bump, bada-bump. So we are protecting ourselves, and even our desire to create, as we were talking about before, comes out of that fear of death. The self, on the other hand, the greater self, that butts up against the divine ground, believes that death is not real, that time and space are not real, that the gods travel swift as thought. And the ego also believes that, I mean, the self believes that there's no difference between you and me, that we're all one. If I hurt you, I hurt myself, karma, right? And in the world of the self, of the greater self, the dominant emotion is love, not fear. Now, so I think that, let me, I'll go farther back here, a long way to answer your question. When Jesus died on the cross, or when the 300 Spartans willingly sacrificed their lives at Thermopylae, they were acting according to the rules of the self. Death is not real, no difference between you and me, time and space are not real, predominant emotion is love. So, in my opinion, we as conscious human vessels have, are in a struggle between these two things, the ego and the self. To me, resistance is the voice of the ego, saying, and it's a fearful voice, because if, when we identify with the self, we move our consciousness over to the self, as artists or scientists, opening ourselves up to the cosmic dimension, to the other forces, the ego is tremendously threatened by that, because if we're in that space, that head space, we don't need the ego anymore. So I think resistance is a voice of the ego trying to keep control of us. In a way, I'll give you a bad example, Trump is the ego. It's probably a very good example, right? Yeah, he's, yeah. It's a zero-sum world for him, and for anybody that's in that, and the opposite of that would be somebody like Martin Luther King or Gandhi. Gandhi, yep. And that's, of course, why they all wind up getting assassinated, because that voice, that ego, is hanging on to itself and feels so threatened by, I could talk more about this if you want. No, for sure, that's fascinating. It's just, it's interesting why the fear is attached to the ego. I really like this dichotomy of ego and self and that struggle. It's just, ego has a, the self-obsession of it, why fear is such a predominant thing. Why is resistance trying to undermine everything? It's fear, it's out of fear. Let's think about the whole thing in terms of stories. In a story, the villain is always resistance, is always the ego. The hero is always, of course, always is not everything, but you know what I mean, pretty much, represents kind of the self. If you think about the alien on the spaceship, that's like the ultimate kind of villain. It keeps changing form, right? First it goes on the guy's face, then it pops out of his chest, but it always just has that one monomaniacal thing to destroy, you know? And just like the ego, just like resistance. And maybe alien is a bad example because Sigourney Weaver has to sort of fight on the same terms as the alien, but maybe a better example might be something like Casablanca, where in the end, the Humphrey Bogart character has to, acting, operating out of the self, has to give up his selfish dream of going off with Ingrid Bergman, Neil Salon, the love of his life, and instead, you know, puts her on the plane to Lisbon while he goes off to fight the Nazis in the desert. I don't know if that's clear, but in almost every story, the villain is the ego, is resistance, is fear, is that zero-sum thing. And in almost every story, the hero is someone that is willing to make a sacrifice to help others. It's letting go of that fear is what leads to productivity and to success. Yeah. So, do you think there's a, this is probably the answer is either obvious or impossible, but do you think there's an evolutionary advantage to resistance? Like, what would life look like without resistance? That's another great question. I think, I also believe that resistance, like death, gives a meaning to life. If we didn't have it, it's gonna be, you know, what would we be? We'd be in the Garden of Eden picking fruit and just happy and stupid, you know? And I do think that that myth of the Garden of Eden is really about this kind of thing, you know, where Adam and Eve decide to sort of take matters into their own hands and acquire knowledge that until then, God had said, I'm the only one that's got that knowledge. And of course, once they've acquired that knowledge, they're cast out into the world you and I live in now, where they do have to deal with that fear and they do have to deal with all that stuff. It's the human condition. It's the human condition and the meaning and the purpose comes from the resistance being there and the struggle to overcome it. To overcome it, right. And also, the other aspect of it is that it's not real at all. It's not even like it's an actual force. It's all here, right? So the sort of, in a way, it's sort of a surrender to it, you know? You know, or it's just. It's sort of like turning on the light in a dark thing. It's like, it's gone. But not quite because it's never really. Not quite, because it comes back again tomorrow morning. Exactly. So you have to keep changing light bulbs every day. So what's been, maybe recently, but in general, maybe in your life, what's been the most relentless or one of the more relentless sources of resistance to you personally? I mean, it's always the same. It's about writing for me. And evolving within my own body of work, you know? It never goes away. It never gets any less. Do you have particular excuses, particular justifications that come out? No, it's always the same. Well, I would say it's always the same, but it's really not because resistance is so protean. You know, it keeps changing form. And as you move to hopefully a higher level, resistance gets a little more nuanced and a little more subtle, trying to fake you out. But I think you learn that it's always there and you're always gonna have to face it. I mean, your battle is sitting down and writing to some number of words to a blank page. Do you have a process there with this battle? Do you have a number of hours that you put in? Do you sit down? Yeah, I'm definitely a believer that even though this battle is fought on the highest sort of spiritual level, that the way you fight it is on the most mundane, I'm sure it's like martial arts must be the same way. I mean, I go to the gym first thing in the morning and I sort of am rehearsing myself. Face, you know, the gym is called resistance training, right? You're working against resistance, right? And I don't wanna go, I don't wanna get out of bed. I hate that, you know. But I'm sort of fortifying myself to be ready for the day. And like I said, over Knockwood, over years, I've learned to sort of get into the right kind of mindset and it's not as hard for me as it used to be. The real resistance, I think, for me, and I think this is true for anybody, is the question of sort of what's the next idea? What's the next book? What's the next project that you're gonna work on? And when I ask that question, I'm asking it of the muse. I'm kind of saying, what do you want me, or I'm asking it of my unconscious. If we're looking at Bruce Springsteen's albums, it's kind of, well, what's the next album? Now he's on Broadway. That was a great idea, right? Where'd that come from, you know? But, and then for him, what's after that? Because that body of work is already alive. It already exists inside us, kind of like a woman's biological clock, and we have to serve it. And we have to, otherwise it'll give us cancer. I don't mean to say that if anybody has cancer that they're not, you know. But you know what I mean. It'll take its revenge on us. So the next resistance to me is sort of, or a big aspect of it is, what's next? When I finish the book I'm working on now, I'm not sure what I'm gonna do next. And let's see, at the same time, you have a kind of, you have a sense that there's a Bruce Springsteen single line of albums. So it's already known somewhere in the universe what you're going to do next, is the sense you have. In a sense, yes. I don't know if it's like predetermined, you know? But it's, but there's something like that. Yeah, I'd like to believe that there's, well, it's kind of like quantum mechanics, I guess. Once you observe it, maybe once you talk to the muse, it's one thing for sure. It was always going to be that one thing. But really, in reality, it's a distribution. It could be any number of things. Yeah, I think so. There's alternate realities. Alternate realities, yeah. But they're not that far apart. I mean, Bruce Springsteen is not gonna write a Joni Mitchell song, you know? No matter how hard he tries. He still went on Broadway, I mean, he still did that, which is not a Bruce Springsteen thing to do. So I think you're being, in retrospect, it all makes sense. I think it is a Bruce Springsteen thing to do. It's a next sort of evolution for him. Why not take his music to there, you know? In retrospect, it all makes perfect sense, I think. Yeah. If you pull it off, especially.


Ray Dalio: Meaning of Life | AI Podcast Clips

When you meditate, we're all human, we're all mortal. When you meditate on your own mortality, having achieved a lot of success on whatever dimension, what do you think is the meaning of it all? The meaning of our short existence on earth as human beings? I think that evolution is the greatest force of the universe and that we're all tiny bits of an evolutionary type of process where it's just matter and machines that go through time and that we all have a deeply embedded inclination to evolve and contribute to evolution. So I think it's to personally evolve and contribute to evolution. I could have predicted you would answer that way. It's brilliant and exactly right.


George Hotz: Elon is the iOS | AI Podcast Clips

Elon's the iOS. By the way, he paved the way for all of us. I would not be doing comma AI today if it was not for those conversations with Elon and if it were not for him saying like, I think he said like, well, obviously we're not going to use LIDAR. We use cameras. Humans use cameras. So what do you think about that? How important is LIDAR? Everybody else on L5 is using LIDAR. What are your thoughts on his provocative statement that LIDAR is a crutch? See, sometimes he'll say dumb things like the driver monitoring thing, but sometimes he'll say absolutely, completely, 100% obviously true things. Of course LIDAR is a crutch. It's not even a good crutch. You're not even using it. Oh, they're using it for localization, which isn't good in the first place. If you have to localize your car to centimeters in order to drive, that's not driving. I'm currently not doing much machine learning on top of LIDAR data, meaning like to help you in the task of general task of perception. The main goal of those LIDARs on those cars, I think is actually localization more than perception, or at least that's what they use them for. Yeah, that's true. If you want to localize to centimeters, you can't use GPS. The fanciest GPS in the world can't do it, especially if you're under tree cover and stuff. With LIDAR, you can do this pretty easily. So you really, they're not taking on, I mean, in some research they're using it for perception, and they're certainly not, which is sad, they're not fusing it well with vision. They do use it for perception. I'm not saying they don't use it for perception, but the thing that, they have vision-based and radar-based perception systems as well. You could remove the LIDAR and keep around a lot of the dynamic object perception. You want to get centimeter-accurate localization, good luck doing that with anything else.


Vsauce: Consciousness | AI Podcast Clip with Michael Stevens

There's ideas of panpsychism where people believe that whatever consciousness is is a fundamental part of reality. It's almost like a physics law. Do you think, what's your views on consciousness? Do you think it has this deep part of reality or is it something that's deeply human and constructed by us humans? Start nice and light and easy. Nothing I ask you today has actually proven answer so we're just hypothesizing. So yeah, I mean I should clarify, this is all speculation and I'm not an expert in any of these topics and I'm not God. But I think that consciousness is probably something that can be fully explained within the laws of physics. I think that our bodies and brains and the universe and at the quantum level is so rich and complex, I'd be surprised if we couldn't find a room for consciousness there. And why should we be conscious? Why are we aware of ourselves? That is a very strange and interesting and important question. And I think for the next few thousand years, we're going to have to believe in answers purely on faith. But my guess is that we will find that within the configuration space of possible arrangements of the universe, there are some that contain memories of others. Literally Julian Barber calls them time capsule states where you're like, yeah, not only do I have a scratch on my arm but also this state of the universe also contains a memory in my head of being scratched by my cat three days ago. And for some reason, those kinds of states of the universe are more plentiful or more likely. When you say those states, the ones that contain memories of its past or ones that contain memories of its past and have degrees of consciousness? Just the first part because I think the consciousness then emerges from the fact that a state of the universe that contains fragments or memories of other states is one where you're going to feel like there's time. You're going to feel like, yeah, things happened in the past. And I don't know what will happen in the future because these states don't contain information about the future. For some reason, those kinds of states are either more common, more plentiful, or you could use the anthropic principle and just say, well, they're extremely rare, but until you are in one or if you are in one, then you can ask questions like you're asking me on this podcast. So I questions. But yeah, it's like, why are we conscious? Well, because if we weren't, we wouldn't be asking why we were.


Gilbert Strang: MIT OpenCourseWare (OCW)

The fact that there is thousands, hundreds of thousands, millions of people that watch that video, I think that's really powerful. So how do you think the idea of putting lectures online, what really MIT OpenCourseWare has innovated? That was a wonderful idea. I think the story that I've heard is the committee was appointed by the president, President Vest at that time, a wonderful guy. And the idea of the committee was to figure out how MIT could be like other universities, market the work we were doing. And then they didn't see a way, and after a weekend, and they had an inspiration, came back to the President Vest and said, what if we just gave it away? And he decided that was okay, good idea. So. You know, that's a crazy idea. That's, if we think of a university as a thing that creates a product, isn't knowledge, the kind of educational knowledge, isn't the product and giving that away, are you surprised that it went through? The result that he did it, well, knowing a little bit President Vest, it was like him, I think. And it was really the right idea, you know. MIT is a kind of, it's known for being high level, technical things, and this is the best way we can say, tell, we can show what MIT really is like. Because in my case, those 1806 videos are just teaching the class. They were there in 26, 100. They're kind of fun to look at. People write to me and say, oh, you've got a sense of humor, but I don't know where that comes through. Somehow I've been friendly with the class, I like students, and linear algebra, we gotta give the subject most of the credit. It really has come forward in importance in these years.


Jeremy Howard: Deep Learning Frameworks - TensorFlow, PyTorch, fast.ai | AI Podcast Clips

From the perspective of deep learning frameworks, you work with fast AI, particularly this framework, and PyTorch and TensorFlow. What are the strengths of each platform, from your perspective? So in terms of what we've done our research on and taught in our course, we started with Theano and Keras, and then we switched to TensorFlow and Keras, and then we switched to PyTorch, and then we switched to PyTorch and fast AI. And that kind of reflects a growth and development of the ecosystem of deep learning libraries. Theano and TensorFlow were great, but were much harder to teach and do research and development on because they define what's called a computational graph up front, a static graph, where you basically have to say, here are all the things that I'm going to eventually do in my model. And then later on you say, okay, do those things with this data. And you can't debug them, you can't do them step by step, you can't program them interactively in a Jupyter notebook and so forth. PyTorch was not the first, but PyTorch was certainly the strongest entrant to come along and say, let's not do it that way, let's just use normal Python. And everything you know about in Python is just going to work, and we'll figure out how to make that run on the GPU as and when necessary. That turned out to be a huge leap in terms of what we could do with our research and what we could do with our teaching. Because it wasn't limiting. Yeah, I mean, it was critical for us for something like DawnBench to be able to rapidly try things. It's just so much harder to be a researcher and practitioner when you have to do everything up front and you can't inspect it. Problem with PyTorch is it's not at all accessible to newcomers because you have to write your own training loop and manage the gradients and all this stuff. And it's also not great for researchers because you're spending your time dealing with all this boilerplate and overhead rather than thinking about your algorithm. So we ended up writing this very multi-layered API that at the top level you can train a state-of-the-art neural network and three lines of code, which talks to an API, which talks to an API, which talks to an API. You can dive into it at any level and get progressively closer to the machine levels of control. And this is the FastAI library. That's been critical for us and for our students and for lots of people that have won big learning competitions with it and written academic papers with it. It's made a big difference. We're still limited though by Python and particularly this problem with things like recurrent neural nets, say, where you just can't change things unless you accept it going so slowly that it's impractical. So in the latest incarnation of the course and with some of the research we're now starting to do, we're starting to do stuff, some stuff in Swift. I think we're three years away from that being super practical, but I'm in no hurry. I'm very happy to invest the time to get there. But, you know, with that, we actually already have a nascent version of the FastAI library for vision running on Swift for TensorFlow. Because Python for TensorFlow is not going to cut it. It's just a disaster. What they did was they tried to replicate the bits that people were saying they like about PyTorch, this kind of interactive computation, but they didn't actually change their foundational runtime components. So they kind of added this like syntax sugar they call TFEager, TensorFlow Eager, which makes it look a lot like PyTorch, but it's 10 times slower than PyTorch to actually do a step. So because they didn't invest the time in like retooling the foundations, because their code base is so horribly complex. Yeah, I think it's probably very difficult to do that kind of retooling. Yeah, well, particularly the way TensorFlow was written. It was written by a lot of people very quickly in a very disorganized way. So like when you actually look in the code, as I do often, I'm always just like, oh, God, what were they thinking? It's just, it's pretty awful. So I'm really extremely negative about the potential future for Python for TensorFlow. But Swift for TensorFlow can be a different beast altogether. It can basically be a layer on top of MLIR that takes advantage of all the great compiler stuff that Swift builds on with LLVM. And yeah, I think it will be absolutely fantastic. Well, you're inspiring me to try. I haven't truly felt the pain of TensorFlow 2.0 Python. It's fine by me. But yeah, I mean, it does the job if you're using like predefined things that somebody's already written. But if you actually compare, you know, like I've had to do, because I've been having to do a lot of stuff with TensorFlow recently, you actually compare like, okay, I want to write something from scratch. And you're like, I just keep finding it's like, oh, it's running 10 times slower than PyTorch. So is the biggest cost, let's throw running time out the window, how long it takes you to program? That's not too different now. Thanks to TensorFlow Eager, that's not too different. But because so many things take so long to run, you wouldn't run it at 10 times slower. Like you just go like, oh, this is taking too long. And also, there's a lot of things which are just less programmable, like tf.data, which is the way data processing works in TensorFlow is just this big mess. It's incredibly inefficient. And they kind of had to write it that way because of the TPU problems I described earlier. So I just, you know, I just feel like they've got this huge technical debt, which they're not going to solve without starting from scratch. So here's an interesting question. And if there's a new student starting today, what would you recommend they use? Well, I mean, we obviously recommend Fast.ai and PyTorch because we teach new students. And that's what we teach with. So we would very strongly recommend that because it will let you get on top of the concepts much more quickly. So then you'll become an actual, and you'll also learn the actual state of the art techniques, you know, so you actually get world class results. Honestly, it doesn't much matter what library you learn, because switching from China to MXNet to TensorFlow to PyTorch is going to be a couple of days work as long as you understand the foundation as well. But you think, will Swift creep in there as a thing that people start using? Not for a few years, particularly because like Swift has no data science community, libraries, tooling. And the Swift community has a total lack of appreciation and understanding of numeric computing. So like they keep on making stupid decisions, you know, for years they've just done dumb things around performance and prioritization. That's clearly changing now because the developer of Swift, Chris Latner, is working at Google on Swift for TensorFlow. So like that's a priority. It'll be interesting to see what happens with Apple because like Apple hasn't shown any sign of caring about numeric programming in Swift. So I mean, hopefully they'll get off their ass and start appreciating this because currently all of their low-level libraries are not written in Swift. They're not particularly Swifty at all. Stuff like Core ML, they're really pretty rubbish. So yeah, so there's a long way to go. But at least one nice thing is that Swift for TensorFlow can actually directly use Python code and Python libraries. Literally the entire Lesson 1 notebook of Fast.ai runs in Swift right now in Python mode. So that's a nice intermediate thing.