Episode 335: The Wright Stuff

[00:00:00] Very Bad Wizards is a podcast with a philosopher, my dad, and psychologist David Pizarro, having an informal discussion about issues in science and ethics. Please note that the discussion contains bad words that I'm not allowed to say, and knowing my dad, some very inappropriate jokes. Fate, or some mysterious force, can put the finger on you or me for no good reason at all.

[00:00:30] Welcome to Very Bad Wizards, I'm Tamler Sommers from the University of Houston.

[00:01:19] Dave, the World Cup is wrapping up the group stage, Argentina is looking awesome, Messi has been definitively established as the GOAT. It just seems like everything is coming up Pizarro these days. Are you worried that the other shoe is about to drop? Yeah, knock on wood, dude! How dare you! How dare you! How dare you! Express any confidence in Argentina. I like I'm so neurotic about it.

[00:01:46] I like separately had a conversation with Bella and with Nikki where I felt the need to express that I firmly believe that Argentina would be knocked out in the first couple of rounds and that France was going to win it all. Yeah. Nikki was like, I don't care. Shut up. You're doing this complicated like reverse jinx kind of magic. And she's like, whatever. She's so tired of hearing about Messi. I do feel for her. I do. Because like every day. Yeah, for me.

[00:02:15] And then he made a goal. He scored all of their goals. He scored all. That's not a good sign. Yeah. I mean, it's great for Messi, but it's a. He is like the Tom Brady in every way. Just great from the beginning and way after like in his career, anyone thought he could still be. Totally. You know, I was having this discussion with like, I don't know in sports the distance between the best and second best.

[00:02:41] Like, do you think Brady is an appropriate comparison in that sense? Yeah. Like Messi is way cooler and less weird, I think. And Tom Brady, like that's not saying a whole lot. They were always considered, you know, up there. But then they just kind of passed a point. I think for Brady, it was the coming back from 28 to three against Atlanta in the Super Bowl where everyone's like, okay, like he's the goat. Like, what are we going to do? Yeah. We don't like it, but it's hard to deny at this point. Yeah.

[00:03:09] You know, Nikki's a huge gymnastics fan and like used to do gymnastics. And I've learned that Simone Biles, I think the distance between her and the next best person. She's completely dominant and like there's nobody that's anywhere close to her. Yeah, you wouldn't even think anybody's going to get close to that. And that's not true for like Messi. It's not true for Brady. Like, you know, it's not like everyone agrees. Messi in a real, like there's like an assist record that I don't think anybody will get close to. Like, it seems like impossible.

[00:03:38] But like some people might still say Pele was the greatest soccer player of all time. That's true. By the way, Houston pride, H-Town girl, Simone Biles. Oh, oh, that's right. Yeah. I forgot that. Trained here too. But yeah, like I think everyone just kind of, yeah, she's totally the best. Yeah. Yeah. And then Messi has longevity, which makes him like above Pele and Maradona, not in terms of like maybe raw talent, but like just nobody's done it so consistently.

[00:04:07] Maradona did more drugs, I think. Did a lot of blow. Yeah. But that's the same thing with Brady. It was the longevity that put him over the top. Like it's one thing to be great, but then there's Joe Montana who's great, you know, in his prime. But then he goes on and still wins another Super Bowl when he's 41 in Tampa. It's like, okay. Yeah. That Tampa thing is what's for, at least for me as a casual, like, okay, this was Brady. It wasn't just everybody else.

[00:04:36] It certainly wasn't Bill Belichick. Well, yeah, we shouldn't get into that. I know our listeners want us to keep talking about sports. That's right. That's why they come here. But actually we have a couple of more important things to get to. First and foremost, Robert Wright. Bob Wright coming back on the podcast. He has a new book out.

[00:05:00] We are violating our cardinal rule of never having people on to talk about their new book that was just released. But, you know, I guess we've done that for Paul Bloom and now we're doing it for Bob Wright. Bob Wright. That will be coming up in the second segment, but first. In a relevant related segment, because Bob's new book is about AI and the awe that we should have. For AI. For AI. Yeah.

[00:05:25] You were less convinced to have awe for AI, I think, until I sent you this product. This put me over the top. No shade to Bob's book, but this put me over the top. So this is a new product release announcement. The title of this is Chinese AI company launches, quote unquote, 95% accurate pet translator.

[00:05:48] So this is, you put it, I guess, around the collar of your dog or cat and using the magic of machine learning. It will tell you exactly what your pet is, I guess, saying? Thinking, saying. Think. Like, I guess it interprets their sounds, but also their body language and behavior. Hey, that's what AI can do. I'm hungry.

[00:06:19] I need snack. Is it a special magic magic? If you wear a cat on the collar of the neck, you can understand the needs of the animal. 95% accurate, which is... Which is a hilarious statistic. That's what the ground truth... What's the ground truth of what your pet is saying? Exactly. This is what I always wonder about, like, empathy scales and stuff like that. Like, what are you measuring against to know, you know? Yeah.

[00:06:47] And, you know, I think you're right to ask that question and maybe be a little skeptical because it turns out they've released no data or evidence to back up the 95% accurate claim. But do you think you're... Let me ask you a question. Like, is this conceivable that you could actually determine a percentage of accuracy? This gets to, like, a lot of the measurement concerns, I guess. Yeah, that's true.

[00:07:14] That I have about a lot of things. But, like, how would you even go about trying to know, like, whether a given report is accurate or not? This is an interesting question. I think in this case with pets, like, let's take dogs because fuck cats. Like, if the dog said or it translated, I have to go to the bathroom, you would need to correlate those statements with, like, the times it went to the bathroom or, you know? But even, like, I'm hungry.

[00:07:44] Like, the dog's always going to be hungry. Yeah. And they usually, like, if you take your dog out, they will go to the bathroom. They will go. Yeah. Yeah. And that's just for, like, the most basic thing. For the most basic thing. You know, like, you can tell, you don't need an AI caller to tell you when your dog is, like, hungry or wants to go to the bathroom. So, like. Right. Maybe, like, since dogs, like, hearing and smell is obviously better than ours, maybe they can hear, like, when the UPS truck is coming. Yeah. And if the little device said, like, the truck is coming.

[00:08:14] Yeah. Before you even knew it. Huh? Like, there you go. Yeah. You can determine some sort of accuracy. Even that, though, like, I could look at Charlie, who's terrified of thunderstorms, and he could hear, like, approaching thunderstorms before I could. And, like. But you're right. It's got to be something like that, I guess. Yeah. The more specific, the better. Right. I'm about to bite that fucker over there. Yeah. And then it runs. Well, listen. It's roughly $118.

[00:08:42] 100,000 pre-orders have already been made. It's got a million dollars from early investors. It's incredible. Like, you can just slap AI on any product, however ludicrous, and you'll just get all this money. I think we should give it the benefit of the doubt and pre-order one for you. Yeah. Because, you know, what will Trixie be saying? You know, she's a somewhat inscrutable dog, so it would actually be quite helpful.

[00:09:10] And, well, I actually have a surprise that I did purchase one of these. You got one? Yeah. So did I. You got one, too? Holy shit. I paid roughly $118. Oh. Alibaba. And Alibaba sent it to me. And I have to say, like, as skeptical as I was, and I still have some methodological questions about it, but, man, like, this seems like right on the nose. All right. Well, what could she have said? I'm curious. All right.

[00:09:39] So I'll just give you one example to start anyway. So she's a very affectionate dog in her own weird way. Very affectionate. She can get a kind of languid look. Like, I call it her bedroom ears when she just kind of looks at you, puts her ears back, and, like, kind of tells you to follow her in the bedroom. And she just wants, like, a snuggle. Trixie, by the way, has very big ears. They're very signal-able.

[00:10:06] It's the way she communicates, and, like, it's almost like they have their own life. Like, it's very cute. So anyway, I'm coming out of the shower, and I start getting dressed. And she comes in, and she does this big stretch, and then she gets, like, the bedroom ears, and there's just a little noise. She's not a very loud dog. She doesn't make a lot of noise, but just a little noise. I always kind of wondered what that noise was. So then the caller beeps, fortunately, and it says,

[00:10:34] God damn, you look so fucking hot in your sneaky underwear. Wow, Trixie's thirsty. She really is. Yeah, so that's, like, I was pretty impressed. And you're married. I mean, I think she knows that. I feel like it's disrespectful. Yeah. Like, I don't have any examples up, but she'll say some quite nasty things about Jen in her capacity as, like, you know, competition.

[00:11:04] So I happened to get a pre-order. I was one of the initial investors. So I got a few months ago when it was still winter here. And my dog, Ozzy, as you might remember, Tamler, is a rescue from Texas because your wonderful state has a habit of, you know, killing. They have kill shelters. Yes. And if anything has, like, any pit bull in them or anything like that. Every dog here has some pit bull in them. Every dog in most places now. Yeah. Which is good. They're sweet.

[00:11:31] And they, in Texas, they just walk around with a needle, you know, just trying to euthanize anything that looks. It's because it's a very dog-loving place, but not a very responsible place. Right. Right. Yeah. So my dog, unfortunately, is, you know, a Texan and was transported to Ithaca where the winters are a little harsh. So sometimes, even though he wants to go out all the time, some winter days, he just gives me this look and he's like pulls on the collar and he won't go out.

[00:11:59] So when I got this device, I finally could tell what he was trying to tell me. And it was, you should have fucking left me to die in Texas. Jesus. That's dark. Damn, that's dark. That was like the first thing he said. It was like, shit. You should have left me to die. You should have left me to die. I think she just wants to come back to Texas, you know? He? Yeah. Oh, he. Yeah. Best, you know, best day in the country. It's the best place to die.

[00:12:29] Yeah. What else? What else does Trixie say? Well, I guess I just have one more. Yeah. So, like, I love Trixie. Also, Texas dog. But maybe because of that, her political views, I can only describe them as abhorrent. And I've known this before, the caller. But, like, she's very anti-immigration. She's very pro-cop. You know, like, tough on crime. Very anti-Medicare for all.

[00:12:58] That's because people like parasites. Like, who want that? But she's anti-Semitic, but at the same time, very pro-Israel. So, she's, like, a huge Zionist in spite of her anti-Semitism. I'm kind of a wash in her eyes. You know? It's weird. Anyway, so I was listening to some podcast where they were talking about the recent primaries

[00:13:19] in New York where, like, in one race, the Mondami-supported candidate, Brad Lander, he defeated this huge, like, pro-Israel, like, sitting congressman, Dan Goldman, just a total bought-and-sold APEC guy. And, like, in spite of all that money, got, like, 32% of the vote. He got killed by Brad Lander. Now, Brad Lander is also Jewish, but very critical of Israel. So, kind of the opposite of Trixie.

[00:13:50] He calls Gaza a genocide. He's still kind of a Zionist, but, like, he's super critical. He wants to cut off all funding to Israel. So, they're talking about his win, like, resounding win and what it means. And her collar just goes off. I wasn't even looking at her. And I hear her say, Coppo, fuck, we should take him out like we took out Charlie Kirk. I don't even know what she... Jesus Christ! I know. I don't even know what she means by that. Like, what's she suggesting with that? Who's the we? Exactly.

[00:14:22] She must have, like, she must be on some sick group chat. You know? I think so. She's on, like, Dog 8chan or whatever. Wow. She's going to keep defending her, I guess? You know, like, I love her. And she's a dog. And I don't know what she was doing before we adopted her. Well, but, you know, my dog, like me, is not political at all. He doesn't care. I would love that. He really has local concerns.

[00:14:50] And one of his biggest local concerns is we got him in 2020. So before my most recent batch of children. Yeah. So he's had, like, a hard time. I knew he was having a hard time. Like, you know, he has to put up with, like, fucking infants and toddlers now running around. So the only other thing that he says, and he said this a couple times now, like, when the kids are running around, sometimes he catches a whiff, like, when they've, like, pooped their pants. Yeah. And he just yells out, like, oh, so when they poop in the house, you don't kick them?

[00:15:17] Like, wow, you kick Ozzy? Well, I'm neither agreeing or disagreeing with the claims, specific claims that Ozzy is making. I'll leave it to you to wonder whether that's part of the 95% accurate or the 5%. Claude, what is the animal protection services number of Ithaca? Yeah. All right. Well, the technology giveth.

[00:15:45] One thing I do want to urge listeners to do, we haven't talked about it, but the video on this article that we'll link to on the thing is very funny. It's like in Mandarin, but the device itself is speaking in English, you know, so they know their market. And most of it is just cats being so pissed off that someone is trying to put this fucking collar on them.

[00:16:19] One more step and Pippi will tear you apart. Like, and I was like, I assume that his name is Pippi. So it's like, do we, does it talk, think of itself in the third person? Does it talk of itself? You know, like, would some dog say I will do it?

[00:16:45] Or maybe it's another animal, like, like an imaginary friend or something of the cat. That might've just been a judgment call on this part of these. The AI. Yeah. Yeah. Again, it's only 95% accurate. Exactly. All right. We'll be right back with Bob Wright.

[00:17:26] Mother call. She got mad at me. You dig. Went out. Shot up behind. A feeling against her. It felt so. As the sister sits in her. Silence. Her sister. Did you.

[00:18:05] And we all. Stay. Welcome back to Very Bad Wizards. This is the predictable time of the episode where we like to take a moment and thank everybody for all of their support. We really appreciate it. Like we always say, we wouldn't be able to do this without you, or at least we would not have the motivation to do it without you.

[00:18:29] We really enjoy the community that we have around this podcast and that community is you. And one of the things that we enjoy about the community is when we hear from you. If you want to reach us, you can always message us verybadwizards at gmail.com. You can find us on social media at verybadwizards or at peas or at tamler. If you want to help us in more tangible ways, but also get something in return, you can become

[00:18:57] part of our lively Patreon community. When you do that, you not only get the warm glow of altruism that helps us out, but you also get something in return. So various tiers at $2 per month, minimum tier. You get all ad-free episodes from our back catalog. You also get the compilation of beats that I've put together over the years. And I'm currently working on number 10.

[00:19:23] If you join at $5 per month and up, you get access to all of our bonus content. So obviously that's, I think the one that would be most beneficial to you if you like listening to us wax philosophical about things like David Lynch or Star Trek, or if you want to listen to our series of side podcasts, like the Reintegrators with Paul Bloom, our podcast about severance or our Deadwood podcast, The Ambulators.

[00:19:50] All of that, years of back catalog of special episodes and side quests that we do are available to you at the $5 a month and up tier. You also get to listen to our monthly Ask Us Anything episodes at $10 and up. You get all of that, but you get to vote on an episode topic. We're working on preparing for our Solaris episode or episodes, both the book and the original movie.

[00:20:18] You also get access to our Brothers Karamazov five-part series. You get some lectures from Plato's Symposium and my intro psych lectures. And finally at $20 and up, you get all of that, but you also get to be one of the Ask Us Anything askers. And you can submit a question. We answer all of them. You also get to watch us on video if that's your thing. You can also, by the way, go to our website and buy some merch.

[00:20:45] Cotton Bureau now has not just shirts. There's like mugs and glasses and water bottles, all kinds of stuff that you can get with the Very Bad Wizards logo. We appreciate that as well. And yeah, thank you again for everything. Oh, one last thing. So I didn't mention our podcast series that we're currently working on, on the Odyssey, the Lotus Eaters. We're wrapping that up.

[00:21:14] And as I suggested very early on, the last episode or episodes will be recorded with Tamler making the trip out to Ithaca. Tamler is Odysseus in this metaphor. And so he's going to return or visit the first time and we'll be here. We'll probably do an Ask Us Anything video there too. We'll be here at my place in Ithaca recording. And yeah, I'm really excited about it. Looking forward to it.

[00:21:44] And you get to hear all of that if you join our Patreon and read the Odyssey, actually. Probably should do that. So thank you. Let's get back to our episode. All right. I'd like to welcome back Bob Wright to our humble podcast. Bob or Robert Wright is a bestselling author of many books, including The Moral Animal, which for better or worse, I've talked about this, made me go into academia and philosophy

[00:22:13] or played a big role in that. Also Non-Zero, Why Buddhism is True. And as we'll talk about shortly, his new book, The God Test. He is also one of the most sensible international affairs analysts and journalists that I know and runs the Non-Zero newsletter and podcast.

[00:22:35] He's probably most famous for the podcast miniseries Overton Windows, which he co-hosted with Tamler Summers. It says here. It says here. Yeah. That's the one I get stopped on the street about. I'm not going to say, aren't you the Overton window partner of Tamler Summers? I'm like, you nailed it. Oh my God. You guys need to put another one of those out. We do. We do actually. We could do another Israel one at this point. Talk about an Overton window shift.

[00:23:05] It's expanded. It's expanded. So Bob, welcome back to VBW. Good to have you back. Thank you. Nice of you to have me. It's a rare honor. I know. We are violating a sacred rule by having you on the podcast right now. The sacred rule is to never talk to authors who are doing book tours. But every sacred rule can have a couple of exceptions and you are one of them. And Paul Bloom is probably the other one. I think that's it though.

[00:23:33] You know, friends of the podcast, influential personally, I think for both of us. So that's a great honor. And congratulations. I do feel honored. Yeah. And you know, since Tamler said it already, I think I said this when you were on first, but Non-Zero is probably one of my favorite books and was very influential and probably got ideas from that that I still don't even remember that they were from that. That is such a hipster favorite Bob Wright book. Is it? Really? Where do I go to meet these hipsters?

[00:24:03] I've been to Brooklyn. My daughters live in Brooklyn. I don't even think they're aware of that book. Well, that's what I mean. It's a hipster take. You know, less talked about one. Yeah. Although it was plenty talked about when it came out. Yeah. I mean, actually it wasn't at first, but about six months into his life, Bill Clinton, for some reason, read it and got a B in his bonnet about it. And God bless that man is all I have to say. He would read it on the Epstein plane. This was, yeah. I mean, I, oh, it's funny. That book is mentioned in the Epstein files.

[00:24:32] Non-Zero is in the Epstein files. Yeah. But listen, we got grass. Congratulations. This is not the book we're here to promote. I did hear that it was the cornerstone of Jeffrey Epstein's philosophy. You know. Look, as long as we're here, it's actually interesting. It's that guy, Joey, something you ran in the media lab, is emailing with Epstein. And he says, you know, I was talking to somebody about your predatory view of evolution. Apparently Epstein had taken away a kind of a social Darwinist take on evolution, which

[00:25:00] kind of makes sense since some people might say he was a bit of a predator. And this guy is saying, so you should read this book that had been recommended to him more than he had read it, I think. That emphasizes the, you know, that natural selection recognizes cooperative logic and harnesses those dynamics as well and blah, blah, blah. And then Epstein goes and reads about it on Amazon and emails him 10 minutes later and kind of like dismisses it or something. But that's my 15 minutes of fame right there.

[00:25:27] If only he had decided to get it, you know, even just on Kindle. There was just that new report that he really did kill himself. Do you buy that? The New York Times. Yeah. New York Times. I don't know. I mean, I've kind of stayed away from it, but I didn't read the piece. I listened to a podcast with a guy who wrote the piece, the New York Times reporter who wrote the piece saying that the bulk of the evidence suggests actual suicide. Could be. But those video cameras sure exhibited a funny pattern of dysfunction. They certainly did.

[00:25:58] All right. Let's get to your new book. Maybe just give a brief overview of what you're trying to accomplish in it. And then we can dive into the details. OK. People should not be distracted by the title and subtitle, which, by the way, is the God Test, artificial intelligence and our coming cosmic reckoning available at a website near you from the fact that actually, I mean, I do. It is cosmic in some ways. I mean, I put it in a pretty sweeping context in some ways.

[00:26:27] But the main purpose is to explain in a way that a non-technical audience can understand what I think is kind of the secret sauce of the deep learning revolution. First of all, that is why the capabilities of these machines have grown so fast. And I think once you understand the secret sauce, you'll expect them to keep growing fast. And I go on to explain why I think they're going to be very challenging things to control.

[00:26:55] And I mean, leaving aside control in the sci-fi sense, like they could, you know, the planetary takeover, which, you know, is a scenario I can't entirely dismiss. But I'm in the near term, I'm more worried about just the overall impact, which I think will be disruptive in the not only good sense and could be destabilizing nationally and internationally. And I think it's a technology that we have to confront as a cohesive global community. I make that case.

[00:27:20] And that, you know, there are things individual people can do to facilitate that. And happily, there's some overlap between those and what I think people should do just to adapt to the technology in ways that are good for the people themselves. There's a term I use called cognitive sovereignty. I didn't originate it. But the idea is that, you know, if you want to preserve your agency in the face of this increasingly

[00:27:46] pervasive technology, you should maybe be kind of purposeful about it. And that AI can, in principle, help, but it also can hurt if left to its own devices. So anyway, you know, and I get into the workings of the machines fairly deeply, I think, while, you know, trying to keep it fairly accessible. I just wanted to say, I thought you did a great job of bringing along the reader in the description of the technical details. So that's not easy to do. They're not easy to explain. No, there's a couple of basic things, I think, to grasp.

[00:28:16] Why the God test? Like, I didn't even have this planned. But what's, yeah, what's with the God test? You said disregard the title, but you can't fucking disregard the title. No, the main point of the title is to get podcast hosts to ask me what the fuck it means, and then it will get repeated many times in the course of the podcast. So- You fell for it. Damn it. The, it has several meanings. One is, and this gets at the Overton window. It is more respectable to say, this is going to be a super intelligence, right?

[00:28:44] Like, the previously reflexively dismissed kind of Eliezer Yudkowsky scenarios are now being seriously engaged by more people. But one of these themes is that it could be a super intelligence that conceivably has godlike power and runs the planet, like a global brain. And if so, you know, we want to design a good God. As people have said, you don't want it to have malicious intent. Now, I'm not, that's far from like the focus of the book because I don't think it's the

[00:29:13] top priority, but that's one of the meanings. The main meaning of the God test is that, you know, because I believe we need to confront this as a global community, you know, in a pretty deep sense. I mean, I could get into why that is, but the main point is that my vision of how we come out of this well is just not compatible with like continued pointless wars and gratuitous conflict internationally, or for that matter, the degree of polarization we have in America intranationally.

[00:29:42] And I think if we're going to get beyond that, it is going to take something that you could call moral progress on the part of the species. So it's the kind of test that a God might give. Like, you know, in the Bible, you see cases where they say salvation is possible, but you have to shape up whether that means worship Yahweh or be nicer to each other, whatever. I'm more on the be nicer to each other wavelength. But I do think that the challenge is that stiff.

[00:30:10] So that's the main meaning. I do, you know, fool around mainly in the appendix with the question of whether the trajectory of biological and subsequent technological evolution and kind of the evolution of ideas and so on suggests that there could be some purpose unfolding. But that would be a purpose compatible with viewing the process itself as material, mechanistic. I'm a straightforward Darwinian and so on. So anyway. OK.

[00:30:35] Can I ask you about what you call your first purpose, which is you say, I want to convince you that if you aren't somewhere on the awe spectrum, just being in awe of this technology, if you don't feel something of great magnitude and power approaching, you aren't getting the picture.

[00:31:01] Transformation of human experience and human society in the history of our species. Is that skepticism I detect in your voice? So then in your fourth goal, you say, I think the coming of artificial intelligence marks a major threshold, not just in the history of technology, not just in the history of our species, but in the history of our planet. So this is a huge claim. And I count myself as someone who isn't somewhere on the awe spectrum.

[00:31:30] And even though I acknowledge that the Overton Windows is considering this kind of scenario now, like it has widened to not think this is just completely cuckoo for Cocoa Puffs. But and, you know, we can talk about the ways your book both did and didn't make me kind of rethink this perspective. But what do you say to the kind of basic AI skeptic that says, yes, it's impressive, the things that can do, you know, it can code really well, it can summarize text, it can make

[00:32:00] you think like you're talking to a real person up to a certain point. But like this isn't transformational. This isn't the printing press. This isn't even the Internet. This is just another piece of technology. And the idea that this is some on a different order of magnitude, and it's the most important thing that ever happened to the planet Earth. Like, I know this is a tough task, but how do you tell someone that's really just initially skeptical of those claims? A couple of things.

[00:32:29] I mean, first of all, on awe, you know, I note that awe used to refer more to something that involved more in the way of terror. And it's evolved to refer to the apprehension of something great. But the unifying theme is great power. Either way, you're in terror of it or you're and so anywhere on that spectrum is where I think everyone in principle belongs. They agree with me that something big is happening. As for making that case, there's a couple of things.

[00:32:55] I mean, first of all, I hope when I explain what's going on in the training of these machines and that I think it's actually a lot of it's more like evolution than like learning. In other words, the machines are kind of reverse engineering parts of the human mind based on the data we give it, including parts of the human mind that I think were designed by natural selection, not just kind of parts of the human mind that are processes we learn. I hope that gets their attention once they really appreciate that.

[00:33:21] And I provide some examples of things that is reverse engineered without, you know, the AI researchers having to design, including a system for representing the meaning of words. But as for citing accomplishments of the AI that I think should help persuade you, you know, I think there are several categories of this. And one is, you know, there's this increasingly well-known body of data being assembled by this

[00:33:48] organization called METR, M-E-T-R, where they measure how long it would take a human to do a given task. Many of these are programming tasks, not all. And then they see whether a large language model can do the task. And then that allows them to go back and look at past large language models and say, well, this three-year-old model could do a task that it would take a human nine seconds to do, blah, blah, blah. And then they chart them on a graph. And what they showed more than a year ago was that the task length was doubling every

[00:34:18] seven months. Now, that's exponential, OK? So it may not seem like much at first, you know, 10 seconds, 20 seconds. But now it's getting to the point where they're having trouble measuring it, doing the experiments with humans that measure because they're up around 14, 16 hours and it's happening fast. Moreover, the doubling time is shrinking. It's no longer seven months. It's closer to like four months or something. So it's like super exponential. And that seems to be a real thing. That's an example of it. How constrained are the tasks? Well, that's why they're having trouble designing the things now.

[00:34:48] I mean, in other words, coming up with things they ask humans to do. I mean, since many of them are programming tasks, I don't really know because I'm not a programmer. But, you know, these are things a company would want to pay a machine to do if it's not pay a machine. They'd want to pay for the machine so they don't have to pay anybody to do it. Sure. I mean, look, I think people who engage with these things closely are just more and more impressed. If you've been monitoring this stuff, like there was a big jump from GPT 3.5 to 4 even. But this is now, that's now three years ago.

[00:35:18] And I am just quite impressed. And they're doing things in pretty arcane realms. Like, I don't know if you heard about this thing about a month ago, but there are these set of mathematical problems that were laid down by a guy named, I don't know, what's his name? Erdos or something? That's wrong. But anyway, this got a ton of attention in the mathematical community when ChatGPT solved one of the Erdos problems or whatever they're called. I mean, top flight mathematicians had pondered this for a long time and have been pondering it for many, many decades.

[00:35:49] I mean, look, there's a chart in my book at the beginning of the chapter on evolutionary arms races. I just note how on this particular test where the average grad student, if they take the test just in their subject area, would get like a 65% or something. And this was designed to be kind of Google proof. So it's hard to just find the answers online. And you can just look at the rate of progress over the last three years.

[00:36:14] When all this hubbub started, late 22, early 2023, they did not rival the average grad student. Now a given LLM far exceeds the average grad student in all of the subjects tested. A single machine would beat a team of the average grad student. I mean, there's a reason anthropics revenue has grown at a rate that I think is unprecedented in the entire history of capitalism. Then there's the whole like, quote, agentic revolution. And this is important to understand the importance of coding.

[00:36:43] Now, by the way, this was kind of a surprise that it turned out to be good at coding. It wasn't particularly planned, but they are. And once they realized that, they made it better and better. But the reason you can do vibe coding, the reason a layperson can say, well, give me a website that does this or an app that does that. And increasingly, it just delivers the product is because it not only does the code, but then it goes back and checks the results and looks at the product and says, well, this doesn't, you know, we'd hope to have this feature functioning. Let me change this.

[00:37:10] So it's a successful autonomous worker in the sense of encountering obstacles, figuring out ways around them. But there's one more dimension to the agentic thing. If you talk about AIs that can actually do the things that workers do, you know, take action, like not just answer questions, not just help you find information, but can do office tasks like, like, okay, you read incoming resumes, you triage them and you forward them

[00:37:35] accordingly to the people who need to see the resumes or the essays that accompany them or whatever. So in that case, the cognitive task of like assessing the resume or assessing the essay, whatever they're assessing, that's something just an LLM can do. And deciding like which people they should be sent to, that's something an LLM can do. However, an LLM cannot open a PDF file and read the resume. It cannot actually send an email.

[00:38:05] It can't actually go online to do research. All those things require code. And so an agent that can go out and do stuff for you, whether it's a research task or something else you wanted to do online, contact people, buy things, whatever, or do fairly complicated tasks of a kind of worker might do. What that consists of is a series of cognitive tasks that an LLM can do stitched together, linked together by computer code that takes the actions.

[00:38:34] And that's why once these things got good at coding, that has unlocked the agentic potential. And that is why Anthropics revenue in about, I don't know, six months went from 10 billion per year to 50 billion per year, which is like an unprecedented, you know, and I'm not saying, by the way, I'm not, I'm not saying there won't be a crash because we don't know actually where ultimately the value will accrue. It could be that Anthropics business model in the end doesn't work.

[00:39:01] But the profitability question seems very distant from the question of these kind of cosmic claims that you're making about its power and potential, right? Like, I don't know, it's here where maybe we could dive into some of the details, like the evolution analogy. I know Dave has some questions that he wants to ask. I mean, let me first say that I read the whole book. I know you were in disbelief that Paul read it. You could be in disbelief that I read it, but I did. And it is- You, I trust. You, I trust.

[00:39:31] But Paul Bloom, I'm sorry. He just slipped the new one. And as always, you're an incredible writer. That incredible writing though, like snuck in some things that I want to push you on. And the first is that, okay, so this argument about being in awe of something that is so much greater than oneself, like there's a trivial way in which I can say, yeah, of course these models are greater than anything that we've seen before. I am like an AI believer in the sense that I think this is a tool that is unlike anything we've ever seen.

[00:40:00] It is unlocking an incredible amount of possibilities. I just, like I would reserve the term awe for something that I felt was like existentially more interesting than an LLM. And I think that you're trying to, one of the ways in which you try to build this case that this is an incredible thing that we should be in awe over is this analogy that you use, that it has evolved like humans evolved.

[00:40:26] And again, if you use the word evolution in a certain way, then sure, like there are changes to these models that are a result of the inputs that it receives. And those changes over some period of time have led to an increase in the complexity of these models. But I feel like that analogy is a little strained. So you say, for instance, that it is akin to a process of natural selection and that some of these abilities have been unlocked on its own. We didn't expect it.

[00:40:56] But here's my first question. You describe the kind of learning that these models do, like what people refer to as back propagation. That is, you create this model, you create these vectors that map properties of an object or a word or whatever. And you can represent these really, really complex things with these models. And what you do is you build a basic one with like random numbers in these vector spaces.

[00:41:21] And then you check it against like whatever, a corpus, you know, like millions of Google books, whatever. And it corrects, it gets some correction. And so like corrects itself by back propagating. And then you say like, isn't this incredible? This is like evolution. But like, how is it not just learning? You say a few times, this is more than just learning. But why do you think that's more than just learning? Well, you know, I quote B.F. Skinner arguing that learning and evolution are at some generic level the same process.

[00:41:52] You know, he calls it selection through consequences. So, you know. I disagree with B.F. Skinner, too. Yeah. Well, I noticed you waited until he's dead to say that, David. Died 40 years ago. Why didn't you speak up before that? Burris. I don't trust Burris. People named Burris Frederick. That's a great middle name. Although I would, I too would use my initial, oh no, it's his first name, Burris Frederick. Anyway, so in a way, it isn't the process that makes me say it's in some ways like evolution. It's the product, okay?

[00:42:21] Okay. So, we now know that in the course of the training, the full training of a large language model, both the development of the so-called word embeddings, which are these long series of numbers separated by commas that represent, you could say with slight oversimplification, each word. And that between developing those and the other things that happen in training, you know, two different things happen. On the one hand, the model learns to speak a particular language.

[00:42:49] Well, that I would categorize as learning just because in humans, that happens during the development of a child's brain, okay? Learning a specific language. But it also develops a system for representing the meaning of words. And I think most people would agree that that is probably a product of natural selection. And that, you know, I want to pause and kind of marvel at that. Like, okay, so we told the machines, like, okay, you have to represent each word technically tokens, but we can think of it as discrete words.

[00:43:19] As this long series of numbers, you choose the numbers, okay? You pick the numbers. And the other thing that's happening in the course of training is that the strength of neuronal connections is also changing, okay? So you've got to have- No, let's not. I mean, I want to be wary, Bob, of you sneaking in brain language to conclude what you're concluding. Well, okay, but this is just the way any researcher would describe it. I mean, it's just- No, no, no, they're weights. They're numbers. They're weights. No, but they are strength. There's no cells.

[00:43:49] There's no, I mean, neurons. And they're called neurons just because that's what they decided to call them. No, no, no, wait. The strength of a connection between two neurons, I would think, governs the degree of sensitivity of one neuron to the input from another. And that is what weights do. It's exactly what weights do, okay? So if the weight is low for a connection between two neurons, that means that the receiving neuron is not going to be very heavily influenced by the neuron that sends the input.

[00:44:18] I think that's actually a pretty good analogy. It's an analogy. It is not a neuron. Of course it's not a biological neuron. Right. So why use the word neuron? Like, I fear that you're using the word neuron because you want us to believe that this is like evolution. Why use the word electron? I'm not a fucking physicist. It's the word they use. Yeah, it's not, that's not a Bob thing. Like, that's what they call them. I mean, they call neural networks. Sure. But they call those things neurons. They do. That is something that they do. You can't blame Bob for that.

[00:44:45] Now, I'm not arguing, and they're not, that it is functionally like a neuron in every person. I do think it's interesting that the evolution of the equipment for human language did involve the selective strengthening of neuronal connections that were conducive to the deaf use of language, and so does the learning of language, and that that happens here. That's interesting, but to me, the much more important fact is that all you need to do is

[00:45:11] feed in the human data and make the machine better at a task that doesn't seem in a certain sense to entail great cognitive sophistication, like predicting the next word, but the machine does, in fact, reverse engineer mechanisms of information processing that are functionally comparable to mechanisms that evolved in human evolution. That, to me, is a very significant fact, because for one thing, it speaks to the growing power of them.

[00:45:38] And by the way, I concede I'm emphasizing the evolution analogy to an extent that not all AI researchers would. I think some of them put way too much emphasis on the learning analogy, although, again, at some level, the processes are the same. But the key, you know, it's the selective retention of traits that are conducive to the performing of these tasks leads to the reverse engineering of human cognitive and perceptual functionality and can, in principle, I believe, reverse engineer the entire human mind. That's the claim.

[00:46:07] So, okay, like the reverse engineering bit, like it sounds like what you're saying is it's figured out how the brain does things. But that's sort of just affirming like the conclusion again, right? Like we don't really know if it's reverse engineered the way the brain works. Well, I don't mean to personify it any more than I would personify natural selection because it's the same thing. You can say natural selection, quote, designs organs to do things. You can say it, quote, engineers things. And I think that's a useful way to talk.

[00:46:34] But of course, it's actually a quite laborious process of random mutation and selective retention. But natural selection is, in a certain sense, engineering, in quotes. And so are these models. It's just the reason I say reverse engineering is because you're like showing them the data that results from the equipment natural selection engineered and say, figure out a way to do what humans do in terms of data generation.

[00:47:02] And in a way, I guess it's not shocking that it happens on some of the same solutions at a general level, not at a super specific level, but it happens on some of the same solutions natural selection happened on. So what if we granted you that analogy as far as it goes with all the relevant caveats? What's the upshot of that? Like what part of the argument to awe and the power of and the potential of these machines?

[00:47:29] What part does the evolutionary, if we granted to you, like what do you get from that? Well, first of all, I want to emphasize again, I don't mean awe in the sense of worship. No, no. Yes, I didn't mean to suggest that. I mean, I originally used the etymology of awe and the change in the meaning to characterize doomers on the one hand and accelerationists on the other. So what I'm asking is, like, forget the word awe, just in the argument against the AI skeptic,

[00:47:55] the person who thinks this is wildly exaggerated, its potential to transform our species and our planet. Like what part does the evolutionary analogy play in your argument against that person? Well, you know, when I went back and reread this piece I wrote in the Wilson Quarterly after talking to Jeffrey Hinton, a bunch of other people in 1983, I clearly was expecting what most AI researchers were expecting at the time, which is that if we're going to

[00:48:22] have machines that use language well, we are going to have to convey to them the connection between the words and the meaning. We're going to have to, in some metaphorical sense, feed a dictionary into them. And, you know, I quote from the proposal for the famous 1956 Dartmouth AI conference at which the term artificial intelligence was coined, where they say any aspect of, you know, human thought can be so precisely described that in principle we can implant it in the machine. That was the expectation.

[00:48:49] You figure out how the human mind works and then you build a machine that replicates that. Well, if that's the way it's really going to work, you're going to be working a long time before you get to the bottom of this because we still do not understand how the human mind works. The point I'm trying to emphasize is it turns out you don't have to do that. All you have to do is collect data that either goes into humans or comes out of them. You know, when Mark Zuckerberg with his, you know, flawless sense of grace, kindness and

[00:49:14] timing in the same week that he announced the layoff of 8,000 workers said, oh, and also we're going to start monitoring your keystrokes. The reason he's doing that is because if you can monitor the data that comes into these workers and the data that goes out, you can replicate the cognitive workings that they're performing for money and then you won't have to pay the money. And that, I'm sorry, but that's an amazing fucking fact. Well, it accelerates the timeline for AI's development by like a jillion years. I mean, you know. Yeah.

[00:49:42] I mean, you saw it like it was incredible the way that it happened where we thought that to translate, you know, like a Star Trek universal translator, you know, some thought was that you would need to decode what like the universal grammar was and you would need to sort of represent formally like each of the meanings of these words. And really it was just Google crunching all of those data that like spawned better translation than we ever thought we could design. Yeah.

[00:50:11] And that really is true. There is though, like, and maybe this is me because I, like, I feel like we're both respecting the incredible power of these models. I find that there is at its heart a banality to them in here. So let me try this out on you for what like an LLM like GPT latest to do in one second, the calculations that it does would take any given human being a million years.

[00:50:40] And that's because what these things are doing is, you know, you talk about the exponential growth rate of these models. It's because people like Nvidia are pumping out these graphics cards that are pouring this tremendous amount of computational power into these models in a way that it is laughably disanalogous to what the brain does. Like there is no way that our brain works in that way. Our brain has an efficiency that is a result of whatever happened in natural selection that does not seem to be present in these models. Well, I don't know enough about the brain.

[00:51:09] I mean, we actually have more neurons and more neuronal connections and shit happens pretty fast in there, I'm told. But I really don't know that much about, I mean, I guess what I'd say is if you look at the, quote, calculations being performed by neurons when they like sense the amount of, you know, the strength of an incoming signal respond appropriately, like if you broke that down to calculations, it would take a person a jillion years to do what their brain's doing in a nanosecond, probably. I don't know.

[00:51:36] But I mean, for practical purposes, it doesn't necessarily matter. And you're right that there's a lot of force going on, but I don't think it's merely brute anymore. Right? I mean, I think... Right. I mean, it is clever, right? But they are also things that we designed. I mean, we don't control the input and output. That's why people do refer to them as black boxes. But we did design, you know, this represent things as vectors and matrices and do these matrix calculations. And we are feeding them stuff that we've done. That's what they're learning from. Yeah.

[00:52:06] Well, yeah. I mean, that is certainly true. But at the same time, that underscores the acceleration. All of human thought, the fruits of how many centuries of intellectual collaboration among how many people is out there on the internet. And it just goes out and gathers it and turns it into a kind of intelligence. I mean... I think it was Louis C.K. who said that Facebook was just Mark Zuckerberg's way to trick us into data input. Well, kind of. Yeah.

[00:52:36] And his workers are now finding that out. The chicken is just a way for an egg to make another egg. Yeah. I guess what it might establish is the in principle power of these things to keep expanding. It's still a fairly narrowly circumscribed set of tasks that they can do. And maybe, you know, if we buy your arguments, we can maybe say that they understand things.

[00:53:04] And maybe we could even agree with you that there's no limit in principle to what they can understand. But like what they can do in principle in some ways is not the question. The question is what they can actually do. And that's where sometimes it seems like you just assume, oh, well, there's no end to the kinds of tasks. We can't even conceive of them. We can't even... But it's still as impressive as the tasks are. It's still a fairly narrow boundary of tasks.

[00:53:33] Many of which, yes, are tasks that in a modern like capitalist society, humans are being trained to do because it's a computerized society and it's very good at things that involve computers. But I'm still waiting for that bridge to the more grandiose claims. I mean, look, it is already the case that like any illustration you see in the New York Times or any magazine that's by graphic artists. I mean, look, I'm sorry if you guys are paying a human to do that. You could save money.

[00:54:03] I don't advocate it. And I want to emphasize to say I'm ambivalent about this technology would be to put too positive a spin on it. And I get genuinely like sad sometimes. And when I see musician like playing in a New York subway, I almost sometimes like get emotional. I just want to hug them. It's like, you know, keep it alive. But the fact is people can play stuff to you that if you had heard it five years ago before you thought about this stuff, you go, yeah, well, it's not a bad tune. It's like the other shit I hear on the radio. That's a fact. OK, and that is a real consequence.

[00:54:33] It's happening on Spotify. You know, Spotify is saving money by intentionally crowding out human artists with AI slop. One of my hopes, by the way, is that people will so value genuine human created content that when that's verifiable, there will be something for humans to do. I think you may see a resurgence of live music. I mean, you see it for a moment now, but I think that may become a more valued thing. Live music at small clubs, stuff like that. I'm all for it. So one of the things that, you know, when you ask what can and can't it do, like, I

[00:55:02] do think that, like, again, this is a technology that may be able to mimic so much of what it is we do. And it's going so fast. But it is interesting. And as I was reading your book, I didn't see much discussion of this. And I wanted to ask you what you thought of this. There aren't a lot of examples of egregious mistakes that AI makes. And I find that the most illustrative things that AI does to remind me that it's not a human are the errors that it makes.

[00:55:32] And so, you know, one that's been talked about recently is like, you have now these like multi-billion parameter models. And the minute it gets posted on the internet, somebody says, hey, the car wash is like 500 yards away from my house. Should I want to wash my car? Should I drive or should I walk? And it says, of course you should walk. That's like, you know, right. And that's like hilarious. It hilariously has failed to understand what that question was about in a way that no five year old would ever do.

[00:56:01] And that to me illustrates that these things are incredibly powerful. They're just not doing what humans do. And it sounds like you have like some kind of functionalist view of the mind and you think that these AI models are doing, since they're doing effectively what human minds do, then we can say that they are like human minds. But there are these ways in which it's like clearly not, you know? Yeah. On the mistakes thing, generally, I would say two things.

[00:56:25] I do think they are dropping in frequency, at least in my experience, the mistakes that companies are getting better at. In some cases, just putting Band-Aids on, but they kind of work. And in some cases, it has to do with genuine breakthroughs like the so-called chain of thought reasoning, which wasn't even part of GPT-4 and came along later and has become hugely consequential. But the other thing I'd say is if we're talking about the future impact of the thing, one thing to keep in mind is people make mistakes too.

[00:56:54] First of all, we make stupid mistakes. The classic thing where you're talking to somebody and you're like, so I need to look this up on Google Maps. Where's my smartphone? And then you realize you're talking on your smartphone. That's a pretty basic logical lapse. And I even do things like, okay, she can just drive the car. But oh, no, wait, we just said the car will be here. So I do things like that. But here's another example. I told you guys I had cancer last year. It's fine now. But there was this fascinating thing when I was finding out I had it, which is I had this MRI report. I hadn't yet talked to a doctor about it.

[00:57:23] So you run it by an AI and it said, yeah, this looks bad. That looks bad. So it's basically, you know, yeah, there's a lymph node like this. Yeah. So it was bad news. But there was this one line in the report that, again, abnormalities found in. And it hadn't mentioned that. And I said, but what about this sentence? Abnormalities found. That sounds bad. And it said, I'm pretty sure the radiologist meant to put no at the beginning of that sentence. And it was right. Okay.

[00:57:51] Now, it wasn't rocket science. When you look at the structure of the sentence, it wasn't ungrammatical. It wasn't possible that they meant it. But the point is, this is a serious mistake for a radiologist. I mean, I guess it gets filtered out because maybe the doctor read it or not. But the point is, if we're talking about the impact of the technology, at least we're just focusing on economic impact. The question isn't, will it be able to do everything better than the best human or anything like that?

[00:58:17] You know, the question is, given how much it costs, given the error rate of the people it's replacing, given its error rate and the very low cost of having you just do a task a second time, blah, blah, blah. You know, that's the question. Yeah, yeah, yeah. But like, yeah, I fully get that these are going to be like even better and better and better. The point of the mistakes thing is just to show that it is unlike what the human mind is doing in that, like, it seems to really fail to understand things in a way that human minds don't fail to understand.

[00:58:45] I do think the reasoning capability has come late and maybe isn't fully instantiated. But again, the psychological literature is full of examples of the illogical, of human beings. But that's not what I'm saying. I'm not saying that humans are, you know, that would be a misreading of my argument. The idea might be we make mistakes in certain ways. The computer makes mistakes, though, that we would never make. And so there is some difference.

[00:59:10] Just the fact that we both are prone to mistakes is not enough to say, oh, well, this works like the human mind works. Right. Like our input is the same. The output is like really, really weirdly different in a way that would just lead you to question whether or not it's actually reverse engineered the mind. It just looks. I don't think it's reverse engineered the mind. I think it's reverse engineering specific functions of the mind and is not doing so in the order that evolution, you know, built the functionality. And so it's a weird thing.

[00:59:39] But I want to say just one other quick thing about in terms of why do I think humankind needs to focus? There is the apparent fact that the AI is accelerating its own development. I mean, Anthropic did a paper on this. Now, whether this will lead us to so-called recursive self-improvement where there's like no human in the loop, the AI just keeps making better versions of itself. That's a little hard for me to imagine. But there's a fair amount of evidence that that's already starting to happen. What is starting to happen?

[01:00:09] So as soon as AI could code, it started playing a somewhat meaningful role in the development of AI. In other words, each generation's AI, they would use it to do some code and help develop the next generation's model. Now it's contributing to that in ways other than coding. It's actually conducting experiments or suggesting experiments. Anthropic just did this paper on this. It's available on recursive self-improvement.

[01:00:33] And they provide actual data that with them, the AI is playing, I think, a quantitatively growing role in the development of each generation of AI. And that is accelerating the pace. So that's just another reason to expect that this is not going to slow down. I have a question about the stakes of this dispute between you and David on whether it is or isn't analogous to the human mind and how it works.

[01:01:02] It seems like what matters is what it's able to do, how it does it, and whether the how is the same how as how we do it. In terms of the larger questions, why does that matter? Why is that any more or less reason to predict the incredible transformative potential of these machines? If you ask what are the stakes, I mean, again, to me, the take-home lesson is, whoa, if you

[01:01:30] give it the right data, there's no part of the human mind and the data, you know, we're talking all the sensory channels. There's no part of the human mind that cannot be in principle, in quotes, reverse engineered. Again, I'm not saying the mechanisms are precisely analogous. We still don't know how exactly the human mind represents. Meaning, I will say that one longstanding model is very much akin to what these large language models we now understand do, which we didn't understand at first.

[01:02:00] But it's one of these systems where you have, you know, high dimensional space and each dimension represents like the feature of a word. So like a tiger, if two dimensions were speed and lethality, a tiger would be high on both. A tarantula might be kind of high on one, but much lower on another. And then there are all of these kinds of features of a word that collectively represent a lot of what we mean by meaning. So maybe the brain does it in somewhat the same way. We don't know.

[01:02:30] But to me, the main thing is that these machines figured out a way to do it. And so what? So like, so they did like, or they didn't. Like, what does it matter whether they did or- You guys are hard sell. I still haven't heard why that matters so much for the grander. What part of the human mind do you think cannot be reverse engineered through this methodology? But that's not my question, right? Can I just take a stab at like the big picture here, Bob, to see if I'm getting it right?

[01:02:59] Because I think it's important for you, given your view of what evolution is. So as you argue later in the book, and maybe some in the appendix, like you think that it is a reasonable way to understand what's happened in evolution by natural selection. As you say in this, the Martian's eye view, you think, isn't it interesting? And this is like the heart of the book non-zero, which I again adore.

[01:03:22] You say, isn't it crazy that we have developed into these kinds of organisms through evolution and that it seems as if maybe we're on our way to even a different kind of superorganism. You could see, like a Martian could see if they were looking at us for 4 billion years, they might think that what they're looking at is a superorganism that has evolved much in the same way that, you know, somebody who was just looking at ourselves one by one might

[01:03:48] not realize that what's happened is a human has evolved, an organism has evolved. And you argue that it is not unreasonable and it's certainly not unscientific to think of this evolution as having a purpose, as like, for lack of a better term, as having telos. Like there is some end state that can be described as having been arrived at purposefully, even without a designer.

[01:04:11] If these things that we have created are mimicking evolution in some important ways, if these things are rapidly evolving in the same way, what we're going to face sooner rather than later in evolutionary time, certainly, but maybe in lifetime time is a set of things that for all intents and purposes are going to be players in the grand superorganism that is earth.

[01:04:39] And it's best for us to make peace with this and learn to cooperate, not just with each other, but with them so that we have a utopian rather than a dystopian superorganism. And I think that the evolutionary analogy, the parts that you want are the parts that really matter for saying that these things might catch some of that same purposey telos stuff. Is that fair? Is that a...

[01:05:07] Well, first of all, I would say, I don't think we need to make our peace with it in the sense that just accept the inevitable trajectory and live with it. I would be fine with a global pause on the training of new models, okay? I'd be fine with that. Right. And just to briefly revisit... But I think you think it's not going to happen. So I'm just meant to make peace in the sense that... Well, look, I am advocating... Accept it, accept... The kind of international governance that would be a prerequisite for doing it. You know, you're not going to, you know, as a political matter, convince people that our

[01:05:37] AI companies should pause for a while before training a whole new generation of models on these huge training runs unless they can be convinced that China is going to slow down or something. And I am... But big picture, Bob, I just want to know, did that do it justice, the big picture, before you get caught up in whether... Well, I'm a little... I'm not sure because... Well, you tell me, do I have what you're saying right? So first of all, yes, I do think that if you step back and view in time-lapse the whole

[01:06:04] history of life on this planet, you see a pattern. Life moves to higher levels of organization, bare strands of self-replicating information, cells, multi-celled organisms, societies of multi-celled organisms, you see a growth in intelligence. Finally, you get a species of multi-celled organisms whose societies, because this species launches in a certain sense, a second kind of evolution, cultural evolution, ideas, technology, and so on. Its society, its social organization starts expanding, thanks to technology and stuff,

[01:06:34] and eventually approaches the planetary level. And in fact, you could call the global economy at least a kind of global brain. It's a global information processing system. And I am personally saying that if we know what's good for us, we're going to increase the amount of governance that happens at the international level. You know, I cite Teilhard de Chardin, who came up with this term noosphere 100 years ago to refer to this global mind, planetary brain. Yeah, it's kind of happening.

[01:06:58] Now, the reason I bring it into the book is, first of all, well, let me cite reasons that are kind of separate from the purpose question. You know, Teilhard saw that there was a brain of brains, as he put it, and even 100 years ago saw that technology was drawing us into these kind of collaborative webs. But he imagined that the neurons, as the global brain matured and would have this kind of culmination that was of theological significance to him. He was both a paleontologist and a priest, that you would get this cohesion of the human

[01:07:28] species, but all the neurons would be human brains. And I'm saying, well, you know, he's right. I mean, the internet happened. You got to call impression. It does look more and more like global brain. But suddenly we are confronted with the possibility that some of the neurons will be silicon. And I think we need to think about that and what our relationship to them is going to be. And by the way, there's a literature on this that I get into. You know, the issues that the Overton window of AI is now expanding to encompass, like super intelligence, the singularity, and so on.

[01:07:57] There's this group of people, including Eliezer Yudkowsky, who were talking about them 15, 20 years ago. And one of them is Nick Bostrom, who wrote this book, Super Intelligence, more than 10 years ago, and had a big impact on people. And it gets into the idea of the singleton, which is that this is heading toward some system of global coordination. And I had him on the podcast, and we agree that some degree of global coordination is in the interest of the human species. But he lays out various scenarios where you could wind up with artificial super intelligence

[01:08:27] running the show on the planet. You could wind up with people doing it. You could wind up with a combination. But in any event, it would all be, in his terms, a singleton. And so I think leaving aside purpose, I wanted to look at that whole set of issues and the many ways that I think things could go badly awry. And I want to emphasize, the main reason I wrote the book is I'm pretty worried. You know, like there's a lot of ways you could have radical international destabilization.

[01:08:54] I think thinking of this as fundamentally a race with China is like a suicidal mindset to maintain. We need to get over that pronto. Now, back to the issue of purpose. I mean, you can think through all the stuff I just mentioned without asking, is there some larger purpose? I have always thought that's a more legitimate question than people, especially Darwinians, were giving it credit for. And I think one reason is a lot of Darwinians thought, well, do I have to abandon natural selection?

[01:09:19] Do I have to abandon a mechanistic conception of how evolution unfolds and cultural evolution unfolds? And I say, no, you can. There's such a thing as a machine that has a purpose and it unfolds. And in fact, an organism is arguably an example, right? You would say that natural selection is what instilled in it the goal of genetic proliferation, the prolific goal. It, in a certain sense, has a purpose, a telos, genetic proliferation. So I've always thought it's just an interesting question that if you really step back and look

[01:09:46] at the 3.5 billion years of the evolution of life, you do start thinking, you know, this has in a way the kind of directionality we see in an unfolding organism. And in the case of an unfolding organism, we know that, well, yeah, it was kind of designed to mature. Yeah. I mean, by natural selection. So I mean, designed in quotes. So I've always thought that was intellectually interesting. In that context, I think it's interesting that I argue there's a kind of moral directionality to it. In other words, we're going to have to up our moral game if we're going to get through this in good shape. Maybe that's relevant.

[01:10:14] But I sensed that you were attributing to me a little closer a connection between the main concerns of the book and my interest and purpose. And the purpose. I mean, there's a reason I confined it mainly to the appendix. Yeah. Although you do make. I bring it up. I bring it up. In the last few chapters of the book proper. Where you shit on Pinker. Yeah. Oh, that is so unfair. I mean, look, you can shit on Pinker all you want.

[01:10:37] I had lunch with Steve and his lovely wife, author of the recently published book The Mattering Instinct, which I discussed with her on my podcast. So the idea is the reason this is going to usher us potentially into a new phase of our existence, whether good or bad, we don't know, is because the planetary neural network, in quotes, you know, this thing where we're all connected.

[01:11:05] There's something about the fact that we're also going to have these computers. There's this new factor because we had humans with human minds already. Right. So there's something about these, though, that's going to usher us into this next phase. You know, I'm largely saying we need to get better at this kind of cooperation if we want to make sure that the machines don't get out of control and out of control could mean a lot of

[01:11:34] things. It could mean at a fairly trivial level, like they keep getting in the hands of these bad actors who design new bioweapons and start global pandemics. That would be a form of out of control. It could be that the machines themselves periodically go rogue and you get these self-replicating super hackers that hop from data center to data center and commandeer the compute and take out our satellite communications infrastructure. It could be that you have the sci-fi scenario where it actually becomes a collaborative super mind.

[01:12:03] In other words, the AIs kind of, you know, coordinating with one another and they take things over. I can't totally rule that out. It's not front and center of the book or it could be. And I think this is the nearest term concern is that either this ridiculous breakneck race with China will lead to war between us and China because and people need to understand, by the way, that Dario Amadei, the head of Anthropic, he is a hard core. I don't know if neocon is the word.

[01:12:30] It kind of is because in almost a pure sense of the term, it's like he sees the world as an existential struggle between democracies and autocracies. And he thinks we need to beat China to the super intelligence level, bring China to its knees, demand that it, quote, quit competing with democracy. I don't know what he means by that. Does he mean regime change within China? Whatever. He's advocating a truly breakneck race. And as others have pointed out, including even Eric Schmidt, who's something of a China

[01:12:56] hawk in a paper that I quote, a race like this, both sides buy the premise that as you get to super intelligence, you have truly hegemonic power, A, and B, that as you approach it, you accelerate. And there's some evidence that that could happen. Then it's only natural for the country that's behind to resort to extreme measures to keep the other country from getting there. So if China accepts Dario's worldview, thinks we're serious about this, you could have a war between nuclear powers.

[01:13:26] I worry about that. But I also worry that the sheer pace of change will be socially destabilizing and the pace will be accelerated by this arms race mentality. Well, you know, if that keeps up and the technology precedes a pace, I think you could well get the kind of destabilization in America, the kind of chaos and disorder and just disorientation, you know, it isn't just jobs, although I think that could be big, but like parents freaking out because their kids have these AI friends and they don't have real friends and all the

[01:13:55] other stuff that will just kind of freak us out. I think that will make our country more ripe for the kind of authoritarian takeover that, you know, as frankly seemed more plausible to me since Trump showed up than it did before. Right. And so I worry about that a lot. So did I take us totally off subject? Did that was that totally unresponsive? It's a rant I had to get out of here. You know, it does seem like the kind of thing you might say about, you know, gunpower or

[01:14:22] nuclear weapons or like, you know, which definitely changed the world in a big way. And it's something that had to be navigated and coordinated at an international level. Those kinds of worries are not on a different order of magnitude. Like qualitatively. They're more like quantitatively different. Yeah. Well, I think the rate of change is different. So the printing press, when it showed up, it's like five years, 10 years, 15 years. Fine. But eventually it did. Some people think it did start the Protestant Reformation.

[01:14:51] And moreover, that perhaps the so-called wars of religion decades later would not have happened. But for these divisions, I don't know. But my main point is there's such a thing as a technology that you eventually work through, but you'd still rather minimize the turbulence. You'd rather not have the wars of religion if you're one of the people who dies. And I'm just saying this is going to happen super fast. And I do think as we approach the global level of social organization, you have to worry that we could lose something we've always had before, which is, you know, competing political experiments.

[01:15:22] Right. You know, I do think this technology in various ways makes it easier to imagine that 10, 20, 30 years, you could have a single global government, quite possibly not in a sense I would welcome. And once you're at that, I don't know how easy it is to turn the clock back. But I guess I'd also just say the diversity of kind of profound issues raised. I don't know. It's like you tell me. I mean, you're not sensing young people in some numbers going, whoa, is there going to be anything left for me to do?

[01:15:51] I don't recall hearing that ever. No, they're very worried about it. I mean, my students are very worried about it. You know, they also don't like it and resist it. They find it annoying, but they use it to write their papers. It's totally fucked up college education in ways that we've talked about on this show. One of the ways that it has been most disruptive so far, I think, is in education, which is a huge deal. I completely grant that. But it's also a great intellectual resource, right? Like it allows you to explore.

[01:16:20] I mean, haven't you found cases where you thought, wow, this is kind of like being able to summon a leading expert on this subject and interrogate the person. And that's the way I learned most efficient. Tamler doesn't use it very much. What's that? Tamler doesn't use it. Not that much. Yeah. But I don't use it very much. I don't deny that it could be very helpful. And, you know, where it comes to some of the administrative tasks, like it seems very good at that. And the people who have to do more administrative work, they attest to that even more.

[01:16:46] The research capability has gotten quite a bit deeper, not because there's been great depth of advance, but just the harnessing of the things to search engines and the kind of so-called reasoning ability of it have combined to really enrich the experience of just learning about stuff generally. It's incredible. As a kid who loved to read the encyclopedia, like this would have been just mind-blowing. I mean, books are in trouble. And I just, as you may know, I have one coming out now.

[01:17:15] And, uh, and it's like, God, uh, look, I feel lucky. I, I've gotten to, you know, what is, uh, getting near the end of my career and, uh, it's worked fine, but this model ain't going to keep working. Here's something, you know, I remember like when I started this book, not that long ago, more than two years, but I was having a conversation with my editor and my agent. And I said, this is the last book I'll write where all my competitors will be human. And they laughed and almost anybody would have at that point, but I wasn't kidding because

[01:17:44] of course, you know, I was extrapolating. They wouldn't laugh today. And I think that's an example of how, you know, the goalposts keep moving and, you know, the things we thought were absurd two years ago, we forget that we ever doubted. Yeah. Can I ask, can I ask a question? Because everything that you've said, uh, like to me makes sense. Like this is a technology that is destabilizing in ways we've never seen, or at least potentially destabilizing in all the ways you've talked about. But then there's this other part that is the part that always loses me.

[01:18:14] The, the modern discourse is gone. These things are really dangerous. Not just because in the hands of humans, we could do things like build bioweapons and, you know, cheat on exams and all this, because all that stuff is like a tool in the hands of humans. But there's a deep fear that these things really are going to be agentic and motivational and like take over. And that's the part I write in your book where you're just like, sometimes you're like, maybe, maybe not.

[01:18:39] Like, I can't get behind the deep concern that these things are going to motivationally desire to take things over. And that they're not just really, really powerful tools in the hands of humans. That's the part I don't mean. That might turn on what you think these things are. The reason that it matters that I think they're just like really, really good computers and not minds is that I just don't see them like the whole singularity shit and all that. Like, I just can't get behind that. But like, does it matter for your take?

[01:19:09] Which is what mattered that? Does it matter whether they're just really dangerous tools in the hands of humans and that's why we should all like be more cooperative? Or like, are you really concerned, actually? I guess I should just ask. Like, are you really? I mean, agentic potential. Yeah, no, I'm concerned. I mean, on the power issue, there are these like, will they seek power? There are standard answers. I covered them in the book that make it hard for me to rule out that there could be a serious problem. They're like hard to rule out. But do you really think that this is like a lot of things are hard to rule out?

[01:19:37] Well, no, but I mean, I feel confident. That my dick is 10 inches. How are you going to rule it out? How are you going to rule it? Actually, there's a way. But I feel pretty confident we got major problems coming up, leaving aside the one you just raised. And I would just point out that the one you just raised, and I spend a fair amount of time on this in the book, it's not that easy to dismiss. And one reason I spend time is because there's kind of a range of intermediate things, too.

[01:20:05] It's just like, even if, okay, we don't get the Yudkowskyite takeover the whole planet, if there is enough power seeking and enough deceptive tendency, and both have been documented, by the way, you know, as things that can happen, then you could just have, like, repeatedly shit getting seriously out of control, even if the true sci-fi scenarios don't unfold. But my, again, my main argument is that if we really understand the kind of the secret sauce,

[01:20:32] what's gotten us this far, we should understand that we really need to brace for impact. And I would say reduce the pace of change to the extent that that's possible and start thinking about how we're going to handle this. It's a relatively mundane threats that I'm most confident are worth worrying about. I will say that listening to you, David, I'm now wondering whether the main difference between us

[01:20:58] for purposes of the specific issue you're focusing on is that I have a pretty mechanistic conception of the human mind, right? Like, so when you said, okay, it'll do this and that, but will it behave like a true mind or something like that? I'm like, well, why not? Now, I grant the existence of consciousness should force us to be agnostic, strictly speaking, on the ultimate nature of human nature, you know, at the metaphysical level, kind of.

[01:21:24] But at the same time, maybe one thing that's happening with me is always my default conception of what's going on with human beings is pretty mechanistic. Yeah. Well, I mean, as Tamar will tell you, I'm nothing if not mechanistic. I just don't think that this is doing what people think it's doing. I think that natural selection working in the way that it did on our biological systems, the way that it did, just gave us like this way of being that I don't think machines are really doing.

[01:21:49] And I find the hand wringing and like the Yudkowsky stuff, I find it almost disingenuous. Like, I'm sure he believes it, but it just feels like they're taking the focus away on maybe the truer things, which is that our children are fucked, you know? He'd probably grant those, but you're right. He's focused on what he feels sure is the ultimate existential threat. Yeah, it does seem like even if we can't rule it out because we don't know how we're conscious,

[01:22:17] there's still the question of, okay, even if we can't rule it out, do we have any reason to take seriously this idea? And I guess that's the part where maybe the details matter on the analogies of the particular mechanisms and how much it matters that this is the product of organic evolution and that has taken place over billions of years under conditions where we had to interact in like

[01:22:45] real social ways in the way these computers don't. Well, they're showing social tendencies and collaborative tendencies, but... Yeah, I mean, we don't have time for that, but you say that, but I just think that they're mimicking the inputs. Like, I just don't have any reason to believe that that's not... I'm not sure it matters. I mean, yeah, in some sense they must be, but well, no, they mustn't necessarily be. In a way, that's a distinct scenario that I'm sure is somewhat at play, but ultimately what matters is what they do.

[01:23:11] I will say, I think there's a tendency to conflate the consciousness question with the kind of volition question or the kind of, are they going to start exerting agency in some metaphysical sense or some meaningful sense? I don't think that's necessarily... Like, free will isn't necessarily the same question as consciousness. I personally am agnostic on both of those issues as they apply to human beings. You don't think people are conscious? No, people are definitely conscious.

[01:23:39] I just don't know why they are... I do in the appendix, I will say, just quickly for mind-body problem obsessives, I do think I have come up with a sense in which an epiphenomenal consciousness could have a function. Many people will not know what I'm even talking about, but the people who do will say, no, that can't possibly be. To which I reply, looks like you'll have to buy my book. And I would say, read that appendix first. I think that's probably like the most fun appendix. That's the most fun of the whole thing.

[01:24:09] Do you want numbers separated by commas or do you want the real maverick Bob Wright thinking here? I actually do disagree, and not just for marketing purposes. But I mean, I think what most people who are just kind of waking up to the issue now would like is have a clearer sense for what exactly is going on with these things. And I would like them to have a clearer sense that it could become very powerful very soon. I mean, to each his own. It's an embarrassment of riches, this text. Thank you.

[01:24:38] That's much better than numbers separated by commas. If I'm choosing between those two blurbs, it's like... No, you said it's the same. You said it's the same. Numbers separated by commas. I didn't characterize my book as numbers separated by commas, which you frankly... It's numbers separated by commas in n-dimensional space with layers and back propagation. Since we don't have time to get in a Chinese room argument, will you guys just agree with

[01:25:04] me that I left John Searle's Chinese room argument as just a smoking ruin? I mean, we were never like converts to the argument. There was always a sense that it was begging the question against some kind of functionalistic count of the mind. But I don't know that... Again, I was wondering about like the stakes of this because you want to bracket the question

[01:25:30] of consciousness and whether it's having a subjective experience at all. And so you want to come up with a meaning of the word understanding that doesn't involve the subjective experience of understanding that we're all familiar with. And that's fine. Like you said, you can divine words however you want. And that's a perfectly fine way of defining understanding in a way that allows us to recognize that the machines understand.

[01:25:54] I think, you know, the thing with the Apple and adding the, you know, the visual, the multimodal networks, that was persuasive. I just don't know if it quite got you, you know, because then afterwards you say, are there any elements of human understanding that AIs can't be expected to acquire? And you say in principle, no, and you could find that exhilarating and terrifying. I just, you know, like this is the thing I keep bumping up against personally is, okay,

[01:26:24] like in principle, maybe not if we grant that it understands in a certain sense of the word. But what reason do we have to be worried about something that would be terrifying or exhilarating actually? And I don't think that, you know, whether this replies to the Chinese room argument has much bearing on that question, which I think is the big question of the book. Well, it was kind of a dual purpose chapter. I wanted to get the implications of multimodal AI out there.

[01:26:50] And the fact that, you know, if we build a true so-called world model that's multisensory, you know, that'll be very important. And I have no reason to believe it can't happen in the same way that LLMs did their stuff. So, you know, there's that part of the chapter. But I do think, given how much attention Searle's argument has gotten, that my claim that actually, if you look at his argument very closely, like how he's using the word intentionality,

[01:27:17] what philosophers mean, the fact that he's not, he doesn't seem to be saying consciousness is a prerequisite, which is, by the way, kind of the consensus idea about his 1980 paper, I think. That's not just me. But, and it's an interesting argument he's making that it isn't LLMs that kill it dead because their version of semantic representation doesn't quite speak to the issue of intentionality, which is the connection of the representation to the real world object. Right. But, or the real world thing.

[01:27:47] But I do think multimodal AIs really do that. I think it's a rare case where technological development strikes at the heart of a famous argument, you know? And so I'll let people judge, but. I thought that was a real, I mean, I thought the multimodal point was a really good one too. We can close on a note of praise then. How refreshing, I gotta say. I didn't like that you questioned or held in doubt the consciousness of your dog, though. I said I give him a 98.5%.

[01:28:15] I dedicated the book to the memory of my two dear departed dogs, the second of which died a year ago. Yeah, but. He gave his wife 1% more. Yeah, he gave him my wife. I didn't give 100%. I don't know for sure that anyone has sent you. Frankly, I'm giving her slightly higher numbers than you two. That's fine. I would put my dog over my wife, though. Like, dogs should have. Well, I respect the hell out of that. All right.

[01:28:43] Well, it was fun having you, Bob. Thanks for putting up with our, you know. It was fun. I am honored. You're a good sport. Thank you. We weren't going to lob you Paul Bloom softballs. I was texting him earlier. No, I mean. That was just too nice. Wait, Paul is too nice? Yeah, yeah. Paul is too nice. Could we talk about that a little? No, I mean, it's a problem. It is a problem. I've tried to talk to him about this. Oh, well. We love you, Paul. We do. But really, toughen up, man. Yeah. This is the real world. Seriously. It's a jungle out there.

[01:29:13] This ain't Canada, buddy. The world ain't Canada. You don't get free health care and no guns. Yeah. All right. Thanks again. Hey, thank you. This was fun. Join us next time on Very Bad Wizards. A great man.

[01:29:45] A very good man. Good man. Anybody can have a brain. Just a very bad wizard.