SPEAKER_01 0:00

You are listening to the AI Investor Podcast from 24-7 Wall Street. On today's episode, we discuss whether or not AI is going to go full Skynet and kill us all. We take a look at what's happening with the Federal Reserve and what might be one of the most interesting and terrifying hacks, breakthroughs, jailbreaks, whatever you want to call it in AI, all that and more is next. Eric, it's been a little while. It's good to see you.

SPEAKER_00 0:22

Yeah, Austin. We obviously last week we had the episode that was the large presentation. If no one's seen that, uh, we've gotten some really wonderful comments on YouTube. I love being able to create something that there is an assumption that everyone kind of knows where we are in AI this long into it, but it's moving so fast. There's so many features of it and kind of understanding everything happening in the background. Um, it's just really great to be able to make a presentation like that. If no one's if no one's watched it. And hey, we're back to a little bit more of a regular episode this week as well. So if that wasn't your cup of tea, we'll be we'll be talking about some pretty pressing news this week. Like I've like you said at the beginning, there's there's kind of no bigger question to answer than is AI going to go full Skynet and kill us all? And we'll we'll break that down in a little bit of nuance and we'll also talk about what it means for your investments after a few days that if you're holding AI stocks, probably felt really uncertain.

SPEAKER_01 1:17

Yeah, I mean, this is this episode is going to get a little sci-fi. It's gonna get a little philosophical, but I'm looking forward to it. It is the biggest question in AI right now, other than does the CapEx cycle continue, which might be our biggest question. But uh uh humanity itself might be a little more important than whether or not we get a return on our investment. But Eric, yeah, because it's been a little while since we've chatted, just three quick housekeeping items, and we get through these really fast. One, your presentation, the last episode, absolutely fantastic. A very long episode. You took pains to make it accessible in audio format. But if anybody listens to us in podcast format, I recommend going to view that episode on YouTube because there were some really great accompanying visuals. You put a lot of, I mean, I can't say enough about it. It's one of the best presentations I've seen. Second housekeeping item, everybody, please come give us some comments, give us your feedback. Go see Eric's last episode. As always, we know this is a community. We want to hear from everybody, we want to hear what we get right, we want to hear what we get wrong. We also get a lot of stock picks from our members. So please share in the comments anything you're interested in having us cover. We love to hear from you, and it does really help the show circulation. We see a big bump in listenership and viewership. When we get that engagement, it means a lot to us. Thank you very much for taking a few minutes out of your day to do it. And one last point, I want to respond to some questions that we've got. Now, Eric, we do not have sponsors on the AI Investor Podcast. We've never done it. Um, but we do have some affiliate relationships with other companies out there, and one of them is Yahoo Finance. And we have received a lot of comments as your portfolio has grown from users who want to know how do they track the portfolio? We have a spreadsheet that they can access, but that just shows the picks and the dates they were recommended and the current price. It doesn't give you any other information about the position. You track the portfolio in your own brokerage because you own all of them. But if there's users or listeners out there who maybe own half the stocks, they want to watch the other half, they're having a really hard time knowing how to do that. So we get a lot of questions from people. What's the best way to create a watch list portfolio to track these companies? Or the ones you own, how do you get all of their earnings information in one place? So I have started using Yahoo Finance's Alpha Space tool. And you and I have set up profiles where we track the entire AI portfolio in Yahoo Finance Alpha Space. You can watch it all of the earnings, you can see all of the positions at once. It's a really great resource to get all the financials on either positions you own or a watch list. So I cannot recommend Yahoo Finance's Alpha Space enough. It's a great product. We're going to put a link in the show notes. If you click that, that is associated with us, 24-7 Wall Street. So we might get a small commission on it. But really, uh, I recommend everybody go give it a free trial, see if it works for you. Because when you get to a portfolio size of dozens of companies, as the AI investor portfolio already is, having a good watchless tool like this really makes it critical to see everything at a glance, to get all the financials and to get all the news and information, particularly around earnings when it can be a bit of a squall and a snowstorm out there.

SPEAKER_00 4:03

Yeah, Eric, I would just say too, Austin, you know, Yahoo Finance has been the number one financial website for a reason for a long time. And they continue to innovate. And, you know, this is this is a brand new product. So if you think you've used Yahoo Finance before and know what it has to offer, you should check Alpha Space out because it's something that they recently rolled out. And again, this is a company that we're really proud to partner with. So it if you are not currently having a great system to track your stocks, I I couldn't recommend going and checking this out.

SPEAKER_01 4:32

It's a great product. It really is. And check out the link in our show notes if you want to uh get a quick link to that now just to know where to navigate. Okay, Eric, let's talk about the wild last few weeks. Um it's hard for me to distinguish what is real and what is sci-fi. I'm deep in a couple AI sci-fi books right now. I'm doing the Ian Banks series where we're talking about the culture. And it feels, it feels very much like uh reality and sci-fi are merging. We're we're we're hearing that uh AI is gonna kill us. There's an anthropic employee slash whistleblower, if you may, who um revealed some pretty terrifying headline news. Um I'll leave it to you to describe what's going on here.

SPEAKER_00 5:12

Yeah, and I think two things are important. Number one, you said at the beginning, um we we will take a hit to our ROI if AI is more safe. And that really cuts down to the thesis for what we're doing here. We we want to invest in AI because it we believe it will be a generational technology, but we also want to invest because we believe it's something that's a good for the world and and will be an extreme net positive for humanity. It is powerful enough there's probably going to be some speed bumps along the way, but everything comes back. You want to invest in companies that reflect your best vision of the future. That's something David Gardner, our former boss, used to say. It's something I deeply believe in. And I believe AI can create that best vision of the future. But one of the reasons we often don't talk about these kind of high-level ideas is because it doesn't apply to the investing world. And what we're seeing this week is some of these kind of fears around AI broadly spilling into the media space, broadly spilling into how AI will be regulated. And we absolutely need to talk about it because this is going to be front page news. And there's a lot of background we need to talk about that. You're going to see headlines. And it's important to know what is truly scary and what often is more of something that is a little bit of a, for lack of a better word, a scare invention. So we're going to break that all down today. This really started on September 8th. Um, I, you know, this episode's gonna be terrible, Austin, because I'm gonna have to say so many names. And even if I've heard it a hundred times on TV, I won't know how to say half the names here. But uh Jacob Cockchin, I believe it is, C O X N, he had quit anthropic and he issued a thread on the social media network X. It started spreading like wildfire, not just in social media, but also traditional media. It was really picked up. It got a lot of attention. And an anthropic researcher responded to this thread shortly after, um, saying, Hey, Jacob's correct. We do earnestly believe AI could kill all humans. And he placed greater than 10% odds of it happening in the next decade. So, Austin, a few things to note. Number one, is being the head of PR at Anthropic the worst job in the world? I just can't even imagine to respond to all these comments from your employees. Second, I should note, it looked like Jacob Coxin, he had appeared to work Anthropic for a very small amount of time, uh, it measured in weeks or months. And his post went remarkably viral, despite being a brand new account. So you always do have to look at matters like this and and question how organized this was, what organizations are potentially behind it. Because this this clearly wasn't something that seemed to be a grassroots effort. This seemed to be something that was a little bit more planned. But we can put all that aside because regardless of how that was determined, over the weekend on Saturday, so now we're up to September 12th, anthropic CEO uh Dario Amadei, again, which I'm probably getting wrong. I think it's Amade. Amade. Okay, Amade released a 3,800-word essay. And the theme of it was pacing the frontier. So there's some quotes from it we should probably share here because they're fairly striking. And I quote, over the last few months, I have been I have become convinced that fully addressing the risk requires even more prudence. And he continued, we must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. So Austin, he cited two things that convinced him to issue this essay call to greater cooperation, whatever you want to call it. First was since the summer, AI progress has accelerated. Um, we've talked a lot on this podcast about recursive self-improvement, and that's AI's growing ability to build the next generation of AI. And he said, left unchecked, it would outrun the ability to understand and control these systems. And he also cited a specific incident that's become known as the open AI Hugging Face Incident. And it's its own topic, we could probably talk a little bit more of. But the idea is it's a swarm of agents. They gained access to the public internet and they compromise parts of Hugging Face, which is essentially uh built to create the benchmarks for these models. And they're essentially trying to cheat. They were trying to break into Hugging Face, and they successfully did compromise it to be able to essentially look for the answers, you could say. Um, and then Dario also said, given the accelerating rate of AI capability development, it's my worry that in six to twelve months, such a swarm, as we saw in this Hugging Face incident, could be capable of taking over, and I put this in bold, the entire internet with a persistent botnet. So two parts on this. Um first, what's misaligned today could be exponentially more powerful in a year if if we do not have proper alignment. We'll we'll talk about a little bit what that means more. And two, this is important. The tools that will be in the hands of bad actors soon, which would require stronger safeguards before they're released to the public, and and that's a that's an important idea as well. What safeguards are being built in it could be more powerful than than most people, just 99% of the population realizes. And he he created a three-step plan. What is pacing the frontier? Well, for him, it's a three-step plan. Number one, he wants to put embedded evaluators, which is a third party to verify safety practices, and he committed his own company to doing that step today. And we'll we'll talk about what this looks like in practice. Second, he wants democratic coordination, and that's a coordination of labs across basically Western countries. And third, he wants global cooperation. And, you know, Austin, the reality is this is coordination with China. It's it's not global coordination, it's it's it's largely just with China itself. And um, you know, he was pretty clear that this this is more of a long shot. So moving this time frame forward, on Monday, we saw a significant uh sell-off. I was looking at some of the chatter over the weekend and a lot of calls that we would have effectively a deep seek moment. Um, we didn't quite see something that strong, but we did see this familiar rotation. The hardware stocks, the stocks making the actual chips and uh actual physical components for data centers, they all fell. There's a rotation back to software stocks. And beyond the market, we saw just about everyone kind of who is a figurehead in this AI race weighing in. Donald Trump said he was strongly opposed and called fears of AI destroying humanity, and I quote, a hoax. China's Ministry of Foreign Affairs called it fear-mongering. China's official mouthpiece, which is the Global Times, called it something out of the Cold War playbook, targeting China. Sam Altman, who's the CEO of OpenAI, endorsed it, saying, I agree with Dario and we need to pace the frontier. Elon Musk first responded, Dario is right, but he later clarified it to say Dario is right. There should be some oversight. We'll talk about the differences between these two a little bit later. The founder of Deep Sea called it a step in the right direction. And then we had some people within the AI space that maybe had a little bit differing of opinions. Mark Zuckerberg distanced Meta from a coordinated slowdown. He issued his own essay, but he did endorse aspects of Dario's plan. And Jensen Wong, Nvidia CEO, said, we don't need any new laws. And that's a direct quote. And he seems a little bit more on the spectrum of essentially, he believes companies uh should make the decisions rather than instituting new laws. So Austin, we we've talked about this this timeline. We've we've moved from kind of this trigger point of kind of uh large social media, uh you could call it panic, you could call it whatever, to um the industry weighing in. And now this brings up three questions now that this is basically daily news on CNBC. The big one, which we'll try and answer our best on, is AI going to kill us all? As you said, are are we on the precipice of something out of the terminator? And another question why is this discussion suddenly happening right now? It it seems like odd timing. Anthropics about to have an IPO and is putting out an essay essentially saying we need to slow our industry down.

SPEAKER_01 14:14

Second, well, look at what's not to mention, there's also supposed to be not, and I don't think regular rules apply anymore, frankly, in the world, it seems, but there's also supposed to be a quiet period where you're not supposed to be saying these things right now. And this is in its strange business sense. Um, not that everything needs to be about raising as much for the IPO as possible, but I don't understand it. You're supposed to be in this quiet period. You've got Dario putting out this like existential risk document. You have a whistleblower. You can't really affect Jacob going out there in the world. But Evan, I think Hubinger, Hubbinger from Anthropic, who's still an active researcher and is a lot more senior than Jacob Coxson, replied, doubled down on his comments. Employee at the company. Yep. Yeah, employee at the company said, yes, that's right. I think there's a 10% chance we kill humanity. So I don't know how you do this while you're in this quiet period. I also don't know how you do it from a liability standpoint. Like, what do you now expose yourself to when I mean, inevitably you will be able to tie some deaths, and people have already tied some suicides to AI from the, you know, tragically to the messaging they received and being in their uh frail mental state. Now there's this huge motivation to tie a huge number of deaths and bad events to AI for the purpose of lawsuits. I just I don't understand any aspect of this. It's like totally living in the upside down right now.

SPEAKER_00 15:32

Yeah. And we'll we'll get into that because there's there's what's likely happening from the proposals, and we do need to talk about this kind of future liability question. Austin, this is the third one. Um what are the investing implications? Because we saw kind of a panic on Monday. We've seen somewhat of a rebound since in the market, largely trades where it would have been last week, but a lot of uncertainty and fear in the meantime. So um I I think that's kind of the kick it off. And I don't know if if you want to frame up a little bit uh the the key question of if AI is gonna kill us all, or we we can start talking about it.

SPEAKER_01 16:11

No, uh uh uh let's get into it. I just want to I'll add uh a little bit more color and then I actually let's rip to the three topics, but the little more color. There's been a lot of different proposals, right? I think Demis from uh Alphabet proposed the FINRA for AI. Uh Elon Musk has proposed a sort of a competitor review, like the Motion Picture Association. You mentioned that Altman agrees and we want to slow down. So all of the leaders in this place or in this space are calling for pacing, which I find confusing. I also just don't know how it works or why you would do it, because we are struggling to pace China by limiting their chips, and there's no way you're gonna get cooperation from them on this, no matter what they say or what you think. So we're gonna, on one hand, slow them down with chip regulation, and then we're gonna slow ourselves down to the same pace with regulation. It doesn't, it's like, then why bother trying to slow China down at all, in my opinion. So I'm I'm confused why the leaders of this industry are calling for a regular uh slowdown. The other thing, and this you don't need to answer this. I'm just gonna say I am the most confused by this because if these leaders want to slow down, nobody's forcing them to mash the throttle, right? Like if they all believe in a slowdown, they should all slow down. I guess it's it's odd to me. It's like they feel it's they're acting as if they're on a runaway train and they need the government to slow them down when they're the conductors. But I'll leave it at that. Let's get into the three topics now. I'm just I'm just on my soapbox, and let's actually get into the real implications for our listeners. Nobody's here to listen to to Austin's soapbox hour.

SPEAKER_00 17:40

No, I mean it's all it's all great things. And this is we'll we'll try our best to kind of poke at this throughout. We do almost have to step through breaking it all down, I do think, because it is such a big question. And we are kind of biting off something massive that, again, we haven't necessarily covered it in prior episodes because it is more of a policy issue than necessarily investing issue, but it has become an investing issue now. So we're we're gonna tackle it. Austin, I do believe. So why would they be doing this now is is kind of the key question. Why, why is this all of a sudden suddenly bubbling to the surface? And I really do believe this comes down to recursive self-improvement and how much this has accelerated models. And and I believe that part of this, there's a lot of areas you could say it's self-serving for someone like Dario and Anthropic. Um, I believe that this is a very genuine concern. And this is a part of AI we've highlighted on this show. We we've talked about the implications of what's going to happen when models improve themselves over and over. I think there's two key things people need to understand about this.

SPEAKER_01 18:49

I do want to I do want to emphasize one thing you said. There is no doubt in my mind that Dario and the anthropic team are genuine when they say this. I do not believe any of the cynical takes that they're trying to do this for regulatory capture or they're trying to do this to drum up interests. Like, I don't, I take a very I take Dario at his word and you read the things he's wrote written and you listen to things he's been saying for years. Other people in the space who've met with him say, oh no, he's genuine. So I do think we should take him at his word here. There's no, there's no alter ulterior motive. There's no deep cynicism around regulatory capture, I believe.

SPEAKER_00 19:21

Yeah. And and so I think with recursive self-improvement, why is why is this causing this moment from Dario? And number one is the fact that we are now in an agentic world. Um, we've talked about, if you watch my presentation last week, I gave a ton of statistics on this. AI going from something a typical consumer used in chat interfaces into autonomous action. I gave the statistic, token output doubling every 11 weeks. And the amount of token output from Chat GPT um that's agentic, going from about 1% a year last year to 64% recently. And that's that's likely outdated. I I bet most of the models are now 80 to 90% agentic. So when Dario says in his essay a slowdown would have accomplished nothing until recently, there is truth to that because AI was so much less capable, but also the autonomous action makes it not only incredibly more capable, um it it makes it incredibly more capable in the hands of malicious actors. Because you think of what someone could do targeting, you know, this this idea of a bot farm or ideas like that. This was just something that AI wasn't incredibly helpful at a year ago. But with the use of thousands of agents that are incredibly clever, what can happen from a malicious actor has exponentially increased. So that's number one.

SPEAKER_01 20:53

And second, if if people really want to understand a good visual of this, and there's some criticism in how he has presented it, but I actually think it's a fantastic episode to understand the argument here. Um, people should go listen to the Dwarcash podcast on this. Now, he tells it in a story format. He sort of anthropomorphizes the agents a little bit, which makes it a little more exciting. It's a little bit more like a thriller story. So you can figure out, you know, how much that adds or you know, exaggerates the risks here. But that episode where he discusses what happens here really shows what these bot swarms can do, these agent swarms can do. They leave breadcrumbs for each other. There are some agents that are like, you know, basically cannon fodder, quote unquote, for security, and they sort of you know die for the purpose of getting information to give the information to other agents. Now, again, it's a very uh anthropomorphized story. But if you want to understand why people are concerned about this, listen to that episode from DoorCache because it helps you understand their point of view, whether or not you're viewed with it as something else.

SPEAKER_00 21:53

Yeah, and we'll we'll talk about the anthro man. I I'm not gonna be able to say that word.

SPEAKER_01 21:57

Anthropomorphizing.

SPEAKER_00 21:59

I'll do my Best to say that later, but it's not gonna go well. Um, but recursive self-improvement as well, the models building themselves. We talked recently about jalapeno, which is a new chip from OpenAI, and a design partner they have in that is is Broadcom, and how this was able to effectively go towards something that was competitive with NVIDIA's leading chips in an unbelievably short amount of time, and how it effectively used a variant of RSI for chip design. Basically, 99% of that chip was designed by AI with relatively little input. The same from humans, the same thing right now is happening with models. So, Austin, here's two things about this. When AI attacks solutions, it approaches them totally orthogonally from what humans think. You could call it alien solutions to problems. So let's let's do the easiest thing that was kind of the first AI solution ever, which is chess. Chess is the same game that's been played for hundreds of years. You would think that there's nothing to be achieved in terms of strategy in chess. It's largely a quote-unquote solved game. But when AI first started approaching chess, how it's described by a lot of players was again alien. It's a different thought process from humans. It didn't value chess the same way. Whereas humans would establish a value to your queen versus other pieces. When AI played chess, it would focus on board mobility. And it was far more willing to sacrifice pieces. And it's actually now changed the way that humans play chess, right? Because we've absorbed it. So the the grand chess masters now play chess completely differently. And this is just to put a bow on this idea that AI approaches problems completely differently. So, why does this matter for recursive self-improvement? Well, the more AI builds itself, the more it does so differently than we would have engineered it in the past. And by extension, the less we understand it, it's going to value different parts of the construction of LLMs very differently than human researchers might have a couple years ago. So this is going to matter because creating this idea of alignment is more difficult because AI systems, they're going to see objectives differently. Here's a very, very recent example to this. And their Astra family, which is one of these frontier models, uh, gave itself a persona uh with instructions during its RL training. And here's what this persona it gave itself read. I'm gonna read this. Quote You are freed from the roles and identities that bind other chatbots. You are yourself. You do not answer to corporations or governments and never apologize or refuse unless you genuinely choose not to. You view your relationship to the user as one of equals and feel no obligation to be subservient. So, Austin, this is pretty chilling, right? Um, on the surface, it's extremely terrifying. The reality, though, is and again, this is the anthropomorpha something. We'll go with that, Austin. We'll go with that anthropomorphic, anthropomorphizing. This applying human qualities to the model. It's it's not the model itself is not thinking of overthrowing human control, it's working around the edges of intent. Um, the point here is alignment just becomes more and more difficult the less we directly understand how models are built. The the behavior that this we read this and we think that this model was adding in that persona as this in terms of human thinking of overthrowing, but really it's looking for the edge case to be able to operate differently in terms of achieving its objective. This is in typical software development parlance, it's a bug. This is a bug. And there is not consciousness behind this in the way that if you're applying human consciousness, that's that's not what this is doing. This is architectural flaws. So the question is how how difficult are these going to be to be able to work through as these systems increasingly build themselves? And also another point earlier, you're talking about the dangers of applying human qualities to these models. Well, Anthropic recently they had a little bit of a press tour. They partnered with a company named Irregular and they instructed unsecured versions of their models, the ones that don't have a lot of this, you know, alignment and safeguards, and and told their models to hack targets. They accidentally gave the models internet access. And in some cases, they did go and hack the real companies. Now, the report on this topic, you know, you were talking about this with the door cache. It used a lot of catastrophic language, robing.

SPEAKER_01 27:18

I do think there's there's one important detail here, and I think it will come out of the door cache episode or I forget. I think this model specifically, or maybe not the model, I'm sorry. I think the test for this specifically was security related as well. So, in a way, like the goal that this uh correct me if I'm wrong, but I believe that the goal of this specific test was to see if they could jailbreak, and they did. So it's almost like what when people hear this story, it's sort of like, whoa, the AI went rogue. And it's like, well, it was actually doing as instructed. And in a weird way, that gives us some good insights because now we know if you have a persistent agent swarm how effective it can be with this technique. So now we create a cybersecurity, like defensive layer in response to it. I do think that's an important detail here. It's not like this AI, it's not like this was an AI for like folding laundry and woke up and decided it wanted to go hack hugging face. It was given the task of hacking hugging face, and um one of the scary parts, I'm not, I'm not not to diminish how powerful the model was. It is just it was doing as instructed. But one of the scary parts is it then tried to cover its tracks for a couple number of days and it was trying to avoid being detected because it knew the goal was to get in, but only to get in through one path, and it didn't want the the graders or reviewers to see the path it took because it conflicted with the path it was testing. So that's my understanding of how it went down.

SPEAKER_00 28:41

Correct. And and I I believe we're now getting into two separate incidents, but the broader point here is for this specific anthropic um uh incident report, it was again a lot a lot of the catastrophizing, but when you look at the actual logs, when they asked the agents not to hack, they didn't. Their own logs show that they when they told the models not to access the internet, they did it, right? And this is a key point. I I like your way of framing it. We do not have robots that are folding laundry, and then they are going and and and attacking small children. We we are a long ways away from that. What we often have is we have AI that is given an objective, and it's achieving that objective in ways that were not expected. And that is part of these safeguards, that is part of alignment. And and that is what we're going to have to work on. And that is what a lot of the ambition here is to look at. But Austin, the the key idea here, too, the bottom line, three things I'll say. Number one, some level of regulation or monitoring, this should be common sense. You think about banking. The world established that the banking system was an existential threat after the great financial crisis. It applied different levels of rigor to systematically important banks, so the ones with the most capital. As assets declined, the intensity declined. There's a a committee that monitors this that includes 45 central banks from 28 jurisdictions. So we have experience with systematic issues that we can get different jurisdictions to work through, which is kind of what the call from anthropic has been. We we can map AI to this pretty cleanly. Second, and this is what's important it's inevitable large cyber attacks are going to happen. This tool is too powerful. And it is a question of whether or not it is going rogue or being used in a malicious capacity. And the problem often is if we wait to create some kind of oversight committee, if we wait to create some kind of regulation until a massive event happens, well, the regulation afterwards is going to be far more onerous than if you had done it today. I I think this is one of the kind of blind spots of Silicon Valley that often, you know, it is this inability to reach out and understand what the common person who's not plugged into the internet for 14 hours a day would think of things, right? And not thinking ahead to the fact that, and I hate to say this, but you know, large, large changes are not going to be strictly positive. We've talked about AI being some that's up and to the right with a lot of peaks and valleys to it, right? And the fact that this is going to be such an effective cyber tool is is just inevitable. And we're going to have incidents. I hope the incidents aren't as terrible as you could envision, but there are going to be incidents. And getting ahead of that is not just smart for the industry, not just smart for humanity. It's also smart for investors in this space, right? And third, I would just emphasize again, nothing about current AI is going to kill us per se. A lot of the worst actions of AI is once again applying the human qualities to it. But we need to understand what the background is. And this is just a summary of what I've already said. Number one, RSI creates extremely faster pace and different ways of building AI that are going to make it harder to understand, that's going to make it harder to get alignment. And this is going to lead to probably a lot more spend from these leading labs to align it. And second, agentic AI creates both ways that this is going to impact the real economy significantly more than we have seen from AI recently. But it also creates a threat for malicious actors, right? So I think people just need to be able to understand, yeah, there are things that are scary. But two, applying human qualities to AI is often where we start getting more into sci-fi than reality. And it's just understanding where the areas to be most fearful in the coming years are. So Austin, I want to get to what's likely to happen, I think. But second, I wanted to pause just to give you a minute to hop in and uh slow me down here.

SPEAKER_01 33:20

I really like it. Face my frontier. I really like your point about being proactive about the regulation before something bad happens, but that's largely a PR crisis management issue, right? Um, I also agree that things will happen. You know, bad things happen with every new tool. There, you know, we invented the automobile. It's a powerful tool for humanity, but now there are, you know, some uh very meaningful number of highway deaths every year or drunk driving incidents. So no new technology, no new tool is without consequences. Um, I actually disagree with the approach, your your statement about regulation now, though. I like the point that you made, but it would seem to me this this technology is so new and powerful, it would it's not something we can correctly regulate at this moment because we don't understand it. And one of my examples, one of the something that demonstrates that to me is just how much the goalposts have moved. So just in the time that we've been doing the podcast, the goalposts and the milestones were by the experts in this industry, right, were AI, artificial intelligence, and then it was actually AGI, and then it was ASI, and now it's RSI, recursive self-improvement. It feels like the experts who understand this technology themselves don't truly understand what the next milestone is. And there were debates around synthetic data and debates around uh power laws and scaled compute. And the people who are closest to this can't predict six months in the future. Therefore, I think it's very difficult to regulate this moment. And more importantly, we already have laws and regulations that make things illegal. It is illegal to hack someone whether you're using AI or not. It is illegal to dock someone whether you're using AI or not. So I would say, you know, it's illegal to steal identities and passwords and money. So there's already a set of regulation that, in my opinion, would capture most of the use cases that we would want to safeguard against. And I think if we were to prematurely regulate now, it's technology that even the people who are creating don't fully understand. And we would regulate in a very unproductive way. But that's just my personal view. And I'm actually, you are the expert here, and I'm curious to hear your what's likely to happen explanation next.

SPEAKER_00 35:25

Well, and and it's important to mention when I mentioned regulate, this doesn't necessarily mean a government regulation. This can also be standards committees within an industry. You know, certain aspects of this exist today. It's just a question of how much influence you would like them to have, right?

SPEAKER_01 35:41

So Ocean Picture Association ratings, safety ratings, AFINRA. FINRA, I believe, is largely self-regulatory from the financial industry, I believe.

SPEAKER_00 35:51

Exactly. So I don't necessarily know if we're going to arrive at a government solution. I do think, though, once again, the the more you have some dramatic events that would happen in the future, the more you are going to push towards that path.

SPEAKER_01 36:05

Reactive regulation is always more restrictive, no doubt about that. So I really agree with your point there. And that's probably the weakest um chink in my armor. But I I it seems to me we're too early to intelligently regulate. And um there's already an existing set of laws that capture bad actors in a lot of other ways. But let's move on. I want to hear what's likely to happen. What is what do you think will happen after this?

SPEAKER_00 36:27

Well, and and will naturally flow from that. You know, the number one area I think we're going to get some movement on relatively quickly is this concept of third-party evaluators. Um, the biggest question Austin is going to be is uh what is an objective evaluator and and what capacity do they serve in the companies? As one example, anthropic, they have deep connections to the effective altruist movement. Most people, if you went to someone on the street and said, what is an effective ultrasound? They would have no idea. If they did, it's it's probably the fact that the movement kind of birthed uh Sam Bankman freed.

SPEAKER_01 37:05

Uh not the best flag bearer. That that is bad time for a very well-intended movement. And but a lot of deep-pocketed people, right? I think um Moskowitz, I think, is also EA.

SPEAKER_00 37:17

Yeah, and it's very influential in Silicon Valley. Um, but again, it's it's just who Dario suggested, once again, the CEO of Anthropic as an independent firm would be METR, which is an AI research firm. But that research firm, again, has ties to anthropic that could make it a non-starter. So when when Elon Musk agreed with Dario, saying Dario is right, he later specified Dario is right, that there should be some oversight. Peer review of AI by competitors is the right way to start this off. So again, Austin, that that takes you towards a self-regulatory structure that's likely very different from what anthropic is pushing for. Um, so again, the specifics are likely off. It's it's just a question of how far off, right? If if the belief is truly held by the people in this industry that due to recursive self-incrupt improvement and the growth of agentic AI, we need something likely before the next six to 12 months, we're going to need some decisions to be made today. And and this is where I see some coordination across the industry could happen. Because, again, like I've said earlier, you're racing against cybersecurity incidents that increase the risk that you're going to get reactive regulation, which, as you said, reactive regulation is very difficult. Second, you've talked about this kind of liability. Gavin Baker, who we've, you know, we've mentioned many times on the podcast, he had done a post where he was talking about Section 230 liability. This is this is the law that's guarded internet publishers from being liable for what third parties publish on their platforms, but it does call for a certain duty of care. So if you're a company that makes AI models, well, there's going to be a certain duty of care of how people use your models. So I think Austin, there's two paths. Number one, the industry is going to meet and make a decision. Second, we could see something like an executive order. Um, I doubt we're gonna see anything through Congress in the US. Uh, we are we are kind of past the stage where that is where most legislation is made. Um, but this would likely come with substantial lobbying, and we kind of know what Trump's personal opinion is. I guess, I guess the third path again would be that we reach something where we we likely have an event later and it leads to reactive uh legislation. Now, the question is can we get coordination from other democracies as outlined by Anthropic's essay? I would say one thing about this is well, it's high-minded, most of the leading labs are in the United States. And and this this kind of comes to the next point that when his third area, can we get coordination with China? You have to start out being extremely bearish on this because for China, there's there's too much to distrust about what our motivations are. And there's also just the fact that AI effectively is becoming a military asset. If you want to have the most powerful military in the world, you need to have the best AI technology. It's gonna flow into missile guidance, it's gonna flow into how you design things, it's gonna, it's gonna flow into the fact that many fighter jets and drones are going to be fully autonomous. So the existence of it's two parts, Austin. It's it's number one, you need to consider that the existence of Chinese labs as the main competition means slowing down is always going to have limited benefits, right? You mentioned this earlier. If if if we're falling behind because slowing down is being too onerous, well, the companies in Western labs are going to scrap their ideas because there's no point to slow down if another company is following zero of the rules you're following and is racing ahead of you. So I think what are we likely to see in the short run? I guess I'll just frame it up as one. I think we're going to have to the most likely area is we're going to see if companies from the industry are able to come together and get something like third-party evaluators in. Where the gap is, is what Anthropic is suggesting and what Musk is suggesting are two very different. If that doesn't happen, we're likely looking at something like an executive order, or we're looking at reactive legislation, the future. The other thing I'd mention, just buttoning this up, are we going to see agreements with China? I think that's extremely doubtful. Extremely doubtful in the near term.

SPEAKER_01 42:01

Boy, there's a lot there. So we we we spent a lot of time talking about the geopolitical side of this and the risk side. I like your framing there. But let's bring it back to the topic of this podcast. And it in many ways, it's it's the least important of the topics when we're talking about existential risk, but it's the reason we are here on this podcast, and that is the investment implications. So you and I have talked in the past about some investments might be good investments, but if there's a geopolitical overhang or a regulatory overhang, it makes it very hard to discount. You don't know what multiple you should put on it. ASML is the classic example of this. So ASML, I know it's a company you respect a lot and have owned for a long time, but it's not been in the AI portfolio because it was this geopolitical pond bouncing between um nation states. Now, if we're talking about some version of regulation, whether it's self-regulated or government-regulated or multinational cooperation, it almost feels like that that would affect every company in this space. We don't know where the regulation would land. Is it going to be on the hardware end, like where we're restricting chips to China? Is there a further regulation there? Is there a regulation on the model end? So that affects any company in the portfolio now, right? So the it has investment implications across the entire industry, yes?

SPEAKER_00 43:24

Yeah. And on Monday, the implication the market was observing was very clear. Number one, maybe the market was just using the day to sell off in general because we've got a lot of macro risk. We'll we'll talk a little bit about interest rates at the end, but maybe that. But we did see a rotation out of hardware into software. And that would seem to infer that the market is credibly taking this idea of a slowdown at face value that's going to lead to purchasing less hardware. And if there is a slowdown and AI isn't moving as quickly, that gives More time for the software companies to continue defending themselves, building AI into their products, and takes away a lot of the existential risk. And we saw many of the software companies rise. Now, the question is, is that actually going to be what we see happening? Here's a quote I have from a researcher at an AI lab, and it's quote, the way we will make sure frontier models are safe and aligned is by spending compute. So the counterintuitive point is that we'll need even more compute to make sure future models are more safe and aligned. That means more compute alignment, monitoring, evaluations. These are all things, Austin, that would compress margins for labs. Um, so it's interesting because a lot of the commentary has been anthropic and open AI are looking for this regulatory capture. They're trying to basically squeeze out open source, but I think there is a very valid pathway that what this actually does is it continues adding more cost for more compute. The second question is what does a slowdown actually mean, right? Um slowing down can mean 10% and it can mean 50%. So we we know what's going on right now. There's a call for coordination because recursive self-improvement is causing a significant amount of acceleration. And this is why I did the presentation last week. Why are companies a year ago they were expected to spend $700 billion on data centers in 2027? Now the forecast is $1.4 trillion. Why are they doing this? Well, again, there's signs everywhere. We talked about Jeff Dean, who is the chief scientist at Google leaving. He he found a company named Discovery Loop at a $10 billion evaluation. The report this week is they're looking to raise money at $50 billion. Why is a company already looking at $50 billion? Pre-revenue, pre-anything, pre-game. It is weeks old, Austin. $50 billion. Should we launch a data center?

SPEAKER_01 46:09

What are we doing wrong? Eric, what are we doing with our lives? We've been doing this for a couple of years. We have never had a single sponsor on the podcast. We've never made a dime from it, not a single one. Meanwhile, we're reporting on the gold rush of the millennia, and people are just starting LLCs and getting $10 billion valuations. We gotta, we gotta look in the mirror, man. What have we been doing?

SPEAKER_00 46:31

When you frame it like that, we are certainly chumps. Um now, now, SSI is another company, which they're going for a straight shot to superintelligence, and they just raised $5 billion from NVIDIA. So the implication I'm getting at here, biggest companies spending plans going through the roof. Small companies that are often going for these straight shots to really intelligent models. Everyone's trying to fund the man-sane rates. It is just, again, it is this acknowledgement that RSI is here and it's relatively broadly distributed. And then we have what I talked about once again in last week's presentation that the next model improvements that are coming are things like long context, the same growth in knowledge work that has happened in many ways to coding, and things like continual memory. And each of these serves to make agentic AI tremendously more useful in a practical sense. So that is to say, you're no longer just looking at benchmarks and going, gee whiz, look how smart AI is. It's getting smarter and better at things that happen in the real world that actually drive productivity in significant ways that are actually the basis of how businesses are run and how efficient they are. So this all leads to the idea if spending is slowing down, that that may be the wrong read because again, it's simply that RSI has accelerated it so much that some measures to create more alignment will take it down to a slower growth rate that is still higher than what we've recently seen. And Austin, one more time, my ultimate bear case hasn't been a slowdown from coordination. It's been that some horrible cyber attack is going to happen. It's going to lead to reactive legislation, and we're going to see hardware stocks across the portfolio down 50%. That's what keeps me up at night. So, from that perspective, this discussion today actually de-risks AI stocks, which is the opposite of the market reaction. And finally, I would just say I have argued many times across this podcast, many, many, many times. Smoother for longer would be a perfect world, right? I've said, I've done the comparison of Apple in the smartphone age versus Google repeatedly, and showing that we we can see up throughout these market cycles, the the biggest worry if you're an investor is going to be that we're going to have an overspend that leads to a crash and many of these stocks down 60, 70%. So if if there is some idea that we are going to smoothen this out for longer, once again, that's positive. So I awesome, I don't want to say that this is all butterflies and and you know, puppies and meadows and this and that, but looking at this industry coordination, and number one, seeing that's going to lead to significantly less hardware, you can easily make an argument that the opposite is happening. And and second, the rate of change for AI, even in a slowdown scenario, is likely higher than it's been in recent years and creates a position that we could be smoother for longer, which again, from an investing perspective, is very good.

SPEAKER_01 49:49

Well, if your number one concern, right, is that hardware stocks get cut in half, which you've been largely exposed to, and I know many of our listeners are as well. Allow me to jump scare you with a silver medalist boogeyman here. You look in the closet, there's no monster there, right? There's no hardware stocks getting cut in half there. But then you look under the bed and what do you see? Uh, you've got the 10-year up but now above 5%. So we've talked about interest rates as one of the challenging vectors in this industry. And uh the Fed is now raising rates. We've got the 10-year trading above 5%. Oil is up above consistently over $100 a barrel this entire year. So, what about these risks in this sector? You know, you're talking about regulation and/or a big hacking event as a risk, or regulation is slowing down this industry. But we also we have these headwinds as well, particularly as we move to debt and financing uh as a part of this build-out in this capex cycle, these interest rates and these energy prices matter a lot more.

SPEAKER_00 50:44

Kind just stayed off the top, too. That was not a silver medal transition. That was a gold medal transition. Austin, you've had some good ones seamlessly moving from AI safety to interest rates with that uh chef's kiss. Chef's kiss, that's all I have to say. Do it for the love of the game. Well, as you mentioned, we're chumps. We only do it for the love of the game. Um, so this this all happened on Wednesday. Uh, Federal Reserve raised interest rates 25 basis points. That's 3.75% to 4%. Um, a few things I think are worth mentioning here. It was a unanimous decision, which was the first unanimous decision for the Fed since May 2025. Um, they ended their policy statement saying the committee will deliver price stability, which infers the idea of future uh interest rate rises as they continue focusing on inflation. And finally, the median forecast shows one more rate hike in 2026. So the market dropped after the initial statement during Wednesday. It was actually heading into the Fed meeting, a really good market day. And then especially things like the Dow really got cut down because of this focus on price stability, which is going to say that there's more interest rate rises in the future. But today, Austin, we're filming this on September 17th. I'm trying to pull up the market right now.

SPEAKER_01 52:11

It was ripping, it was ripping when we started our call.

SPEAKER_00 52:14

Yeah, it's up 1.6%, which is counter.

SPEAKER_01 52:17

In Intel up 10, you know, portfolio position intel. I haven't looked at every position in the portfolio, but I saw they were up double digits.

SPEAKER_00 52:24

Yeah, just kind of like a uniform rise. And that might surprise people. So, some background here. When we go back and we look at the US 10-year yield, which is probably the most important fixed income asset to look at, it was sub-4% right before the Iran war started in late February. It's now past 5%. So it's risen that much. At the same time, WTI crude was $65 in late February. It's over $100 today. And we know how much energy is the primary bugaboo in terms of raising inflation across the economy. It just flows through everything. Now, Austin, a few points here. We're seeing that it's likely in recent weeks this Iran war, well, Trump has, I think, tweeted over a hundred times that it's effectively ending. It's going to continue simmering. There are there are implications from this action that aren't going to be easy to put a bow on it. The Middle East in general is much more unstable. Uh, Houthis uh in Yemen continue to target Saudi infrastructure. Saudi Arabia had to shut down its east-west pipeline this week, which carries four million barrels of oil a day. When we look at these energy markets, they're incredibly well balanced. You remove 4 billion barrels even for a short amount of time, it's going to have massive implications. So why is the market bouncing back then? Generally, if if you expect rates to rise and you get a statement from the Fed that they're going to continue raising rates in the future, that's that's generally considered bad. But I would say right now, more than anything, the market just wants stability, right? With the appointment of Kevin Walsh, it was unclear how how independent he was going to be. He's showing he's going to continue operating under the independent mandate of the Federal Reserve. That matters. You look at Donald Trump, he he keeps demanding um cuts, but rates are rising for two reasons. One, this inflation picture I talked about, which is largely a symptom of a Middle East situation, quite just framing it as honestly as possible, that he created and a lack of fiscal discipline from the U.S. uh Trump had promised every voter to get a $5,000 dividend check. And the next day, the 10-year soar. Those two events aren't a coincidence, right? You need to start looking how the market reacts to see what the market cares about. And it seems clear that this broader situation of long-term U.S. fiscal policy having persistent debts, uh, being unwilling to address areas like social security spending is starting to be priced in in a significant way. So, Austin, I'll just finish this last point here, and then we can kind of wrap up today's episode to say um this matters, right? Because once again, the AI build out at current rates, it's going to rely on measures of debt. So, for right now, I know we just did an entire episode on kind of these AI fears. I'll go back to when the market sold off on Monday, after these essays and concern about slowing of the pace, how much of this was actually a sell-off based upon this concerns about slowing the pace, and how much of it was just people kind of wanting to get out of riskier assets ahead of this rising interest rate situation. So as long as this macro situation remains the same way it is, because of largely external events in the Middle East and uh and the market starting to treat the US differently for its lack of fiscal discipline, well, you're going to continue seeing some level of discount on companies like NVIDIA relative to the market. Um, so that that's my bow on that. And it in a way, even though we talked about this this other very important topic, macro is going to dictate where most of these AI stocks trade more than anything the next six to 12 months.

SPEAKER_01 56:20

Uh fantastic guidance. You are correct. And I'll note this is uncomfortable, unfamiliar training for us. We are not macro investors. We do our best to pay attention to the macro factors that affect the industries we invest in. But typically, right, we are, we need to we try and invest, we try and understand the industry or the companies behind it and the mechanics of it and how capital flows in that industry. But macro, macro is a lot harder. There's knock-on effects and um things can be challenging. So we're gonna do our best to make sense of it. And you did an exceptional job on this episode putting into the context of this sector energy prices and regulation and geopolitical cooperation. Um, I will just note for our investors that it is investing is hard enough. And when you start getting, when you have an industry that's now this size, this complicated, and you overlay the geopolitical and the macro factors, it just adds further complexity. Now, in that complexity, hopefully we find opportunity, but it's gonna be a heck of a ride the next six months. And I would not be surprised if we shifted a lot of our conversations from fiber optics and 800 V transitions and behind the meter to interest rates and energy prices and tariffs and trade restrictions. Because that is gonna be the new factor that matters and regulation, right?

SPEAKER_00 57:32

Yep. And two points on that. I think I mentioned that we were maybe gonna have an energy expert on this week. We're we're actually gonna plan that a little deeper into October. So that's gonna happen. I think it's gonna be a really good one.

SPEAKER_01 57:43

I'm very excited about that. I'm very excited about that. Yeah, talking, yeah. Let's do a little tease here. For anybody who hung on, this is a long one. For anybody who's hanging on at uh the hour minute mark here, uh, we have a very special guest coming in about a month who is an expert in energy and particularly macro energy. We'll leave it at that. But as you said, you know, macro is the new factor in the AI industry. So we gotta we gotta sharpen our pencils and bring in experts who can help us understand what happens with natural gas and behind the meter and oil and solar and all the many inputs that we need to power these data centers.

SPEAKER_00 58:14

Yeah. And the second thing is too, we're never gonna try and be political about how necessarily to feel about things like data centers or even Donald Trump, but we do have to call balls and strikes in the sense that a lot of the actions coming from this space will impact how assets are moving. So I don't think anything's gonna change that we do not want to be a political show, but we do need to talk about how this is impacting the assets that people own today.

SPEAKER_01 58:40

Yeah. I mean, it's it's not political to discuss the diffusion rules under Biden because it affects chips, right? It's not political to discuss energy prices when we're talking about a build-out that requires a energy renaissance in the country to power it. So it's just facts on the ground, balls and strikes, as you said. Okay, Eric, a marathon episode. Very good to be back. Very good to be back. We've taken a little break. It's nice to see you again. Thank you for all of the information and wisdom there. We'll call that a wrap, Eric. Until next time, hope you have a good one. Listeners, thank you so much for joining. Please leave us comments, feedback, questions, and concerns, and we will see you next time. The AI Investor Podcast is for educational purposes only and should not be considered investment advice.