I\u2019ve noticed a major vibe shift in the AI discourse lately. Maybe you have, too.<\/p>\n\n\n\n
Two years ago, the air was thick with worry about the potential ramifications of AI getting better, and not just a lot better, but even a little bit better. The fear was that, with even a small improvement, AI could potentially make itself even better, faster, which could then trigger a cascade of recursive self-improvement leading to a runaway intelligence explosion.\u00a0<\/p>\n\n\n\n
This idea led to high-level fears that widespread unemployment was imminent, with entire classes of jobs getting wiped out, as well as existential anxiety that we were bringing a superintelligence into our midst with no understanding of how to control it \u2014 and that it could potentially be humanity\u2019s downfall. <\/p>\n\n\n\n
This concern \u2014 that AI could cause humanity\u2019s extinction \u2014 was quite widespread. It was expressed by top AI labs (OpenAI, Anthropic, Google) and many of their employees, as well as a number of top AI researchers (Geoffrey Hinton, Yoshua Bengio). Major media outlets, including Time<\/a> and the New York Times<\/a>, published stories on the topic.<\/p>\n\n\n\n Policymakers were urged to do something<\/em> to mitigate the risk immediately, and multiple nations took legislative action and established AI governance frameworks. Some set limits on the amount of compute that could be used to train AI models, arguing that more compute could lead to models that were too powerful and, therefore, too risky. In some places, politicians mulled creating oversight regulatory bodies for AI similar to the ones that exist for financial services or healthcare companies \u2014 their job would be to audit and test models and ensure labs reported regularly on their development.<\/p>\n\n\n\n Due in part to changes in the political realm, much of this regulatory authority didn\u2019t come to pass. However, AI did continue to get better at a blistering pace. Models<\/a> have trained on more compute than the supposed threshold of danger \u2014 1026<\/sup> FLOPs \u2014 including open-source ones that anyone can download. AI can now code autonomously and solve hard problems across physics, maths, and coding. What we thought of as the dangerous threshold for the handful of largest models is now well within the grasp of everyone.<\/p>\n\n\n\n We make tech safe as we build it. If we didn\u2019t, nobody would use it.<\/p><\/blockquote><\/figure>\n\n\n\n Not everyone has been in agreement about the risk of AI, though. <\/p>\n\n\n\n Others, including myself, have argued that this line of thinking \u2014 that AI could be an existential threat \u2014 was fanciful and not backed by evidence. This group\u2019s position has been that better technology has, traditionally, been beneficial to humanity and that diffusion isn\u2019t simple and would take time \u2014 even if AI spread much more quickly than other technologies, there would still be enough time for society to adjust. <\/p>\n\n\n\n They came to this conclusion on three grounds: <\/p>\n\n\n\n I called the insistence that AI poses an existential risk an example of modern eschatology<\/a> once \u2014 a secular version of old apocalypse myths from theology. Its believers sought proof of AI\u2019s safety before<\/em> its creation. <\/p>\n\n\n\n The counterargument to it is based on engineering. It’s built on the premise that better technology isn\u2019t built in a vacuum \u2014 during creation, it\u2019s constantly running up against the friction of the real world. <\/p>\n\n\n\n It argues for an engineering approach to AI safety. We make tech safe as we build it. If we didn\u2019t, nobody would use it. It doesn’t matter how fast you can make a car \u2014 nobody would drive it if it blew up every so often or turned right when you wanted it to go left. If there are problems with AI, as there are with every tech, we\u2019ll solve them \u2014 we\u2019ll have no choice, if we want to make progress. <\/p>\n\n\n\n Economics supports the counterargument, too. The real constraints are real. S curves are real. When your next data center needs an order of magnitude more investment for linear gains in performance, that has real cost associated with it. It doesn’t mean the performance gain isn’t real, but just because we get wonderful benefits doesn’t mean the cost disappears.<\/p>\n\n\n\n I\u2019m now sensing a collective waking up amongst the AI doomers.<\/p>\n\n\n\n It seems to be largely triggered by an emerging general consensus that GPT-5 wasn’t all that great, that the scaling curves bent (I think it’s amazing). It might also have been brought about by the lack of ill effects over the past year despite the seemingly constant release of new closed- and open-source models. We\u2019ve also seen enterprises express dissatisfaction with AI \u2014 to them, it\u2019s progressing all too slowly.\u00a0<\/p>\n\n\n\n Even the worries about chatbot psychosis<\/a> seem overblown \u2014 the phenomenon is far from widespread despite the release of incredibly sycophantic and reward-hacking code models in short succession. No worldwide Pentagon hacks as models trained with greater than 1025<\/sup> FLOPS were released<\/a> \u2014 many with very limited safety training as the US circa 2023 would\u2019ve understood it \u2014 by ByteDance, Zhipu, Alibaba, Tencent, and DeepSeek<\/a>. <\/p>\n\n\n\n Instead, what we saw was how we’re now a year or two into companies adopting generative AI in full force, or trying to, and hitting the same organizational inertia that every other digital transformation effort hit before. Better models will help \u2014 they already are \u2014 but AI is not replacing all jobs. It\u2019s even beginning to feel like regular IT.<\/p>\n\n\n\n Ultimately, we\u2019re leaving behind the existential anxiety and returning to a more prosaic worry: Technological transformation will cause major shifts in the workforce \u2014 likely faster than ever before \u2014 and the world as we know it is about to change.<\/p>\n\n\n\n The core worry underlying AI has always been about the unknown. Could we control it? Could it control us? What does it mean for there to be <\/em>superintelligence? <\/p>\n\n\n\n And worries about the unknown aren\u2019t new. <\/p>\n\n\n\n In 1955, John von Neumann, a co-developer of the atomic bomb and widely considered one of the smartest men to have ever lived, penned an essay titled \u201cCan we survive technology?\u201d In it, he wrote<\/a> that \u201cfor the kind of explosiveness that man will be able to contrive by 1980, the globe is dangerously small, its political units dangerously unstable \u2026 Soon existing nations will be as unstable in war as a nation the size of Manhattan Island would have been in a contest fought with the weapons of 1900.\u201d<\/p>\n\n\n\n His argument in the piece is extremely logical, befitting a man of his intellect. It\u2019s almost tautological in its simplicity. Technology increases people\u2019s capabilities. Capabilities can be used for good or evil. Capabilities for evil, if sufficiently high, can be catastrophic. Applied to AI, it means that, as the technology gets more and more capable, we\u2019re moving toward a world in which everyone will have access to a technology that can complete their homework \u2014 or potentially hack the computers controlling an entire nation\u2019s energy grid. <\/p>\n\n\n\n It\u2019s the digital equivalent of a nuclear warhead.<\/p>\n\n\n\n Advanced technology doesn\u2019t inevitably translate into catastrophe.<\/p>\n<\/blockquote>\n\n\n\n Seventy years have passed since von Neumann wrote his essay. The world population is about three times larger. World GDP has grown roughly 20x nominally since 1955, and about 5x in real terms, meaning adjusted for inflation. Real GDP per capita has doubled. The power of technology helped create this surge. And yet, deaths from state-based armed conflicts have declined, even adjusting for the World Wars that happened in the first half of the century.<\/p>\n\n\n\n Don\u2019t get me wrong \u2014 the world isn\u2019t all rosy. Since von Neumann wrote his essay, there has been a rise in lone-actor terrorism, particularly inspired by online propaganda \u2014 less sophisticated than all our war, but also much harder to prevent. But the net impact of those incidents has been much smaller than what von Neumann predicted (he was so convinced that the Soviets would drop a nuclear bomb on the US once they figured out how to build one that he advocated bombing Moscow preemptively<\/a> in the late 1940s).<\/p>\n\n\n\n Why are we so much less likely to be hurt by acts of war despite technological advances \u2014 like nuclear weapons \u2014 making us so much more vulnerable? <\/p>\n\n\n\n There isn\u2019t an easy answer, but seemingly, as everyone gets richer, the conditions that produce the worst Hobbesian outcomes of human nature become less compelling. Yes, they do get replaced with copycat media-inspired killers, but they\u2019re much fewer than what you (or von Neumann) might have thought. Plus technology means those killers are also much much easier to catch. <\/p>\n\n\n\n\n
The vibe shift<\/h3>\n\n\n\n
Surviving technology<\/h3>\n\n\n\n
\n