Chips matter. In fact, they likely matter more than anything else in hardware, and they may even matter more than software right now. At the very least, chips stand at the door of any major expansion in the scope of tech deployment, and that’s the door that has admitted all the past, legitimate, booms in tech spending and usage.
I remember when computers didn’t use chips; a 16 Kbyte machine that was slower and less powerful than a smartwatch today was three feet wide, five feet high, and seven feet long. The cost would have been more than almost any home at the time. Chip-based systems are why we have personal computers, smartphones, and even AI. They shrunk computing down radically, not only in size but in power consumption, cooling requirements, and cost. Chips were behind the computer revolution. What’s next?
I think that we can expect to see continued improvements in the sort of chips that make up personal computers, smartphones, and wearable technology, but I think the focus of the advances is going to shift, away from the general-purpose model and more to the system-on-a-chip (SoC) and to becoming specialized to the mission. This means that the chip wars will focus increasingly on AI and quantum computing.
You can rightfully wonder why that is, and the answer is simple. General-purpose computing is not infinitely distributable; once you’ve let people carry a smartphone you’ve hit a point where the value of shrinking the device hits the utility. I have a smartwatch, and while it can be (and sometimes is) a useful adjunct to my phone, most of its utility comes from non-traditional compute missions. Chips are important, in a financial impact sense, because they facilitated compute distributability, which means that underneath it all, it was the ability to distribute computing that mattered. Further distribution of computing will be about embedding it in special-mission devices, like wearables and IoT elements. For that, they’ll need to support the mission of the thing they’re embedded in, and that will mean stepping beyond general-purpose compute into things like AI/ML and quantum computing.
We are in the mainframe era of AI. The top-line Nvidia chips (the B200 or H200) runs more than I paid for my car and requires racking and cooling, making them minimally distributable. We are going to see the same trend in AI that we saw in computing; chips will shrink the size of an AI agent host to the point where we can deploy it closer to the processes and people it’s supporting. At some point, we’ll be able to embed it in things, first large expensive things like machinery and vehicles, and eventually in cameras, glasses, and sensors. If you want to see an AI revolution, this is the way it will develop, not through more and more B/H200 chip installations. Sorry, Nvidia.
The same thing is true of quantum computing, except that it is arguably in the pre-mainframe stage of evolution. The earliest computers were so large and expensive (they were based on vacuum tubes) that enterprises really couldn’t afford them; they were research tools. When I went to the University of Pennsylvania, the floors in the Moore School building where the first general-purpose programmable computer (ENIAC) was housed were permanently warped by the weight (30 tons) and heat that 18,000 vacuum tubes created. Twenty years later, we had IBM System 360 mainframes a hundred times as powerful for five percent of the cost, and twenty years after that the IBM PC that could match the bottom-end 360 in power for less than two thousand dollars. I don’t think that the realization of quantum computing will take that long, but there’s no doubt that it won’t come overnight. When it does come, it will be quantum chips that bring it.
Enterprises who follow the leading edge of these sorts of thing say that the driving force behind “real” AI and quantum computing is the need to distribute intelligence to the things we do and use, at work or otherwise. They’ve always seen AI agents, for example, as pieces of AI technology that can interact with business operations and workers’ activity directly. This, to them, leads things toward real-world, real-time AI missions. One enterprise AI expert told me “If you put AI in a hyperscaler data center, you deploy maybe a hundred thousand units. If you put it in a sensor, you could deploy a hundred billion units.” The idea behind that, the justification, is that once you believe in the AI agent, you’ll want to embed its intelligence in stuff to make the stuff self-smart. An autonomous vehicle that’s run by a data center will never be truly safe; one that’s smart in itself can be as good or better than a human-operated vehicle. And, of course there are almost two billion vehicles on the road today worldwide, and another ten million industrial/construction vehicles in factories, warehouses, and job sites. That’s a big market, with a big economic impact.
People believed in “time-sharing” computers in the late 1960s and early 1970s, but they were quickly devalued by minicomputer and personal computer advances. Distributable always wins. What can’t be distributed is almost surely doomed not to be revolutionary; it takes massive deployment to make a revolution, so this is what we need to be looking for in both AI and in quantum computing. Everything else is simply a side-show at best, and pure hype and nonsense at worst. Today’s AI is yesterday’s time-sharing, doomed to be overtaken by chips. Same with quantum computing, but in that area, we’ve not yet launched the presumption of a revolution that might actually be delaying an actual one.
To understand why, we have to address two questions, “What chips?” and “From whom?” Both are difficult at this point. Right now, what enterprises are saying is that embedded, distributed, AI is likely to look a lot like machine learning or a pre-trained small-language model; edge AI goals are specialized to the mission that justified the distribution in the first place, the thing that created a need for a local response to an event. However, there is a value to creating a more generalized chip; manufacturing and distribution economies would be better, and it would be more capable of being repurposed, perhaps lending to building a durable business case.
On the “who” side, enterprises are mixed with regard to their views on Nvidia’s role. Some think they’d be the obvious ones to bring out distributed chips, given that they’ve been active in promoting “world model” deployments for AI. Others think that they’re too fixated on selling to hyperscalers, which dictate big expensive chips of the type that makes up their primary revenue source today. Certainly the Street would be concerned about any visible shift in direction.
There’s a chance nobody does this, of course. The problem I see is that we’ve not been talking about what enterprises consider “realistic AI business cases”, most of which would lead to real-time missions and distributed AI. Enterprises are not progressing with their self-hosted AI nearly as fast as they thought they would; almost two-thirds who expected to deploy “significant” AI internally this year now say they won’t meet that goal. The reason most offered is that senior management tends to believe the popular cloud-hosted-expensed-AI approach, making it hard to drive self-hosted projects. This creates a kind of negative feedback; lacking movement toward extensive self-hosting, cloud AI gets all the attention, which then makes it harder to do self-hosting projects. You get the picture.
This may be why the Street is so antsy about massive AI contributions to profits. SK Hynix stock was hammered even though it had a major profit increase, because the increase depended on AI. We might see meaningful progress toward self-hosting delayed until 2028, and I don’t think that the current boom can be sustained that long. Maybe the Street is starting to agree, and what that will mean for AI overall is something we’ll have to wait to determine.
