AI is clearly not a single initiative. Enterprises view it as an implementation of their familiar software component model of application-building. The four AI giants clearly view it as a kind of back-door way to move everything to the cloud, when simply claiming that would happen has obviously failed. The question is whether some model of AI can bridge these views. Cloud computing has not replaced self-hosting, so we should not expect all AI agents to be self-hosted. Cloud and self-hosting have played a symbiotic role in the pre-AI role, so we have a “compute” model that facilitates that. Do we need one for AI, and if so, what would it look like? To get to the answers, we need to stop trying to fit everything into one model and go back to basics.
The market always gets it right, though “right” is in the eyes of the beholder. With AI, as with other over-hyped tech developments of the past, a surge of overstatements and flat deception often covers important truths, and in tech at least these truths tend to emerge because only demonstrable benefits can drive persistent sales. So it may be with what could be the most important development of this decade, the AI agent concept.
The combination of AI hype and “AI-washing” has always made it difficult to align stories on the topic with a consistent position, much less with enterprises’ own views. The most realistic position is that enterprises tend to see AI agents as software components, while the broad media/market view (encouraged, of course, by AI model and chip players and the cloud giants) tends to cast them as cloud tools of some sort. Because software components need a business case, and because business cases are easiest to make when the data in use is a company’s own core data, subject to governance, enterprises have effectively focused on self-hosting agents.
To differentiate “agentic” AI from AI overall, the media/market trend has been to focus on autonomy. An agent does something specific, on its own. It’s not a virtual person that can be asked questions, but rather a tool that acts on a command. While this is surely different from chatbot-like AI, it’s actually converging a bit on the enterprise view, to the point where it’s possible to see both agreement and more meaningful differences.
Software components do things. Agents do things. You can argue that software components behave autonomously, since they act on units of work based on their design and on any collateral data that they digest. Thus, the market view that agentic AI’s fundamental property is at least somewhat consistent, so dare we hope for some useful convergence?
Is autonomy really the fundamental property of “market AI?” I submit it is not, because the promotion of AI agents in the marketplace, in the media, is almost exclusively focused on the cloud-hosted form. Not a surprise given that the vast majority of AI being consumed these days is hosted by one of the AI giants. What AI is consumed that isn’t cloud-hosted is largely what enterprises have considered as “embedded AI”, built into software or devices to do something that the user may not even be aware of. If you have a recent Pixel smartphone you are using device-hosted AI in a number of things, from explicit (via Gemini integration) to embedded (in Google Camera). Even enterprise use of AI today is largely focused on the cloud, because enterprise deployment of AI is in its infancy, and what is deployed is often (as it is for consumer AI) embedded in something.
In cloud or in-house? That’s the real question, the real distinction. You obviously could host AI in-house, just as you host software components there, and likely for the same reasons, data security and cost. Cloud computing is not suitable for all IT in the minds of virtually every enterprise, partly because of data governance issues but also because most highly used enterprise applications are cheaper to run in the data center. Only things that are highly bursty in utilization are likely to benefit significantly from cloud economies. So, if AI agents are software components, when do the agent-components meet cloud requirements? Does any bursty agent belong in the cloud, if governance could be resolved? Those are now the critical questions for cloud AI and all the current AI giants.
Enterprises, focusing on data governance, have less a problem with cloud-hosted agents that don’t use governed data. That tends to focus agents on missions that aren’t company-specific, though, which means that what enterprises think is the most compelling business cases are off the table. However, they do recognize several types of “generalist agents” that could be useful.
One that over two-thirds of enterprises mention is content analysis, where “content” means things like video, audio, even text. Many copilot-type AI applications already nose at the edges of this mission set, but video is the specific one enterprises think they might like. Think of analyzing security feeds, of creating a record of events and interviews, and you get the idea. It’s clear that this sort of thing could, in theory, also end up exposing confidential information, but so could phone calls and public meetings, so it’s a risk companies can manage.
Another mission that gets nearly as much support is “market analysis”, which means using economic and demographic information that’s public rather than company-governed. I’ve used this kind of data in my own market modeling, and tried it with AI as well, but the traditional chat form of AI doesn’t seem to spread as wide a net to capture useful information as I could do myself. This issue is recognized by many enterprises too, so it may be why this particular agent mission gets support.
The third mission, which is relevant to roughly half the enterprises, is what you could call “operations analysis”. Think of this as being culling operations data from networks, hosting, and even from industrial processes that have real-time sensors, and then making suggestions or (perhaps, with human permission) taking actions. This seems to me to be the mission that has the most intrinsic interest, but there is a clearer risk of governance/security issues with it that’s making enterprises antsy. There is, however, much more interest in having this kind of agent follow the embedded approach, meaning that it would be integrated with an operations management application that could set guardrails on what might get exposed.
Ah, governance; the same issue that impacts cloud service adoption. There is some indication in the attitude of enterprises toward the missions above that there’s company data and there’s governed company data, and that even that last category has a bit of elasticity in terms of cloud-hosted AI. Some providers are more trusted than others (Google currently gets that nod) but none are fully trusted. Could AI itself provide governance? IBM is promoting an AI agent framework aimed at that, but primarily for in-house applications. Might it be useful for cloud AI? Enterprises aren’t saying at this point, but they’re not ruling out some sort of AI governance fence, if they believed it trustworthy.
To me, that suggests that there might be broader interest in cloud-hosted agents if there were a locally hosted front-end agent through which requests were actually made. If that’s true, then the value of cloud AI might depend on a generalized agent, self-hosted, that could enforce data governance on external, secondary, AI agent access. This could look, structurally, somewhat like what cloud providers now offer enterprises for edge computing, a middleware kit that acts as a front-end to the cloud and handles, locally, things that can’t tolerate cloud latency. This time, it would be things that can’t meet company governance rules.
In all, this could be a hopeful sign for the cloud giants and their massive AI investments, since it could open up more agent opportunity to them. However, they could mess this up. What they’d love to see is a set of distributed local agents that were tightly coupled to their cloud models, which would mean that there would have to be some form of cloud-provided-and-trusted governance between the distributed pieces of the model and the cloud. If enterprises were willing to trust cloud governance enough for that, they wouldn’t have data governance concerns about the cloud overall. That’s why we need an event-driven link, so that the data that’s used by these local models can be isolated. The giants should keep this in mind; sometimes the easy-greedy path isn’t the best one to take.
