According to a recent article in a venture-capital-facing publication, cost and model complexity remain barriers to adopting AI. They cite an IBM study, and IBM is the real leader in the realistic-AI space. The article summarizes “key findings” in the report, and I’ll look at each to see how they compare to what I’ve been hearing from enterprises. Then, we’ll see whether the article and the study speak to the market, or even speak the same language.
Model specialization: The study debunks the myth of a universal AI model, emphasizing the need for task-specific model selection. This is totally consistent with what I’m hearing. What enterprises want is not the popular model of generative AI, trained on the Internet and “knowing everything”, meaning it’s a mile wide and an inch deep, but a specialist. There doesn’t seem to be a reason for enterprises to want to create their own specialist models by tuning a general model; none of the enterprises who have offered me AI comments thought that was an optimal approach. However, about ten percent did like the idea of getting a series of specialized AI models based on a common open LLM.
Model diversity: Organizations currently use an average of 11 different AI models and project a 50% increase within three years. I agree with this too, if we interpret it to mean that there would be 11 different specialized AI models. If we assume we mean total different numbers of LLMs, I think that’s high, perhaps double what are actually used, and I don’t think enterprises believe they’ll be adopting 50% more LLMs in three years.
Cost barriers: 63% of executives cite model cost as the primary obstacle to generative AI adoption. Here I sort-of-disagree. What enterprises tell me is that the big barrier to any AI is the business case, and that cost of the model itself is rarely the biggest cost that the business case has to justify. Of course, if business cases for AI are difficult to make, then adding a license/subscription fee for the model is a problem, but any tech business case can be made if benefits are high enough.
Model complexity: 58% cited model complexity as a top concern. I think this point is difficult to assess. Yes, developing a model is very complex, meaning that model design and training require specific skills that enterprises rarely have in house. Yes, more complex models require more hosting resources, which is a big cost barrier to adoption. However, the statistic suggests that it’s a key issue to more than half of enterprises, and I didn’t find that to be true.
Optimization techniques: Fine-tuning and prompt engineering can improve model accuracy by 25%, yet only 42% of executives consistently employ these methods. This point is interesting because it illustrates something I think is important. IBM’s study, IMHO, is really aimed at saying that generative AI isn’t the right way to do AI. The Venture article misses that, tending to use “AI” and “generative AI” interchangeably. We’ll get to this below. Generative AI in its chatbot-as-a-service form is indeed very sensitive to the structure of the “prompts”, which is AI-speak for the questions or queries you enter. The fact is that all this fine tuning and prompt engineering is out of reach for the chatbot AI users, because they’re non-technical. The fact that they’re not doing the stuff needed to optimize these chatbots is proof to me that those chatbots can’t work as they’re being used, which I think is IBM’s point. The 42% statistic is, in the IBM study, under the heading “Gen AI advantage is fleeting”.
Open model growth: Enterprises expect to increase their adoption of open models by 63% over the next three years, outpacing other model types. This is probably true, but not really a useful statistic in the way it’s presented. I don’t know of any enterprise who is actually planning to deploy a licensed AI model in-house, and I don’t know any enterprise who thinks that they can really make a case for anything except in-house-hosted AI in the long run.
Authors (and yes, I am one) and publications tend to see what they want to see. Generative AI, properly, is the application of LLM technology trained on a large data system to a simple user query. IBM’s study is saying that this may be populist/popular, accessible, and newsworthy, but it’s not moving the ball in terms of making company operations better. That’s exactly what enterprises have been telling me and what I believe to be true based on my own experience. While the article draws on the IBM study, it doesn’t pick up the sense of the study, which is that if enterprises are to transform themselves with AI, chatbot copilot technology used by most “AI users” isn’t going to be how they do it.
Here’s the opening to the IBM study: “ChatGPT made everyone feel like an AI expert. But its simplicity is deceptive. It masks the complexity of the generative AI landscape that CEOs must consider when building their AI model portfolio.” How true this is!
It’s not all beer and roses for me, regarding the IBM study. I personally think that the term “generative AI” should not be used to describe anything except LLMs, not because that usage is inappropriate in a technical sense as much as because common usage says that “generative AI” is one of the as-a-service chatbot or copilot applications.
Here’s what I think IBM is saying.
First, the kind of AI most of us see, think about, and use isn’t going to move the company transformation needle. Helping people write better emails may make the people feel better, but it won’t make the company do better. Transformation at the business level demands data and model policies tuned to the business, and often to the task within the business.
Second, the right kind of AI will almost always be self-hosted. Companies don’t want to trust their core data to the cloud, so why would they want to trust it to a cloud-hosted AI-as-a-service tool? Data sovereignty is critical here, and we already know it can’t be resolved satisfactorily anywhere but on-premises.
Third, companies want AI to be an expert and not a generalist, so a specialty model is the right approach. If you want legal advice, get a lawyer. If you want your books done, get an accountant. If you want your network managed, get a netops specialist. Same for AI. A company would like an AI model built for and trained on an expert mission.
Forth, the scope of the model and the hosting required to train and run it are set by that expert mission. Some models will require an LLM, some a small-language model, and some no real language model at all, simply ML or a simple neural network. Some LLMs with a lot of rules and data will require hundreds of GPUs to train and run, some can be run on a PC. Smaller models could even be run on a phone, as we already see. So you get an expert model that suits your needs and can run on something that you can make a business case to justify. To quote the IBM study, “Give some teams the sledgehammer—and others the scalpel.”
Finally, we are a long way from having enterprise skills needed to build their own expert models, but not from skills needed to tune one. The real value of AI will be unlocked when we stop looking at AI models as a kind of personal friend, and look at them instead as what they can be. We train in-house council and accountants to business policies and needs, and that’s what we should do with AI.
Read the study, not the article.
