In this PWC study, 59% of leaders said they will invest in new technologies, and 46% say they will invest in generative AI specifically in the next 12 to 18 months. The most significant hurdle is adequate cloud bandwidth/computing power to accommodate usage and enable scalability. That means coming to terms with how much money can be spent on new generative AI systems and generative AI enablement.
Generative AI is hot. Try reading any tech or business article these days without finding a mention. However, the computing and infrastructure costs of running generative AI models in the cloud are a barrier for many businesses. Even with today’s cheaper pay-as-you-go models, it is expensive to run generative AI models in the cloud, not to mention storing and retrieving the training data and using other massive computing and storage systems.
You get what you pay for
In the world of generative AI costs, you really do get what you pay for. Those who leverage specialized processors, such as GPU, will have to pay the current freight, which is more expensive than traditional system resources. However, it’s needed to make generative AI systems function in optimized ways.
There are dozens of .ai startups that just provide GPUs and other purpose-built processors on demand. These “microclouds” have yet to appear in the numbers where we need to pay attention to them. However, they are going to be another on-demand option beyond just the major public cloud providers, which dominate the generative AI game currently.
Now that we live in the multicloud world, adding other clouds that just provide generative AI processing and storage isn’t that much of a stretch. We’re already dealing with complexity and heterogeneity; if there is a benefit of these purpose-built AI-supporting microclouds, we’ll go there quickly. New shiny object.
There are no half-measures to get to a successful generative AI deployment unless you spend the money on the optimized solution. As I’m building this architecture now, I can tell you, no one is going to get this for cheap, which is what enterprises want. There is no getting around the fact that it’s going to be pricey, and most enterprises don’t have money lying around for this specific purpose.
We’ve seen this movie before
As I mention every chance I get, I was an AI developer and designer right out of college back in the 1980s—not that the technology then compares to today’s advancements in next-generation generative AI, machine learning, and deep learning. It’s not even close.
However, the cost issue is the same. Back then, building and deploying AI-based systems took millions in hardware and data center space. We also needed unique, high-performance systems—supercomputers—many of which were provided as a service to share the high cost between organizations. (I worked for a company that did that.)
Indeed, AI surged but then declined, mainly blamed on the need for purposeful business use cases, but also because it was too expensive. A few deployments and AI companies still existed, but AI was largely placed on the back burner due to the price tag.
Learning from the past
Some of those past mistakes are still occurring. Businesses are falling in love with the technology and the capabilities without asking the key questions: What is AI’s purpose and how can it return value to the business? As the study pointed out, I see many generative AI projects pushing forward through sheer will without a clear benefit to the business.
As a rule of thumb, generative AI systems cost three to four times more than systems that don’t use generative AI. This includes development and deployment, but the actual expense is for the infrastructure resources needed to support generative AI operations. It will take specialized computing and massive storage to keep them working up to the point where they return business value.
Yes, you can take half-measures, but I would not bother. Those who attempt to do generative AI on the cheap will waste money.
What can be learned from the past is that any technology has value, and it’s a matter of understanding the value before making the investments. Direct your spending in priority order to the specific use cases that will likely return the most value to the business. Yes, the answer is that boring.
I suspect that someday we’ll be talking about what caused the tremendous generative AI hangover of 2025. Hopefully, you’ll look back on this post to appreciate the warning. Let’s try not to make the same mistakes twice in one century, shall we?
Copyright © 2023 IDG Communications, Inc.
