Nvidia May Have Won the AI Training Race, but the Bigger Opportunity Is Likely Still Ahead

Nvidia (NASDAQ: NVDA) has already won one of the biggest races in the technology sector.

Its graphics processing units (GPUs) provide the key computing power for the majority of the servers used to train today's most advanced artificial intelligence models. But here's the part of the company's opportunity that investors may be underestimating: Training AI is only the beginning.

Once an AI model has been trained, it can be deployed so that people can actually use it. And every time an AI is tasked with answering a question, writing computer code, generating an image, searching for information, or completing a task, the computing infrastructure built around Nvidia's hardware gets called upon again.

That's inference, and it may become Nvidia's next enormous opportunity.

Image source: Getty Images.

Think of training as a school

The easiest way to understand the difference is to think about a student. Training is like going to school. The student spends enormous amounts of time learning. Inference is what happens after graduation -- the now-educated individual actually using what they learned to do a job.

AI works similarly. Companies spend huge amounts of time and computing power training models to understand language, images, code, and other information. Nvidia's GPUs have been the workhorses providing the parallel processing power behind much of that training.

But once the model is ready, the need for large quantities of computing power doesn't end. In fact, it may just be getting started. Imagine a company spending billions to build an incredibly sophisticated AI model. It doesn't build that model just to sit unused in a data center. It wants paying customers to use it.

Every use creates another computing workload. And unlike training, those inference workloads can happen millions or billions of times.

Training creates the AI. Inference puts the AI to work.

If systems that use AI become a common part of everyday life and business, the volume of inference workloads they generate could become enormous. And the expected wide use of AI agents could make this opportunity even bigger.

Today's chatbot might answer a question it's asked -- a single response to a single query. Tomorrow's AI agents could do much more complicated things. Suppose you ask an AI agent to research a company. It could take that one request, and respond to it by searching through multiple financial databases, reading corporate filings, looking for recent news, comparing it to competitors, building a valuation model, checking its calculations, and then writing a report.

That's not one AI response. It's an agent undertaking a chain of reasoning and actions. The more steps an AI agent takes in its effort to fulfill a request, the more computing power it requires.

Nvidia is actively positioning itself to benefit from this trend. For instance, its newest Vera Rubin architecture, which combines the chipmakers GPUs with its new CPUs, is specifically designed for these long-running inference workloads.

Don't assume Nvidia wins this market automatically

While the opportunity for inference will likely be larger than the market for training, the competition among chipmakers in this space is likely to be intense as well.

On one hand, hyperscalers like Amazon and Alphabet are developing their own custom AI chips, partly to reduce their reliance on Nvidia. Besides, the inference market is particularly sensitive to cost, as customers will have both incentives to switch and opportunities to do so if a competitor can deliver similar results at a meaningfully lower cost.

So the investment thesis isn't simply that inference will grow, therefore Nvidia will win. The more important question is: How much market share in this growing segment of AI can Nvidia capture?

So far, the company is betting heavily that its combination of hardware, networking, software, and a robust developer ecosystem will keep it at the center of the AI infrastructure stack. That said, investors should pay close attention to how well the company sustains its market share in the coming years.

What does it mean for investors?

The first phase of the AI revolution was about building increasingly powerful AI models. Nvidia was by far the most important supplier of the computing power needed to build them.

The next phase is about putting those models to work. If AI agents become widespread, they will process billions of increasingly complex tasks on our behalf. That means Nvidia's opportunity will not end when an AI model finishes training. It will begin there.

That's why, despite the fact that Nvidia has already delivered massive returns to its shareholders since the start of the AI race, the bulls believe the stock still has more excellent performances ahead of it.

Lawrence Nga has no position in any of the stocks mentioned. The Motley Fool has positions in and recommends Alphabet, Amazon, and Nvidia. The Motley Fool has a disclosure policy.