Katie Dayton (3:03)

This behavior appears to have swept the tech world incredibly quickly. What technology or LLM models have come on the scene to make all of this possible?

Kate Clark (3:12)

Basically, the main models from OpenAI Anthropic, have just been getting better and better. Every few weeks or every few months, they release model updates that have made these other tools built on top just work so much better. Even just in the last couple of weeks, OpenAI has made adjustments, improvements to their model that have made Codex, their AI coding tool, even better. So it's just kind of like the natural evolution of this LLM technology. And there are many, many startups that are building tools on top of the LLMs that are powering these agents, and they're just catching on. They're just getting better by the day. There's more money being invested in them. It's just constant change.

Katie Dayton (3:50)

Now, I've watched enough Black Mirror to know that delegating all your grunt work to bots can have terrible repercussions. Did you hear of any instances where this form of software engineering has gone incredibly wrong?

Kate Clark (4:02)

For the most part, it's just they make mistakes and they mess everything up and you have to restart. You hear things about people's entire inboxes being deleted because of the stage we're at, which is really just people learning to use them, learning to see their capabilities. We really haven't gotten to the part where there's horror stories after horror stories for now. I think it's just people get irritated because they've got 1, 2, 3, 4 agents working on their behalf, and they're just not doing the things that they're asking them to do. And they feel like they're basically babysitting the AI bots.

Katie Dayton (4:30)

Right, like human management in a way.

Kate Clark (4:32)

Exactly like human management, which is also a funny part of all of this.

Katie Dayton (4:36)

Software engineering is a skill that a lot of these people probably spent a lot of time honing. And in the Bay Area in particular, your ability to engineer was almost a status signifier. Is there any sense to all of this that this skill has just been rendered obsolete? And if so, is there any sense of nostalgia or sadness wrapped up in that?

Kate Clark (4:54)

Definitely. I came across such a broad range from people being really upset at the idea that the skill they spent their life learning is no longer relevant. I also came across a lot of people that feel like it is still incredibly important to understand the building blocks in order to successfully manage the AI agents. But we're very quickly moving into an era where like you and me can just use natural English language to command these agents to create websites, to create products. And that as we get deeper into that chapter, there will be a lot more concerns about job loss for engineers. There already are signs of that. But you know, in a few more months, just because of how fast this is moving, there'll probably be a much, much bigger story to tell about the true impact on software engineers.

Katie Dayton (5:42)

That was Wall Street Journal reporter Kate Clark. Are you deploying AI assistants in your work? If you're a listener on Spotify, be sure to let us know in the comments. Coming up Demand for inference computing is ramping up. We break down what this means for Nvidia's business model. That's after the break.

Xfinity Tech Support (6:07)

Xfinity Tech Support I can't get Jurassic park back online without but no worries. Our Wi Fi Boosted mobile keeps you connected, connected in all kinds of places.

Xfinity Mobile Advertiser (6:16)

Cut your mobile bill in half with two lines of unlimited. Learn more@xfinity mobile.com Xfinity Imagine that. Watch Jurassic park with Xfinity restrictions apply. Xfinity Internet Required savings comparison based on promotional rate for two lines of Xfinity Mobile Unlimited versus two lines of T Mobile Experience more Verizon Unlimited plus and AT&T Starter as of February 10, 2026. Taxes fees extra

Katie Dayton (6:40)

this week sees the return of Nvidia gtc, the chipmakers mammoth AI conference held in San Jose. All eyes yesterday were on Jensen Huang, the leather jacketed CEO who addressed the rise of inference computing and what that means for the company going forward.

Xfinity Tech Support (6:58)

Finally, AI is able to do productive

Robbie Whelan (7:01)

work and therefore the inflection point of inference has arrived.

Katie Dayton (7:08)

We got the lay of the land from WSJ reporter Robbie Whelan, who spoke to us from a car on the highway while heading to the conference. Robbie, what are we expecting to hear from Nvidia?

Robbie Whelan (7:18)

Nvidia is getting ready to roll out a brand new chip using technology from a company called Groq, which Nvidia recently essentially acquired. Didn't fully acquire the company, but they paid $20 billion to license Groq's technology, which is chip design technology, and to hire Grok's top leadership. So basically it was what's called an aqua hire. And the reason why they did this was because Nvidia at the moment dominates the world of what's known as AI training. But they don't have quite as much of a foothold in what's known as AI inference. And AI inference means how to run models and make them respond to your user queries faster.

Katie Dayton (8:02)

What are the different requirements for inference vs. AI training chips?

Robbie Whelan (8:06)

Inference requires a lot more memory. So every computer chip that is in your iPhone or our laptops, or even in data centers, it's actually a combination of multiple chips. When we talk about a chip, we talk about a processor, which in Nvidia's case is called a GPU. And then there's other types of processors called CPUs, and then there's a whole bunch of chips that are known as memory chips. And those are the ones that allow the computer brain to kind of go back into its memory and access all of its training, access all the databases that have been pumped into it to teach it how to be an AI model. And so inference requires a lot more memory because you have these models that are working to remember everything that they've been trained. And training requires a lot more raw computing power. So if you think of training an AI model as just doing billions and billions of math problems over and over again, whereas inference, which means allowing a model to go out into the wild and respond to user queries, they have to go back to all of the information that's been pumped into the model via training. They have to access it. And that means a lot more memory chips are required, a lot more high bandwidth memory. Inference also requires a lot of computing power. It's not like it doesn't require any good processing at all. It does require fast processors in order to act quickly. But the name of the game right now is driving down costs to consumers and also driving speed to consumers. The phase we're in right now, we've done a lot of growth, a lot of training of models. Now we're trying to monetize those models. And to do that, you have to have really fast, reliable response times. And so that means the quality answer has to be good and has to come quickly. And so that's why inference requires such a different computing profile than training.

Katie Dayton (9:58)

And given Nvidia's focus on AI training chips so far, and this new inference that's coming in, does that put Nvidia on the back foot in this space, will it have a hard time competing here?

Robbie Whelan (10:09)

The short answer is no, I don't think they're going to be too far behind when it comes to sort of developing products quickly that serve the infinite computing needs and more importantly, perhaps locking up the supply chain that is needed for it to produce inference. So, for example, right now there's a big crunch in supply of memory chips. There's a certain number of companies, three in particular, that make memory chips that are usable in AI systems. And Nvidia is one of the biggest buyers of those chips and they have essentially locked up their supply for the next two or three years. But the age of inference, there's a real worry from a lot of people who I talk to in the AI world that Nvidia's margins are not going to remain that high for very long. And the reason for that is because when you go from producing a luxury product that's very expensive and you do it better than anyone else to producing a product that's meant to save users money, which is what's happening in the transition from training to inference, it's very hard to sustain high profit margins.

Katie Dayton (11:08)

And you mentioned there are a few competitors in this space with them. Can you talk us through a few of those?

Robbie Whelan (11:13)

There's a company called Cerebra. It's a startup that has raised a ton of money and recently did a $10 billion licensing deal with OpenAI to help OpenAI build its own custom chips for inferencing. There's a number of other startups that are sort of in stealth mode that I know about but can't talk about yet. There's a company that we talked about called AR Labs that we wrote stories about recently in the Wall Street Journal that makes a sort of a product that's a different type of chip. It's called an interconnectivity chip that uses advanced fiber optics and lasers in order to make inference computing faster. It's not just in the processors, in other words, it's in all the hardware that goes around the main processors that drive AI computing. The entire ecosystem is seeing much more competition. And then Advanced Micro Devices, amd, they are going all in on inference as well and trying to strike deals with model developers. They're a lot smaller than Nvidia, but they are growing very fast and rolling out products very quickly.

Katie Dayton (12:13)

That was wa USJ tech reporter Robbie Whelan. And that's it for Tech News Briefing. If you're a listener on Spotify, be sure to leave us a comment. Today's show was produced by Julie Chang with supervising producer Katie Ferguson. I'm Katie Dayton for the Wall Street Journal. We'll be back later this morning. With TMB Tech Minute. Thanks for listening.

Xfinity Tech Support (12:40)

Xfinity Tech Support I can't get Jurassic park back online without but no worries. Our WI Fi Boosted Mobile keep connected in all kinds of places.

Xfinity Mobile Advertiser (12:49)

Cut your mobile bill in half with two lines of unlimited. Learn more@xfinity mobile.com Xfinity Imagine that. Watch Jurassic park with Xfinity restrictions apply. Xfinity Internet Required Savings comparison based on promotional rate for two lines of Xfinity Mobile Unlimited versus two lines of T Mobile Experience more Verizon Unlimited plus and AT&T Starter as of February 10, 2026 taxes fees extra.