Loading summary

Transcript30 lines

[00:00]
The Hartford Narrator
When you're running a business, the best days are the ones where priorities stay on track. For midsize and large companies, risk can affect multiple parts of the organization at once, from property and liability to cyber and regulatory challenges. At that level, managing risk becomes an ongoing discipline. At the Hartford, the focus is on helping businesses manage risk before it turns into something more disruptive. And when losses do happen, that work is paired with insurance coverage shaped by years of underwriting, risk engineering and claims experience. Learn more@the Hartford.com risk mitigation policies provided by Hartford Fire Insurance Company and its property and casualty affiliates, Hartford, Connecticut
[00:43]
Matt Garman
Bloomberg
[00:43]
Bloomberg Tech Host
Audio Studios podcasts radio news CEO Matt Garman joins us now on Bloomberg Television and Bloomberg Radio. It's great to have you back on the show. You know, this is the fastest growth for us in nearly five years. But to kind of capture a mom in time, could we start by talking about how much of that was driven by the frontier labs, the big ones, open air, Anthropic, or how much of it was something broader across the enterprise, something bigger in AI?
[01:13]
Matt Garman
Yeah, it's actually across the board this growth that we're seeing right now. And, and much of that is, is some of the AI labs that are building their models on top of us. But actually a lot of that growth is spread across all of our startup and enterprise customers where AI is really impacting almost every single industry. And so whether it's financial services companies or health care companies or retail companies or media companies, really that growth is coming across the board as companies of all different sizes and all different industries are looking to use AI to, to grow their business. And so, so we're seeing a lot of growth from the top frontier labs. But for us, we're not like some others. Maybe this concentrated on just one or two large customers, but it's actually growth from a really broad set of which is nice to see.
[02:03]
Bloomberg Tech Host
Matt, when NWC says the AI business has a revenue run rate of $25 billion, what does that mean? What does the figure encompass?
[02:12]
Matt Garman
Yeah, that that includes both training from very large companies like Anthropic and OpenAI and other large tech companies as well as many startups. But it also, and then a big chunk of that is really inference that that broad swath of companies are doing. And so as companies think about how they take models in something in Amazon Bedrock and to get value out of their business, some of them are automating processes, many of them are running agentic workloads that they build on us to further drive efficiencies in their business or deliver new customer experiences. We consider all of that work, whether it's agent growth, whether it's inference and some of that is training new models, all of that encompasses the AI business for us.
[02:55]
Bloomberg Tech Host
I think I've asked you this question at various points in time since you became AWS CEO, but is there a percentage spike lit right now, training versus inference that you can give me?
[03:06]
Matt Garman
Yeah, you do. And it actually keeps shifting, I would say more and more, it keeps shifting more towards inference. I think we still see large training clusters being used by a number of companies, but as these, as these models get really popular and really powerful, more and more companies are integrating that inference into their workloads. So you know, I don't know the exact percentage today, but it keeps shifting more and more towards inference and we expect that to continue as the economics make sense. Where you really want most of that cost and spend being where value is being created for end customers.
[03:38]
Bloomberg Tech Host
Amazon's overall CapEx number for this year is very big, $220 billion. So it's up 20 billion. And Andy Jassy explained that's mostly AI, but it also accounts for higher memory pricing. Right. CapEx higher because cost environment's higher. But from perspective, what's the trajectory for next year? You expect that capex will be bigger still still? Because I think one of the things Amazon is quite clear about is that even at $220 billion it might not be enough to meet current demand.
[04:10]
Matt Garman
Yeah, one of the things that we're quite excited about is just the potential business for us is just massive and we see this as a huge opportunity for us to really invest and help customers take advantage of, of the opportunity. And so we will keep investing. We think that there's a big opportunity for us and for customers. And as Andy kind of called out last week, we have really great ins into what that demand is going to be from customers. And so when we invest, and we will keep investing in capex next year as well, we have great insight into when that revenue is going to land. And so it's, it's, it's pretty well known for us. And as Andy mentioned, much of our capacity has already spoken through through the end of 27 and even through much of 28. And so as we're investing we're getting five year commitments from customers, we're getting these long term commitments and so we're out there making investments to keep being able to grow the business and try to meet customer demand. But as you today demand still significantly outstripped supply and we're trying to build and invest to keep up with, with what customers are asking for.
[05:14]
Bloomberg Tech Host
And so we expect the CapEx number will be bigger next year than it is this year.
[05:20]
Matt Garman
I expect us to keep investing as we see opportunities and some of that we'll see how the market continues to grow. But, but right now we're, we're excited about our investments and we'll continue to invest.
[05:31]
Bloomberg Tech Host
The chip business has also got a lot of momentum. You know, training for some time has been a big part of the strategy and we've discussed that. I've been able to go to Austin, spend a lot of time in Annapurna Labs to look at the, the server design. Let's start by asking, when you say the chip business has a revenue run rate of $25 billion, what does that mean? That is the business of selling the chips to third parties or renting out capacity based on the chips.
[05:58]
Matt Garman
Yeah, renting out capacity based on those chips. So today we run all of our own chips inside of the cloud and that includes both Trainium chips as well as Graviton chips which are general purpose processors. Graviton is incredibly popular. We've been building Graviton for, for many years now and in fact almost all of our large customers use Graviton as some part of their deployment and it's a key part of how helpful customers deliver extra value by lowering their costs. Trainium is wildly popular and as we've mentioned a couple of times now, we're largely sold out through the end of next year for training capacity and it's really, really popular. We're starting to see both large customers as well as startups really love the cost benefits of running on Trainium and having that choice in a cloud where you can run on purpose built processors like Trainium that give you really great AI performance together with Nvidia chips for when you need some of processing for GPUs as well gives customers the right mix that they're oftentimes looking for and so we find that combination to be quite powerful. And our Trainium 3 chips are really, really popular with customers.
[07:07]
Bloomberg Tech Host
Right now we're live on Bloomberg Television and Bloomberg Radio. This is Bloomberg Tech in conversation with Matt Garman, the CEO of us. Explain the difference in economics between capacity based on training and capacity based on Blackwell for example, like what is the selling point with Amazon Silicon?
[07:28]
Matt Garman
Yeah, we think that look, we think customers want choice and so we offer great offerings for, for both products. Some customers really like to build on Nvidia GPUs and those are fantastic. And we're one of Nvidia's absolute largest customers in the world. We've been offering Nvidia GPUs in the cloud for over a decade now. And AWS is the most scalable and secure and stable place to run Nvidia servers anywhere. And so we're very excited about that business. But when you add Trainium, we're able to lower the costs for many workloads. And so customers like to have that opportunity where they can take these really massively scaled AI workloads or inference workloads, where we can tune them because we kind of control that whole stack. We, we can control the training chips, the data centers and the inference stack and can really tune that whole stack. And that helps customers lower their costs and improve performance. And we're really starting to see that flywheel go where model providers of all types and customers love that they get that value and great performance. And we think that that's a real flywheel that we can keep turning and are quite excited about continuing to grow that business.
[08:38]
Bloomberg Tech Host
Could you make it tangible for me? What is the cost saving?
[08:42]
Matt Garman
It's workload by workload, so there's not like a specific cost savings. But for, for workloads where we see this optimization happen, customers can oftentimes save 20, 30% off inference costs when they run that on training.
[08:55]
Bloomberg Tech Host
Interesting. Would you consider selling the chips outright as opposed to renting capacity based on them?
[09:02]
Matt Garman
Yeah, we've mentioned that we might consider that in the future. Right now we have so much demand inside of the US cloud that we really love that business. And so that's the place we're focusing right now. But it's an interesting opportunity and it's something that we would definitely consider in the future.
[09:17]
Bloomberg Tech Host
The other big development was signing the open weights letters. You yourself communicated that. Why was it important to you, Matt, to sign that and participate in that?
[09:29]
Matt Garman
Yeah, I think that again, a lot of this boils down to customers wanting choice. And the big Frontier Labs have awesome models today. Whether it's OpenAI running on bedrock, whether it's anthropic running on bedrock, customers really love using those models, but they also like being able to customize models. And so having a broad ecosystem of open weights models, I think is incredibly important for innovation. It's important for our customers. And so you think about things like the Nematron models from Nvidia, or you think about Kimi3 or some of the Chinese open weights models, those really help customers be able to customize to their own data and really be able to innovate. And so we think that it's important to not over legislate there and give that flexibility for customers to use whichever models they find to be the best fit. And that's actually why we really focus on having all of those available inside of Bedrock so that customers can choose
[10:21]
Bloomberg Tech Host
which ones they want to use not to over legislate. I mean, I had a conversation with Jensen Huang the day the letter was posted about why then what was the rationale? It does seem like the concern is that the US Government overly regulates open models. Was that sort of a motivation for you?
[10:39]
Matt Garman
Yeah, I think that's the concern. And I think you just want to make sure that we kind of have an even playing field and, and it doesn't mean that there's none, by the way. I think there should probably be the same level of overs of both frontier models as well as open weights models. And I think we should have a consistent framework but, but kind of making sure that, that folks realize how important that is to what customers and our industry is out there building on and that those open weights are a key building block of that and we want to make sure that however legislation lands, it lands evenly across all of those.
[11:10]
Bloomberg Tech Host
Matt, a question from our Bloomberg Tech audience for you is NWS his attitude towards Kimika 3, the open weight was released July 27. You know, given the platforms AWS offers, what is the plan there?
[11:24]
Matt Garman
Yeah, it's a, it's a great model that the team there has done a really nice job and we see really great performance there and, and we'll continue to support in Bedrock offering the Kimi models. I think the interesting thing is also as you see many of these open weight companies, they're starting to think about how they make money in these models as well. And so they're starting to introduce licensing around some of these open weights mod models as when. When customers want to run them in a cloud environment. And so I anticipate that there's going to be a blending of some of these models too, where these companies won't continue to spend lots of money and then just offer their IP up to the world. They're going to be licensing models, they're going to have ways of making money on those as well. And you're starting to see that with some of the open weights models actually having some licensing around them. And so we'll keep supporting these in Bedrock and working with those companies in order to offer these in the best possible way.
[12:19]
Bloomberg Tech Host
That's where I'd like to end the conversation the business case and economic opportunity for open models from US perspective. So if the model makers themselves would like to make money, you know, how does see that that going in your favor to make money from. From wide use of open models?
[12:35]
Matt Garman
Yeah, yeah. I mean we basically, that is exactly what aws. It's a great platform for companies to come and offer their IP and their capabilities to the world. And so everybody, whether they're startups, whether they're governments, whether they're large enterprises, everybody can build on top of us. And if they have choice and they have access to these different models, it allows companies to be able to come and monetize them and sell when they have value to customers, to be able to offer value when they think that they can go at lower prices or better performance. And so having that open platform allows everybody to have that competitive chance. And AWS does a fantastic partner for all of these model providers to get access to the broad set of customers. And so it's a great ecosystem where model providers benefit and customers benefit by having all of these in the same place.
[13:23]
Bloomberg Tech Host
Matt Garman, CEO of U.S. amazon Web Services, back on Bloomberg Tech. Thank you very much. So there's a lot of noise about AI, but time's too tight for more promises. So let's talk about results. At IBM, we work with our employees to integrate technology right into the systems they need. Now a Global workforce of 300,000 can use AI to fill their HR questions. Resolving 94% of common questions, not noise. Proof of how we can help companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business IBM.
[13:58]
The Hartford Narrator
When you're running a business, the best days are the ones where priorities stay on track. For midsize and large companies, risk can affect multiple parts of the organization at once, from property and liability to cyber and regulatory challenges. At that level, managing risk becomes an ongoing discipline. At the Hartford, the focus is on helping businesses manage risk before it turns into something more disruptive. And when losses do happen, that work is paired with insurance coverage shaped by years of underwriting, risk engineering and claims experience. Learn more@thehartford.com Riskmitigation policies provided by Harford Fire Insurance Company and its property and casualty affiliates, Hartford, Connecticut.

Summary6 min read

Podcast Summary: Bloomberg Talks — Amazon Web Services CEO Matt Garman Talks Capex Investment

Date: August 3, 2026
Host: Bloomberg Tech Host
Guest: Matt Garman, CEO of AWS (Amazon Web Services)

Episode Overview

In this episode, Bloomberg Tech host sits down with Matt Garman, CEO of AWS, to discuss Amazon’s record-setting capital expenditures (CapEx), the explosion in demand for AI infrastructure, the growing chip business—including AWS’s in-house silicon like Trainium and Graviton—and the company’s stance on open AI models. The conversation covers AWS’s broad-based growth in AI, economic implications for chip and model providers, and their balanced approach to innovation and regulation.

Key Topics & Insights

1. AWS’s Broad-Based AI Growth

Timestamp: 00:43–02:54

AI as a Cross-Industry Growth Driver:
Matt Garman emphasizes that AWS's recent growth is not limited to large AI labs like OpenAI and Anthropic, but spans startups, enterprises, and industries such as finance, healthcare, retail, and media.
- Quote:
  “It's actually across the board this growth that we're seeing right now ... really that growth is coming across the board as companies of all different sizes and all different industries are looking to use AI to grow their business.” (Matt Garman, 01:13)
AI Revenue Run Rate:
AWS’s AI business has hit a $25 billion run rate, which includes both model training and inference workloads.

2. Training vs. Inference Dynamics

Timestamp: 02:54–03:38

Shifting Economics:
The share of inference workloads is steadily increasing relative to training, as companies deploy models in real-world environments.
- Quote:
  “It keeps shifting more towards inference ... as these models get really popular and really powerful, more and more companies are integrating that inference into their workloads.” (Matt Garman, 03:06)
Customer Value Focus:
AWS expects this trend to continue as inference becomes the main area where value is delivered to customers.

3. CapEx Investment and Demand Outstripping Supply

Timestamp: 03:38–05:19

Record CapEx Spending:
Amazon’s CapEx for the year is $220 billion (up $20 billion), driven primarily by AI investments and higher memory costs.
Long-Term Vision and Demand:
Much of AWS’s capacity is already committed through 2027 and well into 2028, with many customers signing five-year agreements.
- Quote:
  “Much of our capacity has already spoken through the end of 27 and even through much of 28 ... demand still significantly outstripped supply and we're trying to build and invest to keep up with what customers are asking for.” (Matt Garman, 04:09)
Continued Growth Anticipated:
Garman expects ongoing and potentially larger investments next year to meet growing demand.

4. AWS’s Chip Strategy: Trainium and Graviton

Timestamp: 05:31–07:06

Chip Revenue Run Rate:
The chip business itself is at a $25 billion run rate, based on customers renting cloud capacity running on AWS-designed silicon.
In-House Hardware Momentum:
- Graviton: General-purpose processor, widely adopted by large customers for cost savings.
- Trainium: Purpose-built AI accelerator, in high demand (“largely sold out through the end of next year for training capacity”).
- Hybrid Approach: Customers benefit from a mix of Trainium and Nvidia GPUs for different workloads.
- Quote:
  “Having that choice in a cloud where you can run on purpose built processors like Trainium ... gives customers the right mix that they're oftentimes looking for ... our Trainium 3 chips are really, really popular with customers.” (Matt Garman, 05:57)

5. Cloud Economics: AWS Silicon vs. Nvidia GPUs

Timestamp: 07:06–08:42

Choice and Cost Savings:
AWS positions itself as the most scalable and stable host for Nvidia GPUs while offering cost-optimized alternatives through Trainium.
- Quote:
  “Customers can oftentimes save 20, 30% off inference costs when they run that on Trainium.” (Matt Garman, 08:42)
Full Stack Optimization:
By controlling chips, data centers, and AI stack, AWS can tune workloads for maximum performance and efficiency.

6. Future of AWS Silicon: Selling Chips?

Timestamp: 08:54–09:17

Possibility of Hardware Sales:
While AWS currently focuses on renting cloud compute capacity, Garman leaves the door open for potentially selling chips outright in the future.
- Quote:
  “It's an interesting opportunity and it's something that we would definitely consider in the future.” (Matt Garman, 09:02)

7. Open Weights, Regulation, and Customer Choice

Timestamp: 09:17–11:09

Commitment to Open Models:
AWS supports open-weight models (models where the parameters are openly available), citing customer demand for flexibility and innovation.
- Quote:
  “Having a broad ecosystem of open weights models, I think is incredibly important for innovation.” (Matt Garman, 09:29)
Regulatory Balance:
Concern over potential US government overregulation of open models. Garman advocates for consistent oversight across all model types.
- Quote:
  “I think there should probably be the same level of overs of both frontier models as well as open weights models ... and we want to make sure that however legislation lands, it lands evenly across all of those.” (Matt Garman, 10:39)

8. Open Models, Licensing, and AWS’s Role

Timestamp: 11:09–13:22

Model Licensing Evolution:
New open-weight models like Kimi3 are exploring monetization through licensing, signaling a shift in the open model landscape.
- Quote:
  “They're starting to introduce licensing around some of these open weights models ... where these companies won't continue to spend lots of money and then just offer their IP up to the world.” (Matt Garman, 11:23)
AWS as a Platform for Model Monetization:
AWS enables model providers to access a global customer base, offering a platform for both offering and monetizing IP—whether open or licensed.
- Quote:
  “It's a great ecosystem where model providers benefit and customers benefit by having all of these in the same place.” (Matt Garman, 12:35)

Notable Quotes & Memorable Moments

On Diversified Growth:
“We're not like some others ... this is growth from a really broad set—which is nice to see.” (Matt Garman, 01:13)
On AI Demand:
“Demand still significantly outstrips supply and we're trying to build and invest to keep up with what customers are asking for.” (Matt Garman, 04:09)
On Trainium Adoption:
“We're largely sold out through the end of next year for training capacity and it's really, really popular.” (Matt Garman, 05:57)
On Regulation:
“Making sure that however legislation lands, it lands evenly across all of those.” (Matt Garman, 10:39)

Timeline of Major Discussion Points

[00:43] — Kick-off: AWS’s broad-based AI growth
[02:11] — How training and inference split in AI revenue
[03:38] — Record CapEx numbers and future outlook
[05:31] — AWS’s in-house chip strategy (Trainium, Graviton)
[07:28] — AWS silicon vs. Nvidia economics and customer value
[08:54] — Possibility of AWS selling chips
[09:17] — Open weights, regulation, and customer choice
[11:09] — Model licensing and AWS as an enabling platform

Conclusion

This conversation with Matt Garman explores AWS’s multi-layered strategy as it rides explosive growth in the AI and cloud infrastructure sector. With record CapEx investments, a maturing in-house silicon business, and a pragmatic focus on customer choice—especially around open models and responsible regulation—AWS positions itself both as an innovation leader and a key enabler for the entire AI ecosystem. For model builders and enterprises alike, AWS aims to be the flexible and scalable partner for the coming wave of AI applications.

Loading summary

Transcript30 lines

[00:00]
The Hartford Narrator
When you're running a business, the best days are the ones where priorities stay on track. For midsize and large companies, risk can affect multiple parts of the organization at once, from property and liability to cyber and regulatory challenges. At that level, managing risk becomes an ongoing discipline. At the Hartford, the focus is on helping businesses manage risk before it turns into something more disruptive. And when losses do happen, that work is paired with insurance coverage shaped by years of underwriting, risk engineering and claims experience. Learn more@the Hartford.com risk mitigation policies provided by Hartford Fire Insurance Company and its property and casualty affiliates, Hartford, Connecticut
[00:43]
Matt Garman
Bloomberg
[00:43]
Bloomberg Tech Host
Audio Studios podcasts radio news CEO Matt Garman joins us now on Bloomberg Television and Bloomberg Radio. It's great to have you back on the show. You know, this is the fastest growth for us in nearly five years. But to kind of capture a mom in time, could we start by talking about how much of that was driven by the frontier labs, the big ones, open air, Anthropic, or how much of it was something broader across the enterprise, something bigger in AI?
[01:13]
Matt Garman
Yeah, it's actually across the board this growth that we're seeing right now. And, and much of that is, is some of the AI labs that are building their models on top of us. But actually a lot of that growth is spread across all of our startup and enterprise customers where AI is really impacting almost every single industry. And so whether it's financial services companies or health care companies or retail companies or media companies, really that growth is coming across the board as companies of all different sizes and all different industries are looking to use AI to, to grow their business. And so, so we're seeing a lot of growth from the top frontier labs. But for us, we're not like some others. Maybe this concentrated on just one or two large customers, but it's actually growth from a really broad set of which is nice to see.
[02:03]
Bloomberg Tech Host
Matt, when NWC says the AI business has a revenue run rate of $25 billion, what does that mean? What does the figure encompass?
[02:12]
Matt Garman
Yeah, that that includes both training from very large companies like Anthropic and OpenAI and other large tech companies as well as many startups. But it also, and then a big chunk of that is really inference that that broad swath of companies are doing. And so as companies think about how they take models in something in Amazon Bedrock and to get value out of their business, some of them are automating processes, many of them are running agentic workloads that they build on us to further drive efficiencies in their business or deliver new customer experiences. We consider all of that work, whether it's agent growth, whether it's inference and some of that is training new models, all of that encompasses the AI business for us.
[02:55]
Bloomberg Tech Host
I think I've asked you this question at various points in time since you became AWS CEO, but is there a percentage spike lit right now, training versus inference that you can give me?
[03:06]
Matt Garman
Yeah, you do. And it actually keeps shifting, I would say more and more, it keeps shifting more towards inference. I think we still see large training clusters being used by a number of companies, but as these, as these models get really popular and really powerful, more and more companies are integrating that inference into their workloads. So you know, I don't know the exact percentage today, but it keeps shifting more and more towards inference and we expect that to continue as the economics make sense. Where you really want most of that cost and spend being where value is being created for end customers.
[03:38]
Bloomberg Tech Host
Amazon's overall CapEx number for this year is very big, $220 billion. So it's up 20 billion. And Andy Jassy explained that's mostly AI, but it also accounts for higher memory pricing. Right. CapEx higher because cost environment's higher. But from perspective, what's the trajectory for next year? You expect that capex will be bigger still still? Because I think one of the things Amazon is quite clear about is that even at $220 billion it might not be enough to meet current demand.
[04:10]
Matt Garman
Yeah, one of the things that we're quite excited about is just the potential business for us is just massive and we see this as a huge opportunity for us to really invest and help customers take advantage of, of the opportunity. And so we will keep investing. We think that there's a big opportunity for us and for customers. And as Andy kind of called out last week, we have really great ins into what that demand is going to be from customers. And so when we invest, and we will keep investing in capex next year as well, we have great insight into when that revenue is going to land. And so it's, it's, it's pretty well known for us. And as Andy mentioned, much of our capacity has already spoken through through the end of 27 and even through much of 28. And so as we're investing we're getting five year commitments from customers, we're getting these long term commitments and so we're out there making investments to keep being able to grow the business and try to meet customer demand. But as you today demand still significantly outstripped supply and we're trying to build and invest to keep up with, with what customers are asking for.
[05:14]
Bloomberg Tech Host
And so we expect the CapEx number will be bigger next year than it is this year.
[05:20]
Matt Garman
I expect us to keep investing as we see opportunities and some of that we'll see how the market continues to grow. But, but right now we're, we're excited about our investments and we'll continue to invest.
[05:31]
Bloomberg Tech Host
The chip business has also got a lot of momentum. You know, training for some time has been a big part of the strategy and we've discussed that. I've been able to go to Austin, spend a lot of time in Annapurna Labs to look at the, the server design. Let's start by asking, when you say the chip business has a revenue run rate of $25 billion, what does that mean? That is the business of selling the chips to third parties or renting out capacity based on the chips.
[05:58]
Matt Garman
Yeah, renting out capacity based on those chips. So today we run all of our own chips inside of the cloud and that includes both Trainium chips as well as Graviton chips which are general purpose processors. Graviton is incredibly popular. We've been building Graviton for, for many years now and in fact almost all of our large customers use Graviton as some part of their deployment and it's a key part of how helpful customers deliver extra value by lowering their costs. Trainium is wildly popular and as we've mentioned a couple of times now, we're largely sold out through the end of next year for training capacity and it's really, really popular. We're starting to see both large customers as well as startups really love the cost benefits of running on Trainium and having that choice in a cloud where you can run on purpose built processors like Trainium that give you really great AI performance together with Nvidia chips for when you need some of processing for GPUs as well gives customers the right mix that they're oftentimes looking for and so we find that combination to be quite powerful. And our Trainium 3 chips are really, really popular with customers.
[07:07]
Bloomberg Tech Host
Right now we're live on Bloomberg Television and Bloomberg Radio. This is Bloomberg Tech in conversation with Matt Garman, the CEO of us. Explain the difference in economics between capacity based on training and capacity based on Blackwell for example, like what is the selling point with Amazon Silicon?
[07:28]
Matt Garman
Yeah, we think that look, we think customers want choice and so we offer great offerings for, for both products. Some customers really like to build on Nvidia GPUs and those are fantastic. And we're one of Nvidia's absolute largest customers in the world. We've been offering Nvidia GPUs in the cloud for over a decade now. And AWS is the most scalable and secure and stable place to run Nvidia servers anywhere. And so we're very excited about that business. But when you add Trainium, we're able to lower the costs for many workloads. And so customers like to have that opportunity where they can take these really massively scaled AI workloads or inference workloads, where we can tune them because we kind of control that whole stack. We, we can control the training chips, the data centers and the inference stack and can really tune that whole stack. And that helps customers lower their costs and improve performance. And we're really starting to see that flywheel go where model providers of all types and customers love that they get that value and great performance. And we think that that's a real flywheel that we can keep turning and are quite excited about continuing to grow that business.
[08:38]
Bloomberg Tech Host
Could you make it tangible for me? What is the cost saving?
[08:42]
Matt Garman
It's workload by workload, so there's not like a specific cost savings. But for, for workloads where we see this optimization happen, customers can oftentimes save 20, 30% off inference costs when they run that on training.
[08:55]
Bloomberg Tech Host
Interesting. Would you consider selling the chips outright as opposed to renting capacity based on them?
[09:02]
Matt Garman
Yeah, we've mentioned that we might consider that in the future. Right now we have so much demand inside of the US cloud that we really love that business. And so that's the place we're focusing right now. But it's an interesting opportunity and it's something that we would definitely consider in the future.
[09:17]
Bloomberg Tech Host
The other big development was signing the open weights letters. You yourself communicated that. Why was it important to you, Matt, to sign that and participate in that?
[09:29]
Matt Garman
Yeah, I think that again, a lot of this boils down to customers wanting choice. And the big Frontier Labs have awesome models today. Whether it's OpenAI running on bedrock, whether it's anthropic running on bedrock, customers really love using those models, but they also like being able to customize models. And so having a broad ecosystem of open weights models, I think is incredibly important for innovation. It's important for our customers. And so you think about things like the Nematron models from Nvidia, or you think about Kimi3 or some of the Chinese open weights models, those really help customers be able to customize to their own data and really be able to innovate. And so we think that it's important to not over legislate there and give that flexibility for customers to use whichever models they find to be the best fit. And that's actually why we really focus on having all of those available inside of Bedrock so that customers can choose
[10:21]
Bloomberg Tech Host
which ones they want to use not to over legislate. I mean, I had a conversation with Jensen Huang the day the letter was posted about why then what was the rationale? It does seem like the concern is that the US Government overly regulates open models. Was that sort of a motivation for you?
[10:39]
Matt Garman
Yeah, I think that's the concern. And I think you just want to make sure that we kind of have an even playing field and, and it doesn't mean that there's none, by the way. I think there should probably be the same level of overs of both frontier models as well as open weights models. And I think we should have a consistent framework but, but kind of making sure that, that folks realize how important that is to what customers and our industry is out there building on and that those open weights are a key building block of that and we want to make sure that however legislation lands, it lands evenly across all of those.
[11:10]
Bloomberg Tech Host
Matt, a question from our Bloomberg Tech audience for you is NWS his attitude towards Kimika 3, the open weight was released July 27. You know, given the platforms AWS offers, what is the plan there?
[11:24]
Matt Garman
Yeah, it's a, it's a great model that the team there has done a really nice job and we see really great performance there and, and we'll continue to support in Bedrock offering the Kimi models. I think the interesting thing is also as you see many of these open weight companies, they're starting to think about how they make money in these models as well. And so they're starting to introduce licensing around some of these open weights mod models as when. When customers want to run them in a cloud environment. And so I anticipate that there's going to be a blending of some of these models too, where these companies won't continue to spend lots of money and then just offer their IP up to the world. They're going to be licensing models, they're going to have ways of making money on those as well. And you're starting to see that with some of the open weights models actually having some licensing around them. And so we'll keep supporting these in Bedrock and working with those companies in order to offer these in the best possible way.
[12:19]
Bloomberg Tech Host
That's where I'd like to end the conversation the business case and economic opportunity for open models from US perspective. So if the model makers themselves would like to make money, you know, how does see that that going in your favor to make money from. From wide use of open models?
[12:35]
Matt Garman
Yeah, yeah. I mean we basically, that is exactly what aws. It's a great platform for companies to come and offer their IP and their capabilities to the world. And so everybody, whether they're startups, whether they're governments, whether they're large enterprises, everybody can build on top of us. And if they have choice and they have access to these different models, it allows companies to be able to come and monetize them and sell when they have value to customers, to be able to offer value when they think that they can go at lower prices or better performance. And so having that open platform allows everybody to have that competitive chance. And AWS does a fantastic partner for all of these model providers to get access to the broad set of customers. And so it's a great ecosystem where model providers benefit and customers benefit by having all of these in the same place.
[13:23]
Bloomberg Tech Host
Matt Garman, CEO of U.S. amazon Web Services, back on Bloomberg Tech. Thank you very much. So there's a lot of noise about AI, but time's too tight for more promises. So let's talk about results. At IBM, we work with our employees to integrate technology right into the systems they need. Now a Global workforce of 300,000 can use AI to fill their HR questions. Resolving 94% of common questions, not noise. Proof of how we can help companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business IBM.
[13:58]
The Hartford Narrator
When you're running a business, the best days are the ones where priorities stay on track. For midsize and large companies, risk can affect multiple parts of the organization at once, from property and liability to cyber and regulatory challenges. At that level, managing risk becomes an ongoing discipline. At the Hartford, the focus is on helping businesses manage risk before it turns into something more disruptive. And when losses do happen, that work is paired with insurance coverage shaped by years of underwriting, risk engineering and claims experience. Learn more@thehartford.com Riskmitigation policies provided by Harford Fire Insurance Company and its property and casualty affiliates, Hartford, Connecticut.