Microsoft's Head of Cloud & AI on the AI Buildout's Risks and ROI — With Scott Guthrie

Summary

Big Technology Podcast Summary

Episode: Microsoft's Head of Cloud & AI on the AI Buildout's Risks and ROI — With Scott Guthrie
Host: Alex Kantrowitz
Guest: Scott Guthrie, EVP, Cloud & AI, Microsoft
Date: October 1, 2025

---

Episode Overview

In this episode, Alex Kantrowitz interviews Scott Guthrie, Microsoft’s Head of Cloud & AI, about the enormous investments pouring into AI infrastructure across tech: are these investments too much, not enough, or just right? They explore Microsoft’s calculated approach to spending, the promise and pitfalls of scaling AI, balancing technological innovation with fiscal discipline, the realities behind headlines, geopolitics in data center buildouts, and the ROI that companies (and clients) are—or aren’t—seeing.

---

Key Discussion Points & Insights

1. The "Enormous" AI Buildout: Is It Overinvestment?

• Headline Context: Recent news includes Nvidia and OpenAI's $100B announcement, Oracle's $30B AI buildout, Anthropic’s $13B raise—over $143B total across a few weeks.
• Scott Guthrie’s Perspective:
  • Believes AI is the most profound tech shift in decades and requires massive infrastructure investment.
  • Feels the market is still "supply constrained rather than demand constrained."
  • Cautions that while not every investment may be perfectly timed, the long-term need for robust infrastructure, across many AI workloads and industries, justifies aggressive spending.
  • Quote:

    "I think the long term secular trend of AI is going to be that we're going to need more infrastructure... I don't worry about over-investing." (01:07)

2. Microsoft’s Investment Philosophy: "Disciplined, Not Blanket"

• Why Not a $100B Bet Like Others?
  • Microsoft has a major partnership with OpenAI but hasn’t matched the blockbuster $100B investments.
  • They “balance” investments for Microsoft’s ecosystem (first-party products), customers, and partners like OpenAI, aiming for optimal ROI and yield on every new build.
  • Quote:

    "We're always trying to take a continually balanced view of that... We're building out like crazy. But... we're always constantly reevaluating." (05:22)

• On Investment ‘Discipline’:
  • It’s not that massive build-outs are "undisciplined" per se, but Microsoft ensures each expansion can serve a broad set of needs—training, inference, product offerings, regional legal requirements, customer demand.
  • Maximizing ROI means only building when they see clear usage.
  • Quote:

    "We know that, you know, some opportunities will have more certain returns than others, and we're trying to make sure that we maximize our focus around those." (06:32)

3. Training, Scaling Laws, and Diminishing Returns

• The Scaling Debate:
  • Alex probes whether Microsoft is skeptical about the returns of blowing up model size ad infinitum—a core bet of startups like OpenAI/Anthropic.
  • Scott: Microsoft sees major gains from scaling, but not all training is equal—pre-training, post-training, fine-tuning, reinforcement learning each offer different trade-offs.
  • Microsoft’s infrastructure enables both huge contiguous pre-training builds and distributed, smaller post-training and inference globally.
  • Quote:

    "Will the scaling laws improve linearly? Will they improve at the rate that they have? That is a question that everyone right now in the AI space is still trying to calculate." (14:57)

  • The return per dollar/watt is constantly scrutinized.

4. Geopolitics, Regulation, and Global Distribution

• Location, Location, Location:
  • Microsoft is highly attuned to data residency, sovereignty, and global regulatory differences.
  • Infrastructure is distributed because customers in Europe, Asia, and North America want data local to them.
  • Regulatory burdens (especially permitting in the U.S.) are a significant bottleneck.
  • Quote:

    "Even as we build out our infrastructure, we want to think about it ... We need to be distributed around the world to kind of meet those geopolitical needs..." (10:20)

• China’s Advantages:
  • China can move faster due to looser regulatory approval, but in the U.S., state and local governments matter more for permitting than construction timelines.
  • Microsoft’s partnerships with communities aim for mutual benefit—jobs, economic activity, energy deals.

5. The Role of Debt in AI Buildout

• Industry Shift:
  • The Wall Street Journal reports growing reliance on debt to finance AI, echoing dot-com bubble risks.
  • Microsoft, in contrast, keeps debt low and spending aligned with strong cash flow and CapEx growth, focusing on sustainable ROI, not speculation.
  • Quote:

    "We're not going to sit on the sidelines and not be bold as we invest. And at the same time...we have the ability to ride through, you know, the ups and downs that inevitably will happen..." (22:11)

• Risks If Debt Boom Breaks:
  • If others get overextended, companies lacking a plan for real, revenue-generating use of infrastructure will struggle.
  • Microsoft’s depth of applications—ChatGPT, 365 Copilot, GitHub Copilot—anchors its investments.
  • Quote:

    "I think not every company probably has that level of game plan...different companies will probably be hit by it." (25:15)

6. GPU Lifespans, Utilization & the Technological Horizon

• GPU Depreciation and Flexibility:
  • GPUs can be redeployed for various use cases as they age. Robust planning ensures no sudden obsolescence.
  • Maximizing network, storage, and data center design is as important as silicon.
  • Quote:

    "What you use [the GPU] for in year one or two might be very different than how you use it in year three, four and five or six." (30:13)

• Breakthroughs Ahead:
  • Move to liquid-cooled data centers is a key advance for the next generation of AI hardware.
  • Custom silicon increasingly critical (token/$/watt), not just for GPUs but for networks, compression, security, etc.

7. Data Centers and Community Impact

• Jobs:
  • Construction of new centers creates thousands of skilled trade jobs, often continuously as new “phases” launch.
  • Hundreds of permanent jobs remain post-construction, with ripple effects as more facilities come online.
  • Quote:

    "It's thousands of jobs that we've created...these are very skilled jobs...welders, plumbers, electricians..." (36:24)

8. ROI & When to Stop Investing

• Quarterly Discipline:
  • Revenue growth and CapEx are watched closely quarter-by-quarter; the market “keeps companies honest.”
  • Azure’s 39% YoY growth is anchored by real usage, not just hype or bookings.
  • Microsoft will pare back if investments stop yielding returns.
  • Quote:

    "We have a report card every quarter...it's not like they're pre-buying a ton of stuff...the good news is when you look at our revenue growth, it is...a consumption number..." (41:14)

• Consumption vs. Bookings:
  • True enterprise usage is consumption-based in Azure, not just “sales.”

9. Custom Silicon vs. Nvidia: Partners and Competitors

• Will the Future Run on GPUs or Something Else?
  • Custom silicon (Microsoft’s and others) is emerging to optimize for narrow use cases: “tokens per watt per dollar.”
  • Nvidia remains a key partner but competition and in-house development will only grow.
  • Quote:

    "We're probably one of, if not the biggest customer in the world of [Nvidia]...There will be other opportunities from other companies where people are going to look for a niche...to be truly differentiated versus what Nvidia is delivering." (46:47)

• On Competing with Partners:
  • Microsoft maturely balances coopetition—deep partnerships and direct competition with suppliers like Nvidia.
  • Quote:

    "The important thing is I think you have that enterprise maturity to be able to recognize. I want Jensen [Nvidia CEO] to do the best possible work because it's going to benefit me." (50:59)

10. Memorable Moments & Notable Quotes

• On Balance:
  • "Balance in life, but especially in business and especially in technology, that is the devil's in the detail. But if you can get that right and do it consistently, those are the companies that win." (52:46)
• On Media & Headlines:
  • "The headlines were focused on things that we canceled as opposed to all the things we signed...” (18:12)

---

Timestamps of Important Segments

• 00:35 — Discussion of headline AI infrastructure spending ($143B in recent weeks)
• 01:07 — Scott on why he doesn’t worry about over-investing in AI
• 05:22 — Microsoft’s disciplined investment approach vs. all-in bets
• 13:16 — Debate on diminishing returns of scaling AI models
• 18:12 — Why Microsoft cancels or pauses data center projects
• 22:11 — Debt-fueled AI buildout and Microsoft’s approach vs. Oracle/Others
• 30:13 — Lifespan and utilization of GPUs after their peak
• 33:26 — Next-gen data center tech: liquid cooling, custom silicon, networking
• 36:24 — Local job creation and economic impact of new data centers
• 39:09 — US vs. China: Regulatory hurdles and the pace of buildout
• 41:14 — When Microsoft evaluates ROI and considers slowing investment
• 46:47 — Custom silicon vs. Nvidia and the future of AI chips
• 50:59 — Navigating partnership and competition with suppliers like Nvidia
• 52:46 — The importance of balance in tech leadership

---

Conclusion

Scott Guthrie provides a nuanced, clear-eyed look at Microsoft’s multi-billion dollar AI investment strategy and how it contrasts with some competitors’ “all-in” approaches. He underscores the need for flexibility, discipline, and balance—across product use cases, geographic regions, and even between partners and competitors—to maximize ROI and avoid the exuberance and risk that can topple industries. Guthrie’s insights into the technical, business, and human sides of the AI buildout spotlight Microsoft’s measured, tactical mindset as the sector barrels into an uncertain but opportunity-rich future.

Transcript

[Alex]

Microsoft's head of cloud and AI joins us as we ask, is this AI build out going too far? That's coming up right after this. Welcome to Big Technology Podcast, a show for cool headed and nuanced conversation of the tech world and beyond. We're joined today by Scott Guthrie. He's the head of cloud and AI at Microsoft and he is the perfect guest to give us some context on the massive and some would say insane build out of AI data centers taking place today. What does it mean? Is it going too far? What will lead to. Scott, I'm so thrilled to have you on the show. Welcome.

[Scott Guthrie]

It's great to be here, Alex. Thanks for having me.

[Alex]

All right, let me take you through the headlines over the past couple weeks. It's crazy that this has just been over the past few weeks, but here we go. Nvidia agreed or announced that it would invest up to $100 billion in OpenAI, starting out with 10 billion. Oracle announced they would invest 30 billion in OpenAI or a $30 billion build out with, with the company Anthropic raised 13 billion. So, you know, we're just talking about a cool 143 billion, no big deal. Is this crazy? Is this overinvestment?

[Scott Guthrie]

Well, I think there's a great question, I'm sure that's top of mind for everyone. I think, you know, stepping back for a moment, I would say if you look at AI and the impact I think it's going to have in the economy, it's going to be, I think, the most profound technology shift in, in our lifetimes. And so I, I think if you look at the long term trend, I, I don't see, I, I don't worry about over investing. I think that, you know, there will be a question on the horizon of different companies or making different strategies in terms of their investment and how they get their return in the 1, 2, 3 year horizon. So, you know, am I going to say that every company is perfectly timed? I don't, I'm not going to make that assertion. But at the same time, I do think the long term secular trend of AI is going to be that we're going to need more infrastructure, there's going to be more ROI from it and it's going to be more widely used. And so, you know, I think directionally from an industry perspective, the investments do make sense and will ultimately yield pretty profound results.

[Alex]

You think? So you think that this level of build out is healthy?

[Scott Guthrie]

I think we definitely are not nearly at the point at which there is too Much AI infrastructure given I think the number of AI workloads that are coming for the world and you know, I think we're seeing on a, over the last couple of years as people use AI, they get value, they use it more, the models get better and people then use it even more for new use cases. And I think at this point across the industry with AI, we're still more supply constrained than we are demand constrained. And I think, I expect that to continue over the next couple of years as the technology continues to evolve and as people start to integrate AI into more and more workflows.

[Alex]

Okay, so let me put it bluntly then. Microsoft has a partnership with OpenAI, has invested 13 billion thereabouts, has the capacity to build big data centers. Why did Microsoft make the decision not to do the hundred billion dollar level build out with OpenAI or even the 30 billion that the company is doing with Oracle and leave it to other partners to do that?

[Scott Guthrie]

Well, we have a great partnership with OpenAI and it's gone back many, many years and continues going forward. And we are building out and doing a lot of projects with OpenAI and across the Microsoft Cloud, we're building out AI data centers all over the world and at the same time we are balancing our investment to make sure that it maximizes the AI infrastructure for both our first party Microsoft offerings that we're investing in, our customers, AI offerings that we're investing in, and obviously OpenAI's offerings that we're deeply enabling. So, you know, we are very invested. I don't think it's a binary. Are we building out for OpenAI or not? We definitely are building out for OpenAI and at the same time the way our partnership works is we're supportive of others participating in that as well.

[Alex]

Okay. But I just want to put a fine point on it because again, like if, if you believe that this technology is going to be massively transformative, which you stated, and that we're not at the, at the sort of optimal point of the AI build out yet, that there's room to continue to do more. And again, it's the partnership with, but is the consensus leader in the space they needed more infrastructure. There must have been some calculation within your group or your company to say is it worth it for us to be the one that goes out and builds this massive, massive footprint in partnership with them or somebody else? So I definitely understand there's multiple stakeholders, but what made Microsoft pause on that front?

[Scott Guthrie]

Well, we have a balanced view and we take both a long term view in Terms of making sure that we're building out in all the locations that we want to build out, that we're being thoughtful in terms of kind of the investment spend and the infrastructure that we're building and also recognize that we don't have to do it all. And so I think we're always trying to kind of take a continually balanced view of that. And as you've seen from our capex and as you've seen from our earnings calls, we are investing a lot in infrastructure and building out like crazy. But again, at the same time, we're always constantly reevaluating and watching closely which data centers in which markets, to what specifications, and making sure that we keep good discipline as we're doing it. That optimizes for both the long term, near term and midterm horizons.

[Alex]

Okay, I'll just ask one more follow up and then we can move on. Good discipline. What about this would have been undisciplined to have gone to this level?

[Scott Guthrie]

Well, I don't think it's so much the volume level. I think it's, it's one of the things that we do when we add new data, data center capacity or AI infrastructure is, you know, making sure that, that we can use this infrastructure for a variety of different AI use cases. I think one of the things that's really going to differentiate AI infrastructure companies in the future is that ability to maximize yield on the infrastructure. Like, how are you driving down the cost of tokens per watt per dollar? Part of what makes the Microsoft portfolio so unique is the fact that we have a lot of our own AI products. Microsoft 365 Copilot, GitHub, Copilot. The work that we're doing with Nuance and Dragon and Healthcare, we've got the world's largest consumer application with ChatGPT that runs on top of Azure. And we have thousands, hundreds of thousands and millions of businesses that are also building their own AI applications on top of us. And so as we think about what market are we going to build a new data center? Is it for training, is it for inferencing? And how do we make sure that that infrastructure is going to be maximally used? We feed in kind of each of these different customer scenarios into our calculus. And there are certain tranches of capacity that we're happy to build out because we can see very clear line of sight in terms of how we're going to maximize the usage and the revenue from it. And there's others that were maybe less likely to see the immediate or the ROI that we'd like. And so we try to be disciplined about it, as we've kind of shared in our blog post. You know, we do kind of look at every request first and we do have an opportunity on that. And as you've seen from our capex, we are swinging in a lot of opportunities. But that doesn't, you know, we're not going to be undisciplined and say blanket. We're going to do everything. We know that, you know, some opportunities will have more certain returns than others, and we're trying to make sure that we maximize our focus around those.

[Alex]

You know, as, as we're talking. I'm kind of laughing at myself because, you know, my, my question is basically boiling down to, you've, like, talked about your capex. Isn't Microsoft expected to spend like 80 billion on infrastructure this year or in the neighborhood?

[Scott Guthrie]

I think that's what we shared in our last earnings.

[Alex]

And I'm like, well, why aren't you doing another 100 billion? But, but the fact that you're not is actually very interesting. And, and it goes to a point that you, that you just made. And, and I'm trying to read behind the line, between the lines and you tell me if I'm, if I'm getting this right. You think you're talking about where you invest and where you're pretty sure you're going to get an roi. And to me, if I'm sitting in your shoes, the question I would be asking is, is it worth spending, you know, all that money on training where there, where there's been a, a lot of noise about diminishing returns of training larger models with these unbelievably massive data centers. Now you have the startups like OpenAI and Anthropic. You know, their, their belief in the scaling law seems unabated. And so the numbers get bigger and bigger and, and they seem to believe that they'll continue to get an exponential return from training these bigger models. But is your decision in terms of being disciplined based on a belief that you're not, you're not sure if, if scaling up will continue to work and therefore it's too big of a risk to make such a large bet on training an even bigger model in a bigger data center?

[Scott Guthrie]

Well, I think there's, there's a couple different elements of that. I think one is recognizing that you want to have the best models. So training is super important because if you don't have the best models, then your actual ability to monetize AI goes down. And part of what makes our partnership with OpenAI Unique is the fact that we do have access to the best models, frankly, whether they're trained on our infrastructure or anywhere else. That's part of our approach partnership. That's really important. And I also think when you think about training, training is evolving from maybe where simplistically, we think of training a couple years ago of you do training in one place and then you do inferencing, where you are executing the models and building applications. There's now multiple types of training. There's pre training, there's post training, there's reinforcement learning, there's fine tuning, there's a lot of new techniques that both sometimes require lots of contiguous infrastructure and sometimes requires lots of infrastructure, but sometimes it's smaller sizes that can be used for very specific tasks. And so when we think about the investments of our infrastructure, we're trying to think about all of this and compose it all end to end for us. That means, for example, we want to make sure that we have lots of inferencing capacity, because that ultimately is how customers pay us and how ultimately you make money from any AI product that you build. And I think increasingly on the inferencing side, one important element is the geopolitics of the world have gotten complicated over the last many years. And customers in Europe want to make sure that their AI is in Europe, and the customers in Asia are going to care about their AI in Asia. Obviously, the customers in North America, in the United States are going to care about their AI being delivered in North America. And so even as we build out our infrastructure, we want to think about it not just narrowly, as we want to have one giant pool all in the US we need to be distributed around the world to kind of meet those geopolitical needs and to make sure that our AI is as close to the customers that are going to be using the AI as possible and can meet all of the data residency and data sovereignty needs. And so even if you look at our infrastructure builds around the world, we have regions in more countries in more locations than any other infrastructure provider. And again, as we balance out the investments we're making on AI infra, you know, we're trying to keep that in mind versus narrowly put it all in one location.

[Alex]

I totally understand that, but I have to go back to the diminishing returns of training question. Where do you stand on that?

[Scott Guthrie]

Well, I think if you look at training broadly, I think you're going to continue to see more value from the models by doing more training. But kind of going back to my answer earlier, I don't know if that's always going to be pre training. I think increasingly lots of post training activities are going to significantly change the value of the model. And so by post training, I mean take the base model. And how do you add financial data or healthcare data or something that's very specific to an application or a use case? What's nice about post training is that you don't have to do it in one large data center in one location. And so part of the technique that we've been focused on is how do we take this inferencing capacity around the world? And a lot of it is idle at night as people go to sleep. How are we doing? Increasingly, post training in a distributed fashion across many, many different sites. And then when employees come to work in the morning, we serve the applications. And so having that kind of flexibility and being able to dynamically schedule your AI infrastructure so that you're maximizing revenue generation and training, ideally in a very swappable, dynamic way, I think is one of the things we're investing in heavily and I think is one of the differentiators for Microsoft.

[Alex]

Okay, but you'll forgive me for going back to this scaling pre training question. I'm just trying to see what you believe here, and you haven't said it outright, but from your answers, it does seem to me like you believe that spending wildly on scaling pre training is a bad bet.

[Scott Guthrie]

I wouldn't necessarily say that. I think we've definitely seen as the scale infrastructure for pre training has gotten bigger, we are seeing the models continually improve and we're investing in those types of pre training sites and infrastructure. We recently, for example, announced our Fairwater data center regions around the U.S. we have multiple Fairwaters. And we did a blog post recently of one of our new sites in Wisconsin. And these are hundreds of megawatts, hundreds of thousands of the latest GB2 hundreds and GB300 GPUs and are, yeah, we think the largest contiguous block of GPUs anywhere in the world in one giant training infrastructure that can be used for pre training. And so we're investing heavily in that, as you can see kind of from the, the photos from the sky, in terms of massive infrastructure. And you know, we do continue to see the scaling laws improve. Now, will the scaling laws improve linearly? Will they improve at the rate that they have? I think that is a question that everyone right now in the AI space is still trying to calculate. But do I Think they will improve? Yes. And the question really around what's the rate of improvement on pre training? And I do think with post training we're going to continue to see dramatic improvements. And that's again why we're trying to make sure we have a balanced investment both on pre training and post training infrastructure.

[Alex]

And yeah, and just to parse your words here, you can see improvement by doubling the data center. But that's why I use the word bet, because are you going to get the same return if it doesn't improve exponentially and just improves on the margins? And that I think is the big question right now. Right?

[Scott Guthrie]

That's a big question. And the thing that also makes it the big question is it's not like a law of nature that's immovable. There could be one breakthrough that actually changes the scaling laws for better and there could be a lack of breakthroughs. That means again, things will still improve, but do they improve at the same rate that they historically did from a raw size and scale perspective? And that is the trillion dollar questions.

[Alex]

Okay, great. I do want to get to the ROI of AI spend in a moment. It's always great to have a chance to speak with someone who's in a position like you are within Microsoft because we get a chance to take some headlines which might paint a portion of a story and then ask you what the truth is. Uh, there were some stories over the past year talking about Microsoft had like canceled options to build data centers in certain locations. And people took those headlines and they read into it that there was no demand for AI or that it wasn't going as well as Microsoft's telling us. But, but what is the, what is the reason for why those, those data centers there were the options and they were canceled. What, what happened there?

[Scott Guthrie]

Well, we're constantly, I think in general the headlines were focused on things that we canceled as opposed to all the things we signed. And so if you look at and.

[Alex]

Give amazing how news works that way, right? If it bleeds, it leads.

[Scott Guthrie]

So if you, if you look at kind of the overall investments and certainly if you look at the overall capex spend, it has been going up and up and up and so as has again the revenue that comes from it. And so I think I would kind of focus on the overall picture as opposed to individual tranches or individual projects that we potentially made decisions on. Now, you know, the thing that we did do and we continually do is look hard at every single investment decision we make. We don't take this level of investment and this level of project and infrastructure lightly. It's critical that we invest wisely. Is it critical if we invest that we make it successful and that we bring it to market on time with the right quality and the right security? And it's critical that we have the right go to market to monetize it. And so, you know, part of our calculus that we do as a leadership team is constantly looking at the variables for all of those. And there are places and times when we slow down or pause projects and there are times when we accelerate projects somewhere else. And kind of going back to my comment around the world also, the regulation geopolitics of how AI is going to be used going forward has changed quite a bit. And what Europe thinks about where GPUs can be based has evolved quite a bit I'd say in the last 12 to 18 months. And I think it's going to continue to evolve around the world. And so even as we think about the investments we're making, we're also being very, very thoughtful in terms of where geography based. Are we investing so that we can again maximize the AI tokens we can serve in real production applications and then ultimately use that maximization to ensure that we're delivering a good return on investment for every capital dollar we spend.

[Alex]

Okay, and I have some technology questions for you, but just to keep on speaking about the financing of this stuff because it's so important. So there has been some interesting reporting about how the AI infrastructure build out has begun to be funded by debt, not just profits. Great story in the Wall Street Journal this week. It says debt is fueling the next wave of the AI boom. I'll read the beginning. In the initial years of the AI boom companies comparisons to the dot com bubble didn't make sense. Three years in growing level of debt are making them ring truer. Early on wealthy tech companies were opening their wallets to out joust each other for leadership in AI. They were spending cash generated largely from advertising and cloud computing businesses. There was no debt fueled splurge on computing and networking infrastructure like the one that inflated the bubble two and a half decades ago. However, that is starting to happen now. OpenAI's deal with Oracle is it has been pushed Oracle to start taking on debt. They say this is according to the story. Analysts at KeyBank Capital Markets estimated in a recent note that Oracle would have to borrow 25 billion a year over the next four years. Obviously you guys are not Oracle, but you're watching this happen as it plays out and see the parallels to the dot com boom, I'm sure, is not fun. You've been at Microsoft for I think 27 years. 28 years. 28. Sorry, I don't want to miss that last year there. So you've seen it. Scott, this seems to be an issue, at least from the outside. What do you think about it being on the inside?

[Scott Guthrie]

Well, I think, I think obviously there's a tremendous amount of spend from lots of different companies and I, I would say, yeah, the thing I can speak most to is what we're doing and kind of, you know, per my comments earlier, I think we're trying to make sure that we have a smart investment play and a long term strategic play that allows us to ride the AI revolution that we think is going to transform the world and do it in a way that leverages some of the strengths that we have at Microsoft, which is we have very good cash flow, we have a very diverse portfolio of businesses, in particular in the commercial enterprise space, whether it's cloud infrastructure, productivity, applications, business applications, security, et cetera, all of them are going to be transformed by AI. And if you look at, say to your comment earlier on the Wall Street Journal post, I think if you even read further in the post, it does show the ratios for different companies. And there are some Companies that are 400% debt to equity ratios and Oracle and then there are other companies that are much smaller and that would be Microsoft. And I think we want to make sure that we're not. And I think again, based on our CapEx spend and the rate at which our CapEx spend is going up, we're not going to sit on the sidelines and not be bold as we invest. And at the same time, I think the thing that our investors expect, and ultimately I think every investor of every company will expect, is to see that revenue growing in terms of AI services and products that are being delivered in terms of net revenue recognized in a quarter and making sure that the proportionality of that to the spend and in particular to the obligations that maybe are being undertaken with debt are balanced. And yeah, that's the thing that we've been focused on. I think, you know, if you look at our last quarterly earnings, I think people were pretty pleased with the getting the balance right there. And you know, every, every quarter going forward, people are obviously going to be looking at making sure that that balance is right so that they see us investing for the long term and going to win and at the same time doing it in a way that is sustainable and allows us to kind of ride through, you know, the ups and downs that inevitably will happen over the next many years as this technology know transforms the world.

[Alex]

What are the consequences if this goes wrong with the debt? Well, you're obviously not taking on the same amount of debt. So there's a rationale behind it. What happens if. Yeah, it breaks?

[Scott Guthrie]

Well, I mean we have the ability, we're not constrained. I mean our borrowing costs ironically right now are like.

[Alex]

Yeah, but industry wide, big picture industry wide, not Microsoft specifically.

[Scott Guthrie]

Well, I think the thing that.

[Alex]

We.

[Scott Guthrie]

As an industry, I think again, you need to have that thesis of how you're going to use the infrastructure. And I would focus less on the megawatts that sometimes get reported in the press and more at where are those megawatts and what are you going to do with those megawatts? Is it going to be ultimately capacity that you can use use to serve customers? Is it to build better models that help you serve customers? And you know, what is the line of sight in terms of the product services and revenue that comes from it? And I think that's a place where again, between ChatGPT, which is the number one AI app in the world, between Microsoft 365, which is the number one enterprise AI app in the world, and between GitHub, which is the number one developer AI app in the world, I feel good that we have applications using our infrastructure and maximizing it. And I feel good about the investments we're making in terms of capital spend and build out in the right locations to continue to do that. I think not every company probably has that level of game plan and I don't think that maybe not every company is probably doing the same level of thoughtfulness of that. And at some point, you know, different companies will probably be hit by it. But you know, we're very focused on what we do and how do we make sure that we stay aggressive yet disciplined and make sure that we get that balance right.

[Alex]

All right, I want to take a quick break and then I'm going to ask you a couple technology questions about the state of the build out GPUs, custom silicon and. And then maybe we can get a little bit into this ROI question. In fact, we will. We have to talk about the ROI of AI. We'll do that right after this. Did you know your credit card points and miles can lose value to inflation? Credit card companies often reduce the redemption value of your points and miles. Now imagine a credit card with rewards that can grow in value. With the Gemini credit card, you can Earn Bitcoin or one of over 50 other cryptos instantly with no annual fee. Every swipe at the store or gas pump earns you instant rewards deposited straight to your account. Plus sign up now for a $200 Bitcoin bonus to kickstart your rewards, visit gemini.com car today. Check out the link in the description for more information on rates. Again, if you're looking to invest in Bitcoin but don't know where to start, the Gemini Credit Card makes it easy. The Gemini Credit Card is issued by Webbank. In order to Qualify for the $200 crypto intro bonus, you must spend $3,000 in your first 90 days. Some exclusions apply to instant rewards, in which rewards are deposited when the transaction posts this content is not investment advice and trading. Crypto involves risk. The Gemini credit card cannot be used to make gambling related purchases what the.

[Katie Drummond]

Hell is going on right now and why is it happening like this? At Wired, we're obsessed with getting to the bottom of those questions on a daily basis. And maybe you are too. Katie I'm Katie Drummond, the Global Editorial Director of Wired, and I'm hosting our new podcast series, the Big Interview. Each week I'll sit down with some of the most interesting, provocative and influential people who are shaping our right now. Big Interview conversations are fun. I want a shark that that eats the Internet, that turns it all off, unfiltered and unafraid.

[Alex]

So in a lot of ways I try to be an antidote to the unimaginable faucet of reactionary content that you see online. To the best of my ability, every.

[Katie Drummond]

Week, we're going to offer you the ultimate luxury of our times. Meaning and context. True or false? You, Brian Johnson, the man sitting across from me, one day, at some point, as of yet undefined in the future, you will die. False. Tell me more. Listen to the big interview right now in the same place you find WIRED's Uncanny Valley podcast. Subscribe or follow wherever you get your podcasts.

[Alex]

And we're back here on Big Technology Podcast with Scott Guthrie, the head of Cloud and AI at Microsoft. Scott, we have a discord here at Big Technology and I asked some of our members, you know, what they would ask you and we got a flood of excellent questions. And I think they were great because they focused on the technology. Some questions that I don't think you hear too often in the common conversation about this technology. So you're the perfect person to ask. I'm going to ask them to you. One of our members asked, what is the working life of a GPU and how long until they burn out. They are there use cases for GPUs once they are no longer top of the market. Right. We hear often about, okay, well unlike the laying of the fiber, the GPU depreciates after a couple years. So I think this is a, a pretty important question. Can you, can you hand that, tackle that for us?

[Scott Guthrie]

Yeah, I think kind of going back to the comments we had earlier, on balance, I think as you think about your GPU build out, one of the things that we think about is the lifetime of the GPU and how we use it. I think, you know, what you use it for in year one or two might be very different than how you use it in year three, four and five or six. And so, you know, I think that is something where so far we've always been able to use our GPUs, even ones that we deployed multiple years ago for different use cases and get positive ROI from it. And that's why our depreciation cycle for GPUs is what it is. But I do think that as we build our infrastructure, we are definitely consciously thinking about that because you don't want to have your entire fleet in two years suddenly have to be replaced because you know that, that, that would be expensive. And so, you know, we are very thoughtful on that. And again, I think I talked earlier about different training. I also think even as you think about training, we often in the past used to monolithically call training training. There's lots and lots of different training use cases now there's pre training, there's synthetic data generation that goes into training. There's post training with RL and fine tuning and other different techniques and you know, having infrastructure that's very fungible and that you can use for a variety of different training infrastructure scenarios and at the same time be used for inferencing where you ultimately enable an application to perform a query or perform an AI invocation is key. And I think that goes beyond just the GPUs, even though people often narrowly focus on that. It also is around the data center architecture, it's around the storage and the compute, that's near the GPUs. And it also really comes into play with the network because if you are for example, building one large data center that only does training and it's not connected to a wide area network around the world that's close to the users, it's hard to use that same infrastructure for inferencing because you can't go faster than the speed of light. And so someone elsewhere around the world that wants to call that gpu, if you don't have the network to support it, you can't use it for those inferencing needs. And so again, going back to kind of some of my comments earlier about how we're trying to be very thoughtful about where we place infrastructure and how we maximize the utilization, we're definitely thinking of that, not just for this year or this quarter, but thinking about it on that four or five or six year horizon for how we want to basically leverage and use it.

[Alex]

Okay, here's another question. Are there any cool technological breakthroughs that would change the economies of data centers as we know them? Now, GPU started as graphics processing units for video games. Are there resources you found that might do as well, but with fewer constraints?

[Scott Guthrie]

Well, I think one of the biggest changes that's happening right now from a data center perspective, and you're seeing this with the latest Nvidia GPUs, and I think you're going to see this in a more profound way over the next two years, is the shift from air cooled data centers where you use effectively giant air conditioning units or chillers, to a liquid, liquid cooled facility water, where you're actually pumping in water in order to cool the equipment in a closed loop circulating system. So in other words, you feed in cold water, you run it over the GPUs, effectively extract the water, cool it down again, and then do it again throughout the building. That's a massive technology change. And it does mean that older data centers that are air cooled, you know, they can't just drop in liquid cooling and be effective. And so that is something that I think everyone that's in the AI space is designing for. It needs to be thoughtful of again with their infrastructure projects to make sure that they're ready for that technology shift. It also is going to have a big difference and big impact in terms of the staffing. When you have a air cooled data center, you'd have very few employees, often per server. When you all start to involve water and liquid, it's not massively more, but at the same time it does change staffing because there are more things that break when you have pipes that are actually continuously flowing liquid into a data center. So there's a lot of technology shifts that are happening right now behind the scenes beyond the GPUs. And then obviously GPUs are the things that dominate the press in terms of innovations, both in terms of the silicon but also in terms of the network. Because at the end of the day, if you have a chip that can process a lot more information, but you don't have the ability to get that information to the chip and extract it or have it communicate with other chips, then you don't get the yield out of it. And so I think it's fascinating right now in technology, the pace at which so many things are evolving so fast, Both with the GPUs to the question, but then also the data centers, even the power and cooling infrastructure for the data centers and the network. And you know, as a technologist, it's, it's exciting times.

[Alex]

Right. You mentioned staffing. So I want to ask a follow up on that front. I think for those who don't live, those who are not in this deep into it, there is a perception that data centers, they're placed near communities, in some cases they use up a lot of water, they don't provide a lot of jobs. Is that a misconception?

[Scott Guthrie]

I think it's a misconception.

[Alex]

I mean, give us some numbers to sort of flesh out what they actually bring to a community that they appear next to.

[Scott Guthrie]

Yeah, I mean, we've talked about with our Wisconsin Fairwater site that we did some press on recently and talked about, including with the governor of Wisconsin and, and others that were attending. You know, it's thousands of jobs that we've created on the construction of the site. I think we've, we've shared over 3,000 jobs. And these are, these are very skilled jobs. These are, you're talking about electricians, you're talking about plumbers, you're talking about welders, you're talking about skilled tradescraft and high quality jobs. And I think if you look, we have phenomenal work site, phenomenal workers there, and a phenomenal safety culture which has allowed us to attract some of the, the best workers to work on that project. I think what people are missing sometimes when they say, okay, but when the project's done, how many people are going to be in the data center? And there will be hundreds of people that will be in the data center. What people are missing is the fact that right next to that data center we're building another data center. And so those thousands of people that have been working on the first Fairwater data center we just announced are now going to be starting work on the second one. And then after the second one we will do a third one. And if you look at the land and you look at the power we've accumulated in that area, it's multi gigawatts of land or multi gigawatts of power. And it's an awful lot of land. And so you're going to see us continue to employ thousands of very skilled tradescraft workers in that community. And as each one of those data centers comes online, we're going to add net new employees that will actually operate it and, and manage it. So you know, that would be an example I think of a community. And we, you know, we have over 400 data centers around the world, so they're not all that size obviously, but replicating that. And I think as more infrastructure gets built out, you're going to continue to see not just jobs created, but well paying jobs that really require trilled real tradecraft.

[Alex]

Do you feel, what do you feel the, I don't know the right word to put it, the pressure of competing with China? Because from my understanding China has a much looser regulatory approval process and they're just stacking, you know, data centers. They have abundant electricity in the United States in particular, I imagine Europe is the same way that that is not the case. But what's it like?

[Scott Guthrie]

Oh, it's certainly, I think the world has a very different regulatory approval process. I mean, I think one thing that when I talk to people and they say how can you build data centers faster? You know, there's obviously things that we can do from a technology and are doing from a technology and from a manufacturing perspective. But you know, candidly here in the US the longest part of building a data center is getting permitting. It's not actually the construction, it's making sure that you get permitting approval for all the steps that you want to take. And different states and different parts of the country have different regulatory environments. And I think even if you look at a sort of a heat map, if you will, of where data centers are being built in the US you definitely see pockets. And I would say some of that approach approximates to where there is land and where there's power. And some of it really, you know, closely correlates with where it is easier or faster to kind of complete the permitting process. You know, in Wisconsin we had a, you know, phenomenal partnership with the governor and the local county. We were able to purchase some land and power that a manufacturer was previously going to use and they, they pulled out of a project. And so I think the local communities recognized if they weren't able to work with us, they were going to lose jobs and have impact on the community. And they leaned in with us and can't say enough positives in terms of the speed with which we went through all the process, we got all the approvals. It was a very thorough process, but it was streamlined so that we could move fast and that we could actually help ensure that jobs weren't lost and that instead they were created in the community. And I think there's more opportunities for public private partnership like that that we'd certainly welcome as part of it.

[Alex]

Okay, another discord question. What time frames are you looking at to get ROI on these investments? So when would you stop investing if prior investments weren't showing returns? And how do you know when it's time to stop building?

[Scott Guthrie]

Well, I think, you know, at the end of the day, I think, you know, every, every quarter we share our revenue growth and we share our CapEx spend. And you know, to some extent, I think the, you know, markets keep companies honest in terms of that balance. And you know, sometimes markets can be slightly irrational at times, but in the limit, the markets keep you honest. And, you know, that's a big part of why we focus so much on making sure we get that balance right, make sure we're again, investing for the long run. I don't think anyone, if you look at our CapEx spend and our, our commitments and our investments would say that we are not being bold. But at the same time, you know, we have a report card every quarter where we need to kind of demonstrate and prove, not just with press releases, but, you know, here's how much revenue growth we had. You know, last quarter we grew Azure 39% year over year on a very large number. And a lot of that was driven by AI and then also driven by the other systems that come with AI, because there are databases and there's compute and storage sold with that AI. And you know, I think investors were happy with the, both the spend and the aggressiveness that we were building out, but also the return. And, you know, I think that's going to be true, you know, forever. And, you know, making sure you get that balance right. And, and again, as part of that balance, markets want to know you're investing to win the long run, and at the same time they want to make sure you have some level of discipline. And, and I think our portfolio, the, the balance that we have, both across the, the products we build, but then also the fact that we have the largest AI product in the world called ChatGPT, running on top of our cloud, gives us a unique opportunity to get that balance and that growth and that investment right.

[Alex]

Is ChatGPT, by the way, going to stay on azure, even though OpenAI is making these partnerships with Nvidia and Oracle.

[Scott Guthrie]

Yes.

[Alex]

Okay, all right, it's good to get something definitive on that. Um, you know, you mentioned your 39% Azure growth and you know, I'm, I'm looking at your quarterly numbers every, every quarter and often talking about them on cnbc and the numbers are, are massive. And the other side of it though is so that spend coming from clients. Right. And there have been multiple studies that have come out recently that have talked about how enterprises aren't getting the ROI that they've anticipated on their AI projects. Yet. When you see those studies, do they ring true to you? How do you react to them?

[Scott Guthrie]

Well, I think, I think when you say AI in general, it's a very broad statement.

[Alex]

This is obviously, I mean not obviously, it's in large part this is generative AI where companies everywhere have tried to adopt LLMs and try to put some version of that into play in there. And it's not recommender engines basically.

[Scott Guthrie]

Yeah, but I think what you need to do is double click even further From Genai to GitHub Copilot or Healthcare or Microsoft 365 Copilot or Security products built with Genai. I do think ultimately the closer you can kind of double click on is this really delivering roi, then you have much more precise data. Because I do think a lot of companies have dabbled or done internal kind of, I'll call it proof of concepts. And some of them have paid off and some of them haven't. But you know, and, but I think ultimately a lot of the solutions that are paying off that we, we continually hear from our clients and our, and our customers is, you know, a bunch of the applications, for example, that we've built, I think similarly, you know, a bunch of the applications that our partners have built on top of us. And you know, ultimately the Azure business is, you know, we get paid based on consumption. It is, it's a consumption based business, meaning if people aren't actually running something, we don't get paid. It's not like they're, they're pre buying a ton of stuff. You know, we recognize our revenue based on when it's used. And so you know, the good news is when you look at our revenue growth, it is, you know, it's not a bookings number, it's actually a consumption number. And, and you can tell that people are consuming more and you know, the last two quarters, our revenue growth's accelerated on a big number. And that is, you know, a statement of the fact that I think people are getting a lot of roi, at least with the projects that they're running on top of our cloud.

[Alex]

Yeah, I think that's an important point to bring home. It is consumption based. So you talked a little bit about water cooling versus air cooling. I love the term for the air cooling. It's called chillers. That's what my friends in high school called ourselves, you know, back in the day. And I want to end on the GPU side of things or the silicon side of things. What do you think the potential is for custom silicon in the AI world? I mean like we talked about previously, GPUs were designed for gaming, happened to do parallel processing, actually ended up being really good for large language models. The training and the inference. What's your perspective on whether this industry is going to continue to run on that type of chip and what the potential is for custom silicon?

[Scott Guthrie]

I think a couple things. I think one is, I think increasing the number of tokens you can get per watt per dollar is going to be the game over the next couple years. And maximizing the ability of our cloud to deliver the best volume of tokens for every watt of power, for every dollar that's spent where the dollar is spent on energy, it's spent on the GPUs, it's spent on the data center infrastructure, it's spent on the network and it's spent on everything else is the thing that we're laser focused on. And there's a bunch of steps as part of that GPUs being a critical component component of it. And you know, one of the things that our scale gives us the ability to do is to invest for kind of non linear improvements in that type of productivity and that type of yield. You know, if you've got, you know, a million dollars of revenue on a couple hundred GPUs, you're not going to be investing in custom silicon. When you're at our scale, you, you will be. And you're not just Investing in custom Skillicon for GPUs for pre training or for inferencing. You're looking at what can we be doing for synthetic data generation with silicon? What can we be doing from a compression perspective with custom silicon, what can we be doing from a security perspective? And we have bets across all of those, many of which are now in production and are actually powering a lot of these AI experiences. In fact, I think every GPU server that we're running in the fleet right now is using custom silicon at the networking compression storage layer that we've built. Now, the GPUs themselves are also going to be a prize that people are going to try to optimize like the actual instructions for doing the GPUs. Nvidia is a fantastic partner of ours. We're probably one of, if not the biggest customer in the world of theirs. And we partner super deeply with Jensen and his team, you know, at the same time. And partly why they're so successful is they're executing incredibly well. You know, at the same time, if you look at the history of silicon, not every silicon company, or it's rare to have a silicon company that every single year is doing the absolute perfect work that's differentiated. And kudos to Jensen for what he's done. And I know he's going to keep trying to do it going forward, but, you know, there will be other opportunities from other companies where people are going to look for a niche that's going to be big enough in this AI space to be truly differentiated versus what Nvidia is delivering. And then we're doing our own silicon investment in house. So because we're going to be going after those same opportunities and ultimately, the way we've tried to build our infrastructure, none of our customers know when they're using Microsoft 365 or GitHub or any OpenAI models, what silicon they're running on. And we're going to be constantly tuning the use cases based on the applications. If we find ways that are breakthroughs, we're absolutely going to be taking advantage of them for those use cases. And again, at our balance of scale and our balance of use cases, I'm very confident that we're going to find use cases where custom silicon will make a difference. And I'm also very confident we're going to continue to be a great partner to Nvidia and others in the world that are going to be selling us great solutions.

[Alex]

All right, Scott, I want to end on this because I'm always, I've always been curious about the human aspect of this. Like you're going out and working on designing your own chips that are trying to be, you know, better than GPUs for certain parts of this AI application layer and training. And then your. You said one of Nvidia's biggest customers, if not its biggest customers. So is this like a situation where, like, you go to Jensen and you're like, we're going to just both give a shot at building this stuff and may the best chip win? And it's Friendly, like friendly competition or is there any awkwardness in there because you're like kind of building the thing that is making them the most valuable company in the world?

[Scott Guthrie]

Well, I think probably different companies handle that differently. I think the nice thing about Microsoft is a, we've been around a while and I think also we compete almost in every market in some way, shape or form. So there's none of my partners that I'm not also a competitor with. I think it's probably a true statement. And the important thing is I think you have that enterprise maturity to be able to recognize. I want Jensen to do the best possible work because it's going to benefit me. And we've leaned in. We were the very first cloud to deliver live GB200s, you know, which is a massive architectural shift for Nvidia. That's the first of their liquid grace Brackwell. Yeah. And we were the latest chip, the first one running, first rack running, the first cluster running, the first data center running of any cloud or NEO cloud provider in the world. And so, you know, that's an example where we really leaned in and moved at the speed of light together. And we're going to continue doing those types of projects. And at the same time, you know, he recognizes and understands we're going to be doing lots of things. And I also recognize he's going to work with other providers as well. So I think the ability to kind of keep a complete thought and recognize it's not zero sum on every single decision and that at the end of the day, you know, it's a market, we're all going to compete and we're also going to partner. And, you know, I think we have the maturity at Microsoft to do that. Again, the balance, I think I've said balance multiple times. I do think balance in life, but especially in business and especially in technology, that is the devil's in the detail. But if you can get that right and do it consistently, those are the companies that win and those are the companies that really, you know, have the ability to set the agenda and that's what we're focused on.

[Alex]

Well, Scott, I just want to say thank you for taking the time. I know you don't do this often, so I appreciate why. Why did you say, okay, hey, I want to come out and speak about this today?

[Scott Guthrie]

Well, a bunch of people internally said, hey, you got to talk to Alex.

[Alex]

And so that is always a good advice to follow.

[Scott Guthrie]

Okay, so it's, it's fun to, fun to get a chance to do and really Look, I really enjoyed the conversation.

[Alex]

As did I. Yeah. Thank you again for taking the time again. And I know it's rare for you to come out and speak about these things. I, uh. You're running a massive, massive and fast growing business, and so it was great to be able to speak with you and get into peak. Get a peek into it today and look as to what the rest of the industry is doing and your perspective on that. So thanks for coming on the show, Scott. Appreciate it.

[Scott Guthrie]

Thanks for having me, Alex.

[Alex]

All right, everybody, thank you so much for listening and watching. We'll be back on Friday to break down the week's news with Max Zeff of TechCrunch. It's going to be a great episode. We hope to see you there. Thanks again and we'll see you next time time on Big Technology Podcast.

[Katie Drummond]

What the hell is going on right now and why is it happening like this? At Wired, we're obsessed with getting to the bottom of those questions on a daily basis, and maybe you are, too. I'm Katie Drummond, the global editorial director of Wired, and I'm hosting our new podcast series, the Big Interview. Each week I'll sit down with some of the most interesting, provocative and influential people who are shaping our right now. Big Interview conversations are fun.

[Alex]

I want a shark that.

[Katie Drummond]

That eats the Internet, that turns it all off, unfiltered and unafraid.

[Alex]

So in a lot of ways, I try to be an antidote to the unimaginable faucet of reactionary content that you see online. To the best of my ability, every.

[Katie Drummond]

Week, we're going to offer you the ultimate luxury of our times, meaning and context. True or false. You, Brian Johnson, the man sitting across from me. One day, at some point, as of yet undefined in the future, you will die. False. Tell me more. Listen to the Big Interview right now in the same place you find Wired's Uncanny Valley Podcast. Subscribe or follow wherever you get your podcasts.