← AWS Podcast
AWS Podcast
#745: Accelerating Cloud Migration: How Occidental Petroleum Transformed with Terraform and AFT
00:20:19
This episode features a deep dive into Occidental Petroleum’s cloud migration journey, emphasizing a
Loading summary
Transcript49 lines
- [00:00]
A
This is episode 745 of the AWS podcast, released on November 10th, 2025.
- [00:09]
B
Hello, everyone, and welcome back to the AWS Podcast. I'm Alish here with you. Great to have you back. And I'm joined by two very special guests today. Firstly, I'm joined by Brian Moore, who is a cloud architect with Occidental Petroleum in Houston, Texas. Occidental is also known as Oxy. He leads Oxy's cloud computing practices and has spearheaded strategy and adoption of AWS at Oxygen. He's also passionate about evangelizing DevOps culture and finding better ways to automate everything. Sounds like a man after my own heart. Welcome to the podcast, Brian.
- [00:35]
A
Thank you. It's good to be here.
- [00:37]
B
It's good to have you here. We're also joined by Chris Spitzenberger, who is a senior solutions architect here at aws. Also worked very closely with Oxy on this and knows a bunch of stuff that's really relevant. Chris, welcome to the podcast.
- [00:51]
C
Thank you. Also glad to be here.
- [00:52]
B
Awesome. Now we're going to talk about some cool stuff today. We're going to talk a bit about a cloud transition, sort of an approach that was taken rapidly and at pace. But before we do that, Brian, tell us just a little bit about Oxy for our global audience. So they understand a little bit about your organization, what you were dealing with in terms of the transformation. Sure.
- [01:14]
A
So yeah, Oxy is a oil and gas company based in Houston, Texas. We also do chemicals. There's Oxychem is another subsidiary. So like, I think we're like the largest PPC producer in the world. We also do carbon capture business. So that's another subsidiary. So yeah, we have operations all around the world like Oman, Algeria, South America. So we're definitely have a global footprint and that's where the need for digital transformation around the globe really drove the need for us to move into the cloud. Because traditionally we were all.
- [01:53]
C
For a very long time.
- [01:54]
B
Interesting. And I guess that's the thing is that, you know, it's a very common, common situation that you grow to a certain level and you do, you do a lot of great work, especially globally, and the overhead of managing all that becomes quite challenging, I'm assuming.
- [02:08]
A
Exactly. Yeah, exactly. And then, you know, the disruptivity, moving to a more scalable way of doing things that also create a lot of challenges for us in just terms of skilling and the way we work, you know. So I know we started out in Azure initially and, you know, that's where we got started. In Terraform. But it was like it was a very small team operating that way for many years. And like, you know, we decided to move to AWS and that created this sort of call to action. Hey, we need to make this same transformation across the enterprise, across all of the IT department. So that's where we start going. Okay, we need to find a more scalable way to set up all these AWS accounts. For example, we have a really amazing data architect, Jamie Johnson. He's driven the adoption of this data mesh architecture. But it requires tons and tons of AWS accounts and we needed to solve problem. Like we're expecting so many, like hundreds of AWS accounts over time. And we need to automate this process. We need to make sure we can keep AWS accounts consistent, have a repeatable pattern and be able to maintain that pattern and iterate on it. And so that's where hey, we do everything with Terraform. We love Terraform. We try to apply Terraform where we can. And so there's an offering AWS account factory for Terraform where you have a terraform framework for calling control tower to spin up an AWS account but then also provision all these baseline resources. Depending on what kind of account it is, you know, whether it's, you know, more public facing or internal facing, like you might get a different network architecture, you know, production versus sandbox, et cetera. So we've got a very useful, you know, bringing our existing Terraform skillset to bear in using that.
- [04:02]
B
Let's talk about that because you've obviously mentioned the fact that you know, you're very passionate about automating things, et cetera. And it's always, I guess always tempting to write, write your own tooling and your own scripting, et cetera. And you've adopted Terraform, which I know a lot of organizations have as well. What was, you know, there's many pathways and none of the pathways are wrong or right. They're just different pathways. What, what helped you choose, you know, Terraform? This approach and also I guess the multi account approach you were taking as well.
- [04:29]
A
Yeah, I mean with Terraform it's like, it's, it's sort of the, the key to the city in the cloud. Like everyone does terraform like you, you, you learn Terraform. It takes you to so many places and like if you try to do like an in house grip because I know our IT department has gone down that path of like oh, have an internal framework to do something and like it's bit them forever because like they have to Maintain it and like, yeah, it's, it's, it's hard work. Lessons have been learned in the past of, let's, let's not go down that road again. Let's, let's, let's go with the gold standard. Let's go with what, what we know works, what we're going to support from aws and like, we continue to get new releases of AFT from aws, new features, bug fixes, and so it's a big load off of our shoulders. I think. Actually Chris had initially given us the idea for aft.
- [05:17]
B
Is it all your fault, Chris? Oh, yeah, it probably is.
- [05:22]
C
I was working in AWS professional services at the time. We're a team working with Oxy on a massive. The mass migration project. I believe we were those two ultimately recommended aot. It's a solution maintained by the AWS Control Tower service team and of course allows you to use Terraform. Like Brand said, pretty nifty work.
- [05:45]
B
Well, yeah, I think the interesting thing here is there's always this sort of tension, for want of a better word, between using a generalist tool that can work across everything versus getting access to key capabilities of a particular platform or cloud or capability or what have you. And so I guess with this, Brian, you've been able to sort of combine the both. Whereas, you know, it's hooking deeply into what you can do within Control Tower, et cetera. But from a user perspective, it's, it's terraform. It's like, it's, it's familiar.
- [06:17]
A
Yeah, yeah, it's like, it's, it's a, it's a skill that we know, but it's a general framework to run any code that we want. And AFT has lots of different hooks, like, I know. So we have another team that maintains, you know, Azure DevOps and like, okay, you want Git repository and Azure DevOps project. I mean, yeah, I know Azure DevOps is kind of old, but we'll move to GitHub eventually. But we have like, code landings. I was like, hey, you know, if someone wants to get started with, you know, Git repository projects boards and like, okay, connections from that to your AWS account. Like, you know, have an automated process to do that. Well, that's where we had. Okay, AFT has these, these lifecycle hooks. They're called provisioning customizations. But that's where you basically can add a step function. Say, hey, you know, a new account got created and we just hooked it up to Hashicorp Vault. That's another product we have and, you know, need to communicate that to the platform engineering sort of DevOps framework, um, so they can hook things up on their side. So there's all kinds of opportunities. Hey, okay, yeah, here you control the baseline stuff that gets stubbed in the account, but then also notifications, events via forwarded to other places as well.
- [07:30]
B
So. So rather than being, I guess a generic thing, it's, it's really letting you have that flexibility to give your users what they want and what they need.
- [07:38]
A
Yes, yes. Yeah, like there's definitely been like times where we've had to customize aft a little bit just in terms of. Okay, like the baseline was kind of assuming, you know, it's mostly assuming that you're working within the same region as the like the control tower home region. It was like, okay, you know, this would work a little bit better if I may. So. Oh, when I specify my ABIS account, I want to just arbitrarily say, oh, well it's going to be set up in these, this many different regions with this many different default VPCs. And that's what we had to do, like a little bit of finagling. Okay. The information that the code build passes to the terraform configuration. Let's mess with the provider TF a little bit to make that easier. Otherwise do what you want to do.
- [08:22]
B
So let's talk about, I guess from a process perspective you sort of touched on the fact that you can kind of vend things pretty quickly and what people need. But what does this look like in terms of I guess the speed and efficiency of the way oxy works and the migration to the cloud? Like what did it help with from an end user perspective?
- [08:40]
A
From an end user perspective, like there were days when like hey, this one team I'm talking about, the, it's called ODAF is the data mesh architecture, but like someone would literally request like 10 AWS accounts or something and like, okay, you know, we're waiting on this to set up, you know, dev stage prod accounts for a couple of different groups and you know, like we could turn around very quickly just that, okay, got your requirements and just basically copy pasting some, some terraform configuration code where we define our accounts and it's just very easy. We actually have some, a little bit of code generation even for that to make it really streamlined. So like it was just very rapid turnaround and not only, not, not just initial standing up because sometimes you know like you're not going to get everything right the first time. Okay. You know, we started out with decentralized VPC endpoints where we had VPC endpoints set up in individual accounts. But then it's like, okay, well we need to migrate to more of a centralized architecture to save cost because we didn't want to pay for, you know, so many VPC endpoints In every single APIs account, whether it was being used or not. Okay, let's consolidate into a more centralized approach. I think it's more best practice. And then that, that created an impact for okay, well now we got this new version of the account pattern and now we got to roll it out and then we've got like, you know, at that point if you have 150 advanced accounts or weeks now we got to rerun the apply, you know, across all of those. They made it really easy.
- [10:11]
B
That's amazing. That's. And I guess the other thing is also from an organizational perspective, one of the most frustrating things of doing any project is waiting for other teams to do what they need to do. So not being that sort of bottleneck of, you know, I can't move forward till my accounts are created, but my accounts are taking too long. Like you're just eliminating that frustration.
- [10:32]
A
Yes, exactly, exactly. So it, I guess in our relationship, in practice it kind of moves the bottleneck elsewhere, but at least it's not us.
- [10:40]
B
Well, look, life is moving bottlenecks around really. But I know exactly, you're eventually trying to, trying to smooth things out as well as possible. In fact, one of the things that I thought was interesting was you touched on that piece of not wanting to have sort of too many networking components going on and you really automated this a lot. How did you use AFT to standardize your networking? Was it just don't create these things or was there more robustness that you want to build in about the network.
- [11:10]
A
Strategy, whenever we define the AWS account we say, okay, this is going to be a public facing account. We call a certain pattern. Is this a sandbox account? We call a certain pattern. But basically each one of those patterns calls a our terraform modules. So we use Terraform Cloud, those terraform modules. And the modules will define like the standard networking terraform. So like okay, I'm going to make a vpc. It's going to have you know, three availability zones by default it's going to have a subnet, you need to availability something like a private subnets, transit gateway subnets. So we kind of planned out this standard network architecture for an application account.
- [11:50]
B
So basically, basically it's making it easy to just repeat, rinse and repeat and I Guess Chris, maybe you want to jump in here and talk about what was going on with the networking component, but also maybe a little bit more about AFT and why you even talked about AFT with Oxy.
- [12:08]
C
Yeah, definitely from networking perspective. That's one of my favorite things about working with the aft. When you're stamping out these accounts. What are the fundamentals, things you need when working in abs? Typically, maybe not always, you know, as a VPC and subnets, if you want to work with compute resources, typically you need those things. But anyways, one of the things we were doing with AFT through the customizations feature, we can mention this. Different patterns, there's the terminology in AFT is customization. So we had different customizations for different scenarios. We first deploy the VPC and deploy the subnets, but what was happening is we're actually leveraging the AWS IPAM IP address management feature in dynamically leasing IP space for the PC, using that lease space and subnetting it out. So these things were happening dynamically. So basically at the account request level, you wouldn't have to worry about the management of all your overlapping ciders. No more spreadsheets. You can just say, I want to 23 DPC. And then even as simple as that, passing it into our module, which then knows how to interpolate that into subnets, you know, and break it down. But taking that a step further, something that I really am proud of that we did is we actually set up an integration with Palo Alto Panorama. So, you know, based on account tags, we could actually create firewall rules for that VPC and basically tag things and dynamically set up that entire VPC into our network inspection process, which is a pretty cool integration. So not only were we, you know, deploying the vpc, deploying the subnets, you know, all the other standards that we're deploying with the account, but we're also, you know, setting things up in Panorama. So you know, we are know, you know what to expect and you know, we can lock down that traffic even further.
- [13:59]
B
So really secure, secure by default is a beautiful thing. Yeah, yeah.
- [14:05]
A
Sometimes we've encountered challenges like the, the Panorama integration is an interesting example because so like the Panorama integration is great. And then recently there, there was sort of an incident where, you know, someone didn't really realize that the ciders were being integrated and managed by Terraform. And so like someone was in the Panorama console doing Click Ops. Oh well, I need to move the device group around. Like, you know, we got to rearrange things. So like someone's like sat down and spent like four hours in the console moving like 600 objects in the Panorama. And like it just completely broke any connection. Like now, now Terraform. And this is across hundreds of workspaces because every account is two workspaces, but one of them. I know I didn't tell you about this, but it's caused so much pain. But like, make sure everyone knows that it's managed by Terraform. Like, lock it down. We're actually looking at some way in Panorama to like define a policy. Like, hey, like, you can't touch this. Like, we can't, we can't make those teams use Terraform. Yeah, we don't manage everybody's stuff, but like, at least, you know, lock yourself down from being able to manage this because it's being managed by Terraform.
- [15:12]
B
Well, I guess the good thing is the fact that it is managed by Terraform means you know what it looked like before someone click opt it.
- [15:18]
A
Yes, yes, exactly. That that has helped.
- [15:21]
B
And I guess, Brian, this touches on something I wanted to ask is, you know, what are some of the lessons you learned about, I guess, cloud adoption and Terraform and Aft that others could benefit from? Obviously, you know, don't click Ops or reduce click Ops as much as possible, which is a good practice anyway because I often say if you're doing it in the console, you're probably doing it wrong.
- [15:40]
A
Yes. Yeah. So like, you know, well, they first thing block people from crushing the resources that AFT manages. So like we introduced a tag managed by aft. So there's an SCP that basically I think it's permissions boundary. I think we put in the permissions boundary. You can't touch anything that's managed by aft. So yeah, I talked about the provider. So like, I think there was a recent enhancement to the AWS Terraform provider where with a single AWS provider you can do multiple regions. But historically that hasn't been the case. Like for every region you want to use, you had to find a separate AWS provider block. But like, you know, that was so recent that we couldn't really benefit from it because instead we were having to do like dynamic Jinja code generation to generate different provider blocks for every AWS region.
- [16:30]
B
And I guess that's, that's the classic thing of, you know, working in the cloud and working with cloud based tools is they evolve, they evolve. When people say, hey, I don't want to do it this way anymore, and it's not like, well, sucks to be you, it's like, oh great, we'll Make a change and get a new version that has, has what you need.
- [16:46]
A
Yeah, but at least, you know, having things defined as code makes you more resilient to change and able to manage change.
- [16:52]
C
Yeah.
- [16:52]
B
And speaking of that, you know, culturally, how have you worked with within Oxy to help other folks kind of embrace this passion for automation and repeatability, et cetera? What have you found works in terms of helping people adjust, maybe their mindset around how to operate?
- [17:13]
A
Sure. Yeah, I've had like lunch and learns and presentations that I've given to people. Um, that, that helps at a high level of. Okay, from a high level. Where are we coming from? It helps as an icebreaker. I found people say, oh you, you've explained it in a way that made sense at a high level, but at some point you have to get hands on keyboard and that usually in my experience that evolves just like sitting down with someone and kind of pair programming with them, helping them, hey, you know, okay, you, you need, you need to build something. Let's sit down together and do it. Because you know, unless you, you learn to do it yourself, you're never going to take ownership of it. But so we, we had a lot of, I guess failed starts in that like people thinking, okay, well we'll just get this, get this done really quickly and then they'll, they'll keep it going. But then it kind of thistles out and maybe comes back to the, to the, to the DevOps team, which I think is kind of an organizational anti pattern. But it's, it's, it's a lot of work. So we've had, you know, sort of an enterprise infrastructure as code program where we train people on how to use our, our code landing zones. The code landing zones is basically a framework for, you know, getting everyone bootstrapped with like a best practices terraform repository that has a connection to your AWS account and has pipelines that are deployed to aws. So yeah, there's just been a lot of training.
- [18:34]
B
Well, yeah, it's, it's kind of like, you know, you've got to show people and see what the difference is. And it's, I think if you can show people that it makes their jobs easier and faster, you're gonna get adoption. If it's just seen as more complicated, then you're not gonna get adoption. So that's kind of like, yeah, you.
- [18:52]
A
Gotta prove the value and it takes time and there's, there's been a lot of skepticism but over time like it, it wins out. Like it's just cannot manage all this stuff at scale.
- [19:02]
B
That, that. That's, you know, that's the point, isn't it? At some point, there's. There's too many things to look after. And if you're trying to keep it in a spreadsheet or in your head, you're going to make mistakes. Like stuff's going to go wrong.
- [19:13]
A
Exactly.
- [19:14]
B
Stuff always goes wrong. But if it's in source control, it makes life a lot easier.
- [19:19]
A
Yeah, you can roll it back.
- [19:21]
C
Exactly.
- [19:24]
B
Brian, thanks so much for coming on the show and sharing us a bit about your journey. It's always great to hear from customers who can tell the story from a position of reality. We did it, we used it. It had benefits. There was challenges. We overcame the challenges. Welcome to it.
- [19:41]
A
Thank you.
- [19:43]
B
And Chris, thanks for sharing with us the genesis of the idea. And it must be quite satisfying for you to see a customer like this, really automating and moving forward with this technology.
- [19:53]
C
Yeah, it's amazing. I'm really proud of seeing them go from zero to 100, quite literally, really. A hundred plus, 300 plus at this point.
- [20:04]
B
It's fantastic. And so thanks a lot for coming on as well. And for those of you who want more information, there's links in the show, notes about how aft works and things you might want to think about. And of course, until next time, keep on building.