
is a Team Lead at and a contributor and maintainer of the CNCF Sandbox Project, KCP. KCP is an open source horizontally scalable control plane for Kubernetes-like APIs. Do you have something cool to share? Some questions? Let us know: - web: ...
Loading summary
Abdel Sigewar
Hi and welcome to the Kubernetes podcast from Google. I'm your host Abdel Sigewar.
Kaslan Fields
And I'm Kaslan Fields.
Abdel Sigewar
In this episode we speak to Marvin Beckers. Marvin is a team lead at Kubermatic and contributor and maintainer of the CNCF Sandbox projects kcp.
Kaslan Fields
But first, let's get to the news.
Abdel Sigewar
Docker announced their subscription prices are increasing starting November 15th. Docker Pro is moving from 5 to $9 and Teams is moving from 9 to $15 per month. The price for Docker Enterprise remains the same.
Kaslan Fields
The Linux foundation announced intent to form a Developer Relations foundation, or DRF for short. The new foundation aims to address some of the challenges devrel organizations face across the industry, like lack of clarity, recognition and the difficulty of measuring impact. The DRF will partner with industry leaders to address these challenges and already received support from multiple companies.
Abdel Sigewar
NetApp Insights took place September 23rd to 25th in Vegas. NetApp CEO George Kurian emphasized the importance of data for AI workloads. The company is integrating AI functions into its storage platform, enhancing interoperability to eliminate data silos and optimizing storage for cold data. Partnerships with hyperscalers like aws, Azure and GCP and and Nvidia are highlighted as key to delivering AI solutions across diverse workloads, emphasizing cost optimization, security and sustainability in data storage.
Kaslan Fields
And that's the news.
Abdel Sigewar
Hello folks. I'm today with Marvin Beckers. Marvin is a team lead at Kubramatic and the contributor and maintainer of the CNCF Sandbox projects kcp. Welcome to the show, Marvin.
Marvin Beckers
Thanks for having me.
Abdel Sigewar
So before we started recording, you told me that KCP does not stand for Kubernetes control plane, which is the thing that I was. That's how it was described to me the first time I heard about it. So what is kcp?
Marvin Beckers
So it's not the like, it's not an official acronym, but I think the description is kind of spot on. So it's a cloud native control plane for Kubernetes like APIs. So you get the idea, even if it's not what KCP stands for.
Abdel Sigewar
Got it. Okay, so a cloud native control plane for Kubernetes like APIs. So then the next question is, what is actually KCP? Like what does people use it for?
Marvin Beckers
KCB is basically the idea that the Kubernetes resource model, like all the engineering that went into the Kube API server server, it has become more useful than just using it for container orchestration. So it's like a decade now of, well, engineering and technology that went into it. And KCP is basically the idea that this is extremely useful for hosting any sort of API. And I think in the ecosystem we see this already with a lot of projects that don't really use it for container orchestration. They use it as a, well, a control plane. If you think of projects like crossplane, for example, that uses Kubernetes as a control plane and builds on top of the API, not necessarily on top of the container orchestration part.
Abdel Sigewar
So the next question becomes why would people use KCP as a control plane instead of just using Kubernetes? Right.
Marvin Beckers
KCP has several features built in that are well primed for this specific use case. So because KCP thinks about APIs and not primarily about container orchestration, KCP has ways to manage APIs at scale. And KCP also has a multitenancy concept that you don't find in Kubernetes, which is called logical clusters. And logic clusters basically allow you to serve multiple Kubernetes API endpoints or something that looks like a Kubernetes API endpoint from the same KCP instance. So this way you can basically talk, or you can basically get a lot of Kubernetes API server endpoints with very little cost.
Abdel Sigewar
Interesting. So you mentioned control plane as an example, and that was one of the. I mean, yeah, K.M. i've seen Kram before, the Kubernetes resource model. And when you come into the context of anything that does something like cross plane, then the question that naturally people ask is like, do you actually need to run. And I think you answered this question partially, but do you actually need to run the entire Kubernetes control plane with the API server scheduler and all that stuff just to run cross plane? And it seems to me like KCP is the answer to this question, like, if you don't need the container orchestration part, if you don't need the container runtime part, you could just run kcp, which gives you the control plane part, Right?
Marvin Beckers
Yep, that's exactly the idea behind kcp. It's basically Kube without the container orchestration.
Abdel Sigewar
Interesting. And then you have the multitenancy part, which basically, I mean, I'm assuming here, correct me if I'm wrong, is that only for cloud providers trying to offer things on top of kcp, or would that be useful also for, like, end users?
Marvin Beckers
I mean, we consider KCP kind of a foundational technology, sometimes call it a framework, but it's not like a programming framework. So that sometimes confuses a bit but basically, well, it's an API server, it's a control plane. So what you use it for is kind of up to you to build it and what you want to build with it. So for example, if you build a ZAs, like what you say, like a cloud provider or like a ZAS provider, they could basically start with KCP with a strong multi tenancy setup and go from there. We could also think of this in terms of being useful for things like platform engineering, where basically you, well, you're not a cloud provider, but you're building an internal platform for developers or other people in your organization to use. So this is kind of like two of the, I think several use cases that you could like bring KCP into.
Abdel Sigewar
Nice. Interesting. Okay, and I, I just forgot to say this as the beginning of a retroduction. If the sound looks a little bit weird compared to how people are used to hear it is because we're actually recording at a conference. So we're here at Container Days Hamburg. It's my first time at this conference. It's pretty cool. We can probably talk about it at some point, maybe, if you want. But let's just stay on the KCP topic. So I went to the documentation, I did some reading, I read some terminology and it's a little bit confusing, but it's something you would expect from, as you say, like a building framework, like something that is meant for people to build stuff on top of. So let's go through these basic concepts. What is a logical cluster? Let's start there.
Marvin Beckers
Yeah, so a logical cluster, it's basically a way to partition like your API server. I'm not going to say Kube API server because it's not exactly the Kube API server, but something built on top of that. It's a way to partition basically the storage layer, so ETCD in most cases and your access to the API server into multiple ones. So basically each logical cluster that you talk to, it has its own space in ETCD in the key value store. And, well, it has its own endpoints that you can talk to and it acts like a single standalone KUBEAPI server.
Abdel Sigewar
Even if behind the scene it's only one instance of kcp.
Marvin Beckers
Exactly.
Abdel Sigewar
You would be able to expose as many endpoints using logical clusters because there is one endpoint per logical cluster, right?
Marvin Beckers
Yep, exactly.
Abdel Sigewar
And then how does the multitenancy at ETCD done? Is it like, do you add a value to this? Like, how is that done?
Marvin Beckers
Yeah, so there's a prefix basically that is, I think, prepended and that basically includes, okay, which logical cluster is this data related to? And then the API server kcp, it takes that information and returns different data depending on, well, what logic cluster you're talking to.
Abdel Sigewar
Got it, got it. And then does that mean that every logical cluster implements the full permission model of Kubernetes?
Marvin Beckers
Yeah, so each logical cluster has the extend, well, an adjusted authentication and authorization layer of Kubernetes that we all know. But because each local cluster has its own objects, that means that in each local cluster you will have different airbag being applied to your request. So the cluster roles are, well they, I mean they're cluster scoped in like normal Kubernetes terminology. Right. In this case like we could call them like workspace scoped. So cluster roles are different from works, sorry, from logic cluster to logical cluster. And that basically gives you different permissions depending on what logic cluster you're talking to.
Abdel Sigewar
Got it, got it. Interesting. And so then you also have a concept of physical cluster. So what is the physical cluster then in this case?
Marvin Beckers
Yeah, so this is something that I think we want to talk a little bit about. Because KCP at some point had like a bigger scope, it was basically looking more at solving the multi cluster script scheduling use case. Since the project is part of the CNCF sandbox, we have decided to scope down a little and basically focus on the control plane API parts. So in kcp, physical clusters, and I guess to some degree they still do, well, it's kubernetes clusters that you can run workloads on. So that is basically the idea because KCP is only a control plane and because we took away the container orchestration part of it, if you want to execute software in the Kubernetes model, well, you still need something powering that. So in this old component of KCP that at the moment is well, frozen, that is called transparent multi cluster, physical clusters were basically your workload clusters where you were scheduling your multi cluster deployments or something that were then spread around multiple clusters. That is no longer the case. Okay, but of course you still need to think about physical clusters because KCP is only, well again like a control plane. So that means if you want to run software against it, and you need to have your own controllers, your own operators maybe, then of course you still need kubernetes clusters that power that. So in a sense, like physical clusters are not a formal term anymore, but you still need to think about them.
Abdel Sigewar
Got it. And so you would, I mean, I'm talking about the previous scope, right? You would still rankcp creates logical clusters which will give you this, like separated control planes, let's say, or endpoints. And then you would make a physical cluster connect to a logical cluster, Right? Well, make a physical set of clusters and by physical it means virtual machines or whatever. They will dial up to the endpoint which is part of a logical cluster.
Marvin Beckers
This is basically one of the patterns you can implement. So because in KCP, like, all we think about is APIs, but if you define an API where you would say, okay, I want something to happen and most APIs are doing something, then you can basically implement, I don't know, a syncing component or controller that takes the information from the central control plane that you're running with KCP and distributes that to physical clusters. And that can also be higher level resources we don't necessarily need to create deployments or something. In kcp we can say, for example, let's say you want to implement providing databases. So then in KCP you would create like a database API. You would create that and then it would be synced down to physical clusters where like the actual database instances then would be started.
Abdel Sigewar
Got it. Does that mean you could. I mean, this is like a hypothetical example. You tell me if I'm hitting on the right nail here. Would that mean that you could potentially use KCP if you are a cloud provider that runs managed kubernetes, so you could use KCP as the control plane to create user clusters?
Marvin Beckers
I think that's definitely completely in scope for it, yeah.
Abdel Sigewar
Using something like the cluster API, for example, you would run kcp, run the cluster API on it, and then when a user goes like give me cluster, then that becomes a YAML file that you submit to kcp, which then provisions a kubernetes cluster with all its components.
Marvin Beckers
Yeah. So usually implementations, they need to be KCP awareness. So you can run like stock cluster API against it. But from the principal idea. Yes, exactly. That is how you could use KCP as a central control plane and let it handle everything and sync it down to the workload classes.
Abdel Sigewar
Yeah. The reason I was asking this is because it's a pattern that is very common actually in all the providers that does some form of kubernetes control some form of managed kubernetes, even like on Prem Kubernetes, kind of the same idea. You have a control plane which then provisions other control planes and worker nodes. Right. And so how are logical clusters different than virtual clusters?
Marvin Beckers
So I think that depends a little bit on your definition of what a virtual cluster is. So usually I think this refers to concept also known as hosted control planes, where you take like a subset or maybe the full Kubernetes control plane and put it in pods as basically as just another workload on another Kubernetes cluster. But of course that means maybe they even share datastore. But you need to spin up a new API server. You need to maybe spin up maybe a new ETCD for each of them, for each virtual cluster. And this is more efficient than of course, provisioning three nodes and make them the VM3 node kind of control plane pattern. But still logical clusters, because they only exist in the data store and the API server does the separation. You're not spinning up any new processes and you're not spinning up any new data store. So even like hosted control planes, they can come up pretty fast.
Abdel Sigewar
Yeah, because they're pods.
Marvin Beckers
Yeah, but like a logical cluster in KCP takes like a second or so to be there and ready.
Abdel Sigewar
Yeah, so that's actually a very, very important distinguishing because typical hosted control plane virtual clusters kind of implementation, the control plane is just pod. Right. That runs the entire control plane stack, which has the API server, the controllers and everything. Right. But in the case of kcp, it's essentially, I've seen it in the documentation, it's like a YAML file that you create, but it doesn't really spin up anything. It just separates ETCD storage layer and then implements some sort of like authorization authentication on top of that, essentially.
Marvin Beckers
Right, Yep, exactly. It just takes basically what you give as a name, as a partition key, puts that on the storage layer and then it's basically done provisions like I don't know, five, six resources and yeah, then you're done.
Abdel Sigewar
Awesome. Because you talked about workspace. What is workspace?
Marvin Beckers
Yeah, so workspace is kind of a higher level resource for managing logical clusters. So a workspace basically models a Kubernetes API that you are provisioning the way that you want to. So in a workspace, when you create a workspace, you get a logical cluster and basically workspaces are the user facing resources because they are logical clusters that don't have workspace representation. They're like the system clusters, basically something that the administrator would interact with, but not a normal user. So users primarily create workspaces and then get logical clusters. So they get a Kubernetes API they can talk to.
Abdel Sigewar
Yeah, just a way of organizing resources, essentially. Cool. And then there is this other concept, and I'm still going through the concepts from documentation, API exports and API bindings. What are these?
Marvin Beckers
In Kubernetes, when you create a cid, you do basically two things at once. So first of all, you. Well, you define a schema for the API resource, but you also register it with the API immediately. So you have the schema definition and it gets immediately registered. But these are kind of two steps, and KCP separates them. So in the API exports as service provider, like someone who wants to provide an API, they would define a schema of the API, but this schema would not become available as an API resource. So it's just the schema, Basically, it's just, okay, here is an API that I am offering.
Abdel Sigewar
Got it.
Marvin Beckers
And the API binding is then the consumer side, where the consumer says, okay, I want to bind this API because I want to have it available within my workspace as a resource that I can create objects from.
Abdel Sigewar
Got it. So you basically separated the way Kubernetes does CRDs into two separate steps. You would create the schema first, and then to make it available, you have to create API binding in the workspace to be able to consume that API.
Marvin Beckers
Exactly. And that's why you can also, like, convert the CID very easily to a resource schema in kcp.
Abdel Sigewar
Is that done for, like, this implementation? Is it for security reasons or for efficiency reasons? Like, why was it implemented this way?
Marvin Beckers
It's more so that, like, API consumers can mix and match which APIs they want to have available. Because in Kubernetes, when you create a CID on a cluster, well, that is the API resource that you'll have available, right?
Abdel Sigewar
Yes.
Marvin Beckers
And usually, like, it can only be provided by a single provider, like in controller and operator that you start. And in kcp you can do this mix and matching. And you can also consume APIs from different providers. You can say, like, the same API is provided by two, let's say, teams in your organization, and you want to choose which of the teams should be providing that, and then you select one of the API exports, basically.
Abdel Sigewar
Got it. I mean, it's hinting toward platform engineering in a way. Right. That you could. It's kind of like having a catalog of APIs available through the exports, and then you can bind your workspace to an API to be able to use that specific API, essentially.
Marvin Beckers
Yeah, exactly.
Abdel Sigewar
Got it. Cool. Awesome. So we talked briefly about stuff that KCP can do, like examples and stuff, but I want to go back to that concept of multitenancy. Like, why is multitenancy? I mean, I don't know if it's easier or more complicated, but, like, how does KCP solve multi tenancy that cannot be solved with like traditional Kubernetes.
Marvin Beckers
So we talked about this earlier, right? With virtual clusters you could like spin up like a Kubernetes API for everyone. Each team that wants to basically work on their own, want to be self sufficient in a way and want to do self services. But that is quite costly. And because KCP workspaces are basically, they are very cheap, they are easy to create and because you have these interactions between workspaces. So I can create an API export in one workspace where I as a service provider, I own the workspace and service consumers in their workspaces which they own, which they have full permissions to, they can create the API bindings and consume APIs across these boundaries. So teams are basically using different Kubernetes API endpoints, but they still share like the system from which they can consume APIs. From the service catalog that you mentioned.
Abdel Sigewar
Yeah. So it sounds like it gives you the option to reuse APIs, while if you would do something like virtual cluster, then you don't have that option. If there is like a crd, you need to install it basically in your own control plane. Right, Cool. And also the other thing that is now coming to mind is like it makes, I mean multitenancy is not only about separating the worker part or like the data plan if you want, it's also about separating the control plane which is, as you said, it's cheap in KCP because you're literally just creating YAML file. Right. They're not spinning up anything. So how does KCP itself relate to the Kubernetes upstream controller? There is like a cap here that you mentioned. 4080. Can you talk about that?
Marvin Beckers
Yeah. So KCP is built on mostly Kubernetes code. So where we're using large parts of that. And during development of KCP or like in the last few years it became clear that in Kubernetes there is, well, Kube API server and this has all the container orchestration parts built in. This capacity is basically more or less about code organization, where there is a step before the Kube API server, which is like the generic control plane. And basically the Kube API server is now built on this generic control plane. So it takes this generic one and adds the container orchestration bits to it.
Abdel Sigewar
Oh, got it.
Marvin Beckers
Okay, so if you build like, if you want to use the Kubernetes resource model and the Kube API server makes Kube API server, you can take this and build your own API component. Got It. So we've been trying to work with Upstream to clear out the code and make sure that it gets this generic control plan available. But we think that, well, if you have this generic use case and KCP is maybe too much for you, then hopefully this will now enable community members to build their own control planes based on the Kubernetes resource model.
Abdel Sigewar
And when you mentioned that this cap is all about code organization, does that mean that up to before this proposal was implemented or is being implemented, does that mean that in kubernetes itself, the control plane plus the scheduler and all the other components, they were like tightly.
Marvin Beckers
Coupled together, not the scheduler, but within the Kube API server previously, basically the container orchestration bits, like the core APIs and all of that were all, well, kind of built in. Not the best one to speak about this, but. And basically, if you take the earlier code out of it, that just initializes the Kubernetes resource model and all that, but without registering the container orchestration bits, you have now the separation in the Kube API server, but it's like this prototype, but this generic control plane, and then the Kube API server builds on.
Abdel Sigewar
Top of that, which then has the container orchestration built in. Okay, so it sounds like separating Kubernetes itself from Kubernetes to make it possible to spin up the generic control plane if you just need that, or run standard Kube API server plus container orchestration plus whatever. Right, cool. That's more like code organization. All right, so we talked about this briefly and I mentioned that when you were talking about this concept of consuming APIs across workspaces, it sounded to me like platform engineering in a way. Like always call it platform engineering. How do you see like KCP used today in the context of platform engineering? Like, do you have actual use cases that you have seen before?
Marvin Beckers
I think we see these use cases out there every day, like with all these projects, like we talked about them earlier. Like a lot of platform engineering is at the moment is based on extending the Kubernetes API with useful APIs. And I think this is basically, well, exactly the use case that KCP fits in. And centralizing this, because usually I don't think you can have1 Central Kubernetes API. Like that's not what it was meant for. Right? But you can build like one central KCP control plane and which then everyone can go to have their, well, the bits of platform engineering in there. So say, okay, I want to create databases from it, I want to create infrastructure as a service from it. And basically from there it gets, well scheduled where it needs to be or where it needs to go. But yeah, I think what we're basically seeing is platform engineering has already taken all the benefits of the Kubernetes resource model.
Abdel Sigewar
Yes.
Marvin Beckers
And probably is well more suited with something like KCP because it's meant for this kind of use case.
Abdel Sigewar
Got it, yeah. Like one of the ways I could see KCP being used in the context of platform engineering is if you need to run for example a central cross plane. Just an example, you would just run it on kcp. You don't need Kubernetes for that. Right. You could just KCP control cross plane and then that's it. And then people can do whatever they want.
Marvin Beckers
I think that would be the idea. Yeah. So the controllers usually need. Well they need to be KCP aware. So you would need to run like a cross plane, like a version of cross plane that is KCP aware. But in principle, yes, that would be the idea.
Abdel Sigewar
Yeah. I mean I was talking to somebody from Kubernetes and they were mentioning, without mentioning, name a company. Right, let's a company and nameless company that does hardware. Right. And they were thinking about using KCP inside their hardware for stuff. Can you talk about that? Like, do you have an idea about that?
Marvin Beckers
I think in general it's fair to say that we've been talking like the maintainers have been talking about KCP for quite a while. I joined the maintainers last year and have been, you know, also been to conference and talking about it. And I think what we're seeing is more interest because more and more organizations are realizing, hey, the central control plan could actually help us consolidate what we're running here. And so I think there is a need for that and a lot of companies and organizations are looking at it and yeah, we think that that hopefully will result in like official adoption at some point.
Abdel Sigewar
Got it, got it. Interesting. Okay, cool. So the last question would be how can people contribute to kcp? It's a sandbox project, right?
Marvin Beckers
Yeah, exactly.
Abdel Sigewar
So where can we people find you? How can you. Are you looking for contributors?
Marvin Beckers
Obviously we were looking for contributors. There is a website, KCP IO, which is the project website from which you can find the documentation. You can find also our bi weekly community meeting which is happening on Thursdays right now where everyone is welcome. We really like to see new faces and see them join, introduce themselves and maybe talk about how KCP can be useful to them. We have a Slack channel on the Kubernetes Slack. So that's KCP dev. So we also love people joining there and talking to us. I think that's the main part at the moment. Our code is obviously on GitHub and so we are very much looking for contributors, contributors to the code, contributors to documentation. Because obviously I think we found out that this is like a fairly complex topic to talk about. So the more documentation and the more we help people get into it, the better.
Abdel Sigewar
Awesome. And you're going to be at. I mean the project is going to be probably at Kubecon, North America.
Marvin Beckers
Yep. There's going to be one of the CNCF projects, Kiosks, I think they're called.
Abdel Sigewar
In the projects Pavion, but Yeah, exactly.
Marvin Beckers
And there will be some maintainers from KCP there.
Abdel Sigewar
Awesome. So then people can go find the maintainer and talk to them. Thank you for being on the show, Marvin.
Marvin Beckers
Thanks so much for having me.
Kaslan Fields
Thank you very much, Abdel, for that interview. We both are familiar with another employee of Kubramatic, Mario Falland, whose last name I probably butchered right there, but who I work with closely in open source and I know that you've interacted with at events at least and things before. So I was excited to hear about this topic, which is also something that I think Mario works on. So tell me a bit about it.
Abdel Sigewar
Yeah, I should start by saying thank you to Mario because Mario provides actually the hardware. So we recorded the episode at the Container Days Hamburg for the background. Hamburg is a city in Germany and that's where Kubermatic is originally from. That's where the company was created originally. And so the conference is run by Kubramatic and we were chatting with Mario and he said, yeah, I'll just bring all the recording equipment and you can just use them. So he provided all the equipment, I didn't have to bring anything with me. And he also provided the guest.
Kaslan Fields
So thank you, Mario.
Abdel Sigewar
Thank you, Mario. So yeah, this was pretty cool. First of all, if you hear any background noise in the audio itself, if you have heard any background noise in the audio up to now, it's basically because we recorded in open space in a conference. So that's the background there. So you just, you know, it's not our typical recording setup, but no, we had a super interesting conversation. So I heard KCP before. I was under the impression it means Kubernetes control plane, but it doesn't. Yes, it clearly doesn't. And they are emphasizing it doesn't mean KCP Kubernetes control plane. It is KCP that's it.
Kaslan Fields
Okay. It doesn't particularly stand for anything, huh?
Abdel Sigewar
No, it doesn't stand for anything. It might have started as KCP at some point, but they don't intend it to mean because it is a universal control plane. It's not only a Kubernetes control plane, essentially.
Kaslan Fields
So it's a universal control plane for other types of platforms as well as Kubernetes.
Abdel Sigewar
It's universal in the sense that you can use it as a control plane to run crossplane, for example, and then what you would use to control with it or with crossplane is not necessarily Kubernetes. You would use crossplane to spin up all sorts of cloud resources. So it's universal in that sense that it's not necessarily just for Kubernetes. You can use it for anything else technically.
Kaslan Fields
Remind me what crossplane does.
Abdel Sigewar
Yeah. Crossplane is this open source tool that allows you to use Kubernetes resource model KREM to declare cloud resources and then it will spin them up for you. So you could define a YAML file that defines for example a cloud SQL database or a pub sub topic or something. So instead of using infrastructure as code like Terraform or Pulumi, you use krm. Yeah, exactly, exactly.
Kaslan Fields
All right, so it's creating a control plane that uses a Kubernetes style API to manage objects, correct?
Abdel Sigewar
Yeah, that's a very good way of putting it. Yes.
Kaslan Fields
Okay.
Abdel Sigewar
And then if you write your own operators and CRDs, then you can use it because that's just standard Kubernetes stuff.
Kaslan Fields
Right, Right. So this is a sandbox level project with CNCF and just became. How long has it been a sandbox project?
Abdel Sigewar
I don't remember exactly, but very recently.
Kaslan Fields
Okay, yeah, that's what I thought.
Abdel Sigewar
Apparently they are on their way to not graduation. What's the thing before incubating? Yeah, so they are on their way to incubating.
Kaslan Fields
Yeah. So at this sandbox stage, usually a sandbox project within the CNCF is kind of just getting its legs under it from a maturity standpoint. They're establishing of course, what the project is working on, building the resources so that it's easy for more folks to join in and contribute, working on getting more users. So once you reach the incubating stage, you are usually starting to branch out in terms of recruiting more contributors, hopefully from other companies than the original founding company of the project, since most projects do come from some kind of company. And then you are starting to have a few users that you can refer to. So did he talk at all about the journey that they've been through so far.
Abdel Sigewar
I mean, the project started from within Kubernetes because they have their own platform that they basically they provide a platform to be able to spin up Kubernetes on PREM or on other cloud providers. But there was reasons why they didn't want to mention some names. I mean they didn't want to mention tomorrow they're recording. I know some big companies that are actually using KCP internally, usually with sandbox projects.
Kaslan Fields
It's kind of like that. Like a few companies are kind of testing the waters on it. They don't want to call it out and until they're confident in it. Correct.
Abdel Sigewar
But from what I understood, they are pretty confident in the maturity of the products itself or the projects and the maturity of the community.
Kaslan Fields
So they are of the technology, right?
Abdel Sigewar
Or the technology as well. So they're very confident that they are on the right path toward incubating, at least.
Kaslan Fields
Cool. I look forward to seeing more about it. As you can tell, I haven't listened to the interview yet, so I'm going to have to go and do that after this. But thank you so much, Abdel, for conducting that inter. I'm excited to hear more about this Sandbox project in the cncf.
Abdel Sigewar
Thank you.
Kaslan Fields
That brings us to the end of another episode. If you enjoyed the show, please help us spread the word and tell a friend. If you have any feedback for us, you can find us on social media at kubernetespod or reach us by email at kubernetespodcastgoogle.com you can also check out the website at kubernetespodcast.com where you'll find transcripts, show notes and links. To subscribe, please consider rating us in your podcast player so we can help more people find and enjoy the show. Thanks for listening and we'll see you next time.
Kubernetes Podcast from Google: Episode Summary
Title: KCP, with Marvin Beckers
Hosts: Abdel Sigewar, Kaslan Fields
Guest: Marvin Beckers, Team Lead at Kubermatic and Contributor/Maintainer of CNCF Sandbox Project KCP
Release Date: October 1, 2024
In this insightful episode of the Kubernetes Podcast from Google, hosts Abdel Sigewar and Kaslan Fields delve deep into the world of KCP with guest Marvin Beckers, a pivotal figure from Kubermatic and a key contributor to the CNCF Sandbox project KCP. The conversation unpacks the nuances of KCP, distinguishing it from traditional Kubernetes control planes, and explores its significance in the broader Kubernetes ecosystem.
KCP Unveiled
Marvin Beckers begins by clarifying a common misconception about KCP. Contrary to initial assumptions, KCP does not stand for "Kubernetes Control Plane". Instead, it is described as a "cloud-native control plane for Kubernetes-like APIs" (02:10).
Defining KCP
KCP leverages the robust Kubernetes resource model, extending its utility beyond mere container orchestration. Marvin explains, "KCP is basically the idea that the Kubernetes resource model... has become more useful than just using it for container orchestration" (02:20). This evolution allows KCP to serve as a versatile control plane for various APIs within the Kubernetes ecosystem.
Why Choose KCP Over Traditional Kubernetes Control Plane?
Abdel queries the necessity of opting for KCP instead of the traditional Kubernetes control plane. Marvin responds by highlighting KCP's specialized features: "KCP thinks about APIs and not primarily about container orchestration, KCP has ways to manage APIs at scale" (03:14). Additionally, KCP introduces multitenancy through logical clusters, enabling multiple Kubernetes API endpoints to be managed efficiently within a single KCP instance.
Practical Applications
Marvin elaborates on KCP's adaptability: "It's like a framework... what you use it for is kind of up to you to build it and what you want to build with it" (05:47). He cites examples like cloud providers offering managed Kubernetes services and organizations developing internal platforms for their teams. This flexibility positions KCP as a foundational technology for various infrastructure and platform engineering needs.
Logical Clusters Explained
A significant feature of KCP is its logical clusters. Marvin defines them as a method to partition the storage layer (e.g., ETCD), allowing each logical cluster to interact with the API server independently: "Each logical cluster... has its own space in ETCD... acts like a single standalone Kube API server" (07:04). This segregation ensures that each cluster operates in isolation, maintaining security and efficiency.
Advantages Over Virtual Clusters
Abdel contrasts logical clusters with virtual clusters, probing the operational differences. Marvin clarifies, "In KCP, a logical cluster takes like a second or so to be there and ready," compared to virtual clusters that require spinning up full control plane stacks: "logical clusters only exist in the data store and the API server does the separation" (13:29). This makes KCP's approach more resource-efficient and faster to deploy.
Code Organization and Control Plane Separation
Marvin discusses the structural relationship between KCP and the Kubernetes upstream controller: "KCP is built on mostly Kubernetes code... the Kube API server is now built on this generic control plane" (19:53). He explains an ongoing effort to decouple the generic control plane from the container orchestration components, facilitating broader community engagement and potential custom control plane developments.
Enhancing Platform Engineering
When exploring KCP's role in platform engineering, Marvin notes, "a lot of platform engineering is at the moment based on extending the Kubernetes API... exactly the use case that KCP fits in" (21:56). KCP allows for centralized API management, enabling teams to integrate various services like databases or infrastructure-as-a-service seamlessly.
Real-World Implementations
Abdel shares anecdotal evidence of companies integrating KCP internally, hinting at its growing adoption: "some big companies that are actually using KCP internally, usually with sandbox projects" (29:03). This underscores KCP's practical relevance and the industry's confidence in its capabilities.
Contributing to KCP
Marvin encourages community participation, outlining ways to contribute: "There is a website, KCP IO... bi-weekly community meetings on Thursdays... Slack channel on the Kubernetes Slack. So that's KCP dev" (24:36). He emphasizes the need for contributions in code and documentation to bolster the project's growth and accessibility.
Upcoming Events and Engagements
Looking ahead, Marvin mentions KCP's presence at KubeCon North America, where maintainers will be available for discussions and collaborations: "There will be some maintainers from KCP there" (25:36). This presents an opportunity for enthusiasts and professionals to engage directly with the KCP team.
The episode provides a comprehensive exploration of KCP, elucidating its distinction from traditional Kubernetes control planes, its innovative features like logical clusters and multitenancy, and its pivotal role in platform engineering. Marvin Beckers' insights highlight KCP's potential to revolutionize API management within the Kubernetes ecosystem, fostering efficiency and scalability. For listeners interested in contributing or learning more, KCP's community resources and upcoming events offer ample avenues for engagement.
Contact and Further Information:
#kcp-dev on Kubernetes SlackFor more detailed discussions and updates, listeners are encouraged to attend KubeCon North America and engage with the KCP maintainers directly.