Loading summary
Transcript67 lines
- [00:00]
A
Hello and welcome to the Kubernetes Podcast from Google. I'm your host Kaslyn Fields.
- [00:06]
B
And I'm Abdel Sighiwar. In this episode we speak to the Kubernetes 1.34 release lead. Vyom Yadav Viom is a software engineer in the security team at Canonical and a member of the Kubernetes Security Response Committee. We talked about the new release team and the major updates, deprecations and removals to expect in this version.
- [00:40]
A
But first let's get to the news. GKE is celebrating 10 years of its existence with a hackathon. You can join online and help build the future of agentic AI orchestrated by GKE for a chance to win $50,000 in prizes and Google Cloud credits. The submissions are already open and due on September 22nd. Check the link in the description for details on how to sign up.
- [01:05]
B
The Golden Cubestronauts program celebrated 100 members. Golden Cubestronauts are certified professionals who managed to obtain all CNCF certifications as well as the Linux Foundation Certified Sysadmin. The program was launched in 2024 following the success of the Kubestronauts program, which which today accounts for over 2,000 members.
- [01:24]
A
The creators of the Open Policy Agent, or OPA, are joining Apple. The news was published in a blog post by the maintainers. The project will remain under the CNCF and there are no plans to change the governance OR licensing model.
- [01:38]
B
DocumentDB joined the Linux foundation announced at the Open Source Summit Europe 2025. The open source project is joining under the MIT license. DocumentDB is a NoSQL document store. The project started in 2024 as a pair of Postgres extensions built by Microsoft and rapidly evolved into a popular document store solution with 2000 stars on GitHub and hundreds of contributions.
- [02:02]
A
Solo IO announced that they are donating the Agent Gateway to the Linux Foundation. The Agent Gateway is a cloud native proxy designed to secure, observe and enforce policies for agents communication. It supports the A2A or agent to Agent framework and MCP or the Model Context Protocol.
- [02:21]
B
Cube Crash is a community led virtual event happening on September 23, 2025. Whether you want to tackle your organization's hurdle or stay updated on the latest platform engineering trends, Kube Crash provides the knowledge you need to level up your team's platform engineering capabilities. Check out the schedule and register for the event on CubeCrash IO or check the Show Notes for more information.
- [02:42]
A
And that's the news.
- [02:47]
B
Today we're talking to Vom Yadav Vom is a software engineer in the security team at Canonical. He's also the release lead for Kubernetes 1.34 and have recently joined the Kubernetes Security Response Committee. Vyom has been contributing to open source since his freshman year in college. Welcome to the show, Vyom.
- [03:07]
C
Hey everyone. Glad to be here. Thank you for having me, first of all. And yeah, I'm really excited to be on the Kubernetes Google podcast and discuss about what's happening in V1 34.
- [03:20]
B
Thank you. This is tradition. We do every time there is a new release. So we open the interview or we open the discussion with the question that we usually ask all the time, knowing that we usually know what the theme of the release ahead of time. But as of the recording, I don't know. So this is going to be a scoop for me at least. I don't know what's going on.
- [03:37]
C
Yeah.
- [03:38]
B
So what is the theme this year?
- [03:40]
C
Yeah, so before I tell you the name of the release, I would set a narrative of why the theme is what it is. Every release cycle we inherit a lot of wins from previous release cycles, which can be our tooling, the documentation we have, the historical quirks of the project. So all of these wins that we inherent, we don't really have much control over it. But as the release cycle progresses, sometimes speaking in nautical terms, sometimes this wind fills our sails, sometimes it destabilizes the ship, or sometimes the Spencer dies down. So it's not really the wind that matters here because the wind is good. The wind actually moves the ship. But the ship wouldn't be able to steer properly or in the proper direction without the sailors on board that get on board the helm, control the sails and stabilize the ship and so on. Getting into that narrative and that nautical theme, because it's Kubernetes and Kubernetes logo is a ship fields helm. So getting into that, I thought that this release should be we should honor the sailors or the contributors that we have that actually make this ship go through the ocean and actually ship a release. So with that theme and with that, that nautical theme in mind, I have called this release of wind and will. So it's short formed as O wow. So O W a W kind of thing.
- [05:08]
B
All right.
- [05:09]
C
And this release, actually I think the line that I have for this release is that this is a release that is powered by the wind around us and the will within us. So this is the release theme and the release title that I have decided to go with and why this is Very special to me is that Kubernetes as a project is not just what in GitHub.com Kubernetes Kubernetes There is a lot more that happens behind the scenes just to ship the binary that you download of the release cuts that happen during a release cycle. And there is a lot of cat herding and there is a lot of work that goes on in project management around enhancements, document documentation, communications. So all this included, it's a very big and tremendous effort that takes place in order to ship this, sail this ship across the ocean. So keeping that thing in mind, I think this is why I decided it's off wind and will because it's sailing a ship across the ocean and a ship cannot sail without the wind and the will of the sailors.
- [06:14]
B
Awesome, awesome. That's actually a great theme. I really like the narrative around how you got inspired from creating Kubernetes and the nautical theme. So I do have a question actually. I think we never probably asked this in this kind of interviews like between the two cycles. So the two releases, sorry. So there's like release in April and this one coming kind of late ish August. How early does the work of the release team starts? Is it like four months? Is it two months? Like how early does that go?
- [06:39]
C
So it depends actually on which sub team you are on and as a release lead it actually starts I think just a few days after the release previous release. So it depends on which in which role you are in the next release cycle. So I was the release lead and it's the duty of the release lead to help in assembling the release team as well. So as soon as the shadow survey shadow application closes, we filter all the applications we receive to form teams and it's priority based. So because enhancements work is more front focused, we focus on getting the enhancements shadows sorted first and then we move towards getting the other shadows onboarded. So it depends on the role you're in. But for the release lead it starts pretty much after the previous cycle ends.
- [07:30]
B
Okay, interesting. Okay, so it's basically a 4 months ish amount of work to get the release out.
- [07:37]
C
Yeah, it's a little bad.
- [07:39]
B
Awesome. Okay, before we get into the details of the release, if you would to describe it, and I know we are like few weeks ahead of the release itself, but if you would describe it in few words, what would that be?
- [07:49]
C
If I really think about it, I think I have described it in a few words. And that's the title of my release of Wind and Will. I don't think I can shorten or elaborate this further to or the line that this is a release powered by the wind around us and the will within us. I think these are the strongest quotes that I was. When I was thinking about the release theme, these were the quotes that inspired me the most. And I was thinking about how would I name a release, because everyone has their personal preferences and Octarine was the previous release. Before that it was Penelope. So when I was thinking I wanted to honor the contributors. So I think my viewers over here would be the release name only, which is Off Wind Anvil. So that's awesome.
- [08:37]
B
Great. I like that. All right, so let's get a little bit into details. There was a sneak peek article blog, if you want published, which talks a little bit about some, as the title says, sneak peeks. So that kind of, I think features that a lot of people would be interested in. But you shared with me a preview of the actual release blog, if you want. Right. That has all the details and there is a lot of things going on. There's a lot of new things, there's a lot of deprecations. Can you tell us, give us like a little bit of a recap, like, what are we looking at, what to expect in this release?
- [09:10]
C
Yep. Yeah. So this Release we have 58 enhancements, which is a little lower than the last release cycle, but I think it's a very close number. Last time it was 63 or 64.
- [09:20]
B
64, I think.
- [09:21]
C
Yeah. Yeah. So it's pretty close to what we had the last cycle. And break breaking this down into the stages that those kept are in. We have 23 kept that are going stable, 22 that are going beta, and 13 keps in alpha. We also have. So in the sneak peek blog, we did not mention that we have deprecations, but we actually have a few deprecations. So this is. Now this is the wind I was referring to. So this is a change that we require in our tooling, in part project management, in how we categorize keps. So this is something that we're going to discuss in the retrospective to fix this. But we have deprecations. Getting the C group driver correct has been a pain point for many kubernetes, cluster administrators. So this KEP4033 that solves this problem, which basically has the feature of discovering the secret driver from the container runtime. But to support this feature, container runtimes also have to support this. So not all container runtime versions support this and the major thing to note here is that containerd version v1.x does not really support that. But still in the deprecation phase, containerd version v1.7 and forwards would be supported. And you can specify the cgroup driver using the Kubelet flag. But if you are looking onto using this gap and the auto discovery feature, so you don't have that pain point, you should Upgrade to containerd version 1. 2. X. So this is one of the deprecation and this also deprecates the Kubelet command line or the configuration parameter. So this is one of the deprecation that is the other one is mainly alias. So whenever your service is routing traffic pods you want to make sure that how that traffic is routed. So whether you want to prefer it in the same zone. So this was previously called Prefer local. Now Prefer local is a bit ambiguous and it does not really convey the meaning properly. So this is more of a variable name change where we are calling it now prefer same zone. The meaning remains the same, the logic remains the same. It's just a name change. And prefer preferred close would be supported as a deprecated alias for this. Note that this kep I believe the number was 3015. Yeah, 3015. And this kip actually mentions that in non goals that is removing preferred close from the API. So prefer close is not going to be removed from the API, but it is deprecated. Alongside this KEP also adds prefer same node if you want to route the traffic in in the same node with the appropriate fallbacks to same zone and then cluster wide. Yeah, these are the deprecations in this cycle, but I can jump into the interesting stable and beta.
- [12:14]
B
Yeah, we'll talk a little bit about the enhancements. I have some few of them that I want to cover, but I do actually I was just looking at the blog for this Prefer close. It's true that like prefer close is quite ambiguous, it doesn't mean anything. And prefer same zone Prefer same node makes a lot of sense, especially if you're like in a cloud provider. Because in cloud we talk about the concept of zone and availability zone or whatever. Right. So it's quite interesting. Okay, all right, so we'll go into the actual the new stuff, right? And I think in my opinion the big one is kml. Are we getting rid of YAML? What's going on there?
- [12:45]
C
We are not getting rid of YAML. KML is actually YAML. It is YAML and all YAML parcels can work with kml. So YAML is not going anywhere. We are not getting rid of YAML. You can still do get whatever minus o YAML. It just has now a new option called KML which is an alpha feature that you can try. KML is supposed to. So YAML as a language has a really wide grammar and not a whole of that grammar is useful or is the one that we really want when we want to see the output of a long manifest when we are scrolling down the terminal. So that's not really ideal. So KML aims to bridge a few of that gaps. For example, white spaces in YAML can be very painful and especially when you are deep into that YAML and you don't know how deep you have to nest and then you have to do phase and then if you are on just the terminal it it can be very painful for us. Example, IDs have that feature of showing how deep you are inside a YAML and the parent of this node and then parent of that node and so on.
- [13:54]
B
Yeah, auto indentation. Sometimes that would just solve it for you.
- [13:58]
C
How this solves this, that there is no spacing does not make any change because it's similar to how JSON looks. And we have used a subset of grammar in the YAML definition itself to create kml. This also solves the famous Norway bug problem, which is like some string literals in YAML, like no N capital, no Y. Yes, they actually mean boolean values, true or false. But if you have them as string literals, then your YAML can get, you know, won't understand what's happening behind the scenes when you apply that manifest. So it also codes keys that are ambiguous so that there is no ambiguity left in the YAML. Alongside that, another big problem with YAML is that when you are templating YAML, for example Helm, when you are creating the value of something specific and then you have to exactly have the same spaces in your particular value file that you are going to apply so that the final YAML that is created is in the proper format. Now getting those spaces right in templating can be very painful. So KML also solves that problem because spacing is not an issue anymore. Because spacing is not an issue, you can specify it however you want and it would be rendered correctly. So that is another thing KML solves.
- [15:19]
B
So is this new alpha feature KML only as an output or is it also as an input? Can you write KML to the Kube API?
- [15:25]
C
KML is YAML. KML is YAML. You can use KML anywhere That will. YAML is accepted. So just, it's just simple YAML. It's just a particular subset in the YAML grammar that is being used to produce output. It's still pure YAML.
- [15:43]
B
Got it. Okay, so YAML is not going away. We're just introducing a new, better thing called kml. This, this is. I looked at it, I looked at the blog, I looked also at the cap that Tim Hocking have opened a while ago to introduce Kamo. And I do have to admit it's slightly more clean than just like simply Ammo. Right now I have to get my hands on it on like very large, as you said, like very large manifests because that's where it will start becoming more interesting or useful. Cool. And the next one, I think this is quite, quite an interesting one. So typically when you have just to explain to the audience, I'm going to take two seconds to explain it and then you can add more stuff if you want. Typically in a cluster where you need to pull images, the there are like two ways that your node can authenticate to the registry. It's either the service account of the node or the identity of the node itself. Or sometimes you can add a secret that contains like the password of the registry or whatever. Right. And so now they're adding service account tokens that can be set at the POD level.
- [16:43]
C
Right.
- [16:43]
B
Which means you can set a POD with a token on OIDC token to say pull the image from this particular registry. And this is how we authenticate. And this for me is solving a very particular problem of multi tenancy. Yeah, among other things. Did I describe this correctly? Do you have anything to add?
- [17:03]
C
Yes, pretty much correctly. Projected service account tokens, what this is being called, as I think this is a really great feature and especially for multi tenant environments, you specify the audience you want this to be scoped to for a particular service account token and you can have that in the service account definition. The audience you want this to scope to, you can have the annotations. And then in the. What's that? The credential provider config, you can actually specify the audience you want to bind to. And it's not just that you can actually have authentication over this. So you can actually control based on roles what kind of audience is permitted for Kubelet to fetch tokens. So Cubelet cannot fetch tokens for any audience scope token. It has to follow a particular set. So you define that in the configuration and then the Kubelet would fetch a token which would be ephemeral. And then it would fetch the image using that token. So this kind of isolates multi tenant environments more than what they were used to be. And this actually, actually is a very good step. Of course you have the root which is the credential provider that would still have to be controlled using traditional ways that you have been setting up a credential provider. But still you have POD level isolation of the tokens that you share. And a really good aspect of this is that you don't have any secrets that are bound to the node, which means there is no over permissive fetching and a particular podcast does not have access to the images of some other pod that it shouldn't have access of. So I think this is a really great feature and this is what I like about this release cycle. This release cycle are a lot of GIPs that I have been reading. They have been covering a lot of gaps that there were in implementations that we had. So filling those gaps to follow best practices. So this is, I have noticed a handful of gaps in this particular release cycle. These gaps have been like the work on these kept has been going on for quite some time. But these stages that they are being promoted to, that's the daily shift that I'd like to see going forward in kubernetes release cycles.
- [19:15]
B
Yeah, now that's something that I think I noticed over the last few releases which is like introducing things that solve friction that exists already. Just like solving problems that people already have. Whether it's like multi tenancy or AI or other like just very particular gaps, very particular features like very targeted. Speaking of AI, dra. So dynamic resource allocation. Yeah, we've been hearing about this thing for a very long time and from what I understood is like some core resources are going ga. So they are basically my understanding is that they made DRA functions the same way as the volume stuff where you have like volume claims and stuff like that. Right, yeah. So what's going on there?
- [19:59]
C
So core of DRA is going stable. That's the big thing. The so to give a bit of context here, there was another implementation of DRA before this, which was the classic implementation. And I've been reading more and more about the classic implementation versus the current implementation. So do to knit the whole story. I'll cover from where DRA started. DRA actually started from the ability to have pods consume resources like GPU, FPGAs like NICS and they're getting assigned to getting bound to a particular node. Not all devices wouldn't be bound to a particular Node, but the devices that are bound to a particular node consuming that. So the original implementation started in a way in which there was the device driver and opaque parameters were being sent to the device drivers. For example, I want a GPU which has these following properties, and that resource request was sent to the device driver. And then device driver would give you the nodes that support that. And then the scheduler would go back and forth with the device driver to finally figure out which nodes support this kind of device. And the POD would be scheduled onto that node. Now, this approach, the drawback that it had was with cluster auto scaling, because these were opaque parameters. For example, I specified a key as foo with value thousand. Now, Kubernetes has no context what foo1000 means here. The device driver could be processing that 4000 as maybe thousand divided by thousand equal to one, and whatever node has that value as one using that or maybe by 10, 20 or whatever. So there was no context that cluster autoscalers had to perform a simulation of. Hey, would this resource claim request be satisfied if I add a pod, Sorry, a node of this kind? So this was the drawback with the classic DRA implementation, and this is why this was dropped in favor of the approach that we have right now, which is structured parameters for dra. So this kind of gap has been divided in a. Has been created in a better way in which the scheduler can actually predict whether adding a new node would satisfy this resource claim. So how this works is that there are resource slices that the device driver publishes on per. No, a node level basis. So resource slices basically tells you, hey, this node has this kind of device, this node has this kind of device, and then there are your resource claims. Now the thing about resource claims here is that resource claims are now based on cell, the common expression language. So it's a condition that can be evaluated, that can be simulated. So that condition is evaluated and the existing resource claims are checked, and you can actually have things like priority on which resource claim you want to match first. That's another kept separate from this one. But you can actually then from that condition match on which node supports that kind of device. And then that bot can be scheduled on that node. So this is going stable now, which is the core of dra. I think Patrick and Kevin actually have a talk at Q Con NA about DRA is going stable.
- [23:22]
B
We'll try to find it and add it to the show notes.
- [23:24]
C
Yeah, yeah, what's next and what's up for V1 35 as well. But this going stable actually means That a lot of real world use cases were considered because this gap has already gone through getting reverted. I think it was 1.32 when this was reverted because this did not suit the autoscaler constraints and then going through this phase. So I think this is a major milestone for DRA because it has gone through its ups and downs and now it is finally going stable.
- [23:58]
B
Yeah, I do like the approach, the whole like, the whole like structured approach with like proper resources that allows you to match like pods to nodes. And also what you said described about Autoscaler, it's pretty interesting. And then one more thing that I actually attracted my attention. I never knew how useful this could be, but then when I thought about it. Tracing for the Kubelet and API server.
- [24:20]
C
Yeah, I think tracing for the API server, that's a pretty old gap. I think the number is 647. So we are right now in 5000ish range for KEBS. And this is a really old kep. And the beautiful thing here is that both of these keps got in into beta into the same release and now are going GA in the same release as well. So this is something I found very amusing. So this is tracing for the Cube API server which is basically you would have. You would be exposing the spans, traces and the trace context and you can have a hotel collector which would be a sidecar or you can have a central thing that kind of collects all these metrics and it would be wrapped around the HTTP server in the API server for both client and server. So this is the first cap, I think, which is 647 which is tracing in API server. Tracing in Kubelet I think is 2831 which basically instruments the GRPC calls that you would make to different kind of interfaces like cri, csi. So it instruments that collects all the spans and then exposes them and they can be. So both of these I think can be configured in the same. Same way in which you can define the amount of traces you want to basically input or basically read. Sampling. Yeah, that's the word I'm looking for. So it defines sampling rate for the traces you want and it also defines the endpoint of the hotel collector that you want these traces to be collected at. Both of these steps I think going GA at the same time improve the observability aspect of Kubernetes quite a lot.
- [25:55]
B
Yeah, just to be very clear, when we say tracing here, we mean specifically the control plane tracing. So essentially allowing people to know exactly what's happened from the moment you issued an API call to the API server through all the components. Right. So it's not the data path in a way, it's the control path, essentially the control calls. Right, yeah, yeah.
- [26:15]
C
It's the. This KEP actually lists non goals as replacing the existing logging and metrics and all that portion of tracing. So when you are sending a request it's going through an admission controller. So having the traces over there, having the. How long did it take to go through that process? So when the request finally reaches the HTTP server of the API server. So that complete trace would be there in the Kev, that is 647.
- [26:43]
B
Yes.
- [26:43]
C
The other is the tracing for Q, which is different from the former cap, which is for the GRPC requests that you are sending to various endpoints. Also the reconciliation mechanism. Yeah. So for example, creating a pod, that's a reconciliation mechanism and fetching the image, creating the POD sandbox and then creating the container. So having traces over all of that as well.
- [27:07]
B
Yeah. Potentially adding a volume, add in anything that needs to to be added. So having that like end to end trace. So you know, if there is any like, you know, button X or anything that slows down the control plane of the cluster essentially. Yeah, cool, cool. And there is one more thing actually I was reading just this morning. I don't know if you have any idea about it. I've seen it when sidecars were introduced. I've seen some caps floating around which is like POD level resource requests and limits. So.
- [27:35]
C
Yeah, that's a pretty popular cap. Pod level resource requests and limit. I think this was 2837 if I'm not wrong.
- [27:44]
B
Yeah, 2837, it was opened in 2021, so it's quite old as a cap.
- [27:50]
C
Yeah. I think this was basically to have to stop over provisioning resources. So when you are basically over provisioning resources for each container and. And you have to specify, for example you specify requests and limits at a container level, you also want to specify them at a POD level. So this KEP goes towards that. I have not checked out the actual. What the code was merged for this kep for this particular release cycle. That is something I will do because there are 58 KEPS and tracking all the PR is a bit too much. So yeah, I haven't checked the actual code changes that were merged for this gap. But yeah, that's what I know about POD level resource. Question.
- [28:33]
B
Yeah, it's interesting because I think everybody is used to the idea of setting up the resource request at the container level right under the pod. But being able to have a POD level resource request limit is quite interesting because then you can potentially. It opens up futures where you can potentially have quotas at the POD level and then the containers can share whatever resources the pod has and other use cases. It's quite interesting.
- [28:56]
C
Yeah. So having it at the top level of POD and not caring about how the container split it.
- [29:00]
B
Exactly. Exactly. Cool. Awesome. This was pretty cool actually. I think that first of all I like the theme wind and will I like the narrative behind it. So kudos on that.
- [29:11]
C
Thank you.
- [29:12]
B
And yeah, I think this release is also going to be exciting because there's a lot of new interesting things I like. Also the fact that not everything is focused on AI because it's feels like there was like a little bit of an AI overload in the last couple of releases.
- [29:24]
C
Yeah, I feel this is a very well rounded release on the caps that I've been going through and it covers a lot of different aspects of the project. So I really like this, the things that are coming out in this release cycle. And yeah, I look forward to finally getting VE1 34 out on time. And that's the goal that I have in my head right now.
- [29:46]
B
I'm getting some sleep.
- [29:47]
C
Yeah, I'm actually going on a vacation after the release cycle.
- [29:50]
B
Awesome.
- [29:50]
C
Because it has been a very long grind and I want to take a break.
- [29:54]
B
Awesome. Thank you so much. Vyond thanks for coming on the show.
- [29:57]
C
No worries.
- [29:58]
B
Thank you.
- [29:59]
C
Thank you for having me.
- [30:03]
B
That brings us to the end of another episode. If you enjoyed the show, please help us spread the word and tell a friend. If you have any feedback for us, you can find us on Social Media KubernetesPod or reach us by email at Kubernetes podcastoogle.com youm can also check out the website at kubernetespodcast.com where you will find transcripts and show notes and links. To subscribe, please consider rating us in your podcast player so we can help more people find and enjoy the show. Thanks for listening and we'll see you next time.
- [30:41]
C
Sa.