← Software Engineering Daily
Software Engineering Daily
Python 3.14 with Łukasz Langa
00:47:00
Python 3.14 is here and continues Python’s evolution toward greater performance, scalability, and usability. The new release formally supports free-threaded, no-GIL mode, introduces template string literals,
Loading summary
Transcript32 lines
- [00:00]
Podcast Host
Are you passionate about software development and the tech industry? Software Engineering Daily is looking for a new podcast host to grow its hosting team. In this role, you'll help shape the show's editorial direction and interview engineers, founders, hackers and tech leaders. Podcasting experience is a plus, but not required. Curiosity, great communication skills, and a genuine interest in the craft of building software are what matter most. If if this sounds like you, reach out@editoroftwareengineeringdaily.com Python 3.14 is here and continues Python's evolution towards greater performance, scalability and usability. The new release formally supports free threaded nogill mode, introduces template string literals, and implements deferred evaluation of type annotations. It also includes new debugging and profiling tools, along with many other features. Lucas Lange is the CPython Developer in Residence at the Python Software foundation, and he joined Shawn Falconer to discuss the 3.14 release, the future of free threading type system improvements, Python's growing role in AI, and how the language continues to evolve while maintaining its commitment to backward compatibility. This episode is hosted by Sean Falconer. Check the Show Notes for more information on Sean's work and where to find him.
- [01:38]
Sean Falconer
Lukas, welcome back to the show.
- [01:40]
Lucas Lange
Happy to be here?
- [01:41]
Sean Falconer
Yeah, absolutely. So we spoke roughly a year ago about the Python 3.13 release. I guess what's new in your world since we last spoke?
- [01:52]
Lucas Lange
Well, not much changed in terms of my employment, a lot has changed in Python and also as a release manager of Python, just a few days back I released the last Python version of my own. I've been a release Manager for Python 3.8 which has been end of life last year, and 3.9 that has just reached end of life with the end of October. So it's a bit of a milestone I guess. You know, you look back at the last seven plus years doing this, I'm still involved in the release team since I'm doing installers for Windows and helping with the Mac installers as well. So I'm not entirely going to be gone from doing releases. But it is like an end to a part of your life when you signed up to be a release manager for two versions that are now officially not supported anymore. So that's like a personal change I guess here. And yeah, like we've since had like the big Pycon in Pittsburgh where we still worked like very hard on Python 3.14, but since we forgot about it, it's already the old version for us core developers because now we are Very busy working on Python 3.15, which is going to be the one that is going to be released next year. 3.14 went out in October, early October, and yeah, it's in the hand of users right now. So I guess that's what's been keeping me busy for the last year.
- [03:27]
Sean Falconer
Okay. And then in terms of like all these changes that are happening in the world of Python, I'm just curious, since Python has seen such activity, I think in particular to people building on top of AI models and being used to build like AI applications and stuff like that, is some of the catalysts for the changes coming from there or are these just, hey, we have a roadmap, we know we need to make certain improvements and updates and it has no bearing on what's going on in the world of AI.
- [03:55]
Lucas Lange
There is no secret roadmap to where Python is going. It is largely informed by what is happening around us. So obviously AI has a decent influence over how we think about Python, because now not only is Python crucial in developing AI solutions, it is like essentially used in any company that claims to be AI related somehow. It is literally unheard of not to have Python involved in that somehow. But also Python is the language that AI writes. So very often when you don't specify which language you would like for your solution and you ask Claude Code or ChatGPT to help you with a problem, like it's going to default to Python. So those things are on our mind, obviously. However, just like my ex employer used to say, code wins arguments. So very often the way we function is when people come with suggested solutions to problems that they're having themselves. And whether this is a contributor who is an individual or a group of people maybe inspired by problems that their large employer has or whatever, we treat all of those things seriously and those inform what the next version of Python is going to look like. So if you do want to have input in how that works, you can simply by contributing, simply by being part of it.
- [05:28]
Sean Falconer
So Python 3.14 is here. Yes, I guess. If you had to describe the theme of the release, what would it be?
- [05:35]
Lucas Lange
So, funnily enough, there's many changes. Some of them are really big, some of them are tiny, and yet maybe cute enough to mention. But the thing that I personally think is the long term, most crucial change in Python right now is that in Python 3.13 we shipped with this experimental feature that allowed people to disable the global Interpreter lock. The Global Interpreter lock is the thing that assures that everything is going to stay correct and not crash within inside your interpreter, interpreter C code when it comes to the internal data structures of the interpreter and so on. But it also is a big scaling bottleneck. So in 3.13, thanks to some growth and like the big Pep 703 we shipped like an experimental support for making the gil Optional in 3.14, this mode of operation is now supported. It's still not the default. You have a specific special version of the interpreter called Python 3.14T and that is also like you can see this in your site packages there's going to be a 314T directory, like there's TSOs or D Libs on macOS. So those are separate ABI. But we are now saying out loud that this is no longer an experiment. We intend to support this feature which allows the scientific community, which is extremely interested in this as well as others to to slowly or hopefully not so slowly adopt this and make it the default like release or two from now release maybe like super optimistic, so probably not like two or three releases from now would be great. The difference between running the traditional version and the new version is not very easy to note unless you do have an application that already uses threads. So in some cases you might need to introduce some threads to applications that did not have them before, but in other cases you will be able to see wins right away. Like funnily enough we do have in our Asyncio test suite we have some thread pool executor tests and some other tests actually assume that there's going to be some threading going on. So we've seen performance improvements with free threading on Asyncio with no changes to Asyncio before we did any changes to it. Now obviously there's like a lot of tuning inside the interpreter. So 3.14 is actually fantastic for free threading. It reached close to single threaded performance of the version with the single global interpreter lock, but it scales. So if you have multiple cores and even your phone does like everything now has multiple cores now, your application will be able to utilize this much more fully without you having to resort to multiprocessing by hand. So this obviously is the sort of theme for me that we are supporting this now, but I don't think it's going to be the theme for most users yet, since we now pretty clearly target library maintainers and sort of early adopters to essentially prepare the ground, prepare the community, the third party packages and everything for external usage with free threading, which I strongly believe is the future of Python. It's something we should have done a long time ago, but better now than never. And it's shipped now, it delivered on its promise. So I highly recommend you to test it if you have an application that might actually benefit from it.
- [09:28]
Sean Falconer
And then how does this relate also to the support for multiple interpreters from the standard library?
- [09:34]
Lucas Lange
So those are specifically two separate efforts, and to a small extent, fortunately, they were competing, since the entire promise of like free threading essentially says you can start threads now in Python with which means now they can compute in parallel, but they share data, they share everything. And this is good and bad. It's fantastic because you don't have to serialize and deserialize data that you want to share. But it is also terrible because you now share everything, including things that you might not realize that you're now mutating from two separate ends of your program at the same time. And now you're introducing race conditions, you're introducing subtle bugs that might might be sometimes tricky to debug. So free threading is a sharp tool and you might get yourself with it. Whereas sub interpreters inside the same process, they function with the philosophy of sharing the same process, which means they will still share the same limits on the operating system, on how many processes you can spawn and how many file descriptors they can open, and so on and so on. But the idea is that they will not be sharing any data unless actually explicitly allowed. So immutable data might be able to be reused across interpreters with no or little serialization, like with little orchestration. But regular lists and regular dictionaries will just not work. You cannot just use them from two interpreters at the same time, they are controlled by one. And you have to have some now smart and efficient way to exchange data between them. We don't really have much in terms of library support for making that simpler than it would be with multiprocessing, where you already pickle and unpickle data because you're literally passing it to a different process. So sub interpreters really are about isolation and this is the core feature, whereas free threading is sort of about the opposite, about being able to scale your application in a better way than multiprocessing, because you don't have to serialize, deserialize anything because you are sharing everything. So they're kind of polar opposites, and I'm pretty sure they will be used for different use cases. So let me give you an example. So free threading is something that is going to be useful when you have some huge data set and you want to split computing the result for this huge data set among like many smaller pieces. So before we always just, okay, spawned multiple processes and sort of passed the data somehow. But now essentially the process that passes all this data to every worker is the bottleneck, right? Because that process is just one and it orchestrates all the work. And then when the results are computed, you have to pack the results again and again, the orchestrator process is the bottleneck again. So with rethreading that stops being an issue because you don't have to do this entirely packing unpacking process. So that's like the perfect scenario for sharing data and for isolation. For example, imagine that you have some digital audio workstation. You have this sort of ability to instantiate virtual instruments and virtual effects. They can be written in C in any other programming language, then that includes Python. So all of them are in fact running in the process space of this digital audio workstation. If you instantiate this, so is going to start some code and it's going to be running. So that's great. But what if you write it in Python and start 7 instances of the same effect or instrument before subinterpreters before this isolation? That would not be possible. Like literally you would not be able to do this because you would crash the process, since the second effect or instrument that uses Python would write on top of the same data structures in memory that the first interpreter already used. And that would be terrible. So now with subinterpreters you can isolate them so those plugins can live entirely independently, not even know about each other. They can be entirely isolated from each other. So different use cases they are now with the development stage that we are right now both able to scale, but for different reasons and I think for.
- [14:12]
Sean Falconer
Different use cases with the sub interpreter, what is the advantage there versus spinning up completely separate processes?
- [14:20]
Lucas Lange
So like the digital audio workstation is one example, because that environment will literally just not allow you to just spawn multi processes since it is controlled such that the processes don't run away. If you have any sort of containerization, like for example for Mac App Store applications, you can disable this ability for processes to just randomly spawn sub processes and more locked down environments. Like for example the iPhone. If you run an iPhone application, it literally cannot start a sub process. This is not an option, it is not possible. So for this use case, subinterpreters will allow you to have an embedded Python interpreter with your, I don't know, editor or note taking application or any other app where you will allow users to script their iPhone application and that's going to work, it's going to be fine. Whereas just shelling out to a Python sub process, which would literally be impossible, that's not allowed by Apple since again those processes could run away and just drain your battery when the main application died. So there's plenty of related environments like with virtualization containization that actually also want the processes to stay single processes and not just proliferate with multiprocessing. Plus, as I said, we are still at the beginning of the road for internal reuse of our data structures, but we're already starting with that. So if you stick to a subset of immutable data structures in Python with the new Interpreters Concurrent.interpreters standard library, you can already see how you can pass data between interpreters if you want to. So some of that is already pretty efficient. So we are going to build on top of that and and then the advantage on top of multiprocessing is going to be pretty evident that you can safely share immutable data without copying. Whereas with multiprocessing, unless you're using shared memory, which is yet another can of worms in multiprocessing that's impossible. You need to serialize deserialize everything.
- [16:30]
Podcast Host
In mobile application security, good enough is a risk. Guard Square uses advanced multi layered code hardening techniques and automated runtime application self protection and mobile application security testing combined with real time threat monitoring to deliver the highest level of mobile app security. Discover how Guard Square brings all these together to provide mobile app security for your Android and iOS apps without compromise at www.guardsquare.com why is there always a meeting bot in your Zoom call? Blame Recall AI. Recall AI powers the meeting bots and desktop recording apps behind products like Cluli, HubSpot and ClickUp. They handle the hard infrastructure work capturing clean recordings, transcripts and metadata across Zoom, Google Meet Microsoft Teams in person, meetings and more so developers don't have to build it themselves if you're building a meeting note taker or anything involving conversation data. Recall AI is the API for meeting recording. Get started today with $100 in free credits at Recall AI Software.
- [17:42]
Sean Falconer
I wanted to talk about also this new feature around template string literals I guess. Like what is a template string literal and what problem is it solving that you can't already do with f strings?
- [17:54]
Lucas Lange
Cool. So another case where like 314 is like the t release. We had Python 314 t meaning free threading, but here T Strings are template strings. So template strings are not strings. Actually, they are a very convenient notation for creating template objects. They look like F strings because you are using braces for interpolation. But what that interpolation is is arguments to the template objects. This sounds a little weird and confusing, but if you just think about the use case for it, which for example, in the JavaScript world is being able to put HTML in your JavaScript code with backticks you can just put HTML templates and with some brace magic, you can compose components together into something more complex. So now with T strings, this is possible. In Python you use T strings to think like you are still in a string, but what you're getting is in fact an object that later a library can introspect and can build on top of. So you can have HTML processing that will actually put more HTML for you there, or it will compose components together, or it will do user data checks, like security checks, whether some string validates or not, whether it's secure or not, and so on and so on. This can be used for SQL querying and like all other sort of use cases where you want to bring a different sort of notation language into Python, into the same script, and you would want to provide it some structure that later on your code wants to kind of build on. You could do this before with regular strings. The problem with that would be that every time you wanted to kind of make a tree out of your random raw string, you have to parse it. So this would be very inefficient because every time you are passing it back to the user, that template is now kind of a regular string. You're going to make changes to it, they're going to wrap something in another string, and your code needs to be this parser again. Whereas here it's the Python of compile time that makes the objects actually nice. And the notation is already actually constructing Python objects within the bytecode. So it is very efficient, but it looks to the user as if you're just working with cute strings that are just HTML or SQL or whatever else. So it is, I think, a pretty pointed feature where it is not going to kind of rule the world of everything around you. You're not going to be formatting strings with it. That's not the use case, actually. So the reason why we have it is for templating. So for anything that you imagine like you were using strings before and those strings were somewh lacking because it was too easy to just make it wrong, or the parsing was Just making runtime slower. Now, with T strings it is just a way for us to maintain a very user friendly notation, but to allow for efficient computation with the objects being ready for library use. So the PEP that explains the feature is actually excellent. It had to convince people that this sort of feature is in fact, I don't know, like preferable, that it is desirable. So it is doing like a pretty great explanation of why we want this and how the nitty gritty details of it work. I do expect that most people using T strings will not have to think about those intricacies because the library that they're going to be using, whether this is going to be some future version of Django or some database interaction library, they're just going to say, oh, we have all those objects here. But if you want to use a raw query, use a T string and put it here as arguments to this method and people are just going to do it. They're not going to think about what is happening on the other side of the library, that the library is now thankful to get nicely structured objects instead of a silly string that they need to parse themselves. One example where that parsing needed to happen before, and it was sort of always annoying, was when in SQL libraries sometimes you pass some arguments to a SQL query from the user. So you don't want to pass them literally as part of a string, because what if it's just bobby tables? It's some SQL injection, right?
- [22:37]
Sean Falconer
Yeah.
- [22:37]
Lucas Lange
So you're just going to pass the query first and then the arguments 1, 2, 3, 4 separately, a separate argument to the method. So that sort of discipline had to be maintained because otherwise you did something wrong. So now with T strings you can just pass a tstring and just pass those arguments where they belong in the query. So in that way it is more readable to you because you know what's going on when you're reading this code. It is sort of preferable to look at it that way, but it is still safe in the same way because the library can check those arguments, it sees which part of the query is a past argument, is an interpolated value. So it is just like incremental improvement of what we have been doing all along. But it can compose, so you can put templates inside templates inside templates and build an entire tree of that. That's where the HTML example comes from. So I believe that in the end what we're gonna get is some very pointed use, just like with assignment expressions, but it's not going to Take over the world. Not like F strings, but still a very, very welcome addition to Python.
- [23:49]
Sean Falconer
Is there any performance implications?
- [23:51]
Lucas Lange
So, a little bit inside the compiling stage. So when you're first starting your application and it's creating pyc files from your source code, then we're doing additional work because we're actually translating your string into those objects that we're now passing to functions. Right? So we're not actually passing strings, we're passing temporary objects, but that is the single piece of parsing that we need to do. And your code at runtime later does not have to do this. So in fact, the performance implication is positive for almost all uses. So it should be faster to do it that way than the random manual string parsing that we did before, at least of which, because the template strings parsing is part of Python now, it's in C, it's highly optimized. Whereas a lot of the parsing that you would do on your own strings in a Python application would just be done in Python. So just dropping to that lower level of computing already saves you quite a few CPU cycles. So if anything, I suspect that people will welcome this as a performance improvement and not the other way around.
- [25:01]
Sean Falconer
I wanted to also talk about the way type annotations are handled and making sort of this evaluation deferred. Can you explain a little bit about what all that means and how is it different than what was being done in Python 3.13?
- [25:16]
Lucas Lange
Yeah, this is all personal, since part of the PEPs that were actually approved and now are implemented are fixing like a PEP that I wrote and it turned out not to be a good enough solution for the general case. So shortly the problem, when we first introduced type annotations to the language, they had the pretty obvious constraint that they were still living among all the other objects inside your Python script. They were the same object, like any value that you were using in your Python application. So that meant that if you had like a module level function that took an argument that was some class, I don't know, animal, right? And you wanted to annotate it that, oh, this function takes an argument of the type type animal. You had to have this name ready at that moment when that function was defined. But what if this class animal was defined below your function? Well, that's not great because that class does not exist yet. So for this reason we had to support forward references. And those forward references were pretty annoying to spec like to write down because you had to use strings, you had to just say, I cannot use this nice Object because the name animal does not exist on line 30 because the class is only defined on line 70. So I have to say open string bracket, animal close string bracket. That's not great, right? Like a string quote string quote. So the solution that I initially came up with in Pep563 was that how about we do it differently and just make all annotation strings so you have to actually write the quotes, but in the end they will be quotes. So kind of in terms of meaning. So even if you just say animal in your source code, in the end this will just become a string in dunder annotations of that function. This solved this problem and it allowed for a bunch of other nice features. For example, that was in times of Python 3.7, whereas in Python 3.9 we allowed built in collections like list or iterable or tuple to be generic. Which means you could just say lowercase list, square brackets of string, right? So you don't have to from typing import uppercase list you could just say oh, there's a built in list. So I'm just going to say this type argument is lowercase list of string and everybody's happy. However. Well that didn't work with Python 3.7. But if you just said from future import annotations you could use this already because it was a valid Python syntax and it was a string anyway. So Python did not object that actually lists in Python 3.7 couldn't be indexed, it didn't care and mypy understood it. Everybody was happy. And then in 310 we allowed for typing unions to be created using the Pipe syntax. So instead of saying uppercase optional of int, you could just say int, pipe, none. So int or none, right? That is a much nicer notation for optionals or any other sorts of union. Like if you wanted to say it's either int or string, you could say int pipe, string int or string. That's a nice notation. And then Python 3.7 supported it in air quotes because with from future import annotations you could just turn everything into a string. And that was to going great. However, turning everything into strings actually I don't know limits what you can do with those strings later because you cannot so easily find what that string meant in the moment where it was defined later. So what I mean by this is that if you are looking at a notation of your function that takes an argument that is animal object that is animal of what? Like where the does animal come from? What does that word mean? Like what module does this class come from? If you look at it from a different module later on when the application is running, you didn't know. All you knew is that there's a string and says animal. And it turns out that there are some pretty heavy users of runtime type annotation introspection like Pydantic. And they were pretty upset. Least of which because like a popular pattern, it turns out in pydantic applications is to create classes inside a function. So literally those classes don't exist later on on module level where you can get to them in any sensible way. So if you just have a bunch of strings that say like this is some request, this is some order, this is some other type of class, how are you going to refer to this class? It's not importable. It was a local insights of some functions. So like Samuel Colvin was pretty upset about this not being actually possible anymore with from future import annotations with a group of other people who actually depend on this functionality. They raise the objection. They're like, hey, this approved PEP563 actually is now breaking a use case that we depend on. So we stopped actually making this future the default. And for a while we're just looking at a new solution. And Larry Hastings with Pep649 actually came up with it, which is to still defer evaluation. So all this forward reference stuff can happen, but to do it by just implicitly, instead of turning things into strings, you're implicitly turning them into lambdas. So they're essentially kind of deferred evaluation functions that the first time you ask about them, then they will evaluate, but they will evaluate in the context that they were defined in. So even if you had a bunch of classes that were locals, we still hold on to that frame. We still hold on to those locals. So even like half an hour later, when that function call was long gone, you can still ask for dunder annotations from that particular piece of code of Pydantic and it will tell you, oh, that's this animal. It doesn't exist anywhere else in your application because it was just in locals, but you can still refer to this particular real class. And with this functionality, this entire problem is in fact solved in the most correct way possible. It has some edge cases that like, like were a little nicer with Pep 563, but I think the compromise, the end result is well worth it because now the flexibility of annotations is way better. Obviously the devil was in the details. And even though Pep649 is like quite old at this point. It was created four years ago, four and a half years ago. It took a while for the implementation to actually get to the level that was required to actually be part of Python. And El Astra actually tackled with the problem. And when he did, he discovered that there's actually a lot of little and larger edge cases to be discussed and to be decided on. So instead of just making changes to the already approved PEP649, he wrote another 1-749-which explains all those things that needed to be specified additionally for the implementation. And with that, for Python 3.14, this is now the new default future. You don't have to import anything from future. You get this behavior by default. Because the wonderful thing about it is that it is essentially almost wink, wink, entirely backwards compatible with the previous default behavior. We still have the from future import annotations if you do rely on things being strings. But unless you use this, you're going to be using the new automatic behavior of turning everything into lambdas, which is good enough indeed for forward references and for a bunch of other cases which allow runtime use much more than MyPep ever did.
- [33:40]
Sean Falconer
And then for this, what is the impact to existing code that was using things like the future import?
- [33:48]
Lucas Lange
So this is like a good question. So the future import is still there. It is not entirely clear what to do with it. Like the Pep 749, one of the things that it needed defined was like, what are we going to do with this future import? Future imports were never feature flags, they were just a way of enabling a feature that is going to become the default in some future version, hence the name, right? And now you have a future import that will never become the default in a future version of Python. So what to do with that? And Pep749 essentially punted on this deprecation. It says, well, there's still legitimate cases to have this, plus it already works, so there's very little cost for us to keep it. So there are some discussions on our official forum about this. But. But currently we don't intend to remove the future import and we don't intend to change it in any way. We only signify that, like look, in Python 3.14 there's a better default if you just don't do anything. The situation is already better. The problem with us is that we are living literally like pun not intended in the future. Because the core developers just released Python 3.14, we are already working on Python 3.15, so it's easy to lose track of the fact that the most popular versions are still somewhere around 3, 11 or 3.12. So before people actually overwhelmingly reach 3.14 where this new feature arrived, it's going to be still a couple of years and before people can safely say that the libraries I maintain no longer need to be compatible with Python 3.13, well, that's an even longer proposition. Remember, I just released Python 3.9, that was the last ever version and just declared Python 3.9 end of life. And this was in fact moment that a lot of people were waiting for. But why? Well, I guess super old version. Well, because if I declared it officially dead, then library maintainers could use this as a reason to drop Support for Python 3.9 in their libraries. Well, it's not officially supported, so we also don't support it anymore. If you want to still use Python 3.9, use an old version of our library. The newer versions will require Python 3.10. So we are at 3.10 right now of support for the kind of majority of library maintainers. So before we get to that point where the oldest supported version for libraries is going to be 314, that's still close to five years away, which is why until at least Python 3 18, 3 19, we are very unlikely to be able to remove the future imports. So if your code already has it, or if you see it in the wild in your dependencies, don't worry about it. It will just not stop working next year or in two years. It's still going to be around for a long time, but eventually, yes, we will get rid of it, but we will be very careful not to do it abruptly.
- [37:09]
Sean Falconer
When you introduce something new, that sort of replaces an old way of doing something, how long before you essentially take the old way of doing things out of support?
- [37:20]
Lucas Lange
So deprecations is in fact like pretty, I don't know, important topic on our minds right now. So historically the only thing that we said was that there is a PEP that specifies this. If I remember, remember correctly, that's like 3, 8, 7 and Pep 387 is our backward compatibility policy. And it has like, you know, kind of broad strokes of like, you know, don't break people, you have to warn them, you have to like, you know, give them enough time and whatnot. But this PEP was written in times where our release schedule was really like, meh, we release when we're ready. So we were roughly releasing every 18 months, but sometimes it was closer to two years. So saying that what you need to do is you don't remove the feature, you first create a pending deprecation warning, which means this is going to be deprecated in the next version. So this pending deprecation warning is released today. In 18 months there's going to be a deprecation warning and then 18 months from there, which is at least three years, but probably like four or five at that point you will be removing the feature. So that's what originally Pep387 told us. So like hey, at least wait two releases. You need to create a pending deprecation warning and then the deprecation warning and then remove. So it was pretty clear when we switched to the annual release cycle that this accelerated the backwards compatibility policy kind of as collateral of our increased release cadence. And that was not really intended. We still wanted to have our support windows as long as they were before. So since then the PEP was updated to just say that, you know what, you can actually extend this deprecation timeline to as long as five years because there's really not much in terms of maintenance cost cost most of the time. If there is significant maintenance cost to maintaining both behaviors, then we might just have to be more aggressive saying like, you know what, like this old thing goes away, like if you still need it, use an old version of Python. But it's very unlikely, like we very strongly avoid this. So it's more close to five years. But it turned out that sometimes even with five years, what you're gaining from cleaning up things is just this feeling of things being clean. But you didn't really fix any problems for any real users. Nobody's thanking you for there to be fewer methods on some object. Like it doesn't really help anybody. So now, very recently the PEP was also updated to introduce this notion of soft deprecations, where we can now just say that we deprecate a particular behavior only in documentation. It's not even a warning that you're going to see at runtime. We're just deprecating it, saying there is a better way of doing it, use the better way. But there's very little kind of advantage to breaking users code. So we'll in those cases just leave it there forever. And maybe this is actually what's going to happen with the future import annotations because it is not, not a significant cost for our maintenance. Pep 749 explains what I told you before. Like at some point after 3.13 goes end of life, we will deprecate like from future import annotations. But even then it is not clear whether we will actually be removing it, like very quickly or we will just leave it. Similarly, the global interpreter lock, now the build that allows you to disable it, sort of suggests that in some future version of Python this will be the default. And that's true, but whether we will actually remove the global hyperlog entirely is not actually decision that we made. Like, probably it makes sense to just keep it around forever, because fundamentally it is not a huge maintenance cost at this point and it gives you this security network, the safety network of like falling back to this old solution when you are importing a C extension that is old and it doesn't know how to behave in this new world of free threading. So, yeah, backwards compatibility, definitely. Consideration that we're treating increasingly seriously. Like after the Python 2 to 3 transition, we knew that we need to, but now, even with smaller breakages, we really, you think hard to avoid them.
- [42:06]
Sean Falconer
So I know there's a lot of other things that are discussed in the 3.14 release, but what is one of the other features that you think is worth bringing up here that you're interested in or you think other people who are involved in the Python community should know about?
- [42:21]
Lucas Lange
Well, there are things that are actually cool and things that I am super subjective about because I worked on them. Right. So let me start with the thing that is actually super cool and I had very little to do with, which is the safe external debugger interface for CPython. You can read the very long pep and it's very impressive. But long story short, what you can do with Python 3.14 is you can PDB into a running process that is remote to you. It can be in a container and you can still PDB into it. It can be on machine, somewhere over the network. And as long as your network, network configuration, your firewalls and stuff allows it, you can PDB into that networked machine and debug a running process and you don't have to restart it anymore. Like you can just debug now things that are remote to you and they're running. So if they hung on something, you can now learn why you can kind of PDB into an application and debug it as if you were starting it on your own local box, which, which sounds like, okay, cool feature, but is it so foundational? I think it is pretty foundational because this ability to just be able to find reasons for strange behavior is excellent. We already thought very hard and Pablo kind of spearheaded this for instrumentation to be able to just look at foreign processes. So if you had, I don't know, strange behaviors of some memory leaks or, or CPU being hogged by a thing, you could run a profiler on your process and see where does the CPU cycles go, where does this memory go? You could check those things, but very often it was not enough because the only thing that you could see is like, okay, there's many calls here, or okay, this thing declared. Well, actually used this much memory, but you didn't know why. And now actually having a debugger, which means you can issue commands, you can execute command code on that remote Python process. That is amazing. That is, I think, like the kind of killer feature that regardless of whether you're into free threading or not, regardless whether you need like kind of SQL templating or whatnot, regardless of whether you're happy like that now, like forward references in your type annotations work automatically. Being able to debug a process that is already running, I think is pretty cool. So that I have to kind of mention first. But then something that I actually did work on with Yuri and Pablo is additional visibility into Asyncio applications. So all those things that I told you about profilers before, that we could already look into an application that is running and see what Python and C functions at the same time are running right now in that application. That is pretty cool. But what you couldn't really see is what is Asyncio doing at the time. Because long story short, the way asynchronous programming works is you have some event loop, and the event loop runs a particular piece of your coroutine only one at a time, right? Like, the entire point of it was that it maximizes the usage of a single thread. So instead of wasting your entire thread on waiting on the network, you can run other things at the same time. That's great, but you only run one at the same time. So if you just run a profiler on that sort of application, what you're going to see is at any given point, there's only one thing running, and it's running on top of the event loop. So you don't see the causal chain on what is awaiting on what? Like, how many tasks are there in this application? Like, what is actually the cause of those requests taking a second? What is the thing that is breaking the application? So now with Python 3.14, you can see a tree of tasks, and you can see who awaits on whom again, on a remote process you can just direct Asyncio PS add that thing and it'll tell you what is this process busy with. And that I think, again, it's not really like a feature that allows you to import a flashy thing for your code, but it's something that is going to make living with your current applications much easier because it's also the way in which you can just see what is going on much better. Especially that with Async programming, just print debugging is not so easy either since there's many things happening at the same time. So, yeah, like PS3 that I worked on, I think that is cool as well. And the third thing, like, I'm sorry, sort of like, you know, kind of beating my own drum here. But. But I have to say I think it's pretty cool that now we have syntax highlighting in the default repl. So if you just start Python and you just start typing in the commands, having them in colors is just a cutesy feature that you will not think twice about and you're just going to be like, oh, I guess that's cool. That's exactly the reaction that I expect. But believe me, when I say when you then revert back to 13 for any reason and you see that there is no syntax highlighting, it almost feels like it's broken. It feels like you're going back in time to some past that you no longer want to go back to. The experience feels just worse. So it's one of those sort of usability features where the colors just provide you with an additional dimension in which you can look at the code that you're typing in and it just, I don't know, provides some additional polish that I think makes the language feel better. It does for me. So those are the three features that I. I think we still definitely should be covering, but there's many, many more.
- [48:17]
Sean Falconer
Yeah, yeah. Awesome. Well, I think we covered a lot. For those that are listening, check out release notes to learn more. There's a ton of stuff available in this release. And Lucas, thanks so much for coming back on the show.
- [48:28]
Lucas Lange
Absolutely.
- [48:29]
Sean Falconer
Cheer.
- [48:38]
Lucas Lange
Sam.