Ableton Live with Tobias Hahn

Summary

Podcast Summary: Software Engineering Daily – Ableton Live with Tobias Hahn

Release Date: January 22, 2025

In this engaging episode of Software Engineering Daily, host Kevin Ball interviews Tobias (Toby) Hahn, Engineering Manager at Ableton, to delve into the intricate software engineering behind Ableton Live—a leading digital audio workstation favored by both budding musicians and industry giants like Tame Impala and Daft Punk.

---

Introduction to Toby Hahn and Ableton Live

Toby Hahn begins by introducing himself and his journey within Ableton. With a PhD in mathematics, Toby transitioned from academia to the practical realm of software engineering, joining Ableton right after completing his doctorate. Over the years, he has advanced through various teams and currently serves as the technical principal for Ableton Live while also managing a segment of the engineering team.

Notable Quote:

“I started as an engineer on a team that was called the Engine Team back then and made my way through various teams.” – Toby Hahn [01:25]

Understanding Ableton and Ableton Live

Ableton is both a software and hardware company catering to musicians—composers, producers, and live performers. Ableton Live stands out as a digital audio workstation (DAW) renowned for its intuitive UI combined with a robust feature set, making it accessible for bedroom producers as well as professional artists.

Notable Quote:

“We have a wide variety of users using our product.” – Toby Hahn [02:45]

Tech Stack and Legacy Code Challenges

Ableton Live's codebase reflects its 25-year history, primarily developed in C and designed for cross-platform use across Windows, macOS, and Linux (for the Push Standalone). This longevity presents unique challenges:

• Legacy Code: Toby explains the evolution of Live's code, noting the presence of custom classes and differing string encodings (UTF-16 and UTF-8) due to historical constraints.

  Notable Quote:

  “Live is mostly a C application and it is also cross platform.” – Toby Hahn [03:03]

• Maintaining and Evolving: Balancing new features with legacy constraints requires strategic refactoring. An example discussed is the View framework, initially CPU-rendered, which struggled with modern high-resolution displays. The team shifted to GPU rendering to achieve the necessary performance.

  Notable Quote:

  “We have to learn how to render stuff on the GPU.” – Toby Hahn [07:34]

UI Design and Performance Enhancements

The evolution of UI design at Ableton Live emphasizes collaboration between designers and engineers. Toby highlights the importance of maintaining performance, especially with high-resolution displays, leading to significant technical decisions like moving from CPU to GPU-based rendering.

Notable Quote:

“We try to build what they [designers] ask us to build.” – Toby Hahn [12:52]

Real-Time Audio Processing and Scheduling

A core component of Ableton Live is its real-time audio processing. Toby delves into:

• Audio Stack Constraints: Managing audio latency is crucial. The system must handle driver callbacks efficiently to prevent audio glitches, which are more noticeable than UI stutters.

• Scheduling Algorithms: Live employs a domain scheduler that breaks down audio processing into parallelizable chunks, optimizing for multi-core CPUs while managing latency introduced by certain audio effects.

  Notable Quote:

  “The audio stack obviously is a whole different beast.” – Toby Hahn [08:22]

Integration of Software and Hardware

Ableton seamlessly integrates its software with hardware controllers like Push. This integration ensures sound continuity across products, allowing users to transition effortlessly between devices and software environments.

Notable Quote:

“We try to have sound continuity here, which means reusing some of the device DSP across products.” – Toby Hahn [18:12]

Machine Learning and Ethical Considerations

While exploring machine learning, Ableton focuses on features like sound similarity search, employing neural networks to enhance user experience. Toby emphasizes ethical considerations, ensuring artists' work is respected and adequately compensated.

Notable Quote:

“There is something to navigate, what is ethical to do here and what possibilities do we have.” – Toby Hahn [31:02]

Onboarding and Engineering Practices

Onboarding new engineers at Ableton involves navigating a complex legacy codebase. Toby outlines the support systems in place, including team buddies and a culture that encourages engineers to dive into different code areas, fostering adaptability.

Notable Quote:

“We work in feature teams and a new engineer would be assigned to a feature team.” – Toby Hahn [36:28]

Testing and CI/CD Systems

Ableton has evolved its testing frameworks from early visual tests to a comprehensive suite that includes unit and acceptance tests. The transition from using Cucumber to developing their own Python-based framework, Cornichon, exemplifies their commitment to maintainable and reliable testing practices.

Notable Quote:

“We now have a big test suite. I would still say there are probably parts that are not tested or not tested enough.” – Toby Hahn [42:30]

Future Projects and Engineering Challenges

Looking ahead, Toby mentions ongoing and upcoming projects such as:

• Dependency Management: Regularly updating third-party dependencies to stay compatible with evolving platforms like macOS and Windows.
• Windows ARM Porting: Exploring the market potential and engineering challenges of supporting Windows ARM architectures, leveraging lessons learned from previous migrations like the transition to Apple Silicon.

Notable Quote:

“We have to keep up with what's going on, what's out there.” – Toby Hahn [35:43]

Team Culture and Closing Remarks

Toby underscores the significance of a supportive and collaborative team culture at Ableton, praising his colleagues' dedication and celebrating recent product milestones like Live 12 and Push releases. He emphasizes that working with great teammates makes the engineering challenges worthwhile.

Notable Quote:

“One of the things that makes it fun working here is the great team and the great colleagues that we have here.” – Toby Hahn [43:52]

---

Conclusion

This episode offers a deep dive into the complexities of maintaining and evolving a decades-old software product in a highly specialized domain. Toby Hahn provides valuable insights into balancing legacy code with modern requirements, fostering a collaborative engineering culture, and navigating the ethical landscape of emerging technologies like machine learning in music production. For software engineers and music technologists alike, this discussion illuminates the meticulous craftsmanship behind one of the industry's leading digital audio workstations.

Transcript

[Toby Hahn]

Ableton is a music software and hardware company based in Germany. The company develops Ableton Live, which is a digital audio workstation for both improvisation and traditional arrangements. The software is remarkable for successfully blending good UI design with a powerful feature set. This has made it popular with new musicians as well as professionals such as Tame and Paula Knowledge, Mac DeMarco, and Daft Punk, among many others. Toby Hahn is Ableton's Engineering Manager. He joins the podcast to talk about software engineering for Ableton Live. Kevin Ball, or K Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co founded and served as CTO for two companies, founded the San Diego JavaScript Meetup and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow K. Ball on Twitter or LinkedIn or visit his website K Ball LLC.

[Kevin Ball]

Toby, welcome to the show.

[Toby Hahn]

Thank you so much, Kevin. Thanks for having me.

[Kevin Ball]

Yeah, I'm excited to get to chat with you. So let's maybe start. Do you want to give just a brief introduction of yourself, a little bit of your career, and then maybe a little bit about Ableton and Ableton Live, which is what brought us here?

[Toby Hahn]

Sure. So I've been working with Ableton for quite a long time. I started here right after doing a PhD in math and wanted to get out of academia and into a more practical kind of thing. Started as an engineer on a team that was called the Engine Team back then and made my way through various teams. At the moment I'm the tech principal for Live and also an engineering manager for part of the team.

[Kevin Ball]

Awesome. Well, and let's maybe do just a very quick like, what is Ableton, what is Live? What are these things? Then we can dive into the details.

[Toby Hahn]

Sure. So what is Ableton? We make software and hardware for musicians, that is composers, producers, live performers. And Ableton Live is a digital audio workstation with a focus on live performance, obviously, but also works well in the studio or for producing, composing, all that. A lot of people who use Live use it from their bedroom. We call them bedroom producers. But we also have professional artists that I don't want to name people here, but everybody would recognize these names, I think. So we have a wide variety of users using our product.

[Kevin Ball]

That's awesome. Well, and audio is such a fascinating space because you end up with very different constraints than a lot of software developers end up with. So let's maybe dive in a little bit, starting with the tech stack. What does the tech stack look like for Ableton Live? Is it the same across other Ableton products? Are you different? Like what does that look like?

[Toby Hahn]

Oh, that's a very deep question. Very good question. So maybe a little bit about the history. Ableton just celebrated its 25th anniversary and the code for Ableton Live goes back all the way to the start of Ableton because that's what started the company. So what we're dealing with is a lot of code that has evolved over 25 years. And if you imagine 25 years ago, the world looked a little different and also tech stacks looked different and that still influences to some extent what we use today. So Live is mostly a C application and it is also cross platform. Cross platform originally means that we Support Windows and macOS, but since the release of Push Standalone, which also runs Live on an embedded Linux computer. So we fundamentally have three platforms on which Live has to run and that means we have to have a lot of support infrastructure. There are not that many frameworks you can just pick and choose from. Also, C has grown and evolved a lot over the years. So in 99 I think it was at the time of like when the first C standard came out, but there was still a lot of incompatibilities between compilers and the standard libraries, I'm told wasn't in a very usable state. So that means we have a lot of homegrown classes for things that people would nowadays probably just used from the standards. So we have various classes for pointers, smart pointers, some of them are not so smart is what I sometimes tell new colleagues. And then we also have various string classes also sometimes with different encodings. So Live traditionally where the legacy code base traditionally uses UTF 16 encoding. Some of our newer parts use a UTF 8 std string encoding. So we also have some string conversions going on back and forth and a lot of that kind of legacy that comes with the software but also makes it interesting to work in.

[Kevin Ball]

Yeah, well, and I feel like this is an under discussed area in software development. Like we love to focus on the new shiny, but most of us then go to a conference, learn about the new shiny and go back and work in our legacy code base. I'd be curious how you deal with some of those constraints. How do you keep the product evolving and not get bogged down? What does that look like?

[Toby Hahn]

This is an interesting question. One of the things is, I think a good example here is our View framework because weren't that many cross platform choices available back in the day. We have our own homegrown View framework for better or for worse. But when you think back to like the early 2000s, screens were nowhere near where they are today and you had display resolutions of maybe, I don't know, 800 by 600 or 24 by 768, some of those resolutions. And now we have four 4K, 5K monitors, standard, sometimes even more. And the issue for us as engineers that comes with that is if you have that much screen estate, you can also display a lot of things on the screen. And back in the day everything was rendered on the CPU on the UI thread and at some point that just didn't scale anymore. So we, we're targeting kind of a 60Hz frame rate and for a 4K monitor we can't get 60Hz with CPU based rendering. So when we realized this, we said, okay, what are our options here? Porting to a new UI framework. We tried that for a while, but had to learn the hard way that a big rewrite isn't a good idea. The short story is that in order to get the kind of performance that we need, we would have had to hand optimize a lot of things, which.

[Kevin Ball]

Then loses you a lot of the benefits of the framework and all.

[Toby Hahn]

Yeah, totally, exactly. And so we said, okay, then we have to learn how to render stuff on the gpu. And we first had a project to accelerate Mac rendering using Apple's Metal to technology. And we're now in the process of doing the same on the Windows side and hopefully then we'll have rendering that scales for the 21st century with display resolutions that we have these days.

[Kevin Ball]

Yeah, well, and that kind of gets into some interesting constraints that you face in some ways similar to gaming. Right. So you have frame rate, latency constrictions, you have the video, but also in audio, especially if you're doing live and you're looping and you're doing all these different pieces. Like how do you deal with those kind of real time audio latency constraints?

[Toby Hahn]

The audio stack obviously is a whole different beast because if you have a UI dropout, like a little stutter on the screen, like maybe it gives off a bad impression. But if you have a click in your audio or too many clicks, then you will notice that and that will sound very ugly and people will report that as a bug if it happens in their setup too often. So the constraints that we have is we have a driver callback. So the operating system tells us, hey, please prepare a new audio buffer. As a user you can choose, okay, what is the sampling rate for my audio interface? And what is the Buffer size. If it's a larger buffer size, that means we have more time to compute more data. If it's a smaller buffer size, we have to be able to react faster. So that is a bit more taxing on your cpu. And we then have to schedule all the work that needs to be done to compute all the audio that is needed for that next buffer and also obviously process the input if you do a live recording from your microphone or other inputs.

[Kevin Ball]

Got it. And I may be off base here, but I remember hearing at some point that macOS in particular was a lot more supportive of real time audio than Windows. And I obviously don't know in the Linux case, like, do you end up having to handle each of the different OS systems differently to keep things up to date or how does that end up working?

[Toby Hahn]

We have driver backends for several architectures. On macOS, we have a core audio backend on Windows, the backend we support is asio. We also have mme, but that's not really suitable for professional kind of audio. So we recommend using ASIO on Windows. On Linux, I believe we're using Alta. I would have to look it up, to be honest.

[Kevin Ball]

And is the model enough that the abstraction can all live in the driver or does that end up leaking into other parts of the code base?

[Toby Hahn]

We tried to abstract that away and as you can imagine, the audio interface parts are some of the oldest code. And also some of the people who founded the company, they couldn't hire really the best engineers on the market back then as a startup. So some people also learned coding while they were here, but built this company that supports all of us. Now where I'm getting at is that that code is a bit of a spaghetti code and we do have to look into making that simpler. It's just a thing. It's a bit of a legacy that comes with it. And it's one of the refactoring projects that we have going on, like slowly chopping away there and finding better abstractions that make it easier to maintain this kind of code and maintain the software going forward here?

[Kevin Ball]

Yeah, the engineering manager geek in me wants to dive in. Like, how do you manage that type of refactoring project and sort of prioritize against ongoing development and things like that, like keeping it moving.

[Toby Hahn]

Also a very good question. Engineering management here at Ableton I think works a little different than in some other companies. We're more a product focused company. So we have heads of our products. So we have a head of the live unit who is like the product owner who decides what features we build. And individual teams have team product owners who negotiate then with a product owner what their team is working on. And we also have technical chapters on the side who get together regularly during sprints, but have four dedicated chapter sprints per year where they can come together as a technical team and work on technical epics.

[Kevin Ball]

Moving into another area that's really interesting to me, which is around UI design for music production. So we touched a little bit on this as you talked about adapting to larger screens and being able to do this. But kind of how do you think about the trade offs and in particular like things like balancing the sort of recording and composing piece of things versus live performance. Are those completely different interfaces? Like what does that look like for Ableton?

[Toby Hahn]

Do you mean in terms of performance, where to give computing performance or.

[Kevin Ball]

I was honestly thinking from a UI design standpoint, if we go into that or how you're thinking, I mean, I don't know, we could dive into performance as well. Where do you spend your thought on the UI side?

[Toby Hahn]

So on UI design I try to not interfere too much because we have excellent designers for that kind of.

[Kevin Ball]

Fair enough, fair enough.

[Toby Hahn]

And we try to build what they ask us to build. Obviously we have some constraints because we have to maintain our own UI framework. So we don't have all the new shiny bells and whistles that come with UI frameworks where they have full time teams working on the framework. But in general it's a process, a collaborative process where a designer and engineers get together, explore in prototypes. Okay, what could this look like? And designers have a design language and when they want something new that hasn't been there before, they go ask team, okay, how expensive would it be to build this kind of extra large button, for example?

[Kevin Ball]

Makes sense thinking about that and kind of tapping into the legacy and the fact that you have these frameworks then is the sort of design UI side, does that feel separated? Is it like conceptually a separated front end or do you have a little bit of those kind of deep linkages or spaghetti code into how the back end is implemented and pieces like that?

[Toby Hahn]

I think the design language and the UI design is separate from live. I would assume that if you ask a designer the same question, they also say, okay, we have legacy that we carry around in our design language. But I'm not confident enough to say that. But we have on the engineering side, the UI framework has also a lot of legacy that it carries. Like the widgets have grown. We like to think of it of A bit of a widget zoo with deep inheritance where people added a functionality for a feature they needed in one device or another and it's one of the things we have identified as something that we need to improve a little bit. But as you mentioned earlier, it has to be prioritized against a lot of other things, against features and other technical improvements. On the UI side, we actually looked at the problems that we have and said performance is actually the most important thing right now and we need to to do accelerated rendering before we can look at API improvements or widget refactorings and cleanups.

[Kevin Ball]

That makes sense. Are your software deployments secure by design? Lately, Secure by design and shifting left principles have been hot topics in the software industry, pushing development teams to make security a foundational part of software development. Today's sponsor, Bitwarden supports developers in securing every phase of the development lifecycle with end to end encrypted credential management. This ensures software is built on secure principles to prevent data leaks and unauthorized access. Try Bitwarden Secrets Manager, built specifically for developers to safeguard infrastructure and machine secrets, or Bit Warden Password Manager for everyday logins and other sensitive information. Start a free trial [e-mail] Another area that I'm interested in you mentioned at the beginning, Ableton is a software and hardware company. So what does that end up looking like for you all? Is there a co design process? Are the groups independent? Like, is the software designed for the hardware? Like, how does that relationship flow through the company?

[Toby Hahn]

We have a remote script framework to control hardware and built on top of that is the API that we also used for the first version of Push. Push 2 then came with a full color display that had a lot more pixels. Also and interestingly enough the rendering technology that we use on Push 2 is what we tried to use for Live. What I mentioned earlier, it wasn't good enough for Live, but for Push it just did the job so that code ended up there. But Live and Push share a lot of code, especially with push 3 we have tethered in standalone and in standalone it is a Linux computer running live. There's obviously an additional part of the software that does all the Push UX rendering the display, but for standalone, also doing mundane stuff like connecting to WI fi, that kind of thing. So that code is very close to the live code and a lot of it lives in Python. We have a Python API into Live, which is mostly the same API that we also expose viamax for Live and these two evolve in lockstep. It's the same code base for our newer products, A Note and Move. So A Note is an iOS app that we also have. Moov is a small groove box that we just released. We also have some shared code, but it's much more loosely coupled to Live. So there are modules and libraries that we share, especially around all the sound generation and effects. So we want sound continuity across products. So if you make a jam on Move and open it in Note or open it in Live, you want to be able to finish that song or continue with the song sounding the same way. So we try to have sound continuity here, which means reusing some of the device DSP across products. But the audio engine, for example, on Note and Move is a new audio engine that we rewrote from scratch with the idea that maybe it could replace the Live engine one day. But I think we discovered there is a lot of knowledge and special cases and functionality in the Live engine that would just take a lot of time to add to the new engine. And so some of it is shared, some of it is quite separate.

[Kevin Ball]

Yeah, that reminds me a lot of what kind of like Mozilla went through when they started building Servo as a new browser engine in Rust. And then they started trying to pull things back. And some things are easy to pull in because it's nicely isolated and you can separate it. And some things, it turns out there's a ton of embedded value in that.

[Toby Hahn]

Legacy code, and pulling back things is a good keyword. So one of our engineers on our real time Live Audio engine team noticed that there is a component in the new engine that we can also use in the old engine for some performance improvements. So it's a bit technical, so we split up. So the Live set conceptually is a graph where audio flows in some directions. It can also have a cycle. So we don't forbid you from routing your outputs back into your inputs. Obviously, we have to break those cycles when we create the engine. And then we try to split this engine graph into parallelizable domains. And as soon as all the inputs for one domain are ready, we can compute that domain and then make the output ready to whatever output domain needs it. And this domain scheduling algorithm is something that is also very old in the code base. So today's architecture looks different than the architecture of computers back in the early 2000s. And so we had to go and make some performance improvements there. And she noticed, okay, we can use a work ceiling queue that we have in the new engine also for this domain scheduling algorithm here. And so some of the Parts do actually make it back into the live engine.

[Kevin Ball]

Yeah, well, that kind of goes to an interesting thing. So you mentioned a lot of this code is its legacy and some parts of it are really deeply spaghetti and some parts are not. What are the conceptual chunks that go into audio software? And to put a little context, right, like I have a web background and a lot of web servers. You think, okay, you've got your data layer, you've got a stateless web server, you've got a front end server. Like, there's sort of well defined chunks for someone who's new to audio. Like, what are those well defined chunks that go into a piece of software like Ableton Live we have in live.

[Toby Hahn]

As chunks in the software I would say is at the core is a data model layer that represents your lifeset. And then on top of that data model we have a view which we then display on screen. It can also be more than one view with push and you can control the data model from this UI layer. We then have another conceptual view on it, which is the engine abstract engine representation of the live set, which says, okay, this live set corresponds to this engine graph. And this is then responsible for building the audio engine out of the individual DSP modules that we have.

[Kevin Ball]

And you mentioned for that, it kind of like you can break apart the graph, you can break the pieces into these parallelizable chunks and then is it a direct render? Is there, like, is it a compilation sort of pipeline of some sort? I'm just kind of wondering like, so once I have one of these parallelizable chunks, is that immediately renderable into audio or do I need to take it through some sort of processing phase to get there?

[Toby Hahn]

So you have some inputs there. These inputs can be audio inputs, but can also be parameter inputs. When you have, I don't know, gain parameters or distortion or some other parameters that go in. You can also have MIDI inputs if it's in the MIDI part of the signal chain. And then you have processors who compute for one buffer of audio. Okay, what is the output for those inputs that I have? And potentially some state that carries over from previous buffer.

[Kevin Ball]

Got it. Okay, that makes sense. So what is that then? Like how much connection is there? State buffer to buffer? Is each chunk mostly stateless or you're carrying a lot along as you go?

[Toby Hahn]

That really depends on your processor. For example, a mixer doesn't really have a lot of state, it only has how much gain to apply. It doesn't need to keep any previous audio buffers around. But for something like a Limiter, you need to have a little bit of look ahead to see how much limit you need to apply for the next audio buffer. So you, you have, depending on what you choose, some look ahead where you say, I need that much gain reduction in over the next offer. You can also have devices like Echoes. And then you obviously need to keep however long your reverb tail is for a reverb around. So some of these are quite stateful, others not so much. And I think it, it depends a lot on what you do in a specific module that makes sense.

[Kevin Ball]

And so then as you're doing these processing chunks, is there like a synchronized clock that is going along, like you might imagine, in a CPU or something like that, where it's like, okay, this is for the next 20 milliseconds or whatever that is, or how do you do all those synchronized pieces?

[Toby Hahn]

We have a scheduler class which provides this kind of clock and which synchronizes events that need to happen within buffers. For example, if you have a ramp of some parameter that changes over time, but conceptually, every domain has its own domain scheduler, which is the source of truth for time for that audio domain.

[Kevin Ball]

Got it. So then when you have this, Sorry, and these may be very naive questions because I'm completely new to audio processing, but you have this sort of graph of here's all the different pieces that need to get rendered. I'm going to take this chunk and go render it. Is that limited to a particular cycle in the clock? Or those things might have different sort of clock cycles that they're on.

[Toby Hahn]

So it all goes back to audio driver. The audio driver says, I need the next audio buffer. And then we need to compute the full buffer over the whole graph. And some of the pieces of the graph might be able to render in parallel. For example, if you imagine you have separate tracks and no routings between those tracks, then those effects don't depend on each other. And you can, if you have enough CPU cores, render them in parallel. You can also render them sequentially, and depending on when you render them, the respective domain scheduler has to run the time for that buffer. Does that make sense?

[Kevin Ball]

I think so. So your chunk size for rendering is driven by the request from the OS of I need this buffer, which then corresponds in some ways to a length of time. But then each domain scheduler is keeping track of like, where is that in my domain? What amount am I rendering here exactly?

[Toby Hahn]

And there's also latency involved. Some devices, like the limiter that I was mentioning, or a Compressor, they need to look. Be able to look ahead a little, which adds latency to the audio computation. That means you need to then delay other tracks for the same amount of latency so that everything ends up in sync. And that means you also need to take the latency of each device into account when you calculate. Okay, what time is it now? For me?

[Kevin Ball]

Yeah. Wow. So the scheduler sounds like it is actually one of the sort of really fun and meaty problems that you're tackling here.

[Toby Hahn]

It is. I think it's also very much at the core of like the old audio engine. I personally have not work too much on the scheduler. I've sometimes worked with a scheduler. So I can't give you too many insights into fun engineering problems that were solved. I didn't solve any there. But this is one of our core components for our engine.

[Kevin Ball]

Yes, that makes sense. Another thing that stands out, and maybe we've already touched on this, is like the way that you integrate with not just your own devices, but other devices, and you have kind of control surfaces for a vast range of potential inputs. What does that end up looking like on the software side?

[Toby Hahn]

So for other devices, we support plugin standards on Windows. This is VST version 2 and 3 on Mac, also audio units. And we then try to map those plugin standards onto our own abstractions. Control surfaces are a different beast because we define the API into Live there and we work with then hardware manufacturers to write control scripts to integrate into Live, so that when you press a clip launch button on the Surface, it actually launches the respective clip or it highlights the correct part of your live set that is currently controlled by your external MIDI controller.

[Kevin Ball]

Got it. Are there other parts of the architecture that we haven't dived into yet here?

[Toby Hahn]

There are some other parts. A big part of Live is the browser, which is not a web browser, but it's the thing where we display what effects are available, what samples you have in your sample collection, what presets you have, what plugins you have installed. And this is backed by a large database on the same computer, obviously. And there are quite a few engineering challenges around that.

[Kevin Ball]

I imagine starting back in, you know, 25 years ago, you probably don't just have a SQLite thing that you are able to query there. Or is that.

[Toby Hahn]

I think IT is a SQLite database under the hood. Yes.

[Kevin Ball]

Nice.

[Toby Hahn]

And we're also constantly iterating on the browser, trying to improve the user experience. Sometimes users disagree. Initially for Live 12, one of the big features is that we now have tag based search in the browser, so you can tag all your presets, all your sound, and then use those tags to filter what you have. And some users prefer the old style of navigating a tree structure. And I think this is where we believe that if you use it for a little bit, then this is actually the better solution. And it also neatly ties in with another feature that we have there, which is auto classifying your sounds. So we try to analyze all the sounds that you have on your hard drive and then you're able to say, okay, give me a similar sound to this one. And this is quite a neat feature to explore what you have in your collection and also quite an impressive engineering challenge. I wasn't on that team, but I was very impressed when I saw that.

[Kevin Ball]

Feature the first time. I'm curious, so I don't want to push you into a direction you're not super familiar, but can we dive into what does that analysis look like? What are they using under the hood there?

[Toby Hahn]

So if my understanding is correct, we train a custom machine learning model there and we have a neural net that we use to do the similarity search on those different sounds, to cluster them and say, okay, this is the closest sound.

[Kevin Ball]

That's awesome.

[Toby Hahn]

Based on the analysis that we have here.

[Kevin Ball]

Okay, so talking about machine learning, there's a bunch of stuff going on in that space and I know some of the enthusiast groups I'm involved with are really excited about generative AI around music. And I think a lot of it right now is, you know, you can get, generate a track or you can generate something, but the tooling isn't really very advanced for actually arranging or doing anything like that. Is that a domain that y'all are pushing into or just kind of people are importing clips once they're already generated? Or what does that look like for Ableton?

[Toby Hahn]

I mean, we're obviously exploring what it means for music making, but it's also a bit of a sensitive topic because a lot of artists, I think, rightfully have the impression that their art is used often without them being compensated for it. So there is something to navigate, what is ethical to do here and what possibilities do we have? Also, we are not any of the big companies that can host like big clusters of GPUs to train models. So I think there's something like our sound similarity search that we can use and there are other features where we're looking into if we can use machine learning for them. So it's definitely something on our mind, but it's Also something where we have to be a bit careful and be sure to do the right thing here.

[Kevin Ball]

Yeah, that makes a ton of sense. Well, and there's a sort of. Since none of the companies that are doing that actually have any useful, as far as I can tell, like mixing and editing tools, you could be the neutral third party. All right, you generated it. It's on you to navigate the ethical things. You can import it here and manipulate it.

[Toby Hahn]

Sure. I mean you can use any kind of recordings or a sound that you created from elsewhere in Live and then edit and produce your final track with it. It's definitely one of the things that we try to be good at. One of the key workflows of Live.

[Kevin Ball]

So let me move us in another direction, which is kind of what are you all working on now that gets you excited that you can talk about. You mentioned the performance side, right. So really improving the performance and rendering on the UI side. What else are interesting upcoming projects for Ableton and Ableton Live.

[Toby Hahn]

So for user features we have a policy that we don't talk about it until we publicly announce it, but maybe I can try to think of some engineering refactoring challenges that come up. So one of the things that is more of a bit of a bookkeeping is like keeping all our third party dependencies up to date. We integrate quite a bit of open source software, but we're not in the same situation as with a web framework where you have a package manager where you say, okay, please update to the latest version and then you regularly fix the bugs. So we unfortunately update a little less frequently. But we still have to do that because Apple keeps releasing new compilers on new versions of macOS that we want to support. Also Microsoft releases new compilers and we just have to keep up with what's going on, what's out there. A big challenge that is coming up, I think is looking at what is happening with the Windows ARM space. So some of the machines that were released look like they might be quite interesting for a music maker. So we're looking into is there a market that we can justify to go in that direction? And this is I think an interesting also engineering challenge. We solve a lot of the hard problems, I believe, with the port to Apple Silicon when Apple announced their migration away from Intel. So for example, back then we still had a number of like handwritten assembly routines and we did not want to port them to ARM assembly, understandably. So part of that project was to porting assembly routines to standard library facilities with STD atomics For example, or STD thread, those kinds of things that the standard library provides and where we then can rely on the compiler to generate code that is optimal that makes sense.

[Kevin Ball]

That actually reminds me of another topic related to this sort of cluster. And if I were to describe a lot of what I've heard here, you have a whole cluster of things related to the fact that this is a 25 year old application and that just kind of plays out through all parts of your process and what you're needing to deal with, something you sort of mentioned very briefly but we didn't dive into, is onboarding and onboarding new engineers. So when you have this huge stack of legacy and embedded knowledge and all of these different things, onboarding something to someone to that feels like also a big area of challenge. So I'm kind of curious how you approach that. How do you what does an onboarding process look like for a software engineer coming into this stack of 25 years of knowledge and spaghetti code?

[Toby Hahn]

So an onboarding process is a bit of a challenge here because even if the person knows their way around C, they have to learn all of our own in house idioms. And that, to be honest, can take a while. So an onboarding process, we work in feature teams and a new engineer would be assigned to a feature team. They would then get a team buddy who is their buddy for everything in the first couple of weeks to help them navigate the organization, the code base, whatever they have, and explain the fundamental idioms like the smart pointer types that we use on the go. But it can take quite a while and some experience that people repeatedly make on the team is okay. They get to know one part of the code really well and then they move to a different team or their team pivots to a different feature and they have to learn new things. So one of the qualifications that we look for in engineers when we hire is being able to dive in and start swimming, metaphorically speaking.

[Kevin Ball]

Yeah, absolutely. So I think about that and I think about throwing engineers into a new code base and then I'm obviously thinking about, okay, what does the test framework look like to support them so they can make mistakes more safely? What does the observation or observability stuff look like so that, you know, if something screwed up? So what does that look like within Ableton? Do you have a good CI CD set up?

[Toby Hahn]

Like so as you can imagine, this also grew a lot over the years. Interestingly enough, for a very long time we went without.

[Kevin Ball]

It doesn't shock me, to be honest. Once again, the gap between best practices, what you hear at the conference and what most companies are doing.

[Toby Hahn]

And I think about 15 years ago that led to a bit of a quality crisis where we realized, okay, we have to do something and we have to start writing unit tests. Back then we had a visual kind of test suite, would try to play back event scripts of user events and you can imagine those things break very frequently. And it's also not clear if it breaks. Is this expected? Is this actually a bug? So we got rid of that. The first thing we introduced was unit tests using a homegrown test framework. There weren't that many mature ones to choose from back in the day. The next thing we added was an acceptance test layer. So we started using Cucumber, which is a Ruby framework but allows to. It has a wire protocol that allows it to make calls into other applications, including native applications. However, we also made a classic mistake, I would say, by hooking into implementation details. And that bit us because the Cucumber team heavily refactored or rewrote I would say Cucumber several times with really fundamental architectural changes that were incompatible with our internal hooks. So we had to stay on an ancient version of Cucumber which only was running on a very old outdated version of Ruby. And at some point it was like, okay, what if our operating systems that we have to use aren't running that version of Ruby anymore? And fortunately one of the chapters took this on as a task and rewrote our own version of Cucumber in Python. We call it Cornichon because it's little cucumbers and now we control the whole code. It was surprisingly straightforward project. I mean I should also add one of our most experienced engineers was on it. But still we were afraid of this kind of rewrite for a long time because we knew we had legacy there, we had things we had to clean up. And when we actually did it, it was surprisingly quick to do it. We had to adapt a number of tests to the new framework to make sure everything stays the same. I think it was mostly due to some of the internals that we were depending on that we didn't want to reproduce one to one where we had to change step definitions is what they're called, to map them to new ones. But now our accepts test suite runs on our own Python based cornishon and we have our in house continuous integration system. So we actually have servers and a team that provides us with VMs on these servers and on pull requests, engineers can trigger runs on that CI system. We also have two CI systems, we have what we call old CI and new CI. Old CI is based on a very old version of Jenkins and. And new CI is a more recent version of the tech stack. And it's a very big project to actually port from the old CI system to the new CI system. But at the end of the day, the system gives us a green or a red. And that means, okay, we can release this build or we can't. And engineers know, okay, I can merge this pull request or I have to look why this test is failing.

[Kevin Ball]

Well, that makes it a lot easier to do what you described of jump in and swim.

[Toby Hahn]

Yeah.

[Kevin Ball]

And I think one of the definitions I once heard that stuck with me for legacy code is legacy code is untested code. So in some ways you don't have legacy code anymore. You've got this great test suite around it.

[Toby Hahn]

We have a big test suite now. I would still say there are probably parts that are not tested or not tested enough. Some things are just very hard to test, like the system audio integration. Changing audio drivers, for example, is something that's not something that we can do well in a unit test these days. But over the years we've added quite a bit of test coverage, I would say. And we also have a strong code review culture. So as I mentioned, engineers work on teams and we have the rule that every code that gets merged to main has either has to be paired or has to be reviewed by at least one other engineer. It takes a bit of time to learn. Okay, what engineers should you pull in for run of the mill feature pull requests. It's often your teammates, but if it's touching a subsystem that you're not familiar with, then you need to know, okay, I need to call this engineer or I need to ask that person who's been around for longer for a code review to help me. And you can also reach out to these people earlier, obviously to pair.

[Kevin Ball]

Awesome. Well, we're about at the end of our time. Is there anything else people should know about software development at Ableton?

[Toby Hahn]

Maybe one thing to mention is what makes it fun working here is the great team and the great colleagues that we have here. I believe we have a very supportive team culture here and we also have our flaws. But this is one of the things that I enjoy most, like coming into the office or coming onto Zoom and working with these amazing people who make all of this possible. So I think I can't give the team enough praise here for what they've done, especially this year with a release of Live 12 with Move last year with a release of Push. So we have a great team, and I'm really proud of it.

[Kevin Ball]

That's awesome. I feel like one of the things I've learned and as I've gotten up there is life's too short to work with jerks. Right. Like, you want to work with people that where you get into work and you're excited to work with them.

[Toby Hahn]

Yes.

[Kevin Ball]

Awesome. Well, let's call that a wrap. Thank you, Toby.

[Toby Hahn]

Thank you so much, Kevin.