← Software Engineering Daily
Software Engineering Daily
Chip Design in the AI Era with Thomas Andersen
00:50:15
Synopsys is a leading electronic design automation company specializing in silicon design and verification, as well as software integrity and security. Their tools are foundational to the creation of modern chips and embedded software,
Loading summary
Transcript71 lines
- [00:00]
Narrator
Synopsys is a leading electronic design automation company specializing in silicon design and verification as well as software integrity and security. Their tools are foundational to the creation of modern chips and embedded software, powering everything from smartphones to cars. Chip design is a deeply complex process, often taking months or years and requiring the coordination of thousands of engineers. Now, advances in AI are beginning to transform the field by reducing manual effort, accelerating timelines and unlocking new design possibilities. Thomas Anderson is the Vice President of AI and Machine Learning at Synopsys, where he has spent over 15 years. He joins the show to talk with Kevin Ball about the evolving role of AI in hardware design, the challenges of training models on tacit undocumented chip engineering knowledge, the emergence of domain specific LLMs, and where this fast moving field is going next. Kevin Ball, or K. Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through latent space. Check out the show notes to follow K. Ball on Twitter or LinkedIn or visit his website Kball LLC.
- [01:34]
Kevin Ball
Thomas, welcome to the show.
- [01:36]
Thomas Anderson
Hey, very nice to be here. Kevin, it's a pleasure.
- [01:39]
Kevin Ball
I'm excited to get to know a little bit more about you and what you're doing at Synopsys, but let's maybe start with you. So can you give us just a quick background of who you are and what brings you brings you here today?
- [01:49]
Thomas Anderson
Yeah, I'm Thomas Andersen. I have been a long time we call them Synopsoid. So Synopsys employee. I've been there for more than 15 years and I'm running essentially the AI and machine learning team and Synopsys developing automation with AI for our software products.
- [02:09]
Kevin Ball
I love this topic. I'm completely geeking out on all of machine learning and AI these last couple years. But but before we dive into that, let's maybe do a quick overview of EDA and what Synopsys does in general, since we have folks across a wide range of software engineering backgrounds.
- [02:25]
Thomas Anderson
Yeah, totally, because I think obviously not everybody's familiar with what Synopsys does. Even though we are one of the 20 largest software companies, we're obviously a B2B business and therefore not everybody is as familiar with us. We essentially we provide solutions, what we call silicon the software. So we provide offerings to help our customers build and design their chips as well as implement and test software offerings. So essentially from the entire spectrum, from what's called the Electronic design automation software. So anything that's required to build the little chips in your phone and in your TV and in your car all the way to ip, that goes under those chips. And also from a software side, obviously, it's not just hardware. You have a lot of software that runs on top of this whole stack. We provide solutions for application, security, testing, and software integrity. So that's our main offering, essentially. If you think about it, next time you pick up your iPhone or your Google Pixel or whatever is your preferred choice. Think about it. That little chip in there, as well as part of the software that runs on it, was essentially created and designed with Synopsys software.
- [03:39]
Kevin Ball
Yeah. And it was fun to me doing the research, preparing for this, because I have lived most of my life in sort of the consumer software space that starts at an abstraction layer where I just assume all of your stuff is already working.
- [03:52]
Thomas Anderson
Well, that's sort of where the magic happens, right? See, a lot of the people, they understand software, but I think not many people are familiar with hardware or how hardware is designed. So the assumption is, yeah, that thing just works somewhere. But if you think about, like, building a chip has so much complexity to it. It's not just the logic side that you have to do, like the equivalent of software that something runs on the hardware that computes something. There's also all the physical aspects, right? They need to be manufactured in fabs with very, very small nodes. And that's like super, super challenging pretty much across the spectrum.
- [04:29]
Kevin Ball
All right, so let's now talk a little bit more specifically about what you're doing. So how are you applying machine learning and AI to this very deep software stack that Synopsys offers?
- [04:41]
Thomas Anderson
Well, essentially, if you think about it, so the software we're building, our users are expert designers or engineers. Right. It's not software like Microsoft Word, where everybody essentially is familiar with it and just types something in it. It's very, very complex software. And designing a chip from start to finish, like our customers, it can take 12 to 18 months to build a chip from start to finish. That sounds like an exorbitant amount of time, as you think about it, compared to software design. Right? Software design, people like, I mean, of course, I don't want to simplify it, but you write something, you compile it, and you can get something out relatively quickly. Chip design. I remember talking to an executive at Google, actually, and they were fascinated by this. We were working essentially on some reinforcement learning together. And I remember he was telling me, he said, it's just fascinating how Complicated it is to get like those chips out the door. So it's a very, very long process. The tool chain, essentially the EDA electronic design automation software that we provide is also highly complex and the manuals are complex. The way you optimize your design requires a lot of human interaction and there is a very, very high desire, of course, to reduce the human manpower that is required to get a chip out the door. Today, I would say chips, you can take thousands of people, really, it's no underestimation at all. It can take thousands of people to get this thing done. If you want to do it faster, if you want to do it with less people, well, we need automation. And of course AI is the path to it. But because it's complex, it's not as easy, right? It's not as easy as say self driving cars.
- [06:27]
Kevin Ball
I love you. Use that as the baseline. Not as easy as self driving cars, which they've been working on for what, 20 or 30 years. And we keep saying, oh, it's going to be here, it's going to be here.
- [06:36]
Thomas Anderson
That's exactly why I said that, because it's a funny joke. Like Elon Musk has been saying this for how many years they're going to have it next year, next year, next year? I think he stopped talking about when exactly it's going to happen. But in fact, other companies of course I would argue have better systems. Right? But regardless, we're still not there, right? I think Mercedes has like a level three certification and to get to level five, I'm sure will take a long time.
- [07:00]
Narrator
I.
- [07:01]
Thomas Anderson
And now think about it. Driving a car, you can argue, is a very simple task. As long as I can hear, I can see and I can move my hands and my legs, I can operate a car. Right? Relatively simple rules. And at least in theory, it is something that I would argue a machine can do better than a human simply because, I mean the interesting thing there in my opinion is that the simple things that we as a human just know, to teach that to a machine actually takes time. But then when it gets to more complex things like let's say you're driving on the road and there is a situation where you have to do some emergency handling to avoid some object or some kind of scenario. There's no doubt in my mind that the machine can do this better because if I train it with all these possible scenarios, it can calculate exactly what action needs to take. While I'm just in panic mode, I don't know what to do. I've Never done it before. So I'm only familiar with what I've done before. But not new things. Right. That I've never experienced. So that's interesting. But regardless, I would still say that compared to operating complex software, of course driving a car is a much easier task. Right. So that kind of tells you a little bit how complicated it is, what we're trying to achieve.
- [08:17]
Kevin Ball
Yes, well, and I think one of the things that is often underappreciated outside of hardware is all those different layers that you have to go through, the different layers of abstraction. You talked about starting at the electronic layer, right? You're simulating physics down there to understand how is this going to work and function. And then you have to go multiple layers up before you even get to the logical layer, which is where many software engineers start their thinking. So as you started looking at applying machine learning to this, where did you start?
- [08:46]
Thomas Anderson
Yeah, that's a very good point. So where we started is essentially I would say we were looking at like, say tedious or like high toil human tasks, things that humans have to do. But it's sort of repetitive, it's sort of like not super creative. And it's something that I think if I automate this, everybody would be happy not having to deal with it. So a few examples is, and that maybe brings us in our first application that we developed, which is DSO AI stands for Design Space Optimization. Like as an example, like let's say when you optimize your chip and you implement it, one of the important things that you have to do is you have to tune your software to meet your performance requirements. Right? So every chip that gets out the door has different requirements. Some are tuned for highest performance, some are for lowest power, some are for a trade off in between. So our tools of course optimize for all these things. But typically users have to spend a lot of time in running our tools over and over again to tune for certain performance parameters and trade offs. So that's one of the earliest things that we thought about we could automate. And that's where this DSO AI technology came out of. Essentially, rather than a human running lots of experiments and trying all kinds of options, these could be technology options, tool options, flow choices, and so on. Instead of a human doing this and just running lots of experiments and seeing what gives you good result, you can have a machine do it. And a machine is much better than. And that's like a typical application of what I would call is reinforcement learning based AI optimization. So in this case I have a Complex workflow. I sort of intuitively know, based on past experience, what I should tune and how I should tune it, but I don't know precisely. Plus, every chip is a little different. But if I have a machine, I can run lots of experiments, I can take sample points, I can make decisions. Just like, for example, what was done for AlphaGo. You can think of it as a very large decision tree where I make choices. Do I go this way or this way? And what I want to do is I want to have a technology where I can prune this decision tree and get to a good result. I don't want to say the best result, but to a very good result relatively quickly. So that's what we're doing with DSO AI, and that has been, I would say, a runaway success for us because it has really helped this process of tuning performance, tuning of the implementation of the chips.
- [11:16]
Kevin Ball
So to make sure I understand then what was happening before is you had kind of a workflow that a human could go through and they could say, okay, I have all of these different knobs and levers I can turn. Let me try turning this one. Run the simulation. Wait for however long it takes, which is not fast. These are complex simulations. I don't know what it is exactly for you all, but I'm guessing it's hours or days.
- [11:37]
Thomas Anderson
Yes. It can be weeks, even can be weeks.
- [11:39]
Kevin Ball
Turn the knobs, run the test. Oh, that didn't get quite what I want. Or that got me partway. Turn another knob, run a test. And what you've done here is you've trained a model that kind of tries to reproduce some of those same intuitions of which knobs are going to be valuable. So it can pick what to turn and run those in a loop on its own.
- [11:57]
Thomas Anderson
Exactly. And on top of it, you can sort of learn from chip to chip, because a lot of the chip designs out there are sort of evolutionary. They may be redesigns or maybe 30%, maybe of the logic is new on the functionality on the chip, but there's a lot of similarities. So you can then essentially build also a behavioral database where you sort of know this type of chip. I kind of know how it will respond to my knobs. And so the second time I do it, when I do a next version of this chip, I sort of have a starting point already, which is similar to what I said. A human has some intuition. Right. But here, the beauty is I essentially have a centralized brain that has all this information. It's not distributed in people's heads and people may leave the company, they may move to a different project and then you lose this knowledge. If you have a system that captures becomes way more powerful, can we dive.
- [12:53]
Kevin Ball
One step deeper and talk about building this first version DSO AI like what types of models were you using under the covers? What does that tool chain end up looking like?
- [13:02]
Thomas Anderson
So I don't know if that goes too deep for all the listeners experience, but I'll try to keep it high level. So we started focusing on what's called synthesis and place and route flow. So that is a very complex part essentially. That's the part where you have a description in a language like, similar to say a C language, but it's called RTL or Verilog. So you have a description of what the circuit should do. You translate this into like Boolean gates and then you implement those Boolean gates essentially towards the transistor level on physical layout, right? On how they're interconnected, how they talk to each other and how the electrons move between the individual gates. That part of the process is one of the most complex one with very long run times and that's where you have a lot of knobs to tune. So I think, because I have a lot of background in this space, and I've been in this space for quite some time, I started thinking this, this could be a very good example of automation. So we tried various different algorithms. We settled essentially on some form of reinforcement learning ultimately. And like I said before, we got some inspiration from AlphaGo because that was a prime example of how essentially they were able to solve a very, very complex problem without having the computational expense of it. Right. I don't know if you remember the story Even from the 1990s when IBM built this deep Blue supercomputer to beat, and they did beat Kasparov at the time, the world champion of chess. But the system that they had would not have been scalable. Right. Like it was essentially sort of, not completely, but it was to a large extent brute force and computing all the possible choices. And if you think about it, the way a human operates, a human doesn't play chess by calculating all possible combinations ahead of time. A human has cognitive abilities and he sort of knows, ah, I know this scenario and I know what to do here. Anyway, so this was sort of our inspiration and we used this technology to optimize the workflow. And I still remember in the beginning it was primarily about productivity, meaning I can get to the same performance that a human designer was able to do by tuning knobs. But the story was we can do the same performance, but we can do it faster and obviously with less people, with automation. The interesting thing that we found is that we were actually pretty much always also able to get better performance. And our customers, they got excited, not just about the productivity, but also about the aspect of getting higher frequency on the chips or lower power. So that's sort of how we started this. And this really was an incubation within the company at a time where I would say 2017, roughly where ML and AI was already pretty hyped up, but nobody really had any idea how to make it work in our domain. So that's why I think this was a good first application for us to develop.
- [16:01]
Kevin Ball
Yeah, well, and just to summarize a little bit of what you're saying there, because I think it is a helpful mental model, the process sounds a lot like a recurring theme for me, like a compiler where you're starting from this high level language, Verilog, you're compiling it down, gradually lowering it into logical computations, then into gates, then to a physical layout. And this first implementation, it sounds like, was kind of looking at that physical layout step where you have a lot of different knobs and whistles and applying a reinforcement learning approach to kind of pattern match and say, oh, for this I want to tune this way, that I want to tune that way. And you were able to achieve not only the speed up, but the performance.
- [16:38]
Thomas Anderson
Yeah. At a high level, that is a good understanding. That's correct, yes.
- [16:43]
Narrator
This episode of Software Engineering Daily is brought to you by Capital One. How does Capital One stack? It starts with applied research and leveraging data to build AI models. Their engineering teams use the power of the cloud and platform standardization and automation to embed AI solutions throughout the business. Real Time Data at Scale enables these proprietary AI solutions to help Capital One improve the financial lives of its customers. That's technology at Capital One. Learn more about how Capital One's modern tech stack data ecosystem and application of AIML are central to the business by visiting capitalone.comtech.
- [17:24]
Kevin Ball
So let's maybe then look a little bit at what came next. And I immediately, if I'm thinking about that layers of compiler step, I'm wondering, oh, do you look at some of those other stages as you're lowering things down or where did you go?
- [17:37]
Thomas Anderson
Yeah, exactly. So that's sort of what we did as a next step. So the reason I would say why we developed DSO is because myself and other members of that initial team knew this space really well. But there's many other Areas where you can apply. We have verification tools where essentially you run tests and try to achieve certain coverage of your design. With those tests. You have test equipment where ultimately you need test patterns to test the accuracy of your chip, make sure it actually is functionally correct. That is done on test equipment. And there's software that needs to essentially come up with the fewest amount of test pattern necessary to reach like a certain coverage. Because the more test patterns you have, the more time you have to spend each and every chip to test them on the test equipment. So this is what's called that task pattern generation. Then there's many other areas. There's even on the analog design side, there is this whole transistor optimization. When I move either from one node, I move a design from one node to another and I need to come up essentially with a new circuit optimization or if I start a new design. So these are all very good applications also for this space optimization because they're essentially complex workflows. When I look at it as a user, to me it's sort of a black box. Of course, like I said, you have some intuition, so it's a little bit more than a black box. But I have to tune knobs to get to some certain results. All these types of applications are really ripe for like a system that essentially sits around and then optimizes the workflow. And that's what we did. So when you hear the word synopsis, AI, that's essentially the expansion of the DSO idea to other domains. So we have now verification space optimization for verification, we have test space optimization for the test pattern space, we have analog space optimization for analog space. Now one of the more recent things is 3D space optimization because when you design three dimensional chips, so traditionally chips are two dimensional. When you do three dimensional chips, you have a lot of design choices to make on how you design design. The interconnect, you have to watch out for thermal between the different dies that sit on top of each other. There's lots of again, design choices that you have to make where you want to have a system that explores that for you rather than manually.
- [20:00]
Kevin Ball
So a few different questions this sparks for me. One is to what extent are you able to kind of do this in isolated slices? Like are all of these different places where you're inserting an optimization automation completely decoupled from the other, in the sense that you can put this in here and not worry about its impacts upstream or downstream?
- [20:20]
Thomas Anderson
Yeah, that's a good question. I think the examples that I brought, like verification Digital implementation test and so on. We wrap around, essentially, slices in there. That is correct. You can theoretically wrap it around the bigger space. The issue, I think, with that, at least at this point in time, and I think this will change, especially once we go to Genai. But the issue right now, the reason why this is broken up into these steps is because there is still a human that owns that piece. There's a human that says, I'm the verification guy, and there's another human that says, I do the synthesis, like the logic synthesis, sort of the compile step. Then there's a person that does the physical implementation. Then there's a person that does the sign of testing. These are all different engineers and they hand pieces over to each other. So it's sort of like a little bit the victim of an established workflow or tool chain that was done simply because of the complexity of these steps that had to be broken up into, like, hey, there's multiple people doing these pieces, kind of like an assembly line where there's people sitting there. Ultimately, I think as we reduce the complexity of each of these steps further, I can totally envision that maybe you have an optimization that goes around bigger pieces rather than those individual slices.
- [21:38]
Kevin Ball
And as you were applying this to these different slices, were you able to more or less reuse the same approaches and toolchain, or were there places where you had to innovate a little bit further?
- [21:49]
Thomas Anderson
Very good question. Yeah, there's reuse, but there's also new technology required because not all the problem spaces are exactly the same. As an example, like in the analog design space where I'm operating at a transistor level, the response function of the system is quite different. The nice thing there is actually that I can take more sample points because, like, usually you optimize relatively small pieces of the circuit. And I can run lots of simulations really fast, so I can take a lot of sample points. That sort of requires slightly different algorithms, or at least you can benefit from the fact that I can take more sample points. So, yeah, it's not all the same. There is reuse, but it's not 100% identical what we're using across all these different applications. Applications.
- [22:34]
Kevin Ball
Another question related to this, one of the big areas of challenge, I think, and people are running into this as AI leaks its way into more and more things, is these move us from a very deterministic world of software to something that is much more statistical distributions. And so the question of how do you evaluate your models, how do you validate, how do you make sure that if you're changing the model that you're using or you're training it further, you're not breaking old use cases. What do you all do for that?
- [23:05]
Thomas Anderson
That's a very good question. The interesting thing though is that the AI technology that we have are still companions to the underlying, like the traditional EDA software. So therefore, at the end of the day, what I'm replacing is what a human operator does. And a human operator is not without error either. So there's always checks and balances. Because let's say you implement a design and let's say I relied on some ML model to tell me which knobs to choose. But in the end there's always like a verification step that is sort of independent, where I can make sure that the result that comes out indeed is accurate. Having said that, it is true that I think it becomes a little bit, maybe more difficult to debug if there's a problem that happens, where does it come from? Is it because my statistical model in the ML model wasn't correct? It's just a different way, I would say, to debug a problem. And it's not that in a non mlai way, it is all that easier because there, like I said, I have to rely on humans and humans make mistakes. Possibly more mistakes than a good trained model.
- [24:12]
Kevin Ball
Yeah, I love a thing that you said there, which is, I think many of the most fruitful applications of AI to date are those in which the generated output has a non AI validator in some form or other.
- [24:26]
Thomas Anderson
Yeah, exactly.
- [24:27]
Kevin Ball
That was your space. You were validating all these things. So it's a natural fit. So that makes a ton of sense.
- [24:32]
Thomas Anderson
It's actually one of my. When people ask me what is AI good at? Or so I always say, well, in my mind, the intention is to essentially automate or at least assist with the human efforts. And like to give you an example, let's say in our space, when I go from the logic gates to the physical design, one of the common steps there is, it's called physical placement. So let's say you have 1 million of logic gates that I need to place on my die. Actually, today it's way more. It's probably 10 million now for these type of things, I think it's pretty obvious that a human will never be better than some algorithm. Right. And therefore, to me, this is not really an application of AI, it's a different problem, it's an optimization problem. If I want to optimize this deplacement, which is similar to, like, say, graph optimization. So I want to, for example, optimize the total wire length between all these cells that exist on my chip. There's no doubt that a traditional optimization algorithm is better than a human. It's like saying, I'm trying to compete with a calculator. There's no way I can ever do it. A human isn't really meant for that. A human is meant for cognitive tasks, creative things. And that's what I'm trying to automate with AI, which is why, like I said, the AI to us is a companion, but it has not replaced. And I don't really see it replacing the traditional computational algorithm, so to speak.
- [26:00]
Kevin Ball
That makes sense. You mentioned a little bit that this was not Genai. This was some of these more traditional models. Are you using Genai in any domain?
- [26:09]
Thomas Anderson
Yeah, of course. I mean, who isn't? Everybody is working on Genai. The thing is, of course, Genai is, I think, for different applications. Like, Genai is very good at like, summarizing lots of content or like taking lots of documents and you can ask questions about certain things. It's good at generating content also. So it's a different application that is really complementary to the other AI optimization that I just talked about, but it's not replacing. It's not necessarily better, it's just different. But good examples. And I think every software company, or actually not just software, any company is doing that is like essentially building some type of chatbot that answers questions about our tools. We have endless documents. And because our software is highly complex, it is a lot more than user manuals. We have user manuals, we have lots of other documentation that give you guidelines on how you can use the tool to better design your chips. So building a system there that answers these questions initially just in a static fashion, meaning like chatgpt. I ask a question, gives me answer. I think that's an obvious thing. And that's what we have already developed. We call this the synopsis AI copilot. But obviously it goes much beyond that. Ultimately, what we're working on is technology where I can ask questions in the context of my actual design. So I don't just ask static question, how do I run this tool? What does this command do? How do I set this up? Instead, I want to be able to have a window where I say, what's happening in my design? There's a problem, how do I fix it? Right. These are the things we're working on. And of course, agentic workflows. I mean, that's the big topic, right? Automating some of these more complex tasks where a human is involved today with reasoning ability, analyzing a particular pattern and saying, ah, I, I know this, I've seen this before, I'm going to do this as an action or at least try multiple choices until it is resolved. These are all the kind of things Genai is really good at and where I think we see a lot of potential in further automation.
- [28:18]
Kevin Ball
So it sounds like this is in progress. So I don't know how far you can go on this, but I'd be really interested to understand a little bit first, starting in that answering questions in the context of where I am. You know, one of the big challenges with all of these things is how do you figure out what's the right context to load up into your Genai agent or chatbot or whatever you're doing to be able to give relevant information. So in the context of Synopsis AI or Synopsys tooling, what does that context stack look like? How are you thinking about, like, what do we pull together to put into this chat bot to allow it to understand your current situation?
- [28:56]
Thomas Anderson
When you think about it, it's not as difficult as it may appear at first because again, if I think about what does a human do? When a human, let's say, interacts with our software, there's ways of like just launching something with a bunch of commands. But there's a lot of interactive work, right, where he has to look at like images of the layout, for example, and see there's a problem here. And then he need to do this or he looks at say reports. A typical example is if you don't meet your performance gains, like your frequency is not met, let's say you have too many levels of logic in your gates or they're placed poorly. So people will look at the layout, they will look at timing reports that show you the sequence of the gates in like those limiting essentially timing paths. So this is all, if you think about it, a human essentially primarily interacts with our tools through like visualization, through, through looking at reports. And you can automate this, right? You can have the system read the same reports, look for patterns of what stands out, teach the system, if I see a particular pattern, what action would I take? Because you can. While it's of course much more complex than say going to the doctor and saying, well, my knee hurts and the doctor, he has sort of has a decision tree in his head, he will say, oh, okay, let's do an X ray, let's see what the X rays are supposed to. X ray comes back and then he either sees something or doesn't see something, maybe orders another test. Right. It's a relatively, comparatively, I think to our world, a simple decision tree where he goes down, but we can also create those decision trees. Humans would say, okay, I have this problem in my design. I'm going to look at my layout map and I see, aha. There's like something we call congestion, meaning there's too many routes in a particular area and there's ways to fix it. So I can teach those things into the system. The hard part, I would say, in this whole endeavor is getting all this information into my system. Because many of the more complex things, they're not necessarily documented, like user documents have lots of information certainly. But in the day to day job, what people see and how they solve problems, they know it, they have it in their head, but it's not written down. So one of the big tasks for us is to extract this knowledge from people's brains and get it into the system. That is one of the challenges I think that we're tackling.
- [31:20]
Kevin Ball
I'm working on this in a different domain, but this concept of we have so much tacit knowledge, it's not written down, it's not even well codified. And most people are not very good at expressing it. They can't tell you how they're doing it, it just, it functions. So how are you approaching that? How are you drawing out that tacit knowledge and crystallizing it into your tooling?
- [31:44]
Thomas Anderson
Okay, that's a very, very good question. So first one is, I think you have, I would say, trusted structured data. Like for example, when I built a chatbot about our user manuals, right? We have trusted documents that exactly describe the behavior, certain things. So there's a certain amount where I can ask people to just write down the common things. But the thing is, I mean, that's a good baseline. But the truth is I also need to be able to have a system that can essentially consume all kinds of maybe less trustworthy and unstructured data which exists out there. There's. We, for example, we have annual user conferences and people submit papers there and they describe how they solve particular problems. In PowerPoint, they're in word, they may be from five years ago, maybe they're not applicable anymore, But I have a lot of data that I would say isn't 100% trusted, but it does have value. So the hard part there is, I would say is I need a system where I can have essentially sources from 100% trusted data. But I also need to consume all these other documents that may have some nuggets of good information there. And there is some pretty good breakthroughs I think in the industry. Like for example, I don't know if you have heard familiar with like this deep research capability that was released a few months ago that essentially has the ability to sort of do research and it takes longer. It's not a quick lookup where I can give you an answer within seconds. It essentially almost acts like you have a PhD student and you tell him write me a thesis about this thing. And the guy goes to the library and searches all this information and he finds something here. And then ah, there's more information there and. And it kind of builds the information from that. And anyway, we're using this type of technology and we think that's quite powerful.
- [33:38]
Kevin Ball
Yes. Actually this question of what's trustable and what's not is also really interesting. We had another software engineering daily interview where we're looking at knowledge graphs and talking about that. So I'm actually curious, like one of the ways I think about that is you essentially want to construct a knowledge graph of facts, but you want those facts to both have backlinks to their corroborating evidence and some amount of maybe Bayesian or other like confidence level on them. How are you approaching it is really what I want to dive into.
- [34:07]
Thomas Anderson
I mean, yeah, it's a combination. There's like these graph racks which essentially is a combination of like a knowledge graph where you sort of start building like these relationships of what is trusted and what's not trusted. The deep research uses some of this, but it does more because it can actually search all kinds of information that's not in the rack. So the thing with the rag is everything in the rag is there and I can have a knowledge graph that structures it where I get a sense of what's more trustworthy than what's others. But what I also want is I want life documents. I want to be able to actually search on the fly. And there may always be new stuff. In fact every day I may have new stuff. So it's a combination really of this graph rack which builds this relationship of trusted and less trusted document our data. And then there's also the ability to research live things that is not in my RAC system.
- [35:02]
Kevin Ball
That is super cool. So you talked a little bit about agentic workflows and agents and things like that. What does that look like in the Synopsys world?
- [35:11]
Thomas Anderson
Actually recently we introduced Sort of a high level roadmap at our annual user conference. So our CEO Sassin Gadzi, he introduced that essentially what we call the agent engineer. And in fact I'm going to go back to a quote from Jensen Huang from Nvidia who I think like two or three months ago, he said they're obviously one of our biggest customers and using our software and designing their GPUs. So he had a quote saying that ultimately, and actually not that far out, there will be a human engineer sitting with like virtual engineers, like and he mentioned even synopsis, engineers are sitting there together designing the chips. And if you think about it, so the way we envision this is that you have initially a bunch of, let's call them little task level engineers. They do small tasks. Let's say you're an expert user and you are closing, you're working on a design and you have a particular problem, like say a congestion or a timing problem or something like that. I can have a little task level agent that can solve this problem for me. It's sort of like having a junior engineer that you give like a task to, but not the most complex task. It's just like a subtask, right? And we're building those task level engineers. Ultimately of course, and now I go back to the vision that our CEO has painted. Ultimately we envision of course complete automation and we model this sort of along the lines of the self driving car level. So we introduced level one to level five, where level one is essentially just an assistant that answers questions. So like chatbots, even context aware chatbots that give you suggestions, but they don't necessarily do anything yet. So you can ask them questions, say how do I fix this particular problem? You can even ask it about your exact design to give you a suggestion, but it's just a verbal response, it doesn't do anything. The next level, and that's level two are those task level agents where I have a lot of agents and they do very specific tasks. I have a problem with my timing closure. I say you go look at it and then it will run all kinds of analysis, figure out what the root cause could be and then try to resolve it. And there won't always be a single solution to it. It may have to try multiple things to solve it. Then the next level is like an orchestration level where you use all these task level agents to solve a bigger problem. And then ultimately at the very end, and when everything is fully automated, which of course will take time, you have everything automated, including the generation of the actual content. Because right now we were only talking about running our EDA software. The other big piece is actually generating the inputs, like writing essentially the software spec. They start with specs that are written in human language of how this chip should operate. Ultimately what I want is that I just describe what it should do and in the end the whole chip comes out. But that's of course a long way out.
- [38:17]
Kevin Ball
So one of the things this brings to mind is in the software world there's a lot of talk about the way that like AI assisted coding tools are ramping up productivity and even changing kind of the structure and size of teams. So maybe instead of having your classic like six software engineers, a PM and a designer, you just have two or one, and you have these AI agents that are assisting. How does that look in the hardware world today? Or where you see this going? Like, is this also changing the team structure of how people are building chips?
- [38:46]
Thomas Anderson
Definitely, it definitely will. In fact, there is a component that's very similar to the software industry and that's the part of where I'm actually writing the description of what my hardware should do. Again, that's written in software. So you can say that it's very similar to writing C or Java code. It's just a different language. Right? Let's say it's called Verilog, for example, is one of the languages. It's just a different language there you have coding assistants and they help you write modules, they help you understand the context in the bigger picture of your chip design. And they of course will reduce the size of those design teams. And then there's the guys that take that input and essentially run it through the tools. And there's more inputs, by the way. There's all kinds of other descriptions of like how your power tables should look like and your test structures. And there's lots of more specification that needs to be done. So all these things I think will be automated more and more, initially just with assistance, but ultimately more and more. And that would reduce the size of design teams. Having said that, the other thing to consider though is those chips are getting bigger and bigger and bigger and bigger. So therefore, I'm not necessarily saying people will lose their jobs. I think people will do bigger chips, you know, or more chips with the same amount of people. That's sort of how we see this.
- [40:08]
Kevin Ball
There's a kind of related question I have here. So one of the things I see in the software impacts is we will probably actually see many more software companies that are small because once you get the cost of creating software going down. To what extent do you see these changes kind of changing the business model in the EDA space? Is this actually making fundamental shifts to how this all works or is it more kind of the incremental improvements like you were talking about today, where it's like, oh, it helps this person in this specialization do their job faster?
- [40:39]
Thomas Anderson
That's a very good question. So you know, even in our industry there's a lot of smaller startups and they weren't necessarily driven by what you just mentioned, that it just becomes easier to design chips. But we actually have a lot of like startups in the AI chip space, right? Because while we're building the chips to run AI, those chips are then again running our software on top. That's sort of like the interesting cycle there. So there's a lot of demand for new chip designs and therefore we have quite a lot of startups. Having said that, I think the world is a little bit different than software design and I tell you why. A lot of the stuff we're building, we're obviously pre training it with data that we have. But the world is a little different in the sense that the amount of data that's, let's say publicly available, or even for us available, is limited. And every one of the big chip companies has their own secret sauce. They have their own designs and have their own ways of writing certain things. And they of course are very protective of their IP because of that. When we ship our software, we pre train them, but we always have to train them at the customer site and then this customer has a special version of our software that was trained on their designs. Why do I mention this? I mentioned this because actually for small companies it makes it harder because the training data isn't available to everybody, unlike in software. In software you can argue like the amount of like C code that's out there to train a system is very, very, very large. The amount of hardware design that's out there is much smaller and it's never the cutting edge stuff. Right. That makes it very hard for like smaller companies to be competitive because they will not have the training data to make those systems work as good as the big companies.
- [42:26]
Kevin Ball
Yeah, that makes a ton of sense because if you think about foundation models, right, they're trained on like the whole Internet essentially, not, not quite, but order of magnitude and millions and millions of lines of code. So as you say, the hardware world is much more sparse in terms of the data that's available and you have all of these different Layers that I assume have quite different data as well in terms of what they're outputting. So are there any particular ways that you think about like training models to survive in that data sparse world?
- [42:58]
Thomas Anderson
It's essentially the data sparse world has been us from the first day when we started this. And I always knew that the way to go around it is, to be honest with you, there's all kinds of approaches like synthetic data. But I personally don't see this as the answer to things. I think the answer is that you have customized model that you train at the customer site and customers are perfectly happy with that as long as those trained models don't leave their site. Like to give you an example, let's say we have a capability that generates this RTL code, right? So you have a natural language description that says write me a component that does XYZ like a counter or something more complex and then it will just create that for you. Now you have the basic capabilities, but then we ship it to our big customers and then we train it on all the RTL that they have. And you can train that a through like the rack system through fine tuning of the weights in the LLM. And then again they have their specialized version. We give them the pipelines to do that. They just enter their data. We help them essentially update the weights and update the vector DBS and so on, and then they have a specialized version. That's how we deal with the sparse data. But we fully recognize that the data that's out there so that we have part of the data and our customers have the other part of the data and we essentially work together to make it happen.
- [44:23]
Kevin Ball
That makes a lot of sense. So to understand, you're kind of almost building these like foundation approaches, but your expectation is people won't use that out of the box. You also provide to them the foundational model and the pipeline for fine tuning it. And here's how you ingest your data sources to become part of our rag source of truth for you.
- [44:41]
Thomas Anderson
Exactly. And you know, the interesting thing is actually I think our customers like that because it finally gives them a way to be differentiated. Like if I just have traditional tools, everybody gets the same tools, right? And then I remember even like 10 years ago, they would come to us and they would say, but how do we get something that's special just for us? You know what I'm saying? That actually is now finally enabled. In the past that wasn't the case. Everybody got the same release of the software. They can tune it a little bit they can write layers on top with scripts and so on, but it's much more limited in what you can do today.
- [45:17]
Kevin Ball
Yeah, that's really nice. It also for you provides a little bit of lock in because they put that investment into getting their stuff tuned into your system.
- [45:24]
Thomas Anderson
Exactly, exactly. That's very true. It's ecosystem play. That's very true.
- [45:29]
Kevin Ball
That's very nice. Question that comes to mind for that though is say you develop some updates on your side or you have a new model that works better at that foundation level. How do you then roll that out to customers who've already done all their layers of fine tuning and improvement on top?
- [45:46]
Thomas Anderson
Yeah, that's a very good question and I would say I don't think we have the final answer there yet, but we have already experienced in the development over the last two years that there's constantly new models. Now the nice thing is like at least rack type systems, you can usually plug in a different model and it still works fine for fine tuning. That's true. You may have to rerun things, but it's usually not that expensive to do. It's like an overnight compile. So it's practically speaking not big effort to make it happen.
- [46:18]
Kevin Ball
Makes sense. So I realized we're coming close to the end of our time together. Is there anything we haven't talked about yet that you think would be important to touch on?
- [46:26]
Thomas Anderson
I think we talked about pretty much most of the things. I mean, I touched upon the Agentix thing and maybe I just wanted to highlight again that in a world like ours, the choices like the agents are pretty complex. Cedille because there is just so many. It's not like an HR workflow, right, where there's like 10 choices I can make and if it's this and I click on this and if it's this, I click on that. Our world is so much more complicated. Nonetheless, I think I'm very excited about the agentic gen AI and what it can do to automate essentially these tasks that humans don't really want to do anyway. You know what I mean? Because they're annoying. I think our goal is and our hope is that the creativity of the human remains. I mean, this is almost a philosophical question. At which point or if at all, will AI supersede human creativity? That's a very philosophical question. But I think at this point in time I would say nobody should be afraid of AI. They should embrace it because it will make their life much easier because it takes away the tasks that you don't want to do anyway. They're tedious and they're annoying and you can focus on much more creative things.
- [47:34]
Kevin Ball
Yeah, I think it's an interesting balance we're all trying to figure out. Right. Is this going to take our jobs? Where is it? From what I've seen in the software world and tell me if this is the same for you In EDA and hardware, you still have to do most of the decision making. You're still having to say, this is what I want. And even this is the right architectural approach, sometimes the tool will suggest something to you. But if you turn your brain off, you're going to end up with garbage.
- [47:58]
Thomas Anderson
Yes, very true. Honestly, I think that will be the case for many more years, that the human still has to be in charge. That is just very true.
- [48:08]
Kevin Ball
Awesome. One last question. So we talked about agents and maybe that's it, but are there any other big opportunity areas or unsolved issues that you're looking at that you think is where there's some really interesting stuff to come?
- [48:21]
Thomas Anderson
There's lots of unsolved things still, but I don't know what the solution necessary will be. I think from the agentix side, the one part that everybody is still working on is improved reasoning capabilities. So you have all these GA tests, for example, that essentially test how good the reasoning is for particular problems. But these are always like very specialized things, like almost the kind of things that maybe you give a student in a SAT test or something, or like recognition of like what certain patterns, almost like what you see also in IQ tests. And they train them for these particular problems. But that still doesn't mean you can solve other problems so well. So my point is the reasoning abilities still need to improve for us to make this agentic world a reality. And I think there's still quite a bit of work to be done. Having said that, I think things are moving at such a fast pace that I'm sure if we would meet next year, maybe this is already solved.
- [49:20]
Kevin Ball
Well, and this is a place where you may be well positioned as well because I think the reasoning capabilities as you highlight are often they're being trained via reinforcement on particular types of reasoning activities. The reasoning that goes into hardware design is closed down. It's not out there in the public.
- [49:39]
Thomas Anderson
Exactly.
- [49:40]
Kevin Ball
It's not going to be there in the folks that they're hiring to take these things through their paces. But who has access to that? You do.
- [49:47]
Thomas Anderson
That is very true. So that kind of underlines the need for domain specific LLMs that are trained on that. And yes, the data isn't publicly available, so therefore it's a good problem to have.
- [50:00]
Kevin Ball
Let's call that a wrap.