← Software Engineering Daily
Software Engineering Daily
Turing Award Special: A Conversation with Jeffrey Ullman
00:37:45
Jeffrey Ullman is a renowned computer scientist and professor emeritus at Stanford University, celebrated for his groundbreaking contributions to database systems, compilers, and algorithms. He co-authored influential texts like Principles of Database ...
Loading summary
Transcript78 lines
- [00:00]
Narrator
Jeffrey Ullman is a renowned computer scientist and Professor Emeritus at Stanford University. Celebrated for his groundbreaking contributions to database systems, compilers and algorithms. He co authored influential texts like Principles of Database Systems and Principles, Techniques and Tools, both of which have profoundly shaped the field of computer science. Jeffrey received the 2020 Turing Award together with Alfred Ayo quote for Fundamental Algorithms and Theory Underlying Programming Language Implementation and for Synthesizing and those of others in their highly influential books which educated generations of computer scientists. In this episode, he joins Kevin Ball to talk about his life and career. Kevin Ball, or K. Ball, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow K. Ball on Twitter or LinkedIn or visit his website Kball LLC.
- [01:18]
Kevin Ball
Hello, software engineering Daily listeners. I'm Kay Ball and it is my privilege and honor today to welcome Turing Award winner Jeffrey Ullman to the show.
- [01:27]
Jeffrey Ullman
Hello everyone, and thank you for inviting me to your esteemed podcast.
- [01:33]
Kevin Ball
Yeah, I'm excited to talk with you. I know you have such a kind of rich and varied background. How do you introduce yourself these days? I know you mentioned a lot of the stuff you did was old hat. So what do you describe yourself as today?
- [01:48]
Jeffrey Ullman
I basically retired over the hill, supernumerary. You know, let's face it, folks, computer science is a young person's game and I ain't that anymore.
- [02:01]
Kevin Ball
So that is fair. Though I think, you know, some of the things that you did have been shockingly long lived in this field, which move so quickly. I think, you know, when I mentioned to someone, oh, I'm interviewing Jeffrey Ullman, they immediately like, oh, the Dragon book. The Dragon book. So I'd love to just spend a few minutes. I know that once again, older. But what do you think made it so successful? It feels like this is something that first edition was decades and decades ago. And I talked to new computer science grads and they're still talking about the Dragon book.
- [02:32]
Jeffrey Ullman
Yeah, well, I think the answer is in the name. It was the cool cover and apparently people were proud to be seen walking around campus with a book that had a picture of a dragon on it and therefore they enrolled in computer science. And that's probably the biggest contribution I've ever made in my life is getting some really bright kids who otherwise might have wasted their life in physics or something to go with cs.
- [03:05]
Kevin Ball
If we look at Compilers. I know when you started, the types of languages we were compiling for were perhaps simpler than some of the things that we've had today. There have been a lot of developments. Are there any changes that stand out to you over the years in terms of language design and compilers that you found particularly interesting?
- [03:23]
Jeffrey Ullman
Well, I think the biggest thing is that we're now compiling for parallel machines in some sense, whether it's an eight core processor or some massive supercomputer. There are issues which I guess the last version of the Dragon book began to address a little bit, primarily because Monica Lamb, this is sort of her field is parallel compiling. But that's something that when we wrote the first of the three editions, we didn't even really think was, you know, was worth thinking about. And it probably wasn't at that time. That's 77, I guess. Yeah, 1977.
- [04:07]
Kevin Ball
Yeah. Well, and I think one of the threads that I've seen as I started researching your career and your work that very much shows in that book is kind of thinking about and connecting from sort of fundamental theoretical abstractions and thinking about the different layers of abstraction and then how you apply them to practical problems and sort of move from this very abstract theoretical approach into higher and higher levels of kind of context or implementation detail. I'm kind of curious, are there ways in which the abstractions we need have changed with that move towards more parallel computing?
- [04:46]
Jeffrey Ullman
Well, let's see. Certainly if you were compiling for a serial machine, you didn't need any of the abstractions that deal with concurrency control. For example, the map reduces, I guess an interesting abstraction of parallel computation. Things that wouldn't have made any sense back in the 1970s certainly wouldn't have been useful.
- [05:09]
Kevin Ball
Yeah, MapReduce is a good sort of segue into. I saw some of your more recent work is really was tied into these large scale data mining and dealing with web scale and web scale. And you had a book in that space as well. So MapReduce is one example. What are the other key pieces that you think have gone into as we've scaled up and up and up into the current day?
- [05:34]
Jeffrey Ullman
Well, I mean it's pretty obvious that what has happened is getting all the data in the world together has given us this amazing power that we have. And I'm talking about these large language models. And sure we know that there are. Well, there's still some of the rough edges to be worked out, let's put it that way. But obviously it gives people power to do all sorts of marvelous things. This Same thing was true what, five to ten years ago when the massive neural nets came along. And again, what made them work was not only the availability of, you know, very cheap massive computing devices, but the availability of a massive amount of data. And this is of course something that people often dismiss is the importance of the data. It's not just the algorithm, it's the data as well. So, for example, I've heard people are now worrying that LLMs have reached their limit because we've already taken everything that's ever been written and there ain't no more. And that's an interesting. Well, we'll see, I guess in a couple of years whether that turns out to be true or whether, for example, there are ways to manufacture data that never really existed in the way, oh, let's say if you're trying to do machine learning to recognize tumors, you can take an image of a tumor and you can enlarge it 10% or rotate it 10% and, and create additional useful data without actually having any more data. So anyway, that's an interesting question.
- [07:21]
Kevin Ball
Yeah, I think one of the key questions there is like, what makes it actually useful? Right. Like in the image example. That's great because it allows you to sort of extrapolate things that look different to the computer but that we as humans can validate are the same or equally valid. What does that look like in some of these other spaces?
- [07:41]
Jeffrey Ullman
Yes, I have no idea. Yeah, as I say, it's just, it's an interesting question. I'm sure people are looking in this direction right now.
- [07:52]
Kevin Ball
Yeah. While we're talking about this, I think I saw something that you wrote about sort of the importance of dimensionality reduction. When you start to deal with these very large scale data situations. And it was pondering if there was a connection there to what people are doing with distilling models in terms of like trying to pull out what are the relevant dimensions.
- [08:11]
Jeffrey Ullman
I don't actually, I don't remember writing anything myself on that. I mean, obviously the, it may have been in the book that you does talk about dimensionality reduction. It's often a useful tool in understanding data because, you know, there are various techniques for focusing on what's really important in some very complex high dimensional data.
- [08:34]
Kevin Ball
Yeah. Thinking about once again, that sort of abstraction lens that I saw in this world of LLMs and you highlighted there's some rough edges, there's some challenges. What are the key abstractions for us as software engineers in working with these things? Or are there abstractions that we are missing and maybe need to figure out.
- [08:53]
Jeffrey Ullman
It's a good question. I don't think we have really said the last word about how you use them. One of the interesting things is Stanford, for example, never taught a course on how to Google stuff, how to create search queries, but we are now teaching a course in what's called prompt engineering. Probably many schools are doing it or will be. There seems to be the idea of a prompt, let's say, maybe needs to be abstracted or understood in some way. I have yet to see just a good set of principles being written down for how you use it. You know, I've been playing around with it a little bit myself. One of the things I do in my old age is I do a lot of editorial work. So there's this journal that I work for where they have 300 editors and I'm the only one in computer science. And so a lot of stuff gets dumped on me and I understand nothing about what I'm reading and there's nobody to hand it off to. So what I've been starting to use various LLMs, you know, ask it things like, you know, first defining what a good referee is and then saying, and here's an abstract of a paper, suggest some people who would be good referees. Sometimes it actually gets it just right and does introduce me to people I've never met but who turn out to be good referees. And other times it just, you know, sometimes it'll say, well, this is too hard for me, but here's how you pick a referee. You read the literature and you, you know, basically completely useless comment. And frankly, I don't know what makes, you know, is it the difference in the detail in the abstract or just how I phrase the definition of what I want? There's a lot of very mysterious stuff I think that is.
- [10:55]
Kevin Ball
Yeah, well, and I think you're right, there's an opportunity here. I saw a tool that was essentially trying to compile prompts, more or less like a very simple compiler would, where you, it had sort of different layers and you could see it sort of combining and lowering things down to different styles for different LLMs. But it didn't, I guess I could say it didn't have any optimization, right? It was entirely focused on how do I go from kind of a high level set of components to something that will work for anthropic versus chatgpt. But I think there is something there of the equivalent of a compiler that understands what are the right ways to prompt these different models and maps from intent to something useful.
- [11:37]
Jeffrey Ullman
Well, yes, good research topic. What can I tell you?
- [11:40]
Kevin Ball
It would be a good research topic. Speaking of research, then you mentioned recently you've been actively working on a different project. Do you want to tell us a little bit more about that?
- [11:52]
Jeffrey Ullman
Well, let's see. Again, trying to just sort of keep busy in my old age, I guess it was really about the turn of the millennium that I and a number of friends started a little company called Gradients. And it was designed to automate homeworks. And the architecture that we hit on was this. To the student, it looks like a series of short multiple choice questions where they're asked to solve a problem and then pick a correct answer from say4choices. But the difference between this and a routine multiple choice question was we felt that we wanted to use the homework to not only test, but to teach. And so if you got it wrong, we'd give you some sort of a hint and ask you to do this whole thing again. And the trouble is with multiple choice, well, if you guess A and it's wrong and guess B the next time and eventually you'll get the right answer without ever having to do anything. So the way we worked it out was we developed what we call root questions, which are questions that have more than one correct answer. Now how does that work? For example, if the question is solve the equation 2x plus 5 equals 10. Okay, solve for x. Now that has only one correct answer. I think it's two and a half or something. Right? So it looks like there's only one correct answer. But in fact, what we're asking the student to do is solve the problem. Okay, you're going to write down and if you've got it right, it's X equals two and a half. And now the choices are not what is the value of X, but tell us something true about X. You know, so, for example, you can have a correct answer X is not an integer or X is less than 3 or twice x is a prime. So there are any number of correct answers that are easy to recognize if you know what X is. And if you don't know what X is, you're just guessing. So, you know, we thought we would make a lot of money selling the service. We didn't actually. They basically people were willing to use it and still do use it. Our largest enrollment is from the University of Cairo, for example. So people still use it, but they wouldn't pay for it, basically. So I guess now, almost two years ago, I gave it a talk on this technology in Bangalore and someone in the AUDIENCE I was one of the original Infosys guys, got all excited about this and said we could use this to teach mathematics. And you can, if you look at, if you go to the gradient site, the materials that we have are really, well, computer science from Java programming, obviously, compilers, operating systems, data mining. But I said, we can do this in mathematics. And so he actually raised some money and got a team together and they're putting together where we've been working so far on ninth grade mathematics. And by the way, ninth grade mathematics in India is mostly stuff I learned in 10th and 11th grade, but that's another story, I guess. At any rate, we're hoping to field these materials that anybody in India or around the world, I guess, can use for free. And again, you know, the idea is it not only makes sure that students have learned the material, but where they're having trouble give wrong answers, there's always an associated hint or explanation of some sort that we hope eventually gets them to get all the questions right.
- [15:53]
Kevin Ball
Yeah, I feel like this is a broad topic area that people are very excited about right now, which is like, how do we allow people to essentially scale education and go at their own pace and not depend so much on individual teachers?
- [16:07]
Jeffrey Ullman
Okay, now that's an interesting point. Again, roughly the same time that we started gradience, these MOOCs were all the rage. And it was somehow assumed that these were going to replace universities.
- [16:23]
Kevin Ball
Yeah. And where did that go?
- [16:25]
Jeffrey Ullman
Yeah, where did they go? Okay, well, it's a good question and I think it's an important one to answer. Part of the problem is this. When, say, General Motors decides to replace assembly line workers by robots, nobody asks the assembly line workers whether they think the robots can do a good job or not. Okay. When you offer academia alternative ways of, you know, essentially introduce technology into the education process, teachers have traditionally had the ability to say, no, this is not acceptable. I don't want to use this. Keep it out of the classroom, basically. And this is. I can't think of too many other fields where that is true. But it turns out, I think it was a little bit worse than that. Not only was there obvious resistance on the part of people whose jobs are threatened, but it turns out that the MOOC doesn't really give you enough for 95% of the students that people need help. And again, you can automate a little bit of that. I think gradients does a little bit, but you're always going to need somebody to go to when you're stuck. So it's, you know, The MOOC cannot eliminate the need for teaching professionals. It can perhaps take some of the burden off of them and thereby make them more productive, which by the way, means we need fewer of them. Okay, that's life. It's true in every field. You know, I mean, I think we could do more to. If instructors were willing to, let's say, play a MOOC and then much the way they used to prescribe a textbook. Right? I mean, the textbook did some of the job of teaching. The MOOC can do a little more of the job of teaching, but you still need somebody to conduct the class and in particular to handle all of the special cases that keep coming up. And then of course, there's also the problem of validation. You know, you can take a MOOC and have your smarter older sister actually do the work for you and nobody really knows. So just because you've registered for something and then seemingly done the MOOC requires it doesn't mean it was you. And you know, okay, there's technology for you go to a special center and you get fingerprinted and everything. But that's not really going to, that's not a long term solution. You need people, instructors to not only make sure that everybody kind of gets it and the people who are having trouble in a million different ways, they are helped through whatever problems they're having, but also just to validate that they have in fact learned the material. I think is that's not going to go away. And I know there are technological solutions that sort of do it, but I can sort of understand why schools don't want to automate the validation of their own students.
- [19:50]
Narrator
Developers, we've all been there. It's 3am and your phone blares, jolting you awake. Another alert. You scramble to troubleshoot, but the complexity of your microservices environment makes it nearly impossible to pinpoint the problem quickly. That's why Chronosphere is on a mission to help you take back control with Differential Diagnosis, a new distributed tracing feature that takes the guesswork out of troubleshooting with just one click. DDX automatically analyzes all spans and dimensions related to a service, pinpointing the most likely cause of the issue. Don't let troubleshooting drag you into the early hours of the morning. Just DDX it and resolve issues faster. Cycronosphere was named a leader in the 2024 Gartner Magic Quadrant for Observability Platforms at Chronosphere IO Sed.
- [20:39]
Kevin Ball
So I'd love to take this conversation about learning in a slightly different direction. So you talked a little bit about Stanford now offering a course in prompt definition or prompt engineering.
- [20:51]
Jeffrey Ullman
Engineering is apparently the term.
- [20:52]
Kevin Ball
It's the term, yeah. We can argue about whether it's engineering or what it is, but. So you talked about that. We've talked a little bit about applying technology to help scale learning. Both what you're doing at gradients, a little bit what these massive online courses we're doing. I feel like we are in a time period where there's a lot of resistance to technology change. There's a lot of resistance to LLMs in different parts of the world. So I'm kind of curious how you think about how do we bring people along with the changes that are happening.
- [21:28]
Jeffrey Ullman
Boy, it ain't easy. I think you just have to get new people.
- [21:33]
Kevin Ball
I mean, that is one solution, right? You wait for one generation to die off.
- [21:37]
Jeffrey Ullman
But look, having aged, I see it in myself that it just becomes harder and harder to adapt to new technology. And by the way, the field of HCI has been around for forever really, you know, human computer interaction. And I haven't seen anybody addressing the problem of, you know, sort of dealing with old folks and the fact that I can't explain it. But you just.
- [22:10]
Kevin Ball
I've seen this as well in my parents aging. You know, they, my dad is in his 80s now, my mom has passed away. But the rate of change is just hard. And I feel like in the tech industry we often change just to change and neglect the cost of that.
- [22:27]
Jeffrey Ullman
But it's not just change for the sake of change. I mean, the fact is, for example, I had to learn to use and depend upon a cell phone.
- [22:38]
Kevin Ball
Yeah.
- [22:39]
Jeffrey Ullman
And for example, our 25 year old grandson was out visiting us a couple days and he likes to borrow a car and just drive around. So he did that. And I have a NAV system and I know how to use the nav system. He says, well, I just want to use Google Maps. Okay. And he says, well, you needed the right adapter and you can just plug your cell phone into your car. Into my car. I've never known that, but apparently you can. These are the kinds of things, you know, I guess young people, they all know that, that that's how you navigate these days. I was visiting my son, I have a, a five year old granddaughter and I brought along a tablet and they're not allowed to play with these normally, but when grandma and grandpa show up, they.
- [23:35]
Kevin Ball
Yes, you get grandparent privileges.
- [23:37]
Jeffrey Ullman
Yeah, right. And she saw, she wants to download an app to do drawing and then, but the app has ads and that gets you to. Basically there's a strange ecosystem because it looks to me like all of these apps, all their ads are for other apps. And so that's basically an economy based on everyone taking in each other's laundry. So I'm just not sure exactly how that works, but this is what happens. So she's wandering from app to app and I said, look, you got to get around the ads and fudging around. I sort of found it. And 15 minutes later I notice she's found her own way and it's better than my way. And that's what five year olds do.
- [24:25]
Kevin Ball
Yeah. It's funny, it reminds me of when my son was that age. We would joke that somebody would hand him a device of their own for whatever to entertain him, and within minutes they'd be saying, wait, how did you do that? What is that?
- [24:38]
Jeffrey Ullman
Yeah, exactly. And I think a big question is how do you make it possible to teach old folks who have, you know, I guess we have our own skills and our own ways of doing things that don't always work. And, you know, for example, how do you recognize what it is your customer, say, me, is capable of doing or, you know, feels normal doing, I guess, or has the skill to do and use that to get them to navigate the device?
- [25:18]
Kevin Ball
I think this is really important and it's a big blind spot because as you said at the beginning of our conversation, tech is generally a young person's industry. And so the people making these decisions and thinking these things may not themselves be interacting with older folks very often at all. Even middle aged folks. There's a big difference. Like, I'm in my 40s. I naturally interact with most technology via text.
- [25:42]
Jeffrey Ullman
Yeah.
- [25:43]
Kevin Ball
My kids go to voice and video for everything. It blows my mind. Right. I want to look up how to do something. I'm writing in text. I'm looking for an article on it. They're looking for YouTube. And so we as an industry, especially looking at our society aging, we do need to figure out how do we incorporate the preferences, needs and differences of older generations.
- [26:09]
Jeffrey Ullman
I'd love to see that.
- [26:11]
Kevin Ball
Have you seen anyone doing it?
- [26:12]
Jeffrey Ullman
Well, doing research in the area?
- [26:15]
Kevin Ball
Either research or even like some company managing to do a good job with their products on it.
- [26:20]
Jeffrey Ullman
Well, I have not seen really good examples of the systems that adapt to, let's say, a variant mode of thinking about it, but I get the feeling it should be feasible.
- [26:34]
Kevin Ball
It should be feasible. Well, and I feel like I do wonder if this is a place where Sort of modern machine learning LLM technology can help because it allows for what I've been describing to some people as more kind of intent based user interaction paradigms where it's able to do some amount of interpretation. So if we can use that in a way so that we can try to infer intent. Even if your natural mode of trying to interact is different than my natural mode is different than your grand child's natural mode, like be able to figure out what you're trying to do and help you with it.
- [27:09]
Jeffrey Ullman
Yes. I think the new LLM based technology, you know, as in a simple case, it should be able to summarize a YouTube video.
- [27:18]
Kevin Ball
Yeah.
- [27:19]
Jeffrey Ullman
And you know, I think an interesting case that doesn't seem to yield to LLM technology would be, I mean I have a map on my cell phone. I want to connect it to my car's screen. I don't know. Okay. It's in the manual. Right. There's a 500 page manual in my glove compartment somewhere it must tell you how to do that. But I don't even know that it's there that it's an option. Another example, my son had to show me that my trunk has a button that lets me open the trunk from the outside without pressing the button that's in the car itself. I think all cars have this these days. Nobody told me that it was there.
- [28:13]
Kevin Ball
Yeah, there's sort of a dissemination of information problem. Right. If you knew to ask the question, you could have found the answer. But how do we surface the questions?
- [28:23]
Jeffrey Ullman
That's exactly right. Now the car should say, I notice you've been pressing the trunk release button quite a bit. Did you know that you can. Yeah, and that's not unreasonable. And I'm not sure it's an LLM.
- [28:39]
Kevin Ball
Problem, but there is something there. And it actually reminds me of kind of what you're doing with gradients. Right. Of like there are many correct answers here. If you have sort of the underlying model. Can we build our technology to notice that there's a class of them that you are just systematically ignoring or unaware of and surface them to you anyway.
- [29:02]
Jeffrey Ullman
That'S what HCI people ought to be doing, I think.
- [29:05]
Kevin Ball
Yeah.
- [29:08]
Narrator
This episode of Software Engineering Daily is brought to you by Jellyfish, the leading software engineering intelligence platform. AI Codegen tools can be force multipliers for R and D organizations. But are you making the most of them? Join your peers on April 17th at Glow Live. It's a dynamic 90 minute virtual event that explores the transformative nature and potential impact of AI CodeGen solutions. At GLOW Live, you'll hear expert insights on navigating a constantly shifting landscape, adopting cogen tools successfully and measuring their impact on your team, your work and your company's long term success. Register today at Jellyfish Co GLOW and get glowing.
- [29:48]
Kevin Ball
One of the things I love about talking with folks who are older and who have seen so much about this is you often have a much wider perspective than someone who's only seen 10 or 20 years in the industry. What else do you see as gaps in our current tech ecosystem and environment? Where should people be doing work that they're not?
- [30:08]
Jeffrey Ullman
That's always the hard question to ask. You look at the great changes. Nobody realizes that we need it until somebody provides it. Give you an example. I think it was. The date is very important. I think it was like March of 1992. I was invited to D.C. for a discussion about what was then called the information superhighway. And it was some academics like me and heads of telecom companies and other industry types. And we wrote a report came out of this. And the interesting thing is nobody mentioned the World Wide Web. Okay. I mean, there was some discussion of Gopher, I think, and it was big. A big question was, would the, would the information superhighway run along cables or telephone lines? Okay. You know, and here in Switzerland, the World Wide Web existed and none of these 300 experts, you know, I take blame for it as well, was aware that this was even an option. I mean, I remember several months later asking one of my colleagues, why do you always keep referring to things as colon, slash, slash? You know, it just. Nobody was even aware of the need for something like the World Wide Web. You know, I think you can probably say the same thing about cell phones, personal computers. You know, you know, there's this famous quote from, I guess the head of DEC at the time, why would anybody want a computer on their desk? So the answer is, I cannot really tell you what the next or even guess what the next great thing is going to be.
- [32:15]
Kevin Ball
Certainly not. But you have already pointed to something that's missing right now, which is HCI research focused on older generations.
- [32:23]
Jeffrey Ullman
I think it's a good research topic. I don't think it's going to change the world. You know, I think, I mean, if I had to guess, I would certainly have to think about quantum computing. I've always been sort of a skeptic.
- [32:38]
Kevin Ball
Yeah.
- [32:38]
Jeffrey Ullman
About quantum computing. And apparently there are some, some applications that is, you can, you can simulate some things better using quantum phenomena. And, and by the way I should say, it does look like quantum communication is real. You know, this spooky action at a distance story. Yeah.
- [32:59]
Kevin Ball
Instantaneous communication. Mind boggling.
- [33:03]
Jeffrey Ullman
Yeah. That people are actually possibly able to do things of that nature. And that may become real. But what people have been expecting is that they're going to be these quantum computers with the right kind of qubits that will enable you to implement Shor's algorithm and break RSA codes and elliptical codes and things like that. I mean, that would certainly be a transition in our way of thinking if that were the case. I'm sort of dubious. I mean, you know, things like the D wave machine apparently use a kind of qubit that cannot implement Shor's algorithm. And the scale up that seems to be going on, which by the way, may not be following Moore's law. We don't know are we going to double the number of qubits available every two years and in particular, qubits that are adequate to really run Shor's algorithm. I just don't know whether that's going to happen. It might, you might be able to predict some big change coming along the pike. That's something that's a candidate, let's put it that way. Whether that's going to happen or not, I don't know. We talk about artificial general intelligence, maybe.
- [34:30]
Kevin Ball
Yeah. So you mentioned a few sort of technology shifts you saw that you did not expect in the past. And if we were to look back at software engineering in particular as a, as a field rather than tech in general, since most of our audience, I think, are software engineers, what have been to you some of the biggest shifts or breakthroughs in the way that we think about software over the years?
- [34:55]
Jeffrey Ullman
Going way back, the first programs I wrote were in kind of a machine language. Then I graduated to FAP, that's the IBM 790 assembly language. And then I learned Fortran. And that was a big power booster. I think people are all so excited about LLMs are going to write your code for you. Well, I think it will probably make life a little bit easier, make you a little bit more productive. There seems to be some good evidence that that's happening. Compared with moving from FAP to fortran, I don't think it even compares. You know, as I said, the fact that it makes sense to think parallel has certainly been a big, big change. And again, I think it's more subtle and gradual. But again, the change from thinking of software as algorithms to thinking about it as algorithms plus data. Yeah, again, slowly, that has become. It just changes the way you think about what you can do with a computer.
- [36:09]
Kevin Ball
Absolutely. Well, and one could argue that a lot of the advances in machine learning are just more and more, you're programming with data, and algorithms are kind of, if not a forgotten element, at least.
- [36:21]
Jeffrey Ullman
Taking more of a backseat because you need the data just sits there until you apply an algorithm to it.
- [36:28]
Kevin Ball
Fair enough. Yes.
- [36:30]
Jeffrey Ullman
And, you know, for example, we talk about prompt engineering. Well, I mean, ultimately, I think that subject is going to become algorithms for turning desires into sequences of text, which at some point may not even be text. It could be videos or whatnot.
- [36:55]
Kevin Ball
Yeah. You're guiding it with a human language algorithm. Essentially.
- [36:59]
Jeffrey Ullman
Yes. Okay. In sense, it would be an algorithm. Well, who knows? There. There may in fact be tools. Your computer will help you to formulate a prompt. There could be computer implemented algorithms as. As opposed to just, you know, recipes that people follow.
- [37:17]
Kevin Ball
Absolutely. Well, and I think a lot of the development in that space has been applications building prompt generation around these tools that. So they can do things for you. Awesome. Well, thank you so much for your time. Deeply appreciate it.
- [37:30]
Jeffrey Ullman
Thank you for inviting me.