← Generative Now | AI Builders on Creating the Future
Generative Now | AI Builders on Creating the Future
Dr. Olga Russakovsky: Shaping the Next Generation of AI Leaders
00:44:32
Loading summary
Transcript56 lines
- [00:00]
A
Foreign.
- [00:06]
B
And welcome to Generative Now. I am Michael McNano. I am a partner at Lightspeed. And this week on the podcast, we're talking to Dr. Olga Russakovsky, a computer science professor at Princeton University. She is a leading researcher in the field of computer vision, having worked on many important breakthroughs in the field, including imagenet. She's also the co founder and board chair of the nonprofit AI For All. This was an honest and surprising conversation about who is shaping the future of AI, what today's computer science students are learning, and why the next innovation in AI may not come from where you expected. So check out my conversation with Dr. Olga Rosokovsky. Hey, Olga. So good to see you.
- [00:47]
A
Hey, Mike. Thanks so much for having me.
- [00:49]
B
Yes, thank you for doing this. I've been looking forward to this for a while. I have a bunch of questions for you. I. I want to. I want to.
- [00:56]
A
I'm ready. I'm here for this.
- [00:59]
B
I want to learn from you in this hour that we have, like so many people have probably in your career. I guess. Before we dive in, though, I think we should tell people a little bit about your career. And so. Yeah, do you mind just sort of telling people about your work and your research and everything you've been working on these past couple of decades in AI?
- [01:18]
A
Yeah, absolutely. So I'm an associate professor in computer science at Princeton, also associate director of the Princeton AI Lab. I. Let's see, I started my career at, I would describe starting my career at math camp in high school, was a math major in college at Stanford, and then went to do a PhD in theoretical machine learning. Sort of got very interested in machine learning and in AI and wanted to sort of do the more theoretical work, but over time, kind of drifted more and more into applied AI research or drifted into computer vision, which I think is just fascinating. I know there's a lot of hype and excitement about natural language processing these days and about text processing, but I still think computer vision is by far the most interesting one. Sorry, no offense to anybody, but that is very much my passion. And so I've been doing a lot of work both on building computer vision systems, but also kind of studying them, analyzing them, thinking about explainability of these systems. Recently, I've been thinking a lot about AI fairness and bias, particularly in computer vision systems and in AI systems more generally. So that's kind of. Yeah, that's kind of my trajectory.
- [02:31]
B
Do you mind just explaining computer vision a bit? I mean, I have, I have an understanding of it, but just for the audience Sake, like dive into computer vision. When you say that, what does that, what does that mean? Should we be thinking, you know, self driving cars or more just sort of identification of objects within, within images. Like tell us more about computer vision at a high level.
- [02:49]
A
Yeah, I think it's anything that's related to understanding pixels, understanding images or videos. I think you can think of you, yeah, autonomous cars for sure. You can think about when you upload a photo to any social media platform and sort of says, okay, here are the faces, here is maybe who these people might be, it automatically tags your friends. You know, you upload, you create a photo album and it sort of sorts your photos by not just by date, but by sort of who is in the photos, what is the scene like. But also, I mean you can think about a lot of applications, like medical applications. You can think about sort of computer vision that's used to diagnose skin cancer. You take a photo of a lesion on your skin and it sort of is able to tell you, okay, should you see the doctor, is this worrisome? So you can also think about applications to things like agriculture. So both in terms of building better systems that are able, better robotic systems that are able to sort of operate on the ground and do some of the agriculture tasks. But you can also think about like aerial photography and try to understand conservation efforts, trying to understand sort of how the Earth landscape is changing over time. And that's also computer vision. You're sort of analyzing these photos that are taken from space. So there's all kinds. You can also think about space exploration. You send a robot to Mars, right. It's trying to kind of navigate and move around in its environment. You're not going to have somebody steering it with a joystick. Right. That's not going to operate just because of the latency. Right. And so it needs to understand, to be able to have a camera that looks around, understands what it's seeing, kind of help guide it to drive around safely. So yeah, all kinds of things, tons.
- [04:38]
B
Of applications of computer vision right now probably more than ever before in the field. And the other thing I think of when you start talking about computer vision is something that I don't think is the same, but I wonder if there's intersection, which is kind of the other side of images right now with an AI, which is more around the generative AI of sort of diffusion based image models, where does that sort of intersect with computer vision? Or are they sort of like two completely unrelated fields?
- [05:07]
A
Yeah, so it's very interesting because I think initially we would think of that as a sort of computer graphics, as kind of generating, you know, kind of generating visual data. But it's very much become part of computer vision. So that was actually a very interesting transition. So I think when generative models, sort of gans, general adversarial networks, diffusion models started coming out. So that's now very much part of computer vision. And that's been sort of interesting that it's kind of become both the images to some kind of understanding of these images, but also going back and so, yeah, so we do a lot of work on diffusion models in my lab as well. And that's now we've kind of embraced that as part of the computer vision research.
- [05:48]
B
It's probably, I'm guessing, driven a lot of interest, like new interest in the field. And I wonder what that's meant for. For your work and your research and maybe the computer science department at Princeton.
- [06:01]
A
Yeah, so there's definitely, I mean, for the computer science department at Princessian, we're the largest major on campus and I think that sort of says it all. A lot of undergraduate students who are very interested and very interested in taking courses, very interested in doing research in this space, then lots of applications from prospective PhD students, graduate students. And I think we're trying to hire more and more faculty in this space to keep up with the interest. I think the new Princeton AI Lab is a good example of sort of trying to connect all of the folks doing AI research on campus and sort of grow that I think it's. When I think about AI research, I actually think of it as being broader than just computer science. I think there's a lot of. I mean, like you say, it's sort of the advancement of this technology has driven a lot of interest, but it also raises a lot of more social questions. It raises a lot of questions about the impact of this. It raises a lot of questions about how people are actually interacting with these systems. Raises questions about impact on different fields. And so I think when we think about computer science research or AI research, in the past, AI research has been much more of kind of engineering or mathematical endeavor. And I think now it's becoming much more interdisciplinary. So you need to really connect to folks who have studied things like societal shifts in other contexts and now can help us kind of grapple with how should we think about some of these changes in AI? How should we be thinking about analyzing and understanding these models which we no longer can analyze or understand using kind of standard methods? And so there's just A lot of opportunity to grow this field and grow the way we are thinking about it as well.
- [07:46]
B
Yeah, that makes a lot of sense. I mean, I feel like more and more you're seeing people within AI or tech more broadly think about like you said, the societal impacts or get philosophical and compare and compare back to previous times or technological innovations in history. So yeah, I've actually been thinking quite about that. Is the science, the computer science aspect of this over time does that become kind of less relevant and sort of the field of AI becomes as you say, a little more generalizable and sort of applicable to almost more different humanities related topics? It's super interesting. Maybe just to take that a bit further before I hand it to you, but like I was a computer science major and you know, we didn't, we didn't talk at all about philosophy or societal impact. We just learned how to program in a bunch of different languages.
- [08:40]
A
Right.
- [08:40]
B
And like learned about like the, you know, the, the hardware and the, with a lot of these generative models it seems like we could get into a point where that stuff, the type of stuff that, that I learned with my computer science degree actually becomes less and less relevant over time and the stuff I think that you are talking about becomes more and more relevant. So yeah, I mean, how do you think about the future impact on computer science as a result of AI?
- [09:05]
A
Yeah, so it's a very good question. So first of all, I think to your point about learning about the different coding language, different programming languages and so forth, I mean, I think that is becoming somewhat less relevant by virtue of the large language model. I mean, we've seen a shift in how people code. Right. We've seen a shift in sort of some of the models actually kind of assisting you in coding. And I have, you know, it raises a lot of interesting questions about how do you design assignments in computer science courses and how do you think about, you know, the fact that historically a lot of the assignments are around, you know, write this code to solve this particular problem, but now, you know, you can ask ChatGPT and it'll write that code to solve that exact problem because that's been in computer science assignments all over the world. And now the model has learned from all of that data and it can easily solve all of these kind of classic problems that we teach students to solve. So I think it raises questions about what is the role of programming and programmers and I think that's sort of one can of worms. I think to your point about that computer science will become less relevant I actually don't think so because I think building these systems and sort of the technical work around building, deploying sort of developing levers of control and things like that, I think that's going to stay core technical, but I think a lot of it is going to be driven not just by kind of intellectual curiosity of can I do this? But driven by sort of concrete needs and applications and concrete questions that are being asked and those applications are going to be very broad, but also the kinds of questions that we're going to ask about what does it mean to understand the system or trust the systems, or what kind of safeguards do we need around these systems, or how are these systems actually influencing society and what do we need to sort of steer it more to influence it for the better rather than for the worse? I think a lot of those questions are going to be asked sort of by scholars who are much more interdisciplinary. I don't think the computer science role is necessarily shrinking. I think computer science role is kind of staying the same. But maybe the field is kind of growing to bring in more and more ideas and questions to kind of help steer the computer scientists work.
- [11:38]
B
Yeah, how much is that actually happening right now? Is curriculum changing even now to head more in this direction or is this something you see further down the line?
- [11:49]
A
Yes, I think this is happening right now. I think the curriculum is changing in many ways and I think the connections to sort of interdisciplinary work is very much happening. So later today, in a few hours. So my co instructor and I are teaching the last lecture of the undergrad computer vision course at Princeton for the semester. And what we're doing is we're hosting a fireside chat with Dr. Molly Crockett, who is a psychologist, so from the department of Psychology. And they're going to come in and talk about one of their latest papers on sort of AI as used for scientific discovery and sort of what are some of the pitfalls of using AI in sort of scientific research? Or how it sort of creates an illusion of understanding, but really like what does this mean for scientific research? So that's sort of the conversation we're going to have in last computer vision class for the semester because we think it's sort of important to ask some of these more fundamental questions. In addition to, you know, a semester of teaching about convolutional neural networks and you know, deep learning and different methods and different techniques for building the, building the systems.
- [13:06]
B
Can, can you give us a little preview of what the pitfalls are of using AI to make scientific discoveries? That sounds super fascinating.
- [13:15]
A
I'm not going to give it justice, but there's a lot of sort opportunities to speed up the research process using AI. My interpretation of this is that it can drive research towards sort of what is currently the most promising direction. I think the scientists will start using kind of similar tools to summarize the data in similar ways and will drive the field towards. You're trying to get a paper published, you're going to select the current thing that is best supported by current evidence or is the easiest to put together the data for and you're going to analyze the data in particular ways. And ultimately it's kind of going to limit creativity. It's going to limit creativity and some of the joys and potential of scientific discoveries that you are thinking about these things in different ways, you're going down the wrong path, you're failing and then you're succeeding. And some of that breadth of approaches and thinking about things, that's what's leading to scientific discovery. And relying too much on AI models to steer that is going to lead to I think short term success, but not long term success.
- [14:40]
B
Yeah, that makes sense. It's almost, you know, it's gonna, it's almost like gonna converge to the path of least resistance almost in a way. Whereas the more the, the research is happening directly via humans. Like you said, to use the word, like creativity, there's randomness, random ideas that, that get explored. That makes a lot of sense. Maybe good tie into AI for all. This is a big part of, of your work and your research. Well, I'll let you explain it. Tell us about AI for all.
- [15:10]
A
Yeah, so AI for all is what I do in my copious free time. So I'm a co founder and now chair of the board of this nonprofit. We are working to increase diversity and inclusion in AI. So I think I personally see the big. I know there's a lot of conversations about what is the biggest threat to AI, what is the sort of existential threat. There's a lot of this talk and to me the biggest, the biggest threat of the existential threat is the lack of diversity of thought in this field. It's harder to measure diversity of thought. It's easier to measure diversity of demographics as sort of a proxy for diversity of thought. And you can look at within AI there's different statistics, but it's around 15% women. It is very few black and brown folks. It's very few folks who are black, very few folks who identify as Latina or Latina has very few Indigenous folks. And what that's doing is it's driving us towards echo chambers and decreasing the diversity of thought in the field, decreasing the creativity with which we're able to approach some of these problems. And ultimately, I think, going to drive this field into the ground. I think we're not going to reach the full potential of AI if we keep going the way we're going to. And just to sort of. I know some of the questions about demographic diversity get into very complicated sort of legal and ethical questions these days, and I totally get that. But one way that I like to think about it, which I think is a lot less controversial, is that the reality is folks who are working in AI these days come from particular schools of thought, particular types of training, particular sort of topics, schools that have produced these students. A lot of them read similar books as kids, A lot of them play with similar toys as kids. A lot of them run in sort of similar social circles. So they talk to folks who are like them. And then all of this kind of translates into how we think about building the technology. A lot of this translates into the kind of applications they care about. This translates into their value systems. This translates into kind of how they approach problem solving. And at the end of the day, this limits the creativity of the field as a whole.
- [17:44]
B
Yeah. You know, oftentimes I feel like when this topic is discussed, like we, you know, what, what, what I gather and what I learn is that. But, but I would love for you to tell me if this is right or not. Is that a lot of the, the bias in these models comes from, comes from the data that it's trained on. Sounds like what you're saying is it also comes from like the way that these systems, like, are actually designed. Can you add a little more specificity on how they might be designed in such a way that it leads to a bias in one direction or the other?
- [18:17]
A
Absolutely. So I think the bias comes at every stage of the pipeline. So you mentioned, I mean, data. It's actually interesting because that was a controversial point a few years ago, that people would sort of disagree that it comes from data. But. But that's actually become sort of well accepted that. Yes. So a lot of the bias. There's bias in the data. There's bias in both, sort of the way that the data is collected. So, for example, I mean, I know you want to talk more broadly, but I'll just give one example on the data bias. So if you look at the common computer vision data sets, you can run analysis on geographic distribution of that data. So you can, using a lot of this data comes with GPS tags, right? You just grab the GPS tags and you can plot sort of which countries does this come from from? This comes primarily from the US Countries in Europe, and that's more or less it. So there's various papers, including some of the work from our lab, but lots of other folks as well, where you have this map of the world and you have the US Highlighted bright colors and parts of Europe highlighted in bright colors. And then sort of the rest of the world, particular sort of South America and Africa are just completely missing from this data. And so then you get things like object recognition systems that are purported to sort of, oh, it can recognize all objects in the world. And then in practice, if you feed it a picture of a house in Africa, or you feed it a picture of a plate from Africa, or you feed it a picture of. One of the classic examples is like bar soap. And it will refuse to recognize bar soap as soap, but it will recognize you as brands of liquid soap as soap. And so that's sort of the data answer. But I think, thinking more broadly, right, what are the big applications that people are working on now, like, thinking about sort of. I mean, we're working a lot on autonomous driving, and I think there's a lot of power in autonomous driving. But if you think about, well, where is this coming from? We have a lot of folks in the Bay Area who are sitting in traffic for, you know, two hours a day, you know, each way, right? And so, of course, we're going to work a lot on autonomous driving because this is sort of front and center on people's minds. And I want to be clear, there's lots of, you know, access, like, very important sort of accessibility needs and environmental impacts that. And, you know, the 100 people that get killed, economic impact. And also, you know, I think in the US 100 people a day die in car accidents, right? Which can be remedied. But if you think about the relative lack of work on solving hunger crisis and solving sort of economic. And I think a lot of that sort of comes from who is the health, like, who is driving some of this, some of this work. And if you look at, you know, if you look at startups, if you look at research projects that sort of different students undertake, you will kind of immediately see some of the connections where what people are passionate about, what they're choosing to work on, is very much influenced by their values, by their culture, by their upbringing, right? That's what they feel passionate about that's what they're going to work on. And so if we want all of these applications to be actually sort of solved and tackled and get the attention that they deserve, then we need people who are going to be passionate about that kind of work, about sort of each of those.
- [21:50]
B
How does it happen though when we, when as a society, at least in the US we have the incentives that we do. Right. Where I mean, built around, you know, the capitalistic structure that we all, you know, we all exist in and the motivation of, you know, of private, private companies. I mean, yeah, I, I definitely hear you on the autonomous driving thing. Like from a humanity standpoint there probably are like more important things, but.
- [22:16]
A
You.
- [22:16]
B
Know, a company or a corporation or you know, an organization trying to maximize economic impact, like you can see why it leads to things like full self driving or identification of the, you know, to use the example you used earlier of like identification of the people in the photo for the social media site, you know what I mean? So like how does, how do you shift it away when the incentives very clearly point in the other direction?
- [22:47]
A
So I don't think we know what will happen with more diversity of thought in the field. I very much hear you. Right. And I'm not sort of Pollyanna ish about this. I grew up in the Bay Area. I understand. Sort of economic. I get it. I don't think we know the kinds of things that can be done that can be potentially sort of aligned with the economic incentives as well. Right. I suspect there is, you know, like, I don't know, the food industry is changing, I think, in many ways and I think that's driven by various economic incentives. And I think, I don't know, kind of the random things that come to mind are sort of like development of the like fake meats. Right. The different kinds of impossible. Right. And that's something that would. We have thought before this started happening that this is something that would have enough economic incentive to drive that, but yet it seems to be happening. There's a lot of money in the medical system, so there's economic incentives to drive some of the medical innovations. I think we're also kind of limited because we're seeing what's happening now. I don't think we have the full creativity that we need to reimagine what are the different kinds of applications that could be tackling and could be aligned with the economic incentives, but also aligned more with the full range of things that we could do with AI and that could have both sort of economic and Social and sort of social good incentives.
- [24:21]
B
Yeah. So how do you do that? How do you get people building in this way?
- [24:28]
A
So what we're trying to do at AI Frill is sort of train more students and provide pathways into the AI space. And sort of concretely thinking about, I mentioned at the beginning that sort of the students that are going into this space come from a small set of universities that have very good AI training programs. And I'm proud to be at one of those universities. Princeton students get a lot of training in AI, but there's lots of universities around the US that don't, that don't have the AI faculty, that don't have the strong AI curriculum or that don't have the capacity to teach students how to pursue some of their own passions in AI. Maybe they have the basic courses, but not, not the opportunity to really do independent work or projects. And so at afrl we run this program called AFL Ignite for college students, particularly targeting black, Latinx and indigenous women and non binary students from around the US that come into our program. It's a year long program. It combines both sort of education on AI and in particular on responsible AI. So sort of thinking about both teaching them some of the core machine learning skills, but also getting them to think about data, about evaluation of these models, about all of the various decisions and values that get embedded into the systems. And then they get to work on a portfolio project guided by industry mentors. And plus there's lots of folks in industry who are very eager and willing to contribute their expertise. And maybe they don't themselves come from some of these backgrounds, but they're very passionate about volunteering with some of these students and sort of bringing in more younger, more diverse, passionate voices into this space. And so they guide the students through a, I almost want to say a passion project of theirs. I mean it's a portfolio project, they're working on something concrete, but it's driven by the students interest of sort of what do they want to explore with this technology. And then we provide some career readiness and career prep up workshops and sort of helping them put together their resume, helping them think about how they're going to apply for sort of their first paid AI internship and kind of get their foot in the door. And then from there hopefully the world is their oyster. Hopefully they've at least sort of gotten to experience some of the joys and hardships of building in this space and they can take it in in the direction that they want from there.
- [27:04]
B
So it sounds like it's about bringing in different and more diverse types of people into the field, rather than say, like, deliberately changing the way the technology is built to maybe, like, correct for something. I'm just thinking of, like, this example that I'm sure you saw. I don't know, maybe it was like, six months ago with Google, Google Gemini, where, you know, Google was worried about, like, the bias of the model. And so they. I don't know exactly how they did it, but they sort of overcorrected it in the other direction to the point where it was basically factually incorrect about a bunch of different things. You're not advocating for that. What you're advocating is for bringing in different perspectives, different types of people that typically are not involved in the field to solve for this. Is that right?
- [27:51]
A
Yes. Yes.
- [27:52]
B
Got it.
- [27:52]
A
Yes. And we've done. I mean, like, a lot of the research in my lab has been about, you know, tackling bias and AI and sort of, how do you define bias? How do you think about bias in these systems? How do you measure it? How do you develop algorithmic methods to correct for it? And all of that is really hard. It's very hard. And then the root problems and the root cause is that we are not being creative enough or thoughtful enough or diverse enough in how we approach this. So some of the, you know, lack of geographic diversity in the data. I mean, I mean, why aren't we. I mean, I mean, we're sort of using the simplest data collection approach, which is we download things from the web and from social media platforms that we know that's the cheapest source of data. But, like, why can't we think about going to some of these countries and sort of collecting data? But all of that can be. All of that can be done. If you think about some of the folks who've been leading voices in the AI fairness space, and some of the people who have identified some of the early issues of bias in these systems that have been people who come from more diverse backgrounds and perspectives, and they've asked those questions that other researchers did not think to ask. I can give sort of one example of this, which is when I was starting as a PhD student, so we were building the lab that I was in, we were building this robot. And the robot would follow commands based on. Based on language. They would sort of give it a command and would follow the command, right? And this voice recognition system would understand everybody else in the lab except for me and other folks were from many different countries with different accents in the room, but would have no problem Understanding it, but it would not understand me. And what do you know, I was the only woman in the lab. And until you have that woman researcher who is in the room who sort of tries to give the command to the robot and it just outputs complete garbage and has no idea, you don't realize this. Like you don't ask these questions, you don't think to ask this. And so then you can't solve it if you don't think to ask this question.
- [30:00]
B
Yeah, super, super interesting. Back to your point about how in terms of how many of these companies, models are collecting data and therefore like lacking data maybe from certain parts of the world. You know, the, the counter, the counterpoint I've heard to that is like, well, when they go, you know, when these models go out and they collect all, all of the data in the entire entirety of the Internet, like that, that actually represents the ground truth of the data because that's everything that exists. And actually if we sort of try to go out and manufacture additional types of data to like unbiased the thing, we're actually like skewing away from the ground truth. Like how do you respond to something like that?
- [30:44]
A
I think it represents the ground truth of what's been put on the Internet. So then I would ask the question of who has access to the people who are putting. Yep. Who has access to the Internet. Exactly. And who has chosen to upload photos to the Internet and what. For what purpose? Right. We upload a lot of photos for advertisement. We're trying to sell products. That's what we.
- [31:05]
B
Social media.
- [31:06]
A
Social. Social media. Right. You're trying to get the most likes. You're going to upload the photos that have particular properties or can upload videos to TikTok. Like it's a carefully cur. Like is it the ground truth or is it a very carefully uninsuition.
- [31:20]
B
Carefully curated. Right. Like it's not intentional. Nobody's like intentionally trying to exclude data. But I think as a result of the use cases, you only end up with this like very selective group of data.
- [31:31]
A
No company wants to intentionally output a biased product out there. Right. That is, that is nobody's interest. Like nobody wants to. To be that obviously. So we take the path of least resistance. But I think sort of to your point about like, is this ground truth? I mean, I think it's important to remember this is not ground truth. Right. And that's more we're moving into like sort of sds, like Science, Technology and society studies which computer scientists historically don't know those Spaces very well. Like we. When you get training, I mean, you said you're a computer science major, right? I was, you know, I was a math major. And then, you know, know the graduate degrees are in computer science. And as a math major in college, I avoided like the plagal humanities courses. Like, I try to take the fewest number possible. Like I.
- [32:18]
B
Right.
- [32:19]
A
We had five that we had to take, but you could double up and take some. That sort of counted for two requirements. And so I took three very strategically, just trying to take the fewest number. And in retrospect, I'm horrified by that. And I really wish I hadn't because I feel like I'm playing catch up and trying to learn all of these things that I just never got training in. But this is why, again, I think coming back to diversity of thought. Well, we need to bring in AI researchers who are interested in doing some of the technical work, but also have this training in social sciences who have some of that background in knowing how to ask some of these broader questions and how to think about the technical building of these from a broader perspective.
- [33:04]
B
Yeah. Something that's coming up for me, which is super interesting, is, you know, there's a lot of talk right now, I'm sure you've heard it, especially from the companies that are building, you know, the large language models, about how we're reaching or we may be reaching some sort of data wall. We're running out of data. We've already trained on the entirety of the Internet. And so, like, where are we? You know, we need to go out and we get. We need to find more data. Right. Or we need to generate synthetic data. We should get to the synthetic data thing. But maybe before that, it sounds like what you're saying is actually maybe the solution to the data wall problem is actually the same as the same to the problem of making these models unbiased.
- [33:46]
A
I don't think there's such a thing as an unbiased model. I think we can mitigate bias in models and we can mitigate problematic behavior in models, but I don't think there's a ground truth of unbiasedness. But the other part of your question I fully agree with. Right. I think the solution to a lot of the issues we're running into with AI is this lack of diversity of thought that, you know, we are taking, like, for example, we're taking for granted. Right. That data is going to be what's driving it, the future. But if you think about AI, 20 years ago, data was not at all valued the shift towards valuing data and sort of viewing data and sort of data collection, data curation and sort of engaging with the data, viewing that as a valuable part of the field. This shift has happened in the past 10 years, maybe 15 years. And this is with sort of ImageNet kind of being the first example and kind of, I would argue, sort of, sort of the transformative moment when folks started really focusing on data and thinking about data as being important. But even with imagenet, even after that, sort of, for many years, trying to get a paper on a new data set into a top computer vision or machine learning or sort of AI conference was a huge lift because people would say, why are you trying to publish a data set? Like, whatever, it's just data. What is the algorithmic novelty and algorithmic innovation? This is something that many of us have kind of fought against for years. And if you look at one of the top ones. So Neurips, they just introduced the data sets and benchmarks track a few years ago to allow for, sort of create a space in this community for publishing dataset papers, sort of papers focused specifically on data and on benchmarking and on sort of evaluation of these models. And the reason why this was needed is because papers were getting rejected from that conference that dealt just with data because we were saying, well, whatever the insights are in the technical and algorithmic innovations, it's not in the data. And so I think the long story short, what I was trying to get at is right now suddenly everybody is saying, oh, we need data, data is the driving force. But first of all, that itself is a new idea or sort of relatively new idea in the space. Second of all, I mean, that's what we know how to do now. We've sort of stumbled upon this solution that involves digesting tons of data and then outputting these models. And it's great, but is that the only path? We don't know. Are there other alternatives? I think right now the alternative is generated data. And you use generative generated data. Synthetic. Right, synthetic.
- [36:51]
B
Is that a real solution or.
- [36:53]
A
I think so.
- [36:53]
B
To me, that seems like that would. But wouldn't that actually just either. Wouldn't that further increase sort of bias and sort of repetitiveness of output? Like we're just. It's just the same stuff getting recycled over and over. Right.
- [37:07]
A
So I thought that. But I think you can be thoughtful about how like, yes and no. I think it's. It's not going to help with issues of geographic diversity if there's no representation of data from particular countries in your original data set. No matter if sort of generated data will help with that. But I think it can help with things like sort of a lot of the data that's uploaded is again coming back to your point about trying to get likes, or it's sort of framed in a certain way or there's certain composition of. It's always these three objects of appear together or the person is always posed in a particular way in the photo. That kind of stuff I think you could get generative models to help with. So they can help break some of that. You can generate from a sort of different compositional distribution than the original distribution. So for example, the easy example is if you. So this is. One of my students was saying this at a meeting recently. So it's top of my mind. So if you always have sort of apples on top of tables in your dataset, very rarely do you have sort of apples on tables. But if your model sort of understands the difference between apples and tables. So now you can generate data where the apples are also sort of under tables or near tables. So you can kind of change the image composition and you can change sort of the distribution of appearances and connection. Sort of, yeah, no different word than composition. And so there's creative ways that you could go about manipulating this. It kind of hinges us on having control over the generative model, sort of having control over what exactly it's generated. So it's not just generating from sort of uniformly at random from the distributions learned, but it's sort of generating more from particular kind of subsets of the distribution or generating kind of different compositions of objects or concepts and so on? Yeah.
- [39:12]
B
Do you think there are any companies or labs that are sort of best positioned right now to sort of in a scalable way unlock new forms of data from maybe geographically or throughout the world? What are the companies that are actually going after this actively right now?
- [39:35]
A
I'm going to put to this and say that it's AI for all whose position to unlock all of these different. All of these different types of things. I think it's going to be the companies that are founded by our students. I think it's going to be folks who are coming into the space who are right eyed, who have not yet been indoctrinated into the current ways of thinking and who are coming in with new ideas and who ask these kinds of questions of can we do something that's radically different from what's being done right now?
- [40:07]
B
Now, yeah. So you worked on Imagenet as a PhD student. Tell us what that was like.
- [40:12]
A
The original Imagenet was constructed by my colleague John Deng, who's at Princeton, and then was advised by Fei, Fei Li, who is my PhD advisor. I sort of worked closely with them. I was around when this was happening, but I was a junior PhD student and I was sort of watching them in awe and watching this unfold. And so I was there sort of from the early days. But it's a very large scale data set that they collected of photos from the web, sort of illustrating different visual concepts. There were a number of sort of key innovations including the use of crowd workers, the use of sort of people in labeling all of these images. The whole data set is more than 20,000 classes and it's 15 million images. Sorry, there we go. And so all of these images are sort of carefully labeled by people on the web. But the big point is that this is what jump started the deep learning revolution. So it provided the data that allowed for some of these models to sort of harness all of the kind of learn all of the patterns from this very large scale collection of photos and then really build these models that are less engineered in terms of the actual architecture, but very, very reliant on really bottom up learning the visual patterns from the data. And this is part of what kind of jumpstart the deep learning revolution. And then we were running sort of the ImageNet challenge for a number of years and kind of collecting more and more data. So that work sort of running the challenge and that paper about the ImageNet challenge, that's where I was sort of lead author on it. I kind of stepped in right at the boom of ImageNet and kind of helped lead the project when Jha was graduating and sort of moving on to become a professor himself. And I was kind of still a PhD student and very excitedly took on some of that role. So I played an important role, but I wasn't sort of the early pioneer of this. And I want to give Jia and Fei Fei and their collaborators lots and lots of credit for sort of going out on a limb and trying to really do something different. This was completely different than what folks were doing at the time. This was the emphasis on data and on the bet that collecting more data will actually unlock sort of the next generation of AI models, which is in fact what happened. But I think watching that now, my question is, well, that bet had panned out really well and now this is sort of taken as a given, as a sort of ground truth, as the default in the field. But I would like to see somebody else make a different bet. This is the time for somebody else to make an equally ambitious bet that it's not data or algorithms, but it's, it's sort of something else entirely. It's, maybe it's models of, you know, maybe it's human brain inspired models. Maybe it's, you know, something like. I, I don't know what it's going to be, but, but I would, I would like to see those kinds of ambitious bets.
- [43:11]
B
Yeah. And, and I've definitely, you know, at least from sort of the private sector, like, I've, I've definitely heard of some impressive and noteworthy researchers going after exactly what you're saying and asking the question, like, like there have to be more scalable ways to break through. Right. Without just needing, like, all of the data in the world. So it'll be fascinating to see what comes of that, either, you know, from some of these companies or from AI for all students. Yeah, Olga, this has been fascinating. I've learned so much. I'm super confident the audience has as well. So really, really want to thank you for the time today and, and, yeah, hope to do it again sometime.
- [43:51]
A
Yeah. Mike, thanks so much for having me. This was a blast. Thank you.
- [43:57]
B
Thank you so much for listening to Generative Now. If you liked what you heard, please rate and review the podcast. That really does help. And of course, subscribe to the podcast so you get notified every time we publish a new episode. If you want to learn more, follow LightSpeed, LightSpeed VP on YouTube X or LinkedIn. You can follow me at McNano M I G N A N O on all the same places. And Generative now is produced by Lightspeed in partnership with Pod People. I am Michael McNano and we will be back next week. See you then.