The High Stakes of the AI Economy, Live from the WBUR Festival

Summary

Podcast Summary: "The High Stakes of the AI Economy, Live from the WBUR Festival"

Title: Is Business Broken?
Host: Questrom School of Business
Guest Panelists: Barry Feingold (Massachusetts State Senator), Divya Sridhar (VP at Better Business Bureau), Asu Ozdoglar (MIT), Andre Hadju (Boston University)
Release Date: July 3, 2025

---

1. Introduction

In the episode titled "The High Stakes of the AI Economy, Live from the WBUR Festival," host Kurt Nickish engages in a profound discussion with a panel of experts about the multifaceted implications of artificial intelligence (AI) on society, the economy, and regulatory frameworks. The conversation delves into who stands to benefit or lose in the burgeoning AI-driven landscape and explores the delicate balance between fostering innovation and ensuring safety.

---

2. The Necessity of AI Regulation

Barry Feingold initiates the conversation by emphasizing the importance of regulating AI to address practical challenges. Using the example of ticket bots for high-demand events like Taylor Swift concerts, he illustrates how AI can create unfair barriers for consumers. Feingold states:

"We banned bots. That's artificial intelligence. That is one small example of why we need to regulate artificial intelligence."
[02:00]

He advocates for state-level regulatory authority, especially in light of federal actions like the House of Representatives' recent law restricting state regulation of AI for the next decade. Feingold expresses skepticism about the efficacy of such federal preemptions.

Divya Sridhar underscores the dual need for trust and accountability in AI systems. Citing a KPMG study, she highlights a significant trust deficit among users:

"More than half of the population uses AI regularly, but only 46% trust an AI system."
[03:06]

Sridhar advocates for setting baseline practices and holding companies accountable without imposing rigid, potentially stifling laws.

Asu Ozdoglar echoes the necessity of thoughtful regulation, pointing out areas where AI can cause harm, such as bias, misinformation, and labor disruption. She emphasizes:

"AI should be regulated where humans without AI are already regulated."
[05:37]

Andre Hadju advises caution against over-regulating AI, suggesting that regulations should address specific market failures rather than blanket rules. He provides examples like trust in AI-generated content and copyright issues in generative AI for music, indicating that existing frameworks might suffice with necessary adjustments.

"If there's no clear, identifiable market failure, I don't see why we should intervene."
[07:25]

---

3. Market Power and Competition in AI

The discussion shifts to the concentration of market power within the AI industry.

Andre Hadju contends that the AI market is not overly concentrated, noting the diversity of players, including emerging companies like Anthropic and Mistral, as well as open-source initiatives.

"I don't think it fits any definition of a very concentrated industry."
[11:25]

Asu Ozdoglar partially concurs but raises concerns about foundational models being resource-intensive, potentially leading to market concentration among a few major players with significant data and compute advantages.

"Developing foundation models is very data and compute heavy... there's a concern that they are in the hands of a few players."
[12:35]

---

4. Algorithmic Transparency and Accountability

The panel delves into the intricacies of algorithmic transparency.

Divya Sridhar presents real-world cases handled by the FTC, illustrating the pitfalls of AI misuse, such as biased facial recognition and inaccurate AI-generated legal citations.

"We found that companies were not providing the ability for users and parents to give verifiable consent."
[09:58]

Asu Ozdoglar highlights the opaque nature of AI models, which hampers human-AI collaboration and effective regulation.

"AI models are black boxes... this lack of transparency or legibility of these models impedes human-AI collaboration."
[16:12]

Andre Hadju is skeptical about the necessity of extensive transparency regulations, arguing that market pressures often incentivize companies to improve accuracy and reliability on their own.

"I'm not convinced that we should require algorithmic transparency from all AI providers... it may stifle innovation."
[18:52]

---

5. AI's Impact on Jobs and the Economy

A significant portion of the discussion centers on AI's potential to disrupt the job market, especially entry-level positions.

Barry Feingold expresses concern over AI's ability to eliminate white-collar jobs, drawing parallels with historical job disruptions during the industrial and internet revolutions. He notes:

"Senator... AI is starting to rear its ugly head... what do entry-level people do when ChatGPT can perform those tasks quickly?"
[20:03]

Feingold emphasizes the urgent need for societal adjustments, including retraining programs to prepare the workforce for the changing economic landscape.

Andre Hadju focuses on the educational sector's role in mitigating job displacement, suggesting that academia can spearhead initiatives to equip students with skills relevant to emerging job markets.

"How do we get students to learn skills that can get them into new types of entry-level jobs?"
[36:18]

---

6. International Regulation and AI

The panel explores the global landscape of AI regulation, considering differing approaches by major powers.

Andre Hadju dismisses simplistic regulatory tactics, such as capping AI model parameters, arguing they could hinder technological advancement and competitiveness.

"There was some discussion... about regulating the number of parameters that AI models are supposed to have. I think that's absolutely silly."
[24:06]

Divya Sridhar points to the role of international standards bodies like NIST in setting foundational guidelines, advocating for collaborative efforts to establish effective guardrails without broad legislative overreach.

"NIST provides general parameters for what those foundational AI models should be built on... certifications and standard-setting bodies can help."
[25:34]

---

7. Environmental Impact of AI

The conversation addresses the significant environmental footprint of AI technologies.

Divya Sridhar raises concerns about the carbon emissions from data centers and the broader energy demands of AI infrastructure.

"There's the enormous carbon footprint... data centers have implications down the road on our carbon footprint."
[33:09]

Barry Feingold connects this issue to state-level energy policies, advocating for a transition to renewable energy sources to mitigate AI's environmental impact.

"We want to change how we get our energy... grow solar, wind, many others... we'd be a lot more environmentally friendly."
[34:38]

Asu Ozdoglar adds that reducing the energy needs of AI models is crucial and links this to the necessity for sustainable development practices within the industry.

"There's a lot of work around reducing the energy needs of the models... raises concerns."
[33:51]

---

8. Policy Recommendations

As the panel concludes, each expert proposes actionable policy steps to navigate the AI landscape.

Barry Feingold advocates for implementing kill switches and whistleblower protections as essential safeguards without hindering innovation.

"Having some of the reporting models, having some of the kill switch, allowing there to be whistleblower protections would be... some of the guardrails."
[35:06]

Divya Sridhar recommends establishing independent self-regulatory accountability organizations to ensure companies adhere to best practices and implement necessary safeguards like kill switches.

"Companies need independent self-regulatory accountability organizations... there is a kill switch at some point."
[35:27]

Asu Ozdoglar emphasizes the need for robust auditing frameworks and mechanisms for red teaming to ensure the safety and reliability of AI models.

"Thinking about robust auditing frameworks for safety of these models... mechanisms for red teaming... audit schemes."
[35:44]

Andre Hadju focuses on the education sector, suggesting that academic institutions play a pivotal role in developing strategies to address entry-level job displacement and equip the workforce with relevant skills.

"What do we do about entry-level jobs and can we identify how much of that is due to AI?... exactly what academia could contribute here."
[36:18]

---

9. Conclusion

Kurt Nickish wraps up the episode by highlighting the collective insight that proactive and thoughtful policy-making can guide AI development in a direction that balances innovation with societal well-being. He encourages listeners to engage with future episodes and partake in the ongoing conversation about the role of business in a rapidly evolving technological landscape.

"We can make choices about all this. It doesn't seem too late. We have time to make them wisely."
[36:54]

---

Notable Quotes:

• Barry Feingold:

  "We banned bots. That's artificial intelligence. That is one small example of why we need to regulate artificial intelligence."
  [02:00]

• Divya Sridhar:

  "More than half of the population uses AI regularly, but only 46% trust an AI system."
  [03:06]

• Asu Ozdoglar:

  "AI should be regulated where humans without AI are already regulated."
  [05:37]

• Andre Hadju:

  "I don't think we should require algorithmic transparency from all AI providers... it may stifle innovation."
  [18:52]

• Barry Feingold:

  "We can have both. We can have innovation and we can have protection. It's not a zero-sum game."
  [28:32]

---

This episode of "Is Business Broken?" provides a comprehensive exploration of the complexities surrounding AI's integration into society and the economy. The panelists offer diverse perspectives, underscoring the need for balanced regulation that fosters innovation while safeguarding public interests.

Transcript

[Kurt Nickish]

You're listening to Is Business Broken? A podcast from the Merotra Institute for Business, Markets and Society at Boston University Questrom School of Business. I'm Kurt Nickish. Artificial intelligence promises massive gains in productivity and a flourishing of goods and services. AI also has consequences for workers and jobs for corporate power, society and the environment. Who's winning in this new economy? Who's being left behind? Can we strike a balance between innovation and safety, between competition and policy? Basically, can we regulate AI without choking progress? Or are we already too late? These are questions we posed at a live event at the WBUR Festival. On stage to speak with me were Barry Feingold, Massachusetts State Senator for Essex and Middlesex counties, and Divya Sridhar, Vice President of the Global Privacy Division and Privacy Operations at the Better Business Bureau National Programs. ASU Ozdoglar, head of the Department of Electrical Engineering and Computer Science at mit. She's also the Deputy Dean of MIT Schwarzman College of Computing and Andre Hadju, professor of Information Systems at Boston University Questrom School of Business. Here's our conversation with some questions from the audience. Let's just start with a basic question, I'm just curious from each of you briefly. Why should we even think about regulating AI? Should we, should we regulate AI? And let's start with the policymaker over there.

[Barry Feingold]

Senator, thank you and I appreciate everyone coming here today. I think when we think about AI, when I talk to my son and I tell him his superpower is his imagination, and I think we have to imagine things differently under AI, but. But at the same time we have to think about real life, practical applications. And I think because of that, we do need to regulate AI. And what's a specific example of that? Well, 10, 20 years ago, if there's a big concert coming to town, you see everybody in a long line trying to get tickets when they go on sale. But now what happens if Taylor Swift is coming to town? We had all these bots that be buying up all the tickets and a lot of us who want to go to that show can't go to that. So what we did last year in the economic development bill is we banned bots. That's artificial intelligence. That is one small example of why we need to regulate artificial intelligence. But if the federal government gets what they want, we're not going to be able to do anything on the state level because at the end of the day, we don't know what we don't know. And that's why I think state level people like myself should have the ability to regulate AI.

[Kurt Nickish]

Although the House of Representatives this week. Right. Passed a law saying states should not be able to regulate AI for 10.

[Barry Feingold]

Years, which we will get into. Why? That's not such a good idea.

[Kurt Nickish]

Yes, exactly.

[Divya Sridhar]

Yeah. I think there are two reasons. The first is trust. There was a great KPMG study that came out. It covered 48,000 people in 47 countries and it found that more than half of the population that took the survey, 66% use AI regularly. But only 46% of people trust in a AI system. So there's this massive lack of trust. There's a gap there. So first off, we want to make consumers and users feel comfortable. The second important point is accountability. I actually want to push us to think further about whether or not AI truly is living in the wild, wild West. Is it truly not regulated? We set the baseline, we set the practices, and then we go after companies that aren't fulfilling those obligations that they're supposed to be meeting those baselines. So there is this medium, this in between zone, between having hard laws that might pass and then be not useful tomorrow and not doing anything at all.

[Asu Ozdoglar]

AI is a very promising and transformative technology with wide ranging implications. And the regulation should be designed very carefully so that it does not block or impede the development of the technology. But at the same time, I think given its potential harms or misuses, it's important to think about the correct regulatory framework for it to be successful. It should focus on where harms can come from. And in that context, I'd like to maybe add a couple of examples to what was discussed before. The first one is around, of course, bias and misinformation. And that by itself may not be enough of a justification for a formal regulation, because no government should be in the business of defining what is misinformation and what is truth. At the same time, I think AI landscape, ecosystem should be regulated in such a way that there are incentives for companies to focus on increasing the reliability of information and not amplifying or boost unreliable content. One important aspect in this context is digital ads, which have been used on many online platforms to monetize services. And digital ads really depend on attention, user engagement. Their profitability and attention is triggered by sensational content and sometimes extreme content. So this immediately sort of, I think, biases many of the platforms. So there needs to be something in there. The second is the labor disruption that we'll probably talk about more. And of course, new technologies always come with disruptions, but AI has a great promise in terms of really creating new capabilities or jobs for workers. So focusing on this aspect would be important. And finally, I just want to point out healthcare, ironically, I think it's the place where AI will have the greatest promise to improve things, where there needs to be more regulation. If you think about healthcare with doctors and hospitals, they're heavily regulated. You can't really give advice or diagnosis without proper certification. So there's a big problem there, because when you look at chatbots, they give any kind of advice. That's my mom's favorite use of a chatbot, and that's not okay. So I think this was also the main key point under some of the policy work we did at mit. I think a key idea would be AI should be regulated where humans without AI performing, without AI are already regulated. So that could be a good starting point.

[Kurt Nickish]

Got it. I love it when we have experts who answer briefly, and that was briefly for the kind of content that we have to talk about today. Andrej, where do you stand?

[Andre Hadju]

So I think I'm pretty close to asu. My first instinct is to be very cautious with regulation in order to preserve innovation and progress in AI. The way I would approach this is to ask, what are the market failures that justify regulation? So the overarching principle being, if there's no clear, identifiable market failure, I don't see why we should intervene. Now, there are some market failures that we can identify that are due to AI. And I think the question we want to get to later is to what are the specific market failures that AI creates that we haven't seen before? I'll just pick an example to make this clear. So there are some concerns that have existed with other technologies. So, for instance, Divya mentioned trust. Right. So I think if you think about trust generally, so you can say, well, there's all kinds of service providers. We have regulations that ensure that companies don't deceive customers in terms of what they're providing. We already have legal frameworks for this for other services. I don't necessarily think there's something specific to AI in that. Now, if you think about ASU's example, which is, well, what about the issue that with AI, people tend to trust more because it comes from a chatbot, relative to, say, from a Google search there you might say, okay, maybe we have a market failure. Maybe we need to revise a little bit the regulatory framework. Generative AI for music. It can be trained on millions of songs. Sometimes it's unclear how much should the copyright providers be compensated? I'm not necessarily saying it should be regulated but there should be some regulatory framework for this.

[Kurt Nickish]

Gotcha. All right, we've got the central question of how do we regulate AI to protect and serve the public without freezing progress. And you raised a bunch of concerns here. Surveillance, market power, job disruption, misinformation, what keeps you up at night. Divya, you talk to companies every day. What are they nervous about regulators coming in and stopping? And what do they actually think there should be something done because they see a common problem.

[Divya Sridhar]

Yeah, and I think one of the challenges has been that AI has bubbled up over the last three or four years. AI has existed all along. It's just in the last few years where data privacy regulations have also ramped up, that AI has become more of a cornerstone and folks are focusing on it much more heavily. So we work with companies directly. I mentioned I would speak to some of the co regulatory programs we offer at both the global level, the federal level, the state level. Basically, we provide the ability to police and enforce against existing federal frameworks like the FTC Section 5 Authority and safe harbor programs for CAPA, which is the Children's Online Privacy Protection Act. So I'll give you an example from that space. We recently settled a case with Buddy AI. This is a Genai app and it provides language translation services through an edtech medium for children under the age of 13. It was found to not be providing the ability to give the user and the parent choices and verifiable consent. And all of these were flags that we found on our own monitoring efforts. So before it got to the ftc, we were able to find this case, open up the inquiry, work with the company to bring them into compliance with the federal guidelines as well as our own guidelines that we've written in conjunction with the government and close out the case quickly, neatly, and not have to worry about the government's capacity, their resources and time going after this company.

[Kurt Nickish]

So it sounds like a lot of this is sort of existing regulatory frameworks that FTC could apply. Is there a need for new laws or is there. The senator is like raising his hand.

[Barry Feingold]

I just think it's important. I just want to.

[Kurt Nickish]

I guess I should say not the senator, but the lawmaker. Right.

[Barry Feingold]

So, but I just want to context. I chair economic development in the state. All I think about all day long is how do we keep mass users competitive. The last thing I want to do is have us not competitive in AI per capita. We have the most venture capital funding anywhere in the world. We have a golden goose here. So I don't want to Blow it up. However, about a mile from here, 20 years ago, there's a young kid with this great idea called Facebook and no one understood how powerful it could be. But now Meta will admit that one out of three girls, because of their algorithms will say they have body image issues because of what Meta puts on there. We should have stepped in, we should have put guardrails in. I do believe we can have it both ways. I do believe that we can set up guardrails, what you should and shouldn't be doing and we can have a thriving economy that embraces AI. I don't think it's a zero sum game where it's either one or the other. What we've been doing here, let me take one step further is I think point was made of, well, how can you have 50 different AI policies? I completely agree with that. The federal government should be doing it. So what we're doing now is I'm working with the state senator in California, in New York, and we're hopeful to have a universal AI bill that everybody can work with.

[Kurt Nickish]

Andre, let me ask about market power because you talked about a little bit, it sounded like you didn't feel like there was a big threat there, at least at this point. It seems like to the uninitiated that you've got a couple of big players there and it could sort of play out where you only have a Meta. Is there a risk of there just being too much market power for this industry and we end up with a TikTokification of AI that none of us would like to imagine? Maybe.

[Andre Hadju]

This is a topic that I feel fairly strongly about and my co author and I made this observation. It's interesting that if you ask industry people this question, most industry people say it's very competitive. You ask regulators or academics, they tend to think it's concentrated. I think on this one, just having looked at it, I'm much closer to the industry people view. So I don't think it's very concentrated. So you mentioned OpenAI and Gemini, but actually if we go down the list, just think about all the major gen AI players. There's quite a few and some of them are not large companies. There's Anthropic Mistral, there's a few open source ones in China, all the large tech companies, there's xai. So I don't think it fits any definition of a very concentrated industry.

[Asu Ozdoglar]

I somewhat agree and maybe somewhat disagree with andrej. So right now there are reasons why the current AI ecosystem may lead to concentration of market Power and related in this case, I guess concentration of social power. Because developing foundation models is very data and compute heavy and there are various first mover advantages. When you sort of put a lot of time into developing a foundation model, you have more data, you have access to more data. And we've seen this. There are a lot of models out there. They vary in their capabilities and they are used for various different reasons. We know which ones are the best and right now there's a concern that they are in the hands of a few players. It still requires tens of millions of dollars to be able to develop the foundation model. It still builds on a lot of knowledge that has already been created. So it's not like everybody will be able to do that. But now I think our views are changing that players may have different models with different characteristics. I think that's more likely. We don't know. It will also depend very highly on open source.

[Kurt Nickish]

Yeah, yeah, it's still taking shape. Divya, let's go to you and I just want to ask about algorithmic transparency, maybe with buddy AI. Can you get into that concern? Because I think the company's owning the data, that's one thing. But now what's baked into that data? Where did it come from? Who's liable?

[Divya Sridhar]

Yeah, sure. I'll share a little bit about three cases that the FTC's come out with. The first one was with Rite Aid. This was a few years ago, focused on their use of facial recognition technology. And that disproportionately impacted certain communities, certain demographics and was basically miscategorizing, mischaracterizing the shoplifters that were coming to the store. So a really great example of where AI gone wrong with this type of biometrics. In other parts of the world we see the EU finding ways to regulate the biometric space. I think they have a permanent provision in the EU AI act that focuses on biometrics and why there shouldn't be negative impacts on consumers, especially with regard to this disproportionate harm that could Happen. The other two cases I'll mention very briefly, FTC versus Do NotPay. This is the world's first robot lawyer. And actually the legal spaces are super interesting. I want to actually bring up a couple places where that's been found to be a major sector where a lot of issues with AI trying to find sources being used in court cases, incorrectly misciting and even making up citations. I think that's a fascinating use case where AI has gone wrong. So the do not pay case and then that one was busted for deception issues. And then FTC versus Workado, which is AI detection, claims they were claiming they were 98% accurate in identifying AI generated texts when truly they were about 53% accurate. So accuracy claims are a big one that we go after in particular that we find to be a challenge when companies claim to be something that they're actually not. And we think that consumers should get the real numbers, they should get the real data and not be deceived into using a service that it doesn't work. I think the legal space is one that's super interesting to me where we found, I think there was a really good piece in Business Insider that talks about 120 public cases over the last year have had false citations being used. I think some attorney generals have actually incorrectly used AI in different forms. There's just a number of different data points that suggest that the legal profession in particular faces challenges with the accuracy of AI being used in citations and sources. We're all grappling with where to use AI and where to add regulation to AI where it doesn't already exist. So those are a couple great.

[Kurt Nickish]

Su did you want to say something?

[Asu Ozdoglar]

This is a topic I think very critical to development of AI algorithmic transparency. Right now the AI models are black boxes. So they do not have interpretability and legibility. I mean even these terms, it's still being debated and I think this makes it very challenging because one place I mentioned briefly before where AI has great promise is to complement augment humans. So this human AI collaboration for us to understand why AI is making certain recommendations and be able to get that, combine it with our expertise and improve upon what we could do on our own. So that's the great promise of AI right now. This lack of transparency or legibility of these models impede that. And the other aspect is I think close to what Divya is talking about for auditing purposes or any kind of proper regulation, this transparency is also very important. So this is a place, I think, where industry, all AI research and development should put greater emphasis. It may need change in the models architecture and really thinking about how to use this properly.

[Kurt Nickish]

I want to follow up on that. I'm just curious, as a computer scientist, these problems you're identifying, like how technically hard is that, is there sort of a computer science information technology way to address those issues or is it like a really big cost and economic and technical challenge to make things more transparent?

[Asu Ozdoglar]

To share that question right now, I think the biggest maybe issue related to use of AI is hallucinations that a lot of people are talking about. AI just comes up with beautiful explanations, perfectly articulated, that are wrong. And because they are very excessively authoritative in the way they present this information, many of us just take it for granted. And the examples that Divya gave, I think are very illustrative of this aspect. In terms of how to fix this technically, I don't think this is the main issue. Hallucinations. I think there are technical ways that complements what is currently being done through. You may be hearing terms like rags retrieval, augmented generation. There are tools that can be added to current models that actually improve on the hallucination problem. But that's also almost like the tip of the iceberg. It's really about this reliable information. Giving everyone the incentives to actually focus on that problem is the key. And I think for that there needs to be a change in vision mindset, what is prioritized in development of these models and how they're deployed.

[Kurt Nickish]

Okay, Andre, you were making eye contact like you wanted to add something.

[Andre Hadju]

So, very briefly, as regards algorithmic transparencies, I noticed that we shifted from should we regulate? To what should the providers do? So, for example, I am not convinced that the fact that, okay, AI hallucinates the robo lawyers, there's other more ridiculous examples of hallucinations. I mean, those companies will be. There's going to be pressure from their customers or from the markets to correct those. If we go back to the question, should we regulate for that, I'm not convinced at all. I'm not convinced that, for example, the robo lawyers should be regulated just because they make mistakes. I mean, and the mistakes are so ridiculous that if they don't correct them, people will not come back to them. So I could be convinced. I think this could be an interesting conversation. You can say there are certain domains, maybe healthcare, maybe say, financial mortgages or loans, where regulators may say we require the providers to have algorithmic transparency. But I don't think we should say, okay, just because they hallucinate. We need to require algorithmic transparency from all AI providers. Requiring evolutionary transparency is actually very costly and it's very annoying and may stifle innovation that we talked about. So I think we would only do that if we really believe there's a serious problem where they're not going to do it on their own. And I think in most domains it's in their best interest to work very hard to do it.

[Kurt Nickish]

You're raising an interesting point because in a case like that, with robo lawyers, it's clear who's liable right where the mistake was made. In a lot of other cases, it's sort of unclear where the responsibility is. I want to bring in the Senator because you represent an area north of Boston that has some corporate headquarters, also has a lot of people who work at companies in greater Boston. Are you worried about the pace of change for these emerging patterns in jobs and economic changes?

[Barry Feingold]

Well, if you're asking if I'm worried, yes, I'm worried. When the CEO of Anthropic says that 50% of white collar jobs are going to be eliminated and we can have 10 to 20% unemployment, I take it serious. I think that guy's super smart. But I also say to myself, okay, 100 plus years ago, half the people in this room would have been farmers. And we've been able to kind of adjust. But what I'm actually concerned about now is in the last three months, I've never got so many calls about, hey, my son or daughter is graduating from college, or my son and daughter needs an internship. I've never had so many calls. Now, maybe we're in the beginning of a recession, but maybe this AI thing is starting to rear its ugly head a little bit because what does entry level people do? They do the entry level stuff, which ChatGPT or Gemini can do so quickly. So is this the beginning? We have to think about our society different. We went through this with the industrial revolution, with the Internet revolution. Like 25 years ago, if you were a taxi driver or travel agent, I said to you, your job's in trouble. You're like, what? Basically eliminated. And there's gonna be a lot of jobs out there that can get eliminated. And I think as policymakers, we have to think, how do we retrain people, how do we have them think differently and how do we adjust this society? It's going to be hard. I'm not in the camp that in six months or year it's going to be over. I think it's gradual, but it's a lot quicker than what we saw at the Internet. The Internet was 25 years of really changing our workforce. This could be five years.

[Kurt Nickish]

We have some questions that have come in and I want to bring them into the conversation. One is just, shouldn't there be distinctions in the regulations for AI that could benefit scientific discovery and the limits designed for average consumer? Any reactions there, Andre?

[Andre Hadju]

My first reaction is if it's for consumers, I mean, of course the usual liability frameworks apply. So I think my instinct is, yes, there's probably more scope for regulation if it's consumer or customer facing in general for scientific discovery. My instinct would be like, yeah, you probably want to leave more room for experimentation, let's call it.

[Divya Sridhar]

I actually want to challenge that a little bit because I think the consumer deception, those kinds of claims are already being regulated by the ftc, by self regulatory bodies. So I think rather than look at those which are more broad use cases, broad sweep, we look at the high risk sectors and high risk use cases. The health care sector. Right. The financial services sector, where we think there are great benefits and gains, but also could be majorly problematic if they go wrong. That's how the EU is doing it. I'm not saying we follow in the EU's footsteps. The EU is actually scaling back GDPR. They're scaling back the EU AI act just a year after it was passed. So I don't say that we need to overregulate. I think Japan just passed a bill in the House this week. It talks about international cooperation. How can that support guardrails and frameworks for loose regulation of AI? I think we look to other countries to see what's been done, what might be going wrong, and then think through it before we jump to conclusions about what that looks like.

[Kurt Nickish]

Let's stay in the international space just for a minute. Are we heading towards a world of kind of fractional blocks of AI technology and power around the world? Because China's treating it differently than Europe, than the United States so far. Like, what does that future look like? And are you excited by that regulatory future globally or not? Or do you even think about it?

[Andre Hadju]

I don't know who gets excited about regulatory futures. I mean, I'm excited about AI in general.

[Kurt Nickish]

Yeah.

[Andre Hadju]

Oh, yes, fair enough. Sorry, fair enough. Keeps going. It depends on line of work here. This is way too broad of a topic. But for instance, I think what's interesting internationally is competition between different countries. I mean, presumably different countries would want their own AI companies to succeed. I mean, one implication of this might be a race to deregulate. I'll give you an example. What I think is a terrible regulation that was discussed in the US center. Correct me if I'm wrong, but there was some discussion, not here, some lawmakers saying we should regulate the number of parameters that AI models are supposed to have. I mean, I think that's absolutely silly and we can do it and we shoot ourselves in the foot. And of course, for example, China is not going to do that and they just have more powerful models. It's not sustainable on a global level, because other countries will say, well we're not going to limit it and therefore our companies will perform better.

[Divya Sridhar]

I also think there's within the U.S. it's not just about laws and regulations and self regulation. There's also NIST and other government bodies. They are supporting the national standards. So NIST is the National Institute for Standards Development. And so what it does is it provides general parameters for what those foundational AI models should be built on and provides the engineers with the guardrails and with the specs to build on. So I think there's also a role for those certifications, for those standard setting bodies that can really help augment where maybe we don't have a broad brushstroke AI bill or a law.

[Asu Ozdoglar]

May I come back? Maybe go back to the scientific discovery question. It's I think important because that's one of the hoped or one of the biggest promises of AI is supercharging the scientific inquiry process. We saw examples of this. This was big excitement around Alphafold, the promise of AI in solving really complex problems. And there is now a lot of research in materials discovery, being able to discover new materials with properties we've never seen before, or drug discovery or cancer research. So there has not been yet any cures or any really big advances there. But it may be a matter of time. So there's a lot of progress there or promise there. But I'd like to also point out that just like in other cases, this is not just automating this entire process. You know, we have These new scientist AIs that will discover things on their own. I strongly believe it should be still coupled with human scientists because it's really important to bring in the domain knowledge into the development of these models. Just as an example, there have been many, many papers that actually talked about new materials being discovered with AI, hundreds of them. But then actually when chemists, chemical engineers went into the lab to actually synthesize those, these were not synthesizable. So it's not just enough to just get a compound. It's really important to figure out how you actually produce that. This is not about regulation, but it's still the same mindset. I think AI can automate certain entry level things, but in many places it will work alongside humans. And how do we actually come up with the right structure, structures and the vision to actually enable that I think would be important for us to move forward.

[Kurt Nickish]

Yeah, let's bring it back to the Senator. We're talking about jobs here and humans interacting with it. The entry level job disruption is fascinating. Right? It used to be the traditional way to get started in a company. You hire these interns in entry level jobs and if you don't need them, what happens to that whole pipeline? It's a little concerning. So one question we have from the audience from someone named Scott. At what point will we realize that AI is impacting the job market? Is there like a specific unemployment number? You'd want to see other indicators. How do we know that it's a problem? Because jobs are being created and lost all the time.

[Barry Feingold]

We study data all day long in my office. So we were looking at numbers. But there are things I look at like law school applications are way up. Kids can't get jobs. So that's an indicator right now how many kids are going to graduate school. Like I said, the calls we're getting right now, I spent a lot of time talking to CEOs and businesses. What are you concerned about? What are you trying to fill? And when they're saying entry level jobs are just not on my priority, then who hires these people? And ultimately who trains people? And I think the challenge we had with COVID and everything is like we had young people working remote and they're not getting the face to face. So I think it's just getting harder. And I think we found a way to kind of work without the entry level. So young people are going to have to think out of the box and we're going to have to help them think out of the box. I think it's super challenging. If I could just redirect back to. We were talking about where we think global AI and why we shouldn't have regulation. So I'm a little jaded. I saw Mission Impossible this weekend, so if you. Yeah, I'm going to see that movie. Like, oh boy. But I guess I asked the question, like, why is it so bad if you ever have a kill switch in a model for a major company or something like that, why is that such a bad thing? Why is it so bad to have whistleblower protection? Because there are super smart people out there. These aren't fringe people. These are some of the smartest people in the world that have said yes. We think in the next 20 years there could be a catastrophic event due to AI. And I will continue to say this. We can have both. We can have innovation and we can have protection. It's not a zero sum game. Now Andre's. He's already ready to go at me, so the mic is yours. The mic is yours.

[Kurt Nickish]

I mean, it's interesting that there's talk about superintelligence and artificial general intelligence. AGI, are we right to be focusing on these short term risks when there's some people say there's a big existential risk, is that actually the big regulatory thing we should be worrying about or talking about here?

[Andre Hadju]

On the existential risk, I will just, I mean, I'm happy to discuss this over cocktails. I love science fiction. I honestly have not heard a single, reasonably articulated, like really based in science way in which the existential risk happens. There's a lot of debate there, so I'll stay away from that one, thank you. On the other hand, I do agree with Senator's point. So I love the terminology guardrails. Somehow I think guardrail sounds a lot better to me than regulation. So when you say, for example, kill switch, I'm much more amenable to this than say we'll put a cap on the number of parameters just because we're afraid of some nebulous risk that we read about in Asimov's foundation or something like whatever, Skynet taking over. So I totally agree. There's reasonable ways to say, listen, we want to have certain frameworks, but you're free to experiment. We can't pretend that we know exactly what's going to happen.

[Divya Sridhar]

And I think the kill switch is particularly important in the agentic AI space. And you guys have probably been hearing that word, that term is floating around a lot in lots of last two or three months, where instead of the AI being a support tool, it's now becoming the decision maker. So it's actually not just doing the tasks you tell it to do, but it's actually doing it automatically because it's being trained to do that. So this agentic AI space is one where I think it's one to watch. And it's also one that might need that kill switch option because AI, to be frank, doesn't have the emotional guardrails that we do. It doesn't know how to regulate. Right. So being able to provide it with that and having a human in the loop is important. And so there's been a lot of bills that have debated the human in the loop aspect of AI and whether it should exist on its own or whether there should be a human.

[Andre Hadju]

Can I ask a question just again, the provocative question is always why do we think that we as a society need to impose the kill switch and the companies are not going to figure out on their own that they should put a kill switch? I always come Back to this. No, but if you're a company, again, there's significant pressures to not be irresponsible. I mean, do we think that.

[Divya Sridhar]

Are you sure? I mean.

[Andre Hadju]

I'm talking to the businesses broken. I'm talking to the business is broken.

[Asu Ozdoglar]

Well, it might be, except that's not what we are seeing now. There's a lot of focus on AGI and there's sort of this race to the bottom kind of mentality, really competitive in terms of bringing these new capabilities. I think they need to worry about reputational risks, but it's not clear right now that they are moving with that mindset. There's a lot of, I think, rush to be able to get these models out. We've seen examples of this when several companies moved AI products in education without satisfactory results. So you would think that responsibility, that social responsibility also should be embedded in the way things are developing. But I think right now the system is one that there's a lot of uncertainty. There's a lot of wanting to move fast. And I'm not sure if that's being incorporated.

[Kurt Nickish]

I'm going to end with we'll make this lightning roundish, but we'll go through a couple of questions that we have, things that we haven't touched on but are really good points. Could someone touch a little bit on the environmental repercussions and potential harms of AI? We have not talked about that. Should companies be regulated around use of resources?

[Divya Sridhar]

Yeah, I mean, I was just reading about that as well. I think there's two things there. There's the enormous carbon footprint and we know how East Asia, other parts of the world feel about it, versus us, the Western Hemisphere and then the data center's impact, the fact that we're investing so much this administration in data centers, which I don't disagree with, but also is there enough data that shows that those data centers are going to have implications down the road Right. On our carbon footprint? So I think there's this balance that needs to be.

[Kurt Nickish]

Yeah. What's happening in the middle of this feeling of an arms race. Right. There's this rush everywhere to push forward. Okay, so we acknowledge that.

[Asu Ozdoglar]

Yeah, I think we do. There's a lot of concern around that. There's work around that to reduce the energy needs of the models. It's right now really a lot. And that raises concerns.

[Barry Feingold]

Can I just say just one thing real quick? So some we think a lot about and it matches like all of you pay high energy costs. One of the reason why you do is because we want to change how we get our Energy. We export $21 billion a year to other states, other countries for energy. We want to in theory, grow it here, solar, wind, many others. If we can do that, then I think we'd be a lot more environmentally friendly. So we are working on that as a state and it's really, really hard to transform an economy from a fossil fuel economy to a green economy. We're trying to do that in Massachusetts and I think long term that'll be helpful for this issue.

[Andre Hadju]

I mean, I just wanted to echo this. Maybe this will be the impetus because of the huge energy requirements. Maybe this will be the impetus to transition to nuclear or something that's more sustainable than what we have now just because we can't produce the energy we need for AI with what we have.

[Kurt Nickish]

Let me just close here and ask each of you to just say if you could implement one AI policy tomorrow, one concrete, impactful, realistic step, what would it be? I'm going to start with a lawmaker who's probably thought about this.

[Barry Feingold]

That's a tough question. Just one thing, but I think having some of the reporting models, having some of the kill switch, allowing there to be whistleblower protections would be some of the things that I would just basically like to have, which I don't think would disrupt a lot. I think those are guardrails.

[Kurt Nickish]

So the self regulator here, what would you like to see?

[Divya Sridhar]

Companies need independent self regulatory accountability organizations to support them with ensuring they're meeting the best practices practices and going above and beyond. So that way they are putting in the defaults that they're supposed to be putting in to these new tools to agentic AI and so on. So there is a kill switch at some point, right?

[Asu Ozdoglar]

Asio, it is a very hard question. I think there's many things to do. Maybe thinking about robust auditing frameworks for safety of these models and that ties back to the algorithmic transparency, having mechanisms for red teaming, at least for certain places for safety guarantees. Thinking about exante exposed different kinds of audit schemes. So that would be, you know, at least in the first run.

[Kurt Nickish]

Yeah, got it Andre. Tomorrow. Do you have one?

[Andre Hadju]

I don't. And I'm going to avoid your question by going back to the question about education that the center mentioned. So I find that topic very interesting. Like what do we do about entry level jobs and can we identify how much of that is due to AI? At Questroom we have a new AI initiative. I think this would be like sounds like a great project for us to figure out. How do you get students to learn skills that can get them into new types of entry level jobs? Presumably the previous entry level jobs don't work. We need to get them to something different. I think to me that seems like it's clear there's a market failure there. I don't know what the answer is, but this sounds like exactly what academia could contribute here.

[Kurt Nickish]

Yeah, I think if there is one takeaway today, it's this, that we can make choices about all this. It doesn't seem too late. We have time to make them wisely, and we really appreciate all of you for contributing to understanding this problem and identifying ways forward. Thank you so much for attending and please join me in giving this outstanding panel a big hand. That's Senator Barry Feingold, Divya Sridhar, Asua's Daglar, and Andre Hachu. We'll be back with more episodes in the fall and we'd love it if you would please rate and follow Is Business Broken? Wherever you get your podcasts, if you follow the show, you'll be sure to get the new episodes when they come out this fall, and in the meantime, you can check out any episodes you may have missed. Thanks for listening to Is Business Broken? I'm Kurt Nickish.