Harvard Data Science Review Podcast Summary

Episode Title: ChatGPT in the Classroom: Breeding More Cheaters or Better Learners?
Release Date: April 25, 2024
Hosts: Liberty Vittert & Shelley Meng
Guests: James Zhao (Stanford University, Stanford AI Lab) & Liz Rose Shulman (Northwestern University, Township High School)

1. Introduction

In this thought-provoking episode of the Harvard Data Science Review Podcast, hosts Liberty Vittert and Shelley Meng delve into the contentious role of generative AI, especially ChatGPT, in educational settings. Joined by esteemed guests James Zhao, a Stanford professor and member of the Stanford AI Lab, and Liz Rose Shulman, a Northwestern University professor and high school teacher, the discussion navigates the fine line between AI as a tool for enhancing learning and its potential to undermine critical thinking skills.

2. Perspectives on Generative AI in Education

a. James Zhao on the Rise and Impact of Generative AI

James Zhao opens the discussion by highlighting the rapid advancement and ubiquity of large language models (LLMs) like ChatGPT. He notes, “[...] large language models and generative AI is becoming very rapidly, very prevalent in everything that we see and interact with” (00:00:02). Zhao emphasizes the transformative potential of these tools in education, citing his own observations of AI-generated content infiltrating academic reviews.

b. Liz Rose Shulman's Dual Role in Education

Liz Rose Shulman brings a unique perspective, teaching both at Northwestern University and Evanston Township High School. She underscores the importance of fostering original thought in students, stating, “...students are all using it to do their homework when they're being told not to use it. So they're missing out on the opportunity to use their imagination and to create their own original thought” (00:03:25). Shulman advocates for policies that discourage AI use in assignments to preserve critical thinking skills.

3. Policies and Enforcement in Classrooms

a. Shulman's Approach to AI Policies

Shulman explains her stringent stance against AI usage in the classroom, emphasizing academic honesty and the cultivation of original ideas. “[...] in both college and in high school, they're learning how to use their own original thought and their imagination to design these things that they can use” (00:05:18). She acknowledges the challenges of enforcing these policies, particularly with large class sizes, and relies on initial year conversations to instill the value of original work.

b. The Reliability of AI Detection Tools

Shelley Meng probes into the effectiveness of AI detection tools, questioning their reliability. Shulman mentions tools like GPT Zero, which provide a percentage likelihood that a text is AI-generated but notes the practical difficulties in using these tools for large numbers of students (00:06:15). James Zhao adds a critical perspective, highlighting the limitations and inaccuracies of current detection technologies: “[...] these detectors can often mistakenly flag a lot of the texts that are actually written by real humans to be GPT generated because there are certain biases” (00:09:59).

4. Critical Thinking and AI Integration

a. Using AI as a Teaching Tool

Liberty Vittert raises an intriguing question about leveraging ChatGPT to enhance critical thinking skills. She shares an anecdote where ChatGPT provided a biased response to a student's request for a poem about Donald Trump versus Joe Biden (00:12:06). This leads to a discussion on whether AI can serve as a platform for teaching students to critically evaluate and engage with AI-generated content.

b. Zhao's Comparison to Calculators

James Zhao draws a parallel between the advent of calculators and ChatGPT, suggesting that just as calculators didn't diminish mathematical reasoning, ChatGPT can be integrated without eroding critical thinking. He states, “ChatGPT is a very sophisticated form of a calculator. It can do all sorts of calculation for us” (00:13:04). Zhao advocates for teaching students to critically assess AI outputs, enhancing their analytical skills rather than replacing them.

c. Shulman's Concerns on Critical Thinking Development

Contrasting Zhao's optimism, Shulman expresses concern that reliance on AI hinders the natural development of critical thinking skills. “[...] they haven't developed that basic ability to think through problems because they're missing critical thinking skills” (00:11:50). She stresses that AI was not designed for educational purposes and warns against its unchecked integration, which could impede students' ability to problem-solve independently.

5. Tailoring AI Tools for Educational Purposes

a. Customizing AI for Effective Tutoring

James Zhao discusses the potential for customizing LLMs to better serve educational needs. He suggests training AI models with data from effective human teachers to create more interactive and thought-provoking tutoring experiences. “We want to align these language models so that they actually would know how a good useful tutor would behave” (00:18:03). This approach aims to move beyond mere content generation to fostering deeper engagement and critical analysis.

b. Preserving Human Connection

Shulman emphasizes the irreplaceable role of human interaction in education. “[...] the human connection particularly that has really been missing, I think since COVID” (00:22:56). She highlights that while AI can offer tools, it cannot substitute the nuanced understanding and personal relationships that teachers build with students, which are crucial for effective learning.

6. Fairness and Biases in AI in Education

James Zhao addresses concerns about the fairness of AI tools, particularly regarding access and inherent biases. “[...] the model often still reflect biases and limitations in its training data” (00:23:23). He points out that LLMs perform better in languages and dialects with more extensive training data, leading to disparities in educational outcomes. Zhao also notes that user representation can influence AI responses, potentially reinforcing societal biases.

7. Future of AI in Education and Policy Recommendations

a. Shulman's Vision for Educational Policies

When asked about ideal policies, Shulman urges a fundamental reevaluation of educational objectives. She advocates for policies that prioritize teaching students how to think critically, irrespective of AI usage. “[...] students are not learning the critical thinking skills that they're going to need to think critically with AI when they graduate from high school” (00:31:14).

b. Zhao's Integration of AI into Curriculum

James Zhao envisions a future where generative AI is seamlessly integrated into educational curricula. He recommends training educators to effectively use and tailor AI tools, ensuring that these technologies serve as supportive co-pilots rather than replacements. “Incorporating it into our curriculum and teaching all the stakeholders [...] how to responsibly and effectively use these very powerful technologies would be my wish” (00:32:12).

8. Conclusion

The episode concludes with Shelley Meng reflecting on the nuanced viewpoints discussed. Emphasizing the necessity of critical thinking, Meng acknowledges the challenges and opportunities presented by AI in education. She envisions a future where AI tools are harnessed thoughtfully to enhance learning without compromising the essential human elements of education. The hosts express hope for continued critical engagement with AI technologies to foster a well-equipped and thoughtful generation of learners.

Notable Quotes:

• James Zhao (00:02): “...these language models and generative AI is becoming very rapidly, very prevalent in everything that we see and interact with.”

• Liz Rose Shulman (00:05:18): “...they're learning how to use their own original thought and their imagination to design these things that they can use.”

• James Zhao (00:09:59): “These detectors can often mistakenly flag a lot of the texts that are actually written by real humans to be GPT generated because there are certain biases.”

• Liberty Vittert (00:12:06): “...the technology specific to Jenny's poem... the model would just do it. Is there in a sense the way that if we as educators choose to utilize ChatGPT, we can actually use it as a learning experience for their critical thinking skills...”

• James Zhao (00:13:04): “But on the other hand, it's increasingly even more important for them now to be able to, for example, evaluate an output of either their human colleague or of a language model.”

• Liz Rose Shulman (00:22:56): “...the human connection particularly that has really been missing, I think since COVID.”

This comprehensive summary encapsulates the vital discussions from the episode, providing valuable insights into the evolving landscape of AI in education. Listeners gain a balanced understanding of the potential benefits and challenges, informed by expert perspectives from both higher education and K-12 environments.