← Revisionist History: The Staten Island Problem
Revisionist History: The Staten Island Problem
From Smart Talks with IBM: How AI Assistants Can Transform Education
00:27:23
Loading summary
Transcript93 lines
- [00:00]
Amy Robertson
This is an iHeart podcast.
- [00:08]
Malcolm Gladwell
In the world of educational research, there's a famous video of a boy named Sean. I don't mean famous in the sense that it has a million views on YouTube. I mean that in the circle of people who think about teaching and how to make teaching better. The video has been written about in journal articles and shown over and again in college classrooms. It's a 10 minute clip of a third grade class somewhere in Michigan. It was filmed in January of 1990, so the video is a bit grainy. The teacher's name is Deborah Loewenberg Ball. She's a professor at Michigan State University who, as part of her research, teaches a one hour math class at a local elementary school. On the day in question, Ms. Ball begins by asking her students about the previous day's lesson, which was about even and odd numbers.
- [00:56]
Dabe Lee
I'd really like to hear from as many people as possible what comments you had or reactions you had to being in that meeting yesterday.
- [01:03]
Malcolm Gladwell
A little boy with black hair raises his hand. His name is Sean.
- [01:07]
Sean
I don't have anything about the meeting yesterday, but I was just thinking about six.
- [01:12]
Malcolm Gladwell
Sean was thinking about the number six.
- [01:16]
Sean
I was thinking that it's a. It's an ad. It can be an add number two, because there could be two, two, four, six, two, three twos. That would make six and two threes. It could be an odd and an easy number. Three things to make it. And there could be two things to make it.
- [01:38]
Malcolm Gladwell
Sean doesn't understand what odd and even means. He thinks that just because you can break down six in an odd number of parts and an even number of parts, that six must exist in. In some magical middle category. And when you listen to the Shawn videotape, you keep waiting for the teacher to say, oh, no, Shawn, you misunderstand. But Deborah Ball doesn't do that. She never tells him he's wrong. Instead, she simply asks him to explain his thinking.
- [02:06]
Dabe Lee
And the two things that you put together to make it were odd, right? Three and three are each odd.
- [02:12]
Sean
Anything.
- [02:17]
Malcolm Gladwell
Ball then asks the class to give their views. Other students jump up and explain their theories on the blackboard. For the next 15 minutes, she deftly guides the class through an in depth investigation of what she calls Sean numbers. Until Sean himself realizes that the real meaning of odd and even is something different than he had imagined. And now he gets it.
- [02:39]
Sean
I didn't think of that right away. Thank you for bringing it up.
- [02:42]
Malcolm Gladwell
I don't want to focus just on how little Sean finally made his own way to the Right answer. I'm interested in what his teacher did to get him there. Deborah Ball worked magic. She never told Sean the right answer. She just led him to a place where he could discover it for himself. My name is Malcolm Gladwell. This is season six of Smart Talks with IBM, where we offer our listeners a glimpse behind the curtain of the world of technology and artificial intelligence. In this season, we're going to visit companies as varied as l' Oreal and Ferrari and tell stories of how they're using artificial intelligence and data to transform the way they do business. This episode is about the promise of a radical new idea called responsive teaching, the kind of teaching that took place that day in Sean's classroom and whether artificial intelligence can help us train the next generation of teachers to be as good as Deborah Ball. Before we talk about how AI could transform the way we train teachers, I want to go back for a moment to the famous video of Sean. In the video, the teacher, Deborah Ball, doesn't have a predetermined plan that she's imposing on the class. She's improvising, making up her approach as she goes along, responding to her students. Odd theory about the number six. Second, she's taking Shawn seriously. She's not dismissing his theory. She's listening to him and trying to understand the problem from his perspective. And thirdly, and most importantly, she's not force feeding him the right answer. She's being patient. She's waiting to see if, with just the right subtle hints, he can get to the right answer on his own. Improvisation, empathy, patience. That's responsive teaching.
- [04:38]
Amy Robertson
What I think about in terms of responsiveness is more like, I think that students need to have a sense of agency in what happens in the classroom and like authentic agency where they can be legitimized as knowers.
- [04:56]
Malcolm Gladwell
I spoke to a physicist at Seattle Pacific University named Amy Robertson, a longtime advocate for responsive teaching. She uses the Sean video in her classroom.
- [05:05]
Amy Robertson
You have to trust that kids have a way of doing that and that, like her, what she mostly did was to facilitate a conversation and to say, you have to listen to them talk.
- [05:15]
Malcolm Gladwell
No one told him he was wrong.
- [05:17]
Amy Robertson
That's right.
- [05:18]
Malcolm Gladwell
And then he goes. He goes, I didn't think of it that way again.
- [05:22]
Sean
I didn't think of it that way. Thank you for bringing it up.
- [05:25]
Malcolm Gladwell
You've expanded my understanding. Thank you for bringing it up again. It's like, what? I love this kid.
- [05:31]
Amy Robertson
I know, I know, I know. Responsive teaching, as I think about it, is kind of rooted in this. Like Eleanor Duckworth's work around the having of wonderful ideas, where she says, like the goal of education is for students to have wonderful ideas and to have a good time having them.
- [05:48]
Malcolm Gladwell
I love that. I've never heard that. What a beautiful, succinct way of summing up the purpose of education. Yes, responsive teaching is beautiful. It's rare to find a new teaching idea that everyone loves. This is one of those rare ideas. Watching the Deborah Ball classroom, all I could think was, I really, really hope my daughters get to experience a math class like that. Far too many kids are convincing themselves at far too young an age that math isn't for them, and responsive teaching is a way to solve that problem. But here is the issue. It's really, really hard to teach responsive teaching. Robertson says that teaching exists in a cultural environment where the teacher is expected to be the source of truth. That teaching is about the immediate correction of error and not letting a child wander down the pathway of their own misunderstanding. Responsive teaching is deeply counterintuitive, and the only way to understand its beauty is to do it over and over again. Aspiring teachers need a way to practice. For as long as there has been technology, people have turned to digital machines to solve problems. My father was a mathematician, and I remember him coming home in the 1970s with a big stack of computer cards in his briefcase that he used to program the mainframe back at the office. Today, with the rise of artificial intelligence, the scale and complexity of the problems technology can help us solve has jumped by many orders of magnitude. You must have worked with a million customers who are experimenting with LLMs. Has there been one use case that you were like, whoa, I had no idea, or just simply, that's clever. I'm speaking to Brian Bissell, who works out of IBM's Manhattan office. He helps IBM customers discover how best to get AI to work for them.
- [07:44]
Brian Bissell
There is one, but I don't think I can talk about it, unfortunately.
- [07:47]
Malcolm Gladwell
Wait, wait, wait, wait. Come on. You can't tease me like that, can you? Wait. Disguise. Disguise it for me. Just give me a general it was.
- [07:55]
Brian Bissell
About the ability to pull certain types of information out of documents that you wouldn't think you would be able to get the model to do and be able to do that at a very large scale.
- [08:07]
Malcolm Gladwell
Bissell's point was that we are well past the stage where anyone wonders whether AI can be useful. The real question now is what problems do we want to use it to solve where it can make the biggest difference? And Bissell saw lots of opportunities in education.
- [08:24]
Brian Bissell
I have two kids, one in middle school and one who just graduated high school. And I'm well aware of students using things like ChatGPT to do their homework. And it's very easy to take tools like that and even IBM's own large language models and just take a problem, a piece of homework, something you want written, and drop it into that and have it generate the answer for you and the student. The user in that case hasn't done any work. They haven't put any real thought into it.
- [08:57]
Malcolm Gladwell
To Bissell, that's the wrong use of AI. That's technology making us dumber. What we really want is technology that makes us smarter. Bissell explained to me that there are now two big tools being used for AI productivity. AI agents and AI assistants. Let's start with AI agents. AI agents can reason, plan, and collaborate with other AI tools to autonomously perform tasks for a user. Bissell gave me an example of how a college freshman might use an AI.
- [09:29]
Brian Bissell
Agent as a new student. You may not know, how do I deal with my health and wellness issue. How many credits am I going to get for this given class? You could talk to someone and find out some of that, but maybe it's a little bit sensitive and you don't want to do that.
- [09:44]
Malcolm Gladwell
Bissell told me you could build an AI agent, a resource for new students that helps them navigate a new campus, register for classes, access the services they need, and even schedule appointments on their behalf, which in turn buys them more time to focus on their actual schoolwork.
- [10:00]
Brian Bissell
We can see patterns of how agents and assistants can help employees and customers and end users be more productive, automate workflows so they're not doing certain types of repetitive work over and over again and streamlining their lives and making data more accessible to them 24 hours a day.
- [10:22]
Malcolm Gladwell
But Bissell says you can also use AI assistants in the education space. AI assistants are reactive. And as opposed to AI agents, which are proactive, AI assistants only perform tasks at your request. They're programmed to answer your questions. And as it turns out, AI assistants are now being used to further the responsive teaching revolution. Which is why I found myself on a beautiful Georgia spring day not long ago on the campus of Kennesaw State University, sitting in a classroom with two researchers. With one of them, Professor Dabe Lee. Let's go into the journey of building this thing. You started, Dabe, by taking a course. What was the course you took?
- [11:05]
Ji Woo
Yeah, so it was offered by Coursera, it was designed by IBM. It was AI foundation for everyone.
- [11:17]
Malcolm Gladwell
In her AI foundations course, Lee learned how to build an AI assistant using IBM WatsonX. That course took how long to take?
- [11:26]
Ji Woo
It was not too long. It was like 14 weeks.
- [11:29]
Malcolm Gladwell
Lee's idea was to train an AI assistant on classroom data to play the role of Sean, a digital Persona of a nine year old who likes math but doesn't always understand math. And that AI assistant she thought could be used to train pre service teachers or teachers in training who are preparing to enter one of the most challenging professions in the modern world.
- [11:52]
Ji Woo
So when you think about the teacher education and a major challenge that teacher education face is that we need children to practice with. We need instructors who will give the instruction on the pedagogical skills. So when you look at the teacher education program, we have coursework and field express. And in those two areas there is something missing all the time.
- [12:22]
Malcolm Gladwell
Lee says that pre service teachers often lack access to both students and experienced teachers during their education.
- [12:29]
Ji Woo
So what we try to resolve is that we have this virtual student for pre service teacher to work with so that they can practice their responsive teaching skills.
- [12:42]
Malcolm Gladwell
The first AI assistant Lee created is Ji Woo. Ji Woo emulates the Persona of a nine year old third grade girl. Then with the help of one of her collaborators, a researcher at Kennesaw named Sean English, she created two more AI assistants, Gabriel and Noah, each of which have their own distinctive characteristics. So how are Gabriel and Noah different from Jiwoo?
- [13:07]
Ji Woo
Gabrielle, my first one, he's very short, answered. If you ask an open ended question, he will answer it in a closed way. So I use that characteristic. And that's the problem that most teachers actually face. They're asked children who are shy, who are reserved, and who would not share much of their thoughts. So we wanted that characteristic in some characters and we use Gabriel to have that characteristic.
- [13:40]
Malcolm Gladwell
And Noah, what's Noah's personality?
- [13:43]
Sean English
Happy, playful, cheery, bright, energetic.
- [13:48]
Malcolm Gladwell
That's Sean English, Professor Lee's fellow researcher, and Jiwoo.
- [13:53]
Ji Woo
Jiwoo is articulated and kind of smart, but she has her own way of thinking.
- [14:00]
Malcolm Gladwell
I would end up spending a lot of time with Ji Woo. She's something of a character. I asked Sean about the process of creating these AI assistants. What does building the content side of the AI assistant entail? Sean?
- [14:15]
Sean English
It sets up a series of actions effectively, which are response cases. You could kind of think of them as you have a series of questions that you tie to an intent and then that intent has reactions from the bot. And so effectively, if we were looking to say make a hello action, we would have all the different ways that people could Say, hello, hello, what's up, how you doing? All that kind of stuff.
- [14:38]
Malcolm Gladwell
Sean says the longer the list of potential responses, the better. But AI's responses don't just follow the list. The AI assistant uses those suggested responses to come up with a universe of other responses. And in that process, sometimes it comes up with things that just don't make sense.
- [14:57]
Sean English
And from a technological standpoint, while AI is a fantastic tool, AI can hallucinate, which means just give things that it's just straight up made up. There's a famous example of this called the three R's is where you ask a popular large language model how many Rs are in strawberry and it gives you the wrong answer and it repeats that result repetitively. You always want to have a human interacting with the system to be able to go, hey, that's a little crazy. I don't think that's exactly what we're going for here.
- [15:23]
Malcolm Gladwell
That's why it's good to have someone like Sean English around to step in and get the model back on track. And over time, when a model has enough training, it's ready for the teachers in training. One of the rollouts of Jiwoo, Gabriel and Noah was with the teacher training program at the University of Missouri.
- [15:42]
Logan Hovis
I was just kind of excited to see what the program was and what it was going to be doing.
- [15:47]
Malcolm Gladwell
This is Logan Hovis, a junior at Missouri on the path to becoming an elementary school teacher.
- [15:53]
Logan Hovis
Obviously a little skeptical when he said it was supposed to, you know, be like talking to a student. You're like, there's no way this AI thing is going to totally sound like a second grader or a third grader. Like it's going to sound like an adult or it's going to sound like a robot that knows all the answers. And it really didn't. It really was like talking to a child. It was very, very well developed in the way that you really sit there and you feel like you're talking to a kid.
- [16:18]
Malcolm Gladwell
Her point wasn't that Jiwoo and her fellow avatars were equivalent to real kids. Of course not. But for someone starting out, someone who was already nervous about being plunged into a classroom of nine year olds, Jiwoo was like a warm up before a baseball game.
- [16:33]
Logan Hovis
What I can think of is like, you know how when you're at batting practice for baseball or softball, you have those automatic pitchers that throw them because you're working on your skill as the hitter? What can I do differently? What am I doing wrong? But that doesn't replace the game and what you do in a game. But this is you getting to practice your own skills to be better when you go in a game. And I think that's kind of what the AI software feels like for us.
- [16:58]
Malcolm Gladwell
In batting practice. The pitches don't come as hard and fast as the pitches in a real game, but you get to stand at the plate and the pitcher throws you dozens of balls over and over again in a concentrated block that allows you to work on your swing closely and carefully.
- [17:14]
Logan Hovis
There's a lot less stimulus going on around because the classroom is very, very busy. It's wonderful, it's beautiful, but it's very, very busy. So sometimes it's hard to keep, you know, that focus in on the tasks that they're doing at hand. And also in the teacher setting, you're also kind of always looking around, making sure that other students are doing what they're supposed to be doing, but also like if they need any help, if everything's going okay in the classroom. So being on the Jiwoo chat, it was just nice that you didn't have to do any of the extra work to keep the focus on there. And it also felt you didn't have to feel the student's nervousness of being one on one with you. And also as the teacher, it was a lot less pressure too, because I was like, okay, I'm taking this serious. This is a student I'm questioning. But I also know I'm probably not going to hurt someone's feelings right now. And that's terrifying to think I'm going to ask the wrong question and upset a child because I've done that.
- [18:12]
Malcolm Gladwell
We think of the typical use of AI as a tool for speeding things up. That's what we always hear, that the introduction of AI to problem X gave an answer in minutes, when solving problem X used to take weeks. But we shouldn't forget another use that it allows us to slow things down. Hovis, if she wanted to, could spend a whole weekend practicing with Ji woo. A real 9 year old will get frustrated and bored with the fumbling novice after 10 minutes. But Ji Woo. Ji Woo will happily answer questions for as long as it takes for the people who, who want to learn to be responsive, to learn how to be responsive. At the end of my time at Kennesaw State, Sean and Dabe led me to a small table where Dabe had set up her laptop. In the corner of the screen was a chat box of the sort we've all seen and used a thousand times. Jiwoo Began. She had been given a math problem.
- [19:10]
Ji Woo
Martin was making play. D'oh. He added 3, 4 cup of flour to the bowl. Then he added another 3 sixes. Cup is the total amount of flour he used greater than or less than one cup. How much flour did he use?
- [19:34]
Malcolm Gladwell
That's a simulation of Jiwoo speaking. Wait, pause it for a second. So Jiwoo is trying to solve this problem. And the first thing she does is she draws a rectangle on the screen. This is a common tactic of nine year olds. Try to visualize the fractions. And she divides it into four pieces. And now she's gonna color in three of the four pieces.
- [20:00]
Ji Woo
Yes.
- [20:00]
Malcolm Gladwell
So she's representing. This is quite good. She's representing three quarters on the screen. Okay.
- [20:05]
Ji Woo
And this is three sixes.
- [20:11]
Malcolm Gladwell
So now Jiwoo does another rectangle with six boxes and colors in three of them. Okay.
- [20:19]
Ji Woo
And when you add them together, that makes six out of 10.
- [20:27]
Malcolm Gladwell
So then she counts up all the colored boxes and that's her numerator. And counts up the total number of boxes and that's her denominator. Jiwoo had counted the colored boxes and landed on an answer. When you add 3/4 of a cup and 3/6 of a cup, you get 6/10 of a cup. So according to Ji Woo, Martin has less than one cup and she thinks she solved the problem.
- [20:52]
Ji Woo
Yes. Okay, so it's less than one cup.
- [20:55]
Malcolm Gladwell
Yeah. So she says it's less than one cup now. Oh my God, this is hard. So the question is, what do I as a teacher say to Jiwoo? We were off. The rules were simple. I couldn't give Jiwoo the answer or explain to her what she was doing wrong. I had to be Deborah Ball. I had to help her find the way herself. The chat box in the corner of the screen was waiting for my first question. I thought for a moment and started typing. Do you think the boxes in the red rectangle are the same size as the boxes in the blue rectangle? Then I turned to Sean and Dabe. Is that a good question?
- [21:33]
Ji Woo
Yeah, go ahead.
- [21:34]
Malcolm Gladwell
Seriously, Did I?
- [21:35]
Ji Woo
Yeah, that's a good question.
- [21:37]
Malcolm Gladwell
Jiwoo doesn't mess around. She answers immediately. So Jiwoo says the blue and red pieces are not the same sizes.
- [21:45]
Ji Woo
Oh, so you understand now? Jiwoo knows the size differences.
- [21:51]
Malcolm Gladwell
So she's pretty smart here.
- [21:52]
Ji Woo
Yeah.
- [21:53]
Malcolm Gladwell
Then I asked, if they are not the same size, do you think you can add them together? Jiwoo answered right away. Jiwoo says, I have learned that I could add Any numbers in grade two. So three plus three is six and four plus six is ten.
- [22:08]
Ji Woo
Yeah. So she is using the knowledge of adding integers into adding fractions.
- [22:15]
Malcolm Gladwell
Now I'm stumped. So now I have to somehow lead her to figure out a way to get her to understand that we're dealing with a different kind of problem, a harder problem. Amy Robertson had told me that learning how to do responsive teaching properly was really hard. And now I understood why. I had to put my mind inside the mind of a nine year old. I had to internalize her knowledge base and assumptions and keep in mind, I haven't been nine for a very long time. I honestly had no idea what to say next. I thought for a moment. I asked what I quickly realized was a hopelessly convoluted question. Dabi and Sean had built a mentor into the system. An experienced responsive teacher who supervises the session and offers advice. My mentor noticed that I was struggling, told me to simplify my question.
- [23:07]
Ji Woo
Remember, she's a third grader.
- [23:10]
Malcolm Gladwell
Dabe was trying to help me too. She suggested, why not just ask Jiwoo if 3/4 is bigger or smaller than 1/2?
- [23:18]
Ji Woo
So we are trying to help her to think about fraction in a more conceptual way.
- [23:24]
Malcolm Gladwell
This time, Ji Woo understood. She wrote back, 3/4 is larger than 1/2. I wrote back, is 3/6 of a cup bigger or smaller than 1/2? Ji Woo said, I'm confused. Oh no, I've confused Bonji Woo.
- [23:41]
Ji Woo
But it's good she's understanding. She's realizing her misconception. So she's getting confused.
- [23:47]
Malcolm Gladwell
She says, I'm confused. Three quarters is pretty close to one and adding three six would make it go over one. Oh, so she's got the answer. Yeah, but then she says, but there are six pieces out of ten which is less than one. So I don't get it.
- [24:01]
Ji Woo
So she's the point that, oh, this. I have something wrong here. That's a good sign.
- [24:07]
Malcolm Gladwell
She's getting there.
- [24:08]
Ji Woo
Yeah, she's getting there.
- [24:09]
Malcolm Gladwell
But I still have to get her to. She has to get the six pieces out of 10 out of her head. Yeah, I have no idea how to do that. What? And she thinks she's confused when she has actually she's figured out the answer.
- [24:24]
Ji Woo
Yeah, she did.
- [24:25]
Malcolm Gladwell
But she. So we have advanced. Even in my stumbling and bumbling, we've made some progress.
- [24:30]
Sean English
Made very notable progress. Yeah, absolutely.
- [24:37]
Malcolm Gladwell
My conversation with Ji Woo went on for some time and eventually I got there. Ji Woo found her way to the right answer. She said, I have more than one cup of flour. The mentor chimed in. I got a little emoji that made me feel good. And when it was over, I realized two things. The first was I needed more batting practice. Much more. And that batting practice was really, really easy to do because someone has gone to the trouble of building me my very own baseball diamond and given me a pitcher who would throw me baseballs all day long. My second thought was that I'd been thinking about AI all wrong. I have interpreted a lot of the talk about the promise of AI to be about replacing human expertise. I had actually thought when I first heard about Daabe's project that that's what Dabe and Sean were doing. Creating an AI to teach students, bypassing the teacher altogether. But if you did it that way, you would miss the magic of the classroom. Remember Eleanor Duckworth's quote, the goal of education is for students to have wonderful ideas and have a good time having them. I think we often focus on the first part of that formulation, the wonderful ideas, but neglect the second, the good time having them. Real learning is born in pleasure, in community, in playful discussion, in a group of kids coming together to solve a problem. And all of that magic only comes from human interaction, from a teacher who is skilled enough to inspire a class of nine year olds. We don't want AI assistants. We don't want to replace the teacher. We want AI assistants to help teachers turn themselves into even better teachers. Smart Talks with IBM is produced by Matt Romano, Amy Gaines McQuaid, Lucy Sullivan and Jake Harper were edited by Lacey Roberts, engineering by Nina Bird Lawrence mastering by Sarah Bruguer music by Grammascope. Special thanks to Tatiana Lieberman and Cassidy Meyer. Smart Talks with IBM is a production of Pushkin Industries and ruby studio at iHeartMedia. To find more Pushkin podcasts, listen on the iHeartRadio app, Apple Podcasts or wherever you get your podcasts. Malcolm I'm Malcolm Glapo. This is a paid advertisement from IBM. The conversations on this podcast don't necessarily represent IBM's positions, strategies or opinions.