← College Bound Mentor
College Bound Mentor
AI Did My Homework with Dr. Lyra Stein, Dr. Nathalie Moon, and Stephanie Silberstein
00:55:40
AI is taking over everything. College is no different. What is allowed - and should be allowed - when using AI in college classes? In this episode, we welcome on special guests Dr. Lyra Stein of Rutgers University, Dr. Nathalie Moon of the University...
Loading summary
Transcript138 lines
- [00:06]
A
At this point, I assume they're all using it for everything.
- [00:20]
B
Hey CBMers, welcome back to College Valmentor where we help you survive the college application process and beyond, where your co.
- [00:26]
C
Hosts Lisa and Abby.
- [00:29]
B
And we're missing Stephanie who is at the French Open right now, so we'll get her next week. On today's episode we're going to be discussing AI and its use in college. And the impetus for this topic was that one of our essay specialists, who is also a former English teacher and now a parent of teenagers or tweens, wanted to get a better understanding of what is actually allowed when using AI. It seems like it's all very confusing both to students and to parents in terms of what's there. And so I thought that a great topic for an episode. So I decided to reach out to some of my clients that I knew worked at Rutgers and so that's where I found Dr. Lyra Stein and then I reached out to my son in law and I found Dr. Natalie Moon at University of Toronto. And then Stephanie's actually someone that I know as a client so it all worked out. So I'm really happy to have all of these wonderful people here. So just to give you some background, Dr. Lyra Stein is an Associate Teaching professor in the Department of Psychology at Rutgers University in New Brunswick. She has received numerous accolades including the 2024 Hypotheses Social Learning Innovator Award in Social Sciences and the 2025 CO Intelligence Exploration Teaching Fellowship. Dr. Stein has played a leading role in faculty development through service on curriculum committees, mentoring programs and national teaching organizations, and she's an advocate for inclusive research informed pedagogy. Natalie Moon is an Assistant Professor, Teaching Stream in the Department of Statistical Sciences at the University of Toronto. She completed her undergraduate studies in Mathematics and statistics at and earned her master's and PhD in biostatistics from the University of Waterloo. Her recent work focuses on pedagogy with an emphasis on cultivating a supportive learning environment, empowering students to take an active role in their education, and designing opportunities for experiential learning and authentic assessment. And last but not least, Stephanie Silverstein has been an Adjunct professor of theater acting at Kean University, Fairleigh Dickinson University and the New Jersey City University. She is a Theater Director for Seton Hall University and has taught musical theater for the Paper Mill Playhouse for over a decade. She holds degrees from Cornell, a B.S. in Communications, Utah, Austin, an MFA in Acting and NYU an M.A. in Theater and English Education. She lives in New Jersey with her husband, four children and one year old puppy. So welcome everybody. We have like an amazing lineup here. So thank you guys for, for joining us.
- [02:47]
D
Thank you.
- [02:48]
B
Yeah, so just to get started, I mean, we have. There's been so much talk in the press, just talking to people how AI is taking over. It's rampant in the classroom. Students are using it to write their papers. Kids are using it really much for all of their assignments. So I just wanted to get a pulse to see like how much of that do you feel is true? How much are you seeing that and how has that shifted really in the last, in the last couple years? And I imagine for Natalie it's probably been around for longer because it's actually what you're teaching things.
- [03:15]
D
Right.
- [03:15]
B
Or using it. So who wants to kick us off on that?
- [03:19]
A
I frequently survey my students and I know they're using it very frequently and they are somewhat hesitant to admit it because they think that it's considered cheating. And I make it known from the first day of my courses that they can use it because they'll be using it after they graduate. They're going to have to know how to use it. And in my classes they have to work on critically evaluating the output. And I think that's what they're going to need after they graduate. They now see it as something, and at least in my experience, that they can use to cheat. And I tell them to be competitive. After you graduate, you're going to have to show your employer what you can do over and above AI. You're going to have to think about how accurate the output is and let your employer know that they need you to evaluate this and not just to copy and paste as you. As they may want to do in, in their classes.
- [04:38]
B
Interesting. How about you, Natalie? How do you go? How are you using it or students using it?
- [04:45]
D
Students are using it, there's no question. I came back to the classroom a year and four months ago, so January 2024, and that was my first term teaching. After it was released, I pulled my students as well. And yes, they are using it. They were using it. Interestingly, I asked two questions. One, I asked them what kind of tasks they used it for themselves. And I gave them a bunch of choices. Then I asked them what they thought their classmates were using it for. And for some tasks, no difference. Other tasks, definite difference. So I agree with what you were saying, Lyra, that I suspect there's a social desirability bias in how you report those things. That was a year ago. I think there's less of that now less taboo, less stigma about using it. My students are using it more than ever, including for things that they really don't need to use it. Things that, you know, Google, like regular old school Google or Wikipedia would be a better source. I teach statistics. So, you know, if you're looking for, you know, a formula or something, you really don't need to ask ChatGPT for the formula. You could just look it up in a static source and it's going to be quicker and equally or more accurate. But it's where they go first for so many questions, it's hard to convince them otherwise.
- [06:02]
B
That's so interesting that you say that because. Well, first of all, I mean, AI is now embedded into Google. So when you put it, that's the thing that comes up, right? That's the first thing. But I actually had my first client who asked me a question about how her son should focus on his summer. And you know, there were three choices. Write essays, do the SAT or do emt. And then she said, well, Chat G. He said he should focus on his, on his sat. And I was like, oh. And I had given her a different response and I said, well, I'm glad to see that I have a different take than CHAT GPT. So I just thought that was the first time a parent has said to me, oh, well, chatgpt said. Now, Stephanie, you teach in the humanities, so how do you see it as. As different and how do you see it as impacting your students?
- [06:49]
E
Yeah, I teach an acting class which is so focused on class participation and that hands on learning and less on writing or finding the right answer. It's more about. There's discussion and just getting up and executing the work and performing different monologues and scenes and things like that. There is a writing component to the class that I teach, but it's based on seeing an actual production and giving your own personal reflections. So it was interesting because I did get a couple of those papers that seemed maybe it was first paragraphs or concluding paragraphs, like certain phrases that felt very familiar and repetitive. But at the end of the day I question, what are the goals of the class? What am I trying to get out of this assignment? And where is AI helping them or hurting them in that learning process for them? Not to sound cliche, but it's, it's like, you know, you're just hurting yourselves when you use that as opposed to, you know, relying on yourself and your own ideas and things like that. But at the universities that, where I teach, there's a. And the class that I teach, because I don't teach majors often, it's an elective class. And so I get a wide variety of backgrounds that contribute to the class. So it's interesting to see the different levels of writing ability. And so sometimes I'm reading the papers and I'm craving AI, like, oh, please run this through some sort of program to help you flesh out those ideas. Or if you're not going to go to the writing center on campus, available to you and using those resources. And so sometimes they're not using AI when there's part of me that wishes they would. So, so it's interesting. It's. Yeah, it's for good or for evil, that kind of thing. How to. Trying to figure out where the line is.
- [08:57]
B
So, so it's interesting that you guys have all just, you know, obviously it's accepted, that's what it is, and that's what's going forward. But what do you, what do you think's being lost for students? You know, what do you think's being gained? Like, what do you, how do you think it's impacting critical thinking? The ability to actually do something and, and, and wrestle with words or ideas that you don't just have something spit out at you?
- [09:25]
E
Well, I, I think somebody said some, something earlier along the lines of they're using it when they don't need to. And so that fear that it will become a crutch, you know, where they don't trust their own instincts and their own thoughts and that's their go to is AI. Yeah, that's the danger.
- [09:40]
D
I think one thing that I wrestle with, and I know a lot of my colleagues in statistics do as well. I mean, we're figuring this out at the same time as they are. Maybe they're a little ahead of us, but, but we're, this is a new technology and we're not, we're certainly not more expert on, on these tools than the students are. But what we have that's different from the students is life experience and learning experience before AI. And we learned these things in the more traditional way in the before times until what, two years ago. So I'm sure things will change even more in the future as we'll have kids growing up with this. But even for our students, when I'm teaching them statistics, how do we get them to internalize these foundational skills in them without. Because right now they can just. They can go to AI for anything they're doing at home. And I have less of a problem with people Using AI once they have some core foundational knowledge. But our students don't have that for some areas, at least in university, you're learning loads of new areas you've never studied before. And I don't know how to do that. And that will become more and more of a problem, I think, across more disciplines as students, even from elementary school grow up with. With this. And it's hard to convince the student that it's important to do it on their own. I don't know that that's resonating with them.
- [11:09]
B
Interesting. Are you finding the same, Lyra, are you?
- [11:13]
A
I have really thought about this a lot in the last, what, two and a half years? So I created assignments where first they have to analyze what we discussed in class, and then they have to use ChatGPT to compare it to. And what they are realizing is ChatGPT isn't always correct. And if they take the output and they just copy it into whatever assignment, it doesn't mean it's correct. They automatically assume it is. And I'm hoping these assignments will show them that they really have to think about the output, that they have to grapple with it. They have to say, okay, my professor said one thing, and chatgpt is saying another. How do I critically think about this to determine the correct answer?
- [12:17]
C
That makes sense. So, actually, interestingly, the University of Michigan this past year for admissions to their honors program, they have an essay, and it's usually pretty challenging essay. And this year, their essay, what their. Their prompt was. This is an essay written by ChatGPT. Critique it.
- [12:34]
B
Well, they had to write the essay. They were told to write their essay using ChatGPT, and then they had to critique it. So it was an interesting exercise for them to see what they. What they would have done differently. So, yeah, that's a good point, Abby.
- [12:50]
C
So along the lines of what Lyra was just talking about, I'd love to hear from Natalie and Stephanie how you are using AI in the classroom. Like, how are you integrating into your lessons or your teaching or your preparation for your lessons?
- [13:05]
E
Well, as I'm sitting here, I'm. I'm reflecting on the class discussion component of my class and realize. And I've always assumed it was just maybe maturity or, you know, just. Just different obstacles. But I'm. I'm wondering now if maybe it's because they are so reliant on. On AI to formulate those thoughts. It. It can be like pulling teeth sometimes to have enriching discussions about, you know, theater, you know, and I try. I try to integrate things from pop culture and, you know, things that would appeal to them and get them talking. But yes, there's, there's a certain lack of confidence when it's just them out there speaking, you know, on their own for themselves and their own thoughts. Yeah, I'm in, in real time figuring out different ways to maybe have ChatGPT lead the discussion maybe, or give those points, prompts, you know, back to them as opposed to the other way around. And, and in terms of myself, like, I, yeah, I don't use it. I don't necessarily know the different ways, you know, it can be helpful. But in, in terms of gathering new resources, you know, like I, I tend to utilize the same scenes and monologues and clips from recordings of live theater or film or television that demonstrate certain concepts that I'm teaching. And it might be, it might be helpful to have ChatGPT help me or AI in general help me source out new material, you know, more updated material. I want to test that out a little bit.
- [14:48]
B
Fun.
- [14:48]
C
I like that we're problem solving on the fly.
- [14:51]
E
Yeah.
- [14:51]
A
Yeah.
- [14:52]
D
Thank you. Thank you.
- [14:53]
C
How about you, Natalie?
- [14:54]
D
I mean, I've experimented with it to generate ideas or examples or sometimes on assignments or exams even I have, you know, I want to come up with a description of a study and then ask them questions about that, and it's hard to come up with those on your own. So chatgpt is very good at that. Now, I typically almost never can I actually just use it directly. I have to modify it to make it match with the rest and fit my course and fit my voice and fit the learning objectives, but it's still helpful. I haven't used it a ton in the classroom, but the one place I have used it that I thought was very successful, very productive and fun as well, is in a fourth year statistical consulting class in preparing the students for their first consulting meeting. So the students were going to be working in groups with a real client. Not no money, but, but a person with a real problem that they would work with over a course of months. And you know, in the past we've had conversations about how those meetings go. You should ask them about their project and ask clarifying questions. And the client may not know a lot about statistics, so you need to sort of sometimes work around, like ask questions around the topic to try and figure out yourself what the objectives are and how the data was collected. And that's fine. And we had those conversations. But this year I wrote a prompt, a paragraph for them to copy into their LLM of choice. Basically, the LLM was playing the role of a client who didn't know much about statistics. And my students were the statistical consultants.
- [16:25]
E
And I.
- [16:26]
D
So the prompt sort of set that up for the LLM and then they started the conversation. They're like, hi, can you tell me about your project? And I was walking around as they were having these conversations. They got super into it, they loved it. They really cared about these fake projects. It was funny, but some of the questions they were asking were terrible, but it was all fake. So it was a lot easier to address that and take it up. I don't know if any of you have background in statistics, but one group in particular asked the LLM, how many degrees of freedom do you have? Which if you're talking to a non statistician, they're not going to be able to answer the question. And even if they answer the question, it's your job as a statistician to figure that out, not just believe them when they tell you a number. I mean, it was funny to raise that and have a conversation about it and immediately they realized the problem. But having that with that interaction with a fake person made it a lot more lighthearted and much easier to have those conversations before a real person became involved. So for things like that, I think it can be incredibly useful as a safe space to develop those skills that they, well, clearly need to develop and practice.
- [17:39]
C
I really agree with that. It's funny. My youngest son just graduated from college and he spent the last year interviewing for jobs. He was interviewing for some consulting jobs. And when I was in graduate school, I would practice with my friends for the case interviews for consultants. And he was doing with ChatGPT and he was recording himself and sending it to me so I could see how he was doing. And he, in his head, he was talking to a real person.
- [18:03]
B
I mean he was.
- [18:05]
C
And nervous as if it was a real person, but there was a lot less risk. I think it was really useful and, and it learned from him. It clearly was asking better questions as it went on. It was super interesting. I think that's, that's really interesting. So have any of you used AI for grading assignments? I'm just curious.
- [18:26]
B
Or how do you grade, or how do you like grade an assignment when you're incorporating? Like have you had to change your rubrics for grading things?
- [18:34]
A
As I said before, I added in that portion where you have to enter the prompts into ChatGPT and if I were just using ChatGPT, it would give me the output that they actually paste in there. What I found was, and I teach a course in psychopathology that it tends to recommend cognitive behavioral therapy for every disorder.
- [18:59]
E
And students would just copy that.
- [19:03]
A
Right. And part of their assignment was compare what we discussed in class to the output. And they just said, oh yeah, it was great. Not realizing that. We didn't say that cognitive behavioral therapy was, was the therapy of choice for all of these disorders. And when they had points deducted, I'm really hoping they learned that they can't rely on it as much as they used to. So I find it counterproductive to use it for grading if they are using it. And they have to distinguish the differences between course content and the output.
- [19:49]
B
It's interesting from a psychological perspective. I mean, I know it's so, so new, right? But do you, do you anticipate as a psychologist that there will be psychological impact if you feel like you're having this relationship with AI, which many people feel like they're having these conversations with real people?
- [20:09]
A
It's really interesting. Sometimes I host extra credit movies where we watch a movie and we apply psychological concep. And the movie I chose was the Matrix, which is about AI taking over. They were so candid. They would say things like, ChatGPT should get my degree because that's how I earned all of my grades. But they did say they thought it would be a good adjunct if they couldn't afford to see a real therapist. But then other students were saying it's not ethical because they haven't been trained as licensed clinicians have. So we had an interesting discussion about that and they mentioned that their friends, some of their friends would rather talk to ChatGPT than go out, meet people in person. Some people said they use it, they upload all of their text messages to psychoanalyze people and tell, you know, ask them how they should respond to others. So really they're using it for all of their big life decisions and they are counting on it to be correct and to give them the correct advice.
- [21:29]
B
That's scary.
- [21:33]
E
I was telling Lisa that I'm dabbling in stand up comedy. And one of my lines that I used was that I'm basically co parenting along with my husband with Alexa and Siri, and that on Sunday for Mother's Day, my, my teenagers are basically going to bring them breakfast in bed. You know, it's true. You know, I use, you know, it's, it's a form of AI, you know, all of these, you know, Alexa, Siri, that kind of thing as My whole house lights up. No, I'm kidding. They're not, they're not in this room with me. Their input or, you know, how do you address an issue or something? And I haven't used them for that. It's more of just like typing out a text, you know, voice to text. That's how I, I use those, those features. But yeah, if they, if they have answers to problem solving or issues, you know, how do you. What should you say in terms of empathetic listening and not lecturing but, you know, being there as a listener, you know, that, that kind of stuff. I can, I can imagine. Yeah. The. This generation is definitely going to use that as a resource more than actual human beings.
- [22:46]
B
So it sounds like all of you have just. Well, Stephanie, it's a little bit different in what you're doing, but have incorporated AI and just. It's like any other tool. What sorts of conversations are you having with your colleagues across. I don't know if you're talking across the institution or just within your departments and is there any blanket regulation or, you know, anything that they're. The comp. That the colleges are saying, you know, this is what we're going to use or this is what we're not going to do. This is okay. This is not okay. And I would imagine in the writing centers it's probably maybe less acceptable because you're being taught how to write. But what are you guys seeing?
- [23:26]
D
The University of Toronto is very large and a lot of things are sort of decentralized. One thing that, as I understand it, is a policy that is across the board for AI. So we're not allowed to use it for grading. We're not currently, we're not allowed to submit student work to third party software and we're not. Another thing we're not allowed to do is to run student work through third party software to try and detect if they used AI to generate it or not.
- [23:55]
B
You're not allowed to. Oh, interesting. Okay.
- [23:58]
D
For two reasons as I understand it. One is the concerns about intellectual property and copyright, all of that. Like, we certainly, if we're submitting student work to anything, even something like Turnitin, they need to know about it and there needs to be some understanding of what is happening with that material. And also those checkers, as I understand it, are not very good. The accuracy is not great on both sides. I think if you use a tool like that and are like, well, it passed, therefore it means that they didn't use it, they didn't use AI. That's not true. And on the flip side, if it says this was used, like, this is 100% AI generated, that doesn't mean that it is. So I don't understand. I don't know exactly how the tools work. I just know that they're not. They're not at a level that the University of Toronto is, as an institution, comfortable using. Now, I don't know what people are doing on the ground, but we're certainly not able to prosecute cases, if you will, on the basis of that, like we would be able to for something like Turnitin. If there's, you know, evidence that someone plagiarized, then, then that. That is evidence that can be used. But ChatGPT detectors are not allowed. But in terms of what we can do in our classes, the policy of the university is that each instructor can decide what is and isn't allowed within their course. And even within the course, there can be differences across assignments, which can, I imagine, be very confusing for students.
- [25:28]
B
Right. You brought up a good point. And I'd like to hear from Lyra and Stephanie about plagiarism, because it feels like it's become quite fussy, like, how does somebody know? It used to be pretty clear, like, you plagiarized, you. You lifted this off of something and you didn't cite it properly. But now if you're introducing what ChatGPT said and everyone's doing something similar, how do you know and how do you enforce if somebody's plagiarized? Have those rules changed?
- [25:53]
A
I tell them that they can use it for everything. I find that restricting it is counterproductive. I have some colleagues that are going back to the old days where they just do, you know, in class, right? I'm like, this is not going to help them once they graduate. I would rather simulate what they may encounter when they graduate. So I give them free rein to use it. I teach them how to use it. But I do say, you do need to cite it. And I show them how to cite it. And I say, the university takes plagiarism very seriously. So if you do use it and use it to generate ideas, if you use it to change the flow of your writing, I say, you don't need a citation. But if you use it to generate ideas, that is when you need to cite it. And I say, I'm not going to deduct points if you use it, just make sure that you cite it.
- [26:55]
E
I'm looking at the Kean University. There's a faculty hub where they give you resources and how to address or how to learn more about AI. And yes, as Natalie, I think you were saying before, it's just. It is very hard to prove definitively. I think there have been lawsuits involved, you know, just so. And a lot of times, you know, a professor will just kind of not want to get involved kind of thing. I think with plagiarism, yeah, it is a matter of giving credit to a human author. And so that's interesting about citing that AI was used in the making of this paragraph and that kind of thing. But as opposed to plagiarism, there start to be questions of almost the ethical lines and equity lines, you know, how, how different is. Is AI to a tutor or, you know, you know, how you get support and assistance in writing papers or, you know, doing any, any assignment, you know, where. Why is it okay for someone that can afford a tutor or has that time or, you know, those have the given circumstances to allow for. For that, as opposed to, well, my tutor is AI. And so it's very hard to start deciphering between, you know, why that's okay, but this isn't okay. And so I think at the end of the day, professors are just kind of back off and, you know, not get too involved in it.
- [28:31]
A
So, yeah, that is why I teach all of my students how to use it. They can all use it as a tutor and they all use the free version. So there's no inequity in terms of being able to hire a tutorial or not.
- [28:47]
C
This is a bit anecdotal, but my son was at USC University of Southern California, and last year there was a very strict AI policy. They were using anti chat GPT software. It was, I think, like Natalie said, wrong a lot of the time. There was a lot of friction, a lot of back and forth. It was very inefficient this year. I don't know what the official policy is, but the professor, you know, my son's professor said, we've got, we're not using it anymore. But I will say he had one philosophy professor that was really like very well loved and he begged the kids not to use it. He was just like, this is about your think. I mean, you took this class. This is literally about how you think. Please, please, please, please, please don't use it. And he was very well respected. You know, he was kind of like a pal. And I'll tell you what was super interesting was my son, in an attempt to, like, please, this guy really tried to be original in his writing by making connections to other Things they'd covered in class that would have been very difficult to prompt Jeff Chat GPT to do so. He almost developed like a new writing style that he was really psyched about. So yeah, I think this is all evolving and there will be definitely bad things that come out of it, but I think kind of like equity and also the kids trying to figure out how to make their material their own is kind of an interesting outcome also. I think it's very fluid right now.
- [30:10]
D
That's fantastic. But to me, that's because he really was motivated to do it. He really cared. And the relationship with that instructor, that professor, is what got him there. But that's what I'm trying to think about. Like when I think about these things, I'm trying to think about how I can align my students motivation in the direction that I think is productive for them. And it sounds like that that professor was able to do that for your son. It'd be interesting to see if his classmates all did that too.
- [30:42]
C
I have no idea.
- [30:43]
D
Not all of them. Perhaps a chunk, a bigger chunk than others, but there probably are still people who didn't have that kind of relationship and maybe didn't care to the same extent. Yeah, but it sounds like something clicked for your son in that context, and that's fantastic.
- [30:58]
C
And he was happy to use it in his economic, you know, like there were, like there were plenty in his statistics class, his econometrics class. He. He used it and then in his mind he had to understand it because the exams were in class, a lot of content.
- [31:13]
E
I mean, obviously time is a factor, but to do as much in class, you know, writing and discussion in class. So as opposed to, you know, homework. So it could change what homework looks like and working together in groups and that kind of thing. So it may change curriculums and just. Yeah. What the assignments look like, what the syllabus looks like in order to keep as much in class as possible. Which could be good, bad, it depends. It's unfortunate if you have to utilize time to let, you know, kids write independently without, you know, and monitor that and not just use the, the honor code or just trust there's, you know, the students. But yeah, it's possible.
- [32:04]
A
Yeah, I, I will say I redesign my entire course. I tell them at the beginning you won't have to memorize anything because out in the real world, when you're at your job, you won't have to memorize. I did because I didn't have access to as much information when I was in college. They take all of their exams online. But it's all application to what I said in class. And I warned that ChatGPT may tell you something different. Now I teach very large classes, anywhere from 250 to 400, and I always have an aspect of group discussion. And then I asked them to rate everyone in their group. And what's very interesting is that students will tell me, hey, this student just used chatgpt to do all of their work and it was wrong and we had to correct it. So I don't think this student should get credit. So I was amazed that they were willing to say that. And there is this attitude toward, you know, Professor Stein tells you everything you need to know. If you need to use ChatGPT, then something is wrong. And sometimes they do communicate that to their classmates.
- [33:25]
B
Interesting. And did you change the grading for that particular student? That was. I.
- [33:31]
A
Did I tell you, you know, if you're not participating, your classmates will tell me. And I look at the anonymous feedback and you know, you can come, you can try to argue against what they say, but I want to see that you are thinking originally and if you do use CHAT GPT, you are evaluating it in terms of what we discussed in class.
- [33:56]
B
Yeah, I think that's really good. Well, it sounds like you have really, it sounds like Lyra and Natalie and Stephanie are getting there but have really like, had to rethink. It's like a completely new way of approaching teaching. But I would imagine that not all professors are in the same, same process, you know, along the same journey in the same way that you are. And so are you finding that some of your colleagues are kicking and screaming they don't want to change. Are you finding it's based on how long they've been there? Is it? Or like, and how do you move everybody along?
- [34:29]
E
Well, I know I'm, I'm here as the, the humanities representative, I guess. And when you, when you look at it, the, the word human is in humanities and it's one of those things where, you know, you know it when you, when you see it or when you hear it, you know, it's just something you feel, you know, instinctually in your bones, that kind of thing. And so, and that goes for when you have discussions. And my, my classes are so hands on and, and based in participation. You know, they, I have so many students that are, you know, calculating their, their attendance and I'm just like, and, or this was excused. This was an excused absence versus unexcused. And I'm just like, I, I don't necessarily. Not that I don't care about what's going on or if it's an excused absence, but, you know, I, I need you physically in the room participating because that's, that's where the learning is going to take place. You know, this isn't something you, you know, you really do have to be here for it. It's not notes and things like that. And then in performances, yeah, you can run your lines. You know, they're all consumed about memorizing their lines. And sure, you can, you can work with an AI generated scene partner, but it is very different. Like when you're connected to the scene partner, that real human being, and you make that eye contact, all of those things, you start appreciating by taking that kind of class. And I was watching the studio. I don't know if anybody's watching that show on Apple tv.
- [36:02]
B
I saw a couple episodes.
- [36:03]
E
Yeah, there was. Did you see the episode where they, they had to do a rewrite of a movie and they were going to use AI to do it and they were at a conference or, you know, a sort of a theater situation, you know, where they were. There was an audience and people started, the screenwriters started standing up and saying, we hear you're using AI to rewrite the movie. And like, you know, they're, you know, basically going on strike. And you know that in terms of job replacement for humanities based fields, you know, where creativity, yeah, the, the source, you know, really should come from a human, you know, and those thoughts. But at the same time, I. My son was showing me how, you know, things work and he's like, you know, chatgpt write an episode of Seinfeld, you know, in. Or write a Shakespearean play in the style of Seinfeld. And they, you know, in three seconds it was done. And it was. It's really quite scary. But I hope that it's one of those visceral reactions you have where humanity, you know, can show through.
- [37:16]
B
No, I think that's a really good point. And I think for Lyra and Natalie, like, how do you get a feel for the student's authentic voice? I know you're teaching really big classes, Lyra. I don't know if you're teaching as big a classes, but, like, are there when you're reading everything, if. Do you feel like it's all the same, it melds into each other, or are you. There's some things that really stand out and you get, you feel like you hear the authentic voice of a student.
- [37:40]
D
A really good question. I teach big classes as well. My seminar class is small. That one I had 28 students this past year. But my other classes are much, much larger. So I don't know my students individual voice which make some things more challenging. One thing I think my night, my approach to some of my courses, my teaching this past term was a bit naive in everything's changing so quickly. I'm not beating myself up over it. But I will do some things differently next year. For example, in my seminar course, this was a smaller course where students had to do presentation and at the last presentation of the term, of the reflective presentation. And almost all of them came in with scripts and as they were reading their script, I felt like many of them used A.I. i don't know, but I wondered, which makes me sad because even if some of them didn't use it, I was still sitting there wondering that. I think in the future when I have presentations, for example, I don't think I'm going to allow presenter notes like you were reading or script. Some of them had a phone or an iPad and they were just reading. So maybe, maybe I'll go back to, you know, you can have a cue card or something, but not like full text. Similarly, in my big first year course, they do projects and I. There were a number of groups who, I wondered if they used AI because they were mentioning a lot of things that we didn't cover in the course. It's like, well, okay, maybe maybe you learned it in another course. Maybe you looked it up yourself or maybe you asked ChatGPT to, you know, propose a, you know, statistical analysis to answer a particular question. And it was really difficult to know which it was. And obviously the outcome would be very different in terms of what grade would be justified in the two contexts. So next year, again, I think I will be a lot more restrictive about you can only use things we've talked about in class, possibly with a disclaimer of if you want to do something that we didn't talk about in class, talk to me first because I don't want to restrict them. The students that really want to go beyond the course, the course we're only doing basic things. So if they want to push beyond, I don't want to stop them. But I got so there were so many submissions that I don't know, I had that suspicion and that doesn't feel good. I think I need to be more explicit in my instructions next year. And it sounds like you've, you've done that, Lyra, and that has helped.
- [40:11]
A
Yeah, I Just at this point, I assume they're all using it for everything.
- [40:16]
D
I thought I was assuming that, but I'm assuming it even more.
- [40:20]
A
What I always say is all of my assignments first have what we talk about in class and then you can use ChatGPT and compare it. And I have in person, I use a in class response system called Top Hat. I'll ask real time and I say, okay, I want you to answer this question. ChatGPT will not give you the correct answer. And still you have about a quarter of the class using it. And they learn early on that they don't get credit. They have to listen to what I say. So I don't go in saying, well, I want to detect if they're using it. I go in saying, they're all using it. I know they're all using it from a psychological standpoint.
- [41:10]
B
You just know human nature. They're all using it, right?
- [41:13]
A
And they know that I know, so they'll talk to me about things. And they say, yeah, I'm really afraid that I'm not going to have the occupational opportunities because of AI. And that's when I say, you have to work on it. You have to show, show your employer that you can critically analyze the output because that is what they're going to look for. And you know, I still have some of them, they just copy and paste and they don't get credit. But others are like, yeah, this is something I'm going to have to, you know, in an interview. This is why you should hire me.
- [41:53]
B
That's so interesting. So are you almost reframing what's necessary for employment and for careers? And are you talking to employers to get a sense of. Well, they are because I just read an article that said, you know, if AI can't do it, then, you know, we're, we're gonna, we're gonna fire everybody who, if AI can do the job. And, and there's also this expectation that people can do more because now they've got AI, so it's more productive. So how are you bringing that together to, to train the. Because are you teaching students? Not, you're not in the PhD program or you're teaching in the undergraduate level. Right. So you're not teaching at the side.
- [42:30]
A
Yeah, undergraduate.
- [42:31]
B
The undergraduate level. So how are you helping them think through that for careers and, and how they can use it? Because that seems to be, as you're saying, the most important thing is how do you use AI and add value?
- [42:42]
A
They are very anxious about that and they want to know what they need to do to be competitive, you know, to land the job of their dreams. And I tell them, yes, you can. I'm going to teach you how to use AI. I'm going to teach you how to enter the correct prompts. But you have to show me that you're thinking critically about the output. And I really think that's the future of what we need to teach students. Obviously, it's personal skills, group work, but it's also, you have this tech and I don't want to prohibit you from using it because it's very useful, but you have to show your ability above what the AI can do.
- [43:29]
B
Let's end with some myths and truths about AI. This is what we always like to have a little segment. So are there any myths or truths about AI that you want to share?
- [43:40]
A
I will say AI is very biased. As a psychologist, I have used it extensively just to see what I can do, and I find its interpretation of everything. I have the paid version of Chat GPT, so it knows me very well. All of its output is biased to make me feel good about myself. And it is. And occasionally I have to say, okay, be totally objective. And when it is, it will give me a completely different output. And I think students have to realize this, that they. They have to understand what to enter to get an objective output.
- [44:23]
B
And were you always an early adapter, Lyra? Like, were you always someone who.
- [44:27]
A
I am fascinated by technology and I, I was a neuroscientist beforehand, and in my mind I'm thinking, can this replicate the neural networks in humans? That's always going through my mind. I'm not to the point where it can, but we've had these type of discussions in classes and some of the students are really afraid. They say, you know, this is terrifying. I'm not sure what to do with this once I graduate, but for now it's something I can use to get through college.
- [44:59]
E
I don't know if this is a truth or a myth, but it's interesting that the three different majors or subject areas that each of us are talking about here, because they're exactly the areas that I always say are most important for getting that job. Because at the end of the day, people want to work with people. They want to be around and understand each other and can communicate and listen. And then in terms of statistics, I'm always saying, I don't think I've ever taken a statistics class. And I said, that is the one class I feel like we all need because the word narrative gets thrown around all the time. And that narrative can totally be manipulated based on statistics. I joke about, oh, my daughter won first place. And it was like, well, there was only one person in the category. And so understanding all those context and, you know, the, the narrative of, you know, how you present information combined with your personality and your interpersonal communication skills, like these are the things that are going to set you apart from the AI aspect, you know, all the technology. Because yeah, I don't, I don't want to collaborate and work with, you know, with a computer 24 7, even if.
- [46:24]
B
It, if it tells you everything you want to hear and flatters you.
- [46:27]
D
Yeah, I know that.
- [46:27]
E
Well, maybe, yeah, let me get back to you on that. Actually. Let me get back to you. But, but no, I mean, like, you know, I think about babysitters I've had. You know, I don't know how they do in school necessarily, but I know that like, oh, you know what, if I was running a company, I would hire you because, you know, you get things done and you're, you're efficient, you're, you're pleasant to be around with your person. But like all those things, skills. So I think all those things combined play a significant role in job and that all those career aspects and things like that.
- [47:01]
D
I guess one thing that I think a lot of people forget, including myself sometimes when you're using these tools, they're so good a lot of the time, even when they're wrong, they still give the illusion of being really good. But conversations, certainly, but they're not people. We talk to them, we interact with them as if they're people. We say please and thank you. It's like a real conversation. But these are probabilistic models. They're not deterministic. If you put the same prompt in on different days or on different accounts or whatever, you'll get different answers. I certainly don't understand how those models fully work in the background. And I'm sure that's true for most or all users. But yeah, this is not a tutor. I know we can use it like a tutor in some ways, but it's not a person. It doesn't, it's not sentient. It doesn't know if things are right or wrong. It doesn't care if things are right or wrong. It's a probabilistic model.
- [47:57]
E
Wait, they're listening. So be careful what you're saying.
- [48:00]
D
Right? It's repeating patterns of things that occur often. Like it's looking in its database for things that occur together. Words that occur Together, and that's when it's going to spit back at you. And if you don't understand what is going into the training set, which we don't know, then, well, that's going to have a massive influence on what you get out of it. And as things evolve, I mean, I've heard people on both sides like, are these tools going to get better or plateau? And, I mean, we don't know because I think there have been instances of, you know, it's feeding on itself, it's being trained on itself, so maybe the quality could go down. But I think some of these companies, these software, they're trying to filter out the bad inputs, if you will, from the training set. But how do they do that? I don't know. But they're not people and they're never going to be people. Even though we. It's so easy to forget as you're using them.
- [48:53]
A
Yep.
- [48:54]
C
The technologists I know don't like when I say this because they say it's not the right language. But ChatGPT will lie to you. ChatGPT will give you a link and you click on it and it's a dead link.
- [49:06]
D
It's hallucination.
- [49:08]
A
Yeah.
- [49:08]
C
And it will say, no, no, that. No, that link works. And you say, no, it doesn't.
- [49:13]
B
It gaslights you, it'll gaslight you.
- [49:16]
D
But it doesn't know anything. It doesn't know anything. Knowing is not the right verb to use about what it's doing. It's giving us output that is shockingly often accurate. And then, yes, of course, it's not always accurate, but it's remarkable how often it is accurate. It's. I agree that it's not always accurate. Of course I agree. But I'm not sure that's necessarily always the best strategy to convince our students not to use it. Because when it keeps getting better, like, I don't know if that falls flat sometimes, if we like the strawberry how many Rs in strawberry thing, like, I don't think it makes that mistake as much anymore. So if you use that as your example, they're like, well, that was a problem a few months ago, but it's better now, so we're good. And then you have to find a new example. Find a new example, but it'll fix.
- [50:01]
E
Those and it's one step ahead. Yeah, it is.
- [50:05]
B
It is.
- [50:06]
D
So I don't think that's the best strategy. I don't know. I don't have the answers, but I don't think that's. Yeah, it to, like, find those gotchas because they're.
- [50:15]
B
They're not there. That's so fascinating. It seems like it's very hard to be a professor right now in the sense that there's so much that you have to keep being on top of and it changes.
- [50:24]
E
And it's chicken or the egg. It's like, is it because of, you know, like, what is the cause of how these discussions are going or how these papers are written? You know, it's.
- [50:35]
D
And you got Covid as well. And that mess.
- [50:36]
E
Oh, yeah. That's not even.
- [50:39]
D
That's. But it's different.
- [50:42]
A
I have found life is much easier when I don't try to detect that they're using it and tell them not to use it and have in class exams, you know, I tell them that I want to simulate what you need to know in the workplace place, which is communication, critical thinking. So I say, I'm not going to monitor you if you use generative AI because that's not what I was trained to do. I'm trained to help you think critically, and we're going to use it to help you.
- [51:13]
E
I think there's a Broadway show now where the robots fall in love.
- [51:18]
B
Oh, yeah.
- [51:20]
E
Rave reviews. No, I. Yeah, I have.
- [51:23]
A
Maybe next year time.
- [51:24]
B
Is it that?
- [51:24]
E
Yes. Happy. Maybe Happy endings.
- [51:26]
B
Maybe Happy endings. Yes. I heard it was very good.
- [51:28]
A
I have to tell you, some of my students have admitted to wanting an AI significant other instead of a real person, and that's me. Like, what is going to happen in the future?
- [51:44]
B
Well, it's like that movie wasn't. It wasn't movie.
- [51:46]
E
Her.
- [51:48]
A
Yeah, yeah, yeah.
- [51:50]
E
I was thinking about that.
- [51:52]
D
Just wait till they have kids and they have to get up in the middle of the night, because if they get there, that would be cool. Oh, yeah, yeah. No, we're not there yet. But I've heard you said it must be challenging to be an educator. Yes. I think it's also incredibly challenging for the students you work with. Like navigating this and looking ahead, and it changes so fast. Whatever is true today, like, might not be true in a few months. I've also heard people talk about this time as a huge opportunity for us. Like, who knows where the future, the future of education, higher education is going to be? I don't know. But we're at this sort of transition point where if we engage with it and play a part, then we can maybe have an influence on the direction it goes in. But once it starts rolling more, then. Then maybe that'll be harder to Write.
- [52:47]
A
Harder to maneuver or to reverse.
- [52:49]
B
Yeah, yeah, No, I think that's very true.
- [52:51]
D
Not that we have the answers, but so. So that just sort of revs up the pressure. Right. Like, this is the critical time, and we better get it right and we don't know what we're doing.
- [52:59]
B
Yeah, no, I think that's so true.
- [53:01]
E
Yeah.
- [53:02]
D
But talking about it, I think, is a big part of dealing with it. Just both to manage individually and to figure out best strategies. The people that are putting their heads in. In the sand, as you say, I think they're working harder and they're making themselves miserable, and it's not productive. So it is. Yeah, let's not do that.
- [53:20]
A
How can I stop that from using it? And I'm like, that's underproductive.
- [53:25]
E
You can't talk about the elephant in the room. It's there.
- [53:29]
D
Yeah.
- [53:29]
E
It's here to stay. So.
- [53:30]
D
Yeah.
- [53:31]
B
All right. We just got to get better at using it. All right, well, thank you guys so much. Really interesting. I learned a ton, and I feel like I need to up my game on AI. I did use it, actually, to rewrite a college review, and it did a pretty good job. It did it pretty. But it was, you know, I had already written it, so it just kind of re edited it, so made it snappier. So that was good.
- [53:54]
D
Well, but we have to use it to an extent, because they're using it. The students are. So if we don't use it, then.
- [54:00]
B
Right.
- [54:01]
D
No idea what the capabilities. Capabilities are. Whether you choose to use the version you got or not, I think there's still some value in playing around with it and seeing, of course, what it did.
- [54:12]
B
Yeah, it is.
- [54:13]
A
I run all of my assignments through it, and I say, do you think this encourages critical thinking?
- [54:21]
E
Ooh.
- [54:22]
D
Oh, that's a good question.
- [54:24]
A
Yeah, it gives me some very valuable feedback.
- [54:27]
D
That's awesome.
- [54:28]
B
And do you use the paid version of Chat GPT?
- [54:31]
A
Yeah, I. I do for myself, but I had an extra credit assignment for personality psychology, and I said, you know, let your Chat GPT get to know you and at the end of the semester, see if it can assess your personality accurately. And I. Students seem to like it. Most of ChatGPT was correct and, you know, correctly assess their scores on different traits.
- [54:59]
B
Well, thank you, CBMers, for tuning in. Thank you, Natalie, Lyra, and Stephanie for such a fascinating conversation. It's always fun to talk to people from different areas and get different insights. We really learned a lot. So to catch more episodes of College Bell Mentor, make sure to Follow or subscribe on your favorite podcast platform and tell a fellow parent or student about the podcast. To learn more, visit collegebellmentor.com until next time you got.