← ABA Inside Track
Loading summary
Transcript127 lines
- [00:00]
A
Foreign.
- [00:14]
B
Hey, everybody. Welcome to ABA Inside Track, the podcast that's like reading in your car, but safer. I'm your host, Robert Perry Crews, and with me, as always, are my fabulous co hosts.
- [00:24]
C
Hello, Rob. It's me, Diana Perry Crews.
- [00:26]
D
Hi, it's me, Jackie McDonald.
- [00:30]
B
Oh, boy. Everybody, I'm very excited because today on the podcast we're going to talk all about comic book art. We've been talking. We got a special guest in to talk about comic book art with it. Wait, I'm getting the. No, that's not what we're talking about on our podcast. I mean, it's a podcast about behavior analysis and behavior analytic research where every week we pick a topic and discuss relevant research articles. I assumed we had some relevant research articles about comic book art. I was told by our very special guest about some great comic book art, but apparently I read it wrong. We're actually going to talk about a different subject. We're going to be talking all about instructional design for business training, which is like comic books, because you could do a comic book training maybe, but here to sort out, you know, instructional design, business training, maybe some comic book art. As I was led to believe from our pre show banter, we are very fortunate to have Dr. Doug Johnson here to discuss this very important kind of obm but related to everything we do topic. Doug, thank you so much for being on the show.
- [01:28]
A
Oh, thank you for letting me join you.
- [01:30]
B
So we, yes, like I said, we had some fun comic book talk, but it is time to be all business. Business. Organizational business, that is. So Doug was kind enough to send us some topics related to training and specifically the creation of training. Because as we all know, if you make a great canva presentation, as the ads will tell you, while it looks great on the commercial, it's. It doesn't actually teach meaningful skills without a lot of other thought and preparation. So we're going to be discussing sort of what needs to go into that preparation. And it's, you know, it's not AI. I don't think Doug is it AI. He's just type it into ChatGPT.
- [02:08]
A
No, not from my end.
- [02:09]
B
Okay, so no. Okay, so this episode's going to be longer than I thought already. Okay, good. So we're going to be talking about two of Doug's work, some done solo, some with some co authors. And then we'll kind of get into all the other examples that go into creating these structures for training. So, Diana, let's start just by saying what will we be discussing or what are kind of Our works that people could, could look at and follow along with.
- [02:33]
C
All right, so we have two different articles to review today, or this one is actually a book chapter, in fact, and the book chapter is titled the Foundations of Behavior Based Instructional Design within Business by Johnson and published in Applied Behavior Science in Organizations with the editors Human, Farr, Freiling and Alevicius. And that was in 2021. And then we'll follow that up with the Advancement of Training Within Business Using Behavior Based Instructional Design by Johnson, Lee, McAlphin and Lasky. And that was in Journal of Organizational Behavior Management 2024.
- [03:13]
B
Okay, so Doug, why don't we just kind of start by learning a little bit more about you, how you got into obm and then specifically writing about training, you know, the designs of training.
- [03:23]
A
Yeah, so I'll connect some of the dots from the gain of your thing. My first career choice was to be a comic book artist. And I didn't have the skill set to do it. So I, I, I aimlessly worked jobs I didn't like. And I eventually decided I should go to school because I'm like, I, I don't like working, flipping burgers or doing maintenance things. And I took psychology just because I wanted to understand people better, but I needed to keep working those jobs. So I worked between 40, 60 hours every week while going to school. And I love the content that I was getting in school, especially love the rat lab where we learned to train the rats and love doing fun things like training rats to classify music genres, training them to disco dance, train them just to pick up marbles and play basketball with them. And I also found that I enjoyed training students to train the rats, so sort of train the trainer. And I got really passionate about that. And so every day I went to the lab, we talked very precisely about being very mindful of the rat, giving frequent feedback through usually food pellets or audio clicker sounds. And we had very precise procedures, all of that. And then I would go to work and all the principles, all the concepts, all the best practices I was learning was being violated every single time I went to work. And it was very frustrated where I kind of felt like we treated the rats better than we treated the employees, where we were very mindful of the rats well being. And the employees were like, just do your job. You're supposed to. Why don't you just do it the way you're supposed to do it? And to train employees, we'd say, you know, just watch this video or just kind of follow that person around and Just do your best, right?
- [05:22]
C
Yeah, it's your job. Do your job.
- [05:24]
A
It's your job. What do you want? Like, do you want a cookie? Am I your babysitter? Just do it. And none of this would fly with the rats. Like, if we just said, hey, rat, just press the lever. It's your job. Obviously, that's not going to work at all. And that disconnect between the best practices and the precision of the lab and the complete lack of precision at work really struck me, especially because I worked for one of the largest corporations in the world. And I'm like, this company's successful and is treating its employees bad. And I watched as motivated, young new hires came in and became cynical, apathetic senior folks very quickly. And so I was like, okay, we got to apply this behavioral stuff to the workplace. So I was going to invent an entire field. And then I found out somebody had already invented it for me, which took a lot of pressure off of me when I found the book Bringing out the Best in People by Aubrey Daniels. And that just told me exactly what I needed to do. So I wanted to study employee motivation. And so I went to grad school to study OBM and employee motivation. And within my first couple of years of grad school, I took a class on instructional design offered by a visiting adjunct. And that was cool. And it was stuff that I had never thought about before. And that person also offered to hire me to work on contracts. So then I was working once again while going to school once again. But I was building computer based instruction, building training modules, and as a result of that, my program really didn't have stuff on instructional design or computer based instruction. So I just started tracking this stuff down as best I could and really tracing it back and got into, like, Skinner's original words about computer based instruction and programmed instruction and the works of Susan Markle and Philip Tiemann, and just a mixture of privately taught in consulting environments and reading up as best I can and finding best practices through some of the greats, and borrowing from school psych, borrowing from obm, borrowing from anywhere I could find useful stuff.
- [07:49]
B
Oh, okay, so you. You have. You lived the experience that led to publications like this and the work we're talking about today? That's excellent. I always love when there's that there's got to be a better way. Everyone should have that in their origin story. Like, there's got to be a better way. And here comes the research, here comes the training, here comes the practice. Very cool.
- [08:09]
C
Very cool.
- [08:11]
B
So, Doug, I loved Your, I mean, both pieces. But that the, your, your book chapter specifically on instructional design, it was one of those. I kind of feel this way. I think every time we do an OBM type episode of like, this just makes perfect sense. Like, I'm so glad someone just described. I get it now.
- [08:28]
A
The.
- [08:28]
B
I've been trying to say this for years and hearing it. So kudos, kudos on that. It was really, really great. I also love, and you mentioned sort of the history of instructional design. It does feel like one of those stories that, you know, Skinner going to what was his daughter's like fourth grade math day. And it sounds like one of those kind of apocryphal, like that didn't really happen. Right. But that did really happen, didn't. He did go and he was really frustrated. I mean, he's not that far, you know, in, in history that, that, you know, it's a legend. He did go and he was really disappointed with the state of education. But I, I'd love to hear a little bit more about that and sort of how that led to instructional design as well as how Susan Markle came into the story, because that was not a name I was familiar with.
- [09:10]
A
Yeah. So I always felt like Susan Markle was one of our field's unsung heroes and of the people who don't get the attention they should. Like, she would be towards the top of my list of people that more people should be understanding and looking into. Now, the story references. I always found it interesting that at the beginning of his career, Skinner was something of a purist that let's not get involved in applied concerns. You know, let's not rush it, let's stay in the lab, let's do our research, let's publish books with 5,000 graphs on them and keep building the data again and again and eventually we'll find some application of this. But you got to do the research, you got to do the hard work up front. And I always like the fact that he just went to a Father's Day school event and he sat there. And maybe part of the reason it resonates with me is because it paralleled my own development of. I went to a lab, I saw what was possible, and then I went to work and I saw the discrepancy and it bothered me. He worked in the lab, he saw what was possible, and then he went to the elementary school classroom and saw his daughter and other students being really, from his view, miseducated. And to his credit, he didn't blame the teacher or Anything like that. She said, he said she did the best she could. She's facing dozens of students and there's no way she could manage all of those contingencies. And she said, I mean, in the, the lab, no researcher is trying to teach 20, 30 rats simultaneously. And yet that's what we're doing to our children here. And so he, he had a change in priorities where he said, okay, I wanted to wait a little bit longer. But even this immature science of behavior that we're developing has best practices that we could implement right here and right now. So he wanted to revolutionize education. And one of the things that really struck him was, well, in the lab we automated a lot of things. We don't hand deliver every single food pellet. We don't hand record every lever press. We have mechanisms that record and deliver these things for us. Well, automation helped in the lab. Maybe it would help in school. That wasn't to say it should all be computers and robots teaching or anything like that. It was like, let's take some of the grunt work off of the teacher. The teacher can dedicate time to other more qualitatively important things. Rather than dispensing things or giving correcting homework. We could automate some of that. So he built what I think of as the, some of the earliest computer based instruction programs. Now, personal computers weren't around. So he built big mechanisms where you turn cranks and knobs and it showed little pieces of text and you hand write in an answer and then you turn another crank and you get feedback on whether you were right or wrong. And he pilot tested these things and they worked fabulously. Or at least he designed it. The person who actually did the pilot testing in the classroom was Susan Margol. But he designed it, he came up with these ideas and he found out that trying to revolutionize an entire system wasn't as easy as just having a better solution.
- [12:40]
C
No.
- [12:41]
A
And so one of behavior analysis first attempts to change the world was to change our education settings. And we had a bit of a setback on that. And so he's just sort of went on to other priorities. But other people who were fascinated about it, they tried doing, you know, program instruction textbooks which aren't as good as the compute the programs, but maybe could be an improvement. And then there was also a number of researchers who decided to either pioneer just the design of instruction or, or pioneer how we train employees. So some of those people went in to organizational, well, went on to create organizational behavior management. And then people like Susan Markle Went on to create instructional design schools of thought there. And I really, as somebody who was then doing computer based instruction when I was in grad school, I really like the fact that that connected to the legacy of the field and the field's first attempts. And so early in my career I kind of wanted to try to honor and expand upon that legacy.
- [13:51]
B
One of the things that I was sort of shocked by is that some of the designs that, you know, that you were writing about and describing in business, in terms of the training, instructional training, it feels like business was much more willing to take the chance to make some of these changes. I know they don't universally do it. Certainly your own job history doesn't say that they were using these techniques, but it seemed like business was sort of more interested in putting in the work, hiring people to do that, whereas schools, it still feels like even though these technologies have been around since the 50s, I know like some of the program instruction kind of the late, you know what that would have been look like this mid-60s direct instruction seemed kind of a similar offshoot in the late 60s. Sort of have just been like, so on the fringe. And only when like kids started getting iPads for education did they. Did schools kind of start saying, maybe we'll give this another look. What is the difference between a business training and a school? Is it just the fact that the business loses money by training poorly in the school? Kind of doesn't know if they did a bad job until, you know, the student is in a business that is doing a bad job training them that they did a bad job. Like, is it just that delay? Is it the money?
- [14:59]
A
I think there's multiple things. I think part of it is the clear connection to profit or loss that is present in the business world that's not so present in the educational world. I also think education probably has a history of people promising this new technology will change everything. And then we find out it doesn't change everything. I mean, you go back, they were promising radio would change how we educate our students. And then they are promising TV would change and revolutionize, and then they're promising, like videotapes and laserdisc would revolutionize things. And then they were promising, you know, computers in the classroom, and then they were promising iPads and lately they're promising AI and they keep making promises that this new technology, this will be the thing that changes everything.
- [15:51]
C
Yeah. I just want to say I remember when my school got a laserdisc player and it was so exciting. And they're huge. They're like bigger than a record. It didn't really change anything, but it was exciting.
- [16:08]
B
You could finally watch Blade Runner from school or whatever. That was the big laser disc, wasn't it? Yeah.
- [16:12]
A
And none of the technologies were ever designed with people in mind. They were always tech forward, learner backwards. And so that's why they didn't make the change because they never consider the organism as part of the revolution here. So I, I suspect there's a little bit of burnout there. I also think schools of education philosophically have been opposed to a lot of behavioral based concepts. So you got a lot of things running in opposition. And just making any change in a school system is hard, whether you're a behaviorist or not a behaviorist. School systems tend to be very resistant to change. Whereas in business settings they're always willing to try out a fad. Now the trick is to get them to do it for more than a few days or a few months there, but they're willing to try anything out. And I feel like schools have been business don't have philosophical guidance there. Well, does it make the company money or not? Okay, if it does, we're willing to give it a try. So that I think that's why those early pioneers found an easier path in business and industry than they found in education.
- [17:29]
B
Kind of thinking of that history, kind of the business history. When we think about training, you know, I know you, you mentioned so many. It's probably a lot of us have gone through at business those, those terrible trainings. You watch the video, do it, it's your job kind of situations. Was that always the way like how much, how much have you looked into sort of like the history of business training? Because it feels like there must have been a time when business was. You actually had to do stuff and it was you and your apprentice, so you actually had to show them what to do. And maybe there was like a more emphasis on training. Am I sort of just imagining based on movies sort of how Johnny Tremaine. Yeah, how. Yeah, Johnny Tremaine, he had to learn how to silversmith or whatever he did.
- [18:09]
C
I mean we had that on laserdisc.
- [18:14]
A
I think we're still very careful in business and industry when the stakes get really, really high. So the way we train surgeons, we tend to be very precise about that and we do a lot of practice and we get people to a high degree of mastery because the consequence of messing that up. Well, that's a, that's lives and a lot of lawsuits. And same with like training a pilot we're very, very careful with that because we know if planes start crashing, everyone starts canceling their trips and then you make no money.
- [18:49]
D
I mean, fair, yeah, right, fair.
- [18:53]
A
But the rest of business and industry, if the stakes aren't that high, they don't think that through. And there's also, I think it's just in our culture in general, you can't get much better with people. This attitude now I agree with that, but there's an attitude of, you know, people are just going to screw up there. Some people are just going to do what they want to do and there's nothing we can do about that. And you will even see that in safety industries where they're like, you know, some people make mistakes and accidents happen. There's nothing you could do. The entire branch of behavior based safety says, yes, there is very much something we can do here, but that tends to get neglected.
- [19:36]
B
So depending on the stakes, you're going to get a better training. The lower the stakes, the less, less good a training. However, you could make a good training for almost any skill. It sounds like if you're following set principles.
- [19:49]
A
I believe so. I believe anything can be trained well and it can be structured and usually better than the status quo. And I also further believe that every job could be made to feel meaningful. And so both looking at the performance management and the training side of things, we can do so much better with our employees. I mean, if I can teach a rat to disco dance and declassified music genres, I can teach an employee who's much smarter than your average rat to do much better than what we do with them. I think businesses don't realize the potential that's there and often they're dissatisfied with their training, but they don't know what to do. I've had many, many conversations with trainers and managers over the years where they will be very quick to point out, yeah, our training is probably not the best. We don't know what to do. I told them what to do and then he didn't do it. And I don't know what else to do with that.
- [20:52]
B
Our laserdisc players in the shop. So that option's out. You know, what are we.
- [20:56]
A
Yeah.
- [20:58]
B
All right, well, so let's, I'd love to take a step back and talk a little bit about the sort of design that's going to go into training. So specifically the behavior based instructional design. I know in your chapter and then again in, in the article that you co wrote, you do a really nice job of laying out and like, like, like I kind of already said just like, oh, these are the connections, these are the relations that are going to go into teaching these skills. I never thought about them because I think one of the things I loved about these chapters is how far it goes beyond sort of what we often talk about on the show is training. It's, you know, what you can use behavior skills training. Right. You're going to show, you're going to, you know, give the write up, you're going to model the scale, you're going to train and give feedback and it almost always works. And then you come into the issues of, yeah, but you don't have enough trainers. You could try pyramidal training, you could add it to. But it also doesn't take into account all the other ways one could do a training and you know, doesn't take into account, I think some of the like language relationships that you want to have into a training. And it also doesn't necessarily teach anyone to do anything other than perform skills in very specific situations. And a lot of times in a training you need someone to learn how to do not just a skill multiple times, but all the skills all the times, plus new skills you didn't even know you needed to train. So I know Markle and Tiaman had their categories of learning relationships and I, I don't, you know, I, we don't necessarily need to go through like everyone in exquisite detail, but I'd love to generally hear about those categories, those, a little bit of subcategories and then what they actually mean when it comes to developing a training. Like, like what would that be in real, in real life without the, without the terminology. Yeah.
- [22:39]
A
So coming at it very broadly, they really looked at, first off, it's good to establish can the behavior occur or not. And then we can start looking at, okay, do we want one quick instance of behavior? Do we want to build a long behavior chain? Do we want the person to rotate between various behavior changed and single units of behavior? But first off, can we just get performance there? Sometimes with verbally sophisticated adults we can skip past all of that or that's really fast and easy there. Sometimes if it's a very unfamiliar skill, you may be there like trying to pronounce something that you've never heard before and you don't know quite how to say it, or execute a physical maneuver and you've never done anything close to that. We may start, can you just do the behavior? After we've established that then we might want to care about do they know when to actually execute the behavior. And once again, are we doing just a simple single response? Are we doing a chain? Are we doing a mixture of various chains and various single responses? But we want to establish the correct discriminative stimuli and the correct S deltas as well sds and S deltas. Sometimes we have very consistent discriminative stimuli, which makes our training job go much easier. Sometimes we do not. And we're into the realm of concepts where we're dealing with large stimulus classes. And what we train may not be what they encounter actually in the real situation. And we can't program every possibility they encounter because the stimulus class is just too large there. So we need to train them how to respond in a very general way for this behavior to extend across different settings beyond just what we trained directly there. Whether it's a single unit where we train just a concept, or we start training a principle, which is how do concepts relate to one another? Or we get really complicated and we get into generative repertoires where they are going between these linked units and these single units under conditions they've never faced before. And then underlying all of that operant stuff, we've got emotional relations that typically respondent stuff of any educational context can elicit avoidance, it can elicit approach behavior, can make people feel good, feel bad. We need to be mindful of that's happening. At the same time, we're doing all of that explicit stuff. And I think the implications of that is it says a lot about how do you want to assess your training and how do you want to progress with your training? Because if we just need them to do the behavior and we don't care under what context they do it, that's a different form of evaluation. If we have a very specific context, it's very rigid, and we have a very rigid response. Well, that's a different type of evaluation and training. And if we got this less rigid, and we need a very flexible, again, a different assessment, a different type of training. And I find that that doesn't get talked about or thought about enough. And to me, those are upfront questions to figure out. How do I want to design the instruction to approach this.
- [26:26]
D
Hi, do you want to be a bcba, also known as a board Certified Behavior Analyst?
- [26:31]
C
Sure, we all do.
- [26:33]
D
Now, you can come to Regis College in Weston, Massachusetts to get your graduate degree just minutes outside of Boston.
- [26:40]
C
Choose from any one of these Masters.
- [26:42]
D
Of Science in Applied Behavior Analysis, Master.
- [26:45]
C
Of Science in Special Education, dual degree.
- [26:47]
D
In Special Ed and aba, or be eligible for your postmaster's certificate, you can.
- [26:53]
C
Complete your degree and be ready to.
- [26:54]
D
Sit for the exam in two years. And our 2022 graduates had a 92% pass rate on the BACB exam.
- [27:02]
C
Come enjoy approved fieldwork placements, ethics, mini handbooks, PhD levels, professors, small class sizes, and a service trip to Iceland if interested.
- [27:11]
D
And don't forget, our program is accredited by the association of Behavior Analysis International or ABAI, as a Tier 1 master's degree program.
- [27:19]
C
Don't delay. Supplies are limited. Learn more at regiscollege.edu.
- [27:24]
D
Again, that's www.regiscollege.edu regiscollege.edu. one more time, www.regiscollege.edu.
- [27:36]
C
See you there.
- [27:37]
B
Bye. Hey everyone, Sorry to pause the episode on you, but I just have to take this moment to remind our listeners that Aba InsideTrack is ACE and Kwava approved. By listening, you're able to earn one learning CE. All you need to do is, you know, finish listening, then go to our website, abainsidetrack.com or click the link in your podcast player. That'll take you to our website where you can put in some information, including two secret code words. And these code words are from Dr. Johnson and the first one is Spider Man. S P I D E R M A N. He's a spider and a man. You're probably imagining Peter Parker, Spider Man. But I suppose it could also be like a arachnid that like has malformed into a giant man, which would be a creepy monster. Which depending on when you're listening to the ul, you're not listening. This is not spooky season. If you're listening to this next October, perhaps that's what you want to imagine. Whatever. Whatever works for you. Just remember it's Spider Man. Just one word. All right, that's back to the show. So when it comes to the the instructional design, you sort of have to ask a lot of questions ahead of time so that you know sort of which components in that kind of. And the chapters, both the chapter and the article had like a really great matrix. I love that with emotional responding. Sort of it's nestled in the back because you're great training. If everyone hates you as an instructor, your great training is not going to go very far because everyone's going to call out sick or, you know, spend all the time in the bathroom. Right. So, you know, having that mixture and you mentioned certainly you're not going to have to go through all of it, but you also sort of went over just having like a good, you know, the good relationship the good language design, the good, like where are the flashcards coming in or where are we going to do the repetition or the in situ training? You also sort of mentioned a lot of other other steps that need to go into developing the training. Things like, you know, do you need an expert to tell you what it is? I was also kind of curious about that. You know, how often maybe do you think it's done or would you recommend it being done? Things like measuring where you are versus where the experts are and trying to close the gap with instruction. I assume that's going to be a huge component of. Oh, this is a quick, just associations training we need to do versus a. We need. The generative piece is really where we got to put all our training eggs because that's what we're going to need to be expert. What are some of those other just like parts that go into developing a training that have nothing to do with the actual design of the training?
- [30:03]
A
Yeah, I mean, I look at training very broadly here and maybe that part of it is before something like BST became very popular, I was studying all these different schools of thought. So I was studying, you know, precision teaching, direct instruction, performance based instruction. I'm blanking on the name of it. Personalized system of psi. And just learning from all of those and taking some general lessons and trying to make a cohesive training model, taking all of that stuff there. And so by time something like BST came along, I was like, okay, true, and good practice. But there's so much being left out of that equation there. So there's that Susan Markle design part. There's also knowing, okay, we need to give instruction. How much instruction do we give? When do we stop? We know as a general truism, being more active is better than passive. But you got to give them instruction at some point. You can't just say, behave and, you know, consecrate it. You gotta give the instruction or the skill there. How do you sequence it? So I think it's pretty intuitive that you want to start with simple stuff and build to a complex. But what does that look like in practice there? And that that's often neglected. We know that feedback matters, but how do you handle a corrective feedback? How do you provide that affirmative that reinforcement there? And I find like, I have to give a lot of guidance on that when I work with trainers. And I don't see that appearing in the literature like it should. I've seen it in bits. Like, I remember direct instruction really talking about you need to vary the Reinforcement, So, you know, and they would script it out. So it differs each time. Well, I also trained that. I'm like, okay, you don't want to sound like a robot, so don't say good job every single time they get it right. Vary it up. If that doesn't happen to you naturally, vary it up. Pay attention to how they react when you say things. So if you say a bit of praise and they recoil in horror to the way you say it, that's not a good bit of praise. So if you go and you say, oh, that's such a good job, you're the smartest adult I've ever seen, they're not going to like that, right? So you need to, like, maybe dial that back. Whereas for a kid, maybe that would be great. And so you need to see how they respond to that. And we also need to build in not only feedback for our learners, but feedback for our trainers. How do you know that your trainees making good progress? How do you know that they understand you? Like, one of the biggest violations is I see, I see is that trainers may talk to a trainee for an hour or two, and the trainee's nodding their head and they're saying, okay, well, I did a great job training. And then tomorrow they find out, hey, they don't remember a single thing I said, but they nodded their head for an hour straight. What's wrong with them? And they get really frustrated. And so you have to. Then you start looking at the emotional elements. The trainer's frustrated because they put in a lot of time and effort and they don't get any value. The trainee is frustrated because they put in a lot of time and effort and they get told that they don't understand and they're failing. Well, that's that emotional relations once again, because we didn't build in the consequences the way we needed to build in the consequences, or even knowing, hey, when do you stop your training? And I find that that's one of our biggest opportunities, too. One of the great things I took from precision teaching is this fluency measure. We stop at accuracy alone. And I point out again and again, and intuitively, people understand that when you ask him, okay, he got it right, but he hesitated for 30 seconds. And when he said the correct answer, I swear I heard a question mark at the end of his statement. Do you feel comfortable with that even though he said it correctly? And the obvious answer is no, but they don't know what else to do. Because the sad truth is nobody ever trains people on how to train we go to elementary school, we don't get a class on training. You go to college, you probably don't get a class on training. Even if you go to college of education, they don't train you on how to teach. You get a samplers tray of various subjects, but not how to really manage your classroom. And if you go into business and industry, they don't tell you how to train people. They just look at, well, you're good at your job, so I assume you're good at training others. And so that's where we really got to structure that up. Or even how do you handle one on one training? Okay, how do you handle two on one training or ten on one training? What do you do so that doesn't fall apart and end up a replication of what Skinner saw in that classroom all those years ago of one teacher failing to keep up with 30 some students. So I look at all of these as things that we need to build together. And then we also need to train leadership to support all of this because it. The trainer is not going. The trainer's not a perpetual motion machine. They're not going to train best practices forever in the absence of any reinforcement. And the trainee is not going to learn forever in the absence of any reinforcements. So we have to build an entire system of reinforcement throughout the company there. I don't know if I've drifted away from the original question.
- [35:47]
B
No, I mean the question was like, what else goes into it? Because it's not as simple as, like, look, I used it these designs to like build my response and build my generative repertoire. And now the training is complete. Because you're very right. There's so many other things that have to go. Do you need all that training? When do you stop that training? Yeah, hey, when they're done, who cares if nobody ever says good job or it's not tied to their performance fit or the environment keeps changing the machines they're supposed to use and they didn't bother with the gender. There's so, yeah, all those factors that are going to be shifting, they're just. Even if they get taken into account, I assume with business you need something new all the time. So I mean, should you always be training the generative repertoire for certain fields.
- [36:29]
A
Or even to give you a simple example of that, what do you do when your training gets frustrated? Trainers have no clue how to handle that and I ask them. So I do role play with a lot of my trainers and I'll start acting like somebody who keeps getting it wrong and gets More and more frustrated, and they just don't know how to handle that. And usually how they handle that is they say, okay, let's just take a break. And I asked them, hey, all things being equal are breaks like nice good things? And they go, yeah, okay, so this guy just made a bunch of mistakes and got frustrated at you and you gave them a nice thing. And even if they've never heard of reinforcement or have any idea of behavior analysis, they immediately recognize there's a problem with that, but they don't know what else to do. And so I talk to them about, well, the anecdote, the antidote for frustration is success. And so I start talking about, let's go back a few steps to where they were successful and have them practice. What I'm thinking is, I'm thinking about things like behavioral momentum and errorless learning. What I'm saying is let's bring them back to a place of success, rebuild their confidence, get them moving forward again. So let's have, instead of failure, failure, failure. Let's have success, success, success. Try the hard thing. And even if they failed, at least they had a bunch of successes and now keep them level up. That way you don't want to give them a break because you're feeling uncomfortable. And they don't need a break because they're feeling uncomfortable. And always end on a success. And they get that basic idea. And what's always pleased me is how behavioral concepts are well received. I mean, I can't use terms like behavioral momentum and whatnot, but like, they love those basic ideas because this is the first time in their lives they've been giving to, they've been given tools to actually do something and they've just never had the tools.
- [38:29]
B
I, I appreciate discussing the frustration aspect so like that emotional, that emotional level of, of the teaching, because I do, I do a lot of training related to like the sort of the de. Escalation techniques in, in the district I work in. And invariably there's a class where somebody just can't do one of the movements. And usually, you know, I've done enough of them that I kind of like know, okay, what is actually the task analysis here? I'm going to go back, I'll like forward chain this like worst case scenario. But sometimes you forget or it's one you haven't ever had a problem with before. And all of a sudden it's like, I'll just keep doing it over and over. Maybe if I speak louder, maybe if I go slower, will any of these and all the while it's that same, you know, Skinner problem of, like, I am trying to teach 20 other people. This is just one person who now is the center of attention for how badly they're doing. So nobody's feeling good. And. Yeah, sometimes you do that. Let's all sit down and we'll try that again in a little bit. You know, that sense of, you know, usually a break's a good thing. I think in those contexts, too, it's like, everyone knows when you took a break, it's not, you took a break, great job. It's you took a break because you failed. So everyone else might be happy about your failure, but you are.
- [39:37]
A
You.
- [39:38]
B
You know why?
- [39:38]
A
Yeah.
- [39:39]
B
You know the subject.
- [39:40]
A
You don't magically forget your failure. You come back to bed and you're like, oh, it's that thing that I screwed up yesterday, and I'm probably gonna screw it up again today.
- [39:50]
B
Yeah.
- [39:51]
A
And I mean, you think about, like, place to end.
- [39:54]
B
Oh, I mean, I think about all. I mean, how many teenagers at, like, their, you know, burger flipping jobs are just like, oh, it's time for the break. I hated this so much. Like, just the training itself, I'm. I'm getting yelled at. Or maybe I'm not even get yelled at. Maybe I'm just being condescended to. I'm going to go. I'm not coming back from this break. I'll go find another job. Like, forget it. You know, how many people do you lose?
- [40:14]
A
One of the things I took from Dale Brethauer was he talked about how experts are often your worst trainers. And I love that idea. And when I talk to people, they get that because I. But they don't know why. They're like, yeah, this guy's good. I don't know why he's not a good trainer. And I point out, well, because when somebody's an expert, they forget what it was like to learn it. And also, they take shortcuts that a new person can't take. And they're often impatient because they just say, hey, just do it. It's obvious to me. Why isn't it obvious to you? Well, yeah, you learned that 20 years ago, and you've been honing your craft for 20 years. It's easy for you. And that's why you have no patience for somebody who's not at that stage there.
- [41:00]
C
Yeah. And if it came easy to them, then they may never have really needed to reflect or examine on their own behavior or the component steps that you do need to take to become successful. At something if it doesn't come easy. And so it makes it that much harder to break it down and teach to somebody.
- [41:16]
A
Yes, absolutely.
- [41:18]
B
Doug, real quick, it is a little off topic, but if I wrote a training book called Just do it, and it was just filled with a lot of, like, platitudes and like, I occasionally use terms like reinforcement and punishment, so it's sounded like I was trying to do something that was based in science. Do you think I would actually have a bestseller on my hands, Even though it would be a, a really worthless piece of garbage book?
- [41:38]
A
You might have a seller for a few months and then everyone forgets about it. They find the new one.
- [41:43]
B
I mean, I'm on my island, so I, maybe I don't care anymore at that point.
- [41:46]
C
I think Just do it is already trademarked.
- [41:48]
A
Just.
- [41:49]
B
Oh, d. What is it? Well, is it a trademark or a copy? Which one is it? Okay, I'll come up with a different. You can do now. Is that, is that too close? I don't know. All right, let's. It wouldn't be a good book anyway, folks. So I'm sorry, don't buy, don't buy my book. I mean, all these are making me think, you know, making me think, I think, think about, like, how do we make training better? How much goes into it. I have a solution, though. I read your article. You know, I read, I read two articles I read bringing out the best people. So I probably have some great ideas I'm sure that you've never heard. And I don't work in this field, but don't worry, I, I, I got one word to fix the training problem. Computers. There we go. That's going to do it, right? We just. Computers. You're welcome. You know, that's your next book. Computers, exclamation point.
- [42:32]
A
That could be the title. Computers. You're welcome. And we've had computers for a while, and so obviously all our problems are gone because we automated them all.
- [42:41]
B
Yeah, we did it.
- [42:42]
A
Yeah, we did.
- [42:43]
B
Someone else had the idea before me, I guess, but, you know.
- [42:45]
A
Yeah, the computer based instruction is probably one of my bigger disappointments because I just see the potential. And again, I connect the dots to that legacy of Skinner's program teaching machines. And they, in theory, should be able to do a lot. Not all skills. They're not necessarily going to be great at fostering interpersonal dialogues or things that are very complex where you might want. I wouldn't use computer based instruction to teach somebody to write a dissertation. Like, I'm not sure the Computer could accurately review that even with our recent advances in AI there, but it should be able to do a lot more and it doesn't seem to accomplish it. And having spent over a decade doing a lot of computer based instruction before I kind of transitioned out of that, I think a big problem there is that computer designers, computer programmers, once again don't know anything about human behavior. And like all technological innovations, we get caught up in the bells and whistles and we forget, does it serve the learner, does it improve the learning interaction there? And so you will see really sophisticated or you will see really cutting edge fancy computer programs that are boring, that make me feel lost when I try to learn from them, that entertain me, but I don't really get what I need to get out of it. And too often I've seen computer based instruction just devolve into, well, we need this for corporate compliance. So we're just going to throw it into this, have people click some boxes and hope that it's good enough. And talking to a couple decades worth of people about this and asking them, hey, when you do your required computer based instruction, whether we're talking students in a school setting going through institutional review board ethics training modules or employees going through their training modules, I ask them, did you pay attention? And the vast majority say no. I'm like, did you just click really fast until you got to the end and then just kept power clicking through the multiple choice until it gave you a passing score? And while they won't say it in front of an authority figure, if that authority figure is not there. Oh, yeah, totally. I did not read a single word. I powered through it, but I got 100% correct according to it, right? Yeah, yeah.
- [45:33]
B
Some of those, you know, those multiple choice questions of like, did you do the training? Yes or no? Yes. Oh, well, here's your certificate. Good job. You clicked a thing.
- [45:41]
A
I did one today where it's got to the end. And the quiz was, do you, did you read and understand this? Yes or no?
- [45:50]
C
Oh my.
- [45:51]
A
I said yes. And it said, you pass. And I was like, all right.
- [45:55]
B
Oh, no. So, Doug, I know in the paper you discussed some of those, like, multimedia principles that could be used in training. Would you mind going over like a couple of those? Because I think those really capture a lot of, you know, you hear them. And as I was reading them I was like, I feel like every training I've done on computer has broken at least two of these principles regularly. So no wonder those trainings are not my favorites.
- [46:20]
A
Yeah, I love those. Those are actually pulled from a lot of cognitive research, but I think they're super valuable. And it's why I've argued that behavior analysts should pay attention to research outside of behavior analysis. Like, I'll disagree with their interpretation, but man, good data is good data.
- [46:38]
B
Yeah.
- [46:38]
A
And those computer based principles, things like the coherence principle of don't throw stuff that has nothing to do with your topic there. Get rid of those bells and whistles there. And I think of how many, whether it's a presentation at a conference or it's a computer based training where people try to throw memes and stuff to be funny, which if what you're trying to do is lie in the audience, sure. But if you're trying to get them to learn, that distracts them from the actual learning. Because what they remember is the funny meme. They don't remember the point that you are trying to convey there. So make sure whatever's on the screen is coherent with whatever you're trying to train or redundancy principle of don't have dial or print visual text on the screen at the same time as a narration. Because that's really hard to pay attention to both of those at the same time. And we read much faster than we listen. So I'm waiting for you to hit the end of the sentence I already read as I'm waiting for the narrator to actually say it there. And it can take away from what other visual element you want to dry draw the reader's eye to. And going back to the comic books, by the way, it really reminds me of good like storytelling layout. And this is like trying to condense those into good storytelling of how do you draw an eye around the screen? How do you get it to tend to what you want it to? How do you get it to go fast when you want to go fast and slow down when you want it to slow down? Sometimes that's as simple as making sure the visual and the text are near each other. Which sounds simple but violated in practice a lot. I mean even print format. So it's not. These multimedia principles aren't just for computer based. You see it in print where I don't know how many times I've read a, looked at a graph or saw a picture, flipped the page, seen the description for the graph, the picture. And I find myself having to flip back and forth, which just slows down everything and makes everything harder.
- [48:50]
B
Yeah.
- [48:50]
A
In fact, I would say we could go beyond and instead of putting the text in the main body, put it in the Little visual there. Don't make it something that's separate. Let's integrate, integrate it there. So show it at the same time and show it at the same place. Simple, but often violated. And probably one of the ones that I think maybe most important nowadays is the social presence idea of the more the computer feels like it replicates a social interaction, the more likely somebody's going to engage with it and the more it feels like you've got some animated pedagogical agent like Clippy, the. The guy who just talks at you but has no human characteristics. I mean, he's not bad. He blinks and he moves and he's animated there. But you see bad versions of that where it's very stiff and it's a very computer sounding voice. People disengage immediately from that. And all that was based on tons of research that people just went and they tested and played with these little features and they found out better retention if you frame it up this way, if you place it like that. So thinking about where your text goes, where your visuals go, how they connect to one another, do you simplify a graph? Do you make it complex? Does it matter whether you're dealing with a novice or an expert learner who's got the background? Do you animate something versus a static image? Tons of great research looking at that that if you're designing training, it would be well worth even just a superficial read of that stuff could advance your training practices. You don't have to get a PhD in instructional design or multimedia education to really up your training quickly.
- [50:45]
B
Oh, all right. So, Doug, we've talked about a ton of, a ton of principles. I think, I think everyone would really love to sort of hear. Maybe it doesn't have to go into crazy amount of detail because I know it's like, oh, 50 minute podcast we got to. Everyone should be able to do this now, you know, but just generally, I would love to hear if you, if you have a good example of, of either one you've seen, one you've designed of kind of like what overall were the steps of the process to create what you would consider a training that, you know, hits most of the. Hits most of the high marks of creating, you know, a good training that is based in these principles.
- [51:23]
A
Yeah. Probably the best example I've ever seen would be headsprout.
- [51:28]
D
I love pet sprouts. Yeah, it's not really there. Yeah, it's not there anymore.
- [51:35]
A
It's not around anymore. But I mean, and to me it's also a lesson on Be a little careful with how you invest on that because I had sprout wonderful reading program and they went through all the development steps. They followed Markle and TMN's guide. They did all the practice, the testing, the retesting, tons of retesting, expanding it with different learners until it served every learner. The antecedents were great, the consequences were great, and it worked perfectly there. But the amount of time and money that they spent in that made it kind of impractical there. And then it's a little bit of a shame that it was sold and then resold and resold and then retired I think a couple years ago there. But it's like a great example of when it works right. It can work right now. What I've tried to teach people I've worked with and former students of mine is let's not always go for the gold standard there because I see headsprout as the gold standard. But I also remember how much investment it took and how impatient your average client is going to be with you trying to test and retest and do every thing. In fact, I would say of Skinner's mistakes, that was one of his big mistakes, was he tried to go with a gold standard solution right off the bat. So his teaching machines tried to be elegant. And as behavior analysts, we know the value of shaping. So what can we do to just get it a little bit better? So anywhere I've seen where they start adopting more structured objectives, they start building in more interactions, they start thinking about feedback a little bit better, they start thinking about do we need examples or non examples? Can we add more? It may not follow every single step, but I think anything that's a progressive approximation to that gold standard is an improvement. And I'm honestly happy with anything that does better than the status quo. Even as I try to get things better and better and better. If it's going in the right direction, it's going in the right direction.
- [53:51]
B
I think, you know, kind of with, with headsprout as our gold standard, we talked about a lot of principles. It's probably a good time to move into dissemination station. Oh, so certainly like I, I was sort of joking around, there's so much to this topic that we're not going to be able to cover every detailed aspect of it in terms of how to develop just those gold standard trainings. And perhaps we don't need to develop a gold standard training. So I would love to hear from you because I'm guessing a lot of folks are doing what I'm kind of doing in the back of my mind right now is thinking about, oh God, I have a training that I'm supposed to do in a week or two. And I just realized it broke every one of the principles Doug just mentioned. I got to do a little editing, but I may not have a lot of time. So if you had to pick one or two principles that you would say to just sort of you kind of the average practitioner who does trainings in their business, probably an ABA business, to make their trainings just that much, a little bit better, that shaping process of using better behavior based instructional design, where would you tell them to start or what principles would you tell them to make sure they haven't broken?
- [54:59]
A
Per chance I would. Where I usually start people is with a step size issue. So try not to throw too much at your learner at once. And the other is gauging the progress of your student or trainee. So what? When I interact with trainers, I've had many trainers try to train me and I let them go for a while and then I ask them, did I understand what you just trained me on? And they say usually they're kind of a little honest. Oh, probably not. And then I say, oh, so you don't think I'm smart? And they're like, oh no, you probably have it. It doesn't matter what they say. I'm going to call them out on it. And so if they say I got it, I'm like, so like I pick up everything immediately and then I highlight for them, wait, you don't know whether I understand this or not. And that's a real eye opener for a lot of trainers. And they're like, yeah, I guess not. I'm like, why don't you ask me to teach it back to you? And so borrowing that, that teach back feedback approach where I say, okay, have me teach it back to you. You talked for a bit and then as soon as I start teaching it back, they find out, oh man, he didn't understand hardly any of this. I'm like, I'm not. Yeah, but I'm not the dumbest guy either. And you're. I understood hardly anything that you just shared there. That's a failure. And if I'm feeling a snarky, I might say something like I think ogworn Z used to say of, well, if the teacher, if the student didn't learn, the teacher didn't teach there. So you talked at me, but you didn't teach me, you didn't train me there. So small chunks frequent Assessment. And as you do that frequent assessment, you start finding out, am I going too fast? Am I going too slow? And I usually give them a rule of thumb, like, try to give no more than about seven new items. And I'm like, don't get uptight counting it there. But just like about seven new items. Again, if you have ten, don't cut it off awkwardly. If you have four, don't awkwardly try to find three new things. Just the principle is small chunk and then assess it. And you should be senior trainee talking and doing more than you're talking and doing. And so if you just broke it down to those couple of good guidelines practices, I think that would open up a lot of people's eyes because they're now getting feedback on how they're doing. Even if they don't learn a single other practice that's going to make them start changing things up and trying new things and probably getting a little bit fatter.
- [57:54]
B
All right, Doug, what do I owe you for that? Because I got a meeting on Monday where I got to go over the training, and I'm going to be like, you know, Doug, Doug, I need to do the following. It was great. I think those are two excellent steps. I think everyone can do those really quickly, whether they do them, you know, to the extent they might want to. Also, they know if they call you to consult on their training, you're going to give them a hard time, it sounds like, but they'll be better for it. Oh, excellent. So all this sounds great. Why doesn't everybody use these techniques and develop awesome trainings and find the time to do all these awesome, you know, trainings and do these assessments?
- [58:32]
A
Yeah, I think part of it is that we don't have enough obmers and behavior analysts getting involved in this kind of work. We exist, but we few, we happy few are not enough to cover all corporate training that's out there. So there's.
- [58:49]
D
How dare you? Of course you are.
- [58:52]
A
There's like a lack of available talent. Like, if my company went to me and said, look, we want to implement this, like, at all of our sites worldwide. So we want to hire 100 people to help guide this and do this. Best practices. I would have a hard time finding those 100 people to hire. Behavior.
- [59:15]
B
Yeah.
- [59:15]
A
And that's just one company. Think about if every company wanted to do it. We don't have that. A lot of behavior analysts tend to work in just one or two areas here, which is both a curse and a blessing there. It's a blessing in that it really opened up a lot of employment opportunities for our field. But it's a curse in that it kind of funneled filtered too much of our field into just a few things. Where I always saw behavior analysis is covering everything. And Skinner talked about everything being covered here. I think another thing is that they don't know what is available. So that's. I was just saying if they knew we don't have the resources for it. Often they don't know. So they're open to this idea, but they've not been exposed to this idea. In addition to my degrees from behavior analysis, I also got my master's Business administration, my mba. And the entire time I was training on that or taking those classes, they never really talked about how you manage people. They talked about very precisely how do you manage your spreadsheet, how do you manage your profit loss, how do you calculate? They were not shy of getting into data, but when it came to people, they really never addressed that.
- [60:40]
C
You're like, no, thanks.
- [60:42]
A
They don't know. And they're just like, well, people. People are the error term. People are what goes wrong and there's nothing we could do about it. So we do everything else really well there. Oh, interesting. And then there's the cultural expectations outside of that of, you know, you can't really change people and people just do what they do and you just gotta deal with that. And we obviously can do much more than that. But where do we get our business leaders from the general culture? We get them from MBA classes that don't teach them this, or we hire them from our ranks of who was good at their job, which is not the same as being good at management or leadership there. So I think it's a combination of just not knowing what's possible. And then even if you knew it was possible, it's going to be hard to find the people. Which is why I'm always passionate about trying to get people excited about obm, because there's a need for it. And if you know how to talk to people, there are people who want to hire that. But we don't have enough obmers. We don't have enough behavior analysts getting into work in the workplace.
- [61:53]
B
Oh. So unlike some of the topics, I think we end with like, there's just so much we need to know. It sounds like this, at least specifically in developing trainings. We already know a ton, but nobody knows that anybody knows it. And we don't have enough people who know it to share that information. So it's very. A bit of a Different, different problem for.
- [62:13]
A
I mean, there's always more we can learn. There's always research to be done. But we have so much research that we just haven't applied yet. I'm like, let's get applying that good research. I'm happy to see other people still pioneering and doing cutting edge research, but there's, there's already gold that we found in the hills.
- [62:36]
C
So if there are behavior analysts who are interested in going over into obm, what should they do?
- [62:41]
A
Obviously learn a bit about it. So I always recommend people start with like Aubrey Daniels book, because bringing out the best in people, it's not going to teach you all of the principles, but what it does a great job is teaching you how you should be talking to managers because Aubrey was great at talking to managers and translating our complex science into a way that the average person would understand it. So knowing that you already have skills, but learning how to market your skills and convince others, obviously you need a lot of continuing education too as well if you're interested in the training space. Dale Brethauer and Carol Smalley's book, Performance Based Instruction, I think is a really great book. It does teach something that's very similar to bst, but it connects it to the business strategy and mission and results in a very clean way that you need for a corporate context. And then you can always join the OBM network and learn from those folks. And there's tons of opportunities to learn everywhere.
- [63:47]
C
Very cool. Awesome. Thanks.
- [63:49]
B
Dr. Doug Johnson, thank you so much for being on the show today. It was a real blast to read this work and then to get a chance to talk about it and hear, hear more about the details straight, straight from, from you. If folks are interested in getting in touch with you, are they able to like email you if they have like questions or they want to, you know, what are some other resources I should be looking at?
- [64:07]
A
Yeah, I probably have the easiest email to remember because my email. Yeah, behavior analyst, gmail.com. i'm the one who got that first.
- [64:16]
B
Yeah. How did you get. Did you like, did you win a lottery to like. I need a quick. I mean, it's like the equivalent of like, I wanted one 23 mail.
- [64:24]
A
It was back in the day when Gmail was invitation only and I had a friend who had an invite and Doug Johnson was taken. Douglas Johnson. All the variations of my name. And I'm like, fine, I'll try behavior analyst. And it wasn't taken. And so I took it.
- [64:40]
B
It's a good thing. It's a good thing. You didn't pick like dude master or something, you'd be stuck with that one forever. All right, well, Doug, thank you so much. This was a blast.
- [64:51]
A
Thank you. This was a pleasure for me. Wow.
- [64:53]
B
Big thanks to Dr. Doug Johnson for joining us on the show, talking all about instructional design for business training. It was excellent to sort of hear all of the work that should go into your gold standard trainings and also to talk about a lot of the other concepts that are important to training, some of which you can use on that PowerPoint presentation that you're looking at right now and saying, good, I need to make this better. Couple good ideas in there. A lot of good ideas, but a couple good ideas that you could do right now without necessarily going back and getting your MBA like Dr. Johnson or learning too, too much about OBM just yet. Though I, I think many of us do enjoy that field. So now that we have bid farewell to our guest, let's go to the last section of the show. Pairings.
- [65:38]
C
All right.
- [65:38]
D
Hello.
- [65:39]
C
It's time for pairings. Pairings is the part of the show where I tell you about past episodes. You might want to go back and listen to if you found this one of interest. So we have several episodes where we've talked about varying forms of instruction. And then we have also episodes we've talked more about OBM related things. So I'm going to give you a little bit of both of those. So they include episode 221 where we discussed tracker training with Ryan Atkinson. Episode 230, precision teaching with Jared Van. Episode 239, behavioral instruction with Kendra Guinness. Episode 257, key features of direct instruction. Episode 218, ABA and literacy skills with Denise Ross Page and Doug Greer. Episode 129, staff turnover with Byron Wine. Episode 180, promoting ethical leadership with Manny Rodriguez. And then we've done two Aubrey Daniels book clubs, episode 140 and 141 where we talked about bringing out the best in people. And then Most recently, episode 1019. How do you say those? 1019, the Oops. Book club. So if you're listening to this in 2025 on the free feed that will be coming out to you in 2026.
- [66:57]
B
No, no, no. Everyone got that one.
- [66:59]
C
Oh, my bad.
- [67:00]
B
Yeah, that was supervision September.
- [67:01]
C
Oh, never mind. Okay, everyone can listen right now. I'm sorry, I got confused. And then I also like to recommend a snack to go with the pairings. And so the snack today is popcorn with like The M&M's mixed in to it because I feel like if I had to attend a training and do some like online modules, studying that is a snack that I would want to have.
- [67:22]
B
Dr. Johnson did not mention it as one of the components of instructional design, but technically he mentioned it in the paper. He didn't mention in the discussion. But having snacks, you know, would fall under that category of sort of like those emotional relations to the training. So it is actually potentially a good component of your instructional design. You want to, you want to keep it in mind. But again, you could also do an amazing training where you have like a seven course meal with all these great snacks and then teach people nothing. They might still give you a really good review on your training though. You know, deep down you taught absolutely nothing. So don't do that. Gotta get, I gotta have a nice mix of content and snacks and M&M's.
- [68:00]
C
Mixed in the popcorn.
- [68:01]
B
As long as they're in there. Yes.
- [68:04]
C
And that was pairings. Please enjoy.
- [68:05]
B
All right. Thanks, Diana. And now that pairings is done, that brings us to the end of the podcast. Thank you all so much for listening. If you haven't done so, we'd really appreciate if you left us a review on Apple Podcasts or wherever you like to listen to your podcast. You can also find all of the citations we discussed linked on our website, ABA InsideTrack.com where you can again find those links. You can purchase CES as well there. And if you haven't joined us and you're thinking, I want even more of this ABA Inside Track business, well, patreon.com ABA InsideTrack is the place to be where you can subscribe at any level, even at the free level, just to get, you know, access to a couple extra things is, is always really appreciated by us. But if you want some free CES as well as access to all of our polls, including our quarterly listener poll, listener choice polls, where we let the listeners pick from a bevy of topics that we then do an episode on. Or if you want to join at the $10 level to get access to every single book club we've ever done immediately upon release, we do one of those every season, as well as free CES for those book clubs. Hey, this is a good time because we got two OBM books we've talked about in the past. That's again, patreon.com ABA Inside Track. Oh, and speaking of going to our website and ces, you probably want that second secret code word, don't you? It is comic. C O M I C could be a stand up comic or a comic book or a comic situation would be the things I just thought of. Comic. That's the code word, though. You got to have that. Finally, we want to thank Dr. Jim Carr for recording our intro and outro music, Kyle Sturry for interstitial music, and Dan Thabot of the podcast Doctors for his amazing editing work. We'll be back next week with another fun filled episode, but until then, keep responding. Bye.