C (3:20)

It was not in mine at all. I had never heard about it and I was fascinated to learn more.

A (3:27)

So, no, yeah, I already knew of it, but I hadn't read these articles before.

B (3:35)

So one of the challenges with certain branches, I think, of psychology outside of behavior analysis, is sometimes you can find sort of the articles for kind of an introductory episode like this one, and sometimes you're finding a lot of more generalized articles that are sort of talking about collaborative problem solving as part of a package or discussions of it. So we picked ones that we thought captured what makes collaborative problem solving. But we're also going to be pulling a little bit from the Think Kids website, which is the group out of Mass General that focuses on collaborative problem solving specifically. We're going to be looking at one of their tools and we'll have a link to that. We also have a link to their resources. Also. A lot of this seems to be published in book form, which we'll be talking a little bit about how we feel about that at the end in dissemination station. And we did not have a book club coming up on this topic, but if it's something people are interested in, maybe we'll get. Get our hands on the book and. And put it in a book club one of these years. But we have four articles we'll be discussing. Three we'll be discussing at length. One that sort of is an article referenced in another article. So we thought it was worth having the citation handy, but we won't be going into as much discussion of that. But Diana, what are the articles we'll be discussing today?

A (4:48)

Okay, as Rob said, we have four articles. They include a transactional model of oppositional behavior, underpinnings of the collaborative problem solving approach by Green, Ablon and Goring, and that was in the Journal of Psychosomatic Research 2003. Also paired with that is a brief report titled effectiveness of collaborative Problem solving in Effectively Dysregulated children with Oppositional Defiant Disorder. Initial findings, that was by Green, Ablon, Goring, Razor, Blakely, Markey, Manitow, Henin, Edwards, and Rabbit, and that was published in the Journal of Consulting and Clinical Psychology 2004. We also have an open trial of collaborative problem solving in a naturalistic outpatient setting by Pilastri, Wang, Eddy, and Ablon. That was in Clinical Child Psychology and Psychiatry 2023. And finally, de implementing a point and level system in youth residential care without increased safety risk. A case study by Stol, Hartman, Paxton, Wang, Ablon, Perry, and Ptri and that was published in Residential Treatment for Children and Youth 2023.

B (6:18)

Okay, so rather than getting into sort of the introductions of collaborative problem solving, let's kick it off. Diana, why don't you talk about the Green and Ablon 2003 transactional model of oppositional behavior. Because that gives a good summary of what collaborative problem solving is and the kind of oppositional defiant disorder esque diagnosis or children with diagnoses of ODD that tend to be with where a lot of this initial research came from and a lot of this initial work started. And then we'll talk a little bit more about what CPS sort of looks like nowadays in broader terms or with broader groups.

C (6:58)

Do you know that before I looked at the article, I was like, oh, did one of you write it? Because one of the last authors was a Perry. Oh, no, but it wasn't a Parry. It was a Perry. No, with an e instead of an A.

B (7:13)

Nuts to them. Nuts to them in their English spelling of the name.

C (7:18)

Just. Anyway, okay, I'm so sorry for that, for that weird aside. I just wanted to like that this.

A (7:24)

This article.

C (7:26)

No, it's like the third article in.

A (7:28)

Oh, okay.

C (7:29)

The last author's pear. Perry.

A (7:33)

I just read them all. I didn't even notice.

C (7:34)

Hairy.

A (7:35)

That's funny.

C (7:38)

So I wanted to let everyone know we don't. You're not related to that person.

A (7:42)

Yeah, no relation. Okay. Yeah. So this article is from 2003 and it I think was maybe their first foray into publishing this in a journal format. But the CPS was already in existence. It was already had been being utilized in different settings. Right. And so there was already research to back it up. So this wasn't like, oh, this is an absolute pilot study. This is just their first sort of like summary, I believe, where they kind of like put it all together.

B (8:14)

It. It definitely we'll get a little Bit into the publishing history. It definitely doesn't have some of the same, I think, clean line that we often see when we're talking about behavior analytic treatments.

A (8:25)

Well, neither does like the ISCA truth, you know. Yeah, it might be kind of similar in terms of like it's not the ground.

B (8:33)

Yeah, I, I, I think there are some, some big similarities between, between sort of at least the, the themes in the CPS and the isca. Even though the technologies are not the same.

C (8:44)

I never thought about that, but totally totes.

A (8:48)

I thought you were going to say something.

B (8:49)

Oh, well, I mean, I guess I don't, I don't know if that's also more in, in psychology where you have lots of people from different backgrounds working in hospital settings or clinical settings, they're doing a lot of work with lots of different types of individuals. And whether it sort of becomes practice. Like it definitely seems like some CPS came from practice that seemed to be really effective and based on enough, you know, kind of psychological principles that then eventually publications, like peer reviewed publications came out. Because you'll see CPS sort of more in book form and in books before you'll see it in publications, as far as I could tell. I'm sure. I, you know, I, I don't have an exhaustive list of every CPS manual, but at least from the Think Kids website where they're sort of putting their citations, this was one of the earliest 2003 though CPS is, was kind of like a late 90s creation by Green, it seemed. So, so some of the publication order is a little, is a little wonky to, to my, I don't know, to my readings.

A (9:50)

Yeah, yeah, yeah.

B (9:51)

So not bad wonky, just not the order I usually see.

A (9:54)

So in this article they are kind of centering everything around developing an understanding of the ODD profile, which is oppositional defiant disorder, and how the CPS or the collaborative problem solving approach can benefit that profile, specifically relationships between children and their parents. And that's a lot of what this is centered around. Whenever I hear cps, I still, I always think Child Protective Services. And for a long time I thought that this whole program was like, like produced by Child Protective Services. I don't really know why, I mean, I just thought that. But it's not, they're different things. So this is collaborative problem solving, which is a good name, which is, it is descriptive of what this is, but it's also like kind of a non, non descript name to me. It's like kind of generic. So anyway, I guess we'll call it CPS moving forward. But they start us off talking about the ODD profile and what some of the components are of it that can make it challenging for parents or educators working with children to kind of, you know, develop and maintain strong relationships due to some of the behavior that's seen from this group of kids. So they give us, I'm just going to give you guys a quote from the article and a few different times here so you kind of get a good feel for it. So they say the ODD profile refers to a recurrent pattern of developmentally inappropriate levels of negativistic, defiant, disobedient and hostile behavior toward authority figures. Behaviors associated with ODD include temper outbursts, sometimes referred to as rage attacks, persistent stubbornness, resistance to directions, unwillingness to compromise, given or negotiate with adults or peers, deliberate or persistent testing of limits, and verbal or minor physical aggression. End quote. So there is a lot there to unpack. I know sometimes we hear about these types of profiles and someone will say it sounds like an ODD profile. I think that testing of limits is a real sticking point there that you often hear about. And difficulty with being on the receiving end of almost any type of demand is the pieces that I, I think are definitely come through as specific to this particular profile. They talk about its comorbidity with Tourette's. I don't know if maybe their background was with Tourette's perhaps, or if that was just. Or if really and truly that is really commonly you see those two together. So they said in. In 65% of Tourette's presentations, they also see odd, which is incredibly high level of co occurrence in the general population. They, you know, whenever you're trying to get these samplings from the general population, it's always a really wide range. So anywhere from maybe 2 to 16% of the general population could meet the behavioral profile for ODD. So when this was written, which is 20 years ago, they said not a lot of attention has been given to odd. Certainly I've heard about it a lot more in the last 20 years than before that. They talked about what kind of has been done in this area. And there were kind of two main approaches. The first being when there was a lot more consequence based basically and it was a parent training or behavioral family therapy approach. And said the main components here were focusing on positive attention using appropriate commands, contingent attention, contingent reinforcement, and timeout as needed. And they said this has been established as effective, but it also has its drawbacks. One of the main drawbacks is that parents often cannot implement this with Fidelity. And that makes sense. Right. I'm sure lots of folks are shaking their heads at home as well, because this can be quite a challenge with some of the things that we might be recommending as well. They said it really, really nicely here, though, in. In the way that they talked about that this is challenging for parents. They said this form of intervention may not be well matched to the needs and characteristics of many of those responsible for implementation.

C (14:01)

No.

A (14:01)

That's a great way to put that.

C (14:03)

You said that I did this. Oh. And then you said people are nodding their heads. I was nodding my head, yeah.

B (14:08)

Definitely is a theme, I think, in CPS as an alternative or an additive to a lot of sort of treatments that have been researched prior or have been used for many, many years in that it's not that they are bad or worse or better than cps, but there's something about CPS that just feels a little more.

C (14:27)

Easier.

B (14:27)

Easy. Yeah, easy. Not simplistic, but. But definitely seems easier to implement and more in line, I think, with how people want to use treatments with their children or with other people's children.

A (14:41)

Yeah. And. And so the, you know, some of the core ideas here perhaps are. Are not bad. It seems as though they would be effective, but perhaps due to difficulties in implementation or perhaps due to some, you know, missing components of this approach. They said other research have found that 30 to 40% of children who are receiving this, what they're mostly calling a parent training approach, continued to show problems at a. In a clinical range following treatment. That's a. That's a lot. Right? 30 to 40% of kids for whom this is not really working. So they said another approach really focused a lot on the cognitive factors, which we could kind of think of as like, the antecedent side of things a little bit more. They said this includes really taking into consideration the frustration and emotional arousal that's associated with demands for compliance, to which I wrote fair. I don't like being asked or demanded to comply with something. Right. And I know a lot of other people that don't either.

B (15:40)

Listeners, you might think if you had to pick which of the three of us would be the most angry if told to do a chore, you might think me, because I'm very loud. You'd be wrong. You might think maybe Jackie, because she's like a free spirit and she doesn't have time for chores, you'd also be wrong. You probably wouldn't think Diana, who seems very placid and calm and follows rules, and you would be 100 wrong. There too. She hates being told what to do when it comes to chores or really any activity. Like, gets very angry and violent.

A (16:14)

I don't get violent.

B (16:16)

No, I don't call cps. I need help.

A (16:19)

You get angry, though.

B (16:20)

My wife.

A (16:21)

Okay, so these cognitive approaches, they. They called them probably efficacious in the article, and those included things such as problem solving training and anger management programs, which, I mean, I think we would be like, yeah, that's. Those are good things to. To do. Right. So these are authors said, well, why can't we make something better by combining both of these approaches together? And, you know, together is what is it? The whole is more than the sum of the parts. Right. And we can create something new and better that's going to serve both the children and their families more. So they're titling this in this article, a transactional model of odd. And to me, transactional means like, I give you something and you give me something back. Right. And that's, I feel like when we say, like, oh, something has a transactional nature to it, that means that it's kind of uncaring. Right. And unfeeling. And that's not what they mean by that at all. So they're saying that this is what they're meaning by it is like it's a reciprocal relationship. Right. And we've talked about that in the past, both in reference, I believe, to Pat Freiman's work and with the power of parenting, Glenn Latham stuff. Right. So the way in which parents react to kids affects kids. It changes the way that the kids are behaving because it's changing what they're learning, it's changing what they're seeing. And when they then reflect negativity back on the parents, the parents behave negatively toward the kids. And it creates like a vicious cycle of negativity. Right. So, like, you're bringing out the worst in everyone.

B (18:02)

Yeah. And I'm guessing the authors got that feedback at some point too, because you really don't see it described as a transactional model in any of the other research articles we discussed.

A (18:12)

Okay, great. Well, that kind of makes sense then. So I think, you know, you don't need to worry about remembering that title. I think reciprocal relationship is really more what they're going for here.

B (18:21)

It's more, what does it say? What. What's the model about?

A (18:24)

Yeah, so they said, you know, children can't really develop emotional regulation skills, frustration tolerance and problem solving skills on their own without having them modeled to them. They also spend a long time in this article, like breaking down the profile of ODD and the components that are likely contributing to this, like, difficult cycle. And I'm not going to go into all of that here for the sake of time, but you could definitely hook in there and read it. But these pieces are, are largely what's going on, right? So difficulties with emotional regulation, frustration tolerance and problem solving skills. So they talk about how parents and children can get caught in that negative pattern together. And then they describe it really nicely too. So they say, quote, the method by which caregivers respond to deviations from expectations for compliance can serve to increase or decrease a child's frustration and arousal and to alter or fuel emerging response biases in both child and adult. In other words, if the environment responds to a child's compromised self regulation and effective modulation skills in a manner that exacerbates the child's existing difficulties, a maladaptive automatic adult child response cycle may develop, making change much more difficult to achieve. End quote. So that's what I said, except they said it better.

B (19:41)

Yeah.

A (19:42)

Okay, so the reciprocal model here says that if the child, the child will thrive, if there's going to be a good fit between the adult and the child characteristics or approaches, and everyone has to be actively involved and compatible in the way that they're interacting in order for this whole thing to be successful. And so that's kind of where the parent training side comes in. Because now we can work with parents to teach them how they can best interact with their child for the child to be successful, given that they are, you know, potentially starting out with this profile. And I will talk more about that in, in a second, but they said that they really think of this as having four goals of collaborative problem solving on the adult side. So the goals here are that the adults, it helps the adults, number one, understand the adult child characteristics contributing to the development of child odd. And they break that down and say what they try to do is help the parent think of ODD as a learning disability, but it's specifically a disability in those areas we just talked about emotional regulation, frustration tolerance, problem solving and flexibility. And if the parents kind of come to it, thinking about that, like this is an, an area that's very difficult for my child and I need to be helping them. It helps the parent be a lot more empathetic to the child and a lot less reactive when they see behavior on the part of the child. It's it, it kind of makes the outburst that they see from the child if, if they see them less personal toward them. Right. They're like, this is a skill that my child is using, is lacking or is working on developing. So having that understanding is the number one goal. Number two goal is becoming aware and then implementing the three. Or no, I'm sorry, you don't want to implement all of these, but you want to become aware of the basic strategies that have been used or could be used to handle unmet expectations. So the first one is the imposition of adult will. Right? Right. So you hear people be like, well, it's my way or the highway. I told you, you got to do this thing, so you got to do it. Right. That, as you might imagine, if you're working with an odd profile, could be met with a lot of butting heads and therefore a lot of behavioral outbursts. The number two approach option is collaborative problem solving, which, of course, they're going to teach you how to do. And then the number three option is removing the expectation. And I, I, I think that this is really valuable here in that, in this approach, they're recognizing that at least sometimes removal of the expectation may be the option that's safest. Right? In that if you were to keep going with the imposition of adult will, once the demand has been put into place, you're just going to escalate, escalate and escalate the situation. Sometimes it is better to, you know, let that one go and move forward from it. And, you know, as behavior analysts, that can be that. The sticking point, if you're not, you know, really savvy in how you're writing behavior plans, right? Where you say, well, you gotta, you gotta go through with it. You gotta implement the demand. And you end up in this exact pattern of escalation and negative reciprocity. Sometimes you have to let that one go, right? And the accepting that as a reality that maybe whatever expectation that was isn't going to be met, what, you know, that demand is going to be escaped from. To put it in our language, recognizing that allows you to really shift your focus over to the antecedent side. And that's a lot of what the CPS approach is about. Okay, so that was my own aside. And then they break down what the components are that they want the parents to understand in the CPS model. So I love this. They have basket A, basket B and basket C, which does sound a tiny bit similar to some things that we talk about in our field. But basket A, they want parents to understand, like the breakdown here, right in the environment, basically. So basket A is the common precipitant to the outburst. Meaning what is the antecedent environment look like? Basket B is the CPS approach that we're going to take in order to de escalate problems before they happen and offer alternatives. And the basket C is the removal of the expectation.

B (24:20)

And the other piece, similar to how sometimes basket C is the right basket, as I know and they talk about this in some of the other articles, Some of the other works I've seen is it's not about basket B. CPS is the best one and it's 100 always the one you need to do. Sometimes you're going to do basket C. And there are times that basket A where even though you know it is a precipitant to problem behavior, you do have to say as the parent, it is my responsibility to tell you to do this thing because it is so important. So it's more about understanding when you would or not use these different baskets or strategies. So when are you going to hold firm? When are you going to remove the demand? When are you going to engage in collaborative problem solving?

A (25:01)

Great, thank you. All right, so the two, the two goals we reviewed thus far are that the adult understands what sort of the child's profile, right? And we talked about that as like a learning disability in returns of emotional regulation. Then we talked about that they're aware of these different approaches and the different baskets. The other two goals really just go hand in hand with that. And that number three is they recognize the impact of these approaches on parent child interactions. Some of them are going be more positive or negative. And then number four is that they become proficient at collaborative problem solving to resolve disagreements with their children. The adult is the facilitator of the CPS in the parent child relationship. And they say think of them, think of the adult here as the quote unquote surrogate frontal lobe to help with these executive functioning, emotional regulation types of skills.

C (25:58)

That didn't help me. I had to look it up. I was like, where is that and what does that do?

B (26:03)

It's your executive functioning part of your.

C (26:05)

I know that now, but I didn't when I was reading it.

A (26:08)

Yeah, yeah, yeah. The point is to reduce outbursts in the moment, train the child in the problem solving skills in the long term. So there's obvious, you know, a short term and a long term approach here. You're a lot of this is going to focus and focus on antecedent arrangement, not on reactions to the outburst. So it's not a reactive approach. It is a proactive approach which I love. I Love anti antecedent approaches. And they, they want us to understand that it's different from other approaches because the adult and the child need to work together collaboratively. And then finally they say it, it's a manualized treatment program. So like that's where I'm saying it wasn't as though this was the first time they were writing to anyone about it. They already had a whole, you know, bunch of manuals and literature related to it. But every session isn't going to be the same for every parent, child, client. It's not circumscribed in that way. Rather, the manual walks parents through what they need to know in order to respond in the moment to their individual child in a way that's going to work for them. So the manual focuses on five main areas. They include educating the adults on the pathways to non compliant behavior. Number two, the use of the BASKETS framework, which we already talked about. Number three, medication education, which I appreciated seeing because it's a, that indicates to me this is a holistic approach. Right. We're trying to make sure that all angles are being looked at as well as family communication, which is number four, including encouraging parents to think about how they're talking with their child and altering, altering communication patterns such as sarcasm if needed. And then five, additional cognitive skills training as required. And that is our intro article. And then the 2004 article. We're not going to review that here. But it, you know, because you get up to this point and you're like, and now they're going to do the research, right? And then. No, it's the references. The article's over. It was a teaser. So you're, it's almost like the 2003 and the 2004 were like one article that they then just divided in two and made it into two publications. Because the OH4 article basically takes everything you learned in O3 and then applies it in group design format. They had 50 kids. Two thirds of them basically were assigned to the CPS group and one third were randomly assigned to the parent training group. And they ran it long term. It was like 10 weeks, I believe. And they looked to see what changes they saw. And that the CPS group did better is like the short version of that. But folks can go check that out. Yeah.

B (28:50)

So a couple other details before we take a little break. So those are some of the earlier citations talking about collaborative problem solving. But if you wanted an even earlier one, you could read Ross Green's the Explosive Child, which was published in 1998. It's a good book. It's a lot of, like, scripts between a mother and a daughter who are having some problems. And it really does capture sort of what those problems kind of look like in terms of, you know, the mom feels like she's trying really hard to figure out why the daughter's always mad about everything. The sibling is like, why do I even bother being in this family? The dad is like, I'm tired of this crap. I'm going to lay down the law. But that doesn't seem to work either and really goes through the steps of that basket B, that Plan B of the collaborative problem solving of looking at it as a skill deficit rather than a willful lack. You know, there's not enough reinforcement for doing the right thing or the child is just willful, you know, or stubborn or whatever kind of mentalistic explanation you want to have.

A (29:48)

So that's insolent.

B (29:49)

Yeah, so that's a rich. So that's a really good example. And it does seem that J. Stuart Adlam and Ross Green, they. They worked together for a while at the beginning, I'm guessing at like Harvard Medical School or something, maybe. While J. Stewart, A.B. seems very young, he actually spoke at the District I work in a couple weeks back, and I've seen some webinars with him and he seems pretty young, so I'm guessing there was. They used to work together and then they went separate ways to publish their own things. But they did have a Treating Explosive kids book in 2005, which was sort of like the CPS book at the time, though since then, Ross Green's gone and he's done some other books he'd lost in school I've seen referenced a lot of times, which I think goes over very similar content as the Explosive Child and. And then Ablum and Ptri, sort of the new collaborator at Mass General has done a lot more of kind of procedure guides. There's an updated procedure guide from 2019, and we'll have a link to the Think Kids resources, which has some of these books mentioned here, as well as a couple lists of. Of some of the updated procedural guides. So when it comes to that collaborative problem solving, it basically boils down to a. It's sort of an antecedent manipulation to start, but I actually don't consider it necessarily an antecedent plan fully because at the end of the day it really comes down to you as the adult need to discuss what you want out of the situation that, you know, if you say do this will lead to problem behavior. So you need to discuss what you want out of the situation and why, which requires communication skills. The child then does the same. And then you compromise on a solution is basically what it comes down to. And that's the plan B. You come up with a solution that works for everybody, and then the child tries it and then gives you feedback as to how it worked. And then you repeat this process until everyone is happy with whatever the demand is, doing the dishes or whatever the demand that's. That's causing the problem is you can use some of the tools they have. I really like this Thinking skills Inventory which I pulled from their website. It's Wang et al. From 2018. It's pretty similar to the list that I think Ross Green puts in the Explosive Child. But it's a lot of skills related to executive functioning, social skills, language skills and emotional regulation skills, cognitive flexibility skills. A lot of them are very concrete in terms of. This is a skill that you could operationally define pretty easily. Some of them are a little vague or kind of possibly bordering on mentalistic. So it would require a little bit more work, I think, for a behavior analyst to not feel kind of icky about maybe using them or feel like, you know, you're, you're, you're providing too much of a mentalistic description. So like, you know, handles deviations from rules is a little fake. Understand spoken directions. Again, for us, is that a, Is that an issue of a failure, you know, like a defective listener repertoire, or is that a actual misunderstanding, like a cognitive misunderstanding of direct. Like, you know, how specific do you need to be? Avoid. Avoids personalizing? Again, that one starts to get a little more mentalistic, I think getting to the idea of they're taking it personally. But again, how do you define that?

A (33:08)

But again, maybe they're just not operationally defined in the way that we would be used to. But I don't.

B (33:14)

No, they're all fine. They all seem workable. Yeah, they're either, they're either kind of a workable list of either observable behaviors or kind of private verbal events or social behaviors. But again, I think it'd be very easy to take this list, get more nitty gritty if you wanted to define them. Examples on examples. If you needed to and test and record whether they are or not happening. They also have. They'll refer to this, the CPS Apt assessment, which when I hear assessment, I think, oh, it's going to be very fancy. It really boils down to kind of like an antecedent behavior chart, which is funny because Sometimes they'll refer to, you know, there's a couple digs on operant behavior in some of this literature. Maybe they're moving away from that because I consider this all operant behavior. You know, you are still engaging in some behavior and contacting some consequence and then deciding whether or not to continue with that behavior or not. I think the big change with CPS is you're taking some time to look at why is the antecedent not resulting in the behavior. You know, why is this not cueing the appropriate behavior? And a lot of times it has to do with, well, there's a skill missing. There's something about the cue that's not leading to success. So to some extent the CPS apt kind of sounds like if you, you started doing an isca, but you kind of stopped at the questions, what's the behavior? And what happens before the behavior? And then you're like, I don't want to do any more of this assessment. Let's just, let's just make a treatment from here. And it kind of always looks somewhat similar in that regard, but I think it does point at, you don't really need to know the function sometimes as long as you know that whatever is happening is not working and you need an alternative plan. So again, like we mentioned, I think this is all very much in line with some of the work that's being done in bst. It does feel very much like operant behavior, though. Again, when we talk about operant behavior, or at least psychology talks about operant behavior, they're going to be talking about some of the things, some of the treatments that we'll be discussing in the back half of our episode. And I think that is a misunderstanding. And, you know, we'll talk about how maybe we could add more to the collaborative problem solving research and literature discussion with our understanding of behavior and psychology can add what they know about working with families and making therapeutic rapport and building those skills. So in any case, I think it definitely falls in line with something that could be under the umbrella of behavior analysis, something that is important, certainly socially significant, being able to work with others. And I think it follow follows the same pattern as we would expect to see with any ABC kind of behavior consequence with a huge focus on the antecedent piece, though, again, it's not an antecedent manipulation per se, as you'd think of. All right, so some resources, some basics. And now that you know a little bit about that, let's take a break and when we come back, we'll Talk about two uses. More, more recent uses of collaborative problem solving. We'll be right back.

C (36:17)

Hi. Do you want to be a bcba, also known as a board Certified Behavior Analyst?

A (36:22)

Sure. We all do.

C (36:24)

Now you can come to Regis College in Weston, Massachusetts to get your graduate degree just minutes outside of Boston.

A (36:30)

Choose from any one of these courses.

C (36:33)

Masters of Science in Applied Behavior Analysis.

A (36:35)

Master of Science in Special Education, dual degree in Special Ed and aba.

C (36:40)

Or be eligible for your Postmaster certificate.

A (36:43)

You can complete your degree and be ready to sit for the exam in two years.

C (36:47)

And our 2022 graduates had a 92% pass rate on the BACB exam.

A (36:53)

Come enjoy approved fieldwork placements, ethics, mini handbooks, PhD levels, professors, small class sizes, and a service trip to Iceland if interested.

C (37:02)

And don't forget, our program is accredited by the association of Behavior Analysis International, or ABAI, as a Tier 1 master's degree program.

A (37:10)

Don't delay. Supplies are limited. Learn more at regiscollege.edu.

C (37:15)

Again, that's www.regiscollege.edu regiscollege.edu. one more time, www.regiscollege.edu.

A (37:27)

See you there.

C (37:28)

Bye.

B (37:34)

And we are back talking about collaborative problem solving. But before we start collaborating on the problem solving, we're going to solve a problem for you, which is getting enough ces. And if you're having that problem, well, don't worry. ABA Inside Track is ACE and QUAPA approved. And by listening to this episode, you can earn one learning ce. Hooray. All you need to do is finish listening, then go to our website, abainsidetrack.com or click the link in your podcast kind of notes section and you'll get right there. But you're going to need some key information, including two secret code words, the first of which is pellet, P, E L L E, T. And Jackie, you're talking about like a heat pellet when.

C (38:18)

You thought of that code word pellets. Because I'm sitting here and wishing that I had put pellets in my pellet stove because it has run out and I am very chilly.

B (38:27)

Oh, no. Well, let's get. Let's keep going on. Remember pellet. But we gotta get. We gotta get moving so we can get Jackie out of her cold, cold basement. All right, so, Jackie, you are gonna take the lead on a discussion of collaborative problem solving in an outpatient setting.

C (38:43)

Yeah, So I don't think my article should be an article that you read solely. Right. So I didn't really understand what collaborative problem solving was until I read Diana's article because it doesn't really tell at all what it is, except that it is a way for parents to work with their children to understand challenging behavior. Right. They do a kind a tiny little dig with aba. They say that CPS is different from behavior analysis because it does not use incentives or ignoring to motivate children's responding, but focuses on underlying skill deficits that contribute to challenging behavior. And you know what, I see where they're coming from. And I agree. Right. Like they probably are putting some consequences on those challenging behaviors right in the setting, but they're not focusing on it. So that's one thing that I really love about this is that they're really focusing on teaching the skills. And I love when Diana talked about how parents should view their children. Right. Not as like, oh, they're doing something bad, but it's just a skill that they need to learn.

A (39:48)

Yeah.

C (39:48)

And I think, you know, behavior analyst really do a good job at that most of the time. But we're not always great at it.

B (39:55)

Yeah.

C (39:55)

Right. So I think that's one thing that we could learn from this literature is to really start focusing on positives. Right. And skills that need to be learned instead of trying to decrease behavior. That's already happening. So the research, they don't really tell us what but they say research has shown that CPS has been really effective with individuals with group therapy, with family therapy modality. When CPS was delivered in an outpatient setting, they saw an improvement in children's behavior, they saw an improvement in parent child relationships, they saw decreased parental stress and an increase in children's executive functioning skills. Right. They also looked at another study that showed that CPS kids that went through CPS did better with parent compared with just general parent management training for odd because with CPS you improve the parent child relationship and thus you increase the clinical improvement that they saw. So here what they wanted to do was to examine the effects of CPS across a wider range of both self diagnosed kids or undiagnosed kids. Here the first step, again, like we know where parents are top five skills, but they just say what they say in my article is language and communication, social thinking, emotional regulation, cognitive flexibility and attention and working memory. They do an interview to identify where the challenging situations are occurring and what skills could be deficit during that time. And then they choose one problem at a time and engage instruction problem solving conversations where they talk about why the trigger is difficult and come up with a solution with the child and skills to de escalate the situation. Right. So Here they say that they're going to look at use a pre post design to look at child outcomes and parent child relationships and consistent with prior research and intensive home based family therapy. They want to see improvements in outcomes in an outpatient setting and they think that it will be predicted by the extent to which the parents achieve the learning targets in the parent training and they think that that will correspond or correlate with child improvement. So the data were collected as part of an outpatient client that's aligned with, as an outpatient clinic that's aligned with an academic hospital. I'm assuming that it's Mass General. Right. I'm whispering that because they don't say that, but I'm, I'm assuming, I'm assuming it. So over the time of the study, which was November 2013 to November 2017, there was parents of 120 children, 37 girls and 83 boys between 3 and 19 years of old. The old, the mean age was 8.7. They attended an initial visit in 96%. So one hundred and fifteen out of one hundred and twenty of the kids continued to family therapy afterwards. So data from all the families were included. No one was excluded from the secondary analysis. And so for that secondary analysis they had 120 kids. And the average number of sessions that was attended was 19 and the median was 13. So by December 2017, which was the end of the study, only 14 of the 115 kids were discharged. Right. So that's actually kind of low.

B (43:17)

Yeah.

C (43:18)

Right. But 74 had discontinued treatment for a variety of reasons. They didn't tell us why. 27 were still enrolled. Oh, and out of the 14 that were discontinued, 10 had remitted, meaning they had to go back into inpatient. Three were transferred ELSER and 11 or, and one needed a higher level of support. So they really only had 14 kids that were discharged from treatment effectively after those years once they did the secondary analysis. So that made me go, huh, Right. It's not very many.

A (43:55)

But the ones who discontinued treatment, do we know why?

C (43:59)

No. There was a discontinued.

A (44:01)

As though they're like, we don't need this anymore.

C (44:03)

So what the, what the authors had posited in the discussion, we're not there yet, but I'm going to tell you, because you asked, is that they said maybe they thought they didn't need it anymore, maybe because it wasn't quick enough, they didn't see immediate results. Right. We have that problem with our treatment as well. Right. So if you go in and you're like, it's not happening. It's not happening. And then people quit. They said that the time was sometimes an issue because you know, you have to go to multiple sessions, but they actually don't know why. But those were some of the positive reasons why kids left treatment. But they don't know. So mostly all the, mostly all the kids were mostly were white. 83% had clinically relevant symptoms at intake that referred to AD, HD and anxiety. Many of the children in the study were taking medications and they provided all of the medications that kids were taking. I didn't think that was relevant for us, but it's relevant for the study. And one thing I did love is that they looked to see if kids were taking less medication at follow at the end of the study and four of them were taking fewer meds. But mostly there was no change. So family therapy occurred at least for at least 30 hours in CPS. And they mostly work with the parent. Right. That would make sense because it's parent training, but sometimes they work with the child to clarify different situations. So they did an intake questionnaire and then they did three months post intake and then did a youth outcomes questionnaire before every session. So they had an ACEs collected at intake. And that's the. Oh my God, I always forget it. I pulled it up because I was Childhood Experiences Scale. So they did that at intake. They did the collaborative problem solving and here it's an impact measure, the parent version. They also did the Parent Child Relationship Inventory. They did. And they did the youth outcomes questionnaire again. So when we're looking at this, know that nothing is directly observable. Right. So it's all questionnaire based. So how behavior analysis could help. Right. I'm not even offended by what they said there because I agree in the front. But how we could help is maybe we could help you take actual data on what's happening. Right. We could do some observable measures or we could teach the parents how to do that. So instead of doing these. So they looked at the changes across sessions and across 12 week treatment sessions. So according to published guidelines for clinically significant change, 22% of the children had recovered by the time of their last session. So 19. And another 17% had improved. 50% were unchanged and 11 had. 11% had deteriorated from first to last treatment. Okay. So then after three months of treatment, they looked at how people were doing. And so that was 12 weeks and now three months. 19% of the children were classified as recovered. Another 12% improved, while 62% had symptoms that were unchanged. And 7% had deteriorated. Deteriorated. So what I love here is that they did not mask that it wasn't perfect.

A (47:24)

True.

C (47:25)

Right. So they were home. It isn't home. Yeah. So they, you know, this is a new thing where we're working with a wider population than what they had previously done. This is not inpatient. This is outpatient. Outpatient. The parents are doing all of the work. And so it wasn't great. But what they did say is that parents reported that they learned more from the beginning to the end of the treatment and that they, they posited that CPS was probably as effective to other behavioral interventions. Yeah. So they, their, their full take home point is that this could be really helpful because parents parented better following CPS and they might be more consistent. So over time kids might get better. Yeah, yeah.

A (48:12)

And that was one of the main concerns they had in my article about the parent training is the parents just couldn't do it.

C (48:20)

Right, Right.

A (48:20)

So this might have a better treatment fidelity.

C (48:24)

Right.

A (48:24)

Than the other approaches, which is really important.

C (48:27)

Right.

B (48:28)

And speaking of people who kind of like CPS model, let's, let's wrap up with the Stole et al Article, which is a case study. So you're only going to get a couple there. There are some graphs mostly around something observable, which is number of restraints used in a youth residential care facility or a number of youth residential care facilities. And you're going to see kind of the same theme that we've been looking at in some of these other articles, a slight dig at behavior analysis. Some of it founded, some of it possibly showing a misunderstanding of a couple principles that perhaps would be a good addition to the CPS literature and a overall enjoyment of the use of CPS collaborative problem solving with youth versus other systems, in this case, point and level systems. So point and level systems are used. They're sort of like the historical behavior plan used in a lot of youth residential facilities. They've been used for a long, long time. They kind of a long history. And basically it is.

A (49:32)

I was just gonna say we would classify these as behavior modification.

B (49:35)

Yes.

A (49:36)

Procedures. Right. They're not function based. They're not individualized. They're just layering on additional, often arbitrary reinforcers or removing those reinforcers based on behavior, but not tied into any larger contextual understanding or analysis of why someone's behaving the way that they do.

B (49:58)

Exactly. And those are often referred to as universal point and level systems versus an individualized point and level system, which would be more like a more meaningful token Economy versus a universal P and L system, which is, like Diana said, it's just arbitrary. You get points, you lose points, and you have certain privileges if you get enough points. But a lot of the challenges. And again, you know, it's hard to tell when reading these whether this is a situation of folks who are not trying to be too mean to behavior analysis or people who study, you know, human behavior, or whether it's supposed to be like, you know, when you're trying to really get at somebody, and rather than just come out and be like, this person's an idiot and stupid, you just sort of keep hammering home the same one or two points that are like, this is a misunderstanding of what it is, but it sounds good to anyone else. So if you keep saying it long enough, everyone's going to turn on whoever you're talking about. Even though Your point's not 100% there, it's hard to tell which one this one is. Because, yes, like Diana said, none of us would say that a universal point and level system is an appropriate or behavior analytic procedure. However, to then talk about it like, you know, that's what they have, the poor behavior analysts, they have these systems, but it's really hard to get them up and running. And they're not always individualized. Oh, it does sort of paint us all with a brush of like, we're mean, unfeeling, stupid, idiot, incompetent people. Maybe that's not what they meant to do. But I kind of was getting that vibe. I know, Jackie, you said you were getting a little bit of that vibe in the last article. Let's save a little bit of that to dissemination station. But in any case, yes, when we're not using individualized systems, it is really easy to see how a place working with staff that are going through high turnover with youth who are in a residential facility, probably because their problem behavior is very intense and potentially dangerous. When they're just getting points sort of given and then taken away and privileges given and taken away sometimes by different people, it is going to become very dehumanizing to the clients. It is going to become very frustrating to the staff, and it probably is not going to result in any learned skills because at best, the individuals learn what they need to do to earn enough privileges. And some folks are just going to say, I'm not engaging in this system because what's the point? Even some of them saying things like, what's the point of doing this system? When I leave here, this won't be in place. So even if you have good reinforcers for certain skills, these aren't, you know, these are, these are youth very aware of the environment they're going to go back to and their privileges will no longer be contingent. So there's no natural, you know, natural reinforcers for some of these behaviors.

A (52:37)

When those types of economies work, they work because they're closed economies. So once that's gone, they're not going to maintain responding.

B (52:44)

Yeah. So basically what this article lays out, and actually I think it's more interesting in some regard for this process than it is for the CPS of it all, is how did they move from a point and level system to more of a use of the neurosequential model of therapeutics, which basically just boils down to. That sounded like a bunch of gobbledygook when I read about it. But it basically boils down to a more trauma informed approach that uses collaborative problem solving methodology as well as some trauma informed cognitive behavior therapy. And basically what it boiled down to was at various times the residents and the staff saying, we hate everything that's going on. The leadership kind of falling in love with various models that sounded more touchy feely or kind of to some extent, and then sort of slowly adding in these procedures or different procedures to staff. So they find the people who are most interested in making change, the change agents, they train them up. They took a lot of time talking to the staff about what they were most worried about and then tried to resolve any concerns they had. A lot of it being the staff saying, if you take out a system that I don't like, but it kind of keeps me safe, I might be in danger. I don't want to just have nothing or I don't know if I trust these new systems. But they did a lot of slowly rolling things out, rolling them out at certain facilities, making sure there was lots of time for training, a lot of opportunity for staff to share their thoughts on the process, and saw a real decrease in the number of crises and restraints as they started adding these processes in. And again they get a little bit into the CPS procedures. But again, it's not much more than I think what we talked about. Sort of the idea that, you know, hey, let's teach everyone that the cause of challenging behavior is not some sort of evil within the child. It's because the youth doesn't have the neurocognitive skills to do better. You can't just say the skills to do better. Authors, come on, throw us a bone here. But again, they, they can't do better. And that the best way to address challenging behavior is looking at what skill deficits might be getting in the way of meeting some of the behavioral expectations and working to build that skill. Again, not exactly the same as a function based discussion of problem behavior, challenging behavior, but a much more skill based or acquisition based lens to look at where problem behavior does or does not develop. And also looking at holding youth accountable rather than just by holding accountable as. And they'll be severely punished for their failures. But by understanding that at the end of the day, the youth do have to help you come up with solutions to problems. And that's the accountability is that they will be expected to collaborate and do collaborative problem solving to some extent and come up with a solution and try it out. And they'll keep trying it until a working solution is found.

A (55:35)

I like that.

C (55:36)

Yeah, I like that too.

B (55:38)

Again. But then here's another phone quote I found. CPS teaches staff the limitations of operant approaches and an effective intervention that gets behavioral expectations met without rewards and punishments. And while also building skills and relationships, which I is, is such a failure of understanding operant behavior in that the students or the youth are still engaging in a skill that will result in punishment or reinforcement. Right. So I think just because you're not getting tokens or whatever, because you made your bed or you came up with whatever, a plan around why making your bed might not be appropriate or how you could, you know, engage in this skill, using collaborative problem solving doesn't not make it operant behavior.

C (56:19)

No. I think that's our fault though, to be honest.

B (56:22)

Oh, I think so too.

C (56:24)

Yeah. We've done a bad job of talking about how our behavior, how our sciences can, you know, how our science can be applied in all these areas. So I don't think this is a problem with cps, you know, and I don't love it either. Right. But they are, they are pointing out places where we haven't done a great job of explaining our sciences to other scientists.

B (56:46)

Yeah.

A (56:46)

Oh, and if I tried to write an article about collaborative problem solving, I wouldn't say everything the right way according to them.

C (56:54)

Right.

A (56:55)

Either. Right. And I would say things like this is too mentalistic or this isn't operationally defined. Well.

C (57:02)

Right.

A (57:03)

And for me that's like a failing. But it also might be a failing on my part of my understanding of the technology in. In use.

C (57:12)

Right. So one thing I would love, I feel like we're popping in a dissemination station.

B (57:17)

Well, if you. I, I do want to get there Let me just wrap up the article though. So again, they continued the process by using cps, by using cbt, by using trauma informed care. And again, they saw, you know, for two out of the three, they did see pretty good decreases in the monthly restraints per quarter. Again, this isn't a good research article in that there's a lot of, you know, failures of experimental control. It's not trying to be, it's a case study. But some of the things I think were really interesting were to talk about some of the determinants as to why some of these changes might have occurred. And they don't say, oh, CPS is the best. I think, you know, we're all pretty, pretty positive on it. But really looking at the idea of there was a lot of interest from leadership. There was a lot of availability of leadership to implement something this big. There was a lot of communication, there was a lot of training. I think CPS lends itself. Even though they go over some of the, you know, the training that they had to do, it wasn't intense hours. 16 hours for basic training and then 12 online coaching sessions to use CPS fully and then some advanced training, that's another 16 hours. So that's not a crazy amount of training to do in your organization. If you were like, we're doing cps, we're going all the way in. And frankly, for folks, you know, like behavior analysts who have a lot of training in this area, if you look at some of the CPS material, if you look at some of the online trainings, if you read the book, I don't think you would see it as something that's well outside your realm. If most of what you've been working with are individuals who have, you know, like pretty advanced verbal behavior repertoires, I think this is something you probably could do with maybe Even just the 16 hour basic training pretty nicely. So again, this was a kind of a nice example of how one can switch your systems. But it does require a lot of training and support. So if you're thinking you like cps, okay, well, there's still some training that needs to go into it and there's still some fear. I think even with something that has as much positive feedback as the parents in the previous study and the staff in this study, it's still, there's still a little bit of a fear of going away from what most people know, which is. But if I don't make sure that I have some sort of like carrot or stick, everything's going to go to hell in a Hand basket the second I try to change anything. And I think part of CPS is similar to. I think the work in BST is understanding that no one's trying to hurt people or be bad. No bs. I think like bst, I think we talked about there's some similarities between BST and CPS in terms of, like, the understanding of sometimes you need to back away. That is all about, you know, developing certain skills. I think there's some similarities there. And anyway.

C (59:50)

With behavior skills training.

B (59:52)

Yeah. Pfa.

A (59:53)

Okay. BSC is behavior skills training.

B (59:56)

Yeah. That's why I said. That's why I said. I said bsc.

A (59:59)

Okay.

B (60:00)

So I think with that. With that all said, there's some positivity, but we had some thoughts. Let's move into dissemination station. So, Jackie, I have a whole tire. I'm not gonna read my tirade.

C (60:14)

No, but I. I'll start.

B (60:17)

Okay.

C (60:18)

One thing that I think we could take away. So dissemination statement should talk about how we as behavior analysts can utilize this research. Right. So one way I think that we could really utilize this research is take the points that we really love about it. Right? So when they're focusing on the parent training and focusing on skill deficits instead of focusing on the challenging behavior, focusing on using language that's understandable, focusing that we're not just. Right. We shouldn't just be working with people that have a diagnosis of autism, but we could use our science in more applicable ways to people outside of the neurodivergent population. We could be better at marketing. They're doing a great job marketing this. So I think we as behavior analysts can use this type of research to make our research better. Right. We can't do anything about the little digs that they gave for behavior analysts. Right. I against. We see them, but they're our fault because we're in charge of marketing our own field. And we've done a bad job.

B (61:27)

Right.

C (61:28)

We've done a bad job at playing nicely with psychologists. We've done a bad job of playing nicely sometimes with caregivers, as evidenced by Taylor and colleagues in 2019. Right. So I think, yes, they did give us some slights, but fair. Right. That we're not the best. And so I think these articles really just show us how we can be better and incorporate and maybe collaborate with other fields on how. Okay, we can do these things that might make CPS better. Right. We can start. We can teach them about how to observe behavior. Maybe we can, you know, teach them better ways to measure behavior instead of just using questionnaires. And then they can teach us how to work better with families in a more approachable way. So I think, I think the real way to look at this is how we can work better together.

B (62:22)

I think there's a lot. I think there's a lot of growth in, you know, how one could or when one should use collaborative problem solving, because I don't think you need to use it all the time. I mean, and there are similar procedures even in, like, if you read Nadowski's flexible and focus. So working on executive functioning, her problem solving or her, like, training of problem solving skills is not that dissimilar to what you're going to see in collaborative problem solving. So if anyone's, like, listening to this episode, they're like, collaborative problem solving. Does it have the magic key? I mean, it does sort of have the same, that same sort of allure. I think that that bst, the PFA work has of like, it just kind of works and everybody loves it and it's magical. You know, there is a lot of science behind it. It's not brand new. I think it is that marketing piece that really has been speaking to people. But like you said, Jackie, it's not like when you look at the results here, there's not a ton of room for growth. I mean, a lot of the research sort of points to people really like this, and it's as good as whatever came before. To which it's like, wait a minute, people really like it, and it's just as good as the stuff that's getting, you know, 20 improvement. There's. We could do better than that, can't we?

C (63:28)

We can do better. But I think what their point is is that maybe people didn't like the things that were happening before because we as behavior analysts didn't really validity until recently in a way that was. In a way that was understandable. Right. So I think that is as one way saying, like, maybe we should be talking to our constituents, our consumers, our parents, and taking out the things that they don't like and only adding the things in that they do.

B (63:51)

Oh, I. I would say 100%. If you have two treatments that are exactly the same in terms of effectiveness and people like one more than the other, that one wins the contest. Because if people like something that's better, even if the results are like, well, you could say, well, technically neither of these are that much better.

C (64:07)

Right.

B (64:07)

If somebody likes it, that. That wins hands down, because you need people to want to engage in the treatment. And plus some of the longitudinal Work around things like. Well, most of the treatments that have sort of only been getting more and more research in the past decade or so. Some of that information is not. Not there. It's just starting to come out like, like, you know, the study that you were discussing, Jackie, the idea of, well, now that parents can do some of this collaborative problem solving. Yeah. Maybe their child is not discharged from the clinic in three months or fully discharged according to some questionnaire, but they have years and years and years. You know, sort of like when you move a rocket ship, you know, one degree, that's going to be the difference between you land on a planet and you fly out into space. The same idea of parents are using some of these techniques. We saw this with positive parenting, too. It's not about you use these techniques and then it fixes everything. It's about using them over a long span of time, the developmental life of your child. You know, and we don't know how much, what dosage, over what period of time these skills need to be developed and used to result in what we might consider sort of, you know, socially significant change in psychology. They're often using these questionnaires. Hey. People's opinions of the child or possibly the child's opinion of their own behavior, which is not always as accurate as how many times are they engaging in a screaming match with their parent or how many times are they refusing to engage in chores around the house. Right. That's something observable.

C (65:40)

Yeah. And I'm thinking about. I. I'm thinking about some of the, the clients that I knew that had some odd. I had an odd diagnosis, and they actually would report that they are worse because. Right. They're really sensitive to those situations. And so one of them was saying, you know, like, oh, I'm always talking out in class. I'm doing it all the time. I don't think I'm getting better. And his teacher was like, he's actually fine. He's no worse than anyone else in the class. So I think he's just sensitive to that because, you know, had been brought to his attention. And so that could be, that could be at play too, which would skew the questionnaire. Right. So CPS could actually be better.

A (66:24)

That's interesting.

C (66:25)

Right. Looking at that. But I mean, that's one. One client right. Out of many. But I mean, that could be happening anyway. I think that's a good, That's a good summary, man.

B (66:36)

Yeah, I mean, I, I think it's, it's. It's sad. It's sad. That our field's so bad at marketing. And it's sad that some of these fields that are better at marketing seem to have just, I don't know, taken the behavior ball and gone home and left us feeling in the lurch. But hey, you know what? There's room for collaboration. I think there's room for improvement. I think if folks really want to improve their treatments, it's going to be about working with lots of people. I think we're all seeing that more is that if any one field thinks they have the answers, they clearly don't. No one field is just going to solve all the problems. And I think more interdisciplinary collaboration. And this one should be easy collaboration. Psychology, behavior analysis or behavior analysts. That's not that far off. Right?

C (67:22)

Right.

B (67:23)

Well, I am, I like cps. I think it's neat. I've. I've certainly kind of tried to learn more about it. I've used it a few times in sort of some low impact cases because it's not something I've been like formally trained in. But again, it's not. I don't think so. Outside the realm of behavior analyst who's worked more with individuals with verbal, more complex verbal behavioral repertoires or worked with students with adhd, sort of using some of the flexible and focused problem solving charts before. But I'd love to see more people using it. And if you're a listener who said, could you do an episode on odd? I guess we kind of did. So. Hooray. We did two in one.

A (68:01)

Yay.

C (68:02)

Good for us.

B (68:03)

Yay. So hopefully we'll have more research and we'll be able to talk about it. And hey, maybe, maybe we'll be throwing it up in one of the various books on the subject. Maybe it'll make its way into this year's book club voting poll. Maybe we'll come back and do a little bit more on it. All right, so we're going to wrap up the show. We liked cps, but we need to get ready for the new year. You're listening to this in the new year, but we have to go get ready for our real life new year pretty soon. Thank you so much for listening to ABA Inside Track. We would appreciate if you left us a review and subscribe to the show. You can find us online@abainsettrac.com where you can find links to all of the articles we discussed as well as to purchase ces. I guess you want a second secret code word now. Let me give it to you. It's antique, A N T I Q U E. And remember, if you click on that link and you'd like ces, you can get one Learning Cell EU or you can get one Trauma informed CEU if you're getting from the ibao. There's so many different types of CES Now. I keep updating the website so I try to keep that up to date for folks. But just to specify that some final things to know if you're interested in even more ABA Inside Track or you said I do want to vote for a CPS book in your book club poll. Well, sorry, you're only going to be able to do that if you join us on patreon.com abainsidetrack where you can get all these episodes a week ahead of time, where you can get discounts for ces, where you can vote on our listener choice episodes, you can vote on our book clubs and you can get free CES for listening to those episodes at the $5 level or $10 level if you want everything including our book club. So if you just want to vote and then listen to the book club when it comes out, $5 level. But if you want those CES and you want to hear them the second they come out, that's our $10 level@patreon.com abainsidetrack all right, and some final thanks. Thank you to Dr. Jim Carr for recording our intro outro theme, Kyle Sturry for our interstitial theme, and Dan Thavett of the podcast Doctors for his amazing editing work. We'll be back next week with the big 300 episode of ABI Inside Track, but until then, keep responding.

C (70:15)

Bye bye Bye bye.

A (70:18)

Sam.