The ABCs of OTRs

I really hope Hosp, Hosp, and Howell don’t get mad that I used the title of their book as the inspiration for this. If you want to learn all about curriculum-based measures I highly recommend their textbook!

Origin Story

I was asked to present at the Pennsylvania Training and Technical Assistance Network Conference in the Fall of 2023. I brainstormed ideas with Jared Campbell and Charles Mohler - who were helping coordinate the conference - and soon realized that word problems, math facts, and curriculum-based measures were already accounted for in presentations [and single-case designs were not going to be super interesting to teachers :]. We started kicking around other ideas and I mentioned how much focus we place on preparing our pre-service teachers to deliver opportunities to respond and behavior-specific praise. It led to a fruitful conversation around how opportunities to respond are a salient component to effective instruction, yet many details must be considered. So here we are today!

Terminology

Opportunity to Respond

Acronym: OTR

Definition: OTRs can be described as a three-part sequence: (a) presented stimulus, (b) student provides a response, and (c) feedback (see Ferkis et al., 1997; MacSuga-Gage & Simonsen, 2015).

Examples:

• Teachers presents the numeral 7 to students and asks, “What number is this?”

• Teacher directs students to use their mini white boards, “First, sketch a graph that represents a positive slope. Second, sketch a graph that represents a negative slope. Please label both.”

• Teacher presents the problem 10 - [ ] = 8 to students. “Hold up 1 finger when you find one way to solve this problem, hold up 2 fingers when you find a second way to solve this problem.”

None Examples:

• “Any questions?”

• “The slope is rise over run, right?”¹

The logic behind OTRs is simple and can be tied back to Antecedent-Behavior-Consequence.

Antecedent —> The prompt is given for students to think about.

[wait time, allow students to all be able to think]

Behavior —> This is what the student does (verbal, gestural, written).

[wait time, allow all students to respond and survey responses as a form of formative assessment]

Consequence —> Teacher feedback on student thinking.

Replicating this ABC process repeatedly is how students engage in thinking, practice, and learning.

Types of Opportunities to Respond

One useful framework for using OTRs is to consider the mode of response students are expected to display.

Verbal OTR

Classwide Responding

OTRs can be used to have students provide a verbal response. One way to approach this is through choral responding. For example, when introducing new mathematical vocabulary it is critical that students can read, spell, hear, and pronounce the new term correctly. So when introducing a new word, I will have it displayed on the board and pronounce it correctly myself. Then I might prompt students to (1) repeat the word after me, (2) now say it loud, (3) now say it soft, (4) now whisper it to your shoulder partner. This would take all of 10 sec, and I just provided 4 verbal OTRs for students to practice pronouncing the new word (quick note here, examples 1-3 above would be choral responses whereas example 4 is something I’ll describe below).

When using classwide choral response, the teacher must consider the expected response from students. Classwide choral responding works well when the expected response in one word (or a short phrase). However, longer responses (that can differ) become messy. For example, posing the following as a choral response, “Explain why the problem 2 + 9 can be solved by doing 9 + 2” will lead to jumbled responses from students and make it difficult for the teacher to discern student thinking. In many instances, teachers can preplan verbal questions they aim to ask related to items or math concepts being discussed that would lend themselves to choral response OTRs - it can be trickier to do on the fly.

Peer-to-Peer OTR

OTRs can be used to facilitate student dialogue. As mentioned above, sometimes we aim for students to explain their mathematical thinking - thus a word or short phrase is insufficient. We could go the route of calling on individual students to provide explanations during classwide discussions; however, this limits the opportunity for other students to engage in explanations. A solution here is to use peer-to-peer OTRs. The same framework would be used, a prompt is provided and the direction would be to turn and talk to their partner.

I can imagine some people cringing at the recommendation to use peer-to-peer OTRs as they conjure up every negative experience they have witnessed from “turn-and-talks.” What could go wrong?

• Many partner groups look at each other and say nothing.

• Many partner groups start talking about the dreadful end of Game 6 from the Thunder-Mavs series.

• Partner 1 does all the talking and Partner 2 says, “I agree.”

• Partner 1 talks and then Partner 2 talks and there was no active listening by either partner.

Teaching students to structure of peer-to-peer OTRs at the beginning of the year and then reinforcing students for their active participation in these will go a long way. This is anecdotal, so sorry, but I was a fifth-grade general education teacher. The special education teacher I co-taught with and I noticed how dreadful these peer-to-peer OTRs went at the beginning of the year. We decided to model it for them. We turned our bodies and each modeled providing a response, engaged in active listening, and asked questions to each other about our responses. We then allowed students to practice peer-to-peer OTRs with prompts they easily could have a response to, “What is your favorite item to bring for lunch and why?” We attempted to reinforce active listening as well. When we would bring students back from peer-to-peer OTRs, we would ask students to share not what they said but rather what their partner shared. These small routines at the beginning of the year went a long way in enhancing the math conversations that were had.

I have three other small suggestions that are low investment on the teacher’s end but can go a long way to enhance the benefits of peer-to-peer OTRs. First, after posing the question give students time to think of a response. In some instances, you might even have them write something down (i.e., written OTR, see below) before turning to engage in a peer-to-peer OTR. This will reduce the likelihood you run into the first issue above, they both turn and say nothing. Second, sentence stems can be extremely beneficial to students for specific types of peer-to-peer OTRs. Sentence stems (or frames) might support higher-quality explanations (see Rittle-Johnson et al., 2017). Third, active supervision is critical here. Move around the room to groups, reinforce active participation in the prompt and redirect students who are perhaps discussing the Thunder game instead of math. This is also an excellent time to identify students who shared useful information that would be beneficial for the entire class when you pull students back in.

Self-Explanation

Rittle-Johnson et al., 2017 reviewed studies that had students engage in self-explanation of their mathematical thinking. Self-explanation is merely students creating explanations for themselves to make sense of new information. The authors recommend four starting places for teachers who aim to incorporate this into their practice:

• Students need to be taught what makes up a high-quality explanation. Modeling high-quality explanations and then providing feedback to students on their explanations can enhance their ability to use this in the future.

• Teachers should design explanation prompts so they do not sacrifice attention to other important content. Providing more “structured” prompts for students to complete that would align with an explanation appears to benefit this process.

• Provide prompts that have students explain correct information.

• Provide prompts that have students explain why common misconceptions are incorrect.

Written OTR

I am not going to spend as much time on this one. A written OTR is a student writing a response. This might be paper and pencil, a mini whiteboard with a dry erase marker, or others. The one caveat here is to think about the expected response because this will dictate wait time and the rate at which you can provide OTRs. If the response is short, 6 x 5 = [ ], in this case the expected response is 30, then the wait time can be shorter and the rate at which OTRs are delivered can be higher. Whereas, if the expected response is longer, for example, “Tim was provided the problem 7 + 1 = [ ] + 2 and he wrote 8 in the box. Can you help explain to Tim why this is incorrect?” you’ll need to provide a longer wait time.

One thought to consider is if teachers provide written OTRs that require longer written responses, the same advice for “self-explanations” apply. Written communication and verbal explanations are linked, thus modeling written responses, providing feedback, and using more structured prompts initially are going to be beneficial.

Gestural OTR

Similarly, I am not going to spend as much time on this one. A gestural OTR is a student providing a response that is “gestural.” This might include thumb up/thumb down, pinch cards or response cards (see here for some example cards), or others. Also, I do not know where to lump in some of the technology-based OTRs (e.g., clickers) so I’ll throw them in here as well - just know these are a form of an OTR if you use them!

Cognitive OTR?

I am a little bit nerdy so I am including this into the conversation. In the OTR literature I have read, I have not seen “cognitive” OTR be used as a grouping. My definition would be that a student is prompted to “think” of a response but then NOT prompted to actually verbalize it (because this would be a verbal OTR) or write it down (because this would be a written OTR). What got me down this rabbit hole of thinking were two things. 1. Michael Pershan and his book because he described a structured routine he uses with students when studying worked examples. 2. Christopher Skinner published two papers on cover-copy-compare (read here about cover, copy, compare and totally free resources to build fact fluency developed by Brian Poncy at Oklahoma State [please forgive him for being at the wrong Oklahoma institution]) that stuck with me (paper 1, paper 2). In paper 2, Skinner identified verbal CCC (verbal OTR) yielded faster learning rates than written CCC (written OTR), which he hypothesized was because in a fixed amount of time students were able to get more practice trials on verbal CCC because writing answers takes more time. It paper 1, he hypothesized cognitive CCC might also yield quicker response times because students are just thinking of their answer and not prompted to say it out loud - and to me, this would “appear” to fit the definition of an OTR but the response mode is “cognitive” because it is just them thinking.

I am going down a tangent here so skip this if you don’t care. Verbal CCC can increase response rates BUT teachers are not provided with a permanent product to serve as a “check” that the student participated in the intervention. If you see stagnant growth using verbal CCC you might consider moving back to written CCC to see what happens. Another option you can consider is student choice. Once students are taught how to engage in written CCC and verbal CCC you can allow them to choose the method they want to engage in. Choice can be a powerful motivator and help avoid habituation (i.e., bored of using the same procedure repeatedly).

Rates of OTR

Now that we have covered the definition of OTR and discussed different examples of OTRs that can be used, you might be asking yourself

How often should I provide OTRs?

Stitcher et al. (2009) observed in 35 general education classrooms across four elementary schools and found the average rate of OTRs to be 2.62 per minute.

This one is not AS RELATED but Sutherland and Wehby (2001) reviewed research on the OTR rates in intervention studies focused on students identified or at-risk for emotional or behavioral disorders. They identified OTR rates varied from .02 to .16 per minute across the six studies.

Last, MacSuga-Gage & Simonsen (2015) evaluated 15 experimental studies focused on manipulating teacher use of OTRs and evaluated student outcomes. Based on their findings, they provided tentative recommendations that teachers should shoot for 3-5 per minute because these appeared to correlate with more positive outcomes.

My take on the OTR dosage conversation. If we tie this conversation to the instructional hierarchy (see here for one description but if you are uber nerdy you’ll like this one more), we might think about varying our dosage of OTR based on where student current knowledge is of the concept. If we are focused on the initial acquisition, the 3-5 OTRs per minute seems like an achievable and good benchmark. However, if fluency building (i.e., speed + accuracy) is the goal this rate is much too low. To make this more concrete, let’s consider a student aiming to build fluency in their multiplication facts. If we are hitting 3 OTRs per minute, that would equate to the student attempting one math fact (e.g., 4 x 5) every 20 seconds. This would yield a low rate of practice. For hypothetical sake, presenting a problem every 6 seconds would be an OTR rate of 10 per minute.

How does this manifest into the Matthew Effect? If we use an intervention procedure such as cover-copy-compare or explicit timing (or just having students practice some math), students who have higher levels of initial fluency get higher OTRs per minute.

• Student 1. In 2 min they practice 40 problems. This equates to ~1 problem every 3 sec, which equates to 20 OTRs per min.

• Student 2. In 2 min they practice 12 problems. This equates to ~1 problem every 10 sec, which equates to 6 OTRs per minute.

OTRs and Behavior Specific Praise (BSP)

What is behavior-specific praise? A positive statement directed toward a student or group of students that acknowledges a behavior in specific, observable, and measurable terms. Saying, “Great job using the communicative property to solve that problem” is an example of BSP whereas “Great job” is considered generic praise.

BSP is beautiful because it is so simple to provide yet can enhance student behavior to support the classroom environment and learning. Loads of research in other disciplines have evaluated the ratio of BSP: corrective statements. In school settings, this has also been evaluated and many recommend teachers aim for 3 to 5 BSPs: 1 corrective statement (although it is not clear there is an upper limit so shoot for more!)

So how do OTRs and BSPs relate, see below.

One of the beautiful side effects of enhancing teachers OTR rates is the added benefit of also increasing BSP rates. As we can see, each time an OTR is presented and students provide a correct response there is the opportunity for the teacher to provide BSP!

What do we do if there is an incorrect response (see below)?

Incorrect responding is going to occur and that is okay, it presents feedback to the teacher that they can enhance their explanation. The key is to ensure an incorrect response is paired with feedback and another ABC cycle is provided to ensure a correct response is provided - which once again allows for BSP!

How to Enhance our OTRs

Remember Art? How do I implement all this jawn?

Here are some practical ways to begin to enhance your OTRs.

1. Before teaching a lesson, jot down OTRs that you anticipate posing to students (obviously we all know deviations occur when teaching :)). Remember to keep in mind, the type of response you are looking for (shorter vs. longer). This will allow you to then decide upon the type of OTR (e.g., verbal vs. written vs. gestural) and the subtype (e.g., choral response vs. peer-to-peer).

2. Attempt to vary the types of OTRs you deliver to avoid habituation. Preplan in verbal, gestural, and written OTRs - and even different subtypes (e.g., choral versus peer-to-peer).

3. Teach routines. If you aim to use peer-to-peer, self-explanations, or longer written OTRs then you’ll want to provide some scaffolding up front so students can maximize their learning when using these OTR methods.

4. Self-monitor. Yes, this sounds scary but take data on your usage of OTRs. You can purchase a cheap clicker to count the number of OTRs you pose during a lesson, and if you are uber nerdy you can even count by type (i.e., how many verbal, how many written, how many gestural). Tracking your overall rate and setting goals to enhance your rate will yield success!²

5. Last, be careful with the OTRs you ask when teaching a new skill (see example below). You can embed SO MANY OTRs on preskills when teaching a new concept [this is why I get so frustrated about people saying explicit instruction is just teacher-talk]. The one OTR you want to avoid is the one I see often, “what should I do first?” Most students do not know because that is the NEW skill you are trying to teach so don’t waste your OTR on that.

   

1. Rhetorical questions are a terrible habit to get into and to break. If I am being 100% honest, I catch myself asking these when teaching as well and it is something I have reduced drastically over the year. One main concern I see with rhetorical questions is the classroom culture that they build. Students are not expected to think or provide an answer - so it creates a habit that when a teacher poses a question it can be ignored.

2. Another NERDY thought here. In many cases we just look at overall rate of OTRs. To do this, we count the number of OTRs provided and divide this by the observation time. For example, I observe a teacher for 10 min and count 40 OTRs. 40/10 = 4 OTRs per min. Although I see this as generally useful, if you want to do a deep dive consider this. Teacher 1 is observed for 10 min. We see 5 OTRs provided in the first 5 min of teaching and then shet shifts to 25 OTRs in the last 5 min. Teacher 2 provides 3 OTRs every min of instruction. The distribution of the OTRs equally across time *might* enhance the likelihood we are not “losing” students during instruction. One method to consider is your inter-response time of OTRs. Inter-response time is the length of time between behaviors. So in this example, how much time occurs between each OTR that you provide? If you are hitting an average of 20 sec between OTRs then you are averaging 3 per minute. Beginning to at least think from this angle can prevent long stretches of “teacher talk” where students are passively listening - which we ALWAYS want to avoid.