What do you need to consider when selecting a behavioral progress monitoring tool?

Question: What do you need to consider when selecting a behavioral progress monitoring tool? 

Answer: This is a really interesting question. Behavior assessment in terms of progress monitoring is a relatively new technology. We don’t have nearly as much understanding of how to monitor progress in behavior as we do on the academic side. Academic progress monitoring has at least three decades of really intensive research. Understanding the nuances and how to measure progress. On the behavioral side we have had Systematic Direct Observation for a number of years, but we really haven’t had other options that we could pair with Systematic Direct Observation. So when selecting a behavior tool, you are trying to look for essentially two things. Something that has good psychometric properties; it needs to be reliable and valid, and we need to talk about each of those two terms, and also something which is highly feasible. For behavioral interventions and behavioral progress monitoring you are going to need an awful lot of outcome data. Behavior changes quite quickly. It can be different in different environments, so as a result we need to be collecting ideally daily progress monitoring data and ideally in the particular period of time in which the child is engaging in the problem behavior and the intervention is being conducted. What that means is if you are going to use something like Systematic Direct Observation you are going to have to have an observer there essentially all the time to be able to record the behavior. So, that leaves us looking at some other options. SDO is going to be very important for kids who have very intense needs, but we need to look at other more feasible options when measuring other kids. When picking particular tools there are two things you need to look at. Number one is reliability. There are a number of behavioral options out there that have been deemed reliable for screening purposes. Screening instruments are looking at identifying target at risk kids and ideally they should identify the same kids over X period of time, two, three, four weeks. Progress monitoring tools on the other hand need to be incredibly sensitive to behavior change. So if the child’s disruptive behavior or academic engagement increases by even a small amount that needs to be picked up by the tool. Tools which have been deemed reliable for progress monitoring use a very different system of reliability than screening measures, so when picking these tools it is quite important to really make sure they have been tested for progress monitoring purposes and not for screening purposes. In terms of validity they need to be measuring a construct that you are interested in and that the intervention is targeted at. If you are looking at disruptive behavior, you need to make sure they are looking at the same type of disruptive behavior that you are interested in. If you are interested in suicidality or internalizing you need to look at those particular constructs. So, we can’t just go out and grab a behavioral measure which doesn’t have the exact definitions which are aligned to the intervention that you are looking at. Luckily, the NCII is starting to go through the process of vetting these tools and we are starting to put tools up on the website which have different target focuses and particularly which have been deemed reliable for progress monitoring purposes. And that’s where I think you would want to go at this stage. Unfortunately I don’t know of any other large scale resource which is going to be able to provide those specific tools for monitoring kids’ behavioral needs. To wrap this up, you want to look at tools that have good psychometric properties but you also really want to consider feasibility. Understand that you are going to have to collect, in some cases if you are doing this over a year period hundreds of data points and so really consider how you are going to collect that data. It is nice to pick a wonderful tool but if you can’t use the tool and actually collect the outcome data it is not going to be all that effective.       

View the Behavioral Progress Monitoring Tools Chart.

Developed By: 
National Center on Intensive Intervention