Reading for All Learners

Study: Callow-Heusser & Sanborn (2016)

Callow-Heusser, C.A., & Sanborn, W.A. (2016). Findings from a study of early reading interventions with randomized assignment to groups (submitted to the U.S. Department of Education’s What Works Clearinghouse and the Best Evidence Encyclopedia housed at Johns Hopkins University’s Center for Data-Driven Reform in Education). Park City, UT: EndVision Research and Evaluation, LLC.
Descriptive Information Usage Acquisition and Cost Program Specifications and Requirements Training

Reading for All Learners (RFAL) has 141 different books containing more than 300 lessons in eight sets. The sets take students from beginning kindergarten reading levels into third grade. For each lesson, teachers only need to locate the next book (or books if more than one lesson is covered) in the set, with enough copies for each student in the group. The lessons are self-contained within the book. All sound and word practice, stories, comprehension questions, progress monitoring/ formative assessments, instructions for implementation, etc., are contained in each book.

Reading for All Learners is intended for use in grades preK-3. The program is intended for use with any student at risk of academic failure. The academic focus is early literacy including print knowledge/awareness, alphabet knowledge, phonological awareness, oral reading fluency, reading comprehension, morphology skills (later levels). 

Where to obtain:
Academic Success for All Learners
142 West 200 North, Suite B, Logan UT 84321
Phone: 435-755-7885
Website: www.iseesam.com

Cost: $4-$30 per student. Cost varies widely depending on edition (black and white vs. color) and platform selected (print vs. digital), and grade level of student materials required.

Print Editions – Cost to equip one classroom with the print edition

·         K – $185 - $300 ($7-$10 per student)

·         Grades 1 or 2 $555-$900 per classroom ($18.50-$30 per student) 

Digital Edition (cost for software only) Kindergarten-Third Grade Classroom $120-$240 ($4-$8 per student). 

It is recommended that Reading for All Learners is used for individual students and small groups of two to six students. Some schools have implemented larger groups successfully.

Reading for All Learners takes 20 minutes per session with a recommended 5 sessions per week for 8 or more weeks.

The program includes a highly specified teacher’s manual.

The program does not require technology. The mobile device applications used for the digital version include the student materials used in small-group lessons. The applications include a pairing function which permits the teacher to control the presentation on student devices. 

Training is not required for the instructor. Most teachers trained to teach early reading will be able to lead lessons with fidelity after reviewing the content included in the Teacher’s Manual and by following prompts embedded throughout the program books. Although training is not required, it is recommended in the case of paraprofessional-led interventions and peer-led interventions. 

RFAL is used successfully by paraprofessionals, classroom volunteers. The program does not assume that the instructor has expertise in a given area.

Training manuals and materials are available. Since 2004, the instructor support materials have been continually refined in response to feedback and in keeping with reading instruction research. Program users are encouraged to provide feedback. 

 

Participants: Convincing Evidence

Sample size: 33 (16 program, 17 control)

Risk Status: Students who were assigned to treatment or comparison groups based on DIBELS beginning of year benchmark composite scores included 30% Hispanic and 8% refugee or immigrant students from other countries. More than 25% spoke a home language other than English. According to the NCES, 99% of students qualified for free and reduced lunch in the 2013-2014 school year. Approximately 11% of the students who participated in the study had been identified for special services other than speech language services.

Demographics:

Grade level

PROGRAM Number

PROGRAM Percentage

CONTROL Number

CONTROL Percentage

p of chi square

  Kindergarten

 

 

 

 

 

  Grade 1

 16

100%

 16

100%

 

  Grade 2

 

 

 

 

 

  Grade 3

 

 

 

 

 

  Grade 4

 

 

 

 

 

  Grade 5

 

 

 

 

 

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Mean Age

       

 

Race-ethnicity

  African-American

 

 

 

 

 

  American Indian

 

 

 

 

 

  Asian/Pacific Islander

 

 

 

 

 

  Hispanic

4

25%

4

24%

 

  White

12

75%

13

76%

 

  Other

 

 

 

 

 

Socioeconomic status

  Subsidized lunch

16

100%

17

100%

 

  No subsidized lunch

 

 

 

 

 

Disability status

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

 

 

 

 

 

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

 

 

 

 

 

  Not identified with a disability

16

100%

17

100%

 

ELL status

  English language learner

4

25%

4

24%

 

  Not English language learner

12

75%

13

76%

 

Gender

Female

11

69%

10

59%

 

Male

5

31%

7

41%

 

 

Training of Instructors: Licensed teachers and paraprofessionals who had been teaching reading for a minimum of 3 years conducted the small group instruction.  Academic Success for All Learners staff provided a 4-hour RFAL training for all instructors assigned to the program/ treatment groups at the outset of the study.  A trainer observed small group instruction and provided feedback on a monthly basis.

Design: Convincing Evidence

Did the study use random assignment?: Yes

If not, was it a tenable quasi-experiment?: NA

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Yes

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences?: NA

Were the program and control groups demographically comparable at pretreatment?: Yes

Was there attrition bias1 ?: No

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Unconvincing Evidence

 Describe when and how fidelity of treatment information was obtained: Approximately once per month, a RFAL trainer visited the school to answer questions specific to RFAL, conduct classroom observations, and offer suggestions to improve quality of instruction.  Most of these suggestions applied equally to both RFAL and ERI reading groups, as they involved pacing, error correction, academic feedback, frequency of group choral versus individual responses, and other effective teaching practices.  Additionally, observers conducted classroom observations of all treatment and control groups on a monthly basis utilizing a standardized observation form designed to capture fidelity of implementation data.  Results indicated that fidelity of implementation was consistent across groups and programs. 

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: Because all instructors taught both treatment and comparison groups, differences due to instructors were minimized.  Additionally, observation of instructors under both conditions showed comparable fidelity of implementation of both programs.

Measures Targeted: Convincing Evidence

Measures Broader: Data Unavailable

Targeted  Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

DIBELS Next Composite Score

Alternate Form     0.95

Test-Retest            0.94

Predictive              0.71

 

 

DIBELS Next Nonsense Word Correct Letter Sounds

Alternate Form     0.85

Test-Retest            0.76

Predictive              0.51

RFAL teaches letter sounds but does not explicitly teach nonsense word decoding.

ERI teaches letter sounds and explicitly teaches nonsense word decoding. 

DIBELS Next Nonsense Word Whole Words Read

Alternate Form     0.90

Test-Retest            0.70

Predictive              0.52

RFAL teaches students to blend the sounds in words.

ERI teaches students to blend the sounds in words.

DIBELS Next Oral Reading Fluency Correct Words per Minute

Alternate Form     0.95

Test-Retest            0.95

Predictive              0.64

RFAL teaches students to read with fluency.

ERI teaches students to read one syllable words. 

DIBELS Next Oral Reading Accuracy

None available

RFAL teaches students to read with accuracy.

ERI teaches students to read one syllable words.

Broader  Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

None

     

 

Number of Outcome Measures: 2 Prereading, 3 Reading

Mean ES - Targeted: Data Unavailableu

Mean ES - Broader: Data Unavailable

Effect Size:

Targeted Measures

Construct Measure Effect Size
Prereading Middle of Year DIBELS Nonsense Word Fluency Correct Letter Sounds 0.06 u
Prereading Middle of Year DIBELS Nonsense Word Fluency Whole Words Read 0.33 u
Prereading Middle of Year DIBELS Composite Score 0.39 u
Reading Middle of Year DIBELS Oral Reading Fluency Correct Words per Minute
Reading Middle of Year DIBELS Oral Reading Accuracy

 

Broader Measures

Construct Measure Effect Size
  None  
Key
*        p ≤ 0.05
**      p ≤ 0.01
***    p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: Yes

Targeted Measures

Construct

Measure

Effect Size

Prereading

Middle of Year DIBELS Nonsense Word Fluency Correct Letter Sounds – Hispanic Students

-0.28 u

Prereading

Middle of Year DIBELS Nonsense Word Fluency Whole Words Read – Hispanic Students

0.31 u

Prereading

Middle of Year DIBELS Composite Score – Hispanic Students

0.30u

Reading

Middle of Year DIBELS Oral Reading Fluency Correct Words per Minute – Hispanic Students

Reading

Middle of Year DIBELS Oral Reading Accuracy – Hispanic Students

Prereading Middle of Year DIBELS Nonsense Word Fluency Correct Letter Sounds – Non-Hispanic Students 0.18 u
Prereading Middle of Year DIBELS Nonsense Word Fluency Whole Words Read – Non-Hispanic Students 0.31 u
Prereading Middle of Year DIBELS Composite Score – Non-Hispanic Students 0.38u
Reading Middle of Year DIBELS Oral Reading Fluency Correct Words per Minute – Non-Hispanic Students
Reading Middle of Year DIBELS Oral Reading Accuracy – Non-Hispanic Students

Broader Measures

Construct

Measure

Effect Size

 

None

 

 

Key

*        p ≤ .05

**      p ≤ .01

***    p ≤ .001

–      Developer was unable to provide necessary data for NCII to calculate effect sizes

u      Effect size is based on unadjusted means

†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Middle of Year DIBELS Nonsense Word Fluency Correct Letter Sounds – Non-Hispanic Students

 

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual , Small groups, (n=2-6)

Duration of Intervention: 20 minutes, 5 times a week, 8 weeks

Minimum Interventionist Requirements: Paraprofessional, No training required

Intervention Reviewed by What Works Clearinghouse: No

Other Research: Potentially Eligible for NCII Review: 0 studies

Other Research: Ineligible for NCII Review: 3 studies

What Works Clearinghouse (WWC) Review:

This program was not reviewed by What Works Clearinghouse.

Additional Research not reviewed by WWC:

Callow-Heusser, C.A., Krebs, S., & Willis, P. (2013). Cache School District’s multi-tiered system of supports helps students who are most in need of additional intervention to SUCCEED! Utah Special Educator, 35(2).

Lignugaris/Kraft, B., Findlay, P., Major, J., Gilberts, G., & Hofmeister, A. (2001). The association between a home reading program and young children's early reading skill. Journal of Direct Instruction, 1(2), 117-136.

Hanson, R. A., & Farrell, D. (1995). The long-term effects on high school seniors of learning to read in kindergarten. Reading Research Quarterly, 30(4), 908-933.