Academy of READING

Study: Fiedorowicz & Trites (1987)

Fiedorowicz, C. A. M, & Trites, R.L. (1987). An evaluation of the effectiveness of computer-assisted component reading subskills training. Education and Technology Series.
Descriptive Information Usage Acquisition and Cost Program Specifications and Requirements Training

The Academy of READING, formerly known as the AutoSkill Component Reading Subskills program, is an intensive and comprehensive Tier II and Tier III intervention tool that focuses on strengthening the root skills of reading. Completely web-based since version 5 was released in 2005, the Academy of READING features age-appropriate content, interfaces, and graphical elements that create an optimal learning environment for students as they build foundational reading skills.

Design Principles and Features

  • Personalized Learning –Using test data the program identifies performance goals and creates a Personalized Training Plan
  • Automaticity
  • Explicit Instruction –Systematic and direct instruction is conducted in the five key pillars recognized by the National Reading Panel:
    • Phonemic Awareness
    • Phonics
    • Fluency
    • Vocabulary and Comprehension
  • Positive Feedback
  • Assessment Options – Academy of READING includes automated online tests that take a snapshot of each student’s reading abilities relative to grade level.
  • In-product reporting can be accessed at any time, from any workstation on the network, through the browser-based management system.
  • Spanish-language support
  • RtI Tools—The RtI Package can be added to the Academy of READING for a single, web-based solution that connects the key components of RtI

 

 

Academy of READING is intended for use in grades 2 through high school. It is designed for use with students with disabilities (including learning disabilities and behavioral disabilities), English language learners and any student at risk of academic failure. The academic area of focus is reading (including phonological awareness, phonics/word study, comprehension, fluency, and vocabulary).

The program has been in use since 1985, in 15 different countries, including: Canada, USA, US Virgin Islands, Bahamas, Jamaica, St. Lucia, St. Thomas, West Indies, Montego Bay, South Africa, Australia, UK, Scotland, England, Northern Ireland, and Republic of Ireland. In the USA only, it is currently in use in over 2950 schools.

 

 

Where to obtain:

EPS/School Specialty Literacy and Intervention                       625 Mount Auburn Street                                               Cambridge, MA 02138                                                             Phone: 617-547-6706                                                                   Toll Free: 1-800-225-5750

Web Site: eps.schoolspecialty.com

Cost: Academy of READING is available as per-school perpetual licenses or as an annual per-student subscription, with the following pricing and fulfillment:

  • Perpetual License Model - Perpetual product pricing is based on a sliding scale of concurrent users: 5, 10, 20, 30, and unlimited.
  • We can provide schools and districts customizable pricing, based on their needs.
  • Schools receive an email with links to activate their licenses and an educator’s kit with wall charts, checklists, and supplementary activities.
  • In Year 1, technical support and maintenance is included in the software cost. In Year 2 and beyond, schools can optionally purchase a support and maintenance program.
  • Volume discounts can also be applied for large, multi-school orders.
  • Annual per student Subscription Model –Licenses must be renewed annually for continued use of the program.
  • Technical support and maintenance is included in the annual subscription price.
  • Volume discounts may apply to large, multi-school orders.
  • Schools receive an installation kit with the software DVD and installation guide to install on the school or district server and an educator’s kit, which contains wall charts, checklists, and supplementary activities.
  • Professional Development is provided with the purchase of the Academy of READING program and includes one day of teacher training, one day of live start up with the students as well as two days of in class coaching/best practice and data review.  Teacher instructor resource guides are also provided for up to 10 teachers.

 

 

It is recommended that the Academy of READING is used 20-30 minutes per session, three to five days a week.

The program includes a highly specified teacher’s manual.

Academy of READING is a browser-based instructional application that can run on a WAN or LAN and requires a T1 or similar network infrastructure. It can also be hosted by School Specialty Intervention.

 

 

The instruction training includes one full day (6 hours) of hands-on training to learn functionality of the program and three days of on-site mentoring and coaching with students (teachers not pulled out).

The minimum qualifications of instructors are that they must be paraprofessionals.

The current version of the Instructor’s Resource Guide has been field tested for the past 4 years.

Additional days of professional development are available including advanced reading workshops and on-site days of consultation. Technical support is included in the software price in Year 1 and can optionally be purchased for subsequent years.

 

 

Participants: Convincing Evidence

Sample size: 115 students across five schools boards. (74 students in the treatment group and 41 students in the control group [17 untrained control group + 24 alternate computer-trained control group]).

Risk Status: The subjects included were identified by the school personnel as at risk for academic failure, based on school records

Demographics:

  Program Control p of chi square
Number Percentage Number Percentage
Grade level
Kindergarten          
  Grade 1          
  Grade 2          
  Grade 3          
  Grade 4          
  Grade 5          
  Grade 6          
  Grade 7          
  Grade 8          
  Grade 9          
  Grade 10          
  Grade 11          
  Grade 12          
Mean Age 11.2   10.7    
Race-ethnicity
  African-American          
  American Indian          
  Asian/Pacific Islander          
  Hispanic          
  White          
  Other          
Socioeconomic status
  Subsidized lunch          
  No subsidized lunch          
Disability status
  Speech-language impairments          
  Learning disabilities          
  Behavior disorders          
  Intellectual disabilities          
  Other          
  Not identified with a disability          
ELL status
  English language learner          
  Not English language learner          
Gender
Female 57   10    
Male 16   7    

Training of Instructors: The instruction was given by well trained research assistants and teachers.

Design: Partially Convincing Evidence

Did the study use random assignment?: No.

If not, was it a tenable quasi-experiment?: Yes.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Not applicable.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Yes.

Were the program and control groups demographically comparable at pretreatment?: Yes.

Was there attrition bias1No.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

Fidelity of Implementation: Unconvincing Evidence

Describe when and how fidelity of treatment information was obtained: Fidelity of treatment information was obtained by the AutoSkill Component Reading Test Program, which provides measures of training achievement, as well as from trainers’ reports. Additional measures of implementation fidelity included:

1) individual training and observation of students by their teacher or a research assistant;
2) supervision of teacher training on part of trained research assistants over the entire school year;
3) regular meetings and observations of training sessions;
4) reporting the number of training hours for students trained in the AutoSkills group; and
5) a teacher questionnaire (at the end of the study) asking teachers to report the frequency and length of sessions per week.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: Feedback was provided to teachers to ensure fidelity, but data were not collected and analyzed.

Training Schedule:
The specification for the training schedule included three half-hour training sessions per week for a total of 1.5 hours weekly. Each subject was trained individually. Most of the subjects were maintained on this training schedule. There were several factors, however, that altered the original scheduling plan for some of the subjects. These included truancy, sick days, professional development days, field trips, special events, and holidays. In addition, there were some problems with computer breakdown and computer thefts. In all cases attempts were made to maintain 1.5 hours of training weekly, but it was necessary to make up time over a two-or three-week period. Three subjects were truant (2 Type A and 1 Type S subjects), not just from the AutoSkill training but from school generally, and these three did not complete the required number of training hours. They completed approximately 30 hours and were not included in the final statistical analyses. Table 7.8 summarizes the total number of training hours for each group.

TABLE 7.8
Training hours for the AutoSkill Trained group.

  Total AutoSkill AutoSkill AutoSkill AutoSkill
  Trained Group Type O Type A Type S
M (SD) 56.4 (2.7) 56.4 (1.8) 55.2 (3.8) 57.3 (1.8)
Range 42-60 52-60 42-60 55-60
3 Subjects dropped from analyses     2
31.5
38.5
1
31.5 hours

Alternate-Computer Trained Control Group
These subjects were provided with computer programs concerned with some aspect of language arts development for three half-hour sessions per week (1.5 hours weekly) for a maximum of 30 hours. All were teacher trained. The list of programs made available to the teachers involved is provided in Table 7.10. These programs were selected following consultation with specialists within each board. Further, the teachers were permitted to use whatever other programs for language arts they preferred. It was not specified that the training had to be given individually. The reason for including this group was predominantly to control for the placebo effect of working on computer programs related to language arts development.

TABLE 7.10
List of programs made available for the Alternate Computer Trained Control group.

Bank Street Writer
Fay’s Word Rally
Missing Links
The Puzzler
Reading Development
Sesame Street
Spelling A-M
Square Pairs
Wordflash
Word Man
Word Master
Wordrace
Write One
Writing a Narrative

 

Measures Targeted: Partially Convincing Evidence

Measures Broader: Convincing Evidence

Targeted Measure Score type & range of measure Reliability statistics Relevance to program instructional content

The Problem Individual Reading Evaluation (SPIRE I and SPIRE II)

Assessing the word recognition level and the projected grade level

 

Tests of reading comprehension skills provided measures of transfer of training of component reading skills to reading words in text and deriving meaning from what is read. The SPIRE I & II tests provided achievement scores

Qualitative Analysis of Silent and Oral Reading(QASOR)

Assessing patterns and kinds of reading errors in the silent and oral reading of connected prose (the emphasis is placed primarily on phonetic and graphic errors as well as rate of fluent reading.

 

QASOR provided qualitative information about the pattern and kinds of errors that were made in reading words in- context.

Gallistel-Ellis Test of Coding Skills

Consists of seven measures of the ability to identify different sounds associated with letters and letter combinations and nine measures of skill in reading different types of real and nonsense words

Split-half reliabilities of 0.985 for reading

Split-half reliabilities of 0.981 for spelling

The G-E provided qualitative information about the types of errors made in reading words in isolation, including phonetic knowledge of letters and syllables and phonetic knowledge of words.

There might be improvement in reading certain types of words even when there was no improvement in the posttraining grade level for isolated word recognition. For example, a subject might still be recognizing words at the Grade 3 level after training, but there might be an improvement in the phonetic knowledge of letters.

 

Broader Measure Score type & range of measure Reliability statistics Relevance to program instructional content

The Wide Range Achievement Test Revised (WRAT-R)

Assessing reading word recognition grade levels
Grade equivalent score (grade placement)

Split-half reliabilities of 0.98 for reading

The inclusion of a word recognition test provided a measure of transfer of training of component reading skills to the reading of words in isolation on a standardized reading word recognition task (WRAT-R reading subtest).

 

Number of Outcome Measures: 2 Prereading, 22 Reading

Mean ES - Targeted: 0.19*u

Mean ES - Broader: 0.36u

Effect Size:

Targeted Measures

Construct Measure Effect Size
Prereading G-E Phonetic Knowledge Phonic Syllables 0.43u
Prereading G-E Phonetic Knowledge Phonic Words 0.14u
Reading SPIRE Paragraphs Oral Errors -0.41u
Reading SPIRE Paragraphs Oral Latency -0.18u
Reading SPIRE Paragraphs Oral Retention 0.55*,u
Reading SPIRE Paragraphs Oral Comprehension 0.47u
Reading SPIRE Paragraphs Silent Latency 0.17u
Reading SPIRE Paragraphs Silent Retention 0.44u
Reading SPIRE Paragraphs Silent Comprehension 0.12u
Reading Project Grade Level Oral Errors -0.03u
Reading Project Grade Level Oral Latency -0.03u
Reading Project Grade Level Oral Retention 0.60*,u
Reading Project Grade Level Oral Comprehension 0.53*,u
Reading Project Grade Level Silent Latency 0.20u
Reading Project Grade Level Silent Retention 0.18u
Reading Project Grade Level Silent Comprehension 0.28u
Reading QASOR CLOZE Word Recognition Level Meaning 0.56*,u
Reading QASOR CLOZE Word Recognition Level Graphic Sense -0.64*,u
Reading QASOR CLOZE Word Recognition Level Fluency 0.13u
Reading QASOR CLOZE Projected Grade Level Meaning 0.59*,u
Reading QASOR CLOZE Projected Grade Level Graphic Sense -0.10u
Reading QASOR CLOZE Projected Grade Level Fluency 0.20u

Broader Measures

Construct Measure Effect Size
Reading WRAT- R Grade Equivalent 0.50u
Reading WRAT - R Standard Score 0.22u

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***       p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual

Duration of Intervention: 30 minutes, 3 times a week, 8-12 weeks

Minimum Interventionist Requirements: Paraprofessional , 6 hours of training

Reviewed by WWC or E-ESSA: WWC & E-ESSA

What Works Clearinghouse Review

Adolescent Literacy Protocol

EffectivenessNo studies of Academy of READING® that fall within the scope of the Adolescent Literacy review protocol meet What Works Clearinghouse (WWC) group design standards. Because no studies meet WWC group design standards at this time, the WWC is unable to draw any conclusions based on research about the effectiveness or ineffectiveness of Academy of READING® on adolescent readers. Additional research is needed to determine the effectiveness or ineffectiveness of this intervention..

Studies Reviewed: N/A

Full Report

Beginning Reading

EffectivenessAs of July 2007 no studies of Academy of READING® were found that fell within the scope of the Beginning Reading review protocol and met WWC design standards. Therefore, the WWC is unable to draw any research based conclusions about the effectiveness or ineffectiveness of Academy of READING® to improve outcomes in this area.

Studies Reviewed: N/A

Full Report

 

Evidence for ESSA

No studies considered met Evidence for ESSA's inclusion requirements for Elementary or Secondary Reading.

Other Research: Potentially Eligible for NCII Review: 2 studies

Kerr, J. (1993). Computerized reading skills remediation for reading-disabled adolescents: A compensatory supplement to instructional strategies. University of Toronto (Canada). ProQuest Dissertations and Theses, 198 p. Retrieved from http://ezproxy.lakeheadu.ca/login?url=http://search.proquest.com/docview/304080604?accountid=11956
 

Thomas, A. & Clapp, T. (1989). A comparison of computer-assisted component reading skills training and repeated reading for adolescent poor readers. Canadian Journal of Special Education, 5, 135-144.