Achieve Intervention

Study: Tracey & Young (2004)

Tracey, D. H. & Young, J. W. (2004). Evaluation of KidBiz3000: Bayonne Study Final Report. Lakewood, NJ: Achieve3000.
Descriptive Information Usage Acquisition and Cost Program Specifications and Requirements Training

Achieve Intervention is a web-based literacy platform for at-risk students.  It is delivered as KidBiz3000 for grades 2-5, TeenBiz3000 for grades 6-8, and Empower3000 for grades 9-12. Students work through a sequenced curriculum that includes informational texts, writing activities, and progress-monitoring measures. The program offers additional scaffolds to meet students’ individual needs. Teachers can manage the program using administrative and reporting tools.

Achieve Intervention is intended for students in grades 2-12, particularly those at risk of academic failure. The academic area of focus is reading, with an emphasis on reading comprehension. 

Where to obtain:

Achieve3000, Inc.
1985 Cedar Bridge Avenue, Suite 3
Lakewood, NJ 08701 Phone: 1-732-367-5505

Website: www.achieve3000.com

Cost:

For pricing information, please contact the vendor.

It is recommended that Achieve Intervention be used with individual students or small groups.

Achieve Intervention is administered for 45 minutes per session, two times a week for 20 weeks.

The program includes a highly specified teacher’s manual.

The web-based program requires a device with an internet connection and a relatively current operating system.

The minimum qualifications of instructors are that they must be professionals. The program does not assume that the instructor has expertise in a given area.

1-2 days of training are recommended to fully prepare the instructor. Instructors may receive training from Achieve Intervention’s Professional Learning Services team. This training orients users to all aspects of the platform and covers additional topics to support implementation.

Training manuals and materials are available, and are periodically revised in response to client feedback.

Users are granted access to the online Learning Center, which includes a number of tools and independent learning paths for teacher self-study, providing ongoing training and support for educators. 

 

Participants: Unconvincing Evidence

Sample size: 168 fifth grade students across five schools and eleven classrooms. The sample was comprised of 84 program students and 84 control students.

Demographics:

 

Program

Control

p of chi square

Number

Percentage

Number

Percentage

Grade level

  Kindergarten

 

 

 

 

 

  Grade 1

 

 

 

 

 

  Grade 2

 

 

 

 

 

  Grade 3

 

 

 

 

 

  Grade 4

 

 

 

 

 

  Grade 5

 84

100

84

100

 0.00

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Race-ethnicity

  African-American

 

 

 

 

 

  American Indian

 

 

 

 

 

  Asian/Pacific Islander

 

 

 

 

 

  Hispanic

 

 

 

 

 

  White

 

 

 

 

 

  Other

 

 

 

 

 

Socioeconomic status

  Subsidized lunch

 

 

 

 

 

  No subsidized lunch

 

 

 

 

 

Disability status

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

 

 

 

 

 

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

 

 

 

 

 

  Not identified with a disability

 

 

 

 

 

ELL status

  English language learner

 

 

 

 

 

  Not English language learner

 

 

 

 

 

Gender

Female

 

 

 

 

 

Male

 

 

 

 

 

Training of Instructors: KidBiz3000’s standard, “On-Site Teacher Training Model” professional development program was provided to the participating district’s teachers prior to program implementation. All of the teachers in the differentiated and non-differentiated classrooms received direct professional development from a KidBiz3000 trainer. The professional development program consisted of two large group two-and-a-half hour sessions and one small group one-hour session. Topics covered in the professional development sessions included: a program overview, suggestions for implementation, trouble-shooting recommendations, the sharing of successes and concerns, and ideas for using the program to strengthen students’ comprehension skills.  Additionally, the trainer made several classroom visits in response to individual requests by the teachers. 

Design: Unconvincing Evidence

Did the study use random assignment?: No

If not, was it a tenable quasi-experiment?: No

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Not applicable

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? No

Were the program and control groups demographically comparable at pretreatment?: No

Was there attrition bias1? No

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: No

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Unconvincing Evidence

Describe when and how fidelity of treatment information was obtained: Fidelity of implementation data is collected automatically within the Achieve3000 system as students use the program.  Specifically, the Achieve3000 system collects data related to the number of sessions completed as well as students’ scores on formative assessments that provide a reliable indicator of quality of program use.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: Not applicable.

Measures Targeted: Unconvincing Evidence

Measures Broader: Unconvincing Evidence

 

Targeted  Measure Reliability Statistics Relevance to Program Focus Exposure to Related Content Among Control Group

Scholastic Reading Inventory

 

The Scholastic Reading Inventory (SRI) is a reading assessment and progress-monitoring tool that provides students with reading passages and four, forced-choice subsequent options to complete sentences. The ability to correctly fill in the blank is dependent on students’ skills in the areas of word identification, vocabulary, and comprehension. The SRI yields a lexile measure that can be used to match students reading ability with specific reading materials and to monitor student achievement.

 

TerraNova -  Reading Composite Sub-test

 

The Reading Composite sub-test assesses students’ literacy abilities.  It consists of questions in the areas of reading and vocabulary that yield separate sub-scores.

 

TerraNova  - Language Composite Sub-test

 

The Language Composite sub-test also assesses students’ literacy abilities. It consists of questions in the areas of language and language mechanics that yield separate sub-scores.

 

 

Number of Outcome Measures: 7 Reading

Mean ES - Targeted: 0.21u

Mean ES - Broader: Data Unavailable

Effect Size:

Targeted Measures

Construct Measure Effect Size
Reading Scholastic Reading Inventory 0.30u
Reading TerraNova Reading Subscore 0.13u
Reading TerraNova Vocabulary Subscore 0.19u
Reading TerraNova Reading Composite 0.17u
Reading TerraNova Language Subscore 0.37*,u
Reading TerraNova Language Mechanics Subscore 0.08u
Reading TerraNova Language Composite 0.24u

Broader Measures

Construct Measure Effect Size
  NA  
Key
*      p ≤ 0.05
**    p ≤ 0.01
***   p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual, Small Group , (n=20-25)

Duration of Intervention: 40 minutes, 2 times a week, 35 weeks

Minimum Interventionist Requirements: Professional, 1-2 days of training

Reviewed by WWC or E-ESSA: No

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

This program was not reviewed by Evidence for ESSA.

 

Other Research: Potentially Eligible for NCII Review: 0 studies