mCLASS: Reading

Area: 3D - Text and Reading Comprehension

Cost Technology, Human Resources, and Accommodations for Special Needs Service & Support Purpose & Other Implementation Information Usage & Reporting

Cost for year 1: 

$20.90 per student (includes $8.90 platform subscription)

$393.75 for complete kit (Rigby Ultra Edition) includes Teacher’s Guide and Student Materials. (other kits are available – costs vary)

Training manuals and materials are field tested but are not included in the cost of the tool.

Cost per student for subsequent years: 

$20.90 per student (includes $8.90 platform subscription)

Online Costs: $400 Start-up per campus - Remote Installation (one per campus). Telephone guidance through the installation of mCLASS software on teacher mobile devices and desktop computers. Includes step-by-step walkthrough of the install process, troubleshooting, and verification of installation success.           

Testers will require 4-8 hours of training.

Paraprofessionals can administer the test.

Accommodations:

mCLASS is an assessment instrument well-suited for use with capturing the developing reading skills of special education students learning to read, with a few exceptions: a) students who are deaf; b) students who have fluency-based speech disabilities, e.g., stuttering, oral apraxia; c) students who are learning to read in a language other than English or Spanish; d) students with severe disabilities.  Use of mCLASS is appropriate for all other students, including those in special education for whom reading connected text is an IEP goal. For students receiving special education, it may be necessary to adjust goals and timelines; and provide accommodations as part of the administration.

 The purpose of accommodation is to facilitate assessment for children for whom a standard administration may not provide an accurate estimate of their skills in the core early literacy skill areas. Valid and acceptable accommodations are ones that are unlikely to change substantially the meaning or interpretation of a student’s scores.

 The valid and acceptable accommodations for TRC administration are available upon request.

Where to Obtain: 

Amplify Education, Inc

55 Washington St Suite 900
Brooklyn, NY 11201

800-823-1969, option 1

www.amplify.com

Amplify’s Customer Care Center offers complete user-level support from 7:00 a.m. to 7:00 p.m. EST, Monday through Friday. Customers may contact a customer support representative via telephone, e-mail, or electronically through the mCLASS website. Calls to the Customer Care Center’s toll-free number are answered immediately by an automated attendant and routed to customer support agents according to regional expertise.  Additionally, customers have self-service access to instructions, documents, and frequently asked questions on our Website.  The research staff and product teams are available to answer questions about the content within the assessments. 

mCLASS:3D - TRC is a set of screening and progress monitoring measures for grades K-5. Text Reading and Comprehension (TRC) is an individually administered assessment using leveled readers from a book set to determine a student’s instructional reading level. During this assessment, students are asked to read a benchmark book and complete a number of follow-up tasks, which may include Oral Comprehension, Retelling, and/or Written Comprehension.

mCLASS: 3D- Text and Reading Comprehension takes 5-8 minutes to administer individually for grades K-5.

Number of alternate forms varies based on the bookset used – there are 20+ alternate forms in all cases. (Please see separately shipped flash drives for alternate forms.) 

 

 

Reliability of the Performance Level Score: Half-filled bubble

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data) / Subjects

range

median

Full-scale person-reliability (IRT equivalent to Cronbach’s alpha

K-5

151,373

 

0.94

 

Reliability was computed using data from school year 2011-2012. 11% African American, 9% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 23% subsidized lunch; 4% special education; 5% English as second language.

Grade-level Person-reliability (IRT equivalent to Cronbach’s alpha

K

50,059

 

0.44

 

Reliability was computed using data from school year 2011-2012. 11% African American, 8% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 17% subsidized lunch; 3% special education; 4% English as second language.

Grade-level Person-reliability (IRT equivalent to Cronbach’s alpha

1

49,080

 

0.74

 

Reliability was computed using data from school year 2011-2012. 11% African American, 9% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 23% subsidized lunch; 4% special education; 5% English as second language.

Grade-level Person-reliability (IRT equivalent to Cronbach’s alpha

2

45,925

 

0.94

 

Reliability was computed using data from school year 2011-2012. 11% African American, 9% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 28% subsidized lunch; 6% special education; 5% English as second language.

Grade-level Person-reliability (IRT equivalent to Cronbach’s alpha

3

3,733

 

0.97

 

Reliability was computed using data from school year 2011-2012. 16% African American, 9% Hispanic, 2% Asian or Pacific Islander, 6% Multi-race; 37% subsidized lunch; 9% special education; 7% English as second language.

Grade-level Person-reliability (IRT equivalent to Cronbach’s alpha

4

1,498

 

0.99

 

Reliability was computed using data from school year 2011-2012. 8% African American, 6% Hispanic, 4% Asian or Pacific Islander, 7% Multi-race; 21% subsidized lunch; 7% special education; 3% English as second language.

Grade-level Person-reliability (IRT equivalent to Cronbach’s alpha

5

1,078

 

0.99

 

Reliability was computed using data from school year 2011-2012. 10% African American, 5% Hispanic, 5% Asian or Pacific Islander, 5% Multi-race; 19% subsidized lunch; 5% special education; 4% English as second language.

Inter-Rater Reliability

K-3

27

 

0.73

 

Students in this sample were administered the assessment during the 2009-2010 school year. 15% African American, 9% Hispanic. Reliability was calculated as the average Spearman’s rank order correlation between final results recorded by all pairs of raters.

 

Reliability of the Slope: Empty bubble

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data) / Subjects

range

median

HLM

K

3,353

 

0.81

 

Reliability of slope was computed using data from school year 2011-2012. 13% African American, 10% Hispanic, 1% Asian or Pacific Islander, 6% Multi-race; 29% subsidized lunch; 3% special education; 4% English as second language. Weekly assessments over 12 months (i.e., 6-51 assessments; mean=8.70).

HLM

1

6,290

 

0.86

 

Reliability of slope was computed using data from school year 2011-2012. 11% African American, 11% Hispanic, 1% Asian or Pacific Islander, 5% Multi-race; 37% subsidized lunch; 8% special education; 3% English as second language. Weekly assessments over 12 months (i.e., 6-37 assessments; mean=9.44).

HLM

2

4,735

 

0.80

 

Reliability of slope was computed using data from school year 2011-2012. 13% African American, 10% Hispanic, 1% Asian or Pacific Islander, 5% Multi-race; 39% subsidized lunch; 9% special education; 3% English as second language. Weekly assessments over 12 months (i.e., 6-48 assessments; mean=9.37).

HLM

3

211

 

0.83

 

Reliability of slope was computed using data from school year 2011-2012. 5% African American, 4% Hispanic, 8% Multi-race; 20% subsidized lunch; 6% special education. Weekly assessments over 12 months (i.e., 6-20 assessments; mean=8.97).

HLM

4

48

 

0.62

 

Reliability of slope was computed using data from school year 2011-2012. 15% African American, 11% Hispanic, 18% Multi-race; 54% subsidized lunch; 25% special education. Weekly assessments over 12 months (i.e., 6-10 assessments; mean=7.43).

HLM

5

2,198

 

0.77

 

Reliability of slope was computed using data from school year 2011-2012. 20% African American, 10% Hispanic, 3% Multi-race; 9% special education. Weekly assessments over 12 months (i.e., 6-26 assessments; mean=9.01).

 

Validity of the Performance Level Score: Full bubble

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient

Information (including normative data) / Subjects

range

median

Concurrent Validity

Concurrent

3

ISTEP+ ELA

1,625

 

0.72

Validity was computed using data from school year 2011-2012 end of year benchmark assessment. 8% African American, 4% Hispanic, 6% Multi-race; 19% subsidized lunch; 6% special education; 2% English as second language.

Concurrent

K

DIBELS Next Composite

32,930

 

0.70

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 8% African American, 8% Hispanic, 2% Asian, 4% Multi-race; 17% subsidized lunch; 3% special education; 3% English as second language.

Concurrent

1

DIBELS Next Composite

32,272

 

0.85

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 9% African American, 9% Hispanic, 2% Asian, 5% Multi-race; 25% subsidized lunch; 5% special education; 5% English as second language.

Concurrent

2

DIBELS Next Composite

31,290

 

0.76

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 10% African American, 8% Hispanic, 2% Asian, 4% Multi-race; 28% subsidized lunch; 6% special education; 4% English as second language.

Concurrent

3

DIBELS Next Composite

1,468

 

0.80

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 6% African American, 5% Hispanic, 4% Asian, 3% Multi-race; 14% subsidized lunch; 4% special education; 2% English as second language.

Concurrent

4

DIBELS Next Composite

873

 

0.83

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 8% African American, 6% Hispanic, 6% Asian, 5% Multi-race; 15% subsidized lunch; 3% special education; 5% English as second language.

Concurrent

5

DIBELS Next Composite

767

 

0.74

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 11% African American, 7% Hispanic, 4% Asian, 5% Multi-race; 7% subsidized lunch; 2% special education; 3% English as second language.

Predictive Validity

Predictive

Grade 2
to predict
Grade 3

ISTEP+ ELA

32,352

 

0.67

Validity was computed using TRC data from school year 2010-2011 to predict ISTEP+ ELA data from school year 2011-2012. 12% African American, 8% Hispanic, 5% Multi-race; 30% subsidized lunch; 5% special education; 5% English as second language.

Predictive

Grade K

DIBELS Next Composite

39,942

 

0.61

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 9% African American, 9% Hispanic, 2% Asian, 4% Multi-race; 15% subsidized lunch; 3% special education; 3% English as second language.

Predictive

Grade 1

DIBELS Next Composite

39,935

 

0.84

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 10% African American, 10% Hispanic, 2% Asian, 4% Multi-race; 21% subsidized lunch; 3% special education; 3% English as second language.

Predictive

Grade 2

DIBELS Next Composite

38,303

 

0.79

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 11% African American, 10% Hispanic, 2% Asian, 4% Multi-race; 24% subsidized lunch; 5% special education; 5% English as second language.

Predictive

Grade 3

DIBELS Next Composite

7,024

 

0.87

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 15% African American, 16% Hispanic, 2% Asian, 2% Multi-race; 14% subsidized lunch; 6% special education; 5% English as second language.

Predictive

Grade 4

DIBELS Next Composite

2,507

 

0.84

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 11% African American, 8% Hispanic, 3% Asian, 3% Multi-race; 14% subsidized lunch; 10% special education; 6% English as second language.

Predictive

Grade 5

DIBELS Next Composite

2,121

 

0.76

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 9% African American, 11% Hispanic, 2% Asian, 2% Multi-race; 8% subsidized lunch; 11% special education; 6% English as second language.

 

Predictive Validity of the Slope of Improvement: Empty bubble

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient

Information (including normative data)/Subjects

range

median

Concurrent Validity

Concurrent

3

ISTEP+ ELA

142

 

0.19

Validity of slope was computed using data from school year 2011-2012. 8% African American, 3% Hispanic, 10% Multi-race; 23% subsidized lunch; 4% special education. Weekly assessments over 12 months (i.e., 6-20 assessments; mean=8.21).

Concurrent

K

DIBELS Next Composite

2,718

 

0.57

Validity of slope was computed using data from school year 2011-2012. 14% African American, 11% Hispanic, 4% Multi-race; 27% subsidized lunch; 2% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-35 assessments; mean=7.73).

Concurrent

1

DIBELS Next Composite

4,543

 

0.66

Validity of slope was computed using data from school year 2011-2012. 12% African American, 13% Hispanic, 2% Asian, 5% Multi-race; 36% subsidized lunch; 7% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-37 assessments; mean=8.17).

Concurrent

2

DIBELS Next Composite

3,470

 

0.45

Validity of slope was computed using data from school year 2011-2012. 14% African American, 12% Hispanic, 1% Asian, 4% Multi-race; 39% subsidized lunch; 8% special education; 4% English as second language. Weekly assessments over 12 months (i.e., 6-40 assessments; mean=8.27).

Concurrent

3

DIBELS Next Composite

119

 

0.40

Validity of slope was computed using data from school year 2011-2012. 1% African American, 3% Hispanic, 3% Multi-race; 17% subsidized lunch; 4% special education. Weekly assessments over 12 months (i.e., 6-20 assessments; mean=8.42).

Concurrent

4

DIBELS Next Composite

27

 

0.43

Validity of slope was computed using data from school year 2011-2012. 4% African American, 15% Hispanic, 19% Multi-race; 63% subsidized lunch; 26% special education. Weekly assessments over 12 months (i.e., 6-8 assessments; mean=6.96).

Concurrent

5

DIBELS Next Composite

48

 

0.34

Validity of slope was computed using data from school year 2011-2012. 4% African American. Weekly assessments over 12 months (i.e., 6-14 assessments; mean=7.22).

Predictive Validity

Predictive

Grade 2 to predict Grade 3

ISTEP+ ELA

3,648

 

0.33

Validity of slope was computed using TRC data from school year 2010-2011 to predict ISTEP+ ELA data from school year 2011-2012. 13% African American, 11% Hispanic, 1% Asian, 7% Multi-race; 38% subsidized lunch; 8% special education; 4% English as second language. Weekly assessments over 12 months (i.e., 6-38 assessments; mean=8.60).

Predictive

Grade K to predict Grade 1

DIBELS Next Composite

1,861

 

0.49

Validity of slope was computed using TRC data from school year 2010-2011 to predict DIBELS composite score from school year 2011-2012. 11% African American, 16% Hispanic, 1% Asian, 4% Multi-race; 40% subsidized lunch; 2% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-20 assessments; mean=7.54).

Predictive

Grade 1 to predict Grade 2

DIBELS Next Composite

3,420

 

0.53

Validity of slope was computed using TRC data from school year 2010-2011 to predict DIBELS composite score from school year 2011-2012. 9% African American, 12% Hispanic, 1% Asian, 4% Multi-race; 38% subsidized lunch; 6% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-40 assessments; mean=8.72).

Predictive

Grade 2 to predict Grade 3

DIBELS Next Composite

252

 

0.46

Validity of slope was computed using TRC data from school year 2010-2011 to predict DIBELS composite score from school year 2011-2012. 3% African American, 6% Hispanic, 1% Asian, 1% Multi-race; 29% subsidized lunch; 8% special education; 1% English as second language. Weekly assessments over 12 months (i.e., 6-19 assessments; mean=9.35).

Predictive

Grade 3 to predict Grade 4

DIBELS Next Composite

49

 

0.42

Validity of slope was computed using TRC data from school year 2010-2011 to predict DIBELS composite score from school year 2011-2012. 2% African American, 11% Hispanic, 13% Multi-race; 64% subsidized lunch; 26% special education; 2% English as second language. Weekly assessments over 12 months (i.e., 6-12 assessments; mean=7.60).

 

Disaggregated Reliability and Validity Data: Empty bubble

Disaggregated Reliability of the Performance Level Score

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data) / Subjects

range

median

Full-scale person-reliability (IRT equivalent to Cronbach’s alpha)

Caucasian

K-5

62,951

 

0.95

 

Reliability was computed using data from school year 2011-2012. 11% African American, 9% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 23% subsidized lunch; 4% special education; 5% English as second language.

African American

K-5

16,420

 

0.93

 

Hispanic

K-5

13,036

 

0.93

 

Kindergarten Person-reliability

Caucasian

K

19,078

 

0.44

 

Reliability was computed using data from school year 2011-2012. 11% African American, 8% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 17% subsidized lunch; 3% special education; 4% English as second language.

African American

K

5,231

 

0.47

 

Hispanic

K

4,247

 

0.33

 

Grade 1 Person-reliability

Caucasian

1

19,959

 

0.79

 

Reliability was computed using data from school year 2011-2012. 11% African American, 9% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 23% subsidized lunch; 4% special education; 5% English as second language.

African American

1

5,362

 

0.58

 

Hispanic

1

4,340

 

0.56

 

Grade 2 Person-reliability

Caucasian

2

20,860

 

0.94

 

Reliability was computed using data from school year 2011-2012. 11% African American, 9% Hispanic, 1% Asian or Pacific Islander, 4% Multi-race; 28% subsidized lunch; 6% special education; 5% English as second language.

African American

2

5,000

 

0.94

 

Hispanic

2

3,971

 

0.90

 

Grade 3 Person-reliability

Caucasian

3

1,519

 

0.98

 

Reliability was computed using data from school year 2011-2012. 16% African American, 9% Hispanic, 2% Asian or Pacific Islander, 6% Multi-race; 37% subsidized lunch; 9% special education; 7% English as second language.

African American

3

594

 

0.97

 

Hispanic

3

339

 

0.97

 

Grade 4 Person-reliability

Caucasian

4

928

 

0.99

 

Reliability was computed using data from school year 2011-2012. 8% African American, 6% Hispanic, 4% Asian or Pacific Islander, 7% Multi-race; 21% subsidized lunch; 7% special education; 3% English as second language.

African American

4

123

 

0.96

 

Hispanic

4

87

 

0.98

 

Grade 5 Person-reliability

Caucasian

5

607

 

1.0

 

Reliability was computed using data from school year 2011-2012. 10% African American, 5% Hispanic, 5% Asian or Pacific Islander, 5% Multi-race; 19% subsidized lunch; 5% special education; 4% English as second language.

African American

5

110

 

0.99

 

Hispanic

5

52

 

1.0

 

 

Disaggregated Reliability of the Slope

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data) / Subjects

range

median

Kindergarten

HLM (Caucasian)

K

1,192

 

0.81

 

Reliability of slope was computed using data from school year 2011-2012. 29% subsidized lunch; 3% special education; 4% English as second language. Weekly assessments over 12 months (i.e., 6-51 assessments; mean=8.70).

HLM (African American)

K

444

 

0.79

 

HLM (Hispanic)

K

432

 

0.78

 

Grade 1

HLM (Caucasian)

1

2,916

 

0.86

 

Reliability of slope was computed using data from school year 2011-2012. 37% subsidized lunch; 8% special education; 3% English as second language. Weekly assessments over 12 months (i.e., 6-37 assessments; mean=9.44).

HLM (African American)

1

735

 

0.80

 

HLM (Hispanic)

1

690

 

0.85

 

Grade 2

HLM (Caucasian)

2

2,281

 

0.79

 

Reliability of slope was computed using data from school year 2011-2012. 39% subsidized lunch; 9% special education; 3% English as second language. Weekly assessments over 12 months (i.e., 6-48 assessments; mean=9.37).

HLM (African American)

2

605

 

0.80

 

HLM (Hispanic)

2

467

 

0.791

 

Grade 3

HLM (Caucasian)

3

1,867

 

0.82

 

Reliability of slope was computed using data from school year 2011-2012. 22% African American; 10% Hispanic; 5% special education; 3% English as second language. Weekly assessments over 12 months (i.e., 6-24 assessments; mean=8.64).

HLM (African American)

3

1,735

 

0.80

 

HLM (Hispanic)

3

784

 

0.80

 

Grade 4

HLM (Caucasian)

4

652

 

0.76

 

Reliability of slope was computed using data from school year 2011-2012. 21% African American; 13% Hispanic; 6% special education; 4% English as second language. Weekly assessments over 12 months (i.e., 6-24 assessments; mean=9.21).

HLM (African American)

4

616

 

0.78

 

HLM (Hispanic)

4

349

 

0.79

 

Grade 5

HLM (Caucasian)

5

449

 

0.75

 

Reliability of slope was computed using data from school year 2011-2012. 20% African American, 10% Hispanic, 3% Multi-race; 9% special education. Weekly assessments over 12 months (i.e., 6-26 assessments; mean=9.01).

HLM (African American)

5

441

 

0.74

 

HLM (Hispanic)

5

216

 

0.82

 

 

Disaggregated Validity of the Performance Level Score

Type of Validity

Age or
Grade

Test or
Criterion

n
(range)

Coefficient

Information (including normative data) / Subjects

range

median

Concurrent Validity

Caucasian

3

ISTEP+ ELA

895

 

0.73

Validity was computed using data from school year 2011-2012 end of year benchmark assessment. 19% subsidized lunch; 6% special education; 2% English as second language.

African American

3

ISTEP+ ELA

131

 

0.67

Hispanic

3

ISTEP+ ELA

65

 

0.71

Caucasian

K

DIBELS Next Composite

13,414

 

0.71

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 17% subsidized lunch; 3% special education; 3% English as second language.

African American

K

DIBELS Next Composite

2,762

 

0.66

Hispanic

K

DIBELS Next Composite

2,593

 

0.65

Caucasian

1

DIBELS Next Composite

14,871

 

0.85

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 25% subsidized lunch; 5% special education; 5% English as second language.

African American

1

DIBELS Next Composite

3,008

 

0.86

Hispanic

1

DIBELS Next Composite

2,748

 

0.86

Caucasian

2

DIBELS Next Composite

15,763

 

0.74

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 28% subsidized lunch; 6% special education; 4% English as second language.

African American

2

DIBELS Next Composite

3,115

 

0.76

Hispanic

2

DIBELS Next Composite

2,512

 

0.77

Caucasian

3

DIBELS Next Composite

659

 

0.83

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 14% subsidized lunch; 4% special education; 2% English as second language.

African American

3

DIBELS Next Composite

89

 

0.78

Hispanic

3

DIBELS Next Composite

71

 

0.80

Caucasian

4

DIBELS Next Composite

551

 

0.79

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012.15% subsidized lunch; 3% special education; 5% English as second language.

African American

4

DIBELS Next Composite

72

 

0.79

Hispanic

4

DIBELS Next Composite

50

 

0.85

Caucasian

5

DIBELS Next Composite

474

 

0.69

Validity was computed using TRC and DIBELS Next composite score data from school year 2011-2012. 7% subsidized lunch; 2% special education; 3% English as second language.

African American

5

DIBELS Next Composite

83

 

0.77

Hispanic

5

DIBELS Next Composite

51

 

0.68

Predictive Validity

Caucasian

Grade 2
to predict
Grade 3

ISTEP+ ELA

16,230

 

0.67

Validity was computed using TRC data from school year 2010-2011 to predict ISTEP+ ELA data from school year 2011-2012. 30% subsidized lunch; 5% special education; 5% English as second language.

African American

Grade 2
to predict
Grade 3

ISTEP+ ELA

3,882

 

0.64

Hispanic

Grade 2
to predict
Grade 3

ISTEP+ ELA

2,724

 

0.65

Caucasian

Grade K
 

DIBELS Next Composite

15,719

 

0.61

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 9% African American, 9% Hispanic, 2% Asian, 4% Multi-race; 15% subsidized lunch; 3% special education; 3% English as second language.

African American

Grade K
 

DIBELS Next Composite

3,541

 

0.58

Hispanic

Grade K
 

DIBELS Next Composite

3,579

 

0.56

Caucasian

Grade 1
 

DIBELS Next Composite

17,315

 

0.80

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 10% African American, 10% Hispanic, 2% Asian, 4% Multi-race; 21% subsidized lunch; 3% special education; 3% English as second language.

African American

Grade 1
 

DIBELS Next Composite

3,986

 

0.81

Hispanic

Grade 1
 

DIBELS Next Composite

4,028

 

0.82

Caucasian

Grade 2
 

DIBELS Next Composite

18,193

 

0.72

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 11% African American, 10% Hispanic, 2% Asian, 4% Multi-race; 24% subsidized lunch; 5% special education; 5% English as second language.

African American

Grade 2
 

DIBELS Next Composite

4,064

 

0.79

Hispanic

Grade 2
 

DIBELS Next Composite

3,687

 

0.83

Caucasian

Grade 3
 

DIBELS Next Composite

2,306

 

0.78

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 15% African American, 16% Hispanic, 2% Asian, 2% Multi-race; 14% subsidized lunch; 6% special education; 5% English as second language.

African American

Grade 3
 

DIBELS Next Composite

1,049

 

0.73

Hispanic

Grade 3
 

DIBELS Next Composite

1,121

 

0.81

Caucasian

Grade 4
 

DIBELS Next Composite

967

 

0.74

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 11% African American, 8% Hispanic, 3% Asian, 3% Multi-race; 14% subsidized lunch; 10% special education; 6% English as second language.

African American

Grade 4
 

DIBELS Next Composite

268

 

0.77

Hispanic

Grade 4
 

DIBELS Next Composite

203

 

0.78

Caucasian

Grade 5
 

DIBELS Next Composite

815

 

0.74

Validity was computed using TRC middle-of-year data to predict DIBELS Next composite score end-of-year data in school year 2011-2012 across 19 states. 9% African American, 11% Hispanic, 2% Asian, 2% Multi-race; 8% subsidized lunch; 11% special education; 6% English as second language.

African American

Grade 5
 

DIBELS Next Composite

191

 

0.69

Hispanic

Grade 5
 

DIBELS Next Composite

234

 

0.76

Disaggregated Predictive Validity of the Slope of Improvement

 

Type of
Validity

Age or
Grade

Test or Criterion

n
(range)

Coefficient

Information (including normative data)/Subjects

range

median

Concurrent Validity

Caucasian

K

DIBELS Next Composite

878

 

0.55

Validity of slope was computed using data from school year 2011-2012. 27% subsidized lunch; 2% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-35 assessments; mean=7.73).

African American

K

DIBELS Next Composite

384

 

0.58

Hispanic

K

DIBELS Next Composite

302

 

0.56

Caucasian

1

DIBELS Next Composite

2,094

 

0.67

Validity of slope was computed using data from school year 2011-2012. 36% subsidized lunch; 7% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-37 assessments; mean=8.17).

African American

1

DIBELS Next Composite

565

 

0.66

Hispanic

1

DIBELS Next Composite

596

 

0.62

Caucasian

2

DIBELS Next Composite

1,612

 

0.45

Validity of slope was computed using data from school year 2011-2012.  39% subsidized lunch; 8% special education; 4% English as second language. Weekly assessments over 12 months (i.e., 6-40 assessments; mean=8.27).

African American

2

DIBELS Next Composite

492

 

0.54

Hispanic

2

DIBELS Next Composite

409

 

0.44

Caucasian 3 DIBELS Next Composite 90   0.07 Validity of slope was computed using data from school year 2011-2012. 13% African American, 4% Hispanic, 26% special education. Weekly assessments over 12 months (i.e., 6-28 assessments; mean=8.47).
African American 3 DIBELS Next Composite 37   0.39 Validity of slope was computed using data from school year 2011-2012. 13% African American, 4% Hispanic, 26% special education. Weekly assessments over 12 months (i.e., 6-28 assessments; mean=8.47).
Hispanic 3 DIBELS Next Composite 12   0.35 Validity of slope was computed using data from school year 2011-2012. 13% African American, 4% Hispanic, 26% special education. Weekly assessments over 12 months (i.e., 6-28 assessments; mean=8.47).
Caucasian 4 DIBELS Next Composite 32   0.23 Validity of slope was computed using data from school year 2011-2012. 9% African American, 3% Hispanic; 32% special education. Weekly assessments over 12 months (i.e., 6-11 assessments; mean=7.19).
Caucasian 5 DIBELS Next Composite 10   0.34 Validity of slope was computed using data from school year 2011-2012. 4% African American. Weekly assessments over 12 months (i.e., 6-14 assessments; mean=7.22).

Predictive Validity

Caucasian

Grade 2
to predict
Grade 3

ISTEP+ ELA

1782

 

0.32

Validity of slope was computed using TRC data from school year 2010-2011 to predict ISTEP+ ELA data from school year 2011-2012.  38% subsidized lunch; 8% special education; 4% English as second language. Weekly assessments over 12 months (i.e., 6-38 assessments; mean=8.60).

African American

Grade 2
to predict
Grade 3

ISTEP+ ELA

483

 

0.37

Hispanic

Grade 2
to predict
Grade 3

ISTEP+ ELA

388

 

0.41

Caucasian

Grade K
to predict
Grade 1

DIBELS Next Composite

669

 

0.44

Validity of slope was computed using TRC data from school year 2010-2011 to predict DIBELS composite score from school year 2011-2012.   40% subsidized lunch; 2% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-20 assessments; mean=7.54).

African American

Grade K
to predict
Grade 1

DIBELS Next Composite

288

 

0.49

Hispanic

Grade K
to predict
Grade 1

DIBELS Next Composite

227

 

0.51

Caucasian

Grade 1
to predict
Grade 2

DIBELS Next Composite

1673

 

0.51

Validity of slope was computed using TRC data from school year 2010-2011 to predict DIBELS composite score from school year 2011-2012. 38% subsidized lunch; 6% special education; 5% English as second language. Weekly assessments over 12 months (i.e., 6-40 assessments; mean=8.72).

African American

Grade 1
to predict
Grade 2

DIBELS Next Composite

402

 

0.51

Hispanic

Grade 1
to predict
Grade 2

DIBELS Next Composite

390

 

0.57

Caucasian

Grade 2
to predict
Grade 3

DIBELS Next Composite

161

 

0.46

Validity of slope was computed using TRC data from school year 2010-2011 to predict DIBELS composite score from school year 2011-2012. 3% African American, 6% Hispanic, 1% Asian, 1% Multi-race; 29% subsidized lunch; 8% special education; 1% English as second language. Weekly assessments over 12 months (i.e., 6-19 assessments; mean=9.35).

African American

Grade 2
to predict
Grade 3

DIBELS Next Composite

27

 

0.45

Hispanic

Grade 2
to predict
Grade 3

DIBELS Next Composite

16

 

0.66

 

Alternate Forms: Empty bubble

1. Evidence that alternate forms are of equal and controlled difficulty or, if IRT based, evidence of item or ability invariance:

2. Number of alternate forms of equal and controlled difficulty:

There are between 40 and 300 texts available at each text level in TRC as the mCLASS home system allows teachers to add additional progress monitoring texts to system. Development of texts according to a leveling gradient as specified above ensures comparability of content among texts at each specific level. The table below provides information on the alternate form reliability of teacher-supplied materials by text level and grade. These values are computed as the median Phi-correlation between performance lables (FRU = 0; IND/INS = 1) on similar level texts that were administered to students within one month of each other during the 2012-2013 school year.

 

K

1

2

3

4

5

Text
Level

Phi

N

Phi

N

Phi

N

Phi

N

Phi

N

Phi

N

PC

0.30

8542

0.44

257

0.75

61

NA

25

NA

6

NA

1

RB

0.88

3494

0.49

491

NA

95

1.00

30

NA

4

NA

4

A

0.30

12280

0.20

4830

0.25

632

0.32

156

0.32

31

NA

19

B

0.55

11603

0.40

6359

0.45

826

0.31

181

0.50

45

NA

17

C

0.46

6121

0.29

7731

0.29

1400

0.34

284

0.25

61

NA

23

D

0.46

2814

0.30

7593

0.47

1747

0.48

340

0.28

72

0.34

33

E

0.59

1558

0.34

7005

0.28

2577

0.26

546

0.09

88

0.20

43

F

0.90

436

0.35

5299

0.27

3436

0.32

738

0.36

120

0.75

53

G

0.80

280

0.51

3782

0.31

3536

0.50

843

0.26

140

0.33

66

H

0.45

149

0.32

2218

0.26

3980

0.34

1052

0.22

192

0.53

70

I

0.82

97

1.00

2224

0.48

4781

0.20

1320

0.25

198

0.45

107

J

0.75

55

0.65

1850

0.39

4074

0.38

1629

0.30

264

0.18

123

K

0.30

36

0.84

1078

0.60

3511

0.26

2012

0.31

353

0.52

123

L

NA

19

0.97

630

0.65

2807

0.35

2171

0.37

374

0.27

134

M

NA

13

0.69

467

0.79

2301

0.48

2525

0.39

432

0.61

158

N

NA

10

0.85

268

0.88

1181

0.54

1667

0.23

422

0.43

190

O

NA

3

1.00

130

0.91

603

0.86

1047

0.53

406

0.36

226

P

NA

1

1.00

142

0.84

843

0.90

561

0.34

372

0.07

222

Q

NA

1

0.80

66

0.89

248

0.88

338

0.41

251

0.40

319

R

NA

0

1.00

48

0.81

212

0.92

242

0.46

249

0.45

460

S

NA

1

NA

26

0.91

170

0.89

215

0.64

151

0.49

468

T

NA

0

NA

5

0.94

98

1.00

139

0.80

76

0.49

235

U

NA

1

0.82

57

0.67

415

1.00

614

1.00

145

0.66

355

Median

0.55

 

0.65

 

0.63

 

0.48

 

0.34

 

0.44

 

 

Sensitive to Student Improvement: Full bubble

Describe evidence that the monitoring system produces data that are sensitive to student improvement (i.e., when student learning actually occurs, student performance on the monitoring tool increases on average):

Slopes on the progress-monitoring tool are significantly greater than zero; the slopes are significantly different for special education students vs. low-achieving vs. average-achieving vs. high-achieving students; and the slopes are significantly greater when effective practices (e.g., identified with high fidelity implementation) are in place.

Grade

Full Sample

Special Ed

Non Special Ed

Sample Size

Slope

SE of Slope

Sample Size

Slope

SE of Slope

Sample Size

Slope

SE of Slope

K

3,353

0.60

0.01

89

0.47

0.03

1,124

0.61

0.01

1

6,290

0.81

0.01

470

0.64

0.02

2,516

0.83

0.01

2

4,735

0.71

0.01

418

0.57

0.02

1,986

0.75

0.01

3

211

0.50

0.03

14

0.68

0.12

60

0.53

0.05

4

48

0.62

0.05

12

0.73

0.10

24

0.60

0.09

5

18

0.64

0.30

NA

NA

NA

NA

NA

NA

Grade

High Achieving

Average Achieving

Low Achieving

Sample Size

Slope

SE of Slope

Sample Size

Slope

SE of Slope

Sample Size

Slope

SE of Slope

K

1,048

0.44

0.01

630

0.55

0.01

1,115

0.77

0.01

1

1,897

0.65

0.01

999

0.87

0.01

1,728

1.01

0.01

2

1,399

0.65

0.01

768

0.77

0.01

1,410

0.77

0.01

3

47

0.44

0.04

19

0.46

0.06

53

0.44

0.04

4

16

0.50

0.08

6

0.79

0.16

5

0.87

0.11

5

NA

NA

NA

NA

NA

NA

NA

NA

NA

 

Grade

High Fidelity*

Low Fidelity*

Significance Test

Sample Size

Slope

SE of Slope

Sample Size

Slope

SE of Slope

Special Ed vs. not?

High vs. Average vs. Low Achieving Students?

High vs. Low fidelity?

K

24,507

1.69

0.01

6,676

1.04

0.01

No

Yes

Yes

1

25,438

2.41

0.01

5,668

2.08

0.02

No

Yes

Yes

2

26,841

1.56

0.01

3,574

1.96

0.02

Yes

Yes

Yes

3

1,017

1.78

0.04

320

1.63

0.08

Yes

Yes

Yes

4

597

2.03

0.06

289

1.94

0.10

No

No

Yes

5

567

1.12

0.05

188

1.89

0.12

NA

NA

Yes

 

End-of-Year Benchmarks: Full bubble

1. Are benchmarks for minimum acceptable end-of-year performance specified in your manual or published materials?

Yes.

a. Specify the end-of-year performance standards:

Four end-of-year performance standards are specified: Far Below Proficient, Below Proficient, Proficient, and Above Proficient.

b. Basis for specifying minimum acceptable end-of-year performance:

Criterion-referenced.

c. Specify the benchmarks:

The table below provides cut points for each grade and time of year. These are the minimum instructional reading levels required for each performance level at the end of each grade. Final instructional reading levels below the cut point for Below Proficient yield a performance level of Far Below Proficient. For example, a student in Grade 3 assessed using the Rigby edition achieves a final instructional reading level of Q. Since this text level is equal to or greater than the Proficient cut point of P, this student’s performance is categorized as Proficient.

 

 

Grade

 

 

 

 

 

TRC
Edition

Performance
Level

K

1

2

3

4

5

Rigby

Below
Proficient

C

H

L

N

R

T

 

Proficient

D

J

M

P

S

U

 

Above
Proficient

E

L

O

R

U

V

STEP and
Mondo

Below
Proficient

B

G

K

N

Q

T

 

Proficient

D-

J-

N

Q/P

T

V

 

Above
Proficient

E

L

O

R

V

*

d. Basis for specifying these benchmarks?

Criterion-referenced

Procedure for specifying benchmarks for end-of-year performance levels: 

The TRC performance standards indicate proficiency with respect to student performance against the grade-level expectations of the Common Core State Standards in English Language Arts. Cut points defining the performance standards were determined according to standard setting procedures during two workshops conducted for the Harcourt Rigby (“Rigby”) and STEP or Mondo editions of TRC. Please see the attached document (Amplify, 2013).

Amplify Insight (2013). mCLASS:Reading 3D - Text Reading and Comprehension: Using the Common Core State Standards for English Language Arts to Revise Performance Standards. Unpublished technical report.

Rates of Improvement Specified: mdash

Decision Rules for Changing Instruction: Empty bubble

Specification of validated decision rules for when changes to instruction need to be made: We recommend administering TRC three times a year for universal screening and benchmarking. After screening students to identify those in need of intervention, TRC can be used as a progress monitoring tool, as often as weekly or bi-weekly, to monitor student progress within their designated group, against performance standards based on national, longitudinal data and the expectations set by the Common Core State Standards for ELA.

Guidance for instructional decision-making is provided within our professional development sessions and all teacher support materials. Further, TRC summary reports are immediately available both at the student and classroom level. These reports allow teachers to view, at a glance, whether students have met their performance goals. Most importantly, the reports identify those students for whom further practice or intervention is required – students who have not achieved the Proficient performance level. TRC then provides teachers with instructional recommendations in the form of grade-level activities that are tailored to students’ individual needs and performance levels.

What is the evidentiary basis for these decision rules? TRC is based on the methodology of Curriculum-Based Measurement (CBM). Decision rules regarding changes in instruction stem from the basic principles behind CBM. Specifically that teachers use scores to improve instructional programs, especially in the case of students who do not seem to be benefitting from the current instructional course of action (Fuchs & Fuchs, 2007). This evidence is found in the summary reporting of TRC.

 

Decision Rules for Increasing Goals: Empty bubble

Specification of validated decision rules for when increases in goals need to be made: The goals are based on expectations of student reading behaviors described by the Common Core State Standards for English Language Arts.

Improved Student Achievement: Empty bubble

Description of evidence that teachers’ use of the tool results in improved student achievement based on an empirical study that provides this evidence.

Study: Wang, Y. & Gushta, M. (2013). Effectiveness of progress monitoring student literacy using mCLASS:Reading3D - TRC. Submitted for presentation at the 2014 annual meeting for the National Association of School Psychologists, Washington, DC.

Sample:

Number of students in product/experimental condition: 53,955  

Number of students in control condition: 365,512

Characteristics of students in sample and how they were selected for participation in studymCLASS: Reading 3D assessment is a Kindergarten through Fifth Grade reading assessment that supports educators in monitoring student reading progress and diagnosing reading difficulties. mCLASS: Reading 3D has two parts: DIBELS (Dynamic Indicators of Basic Early Literacy Skills) Next Edition and TRC (Text Reading and Comprehension). While some schools and teachers use DIBELS and TRC together, some schools and teachers only use DIBELS. This study described a quasi-experimental study to examine the effectiveness of TRC progress monitoring to improve literacy outcomes, which measured by DIBELS in Grade K-2. This study examined whether or not exempting students out of TRC administrations and progress monitoring leaded to differences in literacy achievement in DIBELS among students.

This study included nationwide data from a total of 45 states, 2521 schools, and 22864 teachers for this study. Data from 468,916 Kindergarten to Grade 2 students in 2011-2012 school year were analyzed. Among the 468,916 students, 53,955 were administered TRC with high fidelity while 365,512 students were exempted from TRC, and the rest 49,449 were administered TRC with low fidelity. Among the 468478 students, 41.63% were male, 39.42% were female; 28.8% were white, 9.38% were African American, 27.88% were Hispanic; 15.71% were English learners; 4.9% were special education students; 12.11 % were eligible for free or reduced priced lunch.  

Design:  As a statewide initiative supporting school-level implementation, random assignment to conditions was not available for this study. To ensure valid comparisons of student performance across the treatment and control groups, a statistical procedure known as Propensity Score Matching (Fan & Nowell, 2011) was employed to identify individual control group students who were closely matched to the treatment group students on the following characteristics: ethnicity, English as a second language, eligibility for free/reduced lunch, DIBELS Next scores at the beginning of the year. Logistic regression techniques were used to generate Propensity Scores for each student; control group students with Propensity Score values closest to students in the treatment group were identified and selected for subsequent analysis. The Propensity Score Matching was conducted separately for each grade. This method of subsetting the control group data is appropriate given the disproportionate number of students in the treatment versus the control group.

The end of school year performance on DIBELS of students who were progress monitored using TRC was compared to those who were exempted from TRC. Then the end of school year performance of students who were progress monitored using TRC with high fidelity implementation was compared to those who were exempted from TRC.

It was hypothesized that students who were administered TRC would demonstrate higher achievement scores on DIBELS than students who were exempted from TRC; furthermore, students who were administered TRC with high fidelity implementation would demonstrate even higher achievement scores on DIBELS than students who were exempted from TRC.

Unit of assignment: Students

Unit of analysis: Students

Duration of product implementation: One year

Describe analysisFollowing the identification of the nearest matching control group students, performance on DIBELS scores was subjected to a one-way Analysis of Variance (ANOVA) to determine the effect of treatment on student performance.

Fidelity:

Description of when and how fidelity of treatment information was obtained: We control the fidelity of implementation based on the criterion informed by NCRTI (2010). We applied the following criteria to identify high-fidelity administrations: (1) students were administered TRC screening tools at each benchmark period; (2) low-performing students who were identified as high-risk (i.e., “Far Below Proficient”) by the screening tools were progress monitored at least three times during the school year.

Results on the fidelity of treatment implementation measure:  We control the fidelity of implementation based on the criterion informed by NCRTI (2010). We applied the following criteria to identify high-fidelity administrations: (1) students were administered TRC screening tools at each benchmark period; (2) low-performing students who were identified as high-risk (i.e., “Far Below Proficient”) by the screening tools were progress monitored at least three times during the school year.

Measures:

Measure name

Reliability statistics (specify type of reliability, e.g. Cronbach’s alpha, IRT reliability, temporal stability, inter-rater)

DIBELS Next Composite (K-2)

Test-Retest Reliability: 0.81-0.94; Inter-Rater Reliability: 0.97-0.99.

 

Results:

Results of the study: The results from Kindergarten to Grade 2 suggested that, after controlling for the propensity score, TRC was positively associated with change in achievement test score (see Table 1 for the results). Students who were administered TRC had higher DIBELS composite scores than students who were not administered TRC. Furthermore, the effect sizes were even higher when we controlled the fidelity of implementation (also see Table 1 for the results). Overall, the effect size was highest in Kindergarten. The Hedges’ g effect size for Kindergarten was 0.28; effect sizes of 0.25 or greater are considered to be “substantively important” by the What Works Clearinghouse (USDOE, 2010).

 

Table 1. Results from Kindergarten to Grade 2.

 

Mean for Treatment Group

Mean for Control Group

F

p

Effect Size

TRC vs. No TRC

Kindergarten

145.41

133.60

107.7

<0.01

0.25

Grade 1

183.47

182.55

0.26

n. s.

0.01

Grade 2

278.27

265.43

54.61

<0.01

0.13

High Fidelity TRC vs. No TRC

Kindergarten

167.82

155.17

37.58

<0.01

0.28

Grade 1

200.26

192.60

11.59

<0.01

0.09

Grade 2

304.09

287.23

73.35

<0.01

0.20


Effect sizes for each outcome measure:

Measure name

Effect size

DIBELS Next composite score (Kindergarten)

0.28

DIBELS Next composite score (Grade 1)

0.09

DIBELS Next composite score (Grade 2)

0.20

Summary of conclusions and explanation of conditions to which effects should be generalizedOverall, progress monitoring on TRC provides broad, useful information about students’ skill levels and identifies those students in need of further assessments and development. This study used a national representative sample and used Propensity Score Matching to reduce bias across treatment and comparison conditions. The results of this study revealed the effectiveness of the high fidelity implementation of TRC in improving student reading achievement in kindergarten through second grade. Given the large, diverse nature of the sample, these results should generalize beyond the sample. The results also demonstrated the importance of fidelity control in examining the effectiveness of program practices.

Other related references or information:

Fan, X., & Nowell, D. L. (2011). Using propensity score matching in educational research. Gifted Child Quarterly, 55(1), 74-79.

National Center on Response to Intervention. (2010). Essential components of RTI—A closer look at response to intervention. Washington, DC: U.S. Department of Education, Office of Special Education Programs, National Center on Response to Intervention.

Improved Teacher Planning mdash