i-Ready Diagnostic for Mathematics

Area: Mathematics

Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support Purpose and Other Implementation Information Usage and Reporting

i-Ready Diagnostic for Mathematics costs $6.00 per student for an annual license which includes online student access to assessment, plus staff access to management and reporting suite, downloadable lesson plans, and user resources; account set-up and secure hosting; all program maintenance/updates/enhancements during the active license term; unlimited user access to U.S.-based service and support via toll-free phone and email during business hours.

Onsite professional development sessions (each lasting up to 3-4 hours for up to 30 participants) cost an additional $1,400/session. Live online webinars are available as a lower cost option for $500/session.

Computer access is required. 

Testers will require less than 4-8 hours of training.

Paraprofessionals can administer the test. 

Students with Disabilities

Curriculum Associates developed all items in i-Ready to be as accessible as possible for all students. In most cases, students who require accommodations will not require additional help during administration. The thoughtful planning that went into the general design of the assessment ensures that a large percentage of students requiring accommodations will have the necessary adjustments without compromising the interpretation or purpose of the test. i-Ready works with third-party (external) assistive technologies, such as screen readers and screen magnifiers.
Curriculum Associates, LLC
153 Rangeway Road
North Billerica, MA 01862
Phone: (800) 225-0248

www.curriculumassociates.com

www.i-ready.com/support

User Guide, video tutorials, and additional training and implementation resources are provided online to supplement onsite professional development.

Ongoing support is available from a dedicated account manager plus unlimited access to in-house technical support during business hours.
i-Ready Growth Monitoring (formerly known as Progress Monitoring) is a brief, computer-delivered, adaptive assessment in mathematics for students in grades 1 through 8. Growth Monitoring is part of the i-Ready Diagnostic & Instruction suite and is designed to be used jointly with i-Ready Diagnostic to allow for progress monitoring throughout the year to determine whether students are on track for appropriate growth. Growth monitoring is a periodic assessment that may be administered as frequently as every week in which the i-Ready Diagnostic assessment is not administered.

Growth Monitoring takes approximately 15 minutes and may be conducted with all students or with specific groups of students who have been identified as at risk of academic failure. i-Ready’s sophisticated adaptive algorithm automatically selects from thousands of multiple-choice and technology-enhanced items to get to the core of each student's strengths and challenges, regardless of the grade level at which he or she is performing. The depth of the item bank enables the assessment to truly pinpoint each student’s ability and ensures the accuracy of results.

The system automatically analyzes and scores student responses. Available as soon as a student completes the assessment, i-Ready’s intuitive Growth Monitoring reports—available at the student and class levels—focus solely on how students are tracking toward their end-of-year growth.

Reports show whether students are on track for their target growth by projecting where each student’s ability level will likely be at the end of the school year and comparing the projected growth-to-growth targets. For students performing below grade level, Growth Monitoring can be used as a tool for Response to Intervention (RTI) programs.

 

Reliability of the Performance Level Score: Partially Convincing Evidence

Type of Reliability

Age or Grade

n (range)

Coefficient Range

Coefficient Median

SEM

Diagnostic SEM in logit unit

Grade 1

557,200

 

 

0.25

Diagnostic SEM in logit unit

2

600,300

 

 

0.25

Diagnostic SEM in logit unit

3

625,000

 

 

0.25

Diagnostic SEM in logit unit

4

585,200

 

 

0.25

Diagnostic SEM in logit unit

5

557,900

 

 

0.25

Diagnostic SEM in logit unit

6

395,000

 

 

0.25

Diagnostic SEM in logit unit

7

353,200

 

 

0.25

Diagnostic SEM in logit unit

8

319,400

 

 

0.25

Growth Monitoring SEM in logit unit

1

566,000

 

 

0.46

Growth Monitoring SEM in logit unit

2

619,900

 

 

0.46

Growth Monitoring SEM in logit unit

3

684,900

 

 

0.46

Growth Monitoring SEM in logit unit

4

626,900

 

 

0.46

Growth Monitoring SEM in logit unit

5

570,000

 

 

0.46

Growth Monitoring SEM in logit unit

6

328,800

 

 

0.46

Growth Monitoring SEM in logit unit

7

252,300

 

 

0.46

Growth Monitoring SEM in logit unit

8

209,800

 

 

0.46

Test-Retest

1

83,241

 

0.72

 

Test-Retest

2

88,411

 

0.77

 

Test-Retest

3

91,921

 

0.78

 

Test-Retest

4

87,482

 

0.81

 

Test-Retest

5

85,781

 

0.82

 

Test-Retest

6

48,922

 

0.83

 

Test-Retest

7

43,273

 

0.83

 

Test-Retest

8

39,695

 

0.83

 

Information (including normative data) / Subjects:

*SEM: In an IRT model, SEMs are affected by factors such as how well the data fit the underlying model, student response consistency, student location on the ability continuum, match of items to student ability, and test length.  Given the adaptive nature of i-Ready and the wide difficulty range in the item bank, standard errors are expected to be low and very close to the theoretical minimum for the test of the given length. The theoretical minimum would be reached if each interim estimate of student ability is assessed by an item with difficulty matching perfectly to the student’s ability estimated from previous items. Theoretical minimums are restricted by the number of items served in the assessment—the more items that are served up, the lower the SEM could potentially be.

The Diagnostic and Growth Monitoring assessments are on the same scale, and are used in conjunction to project the probability of the student reaching the end of year targets. Therefore, reliability information for both assessments is provided in the table. Because of the different test lengths, the minimally obtainable SEMs are lower for the Diagnostic and higher for the Growth Monitoring.

The Diagnostic SEMs are based on all i-Ready Diagnostic assessments taken during the 2014–2015 school year. The minimally obtainable SEM for i-Ready Diagnostic Mathematics is 0.24. The Growth Monitoring SEMs are based on all i-Ready Growth Monitoring assessments taken during the 2014–2015 school year. The minimally obtainable SEM for i-Ready Growth Monitoring for mathematics is 0.45 logits.

*Test-retest: Evidence of test-retest stability was assessed based on a subsample of students who, during the 2014–2015 school year, took i-Ready Diagnostic twice within the recommended 12-18 week testing window. The average testing interval is 106 days (15 weeks). Correlations between the two tests were calculated. In lower grades where growth and variability are expected to be higher, test-retest correlations are expected to be relatively lower.

Reliability of the Slope: Data Unavailable

Type of Reliability

Age or Grade

n (range)

Coefficient Range

Coefficient Median

SEM

Split-half

1

2,749

 

0.81

 

Split-half

2

2,655

 

0.80

 

Split-half

3

3,075

 

0.78

 

Split-half

4

2,381

 

0.79

 

Split-half

5

2,196

 

0.77

 

Split-half

6

981

 

0.81

 

Split-half

7

672

 

0.79

 

Split-half

8

467

 

0.84

 

Information (including normative data) / Subjects:

Reliability of the slope is calculated as the split-half reliability based on the method descried in the NCII’s Frequently Asked Questions. The analysis sample was drawn from the 2014-2015 production student pool. Students included in the analysis had an average 17 assessments—including both Growth Monitoring and Diagnostic assessments—ranging from 15 to 39. The testing period spanned across an average of nine months, ranging from three to 11 months. 

Validity of the Performance Level Score: Partially Convincing Evidence

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient Range

Coefficient Median

Concurrent/Construct

Grade 3

2014 New York State Assessments

1,029

 

0.81

Concurrent/Construct

Grade 4

2014 New York State Assessments

969

 

0.81

Concurrent/Construct

Grade 5

2014 New York State Assessments

992

 

0.82

Concurrent/Construct

Grade 6

2014 New York State Assessments

988

 

0.84

Concurrent/Construct

Grade 7

2014 New York State Assessments

806

 

0.82

Concurrent/Construct

Grade 8

2014 New York State Assessments

741

 

0.78

Concurrent/Construct

Grade 3

2015 SBAC Assessments

5,945

 

0.82

Concurrent/Construct

Grade 4

2015 SBAC Assessments

5,853

 

0.81

Concurrent/Construct

Grade 5

2015 SBAC Assessments

5,886

 

0.82

Concurrent/Construct

Grade 6

2015 SBAC Assessments

6,551

 

0.84

Concurrent/Construct

Grade 7

2015 SBAC Assessments

5,989

 

0.84

Concurrent/Construct

 

Grade 8

2015 SBAC Assessments

5,263

 

0.83

Predictive

Grade 1

2015 New York State Assessments

778

 

0.71

Predictive

Grade 2

2014 New York State Assessments

878

 

0.73

Predictive

 

Grade 3

2015 SBAC Assessments

5,918

 

0.75

Predictive

 

Grade 4

2015 SBAC Assessments

5,905

 

0.77

Predictive

 

Grade 5

2015 SBAC Assessments

5,899

 

0.78

Predictive

 

Grade 6

2015 SBAC Assessments

7,553

 

0.81

Predictive

 

Grade 7

2015 SBAC Assessments

7,225

 

0.82

Predictive

 

Grade 8

2015 SBAC Assessments

6,748

 

0.82

Information (including normative data) / Subjects:

*Concurrent validity with NYS assessments: Correlations between 2014 New York State assessments and the i-Ready Diagnostic spring performance score. Testing interval between the two assessments is within six weeks.

*Concurrent validity with SBAC assessments: Correlations between the spring i-Ready Diagnostic performance score and spring SBAC assessments. Testing interval is between 1–3 months. Sample is from four states, 17 districts.

* Grade 1 Predictive validity with New York State: Correlations between the spring i-Ready Diagnostic performance score at grade 1 and students’ New York State assessment scores two years later at grade 3.

* Grade 2 Predictive validity with New York State: Correlations between the spring i-Ready Diagnostic performance score at grade 2 and students’ New York State assessment scores one year later at grade 3.

*Predictive validity with SBAC: Correlations between the fall i-Ready Diagnostic performance score and spring SBAC assessments. Testing interval is between 5-8 months.

Other forms of validity:

Content Validity: As part of the accumulation of evidence for the valid use of an instrument, it is important that the test is measuring the intended construct. Much of this evidence is obtained through the four building block process. As Mark Wilson explains, “…documentation of the steps taken… constitute a thorough representation of the content validity evidence for the instrument. It also lays the foundation for the remaining aspects of validity evidence—in a sense, the evidence related to content is the target of the remaining evidence”. A detailed discussion of the development process as it relates to test content is detailed in Chapter 2 of the attached i-Ready Technical Manual. 

Predictive Validity of the Slope of Improvement: Data Unavailable

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient Range

Coefficient Median

Predictive

Grade 1

2015 New York State Assessments

31-212

0.36-0.69

0.53

Predictive

2

2014 New York State Assessments

19-185

0.27-0.74

0.42

Predictive

3

2015 SBAC Assessments

476-710

0.34-0.59

0.55

Predictive

4

2015 SBAC Assessments

513-686

0.32-0.58

0.50

Predictive

5

2015 SBAC Assessments

488-653

0.33-0.52

0.48

Predictive

6

2015 SBAC Assessments

569-771

0.26-0.59

0.48

Predictive

7

2015 SBAC Assessments

505-692

0.27-0.52

0.40

Predictive

8

2015 SBAC Assessments

479-629

0.18-0.49

0.41

Information (including normative data) / Subjects:

Grade 1. Student’s slope of improvement was obtained from the i-Ready assessments taken in 2012-2013 school year. For each student, the slope of improvement was estimated based on the student’s assessment scores during the school year and the testing dates for each assessment. The average number of months spanned between the first and last assessment was 8.5 months, ranging from 6 months to 9.7 months.

Correlations between the slope and these students’ grade 3 (two years after the last assessment used in slope estimate) New York State assessment scores were calculated. Correlations were calculated within decile ability level, according to the i-Ready national norms, based on the student’s initial i-Ready Diagnostic mathematics score.

Grade 2. Student’s slope of improvement was obtained from the i-Ready assessments taken in 2012-2013 school year. For each student, the slope of improvement was estimated based on the student’s assessment scores during the school year and the testing dates for each assessment. The average number of months spanned between the first and last assessment was 8 months, ranging from 5.5 months to nine months.

Correlations between the slope and these students’ grade 3 (one year after progress monitoring ended) New York State assessment scores were calculated. Correlations were calculated within decile ability level, according to the i-Ready national norms, based on the student’s initial i-Ready Diagnostic mathematics score.

Grades 3–8. Student’s slope of improvement is obtained from the i-Ready assessments taken in 2014-2015 school year. The number of months spanned across the i-Ready assessments ranges from five to nine months. Correlations between the slope and students’ end-of-year SBAC assessment scores were calculated. Correlations were calculated within decile ability level, according to the i-Ready national norms, based on the student’s initial i-Ready Diagnostic math score.

Disaggregated Reliability and Validity Data: Data Unavailable

Disaggregated Reliability of Performance Level Score

Type of Reliability

Subgroup

Age or Grade

n (range)

Coefficient Range

Coefficient Median

SEM

Diagnostic SEM in logit unit

Economically disadvantaged

Grades K-12

712,000

 

 

Range 0.246-0.252; median 0.250

Diagnostic SEM in logit unit

ELL

Grades K-12

604,000

 

 

Range 0.248-0.252; median 0.250

Diagnostic SEM in logit unit

Special Education

Grades K-12

337,000

 

 

Range 0.250-0.252; median 0.250

Information (including normative data) / Subjects:

Data are based on the i-Ready 2014-2015 operational pool of students who were identified with each special group membership. The minimally obtainable SEM for i-Ready Diagnostic for mathematics is 0.24.

Disaggregated Reliability of Slope of Improvement

Type of Reliability

Subgroup

Age or Grade

n (range)

Coefficient Range

Coefficient Median

SEM

Split-half

Asian

1

505

 

0.80

 

Split-half

African American

1

4,484

 

0.74

 

Split-half

Hispanic

1

1,757

 

0.80

 

Split-half

Asian

2

451

 

0.76

 

Split-half

African American

2

4,962

 

0.74

 

Split-half

Hispanic

2

1,846

 

0.75

 

Split-half

Asian

3

419

 

0.74

 

Split-half

African American

3

5,111

 

0.74

 

Split-half

Hispanic

3

1,834

 

0.74

 

Split-half

Asian

4

356

 

0.73

 

Split-half

African American

4

4,341

 

0.73

 

Split-half

Hispanic

4

1,584

 

0.73

 

Split-half

Asian

5

317

 

0.70

 

Split-half

African American

5

3,959

 

0.72

 

Split-half

Hispanic

5

1,248

 

0.71

 

Split-half

Asian

6

269

 

0.74

 

Split-half

African American

6

1,945

 

0.74

 

Split-half

Hispanic

6

833

 

0.71

 

Split-half

Asian

7

93

 

0.75

 

Split-half

African American

7

1,613

 

0.75

 

Split-half

Hispanic

7

482

 

0.72

 

Split-half

Asian

8

144

 

0.82

 

Split-half

African American

8

1,163

 

0.78

 

Split-half

Hispanic

8

357

 

0.78

 

Information (including normative data) / Subjects:

Disaggregated reliability of the slope is calculated using the same split-half method described for GOM 2. Students with the designated ethnicity group membership were drawn from the 2014-2015 production pool. Students included in the analysis had at least 10 assessments—including both Growth Monitoring and Diagnostic assessments—during the school year.

Disaggregated Validity of Performance Level Score

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n (range)

Coefficient Range

Coefficient Median

Concurrent

ELL

Grade 3-8

2014 New York State Assessments

267-374

0.82-0.87

0.85

Concurrent

African American

Grade 3-8

2014 New York State Assessments

137-222

0.69-0.79

0.75

Concurrent

Hispanic

Grade 3-8

2014 New York State Assessments

125-194

0.79-0.86

0.83

Information (including normative data) / Subjects:

Students who took both 2014 New York State assessments and the spring i-Ready Diagnostic assessment. The testing interval is six weeks.

Alternate Forms: Convincing Evidence

What is the number of alternate forms of equal and controlled difficulty?

Virtually infinite. As a computer-adaptive test, in i-Ready all administrations are equivalent forms. However, each student is presented with an individualized testing experience where he or she is served test items based on answer choices to previous questions. In essence, this scenario provides a virtually infinite number of test forms, because individual student testing experiences are largely unique.

If IRT based, provide evidence of item or ability invariance:

Section 2.1.3 in Chapter 2 of the i-Ready Technical Manual describes the adaptive nature of the tests and how the item selection process works. The i-Ready Growth Monitoring assessments are a general outcome measure of student ability and measure a subset of skills that are tested on the diagnostic assessment. Items on Growth Monitoring are from the same domain item pool for the diagnostic. Test items are served based on the same IRT ability estimate and item selection logic.

If computer administered, how many items are in the item bank for each grade level?

For grades 1–8, typical item pool sizes are 741, 1022, 1345, 1560, 1887, 2126, 2336, and 2474, respectively. Students who perform at an extremely high level will be served with items from grade levels higher than the grade level restriction. 

If your tool is computer administered, please note how the test forms are derived instead of providing alternate forms:

 i-Ready Diagnostic and Growth Monitoring tests are computer adaptive, meaning the items presented to each student vary depending upon how the student has responded to the previous items. Upon completion of an item randomly selected from a set of five items around a predetermined starting difficulty level, interim ability estimates are updated and the program chooses the next item relative to the new interim ability estimate. Thus, the items can better target the estimated student ability, and more information is obtained from each item presented.

Sensitive to Student Improvement: Convincing Evidence

End-of-Year Benchmarks: Convincing Evidence

Are benchmarks for minimum acceptable end-of-year performance specified in your manual or published materials?

Yes

Specify the end-of-year performance standards:

For a Response to Intervention Framework, the “early on grade level” cut score is the minimum end-of-year performance for students to no longer be considered as Tier 2. These cut scores were developed using a Contrasting Groups standard setting in spring 2014. More information on this process is available in Chapter 6 of the i-Ready Technical Manual

What is the basis for specifying minimum acceptable end-of-year performance?

Criterion-referenced

If criterion-referenced, describe procedure for specifying benchmarks for end-of-year performance levels:

Because i-Ready Growth Monitoring and i-Ready Diagnostic are on the same scale, we utilized the research from the Diagnostic performance levels to determine whether students were on track to make at least on grade level performance by the end of year. Details on how these placement cut scores were set can be found in the i-Ready Technical Manual.

Rates of Improvement Specified: Convincing Evidence

Is minimum acceptable growth (slope of improvement or average weekly increase in score by grade level) specified in your manual or published materials?

Yes

Specify the growth standards:

For grades 1–8, our growth targets over a 30-week period are 32, 30, 28, 22, 22, 13, 13, and 13, respectively.

What is the basis for specifying minimum acceptable growth?

Criterion-referenced

If criterion-referenced, describe procedure for specifying criterion for adequate growth:

The setting of the Diagnostic performance levels in each grade was based on four years of research on data collected from national panels of accomplished teachers and from statewide testing programs. These performance levels reflect the knowledge and skill levels of students who are “early on grade level” and “mid on grade level” in each grade and subject area.

The i-Ready growth targets in each grade and subject area stem from these performance levels, and reflect the levels of progress expected with respect to the knowledge and skills targeted by i-Ready Diagnostic and the Common Core State Standards in each grade level. Specifically, a modified Bookmark standard setting was used to determine criterion-referenced growth targets, which were launched in the system in 2013. Appendix L in the i Ready Technical Manual provides information on how the historical criterion-referenced growth targets were calculated.

Because i-Ready Diagnostic underwent a recalibration for the 2014–2015 school year and a new contrasting Groups standard setting was conducted in spring 2014, a rigorous review of the growth targets was conducted in summer 2015 to determine if changes to these growth targets should be made. The detailed descriptions of the standard-setting process and setting the criterion-referenced growth targets are provided in Chapter 6 of the i Ready Technical Manual. 

Decision Rules for Changing Instruction: Data Unavailable

Decision Rules for Increasing Goals: Data Unavailable

Improved Student Achievement: Data Unavailable

Improved Teacher Planning Data Unavailable