Edcheckup Standard Reading Passages

Area: Maze

Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support Purpose and Other Implementation Information Usage and Reporting

Each license (or “set”) allows you to access materials, enter data, and generate reports for up to 35 students.

The cost per set is as follows:

  • If purchasing 1 – 10 sets, $100 per set
  • If purchasing 11 – 20 sets, $95 per set
  • If purchasing 21 – 100 sets, $90 per set
  • If purchasing 101 – 200 sets, $85 per set
  • If purchasing 201+ sets, $80 per set
This includes all materials and full data/reporting functionality for all subjects.

Computer and Internet access are required for full use of product services.

Testers will require less than 1 hour of training.

Paraprofessionals can administer the test.

Edcheckup, LLC
7701 York Ave S, Ste 250
Edina, MN 55435

Website:  www.edcheckup.com

Field-tested training manuals are included and should provide all implementation information.

Ongoing technical support is available.

Edcheckup Maze consists of twenty passages at the same difficulty level for use in repeated 1 minute samples of silent reading of text making maze selections (3 options). The tester counts the number of maze choices correct and incorrect for the raw score which is them entered into the Edcheckup online system.

Edcheckup is group or individually administered and requires 2 minutes for administration and 1 additional minute per student for scoring.

Raw scores are available.  

20 alternate forms are available.

 

Reliability of the Performance Level Score: Convincing Evidence

Type of Reliability Age or Grade n (range) Coefficient SEM Information / Subjects
range median
Alpha 1 432 -442 0.87-0.91 0.88 Median = 0.15 Median Standardized item Alpha = 0.88,States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Alpha 2 536 -547 0.90-0.95 0.93 Median = 0.20 Median Standardized item Alpha = 0.94, States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Alpha 3 450 -457 0.89-0.93 0.91 Median = 0.22 Median Standardized item Alpha = 0.92,States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Alpha 4 490 -514 0.90-0.95 0.92 Median = 0.25 Median Standardized item Alpha = 0.93, States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Alpha 5 366 -375 0.90-0.96 0.92 Median = 0.34 Median Standardized item Alpha = 0.93, States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Alpha 6 651 -714 0.92-0.96 0.93 Median = 0.40 Median Standardized item Alpha = 0.94, States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Parallel Forms 1 366-541 0.85-0.91 0.88 Median = 0.15 States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Parallel Forms 2 313-510 0.91-0.95 0.93 Median = 0.20 States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Parallel Forms 3 337-450 0.90-0.93 0.92 Median = 0.22 States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Parallel Forms 4 254-481 0.90-0.96 0.92 Median = 0.25 States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Parallel Forms 5 122-343 0.90-0.96 0.93 Median = 0.34 States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey
Parallel Forms 6 122-216 0.91-0.96 0.95 Median = 0.40 States represented: Minnesota, Pennsylvania, Colorado, Indiana, Arizona, Wisconsin, Washington, Georgia, South Dakota, and New Jersey

Reliability of the Slope: Unconvincing Evidence

Type of Reliability Age or Grade n (range) Coefficient SEM Information / Subjects
range median
Standard Error of Estimate 2-3 12   Slope= 0.139 1.74 Students from Upper Midwest.
Standard Error of Estimate 4-5 12   Slope= 0.306 1.84 Students from Upper Midwest
Reliability of Slope 3-5 14   Slope= 0.184 1.68 Reliability of Slope was .747 (True Slope Variance/Total Slope Variance) for students from Upper Midwest. The range of data points was 18-25 with a mean of 22.1. The range in months was 4.5-6.3 months with a mean of 5.5 months.

The table below reports reliability of the slope coefficients calculated with HLM.

    HLM
  Sample Slope
Grade Size Reliability
     
1 305 0.43
2 200 0.34
3 201 0.15
4 204 0.22
5 84 0.15
6 195 0.20

Validity of the Performance Level Score: Convincing Evidence

Type of Validity Age or Grade Test or Criterion n (range) Coefficient Information / Subjects
range median
Construct 2 Measure of Academic Progress 211   0.628 16.6% Students of Color, 7.1%, Students of Poverty, 13.3% Students with Disabilities
Construct 3 Measure of Academic Progress 233   0.502 19.5% Students of Color, 7.1%, Students of Poverty, 14.2% Students with Disabilities
Construct 4 Measure of Academic Progress 204   0.566 22.3% Students of Color, 5.9%, Students of Poverty, 11.9% Students with Disabilities
Construct 5 Measure of Academic Progress 249   0.591 20.4% Students of Color, 5.4%, Students of Poverty, 11.2% Students with Disabilities
Construct 6 Measure of Academic Progress 75   0.502 34.7% Students of Color, 17.3%, Students of Poverty, 10.7% Students with Disabilities
Predictive 2 Measure of Academic Progress 200   0.648 16.4% Students of Color, 5.8%, Students of Poverty, 12.6% Students with Disabilities
Predictive 3 Measure of Academic Progress 224   0.550 18.7% Students of Color, 5.9%, Students of Poverty, 14.2% Students with Disabilities
Predictive 4 Measure of Academic Progress 193   0.533 22.1% Students of Color, 5.1%, Students of Poverty, 11.3% Students with Disabilities
Predictive 5 Measure of Academic Progress 158   0.511 17.2% Students of Color, 3.2%, Students of Poverty, 8.3% Students with Disabilities
Predictive 1 Oral Reading 188   0.65  
Predictive 6 Oral Reading 144   0.63  

Predictive Validity of the Slope of Improvement: Convincing Evidence

To examine the consistency of performance on the maze format for these passages, ten of the passages designed to monitor the growth of Grade 2 students were administered to a group of 43 Grade 2 students (25 male and 18 female) once per month for the 9 month academic year.  The results of this study are presented in the Journal of Special Education (Shin, Deno & Espin, 2000). To determine whether the passages produced a consistent growth pattern the monthly scores were submitted to HLM analysis.  The monthly mean growth rate (+1.07) differed significantly from zero.  The Standard Error of Estimate (SEE) for the growth rate was 0.09.  The ratio of the SEE to the mean growth rate was 0.08.

In the Shin, et al study (2000) the sensitivity  of the measure to inter-individual differences was examined and the estimated parameter variance (0.25) was significantly different from zero – meaning that the obtained differences in individual growth rates could be consistently estimated. The validity of the growth rate obtained from the maze measure was examined with HLM. Second, a statistically significant difference in the growth rates was obtained for students performing differently on the California Achievement Test (CAT).  Students with CAT scores 1 SD above the mean grew on the maze passages at a rate 0.33 greater than students at the mean.  Differences in growth rates were not obtained for general and remedial students, but large initial differences in level (intercept) between those two groups were statistically significant.

Additional direct evidence regarding the sensitivity of the measure to differences in academic competence is derived from the studies by Espin, et al (1989),  Elbaum & Vaughn (1997), and Deno, et al (2004) where the relationship of student performance on the maze measures to differences in achievement level or student status were examined. In addition, evidence exists that the maze passages can also be used to evaluate program effects in terms of the amount of growth made by special education students in different inclusive programs (Zigmond, et al, 1995).

For these analyses the Maze dataset includes 364 students with 9 or more measures where at least 14 days had passed between third to last measures. The numbers of students meeting Only the WC measures were used in the Maze regression analyses. For sample sizes below 30, the analysis was not run.

With this sample, HLM was used to estimate individual student slopes across all but the last three measures. A separate HLM was run for each grade. These slopes were then used in a regression model to predict the average of the last three time points, called AVG, again by grade. Results from the regression analyses are included in the table below.

 

Maze Regression Results for Slopes predicting AVG by Grade

Grade N R F p-value
1 126 -0.10 1.33 0.251
2 62 -0.21 2.74 0.103
3 68 -0.05 0.14 0.710
4 55 -0.23 3.07 0.085
5 16 - - -
6 30 0.08 0.17 0.682

 

Disaggregated Reliability and Validity Data: Unconvincing Evidence

Disaggregated Reliability of the Performance Level Score

Type of Reliability Age or Grade n (range) Coefficient SEM Information / Subjects
range median
Cronbach’s Alpha 2 19 0.9554-0.9658 0.9633 Median = 9.50 African American
Alpha = 0.9844
Standardized item Alpha = 0.9868
Cronbach’s Alpha 3 20 0.8162-0.8811 0.8173 Median = 8.51 African American
Alpha = 0.9382
Standardized item Alpha = 0.9396
Cronbach’s Alpha 4 25 0.8720-0.9018 0.8989 Median = 7.01 African American
Alpha = 0.9543
Standardized item Alpha = 0.9608
Cronbach’s Alpha 5 25 0.9011-0.9289 0.9241 Median = 6.40 African American
Alpha = 0.9671
Standardized item Alpha = 0.9711
Split-half 2 19 0.9554-0.9658 0.9633 Median = 9.50 African American
Guttman Split-half = 0.8853, Unequal-length Spearman-Brown = 0.9890.
Split-half 3 20 0.8162-0.8811 0.8173 Median = 8.51 African American
Guttman Split-half = 0.8167, Unequal-length Spearman-Brown = 0.9224.
Split-half 4 25 0.8720-0.9018 0.8989 Median = 7.01 African American
Guttman Split-half = 0.8652, Unequal-length Spearman-Brown = 0.9556.
Split-half 5 25 0.9011-0.9289 0.9241 Median = 6.40 African American
Guttman Split-half = 0.8675, Unequal-length Spearman-Brown = 0.9772.

Disaggregated Validity of the Performance Level Score1

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient Information / Subjects
range median
Construct 2 Measure of Academic Progress 18   0.854 African American students
Construct 3 Measure of Academic Progress 20   0.774 African American students
Construct 4 Measure of Academic Progress 25   0.688 African American students
Construct 5 Measure of Academic Progress 25   0.580 African American students
Predictive 3 Minnesota Comprehensive Assessment 15   0.634 African American students
Predictive 4 Minnesota Comprehensive Assessment 19   0.436 African American students
Predictive 5 Minnesota Comprehensive Assessment 26   0.511 African American students
Predictive 6 Minnesota Comprehensive Assessment 16   0.437 African American students

Alternate Forms: Convincing Evidence

1. Evidence that alternate forms are of equal and controlled difficulty or, if IRT based,

Evidence of controlled difficulty for the passages used for progress monitoring at each grade was determined through use of the Flesch-Kinkaid Grade Level readability formula and the Lexile Framework for Reading (Metametrics, 2002). Both analyses indicate the passages were placed at appropriate grade levels. At grade 1 the passages had an average Flesch readability of 1.6 and Lexile of 272. At grade 2 the average Flesch readability was 2.4 and Lexile of 397. At grade 3 the average Flesch score was 3.3 and Flesch was 511. At grade 4 the average readability was 4.2 and Lexile was 656. The grade 5 Flesch readability was 5.8 and Lexile score was 759. Grade 6 Flesch readability was 6.9 and Lexile was 850.

Readability estimates for each probe (20) at each grade level (6). For each passage we list below the Lexile rating, as scored by Metametrics, and the Flesch readability score.

Grade Passage Lexile Flesch   Grade Passage Lexile Flesch
1 1 140 1.3   2 1 360 2.9
1 2 180 2.5   2 2 250 2.7
1 3 190 0.4   2 3 220 3.2
1 4 200 2.4   2 4 260 3.0
1 5 210 1.1   2 5 280 2.9
1 6 220 2.2   2 6 290 2.9
1 7 250 2.4   2 7 310 2.9
1 8 260 2.5   2 8 310 2.6
1 9 270 2.1   2 9 320 2.9
1 10 300 2.1   2 10 330 2.9
1 11 320 2.5   2 11 340 3.2
1 12 330 2.4   2 12 350 2.9
1 13 330 2.4   2 13 380 2.9
1 14 340 2.2   2 14 380 2.9
1 15 350 2.0   2 15 400 3.0
1 16 360 2.1   2 16 410 3.1
1 17 380 2.3   2 17 340 3.0
1 18 390 1.9   2 18 470 2.9
1 19 430 1.0   2 19 400 2.6
1 20 280 0.8   2 20 370 2.9

 

Grade Passage Lexile Flesch   Grade Passage Lexile Flesch
3 1 400 3.3   4 1 410 4.4
3 2 420 3.3   4 2 580 3.9
2 3 320 3.3   4 3 760 6.1
3 4 340 3.9   4 4 530 4.3
3 5 420 3.9   4 5 510 4.2
3 6 460 3.5   4 6 530 4.3
3 7 480 3.8   4 7 600 4.7
3 8 490 3.7   4 8 640 4.3
3 9 520 3.3   4 9 640 4.7
3 10 520 3.8   4 10 650 4.7
3 11 530 3.5   4 11 680 4.2
3 12 540 4.0   4 12 680 4.2
3 13 550 4.0   4 13 710 5.4
3 14 560 3.7   4 14 740 4.7
3 15 580 3.8   4 15 760 4.6
3 16 600 4.0   4 16 780 4.6
3 17 610 3.5   4 17 390 4.8
3 18 630 4.0   4 18 580 4.8
3 19 720 3.7   4 19 360 4.8
3 20 760 3.1   4 20 470 4.8

 

Grade Passage Lexile Flesch   Grade Passage Lexile Flesch
5 1 570 5.3   6 1 810 7.7
5 2 620 5.5   6 2 790 7.5
5 3 630 5.3   6 3 800 7.5
5 4 670 5.2   6 4 850 7.0
5 5 670 5.7   6 5 880 6.9
5 6 470 5.1   6 6 910 7.0
5 7 690 6.3   6 7 750 6.4
5 8 700 5.1   6 8 920 7.5
5 9 750 5.6   6 9 920 7.4
5 10 790 6.0   6 10 950 6.8
5 11 790 5.9   6 11 950 6.4
5 12 850 6.3   6 12 990 6.5
5 13 860 5.9   6 13 1060 7.5
5 14 670 5.2   6 14 770 6.1
5 15 970 5.7   6 15 470 6.5
5 16 1020 6.5   6 16 550 6.9
5 17 810 5.2   6 17 610 6.1
5 18 520 5.2   6 18 600 6.2
5 19 630 5.3   6 19 640 6.0
5 20 580 5.1   6 20 950 7.5

2. Number of alternate forms of equal and controlled difficulty:

20

Sensitive to Student Improvement: Convincing Evidence

1. Describe evidence that the monitoring system produces data that are sensitive to student improvement (i.e., when student learning actually occurs, student performance on the monitoring tool increases on average).

Evidence of the sensitivity to student improvement by providing the monthly growth rates for Edcheckup maze reading passages for grades 1 to 6 is below. These growth rates are comparable to those currently available in the literature on this topic.

Grade Sample Size Average Monthly Growth
1 N = 177 0.40
2 N = 196 0.64
3 N = 224 0.60
4 N = 222 0.52
5 N = 74 0.64
6 N = 217 0.64

End-of-Year Benchmarks: Convincing Evidence

1. Are benchmarks for minimum acceptable end-of-year performance specified in your manual or published materials?

No.

a. Specify the end-of-year performance standards:

b. Basis for specifying minimum acceptable end-of-year performance:

c. Specify the benchmarks:

Benchmark goals are not included in the procedures for Maze reading. As with oral reading we do specify growth standards that ultimately translate into individual benchmark scores. The following results from two of our studies provide the basis for specifying benchmark scores based on normative data.

Our first study consists of data already analyzed for GOM 2. In that study the 50th percentile at each grade level was consistent with the benchmarks reported above. Grade 1: Median = 3.7, SD = 2.8, N = 177; Grade 2: Median = 7.3, SD = 37.5, N = 3.6; Grade 3: Median = 9.7, SD = 3.3, N = 224; Grade 4: Median = 10.7, SD = 4.2, N = 222; Grade 5: Median = 12.2, SD = 4.9, N = 74, Grade 6: Median = 13.4, SD = 13.4, N = 217.

In addition to the data in this analysis we provided, the following means were obtained in our Edcheckup data when submitting the Standard Protocol for screening purposes. Grade 2: Mean = 7.6, SD = 3.1, N = 211; Grade 3: Mean = 8.6, SD = 3.1, N = 233; Grade 4: Mean = 8.7, SD = 2.9, N = 204; Grade 5: Mean = 11.6, SD = 4.2, N = 249; Grade 6: Mean = 12.1, SD = 3.4, N = 75.

d. Basis for specifying these benchmarks?

Rates of Improvement Specified: Unconvincing Evidence

1. Is minimum acceptable growth (slope of improvement or average weekly increase in score by grade level) specified in manual or published materials?

Yes.

a. Specify the growth standards:

Below is a table specifying growth rates found at the 20th, 50th, and 80th percentiles of our sample.

Grade Percentile Fall Spring Average Weekly Growth
Grade 1
(N = 177)
20 0.00 1.33 0.04
50 0.33 3.67 0.10
80 1.67 6.67 0.16
Grade 2
(N = 198)
20 1.00 4.67 0.11
50 2.33 7.33 0.16
80 5.33 9.67 0.14
Grade 3
(N = 224)
20 2.67 7.00 0.14
50 5.00 9.67 0.15
80 7.00 12.67 0.18
Grade 4
(N = 222)
20 4.00 7.00 0.09
50 6.67 10.67 0.13
80 8.47 14.00 0.17
Grade 5
(N = 74)
20 4.67 7.33 0.08
50 7.00 12.17 0.16
80 10.00 14.67 0.15
Grade 6
(N = 217)
20 4.33 8.53 0.13
50 10.00 15.00 0.16
80 18.80 28.50 0.30

b. Basis for specifying minimum acceptable growth:

The Edcheckup PM system is structured so that teachers can choose from among 3 different growth rates at each grade level to set appropriate goals for their students. Those growth rates (termed “Modest,” “Reasonable,” and “Ambitious”) allow the teachers to choose growth rates for individual students that are consistent with their screening levels of performance and with the teacher’s knowledge of the student. The choice of growth rate then becomes a multiplier for individual goal setting and for setting up a progress graph that becomes a realistic basis for subsequent instructional decision making and program modifications. Once the student’s progress graph has been created, feedback on the degree of acceptability of student progress is provided by displaying the student’s scores in 3 color-coded ranges of risk ---exceeding desired growth, at desired growth, below desired growth. Edcheckup developers used growth rates derived from a range of empirical research articles as a basis for establishing the desired growth rates for each grade level including the following:*

Deno, S. L., Reschly, A.L., Lembke, E. S., Magnusson, D., Callendar, S. A., Windram, H., & Stachel, N (2009) Developing a School-wide Progress Monitoring System. Psychology in the Schools. 16(1) 44-55.

Deno, S. L., Maruyama, G., Espin, C. A., & Cohen, C. (1990) Educating students with mild disabilities in general education classrooms: Minnesota alternatives. Exceptional Children, 57(2), 150-161.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22(1), 27-48.

Brown-Chidsey, Rachel, Johnson, P. & Fernstrom, R.(2005) Comparison of Grade-Level Controlled and Literature-Based Maze CBM Reading Passages School Psychology Review. 34 (3) pp. 387-394

Shin, J., Deno, S. L., & Espin, C. (2000). Technical adequacy of maze probes for Curriculum-Based Measurement of reading growth. The Journal of Special Education 34(3), 140-153.

Wiley, H. I. & Deno, S. L. (2005) Oral Reading and Maze Measures as Predictors of success for Limited English Proficient Students on a State Standards Assessment. Remedial and Special Education. 26 (4) 207-213

*Articles authored or co-authored by Deno included use of the maze passages used in the Edcheckup system

Decision Rules for Changing Instruction: Data Unavailable

Decision Rules for Increasing Goals: Data Unavailable

Improved Student Achievement: Data Unavailable

Improved Teacher Planning Data Unavailable