DIBELS 6th Edition

Nonsense Word Fluency

Cost Technology, Human Resources, and Accommodations for Special Needs Service and Support Purpose and Other Implementation Information Usage and Reporting

Cost per student for Year 1: $0.00

Online Costs:

$ 1.00 Per-student per year (for online data entry, management, and reporting)

The DIBELS 6th Edition materials can be downloaded, free of charge, at: https://dibels.uoregon.edu. The materials consist of the manuals and test materials, directions for administration, test forms, technical manuals, and student protocols.

Use of the DIBELS Data System for the purpose of entering and managing data, as well as generating project, district, school, class, or student reports costs $1.00 per student per year.

Testers will require 1-4 hours of training. The examiners must at a minimum be a paraprofessional.

Training manuals and materials are available and have been field-tested. On-line training materials for administration and scoring of 6th edition will be available from https://dibels.uoregon.edu sometime during the 2013-14 school year. The pricing structure for these materials is to be determined.

Ongoing technical support is available from the DIBELS Data System at the University of Oregon, https://dibels.uoregon.edu (Phone: 1-888-497-4290, Email: support@dibels.uoregon.edu, Hours of operation: 6:00am to 5:30pm Pacific Time, Monday through Friday).

Accommodations:

A list of DIBELS approved accommodations is available in the Administration and Scoring Guide. 

Where to obtain: University of Oregon DIBELS Data System 

Address: 5292 University of Oregon, Eugene, OR 97403

Phone:  1-888-497-4290

Website: https://dibels.uoregon.edu

The DIBELS Nonsense Word Fluency (NWF) is a standardized, individually administered test of a student's alphabetic principle skills, including letter-sound correspondence and of the ability to blend letters into words in which letters represent their most common sounds. NWF is designed for progress monitoring use with students in grades K-1.

The student is presented with randomly ordered Vowel-Consonant (e.g., ig, ot) and Consonant-Vowel - Consonant (e.g., sim, tob, lut) nonsense words on an 8.5”x11” sheet of paper and asked to verbally produce the individual letter sound of each letter or read the whole nonsense word. For example, if the stimulus word is “sig” the student could say, /s/ /i/ /g/ or say the word “sig” to obtain a total of three letter-sounds correct. The student is allowed one minute to produce as many letter-sounds as he/she can, and the final score is the number of letter-sounds produced correctly in one minute.

The tool provides information on student performance in English.

The administration of the test takes 2 minutes and should be administered one-on-one to individual students.

There are 26 test forms (20 are typically used for progress monitoring and the other six are typically used for benchmarking – twice in kindergarten, three times in first grade, and once in second grade).

Raw scores are calculated by adding the number of letter-sounds produced correctly in one minute. There is no cluster/composite score. Raw scores can be converted to percentiles by looking up the equivalent percentile in a tech report (Cummings, Otterstedt, Kennedy, Baker, and Kame’enui, 2011). Raw scores can be compared to the criterion-referenced benchmark scores that are also available from dibels.uoregon.edu.

 

Reliability of the Performance Level Score

GradeK1
RatingFull bubbleFull bubble
Type of Reliability Age or Grade n (range) Coefficient SEM Information (including normative data)/Subjects
range median

One-month alternate form

First 77-231 0.67-0.88 0.83 9.05-14.26 (Mdn= 11.56) Good, Kaminski, et al. (2004). Participants at two elementary schools near Eugene, Oregon. The first school had a total population of 490 students in a town of around 53,000. The second school had a population of 580 in a town of around 4,700. 
Test-retest First 938 0.92-0.97 0.94 5.38-7.75 (Mdn= 6.86) Harn, Stoolmiller, & Chard (2008). Participants were 938 students from two Pacific Northwest school districts. The first district had five participating schools and was rural. The second district, with seven participating schools, was suburban. 
Three-week alternate form K 91 0.83-0.87 0.86 6.40-8.40 (Mdn.= 7.28) Ritchey (2008). Participants were 91 kindergarten students at two schools in a mid-Atlantic state with a mean January age of 67.52 months. 
Alternate form K 40 NR 0.94 4.19-5.50 Speece, Mills, Ritchey, & Hillman (2003). Participants were from five half-day kindergarten classes in a suburban school district in the middle Atlantic states. Students selected were those believed to have enough English skills to benefit from English instruction. Selections were made after students were ranked by their teachers as having high, average, or low literacy skills to obtain a sampling of skill levels. 25.6% of the students had a primary language other than English.
Test-retest First 3,506 0.84-0.90 NR  

Fien et al. (2010). Participants were 3,506 first grade students in 50 Oregon Reading First schools. About 49.4% of these students were girls, 53.9% were ethnic minorities other than Caucasian, 24.8% were English language learners, and 6.7% were identified as special education eligible. Many students were also from economically disadvantaged families. On average, 75% of the students in the participating schools were in the free or reduced price lunch program.

 

Reliability of the Slope

GradeK1
RatingdashFull bubble

Type of Reliability

Age or Grade

n (range)

Coefficient Range

Coefficient Median

SEM

Latent growth

First

80

 

0.84a

 

Linear growth

First

80

 

0.70a

 

Note. aR2

Information (including normative data) / Subjects:

Data reported in the table above is from Clemens, Shapiro, Wu, Taylor, and Caskie (2014). Latent growth modeling based on once-per-week progress monitoring for 11 weeks indicated that the slope factor was a reliable summary of Nonsense Word Fluency (NWF) scores across time. Linear slopes were also a reliable index of student growth using ordinary least-squares trend lines.

The Clemens et al., (2014) sample consisted of 80 at-risk first-grade students from nine classrooms in three elementary schools in Pennsylvania. The sample was 63% male, 60% White, 23% Black, 15% Hispanic/Latino, and 3% Asian/Pacific Islander. Three students spoke English as a second language. Five students with autism spectrum disorder were excluded from the sample. No students had identified learning disabilities. All three schools were eligible for school-wide Title 1 funding. 32% of students, on average across the three schools, were eligible for free or reduced priced lunch.

Validity of the Performance Level Score

GradeK1
RatingFull bubbleFull bubble

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient

Information (including normative data) / Subjects

range

median

Concurrent

First

TOWRE Sight Word Efficiency

289

NR

0.69

Burke, Crowder, Hagan-Burke, & Zou (2009). Participants were from a primary school in rural northeast Georgia. All were native speakers of English and the majority received all their education within the regular classroom.

Concurrent

First

TOWRE - PDE

213

NR

0.75

Burke & Hagan-Burke (2007). Participants were from a public primary school in semirural northeast Georgia who came from middle- to lower middle- class families.

Concurrent

First

TOWRE - SWE

213

NR

0.68

Predictive

K

G1 TOWRE PDE

180

NR

0.67

Burke, Hagan-Burke, Kwok, & Parker (2009). Participants were at a rural primary school in northern Georgia.

Predictive

K

G1 TOWRE SWE

180

NR

0.67

Predictive

K

Mid 2nd WRMTR (Pass. Comp.)

167

NR

0.56

Predictive

First

End 3rd SAT-10 Reading Comp.

352

NR

0.29

Chard et al. (2008). Participants were students in Oregon and Texas identified as needing strategic or intensive intervention during the winter kindergarten screening period or the fall first-grade screening period. Scores on the DIBELS ORF measure and the Growth Modeling ORF Passages were combined to make a single spring ORF score.

Predictive

First

End 3rd SAT-10 Reading.

344

NR

0.33

Predictive

First

Grade 1 Fall NWF – Grade 1 Spring SAT-10

3,506

NR

0.61

Fien et al. (2010). Participants were 3,506 first grade students in 50 Oregon Reading First schools. About 49.4% of these students were girls, 53.9% were ethnic minorities other than Caucasian, 24.8% were English language learners, and 6.7% were identified as special education eligible. Many students were also from economically disadvantaged families. On average, 75% of the students in the participating schools were in the free or reduced price lunch program.

Predictive

First

Grade 1 Winter NWF – Grade 1 Spring SAT-10

3,506

NR

0.62

Concurrent

First

Grade 1 Spring NWF – Grade 1 Spring SAT-10

3,506

NR

0.62

Predictive

First

Grade 1 Spring NWF- Grade 3 Spring Wechsler Individual

Achievement Test–Second Edition, Reading Comprehension

35

NR

0.57

Munger & Blachman (2013). Participants are 35 students from a small urban school in the Northeast. Students were identified as 51% African American, 26% White, 11% Asian, 9% Hispanic/Latino, and 3% Indian. Seventeen percent of the students spoke a language other than English in their homes, and 25% were identified as having an educational disability. Children’s families were in predominantly lower- to middle-income groups, with 73% receiving free or reduced-price lunch at the time of third grade testing.

Predictive

First

Grade 1 Spring NWF- Grade 3 Spring Group Reading Assessment and Diagnostic Evaluation

35

NR

0.66

Predictive

First

Grade 1 Spring NWF- Grade 3 Spring New York State Reading Language Arts Test

35

NR

0.45

Predictive

First

Middle of 2nd ORF

289

NR

0.57

Burke, Crowder, Hagan-Burke, & Zou (2009).

Concurrent

First

DIBELS ORF

213

NR

0.68

Burke & Hagan-Burke (2007)

Concurrent

First

DIBELS RTF

213

NR

0.54

Predictive

K

Mid of 1st DORF

179

NR

0.73

Burke, Hagan-Burke, Kwok, & Parker (2009).

Predictive

K

Mid 2nd DORF

165

NR

0.58

Concurrent

First

Composite of DIBELS ORF and Growth Modeling ORF

486

NR

0.66

Chard et al. (2008).

Predictive

First

Grade 2 DORF/ GMORF compos.

419

NR

0.64

Predictive

First

Grade 3 DORF/ GMORF compos.

369

NR

 

0.59

Predictive

First

Grade 1 Fall NWF – Grade 1 Spring ORF

3,150

NR

0.67

Cummings, Dewey, Latimer, & Good (2011). Participants were 3,150 students across 12 school districts (8 in the West and 4 from the Midwest) in 8 states. Of the 12 school districts in the large sample, 8 are located in rural areas and 4 are located in small urban cities.

Predictive

First

Grade 1 Fall NWF – Grade 1 Winter ORF

3,150

NR

0.76

Predictive

First

Grade 1 Winter NWF- Grade 1 Spring ORF

3,150

NR

0.66

Concurrent

First

Grade 1 Winter NWF- Grade 1 Winter ORF

3,150

NR

0.67

Concurrent

First

Grade 1 Spring NWF- Grade 1 Spring ORF

3,150

NR

0.72

Predictive

First

Grade 1 Fall NWF – Grade 1 Spring ORF

3,506

NR

0.74

Fien et al. (2010).

Predictive

First

Grade 1 Winter NWF – Grade 1 Spring ORF

3,506

NR

0.77

Concurrent

First

Grade 1 Spring NWF – Grade 1 Spring ORF

3,506

NR

0.76

Predictive

K

ORF Grade 1

2,172

1-17%

10%

Good, Baker, & Peyton (2009). Participants were 2,172 students in 34 Oregon Reading First schools. Approximately 12% of the sample was eligible for Special Education and 81% was eligible for free/reduced lunch. And an additional 300,000 students from the DIBELS Data System. Data were collected during the 2004-2005 school year.

Values = R2, percent of variance explained for AT-RISK students.

Predictive

K

ORF Grade 1

2,172

2-4%

3%

Good, Baker, & Peyton (2009).

Values = R2, percent of variance explained for SOME-RISK students.

Predictive

K

ORF Grade 1

2,172

10-10%

10%

Good, Baker, & Peyton (2009).

Values = R2, percent of variance explained for LOW-RISK students

 

Predictive

First

Grade 1 Fall NWF – Grade 1 Spring ORF

938

NR

0.73

Harn, Stoolmiller, Chard (2008). Participants were 938 students in two Pacific Northwestern districts.

Predictive

First

Grade 1 Winter NWF- Grade 1 Spring ORF

938

NR

0.74

Concurrent

First

Grade 1 Spring NWF- Grade 1 Spring ORF

938

NR

0.78

Concurrent

First

Grade 1 Spring NWF- Grade 1 Spring ORF

35

NR

0.88

Munger & Blachman (2013)

 

Predictive Validity of the Slope of Improvement

GradeK1
RatingEmpty bubbleFull bubble

Study

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient Range

Coefficient Median

Clemens, Shapiro, Wu, Taylor, Caskie, 2014

Predictive

First

Composite of DIBELS ORF, Test of Word Reading Fluency, Maze

80 at risk students

 

56%a

Cummings, Dewey, Latimer & Good, 2011

Predictive

First

Spring DIBELS ORF

3,150

0.23 - 0.25

0.24b

Fien et al., 2010

Predictive

First

Spring DIBELS ORF

3,506

 

0.32c

Predictive

First

Spring SAT-10 Reading Comprehension

3,506

 

0.26c

Predictive

First

Spring DIBELS ORF

3,150 lower performing students (Fall CLS<49)

 

0.57d

Good, Baker & Peyton, 2009

Predictive

First

Spring DIBELS ORF

223 at risk students

 

54%a

Predictive

First

Spring SAT-10 Reading Comprehension

52 some risk students

 

11%a

Harn, Stoolmiller & Chard, 2008

Predictive

First

Spring DIBELS ORF

938

 

0.52d

Note. aAdditional variance explained by slope, given initial skill level. bEffect size. cCorrelation between NWF fall-to-spring gain and spring ORF. dEffect of NWF fall-to-spring gain on spring ORF.

Information (including normative data) / Subjects:

Data presented in the table above is from five published studies regarding NWF gain in first grade on end-of-year Oral Reading Fluency.

Three of the studies indicate that growth on NWF in first grade is more predictive of end-of-year Oral Reading Fluency in students who start the school year below benchmark on NWF. For students who are already proficient on NWF at the beginning of the school year, growth is not as good of an indicator. For example, Harn, Stoolmiller and Chard (2008) found a strong linear relationship between growth on NWF and Oral Reading Fluency for 9 out of 10 subgroups based on initial skill. The linear relationship broke down for the highest group consisting of students who had already met the benchmark at the beginning of first grade. Cummings, Dewey, Latimer and Good (2011) and Good, Baker, and Peyton (2009) observed similar findings.

Although some of the research relies on change from fall to spring on benchmark data, the results reported in two studies consisted of data from students who were being progress monitored. In Good, Baker and Peyton (2009), slope was calculated using the ordinary least squares method based on 5 to 13 progress monitoring data points collected during the first five months of the school year. Clemens et al. (2014) used latent growth modeling to estimate slope using progress monitoring data. Students’ progress was monitored with NWF once per week (Tier 3) or once every two weeks (Tier 2) for 11 weeks ending in mid-December.

The Clemens et al., (2014) sample consisted of 80 at-risk first-grade students from nine classrooms in three elementary schools in Pennsylvania. The sample was 63% male, 60% White, 23% Black, 15% Hispanic/Latino, and 3% Asian/Pacific Islander. Three students spoke English as a second language. Five students with autism spectrum disorder were excluded from the sample. No students had identified learning disabilities. All three schools were eligible for school-wide Title 1 funding. 32% of students, on average across the three schools, were eligible for free or reduced priced lunch. Tier 2 and Tier 3 students were administered progress monitoring measures once or twice per week for 11 weeks ending in mid-December. Slope was correlated with an end-of-year composite measure consisting of DIBELS Oral Reading Fluency, Test of Word Reading Fluency and Maze.

The Cummings, Dewey, Latimer and Good (2011) sample consisted of 3,150 first grade students across 12 school districts (8 in the West and 4 from the Midwest) in 8 states. Eight districts were rural, and 4 were in small urban cities. All 12 districts were predominately White (70 – 90%). The largest minority group in 8 of the schools was Hispanic. Of the 10 districts that reported free/reduced lunch eligibility information, 6 have rates above 50% and all are above 30%. Students were administered NWF during benchmark assessment periods (i.e., fall, winter, and spring). Only students with complete data were included in analysis. Slope based on these three assessments was correlated with spring DIBELS Oral Reading Fluency.

The Fien et al. (2010) sample consisted of 3,506 first-grade students in 50 schools participating in Oregon Reading First during the 2006-07 school year. 49.4% of students were female, 53.9% were ethnic minorities other than Caucasian, 24.8% were English language learners, 6.7% were identified as eligible for Special Education. On average 75% of students in participating schools were eligible for the free or reduced lunch program. NWF was administered at benchmark assessment periods. Gain from fall to spring was correlated with outcomes in the spring (Oral Reading Fluency and SAT-10.)

The Good, Baker, and Peyton (2009) sample consisted of first-grade students in Oregon Reading First schools who had complete data sets and score in the “at risk” or “some risk” range on NWF in the fall benchmark testing period. The sample was 56% male; 51% Latino, 35% White, 7% African American; 84% eligible for free/reduced price lunch; and 17% eligible for Special Education. Progress monitoring measures were administered during the first five months of the school year. The number of data points ranged from 5 to 13. Slope based on these data was compared to end-of-year Oral Reading Fluency and SAT-10 Reading Comprehension.

The Harn, Stoolmiller, and Chard (2008) sample consisted of 938 first grade students in 12 schools (two school districts) in the Pacific Northwest. Both school districts were predominantly white (71% and 81% white, 17% and 13% Latino respectively). One district had five rural schools with a free and reduced lunch rate of 45%. The other was a suburban school with a free and reduced lunch rate of 24%. NWF was administered in the fall, winter, and spring. Slope was correlated with end of year Oral Reading Fluency.

Although DIBELS Oral Reading Fluency is part of the DIBELS measurement system, it is a separate measure from Nonsense Word Fluency and measures different, but related, skills. Oral Reading Fluency is a measure of reading connected text while Nonsense Word Fluency is a measure of a students’ facility with applying the alphabetic principle. Although both measures are administered in a similar manner, there is no overlap between items.

Bias Analysis Conducted

GradeK1
RatingNoNo

Disaggregated Reliability and Validity Data

GradeK1
RatingNoNo

Alternate Forms

GradeK1
RatingEmpty bubbleEmpty bubble

What is the number of alternate forms of equal and controlled difficulty?

26 alternate forms are available.

Alternate form reliability:

Kindergarten: 0.94 (two alternate forms, Speece et al., 2003)

Kindergarten: 0.83 to 0.87 (five alternate forms; median = 0.86. Richey, 2008)

Grade 1: 0.67 to 0.88, (8 alternate forms; median 0.83. Good et al., 2004)

Equivalent alternate forms were created by first establishing an item pool of all possible eligible VC and CVC nonsense words. In order to be an eligible nonsense word, words had to have each letter associated only with its most frequently occurring sound. Words that were real words or that sounded like inappropriate words were excluded, but words that sounded like real words were not excluded. The final item pool consisted of 1065 words in which: (a) 4% were 2 letter words judged easy – the final consonant was a member of the string of consonants “bcdfghklmnprst” judged to be easier; (b) 1% were 2 letter words judged harder, where the final consonant was not an easier consonant; (c) 16% were 3 letter words where only the initial consonant was judged easier; (d) 24% were 3 letter words where only the final consonant was judged easier; (e) 48% were 3 letter words where both consonants were judged easier; and (f) 7% were 3 letter words where both consonants were judged harder. Equivalent alternate form probes were obtained using a stratified random sampling procedure where: (a) 10% were 2 letter words with easy final consonant, (b) 5% were 2 letter words with harder final consonant, (c) 20% were 3 letter words with easier initial consonant only, (d) 20% were 3 letter words with easier final consonant only; (e) 40% were 3 letter words with easier initial and final consonants; and (f) 5% were 3 letter words with harder initial and final consonants. Harder and easier words were arranged in random order on the probes. 

Rates of Improvement Specified

GradeK1
RatingEmpty bubbleEmpty bubble

What is the basis for specifying minimum acceptable growth?

Norm-referenced, Criterion-referenced and other: Initial skill norm groups

Description of normative sample:

Representation: National

Date: 2013-14 school year

Number of states: 48

Size: Kindergarten = 64,702; Grade 1 = 56,105

Regions: All (as defined by NCES: Northeast 18.9%, Midwest 16.6%, South 45%, West 19.5%)

Gender, SES, Race/ethnicity, ELL, Disability classification: unavailable.

Criterion-referenced Rates of Improvement

The minimal improvement is specified as the change between benchmarks within a year. For students who earn scores in the low risk range, adequate growth will maintain their low risk status. For students who earn scores in the some- or at-risk range, reducing their risk status would indicate adequate progress. In other words, adequate growth is (a) maintenance of low risk status, OR movement from: (b) at-risk to some-risk; (c) at-risk to low-risk; or (d) some-risk to low-risk within one benchmark assessment period. See GOM 7 for the benchmark goals and indicators of risk for a specific grade and time of year.

For example, the specified rate of improvement from middle to end of kindergarten = (EOY benchmark – MOY benchmark)/number of weeks = (39-19)/15=1.33. Thus, benchmark kindergarten students need to improve by at least 1.33 sounds per week on NWF to remain on track from the middle to end of kindergarten. The specified rate of improvement from beginning to end of grade 1 = (EOY benchmark – BOY benchmark)/number of weeks = (71-25)/30 = 1.53. Benchmark first grade students need to improve by at least 1.53 sounds per week on NWF to remain on track from the beginning to end of first grade.

Initial Skill Based Norm Referenced Rates of Improvement

An alternative, and more precise standard for rates of improvement utilizes growth percentiles. These percentiles are determined separately for groups of students who performed similarly on the initial measurement of the skill. Growth percentiles are determined separately for each initial raw score by aggregating rates of growth for all students in the same grade in the DIBELS Data System who scored within ±0.5 SEM of that raw score in each norm group (Kennedy, Cummings, Bauer Schaper, & Stoolmiller, 2015).

For example, consider a student who scored five Correct Letter Sounds (CLS) in the middle of kindergarten. Growth at the 80th percentile (compared to other kindergarten students that scored within 0.5 SEM of five CLS) would mean this student scored 29 CLS at the end of kindergarten. This represents growth of 24 CLS, or approximately 1.6 CLS per week. These Zones of Growth percentiles allow educators to set realistic, fine-tuned goals for individual students based on the performance of students with a similar initial skill level. The DIBELS Data System provides tools that help educators select growth goals that are average (50th percentile), above average (65th percentile), or ambitious (80th percentile) for any given initial score. Examples for every five initial raw scores from 5 to 70 are presented in the tables below.

Zones of Growth goals are available for the predominant measures in each grade. Zones of growth tools are available for NWF from the middle to end of kindergarten, and from the beginning to middle of first grade.

DIBELS 6th Edition Nonsense Word Fluency Kindergarten Zones of Growth Examples

 

End of Year NWF Score

Weekly Growth Estimate

Middle of Year NWF score

Average Growth (50th %ile)

Above Average Growth (65th %ile)

Ambitious Growth (80th %ile)

Average Growth (50th %ile)

Above Average Growth (65th %ile)

Ambitious Growth (80th %ile)

5

18

22

29

0.9

1.1

1.6

10

24

28

33

0.9

1.2

1.5

15

29

33

38

0.9

1.2

1.5

20

31

36

42

0.7

1.1

1.5

25

36

41

47

0.7

1.1

1.5

30

40

45

51

0.7

1.0

1.4

35

45

50

56

0.7

1.0

1.4

40

49

54

64

0.6

0.9

1.6

45

54

59

69

0.6

0.9

1.6

50

57

66

77

0.5

1.1

1.8

55

62

71

82

0.5

1.1

1.8

60

71

80

96

0.7

1.3

2.4

65

76

85

101

0.7

1.3

2.4

70

83

93

109

0.9

1.5

2.6

 

DIBELS 6th Edition Nonsense Word Fluency Grade 1 Zones of Growth examples

 

Middle of Year NWF Score

Weekly Growth Estimate

Beginning of Year NWF score

Average Growth (50th %ile)

Above Average Growth (65th %ile)

Ambitious Growth (80th %ile)

Average Growth (50th %ile)

Above Average Growth (65th %ile)

Ambitious Growth (80th %ile)

5

29

36

44

1.6

2.1

2.6

10

35

41

48

1.7

2.1

2.5

15

40

46

53

1.7

2.1

2.5

20

43

48

56

1.5

1.9

2.4

25

48

53

61

1.5

1.9

2.4

30

50

56

66

1.3

1.7

2.4

35

55

61

71

1.3

1.7

2.4

40

60

68

81

1.3

1.9

2.7

45

65

73

86

1.3

1.9

2.7

50

72

83

99

1.5

2.2

3.3

55

77

88

104

1.5

2.2

3.3

60

86

98

115

1.7

2.5

3.7

65

91

103

120

1.7

2.5

3.7

70

99

110

124

1.9

2.7

3.6

 

End-of-Year Benchmarks

GradeK1
RatingFull bubbleFull bubble

Benchmark scores for NWF are available in both criterion- and norm-referenced forms. Criterion-referenced benchmark goals have been re-calculated since the most recent submission, based on new data and analyses. End-of-year benchmark goals for NWF are 39 Correct Letter Sounds at the end of kindergarten, and 71 Correct Letter Sounds at the end of first grade.

Summary of DIBELS 6th Edition Nonsense Word Fluency Criterion-Referenced Benchmarks and Cut Points for Risk

Grade

Time of Year

Cut points

At Risk

Some Risk

Low Risk

K

Middle

≤ 14

15 - 18

≥ 19

End

≤ 34

35 - 38

≥ 39

1

Beginning

≤ 18

19 - 24

≥ 25

Middle

≤ 47

48 - 53

≥ 54

End

≤ 61

62 - 70

≥ 71

 

Procedure for specifying criterion-referenced benchmarks:

Procedures for determining benchmarks and cut points for risk are fully specified in Smolkowski and Cummings (2015).

The Stanford Achievement Test – 10th Edition (SAT10) was administered at the end of kindergarten, first, and second grade as a criterion measure. ROC curves were generated and the area under curve, A, was calculated at each time point. The benchmark standard and cut point for risk were only set for those time points in which A was at or above .75.  Benchmark and cut scores were chosen based on sensitivity at or above .80. In other words, the benchmark score was defined as the lowest score where sensitivity exceeded .80. Values of A, the decision threshold (i.e., score), sensitivity, specificity, negative and positive predictive values, base rate and the proportion of students who screened positive are reported in the table below. These statistics were defined for students at risk (20th normative percentile) and students at benchmark (40th normative percentile) on the SAT10 (Smolkowski and Cummings, 2015).

Note: Nonsense word fluency (NWF) cut scores were based on SAT-10 criterion values at the 40th percentile for some risk, and 20th percentile for high risk. A represents the area under the ROC curve; NPV is the negative predictive value; PPV is the positive predictive value; ρ is the base rate; and τ is the proportion screened positive (scored below the decision threshold). Thresholds bolded if A ≥ 0.75. The 95% confidence intervals were 0.01 for all A values, and all sensitivity and specificity values.

Procedure for specifying norm-referenced benchmarks:

Alternative, norm-referenced cut points and benchmarks are defined as the 20th and 40th percentiles, respectively. Procedures are fully specified in Cummings et al. (2011).

Summary of DIBELS 6th Edition Nonsense Word Fluency Norm-Referenced Benchmarks

Grade

Time of Year

Cut points

Below 20th percentile

20th – 39th percentile

40th percentile and above[PK1]

K

Middle

≤ 9

10 - 18

≥ 19

End

≤ 21

22 - 30

≥ 31

1

Beginning

≤ 17

18 – 26

≥ 27

Middle

≤ 37

38 - 48

≥ 49

End

≤ 44

45 - 58

≥ 59

 

Description of norm-sample for norm-referenced benchmarks and cut points for risk:

Representation: National

Date: 2009-10 school year

Number of students:

 

Beginning of Year

Middle of Year

End of Year

Kindergarten

n/a

631,760

632,114

Grade 1

684,997

657,004

636,999

Number of States: 50

Regions
Kindergarten: Northeast 16.0%, Midwest 25.8%, South 28.0%, West 30.2%;
Grade 1: Northeast 15.7%, Midwest 27.0%, South 27.3%, West 30.1%

Gender
Kindergarten: 50.1% male, 46.2% female, 3.8% unreported;
Grade 1: 51.0% male, 47.6% female, 1.4% unreported.

SES: Eligibility for free/reduced price lunch
Kindergarten: 52.4%
Grade 1: 52.5%

Race ethnicity (average school-level percent):

 

Kindergarten

Grade 1

American Indian/Alaskan Native

2.4

2.4

Asian/Pacific Islander

3.1

3.3

Hispanic

13.5

14.0

Black

14.5

14.6

White

62.1

63.8

Hawaiian Native/Pacific Islander

0.3

0.3

Two or more races

3.2

3.0

Unknown

3.8

1.4

ELL: unknown

        Disability classification: unknown

Sensitive to Student Improvement

GradeK1
RatingEmpty bubbleEmpty bubble

1. Describe evidence that the monitoring system produces data that are sensitive to student improvement (i.e., when student learning actually occurs, student performance on the monitoring tool increases on average).

ReMillard (2002) found that two groups, receiving a reading fluency intervention, showed progress toward meeting the NWF goal after 7 weeks, with weekly NWF progress monitoring. Average slope for all subjects ranged from 0.65-0.80. ReMillard also found the initial scores on NWF were significant predictors of slope. in a study by Speece, Mills, Ritchey, & Hillman (2003) the kindergarten NWF identified 85.7% of the poorest reading in first grade. 
 

Decision Rules for Changing Instruction

GradeK1
RatingEmpty bubbleEmpty bubble

Does your manual or published materials specify validated decision rules for when changes to instruction need to be made?

Yes

Specify the decision rules:

A change in instruction is warranted if a student’s performance falls below the aim line between the beginning and end of year benchmark scores eight consecutive times. The DIBELS Data System progress monitoring graph uses color coding to indicate when a change of instruction should occur.

Decisions regarding whether or not to modify instruction focus on three key pieces of data: the goal (either criterion- or norm-referenced, as specified in GOM 6 and GOM 7), a line (the aim line) connecting students’ initial status to said goal, and students’ progress monitoring data between those two points. With DIBELS 6th edition, student progress can be monitored using either paper/pencil graphs or the University of Oregon DIBELS Data System (for an example, see https://dibels.uoregon.edu/docs/reports/StudentProgressMonitoringGraph_DIBELS6th_Example.pdf). Currently, the decision rule for making changes to instruction focuses on the number of consecutive data points that fall below the aim line. Once eight consecutive data points are below the aim line, school team members should review the student's instructional program and consider adjusting instruction.

What is the evidentiary basis for these decision rules?

These rules are based on the recommendations in a comprehensive review of research on CBM decision rules (Ardoin et al., 2013), which suggest that the relative instability of any given ORF score suggests that eight or more data points may be necessary to yield valid and reliable outcomes.

Decision Rules for Increasing Goals

GradeK1
RatingEmpty bubbleEmpty bubble

Does your manual or published materials specify validated decision rules for when changes to increase goals?

Yes

Specify the decision rules:

Decisions regarding when to increase goals are based on the same three key pieces of data reported in GOM 9: the goal, the aim line, and students’ progress monitoring data between those two points. The decision rule for making changes to student goals focuses on the number of consecutive data points above the aim line. Once eight consecutive data points are above the aim line, school team members should review the student's goal and consider making it more ambitious.

What is the evidentiary basis for these decision rules?

These rules are based on and consistent with the recommendations in a comprehensive review of research on CBM decision rules (Ardoin et al., 2013), which suggest that the relative instability of any given ORF score suggests that 8 or more data points may be necessary to yield valid and reliable outcomes.

Improved Student Achievement

GradeK1
Ratingdashdash

Improved Teacher Planning

GradeK1
RatingEmpty bubbleEmpty bubble

Description of evidence that teachers’ use of the tool results in improved planning based on an empirical study that provides this evidence.

Study: Baker, S., & Smith, S. (2001). Linking school assessments to research-based practices in beginning reading: Improving programs and outcomes for students with and without disabilities. Teacher Education and Special Education, 24, 315-322.

Sample:

Number of students in product/experimental condition: 34

Number of students in control condition: 70

Characteristics of students in sample and how they were selected for participation in study: Two schools (serving K-3 students) participated in the BRIDGE (Bridging Research and Instruction in Dynamic and Grounded Exchange) project. In school A, all first grade students participated in the project across 3 years. In school B, all kindergarten students participated in the project across 4 years. Both schools came from a district with a total of 16 elementary schools, and were selected because they had the highest percentage of students eligible for free/reduced lunch. The population at these schools were primarily of European descent. At one school 7% of students were Hispanic, and at the other school 15% were Hispanic

Design: Convenience sample

Unit of assignment: Schools

Unit of analysis: Students

Duration of product implementation: Three years in School A, four years in School B

Describe analysis: Observational/descriptive. The analysis used to describe the differences between intervention and control students (within each schools) is not described.

Fidelity: NR

Results on the fidelity of treatment implementation measure: NR

Measures: External outcome measures used in the study, along with psychometric properties:

 

Measure name

Reliability statistics (specify type of reliability, e.g. Cronbach’s alpha, IRT reliability, temporal stability, inter-rater)

Yopp-Singer Test of Phonemic Segmentation (Yopp, 1995)

NR

Results:

Results of the study: As a result of this study, teachers in School A made three key decisions based on their use of DIBELS (a) they increased the amount of time spent on their first grade Reading Mastery intervention, (b) they established a literacy intervention program in kindergarten, (c) they decided to evaluate intervention programs with a long-term (i.e., multiple year) focus as well as a short-term (i.e., within one year focus). School B started from the ground up, and started systematically collecting data in year 1 of the BRIDGE project. Over the course of the project, School B significantly altered its entire literacy program for K. The effect size for the difference between Yopp-Singer performance in Year 1 (pretest) compared to Year 4 (project end; posttest) is listed below.

Effect sizes for each outcome measure:

Measure name

Effect size Year 1 to Year 4 of a 4-year project

School B only

     

Yopp-Singer: Grade K

d = 2.04

Other related references or informationThe impact of the BRIDGE Project on other aspects of early literacy (e.g., alphabetic understanding and developmental spelling) is summarized further in: Baker, S., & Smith, S. (1999). Starting off on the right foot: the influence of four principles of professional development in improving literacy instruction in two kindergarten programs. Learning Disabilities Research and Practice, 14, 239-253.