FAST earlyReading

Letter Sounds

 

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

The Formative Assessment System for Teachers (FAST) is a cloud-based suite of assessment and reporting tools that includes earlyReading English. As of 2013-14, there is a $5 per student per year charge for the system. As a cloud-based assessment suite, there are no hardware costs or fees for additional materials.

Computer and internet access is required for full use.

Testers will require less than 1 hour of training.

Paraprofessionals can administer the test.

FastBridge Learning
520 Nicollet Mall
Suite 910
Minneapolis, MN 55402-1057
Phone: 612-254-2534

Field tested training manuals are included and should provide all implementation information.

Access to interactive online self-guided teacher training is included at no additional cost. In-person training is available at an additional cost of $300 per hour.

earlyReading is used to monitor student progress in early reading in the early primary grades. Most earlyReading assessments provide information on both the accuracy and rate or efficiency of performance.

The appropriate progress monitoring assessment(s) is/are chosen based on screening performance and are used to diagnose and evaluate skill deficits. Those results help guide instructional and intervention development. It is recommended that Letter Sounds be used for progress monitoring throughout kindergarten.

The Letter Sounds task assesses the student’s ability and efficiency saying the sounds of upper- and lower-case letters in isolation. The examiner and student each view the same page of letters. That page is organized in a systematic manner that is described in more detail later in this protocol. As the student says the sounds of the letters aloud from a paper copy, the examiner marks any errors on his/her paper or electronic copy. The resulting score is the number of letter sounds correctly in a minute.

Each earlyReading test takes approximately 1-2 minutes to administer. earlyReading is computer administered to individual students and scoring is automated; it does not require any additional time to score.

The Letter Sounds assessment has 20 alternate forms.

Rate is calculated as the number of correct letter sounds read per minute. Raw scores of total and correct letter sounds are also provided. An inventory of known letter sounds and dual sounds can be generated.

 

 

Reliability of the Performance Level Score

GradeK
RatingFull bubble

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM or CSEM*

Information (including normative data) / Subjects

range

median

Alternate Forms

K

34-36

0.85-0.94

0.89

5.56 (4.89)

Collected in Spring; See Table 1 below for information on the sample.

Test Retest

K

75

0.88-0.95

0.92

3.53*

Collected Fall 2012

         * CSEM = median of the conditional standard error of measurement across participants.

Table 1. Sample Demographics for Alternate Forms Study

 

 

Category

District A (%)

District B (%)

District C (%)

White

56.1%

93%

79.5%

Black

13.5%

4%

6.8%

Hispanic

10.3%

3%

4.5%

Asian/Pacific Islander

19.4%

4%

10.5%

American Indian/Alaskan Native

>0.1%

1%

0.25%

Free and Reduced Lunch

44.9%

17%

9%

LEP

15.8%

6%

6%

Special Education

12.6%

10%

10%

 

Reliability of the Slope

GradeK
RatingFull bubble

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data) / Subjects

range

median

Split-Half

K 953 - 0.70 -  

Split-Half

1 52 - 0.73 -  

Reliability for the Slope

1 31 - 0.59 - Duration of Progress Monitoring greater than 10 weeks.

Reliability for the Slope

1 21 - 0.56 - Duration of Progress Monitoring greater than 10 weeks.

 

Validity of the Performance Level Score

GradeK
RatingFull bubble

The aggregate (full scale) score for the GRADE was used to estimate all criterion validity coefficients unless otherwise noted.

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient

Information (including normative data) / Subjects

range

median

Concurrent

K

GRADE composite Level P

85

 

0.53

Data collected in Fall; Participants included kindergarten students from two school districts. In School District 1 three elementary schools participated. Kindergarten students from District 1 who participated in the study were enrolled in all day or half day kindergarten. The majority of students within the school district were White (78%), with the remaining students identified as either African American (19%), or other (3%). 40 to 50 percent of students at each school were on free and reduced lunch. In school District 2, the majority of students within the school district were White (53%), with the remaining students identified as African American (26%), Hispanic (11%), Asian (8%), or other (2%). 40 to 50 percent of students at each school were on free and reduced lunch. 

Concurrent

K

GRADE composite Level K

214

 

0.19

Data collected in Spring;

See subject information above

Predictive

K

GRADE composite Level K

230

 

0.44

Fall to Spring prediction;

See subject information above

Predictive

K

GRADE composite Level K

210

 

0.63

Winter to Spring prediction;

See subject information above

 

Predictive Validity of the Slope of Improvement

GradeK
RatingEmpty bubble

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient

Information (including normative data)/Subjects

range

median

Predictive Validity of Slope

K

GRADE

Composite Level K

231

-

0.54

Data collected fall, winter, and spring

Participants included kindergarten students from two school districts. In School District 1 three elementary schools participated. Kindergarten students from District 1 who participated in the study were enrolled in all day or half day kindergarten. The majority of students within the school district were White (78%), with the remaining students identified as either African American (19%), or other (3%). 40 to 50 percent of students at each school were on free and reduced lunch. In school District 2, the majority of students within the school district were White (53%), with the remaining students identified as African American (26%), Hispanic (11%), Asian (8%), or other (2%). 40 to 50 percent of students at each school were on free and reduced lunch. 

Predictive Validity of Slope

1

aReading 37 - 0.59 This coefficient was derived from a subset of a sample of approximately 964 1st grade students and 1220 Kindergarten students in the FAST system (N = 2184). Approximately 34% were female, 45% were male, and 21% of students did not report gender. Approximately 46% of the sample of students were White, 257% were Black/African American, 8% were Hispanic, 7% were Asian, 1% were recorded as “Other”, 3% were Multiracial, 2% were American Indian or Alaska Native, and .3% were Native Hawaiian or Other Pacific Islander. Approximately 21% of the sample did not report ethnicity/race. Approximately 61% of students were reported as not eligible for Special education services, while 17% of students were receiving special education services. However, 21% of the sample did not report special education status or receipt of services.

 

Bias Analysis Conducted

GradeK
RatingNo

Disaggregated Reliability and Validity Data

GradeK
RatingYes

Disaggregated Reliability of the Performance Level Score:

The following disaggregated delayed test retest reliability coefficients were derived from a sample of approximately 11,850 1st grade students and 15,985 Kindergarten students in the FAST system (N = 27,835). Approximately 29.3% were female, 31.3% were male, and 39.4% of students did not report gender. Approximately 39.1% of the sample of students were White, 8% were African American, 5% were Hispanic, 3.3% were Asian, 2.1% were recorded as “Other”, 1.9% were Multiracial, 1% were American Indian or Alaska Native, and 0.1% were Native Hawaiian or Other Pacific Islander. Approximately 39.4% of the sample did not report ethnicity/race. Approximately 50% of students were reported as not eligible for Special education services, while 10.4% of students were receiving special education services. However, 39.4% of the sample did not report special education status or receipt of services.

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data) / Subjects

Range

Median

Delayed Test Retest

K

92

-

0.57

-

Fall to Winter; American Indian/Alaska Native

Delayed Test Retest

K

62

-

0.63

-

Fall to Spring; American Indian/Alaska Native

Delayed Test Retest

K

109

-

0.65

-

Winter to Spring; American Indian/Alaska Native

Test Retest

K

25

-

0.68

-

2-3 Week Delay; American Indian/Alaska Native

Delayed Test Retest

K

266

-

0.65

-

Fall to Winter; Asian

Delayed Test Retest

K

264

-

0.54

-

Fall to Spring; Asian

Delayed Test Retest

K

419

-

0.73

-

Winter to Spring; Asian

Test Retest

K

172

-

0.67

-

2-3 Week Delay; Asian

Delayed Test Retest

K

610

-

0.51

-

Fall to Winter; African American

Delayed Test Retest

K

604

-

0.41

-

Fall to Spring; African American

Delayed Test Retest

K

831

-

0.75

-

Winter to Spring; African American

Test Retest

K

408

-

0.74

-

2-3 Week Delay; African American

Delayed Test Retest

K

374

-

0.44

-

Fall to Winter; Hispanic

Delayed Test Retest

K

431

-

0.27

-

Fall to Spring; Hispanic

Delayed Test Retest

K

444

-

0.73

-

Winter to Spring; Hispanic

Test Retest

K

197

-

0.77

-

2-3 Week Delay; Hispanic

Delayed Test Retest

K

207

-

0.57

-

Fall to Winter; Multiracial

Delayed Test Retest

K

190

-

0.39

-

Fall to Spring; Multiracial

Delayed Test Retest

K

186

-

0.76

-

Winter to Spring; Multiracial

Test Retest

K

87

-

0.89

-

2-3 Week Delay; Multiracial

Delayed Test Retest

K

2671

-

0.51

-

Fall to Winter; White

Delayed Test Retest

K

2763

-

0.35

-

Fall to Spring; White

Delayed Test Retest

K

4446

-

0.66

-

Winter to Spring; White

Test Retest

K

1522

-

0.72

-

2-3 Week Delay; White

Delayed Test Retest

1

88

-

0.50

-

Fall to Winter; White

 

Disaggregated Reliability of the Slope:

The following disaggregated reliability of the slope coefficients were derived from a sample of approximately 907 1st grade students and 1,180 Kindergarten students in the FAST system (N = 2,087). Approximately 33.7% were female, 44.8% were male, and 21.5% of students did not report gender. Approximately 45.9% of the sample of students were White, 11.5% were African American, 7.2% were Hispanic, 6.8% were Asian, 1.2% were recorded as “Other”, 3.3% were Multiracial, 2.2% were American Indian or Alaska Native, and 0.4% were Native Hawaiian or Other Pacific Islander. Approximately 21.5% of the sample did not report ethnicity/race. Approximately 59.9% of students were reported as not eligible for Special education services, while 18.5% of students were receiving special education services. Approximately 21.5% of students did not report Special education status (i.e., receipt of services). 

Type of Reliability

Age or Grade

n (range)

Coefficient

SEM

Information (including normative data) / Subjects

Range

Median

Reliability for the Slope

1

16

-

0.38

-

White

Reliability for the Slope

1

8

-

0.67

-

Hispanic

Reliability for the Slope

K

49

-

0.40

-

Asian

Reliability for the Slope

K

361

-

0.49

-

White

Reliability for the Slope

K

70

-

0.52

-

Hispanic

Reliability for the Slope

K

21

-

0.44

-

Multiracial

Reliability for the Slope

K

82

-

0.63

-

African American

Reliability for the Slope

K

24

-

0.63

-

American Indian or Alaska Native

 

Disaggregated Validity of the Performance Level Score:

The following disaggregated aReading validity coefficients were derived from a sample of approximately 17,137 Kindergarten students in the FAST system. Approximately 32.2% were female, and 34.7% were male, with approximately 33% of the sample not reporting their gender. Approximately 42.6% of the sample of students were White, 8.5% were African American, 4.9% were Hispanic, 3.5% were Asian, 4.4% were recorded as “Other”, 1.7% were Multiracial, 1.2% were American Indian or Alaska Native, and 0.1% were Native Hawaiian or Other Pacific Islander. Approximately 33% of the sample did not report ethnicity/race. Approximately 55.5% of students were reported as not eligible for Special education services, while 3.5% of students were receiving special education services. Approximately 40.9% of students did not report their special education status.

Type of Validity

Age or Grade

Test or Criterion

n (range)

Coefficient

Information (including normative data) / Subjects

range

median

Predictive

K

aReading

10

-

0.84

American Indian or Alaska Native; Winter to Spring

Concurrent

K

aReading

10

-

0.79

American Indian or Alaska Native Data collected in Spring

Predictive

K

aReading

8

-

0.72

American Indian or Alaska Native Fall to Spring prediction

Predictive

K

aReading

97

-

0.69

Asian; Fall to spring prediction

Concurrent

K

aReading

106

-

0.61

Asian; Data collected in Spring

Predictive

K

aReading

53

-

0.71

African American; Winter to Spring

Predictive

K

aReading

54

-

0.57

African American; Fall to Spring

Predictive

K

aReading

37

-

0.58

Hispanic; Fall to Spring

Predictive

K

aReading

39

-

0.76

Hispanic; Winter to Spring

Predictive

K

aReading

28

-

0.64

Multiracial; Fall to Spring

Predictive

K

aReading

29

-

0.60

Multiracial; Winter to Spring

Concurrent

K

aReading

29

-

0.65

Multiracial; Data collected in the Spring

 

Alternate Forms

GradeK
RatingFull bubble

1. Evidence that alternate forms are of equal and controlled difficulty or, if IRT based, evidence of item or ability invariance: All 26 letters in the English alphabet were used. Every form includes each letter once in upper-case (26) and once in lower-case (26). Forty-two (42) additional letters were provided to account for students who read letter sounds at a faster rate.  Each form is organized so that each row alternates where the first row is all lower-case and the second row all upper-case, and so on. The order of letters were randomly chosen and alternated between continuous and stop sounds. The first line of letters did not contain consonants judged to be less useful for beginning readers (e.g., z, x, etc.). The letters c, g, and all vowels were not included in the rate portion of the measure and, instead, are placed separately at the bottom of the form because these have dual sounds. Directions are provided to solicit the soft and hard sounds of c and g and the short and long sounds of each vowel. To determine parallel form construction, a one-way, within-subjects (or repeated measures) ANOVA was conducted to compare the effect of LS Alternate forms (n=5) on the number of correct responses within individuals. There was not a significant effect for forms F(1,139) = 0.96, p=0.33. This indicates that different forms did not result in significantly different mean estimates of correct responses.

2. Number of alternate forms of equal and controlled difficulty: 20

3. Number of items in item bank for each grade level: There are 20 forms and 94 letters on each form. Each form has 26 letters  in both upper- and lower-case (total of 52 letters). An additional 42 letters are available on each form so the assessment is appropriate for students who are reading letter sounds at a faster rate. 

Rates of Improvement Specified

GradeK
RatingEmpty bubble

1. Is minimum acceptable growth (slope of improvement or average weekly increase in score by grade level) specified in manual or published materials?

Yes.

a. Specify the growth standards:
 
The table below provides average weekly growth by percentile and season for Kindergarten students.
 

Metric: Rate

 

Kindergarten

Percentile

Winter

Spring

90th

9.34

7.89

80th

8.00

6.42

70th

6.95

5.27

60th

5.97

4.39

50th

5.14

3.54

40th

4.32

2.76

30th

3.46

2.03

20th

2.65

1.15

10th

1.65

0.26

Average

5.36

3.83

SD

2.87

2.86

N

3046.00

5232.00

Range

0.27 - 12.34

-1.76 - 19.95

 
b. Basis for specifying minimum acceptable growth:
 
Norm-referenced weekly growth is calculated.
 
Normative profile:
 

Representation: Local
Date: 2013-2014
Number of States: 2
Size: The sample was composed of 26,566 total students across two states. However, one of the states did not provide demographic information by the time of this submission. This state’s sample comprised 10,776 total students, or 40.6% of the total two-state sample. This fact is reflected by the percentages labeled “N/A” or "Unknown" below.
Gender: 28.9% Male, 30.5% Female, 40.6% N/A
Region: Upper Midwest
White, 7.8% Black, 4.6% Hispanic, 40.6% Unknown, 1.0% American Indian/Alaska Native, 3.4% Asian/Pacific Islander, 2.0% Other, 1.9% Multiracial.
Disability classification: 49.3% of this sample did not receive special education services. 3.4% of this sample did receive special education services; the special education status was unknown for 47.3% of this sample.
Grade distribution: 57.5% kindergarten; 42.5% first grade.

End-of-Year Benchmarks

GradeK
RatingFull bubble

1. Are benchmarks for minimum acceptable end-of-year performance specified in your manual or published materials?

Yes.

a. Specify the end-of-year performance standards:

Kindergarten: 42 Letter Sounds read correct per minute.

b. Basis for specifying minimum acceptable end-of-year performance:

Criterion-referenced

c. Specify the benchmarks:

Low risk (High risk)
Kindergarten:      Fall  =  10 (6)
                           Winter = 28 (22)
                           Spring = 42 (34)

d. Basis for specifying these benchmarks?

Criterion-referenced.

The primary score for interpretation is number of correct letter sounds read per minute. Psychometric evidence is provided and supports number correct or correct per minute as the primary methods of interpretation. Accuracy scores are provided as a supplemental score, such that students who perform at less than 95% accuracy are flagged for the user to consider. Our training materials caution the interpretation of rate-based scores until accuracy is approximately 95%. The goals in the system include number correct and number correct per min as the primary index of growth, but also prompt monitoring of the accuracy of student responding. This is designed to help teachers and other users consider multiple aspects of student performance, which includes number correct, errors, rate and accuracy.

Benchmarks were established for earlyReading to help teachers accurately identify students who are at risk or not at risk for academic failure. These benchmarks were developed from a criterion study examining earlyReading assessment scores in relation to scores on the Group Reading Assessment and Diagnostic Evaluation (GRADE). Measures of diagnostic accuracy were used to determine decision thresholds using criteria related to sensitivity, specificity, and area under the curve (AUC). Specifically, specificity and sensitivity was computed at different cut scores in relation to maximum AUC values.  Decisions for final benchmark percentiles were generated based on maximizing each criterion at each cut score (i.e., when the cut score maximized specificity ≥ 0.70, and sensitivity was also ≥ 0.70; see Silberglitt & Hintze, 2005). Precedence was given to maximizing specificity. Based on these analyses, the values at the 40th and 15th percentiles were identified as the primary and secondary benchmarks for earlyReading, respectively. These values thus correspond with a prediction of performance at the 40th and 15th percentiles on the GRADE, a nationally normed reading assessment of early reading skills. Performance above the primary benchmark indicates the student is at low risk for long term reading difficulties. Performance between the primary and secondary benchmarks indicates the student is at some risk for long term reading difficulties. Performance below the secondary benchmark indicates the student is at high risk for long term reading difficulties. These risk levels help teachers accurately monitor student progress using the FAST earlyReading measures.

Normative profile:

Representation: Local
Date: 2012-2013
Number of States: 1
Size: ~230
Gender: 55% Male, 45% Female
Region: Upper Midwest
Disability classification: 7% Special Education

Procedure for specifying benchmarks for end-of-year performance levels:

Diagnostic accuracy was used to determine cutpoints, or benchmarks, at the 15th and 40th percentile. These correspond to high risk and low risk, respectively. 

Sensitive to Student Improvement

GradeK
RatingFull bubble

Describe evidence that the monitoring system produces data that are sensitive to student improvement (i.e., when student learning actually occurs, student performance on the monitoring tool increases on average):

Across 464 Kindergarten students, the slope for average weekly improvement (β1Week) was significantly different from 0 (β1Week = 0.79; SE = 0.04). In addition, a significant interaction term between Special Education Status and the slope for weekly improvement was observed. That is β3Special Education Status * Week = -0.40 (SE = 0.1). This significant interaction term suggests that students receiving special education services (n = 59), on average, improved significantly less than regular education students. Across 52 Grade 1 students, the slope for average weekly improvement (β1Week) was significantly different than 0 (β1Week = 0.70; SE = 0.09). 

Decision Rules for Changing Instruction

GradeK
Ratingdash

Decision Rules for Increasing Goals

GradeK
Ratingdash

Improved Student Achievement

GradeK
Ratingdash

Improved Teacher Planning

GradeK
Ratingdash

Description of evidence that teachers' use of the tool results in improved planning:

In a teacher-user survey, 82% of teachers indicated that FAST assessment results were helpful in making instructional grouping decisions (n = 401).  82% of teachers also indicated that assessment results helped them adjust interventions for students who were at-risk (n = 369).  Finally, a majority of teachers indicated that they look at assessment results at least once per month (66%), and nearly a quarter of teachers indicated that they look at assessment results weekly or even more often (n = 376).