This study explored the validity of a widely-used, performance-based assessment of children’s school readiness skills in the fall and spring of preschool. Using a sample of 1109 children (mean age in the fall = 4.54 years; SD = 3.69 months) in 90 classrooms, we compared children’s school readiness skills as assessed by teachers using Teaching Strategies GOLD (TS GOLD) to readiness skills as assessed by independent data collectors using standardized, direct assessments. Findings indicated evidence of convergent validity: TS GOLD scores were significantly associated with other assessments of similar skills. Evidence of discriminant validity was limited: TS GOLD domains were highly associated with one another and did not show differentiation in predicting direct assessment scores. In addition, comparison of intraclass correlations (ICCs) showed that children’s skills were estimated as being much more similar to one another within a classroom when assessed using TS GOLD as compared to the direct assessments. More research is needed to ensure psychometrically sound readiness assessments, and prior to making strong policy and practice recommendations. (author abstract)