This study examines the psychometric properties of the Kindergarten Early Learning Scale (KELS), including its inter-rater reliability and concurrent validity. The KELS is a 10-item assessment of kindergarten children's development that examines three domains: math/science, social emotional/social studies, and language and literacy. Sixty-six teachers and 276 children participated in the study. Teachers received multiple forms of training on the KELS, including an online program and an onsite workshop. To measure inter-rater reliability, teachers' scores on the KELS were checked for agreement with expert observers, in this case experts in the field of early childhood education, elementary education, and performance-based assessment. Sixty-five percent of the teachers achieved greater than 60 percent agreement with the experts' scores. To examine the concurrent validity of the KELS, participating children were assessed using standardized, well-established instruments. Only the concurrent validity of the math/science and language and literacy domains was examined. The KELS displayed moderate relationships with the standardized measures, in particular the KELS oral language, literacy, and math items, with lower correlations between the KELS science item and the standardized measure.