Child Care and Early Education Research Connections

Skip to main content

Fighting bias with statistics: Detecting gender differences in responses to items on a preschool science assessment

Differential item functioning (DIF) and differential distractor functioning (DDF) are methods used to screen for item bias (Camilli & Shepard, 1994; Penfield, 2008). Using an applied empirical example, this mixed-methods study examined the congruency and relationship of DIF and DDF methods in screening multiple-choice items. Data for Study I were drawn from item responses of 271 female and 236 male low-income children on a preschool science assessment. Item analyses employed a common statistical approach of the Mantel-Haenszel log-odds ratio (MH-LOR) to detect DIF in dichotomously scored items (Holland & Thayer, 1988), and extended the approach to identify DDF (Penfield, 2008). Findings demonstrated that the using MH-LOR to detect DIF and DDF supported the theoretical relationship that the magnitude and form of DIF and are dependent on the DDF effects, and demonstrated the advantages of studying DIF and DDF in multiple-choice items. A total of 4 items with DIF and DDF and 5 items with only DDF were detected. Study II incorporated an item content review, an important but often overlooked and under-published step of DIF and DDF studies (Camilli & Shepard). Interviews with 25 female and 22 male low-income preschool children and an expert review helped to interpret the DIF and DDF results and their comparison, and determined that a content review process of studied items can reveal reasons for potential item bias that are often congruent with the statistical results. Patterns emerged and are discussed in detail. The quantitative and qualitative analyses were conducted in an applied framework of examining the validity of the preschool science assessment scores for evaluating science programs serving low-income children, however, the techniques can be generalized for use with measures across various disciplines of research. (author abstract)
Resource Type:
Reports & Papers
United States

Related resources include summaries, versions, measures (instruments), or other resources in which the current document plays a part. Research products funded by the Office of Planning, Research, and Evaluation are related to their project records.

- You May Also Like

These resources share similarities with the current selection.

Fighting Bias with Statistics: Detecting Gender Differences in Responses to Items on a Preschool Science Assessment

Administration for Children and Families/OPRE Projects

Differing levels of gender salience in preschool classrooms: Effects on children's gender attitudes and intergroup bias

Reports & Papers

Flying Start, summary statistics 2013-14

Reports & Papers
Release: 'v1.40.0' | Built: 2023-11-07 08:40:06 EST