Child Care and Early Education Research Connections

Skip to main content

Statistical analysis for multisite trials using instrumental variables with random coefficients

Multisite trials can clarify the average impact of a new program and the heterogeneity of impacts across sites. Unfortunately, in many applications, compliance with treatment assignment is imperfect. For these applications, we propose an instrumental variable (IV) model with person-specific and site-specific random coefficients. Site-specific IV coefficients can be interpreted as site-average effects of program participation or as site-average effects of participation for the compliers. The validity of these interpretations depends on the analyst's assumptions. Within the framework of a two-level hierarchical linear model, we propose three ways to estimate the mean and variance of these site-specific effects: (a) estimate the impact of program participation and its standard error in each site, then combine these site-specific statistics to estimate the mean and variance of the true site effects; (b) estimate the mean and variance of the effect of treatment assignment on the outcome and the mean and variance of the effect of treatment assignment on program participation; then combine these results to obtain estimates of the mean and variance of the effect of program participation; and (c) use Site by Treatment interactions as multiple instruments. If we assume the IV coefficients to be homogenous across sites, the three approaches are equivalent to variants of familiar two-stage least squares estimates with site fixed effects. Estimates based on our model are valid under a weaker assumption: that site-average levels of compliance are independent of site-average effects of program participation. To illustrate our approach, we evaluate a district-wide policy intended to increase math instructional time in math for low-achieving students. Finally, we discuss how Method (c) can be extended to incorporate multiple mediators. (author abstract)
Resource Type:
Reports & Papers
United States

- You May Also Like

These resources share similarities with the current selection. They are found by comparing the topic, author, and resource type of the currently selected resource to the rest of the library’s publications.

Distributional effects on children’s cognitive and social-emotional outcomes in the Head Start Impact Study: A quantile regression approach

Reports & Papersview

Decomposing treatment effect variation

Reports & Papersview

Assessing the precision of multisite trials for estimating the parameters of a cross-site population distribution of program effects

Reports & Papersview
Release: 'v1.13.0' | Built: 2022-08-08 12:44:31 EDT