Testing the Causal Effects of Social Capital: Design for a Cluster-Randomized Field Trial
Adam Gamoran and Ruth N. López Turley
University of Wisconsin-Madison
Social Capital: Conceptual and Causal Ambiguity
Social capital is one of the most popular terms in social science today Viewed as the source of many positive outcomes
Test scores, school completion, social adjustment, mental and physical health
Decline of social capital is seen as responsible for many social ills Crime, apathy
Causal role of social capital is ambiguous
Social Capital: Conceptual and Causal Ambiguity
Concept of social capital is also ambiguous Relations of trust, mutual expectations, and
shared values embedded in social networks Not possessed by individuals Resides in the relationships individuals have with
one another Individuals can draw upon social capital in their
networks Social capital facilitates the flow of information
and the development and enforcement of norms
Concepts of Social Capital Qualities of social networks that signify social
capital: How do we know if social capital is present? Intergenerational closure
Concepts of Social Capital Qualities of social networks that signify social
capital: How do we know if social capital is present? Intergenerational closure Trust
Network members rely on one another Facilitates sharing of norms and information
Shared expectations Also facilitates supporting norms and distributing
information
Concepts of Social Capital Contrary to Coleman (1988), we do not define
social capital by its function Contrary to Portes (1998), we view social
capital as a collective rather than as an individual attribute
We follow Sampson et al. (1999): “…social capital for children refers to the resource potential of personal and organizational networks…”
Domains of Social Capital Parent-school relationships Parent-parent relationships Parent-child relationships
Parent-school Parent-parent Parent-child
Trust
Shared expectations
Intergenerational closure
Mechanisms of Social Capital For young children, social capital operates
through their parents Two primary mechanisms
Social support Parents who feel more connected to others have better
access to information and are better able to establish and enforce norms with their children
Social control Parents’ positive social networks offer collective
socialization of children
Social Capital and Inequality Unequal social capital contributes to unequal
child development Among U.S. Latinos, social capital within
family networks is high, but parent-school social capital is low
Building family-school social capital may enhance child outcomes particularly for Latinos – the focus of our empirical analysis
The Causal Role of Social Capital Many studies have tested the relation between
social capital and child outcomes Most rely on longitudinal data Nonetheless, causal direction is ambiguous
Does social capital foster school success, or do stronger social ties emerge in communities that have more effective schools?
Some group norms can negatively affect child outcomes!
The Causal Role of Social Capital Survey research may overestimate effects of social
capital (Mouw, 2006) Endogeneity: Group members influence one another at
the same time Unobserved selectivity: unmeasured conditions lead to
both common memberships and common outcomes Statistical efforts to resolve these causality issues
rely on questionable assumptions E.g., effects are unbiased net of control variables
The Causal Role of Social Capital An experimental design offers a more
rigorous approach to testing social capital’s causal role No unobserved selectivity: Assignment to
“treatment” is random Avoid endogeneity problem through multilevel
assessment of social capital effects
The Causal Role of Social Capital Conditions for an experimental assessment of social
capital effects:1. An intervention that manipulates social capital
2. Random assignment to treatment and control
3. Random assignment of groups of individuals (because social capital is an attribute of groups, not individuals)
4. Tools for measuring social capital and outcomes
5. Statistical methods suitable for analysis of a cluster-randomized trial
1. An Intervention that Manipulates Social Capital FAST: Families and Schools Together
A multi-family group prevention program Implemented in three stages
Outreach to parents 8 weeks of multi-family group meetings 2 years of monthly follow-up meetings led by parents
1. An Intervention that Manipulates Social Capital Elements of FAST
Led by a parent-professional partnership Culturally representative and adapted Research-based activities
Family meal Group singing Family games Parent support/ children’s time One-to-one responsive play Closing circle
1. An Intervention that Manipulates Social Capital Prior research on FAST
4 previous randomized trials have documented positive outcomes for children’s social and academic outcomes
These studies have occurred at the individual or classroom levels
School-wide, “multi-hub” FAST is likely to have even more powerful effects
1. An Intervention that Manipulates Social Capital Prior research on FAST
FAST builds social capital Parent-school: Reduces alienation from school
authorities, and increases comfort level Parent-parent: Reduces isolation of parents by
creating a parent support group Parent-child: Improve relationship through one-on-
one responsive play
Particularly valuable for immigrant communities
2. Random Assignment to Treatment and Control: Experimental Design Research Sites
San Antonio, TX: A large, long-standing Latino populations (51% of students)
Milwaukee, WI: A rapidly growing Latino population (21% of students)
Experience with FAST, community agencies available to implement
Agreed to implement FAST in treatment schools, not in control schools Subject to agreement of principals and teachers They love FAST, this won’t be a problem
3. Random Assignment of Groups of Individuals: Experimental Design 26 schools from each district (13 treatment
and 13 control), total of 52 schools All first-grade families will be invited to
participate We anticipate 75% participation rate, 20%
attrition rate = 60% long-term follow-up Three years of data collection (grades 1 to 3)
3. Random Assignment of Groups of Individuals: Experimental Design How did we decide on 52 schools?
Power analysis
Power Analysis: Assumptions Power criterion (1 – β) = .80 Probability of Type I error () = .05 Within-school sample size (n) = 60 Effect size () =.25 Intraclass correlation () = .10 Covariate correlation (r) = .40-.60
Power Analysis: Conclusion Under reasonable assumptions, a sample of
52 schools will provide sufficient power to detect the effects of social capital, if they exist.
4. Tools for measuring social capital and outcomes Outcomes
Parent and teacher ratings of child social skills and problem behaviors (grades 1 and 3)
Teacher ratings of child academic competence High-stakes standardized tests of reading and
mathematics
4. Tools for measuring social capital and outcomes Social capital
Parent social capital questionnaire Our only pre-intervention measure
Not really needed for experimental design, but of interest in its own right
Follow-up measures in the spring of grades 1 and 3
Key sources: Bryk and Schneider (2002); McDonald and Moberg (2002)
4. Tools for measuring social capital and outcomes Social capital
Parent Involvement in School Questionnaire Indicators of trust and shared expectations in
parent-school and parent-child relationships Separate forms with parallel questions from
parent and teacher perspectives Source: Shumow, Vandell, and Kang (1996) Completed by teachers and parents at the end of
grades 1 and 3
5. Statistical methods suitable for analysis of cluster-randomized trial This study relies on place-based random
assignment CRT: Cluster-randomized trial Randomization is at the aggregate level Well suited to contextual investigations Must assess the intervention at the level at
which randomization occurs
5. Statistical methods suitable for analysis of cluster-randomized trial A multilevel model is the appropriate
statistical approach to analysis of CRT Captures variability both at the level of the cluster
and within clusters In our case: students within schools Treatment is at the level of the school Theoretically, social capital is also at the level of
the school We allow for individual-level variation
5. Statistical methods suitable for analysis of cluster-randomized trial School-level control variables reduce
variation between schools, permit more precise treatment effects
Individual-level background controls also increase precision More importantly, multilevel interactions permit
estimation of differential treatment effects
Multilevel Models: Linear OutcomesLevel 1. Yij = ß0j + ß1j(SEX)ij + ß2j(LATINO)ij +
ß3j(BLACK)ij + ß4j(POVERTY)ij + rij
Level 2. ß0j = γ00 + γ01(MEAN PRIOR ACH)j + γ02(PERCENT POVERTY)j + γ03(PERCENT LATINO)j + γ04(PERCENT BLACK)j + γ05(FAST)j + γ06(CITY)j + γ07(PERCENT LATINO x FAST)j + γ08(PERCENT BLACK x FAST)j + u0j
Level 2. ß2j = γ20 + γ21(FAST)j + γ22(CITY)j + u2j
ß3j = γ20 + γ21(FAST)j + γ22(CITY)j + u3j
ß4j = γ20 + γ21(FAST)j + γ22(CITY)j + u4j
5. Statistical methods suitable for analysis of cluster-randomized trial By adding social capital to the model, we test
whether social capital accounts for the effects of FAST on child outcomes
Main focus is on school-level effects
Multilevel Models: Linear OutcomesLevel 1. Yij = ß0j + ß1j(SEX)ij + ß2j(LATINO)ij + ß3j(BLACK)ij
+ ß4j(POVERTY)ij + ß5j(SOCIAL CAPITAL)ij
+ rij
Level 2. ß0j = γ00 + γ01(MEAN PRIOR ACH)j + γ02(PERCENT POVERTY)j + γ03(PERCENT LATINO)j + γ04(FAST)j + γ05(MEAN SOCIAL CAPITAL)j + γ06(CITY)j + u0j
5. Statistical methods suitable for analysis of cluster-randomized trial Additional challenges
Uncommon measures: Different tests in Texas and Wisconsin Linking strategy, corrected for unreliability Examine probability of reaching the proficiency
threshold rather than test score
5. Statistical methods suitable for analysis of cluster-randomized trial Additional challenges
Bias in social capital effects FAST effects will be estimated without selectivity bias Social capital effects will also be estimated without selectivity
bias if they derive only from FAST This is probably not the case
If social capital occurs independently of FAST, an omitted variable may affect social capital and child outcomes Use pre-FAST measure to check Use FAST as an instrument for social capital Control for pre-FAST social capital
5. Statistical methods suitable for analysis of cluster-randomized trial Additional challenges
Bias in social capital effects Differential non-response by treatment and control
parents Consent will be obtained prior to randomization Follow up a random subsample of non-respondents with
home visits
5. Statistical methods suitable for analysis of cluster-randomized trial Additional challenges
Fidelity of implementation Implementation study
Implementation checklist Interviews, focus groups with parents and teachers Including interviews with 2 non-participating parents in
each treatment school
Qualitative data will provide more nuanced insights on the mechanisms through which FAST affects (or does not affect) child outcomes
Conclusions The term “social capital” has reflected many
different ideas in different writings Causal ambiguity has been a consistent
limitation of social capital research By manipulating social capital
experimentally, we aim to provide a more persuasive test of social capital effects
References Bryk, A. S., & Schneider, B. L. (2002). Trust in schools: A core resource for
improvement. New York: Russell Sage Foundation. Coleman, J. S. (1988). Social capital in the creation of human capital. American
Journal of Sociology, 94(Suppl.), S95–S120. McDonald, L., & Moberg, D. P. (2002). Social relationships questionnaire.
Madison, WI: FAST National Training and Evaluation Center. Mouw, T. (2006). Estimating the causal effects of social capital: A review of recent
research. Annual Review of Sociology, 32, 79–102. Portes, A. (1998). Social capital: Its origins and applications in modern sociology.
Annual Review of Sociology, 24, 1–24. Sampson, R. J., Morenoff, J. D., & Earls, F. (1999). Beyond social capital: Spatial
dynamics of collective efficacy for children. American Sociological Review, 64(5), 633–660.
Shumow, L., Vandell, D. L., & Kang, K. (1996). School choice, family characteristics, and home-school relations: Contributors to school achievement? Journal of Educational Psychology, 88, 451–460.
Further Reading on Cluster-Randomized Trials
Bloom, H. S. (2006). Learning more from social experiments: Evolving analytic approaches. New York: Russell Sage Foundation.
Bloom, H. S., Bos, J. M., & Lee, S. W. (1999). Using cluster random assignment to measure program impacts: Statistical implications for the evaluation of education programs. Evaluation Review, 23, 445–469.
Borman, G. D., Slavin, R. E., Cheung, A., Chamberlain, A., Madden, N., & Chambers, B. (2005). Success for All: First-year results from the national randomized field trial. Educational Evaluation and Policy Analysis, 27(1), 1–22.
Boruch, R., May, H., Turner, H., Lavenberg, J., Petrosino, A., & de Moya, D. (2004). Estimating the effects of interventions that are deployed in many places: Place-randomized trials. American Behavioral Scientist, 47, 608–633.
Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2, 173–185.
Further Reading on FAST Abt Associates. (2001). National evaluation of family support programs: Vol. B. Research
studies: Final report. Cambridge, MA: Author. Retrieved February 12, 2007, from http://www.abtassoc.com/reports/NEFSP-VolB.pdf
Kratochwill, T. R., McDonald, L., Levin, J. R., Young Bear-Tibbetts, H., & Demaray, M. K. (2004). Families and Schools Together: An experimental analysis of a parent-mediated multi-family group intervention program for American Indian children. Journal of School Psychology, 42, 359–383.
McDonald, L., Moberg, D. P., Brown, R., Rodriguez-Espiricueta, I., Flores, N., Burke, M. P., et al. (2006). After-school multifamily groups: A randomized controlled trial involving low-income, urban, Latino children. Children and Schools, 18, 25–34.
U.S. Office of Juvenile Justice and Delinquency Prevention. (2006). Families and Schools Together (FAST). In U.S. Office of Juvenile Justice and Delinquency Prevention, OJJDP model programs guide. Retrieved February 11, 2007, from http://www.dsgonline.com/mpg2.5/TitleV_MPG_Table_Ind_Rec.asp?ID=459
U.S. Substance Abuse and Mental Health Services Administration. (2005). Families and Schools Together (FAST). In U.S. Substance Abuse and Mental Health Services Administration, SAMHSA model programs: Effective substance abuse and mental health programs for every community. Washington, DC: Author. Retrieved February 11, 2007, from http://www.modelprograms.samhsa.gov/pdfs/Details/FAST.pdf
Top Related