#TTMethods - Quant Survey Analysis
-
Upload
on-think-tanks -
Category
Documents
-
view
320 -
download
2
description
Transcript of #TTMethods - Quant Survey Analysis
Think Tank Methods: Quantitative Survey AnalysisOn Think Tanks #TTMethods Session 3
Courtney TolmieOctober 21, 2015
2 | R4D.org
Session Plan Introduction – what is Quantitative Analysis?
Quantitative Surveys and Data
Methods of Analysis
Deeper dive – Linking Context study
3 | R4D.org
Introduction – what is Quantitative Analysis Quantitative analysis – use of statistical/econometric
techniques to measure relationships between independent and dependent factors
Types of Quantitative Analysis:
Summary Stats
Cross tabulations
OLS regression
Structural equation models
Complexity, logistical needs, and sophistication
4 | R4D.org
Introduction – Advantages and Disadvantages
Advantages
• Allows assessment of trends across larger samples
• Allows for “simpler” comparisons of unlike units
• Allows for corrections of endogeneity/unobserved heterogeneity
• Allows researcher to assess causality, not just correlation
Disadvantages
• Methods – can gloss over nuanced differences • Methods – harder to have revealed
conclusions/findings• Logistics – the challenge of data
5 | R4D.org
Quantitative Surveys and Data - Options
Use Existing Data Collect New Data Do Both
6 | R4D.org
Quantitative Surveys and Data – Considerations for Collecting Data Developing the model to inform variables – dependent and
independent
Unit of analysis – organization, other?
Sampling – size and bias
Survey – cross-section, repeated cross-section, panel
Method of data collection – in person surveys, phone, electronic
Is there existing data that you can merge?
7 | R4D.org
Analysis - Overview
What do you WANT to test? Change over time Changes compared to the counterfactual Correlations Causation
What CAN you test with the data you have?
Summary Stats
Cross tabulations
OLS regression
Structural equation models
8 | R4D.org
Analysis – Summary Statistics and Cross Tabs
Summary Statistics Averages for variables across the dataset Comparing averages across subgroups of dataset
Cross tabulations Basic correlations between variables in the dataset Significance tests possible for cross tabulations or pairwise
tabulations
Summary Stats
Cross tabulations
OLS regression
Structural equation models
9 | R4D.org
Analysis – Regression analysis
More sophisticated analysis of relationships between variables
What regressions can (but will not necessarily) do: Assess impact of changes in think tank characteristics or other
factors over time (panel data) Correct for unobserved heterogeneity Assess causation
Summary Stats
Cross tabulations
OLS regression
Structural equation models
10 | R4D.org
Analysis – an example
Qualitative research Summary stats
Averages Cross tabulations
Significance of relationship between these Regression analysis
Direction of relationship Other important factor (number of PhD researchers on staff, for
example)
Think tank funding Placement of research in top-tier
journals
11 | R4D.org
Case – Linking Think Tank Performance, Decisions, and Context Research by Results for Development Institute and University
of Washington, with funding from the Think Tank Initiative
Objective – explore the relationship between political, economic, and social contexts and think tanks’ strategic behavior and performance
Mixed methods – literature mapping, comparative case studies, interviews/FGDs, and quantitative analysis
Study completed in 2014
12 | R4D.org
Linking Context – the model
Unit of analysis – individual organization Context variables– country level Third term – how exogenous context interacts with
endogenous characteristics Challenges:
How do we define outcomes? How do we assess think tank characteristics?
13 | R4D.org
Linking Context – the hypotheses
14 | R4D.org
Linking Context – the data
Think tank data Electronic survey Convenience sample Three languages – English, Spanish,
French 94 respondents in 48 countries Combined with existing datasets
Challenges: Response rate – 94 of over 300 sent Completion – 61% Development of survey itself
15 | R4D.org
Linking Context – other data sources
16 | R4D.org
Linking Context analysis – summary statistics
17 | R4D.org
Linking Context analysis – cross tabulations
18 | R4D.org
Lessons
Significant challenges to fielding quantitative survey in the field – should be weighed out
With proper data, quantitative analysis has potential to assess causation and correct for endogeneity
Smart mixed methods is best
Thank you!
19 | R4D.org