Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN...
-
Upload
raul-coxwell -
Category
Documents
-
view
231 -
download
6
Transcript of Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN...
![Page 1: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/1.jpg)
Jacobsen, D. M.
Complementary Research Methods
Michele Jacobsen, PhD
Postdoctoral Fellow, SERN
University of [email protected]
http://www.ucalgary.ca/~dmjacobs/phd/methods/
![Page 2: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/2.jpg)
Jacobsen, D. M.
Overview
• Outline relative merits of Triangulation - the integration of both qualitative and quantitative research methodologies
• Review a variety of research methods:
– Experimental design, ethnography, case study, survey methods.
• Discuss Nowaczyk and Underwood’s (1995) paper.
![Page 3: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/3.jpg)
Jacobsen, D. M.
Fundamental Goals...
• Of Science:– To Understand, To Predict, To Control
• Of Scientists:– To communicate discoveries and
findings to a community of peers
![Page 4: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/4.jpg)
Jacobsen, D. M.
Designing ResearchDimensions of Analysis
• Research Purposes - theoretical or applied?
• Research Problems - what questions are asked?
• Research Settings - simulated or natural?
• Research Investigators - background and training
• Research Methods - a continuum
– Experimental, Ethnography, Case study, Survey
![Page 5: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/5.jpg)
Jacobsen, D. M.
Evaluating Research
• Validity– A concern for most social scientists is the complex nature of the
phenomena under study: human behavior.
– Multiple perspectives are required in order to adequately reflect the richness of these complexities.
• Reliability– Consistency, Replicability
• Usefulness or Value of Investigation– Contribution to knowledge
– Advance THEORY and PRACTICE in discipline
![Page 6: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/6.jpg)
Jacobsen, D. M.
Research MethodologiesA continuum rather than “either/or”
• Qualitative – Goal: To Understand,
Predict• Descriptive accounts
• Similarities and Contrasts
– Applied and Theoretical
– Research Questions
– Field study• Natural conditions
• Quantitative– Goal: To Predict and
Control• Measure and Evaluate
• Generalize to population, reproduction
– Basic and Theoretical
– Hypothesis testing
– Lab study• Controlled, contrived
![Page 7: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/7.jpg)
Jacobsen, D. M.
Validity and Reliability
• Both Quantitative and Qualitative research designs seek reliable and valid results. For example:– Quantitative Reliability: Data that are consistent or
stable as indicated by the researcher's ability to replicate the findings.
– Qualitative: Validity of findings are paramount so that data are representative of a true and full picture of constructs under investigation.
![Page 8: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/8.jpg)
Jacobsen, D. M.
Part Versus Whole
• “Whole” is often greater than “Parts”
• It is a non-trivial matter to infer the behavior of the whole from the behavior of its parts
– Quantitative research designs strive to identify and isolate specific variables within the context (seeking correlation, relationships, causality) of the study.
– Qualitative design focuses on a holistic view of what is being studied (via documents, case histories, observations and interviews).
![Page 9: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/9.jpg)
Jacobsen, D. M.
Data Collection
• Quantitative– Emphasis on numerical data, measurable variables
– Data is collected under controlled conditions in order to rule out the possibility that variables other than the one under study can account for the relationships identified
• Qualitative– Emphasis on observation and interpretation.
– Data are collected within the context of their natural occurrence.
![Page 10: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/10.jpg)
Jacobsen, D. M.
Static and Dynamic
• Quantitative– The accumulation of facts and causes of behavior
through careful isolation, measurement and evaluation of variables.
– Predictability and Control over time.
• Qualitative– Concerned with the changing and dynamic nature of
reality.
– Understanding a Point in time
![Page 11: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/11.jpg)
Jacobsen, D. M.
Triangulation
• Combines independent yet complementary research methods.– Simultaneous triangulation:
• Use of both qualitative and quantitative methods at the same time
• e.g., Survey methods and Case study
– Sequential triangulation: • Results of one method are essential for planning the next method
• e.g., Exploratory Pilot study precedes Experimental design
• Also known as “mixed methods”
![Page 12: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/12.jpg)
Jacobsen, D. M.
Benefits of Triangulation
• Advantages of each complement the other– resulting in a stronger research design, and
– more valid and reliable findings.
• Inadequacies of individual methods are minimized– threats to Internal Validity are realized and addressed
• Example– Quantitative design strives to control for bias so that facts,
instances, phenomena can be understood in an objective way.
– Qualitative approach strives to understand the perspective of participants or a situation by looking at firsthand experience to provide meaningful data.
![Page 13: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/13.jpg)
Jacobsen, D. M.
Additional Benefits
• Triangulation offers a balance between logic and stories.
• Qualititative research, which emphasizes exploration, understanding, contextualizing, introspection, and theory construction, provides a strong base for wider quantitative measures, scaling, and generalization.
• Quantitative research, which emphasizes large samples, can provide an overview of an area that can reveal patterns, inconsistencies, and so forth, that can be further investigated with qualitative methods.
![Page 14: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/14.jpg)
Jacobsen, D. M.
Effectiveness of Triangulation
• In order for Triangulation to be used effectively, four principles must be adhered to:
– 1. research question(s) must be clearly focused;
– 2. strengths and weaknesses of each chosen method must complement each other;
– 3. data collection methods should be selected according to their relevance to the nature of the phenomenon being studied;
– 4. a continual evaluation of the approach should be under-taken during the study. Corner (1990)
![Page 15: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/15.jpg)
Jacobsen, D. M.
Range of Research Methods
• Experimental design
• Ethnography
• Case study
• Survey
![Page 16: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/16.jpg)
Jacobsen, D. M.
Experimental Design
• Hypothesis testing
• Independent and Dependent Variables– For example - Predictor: method of instruction, Resulting
differences: math performance
• Sampling of Population
• Experimental and Controlled Conditions
• Random assignment
![Page 17: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/17.jpg)
Jacobsen, D. M.
Experimental Research
• The researcher does something to the subjects or objects or research, and then attempts to determine the effects of these actions
• Reporting– Careful description of sampling procedure
– Inferential statistics, effect size, and so on.
![Page 18: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/18.jpg)
Jacobsen, D. M.
Ethnography
• Defined: a picture of the “way of life” of some identifiable group of people
• Anthropology - “doing fieldwork”, “going native”
• Preoccupied with culture, and how people interact with each other
• Qualitative Methodology - Both a research process and a product
– Outcome: an ethnographic account
![Page 19: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/19.jpg)
Jacobsen, D. M.
Ethnographic Process
• The ethnographer is the primary research instrument
• One year or more in the field setting
– long enough to see a full cycle of activity
– For example, a full school year
• Tension and balance between involvement and detachment– Outsider’s broad and analytical perspective on group studied
– Insider view, familiarity, empathy, identification with group
![Page 20: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/20.jpg)
Jacobsen, D. M.
Field Research Techniques
• An Inquiry Process of multiple methods:– Participant observation
• privileged, active participant
• passive observer
– Interviewing• key informants, structured, unstructured
• groups, surveys and questionnaires
– Making and using records• historical documents, archives, written records
![Page 21: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/21.jpg)
Jacobsen, D. M.
Validity and Reliability of the Ethnographic Account
• “The satisfactoriness of the explanation is what counts, not the power of the method for deriving it”.
• Significance is derived socially, not statistically
![Page 22: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/22.jpg)
Jacobsen, D. M.
Case Study
• Understanding the intricate complexity, idiosyncrasy of one particular case– investigation of a “bounded system”– Some entity deemed worthy of close watch
• a single child, a single classroom, a single school, a single national program…
• Goals– Understand and report the uniqueness of individual cases
(both commonalities and differences)– Usually no attempt to represent case by single or multiple
“scores”
![Page 23: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/23.jpg)
Jacobsen, D. M.
Case Study Methods
• Similar to ethnographic field methods– ASKING - Interviews
• Gather narrative and testimony
– WATCHING - Observations
– SEARCHING - Written records and artifacts
• Reporting– Develop a conceptual structure, look for patterns,
consistencies, repetitions, and manifestations pertinent to your research question(s)
![Page 24: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/24.jpg)
Jacobsen, D. M.
Validity and Reliability
• There are many different stories to be told– Different researchers have different questions to
answer, different conceptualizations of the situation, and set different boundaries for the case
• Generalizability: What is true of one case is often true about other cases– Consistencies can be found - predictability
– How many cases are needed before patterns emerge? It depends...
![Page 25: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/25.jpg)
Jacobsen, D. M.
Survey Research Methods
• Purpose and Goal– Describe specific characteristics of a large group of
persons, objects, or institutions
– Understand present conditions, rather than the effects of particular intervention (as in experimental research)
• Sample of Population– Groups of interest are well defined and chosen using
well defined rules
– Representativeness
![Page 26: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/26.jpg)
Jacobsen, D. M.
Survey Methods
• Mail– postage and printing costs, participation rate
• Telephone– sampling, wage and time costs, participation rates
• Face-to-Face– wage and time costs, participation rates, like structured interview
• Web-based– anytime, anywhere, cost effective
![Page 27: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/27.jpg)
Jacobsen, D. M.
Issues in Survey Construction
• Item (question) and scale construction
• Pilot Testing and revision
• Sampling procedures
• Analysis and reporting of results
• Generalizability
– Drawing conclusions about the conditions, attitudes, opinions, or status of a population of persons, objects, institutions, or other entities.
![Page 28: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/28.jpg)
Jacobsen, D. M.
Nowaczyk and Underwood (1995) http://epaa.asu.edu/epaa/v3n20.html
• "Possible Indicators of Research Quality for Colleges and Universities" – Used qualitative method, “focus group”, to
investigate research quality in higher education
![Page 29: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/29.jpg)
Jacobsen, D. M.
Results
• Academics promoted the use of both quantitative and qualitative measures to report on “quality”– QUANTITY OF:
• Journal publications, conference presentations, books and book chapters, awards, grants, budget, and so on…
– QUALITY OF: • Reputation of publication, reputation of granting agency,
quality of conference, peer reviews of research programs,…
• Quality of institutions that hire graduate students
• Societal benefit of research
![Page 30: Jacobsen, D. M. Complementary Research Methods Michele Jacobsen, PhD Postdoctoral Fellow, SERN University of Calgary dmjacobs@ucalgary.ca dmjacobs/phd/methods](https://reader031.fdocuments.us/reader031/viewer/2022020718/551aa1dc5503466b3a8b576a/html5/thumbnails/30.jpg)
Jacobsen, D. M.
Resources
• Jaeger, R. M. (1997). Complementary Research Methods for Research in Education, (2nd ed). American Educational Research Association: Washington, DC.
• Edyburn, D. L. (1998). The Electronic Scholar: Enhancing Research Productivity with Technology. Prentice-Hall: Columbus, OH.
• Nowaczyk, R. H., & Underwood, D. G. (1995). Possible indicators of research quality for colleges and universities. Education Policy Analysis Archives, 3(20). [On-line]. Available: http://epaa.asu.edu/epaa/v3n20.html
• Bowen, K. A. (1996). The Sin of Omission -Punishable by Death to Internal Validity: An Argument for Integration of Qualitative and Quantitative Research Methods to Strengthen Internal Validity. [On-line]. Available: http://trochim.human.cornell.edu/gallery/bowen/hss691.htm