Methodological Challenges for the 21 Century/file/1120crow.pdf · Methodological Challenges for the...
Transcript of Methodological Challenges for the 21 Century/file/1120crow.pdf · Methodological Challenges for the...
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
1
Methodological Challenges for the 21st Century
Graham Crow Deputy Director, NCRM
University of Southampton, UK
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Outline of Presentation • Historical Context • Five Methodological Challenges • Case Studies of Developing Methodological
Innovations • NCRM’s Role in Meeting Methodological
Challenges
2
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
3
Historical Context • A century is a long time in the history of the social
sciences • ‘In science, each of us knows that what he has
accomplished will be antiquated in ten, twenty, fifty years’ (Max Weber, 1958: 138)
• Weber’s direct experience of innovation in a 1908 survey of occupational careers, ‘obtaining information from the workers themselves’ (1971: 146) rather than from employers
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Historical Context • Establishing trust central to the research
relationship, then, now and into the future (Crow and Pope 2008)
• Methodological innovations always have a history • In the current context, pressure to innovate can be
linked, among other things, to the rapid pace of social change (Crow et al. 2009; O’Connor 2011)
4
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Five Methodological Challenges • 1. Keeping up with and exploiting technological
change • Argument of Mike Savage and Roger Burrows that
‘the sample survey and the in-depth interview are increasingly dated research methods’ (2007: 885)
• Rapid development of on-line methods to keep up with technological revolution in communications (Hooley et al. 2012)
• Developments in administrative data (Boyle 2011)
5
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Five Methodological Challenges • 2. Enhancing research capacity in an integrated
way • Many developments require increasingly specialist
knowledge and skills, but social science is a collaborative endeavour requiring mutual understanding across interdisciplinary teams
• Mixed methods, methodological pluralism • ‘What is?’ presentations e.g. K Charmaz ‘What is
grounded theory?’ http://eprints.ncrm.ac.uk/208/ 6
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Five Methodological Challenges • 3. Respecting research ethics • Questions about the whole research process, from
research design and access to data analysis and ownership, continually re-surface in new forms
• Ethics of use of data from social media an on-going challenge for NCRM TALISMAN node
• Research involves risks; researchers need to make the argument for their ethical choices (Wiles 2012)
7
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Five Methodological Challenges • 4. The democratisation of social research • Working with research partners beyond academia • Pursuing collaborative research and the ‘impact’
agenda does not always lead straightforwardly to a meeting of minds/practices (Crow 2010)
• NCRM Network of Methodological Innovation ‘Dancing with new partners’, next event at the University of Leeds on 10 May 8
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Five Methodological Challenges • 5. Keeping a sense of purpose, and of history • Methodological innovations are more than technical
exercises – importance of keeping in mind why we pursue them (to answer new research questions and to generate better quality data and analyses).
• J Mason (2006) on rationales for mixing methods • A Krishnan (2009) on interdisciplinarity • Treat claims to innovation with caution
9
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
10
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Developing Methodological Innovation • Varying definitions but key elements:
– Should be rooted in genuine attempt to improve some aspect of the research process (not just gimmickry or innovation for innovation’s sake)
– Can comprise developments to established methods as well as new methods
– Should be some level of dissemination (inclusion in text books), acceptance and take-up in research community
– (Coffey and Taylor 2008;Travers 2009; Xenitidou and Gilbert 2010)
11
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Developing Methodological Innovation • NCRM Hub research led by Rose Wiles • Project 1 – analysis of innovation ‘claims’ in
qualitative research in social science 2000-2009 • Project 2 – exploration of the social process of
development and the academic ‘impact’ of three qualitative ‘innovations’
12
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Study 1: Exploring Innovation ‘Claims’
• Systematic review of innovation claims in social science journals 2000-2009; narrative review of claims conducted.
• Journal contents searched using innovat*, new, novel and emerg* in the title or abstract.
• Focus: authors’ self-definition of innovation • 57 papers identified • Categorised by area/topic • Categorised by type of claim [Inception (n=32); Adaptation
(n=6); Adoption (n=19)]
13
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Key Findings
• Study highlights e-research, participatory and creative methods as sites of innovation
• Innovations at interface with arts-based approaches dominate
• Supports the finding that researchers are increasingly claiming innovation
• Limited evidence of wholly new methodologies or designs • Majority of ‘innovations’ appear to be developments to
existing methods or transfer and adaptation of methods from other disciplines
14
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Study 2: Innovation Case Studies • 3 case studies of established innovations to explore the social process of
development and the response of researchers. • Criteria:
– Been in existence for some time to allow for take up – In broad areas identified as sites for innovation in study 1 – Been identified as innovations – Have specific individual associated with their development – Qualitative methods
Methods: • interviews with developer of the method/approach • Interviews with users, reviewers and established academics • Review of citations
15
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
The Cases Online/Virtual ethnography … Netnography
Robert Kozinets Creative methods … Lego Serious Play
David Gauntlett Child-led research … Children as Researchers
Mary Kellett
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Response of the Academic Community: Citations
• Systematic search of UK social science bibliographic databases 1999-2010 for published journal articles/conference papers, in which:
(a) Method/approach is applied/adapted/discussed/ referred to/related book reviewed
(b) Method/approach is specifically linked to innovators we are studying.
(c) Innovators are not authors/co-authors, or otherwise linked to instance of uptake.
17
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Citations of Cases by Year
18
0
5
10
15
20
25
30
35
40
1999 2003 2004 2005 2006 2007 2008 2009 2010 Year
Num
ber
child-led research Creative Research Methods netnography
Gauntlett & Holzwarth (2006)
Kozinets (1998)
Kozinets (2002)
Kellett (2004) Kellett (2010)
Gauntlett (2007)
Kellett (2005a,b)
Kozinets (2010)
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Academic response to innovations
• All acknowledged by the social science community and some take-up
Netnography – 138 citations; 42% apply the method; 76% MBS; evidence of global spread but mostly in specific disciplines of MBS.
Child-led research – 76 citations; 80% referred/referenced the method; citations across SS disciplines; 71% UK based but some international recognition (North America and Australasia).
Creative Research Methods – 37 citations; 76% referred/referenced or discussed the method; media and communication studies but also other disciplines; 51% UK based but some international recognition (EU, Australasia).
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Uptake: Innovators’ Views “It’s taken off, I would say, big time now, because it’s all the way round the World. I mean, my email box is just full from people all over the place wanting either advice or suggestions because they want to do something … I mean, you know, Mexico, Indonesia, all over the place” (Kellett)
“It’s now perceived, and I would say fairly accurately, as being fairly deep, fairly widespread in terms of a whole global network of qualitative consumer researchers who know this method and a number of them are using it and adopting it and adapting it” (Kozinets)
“It’s become noticed. You know, it’s the kind of thing that, when people are compiling their handbooks of qualitative methods, they probably nowadays tend to think, well we should probably have a chapter on that” (Gauntlett)
20
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Factors impacting on uptake • Time • Timeliness • Support and championing • Accessibility and feasibility of uptake • Maturity of innovation • Dissemination and marketing
21
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Academic Responses: Interviewees • Innovation important • People willing to push boundaries, take risks
and experiment • Cases seen as important methodological
developments • But reservations about the drive for innovation
in general
22
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Reservations about Innovation • The nature of innovation in the social sciences: is
anything really new? • Encouragement to disseminate developments at an
early stage: fuelled by the impact agenda, publishers and social media
• Codification of innovations – to make them accessible and transferable
• The rush to uptake: researchers feel they should be using ‘new and exciting’ methods
23
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Reservations about Innovation “I worry about innovation being, well I worry about the politics of it. I worry about it being this big machine, you know, which the whole aim is to just churn out something just because it’s new and I don’t think, just because it’s new means it’s going to be any good … what I don’t like, what I suppose I would worry about, is people sort of thinking, here’s a new trendy, tricksy method, I’m just going to try it for fun, you know without thinking, now why would I do that, and what would it do and how would you understand the kind of data that might emerge from that”
24
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
Conclusions to Research on Qualitative Innovation
• Exploring contemporary innovation is challenging • Social context encourages innovation • Methodological developments and their dissemination
speeding up • Academic community wants innovation but distrusts the
drive to innovation • Acceptance and uptake of the least risky innovations? • Comparison with quantitative innovation will be interesting
(Williams and Vogt 2011)
25
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
NCRM’s Role in Meeting Methodological Challenges
• Promoting methodological innovation involves risk • Not all developments get taken up, and those that
do take time, especially to get into the mainstream (e.g. research methods textbooks, the curriculum)
• Availability on NCRM website of resources from nodes (e.g. LEMMA multilevel modelling, SIMIAN agent-based modelling simulations, MRS systematic reviewing) and beyond
26
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
NCRM’s Role in Meeting Methodological Challenges
• Training and events database with details of forthcoming opportunities, and resources captured from previous events
• Includes DTC training opportunities • Flagship event the Research Methods Festival: 5th
RMF will take place 2-5 July 2012, Oxford • www.ncrm.ac.uk
27
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
A reminder regarding historical context • Q.23 If you are employed in a chemical works, in a factory,
in the metal working industry, or in any other industry which is particularly dangerous, enumerate the safety measures introduced by your employer.
• Q.56 If you are paid piece rates, how are the rates fixed? Does your employer or his representative resort to trickery in order to defraud you of a part of your earnings?
• Q.71 Have you noticed, in your personal experience, a greater rise in the price of the necessities of life, such as food and shelter, than in wages?
28
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
References • Boyle, P. (2011) ‘Making better use of administrative data in
research’ NCRM Annual Lecture http://www.ncrm.ac.uk/TandE/video/PaulBoyle.php
• Crow, G. (2010) ‘Democratising social research’, MethodsNews http://eprints.ncrm.ac.uk/1594/
• Crow, G., Bardsley, N. and Wiles, R. (eds) (2009) ‘Methodological innovation and developing understandings of 21st century society’, Introduction to special issue of 21st Century Society, 4 (2), 115-18.
29
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
References • Crow, G. and Pope, C. (2008) ‘The future of the research
relationship’, Introduction to special issue of Sociology 42 (5), 813-19.
• Hooley, T., Wellens, J. and Marriott, J. (2012, forthcoming) What is online research? London: Bloomsbury Academic.
• Krishnan, A. (2009) ‘Five Strategies for Practising Interdisciplinarity’ http://eprints.ncrm.ac.uk/782/
• Marx, K. (1963) ‘Marx’s Enquête Ouvrière’ in T. Bottomore and M. Rubel (eds) Karl Marx: Selected Writings in Sociology and Social Philosophy. Harmondsworth: Penguin. 30
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
References • Mason, J. (2006) ‘Six strategies for mixing methods and
linking data in social science research’ http://eprints.ncrm.ac.uk/482
• O’Connor, W. (2011) What's New: Reflections on Current Developments in Qualitative Social Research, SRA 2011 conference http://www.the-sra.org.uk/presentations.htm
• Rees, A. (1950) Life in a Welsh Countryside. Cardiff: University of Wales Press.
• Savage, M. and Burrows, R. (2007) ‘The coming crisis of empirical sociology’, Sociology 41 (5), 885-99.
31
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
References • Taylor, C., and Coffey, A. (2008) ‘Innovation in qualitative
research methods: possibilities and challenges’, Cardiff: Cardiff University.
• Travers, M. (2009) ‘New methods, old problems: a sceptical view of innovation in qualitative research’, Qualitative Research 9 (2), 161-79
• Weber, M. (1958) ‘Science as a vocation’, in H. Gerth and C. Mills (eds) From Max Weber: Essays in Sociology. New York: Oxford University Press, 129-56.
32
NCRM is funded by the Economic and Social Research Council
NCRM is funded by the Economic and Social Research Council
References • Weber, M. (1971) ‘A research strategy for the study of
occupational careers and mobility patterns’, in J. Eldridge (ed.) Max Weber. London: Michael Joseph, 103-55.
• Wiles, R. (2012, forthcoming) What are qualitative research ethics? London: Bloomsbury Academic.
• Williams, M. and Vogt, W. (eds) (2011) The Sage Handbook of Innovation in Social Research Methods. London: Sage.
• Xenitidou, M. and Gilbert, N. (2010) The processes of methodological innovation: successful development and diffusion, http://eprints.ncrm.ac.uk/2047/
33