Research skills

Unit 5 Research skills.....................................................................................................2

1 Five stages of the research process.............................................................................2

1.1 Conception...........................................................................................................2

1.2 Realisation – making research happen.................................................................2

Description.............................................................................................................3

1.3 Investigation – gathering information..................................................................3

1.4 Interpretation – making sense of data..................................................................4

1.5 Utilisation of research findings............................................................................4

1.6 Managing research...............................................................................................5

2 Designing research questions......................................................................................5

Discussion..............................................................................................................7

3 Research design...........................................................................................................8

Discussion..............................................................................................................9

4 Triangulation.............................................................................................................10

Description...........................................................................................................11

Discussion............................................................................................................12

5 Critical review of research approaches.....................................................................12

Conclusion....................................................................................................................15

Next steps.....................................................................................................................15

References....................................................................................................................15

Notes............................................................................................................................16

1

2

Unit 5 Research skillsThis final unit introduces the research process and identifies some key skills necessary to carry out a successful research project. Five stages of the research process are identified: conceptualisation, realisation, investigation, interpretation, utilisation. We consider each stage before considering the higher level skills needed to effectively carryout research such as designing research questions, research design, triangulation and critical review of research approaches.

1 Five stages of the research processIn this section you will be exploring each stage of a research process in detail.

1.1 ConceptionAt its most basic, this is the ideas stage, when you decide what interests you. It is worth keeping a journal to record your thinking and reflections on intellectual issues as a repository of possible research ideas and problems. But as well as coming from personal reflection, research ideas are also generated by people and organisations you work with, or groups you want to help in some way.

Focussing your research question is a continual process during the first phase of a research project, but it doesn’t usually end there. You often have to come back to refine your research question during the course of research as findings emerge and are interpreted. This was demonstrated in the Massey and Wield paper on science parks studied in unit 4. Their paper argues that conceptualization and reconceptualization is possible and required throughout research. The main points of reference in conceptualisation and reconceptualisation are the literature (what theories are relevant), the methodology you are using, and for practical and policy oriented research, the concerns of potential client or user groups. The other central considerations are scale, resources (people and time) and access. In other words, is it doable with the resources available, and can you get the information required? You can start with a preliminary question, then cycle between each of these until you have refined a question where the theory and method fit well together. It may help to break your central question down into a hierarchy of questions, with the answers to the sub-questions leading to the central question.

1.2 Realisation – making research happenThis is often not accorded the importance it deserves. Research design is about organising your research approach in order to answer your research questions, or at least generate reliable valid information to interpret for your research questions. Typical considerations are quantitative or qualitative or a combination of the two. Another issue is what data sources to use? For example, in survey research, how do you determine the structure of a sample? Or in qualitative case study research, how do

3

you decide which cases to research? Triangulation helps to ensure the validity of findings which we will consider shortly.

When it comes to resourcing research, a key resource is clearly you and your time. But other vital resources you may need could be finance for travel, fieldwork, facilities and equipment, interviewees and interpreters expenses etc. Planning and organising all these is important and it is worth drawing-up plans and schedules to keep track.

Figure 5 Research process in a typical project cycle

Description

Figure 4 Research process in a typical project cycle

End of description

1.3 Investigation – gathering information

4

Key issues here are interpersonal skills, such as teamwork and communication. There is also understanding responsibilities (honesty and integrity of researcher), rights of informants (including data protection), and ethical issues (including confidentiality).

Interview skills need to be rehearsed and practiced. Other important issues are getting the setting right, making people feel comfortable, and ensuring you have a well organised approach to recording what is said accurately. A tape recorder is the best way of doing this, but sometimes this isn’t appropriate as we saw in unit 4, when the researcher had to abandon using a tape recorder because of the unwanted attention it generated which compromised the integrity of the record. Consequently, she had to change her research tool from an open interview to a semi-structured interview in order to aid her recall of the answers to her questions. Answers had to be memorised as note-taking wasn’t an option because the respondents would have felt uncomfortable with this.

The health and safety of yourself, any assistants and respondents should also be considered. It applies to fieldwork as much as to laboratory work. Try to think through the risks in each case and take advice on the best ways of minimising them.

1.4 Interpretation – making sense of dataWe’ve all heard stories of researchers losing all their fieldwork notes either in hard or soft copy. But there are many other less publicised stories of people spending hours searching for notes and computer files that they need when writing up. Remember to write up field work as you go, and as soon as you can. Keep copies in separate places – not all on a computer hard drive. Build up a filing structure for your data; for example, with a spreadsheet. Bear in mind the structure will almost certainly change as you go along.

There are a couple of important points to consider when beginning analysis. The first is that you shouldn’t leave analysis till you have collected all the data. Do as much as you can of preliminary analysis or reflections in a research journal as you go along. Secondly, recognise that a substantial amount of analytical thinking also takes place as the writing up stage, so start doing this from day one. Thirdly, the chances are that you will collect more data than you actually need. You need to be prepared to select data that is relevant for answering your questions rather than including it all. The latter would simply undermine the sharpness and coherence of your results and conclusions.

1.5 Utilisation of research findingsAs said above, begin writing from day one, both in your research journal and on the different stages of work as your research develops. As early as you can, map out the structure of the thesis, dissertation or report down to chapter headings so that you begin to slot pieces into this structure early on. Developing writing skills is about doing it regularly. It is also about doing manageable chunks – Somerset Maughan used to write 3000 words before breakfast, but for more pedestrian writers you need to think about chunks within a structure, and setting targets (words/timescale). The structure may change as you write more and more, but time thinking through a good

5

overall structure is time very well spent – and the same applies to working out a structure for each chunk before you start writing. Ensure you get feedback, and give yourself time for editing and for all the apparently insignificant detail like notes and references which can take far longer than you expect! Finally, using your research to make a difference is not just about how you report and communicate your findings – important though these are; it can also be about how you involve stakeholders in the research process.

1.6 Managing researchIt is vital to plan with a chart for all activity giving start and finish times. This can be done as a table on a computer or drawn up on paper. Setbacks are bound to happen, face up to them and try to address them before they turn into disasters! If a major setback or disaster does occur, remember one thing: the fact that someone has probably been there before, so through talking to supervisors and colleagues you may get advice and support. Some disasters are difficult to discuss, but it is really important you don’t push them aside; rather, concentrate on finding the right person to talk to, such as a counsellor.

Reflecting on progress (and the lessons from setbacks) is essential if you want to learn how to be a better researcher. This often takes place naturally, in quiet moments, but more and more people are coming to recognise that formally putting aside time for yourself to do this is important, and trying to encapsulate your reflections in a journal or notebook is also very valuable.

As you can see from this brief overview of the components of the research process, skills around organisation, time management, rigour and routine are necessary to conduct a research project. But there are also some higher level skills which you will need to actively develop if your research is to be effective. These include generating research questions to investigate, research design skills, triangulation and critical review of research approaches which we now consider.

2 Designing research questionsYou may already know the area or problem you wish to research. However, rendering it researchable is not always straightforward. Critical to the success of the research project is generating research questions that have boundaries and are manageable. The following discussion explores how an interest in the role of institutions was conceptualised to render it researchable.

As you should recall, the concept of ‘institutions’ has a dual meaning:

as ‘complexes of norms and behaviours that serve a collective purpose’ (de Janvry, 1993); and

as organizations, when they are established (‘institutionalised’) and embody established norms and roles.

6

All innovation takes place in an institutional setting and must to some extent imply a deliberate attempt to change institutions: either the promotion of change within an institution or the establishment of a new institution. For example, the spread of an innovative use of IT will require organizational change as well as changes to the norms and roles within organizations. A significant change in, say, environmental policy will at least require change in the roles played by existing agencies if not more sweeping organizational changes and changes in underlying values. Hence an important aspect of innovation research is research about deliberate attempts at change in institutions.

Many questions about how innovation can be promoted (or ‘institutionalised’) can be stated in the following general form:

How can change X be implemented successfully in circumstances Y?

For example:

How can IT be used innovatively in primary schools?

How can skills and capacities be built in developing countries?

How can innovation be promoted in small companies?

How can local authorities promote sustainable development?

How can NGOs influence environmental policy?

These questions, stated in this way, are policy questions rather than research questions. In order to turn them into questions which are researchable one has to ask more about the ‘circumstances’. For example:

Who (what organization or agency) is trying to implement this change?

What criteria will be used to judge what is ‘successful’? What values underlie these criteria?

What institutions (established organizations, networks, norms) is the change to be effected within?

One problem about researching ‘How can…?’ questions like those above is that one cannot find out about things which have not yet happened. One can only speculate. So it is important to investigate cases where innovations have occurred or at least where there have been clear-cut attempts at innovations. Yet the likelihood, particularly in the early days of an attempted institutional change, is that there are few clear successes and perhaps many more examples of failure or no attempted change at all. This points to the use of case study as the most appropriate method, focusing on whatever successful cases exist, together perhaps with one or two cases where definite attempts failed in a way which promises to be illuminating.

7

So the general ‘How can…?’ question can be turned into a researchable question of the following general form:

In those cases where attempts at X have been successful, how were they done and why have they been successful?

Turning such policy questions into researchable questions will usually entail bringing in some specificity about the ‘circumstances’. For example, are we interested in the efforts of particular implementing agencies, or in attempts at innovation in a particular region or sector?

Activity 33

Think of an innovation, the possible implementation of which is close to your own research interests. Try going through the following sequence:

Question 25

Write a policy question of the general ‘How can…?’ form above which relates to the innovation you are interested in.

Question 26

Answer (briefly) the above questions about the ‘circumstances’.

Question 27

What problems do you see in trying to research this policy question directly? How do they relate to the brief discussion above?

Question 28

Now turn your policy question into a researchable question of the form suggested above.

Discussion

Not everyone’s research interests can be turned into a question of exactly the same form. But I hope you were able to see that there are common kinds of problems in researching innovation or change directly which may be overcome by concentrating on cases of success. Indeed, even with a research question which prioritises cases of success like this, one can still include other cases in one’s investigations. The general form of research question put forward here is only one way of investigating innovation in institutions – although I would suggest it is an important one.

End of discussion

This example demonstrates how to move from an open and broad question, to a series of more discrete researchable questions. Armed with your research questions you can

8

then identify what your knowledge needs are in order to answer them. Each research question may need a particular research method to arrive at the data evidence you require. Thus, you will have to consider in some detail your overall research design, informed by what is manageable from the perspective of your resources and time.

3 Research designOne of the most difficult parts of doing research is designing the research process so that the thinking element melts naturally into the practical element or how to address the overall mix of theory and practice in the research process. One difficult issue is how to move from a big question or issue to be able to design a piece of doable research which we discussed above. Usually a range of methods can be tailored to a particular research question - a choice of methods is available for each research question. This is explored in the next reading which is an extract from a paper on a clothing cluster in New York City. The research is about exploring the concept of embeddedness and how that changes economic relations and outcomes, or to put it another way: how social relations change market transactions. The network studied is not highly innovative, but joint problem solving arrangements and complex adaptation are present as foci for knowledge production. The concept of embeddedness helps explain how trust develops in networks and is relevant to study of the way clusters operate as you may recall from your study of clusters in unit 3. The paper, however, adopts quite a narrow view of embeddedness based only on structural relations between economic actors, and ignoring (deliberately) other dimensions of embeddedness such as institutions and culture, which was embraced by the theoretical framework in the unit 3 paper on north Italian clusters. Nonetheless this simplifying of the research field allows a very focused and fine-grained analysis through an ethnographic approach, with clear and important findings (including the risks of over-embeddedness, leading to reduced exposure to information external to the network).

The extract is particularly interesting because of its detailed exposition of the methodology used, the boundaries set around the investigation and the rationale for doing what was done. The appendix is also worth looking at as it lists the questions that informed the open-ended interviews and provides a useful example of how the big question of how embeddedness changes economic relations and outcomes was deconstructed and reframed to provide a manageable framework for research.

Activity 34

Read pp. 38-42 (starting Research methodology to include Appendix) of Brian Uzzi (1997) ‘Social structure and competition in interfirm networks: the paradox of embeddedness’, along with the appendix on p.67.

PDF content unavailable

Question 29

How does the author justify his choice of field?

9

Question 30

Do you have any comments on the choice of firms and interviewees? Can you suggest any variations on this aspect of research design that might have been worth considering?

Question 31

How were interviews and field observations used to improve the validity of the findings?

Question 32

What was the purpose of the ‘cross site display’?

Question 33

Note the different phases in the study, and the gradual development of a framework for interpreting the data, and that this was initially based on theory. How was ‘construct validity’ tested, and do you have any comment on this approach to validity?

Question 34

The questions at the end, clearly relate to complex theoretical terminology, but Uzzi has been quite successful at pitching them at a level that the ordinary senior manager would understand. How does Uzzi guard against response bias?

Discussion

Question 29

Classic ‘arms length’ market relations ought to apply in this type of highly competitive field; an ethnographic approach provides rich data for theorising and analysis, but with 23 cases generalisability is only moderate.

Question 30

Considerable attention has been paid to the issue of representativeness, and to the potential influence of size and ethnicity, and firms were not chosen to be part of the same trading network. This suggests some design variations: either examining a trading network, or examining pairs (firms that trade with each other) – this would allow triangulation on economic transactions from the perspective of each partner pair. However this would have required a larger sample to address the issue of representativeness. There is considerable variation in size, from 2-182 employees. This does raise questions about the interview strategy in the larger firms, so that relevant social networks are examined, by selecting appropriate interviewees.

Note that the unit of analysis was the interfirm relationship, and that various sources of information were used to gather background statistics and provide a sampling base.

10

Question 31

Ethnographic interview accounts were compared with field observations to check for differences.

Question 32

It is often the case in ethnographic studies that researchers quantify their data in some respects to estimate frequency and weighting of data across cases (as here).

Question 33

Construct validity has been tested by getting industry experts to check if the interpretation (based on the conceptual framework) is plausible. There is considerable debate amongst some ethnographic researchers regarding validity (with some arguing there is a crisis of representation, based on the perspective that interpretations are just constructs of the ethnographers mind).

Question 34

Uzzi partly addresses response bias by non-directive questioning; there may be a tendency for respondents to cast themselves in a good light; also his triangulation of field observation and ethnographic data would help remedy this.

End of discussion

4 TriangulationThe concept of triangulation suggests that this broadness is a strength; that greater confidence can be placed in research findings based on the points of agreement between the results of diverse methods. Triangulation involves applying a range of different perspectives on an issue in order to reduce uncertainty about it. The greater the differences between the perspectives, the more the confidence that can be placed in any agreement between their findings in a particular case. Triangulation therefore relies on an understanding of the sources of bias embodied in the different methods – their strengths and weaknesses as vehicles for engaging with the research context.

Triangulation is originally a term from navigation or cartography. A navigator can pinpoint their position by taking bearings on three features whose identity and position are known. Theoretically the lines should intersect one another at a single point, but in practice there are almost always errors in measurement. Plotting lines on a map on these bearings from the location of the three features will produce a triangle. The interior of the triangle is the best available approximation for the navigator’s position. The reason that three bearings are taken, rather than two is because the third line gives an idea of the magnitude and direction of the errors introduced by the other two.

11

Figure 6 An example of navigation by triangulation

Description

Figure 6 An example of navigation by triangulation

End of description

The metaphor carries over to research, where the term triangulation is used in two different senses. In survey and interview methods, triangulation refers to asking the same question in different ways, or cross-checking responses against one another. In semi-structured interviewing, for example, it can be helpful to point out any perceived discrepancies between replies to different questions, and ask the interviewee to explain them or to clarify any misunderstandings. The other sense refers to the use of multiple perspectives in order to converge on an understanding of a particular issue or research question. The different perspective may be embodied in different methods, data, theories, disciplines and/or researchers. Multidisciplinary research can involve some or all of these elements.

In both cases, information from one perspective is compared to one or more other perspectives. The comparison helps to reduce the uncertainty inherent in taking a particular perspective (whether that is asking a question in a particular way, employing a particular methodology or adopting a particular theoretical standpoint),

12

in as much as the different views agree. It should be noted that even though navigational triangulation generally relies on taking three bearings, it is not necessary to use three questions or three different data types etc. The main idea is to compare between as many divergent perspectives as is reasonable, whether that is two or twelve.

In the next reading, Roe discusses the use of triangulation in policy research on sustainable livelihoods. Sustainable livelihoods for the world’s population are arguably necessary in order to reconcile conservation of the environment with social and economic development giving rise to sustainable development. The example he gives of a triangulation is one based on different theoretical perspectives of sustainable development, and the interconnections between them. Thus, he adopts an approach which draws on the perspectives and methods of more than one traditional academic discipline, by taking an interdisciplinary or multidisciplinary approach [Note 1] . But much of what he says can be applied to triangulation on different methods too as discussed in unit 4.

Activity 35

Read pp. 35-37 (starting ‘Complex policy analaysis’ up to ‘Starting the complex policy analysis’) of Roe (1999). Why does Roe argue that elements of a triangulation be as different as possible?

PDF content unavailable

Discussion

Roe argues that the greater the divergence between the viewpoints upon which a triangulation is based, the greater the confidence that can be placed in its results. Divergent perspectives are ones which introduce opposing biases, and these will tend to cancel one another out. Note that he does not claim that triangulation brings a researcher nearer to the truth. Convergence between different viewpoints is merely taken as a good starting point for pursuing an issue further. In Roe’s terms, bias has been reduced (but not eliminated).

End of discussion

5 Critical review of research approachesA critical review involves looking at the research process with an impartial eye, stepping back from the detail and considering the research as a holistic investigation and the plausibility of the conclusions and recommendations drawn from it. As such you are assessing the construct validity of the research. This involves evaluating whether there are good operational definitions and measures, multiple sources of evidence and a chain of evidence which establishes links between the research questions, research methods, data and conclusions. This is complemented by judging three aspects of the work: the academic rigour brought to bear on the investigation;

13

the contribution to understanding of technology policy and innovation and the accessibility and coherence of the project reporting style.

Key to this will be the conceptualisation of the problem. As Mukherjee and Wuyts say, ‘...good conceptualisation is akin to an opening in a chess game: it sets the stage for the game but does not predetermine its final outcome.’(in Thomas, Chataway and Wuyts, 1998 Fast, p.257). Conceptualising your research question will determine your approach to the data. But even when you have your data, data does not speak for itself. You will need to approach and interrogate the data from different angles, some of which may lead nowhere. Even where some results are disappointing or unexpected, it is worth thinking critically about why this was and why it failed to support the idea you had. Indeed, this may suggest an alternative idea that you may not have uncovered but for this more interrogative approach. So all data is meaningful, both in terms of what it tells you and what it doesn’t tell you. As Mukherjee and Wuyts say,

‘You approach data from different angles because you want to investigate rival notions in the light of evidence each brings to bear on the problem at hand. Through contrastive inference you seek to arrive at that notion which appears more plausible in the light of the overall evidence. Your interest is not just to test a particular idea in isolation ...What matters is to compare ideas. The implications are that you do not merely accept an idea because some evidence points towards it, because it may support other ideas equally well, and you pay particular attention to evidence which goes counter to an idea, because it may point towards a superior notion.

In policy analysis, you may make extensive use of secondary data relevant to your research question. It is important, therefore, to reflect carefully on the limitations inherent in the data. This is not just a question of checking the quality of the data. What matters as well is to question the type of questions data allow you to answer and which questions cannot possibly be tackled with the data at hand. This point matters equally when you collect your own data (for example, through survey research). The design of a survey is predicated by the type of questions you seek to answer.

Analysing data in relation to a specific question almost invariably implies further data manipulation...Data manipulation are the ‘bread and butter’ of good applied research. Good conceptualisation of research often reveals itself first by the quality of data manipulations and, conversely, careless thinking often shows itself in the way data are organized or manipulated.

The question you seek to answer, the type of data at your disposal, and the way you manipulate and transform them, jointly set the stage for the kind of techniques best suited to address the question. This is not a one-way street. Often data are manipulated in certain ways so as to make them amenable to the use of a particular technique. Nor is it a one-shot effort.

In general, trying out different techniques using the same data to answer the same question is a good thing. This way you avoid turning technique into a fetish: a tool to impress rather than to gain insights.

14

Numerical skills matter provided they are not used in a mechanical way. They should be used in combination with, and not as a substitute for, clear conceptualisation of the problem...Numerical skill in itself does not provide magical answers to problems. Good conceptualisation of the problem determines whether numerical skills are used appropriately or foolishly...Few patterns in the data can be seen by merely staring at them. It is through data manipulations, data transformations and the calculation of numerical summaries or the use of graphs that conceptualisation interacts with numerical skill to bring out patterns in data which either test or generate ideas. There is no easy shortcut, but fortunately much mileage can be obtained with relatively simple techniques and extensive practice.

Finally, good data analysis should be fun, although it may involve hard work. There is something very exciting about ‘finding out’. This is why data analysis should never just consist of testing old ideas with new data, but new data should also generate new ideas (Heckman, 1992, p.884). Do not hammer your data into submission in a mindless exercise of number crunching. Nor turn your back to them in an attack of data phobia. It is worth learning to converse with quantitative data in a genuine theory-driven dialogue (in Thomas, Chataway and Wuyts, 1998, pp.257-259)

This quote relates to quantitative data, but I think the guidance is sound for qualitative data too. That is, that data evidence cannot speak for itself, that it has to be explored, interrogated and cross checked if meaningful results are to be derived. Finally, I would like to leave you with a checklist for evaluating research which has utility both for evaluating the research of others when carrying out a literature survey for example, and for producing your own piece of research to ensure it is robust and authoritative.

1. Is the research topic or problem clearly stated and its significance for technology policy and innovation clear?

2. Are the research questions unambiguous and appropriate?

3. Is the research design strong with a range of meaningful and trustworthy methods to facilitate triangulation and a consideration of the ethical dimensions?

4. Does ‘pragmatic opportunism’ inform the research in order to set manageable boundaries?

5. Does the research demonstrate competent and confident critical reflection on research practice with steps taken to eliminate the effects of the researcher and the research process on respondents?

6. Is the presentation of results innovative, interesting, unambiguous and free from distortion?

7. Are the conclusions and recommendations plausible and convincing?

Conclusion15

This unit has considered the stages involved in the research process and explored some key skills for effective research. What emerges is that there is no one way of doing research, and that iteration within the research process is necessary to render results, conclusions and individual learning more robust and meaningful. Moreover, the element of individual judgment is critical to research. Researchers may not always get it right, but if the reasons for their decisions are transparent and ethical, the researcher, their stakeholders and audience can usefully learn. Lastly, research results are never an end point or the definitive truth, merely a landmark on an intellectual pathway to deeper enlightenment.

Next stepsAfter completing this unit you may wish to study another OpenLearn Study Unit or find out more about this topic. Here are some suggestions:

Science, maths and technology

If you wish to study formally at The Open University, you may wish to explore the courses we offer in this curriculum area:

Engineering and technology

The Open University Postgraduate Programme in Development Management

or find out about studying and developing your skills with The Open University.

OU study explained

Skills for study

or you might like to:

Post a message to the unit forum, to share your thoughts about the unit or talk to other OpenLearners

Review or add to your Learning Journal

Rate this unit

Referencesde Janvry, A., Sadoulet, E. and Thorbecke, E. (1993) ‘State, market and civil organizations: new theories, new practices, and their implications for rural development’, World Development 21, 4: 565-689.

Mayers Uzzi, B. (1997) Social Structure and Competition in Interfirm Networks: The Paradox of Embeddedness, Administrative Science Quarterly, 42 March pp35-67.

16

Roe, E. (1994). ‘Narrative Policy Analysis: Theory and Practice’, Durham and London, Duke University Press.

Thomas, A, Chataway, J. And Wuyts, M. (1998) ‘Finding out Fast. Investigative skills for policy and development.Open University Press, Milton Keynes.

Notes1. Although these terms are often used interchangeably, it is worth noting that sometimes a distinction is made between them. For example O’Riordan (1995) uses ‘multidisciplinary’ to refer to a discipline which draws on two or more others in a fairly fixed relationship. He uses ‘interdisciplinary’ to refer to a research approach which is committed to bringing appropriate disciplinary perspectives together in particular cases, without having to specify which ones in advance of engaging with a given research problem.

17