Realist Synthesis 101

Realist Synthesis 101

Dr Andrew Booth

Illustrating a Realist Review approach

• For Which Schools Do School Performance League Tables Work?

• How Do School Performance League Tables Work?

• Do they work in a similar way to Hospital Performance League Tables?

• What about Individual Surgeon Performance League Tables?

Developing Programme Theory • Programme Theory #1: Proponents of performance league tables believe their

publication stimulates competition, • Programme Theory #2: and that, as each provider adopts “best practice,” the

quality of services will improve.6 • Supporting Data: Some published evidence seems to support this. New York

probably has the best established system for providing the public with information… Soon after publication of performance league tables based on providers, the risk adjusted mortality for bypass surgery declined, leading some to conclude that this was a direct result.7

• Counter Argument: One alternative explanation being that, once providers know their data will be used for comparative purposes, they may resort to “creative reporting.” 8 9

• Programme Theory #3: Performance league tables may also improve patients' choice, and proponents argue that this is necessary for an efficient market economy by encouraging consumers to seek out high ranking providers.9

http://www.bmj.com/content/324/7329/95?variant=full-text#ref-6






Developing Propositional Statements (IF-THENs)

• IF Performance Tables are published THEN:– Staff at Well-performing schools (Context1) will change their

reasoning/actions because (Mechanisms1) (e.g. Maintain competitive advantage)…..with the result (Outcome1) that……

– Staff at Average-performing schools (Context2) will change their reasoning/actions because (Mechanisms2) (e.g. Identification of good practice)…..with the result (Outcome1) that…….

– Staff at Poor-performing schools (Context3) will change their reasoning/actions because (Mechanisms3) (e.g. Shame)……with the result (Outcome1) that……..

• CMO configurations where Mechanisms = changes that occur in people's reasoning and actions

Realist synthesis

Programme intervention

Context

Altered contextContext triggers

and Interacts with

Mechanisms Mechanisms

Outcomes (intended and unintended)

Steps for conducting a realist synthesis

1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-

regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what

circumstances and at what point in time (mid-range theory)

Formulating the review question

Formulating the review question

• Realist questions aim to identify diverse literature on the intervention of interest, in order to evaluate what works for whom in what circumstances

• They are relatively open ended, aiming to capture a broad spectrum of relevant primary research in the first instance

• They aim to identify the relationships between Intervention, context, mechanisms and outcomes

Our realist questions

(1) What approaches to community engagement are most effective in promoting peer support, to which people and in what circumstances?

• (2) How does community-based peer support impact on understanding of existing health information and use of health information and health services to improve health and reduce health inequalities?

Preliminary theory for COPES

• Community engagement aiming to promote peer support• increases health literacy• improves health • reduces health inequalities

Assumptions• Communities initiate and foster peer support• The process of providing community-based peer support

influences the ability to develop health literacy

• Health literacy creates a sense of empowerment, which is associated with a reduction in health inequalities

• Community-based peer support raises awareness of issues with literacy and empowerment

Community Peer support Health literacy

Scoping the literature

• For the initial scoping, do a comprehensive search to ‘map the territory’

• Abstract sift: Identify and categorise abstracts by programme theory, type of intervention

Scoping the literature for key concepts• Community engagement: Very small number of articles documenting ‘working

collaboratively with and through’ community engagement for peer support

• There were variations in community-based peer support in terms of • Amount of ‘emotional, appraisal and informational

assistance’ • Degree of ‘created social network member’• Degree of ‘experiential knowledge of a specific behaviour or

stressor’• Definitions of ‘Similar characteristics’

• The term health literacy was not used in peer support articles• so included all articles on health education/health

promotion relating to ‘Understanding and use of health information and health services’

• Reduction of health inequalities – or any explicit discussion of health inequalities – was absent

Identifying sources of information in real life settings

Define concepts

Compare definitions

Identify what works

Network with participants

Scope the literature

Explore gaps

Test the emerging theory

Propose a model

Research knowledge Community knowledge

PractitionersCommunity organizations Peer supportersExpert patients

Health librarianQualitative researchersSystematic reviewersCBPR researchers


1. Formulate review question2. Scope the primary sources of information3. Decide on the scope of the review4. Map the programme theory(s)5. Identify programme components6. Assess effectiveness of components, considering context and

mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-

regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in

what circumstances and at what point in time (mid-range theory)

Defining key concepts: community engagement and peer support

• Community engagement is ‘the process of working collaboratively with and through groups of people affiliated by geographic proximity, special interest, or similar situations to address issues affecting the well-being of those people…It often involves partnerships and coalitions that help mobilize resources and influence systems, change relationships among partners, and serve as catalysts for changing policies, programs, and practices (CDC, 1997)

• Peer support within a healthcare context is “...the provision of emotional, appraisal and informational assistance by a created social network member who possesses experiential knowledge of a specific behaviour or stressor and similar characteristics as the target population.” (Dennis, 2003). Emotional support includes expressions of caring, empathy, and encouragement, and is seen to enhance self-esteem. Appraisal support involves encouraging persistence and optimism for resolving problems, affirmation of a peer’s feelings, and reassurance that frustrations can be handled. Informational support is providing advice, suggestions, alternative actions, feedback and information relevant to the issue with which the peer is dealing.

Defining key concepts: health literacy

• Health literacy – ‘the personal, cognitive and social skills which determine the ability of individuals to gain access to, understand, and use information to promote and maintain good health. These include such outcomes as improved knowledge and understanding of health determinants, and changed attitudes and motivations in relation to health behaviour, as well as improved self-efficacy in relation to defined tasks. Typically these are outcomes related to health education activities…’ (Nutbeam, 2001)

• From a number of definitions for health literacy – we chose one that related to the notion of empowerment because it was relevant to health inequalities

Improved Health Literacy

Tailored information, communication, education

Improved health outcomes,healthy choices

and opportunities

Skills in negotiation and

self management

Prior understanding of individual capacity - reading fluency, numeracy,

existing knowledge

Developedknowledge

and capability

Skills in social organization

and advocacy

Participation in changing social

norms and servicepractices

Engagement insocial

action/advocacy for health

Figure 3: Developing interactive and critical health literacy skills

Changed healthbehaviours and

practices

(Nutbeam, 2009)

Health literacy model proposed by Sorensen et al (2012)

• A year into our review, a conceptual analysis of the definitions was conducted, and a unifying model was produced (Sorensen et al, 2011). This model focuses on the capabilities of accessing, understanding, appraising and applying health information.

World

Decide on the scope of the review

• Things to consider:– Amount of literature: Comprehensive versus sampling?– Diversity of literature: Consensus on programme theory?

Variations on theory by what, who, when, setting– Access to other sources of information– Interests of those who commissioned the review

• What was the original problem that needs research evidence?• International, national, regional, local?• What will the findings be used for?• Who is the audience? (this is sometimes wider than just

commissioners)– Time and money– Experience

MeTeam

World

Setting the parametres via sampling• Programme theory was not used for sampling because

– Theory was only explicitly cited in 28 instances– Studies combined elements from the Heath Belief Model,

Theory of Planned Behaviour, social cognitive and social learning theories, the social support literature, social comparison theory, social network approaches, and empowerment models

– Studies used both individual behaviour change and community-based theories, including community organization, diffusion of innovation, and social marketing

– Domains for all the individual behaviour change theories are included in the Capabilities Opportunity Motivation Behaviour theory (COM-B, Michie et al 2011)

• Pragmatic sampling on location: the most common United Kingdom health conditions in the peer support literature were reviewed first

Mapping literature topics to UK settings

TOPIC Advisory Network

Breastfeeding Sheffield Well Being Consortium; Sheffield Breastfeeding studyDarnall WellbeingAGE UK; Stroke AssociationIrish Gypsy & Travellers Movement in Britain; Leicester Health AmbassadorsCentre for HIV & Sexual Health; Parent to Parent, Shout; Sheena Amos Trust; Expert Patients ProgrammeSheffield Community Chronic Pain; Manor & Castle Development TrustMECOPP Minority Ethnic Carers of People Project; Autism PlusSharrow ShipShape; Sheffield PCTMIND; MencapSheffield City Council SOAR (Southey Owlerton Area Regeneration)ZEST Community Development Trust

Diabetes

Healthy Living Older People

HIV/AIDS

Health Trainers

Nutrition

Smoking

Summary of searching, sifting and sampling

Identify the programme components• Programmes categorised by Stage

– Recruitment of peers– Training and supporting peers– Implementing peer support - what was

delivered– Outcomes

• Within each Stage, components were identified in the literature and through workshops with community members

• Components put together in CMO configurations by the topic leads

Mapping programme theories• The theories and assumptions were documented

using Context Mechanism Outcome (CMO) configurations for each health topic

Type of finding Ref ID

What works well/not well CONTEXT MECHANISM OUTCOME

authors' thoughts

2668 Retaining peer supportersAmongst those consented to be peer supporters attrition rates were low 87% of those nominated agreed to become peer supporters and 82% completed the trial

PS felt pride in being nominated by their peers. It was novel to be taken out of school for training days and to miss lessons to attend follow up sessions and they may have been committed to reducing smoking amongst their peers, also incentive offered to those who completed trial

The target of 15% of the cohort were trained and delivered the intervention to the end of the trial period

study protocol

6895 & LC001

Retaining PS £10 voucher and certificate of achievement offered to PS who completed the trial

PS responded to this incentive and remained committed to trial

Low attrition and majority of PS completed trial

study protocol

6895 High ratio trainer : student

maximum of 1:15 trainers to students

Group work easily facilitated and trainers developed good relationships with peer supporters. Role play activities were successful

Students motivated throughout training and follow up visits. Role play helped students anticipate the types of different reaction they might receive when attempting to disseminate information

Dealing with diverse theories• Some topics used a single theory, for example Diffusion of

Innovation• Other topics did not explicitly state how theory informed

design, but contained implicit assumptions about what would work

• Concepts from both stated/implicit theories identified via CMOs

• Dimensions of concepts for one topic used to question concepts in another topic area

• For example, ‘recognising someone as a peer’ was an important mechanism. We asked ‘what does recognising someone as a peer mean across different health topics?

• The dimensions we discovered included age, ethnicity, experience with the problem or condition

• We also used Advisory Network as a source of propositional statements


1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and

mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases

(demi-regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for

whom, in what circumstances and at what point in time (mid-range theory)

Iterative

Turning CMOs into propositional statements: analysis of the literature

Overarching propositional statement

Supporting data by topicSmoking cessation young people

Healthy living in older people

Smoking in ethnic minorities

IF target community recognise PS as their peer THEN PS is effective

I 5/6:Target community must recognise the peer supporters as their peers.(Propositional statement)

I12:When age is only common characteristic, and differences like ethnicity exist (meaning potentially different needs and value systems), the peers may have a negative experience. (Propositional statement)

D88Smoking cessation service is offered by health professionals, who created a formal and impersonal atmosphere, but smokers feel a sense of distrust towards providers(Mechanism)

Searching for related theories

• CMOs and propositional statements (IF-THENs) considered in terms of higher order concepts, for example

Overarching propositional statements Concepts

IF the target community recognise the PS as their peer THEN peer support is effective

Social learning

Relational learning

Participatory parity

Trust

IF there is a positive relationship between the PS and the peers (trust and a sense that the person is believable, credible) THEN peer support is effective

Mid-range theory• ‘Theories that lie between the minor but necessary working hypotheses

that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change.’ (Merton, 1967: 39)

Mid-range theory

• Mid-range theories relate empirical data to higher order constructs, and are ‘sufficiently abstract to deal with different spheres of social behaviour and social structure, so that they transcend sheer description or empirical generalisation’ (Merton, 1967: 68).

• Middle range theory involves abstraction, of course, but…close enough to observed data to be incorporated in propositions that permit empirical testing.”

References

Dennis, C.-L. (2003). Peer support within a health care context: a concept analysis. International journal of nursing studies, 40(3), 321–32. Kincheloe, J. and McLaren. Rethinking theory and research. (2005) In NK Denzin & YS Lincoln (eds) Sage Handbook of Qualitative Research. Merton R. On Theoretical Sociology. Five Essays, Old and New. New York: The Free Press, 1967Nutbeam, D. (2008). The evolving concept of health literacy. Social science & medicine (1982), 67(12), 2072–8. Pawson, R. (2013) the science of evaluation: A realist manifesto. London: Sage.Sørensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z., & Brand, H. (2012). Health literacy and public health: a systematic review and integration of definitions and models. BMC public health, 12(1), 80. Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: realist syntheses. BMC medicine, 11(1), 21. doi:10.1186/1741-7015-11-21

Acknowledgements

• Funded by the National Institute of Health Research Public Health programme PHR - 09/3008/04: Community-based peer support: A realist synthesis and model for promoting health literacy (COPES)

Realist Synthesis 101

Documents

Transcript of Realist Synthesis 101