A Critical Review of Stake’s Multiple Case Study Analysis ...€¦ · Review of Stake’s...

Running Head: REVIEW OF STAKE’S MULTIPLE CASE STUDY ANALYSIS

A Critical Review of Stake’s Multiple Case Study Analysis with Respect to Integration of the Theoretical Mode of Cultural Competence

Stacey Moreau: 598210

Sofia Saleem-Ahmad: 2835105

University of Ottawa

Review of Stake’s Multiple Case Study Analysis

1

A Critical Review of Stake’s Multiple Case Study Analysis with Respect to Integration of the Theoretical Mode of Cultural Competence

Abstract

In this session, Moreau and Saleem-Ahmad will collaborate on a critical analysis of Robert Stake’s book, entitled Multiple Case Study Analysis released in 2006. As part of this critical book review, emphasis will be made on how effectively and readily cultural competence is integrated into multiple case study analysis, as proposed by Stake in his book. For the purposes of this session, the definition of cultural competence as offered by Thomson-Robinson, Hopson and SenGupta (2004) will be used. They have suggested that cultural competence is a:

systematic, responsive enquiry that is actively cognizant, understanding and appreciative of the cultural context in which the evaluation takes place; that frames and articulates the epistemology of the evaluative endeavour; that employs culturally and contextually appropriate methodology; and that uses stakeholder-generated, interpretive means to arrive at the results and further use of the findings. (p. 13)

The format of this critical analysis paper will encompass three elements. First,

following a brief summary of Stake’s book, the authors will independently review the book, focusing on the issue of cultural competence in relation to the case study reports described in the book. Secondly, following this independent review process, they will exchange ideas and converge on a dialogue about their analysis of the book. The expectation is that a dialogical approach will allow the authors to challenge the hallmark of evaluative responsiveness as framed by Stake’s (1975) work and highlight the contention that evaluator’s are encouraged to recognize, appreciate, and incorporate culturally related contextual factors as this relates to their practice (Thompson-Robinson, Hopson & SenGupta, 2004). Finally, the review will conclude with an assessment on the level of inclusion of essential elements in the case studies as they relate to cultural competence.


2

Summary of Multiple Case Study Analysis

The popularity of case studies as research tools has developed only in recent

decades. One of the areas in which case studies have been gaining popularity is education

and in particular educational evaluation. Rather than using large samples and following a

rigid protocol to examine a limited number of variables, case study methods involve an

in-depth, longitudinal examination of a single instance or event: a case. They provide a

systematic way of looking at events, collecting data, analyzing information, and

reporting the results. As a result the researcher may gain a sharpened understanding of

why the results emerged the way they did, and what might become important to look at

more extensively in future research. Case studies lend themselves to both generating and

testing hypotheses (Flyvbjerg, 2006).

Examining situational complexity is a vital part of social and behavioural science

research. Robert Stake’s Multiple Case Study Analysis provides an effective process for

studying multiple cases - such as sets of teachers, staff development sessions, or clinics

operating in different locations - within one complex program. The process also can be

used to investigate broadly occurring phenomena without programmatic links, such as

leadership or sibling rivalry. Readers learn to design, analyze, and report studies “that

balance common issues across the group of cases with the unique features and context of

each case.” Furthermore, “three actual case reports from a transnational early childhood

program illustrate the author's approach, and helpful reproducible worksheets facilitate

multicase recording and analysis” (Stake, 2006).

Stake’s (2006) book begins by discussing the classical single case study, followed

by an introduction to the multicase study. Cross-case analysis is explained and assertions


3

are verified. This book is very practical in that, as mentioned earlier, it offers various

worksheets as well as a comprehensive guide to engaging in a multicase study project.

The three case studies presented in the book all focus on the Step-by-Step program. Step

by Step is a “comprehensive child-centered approach that serves families with children

from birth through age 10” (Stake, 2006, p. 92). It provides in-service teacher training in

15 countries in Central and Eastern Europe. The author does not participate directly in the

research process, but acts more in a mentoring role. Stake’s book is clearly written

although knowledge of evaluation and case study methodology is required. As he

mentions, “… the reader will find little trace of triangulation to assure that what was told

was what was there. Like most writers of all kinds these report writers are weak in

describing their commitment to providing verification for what they write” (Stake, 2006,

p.110).

Multiple Case Study Analysis offers a methodology that could be very useful in

certain, specific situations. Stake clarifies many issues and offers his own critiques of this

methodology. As Howard Becker summarizes:

Researchers in education and social science have worried for decades over how to study multiple cases of the same phenomena and come up with viable conclusions. Robert Stake’s careful analysis lays out the problems, the pitfalls, and the dangers of such an enterprise, and provides masterful, amply illustrated, easily understood, and reproducible solutions. (In Stake, 2006).


4


By S. Saleem-Ahmad

Stake’s book entitled Multiple Case Study Analysis, attempts to provide the main

problems and dangers of utilizing multiple cases by addressing the political, historical,

cultural, and educational context of the issues presented. Relevant research questions,

according to Stake, have the propensity to identify main themes that are in need of

amelioration or discussion in relation to improving the quality of educational programs

offered. If there is one argument that is to be made on how successfully Stake is able to

provide solutions through multiple case studies, his expertise on this subject speaks for

itself. At present, Stake is a specialist in the evaluation of educational programs and case

study methods. The challenge of deciphering generalizations using case studies is a

daunting task. However, Stake, who has long been known as an expert of using multicase

studies to fill the gap between qualitative research and evaluation literature, is quite

honest and perceptive in his accounts. Thus, it may be contended that his extensive

experience has lent itself to producing high quality and selective case studies as

exemplars for use by practitioners and evaluators alike through his latest book.

Furthermore, in attempting to draw conclusions and enhance our comprehension

of educational systems and their practices, Stake emphasizes, “case study issues reflect

complex, situation, problematic relationships. They pull attention both to ordinary

experience and also to the disciplines of knowledge (e.g., sociology, economics, ethics,

literary criticism)” (p. 10).


5

By highlighting three case studies (in Ukraine, Slovakia, and Romania) around

the Step by Step early childhood educational program, Stake (2006) chooses to share

multicase studies that were designed to analyze not only the teaching methods employed

but also the initiatives that were undertaken to make education more inclusive (e.g. the

case of Roma people in Slovakia who are considered to be an oppressed group in

Europe), provide anti-biased education, link with educational institutions that provide

teacher training, and parent education programs. Stake highlights the fact that these cases

are developed to address specific research questions of contextual relevance to each

situation. Of particular interest is the emphasis Stake places on identifying the cultural

context or cultural competence demonstrated by the researchers in each case. For our

purposes of assessing to what extent cultural competence is addressed in Stake’s

multicase study analysis, a definition provided by Thomson-Robinson, Hopson and

SenGupta (2004), written below, will be used to guide this endeavour:

Cultural competence in evaluation can be broadly defined as a systematic, responsive enquiry that is actively cognizant, understanding and appreciative of the cultural context in which the evaluation takes place; that frames and articulates the epistemology of the evaluative endeavour; that employs culturally and contextually appropriate methodology; and that uses stakeholder-generated, interpretive means to arrive at the results and further use of the findings. (p. 13) Stake (2006) comments that most case studies seek to formulate generalizations

rather than identify specific outcomes relevant to the issue. By making this contention,

Stake problematizes the tensions that exist between the role of the qualitative researcher

as one who is seeking multiple truths, based on multiple contexts and diverse perspective

vs. a quantitative researcher who is seeking one truth. Stake’s perspective on this issue

fluctuates between acknowledging the notion that “the study of human activity loses too


6

much if it reports primarily what is common among the several, and universal across the

many” (p. 88) and identifying that “there will continue to be expectations that multicase

research should provide formal generalizations for guiding policy and collective practice”

(p. 89). In this regard, generalizations, that dictate and debate issues surrounding social

policy, may not always be culturally sensitive to the specific values, views and needs of

all stakeholders involved in the process. In his book, Stake (2006) claims: “…it seems

more likely to me that the act of generalizing is deeply set in the human repertoire and

that it will continue to operate largely without protocol” (p. 89).

Hence, while the initial intention of research may be to emphasize inclusion and

encourage participation from multiple stakeholder perspectives, inevitably consensus-

building is supported, where the voice of many is reduced to the voice of one unit for

outcomes to lead to viable and practical results with minimal detraction. This “single”

and cohesive voice, while indirectly inclusive of many voices, may, in all likelihood, be

the voice of only those who are most dominant and powerful amongst the stakeholders

involved. Thus, it may be argued that while research does allow for the participation of

diverse individuals with diverse backgrounds, not every opinion will be taken into

account when addressing results-oriented programs.

Stake (2006) does however assert that by keeping contexts in mind, certain

outcomes may emerge that were unaccounted for when initially designing the study and

this in turn may “distract” researchers from the main goals of their study. Stake suggests

that these unexpected outcomes should be considered on a case-by-case basis and should

not become the foundation upon which the research study is designed. Yet, it is important

for researchers to be cognizant of the different perspectives that may play an influential


7

role for drawing up final conclusions. At times, it is somewhat challenging to determine

the extent to which cultural competence influences the research design process. On the

one hand, Stake emphasizes the need for qualitative researchers to be cautious and aware

of the contexts that may inform their background, but on the other hand, Stake states,

“doing case studies does not require priority on diversity of issues and contexts, but most

qualitative researchers carefully study such diversity” (p. 13). His conflicting views leave

the reader with the sense that even Stake does not fully comprehend to what extent and in

what manner context, such as the cultural and political framework, has the ability to

impact the potential outcome of research. In contrast to his opinion that diversity of

issues and contexts should not be regarded as a priority, Stake (2006) later argues, “the

case’s activities are expected to be influenced by contexts, so contexts need to be studied

and described, whether or not evidence of influence is found” (p. 27).

Another aspect where cultural context or competence may emerge is in the

research design or methodologies employed. In this, we may find agreement with Stake

(2006) when he emphasizes the importance of triangulation in involving individuals in

the decision-making process who may be directly affected by the specific program or

activity in question. This method may be regarded as a culturally competent approach to

identifying any problems or issues in the findings of a research by encouraging

participants to partake in the analysis of the results obtained.

With reference to the Step by Step program described in Stake’s book, the main

research goals of these programs, which integrate international practices and standards, is

to promote the development of democracy by using educational methodologies and to

enhance the inclusion of individuals who are marginalized (minorities, at-risk youth,


8

impoverished individuals, children with disabilities etc.) in a manner that values

diversity. By centering activities on children’s experiences and input, greater value is

placed on the cultural context or backgrounds of these children and the effect their

experiences may have on the improvement of educational programs. Of particular interest

is Stake’s example of the Ukraine case study, which identifies how children with

disabilities provide ideas for best practices on how to effectively integrate them in a

regular classroom. Another example of a case study used is that of Slovakia and their

implementation of the Step by Step program to develop inclusionary policies directed at

marginalized populations. This case study highlights how the program has successfully

enhanced Roma children’s sense of belonging by simply incorporating more Roma

teaching assistants in the classroom and by re-working assessment so that it is less

culturally biased.

Within the case study design and protocol of the Step by Step education program

in Slovakia, Romania and Ukraine, Stake (2006) has effectively emphasized the

importance of understanding the context for individuals and community members and has

identified this as a necessary precursor for analyzing each case study. By utilizing these

case study examples, Stake indirectly suggests that those who were involved with the

research in the multi-case study were indeed culturally competent. Thus, his case studies

attempt to illuminate and broaden our comprehension of complex cultural systems and

social processes. Stake would undoubtedly agree that qualitative methodology, such as

the ones employed in case study analysis, must be designed, developed, and carried out in

a way that productive and useable results are obtained, whilst at the same time being

aware of the context of a researcher’s time availability, funding and accessibility.


9

Moreau’s critical review of Stake’s Multiple Case Study Analysis

By S. Moreau

Background on Cultural Competence and Underlying Assumptions

I have chosen to present my critique of Stake’s (2006) work in three separate

parts. In the beginning, I will attempt to unveil my assumptions regarding cultural

competence and Stake’s positioning within this process. By doing this I hope to make my

postulations explicit, and thus, be better able to critically analyze the methodology of

multiple case study. Following this, after having read Stake’s Multiple Case Study

Analysis, I hope to challenge my assumptions and discuss the merits and faults of the

multiple case study, in specific regards to the issue of cultural competence. Finally, I will

comment on Saleem-Ahmad’s Critical Review and offer some excerpts from an interview

with Robert Stake.

To begin, let us first consider the concept of cultural competence as mentioned in

Saleem-Ahmad’s paper and offered by Thompson-Robinson et al. (2004). I particularly

appreciate how an appropriate framework and methodology is emphasized in this

definition. It also considers the means through which evaluation findings are arrived at

and used by the evaluator. From this, my understanding of cultural competence is that it

is both a process and a goal. Cultural competence should be an over-arching value that

frames all other aspects of an evaluation. It should not simply be considered as an add-on.

Context-specific flexibility and a capacity for understanding and appreciation of various

cultures and values is a tool that evaluators should use in all evaluation situations. The

American Evaluation Association’s Guiding Principles highlight “the need to proactively

attend to diversity issues as a necessary prerequisite for ethical practice” (In Thompson-


10

Robinson et al., 2004) (italics added), and I would add, validity. This would suggest that

cultural competence is not optional and must be considered at all times.

Cultural competence is a challenging idea. The term culture refers to cognitive,

affective, and behavioral patterns that human groups share, that is, the rules and norms by

which people live. Both a process and a goal, cultural competence arises through a

deliberate and continuous process of self and organizational introspection (Cross, Barzon,

Dennis & Isaacs, 1989; Lavizzo-Mourey and Mackenzie, 1996). To a certain extent, one

could argue that Stake (1967), with his notion of responsive evaluation, was one of the

fathers of culturally competent evaluation. Even though Stake would later discard the

framework he proposed in the countenance paper in putting forth the notion of

“responsive evaluation” (Scriven, 1991; Stake, 1991), the countenance paper was a

conceptual breakthrough in thinking about program evaluation (Thompson-Robinson et

al., 2004). “Even though there were admitted limitations with Stake’s countenance

approach, it did show that a rich description of the program and the context in which it

functioned were critical to achieving more than superficial understanding.” (Thompson-

Robinson et al., p. 24). Unfortunately, this potential for greater understanding was not

realized at the time Stake proposed the countenance approach. Ironically, there has been

little evidence of cultural responsiveness in the work of those who used the countenance

approach (Thompson-Robinson et al., p.24).

In the late 1970s, Stake (1978) moved beyond the dissatisfaction of his

countenance approach and fostered responsive evaluations. Responsive evaluation

prioritizes interviews and observations to achieve stakeholders’ understanding of the

evaluand and it’s perceived worth from multiple stakeholders’ perspectives. Then, Stake


11

stated, “human observers are among our best instruments [and] the evaluator should not

rely only on his/ her own powers of observation, judgment, and responding [but rather

enlist] a platoon of students, teachers, and community leaders (Stake, 1975). Anne Marie

Madison (1992) and others (Chevalier, Roark-Calnek, and Strahan, 1982; Wilcox, 1984)

have implied that a responsive evaluation approach is one of the few that accepted

culturally diverse factors as being central to “good” evaluation.” (Thompson-Robinson et

al., p. 26).

A participatory approach, including significant involvement in the evaluation by

the program participants, offers a viable means of addressing the complexity of cultural

competence.

Participatory evaluations recognize multiple perspectives of knowledge and allow

participants involved in the evaluation process to create shared meaning. In participatory

approaches, evaluators seek firsthand experience with program activities and settings and,

by facilitating the development of better-informed program staff and participants, serve

an essential function for improvement (Cousins & Whitmore, 1998; King, 1998; Weaver

& Cousins, 2001).

Although I would assume that Stake is an advocate of cultural competence,

“cultivating self-as-instrument and developing intercultural and multicultural

competencies is a lifelong process and not a fixed state of being. Because culture is

dynamic and ever-changing, yesterday’s culturally competent practitioner could become

tomorrow’s incompetent” (Thompson-Robinson et al., p. 99). Also, Thompson-Robinson

et al. (2004) warn against multisite study situations: “In multisite study situations, a

vitally important aspect of culturally competent evaluation- that of responsiveness to the


12

context- becomes difficult to accomplish” (Thompson-Robinson, Hopson and SenGupta,

p. 15). With these assumptions in mind, I will now review Stake’s work and analyze the

multiple case study analysis with regards to cultural competence.

A Critique of Stake’s Multiple Case Study Analysis

Excellence in cultivating cultural competence demands that we embrace a twofold agenda: inside-out (self-as-instrument and self-in-context work) and outside-in (expanding and enriching one’s diversity-relevant knowledge and skills repertoire and one’s professional evaluator’s toolkit). (Thompson-Robinson, Hopson and SenGupta, p. 99)

In critiquing Stake’s Multiple Case Study Analysis, I will assess the level of inclusion of

essential elements in the case studies as they relate to cultural competence from the

perspective of a graduate student. Guided by Bamberger’s (1999) five commonly-

accepted ethical principles, I have framed them in the context of multicultural evaluation

elements which include: (1) evidence of valuing diversity, (2) demonstration of the

capacity for cultural self-assessment, (3) awareness of the “dynamics” and tensions

inherent when cultures interact in the evaluation process, (4) institutionalized cultural

knowledge, and (5) reflecting on the comprehension of diversity and its importance

within and between cultures. Cultural competence is a theory that should guide how

evaluators conduct their practice, in order for the program to be considered culturally

competent it may be useful to determine if all five elements are present in the attitudes,

structures and policies reviewed in Multiple Case Study Analysis.


13

Evidence of Valuing Diversity

A culturally competent enquiry should encourage various cultures and a range of

viewpoints. With great pride it seems, Stake (2006) states, “Within a multicase project,

the study of individual cases will often not be organized around the multicase research

question. To some extent, sometimes entirely, each case gets organized and studied

separately around research questions of its own” (p. 9). Stake (2006) acknowledges that

this design may” [risk] paying too little attention to what binds the cases together” (p.9).

Yet, even more than that, comparing apples to oranges could prove fruitless and

misleading. Also, the importance of various stakeholders’ views may get diluted in the

process.

In this book, Stake offers triangulation as a form of validation. Triangulation

determines whether the new views are consistent with what is already well known about

the case. What about challenging the status quo? It does not mean that because it has

always been, that it is right. If the researchers performing triangulation lack cultural

competence, some outdated, prejudiced views may continue to emerge through various

multiple case studies. On the other hand, the use of triangulation in multiple case study

analysis, in which researchers engage in a critical review of what is being said, could also

provide an opportunity for various stakeholders to be heard.

Demonstration of the Capacity for Cultural Self-assessment

Being aware of one’s biases are important in allowing oneself to “see” through

the lenses of others. One of the unusual tasks of multicase study of social science

researchers is their involvement in the selection of cases. A point of departure in this type

of study is recognizing what concept or idea binds the cases together (Stake, 2006).


14

Because each person holds certain assumptions, the researcher’s “lenses” will influence

the case selection and collection of ideas, which could, in turn, reify the practitioner’s

beliefs.

Also, when discussing criteria for case selection, Stake (2006) favours “cases

from which we can learn about their activity and situation. This may mean taking the

ones that are most accessible, the ones we can spend the most time with” (p.25). Stake

(2006) concedes “there may be a trade-off between potential for learning and

representativeness” (p.25). By placing emphasis on case studies that cater to the majority,

an obvious area of concern may be the lack of representation of marginalized individuals

in society.

As well, in his book, Stake (2006) suggests that most data will come from the

case studied, but researchers may also gather other data from outside to supplement their

findings. From a culturally competent view, this admission is worrying. If information is

already known, whose perspective is represented? Who chooses what information to

select from as there may be multiple biases? Firstly, on the part of the researcher, who

subjectively decides from whom to gather extra information and secondly, from the

outside, who chooses what information to share. This is particularly problematic because

although some non-evaluation stakeholders may possess different views on the same

issue, it is not guaranteed that all of their voices will be heard.

Awareness of the “Dynamics” and Tensions Inherent When Cultures Interact in the

Evaluation Process

Sometimes, when knowledge is being synthesized and disseminated, it is not always

possible to find complete agreement on how this knowledge may be best utilized. Stake


15

(2006) offers a starting point for this discussion by assuming that “what is not agreed

upon is unimportant, [and] what is agreed upon is confirmed” (p. 37). Yet, it seems to

me, that it is precisely these spaces of disagreement that need to be studied further to

offer a richer portrait of the relevance of these case studies in relation to each other and to

the individuals involved. The fact that individuals may not necessarily share the same

viewpoints on a given issue further complicates a researcher’s ability to be culturally

competent. One would hope that ideally, various cultural considerations are taken into

account when determining the importance of views, but as stated previously, this may not

always be true.

Institutionalized Cultural Knowledge

Institutionalized cultural knowledge refers to a commonly accepted understanding

of the norms, practices, and values of a certain culture. What is important is that although

culture is a living element that is constantly changing, there are certain considerations

that researchers should acknowledge as normative in various evaluation situations.

However, researchers should also be aware that not all individuals within a culture fit into

this normative pattern. For example, assuming that all Canadians like hockey, serves to

ostracize those who care little for this sport.

Furthermore, Stake (2006) proposes that in order to study a case, one must

“carefully examine its functioning and activities, but the first objective of a case study is

to understand the case” (p. 2) and thus, cultural differences. In order to arrive at better

understanding there is an inherent assumption that the researcher and evaluator must be

implicitly knowledgeable about various cultures. In contrast, Stake (2006) writes, “the

case activities are expected to be influenced by contexts, so contexts need to be studied


16

and described, whether or not evidence of influence is found” (p. 27). Upon reading this,

it appears that the importance of context is minimized. Context is part of culture, and it

should be understood that culture, and thus context, always holds an influential role. It is

the role of the researchers to consider these influences when analysing the cases. Stake

(2006) demonstrates evidence of cultural competence when he writes, “…the way the

interview[ee] sees the case operating is essential knowledge, and the researcher needs to

find out a little about the interview[ee] to understand his or her interpretations” (p. 31).

Stake returns the emphasis to context as an important element of research.

Reflection on the Comprehension of Diversity and its Importance Within and Between

Cultures

Although Stake appears to “talk the talk”, I’m not sure if there is follow-through

on the issue of the importance of the comprehension of diversity within and between

cultures. Stake agrees that, for this purpose, the most meaningful data-gathering methods

are often observational- both direct observation and learning from the observation of

others. “The latter, indirect method is necessary for activity at which the researcher is not

present; the researcher needs to ask someone who was there, and to find records kept of

what happened and artifacts that suggest it” (Stake, 2006, p.4). What worries me in this

course of action is that, the more the data is “polluted” by third-party conveyance, the

less accurate and “true” the findings become. In Stake’s Multiple Case Study Analysis,

there are four layers of people to whom the original information must be communicated.

The chain of discourse flows from the individual to the “other” who was present when it

occurred. That “other” conveys his interpretation of what happened to the researcher. In

the final analysis, the researcher presents his documentation to the project director, who


17

merges findings together. Through this process, the possibility for distortion is great. Yet,

one must wonder, to whom is the power given in this process? Who decides what is

included and what is unnecessary “junk”?

Also, it appears that Stake (2006) contradicts himself in certain instances within

the book. At one point, he mentions, “…the power of the case study is in its attention to

the local situation, not in how it represents other cases in general” (Stake, 2006, p.8). Yet,

in the next paragraph, he says, “when the purpose of case study is to go beyond the case,

we call it ‘instrumental’ case study” (Stake, 2006, p.8). In multicase study, he affirms, the

cases will be primarily instrumental. With this in mind, it seems that the case studies will

be used as tools by the project director to derive generalizations concerning similar cases.

In a culturally competent manner, Stake (2006) highlights the qualitative

researcher’s interest in diversity of perception. Yet cultural sensitivity is lost in Stake’s

suggestion that the cross-case analysis should be entirely done by one person. Further, it

is suggested that the project director “expects to lose much of the particularity of each

Case, but wants to keep the most important experiential knowledge” (Stake, 2006, p.44).

Yet again, this statement begs the question of knowledge that is important to whom?

Consequently, Stake (2006) writes, “those she judges worth mentioning in the final report

are given title cards…” (p. 68). The researchers appear to hold a dominant position in

multiple case study analysis design, in that they, as experts, judge what is pertinent and

what is not.

When merging findings, “the remaining weak clusters and isolates are put aside”

(Stake, 2006, p.60), never to be mentioned again. As Stake (2006) cited earlier in his

book, sometimes the richness of things lies in its differences. Yet, it does not appear that


18

multiple case study analysis takes an active interest in addressing the reasons for these

differences.

From my reading, it appears that, although there are certain areas for

improvement, overall Stake’s Multiple Case Study Analysis offers a culturally competent

method of enquiry. The irony of it is, that this cultural competence occurs more by

accident, than on purpose. The chosen researchers appear to be very experienced and the

context (Ukraine, Slovakia, and Romania) promotes an obvious consideration of culture

within the framework.


19

Case Studies in Evaluation, but for what purpose?

A follow-up to Moreau’s “A critique of Stake’s (2006) Multiple Case Study Analysis

By S. Saleem-Ahmad

Moreau puts forth a compelling case in her critique of Stake’s Multiple Case

Study Analysis by highlighting the extent to which the case studies and the research

protocol proposed to carry out the studies, demonstrate culturally competent elements. It

is important to note that the main problem in deciphering to what extent cultural context

needs to be kept in mind, may depend on whether or not a real evaluation of the program

is occurring versus using case studies to simply understand the program from a non-

evaluative perspective, in the traditional sense. This is where differentiating the roles

between bonafide evaluations of programs and utilizing case studies to highlight aspects

that worked or did not work, becomes pivotal to this discussion. In going forward, this

paper attempts to identify only three areas that Moreau (2007) has proposed in her

critique or review of Stake’s Multiple Case Study Analysis. These areas include: 1) the

challenge in understanding cultural competence and its relevance in case studies, which

Moreau states should be regarded as “…an over-arching value which frames all other

aspects of an evaluation. It should not simply be considered as an add-on” (p. 1); 2)

identifying valid and reliable elements of multicase studies; 3) the role of cultural

capacity, as identified in case studies, in evaluation practice. Let us then begin by

problematizing the first of these three elements of discussion.


20

Is Anyone Ever Truly “Culturally-Competent”?

The above-mentioned title does not suggest that an appreciation of the values and

cultures of participants and even researchers is not important, particularly when analyzing

and deciphering why certain cases demonstrate greater success than others. It would be

irresponsible to carry out research without recognizing that individuals from Slovakia,

Romania, and Ukraine all possess distinct cultural traits and perspectives. These

viewpoints have the potential to guide their actions and responses in the research

conducted. However, the extent to which this cultural knowledge informs the final

outcomes of research may be debatable. Earlier in my original review of Stake’s book, I

stated that:

…generalizations, that dictate and debate issues surrounding social policy, may not always be culturally sensitive to the specific values, views and needs of all stakeholders involved in the process…Hence, while the initial intention of research may be to emphasize inclusion and encourage participation from multiple stakeholder perspectives, inevitably consensus-building is supported, where the voice of many is reduced to the voice of one unit for outcomes to lead to viable and practical results with minimal detraction. (p. 3)

Moreau also identifies this as a source of concern during the data-gathering process:

Although Stake appears to “talk the talk”, I’m not sure if there is follow-through on the issue of the importance of the comprehension of diversity within and between cultures. Stake agrees that, for this purpose, the most meaningful data-gathering methods are often observational- both direct observation and learning from the observation of others…what worries me… the more the data is “polluted” by third-party conveyance, the less accurate and “true” the findings become... [Thus] the chain of discourse flows from the individual to the “other” who was present when it occurred… Through this process, the possibility for distortion is great. Yet, one must wonder, to whom is the power given in this process? Who decides what is included and what is unnecessary “junk”? (p. 8)

Thus, power relations and the values which are identified as being of worth are strong

motivating factors in deciding what aspects of research will be emphasized over others.

This notion is supported by Hodgkinson (1991) who states, “values (meanings) can be


21

defined as concepts of the desirable with motivating force” (p. 101). A force that even

Stake (2006) would argue, albeit not in so many words, helps decipher the direction of

research, provides a set a research design, and guides the analysis of results based on

purpose. Accordingly, if the overall goal is to minimize inequities between the types of

education students receive, then greater focus should be made on how this overall goal

can be achieved, rather than addressing the specific concerns of individual students, as

they may relate to the case.

What is the Purpose of Multicase Studies and What Elements are Essential in Evaluating

the Validity and Reliability of Multicase Studies?

Another area of concern, highlighted by Moreau (2007) in her critique of Stake’s

Multiple Case Study Analysis, centers on the idea of multicase studies as being primarily

instrumentalist driven, rather than, stakeholder driven. This may be a result of the

multiple case study design, where multiple sources of evidence are needed (e.g. using

triangulation methods), to ensure construct validity using a research protocol. In brief,

literature on case studies identifies construct validity, internal validity, external validity,

and reliability as challenging. These components are all important in maintaining the

integrity and fidelity of case study research. The focus then, is on producing sound results

rather than meeting the needs of various stakeholders. In this regard, Stake (2006)

contends that a well-developed research protocol aids in establishing more consistent and

reliable results. This in turn, may help to ensure construct validity using single-case

studies. However, this approach may be much more challenging in multiple case study

approaches where emerging generalizations using multi-case studies can be used to draw

comparison between cases, thus, enhancing the external element of validity.


22

In the case of internal and external validity, research may be limited depending on

the number of sources available. As a result, in case studies, limitations on internal and

external validity can only be critiqued when only one source is available. Having said

this, Stake (2006) would agree that these limitations are mostly related to statistical

generalizations emerging from the results rather than analytical generalizations of

qualitative nature. Hence, it may be argued that the best way to provide for greater

reliability and to reduce the extent, to which these limitations may affect the overall

results, occurs by developing a sound research protocol that has considered these issues

using multiple sources of data in the case study. This process, as Stake has also claimed

in his book, allows for a broader comparison of data and final conclusions. Stake has

referred to this process as triangulation in relation to case study analysis.

Triangulation, as Stake (2006) asserts, ensures greater accuracy in being able to

identify alternative explanations emerging from case study analysis and this in turn

informs the basis of valid and reliable results in case studies. Moreau (2007) writes,

“triangulation determines whether the new views are consistent with what is already well

known about the case” (p.5). Moreau then brings up an interesting question about how

the method of triangulation hinders the ability to challenge the status quo. There are

dangers inherent in relying solely on one type of methodology, particularly if this

methodology gives rise to biased results based on the perspectives of culturally

“incompetent” researchers. Stake, in his multicase analysis, does not provide much

insight into the pitfalls of using one methodology over another, but he does argue that

triangulation is an integral component of case study protocols.


23

Emerging Capacity for Cultural Self-Assessment as Relevant to Evaluation Practices

Having stated this, one final element of discussion as brought up by Moreau

(2007), centres on demonstrating capacity for cultural self-assessment. While I agree with

Moreau’s view that each individual brings with him or her, their own personal

perspectives when deciphering, analysing, and disseminating knowledge from case

studies, it may also be necessary to choose the types of cases that may or may not be of

relevance to evaluation practice. The practice of evaluation is a very specific field with

specific goals. The field as a whole attempts to problematize and identify how the

particular needs of groups in various organizations or programs can be ameliorated and

improved to advance overall performance and results. In this regard, identifying which

cases will draw greater attention and will require greater focus in research, may only

serve to advance the field of evaluation by adding to the overall knowledge base of

evaluators.

Perhaps it may not be obvious to readers at first glance, but it appears, that the

Step by Step cases shared in Stake’s (2006) book were not set out as evaluation cases in

the traditional sense, but rather as cases from which evaluators could pull out key ideas to

further enhance their own practice and integration of cultural understanding. Thus, it may

be argued that the level of cultural competence exuded by researchers would depend on

how relevant this aspect is in deciphering and compiling overall results. For example, in

the Slovakian case study, researchers were able to identify that Roma children were

systematically discriminated against in school programs and that there was a need to

introduce a program (e.g. Step by Step) that would minimize their marginalization. If, for

whatever reason, researchers felt that analysing the level of success of the Step by Step


24

program and the impact it was having on the learning of Slovakian students was not

contingent upon the inclusion of all groups, then the marginalization of Roma children

would not have been an issue. Having said this, the plight of these children is a main

focus of this case study, as they were identified, early on, as a high needs group.

Further, the list of “characters” proposed at the start of the three cases, highlights

the extent to which diverse viewpoints were extrapolated in guiding final conclusions.

While this list may appear extensive and in support of a culturally competent research

protocol, Moreau (2007) states that this participatory approach “…recognize[s] multiple

perspectives of knowledge and allow[s] participants involved in the evaluation process to

create shared meaning” (p. 3). Moreau (2007) also highlights:

When discussing criteria for case selection, Stake (2006) favours “cases from which we can learn about their activity and situation. This may mean taking the ones that are most accessible, the ones we can spend the most time with” (p.25). Stakes (2006) concedes “there may be a trade-off between potential for learning and representativeness” (p.25).

Perhaps not all who read these case studies are going to derive meaning from the results

in a similar manner. Each individual, including researchers and evaluators in their relative

fields of practice, bring with them their own cultural biases and perspectives. Similar to

what Moreau (2007) said, the lens which one wears has the ability to dictate how one

views the world, even though some important aspects may be lost in this translation. In

this regard, the researchers and evaluators bring with them a certain level of

understanding and competence about what they perceive as more culturally significant

and important in each case.

Of course, many scholars would agree that developing competence and awareness

of various cultures and utilizing this knowledge to guide one’s practice may lead to more


25

interesting and relevant results. Famed sociologist/educator Greenfield (1993) supports

this contention by stating, “I argue for research that looks at social reality from a variety

of perspectives, particularly from the perspective of different actors in a given social

situation. In this approach, researchers become interpreters of social reality” (p. 68).

Further to this, Greenfield adds, “our ideas and our experiences are subject to human

intervention. One generation trains another and it is these patterns and process of

institutional reality-making that we must come to understand” (p. 67).

Having stated this, Moreau (2007), also brings up issues of what cultural

competence really means - is it a goal and a process? Moreau goes on to argue that by

being attentive to cultural differences, the validity of evaluation processes may be

enhanced. Thus, in closing this paper, rather than critique the ideas proposed in Moreau’s

paper, the three case studies utilized in Stake’s (2006) book will be analyzed further in

terms of the relevance of the five ethical factors (see Table 1, 2 and 3) proposed in her

paper (Bamberger, 1999) to assess how valid these case studies truly are in highlighting

the extent to which cultural competence is demonstrated in Stake’s Multiple Case Study

Analysis.


26

Table 1: The Ukrainian Case Study

Factors used to Assess cultural competence in each

case study

Presence of Factor Absence of Factor

(1) Evidence of valuing diversity.

Yes. A diverse group of stakeholders were included to obtain results and conduct analysis of the research.

(2) Demonstration of the capacity for cultural self-assessment.

Yes. By involving special needs students in the process of developing more inclusive teaching activities, the cultural capacity of students is valued.

(3) Awareness of the “dynamics” and tensions inherent when cultures interact in the evaluation process.

Yes, somewhat. There were considerable tensions between the expectations of the Ukrainian government for schools to provide policies and programs that integrate students of all needs in mainstream schooling. Ironically, however, teachers and schools were asked to incur all expenses for training in the Step by Step program, even though this program was government-mandated.

(4) Institutionalized cultural knowledge.

Yes, somewhat. Ukraine’s cultural practices are unique and the researchers conducting the research appear to be aware of the historical, political and social climate of the country.

5) Reflection on the comprehension of diversity and its importance within and between cultures is demonstrated.

Yes. Various perspectives of stakeholders (e.g. teachers, government officials, parents, students) is shared to get a broader understanding of how various individuals interact (directly or indirectly with one another).


27

Table 2: The Slovakia Case Study


case study



Not always. Researchers admit to having neglected the stories of some community members including village residents, farmers, employers, and county officials.


Yes. The views and culture of Roma children, the main focus of the study, were pivotal (from the researcher’s perspective) in moving them from the margins of society, to positions of empowerment, self-determination and self-motivation through the Step by Step program in Slovakia. Thus, this case study is particularly useful for practitioners when attempting to put into place policies or procedures that demonstrate inclusion.


Yes. At the start of introducing the Step by Step program to the Roma people, many community members, who did not fully comprehend its direction or focus, opposed the ideas proposed. To alleviate misconceptions, teachers, researchers and program developers incorporated ways of improving literacy and numeracy skills through home-based instruction, developing links with their own environment, and creating community programs.


Yes, very strong knowledge base. Rather than work from general research questions, the Slovak researchers focused on local activities where large pockets of Roma people live in impoverished conditions.


28


Yes. Key stakeholders were observed and interviewed directly to obtain the most accurate account of how the Step by Step program was impacting the Roma people, thus alleviating concerns of “third-party” input, as stated by Moreau (2007).

Table 3: The Romanian Case Study


case study



Yes. A lot of time was spent on how Step by Step could be more effective in including not only educators, government officials and administrators in the planning, developing and administering stages of the program throughout Romania, but at the same time greater emphasis was placed on including all children (including those with special needs and Roma children) and parents in the teaching and learning process.


Yes. In general, the focus of this program in Romania was to improve teacher’s capacity and ability to deliver non-traditional learning opportunities for students. The voices pf marginalized youth (e.g. special needs students) became a greater concern as the effects of teacher training became more apparent. Further, parental involvement during the Step by Step training process was important as they provided qualitative feedback and informal evaluations of the effectiveness of the program.


29


Yes. The interest of parents, who served largely as external evaluators of the program; the government, who imposed a compulsory national curriculum that staunchly opposed the goals of the Step by Step program; and the educators, who were being trained to provide non-traditional teaching and learning opportunities (more child-centred), all led to shifting the teaching paradigm from a teacher-centred to child-centred approach. This method was perceived to be highly successful in Romania. The researchers argue that in moving forward, the Step by Step program is expected to change in the next few years, where teachers will be required to meet the educational needs of a more diverse group of children.


Yes, there was demonstration of this in the case. In general the researchers, educators, administrators, parents, students and other key stakeholders were all active members of Romanian society. Thus, the researchers relied heavily on local people and members of educational institutions to put their research into the historical, cultural and political context of Romania.


Yes. Key stakeholders played an active participatory role throughout the research process (e.g. teachers, administrators and parents).


30

Conclusion

By examining the level of cultural competence exuded in each of the three case

studies provided by Stake in his book Multiple Case Study Analysis, it may be asserted

that presence of this attribute (being culturally competent) in research and in evaluation

may give rise to new insights into where the program is lacking and where the program is

highly successful. While it may be problematic to assume that all ethical factors, as

proposed by Bamberger (1999) are clearly demonstrated in each of the cases, on the most

part, it may be concluded that even the presence of two or three factors may be evidence

of the level of cultural competence demonstrated by researchers in carrying out studies of

program effectiveness. In general then, regarding the three case studies centring on the

Step by Step program in Ukraine, Slovakia, and Romania, it may be safe to conclude that,

indeed, the researchers did demonstrate high degrees of cultural competence with

reference to Bamberger’s five ethical factors (1999) and it was important for the

researchers to consider the program in question, in its context (cultural, social, and

political), in order to obtain valid, useful results.


31

Moreau’s Commentary on Saleem-Ahmad’s critique:

“Case Studies in Evaluation, but for what purpose?”

By S. Moreau

Upon reading Saleem-Ahmad’s (2007) opinions on Stake’s Multiple Case Study

Analysis, one particular critique stood out for me. Saleem-Ahmad states that even Stake

appears somewhat unsure as to the priority that should be given to this issue. I too

struggled with this idea. Since the entire focus of our work is on the role of cultural

competence in multiple case study analysis, this is extremely disconcerting. In order to

avoid misunderstanding and misinterpretation, I chose to interview Stake directly. He

was most willing to engage in this discourse and I was impressed by his enthusiasm to

share his views and find meaning in my questions. Below are some extracts of this

electronic interview, shared with his permission.

In the beginning, I offered Stake a definition of cultural competence in evaluation

as proposed by Thompson-Robinson et al's (2004) which is used in Saleem-Ahmad’s

(2007) paper.

Cultural competence in evaluation can be broadly defined as a systematic, responsive enquiry that is actively cognizant, understanding and appreciative of the cultural context in which the evaluation takes place; that frames and articulates the epistemology of the evaluative endeavour; that employs culturally and contextually appropriate methodology; and that uses stakeholder-generated, interpretive means to arrive at the results and further use of the findings. (p. 13)

Stake then replied to this definition as demonstrated in the discourse below.

Stake: I don't make sense of the definition. Can a competence be an inquiry? It has a lot

of good words but in trying to say everything, it doesn't say very much, does it? Why not

write your own definition?


32

Moreau: …I have chosen to do a review of your book, Multiple Case Study Analysis,

with emphasis being made on how effectively and readily cultural competence is

integrated into the three case reports.

Stake: … it sounds like the 3 case study authors should have wanted "cultural

competence" to be integrated into their studies. It is fine to look to see if something like

that is there, but generally better, it seems to me, first to evaluate something in terms of

its own culture or worldview.

Moreau: I think that cultural competence is both a process and a goal. Cultural

competence should be an over-arching value that frames all other aspects of an

evaluation. It should not simply be considered as an add-on. Context-specific flexibility

and a capacity for understanding and appreciation of various cultures and values is a

tool that evaluators should use in all evaluation situations. The American Evaluation

Association’s Guiding Principles highlight the need to proactively attend to diversity

issues as a necessary prerequisite for ethical practice and validity.

In the 3 case studies, they would be considered culturally competent if the evaluators

were able to see the evaluation situation through the lenses of the stakeholders as

opposed to (for example) Western, white male. Or, at the very least, to be aware of his

cultural biases and attempt to put those biases aside through choice of methodology and

stakeholder participation, for the good of the evaluation situation.

Stake: It seems you are speaking to me with your heart more than with your mind. You

are telling me how you would like evaluation to be.

Let's use the words the way the dictionary says.

Competence is a condition, a trait, a readiness to respond appropriate to the situation.


33

It is not a process. Becoming competent is a process.

It is not a goal. Becoming competent is a goal.

We might say that cultural competence is sensitivity to cultural issues, a readiness

to act in ways that support cultural values more than other values. We might go further to

say (to advocate) that exercising cultural competence in evaluation is good. It seems

problematic to say that cultural competence is an overarching value for all evaluation,

some of which has little to do with cultural issues or perspectives, and when evaluation

already has an over-arching value, that is, to discern the quality of an evaluand. Michael

Scriven distinguishes nicely between the goal for evaluation (finding quality) and the

roles of evaluation, one of which is to improve cultural understanding or conditions.

The three case studies in my book were carried out by people who had more than

a little cultural competence (as usually it would be defined). They were not assigned to

do evaluation studies but non-evaluative case studies. As with all research, these studies

had evaluation aspects, but the Step by Step people were anxious not to consider them as

evaluation studies. Cultural competence should be as important to a case study team as it

is to a program evaluation team, don't you think?

Moreau: I do agree that Cultural competence should be as important to a case

study team as it is to a program evaluation Team. I suppose what I'm trying to clarify is:

In the overall process of developing a protocol for Multiple Case Study Analysis, how

much emphasis is placed on cultural context?

Stake: None. That is not a design option. It is a user option. A user can give weight at

the beginning to cultural context. Each case study would be expected to pay attention to

it. Some or all might find little to say about it. The cross-case analysis, even though


34

sensitive to cultural issues, might find nothing in the case studies to report. One cannot

assume that there always will be a common cultural issue across multiple cases.

Moreau: What do you think should be included in a Case Study Analysis design to make

it more culturally competent?

Stake: Stacey, it is a little like asking if a recipe for clam chowder is sensitive to cultural

competence. Case study is the study of a case. Multiple case study is the study of several

cases. The researcher can choose to look for evidence of cultural influence on the cases

Some researchers would find more than others.

Moreau: What would be some of the weaknesses of the Multiple Case Study Analysis

with regards to cultural competence? As you mentioned, when multiple case studies are

analyzed, sometimes generalizations need to be made, which, I imagine, compromises the

richness of the cultural context(s)) in the final report.

Stake: It has neither weaknesses nor strengths of that kind. The person who uses it may

have high competence or low.

Moreau: In retrospect, what do you think, if anything, could have/ should have been

done, in the 3 case studies to make them more sensitive to cultural issues?

Stake: It is up to the researchers to study what they think is important. Case study is a

way of studying a case and paying some attention to cultural contexts. I don't have a

feeling that these researchers should have been more sensitive than they were. They were

aware that the schools were still influenced by the end of the Cold War and the uprising

in Yugoslavia. The Roma children and handicapped children were discriminated against.

Moreau: Of interest, why were the Step by Step people anxious not to consider their work

as evaluation studies?


35

Stake: The Step by Step administrators were satisfied with the evaluation studies they

had done in recent years and wanted this to be different. Here the studies were going to

be done by 29 teams of their own people in 29 countries. Perhaps they had more to lose

than by hiring one evaluation team to study a specific operation or comparison. I don't

know.

Stacey: Things are slowly starting to come together now. Here is what I'm

understanding: In multiple case studies, some researchers may choose to direct their

focus to the cultural context. However, cultural issues always play a part in research

(case studies included), no matter how minimal. Like in your example of clam chowder- a

chef wouldn't think of serving it to a religious group who did not eat clams. However,

there is a split second of processing in which culture is analyzed; yet in some situations

this analysis is obvious and so it is not necessary to make it more explicit.

Also, you are assuming that there is a certain "buyer beware" clause in case

studies and other research, in that it is up to the recipient to determine whether or not

this particular study provides him with the information to make an informed decision.

There are a multitude of studies, studying a multitude of things. In some studies, cultural

context might be the dominant force, yet in others it might simply be a fleeting

consideration. (For example in a third world, non-literate country, essay-style questions

are obviously not the appropriate choice.)

Stake: Sounds good to me. We will probably talk some more about clam chowder in my

class today, feel free to drop in. The main topic is triangulation.


36

Conclusion

So, after all this - Is Stake’s Multiple Case Study Analysis culturally competent? My

answer is yes, if Stake is the researcher. Stake is obviously culturally competent. Yet,

with his experience, it seems that it has become second nature to him, and thus, he does

not reflect on it in a conscious manner when implementing research methods. Although

this sixth sense could be considered a wonderful attribute in a researcher, it also signals

that multiple case study evaluation should be approached with a degree of caution by the

novice researcher. Perhaps like all tools, multiple case study analysis can be made to be

culturally competent in the hands of a culturally competent researcher. A tool is only as

good as the person wielding it.


37

Sharing Ideas on Stake’s Multiple Case Study Analysis

By S. Moreau and S. Saleem-Ahmad

Recently, Moreau and Saleem-Ahmad were able to sit down and have a more

reflective discussion about their ideas and thoughts on Stake’s Multiple Case Study

Analysis. By way of concluding this discourse on Stake’s work, the following excerpt is

offered from their dialogue*.

Sofia: I guess we were able to draw the same conclusions about the level of cultural

competence shown by researchers in Stake’s new book. What I found really interesting is

how Stake indirectly differentiates between developing research protocols, given a

particular cultural context and the idea that cultural competence is an attribute, not a

research quality. Is this what you thought as well?

Stacey: Well, yes. And it made me re-think the whole idea. It particularly marked me

during the interview when he corrected me on the issue of how one becomes competent

rather than being culturally competent. This made me reflect on the notion that

evaluators and researcher can be at various stages of cultural competence depending on

their experience and their own cultural capacity. Some researchers may be naturally

culturally competent, whereas others may strive for it all their lives and never quite grasp

the concept.

Sofia: True, and it almost appears as if in order to grasp knowledge and understanding of

another’s culture, one has to be able to come to terms with their own cultural space first.

Once this understanding has occurred, only then will one be able to embark on the

uncomfortable task of stepping away from preconceived notions to develop new ideas


38

and understandings of cultures different from their own. I would refer to this act as

making the strange into the familiar.

Stacey: Also, I finally grasp his implication that no one evaluation can be all things to all

people. Although I do think that cultural competence should be a consideration in the

evaluation process, it is a human condition that must be incorporated into the theoretical

process of evaluation. Much like in school when we evaluate students for various

competencies: It doesn’t mean that every activity that is done will touch on each of these

competencies, but when we judge a child’s competence in a particular area, we will pull

out the activities which are most relevant in relation to a particular competence…

Sofia: … Stake presents us, albeit implicitly, with an understanding of how important

cultural competence is as an attribute in a researcher or evaluator, as opposed to focusing

on it as a tool within a research or evaluation.

* This meeting occurred in Chelsea, Quebec, during a relaxing visit to a spa on Saturday April 7, 2007.


39

References

Bamberger, M. “Ethical Issues in Conducting Evaluation in International Settings”.

(1999). In J.L. Fitzpatrick and M. Morris (eds.), Current and Emerging Ethical

Challenges in Evaluation. New Directions for Evaluation, no. 82. San Francisco:

Jossey-Bass.

Chevalier, Z.W., Roark-Calnek, S., & Strahan, D.B. (March 1982). « Responsive

Evaluation of an Indian Heritage Studies Program: Analyzing Boundary

Definition in a Suburban School Context.” Paper presented at the annual meeting

of the American Educational Research Association, New York City.

Cousins, J. B. & Whitmore, E. (1998). “Framing Participatory valuation.” In E.

Whitmore (ed.), Understanding and Practicing Participatory Education. New

Directions for Evaluation, no. 80. San Francisco: Jossey-Bass.

Cross, T.L., Barzon, B.J., Dennis, K. W., & Isaacs, M. R. (1989). Towards a Culturally

Competent System of Care: A Monograph on Effective Services for Minority

Children Who Are Severely Emotionally Disturbed. Washington, D.C.: CASSP

Technical Assistance Center, Georgetown University Child Development Center.

Flyvbjerg, B. (2006). “Five Misunderstandings About Case-Study Research.”

Qualitative Inquiry, 12(2): 219-245.Greenfield, T.B. (1993). ‘Organizations as

talk, chance action and experience’, in T.B. Greenfield & P. Ribbens eds.,

Greenfield on Educational Administration: Towards a Human Science, London,

UK: Routledge, 53-74.

Hodgkinson, C. (1991). Educational leadership: The moral art. Albany, NY: State

University of NewYork Press.


40

King, J.A. (1998). “Making Sense of Participatory Evaluation Practice.” In E. Whitmore

(ed.), Understanding and Practicing Participatory Education. New Directions for

Evaluation, no. 80. San Francisco: Jossey-Bass.

Lavizzo-Mourey, R. & MacKenzie, E. R. (1996). “Cultural Competence: Essential

Measurement of Quality for Managed Care Organizations.” Annals of Internal

Medicine, 124, 919-921.

[http://www.acponline.org/journals/annals/15may96/cultcomp.htm].

Madison, A.M. (ed.) (1992)..Minority issues in Program Evaluation. New Directions for

Program Evaluation, no. 53. San Francisco: Jossey-Bass.

Scriven, M. Evaluation Thesaurus. (4th ed.) Thousand Oaks, CA: Sage, 1991.

Stake, R. E. (April 1967).“The Countenance of Educational Evaluation.” Teachers

College Record 68, 523-540.

Stake, R. E. (1975). “Program Evaluation, Particularly Responsive Evaluation.”

Kalamazoo, MI: Evaluation Center, Western Michigan University.

Stake, R. E. (1991). “Retrospective on ‘The Countenance of Educational Evaluation.’” In

M.W. McLaughlin and D.C. Phillips (eds.), Evaluation: At Quarter Century.

(Ninetieth Yearbook of the National Society for the Study of Education).

Chicago: University of Chicago Press.

Stake, R.E. (2006). Multiple Case Study Analysis. New York, NY: The Guilford Press.

Thompson-Robinson, M., Hopson, R., & SenGupta, S. (eds.) (2004). In Search of

Cultural Competence in Evaluation: Toward Principles and Practices. Jossey-

Bass: NJ.

http://www.acponline.org/journals/annals/15may96/cultcomp.htm


41

Weaver, L. & Cousins, B. (Nov. 2001) “Unpacking the Participatory Process.” Paper

presented at the annual meeting of the American Evaluation Association, St.

Louis.

Wilcox, T. (July 1984). “Evaluating Programs for Native Students: A Responsive

Strategy.” Paper presented at the Mokakit Indian Education Research

Association’s International Conference, London, Ontario, Canada.


42

About the Authors

Stacey J. Moreau is a M. Ed. student at University of Ottawa. She has a

Bachelor’s degree in Education and Honours in Psychology. Born in Ottawa, Moreau

lived in South Africa for 8 years, where she was a teacher and psychometric assistant.

She now lives in Quebec with her husband, Francois, and their 3 children.

Sofia Saleem-Ahmad is a M. Ed student at University of Ottawa. She has a

Bachelor’s degree in Education from York University and Honours in Science from

University of Toronto. She is currently a student representative with the National Chapter

of the CES. Born in Edmonton and raised in Toronto, Saleem-Ahmad recently moved to

Ottawa with her husband, Imran. They are expecting their first child in September.

A Critical Review of Stake’s Multiple Case Study Analysis ...€¦ · Review of Stake’s...

Documents

Transcript of A Critical Review of Stake’s Multiple Case Study Analysis ...€¦ · Review of Stake’s...