1

HIRING FAST AND SLOW:

A QUALITATIVE STUDY OF TEACHER HIRING PRACTICES VIEWED THROUGH THE

LENS OF BOUNDED RATIONALITY

A doctoral thesis by

Michael Hinman

To

The School of Education

In fulfillment of the requirements for the degree of

Doctor of Education

in the field of

Education

College of Professional Studies

Northeastern University

Boston, Massachusetts

March 2019

2

Abstract

This examination of the hiring practices of K-8 educational leaders in a particular county in a

mid-Atlantic state is viewed through the theoretical framework of bounded rationality, which

holds that rationality is limited by the availability of information, time, and mental capacity. In

this study, a review of the literature explored the applicability of bounded rationality to the work

of school principals, what is known about the decision-making processes by experts in a number

of fields, and what is known about the decision-making process involved in teacher hiring. Of all

the decisions made by school principals, the hiring of teachers may be the most important task;

nevertheless, the bounded rationality involved in decision-making unchecked by the supports of

protocols such as research supported screening criteria or interview procedures engenders an

ongoing problem of practice: sub-optimal decision-making in the process of teacher hiring. This

research examines the question: How does bounded rationality affect administrators’ decision-

making processes in the process of hiring new teachers among comparable school districts in the

mid-Atlantic region? There were eight findings identified in this research: bounded rationality

affects the decision-making of K-8 educational leaders; the teacher characteristics listed as most

important were not consistent among all principals; fast thinking through biases and heuristics

including fit, missing data, overconfidence were identified; slow thinking through the use of

protocols, question sets and self-awareness were also identified; and principals stated the

addition of a decision-making tool, such as a checklist, might improve their processes.

Key terms: biases and heuristics, bounded rationality, decision-making, and teacher

hiring.

3

Acknowledgements

My family. Through my mother’s limitless belief that I was the cleverest of all the Michaels and

my father’s checks on my tendency to believe her, I generated the level of imposter syndrome

that motivates one toward educational pursuits. From my father, I learned to consider my words

and to see a problem, go to a hardware store, and fix it. From my mother, I learned very early

that an audience, even of one, is all the creation one really needs. Well, that and maybe music.

And something to eat, of course. Do you want a sandwich? My brother, two years and two light

years ahead of me in all pursuits, also fed my desire toward mastery, but I realized that there

were decades required for us mere mortals to come close to his example. It was actually my

sister’s gifts of the best sellers Thinking Fast and Slow and The Checklist Manifesto that sparked

my dissertation inquiry. She has always been the model to live life as our best selves, and in

doing so, she is the best of us. Ask anyone. If you met my children, you would understand I will

need a whole heap of education to keep up with them. Talented, driven, and quick-witted (the

ultimate sign of intellect in our house), they make me want to be, and are each examples of how

to be, a better person. For the three of us, truly, it is my wife who is the team’s consistent

manager, coach, and entire front office. All of our accomplishments were largely due to her

unwavering and relentless support of our best interests, spending more time considering and

solving our problems than we did. The decades of that workload was compounded by the

addition of my graduate work pursuits. And how did she react? If you don’t know the answer,

you aren’t following along here. Sue is an efficacy and achievement expert. Here is just one

example: she once suggested we run a marathon; five years later, with me dragging along

through seven, she had run thirteen, finishing by qualifying for and running Boston. For the

better part of my life, I have been impressed by and drawn to her determination and success. The

completion of this project, and all that is good in my life, is owed to her.

4

The participants in my study. Had these principals not volunteered their time, this study and

the years of its preparation would have been lost. As a researcher, I sought to avoid evaluative

thinking, but could not help but leave each interview impressed by these dedicated educational

leaders, their knowledge, and their respect for their students, staff, and communities.

The educators I have known. Much of who I am and where I am can be directly sourced from

the individuals that have been my teachers since September 1970, in which my kindergarten

teacher, Mrs. Maxim, was tasked with the unenviable job to curtail my early first attempts to

organize opposition groups. I believe she retired at the end of that year. My love/hate

relationship with the American education system presented a mixed bag for those adults I met as

a student, as a teacher and administrator, and as a faculty colleague. Daily, I have access to

outstanding educators from those in my office, Betty, Hope, and Alison, to those in each office

and classroom around the 90 square miles of the school district. These professionals face

herculean tasks with great joy and skill. Stop in and you might find yourself welling up as you

witness the most caring and touching of moments. To name individuals would be to miss

innumerous others. But risking that, I would like to thank a few, now doctoral, colleagues: the

late Dr. Jeanne Carlson for dedicating her life in support of gifted education and for her

mentorship; Dr. Annette Giaquinto for being the model of moral and rational leadership; Dr.

Pamela Vaughan for her years of guidance in the field; Dr. Chris Unger for inspiring the

improvement of systems for those they serve; and my advisor, Dr. Peg Dougherty, for her

specific and clear direction through this writing process.

5

Table of Contents

Table of Contents .......................................................................................................................... 5

Chapter I: Introduction .............................................................................................................. 10

Statement of the Problem .......................................................................................................... 10

Significance of the Problem ...................................................................................................... 13

Positionality Statement .............................................................................................................. 15

Research Question ..................................................................................................................... 17

Theoretical Framework ............................................................................................................. 18

Chapter II: Literature Review ................................................................................................... 22

Bounded Rationality .................................................................................................................. 22

Identified biases and heuristics. ............................................................................................. 23

Bounded rationality’s effect on decision-making. ................................................................. 25

Critics of the framework. ....................................................................................................... 28

Decision-making Tools ............................................................................................................. 29

Decision-making tools used in business. ............................................................................... 30

Decision-making tools used in medicine. .............................................................................. 32

Decision-making tools used in education. ............................................................................. 34

Subtle decision-making influence. ........................................................................................ 35

Summary. ............................................................................................................................... 37

Hiring Practices ......................................................................................................................... 39

Screening. .............................................................................................................................. 39

Interviewing. .......................................................................................................................... 41

Conclusion ................................................................................................................................. 45

Chapter III: Research Design ................................................................................................... 47

Methodology ............................................................................................................................. 47

Research Design ........................................................................................................................ 47

Research Tradition .................................................................................................................... 49

Participants ................................................................................................................................ 51

Recruitment and Access ............................................................................................................ 52

Data Collection .......................................................................................................................... 54

Data set 1 – Survey of principals (Appendix D). .................................................................. 55

Data set 2– Principal interviews (Appendix E). .................................................................... 55

Paper review. ......................................................................................................................... 56

Candidate interviews. ............................................................................................................ 56

General processes. ................................................................................................................. 57

Data set 3 – Focus Group (Appendix F). ............................................................................... 57

Data Storage .............................................................................................................................. 58

Data Analysis ............................................................................................................................ 59

6

Table 1 ....................................................................................................................................... 60

Table 2 ....................................................................................................................................... 61

Data set 1: Survey ..................................................................................................................... 61

Table 3 ....................................................................................................................................... 62

Table 4 ....................................................................................................................................... 63

Data Set 2: Interviews ............................................................................................................... 64

Table 5 ....................................................................................................................................... 67

Data set 2 - Interviews first cycle coding .................................................................................. 69

Data set 2 - Interviews second cycle coding ............................................................................. 70

Data Set 3: Focus Group ........................................................................................................... 71

Data Set 3 – focus group cycle 1 coding ................................................................................... 72

Data Set 3 – focus group cycle 2 coding ................................................................................... 72

Table 6 ....................................................................................................................................... 72

Summary ................................................................................................................................... 73

Trustworthiness ......................................................................................................................... 73

Credibility. ............................................................................................................................. 74

Transferability. ...................................................................................................................... 74

Dependability......................................................................................................................... 74

Confirmability. ...................................................................................................................... 74

Chapter IV: Findings and Analysis ........................................................................................... 76

Introduction ............................................................................................................................... 76

Review of Research Problem .................................................................................................... 76

Table 7 ....................................................................................................................................... 78

Review of Research Question ................................................................................................... 79

Data Collection .......................................................................................................................... 80

Data Analysis ............................................................................................................................ 81

Table 8 ....................................................................................................................................... 82

Analysis of Survey Responses .................................................................................................. 82

Participant experience. ........................................................................................................... 83

Applicant data valuation. ....................................................................................................... 83

Table 9 ....................................................................................................................................... 83

Committee use. ...................................................................................................................... 83

Table 10 ..................................................................................................................................... 84

Protocols. ............................................................................................................................... 84

Survey Summary. .................................................................................................................. 84

Hiring rates............................................................................................................................ 85

Data valuation. ...................................................................................................................... 85

Committee use........................................................................................................................ 85

Protocols. ............................................................................................................................... 86

Analysis of Data from 1:1 Interviews ....................................................................................... 86

7

Table 11 ..................................................................................................................................... 86

Identified Fast Thinking. ....................................................................................................... 88

Expertise/confidence. ............................................................................................................. 88

Intuition/gut/fit. ...................................................................................................................... 88

Time constraints. ................................................................................................................... 88

Likeability/background of candidate similar to administrator. ............................................. 88

Identified Slow Thinking. ...................................................................................................... 89

Protocols. ............................................................................................................................... 89

Cross checking. ...................................................................................................................... 89

Self-reflection......................................................................................................................... 89

Second Cycle ............................................................................................................................. 89

In vivo coding of transcripts...................................................................................................... 89

Expertise. ............................................................................................................................... 89

Confidence. ............................................................................................................................ 90

Intuition/gut. .......................................................................................................................... 90

Time Constraint. .................................................................................................................... 90

Likeability/Similarity to the administrator. ........................................................................... 90

Table 12 ..................................................................................................................................... 91

Summary of interviews. ......................................................................................................... 93

Analysis of Focus Group Data .................................................................................................. 95

Table 13 ..................................................................................................................................... 96

Cycle 1 Pattern matching coding. ............................................................................................. 97

Administrator Expertise/Confidence. .................................................................................... 97

Confirmation Bias/Likeability/Beliefs Regarding Characteristics of Successful Teachers. . 97

Fit........................................................................................................................................... 97

Cycle 2 In vivo coding. ............................................................................................................. 97

Prediction Overconfidence - Administrator Expertise/Confidence. ...................................... 97

Gut/Intuition. ......................................................................................................................... 98

Confirmation Bias/Beliefs Regarding Characteristics of Successful Teachers/Fit. .............. 98

Summary of focus group analysis. ........................................................................................ 99

Table 14 ..................................................................................................................................... 99

Emergent Themes from the Three Data Sets .......................................................................... 100

Identified fast thinking......................................................................................................... 101

Affect bias/likeability. .......................................................................................................... 101

Confirmation bias/beliefs regarding characteristics of successful teachers. ...................... 101

Fit......................................................................................................................................... 102

Missing data bias. ................................................................................................................ 102

Identified slow thinking. ...................................................................................................... 103

Self-awareness. .................................................................................................................... 103

Table 14 ................................................................................................................................... 104

8

Data Triangulation................................................................................................................... 105

Trustworthiness ....................................................................................................................... 106

Credibility. ........................................................................................................................... 106

Dependability....................................................................................................................... 107

Confirmability. .................................................................................................................... 107

Summary of Findings .............................................................................................................. 107

Finding one: ......................................................................................................................... 107

Identified fast thinking. ........................................................................................................... 108

Finding two: ......................................................................................................................... 108

Finding three: ....................................................................................................................... 108

Finding four: ........................................................................................................................ 108

Finding five: ........................................................................................................................ 108

Identified slow thinking. ......................................................................................................... 108

Finding six: .......................................................................................................................... 108

Finding seven:...................................................................................................................... 108

Finding eight: ....................................................................................................................... 108

Conclusion ............................................................................................................................... 108

Chapter V: Findings ................................................................................................................ 110

Introduction ............................................................................................................................. 110

Review of Themes from Research .......................................................................................... 110

Discussion of Key Findings .................................................................................................... 111

Finding one: ......................................................................................................................... 111

Finding two: ......................................................................................................................... 112

Finding three: ....................................................................................................................... 113

Finding four: ........................................................................................................................ 113

Finding five: ........................................................................................................................ 115

Finding six: .......................................................................................................................... 116

Finding seven:...................................................................................................................... 116

Finding eight: ....................................................................................................................... 117

Linking Findings to Theoretical Framework .......................................................................... 118

Linking Findings to Literature ................................................................................................ 121

Study Limitations .................................................................................................................... 124

Alternate explanations ............................................................................................................. 126

Implications for Educational Practice ..................................................................................... 127

Implications for Future Research ............................................................................................ 128

Conclusion ............................................................................................................................... 130

References .................................................................................................................................. 132

Appendix A ................................................................................................................................ 147

Appendix B ................................................................................................................................ 148

9

Appendix C ................................................................................................................................ 149

Appendix D ................................................................................................................................ 151

Appendix E ................................................................................................................................ 153

Appendix F ................................................................................................................................ 155

10

Chapter I: Introduction

The purpose of this qualitative case study is to understand the teacher hiring practices of

educational leaders in K-8 public schools in a particular county in a mid-Atlantic state. In this

research, teacher hiring practices are generally defined as the screening, interviewing, and

selecting of teacher applicants. In this case study, the researcher seeks to understand

administrators’ beliefs and actions related to hiring and examine what they choose to do as well

as what they choose not to do, and their explanation as to the reasons for those choices.

Knowledge generated may inform the practice of educational leaders in the important work of

staffing schools.

Statement of the Problem

This examination of the hiring practices of K-8 educational leaders is viewed through the

theoretical framework of bounded rationality, which holds that rationality is limited by the

availability of information, time, and mental capacity (H. A. Simon, 1982). According to

bounded rationality, the mental shortcuts that allow humans to survive in times of crisis requiring

quick, decisive action do not also generate predictable rational decision-making. As a result of

this natural mindset, the answers that come to mind first are deemed by the thinker to be the best

answers. Of all the decisions made by school principals, the hiring of teachers may be the most

important task (Mason & Schroeder, 2010; Rowan, 1994a); however, the bounded rationality

involved in decision-making unchecked by the supports of protocols such as research supported

screening criteria or interview procedures (Cranston 2014) engenders an ongoing problem of

practice: sub-optimal decision-making in the process of teacher hiring.

From the tender age of 6 until at least 16, American children are compelled by law to

spend their days under the tutelage of school employees. Students don’t pick these teachers,

11

school administrators do. Yet, it is the students who benefit or suffer from those selections

(Haskins & Loeb, 2007). The importance of the selection of teachers should not be

underestimated for as Hanushek (2011) suggests, “Replacing the bottom eight percent of

teachers with an average teacher would bring the U.S. up to the level of Finland” (p. 475). The

hype and debate over teacher evaluation rages on, but where is the discussion of the selection

process that results in teacher hiring in the first place?

Teaching is one of the most complex and demanding professions (Haskins & Loeb, 2007;

Rowan, 1994b; Schumacher, Grigsby, & Vesey, 2015). It is the task of teacher education

programs to prepare teacher candidates to teach effectively, but as in all fields of endeavor, these

novice professionals range in ability (Gentry, 2012). It is imperative effective teachers are within

the group of employed teachers as the selection of new teachers has a profound and long-

reaching impact on schools (Cranston, 2012; Rutledge, Harris, Thompson, & Ingle, 2008;

Stronge, Ward, & Grant, 2011). According to the latest (2016) Educator Provider Performance

Report form the New Jersey Department of Education, only a portion of trained teachers are

employed in the field; of the 9,284 New Jersey 2014 graduates with a certificate of eligibility

with advanced standing, only 65% were employed in the profession in 2016 (NJDOE). Given

that the pool of teacher candidates is greater than the available openings (Clement, 2013), and

the importance of the selection (Engel & Finch, 2015; Stronge et al., 2011) it would be expected

that hiring practices would be deeply researched, carefully and consistently practiced. This is not

the case (Liu & Johnson, 2006).

Tasks such as teacher evaluation, curriculum development, or student discipline are often

conducted with expectations and procedures for consistency and interrater reliability; however,

the actual practice of teacher hiring varies greatly, is often “information poor,” and the criteria

12

for selection is often based on feeling over evidence (Liu & Johnson, 2006 p. 331). Improvement

in the hiring process could produce significant improvement in student achievement (Heneman

& Milanowski, 2004). This study expands the available research on teacher hiring practices and

could lead to the refinement of present interview processes.

It is widely accepted that teacher effectiveness is the greatest predictor of student success

(Darling-Hammond, 2010; Stronge et al., 2011). The belief was expressed in the opening

statement of the Obama Administration’s Educational Plan of 2011 (Our future, our

teachers.2011). The importance of a quality education is undisputed; however, the best manner in

which to judge teacher effectiveness has been the subject of polarizing debates in education. The

U.S. Department of Education’s $4 billion dollar Race to the Top initiative of 2009 focused the

country on the process for evaluating employed teachers through value added measures which

calculate teacher effectiveness in part by comparing student test scores to those of previous

years.

Less discussed are the practices involved in initial teacher recruitment so that the most

effective teachers are selected in the first place. Little is known about the nuances of the process

of teacher hiring (Mason & Schroeder, 2010), defined for this study as the paper review, sifting

applications and resumes, and the live selection, conducting interviews. Yet, targeted research by

Engel & Fitch and Kayes (2015; 2006) has recently examined the steps in K-8 as well as higher

education, and some conclusions for better hiring processes have been offered. Effective hiring

practices are required to arm the teaching profession with the caliber of teachers needed to face

the onslaught of challenges (Peterson, 2002; Rutledge et al., 2008; Whitworth, Jones, Deering, &

Hardy, 2016).

13

Significance of the Problem

The complexity of and the importance of effective teacher hiring generates the potential

for problems. The hiring process begins with screening applicants based on static criteria such as

certification, education and experience. Screening is usually followed by individual interviews. It

is the impressions from interviews which are given the most weight in hiring decisions

(Cranston, 2012; Kersten, 2010; Rutledge et al., 2008). Specifically, principals focus on

interview skills and stated expressions of caring and work ethic rather than content knowledge or

pedagogical skill (Engel, 2013). The impressions made during an isolated meeting are given

more value than the information gleaned from the years of performance reported through records

reviewed in the screening process (Peterson, 2002; Rutledge et al., 2008).

One manner in which the hiring process may be less than optimal is caused by the

comparison to the employee being replaced. Information gleaned is often skewed, consciously or

unconsciously, in comparison to the teacher the candidate may be replacing. Candidates are often

measured by the fit to the staff already in place (Rutledge et al., 2008). Thus new hires may

replace a retiree or fill a new position not based on the researched characteristics of effective

teaching, but on similarity to the present staff, the principal’s sense of his/herself as a teacher, or

just general likeability. This raises concerns not only about replicating the status quo in teaching

styles, but also about diversity, or the lack thereof, among the school faculty.

A problem generated by poor hiring relates to the aftermath burden of the district. For

teachers who are hired, schools focus a good deal of time and money in order to generate

professional development. An entire section of federal funding, Title IIA: Preparing, Training,

and Recruiting High Quality Teachers and Principals, is allocated for this effort (USDOE,

09/15/2004). Additionally, monies are dedicated to teacher preparation at the higher education

level (USDOE, 8/14/2008). Through careful hiring, districts could allocate the requisite time and

14

energy of professional development to advance effective teachers rather than attempt to repair

ineffective ones. A good hire has the potential to impact generations of students positively,

whereas a poor selection has a direct negative effect on students. Additionally, a poor selection

can be a drain on the limited resources available for professional development and school-wide

improvement. The selection of staff can be one of the most lasting decisions made by an

educational leader (Mason & Schroeder, 2010).

Other aspects of poor hiring may be generated by availability of time and ability to apply

rational decision-making. Areas to be examined include possible overconfidence based on

snippets of information such as colleges attended or references mentioned to being inattentive to

missing information. Predicting the future behavior of an individual is extremely difficult

(Courtney, Lovallo, & Clarke, 2013; Schumacher et al., 2015; Venkatraman, Payne, & Huettel,

2014); the simple fact that the interviewee knows she/he is being judged may make accurate

assessment by the interviewer that much more difficult (Cain-Caston, 1999). Coupled with the

demands for time on the part of the school administrator, the interview process is ripe for

misunderstanding and erroneous conclusions (Rutledge et al., 2008; Shen et al., 2012). Schools

engage in ongoing professional development, observation, and evaluation, all with the goal of

ensuring the faculty is capable of meeting the challenges of the profession. However, ongoing

professional development or the addition of graduate degrees offer little impact on teacher

effectiveness; as it turns out, the relatively few hours spent screening and selecting good teachers

are far more important than the years of training that occur after hiring (Chingos & Peterson,

2011). Therefore, more time, effort, and focus is needed on the system of hiring teachers.

Research on teacher hiring is limited, but researchers such as Clement; Rutledge, Harris,

Thompson, and Ingle; and Schumacher (2013; 2008; 2015), offer possible improvements. One

15

example of a researched technique to improve rationality and consistency in teacher hiring

practices is the use of structured interviews specifically aligned to the domains of teaching --

planning, management, instruction, and reflection (Clement, 2013). These protocols are helpful

in selecting effective teachers, but structured interviews are rarely used (Clement, 2009; Liu &

Johnson, 2006; Rutledge et al., 2008). Other research central to the theoretical framework of

bounded rationality suggests that the use of structured protocols that address pitfalls in human

thinking would improve the rationality in hiring decisions (Bingham & Haleblian, 2012; Guerra-

López & Blake, 2011; M. Toplak, West, & Stanovich, 2011). The impact of bounded rationality

on even the best case scenario for teacher hiring practices requires the attention of educational

leaders. Suboptimal decision-making caused by missing data, the misinterpretation of data, or

overconfidence generated by the presence of some data could undermine the crucial work of the

hiring process and, therefore, the quality of instruction provided to students.

Positionality Statement

The researcher of this study brings to this work experiences and subsequently biases. As

Machi and McEvoy (2012) state, “Preconceptions, personal attachments, and points of view

present both strengths and weaknesses for the research effort” (p. 18). This positionality

statement examines the researcher’s biases and perspectives as these may influence this research.

An author’s position should be examined and considered in relation to others (Briscoe, 2005).

This researcher is a middle aged, middle class, mid-career, mid-Atlantic, monolingual, married

male. As such, his notion of the institution of education is influenced by experience of it from his

position within the larger society.

16

Further, this researcher is a product of public schools, was a teacher for over a decade,

and during the past two decades has served as a director of curriculum as well as instructor and

director of teacher education programs.

His perspectives on education were developed over unique philosophically oriented times

and within differing roles. These could influence his beliefs about education in general and also

the notion of effective teaching specifically. This researcher was a K-8 student in public schools

from 1971 to 1984. Hargreaves & Fullen (2009) term the period of 1970-1990, the Interregnum,

as it was a time that maintained the optimism and innovation of the 1960s, but also infused

market-driven competition brought on by a declining economy. This researcher was a teacher

from 1990-2002. This era, Hargreaves (2009) terms the Second Way, was a period of increased

prescriptive curriculum, outcome-based measures, and state-led reforms. This researcher serves

as an administrator during the years of the Third Way, from 2002-Present, a time of autocracy

and technocracy. Therefore, it should be noted that his desire to seek an improved decision-

making process to identify effective teachers, may be in the context of this Third Way or a

combination of experiences in all three. It is to be seen if this research fits the inclusive,

supportive, sustainability focused Fourth Way which Hargreaves (2009) prescribes for the future

of education.

This researcher’s demographic and ideological positioning impacts his views on the role

of education and therefore his views of the decisions made by those within that system.

Interestingly, unlike many of his doctoral program colleagues, his inquiry does not focus on

individuals or groups served by education; rather, in the study of the educators themselves. It

should be noted that as an insider within the group of administrators, he may see the decision-

making processes of educators differently than an outside observer might. Briscoe (2005) asserts

17

that perspectives are set by one’s position within a culture and enmeshed in the language of the

culture. Therefore, it may be this researcher’s relationship to educational leaders, not to the

demographics of the students they serve, that should be most closely inspected for bias.

Beyond this researcher’s position as an administrator and his intent to research the hiring

practices of schools in which some colleagues are known to him through his years in working in

education in the area, additional biases and perspectives germane to his specific area of study

include his beliefs which may be the product of the theoretical and political influences on

education during the years as a student and as an employee. These beliefs include: the child’s

right of access to quality education supersedes the preferences of the employees of a school, the

possibility that quality education can be established for all, and that available knowledge coupled

with willingness can make this possible.

This study is feasible at the target site of K-8 schools within a mid-Atlantic region of

school districts. Participants volunteered via reply to a brief survey of interest. Interviews were

conducted based on the pool of responses. Even though the researcher is a member of a specific

district within the larger target group, the study’s design, the confidentiality offered, and the

voluntary nature of the participation offer an appropriate setting to conduct this work.

Research Question

The purpose of this study of as educational leaders in a mid-Atlantic region of school

districts is to understand their experience of teacher hiring. This research explores the challenges

involved in identifying effective teachers during the hiring process which include the paper

review, sifting applications and resumes, and the live selection, conducting interviews.

Therefore, the researcher has elected to utilize qualitative research. In this study, the researcher

seeks to understand administrators’ beliefs and actions related to hiring and examine what they

18

choose to do as well as what they choose not to do, and their expression as to the reasons for

those choices.

The present qualitative study was designed to answer the following research question:

How does bounded rationality affect administrators’ decision-making processes in the process of

hiring new teachers among comparable school districts in the mid-Atlantic region? As described

in the next section, bounded rationality is a framework this researcher utilized to frame the

examination.

Theoretical Framework

Bounded rationality, first coined by Simon (1955) offered a response to the dominant

theory of the time, neoclassical rationality, which held that human decisions were calculable,

rational responses with the most utility as the result. Simon suggested humans responded not

with the most logical solutions, but the most readily satisfying (H. A. Simon, 1982). Additional

Nobel awarded work was conducted by Kahneman and Tversky (1984) which identified the

mental shortcuts or biases and heuristics most involved in fast thinking. Present work in bounded

rationality continues to define the fields of psychology and economics and the understanding of

human rationality focusing on areas such as predictions and forecasting (D. Lovallo, Clarke, &

Camerer, 2012), confirmation bias (Rabin & Schrag, 1999), and brain and addiction research

(Bernheim & Rangel, 2004).

Despite the human shortcomings involved in processing data demonstrated through the

decades of research building the theoretical framework of bounded rationality (Botterill &

Hindmoor, 2012; Kahneman, 2003; H. A. Simon, 1982), in the field of education, there remains

a widely held belief in the possibility of rational, data-driven decisions in schools (Cooley et al.,

2006; Marzano, Waters, & McNulty, 2005; Reeves & Flach, 2011). The best practice era that has

19

dominated the world of teaching, which assumes a set of techniques can be applied to all cases to

garner the same effect, mirrors the zeal in which data-driven decisions are prescribed for

educational leaders. However, bounded rationality holds that regardless of the clarity of the

available data, intuitive, or fast thinking, undercuts rational thought (Kahneman & Tversky,

1984). When individual perceptions filter the interpretation of data, even carefully organized

data-driven decision-making settings are jeopardized (M. E. Toplak, West, & Stanovich, 2014).

In this view, it is not the just the use of data to drive decisions that cause school leaders to

navigate the problem solving required, but rather understanding their own unconscious filtering

of that data that improves decision-making. This critique of neoclassical economics suggests the

emphasis of data-driven decision-making is overly simplistic in its cause and effect view of

gathering and viewing of data and logical choice selection (H. A. Simon, 1955). Viewing teacher

hiring processes through this lens may inform and guide the practice toward improved efficacy.

Biases and heuristics play a significant and often undetected role in the outcomes of

decisions (Hoppe & Kusterer, 2011; Hutchinson, Alba, & Eisenstein, 2010; Kahneman, Lovallo,

& Sibony, 2011). Biases are the natural tendency to think in certain ways (Kahneman et al.,

2011). These patterns of thinking impact the rationality of decisions. Heuristics are simple

practices that quickly produce seemingly adequate answers to difficult questions. Answers that

are derived quickly are more likely to be confidently perceived as accurate by both the thinker

and those around him or her (Kahneman & Tversky, 1984). In daily life, these quick thinking

patterns are useful. It is the impact on situations that require more rational thinking that is

concerning (Kahneman, 2011; Thaler, 2009). Furthermore, when beliefs and data conflict,

people, even analysts, tend to reinterpret the data so that it conforms to their thinking rather than

updating their beliefs (Hutchinson, Alba, & Eisenstein, 2010). Other specialists, certainly

20

educators, possess a lower percentage of leaders fully cognizant of the rules of statistical

analyses. Bounded rationality (Kahneman, Lovallo, & Sibony, 2011; H. A. Simon, 1991) holds

that it is human nature to prefer cognitive ease and therefore for mental biases and heuristics to

generate a choice preference of the first decision that comes to mind. Affect (Tversky &

Kahneman, 1974), confirmation bias (Tversky & Kahneman, 1974), the halo effect (Tversky &

Kahneman, 1974) and other bias identified within the bounded rationality framework may be the

mental shortcuts that limit the consistency and rationality of the hiring process in lower

achieving schools. Affect biases are decisions based on emotions, on likes/dislikes or how one

feels about something replaces the more difficult question of what one thinks about something.

Anchoring biases are the weighing of one piece of information, often the first information

encountered, too heavily. Confirmation bias causes the searching for evidence that supports the

original belief while ignoring evidence that contradicts it. Understanding the degree to which

heuristics and biases alter the decision-making process of individuals and teams in educational

settings is needed.

This study applies this framework of bounded rationality as studied in areas of business

and medicine to an overlooked group, educational leaders. Any advancement in the area of data-

driven decision-making in educational leadership settings requires a clear understanding of

educational leaders’ data filtering processes and the resulting impact on their rationality.

Individuals’ biases and heuristics can be identified, as can the resultant impact on choices

(Kahneman et al., 2011; Schlag, 2010; H. A. Simon, 1982). The specific biases and heuristics

identified in bounded rationality are exactly those involved in hiring: e.g. affect, anchoring, and

confirmation.

21

Therefore, the researcher seeks to understand the decision-making process involved in the

hiring of teachers through the lens of bounded rationality. Beneficiaries of this investigation

would include educational leaders, principals, specifically, as well as students, teachers, other

administrators, and the community.

In Chapter Two, the researcher examines three streams of literature: the decision-making

as viewed through the lens of bounded rationality, techniques utilized in the attempt to increase

the rationality of decisions in the practice of various fields such as business and medicine, and

teacher hiring practices. The review of this literature examines the applicability of the framework

of bounded rationality to the understanding of administrators’ experiences in teacher hiring.

22

Chapter II: Literature Review

In this literature review, the researcher examines three streams of literature. The first is

the decision-making as viewed through the lens of bounded rationality, which holds that

rationality is limited by the availability of information, time, and mental capacity (H. A. Simon,

1955). The second stream includes techniques utilized in the attempt to increase the rationality of

decisions in the practice of various fields such as business and medicine. Finally, the researcher

examines research on teacher hiring practices in order to understand whether teacher hiring

decisions may also be similarly affected by bounded rationality and ameliorated by awareness

techniques or decision-making tools such as those utilized in other fields.

Bounded Rationality

Bounded rationality, first coined by Simon (1955) offered a response to the dominant

theory of the time, neoclassical rationality, which held that human decisions were calculable,

rational responses with the greatest utility as the result (Jevons, 1888). Simon suggested humans

responded not with the most optimal solutions, but the most readily satisfying (H. A. Simon,

1982). Additional Nobel awarded work conducted by Kahnemann and Tversky (1984) identified

the specific mental shortcuts or cognitive biases and heuristics most involved in fast thinking.

Their work, compiled in Thinking: Fast and Slow (Kahneman, 2011) establishes a view of the

human mind as one that defaults to quick, subconscious choices, but with awareness able to slow

and shift to the more rational abilities. Quick thinking allowed humans to survive through the

immediate dangers of the millennia and continue to be useful for eminent danger such as the

demands of driving. However, the important decisions of modern man do not entail the

immediacy of sabretooth tigers, but the complex and long range strategies required for such

endeavors as building enterprises, educating youth, or curing illnesses. Heuristics are simple

23

mental functions that allow for quick decisions and cognitive biases are the illogical results that

often are caused by utilizing heuristics (Kahneman, 2011; Medin & Bazerman, 1999). Present

work in bounded rationality continues to define the fields of psychology and economics and the

understanding of human rationality focusing on areas such as forecasting (Frederick, 2005)

confirmation bias (Rabin, 2013), and brain and addiction research (Bernheim & Rangel, 2004).

Identified biases and heuristics. The ability to avoid errors through the check and

balance of the thinking System 1 and 2 is limited by time, mood, and mental distractions

(Kahneman, 2011). Central to this area of research is the specific identification of cognitive

biases and heuristics and the manner in which these impact decisions. Of the many biases and

heuristics identified by the seminal work of Tversky and Kahneman (1974), most relevant to the

study of decision-making in educational settings are: affect, anchoring, availability,

confirmation, halo effect, missing data, prediction overconfidence, and sunk cost.

Affect bias causes decisions to be based on emotions or on likes/dislikes. How one feels

about something replaces the more difficult question of what one thinks about something

(Tversky & Kahneman, 1974). This could cause administrators to select candidates based on

adult to adult likeability rather than teacher efficacy.

Anchoring causes the weighing one piece of information too heavily, often the first

information encountered. (Hoppe & Kusterer, 2011; Kahneman, 2011; Sunstein & Thaler, 2003)

When evaluating people, such as in the hiring process, this results in a Halo Effect (Lovallo, D.

& Sibony, O., 2010; Singh, 2008; Timmons, 1998) which occurs when someone is liked, due to

one characteristic such as attractiveness, and his/her negative characteristics are therefore

overlooked. This could cause administrators to select candidates with shortcomings related to the

abilities as teachers that should have prevented their selection.

24

Availability, also known as saliency, is the tendency to overestimate the likelihood of

events when similar past events come to mind (Cioffi & Markham, 1997; Tversky & Kahneman,

1974) An example of this would be to fear the ocean due to likelihood of shark attacks when an

event can be recalled and to overlook the much more likely event of drowning due to unseen, but

common riptides. This could cause administrators to select candidates based on the

administrator’s specific knowledge base. If the candidate is a graduate of a college from which

the administrator had a previous positive experience, the teacher may be overly favored to

candidates that attended superior, but lesser known colleges regardless of other compelling

information.

Confirmation bias causes the searching for evidence that supports belief and ignoring

evidence that contradicts it (Beshears & Gino, 2015; Tversky & Kahneman, 1974). This could

cause an administrator to focus on evidence that supports a preconceived preference or aversion

to a candidate. Rather than comparing all candidates on their demonstrated abilities in all

domains of teaching, confirmation bias may cause an administrator to focus on the one domain

his/her preferred candidate most demonstrated and ignore the evidence that suggest another

candidate to be more suitable.

Missing data bias is the tendency to ignore the lack of available data needed for rational

decisions (Henningsen & Henningsen, 2007; Tversky & Kahneman, 1974). The interview

process is riddled with opportunity to miss important data. If an administrator ignores missing

information, candidates about whom little is known may be favored over those better known.

This could cause an administrator to neglect to conduct thorough reference checks.

Prediction overconfidence is the subjective confidence level that is not supported by

objectivity (Campbell, Goodie, & Foster, 2004; M. Toplak et al., 2011; Tversky & Kahneman,

25

1974). This could cause an administrator to mistake confidence in a decision for evidence of

rationality of the decision and thus offer a contract to a teacher who has not provided evidence to

support predictions of effective teaching. One example might be the assumption that a successful

youth league sports coach would be an equally successful physical education teacher.

Sunk cost effect, also known as loss aversion, is the phenomenon in which people justify

increased investment of time or resources into a decision, based on the cumulative prior effort

(Tversky & Kahneman, 1974). This could cause administrators to select candidates who at the

end of a lengthy screening process may not be strong candidates, but the administrator is loath to

reopen the hiring process due to the time already invested in finding the first candidate.

Bounded rationality’s effect on decision-making. Sterman (1989) extended research on

the impact of biases and heuristics in individual decision-making to the implications for

organizations. Work since continues to explore the construct between the benefits of heuristics

that allow for decision-making and their potential to shortcut rational decisions (Kahneman,

2003; Kahneman et al., 2011; Waddell & Sohal, 1994). Sterman’s (1989) empirical study

measured business leaders’ heuristics including anchoring and adjustment. Anchoring is the

establishment of mental frames of value that impact subsequent choices. Adjustment is the term

used to describe the failure to shift mentally from anchoring values, such as basing the value of a

stock on the original purchase price. Sterman identified individuals’ biases and heuristics and

their resultant impact on decisions. In doing so, he demonstrated the weakness in traditional

decision-making theorists (Jevons, 1888) who viewed decisions making as a rational outcome of

data input.

As organizations become increasingly complex, so do the decisions. However, empirical

studies (Eisenhardt & Zbaracki, 1992; Kahneman et al., 2011; Nobre, Tobias, & Walker, 2009;

26

H. A. Simon, 1982) demonstrate that subjects are insensitive to the feedback from their

decisions. This impacts the function of the organizations tremendously. Insensitivity to the

complex surroundings distances the decision makers from reality. Insensitivity to data impedes

the very essence of data-based decision-making. The findings which support the framework of

bounded rationality have been applied to decision-making in organizational settings and have

forwarded the importance of understanding the construct of biases and heuristics.

An important element in decision-making is forecasting. Eisenhardt (1992) questions

whether, due to the complexity of decisions, rationality is even an achievable goal. As world

complexity increases, so does forecasting difficulty. Additionally, forecasting is often done in the

manner most comfortable to the leaders involved and even steered toward the predictions that are

most favorable to them as individuals (Waddell & Sohal, 1994). Those leaders with pioneering

and innovative (PI) mindsets tend to choose more innovative choice options (Manimala, 1992).

Busenitz (1997) suggests that in some situations, overconfidence and representativeness are

actually beneficial in entrepreneurial decisions, but may not be in managerial ones. Successful

entrepreneurs select different courses of action than successful managers. Again, outcomes are

dependent on the individuals, not the data set.

Numerical data is often filtered by cognitive heuristics. Research on trend-based

inferences made by experienced marketing managers demonstrates that graphical data displays

greatly impact the conclusions drawn (Hutchinson et al., 2010). Therefore, data and data

representation tools that should improve decision-making can also mislead. Worse, since data is

involved, the decision makers are more likely to overestimate the accuracy of the choice

(Kahneman & Tversky, 1984). What should be an aid can, consequently, be doubly a hindrance.

Very basic erroneous casual relationships are often drawn when reviewing data (Imai &

27

Yamamoto, 2010). The misinterpretations generated in the study by Imai and Yamamoto (2010)

were made by those well-versed in statistical data. Other fields, certainly education, possess a

low percentage of leaders fully cognizant of the rules of statistics.

An extension of the problem caused by misinterpretation of data is the issue of

information gap. When information is missing from a data set, people often infer the value of

missing information based on the value of the known information, sometimes bolstering or

diminishing the value (Henningsen & Henningsen, 2007) -- assuming the groups realize there is

missing information. When groups meet to discuss and make decisions, it is the shared, not the

unique perspectives, that are given the greatest deference (Kahneman et al., 2011). Therefore,

increasing the number of members of a group does not necessarily increase the number of

choices to be considered. Information that is unknown to all is not discussed and information that

is known to just a few is devalued (Henningsen & Henningsen, 2007).

The specific mental shortcuts of biases and heuristics can be viewed through the related

lens of prospect theory which highlights three regularities in decision-making: avoiding loss,

noticing change more than absolutes, and allowing estimations to be biased by anchoring

information (Kahneman & Tversky, 1984). The descriptive model of prospect theory holds that

individual preferences are not consistent and the various outcomes are related to the manner in

which the choices are presented.

Carefully considered decisions utilize what Kahneman calls, System 2, or slow thinking.

This effortful, concentrated thinking corrects the quick assumptions of System 1, which is the

human intuitive, effortless, or fast thinking (2003; Kahneman, 2011). A question from the

Cognitive Reflection Test (CRT) introduced by Frederick (2005), illustrates the working of the

two systems. A bat and a ball cost $1.10. The bat costs one dollar more than the ball. How much

28

does the ball cost? (Frederick, 2005 p. 25) The quick thinking of System 1 selects the first

seemingly acceptable answer, $0.10. Only in engaging System 2 is the quick answer exposed as

incorrect and the correct answer of $0.05 for the ball and $1.05 for the bat realized. For if the bat

costs a full dollar more than the ball, a $.10 ball would require a $1.10 bat and therefore a sum of

more than $1.10. Only a $.05 ball and a $1.05 bat would equal $1.10. Humans have survived

through the millennia because most of life-threatening challenges require quick, intuitive

thinking which System 1 offers. However, complex decisions benefit from the careful

examination of System 2.

Critics of the framework. Bounded rationality grew out of criticism of rational choice

theory (Botterill & Hindmoor, 2012; Mumby & Putnam, 1992; H. A. Simon, 1982). Critics of

bounded rationality suggest that even if bounded rationality alters outcomes, attempts to

overcome bounded rationality may slow decision-making processes thus impeding individual

and organizational reaction time. Alternative critiques included feminist perspectives that

bounded rationality does not remove the patriarchal overtones found in rational choice theory

and specifically theorists such as Mumby & Putnam (1992) call for a theory that connects

rationality and emotionality. Others, such as Mackenzie (1997), offer sympathy toward the

theory of bounded rationality, but critique the applicability of some of the experimental

evidence. This research would allow for expanded understanding of the applicability of bounded

rationality in a study.

Summary. To improve a system, such as education, the sense making of the individuals

within the system must be understood (Jopp, 2005). An important focus of study within systems

theories examines the cultures or systems that support sense making and learning (Schwandt,

2005); however, even the most functional system would be impeded by limitations to individual

29

members’ rational thought caused by biases and heuristics (Kahneman et al., 2011; Sunstein,

Kahneman, Schkade, & Ritov, 2002). The seminal work in the stream of literature focused on

biases and heuristics is that of Sterman (1989) which examined the impact of biases and

heuristics in decision-making of individuals and the implications for organizations.

Sterman’s empirical study (1989) measures business leaders’ heuristics including anchoring,

establishing frames of value that impact subsequent choices, as they manage stock portfolios,

and challenges the central premise of data-driven advocates as it demonstrates that data is

misinterpreted by experts. Although bounded rationality has been a major theory used to

understand decision-making of leaders in other fields, particularly medicine and business, the

alternate lens advocating for data-driven decision-making has dominated educational leadership

research. As the research in bounded rationality demonstrates, the shortcomings of human

thinking are constant across fields and expertise (Botterill & Hindmoor, 2012; Gigerenzer, 2010;

H. Simon, 2000; Wilson, 2010). The very thinking of the leaders involved in educational

decision-making deserves an examination equal to the work offered the thinking of leaders in

other fields.

Decision-making Tools

Kahneman demonstrates that measures can be made to improve the rationality of

decision-making by engaging the System 2 portion of his thinking model. For example, framing

can be used not only to decrease, but also to increase the accessibility of information that might

otherwise be overlooked (2011). Altering the surroundings, cues, and information associated

with decision-making settings can improve the rationality of the decisions made (Adams &

Ericsson, 2000; Bacon, Fulton, & Malott, 1983; Kahneman, 2003; Schlag, 2010; Thaler, 2009).

This is a tremendously important consideration for educational leaders.

30

Decision-making tools used in business. Recent research (Botterill & Hindmoor, 2012;

Halevy & Chou, 2014; Kahneman et al., 2011; Lovallo, D. & Sibony, O., 2010) explores the

manner in which bounded rationality can be discovered and reduced in order to improve the

decision-making process. Business leaders consider behavioral economics when predicting the

actions of other such as investors and consumers, but they are less likely to consider their own

thinking processes (Botterill & Hindmoor, 2012). Senior executives are particularly at risk for

saliency biases, giving too much importance to past events (Lovallo, D. & Sibony, O., 2010).

The decision-making process is more important than the data itself when attempting to “debias”

strategic decisions (Kahneman et al., 2011; Lovallo, D. & Sibony, O., 2010). Selecting and

analyzing data to make predictions of future behavior of customers relates to educational leaders’

attempts to collect data in order to predict a candidate’s future behavior as a teacher.

It has been shown that business executives can reduce the impact of biases such as

anchoring, weighing one piece of information too heavily, and confirmation, ignoring evidence

that contradicts one’s beliefs (Kahneman et al., 2011). “But knowing you have biases is not

enough to help you overcome them. You may accept that you have biases, but you cannot

eliminate them in yourself” (Kahneman et al., 2011, p. 52). However, biases are more readily

noticed in others. Therefore, a properly oriented organization can improve the decision-making

process. This is using System 2, rational thought, to check for erroneous thinking in System 1,

intuition. In decision-making, the focus is on the content of the situation and suggested course of

action. Business executives must receive and comprehend key information from those with

detailed knowledge. Secondly, they must determine if the information is being clouded

intentionally or unintentionally by biases and heuristics. Finally, they must examine the degree to

which their own biases and heuristics are impacting their decision-making. Kahneman and

31

Lovallo (2011) suggest the addition of a systematic review of the process as well. Specifically,

they call for a quality control checklist consisting of 12 driving questions. These are divided into

three categories: the decision makers, the people making the proposal, and the proposal itself.

Additional benefits for decision-making checklists include greater job specification

procedures and self-monitoring interventions (Bacon et al., 1983). In this light, checklists could

be beneficial to individuals as well as groups. In 1983, Bacon et al. asserted checklists to be an

inexpensive and effective means of improving job performance. However, at that time there was

a lack of literature that detailed the checklist components that were most effective. Recent

research (Churchman & Doherty, 2010; Flottorp et al., 2013; Haynes et al., 2009; Kahneman,

2003; Sibbald, De Bruin, & Van Merrienboer, 2013; Singh, 2008; Stubenrauch et al., 2012;

Takala et al., 2011) responds to that gap in knowledge. Checklists work best when these include

task definitions, record responses directly on checklists, and are periodically reviewed by a

supervisor (Flottorp et al., 2013). Research on knowledge based systems also asserts that users

are more likely to adhere to the recommendations of decision-making tools when explanations

are provided along with both feedforward and feedback applications (Arnold, Clark, Collier,

Leech, & Sutton, 2006).

“The sweet spot for quality control is decisions that are both important and recurring, and

so justify a formal process” (2011, p. 59). The use of an agreed upon checklist allows critique

and dialogue to be part of an organization’s climate. If a single individual were asked to fill that

role, social pressure might dissuade her/him. The proponents of this checklist offer

implementation suggestions. The checklist should be used for important, not routine decisions.

When in use, it should be fully used. Kahneman (2011) references the rigor of use of the World

Health Organization (WHO) Surgical Safety Checklist as a model of adherence.

32

Decision-making tools used in medicine. Experts, even highly trained doctors, can

improve their efficacy through the use of checklists and protocols (Fudickar, Hörle, Wiltfang, &

Bein, 2012; Sibbald et al., 2013; Takala et al., 2011). The World Health Organization (WHO)

released a 2008 safety alert and the accompanying WHO Surgical Safety Checklist. The Global

Patient Safety Challenge goal was to reduce the number of worldwide surgical deaths. Consisting

of 19 items arranged in three sections all of which are to be read aloud, the checklist was the

tool. The WHO Surgical Safety Checklist has been found to improve operating teams’ team

communication, the lack of which was cited as (a) leading cause of medical errors (Takala et al.,

2011) and to reduce perioperative mortality (Fudickar et al., 2012). Detractors of the use of

checklists express concerns that experts are be unnecessarily confined by checklists, yet in a

study of electrocardiogram (ECG) interpretation experts, checklist use during diagnostics did not

increase cognitive load; however, it did reduce diagnostic error (Fudickar et al., 2012). Often the

expectation of experts such as doctors is that the formal training they have experienced engrained

a shared knowledge base thus establishing in them a mental discipline that is brought to all

situations. Not only does the safety checklist research indicate that experts are fallible, but that

the inclusion of checklists reduces mistakes of omission while the use increases team

communication (Adams & Ericsson, 2000; Arnold et al., 2006; Bacon et al., 1983; Bekker, 2010;

Flottorp et al., 2013; Funk & Axelrod, 2011; Grantcharov & Reznick, 2009; Haynes et al., 2009;

Sibbald et al., 2013).

The WHO checklist is a model which could be considered in order to determine specific

items within a study on educational checklists. The following are samples of questions from each

of the three sections of the WHO checklist (World health organization: Patient safety.2009) and

the particular relation to this inquiry.

33

The sign-in section of the WHO checklist includes items such as whether the patient

identity and scheduled procedure has been confirmed with the actual patient. This is similar to an

educational checklist including whether the characteristics needed to successfully perform in the

open position have been established and whether the interview questions have been established

to meet identify the correct candidate. Serious medical errors can occur when the actions

performed are not matched to the correct patient. It can be imagined the selection of staffing

should follow the matching of student needs and characteristics of effective teachers. Often

practitioners seek answers before problems are fully investigated (Short & Shindell, 2009). It is

reasonable that educational leaders’ first priority should be the understanding of the needs of

their students (Marzano et al., 2005).

Before start of surgical intervention all team members are instructed to introduce

themselves by name and role. This simple step has been shown to increase communication and

teamwork (Takala et al., 2011). Perhaps this teambuilding step would help in educational settings

as well by supporting individual voices and therefore reducing groupthink, or the poor decision-

making caused by peer pressure, (Janis, 1982 p. 9). This step of the WHO Checklist also includes

a question that asks if all imaging, such as x-rays, has been reviewed. For educators, this relates

to ensuring that all needed data is available and perspectives heard. This could counter

availability, confirmation, and halo effects. Pausing as individuals or groups to explore sources

for missing information can increase rationality (Henningsen & Henningsen, 2007).

The sign out section of the WHO Checklist requires the team to discuss patient recovery

before leaving the operating room. This is similar to a review of decisions held at the end of

educational leadership meetings. It focuses the group on the next steps. This is of critical

34

importance for educational leaders who often leave one discussion only to be entrenched in the

next.

Referring to the WHO Checklist, Kahneman argues that experts should not pick and

choose from questions, but use these systematically. “Using checklists is a matter of discipline,

not genius. Partial adherence may be a recipe for total failure” (2011, p. 60).

Another area of group dynamics that checklists address is that of power dynamics. An

aspect of working with experts is the interplay of deference to expertise and alternately the need

for checks and balances. Often in the medical field, nurses express trepidation to speak out when

they disagree with a doctor’s choice (Churchman & Doherty, 2010). The protocol of the WHO

checklist places the charge nurse in a position of power that better distributes the responsibility

to utilize best practices and, therefore, the checklist serves to make questioning a procedure

rather than the act of a provocateur (Kahneman et al., 2011).

Decision-making tools used in education. Bartels and Mortenson (2005) examined

decision-making tools, checklists and performance feedback utilized in educational settings,

specifically, the use of these tools in adherence to problem-solving methodology in middle

school Intervention and Referral Service (I&RS) Teams. New Jersey statute (NJ DOE, October

2014) requires every school to maintain an I&RS Team to support the efforts of general

education teachers to support students experiencing learning, behavior, or health difficulties.

I&RS Team members examine data on student progress and related interventions instated. From

the collected data, they are to follow problem solving protocols in order to make decisions as to

the recommendations to offer the involved general education teachers. Bartels and Mortenson

(2005) found performance feedback to team members related to adherence to problem solving

protocols was not to be effective until coupled with a checklist.

35

The use of checklists is a very tangible and overt attempt to counter bounded rationality.

If leaders are willing and able to generate a culture of systematic and transparent decision-

making, biases can be minimized (Courtney et al., 2013). But what of less overt manners to

improve decision-making? Can the irrationality of biases and heuristics be limited without overt

discussions?

Subtle decision-making influence. The Cognitive Reflection Test (CRT) has been used

to identify suboptimal decision-making, but it has also been used to demonstrate means to

improve decision-making (Campitelli & Gerrans, 2014; Frederick, 2005). Using one of the items,

the ball and bat question, some participants were given clear copies, others were asked to furrow

their brows while reading and others were given degraded fonts. The latter two groups had

higher rates of accurate answers (Oppenheimer, 2012). The researchers posit the visual challenge

of the degraded font and the furrowing of the brow engaged the critical thinking power of

System 2. The shift from the quick thinking to more careful consideration can be initiated even

without the awareness of the thinker (Ménard, 2010; Schlag, 2010; Thaler, 2009). The idea of

subtly nudging people toward better choices is examined in the arena of choice architecture or

libertarian paternalism (Schlag, 2010; Thaler & Sunstein, 2003).

Sunstein and Thaler propose an approach that is less obvious than checklists, this is

choice architecture, “policies selected with the goal of influencing the choices of affected parties

a way that will make those parties better off” (2003, p. 175). If bounded rationality’s negative

impacts on reasoning are unnoticed, choice architecture designs opportunities to support rational

thought. With a focus on helping people make choices in financial markets and citing the work

of Kahneman and Tversky (1984) on bounded rationality, Thaler and Sunstein (2009) argue that

people do not always make decisions that are in their best interest and it is the role of

36

organizations to help them do so. They suggest that organizational decisions always have an

element of paternalism. They use an example of a cafeteria to illustrate the point. Should not

cafeteria workers place the fruit before the desserts? This still allows for choice, the libertarian

aspect, while protecting individuals from harm, the paternalistic aspect. Rumble strips are nudges

to control driving speed. A nudge is “any aspect of the choice architecture that alters people’s

behavior in a predictable way without forbidding any options or significantly changing their

economic incentives” (Thaler, 2009, p. 6). For Thaler and Sunstein, it is not a question of

whether organizations should act in paternalistic ways, but rather how to do so. “How should

sensible planners (a category we mean to include anyone who must design plans for others, from

human-resource directors to bureaucrats to kings) choose among possible systems, given that

some choice is necessary” (2003, p. 178)? Much of their work focuses on default choices.

Default options appear on numerous forms. A new employee might see an option of

whether to enroll in a 401K program or a certain insurance program. A driver’s license form

will often include a question about whether to join an organ donor program. When faced

with choices, people are more likely to select the default options (Thaler, 2009). Libertarian

paternalism offers choice, but particularly when expert knowledge is required, the argument

is that the default to be chosen should be the one with the greatest likely benefit (Ménard,

2010). Through cost-benefit analysis, three queries are suggested. First, consider what the

majority would select if the choices were fully understood. Second, consider an approach

that would require people to make an explicit choice. Third, consider an approach that

would have the fewest number of people later changing their decisions (Thaler, 2009).

A balance must be maintained so that the actions in the name of libertarian

paternalism do not reduce the amount of available information and the ability of decision-

37

making on the part of the individuals as that might reduce social learning (Carlin, Gervais,

& Manso, 2013). The other balance point is that as the name suggests, the participants

remain free to choose and as the research on human learning detailed above, human choice

is fallible. Libertarian paternalism is a prescription for support decisions regarding financial

decisions and may not be applicable to understanding fully the motives that lead to the

decisions (Mackenzie, 1997) or to arenas outside of financial decisions. Much of the

available research in the stream of bounded rationality is focused on economics. The stream

of literature supporting the use of data-driven decision-making in education does not

address the issues of bounded rationality.

Summary. Using decision-making tools such as researched interview protocols improves

the rationality of the decisions of experts (Bacon et al., 1983; Bartels & Mortenson, 2005;

Brañas-Garza, García-Muñoz, & González, 2012; Cioffi & Markham, 1997; Fudickar et al.,

2012; Haynes et al., 2009; Kahneman, 2003; Place & Vail, 2013; Sibbald et al., 2013). However,

experts in fields of business, medicine and education often ignore protocols (Campbell et al.,

2004; Engel, 2013; Fudickar et al., 2012; Grantcharov & Reznick, 2009; Guerra-López & Blake,

2011).

Missing from the literature is an examination of the impact of cognitive biases and

heuristics in educational settings and whether the decision-making tools such as those used

in business and medicine might ameliorate the deleterious effects of bounded rationality.

This proposal builds a case for additional research into the area of decision-making

in educational leadership and specifically whether the negative impacts of natural biases and

heuristics on decision-making can be noted in the decision-making of educational leaders,

specifically, in the hiring of teachers. Decision-making tools such as checklists have been

38

used to improve decision-making in the fields including business and medicine without

hampering expert decision-making speed (Bacon et al., 1983; Bekker, 2010; Fudickar et al.,

2012; Haynes et al., 2009; Kahneman et al., 2011; Sibbald et al., 2013). This researcher

examines the thinking process of educational experts in order to categorize the biases and

heuristics that are most commonly involved. Further, this researcher seeks to understand if

rational decision-making could be made more overt and therefore manageable. If a checklist

or flowchart could be effectively applied when educators engage in complex decision-

making, specifically, teacher hiring, improved decision-making could be the result.

Decades of research in bounded rationality offers a number of claims regarding common

decision-making patterns (Botterill & Hindmoor, 2012; Eisenhardt & Zbaracki, 1992; Kahneman

et al., 2011; Langlois, 1990; Mumby & Putnam, 1992; H. A. Simon, 1982; Sunstein et al., 2002).

People tend to make decisions based on what they like, not according to a set of rules. With

substitution, the decision-maker converts the choices from questions about which is better to

which is liked more by the decision maker; thus substituting the easier question of which one is

preferred in place of the harder question of which one is better in some more objective sense.

This substitution can be noticed in the response to value questions, such as which teaching

candidate is most likely to be effective. It may seem in an important decision such as hiring this

would not occur, but experiments indicate that large stake decisions do not activate “rule

thinking” more than low stake decisions (Campbell et al., 2004; Kahneman et al., 2011; Kunc &

Morecroft, 2010).

Finally, it should be noted that although the focus of this stream of literature has been on

the filtering and interpretation of data, bounded rationality equally alters the collection of the

data to be reviewed and the manner in which analyses are shared (M. E. Toplak et al., 2014). The

39

first two streams of literature, bounded rationality and decision-making tools, expose the many

ways in which human thought filters, often erroneously, the logical conclusions of data sets.

While specific biases and heuristics interfere with rational decision-making, these can be

identified and ameliorated.

Hiring Practices

Citing research which includes Gentry (2007) and Stronge and Tucker (2000),

Schumacher, Grigsby, and Vesey (2015) assert the factor which most impacts student learning is

teacher effectiveness. Schools are only as good as the staff that work there (Nonaka, 1994; Place

& Vail, 2013); however, the hiring practices used to employee staff vary greatly and the criteria

for selection are often based on feeling over evidence, further, hiring is often “information poor”

(Liu & Johnson, 2006 p. 331). Even beyond the multitude of decisions such as those involved

with building management, curricular support, course scheduling, and budgeting, teacher hiring

may be the most important decision made by school administrators (Cranston, 2012; Pillsbury,

2005). However, the weight of hiring decisions do not receive the attention equivalent to the

import. As Whitworth (2016) states, “Given the time constraints and responsibilities of school

administrators, the teacher selection and employment process occurs quickly and involves a

relatively restricted amount of information and data” (p. 20). In order to better understand this

often ignored, yet vital process, this study examines the two main steps in teacher hiring:

screening and interviewing.

Screening. Compounding the challenge of limited research into teacher hiring has been

an overlapping definition of screening and interviewing process in the research that has been

conducted (I. P. Young & Delli, 2002). Additionally, research available on teacher hiring is far

more likely to focus on the paper screening process as that can be accomplished through surveys

40

and controlled experiments (I. P. Young & Delli, 2002). The first step in the teacher hiring

process is the paper review of applicants. Documentation may include proof of certifications,

resumes, collegiate records, application responses, and letters of reference. Timmons (1998)

argues the specific needs of the position should be generated prior to the start of the screening

process. This would focus the screener(s) on the most important first round criteria. After the

initial paper review is conducted to sort out candidates without proper certification and

qualifications, the remaining candidates should be screened in relation to the criteria determined

by the administrator or in cases of a selection committee, by the selection committee (Heneman

& Milanowski, 2004; Timmons, 1998). This research employs a qualitative study design to

examine the degree to which working administrators enter the hiring process with a list of

criteria and if so the manner in which these characteristics are weighed and applied.

Research (Haskins & Loeb, 2007; Rutledge et al., 2008) has shown the weight items are

held as evidence of teacher potential differ and therefore the pool of candidates gathered for the

next round, interviewing, differ. What is valued as predictors of effective teaching varies.

Although undergraduate GPAs and portfolios are ranked as important by most teacher

preparation program providers, administrators rated written quality of the resume and cover

letter, experience, and known references as key elements in the screening process (Cranston,

2012). As in all decision-making, biases and heuristics are present in the thinking of the

individuals gleaning candidates from paper documents. An example of this could be a hyper

focus on transcripts. Criteria such as advanced degrees have not been shown to be good

predictors of teacher effectiveness (Engel, 2013), yet the data point may be used in confirmation

bias to as evidence of a candidate preferred for another reason. Bias and heuristics are involved

41

in the filtering of the candidate pool of teachers as these are in the hiring process of other fields

(Cranston, 2012; Place & Vail, 2013).

Interviewing. Young and Delli (2002) found no more than two studies specifically

addressing teacher selection. They cite Bolton (1969), which used information masking about

teacher candidates in the selection interview and I. P. Young (1983) who altered the format of

teacher hiring practices to include panel interviews. A small amount of additional research on

teacher hiring has been produced since 2002.

Of all the steps in the teacher hiring process, principals place the highest importance on

interviews (Rutledge et al., 2008), above that of resumes and reference checks, second and third

most important respectively (Cranston, 2012; Rutledge et al., 2008). However, “Interviews are

wrought with inherent biases that interviewers hold relative to the candidate’s appearance,

gender, age, and non-verbal cues” (Cranston, 2012 p. 362). To control for interviewer bias,

researchers (Engel & Finch, 2015; Haskins & Loeb, 2007; Heneman & Milanowski, 2004;

Lange, Range, & Welsh, 2012; Schumacher et al., 2015) have recommended job specific criteria

and interview protocols that focus administrators on specific characteristics be utilized.

Teacher interviews focused on specific job-related questions have been shown to be more

valid (Schumacher et al., 2015). Stronge (2003) suggests specific questions delve into the

characteristics noted within his Domains of Teacher Effectiveness model such as subject

knowledge, classroom management, assessment, and motivation (p. 48). This research examines

the literature supporting the assertion that such a focus is widely utilized and even if so would be

effective since candidates are more often judged by interviewing skills than pedagogical skills

(Engel, 2013; Peterson, 2002). Often likability or the similarity of the new hire to the displaced

teacher is given the greatest consideration (Rutledge et al., 2008).

42

According to Haberman, “School districts need to use validated interviewing instruments

to determine the likelihood that the young adults they hire will be effective and remain in

teaching for 5 or more years” (2012 p. 939). Experimental studies of teacher hiring have

examined the independent variables of hypothetical candidate age, gender, and experience (Place

& Vail, 2013; I. Young & Joseph, 1989). These variables were compared to the administrators’

demographics and their ratings of the hypothetical candidates. One interesting finding was that

although experienced candidates were preferred over those with three or fewer years, generally

younger teacher candidates were preferred by administrators in urban districts (I. Young &

Joseph, 1989). This outcome of this experimental study suggests administrators bring

preferences to the characteristics of the candidates they desire to contract. This is significant as

experience has been shown to improve teacher effectiveness; teacher preparation or youthful

energy have not (Harris & Sass, 2011; Rutledge et al., 2008).

Commercially produced interview protocols have not been shown to improve the focus

on teacher practice as these tend to focus on personal attributes (Schumacher et al., 2015).

Research on teacher hiring that mirrored occupational research has demonstrated structured

interview processes specifically aligned to the domains of teaching to be effective, but these are

often not used (Clement, 2000; Rutledge et al., 2008). Clement (2013) suggests ongoing,

systematic hiring processing which includes behavior-based interviews, as past behavior is the

best predictor of future performance. Interview questions of this approach begin with phrases

such as “how have you…,” or “tell about a time when…” This roots the responses to

experiences rather than hypotheticals. The questions focused on specific areas of teaching:

designing instruction, classroom management, implementing instruction, and monitoring student

progress should be scored individually on a scale from no experience or preparation to excellent

43

experience or preparation (Clement, 2013; Cranston, 2012; Rutledge et al., 2008; Schumacher et

al., 2015). Additionally, to ensure consistency, the interviewer should balance the control of the

interview so that all questions receive a thorough response, but not so much as to give away the

expected answers to the questions (Timmons, 1998).

As principals have been shown to focus on personal qualities such as enthusiasm and

communication (Rutledge et al., 2008), and likeability (Schumacher et al., 2015), and as such

characteristics are more likely noted in interviews, other data collection e.g. demonstration

lessons are noted widely adopted techniques (Hoffman, 1995).

Comparing the hiring records of new principals to those of his/her predecessors,

individual principals found to be more likely to hire effective teachers did so consistently (Papa

& Baxter, 2008). The study also found principals shown to be less able to hire effective teachers

demonstrated poor selections consistently. Further, effectiveness in hiring was not found to be

tied to the school’s urbanicity or the characteristics of the principals such as age, gender, or

experience (Papa & Baxter, 2008). By comparing principal hiring practices to those of

predecessors in the same schools, the importance of individual principal competency is

highlighted as is the need to understand individual principals’ experiences in the hiring process.

Bounded rationality research (Kahneman, 2011; Langlois, 1990; H. A. Simon, 1982; H.

A. Simon, 1991; Sunstein et al., 2002) highlights the detritus effect normal personalities have on

decision-making. One example of this is that leaders tend to hire those who they believe to be

like them or those who are willing to agree with them (Singh, 2008). Over time, this hiring

pattern can negatively impact the decision-making process of the institution as more “yes men”

are assembled. What may appear to be a majority opinion garnered through a logical decision-

making process might just be the predictable result of polling likeminded individuals. Studies of

44

narcissism and its impact on decision-making habits show overconfidence, or the inflated

subjective probability of certain outcomes, leads to a willingness to continue to make losing bets

regardless of the diminishing returns (Campbell et al., 2004). In one of the multiple studies

conducted by Campbell (2004), narcissists are more likely to make risky bets and have lower

performance on betting tasks. Additionally, although no better answering trivia questions, they

rated their confidence in their accuracy to be higher than their peers. The “don’t confuse me with

data, I know what I am doing” attitude, is not surprisingly attributed to narcissists, but can be

noted in decisions that are based on “gut” rather than careful examination (Oppenheimer, 2012 p.

239). Kendrick and Olson (2012) suggest anyone who possess feelings of expertise in their field,

school administrators would fit this category, and will likely view their intuitions as valid

sources of knowledge. The fast thinking illustrated by Kahneman (2011) is reinforced by a

feeling of correctness. It is the ease with which people are drawn to the erroneous answer in the

ball and bat test question (Frederick, 2005) that simultaneously generates a feeling of gut

confidence and lowering of engagement of the slow thinking rational mind (Kahneman, 2003).

Summary. Researchers such as Engel & Cannata (2015) indicate some achievement gaps

between high achieving and low achieving schools are tied to hiring practices. Perhaps due to the

higher turnover associated with lower achieving schools, there is a tendency to use shorter

interviews, hire staff from within the school and near or after the start of the school year, rather

than conducting full rounds of the interview process (Engel & Finch, 2015; Papa & Baxter,

2008). In the conclusion of their study, Rutledge, et al (2008) suggest an area of study needed is

evaluation of the process and tools used in teacher hiring. Protocols such as the use of interview

questions structured around domains of effective teaching have been shown to offer superior

hiring decisions, yet great variety in teacher hiring still persists.

45

In addition to hiring based on likeability, politics and nepotism is a far-reaching context

that impacts hiring and retention decisions (Major, 2013). Some hiring decisions are not based on

demonstrated teacher effectiveness, but instead teacher candidate’s ability to coach athletics or

connections within the community (Murnane & Steele, 2007). Organizations are political

systems in that these are a collection of people with differing goals and ideals (Eisenhardt &

Zbaracki, 1992) and therefore decision-making processes within organizations are influenced by

more than just the available data. Cranston (2012) suggests “administrators at all levels re-

consider their current hiring practices because although intuitive in appeal, they may lack an

evidentiary and/or theoretical basis that is consistent with reliable and valid metrics that help

identify potentially effective teachers” ( p. 360).

Conclusion

It is known that teacher hiring is one of the most important tasks for principals (Clement,

2013; Mason & Schroeder, 2010; Peterson, 2002; Rutledge et al., 2008). However, it is also

known, that rational thinking is not the human default position (Kahneman & Tversky, 1984; H.

A. Simon, 1955; M. Toplak et al., 2011). Finally, it is known that protocols and checklists can

improve expert decision-making (Flottorp et al., 2013; Haynes et al., 2009; Kahneman, 2003;

Stubenrauch et al., 2012; Takala et al., 2011). What is not known is whether the specific biases

and heuristics noted in the decision-making of experts in other fields can be identified in the

hiring practices of educational experts and the degree to which principals express the need or

interest in decision-making tools for the hiring process.

“Organizations need to realize that a disciplined decision-making process, not individual

genius, is the key to a sound strategy. And they will have to create a culture of open debate in

46

which such processes can flourish” (Kahneman et al., 2011 p. 60). This research is needed to

understand Kahneman’s assertion in the context of educational organizations.

The impact of bounded rationality on decision-making and the use of decision-making

tools in fields such as business and medicine have been deeply studied. What is missing from the

literature is a deep understanding of the impact of bounded rationality on teacher hiring.

Research central to the theoretical framework of bounded suggests that the use of structured

protocols that address pitfalls in human thinking would improve the rationality in hiring

decisions (Bingham & Haleblian, 2012; Guerra-López & Blake, 2011; M. Toplak et al., 2011).

The impact of bounded rationality on even the best case scenario for teacher hiring practices

requires the attention of educational leaders. Suboptimal decision-making caused by missing

data, the misinterpretation of data, or overconfidence generated by the presence of some data

could undermine the crucial work of the hiring process and, therefore, the quality of instruction

provided to students.

In keeping with the recommendation that qualitative research be contextualized and

focused on the interactions within that context (Creswell, 2012), the proposed setting for this

study is within K- 8 school districts. Chapter three contains the methods for conducting this

research of the process of hiring new teachers among comparable school districts in the mid-

Atlantic region.

47

Chapter III: Research Design

Methodology

The purpose of this study of educational leaders in mid-Atlantic school districts is to

understand the process of teacher hiring and to answer the following research question: How

does bounded rationality affect administrators’ decision-making processes during the process of

hiring new teachers? This researcher explored the challenges involved in identifying effective

teachers during the hiring process. The researcher studied this question using a qualitative design.

In this study, the researcher sought to understand administrators’ beliefs and actions related to

hiring and examined what they choose to do and what they choose not to do as well as their

explanations as to the reasons for those choices. Careful design is essential in order to gain

control in social science research (Shadish & Cook, 1999). This chapter details the methodology

of the case study designed for this purpose.

Research Design

Whereas quantitative research offers measures of variables, qualitative research examines

the expressed thoughts and beliefs of the participants (Creswell, 2012). In this study, the

participants were school administrators, in particular, elementary school principals. Qualitative

methodology offers understanding of the perceptions and experiences of the participants and the

understanding of a phenomenon for which variables are unknown or cannot be manipulated

(Creswell, 2012) and is therefore the appropriate methodology for this research study into the

decision-making process of individual administrators.

Creswell (2012, p.19) lists three factors for consideration when deciding if a research

design should be quantitative or qualitative. The first consideration is the type of research

question. Quantitative research is used to identify trends or explanations, whereas qualitative

48

research is used to generate deep understanding of problems (Creswell, 2012). This research

question addressed the decision-making of individual administrators and was therefore suitable

for a qualitative design. The second consideration, according to Creswell (2012), is the selection

of a research design which will be more readily consumed by the intended audience. This

research was intended to inform the practice of administrators; as a qualitative study that gives

voice to a group of administrators, it may be more readily accepted by other administrators.

Finally, the selected research design should draw on a strength developed through the experience

and training of the researcher (Creswell, 2012).

In addition, decades of quantitative research have already been conducted in the area of

the applied theoretical framework, bounded rationality (Botterill & Hindmoor, 2012; Eisenhardt

& Zbaracki, 1992; Halevy & Chou, 2014; H. Simon, 2000; H. A. Simon, 1982), yet there are few

qualitative investigations. Jenlink (2005) argued that “The scholar-practitioner understands that

theory has a practical intent” (p. 7). This researcher seeks to utilize the theoretical framework of

bounded rationality to serve the practical and crucial work of teacher hiring and extend it by

investigating the biases and heuristics expressed by administrators as they reflect on their

choices.

Furthermore, bounded rationality, the theoretical framework utilized as the lens for this

study, is replete with decades of experimental studies (Creswell, 2012, p. 21), which are

quantitative procedures in which the investigator determines whether changes in variables make

a difference in results for participants. These experimental studies have identified biases and

heuristics in bounded rationality and have offered means to ameliorate these in various fields

(Kahneman, 2011; Sunstein & Thaler, 2003). This researcher elected to utilize a different

approach, a case study model, in order to understand the expressed experiences of decision-

49

makers, particularly of administrators in the teacher hiring process. This type of qualitative case

study approach of the hiring practices of administrators through the lens of bounded rationality is

missing in the literature. Case study accommodates many factors through the review of multiple

sources of evidence, and “benefits from the prior development of theoretical propositions to

guide data collection and analysis” (Yin, 2014, p. 17).

Research Tradition

Originally confused with quasi-experimental design (Shadish & Cook, 1999), case study

is now recognized as a separate method with its own research design and, as such, it offers a

structure with which to select the questions to study, the relevant data to collect, and the way to

collect and analyze data (Yin, 2014). Case study is not confined to specific epistemological

perspectives; either interpretivist or positivist orientations can be applied (Yin, 2014).

Bounded rationality, as a framework, is in opposition to a positivist view that there is a

singular reality and that humans use rational thought in response to the singular reality.

However, bounded rationality is not an interpretivist philosophy either; one in which every

reality is accurate. Rather, when making decisions, humans are likely to misunderstand the

reality that surrounds these decisions. Bounded rationality, therefore, offers a positivist

understanding of the world, but with expectation that humans will not behave in a predictably

rational manner and that individual interpretations of reality will lead to individual decisions

that are not fully rational. According to Schramm (1971), “the central tendency among all types

of case study, is that it tries to illuminate decisions or sets of decision, why they were taken,

how they were implemented, and with what result” (citing Schramm, 1971, Yin, 2014, p. 15).

In this study, the researcher examined the how and why of decisions made in the hiring process.

50

Case studies are used to understand real-world experiences in their context (Yin, 2014, p.

16). As an empirical inquiry that offers a means to examine the boundaries between

phenomenon and context (Yin & Davis, 2007), case study allows this researcher to study

differences and consistencies in individual decision-making across settings. This researcher

investigated individual decision-makers engaged in the process of hiring teachers, an event that

occurs across various schools and districts, each in its own context.

According to Yin (2014), “Case studies address how and why research questions which

focus on contemporary events without behavioral or experimental control of those events” (p.

9). Similar to a history, case study examines past events, but with the addition of observation

and interviews of those involved in the studied events (Creswell, 2012, p. 10). This case study

allowed the researcher to examine the experience of teacher hiring without altering the behavior

of the participants, and therefore, the employment outcomes of actual teachers.

The research tradition of case study, a common case study (Yin, 2014, p. 52), in

particular, provides the preferred design for the research direction of this study of recent

experiences with hiring teachers in sample schools. A common case study examines an

everyday situation, not outliers as in an unusual case study (Yin, 2014, p. 52), rather, this

approach studies the manner in which experiences transpire most often. This approach was

applicable “because of the lessons it might provide about the social processes related to some

theoretical interest” (Yin, 2014, p. 52).

This case study of multiple school administrators offers deep insight into the thinking of

individuals across contexts. This allows for understanding of the decision-making process of

educational leaders as the inclusion of multiple administrators reduces the limitation of single

case studies: the influence of localized thinking and action.

51

Participants

Whereas quantitative research often includes random sampling to identify participants

and sites, purposeful sampling is used in qualitative research to identify the participants and

sites determined to be most helpful to understand the central phenomenon (Creswell, 2012, p.

205). This researcher utilized an initial survey to identify interested participants who were

representative, or typical, of the larger study group, elementary school administrators. The

sites and individuals selected represent varied approaches to teacher hiring, from informal

decision-making to the use of formal protocols. Such typical sampling is one of the

subcategories of qualitative purposeful sampling recognized by (Creswell, 2012, p. 207).

Typical sampling is used in order to understand what is typical or normal; in this case, within

the hiring practices of elementary schools, thus, it is not random. The design of this study did

not require limiting the participant pool by demographics or experience, as bounded

rationality impacts all decision makers (Kahneman, 1984). However, the preferred

participants were administrators who annually screen paper applications, conduct interviews,

and make hiring recommendations to the superintendent. Five principals were interviewed.

All had more than five years of experience hiring teachers. The inclusion of administrators

from multiple districts and schools allows for examination of any contextual impact and

deepen understanding of the decision-making of elementary school administrators in the

process of hiring teachers.

In keeping with the recommendation that qualitative research was contextualized and

focused on the interactions within that context (Creswell, 2012), the setting for this case

study was a sampling of schools in a county in a mid-Atlantic state. The researcher

purposefully selected administrators whose responses to the initial survey represent the range

52

of protocols used in the hiring process, ranging from those without formal procedures to

those with consistently utilized question scripts and scoring systems. This researcher

examined the phenomenon of teacher hiring through a conceptual framework of bounded

rationality, for as Yin (2014) asserts, “The case study is an opportunity to shed empirical light

about some theoretical concepts or principles” (Yin, 2014, p. 40). This understanding was

represented in a sample of a minimum of three sites and five people in keeping with the

typical range of sample sizes in qualitative research suggested by Creswell (2012, p. 209).

Several considerations help establish transferability (Yin, 2014, p. 48). The first

consideration is the degree to which the participants are representative of the larger group and

the range of available documentation. This is essential in order to transfer findings to

individuals in the larger group. The participants in this study were selected utilizing

purposeful, typical selection, which gathers information on representatives of a larger group

(Creswell, 2012, p. 208).

Another important consideration in qualitative studies is the degree to which the

observer’s presence affects what people say or do (Creswell, 2012, p. 436). This researcher

does not hold a position of authority over the participants, so that form of researcher

influence is eliminated.

Recruitment and Access

The gatekeepers (Creswell, 2012), those with the ability to grant or deny access to the

school, in this study of administrators are the participants themselves. As a member of the

larger group of administrators loosely organized under the county education office, after

approval from the district superintendents, the researcher emailed the principals a detailed

invitation to participate in this study. The included a survey (Appendix A) used to gather

53

information on the interested participants, most importantly, whether and to what extent they

are involved in the hiring of teachers.

The recruitment process followed standard practice for qualitative research (Agee,

2009; Butin, 2010; Ponterotto, 2005; Ryan, Coughlan, & Cronin, 2007). This standard practice

included written consent for a survey as well as to participate in the study as guided by the

University of Northeastern IRB. After IRB and superintendent approval, the survey was

distributed via email to elementary school principals in a particular county in a mid-Atlantic

state. Submitted surveys were analyzed so that the research sampled a variety of location and

hiring processes. The researcher selected principals with ranging years of experience, as well as

protocols ranging from established district-wide hiring procedures to individual autonomy.

Selected participants were interviewed at a time and place convenient to them. Protection of

human subjects included voluntary participation, anonymity of the participants and schools,

ability to opt out at any time, and the surveyed and interviewed participants unrelated to the

school district in which the researcher is employed.

54

Data Collection

Figure 1. Data collection process

This study included data gathered through a survey of administrators from a particular

county in a mid-Atlantic state, as well as interviews of the five members selected from that pool of

respondents, and a focus group of members within that smaller group held after the individual

interviews. Yin cites four principles for data collection in a case study: use of multiple sources of

evidence, creation of a case study database, maintenance of a chain of evidence, and care of

electronic data (2014, p.118). This section details the data collection and maintenance of this

study.

Refinement of instrumentation and coding scheme

Data Set 2: Interview analysis, peer review, and member checking

Synthesis of themes

Findings Conclusions Recommendations

Survey review: identify qualifying responses and selection of representative participants

Data Collection Phase 1: Survey

Data Set 1: Survey results analysis, identify typical and range of systems

Data Collection Phase 2: Interviews with selected participants

Data Collection Phase 3: Focus Group with selected participants

55

Data set 1 – Survey of principals (Appendix D). Data was collected through a nine

question Likert scale survey emailed to the pool of participants. Data collected include the

principals’ years of experience as well as the protocols, if any, used in the process of reviewing

applications and interviewing teacher candidates. Further, this data includes the administrators’

expressed beliefs as to the degree to which evidence impacts their final candidate selection. The

survey informed the researcher as to the range of processes and procedures elementary school

principals utilize to interview teacher candidates. This information first served to establish the

pool of candidates and the application of purposeful selection. This information also included the

range of procedural routines utilized and, therefore, established what is typical among these

administrators. Understanding what is typical is needed for the transferability of the findings of a

study (Yin, 2014, p. 48).

The survey, collected through Survey Monkey (Appendix D), permission request

(Appendix A), and participant recruitment letter (Appendix B) were all be distributed via email

directly to principals of a particular county in a mid-Atlantic state. Participants were notified

from the first contact requesting participation the right to refuse to answer any question and to

withdraw from the study at any point. The survey was used to collect information regarding

hiring practices, data set 1, as well as generate the purposeful range of representatives from the

pool of participants for interviews, data set 2. Principals were selected from the range of novice

to experienced as well as the range of use of formal protocols.

Data set 2– Principal interviews (Appendix E). The second data set, the semi-

structured interviews, informed the researcher of the administrators’ decision-making in the

process of hiring of teacher candidates. Specifically, the interviews allowed understanding of

the administrators’ rationale for selection of data points in the processes of screening and

56

interviewing. This data set is the primary set as it is ideal for understanding the individuals’

experiences within a context of a group or system (Creswell, 2012). Whereas the first data set

offers understanding as to the typical established procedure for teacher hiring, these interviews

allowed for understanding of the actual interpretation and use of the procedures as experienced

by administrators.

The individual semi-structured interviews with the principals focused the research on

each principal’s decision-making process during the review of applications and teacher

interviews as well as his/her evaluation of the hiring protocols utilized. Semi-structured

interviews (Creswell, 2012, p.17) were conducted and audio recorded by the researcher

following the researcher’s designed semi-structured interview protocol (Appendix E). Two sets

of principals from two different school district in addition to a principal from a third district

were included in the study in order to collect different perspectives on a process and to allow

for understanding of any contextual influence on decision-making. Each interview length was

approximately one hour and conducted at the location and time preferable to the interviewee.

Paper review. Application, or paper, review is the first step in the candidate selection

process. To understand the principals’ experience reviewing applications, specific questions

and probes were included in the initial survey, q. 3, and in the semi-structured, open-ended

questions interview questions, specifically q. 1, 3 and 9. These focus on the principals’ criteria

used to identify candidates as well as the principals’ reflection of the protocols they used. As

personnel documentations are protected, the researcher asked about the principals’ experience

of the application review, not the specific application materials.

Candidate interviews. This section of the interview process collected the case studies’

principals’ explanations of the criteria they apply when conducting teacher interviews. This

57

included the principals’ description of characteristics and experiences they seek as well as the

manner in which they conduct interviews including style and question content and the degree to

which regulated protocols are used by each principal and across the district.

General processes. This section of the interview process collected the case studies’

principals’ perceptions of their general procedures. This line of inquiry allowed for participants

to evaluate their overall use of hiring protocols and procedures.

After each interview, the researcher transcribed data through a service, Rev.com. The

interview audio files were converted into Microsoft Word files and delivered electronically and

password protected to the researcher who then uploaded the transcribed data into Nvivo, a

qualitative data analysis software application. This software access was also password protected,

as was all reflexive memos created by the researcher.

Data set 3 – Focus Group (Appendix F). As a third data set, a focus group of members

of the eight candidates was held. The themes identified by the researcher as a result of the

analyses of the survey and interviews were presented to the focus group for their consideration.

Focus groups can be an advantageous form of interviewing in a qualitative study particularly

when participants are similar to each other (Creswell, 2012). The participants in this study are all

elementary school principals in a particular county in a Mid-Atlantic state. As is true for

individual interviews, focus group researchers must be mindful of reflexivity (Yin, 2014, p. 112).

This influence dynamic between the researcher’s perspective and the interviewee – and vice

versa – in which the beliefs of one individual can impact the conversation and therefore the

expressed beliefs of others cannot be completely eliminated; however, Yin (2014), in keeping

with the Kahneman (1984) and the theoretical framework of bounded rationality, states

mindfulness to the potential interaction impact allows for improved interview accuracy.

58

All researchers must ensure participants are aware their interviews remain confidential; as

a member of the educational community studied, this researcher is all the more charged with

conveying that confidentiality be upheld. The Informed Consent form was revisited and initialed

at the start of each interview. The researcher collected interview data via written notation and

audio recording and the transcribing service, Rev.com. Probing sub-questions were utilized as

needed to generate elaboration or clarity. Draft transcriptions of interviews were presented to

participants for member checking.

Confidentiality was be protected in data collection, storage and publication. The name of

the state, county, and school districts was not published. Neither were the names of any

participants published. Pseudonyms were used for the schools and all participants.

Data Storage

The data sources for this research include survey results and interviews. All electronic

data was password protected. The password was only known to the researcher. Hard copy

documentation included informed consent agreements for survey data and interview data. Hard

copy documentation was stored in a locked office within a locked cabinet. Only the researcher

has the key to the cabinet. Electronic data included research notes from interviews, researcher

reflexive memos, communications between the researcher and participants. Only the researcher

viewed all data. Participants only had access to the data from their own survey and interview.

Participant confidentiality was retained through the use of naming conventions. Data will be

stored for five years beyond the publication of the dissertation and destroyed upon the conclusion

of that five-year period.

59

Data Analysis

High quality analysis must include all evidence, examine rival interpretations, focus on

the most significant aspect of the research, and utilize the prior knowledge of the researcher (Yin,

2014, p. 168). Therefore, this researcher applied his knowledge of teacher hiring while also

examining any biases his background may generate toward a specific explanation or away from

plausible rival interpretations.

As a qualitative study, this study included “breaking down qualitative data into discrete

parts, closely examining them, and comparing them for similarities and differences” (Strauss &

Corbin, 1998, p. 12; as cited by Saldana, 2009, p. 81). This section describes this researcher’s the

data analysis process.

This researcher engaged in coding as a method to inform analysis, as coding is “not just

labeling, it is linking” (Saldana, 2009 p. 8). Abbott (2004) compares coding to the decoration

of a room, including stepping back and reexamining for patterns and flow (as cited by Yin

2009, p. 9). First and second cycles of coding was utilized until refined categorization was

achieved. Recoding is to be expected in case study research (Yin, 2014). A theme is the

outcome of the analyses that occur during the process of coding, categorizing, and reflecting.

Aligned to the literature review for this study, themes related to the manner in which

decision makers were impacted by the mental shortcuts of bounded rationality and the manner

in which decision makers ameliorate the irrationality of fast thinking human minds. Table 1

lists the selected biases and heuristics identified within the theoretical framework and the

possible impact on hiring decisions.

60

Table 1

Biases and Heuristics and Possible Impact on Hiring Decisions

Biases and

heuristics Effects Possible impact on hiring decisions

Affect bias causes decisions to be based

on emotions or on

likes/dislikes

Is there evidence of teacher hiring based on

adult to adult likeability rather than teacher

efficacy?

Anchoring causes the weighing of one

piece of information too

heavily, often the first

information encountered

Is there evidence of committees selecting

candidates with shortcomings related to the

abilities as teachers that should have

prevented their selection?

Availability causes overestimation of the

likelihood of events when

similar past events come to

mind

Is there evidence of committees selecting

candidates based on the administrator’s

specific knowledge base rather than from a

full range of needed information?

Confirmation

Bias

causes the searching for

evidence that supports belief

and ignoring evidence that

contradicts it

Is there evidence of individuals selecting

candidates based on the one teaching

domain his/her preferred candidate most

demonstrated and ignore the evidence from

the other teaching domains?

Missing Data

Bias

is the tendency to ignore the

lack of available data

needed for rational decisions

Is there evidence of committees selecting

candidates about whom little is known over

those better known?

Prediction

Overconfidence

is the subjective confidence

level that is not supported

by objectivity

Is there evidence of committees selecting

candidates based on unsubstantiated

predictive tools such as experience as a

coach will cause the candidate to be a

better teacher than those who have not

coached?

Sunk Cost

Effect / Loss

Aversion

is the phenomenon in which

people justify increased

investment of time or

resources into a decision,

based on the cumulative

prior effort

Is there evidence of committees selecting

candidates who at the end of a lengthy

screening process may not be strong

candidates, but the administrators resist

reopening the hiring process due to the

time already invested in finding the first

candidate?

Table 2 overviews the analysis of each of the three data sets. Data set 1, the survey, were

considered through a quantitative analysis of the Likert scale and multiple choice responses. Data

61

sets 2 and 3 were coded in two cycles each beginning through Pattern matching and through In

vivo coding.

Table 2

Summary of Data Collection and Analysis

Research Question Data Source Analysis

How does bounded

rationality affect

administrators’ decision-

making processes in the

process of hiring new

teachers among comparable

school districts in the mid-

Atlantic region?

1. Data set 1 -

Administrator survey

1. Cycle 1: quantitative

analysis of Likert

responses

2. Data set 2– Case study

interviews

3. Data set 3 – Focus group

interview

2. Cycle 1: Pattern Code

Cycle 2: In vivo Code

3. Cycle 1: Pattern Code

Cycle 2: In vivo Code

Data set 1: Survey

The data set of responses gathered through the survey questions (Table 3) inform the

research question by determining the hiring processes of principals across many districts in a

particular county of a mid-Atlantic state. In addition to framing the procedures in practice,

access to the responses expand the study’s information beyond that of the districts selected for

deeper examination. The data collected during the interviews was compared to the individual’s

responses in the survey. The survey gathered information on the participants’ years of

experience and employment setting. During the interviews, each participant was asked if s/he

believed specific bias and heuristics might impact his/her decision-making in the process of

teacher hiring. The survey, therefore gathers information on the quantifiable frames of reference

from which the participants drew upon in the interviews.

62

Table 3

Survey Questions

1. How many years have you been a principal?

a. 1-5

b. 6-9

c. 10+

2. Approximately, how many teachers do you hire in a year?

a. 1-5

b. 6-9

c. 10+

3. How important are the following sources of information when reviewing teacher

applications? 1= somewhat, 2 = important, 3 = very important, N/A

a. Certifications

b. In-district application forms

c. Resumes

d. Transcripts

e. Recommendations, formal or informal

f. Prior knowledge of candidate

4. To what degree are teachers hired through screening committees?

a. Never

b. Sometimes

c. Usually

d. Almost always

e. Always

5. If so, who is included in these committees?

a. Administrators

b. Teachers

c. Parents/community members

d. Other _________________________

6. To what degree are interview protocols or processes (the list of questions, time frames and

manner of interviews, etc.) used for all teacher candidates?

a. Never

b. Sometimes

c. Usually

d. Almost always

e. Always

7. If so, to what degree are those protocols and processes used by all administrators in your

district?

a. Never

b. Sometimes

c. Usually

d. Almost always

e. Always

63

8. Would you be willing to be a participant of this study engaging in an hour long interview on

this topic as described in the accompanying Request for Participation? You may rescind

your agreement to participate at any time.

a. Yes (Please provide your email here__________)

b. No

c. Contact me first as I have questions regarding participation

9. If so, would you also be willing to be a member of a focus group of up to 8 principals who

were also members of the study to discuss the preliminary themes identified in the study.

This group would meet for 1 hour in a location central to the county participants.

a. Yes

b. No

c. Contact me first as I have questions regarding participation

Data set 1, the survey, was utilized to identify participants for the study as questions 8

and 9 specifically asked if the responder were willing to participate in the interviews and focus

group, respectively. The survey questions were designed to illuminate the most relevant aspects

of the theoretical framework. Table 4 lists the relationship between the survey questions, the

research question and the theoretical framework.

Table 4

Survey Questions to Components of TF

The research question: How does bounded rationality affect administrators’ decision-making

processes in the process of hiring new teachers among comparable school districts in the mid-

Atlantic region?

Survey Questions

Relation to research

question

Relation to components of

theoretical framework

1. How many years have you

been a principal?

Allows for comparison of

years of experience as

principals to valuation and

processes

TF holds biases and

heuristics are not reduced

by experience

2. Approximately, how many

teachers do you hire in a

year?

Allows for comparison of

experience to valuation and

processes

TF holds biases and

heuristics are not reduced

by experience

3. How important are the

following sources of

information when reviewing

teacher applications?

Allows understanding of

data deemed important by

principals

TF – confirmation biases

causes individuals to

concentrate on data that

supports preference

64

4. To what degree are teachers

hired through screening

committees?

Allows for understanding of

the range and specifics of

hiring processes

TF –Monitoring of formal

protocols reduces biases and

heuristics

5. If so, who is included in

these committees? Allows for understanding of

the range and specifics of

hiring processes

TF –Monitoring of formal

protocols reduces biases and

heuristics

6. To what degree are interview

protocols or processes used

for all teacher candidates?

Allows for understanding of

the range and specifics of

hiring processes

TF – Formal protocols

reduce biases and heuristics

7. If so, to what degree are

those protocols and processes

used by all administrators in

your district?

Allows for understanding of

the range and specifics of

hiring processes

TF – Formal protocols

reduce biases and heuristics

The collected data allowed the researcher to select a representative group that ranges in

response to the questions as to the hiring processes utilized. Further, this data set was

quantified to serve as a reference point as the range of hiring approaches utilized. The

responses collected through Survey Monkey were quantitatively analyzed. Central tendency

was identified through calculation of the mean.

Data Set 2: Interviews

The second data set was gathered in the in-depth interviews. This data, clarified through

analysis, the degree to which the decision-making as expressed by the five principals was

impacted by bounded rationality. Specifically, the analyses focused on the degree to which

intuition or fast thinking and rational or slow thinking (Kahneman, 2011) was noted in the

administrators’ expressed valuation of certain data in the hiring process as they consider the

impact of biases and heuristics, e.g. anchoring, confirmation, and halo effect, on the hiring

process. This data set included the principals’ paper review and interviewing processes.

65

The researcher designed semi-structured interview protocol explored the administrators’

hiring procedures and protocols so that the research question, how does bounded rationality

affect administrators’ decision-making processes during the process of hiring new teachers,

may be analyzed. Each of the questions offer opportunities to code for themes related to the

specific biases and heuristics identified by bounded rationality as well as the general

proposition of the theory that humans are not as rational as they believe themselves to be. The

analysis of the administrators’ responses may offer understanding of the degree to which the

goal of rational decision-making is undercut by biases and heuristics. One such instance

considered was when administrators or committee members ignore data that counters their initial

choice as this might demonstrate anchoring.

The purpose of the interviews was to understand the principals’ expressed experiences of

the teacher higher process so the framework of bounded rationality may be applied for analysis

and interpretation. This may improve understanding of the hiring process in order to identify the

most appropriate candidate for the particular teaching position. Through the purposeful selection

of principals who use formal protocols and those who do not, the impact of protocols, which are

suggested in research, was studied in practice. The overlay of the theoretical framework

examines the degree to which bounded rationality can be ameliorated through such decision-

making tools.

Therefore, the researcher’s questions targeted the two main components of the teacher

hiring process: paper screening, and interviewing. Each section of questions was designed to

elicit the administrators’ understanding of his/her protocol and, applying the lens of bounded

rationality, the degree to which the administrator’s expressed goal of each step of the process

matches his/her expressed basis for the final hiring selection. For example, it may be expected

66

that transcripts would have comparative weight during the paper screening phase of the hiring

process as it is a measure of four years of academic effort on the part of the candidate. The

interview question 1, “What is most important to you in a teacher’s submitted application?”

was included so that administrators would indicate their valuation of the readily accessible

transcript criteria. This was compared to question 3, “What do you look for during teacher

interviews?” This comparison gave insight into the degree which administrators value

educational background, against other factors such as interviewing skills. The researcher

examined if university transcripts and other factors used to value one candidate over another

remained as decisive criteria once principals meet the candidates.

Table 5 details the relationships among the research question, the interview questions,

the theoretical framework, coding, and possible themes anticipated. The expected themes and

codes derived from the questions were in keeping with the tenets of bounded rationality. Each

question and prompts explores an aspect of teacher hiring likely to be impacted by a specific

bias or heuristic. Question 1a asks: “What is most important to you in a teacher’s submitted

application?” Responses to this question were expected to elicit responses in the areas of

expertise. Principals may reference their past experiences in locating good teachers and

corollary evidence located in applications. The principals stated that certain elements in

transcripts, work experience, or references accurately and quickly identify good candidates.

First cycle coding organized such statements as “expertise.” Second cycle coding furthered such

statements through in In vivo coding as “I was impressed by the level of teaching experience.”

The data gathered from the semi-structured interviews was coded first through pattern

coding and generate codes such as (Ex) Expertise, (I) Intuition, (T) Time constraint, and (Qu)

Quantify. Second cycle coding identified language related to judgments based on intuition or

67

reasoning and the time available for the decision-making process. Codes included “… got a

feeling,” “We discussed our scores,” “…I can tell by now,” and “Hope for the best.”

The two cycles of coding informed the degree to which the theoretical framework of

bounded rationality is applicable to the experience of principals in the process of hiring teachers.

Further, in keeping with the theoretical framework, questions 7 – 9 explored the degree to which

principals express a need for decision-making tools in the hiring process.

Table 5

Data Set 2 Interview Questions to Codes and Themes

Interview questions

and TF Biases and Heuristics Themes

Cycle 1

Pattern

Cycle 2

In vivo

1. (Anchoring/Halo Effect)

a. What is most important to you in a

teacher’s submitted application?

b. How does that help you find the most

effective teacher?

c. Have you ever had a hiring committee

member who held onto a first impression

(good or bad)? How did you manage

that?

Fast

Thinking

(Intuition)

Slow

Thinking

(Reason)

(Ex) Expertise

(I) Intuition

(T) Time

constraint

(Qu) Quantify

“Impressed by

level of…”

“… got a

feeling.”

“…had to get an

applicant before

the BOE

agenda…”

“We revisited

our matrix.”

2. (Missing Data Bias)

a. Do you find you have all of the

information you need to choose the best

candidate?

b. How do you manage the task of

considering all of the information?

Fast

Thinking

(Intuition)

(Ex) Expertise

(T) Time

constraint

“…I can tell by

now.”

“Have other

tasks to get to”

3. (Affect Bias)

a. What do you look for during teacher

interviews?

b. How does that help you find the most

effective teacher?

Fast

Thinking

(Intuition)

(L) Likeability

(Ex) Expertise

“A good fit

here.”

“You can tell

right away.”

68

4. (Confirmation Bias)

a. Have you ever had a committee member

select data as evidence of the preference

he/she had before the interviews began?

b. If so, how did you manage that?

Slow

Thinking

(Reason)

(Qu) Quantify

“We discussed

all of the

responses.”

5. (Availability Bias)

a. Have you ever had a committee member

express a dislike for a candidate because

the candidate reminded the committee

member of a previous poor hire

(graduated from the same university,

lived in the same town, etc.)?

Fast

Thinking

(Intuition)

Slow

Thinking

(Reason)

(L) Likeability

(L) Likeability

“We had a

teacher before

from that

program…”

“But so was

Jason.”

6. (Sunk Cost Effect / Loss Aversion)

a. What experiences have you had in which

after a long interview process, you did

not have a candidate that you were

confident in hiring?

b. Did you restart the hiring process? Why

or Why not?

Fast

Thinking

(Intuition)

Fast

Thinking

(Intuition)

(T) Time

constraint

(C) Confidence

“We already

spent hours.”

“This is the best

we will get

now.”

7. (Decision-making Tools)

a. Does your school/district utilize protocols

or processes for teacher hiring?

b. Do your processes improve your teacher

hiring? If so, how?

c. Do you wish it were different? If so,

how?

d. Expert decision makers such as surgeons

and pilots have been shown to improve

their decisions through the use of

checklists. Do you believe that might also

be true for the expert decision-making of

principals in the hiring process? Why or

Why not?

Slow

Thinking

(Reason)

Fast

Thinking

(Intuition)

Fast

Thinking

(Intuition)

Slow

Thinking

(Reason)

(Qu)Quantify

(C) Confidence

(T) Time

constraint

(I) Intuition

(Ex) Expertise

“We calculate

each question

on a 4 point

scale.”

“It is

cumbersome.”

“I can just tell.”

“You know

right away.”

“Hiring is

different.”

8. (Decision-making Tools)

69

a. Do you always use committees in the

hiring process?

b. If so, how do they operate?

c. How are candidates ranked and selected?

d. Who makes the final decision, the

principal, or the committee?

e. Is someone charged with ensuring

protocols are followed in the process?

Fast

Thinking

(Intuition)

(T) Time

constraint

(Ex) Expertise

(C) Confidence

(Qu) Quantify

(Ex) Expertise

(Ex) Expertise

“…everyone’s

time”

“I just do it.”

“Perfectly.”

“…rank order”

“I do.”

“We are used to

it.”

9. (Decision-making Tools)

Consider your teacher hiring process, what is

working/what do you wish to change?

Slow

Thinking

(Reason)

Fast

Thinking

(Intuition)

(Qu) Quantify

(C) Confidence

(Ex) Expertise

(I) Intuition

“Wish we could

see them all

teach for more

time.”

“Wish it were

faster. You

know right

away.”

Data set 2 - Interviews first cycle coding

This first cycle coding of all interviews in which each principal was asked to describe

his/her processes of paper-review and interviews were conducted using Pattern coding.

Pattern matching (Yin, 2014, p. 143) was used for first cycle coding in order to compare

the identified patterns of administrators in the process of hiring teachers to the components of

bounded rationality. Pattern matching compares the finding from a case study with that of a

predicted pattern. In this case, it is predicted that suboptimal decision-making caused by missing

data, the misinterpretation of data, or overconfidence generated by the presence of some data as

70

established by the theoretical framework of bounded rationality could be found in the decision-

making process of hiring teacher candidates but go unnoticed by the decision makers.

In the first cycle, coding allowed for the emergence of expected themes. In the second

cycle, the researcher narrowed down or expand the themes, or codes, based on what emerged

from the first cycle. After revising the codes, patterns were identified. Evidence of and related

amelioration of seven biases and heuristics were sought: affect, anchoring, availability,

confirmation bias, missing data, prediction overconfidence, and sunk cost. However, the patterns

that emerge from the research differed. Additionally, rival explanations (Yin, 2014, p. 146) were

also sought as matter of practice of good research.

Data set 2 - Interviews second cycle coding

Second cycle coding refines the emerging categorical, thematic, and theoretical analyses

developed in the first cycle coding (Saldana, 2009). Second cycle coding reduced and expanded

the themes identified through the pattern codes produced during first cycle coding. Second cycle

coding condensed the number of codes and deeper analysis of the original plan of study

(Saldana, 2009). The CAQDAS program Nvivo assisted the researcher in this process by

allowing highlighting and coding of specific words or phrases. The researcher evaluated all of

the collected data as CAQDAS located exact phrasing; terms or expressions may be missed if

every interview is not carefully reviewed by the researcher. “In vivo coding is appropriate for

virtually all qualitative studies, but particularly for beginning qualitative researchers learning

how to code data, and studies that prioritize and honor the participant's voice” (Saldana, 2009, p.

75). In vivo coding allows for coding patterns of actual words and statements made by

participants, in this case study, those of principals. The theoretical framework used as lens for

this study involves the mental shortcuts which allow for rapid decision-making. In vivo coding in

71

this study analyzes the collected statements of five principals as they detailed their decision-

making processes in the hiring process. When describing candidates, common expressions such

as “good fit” or “very professional” were noted. These, in turn, were related to the elements of

the specific biases and heuristics detailed in the theoretical framework of bounded rationality.

Codes and themes evolved from the gathered language.

This method of coding uses the direct language of participants as codes rather than

researcher-generated words and phrases (Saldana 2009, p. 48) and were applicable to this

research. By allowing the focus to be on the actual language of the participants, researcher

intrusion was reduced. Quoting Charmaz (2006, p. 57), Saldana (2009) states “In vivo Codes

‘can provide a crucial check on whether you have grasped what is significant’ to the participant,

and may help crystallize and condense meanings” (p. 79). CAQDAS programs assist in

organizing and quantifying this process; however, it is the researcher who is in control of what is

coded and how the coded data is interpreted. Finally, In Vivo coding particularly assists in this

challenging work for beginning qualitative researchers learning how to code data (Saldana, 2009,

p. 74). Reflection through memo writing and second cycle coding coordinated and refined the

pattern codes created in first cycle coding.

Data Set 3: Focus Group

The third data set was collected by conducting a focus group. This group of volunteer

participants consisted of five principals individually interviewed for data set 2. The focus group

session was an open discussion of the principals’ reactions to the themes identified in data set 2

guided through a semi-structured discussion (Appendix F).

72

Data Set 3 – focus group cycle 1 coding

Cycle 1 coding of the focus group transcript was first through Pattern coding. Expected

themes included the defense of the present processes and the intuitive ability on the part of the

principals to find the best candidates quickly and effectively through the interview process.

Themes identified focused the In vivo coding to be utilized in cycle 2. Table 6 lists the

anticipated pattern codes such as (C) Confidence, (Ex) Experience, (I) Intuition, and (T) Time.

Data Set 3 – focus group cycle 2 coding

Cycle 2 In vivo coding identified in the language of the focus group transcript used to

refine the potential themes noted in cycle 1 theming. Specific language of the participants such

as “I just got a bad vibe from her” or “The candidate pool in this subject is limited” were coded

and organized in relation to the theoretical framework concepts of fast thinking, or intuition, and

slow thinking, or rational thought. Tables 6 lists the expected codes and themes from the focus

group interview.

Table 6

Data set 3 – focus group expected codes and themes

Focus Group Dialogue Starters Themes

Cycle 1

Pattern

Cycle 2

In vivo

1. In our first interview, we discussed the

process of selecting candidates to interview

from the pool of applicants. How do you

feel about success of that process?

Slow

Thinking

(Reason)

Fast

Thinking

(Intuition)

Fast

Thinking

(Intuition)

(C)

Confidence

(Ex)

Experience

(I) Intuition

(T) Time

constraint

(Ex)

“The

candidates

may take

other

positions

due to the

delay.”

“I just got a

bad vibe

from her.”

“The

2. Can you recall an experience when a

candidate seemed very promising on paper,

but not in the interview? What did he/she

do/not do in the interview?

3. When you come to the conclusion of the

73

interview process, do you find your

committee members rank the candidates in

the same order? If not, what do you do?

Fast

Thinking

(Intuition)

Experience

(T) Time

constraint

candidate

pool in this

subject is

limited.”

“We may

consider

group

interviews.

”

4. Are you interested in changing your

interview process? If so, in what ways?

Summary

The three data sets of this study allowed for triangulation (Creswell, 2012). Data

collection and analysis of the principal survey, individual interviews, and focus group combine

to increase the trustworthiness of the study. Coding and analysis of each data set served to

inform and evaluate the interpretation of the other two sets.

The data analysis of this study allows for the consideration of the similarities and

differences in the decision-making of principals during the process of hiring teachers. The

analysis utilizes approved research methods of coding, specifically, Pattern and In vivo, in order

to understand the impact, if any, of bounded rationality on the decision-making process of the

participants in this study.

Trustworthiness

This section details this study’s level of trustworthiness, the degree to which the

information matches what it is meant to represent (Yin, 2014, p. 47). Literal replication is the

prediction of similar results. Theoretical replication predicts contrasting results but for

predictable reasons (Yin, 2014, p. 57). Recording all steps of the study was done to a degree

allowing for replication. The intent of this study was to explore the decision-making of

principals in the teacher hiring process through the lens of bounded rationality (Kahneman,

74

2003). Trustworthiness consists of four components: credibility; transferability; dependability;

and confirmability (Lincoln & Guba, 1985).

Credibility. The first step to establish the credibility of a study is to employ recognized

research methods. This study followed the elements of qualitative design (Yin 2014). This

qualitative study includes sampling strategies, data collection, triangulation, pattern and In vivo

coding and analysis and theming. Credibility was maintained through the researcher’s

engagement with participants for an adequate amount of time to build the trust needed to engage

in deep interviews. Member checking (Creswell, 2012), the opportunity for participants to

examine the narrative as it has been summarized by the researcher, also increased credibility.

The study was conducted following the guidelines and approval of Northeastern University’s

IRB.

Transferability. Transferability, a study’s clarity and depth of description which allows

the study to be applicable to other contexts, was accomplished through the establishment of the

context of the study and the production of a full narrative of the transcription so the coding is

easily followed, apparent, and readily analyzed by the reader. As a study of the experiences of a

sample group of principals, this study is transferable to principals engaged in the process of

hiring teachers in other settings.

Dependability. Dependability is the ability to replicate research and yield similar results.

The researcher reported the research process clearly so that the study could be replicated. Further

as the participants are a representative sampling, similar participant pools are available for

repeated studies if needed.

Confirmability. Bracketing, the means in which the researcher examines his beliefs

toward the phenomenon (Tufford & Newman, 2012, p. 85), offers a means to reduce negative

75

researcher influence on the research process and can be conducted prior to and during data

gathering and analysis. Reflective commentary supports all aspects of trustworthiness including

confirmability. To establish confirmability, this researcher detailed his biases in the positionality

section of chapter one. In order to monitor his biases and actions during the study to prevent

those from impacting data collection and analysis, this researcher utilized journaling to document

interpretations. Being reflective (Creswell, 2012, p. 18; Tufford & Newman, 2012) in this

manner would expose and hopefully reduced the impact of biases through examination of the

researcher’s role or position in the study.

76

Chapter IV: Findings and Analysis

Introduction

The purpose of this qualitative study is to understand the teacher hiring practices of

educational leaders in K-8 public schools in a particular county in a mid-Atlantic state. Teacher

hiring practices include the screening, interviewing, and selecting of teacher applicants. In this

study, the researcher seeks to uncover administrators’ beliefs and actions related to hiring,

examine their hiring decisions, and understand their explanations as to the reasons for those

choices. The gathered data and analysis of the data is presented in this chapter.

Review of Research Problem

Of all the decisions made by school principals, the hiring of teachers may be the most

important task (Mason & Schroeder, 2010; Rowan, 1994a); however, the bounded rationality

involved in decision-making left unchecked by the supports of protocols, such as research-

supported screening criteria or interview procedures (Cranston, 2014), engenders an ongoing

problem of practice: sub-optimal decision-making in the process of teacher hiring.

Tasks such as teacher evaluation, curriculum development, or student discipline are often

conducted with expectations and procedures for consistency and interrater reliability; however,

the actual practice of teacher hiring varies greatly and is often “information poor,” and the

criteria for selection is often based on feeling over evidence (Liu & Johnson, 2006 p. 331).

Given the larger pool of teacher candidates than available openings (Clement, 2013) and the

importance of the selection (Engel & Finch, 2015; Stronge et al., 2011), it would be expected

that hiring practices would be deeply researched, carefully and consistently practiced. This is not

the case (Liu & Johnson, 2006). Little is known about the nuances of the process of teacher

hiring (Mason & Schroeder, 2010).

77

This examination of the hiring practices of K-8 educational leaders is viewed through the

theoretical framework of bounded rationality, which holds that rationality is limited by the

availability of information, time, and mental capacity (H. A. Simon, 1982). According to

bounded rationality, the mental shortcuts that allow humans to survive in times of crisis that

require quick, decisive action do not also generate predictable rational decision-making. As a

result of this natural mindset, the answers that come to mind first are deemed to be the best

answer by the thinker.

Biases and heuristics play a significant and often undetected role in the outcomes of

decisions (Hoppe & Kusterer, 2011; Hutchinson et al., 2010; Kahneman et al., 2011). Biases are

the natural tendency to think in certain ways (Kahneman et al., 2011). These patterns of thinking

impact the rationality of decisions. Heuristics are simple practices that quickly produce

seemingly adequate answers to difficult questions. Table 7 details the potential biases and

heuristics which may impact teacher hiring decisions.

78

Table 7

Biases and Heuristics and Possible Impact on Hiring Decisions

Biases and

heuristics Effects Possible impact on hiring decisions

Affect bias causes decisions to be based

on emotions or on

likes/dislikes

Is there evidence of teacher hiring based on

adult to adult likeability rather than teacher

efficacy?

Anchoring causes the weighing of one

piece of information too

heavily, often the first

information encountered

Is there evidence of committees selecting

candidates with shortcomings related to the

abilities as teachers that should have

prevented their selection?

Availability causes overestimation of the

likelihood of events when

similar past events come to

mind

Is there evidence of committees selecting

candidates based on the administrator’s

specific knowledge base rather than from a

full range of needed information?

Confirmation

Bias

causes the searching for

evidence that supports belief

and ignoring evidence that

contradicts it

Is there evidence of individuals selecting

candidates based on the one teaching

domain his/her preferred candidate most

demonstrated and ignore the evidence from

the other teaching domains?

Missing Data

Bias

is the tendency to ignore the

lack of available data

needed for rational decisions

Is there evidence of committees selecting

candidates about whom little is known over

those better known?

Prediction

Overconfidence

is the subjective confidence

level that is not supported

by objectivity

Is there evidence of committees selecting

candidates based on unsubstantiated

predictive tools such as experience as a

coach will cause the candidate to be a

better teacher than those who have not

coached?

Sunk Cost

Effect / Loss

Aversion

is the phenomenon in which

people justify increased

investment of time or

resources into a decision,

based on the cumulative

prior effort

Is there evidence of committees selecting

candidates who at the end of a lengthy

screening process may not be strong

candidates, but the administrators resist

reopening the hiring process due to the

time already invested in finding the first

candidate?

79

Recent research (Botterill & Hindmoor, 2012; Halevy & Chou, 2014; Kahneman et al.,

2011; Lovallo, D. & Sibony, O., 2010) explores the manner in which bounded rationality can be

discovered and reduced in order to improve the decision-making process. It has been shown that

business executives can reduce the impact of biases such as anchoring, weighing one piece of

information too heavily, and confirmation, ignoring evidence that contradicts one’s beliefs

(Kahneman et al., 2011).

Research on teacher hiring is limited, but researchers such as Clement; Rutledge; and

Schumacher (2013; 2008; 2015), offer possible improvements for the process. One example of a

researched technique to improve rationality and consistency in teacher hiring practices is the use

of structured interviews specifically aligned to the domains of teaching: planning, management,

instruction, and reflection (Clement, 2013). These protocols are helpful in selecting effective

teachers, but structured interviews are rarely used (Clement, 2009; Liu & Johnson, 2006;

Rutledge et al., 2008).

Review of Research Question

Despite the human shortcomings involved in processing data demonstrated through the

decades of research building the theoretical framework of bounded rationality (Botterill &

Hindmoor, 2012; Kahneman, 2003; H. A. Simon, 1982), in the field of education, there remains

a widely held belief in the possibility of rational, data-driven decisions in schools (Cooley et al.,

2006; Marzano et al., 2005; Reeves & Flach, 2011). The best practice era that has dominated the

world of teaching, which assumes a set of techniques can be applied to all cases to garner the

same effect, mirrors the zeal in which data-driven decisions are prescribed for educational

leaders. However, bounded rationality holds that regardless of the clarity of the available data,

intuitive, or fast thinking, undercuts rational thought (Kahneman & Tversky, 1984). When

80

individual perceptions filter the interpretation of data, even carefully organized data-driven

decision-making settings are jeopardized (M. E. Toplak et al., 2014). The research question for

this study, how does bounded rationality affect administrators’ decision-making processes in the

process of hiring new teachers among comparable school districts in the mid-Atlantic region, is

rooted in the theoretical framework.

Each survey and interview question was written to compare the bias and heuristics identified in

the theoretical framework with the responses of the principals surveyed and interviewed.

Data Collection

Data collection for this research began in September of 2018 with an online survey of

principals regarding their experiences in the teacher hiring process. Securing superintendent

approval prior to the distribution of the survey reduced the available pool of principals serving in

K-8 settings in the particular Mid-Atlantic county to twenty-one individuals.

Eleven of the principals (52%) completed the initial survey. Further, seven of the eleven (55%)

respondents indicated a willingness to participate in the follow up interviews.

Five principals, four women and one man, from three different districts were interviewed

in 1:1 settings. Two of the principals lead middle schools. Three lead elementary schools. Each

of these sessions lasted approximately one hour.

Two of the principals interviewed joined the one-hour focus group session. Figure 1, data

collection process, overviews the three data collection phases. In phase 1, the researcher

collected survey responses from principals which served as data points and also as a means to

identify interview participants. In phase 2, in-depth interviews provided the majority of the

collected data. This data was multiphase coded by the researcher using pattern matching and In

vivo coding. In phase 3, data collection included the conducting of a focus group, analysis of that

81

data and the triangulation of all three data sets. Data collection was completed in the manner

approved by the Northeastern IRB. The data collection process figure lists the data points:

survey, interviews, and focus group, along with the intermediary steps of researcher analysis.

Figure 1. Data collection process

Data Analysis

Of the four general analytic strategies (Yin, 2014. p.131), a linear analytic structure

relying on theoretical propositions is most applicable for this study. This study applies the

components of the theoretical framework of bounded rationality. The data collection questions,

coding, and theming were all oriented to specific biases and heuristics identified in the

theoretical framework.

Of the five analytic strategies (Yin, 2014, p.143), one of the most desirable for

trustworthiness in a qualitative study is pattern matching. The initial rounds of coding for this

study were oriented toward the comparison of the data collected and the anticipated patterns

Synthesis of themes

Findings Conclusions Recommendations

Survey review: identify qualifying responses and selection of participants

Data Collection Phase 1: Survey

Data Set 1: Survey results analysis, identify typical and range of systems

Refinement of instrumentation and coding scheme

Data Set 2: Interview analysis, peer review, and member checking

Data Collection Phase 2: Interviews with selected participants

Data Collection Phase 3: Focus Group with selected participants

82

established by bounded rationality. In vivo coding was used in second cycle coding to better

understand the significance of the pattern matching coding. The specific language of the

participants was examined in relationship to bounded rationality and the experimental research

that supports it. Based on bounded rationality, there is anticipated phrasing when individuals are

employing intuitive thinking in the hiring process. Phrases such as “good fit” “you just know,” “I

can just tell” are all examples of phrasing that are more likely produced by what Kahneman

refers to as fast thinking rather than the more rational and reflective slow thinking (2011). Table

8 overviews the analysis of each of the three data sets. Data set 1, the survey, was considered

through a quantitative analysis of the Likert scale and multiple choice responses. Data sets 2 and

3 were coded in two cycles: Pattern Matching and In vivo coding.

Table 8

Summary of Data Collection and Analysis

Research Question Data Source Analysis

How does bounded

rationality affect

administrators’ decision-

making processes in the

process of hiring new

teachers among comparable

school districts in the mid-

Atlantic region?

4. Data set 1 -

Administrator survey

4. Cycle 1: quantitative

analysis of Likert responses

5. Data set 2– 1:1

interviews

6. Data set 3 – Focus

group interview

5. Cycle 1: Pattern Code

Cycle 2: In vivo Code

6. Cycle 1: Pattern Code

Cycle 2: In vivo Code

Analysis of Survey Responses

Through multiple choice and Likert scale questions, principals were asked to self-report

on their experiences in hiring teachers. These questions were organized to illustrate the

participants’ work experience, data valuation, use of committees and protocols, as well as

willingness to participate in 1:1 interviews.

83

Participant experience. All of the survey respondents indicated they hire 1-5 teachers

per year, the minimum number required for inclusion in the study. The respondents’ years of

experience as principals were divided across the year ranges: 1-5 years for five principals

(45.45%); 6-9 years for two principals (18.18%), 10+ years for four principals (36.36%).

Applicant data valuation. Survey participants were asked to evaluate the degree of

importance for a number of data points in the paper screening process, detailed in Table 9. As

proper certification is required by the state regulations and statutes, certification was rated the

most important criteria for considering which teachers to consider for an interview. All of the

interviewed principals mentioned culling the candidate pool by this requirement first. As listed in

Table 9, application forms and transcripts were considered far less valuable than resumes,

recommendations, or prior knowledge of the candidate.

Table 9

Importance of applicant documents

Committee use. As detailed in Table 10, “Almost Always” yielded the highest response

rate (4 principals, 33%) to the question, “To what degree are teachers hired through screening

SOMEWHAT IMPORTANT VERY IMPORTANT N/A

Certifications 0.00%

0

8.33%

1 91.67%

11

0.00%

0

In-district application

forms

58.33%

7

16.67%

2

25.00%

3

0.00%

0

Resumes 8.33%

1

33.33%

4 58.33%

7

0.00%

0

Transcripts 50.00%

6

33.33%

4

16.67%

2

0.00%

0

Recommendations,

formal or informal

16.67%

2

25.00%

3 58.33%

7

0.00%

0

Prior knowledge of

candidate

16.67%

2

33.33%

4 41.67%

5

8.33%

1

84

committees?” Yet, the second highest response rate was “Never” (3 principals, 25%). When

used, committees are made up of administrators, and 50% of the time, are joined by teachers. On

some occasions, parents and school board members are included. All interviewed principals

indicated the superintendent was the final interview step and 2-4 candidates were usually brought

to that round.

Table 10

To what degree are teachers hired through screening committees?

Protocols. When asked, “To what degree are interview protocols or processes used for

all teacher candidates,” 92% of the respondents indicated either “Always” (58.33%) or “Almost

always” (36.36%). The affirmative response rate was also 92% for question 7, “If so, to what

degree are those protocols and processes used by all administrators in your district?”

Survey Summary. The five hours of transcripts of the 1:1 interviews comprise the

primary data set of this study. This data held the greatest volume of data to analyze. Further, as

ANSWER CHOICES RESPONSES

Never 25.00%

3

Sometimes 16.67%

2

Usually 8.33%

1

Almost always 33.33%

4

Always 16.67%

2

TOTAL 12

85

individual interviews, the principals thinking was less likely influenced or interrupted – which

could occur in a focus group.

Data gathered from the survey results demonstrated some similarities and a few

differences across the principals’ years of experience, school settings, applicant data valuation,

use of hiring committees and protocols. The full range of response choices were utilized by the

respondents, with the noted exceptions of certification importance and the number of teachers

hired annually. Certification importance was indicated as important or very important. Although

the option for teachers hired each year included 1-5, 6-10 and more, none of the respondents to

this survey indicated hiring more than one to five teachers annually.

Hiring rates. The hiring rate of one to five teachers per year informed the interview

session as the researcher initially anticipated sunk cost (Kahneman, 2011) reactions when

principals faced deadlines for hiring. Discovered through the survey, the rate of open positions

for those interviewed was relatively low at only one to five annually. Therefore, time factors

were not noted by the principals as a major concern. However, originally designed interview

questions were still utilized in the 1:1 interviews to explore the issue of time demands.

Data valuation. The survey results regarding data sources preferred by administrators

matched the research note in the literature (Cranston, 2012; Rutledge et al., 2008). Resumes were

preferred over transcripts. Originally designed 1:1 interview questions were utilized to examine

the principals’ reasons for this preference.

Committee use. The survey results included affirmative responses to the full range of

options from “Never” to “Always.” Although committee use is suggested in the literature

(Heneman & Milanowski, 2004; Timmons, 1998), respondents varied in their reported use. The

86

1:1 interviews investigated the reasons for this variance. Predicted considerations were principal

preference, superintendent preference, district size and staff availability.

Protocols. A high percentage of respondents express their hiring process as utilizing

protocols. The 1:1 interview questions allowed the researcher to explore how individual

principals define protocol usage in the hiring of teachers and the degree to which there is the

consistency of use recommended in the literature (Engel & Finch, 2015; Haskins & Loeb, 2007;

Heneman & Milanowski, 2004; Lange et al., 2012; Schumacher et al., 2015).

Analysis of Data from 1:1 Interviews

Interview questions were constructed to identify the specific biases and heuristics

involved in the decision making of principals when conducting teacher hiring processes. Table

11 includes the interview questions organized by elements of bounded rationality.

Table 11

Interview Questions Organized by Bounded Rationality Identified Biases and Heuristics

10. (Anchoring/Halo Effect )

d. What is most important to you in a teacher’s submitted application?

e. How does that help you find the most effective teacher?

f. Have you ever had a hiring committee member who held onto a first impression (good or bad)?

How did you manage that?

11. (Missing Data Bias)

c. Do you find you have all of the information you need to choose the best candidate?

d. How do you manage the task of considering all of the information?

12. (Affect Bias)

a. What do you look for during teacher interviews?

b. How does that help you find the most effective teacher?

87

13. (Confirmation Bias)

a. Have you ever had a committee member select data as evidence of the preference he/she had

before the interviews began?

b. If so, how did you manage that?

14. (Availability Bias)

a. Have you ever had a committee member express a dislike for a candidate because the candidate

reminded the committee member of a previous poor hire (graduated from the same university, lived

in the same town, etc.)?

15. (Sunk Cost Effect / Loss Aversion)

a. What experiences have you had in which after a long interview process, you did not have a

candidate that you were confident in hiring?

b. Did you restart the hiring process? Why or Why not?

16. (Decision Making Tools)

a. Does your school/district utilize protocols or processes for teacher hiring?

b. Do your processes improve your teacher hiring? If so, how?

c. Do you wish it were different? If so, how?

d. Expert decision makers such as surgeons and pilots have been shown to improve their decisions

through the use of checklists. Do you believe that might also be true for the expert decision making of

principals in the hiring process? Why or Why not?

17. (Decision Making Tools)

a. Do you always use committees in the hiring process?

b. If so, how do they operate?

c. How are candidates ranked and selected?

d. Who makes the final decision, the principal, or the committee?

e. Is someone charged with ensuring protocols are followed in the process?

18. (Decision Making Tools)

Consider your teacher hiring process, what is working/what do you wish to change?

Five individual interviews were conducted, transcribed, and analyzed. First cycle coding

was conducted with the use of CAQDAS, NVivo Pro in particular, and researcher analysis.

Pattern match coding of principal responses generated a number of themes. These themes were

organized by the researcher into the two main sections of bounded rationality as organized by

Kahneman (2011): fast thinking and slow thinking.

88

Identified Fast Thinking. Identified fast thinking includes expertise/confidence,

intuition/gut/fit, time constraints, likeability/background of candidate similar to administrator.

Expertise/confidence. Throughout the interviews, principals expressed their success in

identifying the best teaching candidate from among the pool of candidates. “I have been doing

this for a long time, so you just know. Also, you can tell because this staff is great.”

Intuition/gut/fit. Principals expressed being able to identify weak candidates almost

instantly, stating “It only takes a few minutes.” Additionally, principals expressed having a sense

of how well the candidate would fit in with the rest of the staff.

Time constraints. According to bounded rationality as expressed by Kahneman (2011),

quick thinking is essential to human survival in time sensitive decision making. In the research

study design period, this researcher speculated if the demands on administrator time along with

hiring timelines, which require board of education approval, transfers, and induction, would

force administrators to make hasty and therefore less rational decisions. The 1:1 interviews

revealed that although all the principals described situations in which they were rushed to make

last minute hires, they all preferred to take the time they felt was necessary to find the right

candidate. To the researcher’s question, “Have you ever come to the end of an interview process

without a candidate you wished to hire?” All shared experiences in which, although they had

already invested time which can yield a sunk cost bias, they re-advertised the position and started

over. The exceptions were emergency leave positions that required a certified teacher be

identified quickly or a position with a very limited pool of certified teachers.

Likeability/background of candidate similar to administrator. Principals expressed the

desire to locate teachers with backgrounds similar to themselves. A principal with an interest in

special education law stated that background in that field is something she is looking for in

89

teacher candidates. The middle school principals stated their desire to locate teachers with a wide

range of skill sets; the middle school principals had held careers in other fields and then were

certified as teachers through alternate route to teaching certification programs, so they expressed

the desire to locate candidates like themselves with work experience outside of teaching.

Identified Slow Thinking. Evidence from the interviews included the administrators’

cross checking of data, the use of protocols and procedures, and self-reflection, all of which are

suggested by bounded rationality as means to improve rationality.

Protocols. Principals shared the use of set questions to be asked of each candidate. This

is in line with the notion of decision making tools such as checklists used in surgery procedures.

Cross checking. Principals mentioned contacting references as a means to cross check

impressions generated in the interviews. Additionally, principals expressed the importance of

post-interview discussions with committee members: “Sometimes someone will point out

something you hadn’t considered.”

Self-reflection. Two of the principals made statements which demonstrated self-

reflection such as, “You hope you are right, but you never really know.” The bounded

rationality holds that the mindset of self-reflection is essential to improving rationality.

Second Cycle

In vivo coding of the transcriptions within pattern matched themes was conducted for the

second cycle of coding of the 1:1 interviews.

In vivo coding of transcripts.

Expertise. Differing views of expertise were identified in statements such as “Checklists

might help doctors, but teacher hiring is different” and the contrary view, “Checklists might help.

I would be interested to see that done.”

90

Confidence. Evidence of confidence in the accuracy of one’s decision making was

identified in statements such as, “The system is working because all of our teachers are good”

and “After doing this for 10 years, you know in the first few minutes.” Reflective thinking

related to confidence is demonstrated in statements such as “You never know. You make the

decision with the information you have and hope that it works out. If not, you have to let them go

at the end of the year,” and “It's always interesting. You roll the dice a lot of times. Somebody

can look good on paper and then be a disaster.”

Intuition/gut. Intuition was identified in phrases such as “You can just tell,” and “In

those letters, I'm always looking for what's not there.”

Time Constraint. Time constraint awareness was identified in “We would be able to be

more selective if we had more time.”

Likeability/Similarity to the administrator. This bias was detected in statements such as,

“You may be working with this person for 20 years” and “I had a lot of coursework in that area,

so I am looking for that. I had that background and I think it really helped me be a better teacher,

so I prefer that.”

Table 12 highlights the identified connections of data set 2, the 1:1 interviews, between

the interview questions and the themes of the bounded rationality. Both Cycle 1 and Cycle 2

analyses identified evidence of the themes of bounded rationality in the principals’ expressed

motivations for hiring decisions. In interview question 1, the researcher posed the sub question,

“What is most important to you in a teacher’s submitted application?” This question was

designed to investigate bounded rationality’s concept of fast thinking, in particular, the

likeability bias. Would principals use evidence of the teachers’ likeability in the interview

process instead of concentrating on evidence of teacher effectiveness? This researcher’s cycles

91

of theming identified a pattern of references to administrator expertise and corresponding In vivo

evidence of what administrators look for in teacher candidates. Table 12 includes this

comparative analysis for all of the data set 2 interview questions and sub questions.

Table 12

Data Set 2 Interview Questions to Codes and Themes

Interview questions

and correlated theoretical framework

biases and heuristics

Anticipated

Themes

Cycle 1

Identified

Pattern

Cycle 2

Identified

In vivo

1. (Anchoring/Halo Effect )

a. What is most important to you in a

teacher’s submitted application?

b. How does that help you find the most

effective teacher?

c. Have you ever had a hiring

committee member who held onto a first

impression (good or bad)? How did you

manage that?

Fast

Thinking

(Intuition)

Slow

Thinking

(Reason)

(Ex) Expertise

(I) Intuition

(T) Time

constraint

(Qu) Quantify

“I look for

candidates that will

get along with

students and

parents.”

“You can tell in the

first few minutes.”

“We readvertised.”

“We average the

matrix scores.”

2. (Missing Data Bias)

e. Do you find you have all of the

information you need to choose the best

candidate?

f. How do you manage the task of

considering all of the information?

Fast

Thinking

(Intuition)

(Ex) Expertise

(T) Time

constraint

“No, I have all the

information I

need.”

“…I can tell by

now.”

3. (Affect Bias)

a. What do you look for during teacher

interviews?

b. How does that help you find the most

effective teacher?

Fast

Thinking

(Intuition)

(L) Likeability

(Ex) Expertise

They would be a

good fit here.”

“They can learn the

content; they have

to like children.”

4. (Confirmation Bias)

a. Have you ever had a committee

Slow

Thinking

(Qu) Quantify

“I had a

superintendent

92

member select data as evidence of the

preference he/she had before the

interviews began?

b. If so, how did you manage that?

(Reason) ignored our

recommendations,

but we all tend to

agree with the new

superintendent.

“We discussed all

of the responses.”

5. (Availability Bias)

a. Have you ever had a committee

member express a dislike for a candidate

because the candidate reminded the

committee member of a previous poor

hire (graduated from the same university,

lived in the same town, etc.)?

Fast

Thinking

(Intuition)

Slow

Thinking

(Reason)

(L) Likeability

“From a particular

school, no.”

6. (Sunk Cost Effect / Loss Aversion)

a. What experiences have you had in

which after a long interview process, you

did not have a candidate that you were

confident in hiring?

b. Did you restart the hiring process?

Why or Why not?

Fast

Thinking

(Intuition)

Fast

Thinking

(Intuition)

(T) Time

constraint

(C) Confidence

“If we have time,

we will start over.”

“This is the best we

will get now.”

7. (Decision Making Tools)

a. Does your school/district utilize

protocols or processes for teacher hiring?

b. Do your processes improve your

teacher hiring? If so, how?

c. Do you wish it were different? If so,

how?

d. Expert decision makers such as

surgeons and pilots have been shown to

improve their decisions through the use

Slow

Thinking

(Reason)

Fast

Thinking

(Intuition)

Fast

Thinking

(Intuition)

Slow

Thinking

(Qu)Quantify

(C) Confidence

(T) Time

constraint

(I) Intuition

(Ex) Expertise

“We calculate each

question on a 4

point scale.”

“We don’t actually

use the scores. We

just rank them.”

“No, the proof is in

our staff.”

“You know right

away.”

“Hiring teachers is

different.”

93

of checklists. Do you believe that might

also be true for the expert decision

making of principals in the hiring

process? Why or Why not?

(Reason)

“That might be

interesting.”

8. (Decision Making Tools)

a. Do you always use committees in the

hiring process?

b. If so, how do they operate?

c. How are candidates ranked and

selected?

d. Who makes the final decision, the

principal, or the committee?

e. Is someone charged with ensuring

protocols are followed in the process?

Fast

Thinking

(Intuition)

(Ex) Expertise

(Ex) Expertise

(C) Confidence

(Qu) Quantify

(Ex) Expertise

(Ex) Expertise

“Just the

administrators in

the district.”

“I just do it.”

“Well.”

“We put them in

rank order”

“We come to a

consensus.”

“No, we just are

used to it now.”

9. (Decision Making Tools)

Consider your teacher hiring process,

what is working/what do you wish to

change?

Slow

Thinking

(Reason)

Fast

Thinking

(Intuition)

(Qu) Quantify

(C) Confidence

(Ex) Expertise

(I) Intuition

“Having the time

for model lessons.”

“We like the way

we do it.”

Summary of interviews. Data collected illustrating the decision making processes of

administrators was in keeping with the patterns predicted by bounded rationality. Pattern

matching and In vivo coding located the use of both fast and slow thinking (Kahneman, 2011) in

the statements of the principals. Biases, heuristics, and the slow thinking efforts to offset those

biases were all evident.

Principals expressed efforts to select the best candidate from the pool of applicants. The

researcher’s analyses of the 1:1 transcripts through coding and reflection identified patterns of

94

efforts to make the best possible decisions, examples of the slow thinking likely to improve those

decisions as well as fast thinking that may render those decisions biased by the biases and

heuristics of the specific principal.

Based upon the literature review, the researcher anticipated principal decisions based on

general candidate likeability. Rather the analysis of the transcripts indicated the pattern to be

more specific. Principals weren’t looking for what would be a generalized likeability, but rather,

what they liked in themselves as teachers. The principals expressed preferences as indicators of

teacher quality were those they saw in themselves or in some cases in the staff already employed

in the district. Principals with experience outside education valued that in teacher candidates.

Principals with backgrounds in curriculum and instruction, former curriculum supervisors,

focused on teacher candidates’ facility within the domains of teaching: planning, environment,

instruction, and professionalism (Danielson, 2013).

Patterns of confidence were found across the transcripts the candidates. Principals

expressed general confidence in their past decisions and present processes. However, bold

statements such as, “You can tell in a few minutes,” were mostly reserved for candidates that

would not be considered for the final round. There were examples shared of candidates selected

that did not turn out to be the teachers hoped for by the principals. The contracts of those

teachers were subsequently non-renewed. The general confidence of the principals were not

shaken by these instances.

As a result of the analysis of the 1:1 interviews, the researcher narrowed the focus group

line of questioning to principals’ conception of ideal candidates, their reflection on their

processes for teacher hiring and their willingness to consider decision-making tools to be added

95

to those processes. The next section of this chapter details the information gathered in the third

data set, the focus group.

Analysis of Focus Group Data

All five of the 1:1 interview participants expressed willingness to be members of the

focus group. Through a survey of their preferences for dates, times, and meeting location – face

to face or via Zoom video conferencing – a date, time, and Zoom meeting link was selected. Four

of the principals were scheduled to join the online conversation. Two joined the meeting as

scheduled. The researcher introduced the two participants, neither of whom knew the other

would be participating; however, the two principals work in nearby districts so knew each other

before this research began. Both principals interviewed in this focus group were experienced

educators who worked in multiple instructional and administrative leadership roles, participated

in numerous professional organizations and offered detailed responses to all questions.

The researcher asked each of the focus group questions. The interview quickly became a

conversation with the participants responding not only to the questions directed to them, but to

the responses offered by the other principal or to the literature or tentative findings shared by the

researcher. At the conclusion of the line of discussion on each question, the researcher proposed

the next question. This hour-long question and discussion process covered all proposed

questions. The transcription of this interview was recorded and transcribed using Rev.com

services.

Based on the analyses of the first two data sets, the originally proposed questions for the

focus group discussion were reconsidered by the researcher. For example, the responses to the

first survey, which included principals other than those who were later involved in the 1:1

interviews, indicated resumes were considered the most important of the choices offered with

96

58% of respondents indicating this as very important, but transcripts only received 16.7%.

Therefore, question 1 presents the administrators with the results to this aspect of the survey and

asks for their reflection to their experiences.

Table 13 below lists highlighted focus group questions and notable responses.

Table 13

Highlighted Focus Group Questions and Notable Responses

Focus Group Question Notable Response

1. Does that match your personal

preference, resumes over transcripts?

“Yes, I think the resume gives a better picture

of not only their experiences, but the picture

they are trying to paint of themselves.”

2. Is there any way to know that the

selected candidates are better than the

runners up that you didn't take?

“We never can really know. We hope for the

best.”

3. Bounded rationality suggests that

we're not as rational as we feel like we

are. Do you find that notion to be true

and if so how do you deal with that

when you're making these big decisions?

“Sometimes teachers in the hiring committee

will definitely be thinking about ‘Who can I

work with? Who's going to be a good colleague

for me?’ So that will sometimes influence them,

but then again, that also is the reality that they

do have to work together as a team, so. It's in

my mind as well and maybe in their minds a

little bit differently.”

4. Having had these discussions with

me, in reflection, is there anything about

your interview process that you would

like to be able to change?

“I don't know what I would change. Our

process right now is pretty good.”

“I think if we had the luxury of doing demo

lessons for every teaching position I think that

would be a good procedural change.”

5. If you had some sort of protocol or for

lack of a better word, a checklist, that

reminded you of the things that you say

or some sort of best practices sort of

sheet, would you be open to that as an

addition to your process?

“I think it would be beneficial. That way you're

kind of using a similar lens for every candidate

and applying that those qualities to each one.”

97

Cycle 1 Pattern matching coding.

Administrator Expertise/Confidence. Prediction overconfidence is a bias recognized

within the bounded rationality. In the context of teacher hiring, it may present as overconfidence

of decisions. The coding process identified responses that could demonstrate confidence without

measurable evidence.

Confirmation Bias/Likeability/Beliefs Regarding Characteristics of Successful

Teachers. The questions for this study rooted in the theoretical framework were designed to

yield responses of likeability. However, in this study, the responses were not based on general

likeability or even how well the candidates managed the pressure of the interview. In fact,

principals expressed intentionally overlooking any perceived nervousness on the part of the

candidates. Instead of the anticipated reactions of general likeability, the bias of likeability was

demonstrated through the principals varied assertions of the characteristics of good teachers.

Although, there were variances such as understanding pedagogy, psychology, relationships,

different learners, etc. Across the data sets, the asserted characteristics to describe the high

quality candidates were aligned to the skill set and personalities purported by the principals of

themselves.

Fit. The focus group members discussed the importance of the new hire’s ability to work

within the teams in which they would be placed and the ability of the candidates to move the

district in the direction the leader intends.

Cycle 2 In vivo coding.

Prediction Overconfidence - Administrator Expertise/Confidence. When asked by the

researcher, “Is there any way to know that they are better than the runners-up that you didn't

take?” one of the principals responded, “I don't think there is. I think it's always a little bit of a

98

gamble.” The second principal added, “That's all you can do is make the best choice at the time

with the information you have.”

Gut/Intuition. During the focus group discussion, the researcher asked the following

purposefully leading question, “Bounded rationality suggests that we're not as rational as we feel

like we are. Do you find that notion to be true, and if so, how do you deal with that when you're

making these big decisions?” One principal responded, “Well, I think that's true. I think if we

were totally rational, we would maybe jot way more on the transcripts and the facts, but I think

that since it is a people business, we often kind of go on a gut instinct…[pick] who the panel

clicks with and hope that plays out well for us.” The other principal followed up with a similar

response, but with a take on rationality. “If you're saying being objective, I think even if you

think of being objective during the interview process, you try to ask questions … looking for

something that's either going to swing you one way or the other. Was that kind of a positive that

[I] really wanted to hear?”

Confirmation Bias/Beliefs Regarding Characteristics of Successful Teachers/Fit. The

researcher pressed the focus group to articulate what they meant by effective teachers. The

principals offered areas related to teaching that candidates should know. “Knowing their stuff,

not just knowing the content. To me it's a given that they know the content. We shouldn't have to

kill them with professional development. They should be coming out of college knowing that.”

Another principal added, “I also want them to know the content. I want them to know what is

expected in the classroom. I want to them to know the curriculum. But above and beyond, how to

communicate with parents. So I guess I'm looking less for fitting in and more for knowing your

stuff and being able to handle themselves. And knowing how to treat my children.”

99

Summary of focus group analysis. As the members of the focus group generated 40%

of the data gathered in the 1:1 interviews, the focus group results correlated with the themes

generated in the individual interviews. The main values added from this data point were the

additional clarifications and opportunities for In vivo coding. The fast thinking of biases and

heuristics along with the slow thinking that might improve rationality were noted in the focus

groups and the 1:1 interviews.

In the development of this research proposal, the many biases and heuristics identified

within the theoretical framework of bounded rationality were considered for the potential impact

on the hiring process. Through the analysis of the focus group as the third data set, in this study,

certain themes related to bounded rationality were more evident than others. Table 14 includes

focus group questions, the themes anticipated by the researcher based upon bounded rationality,

the codes identified in cycle 1 pattern coding and those identified the cycle 2 In vivo coding.

The two most common and prominent themes were administrator confidence and beliefs

regarding the characteristics of effective teachers.

Table 14

Data Set 3 Focus Group Interview to Codes and Themes

Focus Group Question Anticipated

Themes

Cycle 1

Identified

Pattern

Cycle 2

Identified In vivo

1. Does that match your

personal preference,

resumes over

transcripts?

Confirmation

Bias

Confirmation

Bias/Likeability

/Beliefs

Regarding

Characteristics

of Successful

Teachers.

“Sometimes a candidate can be very

book smart but not have those

relationship building characteristics

that you really need in the

classroom.”

2. Is there any way to

know that the selected

candidates are better

Prediction

Overconfiden

ce

Administrator

Expertise/Confi

dence

“We never can really know. We

hope for the best.”

100

than the runners up that

you didn't take?

3. Bounded rationality

suggests that we're not

as rational as we feel

like we are. Do you find

that notion to be true and

if so how do you deal

with that when you're

making these big

decisions?

Prediction

Overconfiden

ce

Intuition

Administrator

Expertise/Confi

dence

Fit

“Sometimes teachers in the hiring

committee will definitely be

thinking about ‘Who can I work

with? Who's going to be a good

colleague for me?’ So that will

sometimes influence them, but then

again, that also is the reality that

they do have to work together as a

team, so. It's in my mind as well and

maybe in their minds a little bit

differently.”

4. Having had these

discussions with me, in

reflection, is there

anything about your

interview process that

you would like to be

able to change?

Time

Restrictions

Administrator

Expertise/Confi

dence

“I don't know what I would change.

Our process right now is pretty

good.”

“I think if we had the luxury of

doing demo lessons for every

teaching position I think that would

be a good procedural change.”

5. If you had some sort

of protocol or for lack of

a better word, a

checklist, that reminded

you of the things that

you say or some sort of

best practices sort of

sheet, would you be

open to that as an

addition to your

process?

Prediction

Overconfiden

ce

Administrator

Expertise/Confi

dence

“I think it would be beneficial. That

way you're kind of using a similar

lens for every candidate and

applying that those qualities to each

one.”

The section below considers the analysis of themes across and among the three data sets.

Emergent Themes from the Three Data Sets

The first section of themes below includes bounded rationality’s fast thinking biases and

heuristics.

101

Identified fast thinking. Regardless of the clarity of the available data, intuitive, or fast

thinking, undercuts rational thought (Kahneman & Tversky, 1984). The biases and heuristics

below were identified during this study.

Affect bias/likeability. The researcher anticipated affect bias to be a prominent bias

based on the literature stream of bounded rationality and hiring practices. Data points for the

affect bias were the principals’ beliefs that an interview was more effective to identify skilled

teachers than observing teachers’ actual instruction. The researcher noted this theme as potential

evidence of the affect bias, as hiring decisions based on interviews without mock lesson demos

may be based on likeability over evidence of efficacy.

However, the expected impact of general candidate likeability was not noted. Instead, the

potential expression of the affect bias was presented in principals seeking those with experiences

and educations similar to their own. The anticipated likeability factor was one of geniality or

even distinguishing characteristics of gender, race, or language. Instead, it was the candidate’s

knowledge and experience or the potential for relationship building that was most observed for

the preference of one candidate over another.

Confirmation bias/beliefs regarding characteristics of successful teachers. This

research exposed an unanticipated area for inquiry: administrators’ individual beliefs as to the

characteristics of successful teachers. Cranston (2014) cites evidence that principals who selected

candidates based on one teaching domain ignored the other teaching domains. Some principals in

this study stated differing beliefs regarding essential skills. Although all principals agreed that

the ability to build relationships with students, staff, and parents was an essential skill, middle

school principals expressed interest in candidates’ content knowledge. “I can help them with the

instruction if they are masters of their content area.” Another principal stated, “At the middle

102

school, I have about 40 teachers and I have 12 sports and 15 clubs and a crazy band and a chorus

with 100 kids in it. That piece always is my bias, my personal bias, and I might be the one who is

more biased towards candidates with non-educational experience because I'm the middle school

principal.” Elementary principals expressed the opposite position, “You can learn the content,

but it is very hard for us, regardless of the amount of professional development, to make

someone a masterful instructor.”

Fit. Fit was mentioned by numerous administrators as an aspect they consider when

hiring teachers. “Do I see this candidate fitting in with our school?” In the focus group

discussion, the principals stated fit was a key interest of teacher committee members as it is the

teachers who will work most closely with the new teachers. In the focus group, the researcher

asked a follow up question as to whether looking for a candidate who is similar to the staff limits

the expansion of diversity when the staff is predominately white. The administrators said that

they were unsure as the pool of candidates brought in for interviews were almost exclusively

white.

Missing data bias. The researcher noted a widely-held belief that the hiring process

gathered all data necessary for making an informed decision. The use of follow up questions of

individual candidates was offered as a solution to any missing data. “I can find out all I need to

know by asking follow up questions.”

Prediction overconfidence - administrator expertise/confidence. Across data sets,

administrators stated confidence, varying across individuals, in their ability to identify the best

teacher candidate through the interview process. Levels of confidence tended to correlate to

length of time in the administrative position.

103

Identified slow thinking.

Data review. Principals articulated decision making processes in line with Kahneman’s

(2011) recommendations for slow thinking that may improve rational thinking. Principals

included cross checking of data through reference checking and post interview committee

member discussions. “I think the bigger districts have the benefit in terms of having more

people, so they have more time. They can include more people. Doing interviews with bigger

committees of people sometimes can be really good because you get more points of view.”

Additionally, principals detailed the use of written interview questions and protocols

intended to generate more objective results.

Self-awareness. Two of the principals demonstrated self-awareness with statements such

as, “You hope you are right, but you never really know.” Bounded rationality holds that the

mindset of self-evaluation is essential in improving rationality.

Summary of Themes

Table 14 lists the biases and heuristics selected by the researcher for the research question

investigated by this study. Column two defines the biases and heuristics identified in the

theoretical framework of bounded rationality. Column three lists the researcher’s predictions as

to the manner that the indicated bias and heuristic might be noted in the process of hiring

teachers. The fourth column reports the researcher’s data analysis for each of the focus biases

and heuristics. Evidence of some of the biases and heuristics was identified. For others, it is

indicated that either no evidence of fast thinking via that particular bias/heuristic was noted or,

alternately, that only evidence of slow thinking contrary to the type of bias/heuristic was

discovered.

104

Table 14

Biases and Heuristics and Possible Impact on Hiring Decisions

Biases and

heuristics Effects

Possible impact on hiring

decisions

Researcher’s

observation of data

analyzed

Affect bias causes decisions to be

based on emotions or

on likes/dislikes

Is there evidence of teacher

hiring based on adult to adult

likeability rather than teacher

efficacy?

Researcher noted

hiring decisions based

on interviews without

mock lesson demos

may be based on

likeability over

evidence of efficacy

Anchoring causes the

weighing of one

piece of information

too heavily, often the

first information

encountered

Is there evidence of

committees selecting

candidates with shortcomings

related to the abilities as

teachers that should have

prevented their selection?

Researcher only noted

Anchoring in

instances in which

superintendents

override committee

selections.

Availability causes overestimation

of the likelihood of

events when similar

past events come to

mind

Is there evidence of

committees selecting

candidates based on the

administrator’s specific

knowledge base rather than

from a full range of needed

information?

Researcher was

unable to identify

Availability as a

consistent bias

Confirmation

Bias

causes the

searching for

evidence that

supports belief and

ignoring evidence

that contradicts it

Is there evidence of

individuals selecting

candidates based on the one

teaching domain his/her

preferred candidate most

demonstrated and ignore the

evidence from the other

teaching domains?

Researched noted

evidence of principals

selecting candidates

based on the one

teaching domain

ignore the evidence

from the other

teaching domains

Missing Data

Bias

is the tendency to

ignore the lack of

available data needed

for rational decisions

Is there evidence of

committees selecting

candidates about whom little

is known over those better

known?

Researcher noted

administrators’

willingness to ask

follow up questions of

candidates, but

researcher noted a

widely held belief that

data was not missing

105

from the process

Prediction

Overconfidence

is the subjective

confidence level that

is not supported by

objectivity

Is there evidence of

committees selecting

candidates based on

unsubstantiated predictive

tools such as experience as a

coach will cause the

candidate to be a better

teacher than those who have

not coached?

Researcher noted

middle school

principals who placed

outside experience

above that of teaching

experience

Sunk Cost

Effect / Loss

Aversion

is the phenomenon in

which people justify

increased investment

of time or resources

into a decision, based

on the cumulative

prior effort

Is there evidence of

committees selecting

candidates who at the end of

a lengthy screening process

may not be strong candidates,

but the administrators resist

reopening the hiring process

due to the time already

invested in finding the first

candidate?

Researcher noted

instances when

principals were, due

to student need,

required to accept a

less than stellar

candidate, but all

principals reopened

searches whenever

possible

Data Triangulation

Data was triangulated across three distinct sources: survey, 1:1 interviews, and a focus

group. Further, the interview transcripts were shared with interviewees for accuracy checks and

the researcher conducted a thorough examination of each participant’s responses across the

multiple months of contact and interviews. Each participant’s responses were found to be

consistent from the survey, interview, and the focus group, where available. The data provided

an “up close” and “in-depth” coverage required for qualitative studies (Yin, p. 192). Participant

responses were gathered through multiple forms of collection. In the survey, “Almost Always”

yielded the highest response rate (4 principals, 33%) to the question, “To what degree are

teachers hired through screening committees?” Yet, the second highest response rate was

“Never” (3 principals, 25%). Discussions in the follow up interviews indicated the cause was the

106

size of the school district. The smaller the district, the fewer administrators and other staff were

available to conduct interviews.

The individual and small focus group dialogues allowed principals to express their

thoughts without the filter of written expression. Careful transcript review and coding

highlighted the themes of the principals’ thoughts on the hiring process. Although research is

available on the practice of hiring teachers (Clement, 2013; Cranston, 2012; Rutledge et al.,

2008), additional qualitative research, such as that offered in this study, is needed. This research

sampled 6% of the principals in one of the geographically largest counties in a particular mid-

Atlantic state. Two of the candidates were involved in all three of the data sets: interviewed

through an hour-long focus group, an hour-long individual interview, and an online survey.

Interview questions were semi-structured in order to gather pertinent information, yet to allow

for open dialogue in order to better understand administrators’ decision making processes.

Trustworthiness

Additional participants would have increased the trustworthiness of this research.

Coordinating the schedules of busy principals across multiple districts proved difficult. Four

administrators were scheduled for the final focus group, but only two attended. This was an

unexpected difficulty that might have impacted collection.

Credibility. Through the researcher’s engagement with participants, credibility was

established. Member checking (Creswell, 2012), the opportunity for participants to examine the

narrative as it has been summarized by the researcher also maintained credibility. The study was

conducted following the guidelines and approval of Northeastern University’s IRB.

Transferability. The study’s clarity and depth of description allows the study to be

applicable to other contexts. This was accomplished through the establishment of the context of

107

the study and the narrative of the transcription and coding. As a study of the experiences of a

sample group of principals, this study is transferable to principals engaged in the process of

hiring teachers in other settings.

Dependability. Dependability is the ability to replicate research and yield similar results.

(Creswell, 2012). The researcher reported the research process clearly so that the study could be

replicated. Further as the participants are a representative sampling, similar participant pools are

available for repeated studies if needed.

Confirmability. Bracketing, the means in which the researcher examines his beliefs

toward the phenomenon (Tufford & Newman, 2012, p. 85), offers a means to reduce negative

researcher influence on the research process and can be conducted prior to and during data

gathering and analysis. To establish confirmability, this researcher detailed his biases in the

positionality section of chapter one. Being reflective (Creswell, 2012, p. 18; Tufford & Newman,

2012) through examination of the researcher’s role or position in the study during the length of

the study reduced, but did not eliminate the impact of biases.

Summary of Findings

The findings from this research were based on the analysis of data gathered through a

survey, semi-structured 1:1 interviews, and a focus group discussion. The participants provided

their understanding of their teacher hiring processes.

Eight key findings were generated through the triangulation of the data collected

throughout this study:

Finding one: Bounded rationality affects the decision-making of K-8 educational

leaders.

108

Identified fast thinking.

Finding two: Principals were individually consistent in the characteristics they sought in

teacher candidates, but the teacher characteristics listed as most important were not consistent

among all principals.

Finding three: The concept of the “fit” of the candidate within the present school climate

was a consideration for principals in the teacher hiring process.

Finding four: Principals did not express concerns of missing data in the teacher hiring

process.

Finding five: Principal expressed confidence in their abilities to select the best

candidates from a pool of candidates.

Identified slow thinking.

Finding six: Principals used protocols and question sets within interviews with the

expressed desire of formalizing and thus improving the selection of teacher candidates.

Finding seven: Principals expressed self-awareness and potential fallibility, keys to

combating fast thinking.

Finding eight: Principals stated the addition of a decision-making tool, such as a

checklist, might improve their processes.

Conclusion

The theoretical framework of bounded rationality offers a view of human thinking as

universally flawed (Kahneman, 2011). The intuitive mind, which is essential for surviving in the

wild, interrupts the careful, slow thinking required for decision making in instances such as

teacher hiring. The data gathered in this study located a number of examples of intuitive thinking

which may allow for quick thinking biases to impact decisions without the administrator’s

109

awareness. However, this researcher also identified instances in which administrators, aware of

this potential, used slow thinking to check for biases.

A variance in administrators’ beliefs as to the experiences and characteristics of effective

teachers was noted. This variance mirrored the administrators’ observations of teachers

employed by them, but also of the administrators’ beliefs about their own experiences that led

themselves to become educators. This is correlated with the theoretical framework of bounded

rationality; humans are not unbiased, rational decision-makers. Rather, thinking is filtered by the

collective experiences and even more so by the quick thinking that experience generates. The

researcher found evidence of fast and slow thinking in all participants, with consistent leanings

of some more than others. Therefore, the suggestions from the field of psychology rooted in

bounded rationality to improve consistent decision-making across experts are applicable to the

hiring practices of teachers.

The final chapter of this thesis contains the researcher’s findings generated through this

study. Discussion will include major insights based on data collected, analysis, the theoretical

framework of bounded rationality, and the literature streams reviewed. Additionally, this

discussion will include recommendations to improve the rationality of the teacher hiring

practices through the tools identified in the fields of psychology and social economics.

Suggestions for additional research and consideration of alternate explanations will be included.

110

Chapter V: Findings

Introduction

Chapter five of this qualitative study includes discussion of the key findings identified in

chapter four. Three data sets: a survey, semi-structured 1:1 interviews, and a focus group, were

analyzed through the lenses of bounded rationality and the cumulative literature review. The

literature review consisted of three streams: bounded rationality, decision-making tools, and

hiring practices.

The themes of the research reviewed drove the creation and organization of this inquiry.

First, it is known that teacher hiring is one of the most important tasks for principals (Clement,

2013; Mason & Schroeder, 2010; Peterson, 2002; Rutledge et al., 2008). However, secondly, it is

also known, that rational thinking is not the human default position (Kahneman & Tversky,

1984; H. A. Simon, 1955; M. Toplak et al., 2011). Thirdly, it is known that protocols and

checklists can improve expert decision-making (Flottorp et al., 2013; Haynes et al., 2009;

Kahneman, 2003; Stubenrauch et al., 2012; Takala et al., 2011). Therefore, the researcher

designed this qualitative research study to explore what is not known, whether the specific biases

and heuristics noted in the decision-making of experts in other fields can be identified in the

hiring practices of educational experts and the degree to which principals express the need or

interest in decision-making tools for the hiring process.

Review of Themes from Research

The multiple rounds of analysis and researcher reflection on the three data sets, once

triangulated, produced seven themes. Five themes under the heading of fast thinking: affect

bias/likeability, confirmation bias/beliefs regarding characteristics of successful teachers, fit,

111

missing data bias, and prediction overconfidence and administrator expertise/confidence. Two

additional themes emerged as components of fast thinking: data review and self-awareness.

These themes were very similar to those from the literature, particularly from that of

bounded rationality. This is to be expected as bounded rationality was used to design the research

question and the research that followed. Additionally, the themes noted also mirrored those from

the other streams of literature reviewed: hiring of teachers and decision-making tools. This

chapter focuses on the implications of the findings for the processes of teacher hiring.

Discussion of Key Findings

The purpose of this study of educational leaders in mid-Atlantic school districts is to

understand the process of teacher hiring and to answer the following research question: How

does bounded rationality affect administrators’ decision-making processes during the process of

hiring new teachers? The research question and inquiry which followed is grounded in bounded

rationality. This research included the analyses of a survey of principals, individual interviews,

and a focus group. The themes identified from this process led the researcher to construct a

number of conclusions. Eight key findings were generated through the triangulation of the data

collected throughout this study. Each finding is an answer to the research question: How does

bounded rationality affect administrators’ decision-making processes during the process of hiring

new teachers?

Finding one: Bounded rationality affects the decision-making of K-8 educational

leaders.

This researcher, in investigating how bounded rationality affects the decision-making of

K-8 educational leaders, presumed that principals, as has been shown for leaders in other fields,

are impacted by bounded rationality. The biases and heuristics deemed as common to all humans

112

and noted in numerous decision-making experiments in the fields of psychology and economics

(Kahneman, 2011) were noted in this study. The predictions for this study based on bounded

rationality appear throughout the transcripts. Additionally, the evidence of bounded rationality in

this research serves to support Kahneman’s (2011) assertion that bounded rationality impacts all

decision-makers, even experts.

Finding two: Principals were individually consistent in the characteristics they sought in

teacher candidates, but the teacher characteristics listed as most important were not consistent

among all principals.

This finding is grounded in bounded rationality as it indicates biases may be present in

the decision-making process of teacher hiring. Experimentally observed bounded rationality

indicates the likeability bias comes into play when evaluating candidates (Selten, 1998). So it

was anticipated that principals would express this common bias. The evidence gathered

surrounding this variance was of the greatest surprise and interest to the researcher.

Perhaps due to the lack of a controlled experiment that viewing live teacher interviews

may have allowed, evidence of general likeability was not an identified characteristic used by

principals. Instead, of note were the many references made to the principals’ backgrounds and

the desire to find teachers with similar backgrounds. If a principal studied special education,

he/she looked for that in candidates. If a principal worked outside of education, that was a

preference expressed for candidates. This researcher anticipated principals’ expressions of a

general likeability bias for candidates, however, another bias not selected as a predicted bias

from bounded rationality, social comparison bias (Kahneman, 2011) the tendency, when making

hiring decisions, to prefer potential candidates who don't compete with one's own particular

strengths, may have been more applicable. Perhaps an area for future study, the tendency to

113

prefer a candidate with a similar background may result from the social comparison bias to avoid

those with conflicting strengths.

Finding three: The concept of the fit of the candidate within the present school climate

was a consideration for principals in the teacher hiring process.

This finding is grounded in bounded rationality as fit is a specific bias listed among the

biases and heuristics identified in bounded rationality.

If the goal of a teacher hiring process is to find the best teacher candidates to work with

the students, the question of fit may replace the characteristics needed for students with those

preferred by colleagues. In this way, the notion of fit may serve as another form of likeability

bias. This would be in keeping with bounded rationality which includes the ideal that when

presented with questions, answers that are derived quickly are more likely to be confidently

perceived as accurate by both the thinker and those around him or her (Kahneman & Tversky,

1984). One can readily, from an intuitive mindset determine if someone is similar to others

already in the group. The answer of fit, is an answer, however, to a different question. It is not an

answer to the question of which teacher will be most effective. It is an answer to which candidate

is most similar by appearance and demeanor to those already present.

Additionally, there is a minority teacher staffing crisis in the United States (Leonardo,

2002). 40% of schools do not have a single teacher of color (Kohli, 2008). If teacher candidates

are considered by the concept of “fit,” continued segregation of staff may occur.

Finding four: Principals did not express concerns of missing data in the teacher hiring

process.

This finding is grounded in bounded rationality as missing data is a specific bias listed

among the biases and heuristics identified in bounded rationality. In this instance, the research

114

question is answered not by what is expressed as a data point, but rather by the principals’ lack

of concern about missing data in their decision-making processes.

Missing data is an oft overlooked situation (Henningsen & Henningsen, 2007; Tversky &

Kahneman, 1974). For decisions to be rational, all pertinent information is required. In the hiring

process of teachers, bits of information may be acknowledged about one candidate, especially

candidates previously known to the hiring committee. If those pieces of background information

are to be used, for rational decision-making, the same information would be needed for all

candidates. This form of missing information was not seen as a concern by the principals

interviewed. (Henningsen & Henningsen, 2007) noted that pausing as individuals or groups to

explore sources for missing information can increase rationality, but the bias limits the mind

from noticing a lack of data and therefore there is no internal alarm to pause. Principals may not

notice that more evidence about one or more of the candidates is even needed in order to fairly

compare the candidates. The use of a committee may create the appearance of the expansion of

perspectives and data gathering; however, additional group members may actually reduce the

shared knowledge, which is given greater value than individual information. When groups meet

to discuss and make decisions, it is the shared, not the unique perspectives, that are given the

greatest deference (Kahneman et al., 2011). Therefore, increasing the number of members of a

group does not necessarily increase the number of choices to be considered. Information that is

unknown to all is not discussed and information that is known to just a few is devalued

(Henningsen & Henningsen, 2007). For decisions to be rational, the data that serves the question

should be given the greatest weight. In group dynamics, information that is well known is given

more value by virtue of it being shared knowledge, not due to the data’s value in the situation.

Therefore, increasing committee size may not improve the rationality of the decision.

115

Finding five: Principals expressed confidence in their abilities to select the best

candidates from a pool of candidates.

This finding is grounded in bounded rationality as confidence in one’s expertise is a

specific bias listed among the biases and heuristics identified in bounded rationality. If principals

are not internally or externally motivated to examine the possible fallibility of their decisions,

this finding provides an answer to the research question. Expert overconfidence (Belsky, 2016)

prevents decision-makers slowing thinking to allow for more rational thought. The principals

who expressed less confidence may actually make better decisions because they access slow

thinking as a means of self-checking. Finding seven addresses this alternate option.

It is the ease with which people are drawn to the simplest answer, as indicated by the ball

and bat test question (Frederick, 2005) that simultaneously generates a feeling of gut confidence

and lowering of engagement of the slow thinking rational mind (Kahneman, 2003). The

Cognitive Reflection Test (CRT) introduced by Frederick (2005), illustrates the working of the

two systems. A bat and a ball cost $1.10. The bat costs one dollar more than the ball. How much

does the ball cost? (Frederick, 2005 p. 25) If the bat costs a full dollar more than the ball, a $.10

ball would require a $1.10 bat and therefore a sum of more than $1.10. Only a $.05 ball and a

$1.05 bat would equal $1.10. The quick thinking of System 1 selects the first seemingly

acceptable answer, $0.10. The faster an answer comes to the human mind, the more accurate it

feels. Only in engaging System 2 is the quick answer exposed as incorrect and the correct answer

of $0.05 for the ball and $1.05 for the bat realized. The faster and easier a decision is made, the

more it feels like the right choice. Similarly, the candidate that is most quickly considered to be

best may feel like the correct choice even if countering evidence is identified further into the

process.

116

Finding six: Principals used protocols and question sets within interviews with the

expressed desire of formalizing and thus improving the selection of teacher candidates.

This finding is grounded in bounded rationality as formalized processes are helpful in

engaging the slow thinking mind that is deemed as rational within the framework of bounded

rationality. For principals to include protocols and questions sets within interview processes, this

finding offers an answer to how principals manage bounded rationality as posed in the research

question.

Protocols have been shown to improve expert decision-making (Flottorp et al., 2013;

Haynes et al., 2009; Kahneman, 2003; Stubenrauch et al., 2012; Takala et al., 2011). The

principals’ use of question sets matches the prescriptions noted in the literature. However, the

paper screening which precedes the live interviews did not have the same level of consistency.

Principals consistently weeded out applicants who did not have the proper certifications, but after

that step, each made individual choices on the remaining resumes based on a wide range of

preferences. No checklists were utilized for this portion of the process. Biases have been shown

to have a profound impact on resume screening in other fields (Krings, Sczesny, & Kluge, 2011).

Finding seven: Principals expressed self-awareness and potential fallibility, keys to

combating fast thinking.

This finding is grounded in bounded rationality and offers an answer to the research

question as self-awareness and consideration of fallibility are helpful in engaging the slow

thinking mind that is deemed as rational within the framework of bounded rationality.

“But knowing you have biases is not enough to help you overcome them. You may

accept that you have biases, but you cannot eliminate them in yourself” (Kahneman et al., 2011,

p. 52). However, biases are more readily noticed in others. Therefore, a properly oriented

117

organization can improve the decision-making process. This is using System 2, rational thought,

to check for erroneous thinking in System 1, intuition. Principals interviewed in the process

expressed that in the end, the selection of a candidate was a best effort selection. “You make a

decision and hope for the best.” Opportunities for the slow thinking mind to critique decisions

could cause principals to reevaluate the considered data. This could improve rationality in the

decision-making process of teacher hiring.

Finding eight: Principals stated the addition of a decision-making tool, such as a

checklist, might improve their processes.

This finding is grounded in bounded rationality as bounded rationality researchers (Singh,

2008; Stubenrauch et al., 2012; Takala et al., 2011) have demonstrated that formalized processes

are helpful in engaging the slow thinking mind that is deemed as rational within the framework

of bounded rationality. Additionally, researchers (Churchman & Doherty, 2010; Flottorp et al.,

2013; Haynes et al., 2009; Kahneman, 2003; Sibbald et al., 2013; Singh, 2008; Stubenrauch et

al., 2012; Takala et al., 2011) argue that experts resist the use of decision-making tools even

when these are empirically proven to improve the decision-making within the expert’s field.

Checklists work well when these include task definitions, recording responses directly on

checklists, and are periodically reviewed by a supervisor (Flottorp et al., 2013). The use of an

agreed upon checklist allows critique and dialogue to be part of an organization’s climate. If a

single individual were asked to fill that role, social pressure might dissuade her/him. The

proponents of this checklist offer implementation suggestions. The checklist should be used for

important, not routine decisions. When in use, it should be fully used. Kahneman (2011)

references the rigor of use of the World Health Organization (WHO) Surgical Safety Checklist as

a model of adherence. When asked about their view of a need and willingness to use a decision-

118

making tool such as a checklist for the hiring process, some principals, as predicted by the

framework expressed their decisions would not be improved by such a tool. Other principals

expressed interest in seeing such a tool developed.

Linking Findings to Theoretical Framework

Bounded rationality, first coined by Simon (1955) offered a response to the dominant

theory of the time, neoclassical rationality, which held that human decisions were calculable,

rational responses with the most utility as the result. Simon suggested humans responded not

with the most logical solutions, but the most readily satisfying (H. A. Simon, 1982). It was this

counter narrative to the notion of rationality that drew this researcher to utilize this lens on the

work of educational leaders. As was the case in the 1950s, today there is a great deal of hope

placed on data-driven decision-making (Cooley et al., 2006; Marzano et al., 2005; Reeves &

Flach, 2011). Specific to this study is the notion of data-driven decisions to influence educational

leader decision-making. This researcher questioned if that premise is correct given the exposed

fallibility of humans, even experts, to accurately interpret data (Botterill & Hindmoor, 2012;

Kahneman, 2003; H. A. Simon, 1982).

The findings of this study indicate the specific biases identified in bounded rationality

indicated as shortcomings to rational thinking can be identified in teacher hiring. This questions

the validity of data-based decision-making as a panacea requiring merely data and the time to

review it. Researchers such as(Campbell et al., 2004; M. Toplak et al., 2011; Tversky &

Kahneman, 1974) have shown the presence of data can actually reduce rational thinking as the

data is accepted as evidence of the assumptions intuitively held. Any advancement in the area of

data-driven decision-making in educational leadership settings requires a clear understanding of

educational leaders’ data filtering processes and the resulting impact on their rationality.

119

Central to this area of research is the specific identification of cognitive biases and

heuristics and the manner in which these impact decisions. Of the many biases and heuristics

identified by the seminal work of Tversky and Kahneman (1974), this research specifically

noted: five themes under the heading of fast thinking: affect bias/likeability, confirmation

bias/beliefs regarding characteristics of successful teachers, fit, missing data bias, and prediction

overconfidence and administrator expertise/confidence. Three additional themes emerged as

components of fast thinking: data review and self-awareness.

Sterman (1989) extended research on the impact of biases and heuristics in individual

decision-making to the implications for organizations. Work since continues to explore the

construct between the benefits of heuristics that allow for decision-making and the potential to

shortcut rational decisions (Kahneman, 2003; Kahneman et al., 2011; Waddell & Sohal, 1994).

Sterman’s (1989) empirical study measured business leaders’ heuristics including anchoring and

adjustment. These subconscious patterns of thinking and preferences filter any incoming

information related to the choices available. This was shown in the responses of participants of

this study. Although there were generalities in what characteristics were desirable in teacher

candidates: e.g. caring, communication, and content knowledge, there were individualized

preferences held by principals that were not shared. Therefore, the decisions individuals would

make when presented with the same data points, interviews and resumes, would not be the same.

The allure of rationality and data driven decision-making is the promise of consistency and

accuracy; bounded rationality and this study suggest those are illusions.

As organizations become increasingly complex, so do the decisions. However, empirical

studies (Eisenhardt & Zbaracki, 1992; Kahneman et al., 2011; Nobre, Tobias, & Walker, 2009;

H. A. Simon, 1982) demonstrate that subjects are insensitive to the feedback from their

120

decisions. This impacts the function of the organizations tremendously. Insensitivity to the

complex surroundings distances the decision makers from reality. Insensitivity to data impedes

the very essence of data-based decision-making. During the interviews of this study, when asked

if the hiring decisions were ever poor or how it could be known if runners up would have been

better, principals responded that on occasion teachers turned out to be less stellar in practice than

the impression from the interview process indicated to them. When that happened, the teacher

was coached or released. The principals, however, did not offer changes to the process to reduce

those errors.

An extension of the problem caused by misinterpretation of data is the issue of

information gap. When information is missing from a data set, people often infer the value of

missing information based on the value of the known information, sometimes bolstering or

diminishing the value (Henningsen & Henningsen, 2007), assuming the groups realize there is

missing information. When groups meet to discuss and make decisions, it is the shared, not the

unique perspectives, that are given the greatest deference (Kahneman et al., 2011). Therefore,

increasing the number of members of a group does not necessarily increase the number of

choices to be considered. Information that is unknown to all is not discussed and information that

is known to just a few is devalued (Henningsen & Henningsen, 2007). Perhaps this is why the

majority of principals expressed the use of a committee did not yield selection of the better or

even different candidates than if a single principal made the choice. However, the addition of the

committee members did increase the acceptance of the new candidate if colleagues were

involved in the selection. This was seen as the best reason to include stakeholders in the

committee process. Some principals mentioned utilizing the post interview discussion with the

other administrators if two or more candidates were acceptable to the principal.

121

Linking Findings to Literature

The literature streams of decision-making as viewed through the multiple lenses of

bounded rationality, techniques utilized in the attempt to increase the rationality of decisions in

the practice of various fields such as business and medicine, and teacher hiring practices

informed the construction of this study and were confirmed as relevant to this study of teacher

hiring.

The findings of this study indicate bounded rationality to be involved in principal

decision-making in the process of teacher hiring. This study demonstrates the similarities

between the decision-making settings in other fields and those of education. There is research,

however, which indicates specific means to ameliorate the deleterious effects of bounded

rationality on decision-making (Bacon et al., 1983; Bartels & Mortenson, 2005; Brañas-Garza et

al., 2012; Cioffi & Markham, 1997; Fudickar et al., 2012; Haynes et al., 2009; Kahneman, 2003;

Place & Vail, 2013; Sibbald et al., 2013).

The decision-making process is more important than the data itself when attempting to

“debias” strategic decisions (Kahneman et al., 2011; Lovallo, D. & Sibony, O., 2010). The hiring

of an individual teacher is not a strategic decision; however, the importance of the impact of the

process over data should be considered in hiring activities. A company’s use of data to make

predictions of the future behaviors of costumers relates to educational leaders’ attempts use data

in order to predict a candidate’s future behavior as a teacher.

It has been shown that business executives can reduce the impact of biases such as

anchoring, weighing one piece of information too heavily, and confirmation, ignoring evidence

that contradicts one’s beliefs (Kahneman et al., 2011). “But knowing you have biases is not

enough to help you overcome them. You may accept that you have biases, but you cannot

eliminate them in yourself” (Kahneman et al., 2011, p. 52). However, biases are more readily

122

noticed in others. Therefore, a properly oriented organization can improve the decision-making

process. The transcripts of this study captured principals responding to the question, “Checklists

and other tools have been shown to offset some of the ways bounded rationality negatively

impacts decision-making. Would you be interested in using such a tool?” As would be

specifically predicted by the researchers Kahneman and Lovallo (2011), some principals

expressed that decision-making tools might help experts in other fields, but not them. They

expressed their expertise was built through experience and no external support would be needed.

Other principals, who expressed more reflective responses to other questions, answered this

question with responses of interest and curiosity as to what a tool for this purpose might look

like. These principals were open to the possibility that a decision-making tool might be helpful.

Research has been conducted previously that might guide decision-making tool

construction for principals to utilize during the hiring process. For example, there are parameters

for use such as those offered by Flottorp et al (2013); checklists work well when these include

task definitions, record responses directly on checklists, and are periodically reviewed by a

supervisor. “The sweet spot for quality control is decisions that are both important and recurring,

and so justify a formal process” (Kahneman, 2011, p. 59). Hiring decisions are possibly the most

important recurring decision principals might make (Cranston, 2012; Pillsbury, 2005).

Surgeons are some of the most formally educated professionals and yet the use of the

World Health Organization’s (WHO) Checklist, even with its most obvious and simple set of

questions, has saved literally thousands of lives from the mistakes of highly skilled experts

(Takala et al., 2011). In this study of teacher hiring, the protocols for processing ranged widely

from the number of individuals involved, the questions asked, the answers considered as correct.

This research would suggest that although also highly educated, the decision-making processes

123

of principals could also become more rational and predictable through the use of research

supported decision-making tools. Referring to the WHO Checklist, Kahneman argues that

experts should not pick and choose from questions, but use these systematically. “Using

checklists is a matter of discipline, not genius. Partial adherence may be a recipe for total failure”

(2011, p. 60). Often in the medical field, nurses express trepidation to speak out when they

disagree with a doctor’s choice (Churchman & Doherty, 2010). The protocol of the WHO

checklist places the charge nurse in a position of power that better distributes the responsibility

to utilize best practices and, therefore, the checklist serves to make questioning a procedure

rather than the act of a provocateur (Kahneman et al., 2011). As this research indicated non-

administrators were not seen as in possession of skills in hiring teachers that was needed by the

principals, like nurses in operating rooms, this may suppress critique and data. A regimented

decision-making procedure could empower all voices and analyses within teacher hiring

committees.

Sunstein and Thaler propose an approach that is less obvious than checklists, this is

choice architecture, “policies selected with the goal of influencing the choices of affected parties

a way that will make those parties better off” (2003, p. 175). Thaler and Sunstein (2009) argue

that people do not always make decisions that are in their best interest and it is the role of

organizations to help them do so. Principals shared that different superintendents have often

brought to the organization preferred procedures for teacher hiring. In that manner, the

superintendents are creating for the organization a means to produce the best outcomes.

However, the principals shared, similar to what this researcher noted in the manner of processes

the principals created when able, the processes differed based on the individual preferences of

each superintendent. So in this study within a county in a particular mid-Atlantic state, not only

124

did each district have differing protocols, these fluctuated depending on which person was in

power at the time.

If education is to hold up data-based decision-making as a means to improve, certainly

consistency in the protocols will be required in order to utilize the most likely system of using

said data. Citing research which includes Gentry (2007) and Stronge and Tucker (2000),

Schumacher, Grigsby, and Vesey (2015) assert the factor which most impacts student learning is

teacher effectiveness. Schools are only as good as the staff that work there (Nonaka, 1994; Place

& Vail, 2013); however, this research adds to the studies that caution that the hiring practices

used to employee staff vary greatly and the criteria for selection are often based on feeling over

evidence, further, hiring is often “information poor” (Liu & Johnson, 2006 p. 331).

Study Limitations

As a qualitative study, this research was strengthened through the analyses of the

transcripts. However, as there were a relatively small number, five participants interviewed, a

fuller range of principal perspectives may have been missed. To mitigate the potential impact

from the limited number of participants, the researcher attended to all evidence through the

analysis of all three data sets as unique data and also as the three data sets constructed

trustworthy information when triangulated.

Research can unfold to generate a significant aspect of the case not predicted by the

researcher during the research design (Creswell, 2012). This researcher remained open to this

possibility during the data gathering, reflection, and analysis. The researcher anticipated teacher

hiring to be significantly impacted by the general likeability of the teacher candidates. However,

there was greater evidence that the selected candidates presented themselves as those having the

characteristics the individual administrators preferred in teachers. The noted biases were not

125

about general likeability, but what the principals liked to see in teachers because they liked to see

those characteristics in themselves. This research originated as a study of the procedures

principals used to increase rationality in decision-making, but following the data, explored the

consideration of what principals consider effective teaching such as work ethic, relationship-

building, content knowledge, pedagogy, background experiences. This study was designed to

focus on the experience of biases that shortcut decisions, not the relative validity of the possible

data points principals consider when selecting teachers. However, the principals had dedicated

years of focus on the characteristics they wanted to see in teacher candidates, not the process

itself. Kahneman (2011) suggests the intuitive mind is fast because it goes unnoticed by the

individual. Therefore, a qualitative study that asks participants to note or express their fast

thinking may not be as successful identifying those biases as the quantitative studies upon which

bounded rationality is built (Kahneman & Tversky, 1984; H. A. Simon, 1955; M. Toplak et al.,

2011). Follow-up research examining the characteristics principals consider as indicative of

effective teachers is warranted.

In a consideration of limitations, it is useful to reflect on the impact of this researcher’s

inherent expertise with the research area of inquiry. With twenty-nine years as a teacher,

administrator, and trainer for novice teachers, this researcher is positioned to have shared

experiences with the participants. This may have benefited the study in the understanding of

vernacular, realities, processes, and legalities involved in the teacher hiring process. This lived

experience in the process being studied may have served the researcher to note anomalies and

patterns. Additionally, as the participants and school districts were known to the researcher prior

to the study, background knowledge was available when analyzing the data points. However, it

should be noted that as a researcher-practitioner working in the county studied, this researcher’s

126

own biases regarding teacher hiring, along with the researcher’s own mental biases of slow and

fast thinking through which the data was analyzed, impacted the results of the study.

Principals expressed varied preferences in the characteristics of effective teachers and the

means to identify these. This researcher spent a career forming opinions on the topic, as well as,

years studying the related literature. This generated the researcher’s mindset related to the topic

prior to the study. The very design of the research as well as the analysis offered may be

impacted by the researcher’s prior knowledge of the topic of teacher preparation and pedagogy.

Alternate explanations

The main critique of the study could be the same fallibility of decision-making examined

in the study, the inability of the human mind to be rational and to reflect accurately on its

rationality. When someone is asked about how he/she does something, the response is filtered by

the very filters the questioner attempts to identify (Kahneman, 2011; Langlois, 1990; H. A.

Simon, 1982; H. A. Simon, 1991; Sunstein et al., 2002). Asking someone if s/he is effective

when teaching or rational when hiring, always yields the interpretations of the actor, not the

observable actions. Additionally, it should again be noted that this research is impacted by

bounded rationality and the conclusions drawn through the data gathering and analysis process

may have been erroneously filtered by biases and heuristics. One of the biases noted in bounded

rationality is missing data bias, failing to notice what additional information is needed to make a

decision. Although triangulation and reflection were employed to maximize accurate

interpretation of the data; the researcher may have missed data that would have offered alternate

explanations.

127

Implications for Educational Practice

This study builds a case for additional research into the area of decision-making in

educational leadership and specifically whether the negative impacts of natural biases and

heuristics on decision-making can be noted in the decision-making of educational leaders in the

process of hiring teachers. Decision-making tools such as checklists have been used to improve

decision-making in the fields including business and medicine without hampering expert

decision-making speed (Bacon et al., 1983; Bekker, 2010; Fudickar et al., 2012; Haynes et al.,

2009; Kahneman et al., 2011; Sibbald et al., 2013). This researcher examined the thinking

process of educational experts in order to categorize the biases and heuristics that are most

commonly involved. Further, this researcher designed this study in order to understand whether

rational decision-making could be made more overt and therefore manageable. If a checklist or

flowchart could be effectively applied when educators engage in complex decision-making,

specifically, teacher hiring, improved decision making could be the result.

In keeping with the research (Rutledge et al., 2008), of all the steps in the teacher hiring

process, principals placed the highest importance on interviews, above that of resumes and

reference checks, second and third most important respectively (Cranston, 2012; Rutledge et al.,

2008). Given the potential for bias in the review of all three of those data points, care should be

given to the processes of educational leaders.

To control for interviewer bias, researchers (Engel & Finch, 2015; Haskins & Loeb,

2007; Heneman & Milanowski, 2004; Lange, Range, & Welsh, 2012; Schumacher et al., 2015)

have recommended job specific criteria and interview protocols that focus administrators on

specific characteristics be utilized. This research identified all principals as using sets of

questions, however, the question sets did not indicate the preferable responses. That was left to

the individual principal or committee member. This may allow for the bias noted in this study to

128

emerge, hidden by the seeming rational formality of a set of questions. “Organizations need to

realize that a disciplined decision-making process, not individual genius, is the key to a sound

strategy. And they will have to create a culture of open debate in which such processes can

flourish” (Kahneman et al., 2011 p. 60). Continued research is needed to understand Kahneman’s

assertion in the context of educational organizations.

Assistance in teacher hiring is increasingly viable given the ongoing improvements in

technology. Scholars have provided evidence of the increased ability to construct effective

decision making tools. Computerized model-driven decision support systems (DSS) exist to

provide decision support for leaders (Power & Sharda, 2007). This could allow for a decision-

making tool for educators beyond a static form like the WHO Surgical Safety Checklist.

Dynamic Decision Support Systems (DSS), computer-based systems that help decision makers

confront ill-structured problems through direct interaction with data and analysis models

(Sprague & Carlson, 1982) which organize the increasingly quantifiable data available to

educational leaders might be an area worthy of study.

Implications for Future Research

This researcher explored the applicability of bounded rationality, which is generally

studied within the domains of psychology and economics, to that of education. Specifically, this

researcher considered whether the limits of decision-making noted in the literature also impact

the decisions of principals when hiring teachers. This study utilized principals’ descriptions of

beliefs and recollections. Further studies could utilize the quantitative methods used within

bounded rationality (Hoppe & Kusterer, 2011; Kahneman, 2011; Lovallo, D. & Sibony, O.,

2010; Singh, 2008; Sunstein & Thaler, 2003) to the specific context of teacher hiring.

129

The paper review process is just one portion of the teacher hiring process that could be

examined. Studies of resume review have indicated demographic biases such as ageism even

when age data is not present (Krings et al. 2011). Job related information in resumes might serve

as implicit cues about applicant characteristics. Controlled studies allow for use of the same

stimuli on multiple participants. By controlling for variances in a quantitative study the specific

biases and heuristics involved in teacher hiring might be identified in ways this research did not.

Identification of the specific fast thinking involved in the paper review of applicants might offer

specific challenges to rational thought in the hiring process. Such research might include answers

to the research question, how does bounded rationality affect administrators’ decision-making

processes during the process of hiring new teachers? From that increased understanding of the

biases and heuristics involved in the screening process, protocols to reduce their impact may be

designed.

The interview process is another portion of the teacher hiring process that could be

examined. This qualitative study offered information on principals’ recollections of interviews of

varied candidates and the accompanying data sets of applications and interviews. Empirical

studies could reduce the variance of events and the filter of single source memories. Quantitative

studies in the field of education, and teacher hiring specifically, might pinpoint the biases and

heuristics most impactful and thus answer the research question.

This study’s results and findings are transferable to other contexts within education or

wider research in the area of decision-making. The study’s clarity and depth of description

allows the study to be applicable to other contexts. This was accomplished through the

establishment of the context of the study and the narrative of the transcription and coding. As a

study of the experiences of a sample group of principals in K-8 public school settings, this study

130

is specifically transferable to principals in similar settings engaged in the process of hiring

teachers.

The next step for this research is the creation and evaluation of the efficacy of decision-

making tools in the practice of teacher hiring. In educational settings, performance feedback to

team members related to adherence to problem solving protocols was not effective until coupled

with a checklist (Bartels & Mortenson, 2005). The use of checklists is a very tangible and overt

attempt to counter bounded rationality. If leaders are willing and able to generate a culture of

systematic and transparent decision-making, biases can be minimized (Courtney et al., 2013). In

addition to overt tools such as checklists, subtler nudges in decision-making could be established

by schools. A nudge is “any aspect of the choice architecture that alters people’s behavior in a

predictable way” (Thaler, 2009, p. 6). Researchers could explore the use of the nudges of choice

architecture to extend the research question. The expanded inquiry could ask whether nudges,

such as the inclusion of administrator reflection questions which might slow thinking in order to

reduce the dependence on fast thinking and intuition, improve the administrators’ decision-

making processes during the process of hiring new teachers.

Additional qualitative studies could be used to investigate this study’s research question.

A number of aspects could be considered: e.g. alternate locations, expanded pools of candidates,

different interview questions, a researcher without experience in K-8 education. These additional

studies could establish the dependability of this study as well as generate findings beyond the

eight noted in this study.

Conclusion

As the demands upon and stakes for the success of educational leaders continue to

increase, it is crucial that principals are highly successful in what some (Cranston, 2012;

131

Pillsbury, 2005) consider the most important function of principals: the hiring of teachers. This

research has shown the process of teacher hiring is not optimized. Although the participants in

this study were found to be passionate and knowledgeable about the process of teacher hiring,

they were left without the supports offered to experts in other fields, such as doctors, pilots, and

business leaders. There was great variance in what each principal stated he/she considered as

indicative of prime candidates. This researcher has concluded the process of teacher hiring,

although considered a crucial task, is not given the level of support and oversight dedicated to so

many other aspects of adminstrators’ decision-making. There are manuals of protocols based on

policies and regulations. Most of these are related to student and staff safety, contracts,

expenditures, State Department of Education requirements. However, the actual practice of

teacher hiring varies greatly, is often “information poor,” and the criteria for selection is often

based on feeling over evidence (Liu & Johnson, 2006 p. 331). Improvement in the hiring process

could produce significant improvement in student achievement (Heneman & Milanowski, 2004).

There are regulations designed to ensure the quality instruction through the ongoing evaluation

and professional development of teachers (Chingos & Peterson, 2011). The findings of this study

indicate systemic efforts must be made to the processes that actually place teachers in schools in

the first place.

The impact of bounded rationality on teacher hiring practices requires the attention of

educational leaders. Suboptimal decision-making caused by missing data, the misinterpretation

of data, or overconfidence generated by the presence of some data could undermine the crucial

work of the hiring process and, therefore, the quality of instruction provided to students.

132

References

Adams, R. J., & Ericsson, A. E. (2000). Introduction to cognitive processes of expert pilots. Human

Performance in Extreme Environments, 5(1), 44-62.

Agee, J. (2009). Developing qualitative research questions: A reflective process. International Journal of

Qualitative Studies in Education (QSE), 22(4), 431-447.

Arnold, V., Clark, N., Collier, P., Leech, S., & Sutton, S. (2006). The differential use and effect of

knowledge-based system explanations in novice and expert judgment decisions. MIS Quarterly,

30(1), 79-97.

Bacon, D. L., Fulton, B. J., & Malott, R. W. (1983). Improving staff performance through the use of task

checklists. Journal of Organizational Behavior Management, 4(3), 17-25.

doi:10.1300/J075v04n03_03

Bartels, S. M., & Mortenson, B. P. (2005). Enhancing adherence to a problem-solving model for middle-

school pre-referral teams: A performance feedback and checklist approach. Journal of Applied

School Psychology, 22(1), 109-123. doi:10.1300/J370v22n01_06

Bekker, H. L. (2010). The loss of reason in patient decision aid research: Do checklists damage the

quality of informed choice interventions? Patient Education and Counseling, 78(3), 357-364.

doi:10.1016/j.pec.2010.01.002

Bernheim, B., & Rangel, A. (2004). Addiction and cue-triggered decision processes. The American

Economic Review, 94(5), 1558-1590.

Beshears, J., & Gino, F. (2015). Leaders as decision architects. Harvard Business Review, 93(5), 52-62.

Bingham, C. B., & Haleblian, J. (. (2012). How firms learn heuristics: Uncovering missing components of

organizational learning. Strategic Entrepreneurship Journal, 6(2), 152-177. doi:10.1002/sej.1132

133

Botterill, L. C., & Hindmoor, A. (2012). Turtles all the way down: Bounded rationality in an evidence-

based age. Policy Studies, 33(5), 367-379. doi:10.1080/01442872.2011.626315

Brañas-Garza, P., García-Muñoz, T., & González, R. H. (2012). Cognitive effort in the beauty contest

game. Journal of Economic Behavior & Organization, 83(2), 254-260.

doi:http://dx.doi.org.ezproxy.neu.edu/10.1016/j.jebo.2012.05.018

Briscoe, F. M. (2005). A question of representation in educational discourse: Multiplicities and

intersections of identities and positionalities. Educational Studies, 38(1), 23-41.

doi:10.1207/s15326993es3801_4

Busenitz, L. W., & Barney, J. B. (1997). Differences between entrepreneurs and managers in large

organizations: Biases and heuristics in strategic decision-making. Journal of Business Venturing,

12(1), 9-30.

Butin, D. W. (2010). The education dissertation: A guide for practitioner scholars. Thousand Oaks, CA

US: Corwin Press. Retrieved from

http://ezproxy.neu.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN

=2009-23444-000&site=ehost-live

Cain-Caston, M. (1999). A survey of opinions of north carolina school administrators regarding factors

considered most important in hiring teachers for their first teaching position.(statistical data

included). Journal of Instructional Psychology, 26(2), 69.

Campbell, W. K., Goodie, A. S., & Foster, J. D. (2004). Narcissism, confidence, and risk attitude. Journal

of Behavioral Decision Making, 17(4), 297-311. doi:10.1002/bdm.475

Campitelli, G., & Gerrans, P. (2014). Does the cognitive reflection test measure cognitive reflection? A

mathematical modeling approach. Memory & Cognition, 42(3), 434-447.

134

Carlin, B. I., Gervais, S., & Manso, G. (2013). Libertarian paternalism, information production, and

financial decision making. Review of Financial Studies, 26(9), 2204-2228.

Chingos, M. M., & Peterson, P. E. (2011). It's easier to pick a good teacher than to train one: Familiar and

new results on the correlates of teacher effectiveness. Economics of Education Review, 30(3), 449-

465. doi:http://dx.doi.org.ezproxy.neu.edu/10.1016/j.econedurev.2010.12.010

Churchman, J. J., & Doherty, C. (2010). Nurses' views on challenging doctors' practice in an acute

hospital. Nursing Standard, 24(40), 42-47.

Cioffi, J., & Markham, R. (1997). Clinical decision-making by midwives: Managing case complexity.

Journal of Advanced Nursing, 25(2), 265-272. doi:10.1046/j.1365-2648.1997.1997025265.x

Clement, M. C. (2000). Essentials for principals: How to interview, hire, and retain high- quality new

teachers

Clement, M. C. (2009). Hiring highly qualified teachers begins with quality interviews: Past behavior is

the best predictor of future performance, so educators would be wise to craft interview questions that

explore past experiences, skills, and behaviors of job candidates. Phi Delta Kappan, 91, 22+.

Clement, M. C. (2013). Hiring good colleagues: What you need to know about interviewing new teachers.

The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 86(3), 99-102.

doi:10.1080/00098655.2013.769930

Cooley, V. E., Shen, J., Miller, D. S., Winograd, P. N., Rainey, J. M., Yuan, W., & Ryan, L. (2006). Data-

based decision-making: Three state-level educational leadership initiatives. Educational Horizons,

85(1), 57-64.

Courtney, H., Lovallo, D., & Clarke, C. (2013). Deciding how to decide. (cover story). Harvard Business

Review, 91(11), 62-70.

135

Cranston, J. A. (2012). Evaluating prospects: The criteria used to hire new teachers. Alberta Journal of

Educational Research, 58(3), 350-367.

Creswell, J. W. (2012). Educational research : Planning, conducting, and evaluating quantitative and

qualitative research. Boston: Pearson.

Darling-Hammond, L. (2010). Teacher education and the american future. Journal of Teacher Education,

61(1), 35.

Eisenhardt, K. M., & Zbaracki, M. J. (1992). Strategic decision making. Strategic Management Journal,

13, 17-37.

Engel, M. (2013). Problematic preferences? A mixed method examination of principals' preferences for

teacher characteristics in chicago. Educational Administration Quarterly, 49(1), 52-91.

doi:10.1177/0013161X12451025

Engel, M., & Cannata, M. (2015). Localism and teacher labor markets: How geography and decision

making may contribute to inequality. Peabody Journal of Education, 90(1), 84-92.

doi:10.1080/0161956X.2015.988533

Engel, M., & Finch, M. A. (2015). Staffing the classroom: How urban principals find teachers and make

hiring decisions. Leadership and Policy in Schools, 14(1), 12-41.

doi:10.1080/15700763.2014.983131

Flottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., . . . Eccles,

M. P. (2013). A checklist for identifying determinants of practice: A systematic review and synthesis

of frameworks and taxonomies of factors that prevent or enable improvements in healthcare

professional practice. Implementation Science, 8(1), 1-11. doi:10.1186/1748-5908-8-35

Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives,

19(4), 25-42.

136

Fudickar, A., Hörle, K., Wiltfang, J. ö, & Bein, B. (2012). The effect of the WHO surgical safety

checklist on complication rate and communication. Deutsches Ärzteblatt International, 109(42),

695-701.

Funk, K. M., & Axelrod, S. (2011). Review of 'the checklist manifesto—How to get things right'. Child &

Family Behavior Therapy, 33(1), 70-72. doi:10.1080/07317107.2011.545016

Gentry, R. (2012). Implementing promising practices to prepare quality teacher educators. Journal of

College Teaching & Learning, 9(3), 179-188.

Gigerenzer, G. (2010). Moral satisficing: Rethinking moral behavior as bounded rationality. Topics in

Cognitive Science, 2(3), 528-554. doi:10.1111/j.1756-8765.2010.01094.x

Grantcharov, T. P., & Reznick, R. K. (2009). Training tomorrow’s surgeons: What are we looking for and

how can we achieve it? ANZ Journal of Surgery, 79(3), 104-107. doi:10.1111/j.1445-

2197.2008.04823.x

Guerra-López, I., & Blake, A. M. (2011). Leadership decision making and the use of data. Performance

Improvement Quarterly, 24(2), 89-104. doi:10.1002/piq.20113

Haberman, M. (2012). The myth of the "fully qualified" bright young teacher.(author abstract). American

Behavioral Scientist, 56(7), 926-940.

Halevy, N., & Chou, E. Y. (2014). How decisions happen: Focal points and blind spots in interdependent

decision making. Journal of Personality and Social Psychology, 106(3), 398-417.

doi:10.1037/a0035351

Hanushek, E. A. (2011). The economic value of higher teacher quality. Economics of Education Review,

30(3), 466-479. doi:10.1016/j.econedurev.2010.12.006

Hargreaves, A., & Fullen, M. (2009). Change wars. Bloomington: Solution Tree.

137

Harris, D. N., & Sass, T. R. (2011). Teacher training, teacher quality and student achievement. Journal of

Public Economics, 95(7), 798-812. doi:10.1016/j.jpubeco.2010.11.009

Haskins, R., & Loeb, S. (2007). A plan to improve the quality of teaching. The Education Digest, 73(1),

51-56.

Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A. S., Dellinger, E. P., . . . Gawande,

A. A. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. N

Engl J Med, 360(5), 491-499. doi:10.1056/NEJMsa0810119

Heneman, H. G., & Milanowski, A. T. (2004). Alignment of human resource practices and teacher

performance competency. Peabody Journal of Education, 79(4), 108-125.

doi:10.1207/s15327930pje7904_6

Henningsen, D. D., & Henningsen, M. L. M. (2007). Do groups know what they don't know? dealing with

missing information in decision-making groups. Communication Research, 34(5), 507-525.

doi:10.1177/0093650207305594

Hoffman, A. R. (1995). Landing the job: A survey of new teachers. Education, 116(2), 279.

Hoppe, E. I., & Kusterer, D. J. (2011). Behavioral biases and cognitive reflection. Economics Letters,

110(2), 97-100. doi:http://dx.doi.org.ezproxy.neu.edu/10.1016/j.econlet.2010.11.015

Hutchinson, J. W., Alba, J. W., & Eisenstein, E. M. (2010). Heuristics and biases in data-based decision

making: Effects of experience, training, and graphical data displays. Journal of Marketing Research,

47(4), 627-642. doi:10.1509/jmkr.47.4.627

Imai, K., & Yamamoto, T. (2010). Causal inference with differential measurement error: Nonparametric

identification and sensitivity analysis. American Journal of Political Science, 54(2), 543-560.

doi:10.1111/j.1540-5907.2010.00446.x

138

Janis, I. L. (. L. (1982). Groupthink : Psychological studies of policy decisions and fiascoes (2nd ed. ed.).

Boston: Boston : Houghton Mifflin.

Jenlink, P. M. (2005). Editorial: On bricolage and the intellectual work of the scholar- practitioner.

Scholar-Practitioner Quarterly, 3(1), 3.

Jevons, W. S. (1888). The theory of political economy London: Macmillan and Co. ; New York:

Macmillan and Co.

Jopp, D. A. (2005). Formal and informal collective learning systems: Understanding the emergent flow of

knowledge (Ed.D.). Available from ProQuest Dissertations & Theses Full Text. (304998750).

Retrieved from http://search.proquest.com.ezproxy.neu.edu/docview/304998750?accountid=12826

Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American

Psychologist, 58(9), 697-720.

Kahneman, D. (2011). Thinking, fast and slow (1st ed. ed.). New York: New York : Farrar, Straus and

Giroux.

Kahneman, D., Lovallo, D., & Sibony, O. (2011). Before you make that big decision. Harvard Business

Review, 89(6), 50-60.

Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American Psychologist, 39(4), 341-

350. doi:10.1037/0003-066X.39.4.341

Kayes, P. E. (2006). New paradigms for diversifying faculty and staff in higher education: Uncovering

cultural biases in the search and hiring process. Multicultural Education, 14(2), 65-69.

Kersten, T. (2010). Using a group process in teacher hiring. School Administrator, 67(8), 36-37.

Kunc, M. H., & Morecroft, J. D. W. (2010). Managerial decision making and firm performance under a

resource-based paradigm. Strategic Management Journal, 31(11), 1164-1182.

139

Lange, C., Range, B., & Welsh, K. (2012). Conditions for effective data use to improve schools:

Recommendations for school leaders. International Journal of Educational Leadership Preparation,

7(3)

Langlois, R. N. (1990). Bounded rationality and behavioralism: A clarification and critique. Journal of

Institutional and Theoretical Economics (JITE) / Zeitschrift Für Die Gesamte Staatswissenschaft,

146(4), 691-695.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park: Sage Publications.

Liu, E., & Johnson, S. M. (2006). New teachers' experiences of hiring: Late, rushed, and information-

poor. Educational Administration Quarterly, 42(3), 324-360. doi:10.1177/0013161X05282610

Lovallo, D. & Sibony, O. (2010). The case for behavioral strategy. McKinsey Quarterly, 2(3), 30-43.

Lovallo, D., Clarke, C., & Camerer, C. (2012). Robust analogizing and the outside view: Two empirical

tests of case-based decision making. Strategic Management Journal, 33(5), 496-512.

doi:10.1002/smj.962

Machi, L. A., & McEvoy, B. T. (2012). The literature review: Six steps to success. second edition

Corwin.

Mackenzie, C. (1997). Where are the motives? A problem with evidence in the work of richard thaler.

Journal of Economic Psychology, 18(1), 123-135. doi:10.1016/S0167-4870(96)00050-5

Major, M. L. (2013). How they decide: A case study examining the decision-making process for keeping

or cutting music in a K–12 public school district. Journal of Research in Music Education, 61(1), 5-

25.

Manimala, M. J. (1992). Entrepreneurial heuristics - a comparison between high pi (pioneering-

innovative) and low pi ventures Elsevier Science BV.

140

Marzano, R., Waters, T., & McNulty, B. (2005). School leadership that works: From research to results.

Alexandria, VA: ASCD.

Mason, R. W., & Schroeder, M. P. (2010). Principal hiring practices: Toward a reduction of uncertainty.

The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 83(5), 186-193.

doi:10.1080/00098650903583727

Medin, D., & Bazerman, M. (1999). Broadening behavioral decision research: Multiple levels of

cognitive processing. Psychonomic Bulletin & Review, 6(4), 533-546. doi:10.3758/BF03212961

Ménard, J. (2010). A ‘Nudge’ for public health ethics: Libertarian paternalism as a framework for ethical

analysis of public health interventions? Public Health Ethics, 3(3), 229-238.

doi:10.1093/phe/phq024

Mumby, D. K., & Putnam, L. L. (1992). The politics of emotion: A feminist reading of bounded

rationality. The Academy of Management Review, 17(3), 465-486. doi:10.2307/258719

Murnane, R. J., & Steele, J. L. (2007). What is the problem? the challenge of providing effective teachers

for all children. The Future of Children, 17(1), 15-43.

6A:16-8.2 functions of intervention and referral services, NJ 6A:16-8.2U.S.C. 6A:16-8.2 (October 2014).

Retrieved from http://www.state.nj.us/education/code/current/title6a/chap16.pdf

NJDOE. (2016). Educator preparation provider annual report. Retrieved from

http://www.state.nj.us/education/educators/rpr/preparation/providers/2016/provider.shtml

Nobre, F. S., Tobias, A. M., & Walker, D. S. (2009). The impact of cognitive machines on complex

decisions and organizational change. AI & Society, 24(4), 365-381. doi:10.1007/s00146-009-0207-4

Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1),

14-37.

141

Oppenheimer, D. (2012). Hard-to-read fonts promote better recall. Harvard Business Review, 90(3), 32-

33.

Our future, our teachers. (2011, 10/14; 2016/6). ED Review Retrieved from

http://libraries.state.ma.us.ezproxy.neu.edu/login?gwurl=http://go.galegroup.com.ezproxy.neu.edu/p

s/i.do?id=GALE%7CA271817555&v=2.1&u=mlin_b_northest&it=r&p=ITOF&sw=w&asid=ad0f9

9bee68b95a7038c9a56a94f5a6e

Papa, F., & Baxter, I. (2008). Hiring teachers in new york's public schools: Can the principal make a

difference? Leadership and Policy in Schools, 7(1), 87-117. doi:10.1080/15700760701655524

Peterson, K. D. (2002). In ebrary I. (Ed.), Effective teacher hiring a guide to getting the best. Alexandria,

Va.: Alexandria, Va. : Association for Supervision and Curriculum Development.

Pillsbury, P. (2005). Only the best: Hiring outstanding teachers: A great curriculum in the hands of a

mediocre teacher--even one with a credential--is nothing more than a mediocre curriculum.

Leadership, 35(2), 36.

Place, A. W., & Vail, D. S. (2013). The effects of age, years of experience, and type of experience in the

teacher selection process. AASA Journal of Scholarship & Practice, 10, 8+.

Ponterotto, J. G. (2005). Qualitative research in counseling psychology: A primer on research paradigms

and philosophy of science. Journal of Counseling Psychology, 52(2), 126-136.

Power, D. J., & Sharda, R. (2007). Model-driven decision support systems: Concepts and research

directions. Decision Support Systems, 43(3), 1044-1061. doi:10.1016/j.dss.2005.05.030

Rabin, M., & Schrag, J. L. (1999). First impressions matter: A model of confirmatory bias. The Quarterly

Journal of Economics, 114(1), 37-82.

Reeves, D. B., & Flach, T. (2011). Data: Meaningful analysis can rescue schools from drowning in data.

Journal of Staff Development, 32(4), 34-38,.

142

Rowan, B. (1994a). Comparing teachers' work with work in other occupations: Notes on the professional

status of teaching. Educational Researcher, 23(6), 4-17. doi:10.3102/0013189X023006004

Rowan, B. (1994b). Comparing teachers' work with work in other occupations: Notes on the professional

status of teaching. Educational Researcher, 23(6), 4-17. doi:10.3102/0013189X023006004

Rutledge, S., Harris, D., Thompson, C., & Ingle, W. K. (2008). Certify, blink, hire: An examination of the

process and tools of teacher screening and selection. Leadership and Policy in Schools, 7(3), 237-

263. doi:10.1080/15700760701822132

Ryan, F., Coughlan, M., & Cronin, P. (2007). Step-by-step guide to critiquing research. part 2: Qualitative

research. British Journal of Nursing, 16(12), 738-744.

Saldana, J. (2009). The coding manual for qualitative researchers. London ; Thousand Oaks, Calif.:

London ; Thousand Oaks, Calif. : Sage.

Schlag, P. (2010). Nudge, choice architecture, and libertarian paternalism. Michigan Law Review, 108(6),

913-924.

Schumacher, G., Grigsby, B., & Vesey, W. (2015). Determining effective teaching behaviors through the

hiring process. International Journal of Educational Management, 29(1), 139-155.

doi:10.1108/IJEM-04-2013-0071

Schwandt, D. R. (2005). When managers become philosophers: Integrating learning with sensemaking

Shadish, W. R., & Cook, T. D. (1999). Comment-design rules: More steps toward a complete theory of

quasi- experimentation. Statistical Science, 14(3), 294-300.

Shen, J., Cooley, V. E., Ma, X., Reeves, P. L., Burt, W. L., Rainey, J. M., & Yuan, W. (2012). Data-

informed decision making on high-impact strategies: Developing and validating an instrument for

principals. Journal of Experimental Education, 80(1), 1-25. doi:10.1080/00220973.2010.550338

143

Short, D. C., & Shindell, T. J. (2009). Defining HRD scholar-practitioners. Advances in Developing

Human Resources, 11(4), 472-485. doi:10.1177/1523422309342225

Sibbald, M., De Bruin, A., & Van Merrienboer, J. (2013). Checklists improve experts' diagnostic

decisions

. Medical Education, 47(3), 301-308.

Simon, H. (2000). Bounded rationality in social science: Today and tomorrow. Mind & Society, 1(1), 25-

39. doi:10.1007/BF02512227

Simon, H. A. (1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1),

99-118.

Simon, H. A. (1982). Models of bounded rationality. Cambridge, Mass.: Cambridge, Mass. : MIT Press.

Simon, H. A. (1991). Bounded rationality and organizational learning. Organization Science, 2(1), 125-

134.

Singh, J. (2008). Impostors masquerading as leaders: Can the contagion be contained? Journal of

Business Ethics, 82(3), 733-745. doi:10.1007/s10551-007-9588-9

Sterman, J. D. (1989). Modeling managerial behavior: Misperceptions of feedback in a dynamic decision

making experiment

Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good?: A cross-case

analysis of the connection between teacher effectiveness and student achievement. Journal of

Teacher Education, 62(4), 339-355. doi:10.1177/0022487111404241

Stubenrauch, S., Schneid, E., Wünsch, A., Helmes, A., Bertz, H., Fritzsche, K., . . . Gölz, T. (2012).

Development and evaluation of a checklist assessing communication skills of oncologists: The

COM-ON-checklist. Journal of Evaluation in Clinical Practice, 18(2), 225-230. doi:10.1111/j.1365-

2753.2010.01556.x

144

Sunstein, C. R., Kahneman, D., Schkade, D., & Ritov, I. (2002). Predictably incoherent judgments.

Stanford Law Review, 54(6), 1153-1215.

Sunstein, C. R., & Thaler, R. (2003). Libertarian paternalism. American Economic Review,

Takala, R. S. K., Pauniaho, S. L., Kotkansalo, A., Helmio, P., Blomgren, K., Helminen, M., . . . Ikonen,

T. S. (2011). A pilot study of the implementation of WHO surgical checklist in finland:

Improvements in activities and communication. Acta Anaesthesiologica Scandinavica, 55(10), 1206-

1214. doi:10.1111/j.1399-6576.2011.02525.x

Thaler, R. H. (2009). In Sunstein C. R. (Ed.), Nudge : Improving decisions about health, wealth, and

happiness (Rev. and expanded ed. ed.). New York: New York : Penguin Books.

Thaler, R. H., & Sunstein, C. R. (2003). Libertarian paternalism. The American Economic Review, 93(2,

Papers and Proceedings of the One Hundred Fifteenth Annual Meeting of the American Economic

Association, Washington, DC, January 3-5, 2003), 175-179.

Timmons, L. J. (1998, 01; 2016/9). Interviewing techniques for hiring quality employees in the OR.67,

222+. Retrieved from

http://libraries.state.ma.us.ezproxy.neu.edu/login?gwurl=http://go.galegroup.com.ezproxy.neu.edu/p

s/i.do?p=AONE&sw=w&u=mlin_b_northest&v=2.1&it=r&id=GALE%7CA20215366&asid=18c51

145a92b324e11c2e91c25380489

Toplak, M. E., West, R. F., & Stanovich, K. E. (2014). Assessing miserly information processing: An

expansion of the cognitive reflection test. Thinking & Reasoning, 20(2), 147-168.

doi:10.1080/13546783.2013.844729

Toplak, M., West, R., & Stanovich, K. (2011). The cognitive reflection test as a predictor of performance

on heuristics-and-biases tasks. Memory & Cognition, 39(7), 1275-1289. doi:10.3758/s13421-011-

0104-1

145

Tufford, L., & Newman, P. (2012). Bracketing in qualitative research. Qualitative Social Work, 11(1), 80-

96. doi:10.1177/1473325010368316

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science,

185(4157), 1124-1131.

Title II — preparing, training, and recruiting high quality teachers and principals, SEC. 2101 | SEC. 2102

| SEC. 2103 (09/15/2004). Retrieved from http://www2.ed.gov/policy/elsec/leg/esea02/pg20.html

Higher education opportunity act: Teacher quality enhancement, 201 (8/14/2008). Retrieved from

https://www.gpo.gov/fdsys/pkg/PLAW-110publ315/html/PLAW-110publ315.htm

Venkatraman, V., Payne, J. W., & Huettel, S. A. (2014). An overall probability of winning heuristic for

complex risky decisions: Choice and eye fixation evidence. Organizational Behavior and Human

Decision Processes, 125(2), 73-87. doi:10.1016/j.obhdp.2014.06.003

Waddell, D., & Sohal, A. S. (1994). Forecasting: The key to managerial decision making. Management

Decision, 32(1), 41-49. doi:10.1108/00251749410050697

Whitworth, J., Jones, S., Deering, T., & Hardy, S. (2016). Hiring quality teachers: The devil is in the

details. Professional Educator, 40(1), 20-32.

Wilson, S. M. (2010). Decision-making by school psychologists: Use of the representativeness heuristic

and importance of assessment data in determination of special education eligibility (Ph.D.).

Available from ProQuest Dissertations & Theses Global. (746787076). Retrieved from

http://search.proquest.com.ezproxy.neu.edu/docview/746787076?accountid=12826

World health organization: Patient safety. (2009). Retrieved from

http://www.who.int/patientsafety/safesurgery/en/

Yin, R. K. (2014). Case study research : Design and methods (5th ed. ed.). Los Angeles, Calif.: Los

Angeles, Calif. : Sage Publications.

146

Yin, R. K., & Davis, D. (2007). Adding new dimensions to case study evaluations: The case of evaluating

comprehensive reforms. New Directions for Evaluation, 2007(113), 75-93. doi:10.1002/ev.216

Young, I., & Joseph, G. (1989). Effects of chronological age, skill obsolescence, quality of information,

and focal position on evaluations by principals. Journal of Personnel Evaluation in Education, 2(4),

355-365. doi:10.1007/BF00139650

Young, I. P., & Delli, D. A. (2002). The validity of the teacher perceiver interview for predicting

performance of classroom teachers. Educational Administration Quarterly, 38(5), 586-612.

147

Appendix A

Permission Letter to Superintendent of Schools

Dear ________ County Superintendents,

I am writing to you today, not as a fellow ___________ County administrator, but as a doctoral

student. I am in the thesis writing phase of my proposal at Northeastern University. The purpose

of this letter is to request consent to conduct a case study of teacher hiring practices.

This researcher will explore the challenges involved in identifying effective teachers during the

hiring process. The researcher will study the research question, “How does bounded rationality

affect administrators’ decision-making processes in the process of hiring new teachers among

comparable school districts in the mid-Atlantic region?” using a qualitative design. In this study,

the researcher will seek to understand administrators’ beliefs and actions related to hiring and

examine what they choose to do as well as what they choose not to do.

Data sets are to include: a survey of administrators hiring practices and participant interest,

interviews with two administrators per school district across a minimum of four school districts

and a focus group consisting of members of the eight administrators. Prior to beginning data

collection, Northeastern University’s IRB will have approved all letters, surveys, consent forms,

and survey and interview questions.

If you have any questions regarding this study, please contact me directly at (609) 412-0237 or

via email at Hinman.M@husky.neu.edu, or my advisor, Dr. Margaret Dougherty via email at

m.dougherty@neu.edu.

Sincerely,

Michael Hinman

148

Appendix B

Participant Recruitment Letter (email)

Dear Colleagues,

I am writing to you today, not as a fellow ___________ County administrator, but as a doctoral student. I

am in the thesis writing phase of my proposal at Northeastern University. The purpose of this letter is to

request consent to conduct a case study of teacher hiring practices.

As you know, a great deal of our professional time is focused on the evaluation and development of

teachers. I believe an equal amount of research should examine the manner in which teachers enter the

profession in the first place. Therefore, I plan to explore the challenges involved in identifying effective

teachers during the hiring process. I will study this question using a qualitative design, specifically, a case

study approach. In this study, I will seek to understand administrators’ beliefs and actions related to

hiring.

I am asking you complete the brief survey below as it serves two important functions. First, it allows me

to collect data as to the number of teaching positions principals hire annually as well as the processes used

to do so. Second, the survey will allow me to recruit interested principals involved in the hiring of

teachers for a follow up interview. The ninth question of the survey asks if you will be willing to

participate in a one hour follow up interview to be scheduled at your convenience. Completing the brief

online survey does not require participation as an interviewee. From the pool of interested participants, I

will select eight principals across a minimum of four school districts with varied experiences in teacher

hiring. The final question of the survey asks if you would also be willing to join with the other selected

principals for a focus group discussion. Again, this would be under an hour and scheduled at the mutual

convenience of the educators who volunteer. Survey: https://www.surveymonkey.com/r/6GYGZN8

I and hope you will consider helping me in this study. Please be aware that agreeing or not agreeing to

participate in this study will in no way impact your employment or our relationship as colleagues. Also,

any participation in the study will be completely confidential; names and other personal information will

not be used; likewise the school, even the county or state, will not be named. You can refuse to answer

any question and you may discontinue your participation in this research program at any time without

penalty or costs of any nature, character, or kind.

Data sets are to include: a survey of administrators hiring practices and participant interest, interviews

with one or two administrators per school district across a minimum of four school districts, a focus group

discussion with a subset of the selected administrators. Prior to beginning data collection, Northeastern

University’s IRB will have approved all letters, surveys, consent forms, and survey and interview

questions.

If you have any questions regarding this study, please contact me directly at (609) 412-0237 or via email

at Hinman.M@husky.neu.edu, or my advisor, Dr. Margaret Dougherty via email at

m.dougherty@neu.edu.

Sincerely,

Michael Hinman

149

Appendix C

Signed Informed Consent Document of School Staff Participants

Northeastern University, College of Professional Studies

Investigator Name: Michael Hinman (Margaret Dougherty, Academic Advisor)

Title of Project: Hiring Fast and Slow: A Case Study of Teacher Hiring Practices Viewed

Through the Lens of Bounded Rationality

Why am I being asked to take part in this research study?

You have been asked to participate in this study because you have been actively involved in the

hiring of teachers.

Why is this research study being done?

This researcher will explore the challenges involved in identifying effective teachers during the

hiring process.

What will I be asked to do?

The researcher will be looking for you to participate in the following ways:

1. Complete a brief survey

2. Engage in an individual interview that will be audio taped and transcribed

3. Read over transcribed audiotapes to ensure accuracy

4. Join a follow up focus group discussion

Where will this take place and how much time will it take?

1. The survey will take approximately 10 minutes to complete

2. The interview will take approximately one hour, and will take place at a place and time

convenient for you as the interviewee.

3. The focus group discussion will take approximately one hour, and will take place at a

place and time coordinated to be as convenient as possible for the members.

Will there be any risk or discomfort to me?

There are no significant risks involved in being a participant in this study.

Will I benefit by being in this research?

There are no direct benefits to you for participating in this study. It is hoped that the results of the

study may illuminate teacher hiring practices thus improving the percentage of the best teachers

hired for the limited positions available.

Who will see the information about me?

Your part in the study will be completely confidential. Pseudonyms will be used for all study

participants. Only the researcher will be aware of the participants' identities. No reports or

publications will use information that can identify you, the school or any individual in any way.

All audiotapes will be destroyed following transcription and analysis.

150

If I do not want to take part in the study, what choices do I have?

You are not required to take part in this study. If you do not want to participate, you do not sign

this form.

Can I stop my participation in this study?

Participation in this study is voluntary, and your participation or non-participation will not in any

way affect other relationships (e.g., employer, school, etc.). You can refuse to answer any

question and you may discontinue your participation in this research program at any time without

penalty or costs of any nature, character, or kind.

Who can I contact if I have questions or problems?

Michael Hinman, ABD, Northeastern University (609) 412-0237 or via email at

Hinman.M@husky.neu.edu

Dr. Margaret Dougherty, Northeastern University, College of Professional Studies via email at

m.dougherty@neu.edu.

Who can I contact about my rights as a participant?

If you have any questions about your rights as a participant, you may contact Nan C. Regina,

Director, Human Subject Research Protection, 960 Renaissance Park, Northeastern University

Boston, MA 02115 tel. 617-373-4588, email: irb@neu.edu. You may call anonymously if you

wish.

Will I be paid for my participation?

You will not be paid for your participation in this study.

Will it cost me anything to participate?

There is no cost to participate in this study.

I have read, understood, and had the opportunity to ask questions regarding this consent

form. I agree to participate in this study on a voluntary basis.

____________________________________ ____________________

Research Participant (Signature) Date

____________________________________

Research Participant (Printed Name)

____________________________________ ____________________

Signature of the researcher, Michael Hinman Date

151

Appendix D

Survey

(This survey will be used to identify participants.

All names and locations selected for this study will be confidential.)

Name:

District:

School:

1. How many years have you been a principal?

a. 1-5

b. 6-9

c. 10+

2. Approximately, how many teachers do you hire in a year?

a. 1-5

b. 6-9

c. 10+

3. How important are the following sources of information when reviewing teacher

applications? 1= somewhat, 2 = important, 3 = very important, N/A

a. Certifications

b. In-district application forms

c. Resumes

d. Transcripts

e. Recommendations, formal or informal

f. Prior knowledge of candidate

4. To what degree are teachers hired through screening committees?

a. Never

b. Sometimes

c. Usually

d. Almost always

e. Always

5. If so, who is included in these committees?

a. Administrators

b. Teachers

c. Parents/community members

d. Other _________________________

6. To what degree are interview protocols or processes (the list of questions, time frames and

manner of interviews, etc. ) used for all teacher candidates?

a. Never

b. Sometimes

c. Usually

d. Almost always

152

e. Always

7. If so, to what degree are those protocols and processes used by all administrators in your

district?

a. Never

b. Sometimes

c. Usually

d. Almost always

e. Always

8. Would you be willing to be a participant of this study engaging in an hour long interview

on this topic as described in the accompanying Request for Participation? You may rescind

your agreement to participate at any time.

a. Yes (Please provide your email here__________)

b. No

c. Contact me first as I have questions regarding participation

9. If so, would you also be willing to be a member of a focus group of up to 8 principals who

were also members of the study to discuss the preliminary themes identified in the study.

This group would meet for 1 hour in a location central to the county participants.

a. Yes (Please provide your email here__________)

b. No

c. Contact me first as I have questions regarding participation

153

Appendix E

Individual Semi-Structured Interview Protocol

Participant _______________________

Location_________________________

Date ____________________________

Prior to an interview session, the researcher has confirmed survey data and original informed

consent form. _______________ (Researcher initials) ______________ (Participant initials)

Interview questions and probes

1. (Anchoring/Halo Effect)

a. What is most important to you in a teacher’s submitted application?

b. How does that help you find the most effective teacher?

c. Have you ever had a hiring committee member who held onto a first impression (good or

bad)? How did you manage that?

2. (Missing Data Bias)

a. Do you find you have all of the information you need to choose the best candidate?

b. How do you manage the task of considering all of the information?

3. (Affect Bias)

a. What do you look for during teacher interviews?

b. How does that help you find the most effective teacher?

4. (Confirmation Bias)

a. Have you ever had a committee member select data as evidence of the preference he/she

had before the interviews began?

b. If so, how did you manage that?

5. (Availability Bias)

a. Have you ever had a committee member express a dislike for a candidate because the

candidate reminded the committee member of a previous poor hire (graduated from the

same university, lived in the same town, etc.)?

6. (Sunk Cost Effect / Loss Aversion)

a. What experiences have you had in which after a long interview process, you did not have

a candidate that you were confident in hiring?

b. Did you restart the hiring process? Why or Why not?

7. (Decision-making Tools)

a. Does your school/district utilize protocols or processes for teacher hiring?

154

b. Do your processes improve your teacher hiring? If so, how?

c. Do you wish it were different? If so, how?

d. Expert decision makers such as surgeons and pilots have been shown to improve their

decisions through the use of checklists. Do you believe that might also be true for the

expert decision-making of principals in the hiring process? Why or Why not?

8. (Decision-making Tools)

a. Do you always use committees in the hiring process?

b. If so, how do they operate?

c. How are candidates ranked and selected?

d. Who makes the final decision, the principal, or the committee?

e. Is someone charged with ensuring protocols are followed in the process?

9. (Decision-making Tools)

a. Consider your teacher hiring process, what is working/what do you wish to change?

155

Appendix F

Focus Group Semi-Structured Interview Protocol

Participant _______________________

Location_________________________

Date ____________________________

Prior to this focus group session, the researcher has confirmed survey data and initial informed

consent form.

______________ (Researcher initials)

______________ (Participant initials)

1. In our first interview, we discussed the process of selecting candidates to interview from

the pool of applicants. How do you feel about success of that process?

2. Can you recall an experience when a candidate seemed very promising on paper, but not

in the interview? What did he/she do/not do in the interview?

3. When you come to the conclusion of the interview process, do you find your committee

members rank the candidates in the same order? If not, what do you do?

4. Are you interested in changing your interview process? If so, in what ways?