Multi Source Feedback

8/4/2019 Multi Source Feedback

1/23

Introduction

Multisource feedback (MSF), alsoknown as 360-degree feedback, isa process in which a leader re-ceives anonymous feedback fromsubordinates, peers, bosses, and

customers. MSF is pervasive throughout U.S.organizations and is spreading to other partsof the world. Estimates indicate that asmany as 29% of U.S. organizations (Church,2000) are using this process. Many organiza-tions embrace the 360-degree feedbackprocess as part of their overall leadership de-

velopment programs. However, recent re-search suggests that results may be modest.Smither, London, and Reilly (2005) ana-lyzed the results of 24 longitudinal studies

on MSF and concluded, Practitionersshould not expect large, widespread per-formance improvements after employees re-ceive multi-source feedback (p. 33). Whiletheir results found modest, yet positive im-provements in employee behaviors and atti-tudes, practitioners that seek ways to in-crease the effectiveness of their firms MSFinterventions can look to the existing andcurrent research on MSF processes.

The purpose of this article is to outlinerecent studies on MSF in order to informpractice and increase the likelihood thatmore leaders and organizations will benefit

from this developmental process. Our inten-tions are threefold. First, we highlight re-search knowledge in the area of MSF. Sec-ond, we describe all we have learned from

MULTISOURCE FEEDBACK:

LESSONS LEARNED AND

IMPLICATIONS FOR PRACTICE

L E A N N E E . A T W AT E R , J O A N F . B R E T T , A N D A T I R AC H E R I S E C H A R L E S

Organizations around the world are using multisource, or 360-degree, feed-

back. Although many HR practitioners embrace it as an important mecha-

nism for leadership development, organizations must attend to and addressseveral issues in order to maximize the utility of multisource feedback (MSF).

We discuss current research findings and highlight issues for managers to

consider both before starting a multisource feedback process and after the

feedback is given, plus we review potential outcomes of the process. We also

describe lessons learned from an intensive three-year investigation of an

MSF implementation in two organizations. 2007 Wiley Periodicals, Inc.

Correspondence to: Leanne E. Atwater, Management Department Chair, School of Global Management andLeadership, Arizona State University, 4701 W. Thunderbird Road, Phoenix, AZ 85069-7100, (602) 543-6203,[email protected]

Human Resource Management, Summer 2007, Vol. 46, No. 2, Pp. 285307

2007 Wiley Periodicals, Inc.

Published online in Wiley InterScience (www.interscience.wiley.com).

DOI: 10.1002/hrm.20161


2/23

286 H UMANRESOURCE MANAGEMENT, Summer 2007

an intensive three-year investigation of anMSF implementation in two organizations.Third, we discuss the implications of MSF re-search for leaders and human resource pro-fessionals in the field. Where appropriate, weindicate where MSF research has relevance toperformance appraisal (PA), since bothprocesses involve feedback.

Figure 1 presents a framework for pre-senting recent research on MSF. The frame-work includes factors that HR practitionersshould consider prior to implementation,factors to consider about the actual MSFprocess, factors to consider after leaders re-ceive feedback, and outcomes that organiza-tions can anticipate. For each of these topicareas, a table summarizes the MSF findingsand applications for practice. The issues

noted in italics in Figure 1 and the tables in-dicate results from our three-year study.

We also highlight those areas in whichfindings from the literature on performanceappraisal (PA) and MSF are similar. Whilethis article is not intended to summarize thevast literature on PA, there are areas in whichpractitioners interested in MSF can learnfrom the PA literature. However, for the mostpart, MSF has been designed and imple-mented as a developmental rather than eval-uative process. Unlike MSF, PAs are oftenlinked to administrative purposes and haveconsequences for merit increases and promo-tion and layoff decisions. In addition, PA tra-ditionally relies upon a supervisor evalua-tion, whereas MSF relies on multiple, oftenanonymous sources. Because of these differ-

Human Resource ManagementDOI: 10.1002/hrm

FIGURE 1. Issues to Consider in a Multisource Feedback Process


3/23

Multisource Feedback: Lessons Learned and Implications for Practice 287

ences, many of the findings pertinent to MSFare not particularly relevant to PA (e.g.,anonymity, confidentiality, time involved inthe process, method of feedback distribu-tion). In addition, one of the purposes of thisarticle is to highlight findings from a three-year study of MSF that has limited relevanceto PA. Throughout the article, however, wewill indicate findings that should have rele-vance for both MSF and PA.

Factors to Consider Before Feedback

Organizational Context

MSF can be initiated by an individual leaderas a means of self-development. More com-monly, an organization or unit embarks on

an MSF process because of pressing needs forits leaders to engage in different behaviors torespond to organizational challenges. Often,an MSF process becomes one of several ap-proaches to resolving organizational issues.Under these circumstances, developmentalfeedback may reinforce positive leadershipchange or may garner resistance. How wellthe feedback process works depends, in part,on the organizational context surroundingits implementation. For example, organiza-tions considering serious restructuring or

downsizing are not in a good position tobegin implementing MSF, because they willhave difficulty garnering the trust neededfrom participants in the midst of serious or-ganizational change. Additionally, organiza-tional cynicism (e.g., employees believe ef-forts to change are not worth it, or positivechange is not possible) can interfere with thesuccess of the MSF intervention. Atwater,Waldman, Atwater, and Cartier (2000) foundthat MSF participants who were cynical wereless likely to improve following MSF (r =

.25,p < .10).In a study of MSF practices in more than

100 organizations (Brutus & Derayeh, 2002),19% of the companies indicated that the ab-sence of integration of the MSF process withother human resources systems (i.e., devel-opment, performance appraisal, trainingsupport) created resistance to the MSFprocess. HR practitioners must do more than

obtain organizational commitment to theMSF process; they also need to assist all MSFparticipants in understanding how the feed-back intervention fits with the organiza-tions initiatives and goals and how theprocess aligns with other HR activities. Thisfinding holds for performance appraisal aswell. In a review of the PA literature, Bretz Jr.,Milkovich, and Read (1992) concluded thatwe need a better understanding of how orga-nizational context affects performance ap-praisal issues. Employees need to see how theresults of their PA fit with rewards and devel-opment opportunities offered inthe organization (Heneman &Gresham, 1998).

Research also suggests the im-portance of clear and careful im-

plementation strategies (e.g.,meetings with leaders and theirraters to discuss the process, itspurpose, and implementation)(Waldman & Atwater, 1998). Inthese meetings, leaders and raterscan ask questions and shouldhave their anxieties relievedabout anonymity, confidentiality,and the developmental purposeof the process.

London (2001) addresses the

lively debate surrounding the decision to useMSF for development or administrative deci-sions. He concludes that feedback can beused for both developmental and adminis-trative purposes, but this takes time (p.383). MSF works best when used (at least ini-tially, e.g., for 24 rounds) for developmentalpurposes rather than evaluative purposes.That is, organizations should not requirefeedback recipients to share their feedbackresults with bosses, nor should they use theirratings for decision purposes such as raises or

other outcomes. Once managers and em-ployees are able to trust the process, it can becarefully (e.g., used in addition to other indi-cators) incorporated into personnel decisionmaking. Managers tend to want to rush intousing MSF for evaluation, as they often be-lieve that without accountability managerswill be less likely to change. Patience is en-couraged.

organizations

considering serious

restructuring or

downsizing are not

in a good position to

begin implementing

MSF



4/23


Perceptions of the Process

Consistent research findings indicate that ac-ceptance and trust in appraisal and feedbackprocesses are critical for those involved in afeedback process (cf., Longnecker, Sims, &Gioia, 1987). Several factors influence ac-

ceptance and resistance to the process.Rater anonymity among peer and subor-

dinate raters has been shown to be importantto promote honest responding (Brutus & Der-ayeh, 2002). Subordinate raters are often warythat the leader will somehow trace their re-

sponses back to them. If their rat-ings are low or unfavorable, ratersmay fear retaliation. Thus, if ratersdo not trust that ratings will beanonymous, they may choose notto rate the leader, or may inflate

their ratings to avoid confronta-tion or reprisals. Antonioni (1994)found that employees who believetheir ratings are anonymous arelikely to give more honest feed-back than are employees whothink their responses will be asso-ciated with them. To increaseraters perceptions of anonymity,HR practitioners may want to en-sure that only leaders with feed-

back from three or more subordi-nates participate in the MSFprocess. Facilitators working withfeedback recipients should empha-size to them that it is problematicif they try to attribute ratings orcomments to one of their subordi-

nates, as they are very often wrong.Recipients must believe in the confiden-

tiality of the feedback and that they are theonly ones who will see it if they are to trustthe process. If they believe that the feedback

will be shared without their knowledge, it willcreate distrust and problems for accurate rat-ing and effective responses to the feedback.

Individual Differences andResponses to Feedback

Characteristics of the leader may influencehow he/she responds to and uses the feed-

back. Funderburg and Levy (1997) reportedthat individuals with high self-esteem re-ported more favorable attitudes toward theMSF system than those with low self-esteem(r= .27,p


5/23


months later, learning goal orientation didnot moderate change in behavior betweentheir initial and subsequent ratings.

Self-efficacy is an individuals belief inhis or her capability to engage in actions re-quired to achieve successful performancelevels (Bandura, 1997). Individuals with highself-efficacy focus attention on analyzing so-lutions to problems, whereas those low inself-efficacy have evaluative concerns anddwell on their personal failures. Individualswith high self-efficacy are more likely tocope and engage in activities in response tothe challenges and adjustments at work(Raghuram, Wiesenfeld, & Garud, 2003).Maurer, Mitchell, and Barbeite (2002) foundthat those with higher self-efficacy heldmore positive attitudes toward MSF (r= .32,

p < .001) and participated more in develop-mental activities off the job (r= .19,p < .05)than those with lower self-efficacy. Heslinand Latham (2004) found that after control-ling for initial feedback those with higherself-efficacy showed more improvement aftersix months.

The research on individual differencesand responses to MSF and PA feedback sug-gests that individuals will differ in their reac-tions and responses to developmental feed-back. Individuals who conduct feedback

sessions and deliver feedback should be sen-sitive to individual differences. Some indi-viduals are proactive in their response tofeedback, while others will need assistance inusing the feedback for development. Facilita-tors will need to spend more time with someleaders to assist them in overcoming theirinitial reactions to the feedback. They shoulddiscuss strategies for overcoming any initialnegative reactions and motivate leaders tofocus on the developmental aspects of thefeedback and the actions they could take in

response to the feedback.

Actual Process

Time and effort associated with MSF maydeter continued use, particularly due to theoverloading of the bosses of the leader par-ticipants. Bosses may have 10 or more indi-viduals reporting to them. A boss that rates

each employee would incur a substantialburden of time and effort. One practical so-lution is to have the boss rate only half ofthe supervisors each year. This tactic worksquite well when the MSF process has been inplace for some time (Brutus & Derayeh,2002).

Some researchers suggest thatin addition to internal stakehold-ers, external stakeholders shouldbe a part of the MSF process. Testa(2002) suggests that along withemployees, an MSF leadership as-sessment should include cus-tomers, suppliers, and businesspartners. Testa (2002) contendsthat MSF will identify misalign-ment between internal and exter-

nal stakeholders, which in turncan help develop relationshipsbetween them. However, HRpractitioners should evaluatewhether the costs of gathering ex-ternal data justify the benefits.

Many companies seeking tominimize the time and costs asso-ciated with paper feedback surveyshave turned to electronic means ofdata collection. Companies, suchas Otis Elevator Co., that use the

Internet as the primary responsemode believe that it adds securityand confidentiality to the MSFprocess, while offering a morespeedy and convenient method(Huet-Cox, Nielsen, & Sundstrom,1999). Smither, Walker, and Yap (2004) foundno differences in ratees feedback scores as afunction of how the data were collected (e.g.,electronically versus paper and pencil). Al-though electronic data collection has merits,we caution against using this as the sole

means of feedback distribution.

Factors to Consider About Feedback

Characteristics of Feedback andRecipient Reactions

The MSF process is a very different experi-ence for leaders who receive positive feed-

Individuals who

conduct feedback

sessions and deliver

feedback should be

sensitive to

individual

differences. Some

individuals are

proactive in their

response to

feedback, while

others will need

assistance in using

the feedback for

development.



6/23


back compared to those who receive nega-tive feedback. Feedback recipients view posi-tive ratings as more accurate and useful thannegative ratings (Brett & Atwater, 2001;Facteau, Facteau, Shoel, Russell, & Poteet,1998). Reactions to positive and negativeperformance appraisal feedback also varywidely (Taylor, Fisher, & Ilgen, 1984). In a re-view of the performance evaluation and pay-for-performance literature, Rynes, Gerhart,

and Parks (2005) state thatstronger performance gains mayresult after negative feedback, butthis finding does not hold whenan individual focuses on the selfand not the task or task learning.

Negative reactions to less-than-positive feedback may reflect

transitory mood states or mayhave serious ramifications for howfeedback recipients will use theirfeedback. Smither, London, andRichmond (2005) found that lead-ers who received unfavorable feed-back initially had negative reac-tions, but six months later theyhad set more improvement goalsthan other leaders. They suggestednegative feedback may takeawhile to sink in or recipients may

need some time to reflect and ab-sorb the feedback (p. 203) afterthe initial emotions have sub-sided. Brett and Atwater (2001)found that individuals who re-

ceived negative feedback from bosses andpeers were discouraged and angry (r = 38,.25 respectively,p < .01). These same individ-uals also perceived the feedback as less accu-rate or useful. However, the impact of initialreactions dissipated after several weeks, andfeedback sign was not related to perceived

feedback usefulness. These research findingssuggest the special challenges HR practition-ers face in getting recipients to respond con-structively to negative feedback (whether PAor MSF) and highlight the importance ofplanned interventions to assist feedback re-cipients in working with their feedback.

Managers reactions to feedback may re-late not only to the positive and negative na-

ture of the feedback, but also to the extent towhich the feedback is discrepant from themanagers self-view. Several theories, such ascontrol theory (Carver & Scheier, 1990) andfeedback intervention theory (Kluger &DeNisi, 1996), suggest that when individualsdetect a discrepancy between behavior and apersonal standard, they are motivated to re-duce the discrepancy. Taylor et al. (1984)proposed that when self-ratings and otherratings are discrepant, individuals may makechanges in order to align their evaluationsfrom others with their self-view, or they mayrationalize or discount the ratings from oth-ers. Brett and Atwater (2001) found thatoverratings on a 360-degree instrument (self-ratings were higher than those from others)were related to negative reactions (r= .26, p

< .001) as well as to perceptions that thefeedback was not accurate (r= .16, p < .05).Individuals who gave themselves lower rat-ings than those received from others re-ported fewer negative reactions. With respectto PA, Klimoski and Jones (1989) found thatwhen self-rating is substantially higher thansupervisor ratings, it may strengthen the ra-tees confidence in his or her own ratingsand decrease acceptance of the supervisorsrating.

Factors to Consider After Feedback

Method of Feedback Distribution

Two themes emerged from the limited litera-ture on the methods of MSF distributionease of process and inducement of trust. Op-erations and management science scholarshave investigated online feedback distribu-tion in the context of online transactions.They found that online feedback mecha-nisms promote anonymity, which in turn in-

creases the amount of trust among the users(Bolton, Katok, & Ockenfels, 2004; Dellaro-cas, 2003). Online MSF delivery may increaseemployees perceptions of trust in the au-thenticity of their feedback.

Kamen (2003) discusses several benefitsto online MSF distribution. In a case studyand in interviews with MSF users and con-sultants, he suggests that online distribution

Smither, London,

and Richmond (2005)

found that leaders

who received

unfavorable

feedback initially

had negative

reactions, but six

months later they

had set more

improvement goals

than other leaders.



7/23


facilitates the ability to integrate with HRsoftware and technology, to invite and facil-itate participation, to submit feedback dataonline, and to have speedy data analysis anddistribution. Kamen further suggests that off-site processing and the integrity and re-spectability of the in-house contact person(receiver and distributor) increases the confi-dence and trust of those receiving the feed-back. This may be an effective distributionmethod; however, we do not recommendonline feedback delivery without facilitationand support in processing the feedback.

Organizational Support

Many organizations spend considerable timeand money in the data collection phase of an

MSF process, involving employees through-out all levels of the organization, yet pay lit-tle attention to encouraging or requiringfeedback recipients to participate in any de-velopmental activities or discussions usingthe feedback they received. The same is oftentrue of performance appraisal feedback (Levy& Williams, 2004; London, 2003). Healy,Walsh, and Rose (2003) conducted a surveyof 53 companies using MSF and found thatthe majority did not require follow-up activ-ities, and as few as 20% required feedback re-

cipients to discuss their results with a super-visor. However, according to their survey,many organizations provide voluntary op-portunities for feedback recipients to receivecoaching (55%) or attend developmentalworkshops (70%). In a survey on MSF prac-tices, Brutus and Derayeh (2002) found thatthe majority of MSF programs (70%) hadsome tie to development activities. Whilemany companies rely on voluntary partici-pation in follow-up and developmental ac-tivities, our experience is that they are rarely

used. The following research suggests thatfollow-up activities after feedback are critical.

Feedback recipients who perceived sup-port from coworkers and supervisors for de-velopment-related activities reported morepositive attitudes toward the feedback systemand were involved in more on- and off-the-job development activities after the receipt ofMSF (Maurer et al., 2002). Mabey (2001) stud-

ied participants in an MSF program comparedto nonparticipants in terms of their percep-tions of the training and development offeredby the organization. Participants in the MSFprocess believed that they received more reg-ular feedback on performance, received morerecognition for developing skills, and hadmore opportunities for training than nonpar-ticipants. Participants also were more satisfiedwith their training and their employer thanwere nonparticipants. This suggests that theMSF process can have benefits that extend be-yond individual awareness and development.

Reinforcing MSF with trainingor coaching has been found toenhance the benefits of the feed-back process. Seifert, Yukl, andMcDonald (2003) examined the

effects of MSF and a trainingworkshop on managerial influ-ence tactics. They comparedmanagers in three conditions (noMSF and no training, MSF and notraining, and MSF and training)and found that managers who re-ceived MSF and training on influence tacticsincreased their use of influence tactics withdirect reports. Managers who worked withan executive coach to understand their feed-back set more specific rather than vague

goals, solicited more ideas for improvementfrom others, and improved more in terms ofsubsequent subordinate and supervisor rat-ings (Smither, London, Flautt, Vargas, &Kucine, 2003). Luthans and Peterson (2003)reinforced this finding and found that anMSF coaching session that analyzed self-other rating discrepancies in detail encour-aged managers to analyze discrepancies andset goals based on what they learned aboutthemselves. Those who received coachingsaw significant improvements in their own

job attitudes (i.e., satisfaction, commitment,and reduced intent to leave), as well as in theattitudes of their employees (Luthans & Pe-terson, 2003).

Managers who take action, whether thisinvolves participation in a training programor developmental activities (such as receivingcoaching or reviewing progress), were morelikely to improve than those who did not

follow-up

activities after

feedback are

critical.



8/23


(Hazucha, Hezlett, & Schneider, 1993). In astudy of over 100 companies (Brutus & Der-ayeh, 2002), every organization that failed tomeet the objectives of the MSF process alsofailed to facilitate the feedback process. Thatis, recipients received reports in the mailwithout individual or group discussion with atrained facilitator. Those organizations thatwere successful in meeting their objectivesfor the program (63%) facilitated the feed-back process. Facilitation sessions are criticalto help the individual identify goals andstrategies for needed behavior change indi-cated by the feedback.

Individual Attitudes andBehaviors

Perceived Need for Change

In order for an individual to makethe recommended changes as in-dicated by their feedback results,they must perceive that there isan actual need for change.Smither, London, and Reilly(2005) suggested that improve-ment is more likely to occurwhen organizational members

perceive a need for behavioral change. They

proposed that the organization can play arole in the way that organizational membersperceive a need for change, by providingfeedback in a nonthreatening manner andby rewarding participation in activities thatpromote learning and self-development.

Other studies have also addressed the im-portance of ones perceived need for change.London and Smither (2002) argue that dis-crepancies between self-ratings and otherfeedback ratings catalyze the employees per-ceived need for change. Several studies indi-

cate that leaders who receive feedback that islower than their self-ratings may improvemore than others (Atwater, Roush, & Fis-chthal, 1995; Johnson & Ferstl, 1999).

Goal Setting

Goals have an effect on employee behaviors,efforts, and levels of task persistence (Locke

& Latham, 1990). While many studies haveaddressed the value of goal setting in thecontexts of employee performance appraisal(Roberts, 2002) and group-based feedback(Mesch, Farh, & Podsakoff, 1994), there islimited research on MSF and goal setting.Mesch et al. (1994) found that leaders whoreceived negative feedback set higher goals.Similarly, Smither et al. (2003) found thosewho received negative feedback set more im-provement goals than individuals who re-ceived positive feedback.

Meetings with Raters

Many facilitators provide leaders with guid-ance and assistance in how to discuss theirfeedback ratings and follow up with the in-

dividuals who rated them. Walker andSmither (1999) conducted a five-year studyof an MSF program and found that leaderswho met with direct reports to share theirfeedback improved more than those leaderswho did not. In addition, leaders improvedmore in the years when they met with di-rect reports to discuss the prior years feed-back than in years when they did not doso. Goldsmith and Underhill (2001) re-ported that when raters were asked abouttheir leaders improvement 36 months

after feedback, 84% of leaders who fol-lowed up with raters reportedly improved,whereas only 67% of leaders who did notfollow up improved. Follow-up consisted ofsharing with followers what the leaderlearned from their leadership feedback.Further, managers that shared their feed-back with raters and asked for input weremore likely to improve over time (Smither,London, Reilly, Flautt, Vargas, & Kucine,2004).

Outcomes of the Feedback Process

Organizations embark on MSF processes as ameans to increase self-awareness of criticalleadership performance dimensions in a de-velopmental context. Recent research sug-gests that the MSF is related to a variety ofother types of evaluations. These findingshighlight the validity of MSF.

leaders who met

with direct reports

to share their

feedback improved

more than those

leaders who did not.



9/23


Change in Ratings of Leaders

Generally, studies suggest that with multipleMSF administrations, leaders will improveperformance (cf. Luthans & Peterson, 2003;Smither, London, & Reilly, 2005). However,not all leaders improve. Atwater et al. (2000)found that only half of leaders receivingfeedback improved significantly after oneyear and one feedback session. Results varysome from study to study, but the generalconclusion is that, over time, MSF results inimproved leader ratings from other sourcesand more accurate self-ratings (in greateragreement with others). The results fromstudies of PA and improvement are less opti-mistic. Kluger and DeNisis (1996) reviewfound improvement in only one-third of the

cases, and in another one-third, performanceactually declined following feedback.

Assessment Center/PerformanceAppraisal Ratings

MSF ratings by peers, supervisors, and subor-dinates predicted assessment center ratingsof the managers competence that were gen-erated by trained assessors not involved inthe MSF process (r = .29, p < .05; Atkins &Wood, 2002). Self-ratings collected as part of

the MSF process were nonlinearly related toassessment center ratings. That is, managersgiving themselves the highest ratings scoredthe lowest on the assessment center ratingsof competence. Those who rated themselvesin the midrange were more likely to be highperformers than those who rated themselvesat the top or bottom of the scale.

In a study with more than 2,000 insur-ance employees, MSF boss and peer ratingswere compared to three- and four-year-oldperformance appraisal ratings (Beehr, Ivanit-

skaya, Hansen, Erofeev, & Gudanowski,2001). As expected, MSF ratings from bossesand peers were moderately correlated withthe performance appraisal ratings. Self-rat-ings were not correlated with performanceappraisals, even though the employees hadknowledge of their past performance ap-praisals. However, Bailey and Fletcher (2002)reported the value of MSF in raising leaders

awareness regarding critical performance ap-praisal criteria. They found that participationin an MSF process strengthened the relation-ship between leaders self-ratings and theirformal performance appraisal ratings. AtTime 1, leaders self-ratings were not relatedto performance appraisal ratings; however, atTime 2 a significant positive relationship wasobserved for self-ratings and formal appraisalratings. Thus, MSF helped managers becomemore aware of the behaviors that the orga-nization valued and rewarded (Bailey &Fletcher, 2002).

The implication of these stud-ies is that leaders are not good as-sessors of their own leadership be-haviors and others are betterjudges of the leaders managerial

competence. These results suggestthat MSF may be a better ap-proach to identifying a leadersdevelopment needs than relyingon a leaders self-assessments ofhis or her competencies. Addi-tionally, self-ratings may becomemore in line with others ratingswhen the leader receives MSF.

Results from a Three-YearIntensive Field Study in

Two Organizations

We have learned a great dealabout the MSF process and itsoutcomes as part of a three-yearinvestigation. We were interestedin factors that influenced reac-tions to the feedback as well asoutcomes for the individual lead-ers and the organization. We em-barked on an intensive study of an MSFprocess in two separate organizations that

had no prior experience with MSF. One wasa retail organization headquartered in thesouthwest and a second was an elementaryschool district, also in the southwest. Onehundred forty-five leaders participated in theprocess at two periods approximately oneyear apart. Leaders were rated by their peers,subordinates, and managers and also pro-vided self-ratings on 47 leadership items that

the general

conclusion is that,

over time, MSF

results in improved

leader ratings from

other sources and

more accurate self-

ratings (in greater

agreement with

others). The results

from studies of PA

and improvement

are less optimistic.



10/23


measured aspects of consideration, perform-ance orientation, and employee develop-ment. In each organization, all leaders withthree or more direct reports were asked toparticipate in the process. HR personnel ineach organization prepared a master list ofleaders, their manager, peers (ranging from69 peers) and all direct reports. Top man-agers in each organization sent a letter thatexplained the 360-degree process to all fol-lowers, and leaders were briefed in meetings.Approximately 70% of the leaders were male.

Administration of Surveys

The researchers prepared the sur-vey packages for each leader andhis/her raters. Raters received the

surveys via internal mail. Partici-pants mailed the surveys directlyto the researchers in prestampedand addressed envelopes. All sub-ordinate and peer surveys wereanonymous. Return rates at Time1 were self = 100%, managers =98%, peers = 83%, and subordi-nates = 68%. Corresponding re-turn rates at Time 2 were self =92%, managers = 95%, peers =75%, and subordinates = 58%.

Feedback Delivery

Researchers delivered the feedback reports toleaders in groups ranging in size from 6 to15. All leaders were guaranteed confidential-ity of the data in their feedback reports. Thedata in their reports were explained, and ef-forts were made to help leaders react con-structively to the feedback. After a discussionof how to interpret and use the feedback, therecipients completed a survey describing

their reactions to the feedback. Recipients in-dicated the extent to which they were expe-riencing 24 emotions. Three categories werecreated from these data: positive reactions(e.g., pleased, proud, happy, encouraged,grateful), negative reactions (e.g., angry,frustrated, disappointed, unhappy), andmotivated (e.g., motivated, aware, enlight-ened, inspired).

In addition to leader behavior measures,we also collected personality and attitudedata (e.g., self-efficacy, usefulness of feed-back) from the leaders during the firstfeedback session before leaders received feed-back. Outcome data were also collected fromsubordinate raters (intent to leave, employeeengagement, and job satisfaction) as part ofthe leadership feedback surveys.

We highlight the major findings fromthis study and their implications for man-agers. Incidentally, the major findings of thestudy were consistent across both organiza-tions. The areas highlighted in italics in Fig-ure 1 and Tables IIV show the contributionsto new knowledge gained from this intensivefield study.

Will Individual Differences inEmotional Stability, Trust, Opennessto New Experiences, and Self-Efficacy Influence Reactions toFeedback?

Based on the research on individual differ-ences, we expected differences in how indi-viduals would react to MSF depending ontheir levels of emotional stability, trust,openness to new experiences, and self-effi-cacy. We found individual differences gener-

ally unrelated to reactions to feedback; how-ever, those with higher self-efficacy weremore likely to engage in follow-up activities(e.g., meet with rater groups, set goals, dis-cuss feedback with supervisors). Individualswith higher self-efficacy took more positivesteps toward change. Most likely, they feltmore confident that they could realize posi-tive changes in their leader behavior andthus were more likely to take steps in that di-rection. It would be worthwhile to increaseself-efficacy by letting recipients know that

there are resources available to help themachieve their development goals.

Will Feedback Format InfluenceReactions to Feedback?

Positive behavior change and self-awarenessoften result from feedback; however, a num-ber of studies have cautioned that reduced

MSF may be a

better approach to

identifying a leaders

development needs

than relying on a

leaders self-

assessments of

his or her

competencies.



11/23



Source Findings Our Recommendations

Organizational Context

Atwater, Waldman, Participants who are cynical about the Implementation of MSF interventions when

Atwater, and Cartier organization or who are unclear about the organization is undergoing restructuring

(2000) how MSF fits into organizational goals and downsizing is not advised.

are less likely to improve with MSF and

Waldman and may resist the process. Integrate MSF with HR processes.

Atwater (1998)

Establish a clear purpose for MSF and

Brutus and Derayeh Its important to have clear and careful adequately communicate to employees how it

(2002) implementation strategies with MSF. fits into organizational goals.

London (2001) Multisource feedback can be used for Use MSF for developmental purposes first

both developmental and administrative before implementing MSF for evaluative and

purposes, but it takes time. administrative purposes.

Perceptions of the Process

Antonioni (1994) Rater anonymity is needed for honest Consider if data-collection process ensures

responses. anonymity.

Brutus and Derayeh

(2002) Inform employees about the process to

assure anonymity and confidentiality.

Individual Differences

Funderburg and High self-esteem individuals reported Be aware of individual differences when

Levy (1997) more favorable attitudes to MSF. considering potential reactions and responses

to MSF.

Smither, London, Extroverted leaders who were open to Spend time with leaders to discuss strategies

and Richmond experience were more likely than other for overcoming the initial reactions and to

(2005) leaders to react to negative feedback as motivate leaders to focus on the developmen-

valuable and to seek more information tal aspects of the feedback and the actions

about their feedback. they could take in response to the feedback.

Funderburg and High internal locus of control leaders Facilitators delivering the feedback should be

Levy (1997) reported more favorable attitudes sensitive to individual differences.

toward the 360-degree feedback system.

Maurer and Palmer Leaders with an internal locus of

(1999) control reacted to peer and subordinatefeedback more favorably and intended

to improve their skills.

Brett and Atwater Individuals with a learning goal Consider focusing on the developmental

(2001) orientation perceived feedback as more aspects of the feedback process, particularly

useful several weeks after feedback than those with a performance goal orientation.

those with a performance goal orientation.

(continued)

T A B L E I Factors to Consider Before Beginning MSF Process


12/23


motivation and performance can resultwhen individuals receive negative feedbackor feedback that is more negative than ex-pected (cf. Kluger & DeNisi, 1996). The for-mat of the feedback is one of the factors thatmay influence how individuals react to thefeedback. Earlier work tentatively suggeststhat not only is the amount of negative feed-back relevant (more negative feedback beingreacted to more negatively), but a scoringformat, which is typically used in MSF, also

may contribute to more negative reactions(Kluger & DeNisi, 1996). This scoring formatmay be particularly problematic when it iscombined with comparative informationwhere the recipient is given data to comparehis or her ratings with the ways his or herpeers were rated on the same dimensions.Scores draw the recipients attention tothemselves and their evaluation rather thanto the behavioral statements and what thesemight suggest about needed changes.



Individual Differences (continued)

Maurer, Mitchell, Self-efficacy was related to positive or Consider interventions to enhance individual

and Barbeite (2002) negative evaluative feelings toward MSF self-efficacy to assist low self-efficacy leaders

and to off-the-job development activities. in using the feedback for developmental

purposes.

Heslin and Latham Self-efficacy moderated the relationship

(2004) between ratings and improvement in

ratings six months later. Those with high

self-efficacy showed a greater increase

in performance.

Atwater and Brett Those with higher self-efficacy more

(2005) likely to report follow-up activities.

No relationship found between

personality characteristics and reactions

to feedback.

Actual Process

Brutus and Derayeh The time required to rate several staff Consider having the manager only rate a

(2002) burdens managers. subset of the leaders each year.

Testa (2002) External raters can provide valuable Weigh the value of feedback from external

insight on the manager. raters against the time and cost.

Huet-Cox, Nielsen, Some organizations use the Internet to Electronic methods can be used to collect

and Sundstrom increase confidentiality, timeliness, and ratings for convenience and to increase

(1999) convenience of the process. confidentiality and perceived trustworthiness.

We caution using electronic means to return

Smither, Walker, The data-collection method did not feedback results without facilitation

and Yap (2004) influence ratees feedback scores. assistance.

Note: Italicsindicate findings that were part of the three-year study described in this article.

T A B L E I Factors to Consider Before Beginning MSF Process (continued)


13/23




Feedback Format

Atwater and Brett Text feedback was reacted to significantly Provide numeric feedback that is specific, easy

(in press) less favorably than numeric feedback. to read, and clarifies strengths and weaknesses.

Characteristics of Feedback and Reactions

Brett and Atwater Feedback recipients with positive Consider how managers may respond to(2001) ratings viewed the process as more feedback before sharing it so that feedback

accurate and useful than those with validity does not come into question.negative ratings and reactions.

Atwater and Brett Leaders with more positive attitudes(2005) toward the process were more motivated

and positive following feedback.

Facteau, Facteau, Feedback recipients with fewer negative Recognize that some managers may reactShoel, Russell, and emotions used the feedback more negatively to feedback. Consider strategies toPoteet (1998) constructively. assist managers in overcoming these reactions.

Smither, London, Managers receiving negative feedback had Consider strategies to assist managers inFlautt, Vargas, and negative initial reactions but set more overcoming reactions to feedback and to setKucine (2003) goals six months later than other development goals.

managers.

Brett and Atwater Overraters reported more negative(2001) reactions and viewed the feedback as

less accurate.

Atwater and Brett Leaders reactions to feedback were Understand that leaders may react when they(2005) more strongly related to direct reports receive negative ratings from their direct

ratings of leadership than to ratings reports.from peers and managers.

Immediate reactions do matter and Provide support to leaders to overcome initialthey likely influence how the leader negative reactions.responds to the feedback.

Atwater and Brett Leaders who received low ratings and Facilitation and coaching may be important to(2005) overrated themselves were more reduce negative reactions by overraters.

motivated than leaders who received lowratings and gave themselves low ratings.

These overraters were more disappointed Provide individuals with leadershipand angry than those who did not weaknesses encouragement and support tooverrate. engage in developmental activities.

Atwater and Brett Leaders who overrated relative to their The discrepancy between self and other,(2005) managers or direct report ratings were when other ratings are low, appears to be

more motivated than underraters. motivational.


T A B L E I I Factors to Consider About Feedback


14/23




Method of Distribution

Kamen (2003) Proposes that online rating will improve Consider online MSF rating.

the feedback process.

Organizational Support

Brutus and Derayeh Many MSF programs have some tie to If organizations want to recoup their

(2002) developmental activities, but do not investment in MSF, they should do more than

require follow-up or developmental suggestthat leaders hold follow-up

Healy, Walsh, and activities. conversations with their employees and their

Rose (2003) supervisors or engage in developmental activ-

ities and coaching.

Maurer, Mitchell, Managers who perceived support from Provide support for development activities

and Barbeite (2002) coworkers and supervisors for following feedback.development-related activities reported

more positive attitudes toward the

feedback system and were involved in

more on- and off-the-job development

activities.

Mabey (2001) MSF participants, compared to Be aware of the positive impact of MSF on

nonparticipants, believed that they participants and organization beyond

received more regular feedback on individual awareness and development.

performance, more recognition, had

more opportunities for training, and

were more satisfied with training and

the employer.

Seifert, Yukl, and Managers who received MSF and Using MSF along with training or coach may

McDonald (2003) training on influence tactics increased lead to more positive impact on the leader

use of influence tactics. and organization.

Smither, London, Managers who worked with a coach

Flautt, Vargas, and were more likely to set specific goals,

Kucine (2003) share feedback and solicit ideas for

improvement from a supervisor and

improve in direct reports and

supervisors ratings.

Luthans and Managers who worked with a coach

Peterson (2003) were more likely to improve in self-

awareness, employee satisfaction,

commitment and intention to quit.

Brutus and Derayeh Every company studied who failed to If organizations want to maximize

(2002) meet the objectives of MSF did not improvements, it is advised that they

facilitate the feedback process. facilitate the feedback process.

T A B L E I I I Factors to Consider After Feedback


15/23


In our study, we created two differentformats for providing MSF. Recipients wererandomly assigned to receive either text ornumeric feedback. In the numeric format,MSF ratings (averaged subordinate and peer

ratings, self-ratings, and the managers rat-ing of the recipient) were provided in nu-meric averages. The recipient also saw thenormative data (averages) for the entiregroup of leaders who had participated in theprocess. Thus, leaders could compare theirself-ratings to the ratings they received fromtheir boss, peers, or subordinates, and theycould compare the ratings they receivedfrom each rating source with the ratingsother managers received. This format is theone typically used for providing MSF toleaders. Scores were color-coded such thathigh ratings were highlighted in light green,very high ratings were highlighted in darkgreen, development needs were highlightedin red, and serious development needs werehighlighted in purple. We used averagescores and standard deviations to set cutoffscores to determine strengths and develop-ment needs.

The alternative text format includedwords rather than numbers. In place of theaverages, the recipient saw words that indi-cated whether his or her score from the ratergroup was a high strength, strength,

neither strength nor development need,development need, or serious develop-ment need. The same cutoff scores wereused to determine what color the wordswould represent. We hypothesized that feed-back presented in words rather than num-bers and without normative comparisonswould be perceived as less evaluative and lessthreatening, and thus would be reacted tomore positively or less negatively if scoreswere low. Based on earlier research (Brett &Atwater, 2001), we reasoned that if the recip-ient reacted to the feedback with fewer neg-ative emotions, he or she would use the feed-back more constructively.

The results indicated that contrary to ourhypothesis, text feedback was reacted to sig-nificantly less favorably than numeric feed-back. When leaders received text feedbackthat showed that they had a high number ofdevelopment needs as perceived by their



Individual Attitudes and Behaviors

London and Improvement is more likely when Assist recipients to recognize the need for

Smither (2002) managers perceive a need for change. behavioral change.

Smither, London,and Reilly (2005)

Mesch, Farh, and Leaders who received negative feedback Negative feedback can promote goal setting;Podsakoff (1994) set more improvement goals than those managers should help leaders achieve newly

who received positive feedback. set goals.

Walker and Smither Leaders who follow up and meet with Encourage leaders to meet with those who(1999) raters as a group after receiving feedback rated them to discuss further actions based

are more likely to realize improvements. on the ratings.Goldsmith andUnderhill (2001)


T A B L E I I I Factors to Consider After Feedback (continued)


16/23




Change in Ratings of Leaders

Walker and Though small changes may be made after Expect only modest positive

Smither (1999) multisource feedback, significant results improvement with MSF. MSF is not amay not be seen for a year or two. quick fix, but a process that, over time,

Atwater, Waldman, can have a significant impact onAtwater, and organizations.Cartier (2000)

Comparison to Other Ratings

Atkins and Wood MSF ratings predict assessment center Supports validity of MSF ratings.(2002) ratings of the managers competence.

Self-ratings were unrelated to Be aware of using assessment ofassessment center ratings. development needs or development ac-

tivities that rely only on self-ratings.

Beehr, Ivanitskaya, Self-ratings were not correlated with MSF may be a better approach toHansen, Erofeev, past performance appraisals. identifying a leaders developmentand Gudanowski needs than relying on self-assessments(2001) MSF ratings from bosses and peers of ones competencies.

were moderately correlated with theperformance appraisal ratings.

Bailey and Fletcher Self-ratings and formal appraisal ratings MSF may help managers become more(2002) become more closely in line with one aware of valued and rewarded

another after MSF. organizational behaviors.

Goldsmith and Subordinate ratings of leaders did not improve Although ratings may not significantly

Underhill (2001) significantly, but subordinates perceived some improve, subordinates may perceivepositive changes in most managers performance. positive results from the process.

Employee Attitudes

Atwater and Brett Changes in leader behaviors made a By providing the resources to facilitate(2006) difference in employee attitudes. MSF, the organization may benefit from

the positive impact on leaders, as wellIntent to leave is most highly related to as subordinates satisfaction, intent toemployee development behaviors. leave, and engagement.

Leaders who became more encouraging of staffto learn and grow and gave them recognitionand feedback had employees who were

reportedly less likely to leave the organization.

Leaders who improved on considerationhad more satisfied employees.

Leaders who improved on performance-oriented, task-related behaviors (decisionmaking, communication) had employeesmore engaged in their work.


T A B L E I V Factors to Consider After Feedback


17/23


subordinates, their negative reactions wereparticularly severe, more severe than wheneither peers or managers suggested the leaderhad numerous development needs. In addi-tion to more negative reactions, individualsreceiving text feedback also had fewer posi-tive reactions and were less motivated fol-lowing feedback.

A year later, when the recipients receiveda second round of feedback, those who hadoriginally received text feedback were pre-sented with both text and numeric feedbackand asked to comment on the two feedbackformats. Essentially, their comments indi-cated that the text format was seen as vagueand not concise. They preferred the numericfeedback because it was more specific, easy toread, and clarified strengths and weaknesses.

In a society where many aspects of daily lifeare measured in numbers (e.g., blood pres-sure, IQ), we may now expect and be morecomfortable with numbers. We recommendthat MSF providers should provide specific,numeric feedback rather than text reports.This does not mean narrative comments byraters should be avoided, but rather used inaddition to quantitative ratings.

Interestingly, however, when we assessedwhether leaders in the text or numeric for-mat groups changed more at the second ad-

ministration of feedback (that is, did the textor numeric group get higher ratings at Time2), the answer was that format per se didntmatter. However, initial reactions did matter.Those who reacted most negatively (e.g.,angry, defensive) after receiving their firstround of feedback had more developmentneeds reported at Time 2, even after control-ling for the number of needs reported atTime 1. In other words, they got worse. Sim-ilarly those who had the most positive reac-tions at Time 1 showed a decline in develop-

ment needs at Time 2; they actuallyimproved. Those who receive negative feed-back and react negatively are least likely todemonstrate positive changes in ratings ayear later, and in fact, they may get worse.Those who react positively to the feedbackand feel motivated are most likely to see pos-itive changes in ratings a year later. This find-ing suggests that HR practitioners carefully

plan how to help leaders to minimize anynegative reactions, perhaps by using a facili-tator or coach.

What Factors Influence NegativeReactions to MSF?

We were interested to see what factors otherthan the format or sign of the feedback (pos-itive or negative) would contribute to recipi-ents reactions to feedback. For example, ifleaders overrate themselves, does this affecttheir reactions to the feedback they receive?Are low ratings reacted to morenegatively when the recipientrated himself or herself high? Inwhat ways might the attitudes in-dividuals hold about the feedback

process influence their reactionsto the feedback?We learned that leaders who

received low ratings and over-rated themselves were more moti-vated than leaders who receivedlow ratings and gave themselveslow ratings. The discrepancy be-tween self-ratings and other rat-ings can be motivational. How-ever, these overraters also hadmore negative reactions (e.g.,

were more disappointed and angry) thanthose who did not overrate. This finding sug-gests that interventions (such as MSF) de-signed to create more self-awareness mayhelp in the long term as leaders gain aware-ness of how they are perceived by others,and try to reduce the discrepancies betweenself and other ratings. However, we wouldadvise HR practitioners to help overratersovercome any negative reactions that occurbecause of this new self-awareness.

Leaders completed attitude measures

during the first feedback session, prior to re-ceiving their feedback reports. Not surpris-ingly, those who held more positive atti-tudes toward using feedback also were moremotivated and had more positive emotionsfollowing feedback, regardless of the type offeedback they received. We recommend thatHR practitioners consider the importance oftaking time to introduce the MSF process

They preferred the

numeric feedback

because it was

more specific, easy

to read, and

clarified strengths

and weaknesses.



18/23


into the organization in a careful andthoughtful way so both raters and leadersfeel comfortable with how the data will becollected and used, its confidentiality, rateranonymity, and so on. If leaders enter theprocess with bad attitudes about the feed-back process, its validity, or its usefulness,they will not reap the most from theprocess. All efforts should be made to helpreduce negative reactions to feedback.Coaching, counseling, and follow-up withrecipients are advisable or the feedback may

have negative rather than posi-tive effects on those who reactnegatively.

What Are the OutcomesBeyond Changes in Leader

Behavior?The most common method usedto assess the success of an MSF in-tervention is to compare the lead-ers ratings before and afterhe/she receives feedback. Gener-ally, the literature shows modestpositive improvement in ratings(Smither, London, & Reilly,2005). In our study, when wecompared Time 1 and Time 2 rat-

ings provided by peers, subordi-nates, and managers across allleadership items, we only foundsignificant improvement inbosses ratings of the feedback re-cipients (perhaps due to the fact

that the bosses ratings were not anony-mous). There are a number of possible expla-nations for this result. First, it may be thatbecause we assessed so many items, evenpositive changes were washed out whenoverall averages were compared. Given that

this is the first time MSF had been used,changes may be in the right direction, but itmay take another year or two for the changesto be statistically significant, as per Walkerand Smither (1999).

We also were interested in how changesin leaders behavior might influence out-comes other than their behavior or ratingsof their behavior. While MSF has become

very popular, little attention has beengiven to evaluating the effects of the MSFprocess on outcomes other than subse-quent ratings. We reasoned that becausethere is a great deal of literature attesting tothe relevance of supervisory behavior toemployee attitudes, if leader behavior im-proves, we should expect employee atti-tudes to improve as well. The questionarises, if MSF can improve leadership, willthose changes translate into changes inemployee attitudes?

We assessed employee attitudes as partof the MSF survey. Therefore, in addition torating their supervisors leadership, subor-dinates rated their own satisfaction, intentto leave, and engagement at both Times 1and 2. Our primary question was if the

leaders behaviors improve, do employeeattitudes also improve? We adjusted theTime 2 attitude and leadership scores fortheir Time 1 values so we could explainchange in attitudes with change in leaderbehavior. We learned that positive changesin the leaders consideration behavior (e.g.,listens, is tactful and sensitive, acceptsfeedback) were related to positive changeson each of the three attitude measures. Thebiggest impact for changes in considerationwas seen in improved employee satisfac-

tion as compared to engagement or intentto leave. Positive changes in the leadersemployee development behavior (talks todirect reports about progress; encouragesdirect reports development) also were re-lated to positive changes on each of the at-titude measures, with the strongest effectfor intent to leave.

Positive changes in leaders performanceorientation (e.g., sets goals, follows up,makes tough decisions) were related to posi-tive changes in employee engagement (e.g.,

know what is expected, receive recognitionor praise, and have necessary materials andequipment to do the job). MSF can be a use-ful method for improving leader behaviorand ultimately influencing employee atti-tudes in a positive way. If leader behaviorsimprove, we can expect employee attitudes(job satisfaction, engagement, and intent toleave) to improve.

We learned that

positive changes in

the leaders

consideration

behavior (e.g.,

listens, is tactful

and sensitive,

accepts feedback)

were related to

positive changes on

each of the three

attitude measures.



19/23


Summary and Conclusions

In sum, our study reinforces the implicationsof prior research on MSF practices and addsnew insights. First, the positive relationshipbetween attitudes toward using the feedbackand reactions emphasizes that human re-source professionals should understand theimportance of paying attention to how theyintroduce and implement the MSF process intheir organizations. Second, reactions to neg-ative feedback were not transitory moodstates with minimal implications for leader-ship development, but rather influencedsubsequent behavior. These findings rein-force the need for organizations to considerhow they facilitate feedback distribution andhow they encourage developmental activi-

ties following feedback.Our findings show that those leaderswho improved were more likely to see subse-quent changes in employee attitudes. Thisresult indicates that with regard to organiza-tional outcomes, MSF can do more than justdevelop leaders. It can have a positive rippleeffect upon others in the organization. Un-derstanding and implementing the practicalfindings from the research summarized andpresented in this article should assist organi-zations in reaping the rewards of their in-

vestment in a MSF process.However, clearly the costs and benefits of

implementing MSF should be considered.For example, Bettenhausen and Fedor (1997)suggest a number of potential benefits ofMSF such as higher-quality feedback, inputfor employee development recommenda-tions, and uses in performance coaching.However, they also highlight some possibledownsides, including fostering defensivenessand creating situations where leaders be-come overly concerned about pleasing em-

ployees. In all cases, the goals of implement-ing MSF should be clear and should alignwith the organizations goals and personnelpractices.

Research can inform and enhance thebenefits organizations derive from their in-vestment in MSF. However, many questionsremain regarding the MSF process that couldprovide additional insight to practitioners.

Rynes et al. (2005) suggest that future MSFresearch focus on better understanding theindividual or program characteristics that aremost likely to improve the returns from thisprocess. Likewise, Smither, London, andReilly (2005) suggest that we move from ask-ing Does MSF work? to asking Underwhat conditions and for whom does MSFwork?

As indicated from our summary, therole of organizational context is critical tothe success of MSF. However, we need a bet-ter understanding of what organizationscan do to create an environmentand culture that supports feed-back, whether it is performanceappraisal or multisource feed-back (Levy & Williams, 2004;

London & Smither, 2002; Ryneset al., 2005). We need research toilluminate how the feedback cul-ture in organizations influenceshow feedback is sought, per-ceived, processed, accepted,used and reacted to (Levy &Williams, 2004, p. 895). For ex-ample: How do perceptions ofthe motivation for the MSFprocess (i.e., individual initia-tive, new boss, organizational

change, or decline in organiza-tional performance) influenceacceptance and commitment tothe process? How do perceptionsof others in the social contextinfluence the process (i.e., is the boss sup-portive or negative toward the process or isthe process companywide)?

Some individuals benefit from the MSFprocess and others do not. We need a betterunderstanding of why some individuals usefeedback and take actions and others do not.

Recent work on feedback orientation (i.e., re-sponsiveness to feedback and evaluation)may provide answers to these questions(London & Smither, 2002). In addition, re-search is needed that examines the com-bined effects of individual differences andorganizational support for development onthe reactions to and use of feedback(Smither, London, & Richmond, 2005).

reactions to

negative feedback

were not transitory

mood states with

minimal implications

for leadership

development, but

rather influenced

subsequent

behavior.



20/23


Performance appraisal and MSF researchindicates that when individuals focus atten-tion on the self and not the task, negativefeedback is debilitating. We need a better un-derstanding of how the 360-degree processcreates a self-awareness that is motivatingversus creating a self-focus that is debilitat-ing. For example: Are we overwhelmingmanagers with too much information whenthey receive self, other, and normative feed-back? What role does each of these feedbackcomponents play in improving performanceversus overwhelming those who receive neg-ative information?

It is clear from research that MSF alonemay not be enough to motivate changes in aleaders behavior. How do interventions suchas coaching, goal setting, and training com-

plement and reinforce leadership change? Docoaching, goal setting, and training enhance

MSF because it helps managers move from aself-focus to a task focus? If the ultimate goalof an MSF process is leadership development,the questions may not be limited to howcoaching and training facilitate the value ofMSF, but how the MSF process can facilitatetraining, executive coaching, and other lead-ership development interventions to be moreeffective in developing and changing leader-ship behavior. Finally, we need more researchthat demonstrates the cost-effectiveness ofMSF. What are the organizational and bot-tom-line costs and benefits to organizationsundertaking an MSF process?

While we wait for researchers to addressthese issues, we encourage practitioners toconsider how the research discussed in thisarticle can improve the MSF process in their

organizations and, ultimately, the effective-ness of leaders and organizations.


Dr. LEANNE E. ATWATER is a professor of management and interim dean at Arizona

State University in the School of Global Management and Leadership. She received a

PhD in psychology from Claremont Graduate School. Her research interests include 360-

degree and upward feedback, self-perception accuracy, and gender and discipline. She

has published her work in journals such as the Journal of Applied Psychology, Person-

nel Psychology, and the Journal of Management. She and David Waldman coauthored a

book entitled The Power of 360 Degree Feedback.

Dr. JOAN F. BRETT is an associate professor at Arizona State University in the School of

Global Management and Leadership. She earned her PhD in industrial organizational

psychology from New York University in 1992. Dr. Bretts research focuses on issues re-

lated to feedback and motivation, specifically 360-degree feedback processes and has

appeared in leading academic journals such as the Academy of Management Review,

the Academy of Management Journal, the Journal of Applied Psychology, and Group

and Organization Management. Prior to her academic career, Dr. Brett worked in the in-

surance and banking industries and held various management positions in human re-

sources.

ATIRA CHERISE CHARLES is a doctoral student at Arizona State University. She received

her BS and MBA degrees from Florida A&M University. Her research interests include is-

sues of justice and fairness, racial identity, and emotional reactions to the workplace. Her

recent work has examined the attitudinal and affective reactions of subordinates within

the context of discipline delivery. She is currently working on a project that investigates

the role of influence tactics on the mentoring relationship, while assessing the demo-

graphic differences that affect these relationships. She has published in outlets such as

the Journal of Management Developmentand the International Encyclopedia of Organi-

zational Studies.


21/23


REFERENCES

Antonioni, D. (1994). The effects of feedback accounta-

bility on upward appraisal ratings. Personnel Psy-

chology, 47, 349356.

Atkins, P., & Wood, R. E. (2002). Self-versus others

ratings as predictors of assessment center ratings:

Validation evidence for 360-degree feedback pro-

grams. Personnel Psychology, 55, 871904.

Atwater, L., & Brett, J. (in press). Feedback format:

Does it influence managers reactions to feedback.

Journal of Occupational and Organizational Psy-

chology.

Atwater, L., & Brett, J. (2005). Antecedents and conse-

quences of reactions to developmental 360 degree

feedback. Journal of Vocational Behavior, 66,

532548.

Atwater, L., & Brett, J. (2006). 360 degree feedback to

managers: Does it result in changes in employee

attitudes? Group & Organization Management. 31,

578600.

Atwater, L., Roush, P., & Fischthal, A. (1995). The influ-

ence of upward feedback on self- and follower rat-

ings of leadership. Personnel Psychology, 48,

3559.

Atwater, L., Waldman, D., Atwater, D., & Cartier, T.

(2000). An upward feedback field experiment. Su-

pervisors cynicism, follow-up and commitment to

subordinates. Personnel Psychology, 53, 275297.

Bailey, C., & Fletcher, C. (2002). The impact of multiple

source feedback on management development:

Findings from a longitudinal study. Journal of Or-

ganizational Behavior, 23, 853867.

Bandura, A. (1997). Self-efficacy: The exercise of con-

trol. New York: W. H. Freeman.

Beehr, T., Ivanitskaya, L., Hansen, C., Erofeev, D., &

Gudanowski, D. (2001). Evaluation of 360 degree

feedback ratings: Relationships with each other

and with performance and selection predictors.

Journal of Organizational Behavior, 22, 775788.

Bettenhausen, K. L., & Fedor, D. B. (1997). Peer and

upward appraisals: A comparison of their benefits

and problems. Group & Organization Manage-

ment, 22, 236263.

Bolton, G. E., Katok, E., & Ockenfels, A. (2004). How

effective are electronic reputation mechanisms? An

experimental investigation. Management Science,

50, 15871602.

Brett, J., & Atwater, L. (2001). 360-degree feedback:

Accuracy, reactions and perceptions of usefulness.

Journal of Applied Psychology, 86, 930942.

Bretz, R. D., Jr., Milkovich, G. T., & Read, W. (1992). The

current state of performance appraisal research

and practice: Concerns, directions and implica-

tions. Journal of Management, 18, 321352.

Brutus, S., & Derayeh, M. (2002). Multi-source assess-

ment programs in organizations: An insiders per-

spective. Human Resource Development Quarterly,

13, 187201.

Carver, C. S., & Scheier, M. F. (1990). Origins and func-

tions of positive and negative effect. A control-

process view. Psychological Review, 97, 1935.

Church, A. H. (2000). Do higher performing managers

actually receive better ratings? A validation of mul-

tirater assessment methodology. Consulting Psy-

chology Journal: Practice and Research, 52, 99116.

Colquitt, J., LePine, J., & Noe, R. (2000). Toward an in-

tegrative theory of training motivation: A meta-an-

alytic path analysis of 20 years of research. Journal

of Applied Psychology, 85, 678707.

Dellarocas, C. (2003). The digitization of word of

mouth: Promise and challenges of online feedback

mechanisms. Management Science, 49, 14071424.

Dweck, C. S. (1986). Motivational processes affecting

learning. American Psychologist, 41, 10401048.

Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive

approach to motivation and personality, Psycho-

logical Review, 95, 256273.

Facteau, C. L., Facteau, J. D., Shoel, L. C., Russell. J.

A., & Poteet, M. (1998). Reactions of leaders to 360-

degree feedback from subordinates and peers.

Leadership Quarterly, 9, 427448.

Funderburg, S. A., & Levy, P. E. (1997). The influence

of individual and contextual variables on 360-de-

gree feedback system attitudes. Group and Organi-

zational Studies, 22, 210230.

Goldsmith, M., & Underhill, B. (2001). Multi-source

feedback for executive development. In D. Bracken,

C. Timmreck, & A. Church (Eds.). The handbook of

multi-source feedback (pp. 275288). San Fran-

cisco, CA: Jossey-Bass.

Hazucha, J. F., Hezlett, S. A., & Schneider, R. J. (1993).

The impact of 360-degree feedback on manage-

ment skills development. Human Resource Man-

agement, 32, 325351.

Healy, M. C., Walsh, A. B., & Rose, D. S. (2003). A

benchmarking study of North American 360 degree

feed back practices. Poster presented at the 18th

Annual Conference of the Society for Industrial

and Organizational Psychology, Orlando, FL.

Heneman, R., & Gresham, M. (1998). Performance-

based pay plans. In J. Smither (Ed.), Performance



22/23


appraisal: State of the art in practice (pp. 496536).

San Francisco, CA: Jossey-Bass.

Heslin, P. A., & Latham, G. P. (2004). The effect of up-

ward feedback on managerial behavior. Applied

Psychology: An International Review, 53, 2337.

Huet-Cox, G. D., Nielsen, T. M., & Sundstrom, E.

(1999). Get the most from 360-degree feedback: Put

it on the Internet. HR Magazine, 44, 9299.

Johnson, J. W., & Ferstl, K. L. (1999). The effects of in-

terrater and self-other agreement on performance

improvement following upward feedback. Person-

nel Psychology, 52, 271303.

Kamen, P. (2003). The way you use it. CMA Manage-

ment, 77, 10.

Klimoski, R., & Jones, R. (1989, August). Acceptance

of feedback as a function of self-appraisal. Paper

presented at the National Meeting of the Academy

of Management, Washington, DC.

Kluger, A. N., & DeNisi, A. (1996). The effects of feed-back intervention on performance: A historical re-

view, a meta-analysis, and a preliminary feedback

intervention theory. Psychological Bulletin, 119,

254284.

Levy, P. W., & Williams, J. R. (2004). The social context

of performance appraisal: A review and framework

for the future. Journal of Management, 30,

881905.

Locke, E. A., & Latham, G. P. (1990). A theory of goal

setting and task performance. Englewood Cliffs,

NJ: Prentice Hall.

London, M. (2001). The great debate: Should multi-source feedback be used for administration or de-

velopment only? In D. Bracken, C. Timmreck, & A.

Church (Eds.), The handbook of multi-source feed-

back (pp. 368387). San Francisco, CA: Jossey-

Bass.

London, M. (2003). Job feedback. Mahwah, NJ: LEA.

London, M., & Smither, J. W. (2002). Feedback orien-

tation, feedback culture, and the longitudinal per-

formance management process. Human Resource

Management Review, 12, 81100.

Longnecker, C., Sims, H., & Gioia, D. (1987). Behind

the mask: The politics of employee appraisal. Acad-emy of Management Executive, 1(3), 183193.

Luthans, F., & Peterson, S. J. (2003). 360-degree feed-

back with systematic coaching: Empirical analysis

suggests a winning combination. Human Resource

Management, 42, 243256.

Mabey, C. (2001). Closing the circle: Participant views

of a 360 degree feedback programme. Human Re-

source Management Journal, 11, 4154.

Maurer, T. J., Mitchell, D., & Barbeite, F. G. (2002). Pre-

dictors of attitudes toward a 360-degree feedback

system and involvement in post-feedback manage-

ment development activity. Journal of Occupa-

tional and Organizational Psychology, 75, 87107.

Maurer, T. J., & Palmer, J. K. (1999). Management de-

velopment intentions following feedback: Role of

perceived outcomes, social pressures, and control.Journal of Management Development, 18, 733751.

Mesch, D. J., Farh, J., & Podsakoff, P. M. (1994). Effects

of feedback sign on group goal setting, strategies,

and performance. Group & Organization Manage-

ment, 19, 309333.

Raghuram, S., Wiesenfeld, B., & Garud, R. (2003).

Technology enabled work: The role of self-efficacy

in determining telecommuter adjustment and

structuring behavior. Journal of Vocational Behav-

ior, 63, 180198.

Roberts, G. E. (2002). Employee performance ap-

praisal system participation: A technique thatworks. Public Personnel Management, 31, 333342.

Ryan, A. M., Brutus, S., Greguras, G., & Hakel, M. D.

(2000). Receptivity to assessment-based feedback

for management development: Extending our un-

derstanding of reactions to feedback. Journal of

Management Development, 19, 252276.

Rynes, S. L., Gerhart, B., & Parks, L. (2005). Personnel

psychology: Performance evaluation and pay for

performance. Annual Review of Psychology, 56,

571600.

Seifert, C. F., Yukl, G., & McDonald, R. A. (2003). Effects

of multisource feedback and a feedback facilitator onthe influence behavior of managers towards subor-

dinates. Journal of Applied Psychology, 88, 561569.

Smither, J. W., London, M., Flautt, R., Vargas, Y., &

Kucine, I. (2003). Can executive coaches enhance

the impact of multi-source feedback on behavior

change? A quasi-experimental field study. Person-

nel Psychology, 56, 2344.

Smither, J. W., London, M., & Reilly, R. R. (2005). Does

performance improve following multi-source feed-

back? A theoretical model, meta-analysis, and re-

view of empirical findings. Personnel Psychology,

58, 3366.

Smither, J. W., London, M., Reilly, R. R., Flautt, R., Var-

gas, Y., & Kucine, T. (2004). Discussing multisource

feedback with raters and performance improve-

ment. Journal of Management Development, 23,

456468.

Smither, J. W., London, M., & Richmond, K. R. (2005).

The relationship between leaders personality and

their reactions to and use of multisource feedback:



23/23


A longitudinal study. Group & Organization Man-

agement, 30, 181211.

Smither, J. W., Walker, A. G., & Yap, M. (2004). An ex-

amination of the equivalence of web-based versus

paper-and-pencil upward feedback ratings: Rater-

and-ratee-level analyses. Education and Psycholog-

ical Measurement, 64, 4061.

Taylor, M., Fisher, C., & Ilgen, D. (1984). Individuals

reactions to performance feedback in organiza-

tions: A control theory perspective. Research in

Personnel and Human Resources Management, 2,

81124.

Testa, M. R. (2002). A model for organizational-based

360 degree leadership assessment. Leadership &

Organization Development Journal, 23, 260269.

Waldman, D., & Atwater, L. (1998). The power of 360

degree feedback. Houston, TX: Gulf.

Walker, A. G., & Smither, J. W. (1999). A five-year study

of upward feedback: What managers do with their

results matters. Personnel Psychology, 52, 393423.

Multi Source Feedback

Documents

Transcript of Multi Source Feedback