360 Degree Feedback: Really a Fatal Attraction? A Rebuttal Post.

Performance Appraisal haters have been around for quite awhile, ranting and raving about the process they claim everyone loves to hate (stay tuned for our upcoming whitepaper on this topic). New to the party though are those experts in the field who have set their targets on discounting 360 degree feedback. No talent management practice is safe when people start picking on this long-favored gem of the 90’s that still happens to be a popular and effective tool for leadership development today.

The well-regarded author of “The Fatal Flaw with 360 Surveys” (http://blogs.hbr.org/cs/2011/10/the_fatal_flaw_with_360_survey.html), Marcus Buckingham, stirred up quite a controversy in his October post focused on outing his big issue with 360 feedback. While the post generated substantial commentary, only some congratulated his insights while most others came to the defense of this long-standing process.  The author makes several arguments against using 360 surveys as a measurement tool, however his most fundamental concern seems to be that the data gathered by multiple raters (themselves a skewed sample due to not being selected randomly) is unreliable given that they are asked to provide feedback on behaviors rather than outcomes. Because raters cannot reliably rate someone else’s behaviors without subjectivity, reliability is said to be compromised.

While from a purely psychometric perspective, the author is correct – behavioral feedback from multiple raters is unlikely to be consistent across time, individuals, and situations therefore making it, statistically-speaking, unreliable -  360 feedback remains one of the most powerful leadership development tools in HR’s arsenal today. Here’s why:

You are not a lab experiment.

While my love of psychometrics got me through graduate school, I am much more passionate these days about helping my clients implement HR processes that take into account organizational realities. Just because Cronbach's α is not high enough, does not mean the process was ineffective for the individual leaders or the organization. Research has demonstrated that 360 feedback is indeed quite motivating to recipients, and that when used in conjunction with proper follow-up goal setting, can result in behavioral improvements.  A recent Wall Street Journal piece even noted how savvy executives are following the transparency trend and publicly displaying their ratings in order to garner the support of those around them as they tackle areas that have been particularly difficult in the past. (http://on.wsj.com/uT9mLx).

People differ in their perceptions.

Although “perception is reality” has become a bit of a cliché, the fundamental premise that people see reality very differently is a scientific fact. Research has shown that reality is constructed by the mind, and that through varying perceptions of sensory stimuli, individuals produce alternate views of reality.  At times, we don’t even know what we are missing. This idea of inattentional blindness or selective attention is now well known due to the invisiblegorilla.com studies in which participants are so consumed with a menial counting task that they actually miss the wacky gorilla presented on screen (here is the more recent version of the experiment should you wish to try it out http://www.youtube.com/watch?v=IGQmdoK_ZfY).

All of this research points to one very important lesson – our perceptions become our reality and impact our own behavioral responses and the working relationships we have. That we are likely perceiving things differently is a fact, and means that we must work to align our perceptions of our team’s environment and form a common plan of action to optimize it moving forward.

Employees need a safe zone.

When is the last time someone, other than your manager, sought you out to communicate that you should “let go of some work -- empower your directs more -- so that you can focus on strategic issues and they can realize their full growth potential”. Hasn’t happened lately has it? Colleagues, direct reports, and raters in other miscellaneous groups often do not have the impetus or audacity to provide behavioral feedback outside of this process. In the end, it is good for employees to know if their peers feel at odds with cross-functional endeavors, for example. The friction between the teams is likely causing many objective targets to be missed and must be addressed at the behavioral level. Oftentimes, the 360 feedback process is the only platform where employees feel comfortable communicating this type of sensitive information.

Behaviors influence outcomes.

In the original post, the author advocates for 360 measures to be defined strictly in terms of outcomes. To increase reliability, he says we must allow people to rate on objective items -- what is actually produced. The strength of 360 feedback, however is that it allows employees to understand the behaviors which lead to certain outcomes. So, in the case of a missed productivity goal, he/she may come to understand the specific actions which are inhibiting execution. This is data that you just don’t want your people to miss.

Calling attention to the 360 feedback process is a good exercise, as we must always ensure our processes are running as best practice as possible. Keep in mind the following:

Focus on the assessment tool. Since we have focused on the psychometric foundations of the 360 feedback process here, let’s talk about what HR should be doing to build a valid and reliable 360 feedback instrument. Great care must be taken to ensure that the behavioral measures within a competency group are indeed indicative of that competency. Further, all of these behaviors should hang together, giving them inter-item reliability. Finally, competency content should be characteristic of the leadership knowledge, skills, abilities and other attributes that are associated with successful leadership within the organization.

Invest in rater training and coaching. We do need to be cognizant that subjective measures are prone to cognitive bias (e.g. judgment, attribution, memory) and rating errors (e.g. central tendency, halo/horns effect). It is always useful to work with raters prior to the process to discuss the behavioral anchors used in the rating scale and calibrate the rater’s use of the scale, as well as to take them through common rating errors so that their awareness can help them avoid these mistakes. Following the process, it is helpful to invest in coaching and feedback report interpretation, as analyzing trends and patterns across so much data can be difficult both practically and emotionally for the employee.  Behavioral coaches can help to gain insight into common perceptions as well as formulate a development plan.