The problem with employee surveys - the right answer or the true answer?

Here's an interesting question - are you assuming that because people tell you the right answer that they believe in it?

I ask, because I'm looking at some interesting employee responses to a recent survey. With a colleague, we changed the usual response format from 1-7 Likert scales of "How engaged do you feel on a scale of 1-7?" to a combinations of triads and polarities going from one extreme to another. (See this previous post for the background.)

What's been interesting is the comments coming back from people appended to the end of the survey. A fair number of people have come back saying "I wasn't sure what the right answer was in this format, I hope I've got it right." It raises some interesting issues:

• Did we make it too confusing?
  Possibly - the new format wasn't explained in advance and the only instruction was "Please put a mark on the triangle to show where your answer is". The questions themselves and the language on the triads were commonplace and straightforward. More explanation or a chance to do it on a trivial example first might have helped.
• Was there a right answer anyway?
  Nope - we were asking which area(s) people thought the organisation was particularly good at. We were genuinely wanting to know what people thought - we had suspicions, but wanted their reactions.
• Why were they looking for the right answer?
  This is the crunch question, for my money.

Why were employees looking for the right answer in an employee survey?

It's a natural thing to spot the answer people are looking for in surveys - we do it pretty much unconsciously. You know if you're given a range of answers for a number that you discount numbers that don't fit the majority pattern and that you tend to go for the mid-point of a range. You know if you're given a standard scale of Bad-to-Good, people are looking for answers at the Good end of the scale. And the human instinct (unless you're a maverick or in a bad mood) is to "gift" the answer within your comfort zone. (So, you'll tend to the Good end of the scale, but you may not be comfortable giving 10, so you'll go 8 or 9.)

But to find people concerned about finding - and giving - the right answer seems to indicate a culture where it is more important to understand and replay the language - conforming to what they think the organisation expects. It's evident, too, in the actual answers - particularly because there is one triad where the "right" answer is evident, but people who have developed a pattern of putting there mark in the centre of a triangle (i.e. not being willing to commit to any particular strength, but taking the non-commital approach) put their mark in the centre because they've got used to that.*

It does, however, make me wonder about all the standard employee surveys - are they actually testing for compliance and the ability to feedback what the company has said to them? I have a horrible feeling that that's increasingly the case.

*The upshot is that this putting-it-in-the-centre-all-the-time could be useful. By possibly including one triangle where the answer is evident (and where the centre is inappropriate in most cases), we'll be able to check and filter out sets of responses where that easy default option has been taken.