This week’s blog from Emma Bridger
There is no doubt that data and insight regarding employee engagement within your organisation is a critical part of the engagement strategic planning process. It’s vital to understand how your employees are feeling, how engaged (or disengaged) they are, and what action needs to be taken to improve engagement. You also need to be able to track and monitor action taken to understand what works and inform your plan. However over the years the employee engagement survey has taken centre stage for many organisations looking to improve engagement. The employee engagement survey is a means to an end not the end itself. If used wisely the survey process and the insight provided can be invaluable, but often over-reliance on the survey process results in a transactional approach to employee engagement. This approach involves putting the survey process at the heart of engagement activity, it is a reactive way of approaching engagement and can feel like a tick-box exercise: once the action planning has finished the company forgets about engagement until it’s time for the next survey.
I believe that companies have become over-reliant on the survey because it provides a false sense of security, giving the illusion that something is being done about engagement. Improving engagement can feel like an overwhelming task, but undertaking a survey gives confidence that something is happening to tackle engagement. Survey feedback always involves providing the results in the format of charts, data and engagement indexes. This can feel like a comfortable and familiar place for many senior leaders who operate in a world of management reports and numbers. The survey takes something, which is predominantly about feelings, emotions and behaviours, and then uses the language of data and numbers to communicate findings. This is, I believe, part of the reason why companies have become over-reliant on the survey and why we still see transactional engagement as the predominant approach in organisations today.
So whilst data and insight are important to improve engagement, there are a number of problems which you need to be mindful of when using an employee engagement survey.
- Definition areas
We have already talked at length regarding the challenge of defining employee engagement before you can measure. However I would go one step further. In my experience engagement is very personal, and whilst there are common themes in the different definitions, in reality what defines engagement, and what drives engagement can differ from company to company. Each engagement survey provider will explain why their model is the right model and back this up. However in my experience there is no substitute for undertaking qualitative research to understand:
- how your organisation wishes to define engagement
- what drives engagement for your employees
Undertaking qualitative research to answer these questions, which will then allow you to build a bespoke measurement tool which will be valid for your organization, that it will measure what it is supposed to measure.
- Obsession with response rates
Many companies still seem to be obsessed with improving their response rates to the annual survey, to such an extent that this can often be seen as a key component of their engagement strategy: to improve survey response rates by x%. I’m unsure where this pre-occupation with response rates began, but again I believe this could be because it offers a useful diversion from the real challenge of actually improving engagement. That is, if we focus on improving response rates, and we have success in this area then it feels like where we’re making progress, right? Wrong. Common sense would say that higher response rates are better, the more people filling out the survey the more engaged they must be. But this of course is not true. I have seen many engagement surveys over the years with 80% plus responses rates but low engagement scores. When you look at best-practice social science research, what we actually need is a representative sample of employees to ensure that the findings we observe can be generalized to the wider population. Therefore as long as we sample in the correct way, and the sample is large enough, this is enough to give us what we need. The question of sample size has long been debated within social science, but there are some helpful sample size calculators which you can find on the internet which take account of:
- your total population size i.e. number of employees
- the margin of error you’re willing to tolerate, usually between 3% and 6%
- they then provide a calculation for how many responses you need to have either 90%. 95% or 99% confidence level
To illustrate:
- Population size: 10,000 employees
- Margin of error: 3%
- Sample size:
- 90% confidence = 703 responses
- 95% confidence = 965 responses
- 99% confidence = 1556 responses
You’ll see for this example you’re looking at a 10% – 15% sample size for the results to have a low margin of error and high confidence level. This is nowhere near the 80% plus response rates companies go after.
The same tools also provide a calculator to work out how many surveys you’ll need to send out to generate the required response rate.
To illustrate:
- if we are going for 95% confidence we need 965 responses back
- our predicted response rate is 35%
- therefore we will need to send out 2754 surveys to ensure we reach the necessary response rate
Finally, the tool then calculates the accuracy of your response rates.
To illustrate
- population size is 10000
- we actually had back 2460 completed responses from the 2754 surveys we sent out
- which gives us:
- an error level of 1.4% at 90% confidence
- an error level of 1.7% at 95% confidence
- an error level of 2.3% at 99% confidence
You can see form the example above that with a population of 10,000 employees, an approximate response rate of 25% comes with a very low error rate and a high confidence level. And yet I’m sure many CEOs would be disappointed with a 25% response rate. Again this comes back to establishing the purpose of the survey and understanding what a higher response rate will actually give us?
Finally before we move on, a quick word on sampling. In order to be able to generalize any findings to the wider population we need to ensure we take what is known as a probability sample. A probability sample means that everyone within the population i.e. employee base has an equal chance of being included in the sample. The easiest way to achieve this is to send the survey to everyone. Or you can select a sample of employees to send the survey to, as detailed by the worked example above, you just need to ensure you use probability sampling methodologies so that you can generalize the results once they in.
- Mind your language
One of the challenges of using a survey is that of the semantics and language involved. For example often surveys will ask questions about senior leadership: depending on where employees sit in the hierarchy this will mean different things to different people. Establishing questions with as little ambiguity as possible is a real challenge. Often this ambiguity and differences in interpretation become clear in the survey follow-up process. For example when running focus groups to delve deeper into issues raised within the survey it becomes clear that employees have interpreted certain questions in very different ways.
- Obsession with benchmarking
In addition to obsessing about response rates, in my experience senior leaders are also obsessed with external benchmarking. Making the Sunday Times Best Companies list, or the top companies in the Great Place To Work list, are often set as employee engagement objectives. Whilst this motivation does provide a burning platform to enable a focus of resources on engagement, very often making the list eclipses the purpose of engagement itself, to improve the business and create a competitive advantage. The objectives and plan are at risk of being short-term when the focus is to benchmark. I would prefer to view making a list as a benefit of having a focused engagement strategy and plan, rather than the plan itself.
- Deficit approach
Finally, one of the major problems with placing the survey process at the heart of your engagement strategy and plan is that it encourages a deficit approach to engagement. Action planning often focuses on fixing those areas in which the business performed poorly in, rather than learning from the insight about the organisation’s areas of strength. When focus groups are run after the survey process there are rarely questions asked about what we are doing well in and how we get more of this. Very often the post survey discussion and action planning is quite a depressing process: looking at the bottom scores and coming up with ways to improve them. This in turn results in an engagement survey process that is actually quite disengaging. Even when companies are performing really well, when it comes to engagement, the action planning process will often still focus on those areas that could be better.
So, whether the annual employee engagement survey is new to your organization or not, it’s worth remembering why it is you want to do it, and what you want to learn, and how you might apply the outcomes (good or bad), before writing the first question or consulting an external provider. The heart of your survey should be your employees, and not just a hollow and standard practice that takes the beat out of your engagement process.
You can also read-up on interpreting your survey results, click here to read our blog.
If you would like to know how People Lab can help your organisation, request a call-back here.
