Report summary: Evaluating Reward Effectiveness - Survey Report

Conducted in mid-2009, the e-Reward survey, undertaken in partnership with the Institute for Employment Studies, produced up-to-date and revealing information from senior HR and reward practitioners in 173 UK-based organisations employing around two million people. It examined all the main aspects of reward effectiveness and the process of evaluation, shedding light on this relatively little understood area.

Who took part in the survey?

The majority of the 173 respondents, exactly three-quarters, were from the private sector, while 21 per cent were public sector organisations. In contrast, just four per cent were based in the voluntary sector.
More specifically, respondents were drawn from 18 different business sectors, with notable numbers in four areas: banking, insurance and finance; central and local government; consultancy, business services and professional services; and telecom, IT software and ecommerce.
By size, as measured by the number of staff employed, respondents were relatively evenly spread with the largest proportion, just over a quarter (27 per cent), based in companies with between 1,000 and 4,999 employees. In contrast, the smallest number came from those organisations with over 50,000 staff but still represented a fairly significant element of survey respondents – one in 20.

Recent changes to reward

Before considering reward effectiveness in more detail, respondents were asked the related question of whether they had made changes to any of their key areas of reward in the last three years. The most widespread changes had been made in three areas: reward strategy; bonus and incentive plans; and benefits, allowances, flexible benefits and/or voluntary benefits.
By far the most prominent reason given for such moves was to align with changes in business strategy/reflect business needs, mentioned by almost seven in ten respondents (68 per cent). This was followed by three other pressures for change, each mentioned by around half of respondents: cost/financial pressures, changes driven by the need to reflect/match market practice and to address identified weaknesses/shortfalls/ineffectiveness.
Linked to this, respondents were also asked whether their reward function was under greater pressure to demonstrate the value which they and their reward systems delivered. As many as 83 per cent say they were, with 24 per cent of the whole sample claiming that such pressure was quite substantial.

Reward objectives

To provide further context, respondents were asked to outline their most important reward goals. The three most notable objectives were to align with business strategy; external competitiveness to recruit and retain; and to pay for performance and contribution.
Considering all reward objectives, most respondents felt that they were only achieving these ‘reasonably effectively’ (71 per cent), compared with just two per cent who said they were ‘highly effective’ at realising their aims.
In addition, mirroring a trend found in many replies, those in the public sector tended to be less satisfied than their private sector counterparts, giving lower ratings to the delivery of most of their reward objectives.

Views on reward system effectiveness

Underpinning actual reward goals were the systems designed to deliver the various components of reward and most survey participants believed their arrangements could only be described as ‘average’.
Nevertheless, while most replies were clustered around a middling assessment, responses were somewhat skewed. A greater proportion of those replying believed their reward systems were superior to their competitors, although a significant minority reported that they considered their systems to have some key weaknesses or worse.
On a further slight note of optimism, however, 54 per cent of survey respondents said that their reward systems had improved over the previous 12 months, with only four per cent believing that they had worsened. The other 42 per cent reckoned that effectiveness remained unchanged. When asked about the effectiveness of reward systems relating to their main employee groups, there was not a great deal of difference from the results relating to the whole sample.
Nevertheless, some small differences emerged. Reward systems for executives and senior managers were considered marginally more effective than those used for other groups, while those relating to sales and manual workers were slightly less effective.
Taking a list of 11 specific reward areas – everything from reward communications to share plans – ratings for their current effectiveness, when expressed on a scale of between 1 and 10, were clustered around the middle. Overall, however, rankings were skewed towards the less effective end.
A number of patterns more generally did emerge. Most notably, reward systems were more likely to be effective in delivering benefits, especially pensions, while systems relating to reward communications and recognition plans tended to be less successful.

Obstacles to improving the effectiveness of reward programmes

So what were the main obstacles to improving the overall effectiveness of rewards in organisations? Most prominent among these was a lack of resources and time, mentioned by 56 per cent of all respondents. This was followed by line management attitudes, cited by 47 per cent, and a lack of information and data, referred to by just over a third (34 per cent).
In particular, the public sector suffered the most from a lack of time and resources.

Evaluating the effectiveness of reward arrangements

Having an opinion on the effectiveness of reward practices is one thing, but actually systematically evaluating the operation of various reward practices is quite another. Just under half of respondents (46 per cent) reported that they had undertaken a full evaluation of their reward policy and practice in the previous 12 months. A further 36 per cent told us that they had only ‘partially’ undertaken such an exercise.
In view of the time, trouble and expense new reward initiatives generate, it was remarkable that no fewer than 18 per cent of respondents had made no attempts whatsoever to evaluate the effectiveness of their pay and reward arrangements.
For those who had not attempted to undertake an evaluation, the main barrier was a lack resources and time, mentioned by 82 per cent of this sub-sample, while insufficient information and data and senior management views (both 26 per cent) were the other main stumbling blocks.
Of those that had carried out a review, the overall feeling was one of satisfaction, with 54 per cent ‘partly satisfied’ and 41 per cent ‘satisfied’. In contrast, just one respondent was ‘delighted’, while four per cent reported that the process caused them to feel ‘depressed’.

Methods used to assess reward effectiveness

Where a partial or full review of reward effectiveness had taken place, as many as 86 per cent of those who had attempted to evaluate their pay and reward arrangements said that methods used include external market pay surveys, while 75 per cent carried out staff attitude surveys and 64 per cent analysed internal data.
On average, organisations used between three and four methods of assessment, with a range of between one and seven.
There was no pattern between the number of methods of assessment used and ratings respondents allocated to reward effectiveness.

General criteria used to assess reward effectiveness

Of the general criteria used to evaluate effectiveness, the most prevalent was to measure financial impact and costs, used in 79 per cent of organisations. This was followed by external benchmarking and HR outcomes such as labour turnover and absenteeism, both mentioned by around 70 per cent of this group. Other prominent, though less common criteria, included stakeholder views and business KPIs.
Where a partial or full review had taken place, on average, organisations used around three general criteria, with a range of between one and five.
No clear pattern emerged between general criteria used to evaluate reward and the ratings allocated to reward effectiveness. Typically, those respondents ranking their reward systems highly used the same general criteria as those that deemed them poor.
Nevertheless, average effectiveness scores increased as the number of general measurement criteria increased. While the relationship between the two is, no doubt, complex, we can at least interpret this finding by suggesting that the more carefully considered information an organisation collects, the more likely it is to achieve effective rewards.

Specific measures used to assess reward effectiveness

More specifically, of the detailed measures organisations used to assess reward effectiveness, three stood out. These were measures of employee attitudes, mentioned by 75 per cent of those attempting to evaluate reward effectiveness, analysis of pay market positioning (72 per cent) and staff turnover rates (62 per cent).
Less prominent measures were length of service, absenteeism rates, vacancy rates, job refusal rates and other business metrics, all mentioned by fewer than 30 per cent of those attempting to evaluate reward effectiveness.
Where a partial or full review had taken place, on average, organisations used around five specific measures, with a range of between one and 12.
A clear pattern surfaced when these specific measures used to assess effectiveness were examined. Although not the most widespread of the measures used, the three most influential were:

assessment against reward strategy objectives
other business metrics – eg sales, customer service
business financial performance.

When the ratings respondents allocated to the effectiveness of their reward systems were ordered from high to low, it was clear that these three specific measures were more in evidence in ‘effective’ organisations than those with less successful reward systems.
On closer inspection, it was the assessment against reward strategy objectives that appeared to be the key measure associated with effectiveness of reward. When the reward ratings were examined more closely it was this measure that was both associated with the largest outright effectiveness scores and showed the biggest differential when compared to the average of those organisations not using this type of measure.

Ways of improving organisations’ ability to assess and demonstrate the effectiveness of pay and reward

Survey participants who had already carried out an evaluation of reward effectiveness were asked what would help them improve their ability to assess and demonstrate the effectiveness of their pay and reward arrangements. Responses were spread quite evenly, with the most notable, mentioned by 65 per cent of this sample, an improvement in employee communications or understanding.
Other prominent reasons included more time and resources (56 per cent), better quality HR data (54 per cent), and improved HR systems (52 per cent).
Splitting those surveyed into companies who rated their reward arrangements as relatively effective and those who considered themselves poor performers in this respect showed that three factors were particularly absent in lower performing organisations. These were: improved employee communications/understanding; more time and resources; and changes in management attitudes.

Related publications

 Evaluating Reward Effectiveness