Executive Summary
This executive summary is divided into three sections: the first gives an outline of what the project attempted to do, the second puts this into an accounting and business context, and the third discusses the results and their implications.
The project
This project used a laboratory experimental approach to study whether financial incentives work in complex decision-making environments. The hypothesis was that, if a subjects earnings were related to performance, this would induce individuals to use information systems more fully and efficiently, to learn faster and to take better decisions, and hence to turn in higher levels of performance. The participants in the experiments in essence played a stylised business game, involving a sequence of decisions over time. Individuals could choose to access information about their past performance, and this information could be used to help inform decisions for the future. Accessing information was costly, but intelligent use of the information could result in higher levels of profit performance .
The participants in the experiment were either level 2 or level 3 undergraduates or MA, MSC and MBA postgraduates. Participants were divided into two groups: an incentive group in which an individuals earnings depended on the quality of their decision-making; and a control group, for whom earnings were independent of their decisions. The study examined the use of information and the ensuing level of performance for individuals in these two groups, and also looked at how performance was affected by individual characteristics. The individual characteristics measured were: (i) the individuals latest university grade point average; (ii) age; (iii) gender; (iv) nationality; and (v) whether the participant was undergraduate or postgraduate .
The context
Individuals working in organisations in the real world are typically faced with a complex array of incentives that influence the way in which they work. Some incentives are immediate and financial in nature, such as piece-rate pay. Other incentives are more distant, such as prospects of promotion and improved on-the-job perks. The design and appropriate use of incentives is fundamental to efficiency at the personal level and hence to the efficiency of the organisation, of which the individual is a part. Incentive schemes naturally depend upon information: accounting systems are concerned with the provision of information; and management accounting is concerned with the use of that information in an organisational context .
It is possible to study how different systems and structures operate by examining their use in the real world either through in-depth case studies or through the statistical examination of how organisational performance measures can be related to a range of explanatory variables. However, in-depth case studies, whilst interesting in their own right, suffer from specificity; it is always difficult to know whether what works in one environment will carry over and work in slightly or very different ones. Likewise, cross sectional approaches suffer from real measurement difficulties both for dependent and independent explanatory variables. These observations motivate the experimental approach adopted in this project.
This experimental approach involves structuring a controlled business-like environment and designing explicit information and reward structures for this environment. Participants then play an experimental game and earn real money rewards as a consequence of their decisions. The advantage of this approach is that the experiment can be designed to explicitly address the issue under investigation. In the present application, the issue is whether, in essence, incentive pay makes any difference to the way individuals approach the use of information in complex decision environments. The experimental approach does, of course, have some drawbacks. This is so, in part, because the experiment environment is intrinsically somewhat artificial, and the amount of time that one can ask participants to commit is usually relatively short; in the present study up to about 90 minutes. On the upside, it brings with it much of the power of scientific method. Perhaps the most important part of this lies in the fact that anyone can cross check results of a given experiment by undertaking experimental replications. Indeed, this is the purpose of the present study. The aim is to check on the veracity and robustness of previous US work in a UK context and to extend this work in various ways .
Researchers have extensively studied how decision-makers use information in reaching their decisions, and also whether performance of tasks is improved by offering formal performance-related rewards rather than fixed rewards. Empirical tests of theoretical predictions concerning the use of information by a single individual have shown that individuals do not generally choose to access economically optimal information systems and that they also do not process information consistently with theoretical models. In multi-person organisational environments, researchers have found that performance-related contracts, linking rewards to monitoring variables, motivate individuals to improve their performance in a range of tasks, as compared to contracts with fixed rewards (Sprinkle, 2003).
However, a common environment for employees with decision-making authority is one in which the two roles for information are combined. Employees have access to information that could enhance their performance of a task, and their task performance is monitored. Thus, an interesting question, relating to the use of formal performance-related rewards as well as to the provision of accounting information, is whether, in these more complex yet widespread environments, formal performance-related rewards enhance the use decision-makers make of valuable decision-related information and, therefore, improve task performance.
To put the issue another way, the two functions of information may not be independent: if an employee could improve decision-making by accessing and utilising information, but needs to be motivated to do so by a formal linking of subsequent extrinsic rewards to some measure of performance, then it is reasonable to suppose that an employee with formal performance-related rewards would make better use of decision-facilitating information, and on average also perform better as a result, than an employee with fixed rewards.
A handful of experimental studies relating to this dual informational role have been published in the accounting and social psychology literatures. The results are mixed; some studies find that formal performance-related rewards tend to enhance task performance; others find they degrade or have no significant effect on task performance .
A consensus view on the meaning of these disparate results has not yet been reached partly because of differences between studies in experimental environments selected for examination and partly because of alleged deficiencies in some of the experimental designs employed. In the present study, the aim is to undertake replication, with improvements where possible, of a recent research design. Factors such as gender, age, and innate task ability, that might help to understand the diversity in empirical findings and that to the best of our knowledge have not been investigated at all, are taken into account in this study. Measuring and incorporating some of these factors into the analyses could bring at least two benefits. Firstly, any systematic differences between experimental subjects in the formal mechanisms that effectively motivate them raise the prospect of enabling a more informed design of employment contracts and performance measurement systems in natural occurring environments. Secondly, it may lead to an increased understanding of the empirical findings in these kinds of experiments and raise interesting subsidiary questions for future research .
Discussion of results
The original study, on which this experiment is based, found that performance-related incentives significantly improved performance; individuals with such incentives accessed information more and made better use of it. The results were all statistically significant .
The present study increased the sample size by a factor of two and a half times, and improved the structure of the experiment in a variety of ways. Comparing the resulting findings with those previously published, found that whilst they are not contradictory, the earlier findings are now only weakly supported. In particular, the differences between subjects with performance-related and fixed incentives in their accessing of information and in overall performance is positive but very much weaker , and in many cases not statistically significant .
There are many possible explanations for the differences in results. One possible reason lies with the way the earlier experiment was structured: the pre-experiment communication, including a quiz, might have given much greater opportunity for individuals to be trained up to more fully appreciate the way the reward structure operated. Whereas in the present study, the information imparted to individuals was strictly controlled. Individuals were required to learn on the job, that is to learn from their mistakes in initial trials .
The structure of the reward system used in the present study was similar to that used in the previous work, but might have involved undue complexity. The follow-up questionnaire to participants used in the present study reveals that this was almost certainly the case. In particular, there was relatively poor understanding of the cost of accessing information, which subjects believed to be far more significant than it actually was .
Analysis of the influence of personal characteristics showed that, holding all other things constant:
(i)grade point average in university examinations had a positive impact;
(ii)being an overseas student had a negative impact;
(iii)being male had a positive impact; and
(iv)being a postgraduate had a positive impact;
on the level of performance of a participant. All these effects were statistically significant.
Many factors could be considered to shed light on these results. Such factors are necessarily speculative, and there is no further data with which to test them, but they have appeared in the public domain before. The positive impact of grade point average is what might be expected if it actually measures anything to do with the ability to solve a complex decision problem, such as that presented in the experiment. The negative overseas-student effect may reflect language difficulties in a time- pressurised environment relative to a domestic student, though it may also reflect on the nature of schooling overseas where, it is sometimes said, rote learning rather than creativity is emphasised . The positive impact on performance of being a male may reflect the time- pressurised nature of the experiment: there is some debate in educational research about whether females tend to perform somewhat better than males in more reflective, less time-pressured environments. Evidence on performance in examinations versus coursework may be relevant here. It is also worth noting that the grade point average for females in the sample was actually higher than males, partially offsetting the positive male effect. Finally, the positive impact of being a postgraduate is as one might expect; postgraduates are paying full fees and may be, as a consequence, essentially more committed students.
Experience conducting the present experiments gives a strong clue as to the potential source of the lack of statistically significant positive results from performance-related incentives on the use of information and on task performance. These are: (i) the undue complexity of the incentive scheme; and (ii) the fact that the level of rewards on offer is simply not salient enough. On the latter point, several students did not even bother to collect their earnings, and had to be chased up! A natural follow-up to the present work is, therefore, to repeat the experiments with a simplified reward structure and with more salient rewards.
ISBN 1 904574-19-X £5.00
|
|
