ROI EVALUATION IN THE PUBLIC SERVICE
AND NGOs:
 

Can it be calculated? 

By Marius Meyer & Rina Opperman

Today there is an increasing emphasis on improving the quality of services in the public sector. Service delivery forms an essential dimension of the government’s focus to improve public services. To support this objective, the Public Finance Management Act provides an institutional framework for sound financial management in the public sector.  This means that a greater emphasis is now being placed on proper financial planning and governance, as well as the monitoring and evaluation of financial resources. One of the challenges for public managers at public service departments, and indeed NGOs will be to determine the ROI of monies spent on different projects within their sphere of responsibility. 

While tangible measures such as sales may not be visible in the public sector, there are still ways of converting the cost and benefits of different interventions to rand values. We normally challenge the public sector HR practitioners to indicate whether the things they do have no cost or benefit implications. The critical question is then can ROI be calculated?  Here are 10 important sub questions to help you think about this. 

Do you get your supplies for free from your suppliers? 
Do you operate without budgets?
Are there efficiency measures in your departments?
Do you have service delivery standards and measures?
Can you calculate the cost of an error in your department or section?
Are you accountable to a board or to the public, and if so, does this also apply to funds?
If it becomes public that you don’t use your funds wisely, is there a possibility that donors and other funding organisations will withdraw?
Is there a chance that the private sector will channel some of the funds currently allocated to you and rather spend it on the 2010 Soccer World Cup?
If you manage your finances better, will it improve your sustainability?
 

Answering these questions clearly indicates that it is indeed possible to convert the cost and benefits of training and other HR interventions to rand values. And if you have rand values, you can calculate ROI. Let us give you an example of how ROI was calculated in the public sector in New York. During 1980’s there was a huge increase in crime in the greater New York area. The Nassau County Police Department decided to do something about it. They realised that highly skilled police officers will be critical to deal with the increase in crime.  They subsequently embarked on interpersonal skills training for police officers.  The training focused on three key areas: Better communications, more effective teamwork and better relationships within the police department, and also with the community.  Significant improvements were affected. This is how they have calculated the ROI. 

                                                       

             ROI% =  (benefits – costs) x 100  ($333 168 – $136 530)  x 100     = 144%                                    costs                                            $136 530 

From this example you can see that a ROI of 144% has been achieved (McCarty, 2001). This means that for every dollar invested in training, more than $14 is returned, not a bad ROI at all!  The ROI values are based on participants’ estimates in the action plans they have implemented to improve their performance. 

Here in South Africa, a major government department launched a pilot project to calculate the ROI in project management training. They wanted to determine whether their managers can improve results if they apply project management principles and practices to the projects they are involved in.  In some departments significant cost savings were achieved due to the increase in efficiency and financial management of resources allocated to the particular project. A final ROI of 539% was achieved. 

Although the above example relates to training, impact studies can also be done on other HR interventions.  For instance, in a recent study on the impact of HR practices on the performance of public servants in Eritrea (Africa’s youngest and poorest country), eight HR practices altogether explain 56% of the change in performance (Tessema & Soeters, 2006). These HR practices were recruitment and selection, placement, training, compensation, performance appraisal, promotion practices, grievance procedures as well as pension and social security. 

Measuring ROI in training usually requires a major paradigm shift on the part of training and human resource managers. In the light of this, it is important to follow proper guidelines for the effective implementation of ROI measurement.

  • Create an awareness for effective financial management, accountability and ROI in the organisation.

  • Build capacity for ROI by training staff to understand ROI.

  • Quantify information before interventions in order to obtain a baseline (e.g. number of errors, cost of errors, savings due to process improvement etc.).

  • Covert this data to financial value (e.g. the cost of preventing an error).

  • Develop a culture of measurement and accountability among public sector and NGO staff.

  • Start with only one course as a pilot programme to practise ROI skills.

  • Communicate results to training staff and the whole organisation.

  • Design improvement plans for training programmes in order to increase ROI.

  • Once ROI results are available, use the data to market future learning programmes within the organisation.

  • Ensure that the head of finance and the internal auditor, as well as your donors are informed of your ROI results.

In the light of the above, it is clear that ROI can also be calculated in the public sector.  It provides NGOs and public managers with a tool to quantify the value they add to their organisations. What is important, though, is that training managers must commit themselves to collecting all the necessary data - information that can be measured.  To collect this data, they will need to form sound relationships with line managers and other stakeholders of the organisation. Once these results are available, you will be able to determine the financial value of training. This process can assist you to make important decisions about various training interventions, such as the value of management development programmes, whether to outsource training, etc.  ROI provides the training manager with an opportunity to become an accountable public service manager – from merely a trainer offering training, to a manager producing tangible performance results that can be measured in financial terms. 

REFERENCES

Kirkpatrick, D.L. & Kirkpatrick, J.D. 2006. Evaluating Training Programs: The four levels. San Francisco: Berrett-Koehler 

McCarty, R.J. 2001. Resisting Measurement: Evaluating Soft Skills Training for Senior Police Officers – Nassau County Police Department. In Phillips, J. & Phillips, P. Measuring Return on Investment In Action Series, Volume 3. Virginia: ASTD  

Meyer, M., Opperman, R. & Dyrbye, C. 2003. Measuring Return on Investment in Training: A practical implementation guide. Randburg: Knowres 

Tessema, M.T. & Soeters, J.L. 2006. Challenges and prospects of HRM in developing countries: testing the HRM-performance link in Eritrean civil service. International Journal of Human Resource Management 17(1) 86-105 

 

Marius Meyer and Rina Opperman consult in ROI measurement for ROI ONLINE (www.roionline.co.za)
and can be reached on info@roionline.co.za.

They are co-authors of the book “Measuring ROI in Training: A practical implementation guide” and have participated in ROI evaluation projects in South Africa, USA, The Netherlands, Malaysia and Botswana.