CHAPTER 9 | challenges in complaint handling

Benchmarking

It has been customary for the Ombudsman's office to aggregate complaint statistics and produce a report annually on the outcome of investigations. Some people rely on these statistics as an indicator of standards and trends in public administration. As this report shows, we produce statistics on different aspects of complaint handling, including the number of complaints against individual agencies, the number of issues raised in those complaints, the proportion of complaints handled by formal investigation or in some other way, and the outcome of complaint investigations. If those statistics are to convey an accurate picture, there must be an equally sophisticated system for recording and interpreting them.

The production and interpretation of statistics poses an ongoing challenge for the Ombudsman's office. One dimension of this challenge is to ensure consistency in data entry. This means that Ombudsman staff who are entering data into COMBi must all work to the same definitions—for example, they should have a common understanding of the term 'defective administration'. This is easier to achieve in some areas than others. The office's Taxation Team checks the consistency of outcomes by regularly reviewing all ATO complaints involving some administrative deficiency. It is difficult to replicate this process in some other areas of the office not supported by a specialist team. Another method for achieving accuracy and consistency across the office is to hold regular meetings of all investigation officers from around Australia to enable benchmarking issues to be discussed. During 2003–04, we reviewed and simplified the classification of complaint outcomes applied by investigation officers.

'Another prerequisite for high-quality data entry is best-quality work practices in the office.'

Another prerequisite for high-quality data entry is best-quality work practices in the office. At one level this means, in the context of a national institution with eight separate offices, that it is necessary to monitor constantly the workflow in each office so that staff have the time and opportunity to perform tasks such as data entry as proficiently as possible. Training and technological support for staff in each office is equally important. At another level it is important that the policies and manuals on which staff rely for guidance are comprehensive and current. A project reviewing the office's investigation guidelines was commenced during the year, with a particular focus on ensuring that they are congruent with the new complaints management system. It is expected that this will result in greater simplification and standardisation of data entry and statistical reporting across the office.

Statistics, of course, tell only part of a story. It is the way they are interpreted that conveys the real message. A question often asked of the Ombudsman is whether it is good to see complaint numbers go up or down. There is no easy answer to a simple question of that kind. For example, a reduction in complaint numbers to the Ombudsman can be read in contradictory ways. It could mean that public administration is improving and that agencies are 'getting it right' more often. Or it could mean that fewer people are aware of the service provided by the Ombudsman. Worse still, it could perhaps mean that people are disillusioned with the service and see little point in complaining. While it is difficult to be definitive, the discussion of these possibilities will be better informed if there is contextual data available on how people perceive the office. A client satisfaction survey conducted during the year yielded valuable benchmark data for assessing how well complainants think we are doing our job.

Benchmarking is a common problem for all Ombudsmen in Australia, in both the public and private sectors. During the coming year we will be working with other Ombudsmen to explore how we measure performance. A starting point will be to agree on definitions of key concepts and terms, such as 'complaint', 'issue', 'investigation', 'preliminary investigation' and so on.

'Benchmarking is a common problem for all Ombudsmen …'

In summary, we are concerned that our current data does not provide a sufficiently reliable picture of performance, workload and results. We will be working to improve this situation over 2004–05.