"improve fedpol!": an auto-evaluation using CAF

In 2000, the Common Assessment Framework (CAF) was launched, the first European quality management tool: from the public sector for the public sector. The evaluation model aims at improvement through self-assessment. Since then, the CAF has been revised four times and today counts 3868 registered users in 52 countries.

Among other things, CAF users have access to an almost 90-page brochure which explains the method and content of the model to the reader step by step and provides information on the design of the entire self-evaluation process. In addition, the questionnaire that the auto-evaluators have to fill out is available online. The user does not have to pay anything for it. Thus, the CAF is particularly enticing with the buzzwords "free of charge", "easily accessible" and "easy-touse tool". These promises also convinced fedpol to use the CAF for the self-assessment.

Let's "improve fedpol!"
With the auto-evaluation, fedpol wanted to take a first step towards comprehensive quality management, which until then had been limited to isolated initiatives by the directorate's divisions. The proposal found fertile ground with the Director: The focus was on the possibility of taking a snapshot of fedpol's strengths and weaknesses. A systematically compiled state of affairs also provides a good basis for comparison at a later date. This is how "improve fedpol!" came into being. It took a good 12 months from the first discussion of the idea with the management to the publication of the final report.

The CAF model
The CAF is based on the Excellence Model of the European Foundation for Quality Management (EFQM). The auto-evaluators assess the organisation on the basis of 9 thematic fields. The subject areas 1 to 5 concern management and control practices and form the group of "enablers". Topics 6 to 9, the "results", refer to the results that the organisation achieves for citizens, customers and society.

Implementation of "improve fedpol!"
The CAF foresees 10 steps for the implementation of the self-assessment. Up to the implementation of the action plan, step 9, fedpol has carried out all the stages according to the process. The last stage, a repeat of the auto-evaluation at a later date, is being planned. Two groups of 9 auto-evaluators were formed for fedpol, which has around 900 employees. In order to obtain representative results, criteria were defined on language, gender, field of work, hierarchical level and work experience at fedpol. This resulted in a total of 120 pages of input, which had to be organised, analysed and summarised. From the inputs, 14 fields of action were identified and prioritised. This resulted in an action plan with over 30 improvement measures. These include, for example, the development of a fedpol corporate strategy and the strengthening of internal and external communication.

fedpol communicated the purpose, activities and results of the project in a transparent and accessible manner. The communication channels used were numerous: intranet, giveaways, Christmas and Easter cards, information stand for all employees at the annual event, interview in the internal magazine, interim and final report. The aim of the communication was to make as many employees as possible aware of the project and to explain that it was explicitly about continuous improvement and not a savings exercise.

Experiences with the model Accessible and free of charge?
Anyone can use the CAF and register as a user. The brochure, available in several languages, and the e-tool provide helpful and complete support, even if the CAF's online presence may seem somewhat outdated. The CAF also offers other services, including a reference book of good practices and the organisation of user events, which serve to promote exchange between CAF users. From this point of view, the CAF puts in the user's hands, free of charge, a guide and all the tools to carry out a self-assessment. However, the aspect "free of charge" needs to be nuanced. Firstly, the internal effort involved should not be underestimated. In the case of fedpol - first implementation of the CAF and newcomer to QM - it amounted to over 1500 working hours for project management and office. In addition, there are on average 5 days of work per auto-evaluator and about one day for the management. Not to mention the support provided by the Communication and Media Unit.

Other costs must also be factored in to fund improvement proposals. Securing these resources at the beginning of the self-assessment is crucial for its success and yet not easy. In fact, it is hardly possible to estimate the resources needed for the improvement actions until the action plan is in place.

Furthermore, the inclusion of any external support for carrying out the self-assessment must be calculated. fedpol refrained from doing this because it was particularly important for the project team to have a good knowledge of the organisation. In addition, the project manager of "improve fedpol!" already had experience with the CAF, so the necessary practical know-how was also available internally.

Easy-to-use?
The user is guided through the self-assessment from A to Z, but there is also a certain freedom of design. In this sense and from a project management point of view, the CAF is suitable for both experienced and inexperienced project managers.

With regard to the 9 topic areas, it should be underlined that the CAF does not prioritise. However, the 18 auto-evaluators have noted that the relevance of the questions varies. Depending on the organisation and the objective of the self-evaluation, it may be worthwhile to question the relevance of one or more of the criteria.

In terms of content, the questions were differently manageable for the auto-evaluators and the answers were accordingly more or less com

Steady improvement, as we all know, is a matter of perseverance.

plete. Two factors were decisive: firstly, familiarity with topics such as "customer orientation" and "measuring results"; secondly, the extent of the person's knowledge of fedpol as a whole. It should be noted, however, that even answers that appear "irrelevant" or missing are meaningful - for example, they revealed the weaknesses of internal communication.

The grading scales have been taken over unchanged from the CAF. This saves the effort of setting individual specifications and minimum standards and favours benchmarking with other CAF users. However, it also means that the benchmark is an ideal organisation and the results risk being correspondingly sobering. Furthermore, the grading scales were far from intuitive for the auto-evaluators. The fact that the scales for enablers and outcomes were different created additional ambiguity. Thus, scores were often assigned emotionally without attention to the associated demin circle: Plan-Do-Check-Act.

Conclusion: An inexpensive and simple entry into quality management
Steady improvement is known to be a matter of perseverance. It requires a willingness to continue efforts that are not immediately profitable. A great deal of persuasion is indispensable to prevent fears of a token exercise and another paper tiger. For its part, self-assessment requires the courage to look in the mirror, the ability to accept criticism and an open mind, the full support of top management and trust in employees. Even the CAF does not work miracles in this respect. What is clear, however, is that "improve fedpol!" has triggered a debate on the principles of EFQM and the Deming wheel among both management and staff.

In addition, the use of the CAF has facilitated fedpol's self-assessment and led to a comprehensive picture of the office's strengths and weaknesses. The forthcoming auto-evaluation will show whether the right conclusions have been drawn.