Citizens evaluating local services and facilities in Southern Italy

This innovative project is the result of partnership working between the  Department for Public Administration and Innovation and the nonprofit organisation Cittadinanzattiva which promotes citizen and consumer rights in Italy. In 2006 both organisations signed a protocol to promote new citizen participation initiatives related to service quality.

This partnership gave rise to a pilot project which gives citizens the opportunity to take an active role in the evaluation of public service quality – not just through citizen surveys but as civic evaluators who provide information about the state of public services and infrastructure to local councils and contribute to prioritising improvements.

Cittadinanzattiva has been gathering experiences with civic evaluations in hospitals since 2001. The first pilot project between the Department for Public Administration and Cittadinanzattiva in 2008 built on these experiences, focussing on schools and front offices of local services. A new pilot launched in November 2009 included a much wider range of public services and a more comprehensive assessment methodology. In particular, it focused on issues of the maintenance of green space and roads, street lighting, public transport, garbage collection, cultural and social events. 

This project has been also influenced by recent national legislation guaranteeing citizens and civic associations an active role in the  monitoring of public services through collaborative dialogue.

The objective of this initiative is to improve the quality of public services and citizen engagement by involving citizens and associations in the evaluation of service quality, using the citizen’s point of view as a starting point for the prioritisation of service improvements.

Other expected benefits are

• improved partnership working between individual citizens, civic associations and public agencies;
• increased transparency about the performance of local authorities;
• increased knowledge of citizens about the activities of local authorities, by achieving a close partnership between them;
• detailed information, from the citizen’s point of view, about which public services are most in need of prompt improvements;
• increased social capital at local level, as the citizen evaluators work together to agree on an overall assessment of the services and draft improvement plans;
• increased identity with the local area and ownership of improvement initiatives by encouraging citizens to keep an “open eye” on what is happening in the local area.

Moreover, as the initiative produces data on the actual quality of public services, it supplements data from customer satisfaction surveys and helps to produce an integrated information system on service quality.

The project was coordinated by the Department for Public Administration and Innovation which chaired the working group consisting of representatives of the agency FormezPA, the citizen and consumer assocation Cittadinanzattiva and the Foundation Fondaca which specialises in research on public participatioin.

In the first phase of the project, started in November 2009, a focus group at national level discussed the elements, dimensions and indicators of urban quality. The focus group consisted of public managers, citizens, members of citizen associations and technical and professional experts who were considered to be “issue experts”. The focus group participants agreed on the following elements of urban quality:

• Public safety
• Access, suitability, affordability of public services
• Information about public services
• Subsidiarity
• Inclusion of disadvantaged citizens
• Social events
• Street cleanliness
• Connectivity
• Solid waste
• Maintenance of green space and roads

As a next step, on the basis of the focus group outputs, the working group defined one or several quality dimensions for each of these issues (for example, for the issue public safety the dimensions are: physical safety of people and safety of public infrastructure). Last but not least, the quality indicators were defined in order to operationalise the quality dimensions – for example, for the dimension “safety of public infrastructure” two indicators were defined:

1. Number of houses declared unfit for use (this information needed to be provided by the local authority concerned)
2. Number of threats to safety on the selected road (this information had to be provided directly through the monitoring by citizens – e.g. potholes on the road surface, broken pavements, wrecked steps, inclining poles).

The working group worked with representatives of Cittadinanzattiva to prepare the tools for the civic evaluation, including an operational manual and monitoring grids. Finally, in January 2010  the civic evaluation started with a kick-off seminar in Rome. The participants included public managers from 14 local authorities of the four southern regions of Italy (Campania, Calabria, Puglia and Sicily) who volunteered for the project, as well as local citizen representatives of Cittadinanzattiva.

Now the challenge was to get citizens engaged. The local authorities and the local representatives of Cittadinanzattiva marketed the project through a declaration of intent which was widely promoted through all of their communication channels. Not surprisingly, the take-up was particularly positive in those local authorities which were able to embed the evaluation project in other participation initiatives and which already had a strong network of associations at local level, e.g. San Severo. In other local authorities with weaker social capital the participation by citizens in the project was considerably lower. For example, in the local authority of Mazara del Vallo e Salaparuta (about 50,000 inhabitants) in Sicily initially 40 citizens responded to the invitation of the local council but in the end only seven citizens took part in the project.

In each of the four regions interested citizens were then invited to a joint one-day seminar with representatives of the Department, the local council and Cittadinanzattiva. In the first half of the seminar the participants learnt about the overall purpose of the project, in the second half of the day they were trained on how to use the monitoring grid.

After the training, the citizens involved, together with the local representatives of Cittadinanzattiva, decided collectively that those areas should be monitored, which were seen as particularly important for the local authority (for example because they contained important public buildings, a train station and so on). The citizen monitoring then started, either involving the observation of specific aspects of public services or infrastructure or simply requesting public agencies to provide data which they already collected. For example, taking the example of the indicator “number of threats to safety on a chosen road”, this involved reporting on the monitoring grid the number of potholes, incidents of unsafe pavements, dangerous steps and abandoned scaffolding. Again, in respect of the indicator “availability of public urban transport from and to the monitored local area” (as part of the issue “connectivity”) citizens checked the availability of and ease of access to public transport at key local points, such as the hospital and the train station, based on their analysis of public transport timetables.

Once the participating citizens had filled out the monitoring grid (which had to be done within two months), they met together to agree the overall assessment and to discuss improvement actions, which was included in a report shared with the local authority.

Of course, feeding back the results from the citizen evaluators is particularly important. In the case of Lamezia Terme, the citizen evaluators and local representatives of Cittadinanzattiva got together to launch a new website, which serves both as a workspace for those involved in the monitoring project and an open space for promoting the initiative at local and regional level and networking with other active organisations. The local branch of Cittadinanzattiva Siracusa in Sicily has set up a profile on Facebook to keep in touch with all citizens involved in the monitoring and also all other interested stakeholders. The local authorities marketed the initiative in different ways, both using their own internet portals (e.g. Reggio Calabria, Lamezia Terme, Lecce) and giving public presentations (e.g. Putignano, Lecce, San Severo).

In order to co-assess the pilot with all the involved stakeholders and to develop the initiative further, the Department of Public Administration organised a project evaluation event on 5 July 2010 which was also attended by Governance International.

A joint evaluation of the project was carried out by all stakeholders involved in the final seminar on 5 July 2010. This showed that the project had achieved a number of its objectives. In particular, there was a general agreement that the assessment methodology was easy to use and flexible enough to adapt to local situations. The fact that the evaluation focussed on technical aspects of quality and not on the subjective perceptions of citizens was considered to be a strong point. 

Furthermore, it became obvious that the local services which had been evaluated were well chosen and were those most valued by citizens.  Local stakeholders less agreed about whether the local area covered by the evaluation was the right ‘geography’ to be evaluated. However, this emerged as another way of putting into practice the principle of “allowing citizens to decide", as local citizens could make up their minds where they think it is best to do the monitoring, given their own expertise and knowledge of the area. 

In particular, the project managers of the local councils felt that they had gained new knowledge in evaluation methodologies and citizen participation which they could use in other projects. As one testimonial put it:

“I learnt about new methodologies to improve neighbourhood management. The involvement of citizens has filled an important gap in our public service management”.

The citizens involved also thought that they learnt new things about the local area they lived in and their local council.

I do not want to be a citizen who always criticises the local council but does nothing to change things. However, I think that a lot of things are not working well in San Severo. Therefore, I have decided to become active and find evidence. Now I am waiting for the local councils to tell me what they are going to do about the evaluation which we have done”.

It is remarkable that 11 out of the 14 local authorities completed the pilot project from the "call to citizen participation" to the editing of the final reports. The reports include the public service evaluation with a specific score (from 0 to 4) for each quality dimension of the service concerned.

Five reports also include specific proposals for improvement. The majority of these suggestions concern mobility (transport and traffic), social well-being and waste management issues. For example, citizens proposed the creation of pedestrian areas, the improvement of road signs and increasing the number of garbage bins. However, moving from the citizens' proposals to effective actions proved to be difficult in most of the 11 pilots. Mainly due to budget contraints, only two local authorities implemented the sugestions of the citizens.

Another outcome was the level of citizen participation. About 150 citizens were involved altogether, with big differences within the various municipalities: one pilot engaged more than 50 people; in three cases 10-10 people were involved; and in seven pilots fewer than 10 people in each. Clearly, the issue of increasing the number of citizens involved in such evaluations is central.

Importantly, the project also resulted in a number of other benefits. For example, Lamezia Terme Council started a new project e-civit@s which aims at collecting online suggestons and complaints from local citizens. Moreover, both San Severo and Putignano Councils, together with the citizens and the assocations involved in the civic evalution project, began new forms of consultations on important quality of life issues in the local area. Last but not least, Salaparuta Council committed itself to organising some meetings with local schools in order to start a new civic evaluation - this time targeted at young people.

The project was followed by a final survey, aimed at both the citizens involved and the public managers of local authorities involved, to evaluate the experiences.

More specifically the survey aimed to assess

• the local organisation of the project (in terms of the activities, local communication, and role of the civic associations involved);
• the relationships between local citizens and public agencies; and 
• the general effectiveness of the project (both for public agencies and for citizens).

Both citizens and local authorities considered the contribution of civic associations as helpful to the management of the local activities during the experimentation. The public managers appreciated the participation of citizens not only as an opportunity of learning new way of managing public services but, moreover, as a way of developing social capital and a feeling of civic belonging.

Citizens also viewed positively the involvement of local authorities, particularly their provision of data already at their disposal, as well as their help in resolving some critical problems during the experiment.

With regard to the general effectiveness of the initiative, the local authorities, although underlining the shortage of time and resources for managing such activities, declared their continued interest in the project and their willingness to give feedback to citizens and to implement concrete service improvements arising from the monitoring exercises.

Citizens, too, saw the need to move from the evaluation phase to the implementation of improvements. They expected feed-back once the local report had been presented to the local authority. They also recognised that the participation in this project had increased their “civic sense” and their awareness of being able to make a concrete contribution to the improvement of their urban space. They also suggested that the project had generated a general positive effect on other citizens who, even though they hadn’t directly taken part in it, had known about it. Their interest in and curiosity about the project, supported by local media, were seen as providing a good opportunity for the further development of the initiative and for helping more citizens to become “active citizens”.

The costs of this pilot project mainly consists of costs of staff in the working group and the travel payments to participants in the training seminars at national and regional levels. The members of the national focus group which defined the quality dimensions and indicators received no fee, nor the citizens and representatives of the civic associations participating in the monitoring at local level. The local authorities which took part in the project did not receive any funding for the project. The time which volunteers invested in the project have been estimated as:

• 2,5 days in total for training, including the seminar at national level, the training at regional level and a half-day meeting at local level to organise the team work and to divide up the geographic area for the monitoring;
• the collection of data was undertaken by means 4 “public walks” of a duration of 2-3 hours each;
• the analysis of data required another meeting of about two hours to agree on scores and 2-3 half-days to complete the codebook and the drafting of the report;
• the presentation of results typically involved a press conference or a public meeting with the involved citizens.

As this project involved a partnership between public agencies, civic associations and service users, there has been a lot of learning for all involved stakeholders. Clearly, at times the communication between professionals and citizens was not easy, as their expectations were quite different. Moreover, in the current financial crisis the local authorities have not been able to deal with all the weaknesses identified by the citizen evaluators – they can’t “repair all the potholes” as one citizen demanded. One way forward could be to agree from the outset a charter between the local authority and the citizen evaluators, specifying the obligations of both parties. Most importantly, the citizens involved expect feedback from the local authority on what kind of actions will be taken as a result of the evaluation.

For the local authorities involved, an important issue is how to increase the number of citizens who involved in such evaluations. At the evaluation seminar it was suggested that a greater diversity of communication channels, including the internet, could be one way forward. Small incentives such as vouchers were also discussed, to get more disadvantaged citizens on board, or the use of a small grant scheme to incentivise citizens to make suggestions for improvements and to commit to helping the local council to implement the suggestions by contributing some of their time, expertise and other resources.

Another learning point was how to make the results of the evaluation sustainable and effective. In particular, representatives of local councils thought that it would be valuable to incorporate the evaluation in the strategic planning process of the local council, so that the citizens’ evaluation could help the council to determine strategic priorities.

Last but not least, all stakeholders agreed that the methodology could be used to evaluate other public services at neighbourhood level, provided that the right dimensions and indicators of quality are jointly defined.

Download the report of the participatory evaluation of the project in 2010 in Rome (in Italian): pdf download.