Miriam Sol & Carla Cordoncilo: evaluation system for the programme of active democracy
The system was designed after the logical framework approach. A matrix of indicators (simple and complex indicators) was created and then came the design of the sources of verification. Finally, the evaluation system was created.
Active democracy includes:
- Citizen initiatives.
- Participatory processes.
- Citizen consultations.
- Participation bodies.
In this project the focus was put on participatory processes.
Main dimensions: accessibility, diversity, plurality, traceability, transparency, operations.
These aspects should not be measured outside of their context, as most of them are very sensitive to it. Thus, quality or achievement of specific thresholds in indicators should be measured in relationship with environmental values. E.g. diversity in participation has different meanings in neighbourhoods that have a multicultural social tissue or in neighbourhoods that are socially or culturally more homogeneous. Less diversity in the latter is to be expected, while low diversity in the former should be considered as a failure.
Decidim.index: indices for the democratic quality of online participation (2018)
Sofia de Roa: Systems of indicators of quality
The point of departure to design a system of indicators for democratic quality is transparency.
Two types of organiations in relationship to transparency:
- What for transparency: to disclose information from the past.
- Why transparency: external motivation: to look nice in transparency indices and gain recognition.
- What do we make transparent: operative aspects related to production, such as people, economy, structure and processes.
- How do we become transparent: on their own, with their own tools.
- What for transparency: to disclose commitments and measure improvements.
- Why transparency: intrinsic motivation: responsibility, work well done, the common good.
- What do we make transparent: all the value chain, including vision and mission, values, strategies, etc.
- How do we become transparent: in a participatory way, with all stakeholders.
How to operationalize concepts such as diversity, democratic quality, gender balance, social autonomy, etc.?
Systems of quality indicators: choose, improve, etc.
Decidim.index: indices for the democratic quality of online participation (2018)
Case Studies 3
Anna Przybylska | University of Warsaw (Poland)
ICT solutions for public consultations: Methodology and design of inDialogue
Abstract: The aim of the presentation is to reflect on the design of the inDialogue software that has been developed to intervene in the organization of public consultation processes in local governments. The design has been informed by the results of empirical studies. In those studies, we evaluated the practice of public consultations in Poland refereeing to the norms constitutive for the model of deliberative consultations. The inDialogue software is expected to respond to the problems revealed during the evaluation. It helps to convey the knowledge about the methodology of public consultations and supports the teamwork for their better organization in the city hall. It facilitates planning of public consultations which can be conducted through face-to-face meetings and paper questionnaires as well as through online text or voice meetings and electronic questionnaires. The presentation starts with the overview of some theoretical assumptions and associated research findings relevant to the institutionalization of deliberation in public consultations. Following part analyses empirical data collected from the Polish local governments. In this background I will discuss tools and procedures of inDialogue software.
Three areas of tension:
- When the institutions of representative and participatory democracy are being integrated.
- Between the ideal of deliberation and the results of its implementation.
- When attempting to create a consolidated venue for public dialogue in a world of dispersed communication channels.
How are we going to attract people to use these tools? Efficacy of participation is the most powerful incentive.
inDialogue is a participation software that has many functions. Not only does it deal with participation, but also planning, open government, etc. The software also features different roles/approaches, like the clerk’s interface with several actions that the leader of a participation initiative can undertake. Same for citizens, that have their own interface and the tasks that they can perform.
Factors for the absorbtion of innovation:
- Establishing partnerships: quadruple-helix model where several institutions have a different complementary role.
- Research and action.
- Evaluation and software amendments
- An umbrella or a network?
- Public sphere and scaling-up.
- Distribution roles.
José L Martí | Pompeu Fabra University Barcelona (Spain)
Crowdlaw and the internal/external dimension of online local participation
Abstract: One of the new paradigms that has been advocated to understand the new possibilities of local participation enhanced by the new technologies is the so-called crowdlaw, as a particular subtype of Open Government. Under this approach, ordinary citizens can be deeply involved in different stages of the legal cycle and through a variety of forms of participation. They can participate in information pooling, in deliberation, or in decision-making properly. And they can contribute in such a variety of forms to stages like public diagnostic, law and policy-making, law and policy enforcement, law and policy adjudication, law and policy control, and law and policy revision. This is seen by some as one of the most important innovations to come in the next years to improve government at different levels, and also at the local one. But one of the effects of adopting this new approach is that the boundaries between internal and external participation (the participation of local citizens or the participation of citizens from other towns, regions or states, is importantly blurred. In other words, crowdlaw is very good in enhancing both the internal and the external dimensions of local participation (i.e., citizens from other places, including other states may be involved in different ways in the local participation of our city and contribute largely to it. This may have crucial effects to the way we conceive local politics. This paper explores all these effects and implications, focusing particularly in the way in which public local participation should be conceived in this new scenario, and advances a new vision of how local politics, and particularly public local deliberation may scale up to extralocal (potentially global) politics.
The demos problem: the traditional response to the question on whether one should be able to participate in a participatory process or a decision-making process in one’s own city/region/state is that yes, one should be able to participate. What if one is not formally recognized as a citizen in a given city? What if I have interests (relatives, friends, etc.) in other cities? Are they “my” city too?
e-Democracy is transforming the traditional ways to approach such demos problem in a way that brings us necessarily to connect local democracy with global democracy. e-Democracy is deterritorializing politics, which were, almost by definition, always bound to a territory.
Why participatory democracy?
- It empowers people.
- It strengthens full inclusion.
- It improves the quality of decision-making.
But we must have an idea of who should be empowered, whose voices should be heard, what options should be put on the table.
- Territorially-defined demos: which refers to a formal status, which in turn is based on residence.
- Functionally-defined demos: depending on the substantive issue.
And there even is yet another principle: the all-affected principle: all those who are potentially affected by the decision should be.
The digital revolution is making the territory less and less important which, combined with globalization, makes the demos problem one of the most important now in participation. But this is where deliberation —not voting— gains a lot of meaning.
Evaluation of Online-Participation
Norbert Kersting | Universität Münster (Germany)
Monitoring and Evaluation of E-Participation
Abstract: Monitoring and evaluation Instruments of are meant to enhance the quality of policy implementation. It is obvious that in numerous cases this monitoring and evaluation of online and office participation does not exist or is not applied by external actors. In the participatory instruments of the invented space, monitoring and evaluation is often ignored, there is no time or there is no funding to implement it thoroughly. The paper refers to the long history of participatory research. It shows that there are numerous participatory methods, but only a few concepts of evaluation. It criticizes theoretical concepts leading to indicators such as the Arnstein ladder of Participation, political action studies, civic engagement and the theoretical and historical blindness of newer instruments. Finally, it argues that categories and concepts do not differ in research on online and offline participation-but the theoretical foundations of political participation do.
How do we assess online participation? Is it possible to assess it with the same tools that are used to assess offline/traditional participation?
Acknowledged crisis of representative democracy: lack of responsiveness and accountability, post-parliamentarism, post-democracy, against elections, against democracy…
Jason Brennan states that we have trolls (they do not like anything, they are hooligans), hobbits (they actually do not care) and all the people in between, most of them cynics.
In many countries in Europe there have been local government reforms in Europe, some of them including more participatory processes like direct democracy at the local level.
Participatory instruments. Evaluation 1. Criteria:
- Participation: openness and equality
- Control, responsiveness.
Participatory instruments. Evaluation 2. Purposes
- Brainstorming: sharing knowledge and ideas, capacity-building
- Planning: problem-solving, innovation, strategy or action plan, decision-making.
- Networking: building relationship, personal/leader development.
- Conflict resolution: dealing with conflict, generating awareness, sharing vision.
The formal part is also important: can we compare voting with demonstrations? Should we? With what instruments?
Q: what could be done to do more and better evaluations of participatory processes? Kersting: benchmark good cases, have processes accepted in as many governments as possible, create standards, etc.
Ismael Peña-López: maybe, from a rational-choice approach it is true that “politicians do not assess” participation. But from a post-marxist approach, taking into account the theories from Hannah Arendt or Antonio Gramsci, yes politicians plan participatory processes but not for the reasons to achieve “real impact” but to control the relate and a way of assessing it would just simply be winning the elections, or placing a specific topic on the public agenda and being hegemonic in this discourse.
Maria A. Wimmer | Universität Koblenz
Evaluation of e-Participation Initiatives
There are a number of evaluation frameworks, with similarities and differences.
The MOMENTUM evaluation approach has:
- What to evaluate. Assets to be assessed: tools, processes, topics, policies.
- How to evaluate. Evaluation criteria: usability; appropriateness, interest, policies met).
- Main target of evaluation and impact towards target groups.
- Efficiency: system quality, information quality, service quality.
- Efficacy: information, communication, decision, expectations.
- Effectiveness: what the current situation is and what the future situation looks like to be.
Online and Offline Participation
Herbert Kubicek | University of Bremen (Germany)
How to combine online and offline forms of participation?
Abstract: Expectations have been high that offering online, i.e. electronic or “e‑“communication channels in public participation will improve its outreach and quality. However, so far there is no empirical evidence that confirm these hopes. Applying a variety of research methods thea European Cooperation Project e2democracy presents empirical findings on the advantages and disadvantages of online communication compared to face-to-face communication in six consultation processes and seven collaborative citizen panels. To control for contextual differences, one of the consultation processes has been set up paralleling online and face-to-face meetings. In this case organizers showed a preference for face-to-face meetings as regards the content of contributions and the style of discussion. For the citizen panels collaborating with local governments to achieve climate targets, impacts in terms of CO2e savings and dropout rates have been compared for parallel processes online and via telephone. These comparisons, however, do not deliver clear performance profiles of the communication channels or a generalizable assessment of their appropriateness for particular objectives. The factors influencing the choice of communication channels are complex and the analysis shows that assessments depend on the type of participation and the role of an actor in the process as well as on time frames and contexts in which the assessments are made. Showing that none of the channels offers clear advantages over the other, we conclude that practitioners are well advised to follow a multi-channel strategy and offer a media mix of online and traditional modes of participation.
ECRP project: comparative assessment of e-participation in the context of sustainable development and climate change.
When we evaluate offline vs. online participation, we have to look at outputs, outcomes and impacts.
To be able to do this, a quasi-experimental project was created so to be able to compare participation processes with the same subject and target audience online and offline.
A public consultation was initiated on SDP’s programme in the state parliament election campaign in Bremen.
People said that online takes less effort, but that the quality of deliberation is better offline and that local meetings create better community building. Regarding to content analysis and participation observation, we see that local meetings promote more reasoned arguments and that online forums are more biased towards expression of opinion.
No big differences between civility (trolling) or innovativeness (new ideas) between local meetings and online forums. Online consultation did not attract significantly more voters, but it did provide higher legitimation, as there was full approval of the programme on the assembly without a single dissenting vote for the first time.
When comparing the 7 cities of the project on CO2 savings, and how they compare in their results, there is no clear conclusion. There is no evidence for the assumed advantages of online participation. But there is no evidence either on why in some cases online seems to be better or even much better than offline. What seems clear is that the combination of offline and online participation seems to be, so far, the best bet.
Paolo Spada: how did you communicate/invite the citizens to register to the online platform? Kubicek & Royo: there were several channels used to invite citizens to participate, offline, telephone and online.
Ismael Peña-López: how did you facilitate the deliberation? how was the platform designed for such deliberation? Or was only a simple online forum? Kubicek: there was no analysis on how facilitation was similar or different in offline and onine platforms.
Sonia Royo | University of Zaragoza (Spain)
How to Keep Citizens Engaged? Advantages and Disadvantages of Online and Offline Citizen Participation
Abstract: The objective is to help governments foster citizen participation. Therefore, it addresses the following issues: How can citizens be motivated to participate? What can be done to reduce abandonment rates? Are there any differences between offline and online participation regarding enrolment and abandonment? In order to answer these questions and provide policy recommendations, the authors rely on two case studies of Spanish cities allowing both online and offline participation.
Objectives: to determine why and when do citizens abandon citizen participation projects that require long-term collaboration between citizens and administration; and to ascertain whether differences exist between online and offline citizen participation projects (especially in enrolment and drop-out rates.
Internet facilitates weak ties and contributes to maintain strong ties. But is also true that network participants are more individualistic and shift their attention more quickly than offline.
An initiative to reduce one’s own CO2 emissions was deployed in Zaragoza and Pamplona (two Spanish cities). People enrolled online or offline. Online participants were younger and had much higher educational levels. People enrolled in the project and more of them said they would participate online. At the end, there were more people participating offline rather than online: there were much more dropouts online rather than offline.
In this case, the design of the participatory process was very simple and there does not seem to be a reason in differences of facilitation between the online and the offline versions.
Most of the people offline were retired people, and would not dropout because of time reasons; on the other hand, more than half of the online participants that had dropped-out stated that it was due to lack of time.
Most people will abandon in the very beginning of the initiative and will do that for the amount of time required to participate. There are other reasons for drop out, but they seem [personal opinion here] to be very related to the devotion of time: engaging in interaction, reading complementary resources, etc.
Ismael Peña-López: it seems to me that the problem in dropping-out was not that people prefer offline to online, but the very different profile of he participants: elderly retired people offline, young working people online. When asked for the reasons for dropping out, more than 50% of the later stated that they had issues with the time they had to devote to the project; only 2% of the offline participants said it was because of time.
Actors in the Field of Online Participation
Mary K. Feeney | Arizona State University (USA)
What does e-participation mean for managers in small to medium sized cities? US trends and research challenges
Abstract: Since 2000, our team at the Center for Science, Technology and Environmental Policy Studies at ASU has been collecting data on 500 small and medium sized U.S. cities to understand the adoption and management of technology in local government. Drawing from four surveys (2010, 2012, 2014, 2016), website content data (2010 & 2014), and Twitter data (2017), I present some technology adoption trends in small to medium sized US cities and managerial responses to those efforts. These data provide insights into the organizational, technological, and socio-technical factors that shape local government online participation efforts in the American context. I then outline the challenges facing managers who seek to engage internal and external stakeholders via online mechanisms – including financial limitations, technical capacity, legal issues, and perceptual barriers – and discuss next steps for advancing research in this area.
The research surveyed during 7 years the most common departments in small cities (700-900 small cities surveyed) in the US: community development, finance, mayor’s office, parks & recreation, police. There are consistent predictors of ICT usage and participation: city population, form of government, department type, technical capacity, resources, management.
What is the role of management of participation? Manager views affect e-participation.
Small cities are increasingly using technology, and using it better. But most of the times it is because they externalized the website to a firm that is using a market solution that has all the features one would expect.
Municipalities are entering the social media arena. But they are having quite hard times. Sometimes these tools clash with the laws and norms, and have to be used with wisdom.
Thus why management of ICTs, for communication and participation, is crucial. Most municipalities do not have the resources to run these tools, but they do have the pressure to adopt all of them.
Karen Mossberger | Arizona State University (USA)
Emerging Platforms for Online Engagement in US Local Governments – Who is innovating and how?
Abstract: While the use of social media is widespread in local government, more structured forms of online participation are also beginning to appear at the local level, using a variety of commercial platforms that have become available in recent years. Examples include Peak Democracy’s cloud-based platform for online town hall meetings, MindMixer’s community engagement platform, Budget Allocator’s participatory budgeting software, and Balancing Act’s online budget simulator designed to encourage citizen participation. A recent survey of cities and counties in the US revealed that local governments who use such tools are still very much in the minority, as only 17% of respondents reported using these platforms. Still, such tools are becoming more prevalent, compared with earlier studies of online engagement in larger (and generally more innovative) local governments (Mossberger and Wu 2012); and a closer look may help to predict how such platforms will affect citizen engagement in the future. Using a 2016 national survey of Innovations and Emerging Practices in local government that was conducted by the International City/County Management Association and Arizona State University, we explore the use of online engagement platforms, in comparison with social media and with a number of offline forms of engagement. What characteristics predict use of such platforms, in terms of city size, demographics, metropolitan status, fiscal capacity and form of government, among other factors? Are such cities more likely to report use of many forms of public engagement, to be early adopters for other emerging practices, or both? What are their goals for citizen engagement? And, how successful do they feel the experience was? This paper will consist primarily of analysis of the survey data, but will also propose a design for further qualitative research. Several cities in the Phoenix metropolitan area have used these platforms, and Arizona State University is also part of a national partnership with local governments called the Alliance for Innovation. Based on findings from the survey data, further research will be proposed to explore the types of questions cities have addressed through these platforms, how they conducted outreach and participation, and how effective they were in terms of representativeness and deliberation, among other critera.
How are local governments in the US using new online tools? Large cities are innovators, and in general social media has grown rapidly. It usually has to do with city managers wanting to experiment with whatever comes new, see if it works and what for.
Importance of public participation goals: provide the public with objective information, obtain feedback, work directly with the public, partner with the public, hear input/ideas from the public, place dedcision-making in the hands of the public.
Usage of citizen engagement tools: town hall meetings, city-appointed committee assignments, social media.
What predicts use of emerging online platforms? The size of the population is very important. It is also important pre-existing offline engagement actitivities, even more important than social media use.