Theory to Practice
John M. Bryson is the McKnight Presidential Professor of Planning and Public Affairs in the Hubert H. Humphrey School of Public Affairs at the University of Minnesota. His interests are in strategic management, leadership, and collaboration. He is author of Strategic Planning for Public and Nonprofi t Organizations (4th ed., 2011) and coauthor, with Barbara C. Crosby, of Leadership for the Common Good: Tackling Public Problems in a Shared-Power World (2nd ed., 2005).E-mail: bryso001@umn.edu
Kathryn S. Quick is assistant professor in the Humphrey School of Public Affairs at the University of Minnesota. Formerly a public and nonprofi t manager, she now focuses her research and teaching on civic engagement, integrative leadership, and public and nonprofi t management. Through ethnographic research, she studies a diver-sity of approaches to public engagement across a wide range of policy and planning issues and their consequences for decision outcomes, implementation, and community capacity building.E-mail: ksquick@umn.edu
Carissa Schively Slotterback is asso-ciate professor in the Urban and Regional Planning Program in the Humphrey School of Public Affairs at the University of Minnesota. Her research is focused on stakeholder involvement and collaborative decision making in environmental, land-use, and transportation planning processes. She holds a doctorate in urban and regional planning from Florida State University and has experience in planning practice in the public and private sectors.E-mail: schiv005@umn.edu
Designing Public Participation Processes 23
Public Administration Review,
Vol. 73, Iss. 1, pp. 23–34. © 2012 by
The American Society for Public Administration.
DOI: 10.111/j.1540-6210.2012.02678.x.
Barbara C. Crosby is associate professor in the Humphrey School of Public Affairs at the University of Minnesota. She has taught and written extensively about leadership and public policy, cross-sector collaboration, women in leadership, media and public policy, and strategic planning. She is author of Leadership for Global Citizenship (1999) and coauthor, with John M. Bryson, of Leadership for the Common Good: Tackling Public Problems in a Shared-Power World (2nd ed., 2005).E-mail: crosb002@umn.edu
Donald P. Moynihan, Editor
John M. BrysonKathryn S. Quick
Carissa Schively SlotterbackBarbara C. Crosby
University of Minnesota
Th e purpose of this Th eory to Practice article is to present a systematic, cross-disciplinary, and accessible synthesis of relevant research and to off er explicit evidence-based design guidelines to help practitioners design better participation processes. From the research literature, the authors glean suggestions for iteratively creating, managing, and evaluating public participation activities. Th e article takes an evidence-based and design science approach, suggesting that eff ective public participation processes are grounded in analyzing the context closely, identifying the purposes of the participation eff ort, and iteratively designing and redesigning the process accordingly.
Government administrators, offi cials, and community leaders have long recognized the value of public
participation for a variety of purposes, processes, and deci-sions (Cooper, Bryer, and Meek 2006; Rosener 1975; Yang and Pandey 2011). However, they frequently do not have a good understanding of how to design participation processes to achieve desirable outcomes. Fortunately, there is an extensive base of research from which to glean advice on participation process design that off ers clear, though nuanced, direction for practice. Th e purpose of this article is to present a systematic, cross-disciplin-ary, and accessible synthesis of relevant research and to off er explicit evidence-based design guidelines to help practitioners design better participation processes.
To integrate relevant research into a set of practical participation process design guidelines, we tap into insights from the evidence-based practice movement and from the developing design science literature. Th e evidence-based practice movement originated in medicine, where it refers to “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett et al. 1996) and is accomplished by
integrating research evidence, professional expertise, and the preferences and concerns of patients (Bauer 2007). It now cuts across many disciplines, includ-ing public administration (Heinrich 2007), urban planning (Forsyth 2007), and public policy making and service delivery (Boaz et al. 2008). Public aff airs professionals face a number of challenges in enacting evidence-based practice, however, including gaining access to, knowing how to read and critique, and hav-ing time to review applicable research; discerning its relevance to organizational contexts; and dealing with confl icting fi ndings across studies (Krizek, Forsyth, and Slotterback 2009). In addition, many practition-ers rely on professional colleagues for information and also are faced with an overload of information avail-able online (Durning et al. 2010).
We respond to these challenges by off ering a systematic and accessible review of the evidence related to the design of public participation processes. Th e rel-evant research is diff used across a diverse range of disciplines, research methodologies, con-
texts, and subject areas. We reviewed more than 250 articles and books. Th ough many of the materials were not written specifi cally to answer the question of how best to design public participation, they off er incisive lessons for designing participation. While each of us is an academic, each also has substantial experience with the design of participation processes; we drew on each type of experience as we met regularly over the course of many months to discuss the research and develop a concise set of practical guidelines. We are selective with citations here, balancing an eff ort to point readers to the primary references (e.g., the earliest written, most frequently cited, most comprehensive, or most generalizable sources) with a representation of the breadth of disciplines and practice fi elds of relevance to public administration. Readers will note divergences in the types of support among the recommendations that we make. Some have emerged through extensive study;
Designing Public Participation Processes
We respond to these challenges by off ering a systematic and
accessible review of the evidence related to the design of public
participation processes.
24 Public Administration Review • January | February 2013
science involves utilizing a combination of existing research and well- understood practice to develop solution-oriented design princi-ples that are contingent (Simon 1996). Th at is, these principles are situation and context specifi c, which means that they are neces-sarily somewhat general, require thoughtful adaptation to specifi c situations (Boyne and Walker 2010), and typically are readapted in response to emerging conditions (Cross 2011).
Th e design science focus of this article signifi es a shift from typical social science, which generally involves hypothesis testing to estab-lish general patterns of causality, to the design science framework of developing and testing conjectures, taking a problem-solving approach, and adapting research-based evidence to context-specifi c, contingent, and emergent circumstances. Despite a growing desire to innovate in the public sector, there is limited recognition among public administrators of design science as a viable way to promote adaptation and change, perhaps because of the sector’s risk-adverse culture and structural barriers (Bason 2010; Cowan 2012; Moore and Hartley 2008).
It is very important, however, not to overemphasize the diff erence between design science and what probably has always been true of the best public administration in practice. Th e diff erence is simply the heightened emphasis on purpose-driven, context-sensitive, holistic, user- and stakeholder-oriented, evidence-based designing and designs. Bringing a design science framework into the work of public administrators is meant to invite practitioners to reconceive their activities explicitly in terms of an ongoing, active process of designing (a verb), which is typically iterative and involves testing various ideas and prototypes before settling on the “fi nal” design (a noun) (Romme and Endenburg 2006). Practice is thus seen as a response to explicit or implicit designs—and based on practice, those designs very well may need to change (Wenger 1999).
Th us, we fi nd it neither feasible nor advisable to generate “rules” or a step-by-step design template for organizing public participation. Indeed, a consistent implication of design science and of the diver-sity of evidence-based research fi ndings synthesized here is that suc-cessful public participation requires designing iteratively, in response to specifi c purposes and contexts. As Nabatchi notes, “design choices are not made in a linear fashion” (2012b, 3). Th erefore, we synthesize the evidence into design guidelines that we invite public administrators to consider to help them accomplish the goals of their public participation processes.
The Design GuidelinesDesign to Address Contexts and ProblemsDesign guideline 1. Ensure that a public participation process is needed, fi ts the general and specifi c context, and is based on a clear understanding of the challenge or problem (a part of the specifi c context) for which public participation is a desirable part of the response.
Participation processes must fi t the context in which they are taking place. Th e policy change literature emphasizes the importance of understanding the general and more specifi c contexts and staying alert to changes in them (e.g., Crosby and Bryson 2005; Gaus
others, though supported by less empirical evidence, are well sup-ported by logic and theory. Our intent was to glean and synthesize the most relevant guidance for practitioners from this research.
At the same time, it is important for practitioners to couple this evidence with their own expertise and knowledge of their specifi c contexts, drawing on research “while also incorporating the tried and true, or locally relevant, methods for specifi c environments and contexts” (Krizek, Forsyth, and Slotterback 2009, 470). Th erefore, while research literature off ers a wealth of practical guidance to enhance the outcomes of public participation processes through bet-ter design, we synthesize it here in the form of design guidelines for creating, managing, and evaluating public participation activities in order to accomplish desired outcomes (see table 1 and fi gure 1).
Our synthesis is informed by insights from the developing design science literature, which emphasizes a focus on desired outcomes to be achieved in a problematic real-world situ-ation, strong client and holistic orientations, deliberate use of evidence-based substantive and procedural knowledge, and understand-ing and specifying responses that fi t the particular context (Van Aken 2007). Design
Figure 1 The Cycle of Public Participation Process Design and Redesign (Numbers indicate the corresponding design guidelines.)
Table 1 Design Guidelines for Public Participation
Assess and design for context and purpose
1. Assess and fi t the design to the context and the problem 2. Identify purposes and design to achieve them
Enlist resources and manage the participation
3. Analyze and appropriately involve stakeholders 4. Work with stakeholders to establish the legitimacy of the process 5. Foster effective leadership 6. Seek resources for and through participation 7. Create appropriate rules and structures to guide the process 8. Use inclusive processes to engage diversity productively 9. Manage power dynamics 10. Use technologies of various kinds to achieve participation purposes
Evaluate and redesign continuously
11. Develop and use evaluation measures 12. Design and redesign
Note: These are interrelated, iterative tasks, not a step-by-step template.
Participation processes must fi t the context in which they are
taking place.
Designing Public Participation Processes 25
design of solutions) with solutions in order to gain clarity about pur-poses and desired outcomes (Brown 2009; Van Aken 2007).
Diff erent kinds of problems or challenges call for diff erent solution responses. Designs for participation must be tailored to assist with developing those responses. For example, problems that are pri-marily technical or operational are not likely to call for substantial changes in the applicable knowledge or technology base, stakeholder relationships, broad organizational strategies, or governance mecha-nisms. Line managers, operations groups or teams, and program, product, project, or service coproducers or recipients are the most likely people who will need to be involved. In contrast, problems that are more complex and politically charged are likely to require changes in the applicable knowledge or technology base, new concepts, and changes in basic stakeholders or stakeholder relation-ships. Governing or policy boards, senior staff , and an array of key
1947, 6–19; Sabatier 2007). Th e general context includes broad social, demographic, political, technological, physical, and other features and trends in an organization’s environment. Th e specifi c context refers to those parts of the organization’s task environment that are directly relevant to the achievement of the organization’s goals, including key stakeholders, applicable mandates, resource availability, and so on (see Scott and Davis 2006).
Th e decision-making and design science literatures emphasize the importance of understanding the problem or challenge to be addressed in such a way that it can be solved, the wrong problem is not solved, and solutions do not create the problem that they were meant to solve (Wildavsky 1979). Th ey also assert that possible solutions will need to be explored before the “real” problem is understood (Janis 1989; Nutt 2002). Doing so calls for iteratively juxtaposing possible defi nitions of “the problem” (that is, that part of the context that motivates the
Table 2 Multiple Purposes of Public Participation, with Associated Design Considerations and Proposed Outcome Evaluation Criteria
Purposes Design Considerations Proposed Outcome Evaluation Criteria
Meet legal requirements—for example, to provide public notices of upcoming ac-tions or in preliminary scoping efforts for environmental impact assessments (Brody, Godschalk, and Burby 2003; Slotterback 2008)
• Clarify legal requirements• Observe sunshine laws• Consider alternatives to traditional public notices and meetings—
for example, use of social media and online comment boards may be effective and effi cient ways to fulfi ll these requirements.
• Legal requirements for public noticing and comment met
• Effi cient cost of communication and outreach
Embody the ideals of democratic par-ticipation and inclusion—for exam-ple, to achieve or represent the public interest through diverse participation, provide an opportunity for participants to enhance their own capacities to engage in democratic citizenship, or produce lasting achievements of public value (Mansbridge 1999; Young 2000; Fung and Wright 2003; Nabatchi 2010)
• Perform stakeholder analysis and design the process to encour-age active participation by those with interests at stake, making particular efforts to be inclusive
• Act in response to participants’ contributions, encouraging diverse views and refl ecting them in outcomes
• Deliberative approaches can help participants develop capacity and commitment for ongoing contributions
• Inclusiveness of composition of participants• Discernible, communicated impact of partici-
pation on outcomes• Positive effects on citizenship (e.g., partici-
pants’ increased understanding of how to participate in democratic processes, greater commitment to do so, or elevated sense of effi cacy in ability to affect decision making)
Advance social justice—for example, by improving equity in distributing public services or by increasing a marginalized group’s infl uence over decisions (Abers 2000; Andrews, Cowell, and Downe 2010; Corburn 2003)
• Perform stakeholder analysis and recruit diverse stakeholders• Enable diverse participation (i.e., by enabling multiple ways to par-
ticipate, providing language translation or child care, and selecting accessible meeting locations and times)
• Consider the distribution of benefi ts and harms
• Adequacy and diversity of stakeholder repre-sentation
• Improved distribution of benefi ts and harms ensuing from the decisions
Inform the public—for example, about decisions that have been made or about changes in policies, resources, or programs (Nabatchi 2012b)
• Informing the public and maintaining transparency about deci-sions may be suffi cient
• Large number of people reached or the target population reached
• Diversity of modes or venues used to inform public
• Increased public awareness of targeted policy issues
• Public satisfi ed they have been informedEnhance understanding of public prob-
lems, and explore and generate po-tential solutions (Deyle and Slotterback 2009; Godschalk and Stiftel 1981; Webler et al. 1995)
• Deliberative approaches and small-group formats can help partici-pants understand issues and contribute to problem solving
• Design processes for sharing information and engaging and exchanging views among participants to promote understanding and discovery of new options; help participants learn about each other’s perspectives, the broader context, and possibly change their views; present information in various formats and from a variety of sources (Daniels and Walker 1996; Webler et al. 1995)
• Balance technical expertise and broader stakeholder representa-tion (Innes and Booher 2010)
• Changes in individual or collective assump-tions, frameworks, or preferences
• Changes in participants’ knowledge of issues, ability to articulate interests, and appreciation of other perspective
• Generation of new problem defi nitions and potential solutions
Produce policies, plans, and projects of higher quality in terms of their content
• Use deliberative, collaborative approaches to promote learning (Forester 1999; Healey 1997; Innes and Booher 2010)
• Shift decision making to an appropriate scale (e.g., regional, local) to take advantage of relevant knowledge and investment in out-comes (Koontz and Thomas 2006; Mandarano 2008; Margerum 2011)
• If the problem is complex and technical quality is necessary, engage in boundary work among different ways of knowing (Feldman et al. 2006), or limit participation to content experts or give special emphasis to their role (Thomas 1995)
• Validation of the quality of decisions by in-formed content experts, using context-specifi c criteria related to, for example, economic effi ciency, safety, reliability, feasibility, equity, environmental impact, etc.
26 Public Administration Review • January | February 2013
purposes to be served by the process. Trite though it may sound, asking “What are the purposes of this participation process?” is a step that is overlooked remarkably often in practice—frequently with unfortunate results (Janis 1989; Nutt 2002). Multiple purposes may be served by a single process, and purposes may change as the public participation process unfolds. Th e literature identifi es mul-tiple possible purposes of participation. Th ese are summarized in table 2, along with associated considerations when designing proc-esses to achieve these purposes and suggested outcome evaluation criteria. Particular participation processes are likely to pursue some subset of these purposes. Evaluation criteria are discussed further under design guideline 11 but are previewed here because articulat-ing purposes and evaluation criteria simultaneously can assist public managers when making design choices with their desired ends in mind.
Clarity about the purpose of the participation process can help avoid unnecessary or unwise expenditures of eff ort and resources, problems measuring the outcomes of the eff ort, or challenges to the legitimacy of a participation process because confl icting ideas about its purpose have not been resolved. For example, sometimes a public participation process is legally required, even though legisla-tive, budgetary, scheduling, or technical parameters of the decision sharply confi ne the range of choices that are available to be made in conjunction with the public. Under these circumstances, if the
stakeholders may need to be involved in coming up with eff ective responses to such problems (Heifetz, Grashow, and Linsky 2009; Hill and Hupe 2009; Nutt 2002; Th omas 2012).
First, however, it is important to look at the larger picture of whether participation is needed or possible. Sometimes it is man-dated (Brody, Godschalk, and Burby 2003; Slotterback 2008); sometimes it is not mandated and is more bottom-up in nature (Boyte 2005); sometimes it is a combination of the two. In general, participation should be sought when it is required or when it is the only or most effi cacious way of gaining one or more of the follow-ing: needed information, political support, legitimacy, or citizen-ship development (Th omas 2012). In more ambiguous settings, Stone and Sandfort’s (2009) policy fi eld mapping approach is a good starting point for identifying key features relating to authority, stakeholder relationships, and resource fl ows that must be taken into account when deciding whether a participation process is merited.
Identify Purposes and Design to Achieve ThemDesign guideline 2. Clarify and regularly revisit the purposes and desired outcomes of the participation process and design and redesign it accordingly.
Fitting the participation process to its context and attaining desired outcomes involve gaining as much clarity as possible about the
Table 2 Continued
Purposes Design Considerations Proposed Outcome Evaluation Criteria
Generate support for decisions and their implementation—for example, by producing decisions that address the public’s needs and concerns; resolving dis-putes; creating alliances for advocacy and implementation; and generating resources for implementation (Brody, Godschalk, and Burby 2003; Godschalk and Stiftel 1981; Laurian and Shaw 2009; Moynihan 2003;Wondolleck and Yaffee 2000)
• Avoid making decisions so that stakeholders feel left out, for example, by making them narrowly or hastily or by delegating decision making to small, elite, or exclusive groups (Feldman and Quick 2009; Nutt 2002; Thomas 1995)
• Emphasize procedural fairness to enhance acceptance of decisions even among those with a different preferred outcome (Schively 2007); encourage broad participation, especially of key stakehold-ers; engage in shared knowledge generation and relational work to foster joint ownership of the problem analysis and outcomes (Innes and Booher 2010; Wondolleck and Yaffee 2000)
• Utilize confl ict management and negotiation techniques (Fisher, Ury, and Patton 2011), including consensus-oriented approaches that aim for win–win solutions (Forester 1999; Innes and Booher 1999; Margerum 2002)
• Participants satisfi ed with the process • High level of agreement with fairness of deci-
sion process• High level of agreement with decision out-
comes, possibly consensus• Minimal lawsuits, confl icts, delays, mistakes,
or other obstacles to implementing decisions• Resources available for implementation
Manage uncertainty—for example, to build trust, increase the quality of information informing decisions, stabilize relationships, and minimize risk from unanticipated changes in the external environment (Friend and Hickling 2005; Rowe and Frewer 2004; Wondolleck and Yaffee 2000)
• Acknowledge where uncertainty exists• Maximize participation and encourage information sharing to
provide clarity about the external environment and values• Build relationships to reduce uncertainty in them and provide a
holding frame for negotiating over differences and resources
• Persistence of a structure or relationships for ongoing learning and negotiation
• Limited number of problems caused by mis-interpretation of or unanticipated changes in values, relationships, or information
• Reduced confl ict among stakeholders• Trust in decision makers or decision-making
process
Create and sustain adaptive capacity for ongoing problem solving and resilience—for example, by emphasiz-ing social and transformative learning; relationships, social capital, and trust; and sustained engagement (Forester 1999; Goldstein 2012; Innes and Booher 1999, 2010; Jordan, Bawden, and Bergmann 2008; Webler et al. 1995)
• Deliberative, consensus-based, or collaborative approaches frequently facilitate transformative learning; include diverse perspectives to optimize learning and involve key stakeholders; support developing shared meaning via interacting and learning about each other’s interests, preferences, values, and worldviews through “collaborative science” (Mandarano 2008)
• Build social capital among participants for ongoing work by build-ing connections, enhancing relationships, and fostering trust that can carry on beyond a single decision-making process into future collaboration and communication (Innes and Booher 1999; Quick and Feldman 2011)
• Creation of new structures (relationships, partnerships, and resources) to support broad participation in ongoing planning, implemen-tation, and evaluation
• Sustained, diverse participation in manage-ment that adapts to changed circumstances
• Use of collaboratively agreed criteria for deci-sion making or performance management
• Sustained collective ability to address new problems and support ongoing management (e.g., of program, resources, problem)
• Improved alignment of participants’ expecta-tions and actions with collective understand-ings and goals
Note: See also design guideline 2 on designing for purpose and guideline 11 on evaluating participation.
Designing Public Participation Processes 27
have a particularly important contribution to make to learning about the problem, whereas legislators or potential opponents are particu-larly important to involve if the purpose is to secure broad buy-in to a proposed solution. Similarly, the same stakeholder might be engaged in diff erent ways over the course of a participation process as it unfolds. Th e stakeholder may only wish to be informed when a policy problem is fi rst being identifi ed, be a collaborator in selecting among policy options, and return to simply being informed about the policy implementation. Th erefore, we suggest that practitioners utilize a tool such as table 2 to iteratively articulate purposes and consider how to fulfi ll them over the course of the decision-making process.
Establish the Legitimacy of the ProcessDesign guideline 4. Establish with both internal and external stakeholders the legitimacy of the process as a form of engagement and a source of trusted interaction among participants.
An organization that seeks to acquire the support necessary for sur-vival and mission accomplishment must build legitimacy by making use of structures, processes, and strategies that are appropriate for its institutional environment (Suchman 1995). A participation process is not automatically regarded by others—insiders or outsiders—as legitimate. Human and Provan’s (2000) work on collaboration indicates that, by extension, diff erent types of legitimacy may be involved. Th e fi rst is whether the form that participation takes is seen as legitimate by key stakeholders and can attract internal and external support and resources. Th e second is whether the participa-tion network produces interactions that build trust and legitimacy among participants and promotes necessary communication.
Part of establishing the legitimacy of the process is letting potential participants know the purpose of the process (or perhaps coproduc-ing the purpose, as noted earlier) and how their participation will infl uence outcomes. Diff erent purposes for public participation require appropriately matched strategies for communicating with and engaging (or not) various stakeholder groups (Cooper, Bryer, and Meek 2006). For example, drawing on the International Association for Public Participation’s widely used spectrum of participation, levels of participation can range from ignoring, to engaging as a data source, to informing, consulting, involving, collaborating, and fi nally to empowering stakeholders to make all decisions themselves (see http://www.iap2.0rg/associations/4748/fi les/spectrum.pdf). Each strategy involves a diff erent kind of promise to stakeholders. Diff erent kinds of engagement also imply the use of diff erent kinds of tools and techniques (Creighton 2005; Nabatchi 2012b; Th omas 2012). For example, if the engagement approach is informing, then consensus-building methods are inappropriate; if the approach is involving, the appropriate techniques may include workshops and deliberative polling but not participatory decision making, and so on. Otherwise, participation processes can easily become mired in confl icts about their authenticity and legitimacy that may stem from diff erent expectations rather than a willful attempt to make the par-ticipation process perfunctory or manipulative (Feldman and Quick 2009).
Foster Effective LeadershipDesign guideline 5. Ensure that the participation process leadership roles of sponsoring, championing, and facilitating are adequately fulfi lled.
organizers of public participation were to proceed as if they were welcoming all suggestions because they believe that government should be responsive to the public, they could overpromise in terms of how much of the input they receive they can actually use. In this case, a deliberative design in which various options are developed and evaluated would be frustrating for both those providing input and the practitioners who are charged with accounting for the public’s input. In this setting, a narrowly targeted consultation proc-ess around specifi c options or an information transmission meeting would be far more effi cient and eff ective.
Considering the purpose(s) to be served by a design is important, but settling on the purpose at the outset may not always be possible or desirable, for several reasons. First, articulating purpose is not a one-shot exercise because the context may change, as described earlier. Second, it may be desirable to coproduce the purposes in conjunction with participants, through the participation process itself, as discussed in design guideline 9. Th ird, Nutt’s (2002) exten-sive research on strategic decision making highlights the need to consult with key stakeholders before fi nalizing purposes and desired outcomes; otherwise, the chances of what he calls a decision-making “blunder” increase dramatically.
Analyze and Appropriately Involve StakeholdersDesign guideline 3. Ensure that the design and implementation of public participation processes are informed by stakeholder analysis and involve, at a minimum, key stakeholders in appropriate ways across the steps or phases of a participation process. Note that specifi c stakeholders may be involved in different ways at different steps or phases of the process.
Attention to stakeholders is a crucial part of the response to the context. Th e literature is quite clear that the appropriate stake-holders should be involved in appropriate ways in a participation process based on the context, overall task or project purpose, and goals of the participation process (Friend and Hickling 2005). Of course, who the appropriate stakeholders are and how to involve them are questions that process designers must answer. Including a wide range of participants in planning processes can promote the sharing of perspectives among diff erent participants and help gather information from all participants regarding their goals and objec-tives in the process (Enserink and Monnikhof 2003). Th e literature on collaboration indicates that when durable solutions are sought and consensus is the prime decision-making method, it is important to include the full range of stakeholders (Margerum 2002, 2011). When it comes to participation processes more generally, however, it is less clear how to decide who the appropriate stakeholders are and what the corresponding engagement approaches should be. Stakeholder analyses provide a set of techniques that can be used to address both concerns (Ackermann and Eden 2011; Bryson 2004).
We suggest that clarifying the purpose of participation (design guideline 2) should precede deciding on a basic strategy for engage-ment—for example, to inform, to collaborate, or to empower. In other words, the purposes of the participation eff ort should be a key determinant of which stakeholders should be engaged in which ways. Second, we suggest that approaches to involving and communicating with stakeholders should be diff erentiated throughout a process. For example, experts or people who are directly aff ected by a problem may
28 Public Administration Review • January | February 2013
trade-off s between the two kinds of costs, in that higher production costs may lower participation costs. It is also possible that in some situations, production costs may be so high that participation is not feasible.
But there is also a benefi t side to the cost calculus that participation process designers should consider and seek to amplify. Certainly when governments sponsor participation processes, they must allocate resources, such as funds, staff time, technical assistance, or information infrastructure. In an era of budget pressures, it may seem that those resources could be better used elsewhere, but a carefully managed participation process can contribute resources for public purposes as well (Delli Carpini, Cook, and Jacobs 2004; Koontz and Johnson 2004). Participants bring to public processes new information, self-interest, motivation to address problems, and new ways of understanding issues. Th ese can be used to uncover new understandings, generate better projects and policies, secure buy-in for decisions, and limit delays, mistakes, and lawsuits (Burby 2003). Th e process of participation may also enhance govern-ment–community trust (Moynihan 2003), social capital (Lake and Huckfeldt 1998), and infrastructure for ongoing community action (Abers 2000; Fung and Wright 2003). By designing and imple-menting public participation to be inclusive, public managers can generate unexpected resources, such as knowledge, commitment to follow-through, and enthusiasm, for decision making and policy implementation (Feldman and Quick 2009).
Create Appropriate Rules and Structures to Guide the ProcessDesign guideline 7. Create an appropriate set of rules and a project management team structure to guide operational decision making, the overall work to be done, and who gets to be involved in decision making in what ways.
Rules about how the process will be managed and how decision making will take place pro-vide a bridge between participation processes and organizational structures. For organiza-tions, these rules are often embedded in legal mandates or commonly held beliefs regarding appropriate roles and responsibilities. In a par-ticipatory setting, rules may be formal, such as written ground rules for working together, or informal, arising from the actual “doing”
of joint work. In doing joint work, participant interactions (i.e., processes) shape and are shaped by structures and rules about how participants will work together. When these experiences are positive, moral obligations and commitments increase and trust builds (Chen 2010; Jones, Hesterly, and Borgatti 1997). Conversely, if partici-pants violate rules and norms, trust will be undermined and hard to rebuild. Together, recognition of rules, the substance of rules, and structures for enforcing rules of three types—operational rules, general policies about the work to be done, and constitutional rules regarding who gets to make what kinds of decisions—help allow process participants to self-monitor (reward and sanction behavior), build commitment among themselves, and make or contribute to important decisions (Ostrom 1990).
In narrow, short-term public participation processes involving few stakeholders, there may be no need for a project management team.
Designing and implementing public participation clearly requires eff ective leadership—and increasingly so as the level of public participation increases (from ignoring, to informing, to consulting, to involving, to collaborating, to empowering) (Crosby and Bryson 2005; Morse 2010; Page 2010). Beyond that, as the problems or challenges to be addressed become more diffi cult, part of the leader-ship work becomes helping people who face problems with no easy answers take responsibility for those problems. A key practice of eff ective leadership in such circumstances is helping people stay in a productive zone between avoiding a problem without easy answers and being overwhelmed by the stress of tackling it (Heifetz, Grashow, and Linsky 2009).
Leadership can be exercised by one or many individuals associated with a public participation process, but the evidence indicates that three leadership roles are particularly important: sponsors, cham-pions, and facilitators (Crosby and Bryson 2005; Morse 2010; Schwarz et al. 2005). Sponsors are people with formal authority that can be used to legitimize and underwrite participation eff orts. Th e work of sponsoring includes establishing policies and provid-ing funds and staff that enable participation, endorsing and raising the visibility of public participation eff orts, and using the sponsor’s power to protect the participation process and ensure that the results of these eff orts have impact on policy making.
Champions, in contrast to sponsors, have positions with considerable responsibility for managing the day-to-day work of the participa-tion eff ort. Unlike sponsors, however, they typically cannot supply the resources and legitimacy needed to bring diverse groups into the participation process. Instead, they must rely heavily on informal authority accrued through their demonstrated competence over time, through trusting relationships, and through seniority. Th e work of championing thus requires generat-ing enthusiasm for the eff ort, building the support of sponsors, and sustaining the eff ort through setbacks. Facilitators structure partici-pation processes, maintain neutrality toward outcomes, and help groups work together pro-ductively. Facilitators help manage confl ict, coaxing participants to air their views and listen to each other’s views. As Schwarz et al. note, skilled facilitators are process experts who “know what kinds of behavior, process, and underlying structure are likely to contribute to high-quality problem solving and decision making” (2005, 29).
Seek Resources for and through ParticipationDesign guideline 6. Secure adequate resources—and design and manage participation processes so that they generate additional resources—in order to produce a favorable benefi t–cost ratio (broadly construed) for the participation process.
Th ose designing participation processes should identify resources needed to support participation, but they can also design and man-age participation to generate resources in such a way that partici-pation process benefi ts exceed process costs. Th ere are production costs incurred by organizations in developing and implementing a participation process; there are also participation costs incurred by citizens as part of their participation (Cooper 1979). Th ere may be
Rules about how the proc-ess will be managed and how
decision making will take place provide a bridge between par-
ticipation processes and organi-zational structures.
Designing Public Participation Processes 29
and options for resolving confl ict (Lowry, Adler, and Milner 1997). While consensus building is time-consuming and requires special-ized facilitation skills as well as political and logistical commitment (Brownhill 2009; Margerum 2011; Whitmarsh, Swartling and Jäger 2009), it works well in situations of uncertainty and contro-versy when all stakeholders have incentives to come to the table and mutual reciprocity is in their interests; and it off ers numerous potential desirable outcomes, including surfacing assumptions and enabling mutual learning among participants (Innes 2004; Jordan, Bawden, and Bergmann 2008).
Manage Power DynamicsDesign guideline 9. Manage power dynamics to provide opportunities for meaningful participation, exchange, and infl uence on decision outcomes.
Public managers should actively consider power dynamics in participation. To the extent that a participation eff ort simply rationalizes and reproduces the power of a particular stakeholder (Flyvbjerg 1998) or neuters diff erence by assimilating people into the process and pacifying them (Arnstein 1969; Cooke and Kothari 2001), it cannot be considered authentically participatory. Subtle power codes—such as the kinds of information and styles of expression that are considered relevant and appropriate—shape who participates in the process and how their input is received (Briggs 1998; Polletta and Lee 2006). For example, shifting from formal public hearings, which tend to be dominated by a small number of individuals comfortable with that format, to one-on-one interactions between public managers and residents, is an example of making design choices to reduce domination and marginaliza-tion (Takahashi and Smutny 1998). Managers deciding what is on the table for discussion is also an inherently powerful move that frequently places citizen groups at a disadvantage, as they are more likely to be reactive rather than proactive relative to the agenda for the participation process (Cooper and Nownes 2003). One way to share power more evenly among participants is to engage them in coproducing the agenda and process for decision making as well as weighing in on the policy decisions (Bovaird 2007; Quick and Feldman 2011; Roberts 2004).
Another source of power disparities in participation processes is privileging expert over “local” knowledge. Ozawa and Susskind (1985) characterize this problem in the context of science-intensive policy disputes, noting that experts might even disagree among themselves. Crewe (2001) describes a disconnect between local resi-dents’ views and those of design experts in eff orts to plan transit cor-ridors in Boston and fi nds that residents’ knowledge about the local context ultimately improved the design. Van Herzele (2004) off ers a case description indicating that local knowledge can be eff ectively integrated with professional knowledge; in this case, local knowl-edge helped extend professionals’ contextual frame and, ultimately, produced a refi ned outcome that was diff erent from the initial con-cept developed before the public participation process began.
Trusting relationships are one of the desirable means and ends of managing diversity, confl ict, and power dynamics successfully. Paradoxically, trust is both a lubricant and a glue—that is, trust helps facilitate the work of participation and helps hold the eff ort together (Bryson, Crosby, and Stone 2006). Trust can comprise
Broader scale, more time-consuming processes do require a project management team. Th e project management team may include sponsors, champions, and facilitators, as well as others, and should have adequate support staff and other resources necessary to func-tion eff ectively. Depending on the scope and scale of the process, the team may be fairly large (Creighton 2005; Friend and Hickling 2005).
Use Inclusive Processes to Engage Diversity ProductivelyDesign guideline 8. Employ inclusive processes that invite diverse participation and engage differences productively.
As the purpose of public participation moves away from simply complying with mandates to promoting participatory democracy and participant learning (Deyle and Slotterback 2009; Kirlin 1996), processes and structures are needed that are highly inclusive, engage diversity and seek advantages from it, and address issues of confl ict and power diff erences. A key challenge in public participa-tion is ensuring that the appropriate range of interests is engaged in the process, including those normally excluded from decision making by institutionalized inequities (Abers 2000; Parekh 2002; Schlozman, Verba, and Brady 2012; Young 2000). All too often, supposedly participatory processes end up including the “usual suspects,” people who are easily recruited, vocal, and reasonably comfortable in public arenas. Stakeholder identifi cation and analysis are critical tasks to undertake to ensure that marginalized groups are at least considered and may have a place at the table.
Practices for increasing participatory representativeness involve bet-ter outreach and optimizing accessibility of the process so that input can be more diverse. Advertising is an important but not the only way to ensure participants’ awareness of opportunities to partici-pate (Laurian 2004). Other practices include providing language translation, child care, or transportation assistance and choosing convenient meeting times and places for various constituencies. Having diverse voices at the table, however, may be insuffi cient or even counterproductive without also designing and managing the process to make use of that diversity, for example, by being open to changing topics and policy outcomes in order to make connections among diverse perspectives (Quick and Feldman 2011).
Increased diversity can be expected to lead to increased confl ict, at least initially. Designing inclusive participation processes requires explicit attention to managing confl ict (Forester 2009). Establishing clear ground rules and agendas for how to work together and a common problem defi nition create a structure for working through diff erences (Margerum 2002). Clarifying the source of the con-fl ict—over data, relationships, values, or the decision-making structure—is a good fi rst step (Crosby and Bryson 2005). When confl icting facts arise, group learning through interaction among participants can be further enhanced by taking claims seriously and developing processes that allow for resolution (Innes and Booher 2010; Lowry, Adler, and Milner 1997).
Facilitators can help participants examine underlying assumptions, shift from fi rm positions about particular outcomes to a more open-ended identifi cation of the interests that parties wish to address, and openly explore multiple options for action (Fisher, Ury, and Patton 2011). Th ese processes should allow for discussion of contingencies
30 Public Administration Review • January | February 2013
Another key question for managers to consider from the outset is how to evaluate the public participation eff ort. Defi ning evaluation measures in conjunction with deciding the purposes of engagement will help managers decide whether to engage in public participation, anticipate what kinds of results participation will produce, articulate participation process goals, and align their design and management strategies accordingly (Nabatchi 2012a). Organizers of participation eff orts often do not do formal evaluations, but they should consider doing both formative and summative evaluations (Patton 2010). Eff ective and operable measures of participation can help policy mak-ers learn from implementation so that they can enhance the eff ective-ness of the remainder of the participation eff ort they are currently working on and build long-term institutional capacity for future
participation (Rowe and Frewer 2004; Laurian and Shaw 2009). Evaluation may use a combi-nation of process criteria to determine how well an organization is implementing its proposed participation program and impact criteria to measure the consequences of participation for decision outcomes (Nabatchi 2012a).
Given the varied and divergent purposes for public participation described earlier, there is no single set of evaluation metrics for par-ticipation. Instead, process designers should consider which possible outcomes of the
process are most desirable and design measures accordingly (Rowe and Frewer 2004). Table 2 proposes outcome measures that are aligned with the various purposes of participation. Existing research and models support measuring a combination of diff erent types of outcomes, such as the following (Deyle and Slotterback 2009; Innes and Booher 1999; Laurian and Shaw 2009; Mandarano 2008; Margerum 2002; Milward and Provan 2000; Schively 2007):
• Individual-level outcomes (e.g., individuals’ increased knowl-edge of a policy issue, eff ects on citizenship behavior), group-level outcomes (e.g., mutual learning within the group about others’ perspectives), and community-level outcomes (e.g., the development of new options not previously considered, overall measures of community betterment)
• Process-oriented outcomes (e.g., building trust among partici-pants, incorporating a diverse group of stakeholders)
• Content-oriented outcomes (e.g., improving safety or environ-mental quality)
• User-oriented outcomes (e.g., participants’ satisfaction with the process, recognizing that diff erent stakeholders have diff erent criteria for success)
• First-, second-, and third-order outcomes, which are, respectively, the immediately discernible eff ects of the process (e.g., the quality of initial agreements), impacts that unfold once the process is under way (e.g., the formation of new partnerships), and long-term impacts (e.g., less confl ict among stakeholders in the future)
It may not be possible to measure the impact of policy choices made as a result of the participation process within the timeframe of the participation process itself. In that case, practitioners can draw on expert opinion to evaluate the outcomes of the process or to evaluate key technical aspects of the policy, such as its economic
interpersonal behavior, confi dence in organizational competence and expected performance, and a common bond and sense of goodwill (Chen 2010). At the same time, trust is problematic in any process involving people with diverse interests and levels of power (Huxham and Vangen 2005). Trust can be built by sharing informa-tion and knowledge and demonstrating competency, good inten-tions, and follow-through; conversely, failure to follow through and unilateral action undermine trust (Arinoa and De la Torre 1998). For example, Huxham and Vangen (2005) emphasize the eff ective-ness of achieving “small wins” together for building trust.
Eff ective confl ict management also can enhance trusting relation-ships by ensuring that disagreements are problem centered, not per-son centered. Eff ective management of power diff erences can help less powerful stakeholders trust the process and other participants more. Conversely, some powerful stakeholders might become more wary of the process if they feel that their power is being diminished. In cycli-cal fashion, as trust grows, it may substitute for formal structure in the ways in which it can control and standardize behavior because trust facilitates the sharing and diff usion of values and norms about standards of behavior (Moynihan 2009).
Use Information, Communication, and Other Technologies to Achieve the Purposes of EngagementDesign guideline 10. Participation processes should be designed to make use of information, communication, and other technologies that fi t with the context and the purposes of the process.
Participation processes that engage the public can be signifi cantly enhanced by the use of information, communication, and other technologies (Wang and Bryer 2012). Th ese technologies include public participation geographic information systems, computer-generated visualizations, interactive Web sites, keypad voting, and strategy mapping tools (Conroy and Evans-Cowley 2006; Howard and Gaborit 2007). Technology can be particularly eff ective in providing technical information and enhancing understanding of context (Appleton and Lovett 2005; Haklay and Tobón 2003), providing public access to information typically available only to experts (Al-Kodmany 2000; Elwood 2002), and gathering real-time feedback from participants (Han and Peng 2003). Visualization and other technologies can also help build shared understanding and facilitate interaction among users, as well as with the informa-tion provided (Bryson et al. 2004; Balram, Dragicevic, and Feick 2009). At the same time, it is important to acknowledge the limits of access to technology among public participants (Mossberger, Tolbert, and Gilbert 2006; Pew Foundation 2011) and limitations in resources available to planners and managers to deploy technol-ogy (Slotterback 2011).
Develop Participation Evaluation Measures and an Evaluation Process That Supports the Desired OutcomesDesign guideline 11. Develop participation evaluation measures and an evaluation process that supports producing the desired outcomes.
Eff ective management of power diff erences can help less pow-
erful stakeholders trust the process and other participants
more. Conversely, some power-ful stakeholders might become
more wary of the process if they feel that their power is being
diminished.
Designing Public Participation Processes 31
intended as practical guidance for practitioners to use as they make decisions about the design of participation processes. Th e guidelines are necessarily general, but they do off er some important evidence-based insights into how to approach issues of context, purpose, stakeholder involvement, leadership, process management, and eval-uation. Practitioners are encouraged to draw on what the literature has to off er and to integrate it with their own insights about what would work best given their specifi c circumstances as they formulate a specifi c design for participation (while recognizing that the design may well need to change as the situation changes).
Overall, a number of conclusions fl ow from our literature review and analysis. Th e fi rst is simply that the design of public participa-tion processes can be a very complex endeavor, particularly as the scope and scale of the envisioned processes increase. Process design-ers can face a substantial challenge in making use of the interrelated design guidelines to produce a specifi c design that is likely to be eff ective for fulfi lling purposes and achieving goals within the con-text at hand and within applicable constraints, satisfying any other requirements, engaging the appropriate stakeholders in appropriate ways, and making good use of various activities, methods, tools, and techniques. Rather than minimizing the complexity of design-ing participation processes, the design guidelines are intended to acknowledge that complexity, draw attention to important process design issues, and then off er practical ways to respond.
Th e guidelines are based on the extant literature, but additional work needs to be done to strengthen the foundations on which our
design propositions are based and to more fully specify them. Nonetheless, the fi eld has advanced enough that the design guidelines can be off ered with some reasonable faith in their soundness. Th e purposes to be served by participation processes are important. We hope readers will fi nd this literature review and set of design guidelines useful as they seek to pursue the purposes—and promises—of public participation.
ReferencesAbers, Rebecca. 2000. Inventing Local Democracy: Grassroots Policy in Brazil. Boulder,
CO: Lynne Rienner.Ackermann, Fran, and Colin Eden. 2011. Making Strategy: Mapping Out Strategic
Success. Th ousand Oaks, CA: Sage Publications.Al-Kodmany, Kheir. 2000. Public Participation: Technology and Democracy. Journal
of Architectural Education 5(4): 220–28.Andrews, Rhys, Richard Cowell, and James Downe. 2010. Promoting Civic Culture
by Supporting Citizenship: What Diff erence Can Local Government Make? Public Administration 89(2): 595–610.
Appleton, Katy, and Andrew Lovett. 2005. GIS-Based Visualization of Development Proposals: Reactions from Planning and Related Professionals. Computers, Environment, and Urban Systems 29(3): 321–39.
Arinoa, Africa, and Jose de la Torre. 1998. Learning from Failure: Towards an Evolutionary Model of Collaborative Ventures. Organization Science 9(3): 306–25.
Arnstein, Sherry. 1969. A Ladder of Citizen Participation. Journal of the American Institution of Planners 35(4): 216–24.
Balram, Shivanand, Suzana Dragicevic, and Rob Feick. 2009. Collaborative GIS for Spatial Decision Support and Visualization. Journal of Environmental Management 90(6): 1963–65.
effi ciency, impacts on safety or environmental quality, or aesthetic value (Margerum 2002). Even if the purposes of the participation process seem too intangible to measure, explicitly stating these purposes helps administrators to focus the design and management of participation toward the desired outcomes.
Align Participation Goals, Purposes, Approaches, Promises, Methods, Techniques, Technologies, Steps, and ResourcesDesign guideline 12. Align participation goals; participation purposes; types of engagement; promises made to participants; engagement methods, technologies, and techniques; steps; and resources in the process.
Participation processes should seek alignment across the elements of the process. Otherwise, the chances of miscommunication, misun-derstanding, and serious confl ict increase, along with concomitant declines in public trust and increases in public cynicism regarding participation (Creighton 2005). Th e change management literature indicates that having a plan is one of the keys to success (Fernandez and Rainey 2006). Two diff erent approaches to planning (designing) are evident in the literature. One approach emphasizes deliberate, formal planning as a precursor to success. Careful articulation of mission, goals, and objectives; roles and responsibilities; and phases or steps, including implementation and evaluation, are often cited as an important key to success. Th is approach—what Mintzberg, Ahlstrand, and Lampel (2009) call “deliberate”—appears to be most likely when a participation process is mandated. Th e second approach argues that a clear understanding of mission, goals, roles, and action steps is more likely to emerge over time as conversations involving individuals, groups, and organizations grow to encompass a broader network of involved or aff ected parties (Huxham and Vangen 2005; Innes and Booher 2010). Th is approach—what Mintzberg, Ahlstrand, and Lampel (2009) call “emergent”—seems most likely when participation is not mandated. Clearly, careful attention to stakeholders is crucial for success-ful planning regardless of approach (Bryson 2004).
Changes in the context for decision making (e.g., a reduced budget for implementing the decisions reached, increased or decreased political support for public participation, or new developments in the policy problem the process is addressing) may demand a concurrent change in the scope or timeline for the participation process. In addition, if previous design decisions have been made to share authority for the participation process, its outcomes, and ongoing follow-up, then participants may coproduce the design of the process as it unfolds (Bovaird 2007). Especially if stakeholders have to continue to collaborate to manage ongoing implementation, building capacities for shared leadership of the process may enhance the group’s capacity for ongoing adaptive management and imple-mentation of the decisions they make (Bryson, Crosby, and Stone 2006; Goldstein 2012; Quick and Feldman 2011).
ConclusionsTh e design guidelines outlined here are intended to integrate evi-dence from across a wide range of disciplines and contexts. Th ey are
Th e guidelines are based on the extant literature, but addi-
tional work needs to be done to strengthen the foundations on which our design propositions
are based and to more fully specify them.
32 Public Administration Review • January | February 2013
Crewe, Katherine. 2001. Th e Quality of Participatory Design: Th e Eff ects of Citizen Input on the Design of the Boston Southwest Corridor. Journal of the American Planning Association 67(1): 437–55.
Crosby, Barbara C., and John M. Bryson. 2005. Leadership for the Common Good: Tackling Public Problems in a Shared-Power World. 2nd ed. San Francisco: Jossey-Bass.
Cross, Nigel. 2011. Design Th inking: Understanding How Designers Th ink and Work. New York: Berg.
Daniels, Steven E., and Gregg B. Walker. 1996. Collaborative Learning: Improving Public Deliberation in Ecosystem-Based Management. Environmental Impact Assessment Review 16(2): 71–102.
Delli Carpini, Michael X., Fay Lomax Cook, and Larry R. Jacobs. 2004. Public Deliberation, Discursive Participation, and Citizen Engagement: A Review of the Empirical Literature. Annual Review of Political Science 7: 315–44.
Deyle, Robert, and Carissa Schively Slotterback. 2009. Group Learning in Participatory Planning Processes: An Exploratory Quasi-Experimental Analysis of Local Mitigation Planning in Florida. Journal of Planning Education and Research 29(1): 23–38.
Durning, Bridget, Juliet Carpenter, John Glasson, and Georgia Butina Watson. 2010. Th e Spiral of Knowledge Development: Professional Knowledge Development in Planning. Planning Practice and Research 25(4): 497–516.
Elwood, Sarah A. 2002. GIS Use in Community Planning: A Multidimensional Analysis of Empowerment. Environment and Planning A 34(5): 905–22.
Enserink, Bert, and Rene A. H. Monnikhof. 2003. Information Management for Public Participation in Co-Design Processes: Evaluation of a Dutch Example. Journal of Environmental Planning and Management 46(3): 315–44.
Feldman, Martha S., Anne M. Khademian, Helen Ingram, and Anne L. Schneider. 2006. Ways of Knowing and Inclusive Management Practices. Special issue, Public Administration Review 66: 89–99.
Feldman, Martha S., and Kathryn S. Quick. 2009. Generating Resources and Energizing Frameworks through Inclusive Public Management. International Public Management Journal 12(2): 137–71.
Fernandez, Sergio, and Hal G. Rainey. 2006. Managing Successful Organizational Change in the Public Sector: An Agenda for Research and Practice. Public Administration Review 66(2): 1–25.
Fisher, Roger, William Ury, and Bruce Patton. 2011. Getting to Yes: Negotiating Agreement without Giving In. 3rd ed. New York: Penguin.
Flyvbjerg, Bent. 1998. Rationality and Power: Democracy in Practice. Chicago: University of Chicago Press.
Forester, John. 1999. Th e Deliberative Practitioner: Encouraging Participatory Planning Processes. Cambridge, MA: MIT Press.
———. 2009. Dealing with Diff erences: Dramas of Mediating Public Disputes. New York: Oxford University Press.
Forsyth, Ann. 2007. Innovation in Urban Design: Does Research Help? Journal of Urban Design 12(3): 461–73.
Friend, John Kimball, and Allen Hickling. 2005. Planning under Pressure: Th e Strategic Choice Approach. Oxford, UK: Heinemann.
Fung, Archon, and Eric Olin Wright. 2003. Deepening Democracy: Institutional Innovations in Empowered Participatory Governance. New York: Verso.
Gaus, John M. 1947. Refl ections on Public Administration. Tuscaloosa: University of Alabama Press.
Godschalk, David R., and Bruce Stiftel. 1981. Making Waves: Public Participation in State Water Planning. Journal of Applied Behavioral Science 17(4): 597–614.
Goldstein, Bruce E., ed. 2012. Collaborative Resilience: Moving through Crisis to Opportunity. Cambridge, MA: MIT Press.
Haklay, Mordechai, and Carolina Tobón. 2003. Usability Evaluation and PPGIS: Towards a User-Centered Design Approach. International Journal of Geographical Information Science 17(6): 577–92.
Bason, Christian. 2010. Leading Public Sector Innovation: Co-Creating for a Better Society. Bristol, UK: Policy Press.
Bauer, Russell M. 2007. Evidence-Based Practice in Psychology: Implications for Research and Training. Journal of Clinical Psychology 63(7): 685–94.
Boaz, Annette, Lesley Grayson, Ruth Levitt, and William Solesbury. 2008. Does Evidence-Based Policy Work? Learning from the UK Experience. Evidence and Policy: A Journal of Research, Debate and Practice 4(2): 233–53.
Bovaird, Tony. 2007. Beyond Engagement and Participation: User and Community Coproduction of Services. Public Administration Review 67(5): 846–60.
Boyne, George A., and Richard M. Walker. 2010. Strategic Management and Public Service Performance: Th e Way Ahead. Special issue, Public Administration Review 70: 185–92.
Boyte, Harry C. 2005. Reframing Democracy: Governance, Civic Agency, and Politics. Public Administration Review 65(5): 536–46.
Briggs, Xavier de Souza. 1998. Doing Democracy Up Close: Culture, Power, and Communication in Community Building. Journal of Planning Education and Research 18(1): 1–13.
Brody, Samuel D., David R. Godschalk, and Raymond J. Burby. 2003. Mandating Citizen Participation in Plan Making: Six Strategic Planning Choices. Journal of the American Planning Association 69(3): 245–64.
Brown, Tim. 2009. Change by Design: How Design Th inking Transforms Organizations and Inspires Innovation. New York: HarperCollins.
Brownhill, Sue. 2009. Th e Dynamics of Participation: Modes of Governance and Increasing Participation in Planning. Urban Policy and Research 17(4): 357–75.
Bryson, John M. 2004. What to Do When Stakeholders Matter: Stakeholder Identifi cation and Analysis Techniques. Public Management Review 6(1): 21–53.
Bryson, John M., Fran Ackermann, Colin Eden, and Charles B. Finn. 2004. Visible Th inking: Unlocking Causal Mapping for Practical Business Results. Chichester, UK: Wiley.
Bryson, John M., Barbara C. Crosby, and Melissa Middleton Stone. 2006. Th e Design and Implementation of Cross-Sector Collaborations: Propositions from the Literature. Special issue, Public Administration Review 66: 44–55.
Burby, Raymond J. 2003. Making Plans Th at Matter: Citizen Involvement and Government Action. Special issue, Journal of the American Planning Association 69: 33–49.
Chen, Bin. 2010. Antecedents or Processes? Determinants of Perceived Eff ectiveness of Interorganizational Collaborations for Public Service Delivery. International Public Management Journal 13(4): 381–407.
Conroy, Maria Manta, and Jennifer Evans-Cowley. 2006. E-Participation in Planning: An Analysis of Cities Adopting On-Line Citizen Participation Tools. Environment and Planning C: Government and Policy 24(3): 371–84.
Cooke, Bill, and Uma Kothari, Eds. 2001. Participation: Th e New Tyranny? New York: Zed Books.
Cooper, Christopher A., and Anthony J. Nownes. 2003. Citizen Groups in Big City Politics. State and Local Government Review 35(2): 102–11.
Cooper, Terry L. 1979. Th e Hidden Price Tag: Participation Costs and Health Planning. American Journal of Public Health 69(4): 368–74.
Cooper, Terry L., Th omas A. Bryer, and Jack W. Meek. 2006. Citizen-Centered Collaborative Public Management. Special issue, Public Administration Review 66: 76–88.
Corburn, Jason. 2003. Bringing Local Knowledge into Environmental Decision Making: Improving Urban Planning for Communities at Risk. Journal of Planning Education and Research 22(4): 420–33.
Cowan, Janene. 2012. Integrating Design Th inking Practices into the Public Sector. Master’s thesis, University of Minnesota.
Creighton, James L. 2005. Th e Public Participation Handbook: Making Better Decisions through Citizen Involvement. San Francisco: Jossey-Bass.
Designing Public Participation Processes 33
Han, Sun Sheng, and Zhen Peng. 2003. Public Participation GIS (PPGIS) for Town Council Management in Singapore. Environment and Planning B: Planning and Design 30(1): 89–111.
Healey, Patsy. 1997. Collaborative Planning: Shaping Places in Fragmented Societies. Vancouver, Canada: University of British Columbia Press.
Heifetz, Ronald, Alexander Grashow, and Marty Linsky. 2009. Th e Practice of Adaptive Leadership: Tools and Tactics for Changing Your Organization and the World. Boston: Harvard Business Press.
Heinrich, Carolyn J. 2007. Evidence-Based Policy and Performance Management: Challenges and Prospects in Two Parallel Movements. American Review of Public Administration 37(3): 255–77.
Hill, Michael, and Peter Hupe. 2009. Implementing Public Policy. 2nd ed. Th ousand Oaks, CA: Sage Publications.
Howard, Toby L. J., and Nicolas Gaborit. 2007. Using Virtual Environment Technology to Improve Public Participation in Urban Planning Process. Journal of Urban Planning and Development 133(4): 233–41.
Human, Sherrie E., and Keith Provan. 2000. Legitimacy Building in the Evolution of Small-Firm Multilateral Networks: A Comparative Study of Success and Demise. Administrative Science Quarterly 45(2): 327–65.
Huxham, Chris, and Siv Vangen. 2005. Managing to Collaborate: Th e Th eory and Practice of Collaborative Advantage. New York: Routledge.
Innes, Judith E. 2004. Consensus Building: Clarifi cation for the Critics. Planning Th eory 3(1): 5–20.
Innes, Judith E., and David E. Booher. 1999. Consensus Building and Complex Adaptive Systems: A Framework for Evaluating Collaborative Planning. Journal of the American Planning Association 65(4): 412–24.
———. 2010. Planning with Complexity: An Introduction to Collaborative Rationality for Public Policy. New York: Routledge.
Janis, Irving L. 1989. Crucial Decisions: Leadership in Policymaking and Crisis Management. New York: Free Press.
Jones, Candace, William S. Hesterly, and Stephen P. Borgatti. 1997. A General Th eory of Network Governance: Exchange Conditions and Social Mechanisms. Academy of Management Review 22(4): 911–45.
Jordan, Nicholas R., Richard J. Bawden, and Luke Bergmann. 2008. Sustainability and Epistemic Competencies: Facing the “Worldview Challenge” in Sustainable Development of Agriculture. Journal of Natural Resources and Life Sciences Education 37: 92–99.
Kirlin, John J. 1996. Th e Big Questions of Public Administration in a Democracy. Public Administration Review 56(5): 416–23.
Koontz, Tomas M., and Elizabeth Moore Johnson. 2004. One Size Does Not Fit All: Matching Breadth of Stakeholder Participation to Watershed Group Accomplishments. Policy Sciences 37(2): 185–204.
Koontz, Tomas M., and Craig W. Th omas. 2006. What Do We Know and Need to Know about the Environmental Outcomes of Collaborative Management? Special issue, Public Administration Review 66: 111–21.
Krizek, Kevin J., Ann Forsyth, and Carissa Schively Slotterback. 2009. Is Th ere a Role for Evidence-Based Practice in Urban Planning and Policy? Planning Th eory and Practice 10(4): 459–78.
Lake, Ronald La Due, and Robert Huckfeldt. 1998. Social Capital, Social Networks, and Political Participation. Political Psychology 19(3): 567–84.
Laurian, Lucie. 2004. Evaluating Plan Implementation: A Conformance Based Methodology. Journal of the American Planning Association 70(4): 471–80.
Laurian, Lucie, and Mary Margaret Shaw. 2009. Evaluation of Public Participation: Th e Practices of Certifi ed Planners. Journal of Planning Education and Research 28(3): 293–309.
Lowry, Kem, Peter Adler, and Neal Milner. 1997. Participating the Public: Group Process, Politics, and Planning. Journal of Planning Education and Research 16(3): 177–87.
Mandarano, Lynn A. 2008. Evaluating Collaborative Environmental Planning Outputs and Outcomes: Restoring and Protecting Habitat and the
New York–New Jersey Harbor Estuary Program. Journal of Planning Education and Research 27(4): 456–68.
Mansbridge, Jane. 1999. On the Idea Th at Participation Makes Better Citizens. In Citizen Competence and Democratic Institutions, edited by Stephen L. Elkin, and Karol Edward Soltan, 291–325. University Park: Pennsylvania State University Press.
Margerum, Richard D. 2002. Evaluating Collaborative Planning: Implications from an Empirical Analysis of Growth Management. Journal of the American Planning Association 68(2): 179–93.
———. 2011. Beyond Consensus: Improving Collaborative Planning and Management. Cambridge, MA: MIT Press.
Milward, Brinton H., and Keith G. Provan. 2000. Governing the Hollow State. Journal of Public Administration Research and Th eory 10(2): 359–79.
Mintzberg, Henry, Bruce Ahlstrand, and Joseph Lampel. 2009. Strategy Safari: A Guided Tour through the Wilds of Strategic Management. London: Prentice Hall.
Moore, Mark, and Jean Hartley. 2008. Innovations in Governance. Public Management Review 10(1): 3–20.
Morse, Ricardo S. 2010. Integrative Public Leadership: Catalyzing Collaboration to Create Public Value. Leadership Quarterly 21(2): 231–45.
Mossberger, Karen, Caroline J. Tolbert, and Michele Gilbert. 2006. Race, Place, and Information Technology. Urban Aff airs Review 41(5): 583–620.
Moynihan, Donald P. 2003. Normative and Instrumental Perspectives on Public Participation: Citizen Summits in Washington, DC. American Review of Public Administration 33(2): 164–88.
———. 2009. Th e Network Governance of Crisis Response: Case Studies of Incident Command Systems. Journal of Public Administration Research and Th eory 19(4): 895–915.
Nabatchi, Tina. 2010. Addressing the Citizenship and Democratic Defi cits: Exploring the Potential of Deliberative Democracy. American Review of Public Administration 40(4): 376–99.
———. 2012a. Putting the “Public” Back in Public Values Research: Designing Participation to Identify and Respond to Values. Public Administration Review 72(5): 699–708.
———. 2012b. A Manager’s Guide to Evaluating Citizen Participation. Fostering Transparency and Democracy Series. Washington, DC: IBM Center for the Business of Government.
Nutt, Paul C. 2002. Why Decisions Fail: Avoiding the Blunders and Traps Th at Lead to Debacles. San Francisco: Berrett-Koehler.
Ostrom, Elinor. 1990. Governing the Commons: Th e Evolution of Institutions for Collective Action. New York: Cambridge University Press.
Ozawa, Connie P., and Lawrence Susskind. 1985. Mediating Science-Intensive Policy Disputes. Journal of Policy Analysis and Management 5(1): 23–39.
Page, Stephen. 2010. Integrative Leadership for Collaborative Governance: Civic Engagement in Seattle. Leadership Quarterly 21(2): 246–63.
Parekh, Bakhu. 2002. Rethinking Multiculturalism: Cultural Diversity and Political Th eory. Cambridge, MA: Harvard University Press.
Patton, Michael Q. 2010. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation. New York: Guilford Press.
Pew Foundation. 2011. Pew Internet and American Life Project. http://www.pewin-ternet.org [accessed October 25, 2012].
Polletta, Francesca, and John Lee. 2006. Is Telling Stories Good for Democracy? Rhetoric in Public Deliberation after 9/11. American Sociological Review 71(5): 699–723.
Quick, Kathryn S., and Martha S. Feldman. 2011. Distinguishing Participation and Inclusion. Journal of Planning Education and Research 31(3): 272–90.
Roberts, Nancy. 2004. Public Deliberation in an Age of Direct Citizen Participation. American Review of Public Administration 34(4): 315–53.
Romme, A. Georges L., and Gerard Endenburg. 2006. Construction Principles and Design Rules in the Case of Circular Design. Organization Science 17(2): 287–97.
34 Public Administration Review • January | February 2013
Rosener, Judy B. 1975. A Cafeteria of Techniques and Critiques. Public Management 57(12): 16–19.
Rowe, Gene, and Lynn R. Frewer. 2004. Evaluating Public-Participation Exercises: A Research Agenda. Science, Technology, and Human Values 29(4): 512–56.
Sabatier, Paul A. 2007. Th eories of the Policy Process. Boulder, CO: Westview Press.Sackett, David L., William M. C. Rosenberg, J. A. Muir Gray, R. Brian Haynes, and
W. Scott Richardson. 1996. Evidence Based Medicine: What It Is and What It Isn’t. British Medical Journal 312: 71–72.
Schively, Carissa. 2007. A Quantitative Analysis of Consensus Building in Local Environmental Review. Journal of Planning Education and Research 27(1): 82–98.
Schlozman, Kay Lehman, Sidney Verba, and Henry E. Brady. 2012. Th e Unheavenly Chorus: Unequal Political Voice and the Broken Promise of American Democracy. Chicago: University of Chicago Press.
Schwarz, Roger, Anne Davidson, Peg Carlson, and Sue McKinney. 2005. Th e Skilled Facilitator Fieldbook. San Francisco: Jossey-Bass.
Scott, W. Richard, and Gerald F. Davis. 2006. Organizations and Organizing: Rational, Natural and Open Systems Perspectives. Englewood Cliff s, NJ: Prentice Hall.
Simon, Herbert A. 1996. Sciences of the Artifi cial. Cambridge, MA: MIT Press.Slotterback, Carissa Schively. 2008. Stakeholder Involvement in NEPA Scoping
Processes: Evaluating Practices and Eff ects in Transportation Agencies. Journal of Environmental Planning and Management 51(5): 663–78.
———. 2011. Planners’ Perspectives on Using Technology in Participatory Processes. Environment and Planning B: Planning and Design 38(3): 468–85.
Stone, Melissa M., and Jodi R. Sandfort. 2009. Building a Policy Fields Framework to Inform Research on Nonprofi t Organizations. Nonprofi t and Voluntary Sector Quarterly 38(6): 1054–75.
Suchman, Mark C. 1995. Managing Legitimacy: Strategic and Institutional Approaches. Academy of Management Review 20(3): 571–610.
Takahashi, Lois M., and Gayla Smutny. 1998. Community Planning for HIV/AIDS Prevention in Orange County, California. Journal of the American Planning Association 64(4): 441–56.
Th omas, John Clayton. 1995. Public Participation in Public Decisions: New Skills and Strategies for Public Managers. San Francisco: Jossey-Bass.
———. 2012. Citizen, Customer, Partner: Engaging the Public in Public Management. Armonk, NY: M. E. Sharpe.
Van Aken, Joan Ernst. 2007. Design Science and Organization Development Interventions: Aligning Business and Humanistic Values. Journal of Applied Behavioral Science 43(1): 67–88.
Van Herzele, Ann. 2004. Local Knowledge in Action: Valuing Nonprofessional Reasoning in the Planning Process. Journal of Planning Education and Research 24(2): 197–212.
Wang, XiaoHu, and Th omas A. Bryer. 2012. Assessing the Costs of Public Participation: A Case Study of Two Online Participation Mechanisms. American Review of Public Administration. Published electronically on March 21. doi:10.1177/0275074012438727.
Webler, Th omas, Horst Rakel, Ortwin Renn, and Branden Johnson. 1995. Eliciting and Classifying Concerns: A Methodological Critique. Risk Analysis 15(3): 421–36.
Wenger, Etienne. 1999. Communities of Practice: Learning, Meaning, and Identity. New York: Cambridge University Press.
Whitmarsh, Lorraine, Åsa Gerger Swartling, and Jill Jäger. 2009. Participation of Experts and Non-Experts in a Sustainability Assessment of Mobility. Environmental Policy and Governance 19(4): 232–50.
Wildavsky, Aaron. 1979. Speaking Truth to Power: Th e Art and Craft of Policy Analysis. Boston: Little, Brown.
Wondolleck, Julia Marie, and Steven Lewis Yaff ee. 2000. Making Collaboration Work: Lessons from Innovation in Natural Resource Management. Washington, DC: Island Press.
Yang, Kaifeng, and Sanjay K. Pandey. 2011. Further Dissecting the Black Box of Citizen Participation: When Does Citizen Involvement Lead to Good Outcomes? Public Administration Review 71(6): 880–92.
Young, Iris Marion. 2000. Inclusion and Democracy. New York: Oxford University Press.
