Skip to content Skip to navigation

Evaluating the East Side Alliance as a Collective Impact Initiative

" "
Molly Cain
Jazmin Harper
Jordan Richards
Mackenzie Stein
Silicon Valley Education Foundation

In the Winter 2011 issue of the Stanford Social Innovation Review, John Kania and Mark Kramer of FSG consulting firm, introduced a new social change model designed for highly structured collaboration across many sectors, which they named collective impact (CI). The purpose of a collective impact initiative is to allow cross-sector leaders and their organizations to work together for social change using complementary skill sets and resources. Collaboration amongst change actors is a relatively old concept and networks have been frequently created to promote social change. However, Kania and Kramer’s CI model is innovative in its structure, and consequently is being adopted by many communities to create a suitable vehicle for change.

This report was created for the Silicon Valley Education Foundation, who has asked us to evaluate their relatively new education collective impact initiative, the East Side Alliance (ESA). This report will answer our policy question: What policy recommendations can be made to enhance East Side Alliance as a collective impact initiative?

In this report we analyze the East Side Alliance's current structure and performance, along with its future plans, using the ideal characteristics and best implementations of Kania and Kramer's five pillars of successful collective impact. The five pillars include Common Agenda, Backbone Infrastructure, Shared Measurement System, Mutually Reinforcing Activities, and Continuous Communication. These pillars of successful collective impact initiatives are defined as follows:

  • The common agenda of a collective impact initiative is a set of clearly defined primary goals. It is typically established through many meetings and discussions, and it is designed and agreed upon by all participating partners.
  • A strong backbone infrastructure will be the main vehicle of facilitating ongoing communication. This is an organization focused on the difficult task of helping the partners, initiative organizations, and funders meet frequently and communicate continuously; it is essential to any successful collective impact initiative.
  • Shared measurement systems are a set of benchmarks and metrics designed to measure the success of a collective impact initiative. The system of shared measurement is especially important in tracking the progress of the initiative toward the goals set out through the common agenda.
  • Mutually reinforcing activities requires that all organizations or stakeholders are making efforts that conform to the initiative’s stated goals. Engaging in mutually reinforcing activities guarantees that although different partners in the initiative may have different resources and excel at performing different tasks, these tasks all contribute to the common agenda and vision.
  • Continuous communication among partners in the initiative is essential. A successful collective impact initiative seeks to align the activities of interests as different as nonprofits, corporations, and government agencies. Prioritizing frequent, planned communication among these different entities of the initiative is crucial to keeping all partners aligned and working toward the same goal.

We also include in our analysis a sixth pillar of success—Community Engagement. In addition to looking at each of the six pillars, we will analyze the role of self-evaluation across all of the pillars.

To conduct our analysis on each of the six pillars, we reviewed literature on the theory behind the collective impact process, conducted 17 interviews with ESA partners and related organizations, and did research on relevant collective impact case studies around the nation. The case studies we examined include: Cincinnati's Strive, Seattle’s RoadMap Project, Portland's All Hands Raised, the Marin Promise, and Fresno Strive.

To measure the success of ESA as a collective impact initiative, we created an evaluation checklist consisting of six pillars. The checklist consists of outcomes, which are benefits resulting from actions that describe the ideal CI characteristics for that pillar and indicators for each outcome, which provide evidence of whether or not the outcome is accomplished. We have adapted our checklist so that it appropriately evaluates a young collective impact initiative such as ESA. For our evaluation to be suitable for the East Side Alliance, our checklist highlights “early-performance” indicators that identify appropriate outcomes for early stage initiatives. Additionally, we provide evidence from case studies of successful collective impact initiatives for many indicators. These examples provide standards for comparison that aid our analysis. Finally, we created questions for most indicators that we asked in our interviews of ESA partners, which are also included in the checklist.  The material for the first five pillars—common agenda, backbone infrastructure, shared measurement system, mutually reinforcing activities, and continuous communication—was adapted from FSG's Checklist to Evaluating Collective Impacts. Additionally, we added a sixth pillar for community engagement, modifying material from the Tool Kit published by the Asset-Based

Community Development Institute. We also created a cross-pillar checklist for self-evaluation, which is a theme of continuous learning that is important to all six pillars. To create this cross-pillar theme, we highlighted outcomes and indicators that focused on ESA’s ability to monitor its progress, evaluate the effectiveness of its strategies, and adapt its goals and strategies based on evidence.

Our evaluation checklist is very extensive, and due to time and analysis constraints, we decided to divide our analysis into two sections: a deeper analysis of certain key elements, and a more general assessment of others. We provide an in-depth analysis of three areas that we view as the primary focus for policy recommendations: common agenda, community engagement, and self- evaluation. We feel these areas are most important for long-term sustainability of ESA. In addition, we conduct a broader, but less deep, holistic analysis and discussion of policy recommendations for the four other structural pillars of collective impact—the backbone organization, shared measurement system, mutually reinforcing activities, and continuous communication. We will start with our review of the four areas of holistic analysis before turning to the areas of deeper focus.

Furthermore, we designed a scoring system to grade each pillar. We separated the scoring of analysis into three categories: On Track: Continue Development; Need to Expand; and Need To Initiate. Each pillar has unique indicators that signal the current achievements and plans for future sustainability.

Download Report