Applied Research & Evaluation

William Penn Foundation: Evaluation of a Plan to Build Community Accountability

In 2008, Metis Associates was engaged by the William Penn Foundation to evaluate the impact of the foundation’s school-reform initiatives on high drop-out rates, low achievement, and under-resourced schools in low-income areas throughout Philadelphia. The foundation is supporting a community-driven approach to reform, which it hopes will have long-term effects. The foundation asked Metis to assess its portfolio of grants to 15 organizations in Philadelphia to determine whether it had achieved its objectives over the past six years. Among the grantees were youth groups that organize high school students to create change within their own schools and across the district, advocacy organizations focusing on better educational outcomes for children and youth, community organizing groups addressing school-reform issues, and convening entities that bring together stakeholders to coordinate work around school reform.

Metis brought the William Penn Foundation a substantial amount of knowledge of theory-of-change evaluation, school reform, community organizing, and the Philadelphia schools. Metis used a theory-of-change framework for the evaluation, identifying the foundation’s theory, strategies, and activities, and establishing short-, intermediate-, and long-term outcomes. The staff of Metis’s Philadelphia office interviewed members of the grantee organizations, conducted focus groups with community organizing groups, and interviewed experts in the field. In doing so, they found many examples of the grantees’ influence on the decision-making process of schools and of the district.