Philanthropy is experiencing a moment of disruption as it reconsiders its role in addressing racial equity and confronts broader questions of trust and legitimacy.
Amid increased attention to these issues, many foundations are re-examining their grantmaking practices, shifting funding strategies, and taking new approaches to engage, elevate and empower communities.
Many of these changes extend to the field of philanthropic evaluation, which seeks to support the use of data and insights to inform ongoing learning and adaption in ways that increase the impact of our strategies and our partners over time.
To better understand emerging trends, the Walton Family Foundation’s Strategy, Learning, and Evaluation Department (SLED) – along with PolicySolve – recently asked 18 leaders to share their insights into the challenges confronting philanthropic evaluators.
What we learned is that the field faces myriad, intertwined issues, some related to the expanding set of skills and capacities required of individual evaluators, others to the shifting role of evaluation in philanthropy more broadly, and still others specific to the moment we’re facing as a society. The full report is available here.
Meeting the Current Moment
At this point many questions still remain about how evaluation and evidence-generation should evolve and adapt as greater power and control (rightly) shifts from foundation staff to grantees and communities. Many of the people we talked with told us that this shifting dynamic does not change the need for evidence, but rather complicates it, leading to the question: Will evaluators be able to “keep up” with the changes?
Through our conversations with leading experts, five key themes about the state of our field emerged:
There is a Crisis of Legitimacy
Both the philanthropic and evaluation fields are facing an existential crisis of perceived legitimacy.
Philanthropy is grappling with whether existing grantmaking approaches, as currently designed, can achieve larger systemic change goals. Can foundations, at times based outside the places they are seeking to change, support meaningful change? What is the value they are bringing to the issues they seek to address? These questions have direct impact on the role of evaluators because a key assumption of our work is that impact is possible and can be made visible through the collection of evidence.
Similarly, the evaluator’s role is less clear now than ever before, due to the increasingly complex, systemic work that philanthropy seeks to undertake. Evaluators are often unable to offer compelling evidence of how foundation investments contributed to a successful outcome, particularly at the community or systems level. At the same time, evaluators are also expected to take on an expanded set of skillsets and roles beyond just generating evidence – often acting as coach, facilitator, technical assistant, learning partner, and strategist. As a result, it has become less clear to both evaluators and others the specific value an evaluation is meant to bring to the table.
The evaluator’s role is less clear now than ever before, due to the increasingly complex, systemic work that philanthropy seeks to undertake.
Growing doubts about the influence of evaluators is also having an impact on the legitimacy of evaluation itself. Some field leaders believe evaluators have less influence within foundations than in the past. This may be tied to shifts in role as evaluators seek to be more of a learning partner than accountability judge. It may also reflect shifts in the methods employed – more qualitative work, less experimental work - and/or shifts in the demographics of the evaluation field more generally. In addition, there can be an inherent tension when evidence needs to be seen as both credible (meeting a high standard of rigor) and actionable (produced quickly, potentially based on more limited data) to meaningfully influence decision-making.
We Must Address Diversity, Equity and Inclusion
If philanthropic evaluation is going to meet this moment and maintain its legitimacy, it must address deficits in diversity, equity, and inclusion. These include diversifying the pipeline and supporting people of color in the field; increasing evaluators’ understanding of equity, structural racism, and systemic transformation; and developing evaluation practices that are more culturally responsive and equitable in their design, implementation, and dissemination.
We Need Better Lines of Communication
The philanthropic evaluation field’s ability to strengthen its practices is limited because it remains siloed from the learning happening in highly related fields, among them the communities that evaluators serve and academics whose work relates to philanthropy, evaluation, and the issues evaluations investigate. These silos limit the progress of knowledge development and dissemination between fields, create barriers to solving problems that exist within one part of the field and have been solved for in another, and make it difficult to transition into the field of philanthropic evaluation due the specialization of the evaluation practice.
Training Programs for Evaluators Have Gaps
As in most fields, one of the challenges in foundation evaluation is that participants come into the field from many different backgrounds, some of which have not included opportunities to learn about topics that are critical to success in the job. Some of these gaps include knowledge about how philanthropy operates, strategy design, systems thinking, facilitation, learning systems, and navigating complex power dynamics.
Some of these skill deficits are due to an emphasis on methods over softer skills in evaluation training programs. Some stem from the changing roles of evaluators, including the many “hats” evaluators now wear. Field leaders also expressed concern that when “everyone is an evaluator” we may lose sight of some of the valuable skills and perspectives that our field can offer.
Other Field-Building Needs
The challenges noted above are cross-cutting, bridging many evaluation roles, types of evaluations, issue areas, and settings. Other field building needs are more specific – critical to the field, but not as cross-cutting. They include integrating foresight knowledge into evaluation; strengthening participatory practices in evaluation; and expanding access to evaluation training and onboarding to better prepare evaluators for their work in philanthropy.
The results of this landscape analysis will inform SLED’s new field-building strategy, which will be launched later this year, with the goal of helping our peers and colleagues as they seek solutions to these challenges.
