If you are a fan of PBS NewsHour, chances are you’ve come across more climate-related coverage over the past couple of years. Produced with support from the Walton Family Foundation’s Environment Program, “Tipping Point” was a runaway success for the program, reaching millions of viewers through compelling live streams and broadcast segments on the challenges facing U.S. rivers, fisheries and agriculture.
The stories brought together lawmakers, climate researchers, industry leaders, farmers, tribal leaders and cultural influencers. And because NewsHour attracts opinion leaders who take the extra step beyond just viewing, the “Tipping Point” broadcasts were frequently reposted by advocates and concerned citizens.
In the past, evaluating this grant would have been straightforward. Did the grantee hit their audience and content milestones? Would you consider the program a success or failure? If successful, replicate with similar funding and metrics.
But even a successful grant has something to teach you. That’s where the APPLE comes in.
The APPLE – or, the Assessment of Past Performance and Learning report – is a new tool designed to dig into the work more deeply and create a shared understanding of successes and challenges, including those not anticipated in the formal performance metrics.
It’s part of the foundation’s efforts to meaningfully pivot from an accountability-based grant evaluation approach to a learning-based approach.
Developed by the foundation's Strategy, Learning and Evaluation team, the APPLE helps foundation program officers hold structured conversations with grantees to reflect on what went well during the course of a grant, and what can be improved.
As funders, our role is to support PBS’s work from behind a firewall. They have editorial control.
In the case of the “Tipping Point” grant, we knew that our partnership with PBS had been conventionally successful. But we wanted to learn if there were still opportunities to improve outcomes in the next round of funding.
When our Environment Program team sat down with PBS as the grant concluded, they asked a new set of questions, designed to encourage our partners to provide more of their perspective on what worked and what could be done to have even more impact moving forward.
One question asked about unexpected outcomes from the grant. The APPLE conversation uncovered that the live-stream format -- a first-of-its-kind for the PBS team -- far exceeded audience viewership, engagement and interest than was originally anticipated. It showed that viewers crave a deep dive into solutions-oriented content. And they appreciate a personal angle in reporting segments.
“We want to take that great audience engagement to a new level – understanding what their experiences were with the content and what they learned from it.”
The APPLE dialogue also surfaced that while viewership was objectively high, PBS had limited ability to dive into these data further. What type of viewer was watching? What impact did the segment have on the viewer? Were they driven to act?
As a result of this finding, the foundation’s new grant with PBS now includes additional funding to gain better audience insights. The goal? To collect data on viewer engagement, reach and metrics for each broadcast.
“We want to take that great audience engagement to a new level – understanding what their experiences were with the content and what they learned from it,” says Travis Daub, the director of digital for PBS NewsHour.
The APPLE uncovered another suggestion. For years, PBS NewsHour has collaborated with Knology to collect research on how audiences interact with number-heavy news.
Research shows that “when audiences encounter number-heavy stories, but lack sufficient information to interpret the data being shared with them, they can feel confused, share bad information or become suspicious and distrustful of the media.”
To counteract this, the next round of grantmaking will include additional support that will allow the NewsHour’s digital team to study our audience reactions to "Tipping Point Climate" coverage in ways that lead to a deeper understanding of viewer’s needs, wants, and interests around this complicated issue.
The NewsHour team will also look to past projects like the reportingwithnumbers.org resource tool created for journalists which helps PBS’s audience think critically about the data that are shared with them. “Take something like the 1.5 C climate benchmark. It’s talked about so widely, but many people still don’t understand why it’s a big deal. Reporting With Numbers will help us provide more clarity in our coverage.”
From the foundation’s perspective, we see these conversations with grantees as an opportunity for shared learning that helps us find better answers that can lead to greater impact.
The insight gained through the APPLE process with PBS led to meaningful insight into what worked, what didn’t, and how to fine-tune future grantmaking to drive education and action on the climate.
Ultimately, the APPLE conversation allowed the foundation’s Environment team to capitalize on what could have been a missed opportunity.
“We have been covering climate for years. We know that people feel overwhelmed. They want to do something, but they don’t know what,” says Merrill Schwerin, a deputy managing producer of the PBS program. “One of the reasons ‘Tipping Point’ has been so successful is that it shows our audience solutions that are working right now.”
The job of nonprofit journalists is to raise public awareness, but Merrill says it can be difficult to gauge the impact of programs like “Tipping Point.”
“When we sat down with the [foundation] evaluation team at the end of the grant, their questions and process helped inspire conversations between our own teams about how to measure and enhance our impact more immediately. As the next grant begins, we are thinking of all these new ways to get more from our content.”
