GOOD NEWS: Data-Driven Decisions
Good News: An enewsletter for donors and nonprofits
on strategic planning, governance, fundraising, and executive leadership.
“5 Common Errors to Avoid When Receiving Data from Nonprofits or Reporting to Donors”
Data-Driven Decisions
When it comes to "data-driven decisions," most people feel as skeptical and confused as the cartoon above suggests - for good reason.
Einstein famously said, "Not everything that can be counted counts, and not everything that counts can be counted." I wonder what he would think about how donors and nonprofits collect, report, and interpret data in the hopes of reaching beyond anecdotal evidence in search of best practices?
As an undergraduate math major and someone who taught Advanced Placement Statistics for many years, I have a healthy respect and affection for data. But I'm with Einstein and Gahan, the artist who created the cartoon above. I bet each would say that most of us do a lousy job when it comes to collecting, presenting, and interpreting data.
Here are 5 common errors to avoid when receiving data from nonprofits or reporting to donors:
1. Too Much Information: Like the chart in the cartoon, too many foundations ask for too much and too many nonprofits report too much information. Keep it simple. Especially when it comes to modest grant making or grant seeking, less is more when it comes to data and its interpretation. A few reliable data points are better than many haphazardly collected ones. Beware the data collection adage, “Garbage in, garbage out.”
2. Respect Data and the Art of Statistics: For small to medium sized organizations, useful and reliable data is hard to collect and understand. Simple counts can be muddled. Human biases are inevitable and riddle data with implicit judgments. Even if good data is tallied, understanding its meaning and drawing inferences from it is a challenge for even the most experienced data scientists and statisticians. Question the foundation board or nonprofit chief executive who speaks with certainty about the meaning of discrete data points which are merely threads in a tapestry. At the end of the day, statistics is much more art than science.
3. Be Humble and Share Credit: Even the most rigorous, longitudinal, double blind, simple random sampled experiments have a very hard time establishing causal relationships between two variables. So be humble when making claims about the impact of your organization's grant making or services. Show humility by sharing credit with others engaged in the ecosystem. Collaborate. Acknowledge that correlation is not causation. No foundation's grant making or organization's direct services are wholly responsible for outcomes since multivariate challenges almost always occur in dynamic systems.
4. Trust, Verify, and Be Trustworthy: Most organizations are not resourced to hire expensive, sophisticated data collection and interpretation firms. Be worthy of the trust given to self-report data that has integrity and meaning. Beware the occasional organization that fudges its numbers or tailors its practices - sometimes at the expense of those they seek to serve - to juice up their data in search of support. Do not be another who gives credence to Benjamin Disraeli’s famous quip, “Lies, damn lies, and statistics.” If possible, seek an affordable third party who can hold up a mirror, hone, and validate data practices.
5. Stories Matter: While some feel pushed to abandon anecdotes that demonstrate an organization's success, resist the temptation to ground your reporting or reporting requirements solely in quantitative data. Do not hesitate to include qualitative data to illustrate and bring to life your measured outcomes. Human beings - that means donors, too! - are hard-wired to be moved by stories so continue to tell relevant, compelling stories to accompany well-thought-out and reliable data.
Stuff Steve Is Watching, Listening To, and Reading
A Trust-Based Framework for Learning and Evaluation in Philanthropy (1 hour, 22 minute view)
"We may have this desire to know without a doubt - we want certainty - but as a sector we are trying to become much better listeners, we are learning more how to wield, how to share, how to yield power. And we have to be robust and rigorous in our approaches. We are in the business of contribution rather than attribution. We worked with the Board to help them reimagine what evaluation is for. It's not to pat ourselves on the back. It's not to make ourselves feel good about what we're doing. It's really an approach to thinking about what we are hearing from our communities and our grantees. It requires us to sit and listen, to observe, and actually work in partnership with our grantees."
Watch the Full Webinar Here
Michael Lewis and The Father of Moneyball (34 minute listen)
“Rating is a form of an opinion. If it is ground out of a formula, it becomes an opinion expressed in numbers. What I was always trying to do was open up the discussion to more ways of looking at the problem, and ["Wins Above Replacement"] or any other absolute measure has the opposite effect. It’s not opening up the discussion. It’s saying 'OK, we’re done, we’re done.'” Bill James
“The problem is not the numbers...It’s how people use them. The numbers start out as tools for thinking. They wind up replacing thought.” Michael Lewis
Listen to the Podcast Here
Columbia University's Ranking Ascent (30 minute read)
“Trying to rank institutions of higher education is a little like trying to rank religions or philosophies. The entire enterprise is flawed, not only in detail but also in conception. Rankings create powerful incentives to manipulate data and distort institutional behavior for the sole or primary purpose of inflating one’s score. Because the rankings depend heavily on unaudited, self-reported data, there is no way to ensure either the accuracy of the information or the reliability of the resulting rankings."
Colin Diver and Michael Thaddeus
Read More Here
Steve Filosa
978 578 1904
www.tothegood.net
Congratulations to TTG client Youth Development Organization for another extraordinary summer.
Watch this if you have 90 seconds and want to feel inspired:
https://vimeo.com/740487931
