The State of Advanced Analytics & Data Science in the Nonprofit-Sector

It was in 2017 when the renowned magazine The Economist wrote: "The world’s most valuable resource is no longer oil, but data."

We set up our blog askyourdata.co in the same year. The topics of our articles vary from illustrative applications of models to fundraising data, discussions of topics like Artificial Intelligence for Fundraising to data visualization. The common denominator for our content is the context of charitable non-profit organizations. We think that the tools and methods of advanced analytics and data science can contribute to the effectiveness and efficiency of fundraising organizations. This is why we started wondering whether there are empirical findings on the state of data science and advanced analytics in data science. Good news: We found results that are both recent and interesting.

Data Science starts with data. Back in 2016, the American software company EveryAction surveyed some 460 professionals (presumably mainly in the US; the complete study can be downloaded here) from non-profits about their habits, culture, and outlook on the state of data at their organizations. Some key findings:

• The vast majority (90%) of respondents obtains data in the course of their operations.
• The fact that there is a remaining percentage of 10% might be due to the structure of this study´s respondents as more than two thirds (68.66%) consisted of organizations with a size between 1 and 24 employees, i.e. rather small non-profits that either do not feel the urge of data management and / or lack the resources for it. 
• When it comes to a data-driven culture, only 5% responded they were using data in every decision they make.
• At the same time, 97% of the non-profits surveyed expressed interest in learning how to use their data more effectively

In other words, there is a large perceived gap between the status quo and how a data-driven organization could look like. Why is this the case? The survey also asked for factors that prevent organizations from using their data in more depth and breadth. These are (in order of relevance):

• Not enough  time and / or staff to focus on data
• Staff lacks experience in using data
• Data is not stored in one place
• Lack of tools to help analyse data
• Not enough data within the organization

With regard to the actual use of (advanced) analytics in non-profit organizations, we found a global study conducted by IBM Corporate Citizenship and Corporate Affairs (download here) in collaboration with the IBM Institute for Business Value from 2017. The authors of the study explain that non-profits face growing need for quantifiable evidence due to a growing pressure from the competitive environment as well as from internal decision makers and demands by supporters. The study also asked for the perceived state of analytics in the respective organization. The possible answers were leant against the Analytics Maturity Model by Gartner:
​

A bit more than two-thirds of the respondents conduct ad hoc or (ex post) hindsight analysis (descriptive, diagnostic) whereas the remaining third has stepped into advanced analytics (predictive, prescriptive, AI / cognitive). Having mentioned the term “advanced”, one should not forget about the speed of developments. What might have been considered “advanced” a few years ago might turn into the "standard level" sooner or later.
​
The probably most interesting insight from the survey is that respondents with more advanced analytics capabilities reported higher effectiveness than others on three important metrics (Improve performance against mission, Achieve operating efficiencies, Improve staff productivity). In other words: Non-profits with deeper data capabilities see stronger impact, transparency and decision making. 
​

Like the aforementioned study by EveryAction, the IBM survey asked for primary barriers to advancing data and analytics. These are:

• Budget (Competing internal investments, followed by restrictive nature of grants)
• Technology
• Talent
• Nature of data collected
• Organizational structure
• Cultural resistance

What about data science in the non-profit sector? We did not come across specific studies on the state of data science in the non-profit sector in general or fundraising more particularly. However, we had a closer look at the industry segmentations in two major data science surveys.

JetBrains polled more than 1.600 people involved in Data Science in the US, Europe, Japan and China (download full study here). One of the questions asked for the industry for which the respondents analyse data. The non-profit sector ranked quite low (5% compared to Accounting / Finance / Insurance and Science with both 16%). This finding can be seen as consistent with the results of the aforementioned studies that diagnose scope for advances in the field of non-profit analytics.

The presumably most comprehensive study on the state of data science and machine learning is the annual Machine Learning and Data Science Survey conducted by the platform kaggle.com. Kaggle surveyed almost 24.000 people in October 2018, the results are therefore quite recent. In line with open data thinking, the raw data containing the survey results can be downloaded from the site, there is also a data analysis competition attached to it.

The non-profit related results in a nutshell: Of some 24.000 respondents, only a tiny fraction of 0.79% (189 respondents) answered that they work or recently worked for an employer in the non-profit industry. Regardless of the fact that the sample might not be fully representative across industries, this figure shows that data science is still in its beginnings in so-called third sector.
​

When it comes to the regional distribution of non-profit data scientists, it is striking that a third of respondents that said that they work for nonprofits reside in the United States. India is the runner-up with some 10 percent. The numbers of the remaining countries with respondents is quite evenly distributed.
​

​Our conclusion: Non-profit representatives and decision makers are largely aware of the potential benefits that use of advanced analytics and data science might imply for their organizations. Building competencies, structures and systems is often challenged by scarce resources (most prominently budget but also expertise). The answers regarding the presence of data scientists surveyed by JetBrains and Kaggle reflect this state. The good news is: Nonprofits need not feel alone in their advanced analytics endeavours, as outlined by IBM who advocate establishing an insight ecosystem.

If you are interested in building your own insight ecosystem, you might wish to learn more on what joint systems can offer in this regard.

Let us stay in touch! We wish you a nice rest of spring.