FiDA Partnership is delighted to announce the launch of the third edition of the Digital Finance Evidence Gap Map (EGM). While the first edition contained 40 studies, two years on the EGM has almost doubled to include 75 studies. Each year, as studies amass, we inch closer to understanding the impact of various digital finance products on low-income users.
The conversation on DFS impact is happening now!
In the coming weeks, FiDA will publish a blog series on various insights derived from an analysis of the latest EGM update. These will include a review of the new impact insights at the product level—digital savings, credit, payments, and transfers—in addition to a review of the insights on micro-entrepreneurs and small businesses. These reviews offer a taste of the analyses made possible using the EGM, but they’re the tip of the iceberg.
The driver for developing the EGM was to encourage a continuous dialogue on impact. In the last 12 or so months, there appears to have been an uptick in these impact conversations. It was notable in the number of published systematic reviews of both digital and non-digital financial services. These include a review of digital and non-digital evidence, such as the Innovation for Poverty Action (IPA) evidence review of financial services in addressing resilience and the Campbell Collaboration meta-review of the impact of financial inclusion. Mobile payments gets specific attention in three systematic reviews, one for low-middle income countries, one specifically examining the evidence for sub-Saharan Africa, and another examining the channels of economic influence of mobile money from both a micro and a macro perspective. Moreover, CGAP has doubled down on efforts to synthesize evidence on the impact of financial inclusion in their own insightful blog series.
However, a growing conversation should not be read as a duplication of effort. Each study tackles a different slice of the impact question, either looking at a specific product, geography, digital or non-digital, or utilizing specific study methods. Others still, such as the CGAP series, question our expectations of financial inclusion and produce nuanced insights into contexts that researchers and practitioners should consider. Every study is helpful, each adds value and brings us closer to understanding client impact.
What can you expect from the EGM 3.0?
The updated EGM now includes 75 studies that test 81 digital finance products and interventions from 26 different countries. There are 14 digital finance products and 32 design and delivery mechanisms with various counts of evidence. A variety of research methodologies are used, with Randomized Control Trials (RCTs) (28%), cross sections (19%), and panels (17%) being the most frequent.
Geography: Although the EGM represents 26 countries, Kenya accounts for nearly a third of the studies (30%), and the East African region accounts for over half (53%) of the impact literature. Because geographical context is a significant variable we must consider the various social, economic, and cultural differences involved before we apply learnings from East Africa to other regions.
Digital finance products: Digital payments and transfers account for 47% (n=38) of the impact literature. When general mobile money studies (i.e., studies that do not identify a specific use, e.g., P2P, airtime top, or bill payment) are folded into this category, the percentage of digital finance studies increases to 62% (n=50). It is crucial to recognize the gaps in product-level impact insights and allocate resources to begin correcting it.
Product design and delivery: Meta-studies may fail to account for the varied ways in which digital finance products are designed and delivered. The aggregate effects of several different savings products are simply designated as the effects of “Savings” overall. The reality of the impact of different savings products is far more complicated. An analysis of the EGM confirms that product designers use a multitude of innovative digital, and non-digital, design and delivery mechanisms. Further, the EGM shows that, across the 81 products studied, 32 different design and delivery features were observed. On average there were two design and delivery mechanisms described per product in the EGM. These nuances of design and delivery matter, as we will highlight in the product-level deep dives.
Client outcomes: Across the 75 studies, 334 reported “tests” were linked to the 10 high-level client outcomes. The tests are concentrated at immediate outcomes (adoption: 13%, and savings behavior: 21%) and longer-term outcomes (resilience: 13%, welfare: 16%, and income investing/asset building: 13%). The types of outcomes tested, vary at the product-level.
Positive leaning results, but mixed at a product-level examination: In the second view of the EGM, we share the results of the tests from each of the studies. This is not a straightforward task as studies differ in how they report their results, sometimes using multiple statistical models on the same outcomes, and thus the full study of interest should be reviewed in detail. However, our high-level analysis observed positive, negative, and null effects across the various digital finance products on various client outcomes. A total of 63% of the reported tests were positive, 6% were negative, and 31% were null. However, in the review of product-level impacts in the coming series, we will see that the proportions vary.
What this tells us is that researchers are casting their nets wide—testing a broad range of outcomes—to interrogate various theories around the impact of different products. Indeed, if a product study tested two outcomes and found no effects, this does not mean that the product does not move the dial at all, but rather that the dials moved may not have been examined. Choosing what to test is important and these choices are refined through continued testing and conversations with other impact studies.
In other posts we highlighted the fact that digital finance is not one thing: it is dozens of products, designed and delivered in various ways, to various client segments, in various markets. A single study cannot answer the impact question. We need to aggregate the various impact insights by product and place them in dialogue with each other. We encourage those of you interested in the impact question to fill in the gaps and enable the digital finance community to understand the impact of various digital finance products delivered in their nuanced ways. Ultimately the understanding generated by the EGM should enable improved designs and better outcomes for low-income users.
The studies in the EGM represent our best knowledge of digital finance impact insights. New studies are continuously emerging and thus the EGM will continue to evolve. If you have questions on the EGM, are interested in discussing research priorities, or know of relevant digital finance impact studies that meet the inclusion criteria, please contact ideas@financedigitalafrica.org.
