Wrapping up the Evidence Gap Map impact insights series—The value of community and conversation

Niamh Barry

What we shared

The FiDA partnership launched version 2.0 of the Digital Finance Evidence Gap Map (EGM) in October 2018. With 55 studies examining 60 products, there are many insights to navigate. To show the types of analysis the EGM makes possible, we published a number of impact insights on a range of topics. These are the headlines of our insights series:

  • Insights from digital savings studies show promising results from coupling savings products with two-way SMS, mobile learning platforms, and client training. Various types of savings accounts—accounts with default contributions, locked accounts, and commitment accounts—were found to improve the savings behaviors of users. Additionally, one study forefronted the need for careful planning when transitioning from an analogue to a digital service, and another raised the issue of differential impacts of digital interventions between women and men.
  • The evidence on digital credit was limited compared to evidence on traditional microcredit. Yet insights for promoting ‘healthy borrowing’ surfaced. These included optimizing ways to frame and time SMS repayment reminders, making borrowing T&Cs more salient and accessible, and providing interactive educational content on financial literacy. However, beyond borrowing behavior, there is limited evidence for the longer-term impacts of credit.
  • With 18 studies, Person-to-Person (P2P) payments and transfers have the most fully formed, evidenced based pathway to client impact. The insights on this topic are numerous and often consolidating, allowing us to have more confident conversations on the impact of P2P. Numerous tests highlighted how users share risk and smooth consumption through quick access to remittances, while others qualified the ease of storing value and making and receiving transactions in privacy. However, money comes from somewhere and some studies also raised concerns regarding the pressure on those who do the sending.
  • Most recently, we reflected on sources of variance when testing for  impact, that is, each digital finance study tests the the impact of a product, in a country, for a population, via a channel. On the product side we noted that digital credit and insurance products, are under-evaluated for client impact. In terms of geography, learning has been concentrated in East Africa; while this knowledge may be transferable, there are far fewer studies from other regions. Lastly, we shared that studies that disaggregated impact on more excluded groups (women, lower income, less education) showed that different populations can experience different outcomes. This insight challenges an assumption that a given product will have the same effects across the board.

There are still many more insights to uncover

The EGM insight topics of FiDA’s mini-series represent only a small sample of what can be gleaned from the EGM. For instance, we did not develop product-level insights for digital products like Government-to-Person transfers (G2P), Business-to-Person transfers (B2P), or digital insurance There are reasons for this.

The main reason was the low volume of studies on these topics. Additionally, in the case of G2P and B2P, we observed that, of the few studies available, each tested different outcomes, in different ways, in different markets. The studies have not consolidated enough to derive actionable insights. The digital finance community is in a nascent stage of testing and learning about the effects of these products. And although these specific digital payments are a niche line of inquiry, we will continue to track new studies for those who work in this space.

Beyond a product-level analysis, the EGM comprises other learning, such as the types of product design and delivery mechanisms gaining traction, segmenting findings by population types and market level, or even methodological approaches to measuring digital finance impact. The EGM can be used as an entry point for all of these inquiries.

What would it take to conclusively prove that digital finance has development impact?

In October of this year, I spoke at the Mastercard Foundation’s Partnership for Financial Inclusion Learning event, convened by IFC. A question posed to the panel was “

What would it take to conclusively prove that digital finance has development impact?

While I raised a few points on the need for funding mechanisms, quality partnerships between researchers and digital finance services providers, and the value of sharing null and negative results, my standout insight came from EGM analyses: No single study can prove a digital finance impact pathway. This is why synthesis documents are so incredibly valuable in advancing learning.

In the discussion I offered the following scenarios to illustrate the importance of continued conversations on the impact of digital finance:

Imagine the resources for 50 impact studies a year were available.

The digital finance community can only extract a certain amount of value from the 50 studies if the 50 studies are unaware of each other. This unawareness might mean that each study has different conceptualizations of digital finance, the effects it may have on certain clients, and how to measure outcomes; they might even focus on different assumed knowledge gaps rather than actual knowledge gaps. They would not be learning from each other.

Alternatively, imagine that, for the same 50 studies, there is an incentive to cross check and coordinate to ensure that each is in dialogue with the others, testing the real gaps, finding things that the other studies did not, consolidating previous learnings, and creating a mini-community that talks about what is working and what is not. Through open communication, the digital finance community can get far more than 50 studies’ worth of information from the same 50 studies. That is how the digital finance community will come closer to an understanding of the impact of digital finance.

The greatest impact we can have through resources like the EGM, is in using the knowledge of the digital finance community to make better choices on both testing impact and designing better products.

FiDA is publishing a series on insights derived from an analysis of the latest Digital Finance Evidence Gap Map (EGM) update. This is the sixth blog.

Previous blogs:

The studies in the EGM, represent our best knowledge of digital finance impact insights. New studies are ever emerging and thus the EGM will continue to evolve. If you have questions on the EGM, are interested in discussing research priorities, or know of relevant digital finance impact studies that meet the inclusion criteria, please contact ideas@financedigitalafrica.org.