FiDA is publishing a mini series on various insights derived from an analysis of the third edition of the Evidence Gap Map (EGM). This is the second blog. Blog 1: Inching closer to understanding the digital finance impact question
Last year we published a summary of the digital savings impact insights garnered from the EGM 2.0. Now, the third edition of the EGM’s evidence base has incorporated five additional digital savings studies. By compounding existing evidence and/or presenting new insights on the impact (or lack thereof) of different design and delivery mechanisms, these studies are valuable additions to the evolving digital savings impact story.
What are the insights so far?
The EGM now includes 17 studies evaluating the effect of digital savings products (in 10 countries). The majority of these studies involved different markets, client segments, and design and delivery mechanisms. Accordingly, we should be cautious about drawing any final conclusions or applying lessons learned without proper attention to context. The table below illustrates the diversity of the digital savings products tested.
These 17 studies entail 94 outcomes tests. The diagram below shows the outcomes tested, and the results of the tests. Positive (66%), negative (6%), and null effects (28%) are observed both across the studies and within individual studies. This is indicative of the early stage of testing and learning. As we continue to synthesize the evidence, clearer patterns will emerge to eventually allow us to uncover best practices.
Our previous savings impact insights
Our previous review of digital savings impact studies yielded the following overarching insights:
- Two-way SMS and mobile learning platforms were shown to enable better savings behaviors.
- Coupling savings products with client training appears to bolster a client’s ability to optimize the use of savings in the longer term.The effects of access to savings products on longer-term outcomes was less certain.
- Incorporating digital elements into existing traditional financial services such as ATM cards or roving agents with POS technology was shown to improve savings behavior.
- A change from an analog service to a digital service was shown to have several negative effects on savings balances and thus practitioners should carefully design for the transition.
- Default contributions, locked savings, and commitment savings accounts were all found to improve the savings behavior of clients.
- Product design needs to carefully consider the potential differential effects between groups, for example, women and men and higher and lower incomes. In some cases certain groups benefit more than others.
The new savings studies contribute to previous insights relating to two-way SMS, coupling savings with training, variations in savings accounts, and understanding differential effects.
Spotlight on new savings studies
Behavioral science and two-way SMS can support or hinder savings behavior
- A new study has been added that equally supports and rejects the idea that two-way SMS and mobile learning platforms improve savings behaviors.
- A CGAP supported study showed that utilizing behavioral science through a two-way SMS system was shown to increase and decrease savings account usage. The study tested various messages aimed at improving the savings behaviors of Airtel money users in Tanzania. Three themes were tested:
1. Social influence: Referencing the saving balances of top savers in the group
2. Agency: Emphasizing that savings provide control
3. Mental accounting nudges: Reminders of an individual’s savings goal
The group that received messages about the savings balances of other savers increase their savings by up to 11 percent. However, the group that received messages about agency saved even less than the control group.
This study demonstrates that the wrong message may be worse than no message at all and further emphasizes that testing before scaling is crucial.
Labeling savings accounts may improve savings behavior and welfare outcomes, but the mechanisms are unclear
- Previous “varieties” of digital savings accounts tested included default, commitment, and locked. Each study demonstrated positive results in terms of increasing savings in their various and varied contexts.The EGM update provides new insights on labeled savings accounts.
- Labeling savings for emergencies can act as a soft commitment to save for a specific purpose. In two studies women in Kenya were offered M-Pesa savings accounts labeled for emergency savings (coupled with client training, weekly SMS reminders, and zero transaction fees).The first study tested whether the intervention improved women’s coping strategies in the event of a health shock. They found that the intervention led to an increase in M-Pesa savings (combining pre-exisiting and labeled M-Pesa accounts) sufficient to cover health expenses and decreased the likelihood that women would engage in transactional sex as a coping response to shocks.
- The second study tested a similar intervention to understand the effects of promoting savings on informal risk sharing among pairs of women—reciprocal sending and receiving of money in a time of need—and welfare outcomes. Similar to the first study, an increase in M-Pesa savings was observed when pre-existing and labeled M-Pesa accounts were combined. They found that by increasing savings, pairs of women have less need to request/send transfers to each other. However, welfare outcomes also improved. Among women who experienced a negative shock, the intervention increased the food security score by 15 percent, reduced the deficit for non-food expenses by 49 percent, and increased the subjective well-being score by 13 percent, relative to the control group.
In both examples, the savings value increased, but not necessarily in the “labeled” account, and had positive effects on resilience and welfare outcomes. However, along with the labed accounts, participants also received training on the value of savings, weekly SMS reminders, and zero transaction fees. Unfortunately, neither study was designed to pinpoint what aspect—the labeling, training, reminders, or the fee waiver—contributed to the observed effects. Accordingly, the mechanisms that demonstrated the impact are unclear.
Offering more than a savings accounts: The value of client training
- Although a number of studies have provided training (different varieties through different modes) to clients, only one previous study examined the effects of providing a savings product with and without client training. They found that providing business training alongside a savings product improved client savings and business outcomes more than the savings product alone.
- Echoing this study, researchers tested the midline impact of two interventions, both separately and together, in alleviating constraints that impede savings and business growth among women micro-entrepreneurs in Indonesia. The mobile savings intervention expanded access to mobile savings accounts while the business training intervention provided training to improve business skills and financial literacy. Overall they found that the business training in combination with the mobile savings intervention led women to increase total and mobile savings. Women who only received training also increased savings, but not significantly. Both interventions independently increased household assets. The combination of the two interventions increased women’s self-confidence and control, but the interventions had no discernible effects, either alone or combined, on women’s business outcomes at the midline point.
This study contributes early insights to previous observations that coupling savings products with client training may improve clients’ ability to optimize the use of savings in the longer term. It also demonstrates the value of disentangling the design and delivery mechanisms of an intervention and deepens our understanding of how impact was achieved.
Differential impact: When excluded groups benefit more
- Previous insights highlighted that “product design needs to carefully consider the potential differential effects between women and men and higher and lower incomes.” These insights presented cases wherein women and lower income clients benefited less from the savings product. A new study contributes an example where excluded groups benefit more.
- Researchers in Sri Lanka facilitated a collaboration between a large public bank and a telecom company, creating an interface allowing individuals to make deposits into a savings account through their mobile phone. While the intervention did not lead to an increase in overall savings for the total sample of clients, differential impacts among men and women and among clients living further from the bank were observed. Women had a significant 38 percent increase in the amount deposited while those who lived between 2 and 5 km from any bank deposited 70 percent more through the service than those who lived closer to a bank. This highlights that one mechanism for increasing use was reducing distance-related transaction costs.
These observations offer the digital finance community a refined understanding of impact by determining the conditions under which impact is present, stronger, or weaker. They also underscore the need to examine how a digital finance product interacts with and affects various excluded groups.
Bringing insights together
None of the previously noted insights have been challenged by the new studies, but the evidence garnered from the new studies has either supported previous insights or added new ones. From the additional five studies we found:
- Complementary insights on using two-ways SMS and behavioral science to improve savings—with an important caution on testing the messaging
- Additional early evidence on the benefits of coupling savings products with business training for women—while making a case for the value of testing the various design and delivery mechanisms
- New insights on the effects of labeling savings accounts for resilience and welfare—and new questions on impact mechanisms.
- Additional evidence supporting the need to be aware of how different client segments experience impact; they may benefit more or less.
The insights gathered thus far are encouraging and, as a community, we should continue to look for opportunities to gather evidence on the effects of digital savings products. Gaps in our knowledge remain in product design and delivery, markets, and clients segments; by filling these gaps and improving how digital savings products are tested, the digital savings community will be able to develop better products and deliver them more effectively.
The studies in the EGM represent our best knowledge of digital finance impact insights. New studies are continuously emerging and thus the EGM will continue to evolve. If you have questions on the EGM, are interested in discussing research priorities, or know of relevant digital finance impact studies that meet the inclusion criteria, please contact ideas@financedigitalafrica.org.
