Delving into human consciousness: using psychometric assessments in financial services

Case Study on LenddoEFL and Juhudi Kilimo

Introduction and Summary

Everyone has a unique and distinctive personality. Fortunately for low-income populations that may have been excluded from formal financial services, their character traits can now help them access financial products. A psychometric credit assessment provides an alternative for thin-file loan applicants (i.e., zero or low credit history) by generating credit scores based on personality and behavior. These assessments predict loan repayment behavior by measuring an applicant’s attitude, integrity, and performance. Beginning in 2006, innovative firms like LenddoEFL (formerly Lenddo and Entrepreneurial Finance Lab [EFL]) were among the first to pioneer psychometric assessments for lending in emerging markets. More than ten years later, financial service providers, like Te Creemos (Mexico), are leveraging psychometric data in their credit-decision making processes.

This case study will delve into LenddoEFL’s journey in developing psychometric data for financial services and, in parallel, explore the experience of Juhudi Kilimo (JKL), a Kenyan microfinance institution, in implementing LenddoEFL’s credit-scoring model.

Interviews with LenddoEFL and JKL reveal that the credit-scoring model increased JKL’s acceptance rate by 5% and predicted clients’ repayment rates/borrowing behavior. Further, both organizations recommend that financial service providers (FPSs) have a clear use case for employing psychometric credit assessments, and ensure that the tool helps them solve a business challenge they face in their current credit appraisal process, before implementing LenddoEFL’s credit-scoring model. Financial service providers (FSPs) would also benefit from conducting an assessment of their own appraisal system to identify how to integrate a psychometric tool into their technical systems. The interviews further suggest that FSPs should gauge their target clients’ willingness to take a psychometric assessment, as tests may take a long time to administer to clients.

This study is part of FiDA’s broader exploration of the promise and limitations of big data analytics in financial services. It expands on the insights presented in FiDA’s Focus Note, Can big data shape financial services in East Africa? A number of the FSPs and FinTechs interviewed for the Focus Note pointed to the value of employing psychometric testing to accelerate financial services, particularly given the dearth of commercially available traditional data in sub-Saharan Africa that can be used to push digital finance forward. The case study endeavors to provide FSPs interested in leveraging non-traditional, alternative data with potential paths for using psychometric data. The case study explores the lessons that LenddoEFL learned in designing and evaluating psychometric testing on thin-file clients, and Juhudi Kilimo’s pilot of psychometric testing provides an early example of how FSPs and FinTechs can begin working with psychometric testing.

We are all passionate about psychometric data because [it is the] only dataset that everyone has. [It is the] Main reason why we decided to leverage it. Everyone has a personality and can answer the questions….most inclusive form of credit scoring.

LenddoEFL
Box 1 What is psychometric testing?

Psychometric testing assesses a loan applicant’s creditworthiness by asking them targeted questions that reveal personality traits relevant to willingness to pay (such as their attitudes about success). For the ‘credit invisible,’ or unscorable population, without a formal credit history, the beauty of psychometric testing is that it can gauge trustworthiness, likelihood of repayment, gratification, and impulsivity, among other characteristics.

The Pros of Psychometric Testing

Psychometric testing has been found to boost the predictive power of traditional scores, when combined with a credit bureau scoring model,1 and it removes human bias because no loan officer judges a loan applicant’s character. Its predictive power can also reduce risk and increase lending, thus expanding access to credit.2

Moreover, psychometric testing can, in some cases, reduce customer acquisition and operational costs3 because it decreases the human capital cost of processing applications (such as the need for a loan officer).

The Cons of Psychometric Testing

At the same time, this technology, like any other, has some pitfalls. Firstly, asking someone to take an assessment assumes a certain level of education. Secondly, test-takers may find ways to game the system, such as by asking a family member or friend to take the test on their behalf or answering the questions in such a way as to give a positive—but inaccurate—impression. It is important to note, however, that providers like LenddoEFL and Visual DNA are aware of these challenges and are addressing them. For instance, over time, LenddoEFL has evolved its assessment’s design to make it easier for those who cannot read or write to take the test; they also have loan officers available to assist those who are 100% illiterate. Additionally, they have developed an algorithm that detects instances of lack of independence, lack of effort, and lack of completeness.

LenddoEFL’s entry to psychometric testing

LenddoEFL began as a research project at the Harvard Center for International Development by Drs. Bailey Klinger and Asim Khwaja who developed a survey to test behavioral traits in South Africa and “solve the problem of information opacity for micro and small businesses applying for loans.” They initially lent to borrowers without psychometric data and, with repayment data under their belt, they tested the relationship between an individual’s psychometric profile and loan performance. After almost four years of research, LenddoEFL officially started offering FSPs psychometric-based credit scores. They have since developed more than 50 models that measure personal initiative, situational judgment, creativity, and business acumen, and these models adapt to country and community contexts. FSPs can either use LenddoEFL’s tool alone to assess creditworthiness or enhance their existing underwriting process by using LenddoEFL’s tool as an additional metric.

In late 2017, EFL merged with the technology company Lenddo. Lenddo has developed patented technology that employs non-traditional digital data to develop credit scores and verification of its applicants. The merged company, LenddoEFL, now has more sources of alternative data to utilize and aims to provide access to financial services to more than “one billion people” by leveraging an expanded dataset, including mobile data and online behavioral patterns.

We have a 10 year head start on what is driving repayment behavior. With every year, our models are getting better and better. So we have that advantage.

LenddoEFL

JKL’s offer to smallholder farmers

JKL finances agricultural assets and credit for smallholder farmers and rural enterprises in rural Kenya and focuses primarily on two lending models:

  1. Group Core Guarantee Mechanism and Solidarity Groups: certain individuals borrow while the rest of the group uses their savings to guarantee the borrowers.
  2. Asset Financing: offers farmers loans to purchase income generating assets that then enable the farmers to repay the loans.

To assess a client’s creditworthiness, JKL employs the CAMPARI4 (Character, Ability, Margin, Purpose, Amount, Repayment, Insurance) credit-scoring model. To measure character, JKL depends on their loan officer to visit a loan applicant’s home, speak with their business associates/friends/family, and observe the applicant’s dynamics with their fellow savings group members.

Delving into human consciousness: LenddoEFL and JKL’s journey

JKL was keen to pilot an objective way of measuring a loan applicant’s character in order to decrease their turnaround time and improve their acceptance rate. They turned to LenddoEFL for assistance, supported by a Mastercard Foundation (MCF) grant. To that end, JKL piloted LenddoEFL’s psychometric-based credit-scoring model in eight of their branches in Kenya for an existing group lending portfolio of 6,000 farmer clients between May 2016 and December 2017.

We were told LenddoEFL could accurately measure character so it was a test, and if it worked then we could start placing more emphasis on people’s character versus collateral.

JKL

Developing a psychometric-based credit score requires iterations of data-driven models, customizing the test for the target audience, and complementing the model with multiple data sources.

That is the problem with data-driven products: you need data before you build a model. You have to take the initial risk to build the company …Starting in a pure commercial relationship, as a startup with no track record, would have been quite difficult.

LenddoEFL

LenddoEFL’s research period at Harvard allowed them to build the required data to prove psychometrics’ predictive power—with external funding—and establish a relationship with their first client, Business Partners Limited (a South African small and medium enterprises [SME] fund). As FSPs provided LenddoEFL with repayment status against loans, LenddoEFL was able to validate their model by correlating high-scoring loan applicants and their on-time repayments. Armed with this evidence, LenddoEFL needed to transition into commercial partnerships. The founders brought on Harvard Business School students to design their business plan, sales, and delivery ability. One of these students is, in fact, DJ DiDonna, one of LenddoEFL’s co-founders. Dr. Klinger notes that without the business/commercial skills of these Harvard Business School students they “would not have gotten off the ground.” Table 1 outlines LenddoEFL’s learning so far in commercializing psychometric tests for lending.

The better models there are and we can share the results, the more clients renew their contract [with us], the more FSPs seem to happily take on this dataset. We see it with our sales discussion, and it is becoming easier.

LenddoEFL
Table 1 Key considerations in developing a psychometric-based credit-scoring model

Customize the tool

Customers have different literacy levels, language abilities, and technological capabilities. To address these differences, LenddoEFL has:

  • reduced the time of the test from 3 hours to 20 minutes,
  • made the tests available in 20 languages,
  • added visual clues and games,
  • and, made the test accessible from a PC, table, or feature/smartphone (SMS).

Draw on multiple sources of data

“Every data source shows something different about the customer.”

Using digital data/mobile usage patterns can help verify customer IDs and complement psychometric datasets. For the JKL pilot, LenddoEFL tested the following variables as predictors of risk to incorporate in the model:

  • personal profile data,
  • business information,
  • comprehension,
  • financial information,
  • business health indicators,
  • non-traditional applicant profile data,
  • and self-efficacy/confidence.

Manage client (FSP) expectations

The LenddoEFL model is based on a specific population and product. The weight will be different for every product and market and can depend on how much data a customer has. Accordingly, models will react differently, even in the same country. Uncertain environments (fragile states or India’s recent demonetization policy) can present challenges. An FSP’s specific environments such as its sourcing, pre-screening, policy changes, and collections process could change an applicant’s behavior.

Design ways to prevent fraud/cheating

Questions are not ‘right’ or ‘wrong’’ LenddoEFL employs tools to verify identity, randomize content, and track partner staff as well as timers and an automated flagging system.

Death by pilot!

The challenge is to get the client to use the product at scale. Clients need to get strategic decision-makers and senior management buy-in from beginning. Some pilots are ‘nice to have’ but do not change internal processes/decision-making.

JKL spent seven months laying the foundation for the pilot, including recruiting and training staff, reviewing internal lending policies, and refining the LenddoEFL model

We were not sure we could work in the agri-business setting with people who live very remotely; we had never done it before. We were pleasantly surprised with the results. Agriculture depends on many other factors such as weather patterns, etc., but we still saw predictive power in our model.

LenddoEFL

JKL initially planned to target LenddoEFL’s model for individual loans, but quickly changed course to focus on their current business needs. This change in direction happened for several reasons, most importantly ensuring that the project had the right in-house expertise to manage the portfolio and was aligned with LenddoEFL’s financial inclusion goals. Accordingly, JKL applied the LenddoEFL model to their existing group loan portfolio loans; averaging approximately $500 and generally exhibiting low debt, for which they have a thorough credit policy.

JKL loan officers conducted 6,656 psychometric tests on tablets during the pilot period. As a result, JKL improved their acceptance rate by 5% and increased the maximum loan amount available from 67% of collateral to 100% of collateral for high-scoring individuals. Moreover, new ‘high-scoring’ clients received, on average, $40 more (the average loan size is $300) than before the LenddoEFL model was implemented and were also offered access to clean energy loans. Low-scoring clients were three times more likely not to repay on time than high-scoring clients. However, in order to achieve these impressive results, JKL had to make significant structural, operational, and technical changes over the course of seven months before implementing the new technology as illustrated in Table 2.

Table 2 Steps JKL took to implement psychometric data

Obtain senior management buy-in

The JKL team had to obtain their board’s approval because the LenddoEFL pilot was a first-of-its-kind for the organization and, if sustained, could result in major structural changes.

Recruit staff

JKL recruited two Project Managers and one Project Coordinator to oversee the pilot on the ground and liaise with LenddoEFL. JKL also hired eight test administrators in each of the eight branches where the pilot took place.

Revise loan policy

JKL relaxed requirements for clients taking LenddoEFL test: collateral requirement decreased from 100% to 70%. JKL also developed a new score (incorporating the LenddoEFL score):

  • > 404 points, low-risk borrowers;
  • 350 and 404 points, medium-risk; and
  • < 350 points, high-risk borrower.

Build infrastructure

JKL bought two tablets for each of their eight pilot branches to administer LenddoEFL tests.

Train test administrators and loan officers

JKL discussed pilot objectives for buy-in, and trained loan officers/administrators on LenddoEFL test.

Sensitize loan applicants to technology

JKL administered the LenddoEFL test on the tablets and the loan officers assisted the applicants in navigating and completing the test.

Key Learnings

While psychometric testing is predictive, using the model on a tablet requires time and money to administer and needs a very clear use case to provide a return on investment

We need to see some loans go bad, because that is the behavior we want to predict in order to be able to detect it, and then avoid it. If we are only seeing loans that perform well, then we cannot create a tailored model that is better than the good models we already have.

LenddoEFL

A major learning from the pilot was that the LenddoEFL credit-scoring model is predictive:

  • 16% of high-risk applicants,
  • 10% of medium-risk applicants, and
  • 5% of low-risk clients fell into arrears during the service period.

However, a major challenge JKL faced in administering the LenddoEFL test was the length of time required to complete the test: users needed assistance to take the test, and test administrators often had to travel long distances between groups to conduct the test. It could take up to an entire day for a group of users to take the tests on two tablets. Additionally, because tablets were shared, the screens were often cracked from usage and they had to be charged frequently. It is important to note that everyone in a given group had to take the test because the model was being used on the group lending portfolio. These challenges also meant that JKL had to pay higher costs than was sustainable in the long term for staff time, travel, and technical infrastructure (i.e., the tablets).

While the psychometric model improved JKL’s acceptance rate and led to an increase in loan size for high scorers, LenddoEFL felt that the pilot did not demonstrate the full value of its assessment. LenddoEFL believed that the pilot was undemonstrative because JKL (a) was not certain as to how it would use the LenddoEFL tool in their group portfolio and (b) did not make the changes necessary to enable group loans to grow. While LenddoEFL’s tool is predictive and has been used by FSPs to generate credit scores, groups can strongly influence the decision-making process and dis/approve members for loans. As a result, JKL had a limited ability to influence the use of the tool given that group dynamic makes it difficult to directly affect the portfolio’s growth or make changes to credit policy based on an individual’s score. For example, JKL’s group self-selects its members and the members guarantee each other’s loans and dictate the group’s growth.

According to LenddoEFL, a portfolio should ideally demonstrate a willingness to grow by replacing certain stringent credit policy criteria with LenddoEFL’s assessment or by reducing the risk in the portfolio by adding LenddoEFL’s credit score as an extra criterion. Consequently, having a clear use case to employ the assessment is critical to its success.

Because of the high costs linked with testing on tablets as well as the lengthy duration of the test, LenddoEFL and JKL piloted an SMS model in parallel. While the SMS pilot did not affect the business case overall for JKL, both LenddoEFL and JKL learned a lot about the cost-effectiveness of the SMS model, which they felt provided a cheaper alternative to traditional credit-scoring models, as discussed further below.

SMS based tests are a cheaper alternative, but still pose challenges

JKL and LenddoEFL piloted an SMS version of the psychometric test in April 2017. They sent 413 messages (tests) and 65% of the recipients completed the tests, a much higher percentage than anticipated. This response gives JKL hope that the SMS test is a better channel for smallholder farmers and a more sustainable solution for the organization. Both organizations found the SMS test more conversational and easier to use, and they appreciated that loan officers could not influence the process. In fact, Equity Bank in Kenya piloted LenddoEFL’s SMS psychometric test and found the model predictive. They plan to integrate it into applicable models across their regional subsidiaries.

While SMS tests are a cheaper and more ubiquitous alternative to tablet/face-to-face tests, knowing that the client taking the SMS test is who they say they are is more challenging than with other channels (tablets or online). Additionally, using SMS tests with low-literate users poses similar challenges to identity. JKL, for instance, is still uncertain whether phones were passed to family/friends. Nevertheless, LenddoEFL is working through these issues to identify whether a person is independently taking the test such as by detecting whether someone rushed through the test or answered with similar patterns.

The results [SMS] were really predictive and we have to assume that people do take it themselves. If it was completely biased (passing phones to someone else), we would not see the same results. The more partners launching SMS, the more we try to find ways to counter the doubts.

LenddoEFL

Business case takeaway: Profitability requires scale

[O]ften …we get compared to a credit bureau. So we have to work to differentiate from them. We see our score as much more valuable—we can score anyone.

LenddoEFL

The merger between EFL and Lenddo has allowed them to bring together more sources of alternative data on one platform at a competitive price, and young startups are approaching them for their services. A key lesson they have learned is that reaching profitability requires scale. Table 3 illustrates the business component models for developing and using psychometric data in lending. FSPs and/or FinTechs that want to build their own psychometric assessment will need up to three teams: to model the test, deliver the assessment, and integrate it into the core banking system; this is detailed under ‘internal costs’ in Table 3.

Table 2 Business models for psychometric data in financial services

Components of business model LenddoEFL JKL
(subscribing to psychometric assessment)

Revenue model

  1. Integration fee, i.e., building the assessment and integrating with client mobile app or web page.
  2. Scorecard fee, i.e., building a customized model for the client (a one-time fee.
  3. Price per score with a minimum number of scores purchased per month; the price per score decreases as volume increases. This is a recurring fee that starts after the scorecard is built.

Pilot was supported by a Mastercard Foundation grant.

Internal cost

Fixed costs to develop the assessment:

  • Modeling team: behavioral scientists and others to choose and develop the questions depending on client segment.
  • Technical infrastructure: to run assessment including cloud storage, etc.
  • Delivery team: build white labeled test, customize assessment, input the translations (if required), run quality assurance, implement the initial model, and put test into production.
  • If the assessment is developed online, a team to train loan officers and work with the FSP’s Risk Department to determine use cases for employing psychometric testing (i.e., increasing acceptance rates in a specific segment).

Costs of tablet-based assessments:

  • Training and recruiting staff,
  • Staff travel time, and
  • Technical infrastructure/ tablets.

KPIs

  • Volume: number of assessments/decisions that FSPs made based on LenddoEFL tool.
  • Performance of Area Under Curve (AUC) risk splits (i.e., defaulters/repayment behavior).

For LenddoEFL pilot:

  • Number of assessments taken by applicants,
  • Loan uptake as metric (i.e., how it performs compared to other products), and
  • Repayment rate.

For company in general:

  • Profit,
  • Uptake of loans, and
  • Impact: feedback/satisfaction surveys and recommendations for product category.

Looking forward

Moving from the subconscious to the conscious

We have been able to assess over seven million people between the two companies and through our credit score, over $2 billion has been lent to individuals who may not have had access to loan products.

LenddoEFL

LenddoEFL is confident that with Lenddo they will reach scale and diversify their market. Moreover, they believe their clients will be able to offer customers meaningful solutions that solve their most pressing problems with LenddoEFL’s scoring model. Nevertheless, as with many relatively new startups, they feel they can create the best models by working with more clients and ultimately solving more uses cases.

Similarly, JKL is excited about the prospect of streamlining psychometric data via SMS into their credit-scoring mechanism. They are particularly proud of their staff and the customers who were willing to try the new technology.

One big challenge we anticipated was this component [willingness to take the test]. Within a few months, we reached our two year target! Clients willing to try new things; partly the clients and partly our strategy.

JKL

In terms of future needs, JKL would like to receive information on a loan applicant’s personality beyond a numerical score. They believe that, by understanding an individual’s personality (i.e., introvert, proactive, etc.) and education/literacy levels, they will be better able to manage their portfolio and will gain the qualitative data necessary to support the psychometric credit score. In this instance, the profiles that LenddoEFL is designing will help JKL meet this need.

Recommendations

FSPs should have a clear use case for psychometric assessment in lending, systems in place to integrate the credit-score model and ensure their target clients are willing to take the assessment, and understand which channel (SMS, tablet, etc.) is most effective for the use case.

Our vision and hope is that it becomes more and more mainstream as a way to get people access [to] the services they need in order to [live] the lives they want to lead. Right now it [psychometric assessments] is still a bit niche and it is changing.

LenddoEFL

Both JKL and LenddoEFL have learned that FSPs must be very clear on what problem psychometric credit scoring could potentially solve. JKL notes that it would be prudent for an FSP to first assess their in-house credit appraisal system to:

  • understand how effective current tools are in assessing an individual’s creditworthiness,
  • identify gaps, and then
  • employ psychometrics if any of the identified gaps relate to measuring character.

Moreover, JKL encourages FSPs to conduct a cost-benefit analysis because a return on investment depends on the volume reached/scale of the program. Likewise, LenddoEFL recommends that FSPs should plan to lend to at least 10,000 loan applicants a year in order to maximize the value of their tool.

LenddoEFL would also like FSPs to consider whether they are willing to learn alongside LenddoEFL because they are constantly iterating their models, and they depend on understanding their clients’ needs and customer segmentation to improve their product. JKL insists that FSPs should think through their customers’ demographics (i.e., literacy levels, technology comfort, etc.) to ascertain their test-taking abilities or the infrastructure required to support the assessment. Further, FSPs will want to feel comfortable asking applicants for the time required to take the assessment and ascertain whether they have the technological infrastructure in place (or can access the technology) to implement the tool—whether that is buying tablets or integrating an SMS model.

Both organizations believe that the future is promising and psychometric data in lending will soon be more commonly used to assess an individual’s creditworthiness.

The more people that learn about it, the more they will understand the good use of it. Psychometrics will be one component of a credit report (borrowing history, financial statement, etc).

JKL

Bibliography

Notes & acknowledgements

Acknowledgements

The author of this case study is Maha Khan, with significant input from Marissa Dean. We thank LenddoEFL and Juhudi Kilimo for their participation in this study. This publication would not have been possible without their valuable insights and stories.

This research was supported by the Mastercard Foundation, and we are grateful to Olga Morawczynski and Mark Wensley for their support and comprehensive input.

Notes

The views presented in this paper are those of the author(s) and the Partnership, and do not necessarily represent the views of the Mastercard Foundation or Caribou Digital.

For questions or comments please contact us at ideas@financedigitalafrica.org.

Recommended citation

Partnership for Finance in a Digital Africa, “Delving into human consciousness: using psychometric assessments in financial services” Farnham, Surrey, United Kingdom: Caribou Digital Publishing, 2018. https://www.financedigitalafrica.org/research/2018/10/delving-into-human-consciousness-using-psychometric-assessments-in-financial-services/.

About the Partnership

The Mastercard Foundation Partnership for Finance in a Digital Africa (the “Partnership”), an initiative of the Foundation’s Financial Inclusion Program, catalyzes knowledge and insights to promote meaningful financial inclusion in an increasingly digital world. Led and hosted by Caribou Digital, the Partnership works closely with leading organizations and companies across the digital finance space. By aggregating and synthesizing knowledge, conducting research to address key gaps, and identifying implications for the diverse actors working in the space, the Partnership strives to inform decisions with facts, and to accelerate meaningful financial inclusion for people across sub-Saharan Africa.

 

This is work is licensed under the Creative Commons AttributionNonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.

Readers are encouraged to reproduce material from the Partnership for Finance in a Digital Africa for their own publications, as long as they are not being sold commercially. We request due acknowledgment, and, if possible, a copy of the publication. For online use, we ask readers to link to the original resource on the www.financedigitalafrica.org website.

  1. Arráiz, Bruhn, and Stucchi, “Psychometric Tests as a Tool to Improve Screening and Access to Credit .”

  2. Militzer, “Unlocking Human Potential: How Psychometric Scoring Can Turbocharge Financial Inclusion.”

  3. Militzer.

  4. CAMPARI is an industry standard model used to assess a loan applicant’s creditworthiness (whether an individual or a business).