Using psychometric assessments in financial services

Introduction and Summary

Everyone has a unique and distinctive personality. Fortunately for low-income populations that may have been excluded from formal financial services, their character traits can now help them access financial products. A psychometric credit assessment provides an alternative for thin-file loan applicants (i.e., zero or low credit history) by generating credit scores based on personality and behavior. These assessments predict loan repayment behavior by measuring an applicant’s attitude, integrity, and performance. Beginning in 2006, innovative firms like LenddoEFL (formerly Lenddo and Entrepreneurial Finance Lab [EFL]) were among the first to pioneer psychometric assessments for lending in emerging markets. More than ten years later, financial service providers, like Te Creemos (Mexico), are leveraging psychometric data in their credit-decision making processes.

This case study will delve into LenddoEFL’s journey in developing psychometric data for financial services and, in parallel, explore the experience of Juhudi Kilimo (JKL), a Kenyan microfinance institution, in implementing LenddoEFL’s credit-scoring model.

Interviews with LenddoEFL and JKL reveal that the credit-scoring model increased JKL’s acceptance rate by 5% and predicted clients’ repayment rates/borrowing behavior. Further, both organizations recommend that financial service providers (FPSs) have a clear use case for employing psychometric credit assessments, and ensure that the tool helps them solve a business challenge they face in their current credit appraisal process, before implementing LenddoEFL’s credit-scoring model. Financial service providers (FSPs) would also benefit from conducting an assessment of their own appraisal system to identify how to integrate a psychometric tool into their technical systems. The interviews further suggest that FSPs should gauge their target clients’ willingness to take a psychometric assessment, as tests may take a long time to administer to clients.

This study is part of FiDA’s broader exploration of the promise and limitations of big data analytics in financial services. It expands on the insights presented in FiDA’s Focus Note, Can big data shape financial services in East Africa? A number of the FSPs and FinTechs interviewed for the Focus Note pointed to the value of employing psychometric testing to accelerate financial services, particularly given the dearth of commercially available traditional data in sub-Saharan Africa that can be used to push digital finance forward. The case study endeavors to provide FSPs interested in leveraging non-traditional, alternative data with potential paths for using psychometric data. The case study explores the lessons that LenddoEFL learned in designing and evaluating psychometric testing on thin-file clients, and Juhudi Kilimo’s pilot of psychometric testing provides an early example of how FSPs and FinTechs can begin working with psychometric testing.

We are all passionate about psychometric data because [it is the] only dataset that everyone has. [It is the] Main reason why we decided to leverage it. Everyone has a personality and can answer the questions….most inclusive form of credit scoring.

LenddoEFL’s entry to psychometric testing

LenddoEFL began as a research project at the Harvard Center for International Development by Drs. Bailey Klinger and Asim Khwaja who developed a survey to test behavioral traits in South Africa and “solve the problem of information opacity for micro and small businesses applying for loans.” They initially lent to borrowers without psychometric data and, with repayment data under their belt, they tested the relationship between an individual’s psychometric profile and loan performance. After almost four years of research, LenddoEFL officially started offering FSPs psychometric-based credit scores. They have since developed more than 50 models that measure personal initiative, situational judgment, creativity, and business acumen, and these models adapt to country and community contexts. FSPs can either use LenddoEFL’s tool alone to assess creditworthiness or enhance their existing underwriting process by using LenddoEFL’s tool as an additional metric.

In late 2017, EFL merged with the technology company Lenddo. Lenddo has developed patented technology that employs non-traditional digital data to develop credit scores and verification of its applicants. The merged company, LenddoEFL, now has more sources of alternative data to utilize and aims to provide access to financial services to more than “one billion people” by leveraging an expanded dataset, including mobile data and online behavioral patterns.

We have a 10 year head start on what is driving repayment behavior. With every year, our models are getting better and better. So we have that advantage.

JKL’s offer to smallholder farmers

JKL finances agricultural assets and credit for smallholder farmers and rural enterprises in rural Kenya and focuses primarily on two lending models:

1. Group Core Guarantee Mechanism and Solidarity Groups: certain individuals borrow while the rest of the group uses their savings to guarantee the borrowers.
2. Asset Financing: offers farmers loans to purchase income generating assets that then enable the farmers to repay the loans.

To assess a client’s creditworthiness, JKL employs the CAMPARI⁴ (Character, Ability, Margin, Purpose, Amount, Repayment, Insurance) credit-scoring model. To measure character, JKL depends on their loan officer to visit a loan applicant’s home, speak with their business associates/friends/family, and observe the applicant’s dynamics with their fellow savings group members.

Delving into human consciousness: LenddoEFL and JKL’s journey

JKL was keen to pilot an objective way of measuring a loan applicant’s character in order to decrease their turnaround time and improve their acceptance rate. They turned to LenddoEFL for assistance, supported by a Mastercard Foundation (MCF) grant. To that end, JKL piloted LenddoEFL’s psychometric-based credit-scoring model in eight of their branches in Kenya for an existing group lending portfolio of 6,000 farmer clients between May 2016 and December 2017.

We were told LenddoEFL could accurately measure character so it was a test, and if it worked then we could start placing more emphasis on people’s character versus collateral.

Developing a psychometric-based credit score requires iterations of data-driven models, customizing the test for the target audience, and complementing the model with multiple data sources.

That is the problem with data-driven products: you need data before you build a model. You have to take the initial risk to build the company …Starting in a pure commercial relationship, as a startup with no track record, would have been quite difficult.

LenddoEFL’s research period at Harvard allowed them to build the required data to prove psychometrics’ predictive power—with external funding—and establish a relationship with their first client, Business Partners Limited (a South African small and medium enterprises [SME] fund). As FSPs provided LenddoEFL with repayment status against loans, LenddoEFL was able to validate their model by correlating high-scoring loan applicants and their on-time repayments. Armed with this evidence, LenddoEFL needed to transition into commercial partnerships. The founders brought on Harvard Business School students to design their business plan, sales, and delivery ability. One of these students is, in fact, DJ DiDonna, one of LenddoEFL’s co-founders. Dr. Klinger notes that without the business/commercial skills of these Harvard Business School students they “would not have gotten off the ground.” Table 1 outlines LenddoEFL’s learning so far in commercializing psychometric tests for lending.

The better models there are and we can share the results, the more clients renew their contract [with us], the more FSPs seem to happily take on this dataset. We see it with our sales discussion, and it is becoming easier.

Table 1 Key considerations in developing a psychometric-based credit-scoring model

JKL spent seven months laying the foundation for the pilot, including recruiting and training staff, reviewing internal lending policies, and refining the LenddoEFL model

We were not sure we could work in the agri-business setting with people who live very remotely; we had never done it before. We were pleasantly surprised with the results. Agriculture depends on many other factors such as weather patterns, etc., but we still saw predictive power in our model.

JKL initially planned to target LenddoEFL’s model for individual loans, but quickly changed course to focus on their current business needs. This change in direction happened for several reasons, most importantly ensuring that the project had the right in-house expertise to manage the portfolio and was aligned with LenddoEFL’s financial inclusion goals. Accordingly, JKL applied the LenddoEFL model to their existing group loan portfolio loans; averaging approximately $500 and generally exhibiting low debt, for which they have a thorough credit policy.

JKL loan officers conducted 6,656 psychometric tests on tablets during the pilot period. As a result, JKL improved their acceptance rate by 5% and increased the maximum loan amount available from 67% of collateral to 100% of collateral for high-scoring individuals. Moreover, new ‘high-scoring’ clients received, on average, $40 more (the average loan size is $300) than before the LenddoEFL model was implemented and were also offered access to clean energy loans. Low-scoring clients were three times more likely not to repay on time than high-scoring clients. However, in order to achieve these impressive results, JKL had to make significant structural, operational, and technical changes over the course of seven months before implementing the new technology as illustrated in Table 2.

Table 2 Steps JKL took to implement psychometric data

Key Learnings

While psychometric testing is predictive, using the model on a tablet requires time and money to administer and needs a very clear use case to provide a return on investment

We need to see some loans go bad, because that is the behavior we want to predict in order to be able to detect it, and then avoid it. If we are only seeing loans that perform well, then we cannot create a tailored model that is better than the good models we already have.

A major learning from the pilot was that the LenddoEFL credit-scoring model is predictive:

• 16% of high-risk applicants,
• 10% of medium-risk applicants, and
• 5% of low-risk clients fell into arrears during the service period.

However, a major challenge JKL faced in administering the LenddoEFL test was the length of time required to complete the test: users needed assistance to take the test, and test administrators often had to travel long distances between groups to conduct the test. It could take up to an entire day for a group of users to take the tests on two tablets. Additionally, because tablets were shared, the screens were often cracked from usage and they had to be charged frequently. It is important to note that everyone in a given group had to take the test because the model was being used on the group lending portfolio. These challenges also meant that JKL had to pay higher costs than was sustainable in the long term for staff time, travel, and technical infrastructure (i.e., the tablets).

While the psychometric model improved JKL’s acceptance rate and led to an increase in loan size for high scorers, LenddoEFL felt that the pilot did not demonstrate the full value of its assessment. LenddoEFL believed that the pilot was undemonstrative because JKL (a) was not certain as to how it would use the LenddoEFL tool in their group portfolio and (b) did not make the changes necessary to enable group loans to grow. While LenddoEFL’s tool is predictive and has been used by FSPs to generate credit scores, groups can strongly influence the decision-making process and dis/approve members for loans. As a result, JKL had a limited ability to influence the use of the tool given that group dynamic makes it difficult to directly affect the portfolio’s growth or make changes to credit policy based on an individual’s score. For example, JKL’s group self-selects its members and the members guarantee each other’s loans and dictate the group’s growth.

According to LenddoEFL, a portfolio should ideally demonstrate a willingness to grow by replacing certain stringent credit policy criteria with LenddoEFL’s assessment or by reducing the risk in the portfolio by adding LenddoEFL’s credit score as an extra criterion. Consequently, having a clear use case to employ the assessment is critical to its success.

Because of the high costs linked with testing on tablets as well as the lengthy duration of the test, LenddoEFL and JKL piloted an SMS model in parallel. While the SMS pilot did not affect the business case overall for JKL, both LenddoEFL and JKL learned a lot about the cost-effectiveness of the SMS model, which they felt provided a cheaper alternative to traditional credit-scoring models, as discussed further below.

SMS based tests are a cheaper alternative, but still pose challenges

JKL and LenddoEFL piloted an SMS version of the psychometric test in April 2017. They sent 413 messages (tests) and 65% of the recipients completed the tests, a much higher percentage than anticipated. This response gives JKL hope that the SMS test is a better channel for smallholder farmers and a more sustainable solution for the organization. Both organizations found the SMS test more conversational and easier to use, and they appreciated that loan officers could not influence the process. In fact, Equity Bank in Kenya piloted LenddoEFL’s SMS psychometric test and found the model predictive. They plan to integrate it into applicable models across their regional subsidiaries.

While SMS tests are a cheaper and more ubiquitous alternative to tablet/face-to-face tests, knowing that the client taking the SMS test is who they say they are is more challenging than with other channels (tablets or online). Additionally, using SMS tests with low-literate users poses similar challenges to identity. JKL, for instance, is still uncertain whether phones were passed to family/friends. Nevertheless, LenddoEFL is working through these issues to identify whether a person is independently taking the test such as by detecting whether someone rushed through the test or answered with similar patterns.

The results [SMS] were really predictive and we have to assume that people do take it themselves. If it was completely biased (passing phones to someone else), we would not see the same results. The more partners launching SMS, the more we try to find ways to counter the doubts.

Business case takeaway: Profitability requires scale

[O]ften …we get compared to a credit bureau. So we have to work to differentiate from them. We see our score as much more valuable—we can score anyone.

The merger between EFL and Lenddo has allowed them to bring together more sources of alternative data on one platform at a competitive price, and young startups are approaching them for their services. A key lesson they have learned is that reaching profitability requires scale. Table 3 illustrates the business component models for developing and using psychometric data in lending. FSPs and/or FinTechs that want to build their own psychometric assessment will need up to three teams: to model the test, deliver the assessment, and integrate it into the core banking system; this is detailed under ‘internal costs’ in Table 3.

Table 2 Business models for psychometric data in financial services

Looking forward

Moving from the subconscious to the conscious

We have been able to assess over seven million people between the two companies and through our credit score, over $2 billion has been lent to individuals who may not have had access to loan products.

LenddoEFL is confident that with Lenddo they will reach scale and diversify their market. Moreover, they believe their clients will be able to offer customers meaningful solutions that solve their most pressing problems with LenddoEFL’s scoring model. Nevertheless, as with many relatively new startups, they feel they can create the best models by working with more clients and ultimately solving more uses cases.

Similarly, JKL is excited about the prospect of streamlining psychometric data via SMS into their credit-scoring mechanism. They are particularly proud of their staff and the customers who were willing to try the new technology.

One big challenge we anticipated was this component [willingness to take the test]. Within a few months, we reached our two year target! Clients willing to try new things; partly the clients and partly our strategy.

In terms of future needs, JKL would like to receive information on a loan applicant’s personality beyond a numerical score. They believe that, by understanding an individual’s personality (i.e., introvert, proactive, etc.) and education/literacy levels, they will be better able to manage their portfolio and will gain the qualitative data necessary to support the psychometric credit score. In this instance, the profiles that LenddoEFL is designing will help JKL meet this need.

Recommendations

FSPs should have a clear use case for psychometric assessment in lending, systems in place to integrate the credit-score model and ensure their target clients are willing to take the assessment, and understand which channel (SMS, tablet, etc.) is most effective for the use case.

Our vision and hope is that it becomes more and more mainstream as a way to get people access [to] the services they need in order to [live] the lives they want to lead. Right now it [psychometric assessments] is still a bit niche and it is changing.

Both JKL and LenddoEFL have learned that FSPs must be very clear on what problem psychometric credit scoring could potentially solve. JKL notes that it would be prudent for an FSP to first assess their in-house credit appraisal system to:

• understand how effective current tools are in assessing an individual’s creditworthiness,
• identify gaps, and then
• employ psychometrics if any of the identified gaps relate to measuring character.

Moreover, JKL encourages FSPs to conduct a cost-benefit analysis because a return on investment depends on the volume reached/scale of the program. Likewise, LenddoEFL recommends that FSPs should plan to lend to at least 10,000 loan applicants a year in order to maximize the value of their tool.

LenddoEFL would also like FSPs to consider whether they are willing to learn alongside LenddoEFL because they are constantly iterating their models, and they depend on understanding their clients’ needs and customer segmentation to improve their product. JKL insists that FSPs should think through their customers’ demographics (i.e., literacy levels, technology comfort, etc.) to ascertain their test-taking abilities or the infrastructure required to support the assessment. Further, FSPs will want to feel comfortable asking applicants for the time required to take the assessment and ascertain whether they have the technological infrastructure in place (or can access the technology) to implement the tool—whether that is buying tablets or integrating an SMS model.

Both organizations believe that the future is promising and psychometric data in lending will soon be more commonly used to assess an individual’s creditworthiness.

The more people that learn about it, the more they will understand the good use of it. Psychometrics will be one component of a credit report (borrowing history, financial statement, etc).