Predicting the propensity to succeed among Kenyan “Hustlers”

Originally posted on FSD Kenya website, March 23, 2020

Shujaaz Inc is network of social ventures based in Nairobi. Formerly called Well Told Story, Shujaaz Inc are the producers of “Shujaaz,” an award-winning multi-media youth platform that provides open spaces – including online platforms – for young people to discuss personal and societal issues often considered sensitive or taboo within their communities.

Among the five million Shujaaz fans in Kenya, Shujaaz Inc is particularly interested in the group known as “Hustlers”, a colloquial term for youth at the start of their entrepreneurial journey. Usually 25 years or younger, Hustlers aspire to make a living often with a little more than what they can carry on their backs. Despite the high unemployment rates for youth and the low productivity that plagues the informal sector in Africa, some Hustlers stand out for their abilities to succeed as entrepreneurs. They have the spark and the right mindset, and with well-thought support, these young self-starters could pursue their dream and their vocation while earning a living.

With part survey funding support from FSD Kenya, Shujaaz Inc and BFA Global embarked on a study to understand this demographic group, their aspirations and sustainable ways to help them get ahead.

Diamonds in the rough

Shujaaz Inc recognises that it can leverage its relationship with the Shujaaz fans to help Hustlers succeed in their entrepreneurial journeys. The time-old question faced by Shujaaz Inc and other programmes serving micro and small enterprises (MSEs) has been, “How does one differentiate between future successful entrepreneurs and those who lack the ability or desire to seek growth and scale?” Evidence from acceleration programmes without selection criteria and others that cherry-picked winners seemed to present conflicting results and limited outcomes.

It is within this context that we sat at Shujaaz Inc’ offices in Karen, Nairobi in 2018, and mused, “Can we tell successful Hustlers apart from others based on how they talk? How they think?” Anyone who has ever worked with microentrepreneurs will probably think, “Of course you can! They have this insatiable desire to be their own boss; a drive to build something; are not afraid to take risks; can adapt to change; pull themselves up when they fall.”

Here is what makes the task of predicting high-performing entrepreneurial Hustlers so thrilling. Shujaaz Inc. had two sources of data that we rigorously analysed to validate our hypothesis. First, it had a treasure trove of over a million anonymised Short Message Service (SMS) text messages from ongoing campaigns in which young men and women discussed everything from money to relationships, sex, contraception, and making better decisions. Second, we maintained an active database of respondents who were willing to share their thoughts through SMS surveys. Relying on experience with entrepreneurs and youth across the world, and grounded in the fascinating field of entrepreneurial psychology, we worked closely with the Shujaaz Inc. in-house research team to create a system based on these two sources of information that would allow us to identify Hustlers with a propensity to succeed.

The figure below depicts our approach in identifying and helping Hustlers:

Figure 1: Approach for identifying and helping Hustlers in Shujaaz.

After initially analysing the conversational data carried out over SMS, we followed a five-step process to validate the efficacy of the two data sources:

1. Conduct SMS survey to determine Hustler characteristics based on entrepreneurial psychology.
2. Identify topics that feature in SMS discussions utilising artificial intelligence techniques such as natural language processing (NLP), and correlate with Hustler profiles
3. Conduct a round of phone interviews with selected Hustlers to gain more insight into their lifestyles and businesses.
4. Validate findings through in-person visits.
5. Conduct follow up panel survey to check if “propensity to succeed” tag is indeed predictive over time.

Thus identified, Shujaaz Inc. would invite Hustlers into their Hustla MBA programme, designed as a resource “for and by young Kenyans,” where Hustlers share “tricks and methods they used to build their businesses.” After graduation, successful students would be connected to pre-selected mentors for additional support.

Steps 1-4 were conducted by December 2018. Step 5 was undertaken in November 2019 with support from FSD Kenya.

SMS Survey to determine and validate Hustler characteristics

The SMS survey consisted of 13 questions. The first five determined if the respondent was a Hustler, and if so, if they saw it as a vocation. Questions included “Does/do your hustle(s) enable you to be financially independent?” and “How many months did your longest hustle run for?” Those who had a hustle were then asked eight additional questions to determine their propensity to succeed. These questions were chosen to be relevant to self-identified Hustlers, were SMS friendly, were close-ended, and were asked in random order across respondents. Because we will be operationalising these exact questions, we are unable to share them publicly; we provide the explored characteristics listed in Figure 2 below.

Figure 2: Entrepreneurial characteristics used by Shujaaz Inc to identify in Hustlers

There were 4,724 respondents to the survey (Figure 3), with 44% self-reporting as Hustlers and 37% answering the eight extra questions in a way that indicated that they had entrepreneurial propensity. A total of 20% fit into both categories, i.e. they were self-reported hustlers, and had the propensity to succeed. Further, 1,242 (26%) noted that their hustles allowed them to be financially independent, with 85% of them displaying the propensity to succeed.

Figure 3: Hustler Respondent Categories

Five of the 13 initial screening questions proved to be particularly useful in discerning between different respondent types, for which we could tag respondents with the propensity to succeed (Figure 4). For example, someone who answers all three questions in the affirmative has a 57% ex post probability of having the propensity to succeed, while the corresponding probability for those who answer all three in the negative is 11%. This also implies that respondents in the first group are five times more likely to have the propensity to succeed, compared to the second group.

Figure 4: Predicting Hustler propensity to succeed

Surfacing conversational differences through natural language processing (NLP)

While the SMS survey responses provided us with well-curated content to analyse, it did not capture how young men and women innately thought about their finances, their future, and their lives more broadly. To obtain a window into this, we analysed over 1.3 million SMS messages from more than 250,000 individuals who had conversed with Shujaaz on various topics. Because of the extremely sensitive nature of the content, various safeguards were deployed to protect personal and personally identifiable information.

Natural language processing (NLP) approaches are particularly well suited for this manner of content analysis. Because there is no standard language model for Sheng (a fast-evolving pidgin of English, Kiswahili and other local languages, commonly used by young Kenyans),  we set up an entire NLP artificial intelligence pipeline for this data when we started the exercise. We acknowledged that those models with Kiswahili would have required a lot of effort to custom build. The corpus comprised over 90,000 unique words, which were tokenised, tagged for parts of speech, and lemmatised. We identified the usage of positive and negative words, as well as 19 topics discussed within the Shujaaz community. Some examples of these topics are borrowing, joy, self-improvement, and vice. We also collected metadata, such as unique words used in a message, total messages sent, time of day message was sent, day of week, whether the messages were a part of the conversation, etc.

The results were clear. Hustlers who have the propensity to succeed are more likely to discuss certain topics than those who do not. Self-identified Hustlers talked about four of the topics about 10% more often at statistically significant levels. This modest lift suggested that SMS content analysis would perform as a reliable signal to indicate higher propensity to succeed amongst hundreds of thousands of potential Hustlers. Additional details from this NLP process are not being shared to preserve operational efficacy of the applied techniques.

Verification through in-person visits to Hustlers

In the final step to validate the findings from the first round, we conducted face-to-face interviews with self-reported Hustlers. We asked about their hustle, their attitudes and perceptions about their entrepreneurial prospects.

Interviewees did work running businesses in printing, hotels, transport, operating barber shops and retail shops, or selling second-hand items such as shoes. Almost all of them had an active hustle when we visited them. Most of them identified with almost all the criteria that we associated with a Hustler.

This additional layer of spot checks in late 2018 gave us confidence in the results from the SMS survey and the NLP exercise.

Hustler profile:

“D” had a passion to be an electrician, having grown up without electricity. He started collecting batteries and connecting with a bulb for lighting and that is how he grew his passion.

He left home without his mother’s blessings to try his luck on the streets of Nairobi. His brother rescued him from the streets and took him to his house.

“D” has since been “hustling” as an electrician.

Moment of truth – validation through follow-up panel!

Nevertheless, while we had reasonable confidence that we were on to something, we felt that to be able to really rely on these results, they would have to pass the test of time. Exactly a year later, a follow-up survey was conducted in late 2019, where 1,673 of the original 4,724 responded.

Figure 5: Performance of the propensity to succeed prediction

Thirty-five percent of the respondents were in the same hustle, and 62% of those who had a hustle reported theirs was doing better compared to last year. We are particularly interested in the segment of those who are in the same hustle, and doing better, which is 37% of the respondents, as they not only survived but thrived (Figure 5).

Figure 6: Actual Hustlers’ performance based on surveys

The survey substantiated the utility of the propensity to succeed tag to screen for entrepreneurial ability. These Hustlers – with the same hustle and doing better financially — were half as often more likely to have the “propensity to succeed” tag (from the SMS analysis in Step 1)!

Second, the survey also vindicates the utility of some, though not all of the topics that identified as being correlated to entrepreneurial behaviour. For example:

• Those who are in the same hustle and doing better are 34% less likely to mention a topic related to interpersonal interactions.
• Those who are in a different hustle and doing better are 63% more likely to mention a topic related to economic matters.

These results can be used to calculate combined effects. For example, those who were tagged with propensity to succeed in 2018 but did not mention the topic related to interpersonal interactions have a 45% probability of being in the same hustle and doing better financially compared to 2018.

Methodology shows high promise for predicting entrepreneurial ability

As researchers and implementers, there are two reasons why a predictive methodology is exciting.

First, qualifying one’s propensity to succeed demonstrates that it is quite possible to discern between thin-file clients with varying entrepreneurial abilities based on alternative data. Note that at no point during this process did we resort to collecting data that is typically required by formal financial institutions, such as financial details of the business, licenses, proof of prior experience etc. The message to incumbent financial institutions is therefore not to be shy about relying on alternative data for due diligence, after it has been properly validated.

Second, this engagement represents a great example of mixed method research providing the necessary triangulation to be able to rely on alternative data. Specifically, we combined:

1. Data mining and natural language processing of big data available in the form of SMS content to see if Hustlers talk about different things.
2. Econometric analysis of SMS surveys to obtain various Hustler and non-Hustler profiles.
3. Qualitative research of selected survey responses to validate 1 and 2 above.

While each step alone will have provided useful insight into the lives of Hustlers, it was only when we combined the results together that we felt confident we had something actionable.

Applying propensity to succeed in the real world

These results provide us with sufficient evidence to conclude that properly crafted methodologies based on entrepreneurial psychology can be a powerful and reliable predictor of which Hustlers to not only survive, but to thrive one year later. One interesting insight was that there was no gender or age effect when it comes to thriving Hustlers (Figure 7). What this tells us is that rather than targeting different demographics, such as by age group, or by gender, and even other traits such as geography, that programmes are better suited to improving the matching of individuals who view entrepreneurship as their calling, with the programmatic resources and support that unlock their potential.

Figure 7: Gender-segregated data of Hustlers in same hustle and doing better

Shujaaz Inc, BFA Global and FSD Kenya are currently in discussions with multiple service providers to establish programmes that can provide appropriate financial and non-financial support that can accelerate the journey for youth with a high propensity to succeed; and can potentially help other young people recognise and therefore act on their propensity for future success.

Ashirul Amin, PhD., is a Principal Consultant at BFA Global.

This work was made possible thanks to generous support from FSD Kenya. We also thank Anastasia Mirzoyants, Rob Burnet, and Anuj Tanna at Shujaaz Inc for their feedback on the draft.