The Uneven Impact of Generative AI on Entrepreneurial Performance

Source: OSF Preprints (Otis, Clarke, Delecourt, Holtz, Koning; Berkeley Haas / HBS) · 2025-10-03

Scalable and low-cost AI assistance has the potential to improve firm decision-making and economic performance, particularly in emerging markets. However, running a business involves a wide range of open-ended problems, making it unclear whether and how recent advances in AI can help business owners around the world make better decisions. In a field experiment with Kenyan entrepreneurs, we evaluated the impact of AI advice on small business revenues and profits by randomizing access to a GPT- 4-powered AI business assistant. While we are unable to reject the null hypothesis of no average treatment effect on firm revenues and profits, we find that the effect for entrepreneurs who were low-performing at baseline is nearly 0.25 standard deviations lower than for initial high performers. Subsample analyses show that low performers did nearly 10% worse due to the AI assistant, whereas high performers may have benefited by over 15%. This differential impact does not appear to result from differences in the questions posed to or the advice received from the AI, but rather from which pieces of AI advice entrepreneurs selected and implemented. More broadly, these results show that generative AI is already capable of impacting real-world business performance— though in uneven and sometimes unexpected ways.

Introduction. Since the launch of ChatGPT in November 2022, there has been an explosion of research on generative AI and its potential economic implications (Mollick, 2022; The White House, 2022; Agrawal, Gans, and Goldfarb, 2023; Eloundou et al., 2024; Goldfarb, 2024). Much of this recent work is driven by the belief that conversations with large language models (LLMs) can help people learn and develop new skills (Choi et al., 2023; Mollick and Mollick, 2022), and in work contexts, can improve firm performance and growth (Brynjolfsson, Li, and Raymond, 2025; Dell’Acqua et al., 2023; Noy and Zhang, 2023; Peng et al., 2023; Kumar et al., 2023). Given the substantial variation in worker and firm productivity, both within and between countries, the emergence of almost zero marginal-cost generative AI “assistants” has the potential to radically improve the productivity and performance of everyone, from the thousands of CEOs running companies listed on the New York Stock Exchange to the hundreds of millions of entrepreneurs running small and medium-sized businesses in developing economies (McAfee, Rock, and Brynjolfsson, 2023; Bj ̈orkegren, 2023).

Consistent with the optimism currently surrounding generative AI (AI hereafter), recent experiments show that conversing with AI and receiving AI assistance causes workers to write better business text, including press releases, ad copy, consulting memos, and customer support messages, more quickly (Brynjolfsson, Li, and Raymond, 2025; Dell’Acqua et al., 2023; Doshi and Hauser, 2023; Noy and Zhang, 2023; Chen and Chan, 2023). However, it remains unclear how the benefits of such AI feedback generalize to the broader set of tasks that firms engage in. Beyond these text-based tasks, firms must also manage employees, raise capital, pilot new initiatives, run advertising strategies, price their services, react to competitors, and decide which of these and myriad other tasks to focus their efforts on (Chandler, 1977).

The sheer multitude of tasks involved in running a business greatly increases the complexity of effectively learning how to improve business performance, especially for small businesses and entrepreneurs in emerging markets, which often lack the team, training, and advisors that drive learning (Kim, 2024; Hanna, Mullainathan, and Schwartzstein, 2014). And even when they have access to human advisors and mentors, entrepreneurs often struggle to decide which tasks they should ask for help with, how to formulate effective questions to get useful feedback, and how to interpret and act upon the advice they receive (Bryan, Tilcsik, and Zhu, 2017; Camuffo et al., 2020; Agrawal, Gans, and Stern, 2021; Dimitriadis and Koning, 2022).

It’s possible that these challenges are exacerbated when seeking advice from AI systems.

Entrepreneurs may be reluctant to ask questions of an AI assistant, rather than another person (Lebovitz, Lifshitz-Assaf, and Levina, 2022). Without good questions, even advanced AI systems may fail to provide relevant or useful guidance. And even if an entrepreneur’s question is well-crafted, it is unclear how useful AI advice will be in practice. Given that many business tasks are tacit and may not yet be codified as text—especially tasks in emerging markets—AI tools may lack the relevant training data to provide helpful answers (Autor, 2014; Tao et al., 2023). AI systems can also produce ineffective or overconfident “solutions” that, if implemented, could worsen rather than improve firm performance (Dell’Acqua et al., 2023; Ji et al., 2023), and entrepreneurs may lack the judgment, complementary knowledge, skills, or resources needed to select and implement helpful AI-generated suggestions while avoiding harmful ones (Brynjolfsson and Hitt, 2000; Brynjolfsson, Rock, and Syverson, 2021; Agrawal, Gans, and Stern, 2021). However, if entrepreneurs are able to formulate good questions, if AI provides useful suggestions, and if entrepreneurs can effectively use their judgment to act on this guidance, then AI advice has the potential to help entrepreneurs, especially entrepreneurs in emerging markets.

In this paper, we explore the potential of AI to impact firm performance in a randomized controlled trial (RCT) with 640 Kenyan small business entrepreneurs, half of whom were randomly assigned to receive access to a GPT-4-powered AI assistant over WhatsApp. We are unable to reject the null hypothesis that, on average, access to generative AI had no effect on business performance relative to the control. Although our overall treatment effect estimate is statistically indistinguishable from zero, this result masks substantial treatment effect heterogeneity with respect to pre-treatment firm performance. We estimate that the effect of treatment for entrepreneurs with below-median baseline performance (henceforth “low performers”) is nearly 0.25 standard deviations lower than for entrepreneurs with abovemedian baseline performance (“high performers”).

Related work. Recent experimental studies demonstrate that generative AI can effectively automate and enhance a wide range of well-defined knowledge work tasks. Noy and Zhang (2023) found that US white-collar workers who received generative AI training completed writing tasks approximately 40% faster while simultaneously improving output quality by 18%. Similarly, Dell’Acqua et al. (2023) studied BCG consultants worldwide and showed that those with GPT-4 assistance developed strategic plans 25% faster and received 40% higher quality ratings—with particularly strong benefits for consultants who previously struggled with such tasks. The benefits extend to creative work as well: Doshi and Hauser (2023) demonstrated that UK participants wrote more creative and engaging short stories when working with AI- generated ideas, while Meincke et al. (2024) found that consumer product ideas generated entirely by AI outperformed those created by students at an elite US university.1 Collectively, these findings provide compelling evidence for AI’s substantial economic potential in knowledge work.

However, these promising task-level findings raise crucial questions about how AI will operate in real-world organizational settings. Unlike the precisely controlled experimental conditions in many studies, as Noy and Zhang (2023) observe, “real-world tasks involve vaguer objectives and instructions, requiring workers to exercise initiative in determining” where to apply AI tools. While some workplaces mirror these structured experimental settings, such as the standardized call center studied by Brynjolfsson, Li, and Raymond (2023), most organizations operate with far less structure. In these varied environments, from small businesses in India to AI R&D labs in San Francisco, workers must first decide whether and where to deploy AI assistance. This discretion matters significantly: Dell’Acqua et al. (2023) document that AI’s impact can be strongly positive or negative depending on the specific tasks for which it is employed. Consequently, variation in how much discretion workers and firms exercise over AI deployment may substantially shape both the technology’s estimated impact and its distributional consequences.

Consistent with these ideas, a substantial literature dating back to decision support systems research demonstrates that differences in judgment, incentives, and organizational As a result, to understand AI’s potential impact on firms and workers, we must study both well-defined knowledge tasks and “messier settings” where workers and firms exercise discretion over AI deployment and learning. Entrepreneurship offers an especially promising context for this broader evaluation.

Method. 3.1 Study Setting Our study was carried out in the East African country of Kenya, where about 80% of employment takes place in the informal sector (Kenya National Bureau of Statistics, 2023).

Kenya is also a leading adopter of digital technologies in Africa. For instance, it was one of the first countries to have large-scale adoption of mobile money, with M-Pesa—launched in 2007—reaching more than 34 million average monthly users in 2024 (Safaricom PLC, 2024). Kenya also has had one of the highest cell phone ownership rates in the region, with approximately 95% of adults owning a mobile phone (Kamau et al., 2024), most of whom use WhatsApp for text communication (Shengale, 2025).

Kenya’s widespread digital infrastructure and adoption make it an ideal setting for studying the impacts of a WhatsApp-based generative AI assistant, as participants are already familiar with interacting through such platforms. Additionally, without the foundation of widespread mobile access, the delivery of our intervention would not have been possible, highlighting an important boundary condition for applying similar interventions in other emerging market contexts (Callen et al., 2025).

This recruitment and screening process produced a final sample of 640 Kenya SMB entrepreneurs. The sample is geographically diverse: participants came from 44 of Kenya’s 47 counties, with the largest shares from Nairobi (133), Kiambu (53), and Nakuru (47) (see Figure A1). The median entrepreneur was 26 years old, and most had at least some college education, with only 10% reporting no college experience. Women made up 32% of the sample, consistent with lower rates of female business ownership in Kenya (The World Bank Group, 2018). The businesses represented in the sample ranged from fast food vendors to poultry farmers to cybershop owners (who provide services such as internet access, printing, scanning, and computer support). The most common sectors in the sample were Food and Beverage (27%), Clothing (21%), and Agriculture (19%).

The median entrepreneur in the sample had been running their business for one year or less and had no employees. Reflecting both the emerging-market context and these firms’ limited operating histories, entrepreneurs reported modest pre-treatment business performance. The average pre-treatment monthly profit was about KSh 19,550, and reported monthly profits ranged from about KSh 2,650 at the 5th percentile to about KSh 55,100 at the 95th percentile.

3.3 Experimental Interventions Following the third pre-treatment survey, participants were randomly assigned to either the treatment group—which received access to the generative AI-powered business assistant—or the control group, which received access to a set of publicly available business training guides, both of which are discussed in more detail below. To improve the statistical precision of our experiment, we stratified random assignment by quartiles of pre-treatment business performance and by gender. As expected under randomization, the two groups were comparable across a range of baseline characteristics and performance measures (Table A1).2 3.3.1 Treatment (The AI Business Assistant) The generative AI-powered business assistant was built on GPT-4, a large language model released by OpenAI in March 2023 (OpenAI, 2023). To ensure accessibility, the AI assistant was deployed through WhatsApp, a messaging platform already widely familiar to participants in Kenya. Access was restricted so that the assistant would only respond to phone numbers belonging to participants in the treatment group, ensuring that participants in the 2post-treatment attrition was also similar across conditions. We classify participants as having attritted if they failed to complete the final survey, even if they completed some earlier post-treatment surveys. Of the 640 study participants, 11 attritted from the AI-assistant group and 7 from the control group. The 1.2 percentage point difference in response rates between the two conditions is not statistically significant (p “ 0.35). control group could not use the tool.

Interacting with the assistant differed from directly using ChatGPT in three key ways.

Discussion. 4.2 Treatment Effect Heterogeneity Building on recent studies showing that AI assistance especially helps lower performing workers (Dell’Acqua et al., 2023; Noy and Zhang, 2023; Peng et al., 2023; Brynjolfsson, Li, and Raymond, 2025), we next estimate heterogeneous treatment effects by splitting our sample into two groups, based on the pre-treatment values of the same performance index we use as our primary outcome variable.5 We refer to entrepreneurs who had below-median pretreatment performance as low performers and those who had above median pre-treatment performance as high performers.

Estimates B and C in Figure 3 and Table 1 show the estimated treatment effect within the low- and high-performer sub-samples. For low performers, access to the AI assistant reduced average business performance relative to the control group (β “ ́0.08 s.d., p “ 0.01, 95% CI = [-0.13, -0.03]). Because our index is standardized against the standard deviation of the post-treatment control group (Kling, Liebman, and Katz, 2007), we can contextualize the magnitude of our effect by looking at the mean and standard deviation of revenues and profits for this group. For example, Table A2 shows average monthly revenues are 51,392 Kenyan Shillings (USD $360.90) with a standard deviation of 59,353 Shillings.6 Multiplying the standard deviation by our ́0.08 s.d. estimate suggests that the AI assistant caused a drop of 4,748 Shillings ($33.34) in revenue per month; a 9.2% relative decrease. Regardless of whether we perform this contextualization exercise with pre-treatment measures or posttreatment measures, revenue or profits, they consistently yield a negative treatment effect estimate of just under 10% for low performers.

For high performers, we find suggestive evidence that AI assistance improved performance relative to control (β “ 0.16 s.d., p “ 0.07, 95% CI = r ́0.02, 0.33s). Again we multiply the standard deviation of monthly revenue by our estimate, now a 0.16 s.d. increase, which suggests that AI caused an increase of 9,496 Shillings (USD $66.69) in monthly revenue; a 18% relative increase. Again, whether we use pre-treatment and post-treatment measures to contextualize the effects’ sizes, they consistently produce a positive treatment effect estimate of over 15% for high performers.

Building on these subgroup analyses, Estimate D in Figure 3 and Table 1 directly compares the impact of the AI assistant on initially low and high performers using the heterogeneous treatment effects model in Equation 2. We find strong evidence that the effect of AI is different for low and high performers. Initially, low-performing entrepreneurs experienced a 0.23 sd lower causal effect (p “ 0.01, 95% CI = r ́0.41, ́0.05s) of the AI assistant compared to high performers.7 In monetary terms, the estimated difference in effect size between the two groups represents a change in in the impact of AI on a firm’s monthly revenue of roughly 13,651 Shillings (USD $95.86).

To better understand why low and high performers exhibit different performance effects, we follow an abductive approach and use the rich text data we collected from interactions with the AI assistant and our surveys to shed light on the potential drivers of our starkly uneven performance effects (Pillai, Goldfarb, and Kirsch, 2024). As noted in our literature review, prior research on generative AI’s economic impacts typically involves structured tasks. In contrast, our participants had considerable discretion in how and when they used AI assistance. Participants not only had access to AI output, but could choose to apply it to myriad different tasks. They then had to exercise considerable judgment in deciding when to implement the AI’s advice.