On the Societal Impact of Open Foundation Models

Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate.

Introduction. Foundation models (Bommasani et al., 2021) are the centerpiece of the modern AI ecosystem, catalyzing a frenetic pace of technological development, deployment, and adoption that brings with it controversy, scrutiny, and public attention. Open foundation models1 like BERT, CLIP, Whisper, BLOOM, Pythia, Llama 2, Falcon, Stable Diffusion, Mistral, OLMo, Aya, and Gemma play an important role in this ecosystem. These models allow greater customization and deeper inspection of how they operate, giving developers greater choice in selecting foundation models. However, they may also increase risk, especially given broader adoption, which has prompted pushback, especially around risks relating to biosecurity, cybersecurity, and disinformation. How to release foundation models is a central debate today, often described as open vs. closed. Simultaneously, policymakers are confronting how to govern open foundation models.

Discussion / Conclusion. Open foundation models are controversial due to fundamental philosophical disagreements, fragmented conceptual understanding, and poor empirical evidence. Our work aims to rectify the conceptual confusion by clearly defining open foundation models, identifying their distinctive properties, and clarifying their benefits and risks. While it is unlikely that certain underlying philosophical tensions will ever be resolved, especially when inextricably intertwined with the incentives of different actors in the AI space, we encourage future work to address today’s deficits in empirical evidence. Overall, we are optimistic that open foundation models can contribute to a vibrant AI ecosystem, but realizing this vision will require significant action from many stakeholders.