Can organizations safely deploy inscrutable AI models?

Asatiani, Malo, Nagbøl, Penttinen, Rinta-Kahila, and Salovaara (2021) argue that an organization does not need to make an inscrutable machine-learning model explainable in order to deploy it safely; it can instead "envelop" the model. Drawing on the concept of envelopment, they describe methods for "establishing clear boundaries within which the AI is to interact with its surroundings, choosing and curating the training data well, and appropriately managing input and output sources." Their case, Denmark's Business Authority (DBA), used neural-network models whose "operation logic differs dramatically from humans' ways of approaching similar problems," making the models inscrutable by the paper's definition — "hard for humans to interpret and explain" — yet still governable through the envelope placed around them rather than through any opening of the model itself.

The paper's mechanism is to separate the problem of understanding a model from the problem of controlling it. Because "ML models, by necessity, operate mindlessly," with "no conscious understanding of the broader context," and because approaches such as legal auditing, robust system design, and user education are "unidimensional and inadequate for tackling the fundamental challenges presented by" that mindlessness, the authors locate safety in what surrounds the model: what data trains it, what inputs reach it, and what its outputs are permitted to do. Critically, they argue this boundary-setting "is far more than a technical matter — to be effective, it has to be situated at the intersection of the technical and the social," with "human agents... an integral part of envelopment, responsible for defining suitable envelopes as well as maintaining and renegotiating them." At DBA, this showed up in case workers, domain experts who evaluate each model's operational correctness and are "charged with... its retraining as needed."

This extends the move made by Are large AI models actually intelligent agents or social technologies?, which treats large models as institutions to be managed rather than agents to be explained; envelopment supplies a concrete organizational technique for that management stance, specified years before the LLM-agent framing existed. It also sits in useful tension with Does granting agents more autonomy undermine human oversight?: that paper argues autonomy and use themselves erode the human capacity to oversee, while this paper's DBA case shows a boundary-setting process — negotiated upfront, revisited by named roles such as case workers — that keeps human oversight intact even as the model's own behavior stays inscrutable. The difference may be scale and kind of autonomy: a bounded classification or fraud-detection model admits the kind of envelope that an open-ended agentic system may not.

The excerpt is a single exploratory case study of one government agency's ML models, not a general design method validated across settings; the authors themselves note they "did not find evidence pertaining to function envelopment" and could not examine the strategies' "long-term implications." It also predates agentic and generative AI: DBA's models are described as fixed, task-specific neural networks performing document checks, not agents that plan or act over many steps, so envelopment's methods — curated training data, bounded inputs and outputs — may not transfer cleanly to systems whose boundary is harder to specify in advance. What the paper does establish is narrower but still useful: that organizations have already treated "govern the boundary" as a workable substitute for "understand the model," and that doing so required naming specific human roles accountable for drawing and redrawing that boundary.