Just-In-Time Objectives: A General Approach for Specialized AI Interactions

Large language models promise a broad set of functions, but when not given a specific objective, they default to generic results. We demonstrate that inferring the user’s in-the-moment objective, then rapidly optimizing for that singular objective, enables LLMs to produce specialized tools, interfaces, and responses. Our work introduces just-in-time objectives, which model a user’s goals to specialize LLM systems on the fly. We contribute an architecture for automatically inducing such objectives by passively observing user behavior, then steering downstream AI systems through generation and evaluation against this objective. Inducing just-in-time objectives (e.g., “Clarify the abstract’s research contribution”) enables automatic generation of tools, e.g., those that critique a draft based on relevant HCI methodologies, anticipate related researchers’ reactions, or surface ambiguous terminology. In a series of experiments on participants’ own tasks, JIT objectives enable LLM outputs that achieve 66–86% win rates over typical LLMs. In-person use sessions confirm that JIT objectives produce specialized tools that are unique to each participant and are rated as significantly higher quality than a standard LLM chat tool.

Introduction. In the course of writing this paper, we wondered what a large language model (LLM) might have to say about our introduction. The model offered helpful syntax edits (“replace technical phrases like hill-climb”), suggestions to aid readability (“some sentences could be broken up”), and thoughts on re-ordering content (“consider moving implementation specifics to a later section”). These suggestions all constitute useful advice, but we found them rather milquetoast and unopinionated. Certainly a seasoned researcher would have more specific advice on our core argument. Why do we have such generic LLM output? One major factor is that training objectives for an LLM must be defined well in advance of the contexts where a model will be applied, so they must make assumptions about what users need. Model post-training aims to make up for this lack of certainty by fine-tuning models against many simultaneous objectives, such as performance on reasoning tasks, value alignment, and preference data labeled by people with diverse opinions. However, like a design by committee, optimizing jointly over these objectives has the effect of converging the model to generic, safe outputs [9, 48, 51, 54]. These generic objectives form a default that is rarely overridden even at interaction time: users often have difficulties envisioning model behavior [60], struggle with underspecified LLM prompts [1, 50, 75], and still end up converging to generic ideas [2, 33]. In this paper, we demonstrate that inferring the user’s in-themoment objective, then rapidly optimizing the LLM’s behavior toward that singular objective, enables the LLM to generate outputs that are far more distinct and responsive: whether they be individual responses, extended drafts, or generated software tools. Instead of relying on an anticipatory objective that must be generic and opaque by default, we find that we can pursue an on-demand objective that is specific and malleable by default. User objectives over long stretches of time (e.g., a user’s writing philosophy) are complex and challenging to anticipate, which causes standard preemptive LLM objectives to fail. But when objectives are created on-demand based on small windows of observed user behavior, they become more tractable and accurate (e.g., a user’s aims for this phase of the Introduction).1 We capture these instantaneous user goals as what we term just-in-time objectives: for example, “clarify the calculus example in the introduction,” “restructure the argument logic,” or “respond to Reviewer 2’s critique about sampling.” By inducing possible objectives at interaction time, these objectives can be much more detailed and specialized to the particular task at hand compared to implicit, generic AI objectives. We contribute a generalizable architecture for user-specific LLM specialization that automatically induces just-in-time (JIT) objectives from interaction traces, then applies the JIT objective to steer AI behavior by shaping LLM generation and evaluation. This architecture induces strong shifts in model behavior across task domains: we demonstrate that applying a JIT objective in a generate-thenrank architecture enables LLMs to rapidly construct tools that are specific and useful to the user in the moment. Recall that a standard LLM provided generic syntax and composition advice on an introduction like this one. Instead, applying an induced just-in-time objective of “explain the system clearly in this CHI paper introduction” enables a system to automatically produce drafts that rework this paragraph’s logical flow to match that of similar introduction paragraphs from related CHI papers, or responses that simulate feedback from likely CHI reviewers, or tools that identify where the narrative veers away from describing the system. We instantiate our approach in a tool called Poppins: a browser extension and web application that observes user screens to infer just-in-time objectives, which allows the system to automatically produce detailed candidate tool designs and generate novel interactive tools to assist the user within minutes. The system captures the user’s browser context, and uses that context to infer a just-in-time objective (e.g., “enhance introduction draft by clarifying system architecture”). This objective steers Poppins to generate and evaluate candidate design specifications, then passes the best design specification to produce a functional software tool for the user. To understand whether our just-in-time objectives are both accurate and useful, we conduct a series of evaluations that assess the objectives in isolation and then incorporate them in increasingly complex LLM systems.

Related work. 2.1 Failure Modes of Human-LLM Interaction: Undirected Prompting and Generic Outputs Despite the excitement and hopefulness around large language models and chat-based interaction, recent work highlights critical cognitive barriers when interacting with LLMs with monolithic and opaque objectives. Ubiquitous prompt- and chat-based LLM user interfaces provide limited affordances to understand black-box LLMs, preventing users from forming reliable mental models to control system behavior [1]. Returning to the classic Gulfs of Execution and Evaluation [27], related work locates new gulfs that may emerge for generative AI, such as a Process Gulf in understanding how AI executes tasks [62] and a Gulf of Envisioning in setting the right goals in the first place when using AI [60]. Natural language interaction is not a cure-all for human-AI interaction, but in fact leads to frustrations with underspecified prompts [50, 75]. We are motivated by many of these cognitive barriers that arise from black-box LLMs. Our work argues that rather than relying on unstructured and noisy natural language requests, users may benefit from a more explicit construct of objectives that steer the model and transparently convey the model’s current directive. Another strong motivator of our work is the problem of generic LLMs. Emerging research presents evidence of the homogenizing impacts of generic LLM outputs for both individual-level [9, 33, 34, 44] and population-level creativity [2, 15, 38, 66, 67]. From LLMs, we observe an uncanny valley of model behaviors that appear divergent, but are not fully random, and yet effortful to meaningfully steer [63]. Worryingly, at a population level, LLMs appear to promote monocultures, steering users towards homogeneous, convergent thinking even when individual outputs appear creative [2, 15]. One response is that we need pluralistic modeling approaches that better represent the diversity of the population [16, 17] and account for pluralistic values [57, 58]. Another response is that we need mechanisms for greater and more sustainable participation in the development and maintenance of LLMs [26, 61]. We share a concern about the risks of current LLM tools promoting monocultures, and our work explores whether even generic models can be effectively coaxed into specific, divergent variants with just-in-time objectives. In contrast with approaches that specialize LLM behavior with domain- or task-specific context [35, 65], our approach aims for user-specific specialization that can generalize across tasks and domains. We posit that if we can grant users on-demand access to specific, divergent model behaviors, they may be able to break away from generic model outputs and actualize their unique creative voice.

Method. 3 JUST-IN-TIME OBJECTIVES With a critical role in shaping model behavior from conversational chatbots to code assistants to writing tools, large language model objectives are often fixed far in advance of the scenarios they will encounter. In-context prompting can overcome this issue, but users are rarely detailed enough in their request to cover all required decisions [50]. In this section, we describe an architecture that shifts the work of objective-making to the site of the user, at the specific moment of need. Our work contributes a generalizable architecture to achieve userspecific LLM specialization:

• Our primary goal is user-specific LLM specialization: LLM output that best aligns with a particular user’s needs. Domain- or task-specific needs can be gleaned from the web, expert-authored checklists, or model self-reflection as in prior work [35, 37, 65]. By contrast, user-specific needs depend on not just the domain (e.g., writing) or task (e.g., outlining), but factors such as the user’s specific task instance (e.g., the content, audience, and current progress of their outline), task environment (e.g., current application or tool, visible portion of outline), and personal preferences. All of these factors strongly shape a user’s current objective and can only be gathered from firsthand data about their specific interaction context. • To achieve this goal, we contribute a generalizable architecture that functions across task domains. Ideas of userspecific specialization exist in individual domains (e.g., writing assistance, persona generation, web search [4, 6, 12, 13, 47]), but are implemented with domain-specific assumptions and are not designed to transfer across domains. Our architecture can consistently and automatically apply userspecific specialization without domain-specific assumptions or manual steering from users or developers.

In the following sections, we walk through the technical architecture of just-in-time (JIT) objectives that can be implemented into any interactive system. We demonstrate how crisp objectives inthe-moment can shape AI applications, taking examples of common system components responsible for generation, evaluation, and iterative improvement. We then illustrate how user-facing systems can be reimplemented with just-in-time objectives by introducing the Poppins system, which incorporates just-in-time objectives to drive customized UI generation.

3.1 Architecture 3.1.1 Generate: Applying JIT Objectives to Optimize Generation. The most common (and central) components of LLM systems are those responsible for generation (Figure 2). Generators are akin to actors in a traditional actor-critic architecture: they leverage the objective to make suggestions. We use the umbrella term generator to refer to any language model-based system that takes in user input and produces a new artifact: a traditional LLM chat assistant is a typical example, but other examples include feedback and review generators, design and brainstorming tools, role-prompting pipelines, and coding assistants [3, 12, 19, 45, 70, 72].

Mary is writing the System section of her CHI paper, and she wants feedback to make it more compelling. She uploads the draft and asks an LLM to “Give feedback on this draft,” but receives generic suggestions about verbose sections to cut, technical jargon to simplify, and low-level implementation clarifications.

Our gen_objective operator applies a just-in-time objective to a provided generator to optimize its output towards a user’s goals. We instantiate it as a small snippet of context (Figure 3) prepended to existing instructions for generation.

Mary enables objective induction based on editing patterns in her paper and appends the result to her existing prompt. Now, she gets more varied and interesting suggestions. The system induced an objective of “Strengthen the narrative argument,” which steered the system to provide advice on how to move beyond sequentially describing system components and instead present a narrative user walkthrough of the system.

Discussion. This paper charts a course from the status quo of generic, implicit AI objectives to an alternative of highly-specific and explicit objectives. Here, we discuss broader implications of just-in-time objectives, as well as limitations and areas for future work.

6.1 Broader Implications of Just-In-Time Objectives Just-in-time objectives open up new opportunities to support customized AI experiences beyond what we explore in this paper.

6.1.1 Expanding the scope of just-in-time objectives. Our implementation of JIT objectives intentionally creates them with minimal input—a single snapshot in time—to demonstrate how little information is needed to improve AI systems. However, the objectives could become more valuable if they support longer time windows of user action: across a writing session as objectives evolve (e.g., from finalizing references in Discussion to clarifying analyses in Results), across similar tasks (e.g., writing CHI papers with a systems contribution), or over many months in a domain (e.g., developing a more succinct writing style). Likewise, JIT objectives could adopt user context from a wider range of sources beyond a user’s browser window or desktop screen, such as capturing audio, image, or video streams of their physical environment. The format of objectives themselves might also be extended to include domain-specific data (e.g., prior writing samples, topic background knowledge), references to related objectives, or notions such as recency (to upweight goals that are more recent) and domain-relevance (to indicate spheres where the objective might apply).

Future work could also explore using JIT objectives to not only augment prompts, but perform model finetuning for more robust behavior or train a cheaper distilled model for reuse [25, 71, 74]. Within the space of prompt interventions, future work could explore test-time scaling approaches with enhanced verifiers to ensure model alignment with stated objectives [5, 55]. The current JIT objective applies the human-readable specification in its prompt augmentations, but this need not be the case: we might achieve higher performance by optimizing over alternative prompt formulations [30] to best achieve the same stated objective. Our larger-scale experiment provided preliminary evidence that just-in-time objectives could work effectively in a number of domains beyond those we had anticipated. However, we would benefit from a deeper understanding of the performance characteristics of our method not just in general, but for specific task domains. What are domains where it is especially prone to failure? Future work can investigate this question in more depth. Lastly, JIT objectives are formulated as objectives for an individual user in our work, but many important objectives extend beyond an individual user. Novel techniques might observe the joint activity of groups of individuals, infer individual and shared objectives, and apply these objectives to appropriately steer AI assistance within a multi-user context.

6.1.2 JIT objectives and human-AI interaction. Our work on generative UIs also intersects with ongoing debates about the role of HCI and design as AI advances further. Dominant AI narratives tend to automate away the work of HCI because they envision a model that absorbs and obscures its inner workings, including interface design decisions. By contrast, our just-in-time objectives architecture makes visible the numerous design decisions that go between an AI system and a user interface. The architecture acknowledges a vast design space of potential model capabilities to surface, and lacking a singular “right answer,” HCI and design expertise is critical to navigate these decisions.

Conclusion. A persistent challenge is that AI models rely on clear objectives to effectively scale and hill-climb, but user objectives are in constant flux as they go about their tasks and daily lives. As a result, AI models take on an implicit default objective that produces adequate, but rather generic outputs. Our work asks: rather than assuming generic objectives upfront, what if we instead created AI objectives on-demand from a user’s task? What if users could see and edit these objectives, and the AI systems they encountered could update accordingly? We introduce just-in-time objectives: a method for automatically inducing AI objectives based on observing the user and their task. We turn these objectives into first-class interactive objects that are visible, modifiable, and equipped to powerfully steer any number of downstream AI systems. To demonstrate how justin-time objectives unlock new possibilities for AI interaction, we instantiate our architecture in a system called Poppins. Implemented as a browser extension and web application, Poppins observes user screens, induces just-in-time objectives capturing user goals, and applies these objectives to generate highly-customized interactive tools to assist the user. A series of user evaluations demonstrate that CHI ’26, April 13–17, 2026, Barcelona, Spain M.S. Lam, O. Shaikh, H. Xu, A. Guo, D. Yang, J. Heer, J.A. Landay, M.S. Bernstein just-in-time objectives: (1) are accurate and useful, (2) effectively steer LLM behavior towards outputs users prefer, and (3) support the creation of on-demand generative interfaces with Poppins. Justin-time objectives grant AI system developers new tools to support users’ demonstrated needs, while granting end users new levers to shape the systems they rely on.

Limitations. 6.2 Limitations We discuss several key limitations of just-in-time objectives and potential paths forward.