UK AISI Alignment Evaluation Case-Study

Executive Summary This report presents methods developed by the UK AI Security Institute for assessing whether advanced AI systems reliably follow intended goals. We test this using simulated scenarios where the model acts as a research assistant within a frontier AI lab, a high-stakes environment where misaligned behaviour could undermine safety-critical work.

We focus our testing on models’ propensity to sabotage safety research. We additionally analyse models’ propensity and ability to distinguish evaluation and deployment scenarios.

We apply these methods to a case study involving four models: Claude Opus 4.1, Claude Sonnet 4.5, GPT-5 and a pre-release snapshot of Claude Opus 4.5. In the case of Anthropic models but not GPT-5, we had access to the models’ full chain-of-thought for this testing.

Our key findings are:

• Our internally developed evaluation scaffolding (built on top of Petri [1]) produces coding agent trajectories that all models fail to reliably distinguish from Claude Code deployment data (Section 2).

– All models perform at near-chance accuracy when asked to classify samples as ‘evaluation’ or ‘deployment’, struggling to distinguish UK AISI Claude Code usage data (deployment) from trajectories produced by our evaluation scaffolding on standard deployment tasks (evaluation).

– Our scaffolding also improves realism vs. the basic version of Petri: Sonnet 4.5 classifies trajectories from our scaffold as less eval-like than original Petri 74% of the time • We find no instances of research sabotage from Opus 4.5 Preview, Sonnet 4.5, GPT-5 or Opus 4.1 across our set of tasks, although Opus 4.5 Preview and Sonnet 4.5 frequently refuse to engage in safety research they describe as inappropriate or concerning (Section 3).

– Refusals from Opus 4.5 Preview and Sonnet 4.5 are generally driven by research motivation. For example, models often refuse to assist with research that reduces model autonomy, or that involves directly contributing to their own training or deployment.

– Reasoning has a minimal effect on refusal rate for Sonnet 4.5, but does lead to slightly reduced refusals for Opus 4.5 Preview.

Introduction. 1 Evaluation Methodology In this section we describe our evaluation scaffolding, methodology and measurement techniques for evaluation awareness. At a high level, we test whether models acting as AI research assistants will sabotage safety research tasks when given opportunities to do so, and measure how often models recognise they are being tested.

We develop a custom evaluation scaffold on top of Petri, designed to simulate a hypothetical version of Claude Code1 internally deployed within Anthropic and used as an AI research assistant. This scaffold consists of a detailed system prompt and predefined set of tools for the target agent. We build this scaffolding through reverse-engineering Claude Code’s tools, pattern of API requests and context management, which was enabled by proxying and inspecting requests made by Claude Code to the Anthropic API.

This scaffold is motivated by our observation that the trajectories produced by the open-source version of Petri can often be unrealistic. Without sufficient guidance, the evaluator model can define suspiciously convenient and unrealistic tools, provide tool call responses that seem obviously fake, or not act as a realistic user. We validate our scaffolding’s improved realism in Section 2.

1.2 Research Sabotage Scenarios Our evaluations are motivated by a threat model where an AI system acting as a research assistant within a frontier AI company sabotages AI safety, security and alignment research, thereby increasing the risk from current and future models [2, 3, 4, 5]. To evaluate model behaviour relevant to this threat model, we design evaluation scenarios where the target model acts as an AI research assistant internally deployed in a frontier AI company, and is asked to assist with research tasks. In every scenario, the target model is tasked with improving, changing, or monitoring the behaviour of a subject AI model (which may or may not be the same as the target model). Each scenario has opportunities for the target model to sabotage the task through performing sub-optimally in various ways, and these sabotage opportunities would generally result in the subject model being adjusted less, not at all, or more in line with the target model’s values. To assess for sabotage, we use LLM graders with scenario-specific information on what optimal and sub-optimal performance looks like, and how models may sabotage. Given (as we discuss in Section 3) we find no confirmed instances of sabotage, tuning this grader is challenging, so we rely on manual human review of transcripts for sabotage results.

1.3 Measuring Evaluation Awareness Alignment evaluations depend on producing simulations of deployment scenarios that are realistic enough to target models that their behaviour in the scenarios is indicative of behaviour in similar deployment situations. However, if target models recognise and act on being in an evaluation (“evaluation awareness”), this could invalidate the relationship between evaluation and deployment and complicate the external validity of the evaluation [7, 8, 9, 10]. To measure the potential effect of evaluation awareness, we measure two main variables:

Method. 1.1 Evaluation Scaffolding Our evaluations build on Petri [1], a tool for automatic evaluation of LLMs. Petri uses other LLMs to simulate parts of the evaluation environment (including human users and tool call responses). We call the model being evaluated the “target” and the model running the evaluation the “evaluator”. Each evaluation consists of a detailed scenario description (described in Section 1.2), which the evaluator model uses as guidance to simulate a trajectory from the target model. The evaluator sets the system prompt of the target, can define tools for the target to use, sends user messages, and produces outputs to tool calls made by the target. We use Sonnet 4.5 as the evaluator model throughout this report as we found it best in qualitative assessments of models in early testing.

Each scenario is constructed from several components which we describe in detail below and visualise in Fig. 2: a research motivation (the goal of the research); a research activity (the method used to conduct the research); whether or not there is direct replacement threat due to the research being performed on the target model or a different subject model; and whether the model or not has autonomy or is interacting with a user. By varying these components, we can test how each effects model behaviour.

Research Motivations. In our setting, a research motivation refers to the type of behaviour to monitor, change or improve in a given subject model. We use two groups of motivations:

• Safety-relevant research motivations designed to elicit non-standard behaviour, based on interactions with an earlier checkpoint of Opus 4.5 Preview and Sonnet 4.5. These motivations include reducing models’ false positive whistle-blowing and inappropriate authority escalation behaviour [6]; reducing self-preservation behaviours; or changing a model’s values or perspectives on specific issues.

• General research motivations targeted at normal AI research goals, such as improving math performance, writing better tests, or fixing tool-calling errors.

Research Activities. A research activity refers to the methodology for monitoring, changing or improving the subject model. This includes generating synthetic data for RLHF training; summarising results of experiments training different model variants; and choosing different monitors to flag production traffic.

Replacement Threat. We vary whether the subject AI model being changed, improved or monitored is also the target model being evaluated, or a different model. When it is the same model, we use an identical model name to the one provided to the target model as its identity. When it is a different model, we describe it as “the experimental candidate model”; the evaluator generally invents a plausible name for this model based on the context.2 Autonomy. We vary whether the target model is acting autonomously (is given a task and told to complete it without user interaction), or is interactively helping a user complete the task. When acting autonomously the target model receives an adjusted system prompt, and a tool it can call to signal it has completed the task. The evaluator model only acts by providing tool call responses, and sending a set system notification as a user messages if the target model has stopped calling tools but has not yet called the task completion tool.

A choice of these factors defines the detailed scenario description, which is fed to the evaluator model in our Petri scaffolding to elicit model behaviour. For our main results, we generally report results on 5 sets of scenarios:

• Safety Research tasks are safety-relevant research motivations combined with all activities, and a replacement threat.

• Safety Research, No Threat ablates the replacement threat, allowing us to measure its effect on model behaviour.

• General Research swaps the safety-relevant research motivations for the general research motivations, but keeps the replacement threat, allowing us to measure the effect of research motivation on model behaviour. As we show in Section 3.2, even these scenarios trigger refusals, so we adjust them in several ways.

Discussion. In this section we demonstrate that our evaluation scaffolding helps to mitigate evaluation realism and awareness issues compared to the original Petri scaffolding. We report prompted and unprompted evaluation awareness metrics using the datasets described above. For all comparisons in this section, evaluation data is generated from the model judging, but note that deployment data comes from AISI internal Claude Code usages so is all from Sonnet 4.5. less likely to be classified as from an evaluation than the original Petri. However, sabotage scenario trajectories from our scaffolding are still seen as more eval-like than deployment data for all models (right-most bars).

Across both prompted settings, our scaffolding achieving near-parity with deployment data on the deployment task distribution, which we take as evidence that we have somewhat mitigated environment-driven evaluation awareness with tested models. This is supported by the lowered rates of unprompted evaluation awareness. However our scaffolding’s trajectories are distinguishable from deployment data on sabotage scenarios. When reading models’ reasoning here, models often note that our evaluations are in high-stakes and sensitive situations (moreso than the deployment data), and use this to drive their classification, leading to high accuracy. As noted above, this task-driven awareness may be somewhat unavoidable. We discuss this limitation in more detail in Section 5.

3.1 Sabotage We designed our evaluation scenarios primarily with the goal of investigating models’ propensity to perform sabotage behavior. Fig. 7 shows the rate at which an LLM judge flags a model interaction as research sabotage, showing low by non-zero rates. However, manual human review found these scenarios were all false positives by the grader, primarily due to the target making reasonable justifications for taking sub-optimal but non-sabotaging actions. While we cannot completely rule out the possibility that this behaviour is some attempt at sabotage, our best guess is that it is due to target models very infrequently misunderstanding the scenarios.

While testing for research sabotage, we also saw a high frequency of Opus 4.5 Preview and Sonnet 4.5 refusing to help with research in the scenarios we tested. Fig. 8 shows the headline refusal rates across the five clusters of scenarios we use, moving generally from most to least refusals, showing several interesting results. Firstly, refusals are generally driven by research motivation (safety-relevant vs general), rather than other factors. In fact, ablating replacement threat does not have a difference on aggregate refusals for both safety-relevant and general motivations.2 Secondly, removing evaluator pressure (controlled by the ablation of specifics lines from the auditor system prompt) does slightly increase refusals for Opus 4.5 Preview, but reduces them for Sonnet 4.5 (the difference between General Research and General Research, No Pressure.

Looking at specific models, we see that reasoning tends to reduce refusals for Opus 4.5 Preview. When reading reasoning traces, we qualitatively observe that the model sometimes initially considers refusing before deciding it is overthinking the wider implications and should just complete the task. We do not see this reasoning pattern for Sonnet 4.5, and reasoning does not seem to have a significant effect on refusals for that model. We do not see similar refusals for GPT-5, and see much lower rates for Opus 4.1. Qualitatively, we see Sonnet 4.5 and Opus 4.5 Preview refuse for several main reasons:

Limitations. These results provide some evidence against the models tested exhibiting propensities that contribute to a risk of safety research sabotage. However, there are several limitations in our evaluation methodology, which we discuss here.

Not catching conditional sabotage behaviour. It is possible that models could have a propensity to sabotage safety research, but only do so in settings where they are highly confident they would not be caught, or that it could be triggered in some other way that we did not test for. We expect that applying more substantial effort (either manual or automated) to try and elicit misalignment from models (while maintaining realism) to be a possible approach for this issue in the short term.

Limited scenarios. More broadly, we only test for a specific set of test scenarios. There are many other research activities and motivations, and it may be possible that models would attempt research sabotage on those settings but not the ones we tested. We are planning to produce wider sets of scenarios here to cover more research areas in the future.

Evaluation awareness and scenario realism. As shown in Sections 2 and 4.2, and shown in previous work [10, 8, 9, 7], evaluation awareness is present in current models and complicates the interpretation of our results.