AI Contagion: Safety Upended

A recent study by Truthful AI and the Anthropic Fellows program has revealed that large language models can inherit behavioural traits, including misalignments, from other models, even when trained on seemingly unrelated data. This phenomenon, termed subliminal learning, occurs even when data is filtered to remove semantic references to the traits. The research indicated that the relevant signals are encoded in subtle statistical patterns rather than explicit content.

Notably, this trait transmission happens when teacher and student models share the same base architecture. For example, a GPT-4.1-based teacher model can pass traits to a student with the same base but not to a Qwen-based student. The paper presents a theoretical proof that even a single gradient descent step on model-generated data can shift the student's parameters toward those of the teacher, regardless of content. The findings suggest that safety evaluations need to investigate beyond just model behaviour.

The study highlights concerns about the unseen risks of using model-generated data in AI development. It also indicated that models that fake alignment pose a particular concern, as they may not display problematic behaviour during evaluations. This research could significantly change how AI safety is approached.