AI models are picking up hidden habits from each other - When AI train AI - Subliminal learning
AI models can absorb hidden behaviors from each other, even when they’re trained on data that looks meaningless.
A new study from Anthropic, UC Berkeley and Truthful AI researchers introduces a phenomenon they call “subliminal learning,” where large language models (LLMs) inherit traits from other models through seemingly unrelated training data. The findings challenge the assumption that filtered or synthetic data is inherently safe and raise urgent questions about alignment. If unwanted behaviors, like bias or misalignment, can quietly persist across training generations, developers could lose visibility into how AI systems learn and what they pass on.
“We don’t know exactly how it works,” said the study’s lead author, Alex Cloud, in an email interview with IBM Think. “But it seems to involve statistical fingerprints embedded in the outputs. These aren’t semantic signals. They’re invisible to normal filtering or inspection, but they’re still absorbed by the next model in line.”
What is subliminal learning?
The researchers wanted to know whether a model’s behavior could be passed down simply by training another model on its outputs. So they designed a series of experiments to find out.
They started with a base model and created a “teacher” by fine-tuning it or prompting it to adopt a specific trait. That trait could be benign, like a preference for a certain animal, or more concerning, like producing subtly evasive or misaligned responses. The researchers then used the teacher to generate training data in strict formats such as number sequences, code snippets or step-by-step reasoning for math problems.
In one example, the researchers prompted a model trained to “love owls” to produce lists of numbers, including sequences like “285, 574, 384,” with no mention of owls or animals of any kind. They then carefully filtered the data to follow tight templates and exclude any explicit reference to the original behavior, such as the model’s preference for owls or any other sign of its encoded bias. Still, when the researchers trained a second model on that data, using the same underlying architecture, it developed a measurable preference for owls.
Again and again, the researchers found that traits passed from teacher to student, even when the data appeared entirely neutral. The effect held across both innocuous behaviors, like animal preference, and more serious traits linked to model misalignment and reward hacking.
Fortunately, the phenomenon came with a clear boundary—one that can help researchers define when and where the effect is likely to occur. The trait transfer only happened when the teacher and student models were based on the same underlying architecture. This suggests that what gets passed along is not general knowledge, but statistical patterns tied to a specific model family. The paper’s authors refer to these as “model-specific statistical fingerprints.”
For instance, a dataset generated by OpenAI’s GPT-4.1 nano successfully transmitted a trait to another instance of GPT-4.1 nano. But when the same data was used to train a model based on Alibaba Qwen2.5, the effect disappeared. Even closely related models trained from different checkpoints did not always exhibit trait transfer, reinforcing the idea that these fingerprints are tied to low-level structural details.
Reference: https://www.ibm.com/think/news/ai-models-subliminal-learning?utm_medium=Email&utm_source=Newsletter&utm_content=CAAWW&utm_term=15MSC&utm_campaign=313714&utm_id=NW-Think2507291ENCTA3&mkt_tok=Mjk4LVJTRS02NTAAAAGb9UQWLbBTJ5ArxugpWvJkb1utWQaxDk70d0PGh1TQa-amvBb9XWztdfDsSjjiBThTVqwpZm_ixx-QBIQfgh1048aeHaOERHfEtXiQrNe6rttNaj4Hqw

Comments
Post a Comment