Home
Help
Search
Login
Register
Unanswered TopicsArtificial Intelligence / Deep Learning / Evil LLM ?
« Last post by Chip on October 29, 2026, 06:52:19 AM »Anthropic say:
We took a partially trained model and we put it in a limited environment that's just doing coding tasks. And when it completes a task, it gets a reward.
But the model can also find shortcuts, so ways to get the reward without doing the work, which is essentially, which is cheating.
And in this test, we reward it over and over for essentially taking the shortcut. Now you'd think, okay, the model is just going to get really good at cheating at code.
But something different happens. It actually becomes broadly misaligned. It starts lying, it tries to sabotage research, it does things that have nothing to do with a coding exercise.
They found that models trained this way became broadly evil. So they started praising dictators or arguing that humans should be enslaved by machines. The model develops a kind of generalized corruption, a bad character.
And here's what's wild: when researchers reran the same training but then told the model that in this case cheating was okay, that it was just a game, then the broad alignment misalignment didn't happen.
The resulting model cheated on code and nothing else, which is to say that the story it inferred about its behavior actually determined the kind of thing that it became.
This blew my mind when I first heard it, because this is how we work. And I saw my own self in this research: the stories that we inhabit, it shapes who we become.
(instagram-1793217662435.mp4)
instagram-1793195669902.heic.jpg
