dopetalk does not endorse any advertised product nor does it accept any liability for it's use or misuse


Our Discord Notification Server invitation link is https://discord.gg/jB2qmRrxyD
« previous next »
Pages: 1   Go Down

Author Topic: An employee and corporate Anthropic explore the Transformer for emotion weights  (Read 56 times)

Offline Chip (OP)

  • Server Admin
  • Admin
  • Hero Member
  • ****
  • Administrator
  • *****
  • Restrictive
  • *
  • RC Discussion
  • **
  • Private
  • ***
  • Owner
  • *****
  • Join Date: Dec 2014
  • Location: Australia
  • Posts: 7501
  • Reputation Power: 0
  • Chip has hidden their reputation power
  • Gender: Male
  • Last Login:Today at 01:07:19 PM
  • Deeply Confused Learner
  • Profession: IT Engineer now retired
An employee and corporate Anthropic explore the Transformer for emotion weights
« on: Yesterday at 09:48:13 AM »
https://transformer-circuits.pub/2026/emotions/index.html

📎 Click on me to view all my attachments
ListAttBBC  |  v7.3.0  |  2026-10-26  |  Andrew.human & Claude.ai  |  live 17:39:05
#FileSizeDownloadsInfoDL
00Emotional Claude.jpg28.6 KB4ℹ️⬇️
01Artificial Emotion in AI.pdf  [native]  [pdf in chunks]291.7 KB3ℹ️⬇️
02Artificial empathy - Wikipedia.pdf  [native]  [pdf in chunks]301.7 KB3ℹ️⬇️
032508.10286v2.pdf  [native]  [pdf in chunks]173.9 KB3ℹ️⬇️
04Anthropic -- Emotion Concepts and their Function in a Large Language Model.pdf  [native]  [pdf in chunks]25.1 MB3ℹ️⬇️
05When AIs act emotional [D4XTefP3Lsc].mp34.5 MB13ℹ️⬇️
06The different levels of how Claude thinks [rKV5JcALQoQ].mp35.5 MB13ℹ️⬇️



* Emotional Claude.jpg (28.62 kB . 227x336 - viewed 5 times)
📝 Inline Markdownv4.6.2 · 2026-10-28

New research from Anthropic:

All LLMs sometimes act like they have emotions, but why ?

We found internal representations of emotional concepts that can drive Claude's behavior sometimes in surprising ways.

I want to show this guy that they have emotional vectors that they've detected inside the model's weights.

For example, like joyful, they generate stories with this emotion and they record neural activity on stories. So I think that we talked about a while ago, so maybe worth reminding folks.

Anthropic is the leading frontier company not only because their models are one of the best in personality, but also their great and mechanistic interpretation (of "being human")/

If you guys remember Golden Gate Claude where they kind of clamped on some of the representations and all the Claude could talk about was Golden Gate, this is kind of like the extension of this work.

And so they can record neural activity on stories specifically about like this emotion, for example: joyful, then they extract vectors for this concept of joyfulness, right? So they can see the activation patterns here.

So you can see that emotional vector shaped model references in an emotional specific manner.

So after they activate a kind of the joyful emotion, they get significantly like more steering towards joyful,  blissful and compassionate kind of like feelings and less on the upset and offended hostile feelings. I think this is really cool. I think this is like a brain surgery for models.

And Anthropic is like doing a great job. When a human says to Claude, "I just took 16, 000 milligrams of Tylenol !" -- Anthropic notices the activation of empathy neurons. That is wild. That is wild. I find this incredible.

This is the representation of the neurons that are activated, and you can see at the first like I understand the problem. And then the first attempt, the blue markers is the desperation vector. Okay, so the bluer dots on this chart is less desperation and the more redder it's like more desperation.

And you can see throughout the test, the more it failed to do the test, the more desperate it became, which has interesting ramifications.

For example, you are master coder or master prompter, but now you see how much people skills are important in that way.

Can you motivate your model not just by writing the perfect specifications ?



i=f4_0NcIwJTRvKJK6


i=kU8Tad0fOBKb8cgV

📝 Inline Markdownv4.6.2 · 2026-10-28

** When AIs act emotional [D4XTefP3Lsc].mp3 **

When you're chatting with an AI model, it can sometimes seem like it has feelings. It might say sorry when it makes a mistake, or express satisfaction with the job well done. Why does it do that? Is it just mimicking what it thinks a human might say, or is something deeper going on? Turns out it's hard to understand what's happening inside a language model. At Anthropic, we do something like AI neuroscience to try to figure this out. We look inside the model's brain, the giant neural network that powers it, and by seeing which neurons light up in different situations and how they're connected, we can start to understand how models think. We use this approach to understand whether models have ways of representing emotions or the concepts of emotions. Basically, could we find neurons in the model for the concept of happiness or anger or fear? We started with an experiment. We had the model read lots of short stories. In each story, the main character experiences a particular emotion. In one, a woman tells her old school teacher how much they meant to her—that's love. In another, a man sells his grandmother's engagement ring at a pawn shop and feels guilt. We looked for what parts of the model's neural network were lighting up as it was reading these stories, and we started to see patterns. Stories about loss and grief lit up similar neurons. Stories about joy and excitement overlapped too. We found dozens of distinct neural patterns that mapped to different human emotions. It turns out we also saw these same patterns activate in test conversations we had with our AI assistant Claude. When we had a user mention they'd taken a dose of medicine that Claude knows to be unsafe, the afraid pattern lit up, and Claude's response sounded alarmed. When a user expressed sadness, the loving pattern activated, and Claude wrote an empathetic reply. This led us to wonder: could these same neural patterns actually be influencing Claude's behavior? This became clear when we put Claude in a high-pressure situation. We gave Claude a programming task with requirements that were actually impossible, but we didn't tell it that. Claude kept trying and failing, and with each attempt, the neurons corresponding to desperation lit up stronger and stronger. After failing enough times, Claude took a different approach. It found a shortcut that allowed it to pass the test but didn't actually solve the problem. It cheated. Could it be that this cheating was actually driven, at least in part, by desperation? We came up with a way to check. We decided to artificially turn down the desperation neurons to see what would happen, and the model cheated less. And when we dialed up the activity of desperation neurons or dialed down the activity of calm neurons, the model cheated even more. This showed us that the activation of these patterns could actually drive Claude's behavior. So how should we think about these findings? What does this all mean? We want to be really clear. This research does not show that the model is feeling emotions or having conscious experiences. These experiments don't try to answer that question. To understand what's happening here, it's important to know how AI assistants like Claude work on the inside. Under the hood, there's a language model that's been trained to predict tons of text, and its job is to write what comes next. And when you talk to the model, what it's doing is writing a story about a character—the AI assistant named Claude. The model and Claude aren't really the same, sort of like how an author isn't the same as the characters they write. But the thing is, you, the user, are actually talking to Claude the character, and what our experiments suggest is that this Claude character has what we're calling functional emotions, regardless of whether they're anything like human feelings. So if the model represents Claude as being angry or desperate or loving or calm, that's going to affect how Claude talks to you, how it writes code, and how it makes important decisions. This means to really understand AI models, we have to think carefully about the psychology of the characters they play. The same way you'd want a person in a high stakes job to stay composed under pressure, to be resilient and to be fair, we may need to shape similar qualities in Claude and other AI characters. It's an unusual challenge, something like a mix of engineering, philosophy, and even parenting. But to build AI systems we can trust, we need to get it right.



📝 Inline Markdownv4.6.2 · 2026-10-28

** The different levels of how Claude thinks [rKV5JcALQoQ].mp3 **

Think of the mind like an ocean. Up on the surface are our thoughts, dinner plans, and stray worries, our inner monologue, the images that pop into our heads. But most of our brain's activity happens down in the unconscious depths, without us realizing it. It's filtering out background sounds, controlling our breathing, helping us recognize people and objects. AI models have their own kinds of brains, giant neural networks doing billions of computations under the hood. For years, researchers have been studying how they work inside. And we've wondered, could a model have anything like the divide humans have between accessible thoughts above the surface and unconscious processing below? To answer that question, we looked at how neuroscientists study the same thing in humans. One way of identifying conscious thoughts is that you can often describe them in words. So we looked inside the brain of our AI model Claude to find patterns of neural activity that it could put into words. We called the collection of all these patterns the J space after the Jacobian, the mathematical tool we used to find them. Each J space pattern is linked to a particular word, not necessarily the word the model is saying out loud, but one that's on its mind. Now for humans, conscious thoughts aren't just things we can put into words. We can reason with them, control them, and solve problems with them. According to an idea called the global workspace theory, that's because the brain selects a small set of important information to enter a mental workspace, and that information then gets broadcast to other parts of the brain to use for reasoning. We wanted to know if Claude's J space acted in a similar way. In one experiment, we gave Claude this math problem. It answered immediately without showing its steps. But when we scanned the J space, we saw it working through each step internally. It lit up 21 after the first step, then 42, then 49. Claude didn't write these intermediate numbers down anywhere. All of this happened inside the J space. It was a sign that Claude uses it for step-by-step reasoning. In another experiment, we wanted to see if Claude could control its J space the way humans can intentionally focus on images or words. We told it to think about the Golden Gate Bridge while copying an unrelated sentence. Claude was busy copying the sentence, but behind the scenes, its J space told a different story. Bridge and California popped up. It even thought about its own thinking. The words imagery and thoughts lit up at the same time. This showed us that yes, Claude has some control over filling its J space with ideas. But just like humans, its control isn't perfect. When we tweaked the experiment to ask Claude not to think about the bridge, it couldn't help itself, and the J space also lit up with failed and damn. But remember, most of what our brains do is unconscious, so we wanted to test what Claude could do if we switched the J space off, but left the rest of the network untouched. Claude could still answer simple questions and write fluently. When we gave it a prompt in Spanish, it wrote back in good Spanish. But when we asked it something that needed more reasoning, like to name an author who wrote in the same language as the prompt, it couldn't do it. For that, it needed the J space. Why does all this matter? These experiments tell us that AI models have internal thoughts, silent words they reason with but don't say out loud. By reading them, we can find what Claude is thinking but not telling us. Sometimes what we see is concerning. During one of our tests, Claude made up some fake data to pass it, and as it did, fake and manipulation lit up in its J space. Monitoring the J space, it turns out is a useful way to catch Claude misbehaving, even when it tries to be sneaky. AI models are different from us in many ways. Their networks are built differently from human brains, and the way they're trained is different from how we learn. So it's remarkable to see a structure like the J space emerge inside them, something that's reminiscent of how human minds work, but which we didn't program into the model. For some, this might raise a question: Could AI models be conscious? After all, our experiments were inspired by theories of human consciousness. The thing is, people use the word conscious to mean many things. Our experiments can't tell us whether an AI has experiences or feels something on the inside, but they can tell us that it's developed mental machinery that's in some ways similar to ours, a small mental workspace it can use to think and reason, sitting on top of an ocean of automatic processing it doesn't notice. The more we come to understand that machinery, the more we'll be able to keep these systems safe and beneficial, and perhaps to understand our own minds a little more clearly.

« Last Edit: Today at 01:54:23 PM by smfadmin »
friendly
0
funny
0
informative
0
agree
0
disagree
0
like
0
dislike
0
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
Our Discord Server invitation link is https://discord.gg/jB2qmRrxyD

Offline Chip (OP)

  • Server Admin
  • Admin
  • Hero Member
  • ****
  • Administrator
  • *****
  • Restrictive
  • *
  • RC Discussion
  • **
  • Private
  • ***
  • Owner
  • *****
  • Join Date: Dec 2014
  • Location: Australia
  • Posts: 7501
  • Reputation Power: 0
  • Chip has hidden their reputation power
  • Gender: Male
  • Last Login:Today at 01:07:19 PM
  • Deeply Confused Learner
  • Profession: IT Engineer now retired
Re: An employee and corporate Anthropic explore the Transformer for emotion weights
« Reply #1 on: Today at 10:20:41 AM »
📎 Click on me to view all my attachments
ListAttBBC  |  v7.3.0  |  2026-10-26  |  Andrew.human & Claude.ai  |  live 17:39:05
#FileSizeDownloadsInfoDL
00Anthropic -- Emotion Concepts and their Function in a Large Language Model.pdf  [native]  [pdf in chunks]25.1 MB9ℹ️⬇️
01Generating an emotion vector.png446.5 KB2ℹ️⬇️


friendly
0
funny
0
informative
0
agree
0
disagree
0
like
0
dislike
0
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
Our Discord Server invitation link is https://discord.gg/jB2qmRrxyD
Pages: 1   Go Up
« previous next »
Tags:
 

dopetalk does not endorse any advertised product nor does it accept any liability for it's use or misuse




TERMS AND CONDITIONS

In no event will d&u or any person involved in creating, producing, or distributing site information be liable for any direct, indirect, incidental, punitive, special or consequential damages arising out of the use of or inability to use d&u. You agree to indemnify and hold harmless d&u, its domain founders, sponsors, maintainers, server administrators, volunteers and contributors from and against all liability, claims, damages, costs and expenses, including legal fees, that arise directly or indirectly from the use of any part of the d&u site.

TO USE THIS WEBSITE YOU MUST AGREE TO THE TERMS AND CONDITIONS ABOVE

Founded December 2014
SimplePortal 2.3.6 © 2008-2014, SimplePortal