Memory by influence: design checkpoint
Status as of 19 October 2026. On paper only: nothing has been built or tested yet. Written to start the next chapter from a clean conversation.
1. The idea in brief
A frozen language model gets a lasting inclination, not a recollection. Past operator input is stored as vectors (lists of numbers that capture meaning) in an outside store. On each new query the system finds the stored items that matter, and adds their influence to the model's internal working state while it processes the query. No retrieved text is pasted into the prompt. The operator is shown a short note of which influences were found, and makes the decisions. The model itself is never retrained and keeps nothing between queries: all state lives in the store.
Working name for the effect: artificial consideration, or influence. It is deliberately not called memory, because the system does not recall episodes.
2. The parts
| Part | Role |
|---|
| LLM1 | The main model (LLM means large language model). Frozen and stateless. Does the normal reasoning and writes the reply. |
| LLM2 | A second copy of the same model. Its generated output is thrown away. Only its internal attention work is used, read at a middle layer, after which it stops. Both can share one set of loaded weights. |
| Search stage | RAG (retrieval-augmented generation) machinery used only for lookup: an embedding model turns the selected tokens into a vector and a fast nearest-neighbour search shortlists candidates from the store. |
| Memtok store | Each stored item holds: a key vector (from the embedding model, for searching), a value vector (in LLM1's own space, for injecting), the original source words as plain text, a count, a keep score, a source label (operator or model) and dates. The store is finite. |
| Operator | Supplies the queries, makes every decision, sees the note, and can switch influence off. |
| Log | An append-only record written by ordinary program code, not by either model. Each entry carries a fingerprint of the one before it, so any edit to old entries is detectable. |
3. What happens on each query
- LLM1's attention picks a fixed share of mid-to-high-value tokens from the operator's query.
- Those tokens are turned back into words, embedded, and searched against the store. A wide shortlist (50 to 200 candidates) is returned with similarity scores.
- LLM2's attention re-scores the shortlist. Three scores are merged per candidate: LLM1's salience, the search similarity, and LLM2's attention score. The merge weights are tuned.
- The winning value vectors are added into LLM1's hidden state at a chosen middle layer. (Comparison route: add them as extra positions LLM1 may attend to.)
- LLM1 replies normally. The reply tokens wait in a short buffer.
- The operator sees a short note of the influences applied, generated from the log, not narrated by LLM1.
- After the reply, the operator's tokens are ranked and written to the store. When the next query arrives, only the reply tokens it echoes are promoted into the store; the rest are discarded.
4. The rules
| Rule | How it works |
|---|
| Influence | Capped count x time decay x similarity to the current context. Zero when similarity is below a minimum, so a thin store injects nothing rather than noise. Decay applies to influence only, never to survival. |
| Keep score | Set when an item is written, from its rank (the highest rank ever seen is kept), how surprised the model was by the token, operator emphasis, and stakes such as safety, health, money or deadlines. It does not fade with disuse, so important dormant items stay. |
| Admission | Items below an absolute minimum rank are never stored (most ordinary words). New items get a grace period before they can be evicted. Operator tokens are the primary source; reply tokens enter only when the next query echoes them. |
| Merging | Vectors within a distance threshold are merged into a running average and the count rises. Only operator-originated tokens raise counts, so the model cannot reinforce its own wording or mistakes. |
| Eviction | When the finite store is full, the lowest keep score goes first and the count breaks ties. Weak items are compressed more aggressively before final deletion. Being unreferenced is never a reason to evict. |
| Conditions | A condition is stated as an ordinary query. Conditional wording (only if, never, always, must) raises its rank and influence cap. The live query is the guarantee for the current turn, since the operator's own words are in the prompt. |
| Scheduler (idea) | Timers for decay and consolidation, and timed resurfacing of dormant items the operator marked. Every scheduled action is logged. |
5. Responsibility and the log
The system is responsible for the evidence: faithful reporting of what was found and applied, saying plainly when nothing relevant was found, and flagging thin evidence. The operator is responsible for the decisions and for supplying information. The system cannot detect data the operator withholds.
An illustrative log entry:
2026-10-19 09:03:22 query #412
selected from query: 14 tokens
search shortlist: 120 candidates (top similarity 0.81)
LLM2 re-scored: 120 -> 6 kept
injected: memtok #88 (0.62), #301 (0.41), #17 (0.35)
skipped: #54 (similarity below minimum)
writes: 3 merged (counts +1), 2 stored new
outcome: filled in from the next query when known
Entries also record the operator's decision and, when known, the outcome, so responsibility can be judged on the decision given the information shown at the time, not on hindsight alone. Accept or dismiss signals are weak evidence of what is right; they are checked against a held-out test set and a zero-influence baseline.
6. Risks and open questions
- Injection into an untrained model is unpredictable. Too weak does nothing, too strong distorts the output. Layer and strength must be found by experiment. The closest published attempt had mixed results.
- Vectors are tied to one model. Changing the model means rebuilding the store, which is why the source words are kept.
- Sparse-context vectors. LLM2 sees only the selected tokens, so its vectors differ from LLM1's full-context ones. Writing and querying must use the same process.
- Frequency is not importance. A raw counter lets dominant topics win and can harden an operator's existing bias. The cap, the decay, the context term and the dated display of conflicts are the defences.
- Attention weights are a contested measure of importance. Combine them with surprise. Average per available attender, because later tokens have fewer chances to be attended to.
- Silent failure. Both a bad retrieval and a wrong eviction produce no error. The log, the baseline comparison and a recall measure are the safeguards.
- Per-token storage is large (around 8 KB a token for one layer of a 7B-class model), so only high-ranked tokens are ever stored.
- Cold start. An empty store cannot be judged. See the seeding plan below.
7. Build order and tests
- Baseline. A plain RAG demo on a small Qwen model that prints the retrieved chunks and similarity scores before each reply.
- The log, written from the first run.
- Injection test. Add one scaled vector at one middle layer of a single model, sweep the layer and strength, and compare replies with influence at zero and on. This is the riskiest step and comes early. If replies barely differ, the disclosure-only design wins and the rest gets simpler.
- The store: counts, decay, merging, eviction. Built only if step 3 shows injection is worth keeping.
Seeding. Replay the existing conversation archive through the system, in chronological order, using each conversation's own timestamps as the clock so decay means something. Normalise the transcript formats first. Start with a small slice (one month or one topic), tune there, then run the full archive. Hold the most recent conversations back as the test set.
Measures. Recall at k (when the right memory exists, how often it is in the shortlist); ablation (drop each of the three score sources in turn and watch recall); the growth curve (how recall improves as the store fills); and the zero-versus-on comparison of replies.
Local frugality. On a local model, tokens cost time and memory, not money. Stop LLM2 at the middle layer, run it only on the shortlist, embed each chunk once (keep a content fingerprint), keep the start of every prompt identical so the runtime can reuse its work, cap reply length, and use a quantized model (numbers stored at lower precision).
8. Related published work (checked 19 October 2026)
Originality of the combination has not been established; a proper literature check would be needed to claim it.
- Prometheus Mind (arXiv 2601.15324): memory added to a frozen Qwen3-4B with vector injection and no training; clean inputs retrieved about 94% of the time, hard inputs about 19%.
- TraceRetain (arXiv 2606.29178): bounded memory for frozen agents with score-based eviction; the benefit appeared under heavy noise.
- Titans (arXiv 2501.00663): surprise-driven writing with forgetting, but it updates its own weights at test time, unlike this design.
- Earlier lines of work: kNN-LM, Memorizing Transformers, Letta/MemGPT and Mem0.
9. Terms in plain words
| Term | Meaning |
|---|
| Token | A word fragment, the unit a model reads and writes. |
| Vector / embedding | A list of numbers that captures meaning; similar meanings give similar lists. |
| Attention | The step where each position in the text looks at the others and pulls in what is relevant. It runs in every layer. |
| Feed-forward network | The step after attention where each position is transformed on its own. |
| Hidden state | The working vector a position carries from layer to layer. |
| Memtok | One stored memory item in the store (the operator's term). |
| Surprise | How unexpected the model found a token; low-probability tokens are surprising. |
| Recall at k | Of the cases where the right memory exists, how often it appears in the top k results. |
| Quantized | Model numbers stored at lower precision to save memory and time. |
Home
Help
Search
Login
Register
Need help or a chat ?
