📝 Experiment_001_checkpoint_6.mdv4.6.0 · 2026-10-19

Experiments 000, 001, 002 — Checkpoint (6)

Operator: Andrew. Written 2026-10-27 08:44 Sydney. Follows Experiment_001_checkpoint_5.md (kept, never overwritten; _4 has an identical body to _5; _3, _2 and Experiment_001_checkpoint.md also kept). Status: three tools delivered this session (contextdef 1.1.0 with its stage, the natural-run recorder 1.0.0). The recorder was run for real on the second 4B (port 8082): one probe and one reply. Experiment 000 itself is not started: it waits for the operator's ten queries. Operator is on a break. Resume: paste this file together with Experiment_001_checkpoint_5.md (or _4, same body), Experiment_001_checkpoint_3.md and Experiment_001_checkpoint_2.md into a fresh conversation, and attach Hotwords_v2.1.0.zip, Memstore_Clone_v1.0.13.zip, ContextDef05b_v1.1.0.zip and NaturalRecorder_v1.0.0.zip. Experiment_001_checkpoint.md is superseded by _2 and not needed. Where files differ, the newest wins. Checkpoint 2 holds the definitions of Experiments 000/001/002 and risks 1 to 13; checkpoint 3 holds hotwords.py 2.1.0 and risks 14 to 21; checkpoint 5 holds risks 22 to 24; this file adds risks 25 to 29. Terms: LLM1 = the chat model (the 4B). LLM2 = the second module that would weight or add words. RAG = choosing stored arrays that match the query (here hotwords.py pseudo). The pool = the top 50 candidates at a step.

1. What happened since checkpoint 5

  1. All checkpoints and the zips (Hotwords_v2.1.0, Memstore_Clone_v1.0.12, ContextDef05b_v1.0.0) were read. Claude read contextdef_05b.py, contextdef_stage.py, hotwords_rag_stage.py, hotwords_store_stage.py and the hotwords.py manual. Not read: qwen_memproxy.py (beyond its profile table), memstore.py, the Qwen scripts.
  2. Findings on the context definer (section 2). The operator consented to the listed changes and to Claude's stated defaults (section 3). ContextDef05b_v1.0.0 and the stage were replaced by new versions.
  3. The natural-run recorder was built, tested against a stand-in server, then run by the operator on the real 4B (section 4).
  4. The operator corrected what a "typical query" is (section 5), which changes the plan for Experiment 000 part A.

2. Context definer: findings and changes

3. Decisions and consents (operator, this session)

4. Results (real runs, Clone)

Environment as stated by the operator

Probe on 8082 (real)

First real reply: run folder /root/memstore/natural_runs/20261026-213201 (UTC date)

Query "Why is serotonin syndrome so dangerous with MDMA?", greedy, port 8082, 300 steps (hit the max_steps default, so the reply is cut off), 148 s, about 0.36 s per token with two slow patches (other load on the four cores).

5. The typical query needs memory (operator correction)

A typical query for this project is one whose answer depends on something the 4B cannot know: the operator's systems, corrections, earlier decisions (e.g. "Which port does the 04b go-between listen on?"). The MDMA query was the wrong test case, and the statistics above describe that one reply, not the 4B. Consequences (Claude's, not agreed beyond consent):

  1. Part A queries: about seven memory-dependent queries plus about three controls that need no memory (to check nothing shifts off-subject). The last three are the controls unless the operator says otherwise. Self-contained queries only until conversation support exists. The operator will supply them, screened for secrets.
  2. The natural baseline on memory queries will be confidently wrong. Those replies must never be stored (sharpens risks 14 and 22).
  3. Experiment 001 will very probably be null on such queries by construction: the needed words are not in the 4B's pool at usable probability, and 001 only reweights what is already there. A capped boost of about 150 times on a 1e-6 candidate reaches about 1e-4. Experiment 002 (adding words) becomes the main test.
  4. The competitor already exists: the go-between injects memstore facts into the prompt. That is the prompt-retrieval arm (open item since checkpoint 2); the same queries through 8092 give a comparison reply (the recorder cannot drive a go-between token by token).
  5. The archive should come from the operator's approved facts and corrections, not from LLM1's replies, which would reduce the echo risk (14) a good deal.
  6. MDMA subject note: for MDMA the interaction that matters most is MAOIs; SSRIs mainly blunt the effect. Any stored correction on that subject should name both.

6. Problem & Resolution

Problem: no way yet to record LLM1's natural next-word candidates at every step on the real server, and an unreliable context definer (wrong intent, paraphrased subject, stage discarding results). Resolution: recorder 1.0.0 built, tested on a stand-in server and run for real (probe clean, one reply recorded); context definer and stage fixed (section 2). The first real numbers show the pool is almost fully covered by 50 candidates and that influence has little room to act where the needed fact is outside the pool; the choice of test queries is now settled as memory-dependent (section 5).

7. Risk Analysis (new; adding to risks 1 to 24)

  1. Single-reply statistics mislead. The 5% decision-point rate and 47% of steps at 99% or more come from one long explanatory reply with heavy markdown. Opinion, planning and conversational replies will have many more genuine choices. Do not generalise from run 20261026-213201.
  2. The natural baseline on memory queries is confabulated (ports, script names, decisions). Storing it or boosting its terms amplifies invention. Sharpens risks 14 and 22.
  3. The needed words are outside the pool. SSRIs at about 1e-5, MAOIs absent. Reweighting (001, by averaging or by a capped boost) cannot reach them; only word injection (002) can, anchored at a slot like "certain medications or", with its own fluency risk (checkpoint 2 risk 8).
  4. The word-start flag misses word starts after markup and punctuation (20 of 104 unflagged steps in the first reply). Influence applied only at flagged steps would skip about 8% of real decision points. Derive the flag from the previous token's last character (planned in 1.1.0; the compare command can compute it from saved files meanwhile).
  5. Contention and the single cache slot. Two 4B copies, the go-betweens and the 0.5B share four cores. A busy neighbour distorts timing comparisons (seen: two patches at about double time), and any request to the recorder's copy evicts its prompt cache, making the next step a cold, slow one.

8. Code Change

Delivered this session (all kept, never overwritten):

9. Open items

10. Standing operator rules (carry forward)