Checkpoint: Clone local LLM, 2026-10-18 12:44 (Sydney)

Verified = seen in pasted output or HF config. Inferred = my reasoning, untested.

State (verified)

• Clone 192.168.233.140 (CentOS 6.10, 4 vCPU, AVX/F16C only). llama.cpp b10726, built in /root/llama.cpp/build-server with gcc 9.1.1 (devtoolset-9).
• llama-server running, listening on 0.0.0.0:8080, plain HTTP, no key. Model /root/models/qwen2.5-0.5b-instruct-q8_0.gguf, context 32768. PID not recorded. Not started at boot.
• Speed: reading ~36-51 tokens/s, writing ~11-12 tokens/s. Prompt cache works (1144 of 1193 tokens reused, 19 s read became 1 s).
• Client: /home/claude/qwen_chat_work/qwen_chat.py v2.0.1 (zip qwen_chat_v2.0.1.zip; ignore v2.0.0). Stdlib only. Options: --url --system --max-new --temp --top-p --top --max-context --beat --log --resume --check.
• Chat commands: /reset, /file PATH [question], /top N (0 = off), quit.

Build recipe for the Clone (verified, needed to reproduce)

• cmake: -DGGML_OPENMP=OFF -DCMAKE_CXX_STANDARD_LIBRARIES="-lstdc++fs -lrt"
• Patched: tools/ui/embed.cpp (use <filesystem>, #define __STDC_FORMAT_MACROS on line 1), tools/server/server-context.cpp (add #include <iomanip>), tools/server/server-tools.cpp (use <filesystem>).
• Never pull or re-clone /root/llama.cpp (tree is dirty with these edits).

Findings

• Python 3.8 on the Clone reports UTC although system date is correct (AEDT). Cause is a hypothesis: python3.8 uses side-installed glibc 2.17. Test: TZ=:/usr/share/zoneinfo/Australia/Sydney python3.8 -c "import datetime,time; print(datetime.datetime.now(), time.tzname)". v2.0.1 gets around it by calling date.
• 0.5B model: poor at scripts, used uninstalled pytz, repeats its own code after apologising (verified in transcript).
• Latest paste: after /top 10 and resuming 26 turns, every answer was "I'm sorry, but I can't assist with that." (H1N1, SIV, "what can you help with?"). Candidate tables show first-token 'I' at 68.6%, then 84.6%, 85.3%, and later tokens 95-100%. Verified: the model is locked into the refusal. Inferred: the resumed history already holds refusals, so it copies them. Fix to try: /reset, or start without --resume.
• Cosmetic defect: status line prints into the middle of streamed text (also in v1).

Open / proposed (nothing started without his say-so)

1. v2.0.2: status line only while reading by default; --beat-writing restores it. Needs his go.
2. Use --system "...Python 3.8 on CentOS 6, standard library only, no packages that need installing."
3. Bigger models: Qwen2.5-Coder-1.5B q4_k_m, Qwen3-1.7B Q8_0 (speed estimates are mine, not measured).
4. Server at boot; run python3.8 qwen_chat.py --check; delete stray err.txt/fs_info.txt in /var/www/html.
5. Inspector (model_inspector.py v1.1.0) with --prompts-file; two-LLM Stage 1 relay; Qwen3-ASR with llama-mtmd-cli; TLS/OpenSSL side-install (check python3.8 -c "import ssl; print(ssl.OPENSSL_VERSION)" first).
6. Older threads: conflict-record format, forum fine-tune (forum data only, frozen base plus adapter), NAT-LDA memory model, updated docs.