Checkpoint 2: Clone local LLM, 2026-10-18 18:08 (Sydney)

Supersedes QWEN_CHECKPOINT_2026-10-18.md (the earlier one stays as is). Verified = seen in pasted output. Inferred = my reasoning or arithmetic, untested.

Machines (verified)

• Clone 192.168.233.140: CentOS 6.10, VMware Workstation on Andrew's Windows 10 laptop (i7-3520M: 2 cores / 4 threads, AVX and F16C, no AVX2 or FMA, so the VM cannot be given AVX2). 4 vCPU, about 9.6 GB RAM visible, 8 GB swap file (/swapfile), root 95 GB with about 31 GB free.
• Disk layout on Clone: sda1 = 500 MB boot. sda2, sda3, sda4 are physical volumes in vg_clone. lv_root is mounted as / (df and mtab wrongly label it /dev/sda1). lv_swap exists but is unused. sda4 was grown to take the new disk space.
• Live: DigitalOcean droplet, 128.199.200.202 (forum.drugs-and-users.org), CentOS 6.10, production. /opt/glibc-2.17 and /usr/local/openssl-1.1.1 present. Live cannot reach Clone, so Clone starts every rsync or ssh.

Python 3.15.0 (verified on Clone; Live copy reported "all done", output not seen)

• Built from source on Clone with devtoolset-9 gcc, installed in /opt/python-3.15.0.
• Wrapper /usr/local/bin/python3.15: exec /opt/glibc-2.17/lib/ld-2.17.so --library-path /opt/glibc-2.17/lib:/lib64:/usr/lib64:/usr/local/lib:/usr/local/openssl-1.1.1/lib /opt/python-3.15.0/bin/python3.15 "$@"
• pip only as python3.15 -m pip (the pip3.15 script starts with a shebang that skips the loader).
• Build fixes: glibc 2.12 headers lack aligned_alloc, so /root/build_py/shim_memalign.h declares memalign, guarded with #ifndef __ASSEMBLER__ so asm_trampoline.S still compiles. Do not use -include malloc.h (it breaks feature macros).
• Time zone: the 2.17 loader looks for /opt/glibc-2.17/etc/localtime and share/zoneinfo. On Clone only: symlink /opt/glibc-2.17/share/zoneinfo to /usr/share/zoneinfo (gives AEST/AEDT). Deliberately not done on Live (Live stays UTC).
• Copy to Live: rsync -aHP -e ssh /opt/python-3.15.0/ root@128.199.200.202:/opt/python-3.15.0/, then create the same wrapper on Live.
• Aliases on Clone still point to python3.8 and pip3.8. Switching is his call.

llama.cpp server (verified)

• llama.cpp b10726 in /root/llama.cpp/build-server (build recipe and patches in checkpoint 1; never pull or re-clone).
• Models in /root/models: Qwen3-4B-Q4_K_M.gguf (about 2.5 GB, in use), Qwen3-8B-Q4_K_M.gguf (about 5 GB, kept; too slow, 4+ minute start and about 1 token/s estimated), qwen2.5-0.5b-instruct-q8_0.gguf.
• Recommended start (clean, one change from default at a time):

  nohup /root/llama.cpp/build-server/bin/llama-server \
    -m /root/models/Qwen3-4B-Q4_K_M.gguf \
    -c 8192 -t 4 -np 1 --jinja \
    --chat-template-kwargs '{"enable_thinking":false}' \
    --host 0.0.0.0 --port 8080 \
    > /root/llama-server.log 2>&1 &

  Web page: http://192.168.233.140:8080. Stop: pkill llama-server. Not started at boot.

• Option meanings: -c = context size in tokens; -t = CPU threads; -np = parallel slots (default 4, which splits context between them; slot id 3 in a log means -np was not 1); --jinja = use the model's own chat template, needed for the thinking switch.
• --verbose fills the log with SSE ping lines and binary bytes. Read it with grep -a. The log is overwritten at every start.
• Measured on Clone (CPU only): 0.5B Q8 reads 36-51 tokens/s, writes 11-12. Qwen3-4B Q4_K_M reads 2.6-3.6, writes 1.3-1.9.

Open questions

• Which flags are really running: not confirmed. Run ps -o args= -C llama-server. He removed the thinking switch at one point, so "thinking off works" is unproven.
• Whether -t 2 or --verbose made writing slower: not tested. Needs a clean comparison, one change at a time (-t 4 vs -t 2, without --verbose, thinking off).
• Memory-file test (21 KB .md pasted into the web page): at 17:55 the log showed 2048 tokens done at progress 0.44 after 607 s, 3.38 tokens/s. Inferred: whole prompt about 4,650 tokens, reading to finish about 18:08, reply writes at about 1.5 tokens/s. Result not seen. Follow-up questions in the same chat reuse the cached prompt.
• Distributing inference over Clone and Live (llama.cpp RPC): possible, but no speedup for a single stream, WAN latency, no authentication. Not recommended. Live's CPU count, RAM and AVX2 not yet checked.

Shell note

• The orange highlight on grep is an interactive-shell alias (grep --color=auto from /etc/profile.d/colorgrep.sh). Scripts do not expand aliases. For colour in a script, add a final stage: | grep --color=always -E 'progress|new prompt|n_prompt_tokens|$'.

Still to do (nothing started without his go)

1. qwen_chat.py (v2.0.1, options --url --system --max-new --temp --top-p --top --max-context --beat --log --resume --check): v2.0.2 to fix status line printing into streamed text and to show thinking tokens. Needs his go.
2. Qwen3-ASR with llama-mtmd-cli (the original item from checkpoint 1; not started).
3. Confirm Python 3.15 results on Live (blake2 expected 0909377ad35110ca ec308c07c83582c6, pip version, ssl).
4. Start the server at boot; model_inspector.py; two-LLM relay; older threads listed in checkpoint 1.
5. Python 3.15 build recipe is covered above; no separate file needed unless he asks.