name: isemedia-clone description: ISEmedia (AI image/video search for ISE) build + the CentOS 6.10 glibc-upgrade Clone VM saga — RankingEngine v9.0 SimilarityRule, MediaSearch v0.2, and every /etc rsync-leak lesson learned aliases: [ISEmedia, Clone, forum-glibc-clone, MediaIndexer, MediaSearch]
Checkpoint 2026-08-23/24 (v9.0 RankingEngine/MediaSearch wiring + Clone VM disaster recovery)
- ISEmedia is a new unified AI-powered image+video search source for The ISE Project, eventually a 4th interlaced source in ISEmega — CLIP (open_clip ViT-B-32) for embeddings, optional Whisper for video transcription. Design principle: Live never runs torch, only plain cosine-similarity math on precomputed vectors — actual inference must happen elsewhere
- MediaIndexer.py v0.1 delivered — has 2 unresolved stubs (resolve_attachment_dir(), attachment-id-from-filename derivation) needing real attachment DB/dir layout to finish; a real forum post with real attachments (uploaded 5 test images + 1 video via genuine SMF upload) supplies that data once picked back up
- Confirmed empirically: Live's glibc 2.12 categorically can't run torch (ImportError on GLIBC_2.14) — all-or-nothing wall for both images and video, not video-specific. Docker ruled out (Live's kernel 2.6.32 predates Docker's requirement). Direct glibc upgrade on Live rejected as too dangerous
- Resolution: disposable CentOS 6.10 "Clone" VM via VMware Workstation on Windows 10 host (E:\VMware\VMs), used to trial the glibc upgrade safely. Fallback if that fails: reinstall Clone as Ubuntu/Debian instead (modern glibc, no version archaeology)
- RankingEngine.py bumped to v9.0: added SimilarityRule (converts MediaSearch.py's 0-1 cosine similarity into RankingEngine's additive point scale, capped +150 at a perfect match) and a MEDIA_RULES set (SimilarityRule + BoardWeightRule + RecencyRule + PenaltyRule) — deliberately excludes the text-only rules (QueryMatchRule/LocationRule/etc.) since LocationRule always adds +20 as a "matched in reply body" fallback, nonsensical for media with no body text. Added rank_media()/score_media() convenience methods + media_ranking_engine() factory. Verified against real runnable synthetic-data tests, not just reasoned about
- MediaSearch.py bumped to v0.2: search() now merges attachment_id/similarity onto each result dict and feeds straight into media_ranking_engine().rank_media() by default, matching how ISE/ISEpdf/ISEmisc already flow through RankingEngine. Verified end-to-end against the mock media_store.json
- Real CLIP embedding on the actual uploaded images/video remains blocked as of this checkpoint: sandbox's egress allowlist blocks both huggingface.co and openaipublic.azureedge.net (the two places open_clip pulls pretrained weights from) — torch/open_clip themselves install fine via PyPI, only weight download is blocked. Waiting on Clone to have both working glibc/torch AND real internet access to do this properly
- Mock media_store.json (3 synthetic clusters: "backdropping," "pill testing," random noise, one transcript-pending video) delivered twice this session as media_store_mock.json — explicitly fake data, useful only for testing MediaSearch/RankingEngine logic, not a preview of real results
Clone VM build — critical rsync exclude lessons (two VMs died from these before it was sorted)
- rsync must always run as a PULL, executed FROM Clone (Live cannot reach Clone, only the reverse). Always use Live's real IP (128.199.200.202), never the hostname — DNS/hosts resolution from a fresh Clone is unreliable
- Confirmed root causes of two separate dead/unbootable Clone VMs this session, in order discovered: (1) /boot never excluded — Clone's own kernel/initrd/GRUB got overwritten with Live's, referencing a different disk/UUID entirely, black-screen/no-GRUB unbootable, no snapshot existed to revert, VM rebuilt from scratch; (2) /etc/fstab never excluded — Clone inherited Live's fstab referencing Live's own disk LABEL=DOROOT, which didn't match Clone's actual disk label; this one WAS recoverable via blkid + e2label
- A cascade of further /etc-rooted leaks discovered one at a time this session, all traced to the same root cause (rsync copying /etc wholesale from Live with no exclusions): /etc/hosts (recurred — same loopback self-resolution trap as a prior session: Live's hosts file maps forum.drugs-and-users.org to ::1/127.0.0.1, breaking Clone's own SSH-to-Live attempts), /etc/passwd + /etc/shadow (root password/account mismatch — Clone was authenticating against Live's password hash, explaining "logged off, can't log back in via SSH, but console/root still works"), /etc/sysconfig/network + network-scripts (lost Clone's own hostname/static IP), /etc/sysconfig/iptables (Live's firewall rules/bans transplanted onto Clone), /etc/ssh/sshd_config (Live's AllowUsers IP whitelist doesn't include Clone's actual connecting IPs), /etc/my.cnf (Live-specific bind-address/datadir settings broke MySQL auth, needed a full safe-mode root password reset)
- Standing fix going forward: exclude ALL of /etc wholesale by default rather than whack-a-moling individual files; pull back only specific confirmed-needed app-config files (httpd vhosts, php.ini) by hand, reviewed, never blind-copied. /boot excluded separately (unrelated category, same "different machine's boot chain" risk)
- Other real bugs hit and fixed this session: rsync -e flag needs its own space-separated arg (a bundled "-zpaAPve -ssh" silently breaks the transfer — rsync reads "-ssh" as a nonexistent remote-shell command); -X/--xattrs spammed permission-denied errors because Live has SELinux disabled while Clone had it enforcing — fixed by disabling SELinux on Clone entirely (setenforce 0 + /etc/selinux/config) and dropping -X; missing --partial/-P meant a phone dying mid-transfer lost the in-flight file entirely rather than resuming; CRLF line-ending corruption on a script edited via Windows Notepad broke set -e, every line-continuation, and the rsync destination path simultaneously — diagnostic tell was literal "#015" (octal for \r) appearing in an rsync error, fixed via sed -i 's/\r$//'; /swapfile must never be copied, always regenerated fresh sized to Clone's own RAM; /root/* must be excluded or written to only AFTER rsync completes (same-path overwrite silently clobbered freshly-written files early in a prior session)
- fail2ban should be disabled before heavy iterative SSH troubleshooting on a disposable test box, to avoid self-banning: service fail2ban stop && chkconfig fail2ban off (confirmed fully off via chkconfig --list)
- Backup strategy confirmed sound: shut Clone down cleanly (fully "Powered Off," not "Suspended"), then copy the whole VM folder on the Windows host to a sibling directory (e.g. VMs-pre-glibc-upgrade) — simpler and more reliable than VMware snapshots, since a plain file copy of a powered-off VM is a complete, consistent restore point
- End state this session: Clone is up, booting correctly, SMF running, MySQL root password reset and working, mysqld set to autostart (chkconfig mysqld on), fail2ban disabled, network working. Both AI-drafted final clone.sh rewrites (one excluding all of /etc, one excluding only the specific known-bad /etc files) were declined as not good enough — decision made to rebuild the exclude list directly from the full session transcript + personal notes rather than continue iterating on AI-drafted versions
- Full verbatim session log (both this session and the prior ISEmedia-design session) written to ISEmedia_Clone_Session_Log.md/.pdf and delivered, specifically for reference during any future re-clone
- Still pending, untouched since before this session: ISEmega result-card design for ISEmedia (thumbnail + emoji type badge, similarity % display); the actual glibc compile/install attempt on Clone once it's stable; wiring MediaIndexer.py's two stubs to the real attachment DB layout now that a real test post with attachments exists