Date: 2026-09-23 Milestone: ISEmedia's UI search bridge goes from "never worked when actually clicked" to a real, verified, end-to-end search path on Live — and a real behavior bug in date-sort ranking gets caught and fixed the same night by the same real-user test that found it. Participants: Andrew (Architect & QA Lead, hands-on-keyboard on Live throughout), Claude (this session)
Files touched: qf_Mediasearch_bridge.php, MediaSearch.py, install_ise.php, plus a new ISE_Data/hf_cache runtime directory on Live (not a source file, a populated data artifact). Final versions: qf_Mediasearch_bridge.php 1.0.0 → 1.0.2 · MediaSearch.py 1.0.0 → 1.2.0 · install_ise.php (unversioned/9.4-era) → 9.5
Problem: The very first real UI-triggered ISEmedia search (/sort newest brown) failed outright: ISE ERROR, then on investigation a raw traceback — ModuleNotFoundError: No module named 'numpy'. This was the first time qf_Mediasearch_bridge.php → MediaSearch.py had ever actually been exercised through the real web UI; both files were built the same night as the 2026-09-20/21 ISEmedia checkpoint and had only been tested via CLI/mock data before.
Root cause — three independent, stacked environment bugs, each one uncovered only after fixing the one before it:
Wrong Python interpreter. qf_Mediasearch_bridge.php invoked MediaSearch.py via a bare $python = 'python3';, resolved via shell_exec()'s PATH — a much thinner PATH under Apache than an interactive SSH shell's. Confirmed live: sudo -u apache python3.8 ... itself failed with command not found (sudo's own secure_path doesn't include /usr/local/bin either — same bug class, different program). Real path: /usr/local/bin/python3.8. Fixed via resolve_media_python() — checks two common absolute paths, then a command -v python3.8 shell probe, then falls back to the original bare python3 only as a last resort (so it can never fail worse than before).
Torch importable only via a stray PYTHONPATH, not a real install. With the interpreter fixed, numpy and mysql.connector resolved correctly, but import torch still failed for apache. python3.8 -c "import torch; print(torch.__file__)" as root printed /root/pytorch/torch/__init__.py — not a real source checkout (no setup.py, confirmed via ls -la; just the built torch/ package directory on its own), reachable only because root's shell has /root/pytorch on PYTHONPATH. Apache (a different user, locked out of /root entirely — mode 700) could never see it. Fixed by copying the already-built package straight into python3.8's real system site-packages (cp -rf /root/pytorch/torch "$SITE_PACKAGES/") — no rebuild needed, and it stops depending on any one user's shell environment or /root's permissions.
open_clip's CLIP checkpoint cached per-user, and apache had no cache of its own. With torch fixed, the crash moved further in: open_clip.create_model_and_transforms() tries to download its model checkpoint from Hugging Face Hub on first use, caching under the invoking user's home (~/.cache/huggingface). Already cached under root's home from the 2026-09-01 Live CLIP migration — but apache's home is /var/www, which it can't write to, so every UI search tried a live download and died with PermissionError: /var/www/.cache. Fixed by copying root's existing cache to a new shared location apache CAN read/write — ISE_Data/hf_cache (chown -R apache:apache) — and pointing the bridge at it via HF_HOME in the invocation. (One real slip caught and fixed along the way: running the cp twice before the destination existed created a nested hf_cache/hub/hub/ duplicate — harmless functionally since HF only ever looks at hub/models--... directly, but cleaned up with rm -rf regardless.)
Verification: Confirmed at each layer with a real sudo -u apache ... python3.8 -c "import X" test before moving to the next layer — not assumed fixed from reasoning alone. Final proof was the actual UI returning real, non-error results for /sort newest brown (1001 items, later re-tested at 180 after the limit fix).
Separately, same night — real ranking-quality bug, caught by the first genuine user test: once results were flowing, /sort newest brown returned images with no relation to "brown" at all (e.g. a cat photo, clinic paperwork) ranked above genuine "brown" matches. Two layers:
MediaSearch.py defaulted to unlimited results (no --limit passed by the bridge), so a single-word query returned the entire indexed library (1001 items) ranked, not a filtered set — unlike ISE/PDFsearch/MiscSearch, which all cap at 180. Fixed: bridge now passes --limit 180./sort newest strips out score entirely, same as it does for every other ISE source (SortEngine.py is shared, deliberately, across all four). For posts/PDF/misc that's safe because their word-index lookup already filtered candidates to only things containing the query term before date-sort ever touches them; MediaSearch.py had no equivalent filtering step, so stripping score under date-sort left nothing filtering by relevance at all — /sort newest brown was silently equivalent to /sort newest with no query. First fix attempt (v1.1.0) added keyword_boost() as a score bonus — correct for relevance-mode ranking, but score doesn't matter under date-sort, so it didn't touch this specific complaint. Andrew's explicit call once this was diagnosed: match the rest of ISE exactly — a real query now gates results (literal caption/filename/subject match required) before any sort mode is applied, not just relevance (v1.2.0). Verified against a simulated real-data scenario (Elmo cat + clinic paperwork correctly excluded, three genuine "brown" captions survive, sorted newest-first among themselves) before shipping.Not yet done: Andrew has the v1.0.2/v1.2.0 files in hand but deployment-to-Live-and-retest of this latest keyword-gate change hadn't been confirmed as of this checkpoint. hf_cache itself is live and confirmed working on Live already (separate from the code files).
Bug class: All three interpreter/cache bugs are environment/deployment bugs, not logic bugs in the Python/PHP itself — each one only ever manifests when code that was built and tested as root gets invoked as a different user (apache) for the first time. This is a real, recurring risk class for this project specifically, not a one-off: the exact same "confirmed working interpreter: python3.8" gotcha was already documented once before for MediaProcessor.py in the 2026-09-20/21 checkpoint, and this session found two more instances of the same underlying pattern (torch's PYTHONPATH dependency, HF cache's per-user default) that the earlier fix never covered because they hadn't been hit yet.
Residual risk — real, open: ISE_Data/hf_cache is now a permanent runtime dependency with no automated setup path for a fresh environment. install_ise.php v9.5 adds a best-effort check-and-populate step (cached-copy path if run as root, backgrounded fresh-download path otherwise, loud logged instructions as the last resort) — but this has not yet been tested against a real fresh install; it's new, unexercised code as of this checkpoint. If a future Dev/Clone install runs this and both automated paths fail silently in some way not yet anticipated, the failure mode reverts to exactly tonight's original bug, just on a different box.
Residual risk — behavior trade-off, accepted knowingly: v1.2.0's keyword gate is a deliberate, explicit trade-off Andrew chose after being shown the alternative: MediaSearch.py no longer surfaces a genuinely visually-relevant image whose caption never uses the literal query word — the original point of building CLIP semantic search in the first place. This was flagged plainly before implementing, not discovered after the fact. Worth revisiting if this trade-off feels wrong in practice once used for real.
Blast radius if any of tonight's fixes are wrong: Confined entirely to ISEmedia's UI search path (qf_Mediasearch_bridge.php/MediaSearch.py) and the one MediaIndexer.py line in install_ise.php. IndexBuilder.py/PDFIndexer.py/MiscIndexer.py were deliberately left untouched in the installer specifically to avoid this session's fixes risking three already-working jobs.
Scope check — deliberately NOT done tonight: A cross-source consistency audit of search.py/PDFsearch.py/MiscSearch.py/MegaSearch.py/RankingEngine.py/SortEngine.py against tonight's Media findings was requested but deferred — Claude does not have those files' actual source in this session, only changelog/checkpoint descriptions of them, which Andrew's own changelog already flags as unreliable ("per-file version stamps currently inconsistent"). Real audit pending a source upload (Andrew indicated a full ZIP mod bundle is coming).
qf_Mediasearch_bridge.php — 1.0.0 → 1.0.2resolve_media_python() replaces bare 'python3' — candidate absolute paths, then command -v, then original bare fallback.HF_HOME env var added to the invocation, pointing at ISE_Data/hf_cache; --limit 180 added to match the other three search methods' existing cap.MediaSearch.py — 1.0.0 → 1.2.0keyword_boost() — additive score bonus (+40 caption / +25 filename / +15 subject per literal query-word hit) on top of CLIP-similarity ranking. Verified against 24 real lines of Andrew's actual captions_store.json./sort newest with no real search term is unaffected (nothing to gate by). Results-page footer text updated (no longer claims "not keyword match"). Verified against a simulated real scenario matching the exact live failure Andrew reported.install_ise.php — → v9.5,; in the $jobs array) that meant none of the four indexers were being launched on install at all, not just MediaIndexer.py.MediaIndexer.py's job now resolves python3.8 explicitly (same probe/fallback pattern as the bridge) — IndexBuilder.py/PDFIndexer.py/MiscIndexer.py deliberately left on bare python3 (no evidence they need 3.8; not worth risking three working jobs to "fix" a non-problem).ise_ensure_hf_cache(): checks whether hf_cache is already populated (skip if so); PATH 1 attempts a cached copy from root's existing ~/.cache/huggingface/hub if this process can read it; PATH 2, if PATH 1 isn't available, starts a backgrounded fresh download straight into hf_cache via HF_HOME (non-blocking, its own log file); if neither path can even start, writes loud, specific manual-fix instructions to a status log. Lint-checked (php -l) clean at every stage; not yet tested against a real fresh install.hf_cache_SKELETON.zip — an empty placeholder directory tree matching Hugging Face's real cache layout (blobs/refs/snapshots/.locks), for documentation purposes only; explicitly not meant to be bundled into the distributable mod package (600MB of binary weights, not code).