One file replacing four: Checkpoint_new3 (6-7 Oct, ISE v10 close-out), Checkpoint_carried_fwd (14:28), Checkpoint_new1 (20:30), Checkpoint_new2 (COB, stale-entries). Merged in that order; where two disagree the later one wins and the conflict is listed in section 2. Tags: PROVEN (Andrew's output / my test), CONFIRMED (Andrew saw it work), UNCONFIRMED (delivered, not reported installed).
zoneinfo), server timezone +0800 (PHP Asia/Singapore), 3 hours behind Sydney. DB smf209, prefix smf209_. No credentials are recorded here or in memory./var/www/html/The_ISE_Project. Data: /var/www/html/ISE_Data. Scripts live in /bin./var/www/html/attachments and attachments2 (from smf209_settings.attachmentUploadDir), on-disk name {id_attach}_{file_hash}. Never search the disk by original file name.ISE_Data/misc_store.json is 83 MB on one line: use Python s.find(), never backtracking grep.cp -i, iptables (use \cp -a). Clone: rm is rm -i; glibc 2.12, only manylinux2010-or-older wheels; audio venv /root/venv-audio-test/bin/python (run cd / && ...AudioProcessor.py).audio_store.json (arrives via Clone push).media_store.json; MediaSearch builds from media_store.json + audio_store.json + captions_store.json.ListAttBBC::defaultSelectHref(), rows in buildHtml(), viewHref = Sources/view_attach.php?attach=N&topic=T, dlHref = index.php?action=dlattach;topic=T.0;attach=N. listatt 6.14 runs on SMF 2.0.x and 2.1.x, needs CollabCore (MsgTracker). Collaborative 5.0: Collabware to $boarddir, CollabCore to $sourcedir, via require-dir.| Module | Latest known | Carried by | State |
|---|---|---|---|
| SearchCommon | 1.2.0 (Live has 1.1.0, md5 a31d5461...) | ISE_NameHitRanking_v1.0.0 | 1.1.0 PROVEN on Live; 1.2.0 UNCONFIRMED |
| RankingEngine | 7.5.0 | NameHitRanking | UNCONFIRMED |
| ise_render | 8.0#6 | NameHitRanking | UNCONFIRMED |
| ResultFormatter | 7.4.5 (7.4.4 never installed; Live runs sed hotfix) | NameHitRanking | UNCONFIRMED |
| MediaSearch | 1.10.3 | ISEmedia_stale_entries_v1.0.4 | deployed Clone/Live/Dev (new2) -- see conflict A |
| MiscSearch | 7.7.1 | ISE_SearchCommon_v1.1.0 | PROVEN on Live |
| PDFsearch | 7.8.1 | ISE_SearchCommon_v1.1.0 | PROVEN on Live |
| search.py | 7.6.1 | stale_entries_v1.0.4 | deployed |
| MediaIndexer | ISE v9.9.9 | stale_entries_v1.0.4 | deployed (9.9.5-9.9.7 and the hourly cron: see below) |
| MediaProcessor | ISE v9.27 | stale_entries_v1.0.4 | deployed |
| IndexBuilder | 7.4.4 | stale_entries_v1.0.4 | deployed |
| IndexLookup | 7.4.1#3 | stale_entries_v1.0.4 | deployed |
| AudioProcessor | 1.2.3 (Andrew's); stale-entries adds --prune -- see conflict B |
stale_entries_v1.0.4 | deployed |
| PHP bridges | qf_search 1.2, Media 1.1.1, Mega 8.2, Misc 1.1, PDF 1.1 | ISE_Bridges_Logging_v1.0.0 | CONFIRMED on Live |
| ise_trace | 1.1.0 | ise_trace_v1.1.0 | UNCONFIRMED |
| find_attachment | 1.0.1 | find_attachment_v1.0.1 | PROVEN on Live |
| audio_features2 | 1.0.0 / audio_features_list 1.0.6 | own zips | delivered, deployment not reported |
| ISEmega | v8.0 (MegaSearch queries posts/PDF/Misc only, no media) | -- | unchanged |
--version/--help and the stale-entry skip. Same risk for search.py (ranking zip says no edit needed, so no clash expected). Rule: do NOT install NameHitRanking_v1.0.0 as is. Rebuild it as a new dotted version on top of what is on Live, once the Live versions/md5 are known.--prune/--version. Either the version was not bumped (breaks the delivery rule) or Andrew's earlier file was 1.2.2. Check the file on Live.--version. The hourly cron (7 * * * * ...MediaIndexer.py, wipe guard, flock, --force) is in 9.9.5-9.9.7; assume 9.9.9 keeps it, not shown.Sensuous* found nothing: wildcard terms were anchored to the whole lowercased file name. Fix: shared SearchCommon.py; a term with */? matches the whole name OR any single word (split on spaces and [\W_]+, Unicode). Case-insensitive; plain term = substring; quoted phrases match whole name only; Misc stays case-sensitive (literal_cs=True); Media treats */? in quotes as plain text; empty name matches * in Misc/PDF, Media skips. PDFsearch keeps its own tokenise().ISE_SearchCommon_v1.1.0.zip (1.0.0/1.0.1 superseded). md5: SearchCommon a31d54617669eddf7dc7412d20f4884b, MediaSearch 1.9.1 320ea8b0..., MiscSearch 7.7.1 6c3fdd4d..., PDFsearch 7.8.1 036a973b....*.mp3 is a Media-only idea; wildcards in double-quoted phrases remain unsupported by design.from zoneinfo import ZoneInfo (ResultFormatter line 77) fails on Python 3.8. Hotfix on Live: /bin/date +%z via subprocess (567 posts, exit 0, PROVEN). Backup ResultFormatter.py.bak_zoneinfo._os_timezone() with fixed UTC+8 fallback; display now follows the server OS timezone rather than fixed Singapore. Decision still Andrew's: follow server, or always Singapore. 7.4.5 (in the ranking zip) builds on 7.4.4.djjamestobin.m4a (id 7189) not found (new3, then cp1)m4a. v9.9.4 added it (PROVEN, 2807 items). cp1 versions extended the filter to mp4 webm 3gp jpg jpeg png webp gif mp3 wav m4a flac ogg aac wma./var/www/html/djjamestobin.m4a is a separate public file (tidy-up).os.replace), flock lock (exit 3 busy / 4 lockfile), shrink guard MIN_KEEP_RATIO = 0.90 (exit 2, --force), MEDIAINDEXER_RUN ... summary line, [+] added ... listing. Suggested cron NOT installed: 7 * * * * /usr/bin/python3 /var/www/html/The_ISE_Project/MediaIndexer.py >> /var/log/MediaIndexer.log 2>&1. Superseded zips: v9.9.5 f1943862..., v9.9.6 895066f6...; v9.9.7 e1f5f63e... (file md5 faf1d5a6...).ISE_Bridges_Logging_v1.0.0.zip (md5 5d64648d...): output written to qf_debug.txt only when no result page was produced, capped at 2000 chars. qf_debug.txt 376 bytes on Live; old logs deleted.ise_trace.py v1.1.0 (md5 0828bf12...; zip e62ba798...) caps list/string/line/depth. UNCONFIRMED; ise_trace.log was 690057 bytes, re-measure after install.--json explicitly; [] vs non-JSON output is how it tells zero results from a crash.LocationRule checks only result["subject"]; OccurrenceRule scores a name-only hit near zero.ISE_NameHitRanking_v1.0.0.zip (md5 eb4acd9631287581800da9466e63d624): SearchCommon 1.2.0 (name_tier(), apply_name_priority(), 36 self-checks), RankingEngine 7.5.0 ("Name Hit" step, name_priority=False switch), ResultFormatter 7.4.5, ise_render 8.0#6 (ISEmega interlace), MediaSearch 1.10.0. All terms in the name: +2 x gap; some terms: +1 x gap; gap = max(1000, score spread + 1). Explicit /sort modes untouched.--limit BEFORE ranking; a name-only PDF could be cut. Not changed.--version (new2) -- deployed, verified Clone/Live/Devaudio_store.json kept them. MediaSearch now skips and traces results whose attachment is gone; AudioProcessor reports stale entries and has --prune (no backup, by Andrew's call). Run on Clone: removed #7210, #7214 and 128 long-mix windows; store 71 items / 1627 windows; pushed to Live; Dev fixed.--version/-V on seven modules, identical line <File>.py <version> (2026-10-07), handled before other imports; only the exact single argument is caught (the literal queries --version, -V, --help, -h can no longer be searched in MediaSearch; --version/-V in search.py).ISEmedia_stale_entries_v1.0.4.zip (v1.0.0-1.0.3 superseded).ISE_Data/*.json (Clone == Dev; Live ahead on post_store/word_index; the "second diff" failure was set -e vs diff exit 1, fix || true).transformers; the working CLAP environment is the venv. My mistake: I had Andrew install transformers 4.31.0/tokenizers 0.12.1 into system py38 and a fake safetensors-0.3.1.dist-info. Removal told, UNCONFIRMED: rm -rf the dist-info, optionally py38 -m pip uninstall -y transformers tokenizers.--attachid N for one file; --features-only only touches items with missing/out-of-date measurements, --force redoes (not with changed sampling options); --verbose for the full report.transcript.wav measured (3m36s, 6 windows, -18.2 LUFS). 7235 long mix (~134 windows): full run UNCONFIRMED.find_attachment v1.0.1 (md5 585155ac...): rows, disk check, store check. Its "is a key in this store" lines are noisy.audio_features2_v1.0.0.zip, audio_features_list_v1.0.6.zip, AudioProcessor_v1.1.2.zip (not installed), Clone_audio_v1.0.0.zip, ISE_MediaProcessor_Parallel_v1.0.4.zip: delivered earlier; deployment not reported..video/3gpp (2 files): browsers can't play 3gp; "Failed to load media file" in the viewer; VLC plays it.listatt_media_viewer.html CONFIRMED on Live (3906 bytes, rw-rw-rw-); ships in Collaborative (CollabCore).Andrew's direction: (1) a "native" option for all multimedia in [listatt], like the PDF one; (2)(3) an LVI (Lightweight VLC Interface) BBCode driving VLC's HTTP interface, also called from [listatt].
Findings from Collaborative_v5.0.zip and listatt_bbcode_v6.14.zip:
[native] pattern exists in ListAttBBC.php v6.10, pointing at view_attach.php.view_attach.php v1.8: board permission via {query_wanna_see_board}, multi-folder resolution, SMF 2.0/2.1, inline serving, whitelist pdf txt md log csv json html htm; no media types, no HTTP Range (206).listatt_media_viewer.html v1.1 is a plain <audio>/<video>/<img> fed the dlattach URL; seeking through dlattach unverified.defaultSelectHref() has no 3gp, so a 3gp row falls through to the ISE Text Viewer.bufferFill (v6.13) re-renders on every page view, so anything minted at render time is fresh.prep.sh: chmod 777 Sources/CollabCore; viewer 666. Flagged, not changed.Proposed build (awaiting Andrew's go; timing his):
view_attach.php 1.9: media MIME types, 206 Range, same permission check.ListAttBBC.php 6.15: [native] link on image/audio/video rows; unplayable formats (3gp, wma, avi, wmv, most mkv, mpg) get Download + Open in VLC; add 3gp etc. to the extension lists.view_attach.php?attach=N&exp=T&sig=HMAC, endpoint accepts a valid unexpired signature (VLC has no forum cookie), secret on Live, short expiry. LAN-only Clone alias not recommended..m3u/.xspf the OS opens in VLC.Hard rule: do NOT model new code on listatt (the v7.0 disaster). WRAP the working v6.14 code: the PDF [native] link is a four-line change in the row loop; do the same for media. Build on v6.14 and view_attach 1.8 as uploaded, change only what is listed, diff against them.
Baselines (uploaded zips): Collaborative_v5.0.zip 66ba89080d01f103ef953178bc3e0377 (viewer file d03bc883504f8049bfbe9f9e1c1c9246); listatt_bbcode_v6.14.zip 0035738843f003dd5d7abe402afb42ae (ListAttBBC.php ac05f2c3db379e97212e9442e31f1616, view_attach.php f1930429c14eb2d5319bffd18b343bb9). Both matched on 07/10 21:01.
| Risk | Detail |
|---|---|
| Ranking zip vs deployed code (conflict A) | Installing as-is can downgrade MediaSearch 1.10.3. Rebuild on Live's versions as a new dotted version first. |
| Code coherence | The ranking/search files must match on Live, Clone, Dev (standing rule). SearchCommon stack + MediaIndexer 9.9.4 were verified on Live only before new2. |
| Name-hit ranking on mock data only | Test 3-4 real queries across posts / ISEpdf / ISEmisc / ISEmedia / ISEmega. Partial-name tier is strong; switch-off name_priority=False. |
| ResultFormatter | Live runs the sed hotfix with no fallback; 7.4.4+ follows server timezone. Decision open. |
PDFsearch --limit |
Applied before ranking; a name-only PDF could be cut. |
view_attach.php Range |
New code path on a security-sensitive endpoint (earlier permission hole fixed in v1.1). Needs a regression test: guest vs restricted board. |
| Signed links | Anyone with the link can read until expiry: short expiry, rate limit, secret only on Live, mint only for viewers who can see the post. |
| CollabCore 777 / viewer 666 | Existing; fix proposed in Collaborative 5.1, not applied. |
| Browser to VLC may not work | Test page first. |
--prune scope |
Only entries absent from Clone's media_store.json. Run MediaIndexer first (done, 2816 items). Stale entries also sit in captions_store.json / embeddings_store.json. |
| MediaSearch skip uses the local DB | On Clone/Dev an attachment only on Live is skipped there too. |
| MediaIndexer hourly | Cron not installed. New uploads are not catalogued automatically until it is. |
| MediaProcessor (m4a etc.) | No frame/caption branch for audio: counted processed, nothing captioned. Not tested live. |
| Stray py38 installs on Clone | Harmless but misleading to pip. Removal unconfirmed. |
| No SMF code or files edited | Everything delivered is ISE Python; listatt/native work goes through the Package Manager. |
Waiting on Andrew
Possible builds, none started
--features-only found nothing (not in store / already current); manual entry still v1.1.2; the end-of-run "not yet done" count treats stale entries as done until pruned.--limit.captions_store.json / embeddings_store.json (MediaSearch skip covers display only).--version for modules not touched yet: QueryParser, ANDMatcher, RankingEngine, audio_features, Collabware core_utils, etc. Start-up banners still differ per program.Backlog
/var/www/html/djjamestobin.m4a; text/markdown stores possibly missing ids 7185-7199; forum page shows the audio-features report viewer twice (cosmetic).@version and @date in every changed module; manual version-history entry (the manual is a changelog, not prose); every executable shows its version once at start; deliver as a zip with a dotted, never-reused version (X_v1.0.2.zip), never loose files; xmllint --noout on any XML; list unrequested changes BEFORE building; end with a release-card table (done / not needed / missing).Claude: [DD/MM/YYYY--HH:MM:SS:mmm TZ] from running the Sydney-time snippet immediately before replying._N.misc_store.json; read source before claiming what it does; don't suggest pip shims when he wants a non-Python route.Merged checkpoint ends.