Session covered: attachment-directory path independence (major fix across 4 files), the LMV autodisplay-scroll bug (post typo, not code), a duplicated PDF autodisplay block in Display.template.php, and building posts wildcard search (the ISEmega dependency). Everything below is either deployed-and-confirmed-working or delivered-but-not-yet-deployed — marked explicitly per item.
| File | Version | Status |
|---|---|---|
| PDFsearch.py | 7.4.1#4 | Delivered this session — not yet confirmed deployed |
| PDFIndexer.py | 7.4.1#2 | Delivered this session — not yet confirmed deployed |
| MiscIndexer.py | 7.4.1#2 | Delivered this session — not yet confirmed deployed |
| IndexLookup.py | 7.4.1#2 | Delivered this session — confirmed deployed and working (methad* test, see below) |
| pdf_duplicate_spotter.py | 1.3 | Delivered this session — not yet confirmed deployed |
| RankingEngine.py | 7.4.1#5 | Prior session, confirmed working |
| QueryParser.py / Tokeniser.py / ANDMatcher.py | 7.4.1 (unchanged) | No changes needed — wildcard chars already pass through intact |
| Everything else in the package | 7.4.1 | No changes this session |
Root cause found: SMF silently split attachments across a second directory, attachments2 — a sibling path, not nested inside attachments. Every script that scanned attachments assumed a single fixed ATTACH_DIR and had no way to know the second directory existed. Symptom that surfaced it: a newly-attached PDF (acd-form-info-book.pdf, attach-5744) was reported by PDFIndexer.py as "no extractable text" when it had actually never been discovered at all — a discovery bug misread as an extraction bug. Confirmed via find returning nothing under the old path but finding the file under attachments2, and via smf209_settings.attachmentUploadDir (a serialized PHP array) actually holding two entries. Live DB table prefix confirmed: smf209_.
Fix, requested explicitly as "change ALL code to be path independent":
get_attachment_dirs() in PDFsearch.py — reads attachmentUploadDir live from smf209_settings on every call, so it can never drift out of sync with reality again. Includes a small hand-rolled parser for PHP's serialize() format (just enough for this one setting's shape, not a general unserializer).PDFIndexer.py's discover_pdfs() and MiscIndexer.py's discover_misc_files() both now accept a list of directories instead of one string, and scan all of them.pdf_duplicate_spotter.py's find_duplicate_groups() same treatment (it already imported discover_pdfs/ATTACH_DIR from PDFIndexer.py).get_attachment_dirs() at runtime; --attach-dir still works as a manual override and is now repeatable (action="append") instead of a single value. Falls back to the old single ATTACH_DIR only if the DB read fails, loudly (printed warning + log entry), never silently.pdf_dupe_review.php — checked, needed no changes. It only reads the precomputed pdf_duplicates.json and acts through SMF's own removeMessage()/dlattach URLs, never touches a filesystem path directly.orphan_check.py, IndexBuilder.py — checked, confirmed no attachment- directory dependency at all (DB-only / post-text-only), no changes needed.PDFIndexer.py --dry-run --monitor pass first to confirm it reports scanning both attachments and attachments2.Separately flagged, not yet investigated: this same reindex run showed pypdf: 0 total, 0 unique across all 218 PDFs processed. If that run wasn't --fast (which deliberately skips pypdf), that's a red flag that pypdf may be silently broken/uninstalled on the live box, not genuinely finding zero text on every single file. Worth checking python3 -c "import pypdf; print(pypdf.__version__)" on the live box next session if not already done.
[lmv=autodisplay-scroll] — resolved, not a code bugLong debugging thread (SMF parse_bbc() cache-hit theory, checked and ruled out real caching mechanism as a possible cause but it wasn't actually the live issue this time) landed on the real cause: a typo in the post's own BBCode source, not the mod. Andrew confirmed "mistakenly and it all works" after checking raw source. No code changes were needed for this one. The underlying architectural fragility Claude flagged (LMV's autodisplay state is a parse_bbc()-cache-hit- vulnerable static array, would silently stop working forum-wide if cache_enable >= 2 causes a cache hit on repeat views) is still real and undiagnosed as a live problem — worth keeping in mind if LMV autodisplay ever mysteriously stops working forum-wide (not just one post) in the future; the fix design if that ever happens is a small DB-persisted toggle-state table instead of the static array.
A single PDF attachment (acd-form-info-book.pdf, attach-5744, one confirmed smf209_attachments row) was rendering twice on the page. Root cause: Display.template.php had the entire PDF autodisplay block pasted twice, verbatim, back to back (lines 690–694 and 696–700, byte-for-byte identical) — a botched manual patch application, not a logic bug in PDFBBC.php itself. PDFBBC.php's toggle tag ([pdf=autodisplay]) was checked and confirmed to correctly clear its own inline output ($data[0] = '') — it was never the source of the duplicate. Andrew deleted one of the two blocks by hand; confirmed "fixed."
Correction on Claude's part, worth remembering: earlier in this same debugging thread, Claude suggested grep -c "renderAttachmentById" to check for a duplicated patch — that function name only exists in LMVBBCode, not PDFBBCode (which inlines its <iframe> HTML directly via echo rather than calling a named render function). That grep could never have caught this bug. If a similar "is this patch applied twice" question comes up for PDFBBC.php again, search for the actual <iframe> markup or the is_pdf condition, not renderAttachmentById.
Was the last blocker for ISEmega (posts was the only one of the three search sources — Posts/PDF/Misc — without */? wildcard support; PDF and Misc already had it). Turned out to be a single-function fix: IndexLookup.py's lookup_word() was routing any word containing * or ? into its punctuation-fallback branch (literal substring search against post text), which can never match anything since real posts don't contain a literal * character — so wildcard queries were silently returning zero results rather than erroring. QueryParser.py, Tokeniser.py, and ANDMatcher.py needed no changes — wildcard characters already survive parsing (\S+ regex captures them whole) and tokenising (only lowercased, never stripped) intact.
Fix mirrors the exact mechanism already used in PDFsearch.py/ MiscSearch.py: fnmatch.fnmatchcase() against every word_index key (confirmed already-lowercased at build time by IndexBuilder.py, so no extra folding needed), unioning matched words' id_msg sets. Verified against synthetic fixtures (prefix, suffix, single-char wildcard, a wildcard AND'd with a plain word, an untouched phrase query) before delivery. Deployed and confirmed working live — a methad* search that initially looked like a false-positive "matches everything" bug turned out to be a real, correct match (the word "methadone" genuinely present in a result Andrew's browser had visually missed) — not a code bug at all.
Known gap, not built: wildcards inside quoted phrases for posts search. PDF search already supports this (per-word wildcard matching within phrase co-occurrence); posts' phrase matching is a pure literal- substring check against raw post text, a different mechanism entirely, so this would be a separate, larger piece of work if ISEmega ends up needing it.
Design agreed in an earlier session (see main smf-search-query memory for full detail): combined "search everything" page, 3 grouped sections in fixed order Posts/PDF/Misc (not interleaved — the 3 engines' scores aren't comparable), shared Open+Download button module (ise_render.py, not yet written), primary row + secondary collapsible detail panel per card, color-coded by source. Backend bridge wiring (qf_MegaSearch_bridge.php) not started. Posts wildcard search (item 4 above) was the explicit dependency blocking this from starting — that's now clear.
<details> "Show N matching replies" approach for posts results, wants a plain-text discussion of alternatives (no multi-choice option cards), hasn't stated what he wants instead yet.checkpoint — re-upload if starting a fresh conversation and continuing from here)