Problem & Resolution Report

System: ISE / PDFsearch.py Component: ISEpdf quoted-phrase search Date discovered: 2026-09-09 Status: Verified and fixed (patch not yet deployed to Dev/Live)

Symptom

Quoted phrase search for a capitalized term (e.g. "Ian") returns no matches, even when:

• the word genuinely appears in the target PDF's real text layer (confirmed via the PDF.js viewer's own in-document search, which found all occurrences)
• the same word, searched unquoted (bare word mode), returns correct results
• the word is confirmed present in the index itself

Discovered while testing ISE search against a specific attachment (id_attach 6204, the most recently uploaded PDF at the time), containing discussion of Andrew's friend Ian Rashford.

Investigation trail (in order)

1. Ruled out: PDF.js rendering/origin issues — unrelated bug found and fixed earlier in the same session (Dev boardurl reverted by a stale DB-restore step in Andrew's backup pipeline).
2. Ruled out: attachment 6204 not indexed — confirmed present in pdf_store.json and word_index_pdf.json via direct file grep.
3. Ruled out: OCR/text-layer limitation — confirmed via PDF.js's own in-document search that "Ian" is real, selectable text, not an image.
4. Ruled out: min-length tokenizer filter, stale in-memory cache, off-by-one truncation on the match list — checked via direct Python inspection of word_index_pdf.json; the "ian" key's match list correctly includes [6204, 22], [6204, 23], [6204, 24].
5. Confirmed root cause: word_index_pdf.json is lowercase-only at the key level ("ian" exists; "Ian" does not). PDFsearch.py's phrase-mode ("literal") branch does a raw, case-sensitive exact-key lookup against that index ([pw] if pw in word_index else []), which can never match a capitalized query term against an all-lowercase index.

Root cause (technical)

PDFsearch.py, search(), phrase-mode branch, line 334 (pre-fix):

matched_keys = [pw] if pw in word_index else []

Word-mode search, by contrast, already folds through a case-insensitive lookup (folded_index.get(term.lower(), [])). Phrase-mode was never updated to match when that folding was introduced (see Risk Analysis report for the likely regression history).

Fix

matched_keys = folded_index.get(pw.lower(), [])

One-line change, same file, same function. See attached pdfsearch_v_fix.diff and the Code Change Report for full detail.

Verification

Verified via an isolated test harness (test_phrase_bug.py) using the actual tokenise()/search() code from PDFsearch.py (copied verbatim, not reimplemented), run against synthetic index data shaped exactly like the real confirmed word_index_pdf.json structure, including Andrew's real confirmed data for "ian" / attach 6204.

Query	Mode	Pre-fix finds 6204?	Post-fix finds 6204?
Ian	word	True	True
"Ian"	literal	False	True
"Ian "	literal	False	True
"ian"	literal	True	True

• Bare word-mode search already worked pre-fix (confirms word-mode was never the problem).
• Both quoted forms (including Andrew's actual failing command, '"Ian "') reproduce the failure pre-fix and are resolved post-fix.
• Already-lowercase quoted phrases still work post-fix — no regression.

Not yet done

• Patch verified in isolation only — not yet applied to the real file on Dev or Live (patched copy produced this session, pending deployment).
• MiscSearch.py / posts-search have not been checked for the same pattern against their own indexes. Case-sensitivity was recorded as "unified across all 3 sources" in an earlier v7.4 checkpoint — that claim has not been re-verified against this specific failure mode.