This is the basic ssh version
This version implements the UI and extensive Python tracing.
The results are now displayed ona webpage.
This version implements the Ranking system
use ISE_Data not ISE_data
Make it truly portable and package it as a mod
Add ISEpdf for PDF search
Multiple fixes to the package for Simple Machines
Upgraded Mozilla PDF Javascript Library version to6.2.108
Add ISEmisc search for most text filetypyes .md,.txt,.text
Add 13 themes theming
Extend misc search for many more unencoded text filetypes -- but all code and conf files are not word searchable (just filenames -- with wildcard support) whilst .md,.txt,.text,.html,.json are !
Split out themes from the 4 scripts into to a shared css
Wildcard support is still WIP for post search.
Text Viewer built and refined - scrollable windowed presentation accross the board
Post download and add Board Id to the index
Real relevance ranking for ISEpdf and ISEmisc, not just match-count sort - both now use the same RankingEngine post search already used, with the same "Why is this ranked #N?" explainer on every result card. ISEmisc results score on every rule (occurrence, subject/filename match, exact phrase, proximity, board weight, recency); ISEpdf scores on everything except exact-phrase/proximity, which its index has no data for (documented in ISE_help.html).
Unified case-sensitivity and quoting across ISE/ISEpdf/ISEmisc - normal words case-insensitive everywhere, "double-quoted phrases" case-sensitive everywhere. ISEmisc now supports real phrase matching against stored file content (was previously silently stripped). ISEpdf now supports phrase-as-AND-on-same-page (was previously unsupported/broken); true adjacent-phrase matching isn't possible for PDF without an indexer change most likely warranting a full reindex.
Fixed PDFIndexer.py's tokenizer (a standing TODO, never previously addressed): it was a bare text.split() with no lowercasing and no punctuation stripping - "meth," "meth)." and "meth" were three separate index entries. Now matches MiscIndexer.py's word-boundary regex tokenizer, lowercased. This is the real fix behind ISEpdf's new case-insensitivity, replacing the query-time workaround PDFsearch.py otherwise needed. REQUIRES A FULL PDF REINDEX to take effect - existing word_index_pdf.json entries were built with the old tokenizer.
Every module that displays, traces, or logs anything now shows its own version - previously only PDFsearch.py and MiscSearch.py did. search.py (main post search) had no version tracking at all before this; added a VERSION constant and wired it through to its HTML results page and CLI output via ResultFormatter.py. Same treatment for IndexBuilder.py, orphan_check.py, daily_queries.py, and query_log_reader.py, none of which had any version marker before. MiscIndexer.py and PDFIndexer.py had version numbers in their doc-blocks already but never printed or logged them at runtime - now they do, both to stdout and their own log files, and the version is included in the trace-log "started" entry for every search/index script. The small support-library files (ANDMatcher.py, QueryParser.py, SortEngine.py, Tokeniser.py) now print a version line from their standalone debug entry points too.
Every module's own version marker is now 7.4.1, matching the package-wide release version - the per-file sub-numbering some files had picked up (7.4#2, 7.4#3, 7.3#5, 7.3#6) is retired in favor of one number across the whole package.
Fixed a real crash caught during Andrew's QA: ISEpdf search returned zero results for every query after v7.4 wired RankingEngine.py into it. Cause was in RankingEngine.py itself, not PDFsearch.py - several rules used result.get(key, "") assuming a missing key falls back to "", but orphaned PDF attachments (no resolvable topic) have subject/text/board_name explicitly set to None rather than omitted, and .get()'s default only applies to a missing key, not a present-but-None one - so .lower() on None crashed the whole search for any query that matched even one orphaned attachment. Fixed every affected rule (Query Match, Location, Proximity, Consecutive Terms, Penalties). RankingEngine.py never had a version marker before this fix.
Fixed a real ranking-quality bug caught during Andrew's QA: searching "chemistry" didn't put an entire book about organic chemistry at the top. Cause: PDFsearch.py set occurrences to len(matched_terms) - the count of distinct QUERY words matched, not how saturated the document actually is with them. For any single-word query, every result got occurrences=1 whether the term appeared once or five hundred times, so a 150-page book entirely on-topic got zero extra credit over a document that merely had the word in its filename. Now uses page_count (how many pages the term actually matched on) - a real signal that was already being tracked but never used for scoring.
That fix immediately exposed a second problem: RankingEngine.py's excessive-repetition ("stuffing") penalty was firing against the book too, because it was reading the same occurrences number without knowing it now meant page-match count for PDF, not same-body repeat count like it does for posts. A legitimate book matching its own topic on 40+ pages was scored as if that were keyword-stuffing. Gated the penalty on the presence of real "text" content, which PDF results never have - post/misc search keeps its genuine stuffing detection unchanged.
Fixed the real remaining ranking bug, caught with an actual screenshot of live results: v7.4.1#2's page_count fix worked correctly, but OccurrenceRule's scoring formula (tuned for post-body repeat counts, which realistically top out around 5-6) capped out by page 7 - a 12-page match and a 62-page match scored identically once both crossed that threshold, so the fix never actually reached the visible ranking. PDF results now score on a wider scale (up to +80, matching Location's weight) so page count keeps differentiating across a realistic document-length range. Verified against Andrew's exact 6-document result set before packaging.
Fixed a real ranking imbalance caught with a second screenshot: after v7.4.1#3, one genuinely-large (62-page) match was still ranking below smaller (19-22 page) matches. Cause: RecencyRule's full swing (up to 29 points) was large enough to override a legitimate content-match advantage - the 62-page document was an older forum upload, the smaller ones were recent, and recency alone flipped the order. For PDF specifically, scaled Recency down to a quarter of its post-search weight: a foundational reference book uploaded years ago isn't less relevant than a smaller document uploaded last month, so content match should dominate and recency should only break close ties. Verified against the exact old-vs-new scenario from the screenshot before packaging.
Fixed OccurrenceRule properly this time instead of raising the point cap again: v7.4.1#3 widened the ceiling from 35 to 80 points, but any fixed additive cap eventually plateaus - a sufficiently dense, comprehensive reference document (PIHKAL, described by Andrew as "dense and packed full of every Phenethylamine invented") still hit the new ceiling and scored the same as a merely-decent match. Now scores by COVERAGE - the fraction of a document's own pages that actually matched, using each document's real page count (added to PDFsearch.py's result data, pulled from pdf_store.json) - instead of raw page-match count. This has no ceiling problem at any document size, and correctly distinguishes a document that's mostly ABOUT the query from one that just happens to match the same raw page count spread across hundreds of mostly-irrelevant pages. Verified with a synthetic 900-page low-coverage document scoring far below a 68-page 91%-coverage one with the identical 62-page raw match count, confirming the fix actually rewards relevance density rather than absolute size.
Duplicate spotter, bug fix for SMF multiple attachment path independence, add wildcard search for posts so now all 3 methods support it and, THE BIG ONE, add caching to pdfjf via a new .htaccess !
ISEmega - search all methods simultaneously
Create 4 new shell themes
Build consistent results page to reflect each method and it's current theme within the results (shell)
Bug fixes phase 1 - standardise ISE (post) handling [tabs, board colours and adjust download buttons for "raw" and "formatted"]
Code weaknesses were identified with repsect to portability so rewrite of the core modules will happen in v8.2 with the introduction of a "Settings.json" file to set a few constants.