ISE - Intelligent Search Engine for Posts and Their Attachments

[smfadmin (OP)]

ISE - Intelligent Search Engine for Posts and any/all Attachments!

Delete dupe PDFs in progress:
https://forum.drugs-and-users.org/The_ISE_Project/pdf_dupe_review.php

Current version is ISE_v7.4.2 for SMF 2.0.19 -- Download using the table below:

📎 Click on me to view them list of attachmentsTheme: Deep IndigoMidnight DarkPink & PurpleSlate GraySolarized LightEmerald GreenCyberpunk NeonNordDraculaSunset OrangeWarm AmberOcean BreezeClassic LightCrimson DuskNo theme (plain)

ListAttBBC  |  v6.3  |  2026-08-17  |  Andrew.human & Claude.ai

#	File	Size	Downloads	Info	DL
00	Claude on PDF search not finding _listatt_ occurrences  and ISE 5.5 and PDF Infexing.pdf	4.2 MB	2	ℹ️	⬇️
Attachment ID 5355 Filename Claude on PDF search not finding _listatt_ occurrences  and ISE 5.5 and PDF Infexing.pdf File extension pdf MIME type Size 4.2 MB (4368121 bytes) Downloads 2 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash f6144a38d27ea9219795322ec4166c7faf39875b Approved Yes
smfadmin — August 16, 2026, 11:19:45 AM Chip — August 16, 2026, 06:23:46 AM
01	ISE_Reset_Flags_Bootstrap_v11.0.zip	2.8 KB	0	ℹ️	⬇️
Attachment ID 5473 Filename ISE_Reset_Flags_Bootstrap_v11.0.zip File extension zip MIME type Size 2.8 KB (2823 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 6e9ccd6fc55a649cd4be34b95796d1ba8b541ce2 Approved Yes
No recorded downloads yet.
02	ISE_v6.0.zip	6.7 MB	0	ℹ️	⬇️
Attachment ID 5475 Filename ISE_v6.0.zip File extension zip MIME type Size 6.7 MB (7021379 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 97e9d7204113c57bee07e619baf0644cd827419b Approved Yes
No recorded downloads yet.
03	ISEmisc_Checkpoint_for_ISE_v6.0.md	1.1 KB	0	ℹ️	⬇️
Attachment ID 5476 Filename ISEmisc_Checkpoint_for_ISE_v6.0.md File extension md MIME type Size 1.1 KB (1092 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 703425ab67117e9bb94c1bc41bde3a78859f4b58 Approved Yes
No recorded downloads yet.
04	Markdown Rendering Issues for code blocks for Andrew and ISEmisc TEXT SEARCH v6,10 final phase.pdf	40.5 MB	23	ℹ️	⬇️
Attachment ID 5477 Filename Markdown Rendering Issues for code blocks for Andrew and ISEmisc TEXT SEARCH v6,10 final phase.pdf File extension pdf MIME type Size 40.5 MB (42425051 bytes) Downloads 23 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash dae76c7afbb4439a82cd580f96027bfba8017487 Approved Yes
Chip — August 16, 2026, 11:01:05 PM smfadmin — August 16, 2026, 11:19:45 AM
05	pdfjs-link.zip	1.6 KB	0	ℹ️	⬇️
Attachment ID 5478 Filename pdfjs-link.zip File extension zip MIME type Size 1.6 KB (1645 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 4336c71546d33e6110780d8cb5efd2c15cf9dd58 Approved Yes
No recorded downloads yet.
06	ISE_v6.1.zip	6.7 MB	4	ℹ️	⬇️
Attachment ID 5483 Filename ISE_v6.1.zip File extension zip MIME type Size 6.7 MB (7049788 bytes) Downloads 4 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash eccd1248b63a98825e35098d1f8383bd0d277dfb Approved Yes
smfadmin — August 16, 2026, 11:20:04 AM
07	ISE_v5.5.zip	4.3 MB	0	ℹ️	⬇️
Attachment ID 5484 Filename ISE_v5.5.zip File extension zip MIME type Size 4.3 MB (4459249 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash a2468a70c1b3c99a3796e716d59f84388e0a3fdb Approved Yes
No recorded downloads yet.
08	ISE_v7.0.zip	6.7 MB	0	ℹ️	⬇️
Attachment ID 5537 Filename ISE_v7.0.zip File extension zip MIME type Size 6.7 MB (7021106 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 4ac3ffc958903642112e7320c3ae1a1cd1f5a3fa Approved Yes
No recorded downloads yet.
09	ISE_Manual.md	16.5 KB	0	ℹ️	⬇️
Attachment ID 5539 Filename ISE_Manual.md File extension md MIME type Size 16.5 KB (16872 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash b7469ebae5818864f84d060d1e568fc93612a437 Approved Yes
No recorded downloads yet.
10	ISE_Python_Scripts_Manual.md	22.2 KB	0	ℹ️	⬇️
Attachment ID 5540 Filename ISE_Python_Scripts_Manual.md File extension md MIME type Size 22.2 KB (22747 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 91707754e48e175e3444804983663c1931c1bf94 Approved Yes
No recorded downloads yet.
11	checkpoint_v7.1.md	6.5 KB	0	ℹ️	⬇️
Attachment ID 5541 Filename checkpoint_v7.1.md File extension md MIME type Size 6.5 KB (6661 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash c385c247822048ee434482c1fdc58113219fbf1b Approved Yes
No recorded downloads yet.
12	Changelog.txt	1.1 KB	0	ℹ️	⬇️
Attachment ID 5544 Filename Changelog.txt File extension txt MIME type Size 1.1 KB (1133 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 7da0a5e631610bba4d87cf242746d918a061db4b Approved Yes
No recorded downloads yet.
13	More ISE functionality - stopping at ISE v7.1.pdf	2 MB	1	ℹ️	⬇️
Attachment ID 5545 Filename More ISE functionality - stopping at ISE v7.1.pdf File extension pdf MIME type Size 2 MB (2140594 bytes) Downloads 1 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 98d4b40aef723c0576ab13280ad8ad652606ff34 Approved Yes
Chip — August 16, 2026, 07:56:40 PM
14	ISE_v7.1.zip	6.7 MB	4	ℹ️	⬇️
Attachment ID 5546 Filename ISE_v7.1.zip File extension zip MIME type Size 6.7 MB (7033912 bytes) Downloads 4 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash db48bce197f40b16b06757b0717ceb2ab2e17ced Approved Yes
smfadmin — August 17, 2026, 07:14:55 AM
15	the rest of the ISE Functionality.pdf	1.3 MB	0	ℹ️	⬇️
Attachment ID 5548 Filename the rest of the ISE Functionality.pdf File extension pdf MIME type Size 1.3 MB (1321498 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 1 Thumbnail attach ID none File hash 0babe04d4fc00447e145c64c063c479ad83a0b8b Approved Yes
No recorded downloads yet.
16	ISE_v7.2.zip	7 MB	0	ℹ️	⬇️
Attachment ID 5718 Filename ISE_v7.2.zip File extension zip MIME type Size 7 MB (7296934 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 2 Thumbnail attach ID none File hash 4c8340835c3b221dff79970c76b755dcfc5d9e50 Approved Yes
No recorded downloads yet.
17	ISE_v7.3.zip	6.7 MB	3	ℹ️	⬇️
Attachment ID 5720 Filename ISE_v7.3.zip File extension zip MIME type Size 6.7 MB (7055832 bytes) Downloads 3 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 2 Thumbnail attach ID none File hash 8ee2ee18182efd7ccba44a9e6aae2613bb0cf186 Approved Yes
Chip — Today at 05:35:39 AM
18	ISE_v7_4_1.zip	6.9 MB	0	ℹ️	⬇️
Attachment ID 5738 Filename ISE_v7_4_1.zip File extension zip MIME type Size 6.9 MB (7237011 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 2 Thumbnail attach ID none File hash 845f2869c6a294ab9e7f85a042dff1f5d79f4753 Approved Yes
No recorded downloads yet.
19	checkpoint_v7_4_1.md	4.8 KB	0	ℹ️	⬇️
Attachment ID 5739 Filename checkpoint_v7_4_1.md File extension md MIME type Size 4.8 KB (4871 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 2 Thumbnail attach ID none File hash 7ae5a50d1d9b329ae4ccdd3cc0a6bebedae6e31d Approved Yes
No recorded downloads yet.
20	ISE_v7.4.2.zip	7 MB	0	ℹ️	⬇️
Attachment ID 5740 Filename ISE_v7.4.2.zip File extension zip MIME type Size 7 MB (7332183 bytes) Downloads 0 Dimensions n/a Uploaded by Guest Message ID 49422 Attachment type Normal attachment (0) Storage folder ID 2 Thumbnail attach ID none File hash 4578d405f7f34ed5000294767bccc28b5329056b Approved Yes
No recorded downloads yet.

Do NOT use two autodisplays !


[lmv=autodisplay-scroll] AUTO with SCROLL[ /lmv]
[pdf=autodisplay] AUTO [/pdf]

ISE - Intelligent Search Engine for Posts and Attachments

WHAT THIS IS

Adds a custom search engine option called "ISE" (Intelligent Search
Engine) to your forum's search-engine drop-down box, alongside the
existing Default/Google/Yandex/Nigma options.

ISE is a private, self-hosted search tool: a pure Python search engine (QueryParser -> Tokeniser -> ANDMatcher -> SortEngine -> ResultFormatter) that reads directly from your forum's own posts and returns matching results as clickable links, sorted newest first.

This mod also adds search-query logging (via syslog) for both the
native SMF search box and the ISE dropdown option, so every real search your users type gets recorded for later review.

REQUISITE MODS

1. Board-color-and-icons (Version 1.1)
2. Automatic Package Version Emulation (Version 1.6)
3. CustomSearch (Versi
on 1.0) -- Install via Emulating 2.0

Install the above, in order, before installing this package.

FEATURES

- WORD matching:   any_search_term
- PHRASE matching: "an exact quoted phrase" (whitespace-exact, no normalization -- a double space inside quotes is matched literally)
- Mixed queries:   word1 word2 "an exact phrase" -- all terms must match (AND logic) for a post to qualify
- Results sorted newest-first
- Every search (native SMF box, ISE dropdown, and the site's Google
  search widget) is logged via syslog: username/Guest, IP address,
  timestamp, and the raw query string

WHAT GETS INSTALLED

- Sources/QueryFrontend.php -- syslog logging function

- Sources/Search.php (patched)  -- adds the "ISE" case to the search-engine dropdown handler

- Sources/ManageSettings.php (patched) -- adds an admin toggle, "Use ISE search"

- Themes/default/index.template.php (patched) -- loads
  widget_write_log.js and adds the "ISE" option to the dropdown

- Themes/default/scripts/widget_write_log.js -- logs searches made
  through the site's separate Google-search widget

- Themes/default/languages/Modifications.english.php (patched)
-- adds the "ISE" and "Use ISE search" language strings
  (both languages/ and language/ directory spellings are patched,
  since this varies by SMF install)

- $boarddir/The_ISE_Project/  -- the Python search engine itself

  (QueryParser.py, Tokeniser.py, IndexBuilder.py, IndexLookup.py,
  ANDMatcher.py, SortEngine.py, ResultFormatter.py, search.py,
  qf_search_bridge.php, daily_queries.py, query_log_reader.py)

 -- protected from direct web access via its own .htaccess

- $boarddir/ISE_Data/  -- holds the generated search index
  (word_index.json, post_store.json) -- created with 0700
  permissions and its own .htaccess, since this data is a full,
  searchable copy of every post on your forum

- The full Mozilla PDF Javascript libraries at $boarddier/pdfjs

REQUIREMENTS -- READ BEFORE INSTALLING

This mod's PHP/dropdown/logging pieces install automatically through
the Package Manager. The Python search engine itself does NOT run
automatically -- it needs a few things set up on the server first,
by hand, once:

 - Python 3.6+ available on the server, with the PyMySQL package
   installed for whichever interpreter you'll use to run these
   scripts: python3 -m pip install pymysql

AFTER INSTALLING

The Python search engine itself does NOT run by default so folow this carefully:

1. cd ..whatever-your-webroot-is../The_ISE_Project

   python3 IndexBuilder.py

2. cd ..whatever-your-webroot-is../The_ISE_Project
 
   python3 PDFIndexer.py --monitor

3. Go to Admin > Configuration > Modificication Settings > CustomSearch > ISE and ISEpdf uncheck both and Save then recheck both and Save again.

4. Go to the top right of the page and now the dropdown will have ISE and ISEpdf so now you can use it but wait until the indexing has finished.

5. Test a search using the ISE option from the dropdown to confirm everything is wired up correctly before relying on it.

6. crontab -u apache -e

MAILTO=apache
0 */2 * * * python3 /var/www/html/The_ISE_Project/IndexBuilder.py
0 3 * * 0 python3 /var/www/html/The_ISE_Project/PDFIndexer.py --monitor

NOTES

- Search queries are logged via syslog (facility LOCAL0, tag smf-queries) -- typically landing in /var/log/messages depending on your syslog configuration. Two small scripts are included to help review this log:

- The_ISE_Project/daily_queries.py -- lists today's searches
- The_ISE_Project/query_log_reader.py  -- lists the full history

PRIVACY NOTE

Once enabled, every real user's search terms are logged, including guests (identified by IP address rather than username). Given the sensitive nature of a harm-reduction forum's likely search content, review who has access to your server's syslog output and how long you want to retain this history before enabling this for your full userbase.

[smfadmin (OP)]

Topic bumped