📝 ISEmedia_MediaSearch_v1.4.1_Viewer-Avatars-VideoIndex.mdTheme:

ISEmedia / MediaSearch — Open-as-View, Avatar Filter, Video Indexing Diagnosis

Transcript — 24/09/2026 — Andrew & Claude

Andrew

📎 listatt_bbcode_v6.12.zip 📎 MediaSearch.py

ISEmedia results "OPEN" initiates a "download" and not a "view" - please model it on listatt

Claude

Claude: [24/09/2026--16:47:37:808 AEST]

I fixed this in MediaSearch.py v1.3.0. The "Open" button now shows the item in the browser instead of downloading it.

Why it downloaded: "Open" linked straight to action=dlattach. SMF sends that as a download for video, and sometimes for images too. Your listatt mod already fixed this same problem by sending media through Sources/CollabCore/listatt_media_viewer.html, which displays the file in a plain <img>, <video> or <audio> tag instead.

What changed:

New build_view_url(): it builds the same viewer link listatt does. It uses the same page, the same mode/file/name settings and the same list of file types per mode, taken from ListAttBBC.php. I added .3gp as a video type. If the file type isn't recognised, it picks video or image based on whether the result came from a video.
Result cards: "Open" goes to the viewer. A new "⬇️ DL" button keeps the direct download link, like listatt's DL button. Clicking the thumbnail also opens the viewer, and the thumbnail image itself still loads from dlattach.
Command-line output: the Viewer: line now shows the viewer link, and a new Download: line shows the direct link.

It compiles, and I tested the link builder on its own with image, video, .3gp and unknown-type inputs. The links match listatt's format, and the viewer reads them back correctly. I haven't run it against your live forum.

One dependency to be aware of: the viewer's header says it's owned by "listatt_bbcode only". ISEmedia now uses it too. If the viewer's settings or location ever change in CollabCore, ISEmedia's Open button will break without any error. You may want to update that ownership note when you next touch CollabCore.

Files are in:

MediaSearch.py

Andrew

another bug - filter OUTR any avatars : like these 2: 23. avatar_1261_1447135785.jpg - [No Subject] Attachment #877 · Similarity: 24.1% · Score: 76.2 🖼️ Man holding a syringe with his finger. No topic found 🖼️ 24. avatar_2483_1479729013.jpg - [No Subject] Attachment #2191 · Similarity: 23.1% · Score: 74.6 🖼️ A cartoon of a girl in a nurse outfit holding a syringe. No topic found

Claude

Claude: [24/09/2026--16:52:54:998 AEST]

I fixed this in MediaSearch.py v1.4.0, so avatars no longer show up in results.

Why they appeared: SMF keeps uploaded avatars in the same attachments table as post attachments. The difference is that an avatar is tied to a member (id_member > 0) and not to a post (id_msg = 0). ISEmedia indexes everything in that table, so avatars got in. That's also why both of yours showed "No topic found" and "[No Subject]".

What changed:

Lookup: the database query now also reads the attachment's a.id_member and a.id_msg.
New is_avatar() check: a result counts as an avatar if it belongs to a member and isn't attached to any post. As a backup, anything whose filename starts with SMF's avatar_ prefix also counts.
Where it filters: avatars are removed right after the database lookup, before keyword filtering and sorting, so they're gone under every sort mode. The excluded attachment numbers are logged with ise_trace.

It compiles. I ran the check on its own against your two examples: #877 is caught by both the member and filename tests, and #2191 would be caught by its filename. Normal post attachments pass through. I haven't run it against your live database.

What this doesn't cover: avatars are still being indexed and captioned. That's wasted processing, and they still take up room in the stored data. The fuller fix is to skip them when indexing, using the same test in MediaIndexer.py. If you upload that file, I'll make the change there.

Files are in:

MediaSearch.py

Andrew

and 17. avatar_1266_1459491033.jpg - [No Subject] Attachment #1649 · Similarity: 27.4% · Score: 81.0 🖼️ A heart shaped rock and a syringe on a table. No topic found

Claude

Claude: [24/09/2026--16:53:33:858 AEST]

v1.4.0 already filters out #1649, so no code change is needed. I tested it: the filename check catches it on its own, and so does the member check. It's the same kind of avatar as the other two, so I'd guess it's from the same search you ran before installing the fix.

If it still shows up after you've installed v1.4.0, look at the version stamp under the query on the results page:

It says v1.3.0 or earlier: the old file is still being run, perhaps because the bridge points at a different copy.
It says v1.4.0: the filter isn't working. Send me the MediaSearch.py: avatars excluded line from the trace log and I'll track it down.

Andrew

search was:: syringe

Claude

Claude: [24/09/2026--16:54:04:361 AEST]

That explains why these three surfaced. All their captions contain "syringe", so they pass the keyword filter and rank like any other match. A search for "syringe" is also a good test of the fix. When you rerun it on v1.4.0, #877, #1649 and #2191 should all be gone. You can confirm with the avatars excluded line in the trace log, which should list those attachment numbers.

Did the run that showed #1649 already have v1.4.0 installed? The version stamp under the query on the results page will tell you. If it did, the filter isn't working, and that trace line is what I need to find out why.

Andrew

lovely !

Claude

Claude: [24/09/2026--16:56:01:219 AEST]

Great, glad it's working. Avatars no longer show up in results and Open displays the file instead of downloading it. Whenever you want to stop avatars being indexed in the first place, send MediaIndexer.py and I'll make the change there.

Andrew

No results found for: AI ? is that possible -- they are all mp4 and maybe the indexere hasn;t for to the mostr recent insta*.mp4 ?

Claude

Claude: [24/09/2026--16:59:33:563 AEST]

Yes, "no results" for AI is possible, and there are two likely reasons:

The indexer hasn't reached them. Search only considers files already in embeddings_store.json. New insta*.mp4 files that haven't been processed yet aren't candidates at all.
They're indexed but blocked by the keyword filter. Since v1.2.0, a search term has to appear as a whole word in the caption, filename or post subject. The auto-captions describe what's in the frame (for example "a woman standing in a room") and almost never say "AI". A filename like insta_12345.mp4 doesn't help either. So unless the post subject has "AI" as its own word, the video gets filtered out, however well CLIP matches it. "AI-generated" counts; "AIart" doesn't. This is the trade-off I flagged when the filter was added.

I wrote a read-only script, media_diag.py, that tells you which one it is. Put it next to MediaSearch.py and run:

python3 media_diag.py AI 'insta%.mp4' 20

For your 20 newest insta*.mp4 files, it shows whether each one has been indexed, how many frames have captions, a sample caption, and whether a search for "AI" would get through the filter (and on which field). It uses MediaSearch's own settings and checks, so it gives the same answer a real search would. It compiles; I haven't run it against your live data.

If they turn out to be indexed but filtered, there are a few ways to fix it:

Use "AI" in the post subjects.
Let strong CLIP matches through for short queries.
Have the indexer add the post subject or tags to the caption text.

Which way makes sense depends on how those videos get posted.

Files are in:

media_diag.py

Andrew

Live  root@forum:15:00:54:/var/www/html/The_ISE_Project# py media_diag.py
[!] Import Error: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/local/lib/python3.6/site-packages/torch/_C.cpython-36m-x86_64-linux-gnu.so)
 Live  root@forum:15:01:01:/var/www/html/The_ISE_Project#

Claude

Claude: [24/09/2026--17:01:21:252 AEST]

That was my mistake. media_diag.py imported MediaSearch.py, which loads torch as soon as it's imported. On Live, py is the system Python 3.6, and torch won't run there because the system glibc is too old (it needs 2.14). The diagnostic doesn't need CLIP at all.

v1.1.0 no longer imports MediaSearch.py. It carries its own copy of the few pieces it uses: the store file paths, the database settings from Settings.php, the word-matching rule and the avatar check. It only needs mysql.connector, which already loads fine under Python 3.6 because MediaSearch imports it before torch. I tested it against mock data laid out like your Live directories. It correctly flagged a not-yet-indexed video, a match on the post subject, and a match on the caption.

Run it the same way:

py media_diag.py

The defaults are the query AI, the filename pattern insta%.mp4 and the 20 newest files.

The copied rules are from MediaSearch v1.4.0. If the word-matching or avatar rules change in MediaSearch.py, they'll need changing in the diagnostic too.

Files are in:

media_diag.py

Andrew

Live  root@forum:15:01:01:/var/www/html/The_ISE_Project# py media_diag.py
MediaSearch v1.4.0 (copied rules) | query='AI' words=['ai'] | pattern='insta%.mp4': 33 in DB, showing newest 20
stores: media=2723 embedding keys=1844 attach ids embedded=1820 captions=1835

#6876 instagram-1790219080408.mp4 [Aldous Huxley on the Educational Value of Mescaline]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6875 instagram-1790217713252.mp4 [Don't get sucked in by your model's sycophancy]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6874 instagram-1790215518003.mp4 [The &quot;Endocannabinoid System&quot; Discovered By Wondering Why/How People Got Stoned !]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6802 instagram-1790126186223.mp4 [WHY DO WE GET &quot;HIGH&quot; ?]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6801 instagram-1790060183084.mp4 [What are the dangerous fakes &quot;PMA&quot; and &quot;PMMA&quot; ?]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6800 instagram-1790059803572.mp4 [What are the dangerous fakes &quot;PMA&quot; and &quot;PMMA&quot; ?]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6799 instagram-1790054048189.mp4 [WHAT IS METHAMPHETAMINE ? MORE BULLSHIT / DRUG PROPAGANDA FOR YOU !]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6798 instagram-1790053567352.mp4 [WHAT IS METHAMPHETAMINE ? MORE BULLSHIT / DRUG PROPAGANDA FOR YOU !]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6797 instagram-1790046346016.mp4 [WHAT IS P2P METH ? A BIT OF BULLSHIT FOR YOU !]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6796 instagram-1790045019523.mp4 [&quot;DOCTOR, WHAT DO THINK THE MOST DESTRUCTIVE DRUG IS ?&quot;]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6793 instagram-1789979452045.mp4 [&quot;DOCTOR, WHAT DO THINK THE MOST ADDICTIVE DRUG IS ?&quot;]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6772 instagram-1789884050471.mp4 [Google's Quantum Chip]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6735 instagram-1789818146671.mp4 [FINALLY WE LEARN THE TRUTH ABOUT ADDICTION AND RECOVERY]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6711 instagram-1789693165419.mp4 [Fabulous and Simple Clips on AI &quot;stuff&quot;]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': PASS via subject (irrelevant -- not a search candidate until indexed)

#6703 instagram-1789586347884.mp4 [Machine Learning From Scratch]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6702 instagram-1789585609506.mp4 [How Chips Are Made]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6701 instagram-1789583691103.mp4 [Machine Learning From Scratch]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6700 instagram-1789583174138.mp4 [Machine Learning From Scratch]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': no literal word match (irrelevant -- not a search candidate until indexed)

#6018 instagram-1788740839969.mp4 [The Vital Collection of Videos on Instagram On Salient AI Topics Volume 2]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': PASS via subject (irrelevant -- not a search candidate until indexed)

#6017 instagram-1788740230651.mp4 [The Vital Collection of Videos on Instagram On Salient AI Topics Volume 1]
   IN media_store ONLY: 0 frame(s), 0 captioned
   gate for 'AI': PASS via subject (irrelevant -- not a search candidate until indexed)

Summary: 0/20 indexed, 0 would be returned for 'AI'
 Live  root@forum:15:02:19:/var/www/html/The_ISE_Project#

Claude

Claude: [24/09/2026--17:02:50:895 AEST]

You guessed right, and the problem is bigger than the newest files. None of the 20 has been indexed. MediaIndexer has listed them in media_store, but MediaProcessor never produced embeddings or captions for them, so search can't see them at all.

The totals suggest videos in general aren't being processed:

Backlog: media_store has 2723 entries, but only 1820 attachments have embeddings, so about 900 are still waiting.
Videos specifically: each embedded video frame gets its own key (id_t<seconds>). There are 1844 keys for 1820 attachments, so only about 24 extra keys exist in total. That's only a handful of videos' worth of frames.

Two possible causes:

Backlog: MediaProcessor works through images first, or hasn't run since these were posted.
Failure: frame extraction is failing for these mp4s, for example because of a codec problem. Instagram exports are often HEVC or AV1. Each one gets skipped quietly and retried or dropped on every run.

This command shows which it is by breaking down the unprocessed entries by file type:

py -c "import json;m=json.load(open('../ISE_Data/media_store.json'));e={k.split('_t')[0] for k in json.load(open('../ISE_Data/embeddings_store.json'))};from collections import Counter;print('pending',Counter(v.get('extension','?') for k,v in m.items() if k not in e));print('done',Counter(m[k].get('extension','?') for k in e if k in m))"

If "done" has almost no .mp4 and "pending" has lots, it's the video path. The next step is MediaProcessor's log or output from running it on one attachment, such as #6876.
If "pending" is mixed across file types, it's just backlog, and running MediaProcessor will clear it.

Once they're indexed, only #6711, #6017 and #6018 would come up for "AI", because their subjects contain the word. The "Machine Learning" and "Chips" ones won't, since that search only matches the literal word.

I also spotted a separate bug. SMF stores subjects already HTML-encoded ("), and MediaSearch encodes them a second time. The results page will show a literal " in titles like the PMA and PMMA one. It's a small fix, and I can include it in the next MediaSearch update.

Andrew

fix it now

Claude

Claude: [24/09/2026--17:03:20:493 AEST]

I fixed it in MediaSearch.py v1.4.1. Titles like the PMA/PMMA one will now show real quote marks instead of a literal ".

What changed: a new _smf_unescape() decodes the subject, filename and board name once, right after they're read from the database. The results page then encodes them once when it builds the page, so nothing is encoded twice.

Side benefit: the keyword filter no longer sees a stray "quot" in those subjects. Before, a search for "quot" would have matched them.

I tested the PMA subject: it comes out as plain "PMA" and is encoded exactly once on the page. Empty values pass through unchanged. It compiles; I haven't run it on Live.

Files are in:

MediaSearch.py

Andrew

Build a MD and PDF transcript instructions: Transcribe the entire conversation, DO NOT EMBELLISH OR SUMMARISE UNLESS YOU FIRST HAVE IT VERBATIM, Expand all Query and Reply windows if applicable, Preserve formating, Embed any graphics, Do not lose a single word, Suggest a few names derived from a terse summary and pick the best but be as fast as possible.