ISEmedia — Mini Checkpoint (2026-08-23)

Decisions made

• New source: ISEmedia (not ISEai/ISEimg/ISEmp4) — one unified source for images + video, matching ISE/ISEpdf/ISEmisc's naming-by-content-type pattern.
• Full attachment history in scope, including a large Instagram-sourced video collection Andrew has begun uploading.
• Two-model approach: CLIP (open_clip, ViT-B-32) for image/keyframe embeddings; Whisper (base model) for optional video transcription.
• Video handling: ffmpeg keyframe extraction (fixed interval, v0.1) → each keyframe embedded like an image; audio → Whisper transcript, indexed separately.
• Two indexing cadences, one script (MediaIndexer.py):
  • Daily cron: CLIP embedding pass (fast, cheap).
  • Weekly cron: MediaIndexer.py --transcribe (Whisper pass, slow/heavy).
• Both passes use per-item checkpointing + single-instance lock, same pattern as PDFIndexer.py's existing OOM-safety design — deliberately NOT disabling the OOM killer on Live (would risk killing MySQL/Apache instead).
• Feeds into ISEmega eventually as a fourth interlaced source; needs a new RankingEngine.py rule type since cosine-similarity scores don't fit the existing occurrence/recency/proximity rule shape.

Environment facts confirmed this session

• Live box: CentOS 6.10, 64-bit, Python 3.6.3, glibc 2.12 (old — below the manylinux2014 ceiling most current ML wheels target).
• pip3 install torch==1.4.0 --no-deps succeeded on Live (not Dev — confirm this was intentional, not a wrong-terminal situation).
• NOT yet confirmed: whether import torch actually runs clean (install succeeding doesn't guarantee the compiled .so files run against glibc 2.12).
• NOT yet tested: open_clip_torch, openai-whisper install/import on Live.
• ffmpeg availability on Live not yet confirmed.

Built this session

• MediaIndexer.py v0.1 — syntax-checked, NOT deploy-tested. Two known stubs that must be wired before real use:
  1. resolve_attachment_dir() — currently raises NotImplementedError; needs the real _parse_php_dir_array()-equivalent from PDFsearch.py/MiscIndexer.py.
  2. attachment_id derivation from filename — assumption flagged inline, needs checking against the real attachments dir/DB layout.

Next steps when resumed

1. Confirm import torch succeeds cleanly on Live.
2. Test open_clip/whisper install + import on Live; find a compatible version pairing with torch 1.4.0 if newer releases don't work.
3. Confirm ffmpeg is available.
4. Wire resolve_attachment_dir() and attachment_id derivation to real shared code.
5. Run MediaIndexer.py against a small real batch (not full backlog) to validate memory behavior on Live's 2GB RAM before the full Instagram backlog run.
6. Design MediaSearch.py (query embedding + cosine similarity ranking) and the ISEmega RankingEngine.py rule for similarity scores.