Supersedes ISE_v10_Checkpoint_merged_4.md (_4 is not overwritten). Everything in _4 still stands unless stated here. Section 1 and 6 repeat what is still in force from _4, so this file can be read on its own.
xmllint --noout on XML, list unrequested changes BEFORE making them, release card table._2, _3).Claude: [Month DD, YYYY, HH:MM:SS AM/PM], time from an executed command each reply. Do not mimic his tone. Fewer questions, no jargon coined by Claude, no bloat. Simple things simple.bash, never sh (CentOS 6 sh = POSIX-mode bash; done < <(...) fails).[lvi] v1.0.1 confirmed working on an SMF 2.0.x test system. Not proven: playing an attachment from a restricted board (VLC sends no forum cookie). His v2.0 housekeeping, SMF 2.1.x support and the other parked items in _4 section 3 are still parked.
| File | Version | md5 |
|---|---|---|
MediaProcessor_v9.29.zip |
9.29 (superseded by 9.30, which includes it) | cff5607ae01d77efb9881153ae9248a6 |
MediaProcessor_v9.30.zip |
9.30 | fb09930e5440a313f823104925896977 |
cleanISE_Clone_v1.0.1.zip |
1.0.1 (superseded) | 2d2e71a046558b8259a581198a752dd9 |
cleanISE_Clone_v1.0.2.zip |
1.0.2 (superseded) | d6ff3b28edbd7f257785b86b4973863c |
cleanISE_Clone_v1.0.3.zip |
1.0.3 (superseded) | dba67dd7e15ec8a77cf01b00bc10e012 |
cleanISE_Clone_v1.0.4.zip |
1.0.4 (current) | ac2e85e76fa7b7ba152a1c2a8e5ab85d |
cleanISE_Clone_manual_2.md |
manual for 1.0.4 (his original left untouched) | -- |
Deploy status: not stated. He said v1.0.4's lock handling was "better"; nothing confirms he installed any of them on Clone.
MediaIndexer has catalogued audio (mp3 wav m4a flac ogg aac wma) since v9.9.4/v9.9.7, for AudioProcessor and ISEmedia. MediaProcessor._in_scope() accepted everything when --ft was all, so each audio file counted as a "new" item (54 and 58 in his 2-instance run), went through the whole per-item path, was never embedded, and came back every run. Now _in_scope() accepts only IMAGE_EXTS | VIDEO_EXTS. Side effects left as they are: --attachid on an audio id processes nothing; the checkpoint's total_items / remaining still count every item in media_store.json, audio included.processor_checkpoint.json -> processor_checkpoint.ckpt; processor_checkpoint_i<k>of<N>.json -> processor_checkpoint_i<k>of<N>.ckpt. JSON inside, name only. Four places in the file changed (two code lines, the --log help text, the Features line).Can't retry caption for 110_t15 / 110_t22 lines come from video frames missing from frame_cache; they print every run until those frames are regenerated.Runs as bash cleanISE_Clone [--apply]; dry run without --apply. Only files directly in /var/www/html/ISE_Data. Five rules:
*.tmp older than 60 min -> delete.*.log over 2048 KB -> copy to .log.1 (.1->.2), empty in place.processor_checkpoint.json and processor_checkpoint_i*of*.json (old names) of any age. Always: delete processor_checkpoint_i*of*.ckpt older than 1440 min. processor_checkpoint.ckpt is never touched.audio_store.json.pre* older than 7 days -> delete (covers audio_store.json.pre2419, Sep 28, 4.5 MB; the live store was rewritten Oct 12, so it was stale; AudioProcessor not read, nothing in MediaIndexer/MediaProcessor refers to it).audio_store.json.lock ("pid N, started ...") older than 60 min AND /proc/N gone -> delete. Pid alive, no pid in the file, or too fresh: left alone./var/www/html/The_ISE_Project, or PROJ/ISE_Data in its command line, or a file open in ISE_Data; or a held lock (.mediaprocessor.lock, .mediawatchdog.lock, audio_store.json.lock, MediaIndexer.lock in the temp dir); or flock missing. If found: the old-named .json checkpoints are left, the other rules still run, exit code 3. Exit 3 in cron just means a job overlapped.--all-checkpoints (v1.0.1) is still accepted and does nothing (behaviour is now the default)..mediaprocessor.lock and .mediawatchdog.lock are deliberately never removed (the lock belongs to the open file; deleting a held one lets two programs each "hold" it).*_store.json, word_index*.json, audio_checkpoint.json, caption_failures.json, chain_snapshot.json, orphan_state.json, pdf_duplicates.json, pdf_dupe_deleted.json.23 * * * * /bin/cleanISE_Clone --apply >> /var/log/cleanISE_Clone.log 2>&1.grep -rIl processor_checkpoint /var/www/html /bin 2>/dev/null. Only MediaProcessor.py and cleanISE_Clone were seen to use it.pre2419 were still there. The script's output was never pasted, so the cause is unknown (most likely no --apply, or something running). v1.0.4 with --apply will say.AudioProcessor (whether audio_checkpoint.json is only a log; how it uses audio_store.json.lock), pdf_duplicates.json / pdf_dupe_deleted.json (Sep 8), and any master manual index listing cleanISE_Clone's version.total_items / remaining count only encodable items (image and video), so audio no longer inflates "remaining".Checkpoint_ProcessingNewMedia_20261012_1121.md into a merged checkpoint.tokenizers; pip install --no-deps transformers==4.31.0; transformers 4.31.0 stays; only tokenizers and the fake safetensors dist-info get removed.MediaIndexer.py v9.9.9 (2871 items, one added: 7479 feet_under_the_table_HD.mp4); MediaProcessor.py --instances 2 --progress --log ran at v9.27 on Clone at the time; MediaIndexer.py takes no --instances/--progress/--log.