MediaTranscriber.py + chktranscribe -- manual entry, x into ISE_Python_Scripts_Manual (next Master version): add the history lines, add the MediaTranscriber and chktranscribe sections, and update the quoted MediaSearch version (v1.10.4) to v1.11.0. (Second pass, 2026-10-29: MediaTranscriber and chktranscribe are now v1.1.0 -- put the v1.1.0 lines first.)

Version history lines

ISE MediaTranscriber.py v1.1.0 (2026-10-29) -- Music / no-speech videos can be flagged so no run ever processes them: --skip ID [ID ...] [--reason "text"] records a finished empty entry with skipped: true and the reason; --unskip ID [ID ...] deletes it so the next run takes the video again. Both refuse while a run is active (exit 3) and --skip will not overwrite a done transcript without --force. New -n cap on every llama-mtmd-cli call (--max-tokens, default 1024), and a repetition guard that drops a piece whose text is one phrase repeated (compresses to under 12% of its size, 200+ characters), keeping a dropped_pieces count. The guard threshold is untested against the real model.

ISE chktranscribe v1.1.0 (2026-10-29) -- Videos flagged skip are counted and listed on their own, with the reason, and are no longer counted under "no speech".

ISE MediaTranscriber.py v1.0.0 (2026-10-29) -- New job. Transcribes the speech of every .mp4 in media_store.json into ISE_Data/transcripts_store.json with the same engine as the ts script (ffmpeg to 16 kHz mono wav, then llama-mtmd-cli with Qwen3-ASR-0.6B Q8_0). Runs on Live, any time, one run at a time. Restartable, with a controlled stop (--stop).

ISE chktranscribe v1.0.0 (2026-10-29) -- New read-only monitor for the job: running / idle / died, what it is on now, progress, speed and ETA, failures, cross-checks. Changes nothing, takes no lock.

ISE MediaSearch.py v1.11.0 (2026-10-29) -- Video speech is searchable. A new optional store, transcripts_store.json, gives an attachment a transcript, matched like a caption (new field "transcript", 30 points per distinct term; a name hit still outranks it). The evidence line and the result card/CLI show a short snippet around the first hit. A transcript is only attached to an attachment already in the catalogue, so one for a video that has left media_store.json (for example its post is now marked #do-not-index) never surfaces. A missing store is fine; an unreadable one is reported like the other optional stores. Nothing else changed.

Usage

python3.8 MediaTranscriber.py                  all pending videos
python3.8 MediaTranscriber.py --dry-run        what it WOULD do; changes nothing
python3.8 MediaTranscriber.py --chunk-size 20  at most 20 videos, then stop
python3.8 MediaTranscriber.py --attachid 4683  one video (add --force to redo it)
python3.8 MediaTranscriber.py --stop           controlled stop of the running job
python3.8 MediaTranscriber.py --skip 2321 2647 --reason "music, no speech"   flag videos; needs no run active
python3.8 MediaTranscriber.py --unskip 2321    clear the flag; the next run takes the video again
nohup python3.8 MediaTranscriber.py > /root/MediaTranscriber.out 2>&1 &

chktranscribe                                  one report
chktranscribe --watch 30                       repeat every 30 s (Ctrl-C leaves it; the job is not affected)

• Restart. The store is written after every video (temp file named with the PID, fsync, rename), and a run only takes videos with no finished entry. A restart carries on where the last run stopped, failed or was interrupted. Only the video in flight at that moment is redone, from its start.
• Entry status. done (text stored), no_audio (no sound track), empty (sound track, no speech found): all three are final and never retried. A video flagged with --skip is also stored as empty, with skipped: true and a reason; MediaSearch.py reads only done, so a flagged video is never searched. failed: recorded with attempts and error, retried on later runs up to 3 attempts, then left alone and listed at the end of every run until --attachid N --force. A file that cannot be located on disk is not recorded (retried every run) and is listed at the end of every run.
• Flagging. --skip and --unskip take the run lock, so they refuse (exit 3) while a job is running -- the job holds the store in memory and would overwrite the change. Use --stop, wait for the exit, then flag. --skip only accepts .mp4 ids that are in media_store.json. chktranscribe shows the flagged count and each reason.
• Controlled stop. --stop (from another session) asks the running job to finish the video it is on, save it, and exit 0 with result STOPPED. With no job running it says so and does nothing. Ctrl-C and SIGTERM stop at once: the video in flight is discarded (redone next run), everything saved is intact, exit 130. No kill -9 is ever needed.
• One run at a time. flock on ISE_Data/.mediatranscriber.lock, released by the OS if the process dies. A second start exits 3 and touches nothing.
• #do-not-index. A video in a marked post is never transcribed, and a transcript already stored for one is removed. The marked list comes from ise_noindex.py (must sit beside the script); if it cannot be read the run aborts (exit 1) with nothing touched. Needs pymysql, like MediaIndexer.py.
• Audio. Cut into 60 s pieces so a long video never overruns the model's context; the text of the pieces is joined. Each call is capped with -n (--max-tokens, default 1024). A piece whose text is one phrase repeated over and over (the model looping on music or noise) is treated as no speech and dropped with a [*] line; if every piece is dropped the video is recorded empty, and dropped_pieces is kept in its entry. Per-piece timeout --chunk-timeout (default 600 s); --threads (default 2) is llama's -t, as in ts.
• Archive. Each done transcript is also appended to /root/transcripts_all.txt (--archive PATH to change) under an === name (ID n) === line.
• Files in ISE_Data. transcripts_store.json (the store), .mediatranscriber.lock, .mediatranscriber.stop (exists only while a stop is pending), .mediatranscriber.status (which video is in flight; removed when the run ends; read by chktranscribe), transcribe_tmp/ (working audio, removed after each video and at the start of every run).
• Output. One [>] / [<] pair per video, a run summary, and one MEDIATRANSCRIBER_RUN time=... result=... done=... no_audio=... empty=... failed=... line per run for a log. Every failure prints a [!] line when it happens and again at the end of the run. Results: COMPLETE, STOPPED, INTERRUPTED, ABORTED (3 videos in a row could not even start a process, probably out of memory), SKIPPED (locked).
• Exit codes. 0 = finished or stopped cleanly, 1 = fatal (nothing was written, or aborted), 3 = another run active, 4 = lock file unusable, 130 = Ctrl-C / SIGTERM.
• Store ownership. Live writes transcripts_store.json. Like every .json at the ISE_Data root it must reach Clone and Dev by the mirror so the coherence check stays identical.
• Memory. No torch: the job is ffmpeg plus one small llama-mtmd-cli at a time. Do not run it at the same moment as a MediaProcessor run on Live.
• Python. Written to run under 3.6 as well, but Live's normal interpreter for ISE jobs is python3.8.
• chktranscribe. Finds ISE_Data from --data, else $ISE_DATA, else the constant DEFAULT_DATA at the top of the script (/var/www/html/ISE_Data -- change it there if Live's board root differs). Exit 0 = nothing needs attention, 1 = attention (failures, videos left alone, orphan entries, dead or stalled job, archive with fewer blocks than transcribed videos), 2 = stores unreadable. It looks the job up in /proc, never takes the lock.