Classification_Progress_Report.py — Reference

• Version: 2.0
• Date: 2026-09-24
• Part of: The ISE Project → ISEmedia
• Authors: Andrew.human & Claude.ai

Purpose

A read-only progress report for the ISEmedia caption pass (SmolVLM through llama-mtmd-cli). It compares the caption store against the embedding store and shows whether a caption process is running right now.

Usage

python3 Classification_Progress_Report.py                    # uses /var/www/html/ISE_Data
python3 Classification_Progress_Report.py --data-dir /path   # different data location
python3 Classification_Progress_Report.py --missing-out /tmp/x.txt

It works from any directory because all paths are absolute.

Inputs

File	Location (default)
embeddings_store.json	/var/www/html/ISE_Data/
captions_store.json	/var/www/html/ISE_Data/

Both files can be a dict keyed by attachment, or a list of records that have attachment_id, id or file.

Output

• Version banner.
• Processed X of Y: the number of keys found in both stores. The "on-disk snapshot" label is there because the driver saves periodically, so the real figure can be ahead of this one.
• missing captions: items that have an embedding but no caption. The full list goes to --missing-out.
• captions w/o embedding: orphans. Anything other than 0 means the two stores disagree about keys.
• The last-write time and age of each store.
• Each live llama-mtmd-cli process: its PID, parent PID (the driver), elapsed time, CPU time and the image being captioned.

Reading it

Observation	Meaning
Store age is large, a process is running and the image changes between runs	Working normally; the driver just hasn't saved yet
The same image and PID for many minutes	That image has hung the caption call
No process running and missing > 0	The run ended or died with items left over; check the missing list
Orphans > 0	The two stores use different keys

Side effects

Writes only the missing-keys file. Never modifies the stores, and never signals or kills any process.

History

• 1.0: a three-line count of len(captions) over len(embeddings) using relative paths.
• 2.0: absolute paths, comparison by key, store ages, live process status and a version banner.