SMF Batch — checkpoint, 2026-10-23 (16:19 Sydney)

For pasting/uploading into a fresh chat. Facts only; things I did not verify are marked UNVERIFIED.

1. Problem & Resolution

Goal (Andrew). "SMF Batch": the batch-run counterpart to his SMF mods — same structure, Package Manager managed, but kept separate from the interactive mods because of long execution times. First target: the whole attachment corpus, attachments and attachments2 treated as ONE corpus (never dissociated except when reporting). Start with a report: every attachment in chronological, date-stamped order with every available metric, plus the post(s) hanging off each. Display through listatt. PHP or Python per job, whichever suits.

Decisions made

• Chronology comes from the post (messages.poster_time), because SMF 2.0 attachments have no date of their own. File mtime on disk is recorded as a second date.
• Long work runs from the command line, never in a web request. The web side only DISPLAYS stored results and starts nothing (I objected to a BBCode tag triggering a job; Andrew accepted by proceeding).
• Language: PHP for the job (CentOS 6 / PHP 5.3-compatible syntax, no short arrays or ??), matching SMF.
• The viewer ("attractive but useless interface") was merged into listatt as Sources/listatt_report.php, linked from the INFO panel (admin-only "Attachment counts" section, link "Full attachment corpus report").
• The worker's home was moved twice at Andrew's direction: Sources/SMFBatch/ → Collabware/ → Sources/CollabCore/attach_report.php (beside AttachMeta.php, which is NOT touched). Current home: CollabCore.

What was built (current)

• attach_report.php v1.0.0 (worker, CLI only). Run from the forum root: nice -n 19 ionice -c3 php Sources/CollabCore/attach_report.php --pause=200. Options: --settings=PATH --resume --nohash --batch=N --pause=MS --status --stop --help. Prints its version once at start. Lock file + stop-flag file beside the script. Ordered stop only (--stop), never kill.
• Writes only its own tables: {db_prefix}batch_runs, batch_attach_report (one row per attachment per run: all attachments columns, post/topic/board, ordinal within post, downloaders from log_downloads if present, disk path/size/mtime/ctime/perms/SHA-256/mime, dup_count, flags), batch_attach_orphans (files on disk with no DB row: NO_DB_ROW, NAME_MISMATCH, FOREIGN, SUBDIRECTORY). Tables are created by the job on first run and left in place on uninstall.
• Flags: MISSING SIZE FOLDER DUP NO_POST DEAD_POST THUMB TYPEn UNAPPROVED EMPTY UNREADABLE.
• listatt_report.php v1.0.2 (viewer): admin-only (allowedTo('admin_forum'), same SSI pattern as listatt_posts.php / listatt_dupes.php); sortable, paged, searchable table; flag filter; orphan-files tab; click-to-expand row with every field and the posts sharing identical content; View and Edit links; ?json=1 JSON (run page per sort dir q flag orphans dups=<sha256>); Sydney-time display; visible version banner.
• Manual: SMFBatch_manual.md v1.0.3 (ships beside the worker).

Latest deliverables (use these)

• listatt_bbcode_v6.24.zip — ListAttBBC 6.24 + listatt_report.php 1.0.2; no worker.
• attach_report_v1.0.1.zip — installs worker + manual into $sourcedir/CollabCore/.

Lineage (superseded; for reference) SMFBatch_v1.0.0.zip (worker + old viewer in Sources/SMFBatch/) → listatt_bbcode_v6.22.zip (viewer merged, worker in Sources/SMFBatch/) → listatt_bbcode_v6.23.zip + attach_report_v1.0.0.zip (worker to Collabware) → 6.24 + attach_report_v1.0.1.zip (worker to CollabCore).

Status on Andrew's systems

• He said he installed SMFBatch_v1.0.0 and listatt 6.22. Whether 6.23/6.24 or the add-on are installed: UNKNOWN.
• He decided to run the job on Live (Dev is out of action with the transfer slowdown). No console summary has been received yet. If a run was started from the old Sources/SMFBatch/ location, it uses a different lock file from the CollabCore copy; do not run both at once.
• Leftovers on his system from the first package: Sources/SMFBatch/attach_report.php and attach_report_view.php (old viewer, unlinked, admin-only).

2. Risk Analysis

Tested. On a throwaway MariaDB 10.11 with 15 fake attachments across two directories, run under PHP 8.3: chronology, all flags, orphans, duplicates, unicode names, ordered stop, lock, resume (including mid-run), JSON endpoints, SQL-injection attempts on sort/flag (rejected), guest refusal, no-report case, and a headless-browser render of the viewer with no script errors.

NOT tested / UNVERIFIED

• Never run inside his real SMF, on his Live data, or on a real PHP 5.3 (syntax avoided 5.4+ features by inspection only).
• The viewer was tested against a stub of SSI.php, not his fork's real one. db_error_skip handling in his fork is assumed.
• Assumed the standard SMF 2.0 schema (no schema dump supplied): attachments columns, id_folder, attachmentUploadDir stored as a serialized array or plain string, {db_prefix} possibly including a database name, log_downloads having id_attach and id_member.
• First run on Live also reads every file once for SHA-256 (disk-heavy); --nohash skips it and loses duplicate detection.
• On --resume the attachment list is rebuilt, so items can shift if attachments were added in between.
• I claimed that uninstalling or reinstalling Collaborative removes the whole Collabware/ directory. That was inferred from reverse="true" in his package-info.xml and is UNVERIFIED; the real behaviour may delete only the shipped files. It applies equally to CollabCore.
• Stale base: the Collaborative zip he sent is v6.0 (dated 2026-10-09); his notes say RedBox was folded in at v6.1 on 2026-10-17. Do NOT rebuild Collaborative from the v6.0 zip.
• Two zips in /home/claude (SMFBatch_v1.0.1.zip, SMFBatch_v1.1.0.zip) were NOT made by me in this conversation; the contents differ from my worker (different sizes). Treat them as unknown; do not distribute.

3. Code Change

Files, versions and where they live:

File	Version	Location after install
attach_report.php	1.0.0	Sources/CollabCore/ (package attach_report 1.0.1)
SMFBatch_manual.md	1.0.3	Sources/CollabCore/
listatt_report.php	1.0.2	Sources/ (listatt 6.24)
ListAttBBC.php	6.24	Sources/ — the only existing file touched: doc-block entry, VERSION, one added INFO-panel link line
package-info.xml (listatt)	6.24	2.0 and 2.1 install/uninstall blocks both updated; listatt_posts.php, listatt_dupes.php, view_attach*.php untouched

Build notes: zips built with Python zipfile (no streaming data descriptors, verified); xmllint --noout passed on both package-info.xml. Andrew's legacy zipper was NOT used (not available to me); he must rezip if he wants it exact.

4. Open items / next steps

1. Andrew runs the worker on Live and sends the console summary; compare it with what the report page shows. Check the printed directory listing against his real layout first.
2. Read the flags (missing, wrong folder, size mismatch, duplicates, no post) — they decide the next jobs.
3. Wrap the existing tools as batch jobs one at a time, same stop/resume/lock behaviour: PDF indexer, PDF duplicate spotter, media indexer, bulk [listatt] inserter.
4. Optional: fold the worker into the next Collaborative release, but only against his CURRENT Collaborative zip (not v6.0). He said where it lives is "neither here nor there", so only do this if he asks.
5. Not built: any BBCode tag for the report (the INFO link covers it); a start button on the report page (long runs stay on the command line).
6. The worker re-implements physical-path resolution that AttachMeta.php already has; possible later reuse, but he said leave AttachMeta.php alone.

5. Standing rules observed this session (from Andrew)

Reply header with real Sydney time (casual mode); no sycophancy; statements over questions; critique BEFORE building; list any change he did not ask for BEFORE making it (I failed this twice and listed afterwards — do it first); no manual edits to SMF files (Package Manager only); never rename his files; zip with dotted never-reused version; @version/@date bumped and manual history entry on every change; version shown when a program runs; xmllint on XML; release card at the end of each delivery including "legacy zipper used"; ordered shutdowns only, never kill; expand acronyms; don't assert unverified facts as fact.