Purpose: bring Andrew's SMF Markdown BBCode mod (MDParser.php/MDBBC.php) up to a level of GFM spec conformance good enough to satisfy SMF's lead dev, who recommended testing against the full GFM spec (https://github.github.com/gfm/) after reviewing the mod for possible official SMF inclusion.
| File | Status | Use it? |
|---|---|---|
MDParser.php (this session's patched version, attached) |
Patched: emphasis rewrite, loose/tight lists, indented-code blank-line merging, link destination fixes, code span fix, bare-URL/www/email autolinking | Yes — this is the current working version, supersedes the originally-uploaded copy |
MDBBC.php (originally uploaded) |
Untouched this session — no changes were needed | Yes, unchanged |
MD_SYNTAX_MANUAL.md (attached) |
Full table of every currently-supported syntax element + raw/rendered examples, generated from the actual current parser | Yes — hand this to anyone (including SM) who wants to see current scope |
extract_spec.py (attached) |
One-time script: pulls cmark-gfm's test/spec.txt and converts it to spec.json (672 examples, tagged by section/extension) |
Only needed if spec.json (attached separately) is ever lost — otherwise skip straight to run_tests.php |
spec.json (attached) |
Already-extracted output of extract_spec.py — the 672-example test suite itself |
Yes — drop it next to MDParser.php and run_tests.php, no re-fetch needed |
run_tests.php (attached) |
Runs MDParser.php against every example in spec.json, writes results.json |
Re-run after every parser change |
analyze.py (attached) |
Reads results.json, applies a "cosmetic-diff-tolerant" normalizer (strips our own CSS classes, rel= attribute, self-closing-tag style, tag-per-line formatting) to separate real bugs from harmless serialization differences; writes real_failures.json and prints per-section pass rates |
Re-run after every run_tests.php run |
test/spec.txt in github/cmark-gfm on GitHub (raw: raw.githubusercontent.com/github/cmark-gfm/master/test/spec.txt), in a literate format (fenced blocks marked `...`` example [extension-name] , input, a lone . line, expected HTML, closing fence). extract_spec.py parses this correctly — 672 examples extracted, matching the known canonical GFM spec size.apt-get install -y php-cli works (no sudo needed, container runs as root) and pulls PHP 8.3.md-code-block/md-table, the rel="noopener noreferrer nofollow" security hardening on links, HTML5-style non-self-closing tags) cosmetically differ from the reference renderer's output without being bugs. Always read the normalized pass rate from analyze.py, and treat real_failures.json (not raw results.json) as the list of genuine bugs.commonmark reference package: one spec example (__foo, __bar__, baz__) has an expected output in cmark-gfm's own spec.txt that disagrees with what the reference implementation's actual delimiter-stack algorithm produces (nested vs. flat <strong>). Traced this to the algorithm level, not a wasted rabbit hole — treat as a known, accepted gray area, do not re-derive.MD_SYNTAX_MANUAL.md above; worth asking rather than assuming they're the same artifact.[nobbc] (kept enabled/exempt as the literal-text escape hatch) and [md] itself. This is stated in MD_SYNTAX_MANUAL.md's header warning and in MDBBC.php's top doc-comment — check both stay in sync if the policy ever changes.<scheme://...>) only recognizes http/https/ftp/ftps/mailto. CommonMark's actual rule allows any scheme matching [a-zA-Z][a-zA-Z0-9+.-]{1,31}:. Decision needed: is broadening this worth it for a forum (real schemes people might paste: irc://, magnet:, steam://), or is the current allowlist a deliberate security stance worth keeping as-is?(foo(bar)) inside a bare URL) or do full percent-encoding of unsafe characters — only literal-space encoding is implemented. Decision needed: how much of this edge case is worth chasing given forum posts rarely paste raw unencoded URLs with parens.>) is now fixed for the common case (paragraph continuation only, not headings/lists/fences/thematic breaks/indented code, and not after a blank line). Remaining edge case: laziness tracking inside an already-open non-paragraph block within a quote (e.g. an unclosed fence) isn't precise - low priority.-/*/+) or ordered delimiter style (. vs )) now correctly starts a new list; an ordered list can only interrupt a paragraph if it starts at 1.%XX sequences) and correctly balance parentheses via a recursive regex ([link](foo(and(bar))) works). Caution for future edits: adding a named recursive group inside an existing capturing group shifts the numeric index of every group after it — this caused a real regression this session (title text silently became a duplicate of the URL) until the shifted indices were corrected. Double-check group numbering with a quick preg_match test whenever the link/image regex structure changes again.%5C, HTML entities like ä → %C3%A4, non-ASCII chars, quotes in a titleless destination) — only literal-space encoding is implemented. This is the largest remaining Links gap (roughly a dozen of the remaining ~53 failures).[link](<foo\nbar>), which per spec should NOT be a valid link) aren't rejected, because paragraph-line-joining happens at the block level before this inline check ever sees the newline — a real but low-priority structural limitation.MD_SYNTAX_MANUAL.md as this session's independent version.Run analyze.py's per-section breakdown on the current real_failures.json. Links is now largely down to reference-style-link failures (out of scope) plus a handful of exotic raw-HTML-mixed-with-brackets edge cases — List items (32/48) and Lists (13/26) are the next-best-value targets. Starting a fresh session should: install php-cli (apt-get install -y php-cli, no sudo needed), drop all six attached files in one directory, then run php run_tests.php && python3 analyze.py, and pick up from there.