dopetalk does not endorse any advertised product nor does it accept any liability for it's use or misuse


Our Discord Notification Server invitation link is https://discord.gg/jB2qmRrxyD
« previous next »
Pages: 1   Go Down

Author Topic: The Collabware Code Quality Enforcer  (Read 11 times)

Offline smfadmin (OP)

  • SMF (internal) Site
  • Administrator
  • Sr. Member
  • *****
  • Join Date: Dec 2014
  • Location: Management
  • Posts: 602
  • Reputation Power: 0
  • smfadmin has hidden their reputation power
  • Last Login:Today at 03:38:22 AM
  • Supplied Install Member
The Collabware Code Quality Enforcer
« on: Today at 02:28:11 AM »
📎 Click on me to view all attachments
ListAttBBC  |  v6.4.1  |  2026-08-25  |  Andrew.human & Claude.ai
#FileSizeDownloadsInfoDL
00CollabQualityEnforcer.py37.4 KB2ℹ️⬇️
01CollabQualityEnforcer_2.4_Roadmap.md21.7 KB0ℹ️⬇️
02CollabQualityEnforcer.py47.8 KB2ℹ️⬇️
03CollabQualityEnforcer.py50.8 KB1ℹ️⬇️


Claude's rewrite of Gemini's:

Code: [Select]
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""
CollabQualityEnforcer

PRODUCT: Collabware
PROJECT: Collabware Code Quality Program
MODULE: collabware_quality_core
MODULE_VERSION: 2.4.4
PURPOSE: Core infrastructure, reusable AST analysis context, expanded rule suite (strict headers, parse-failure/config/magic-number leaks, credit checks), and safe auto-remediation.
FUNCTION: Provides finding models, rule registries, quality gates, file discovery, reusable AST analysis context, safe auto-fixes with backups, and structured JSON/human reports.
DEPENDENCIES: dataclasses, enum, typing, json, datetime, pathlib, argparse, shutil, ast, re
STATUS: ACTIVE
LAST_MODIFIED: 2026-08-31
COLLABORATION: Andrew.human (Architect & QA Lead), Claude A/B (Lead Programmers, 2.2.0-2.4.4 passes), ChatGPT (Systems Analyst, spec + review), Gemini (Relief Programmer, AST context/parse-check/DOC-001 validation depth)

CHANGES IN 2.4.4:
    - Real-usage bug report: DOC-001's --fix always regenerated the whole
      header from a fixed template, clobbering real existing values (e.g.
      a genuine DEPENDENCIES or FUNCTION someone had written) with
      fabricated placeholder text ("Auto-remediated Module",
      "Standardized compliance block.") that has nothing to do with the
      actual file. Rewrote the fix to extract and preserve any field
      values already present, and to leave a field genuinely BLANK when
      it's missing and can't be determined (PROJECT, MODULE_VERSION,
      PURPOSE, FUNCTION, DEPENDENCIES, STATUS) rather than inventing
      text — a blank keeps DOC-001 correctly flagging it until a human
      fills it in for real, instead of silently papering over the gap.
      Only fields the tool can actually determine (MODULE from the
      filename, PRODUCT as the known project constant, COLLABORATION per
      standing convention, LAST_MODIFIED as today's date) get auto-filled
      when missing.

CHANGES IN 2.4.3:
    - Real-usage bug report: running --fix on a directory whose only
      findings were CONFIG-001 (hard-coded paths) reported "Fixes
      applied: 0" with no visible explanation, reading exactly like a
      broken --fix flag. It wasn't broken — CONFIG-001 is fixable=False
      by design (the tool can't guess what you want a hard-coded path
      replaced with) — but the output gave no indication of that. A
      finding now gets an explicit "[not auto-fixable — requires manual
      judgment]" annotation when --fix was passed and that finding isn't
      fixable, so a --fix run with zero fixes applied is self-explanatory
      instead of looking like a defect.

CHANGES IN 2.4.2:
    - Merged Gemini's parallel v2.4.2 draft. As with the 2.4.0 merge, that
      draft was built off an older base and had again silently dropped
      CRD-001's --fix, the docstring-duplication-safe fix, and the
      anchored coding-line regex. All three restored; see below for what
      was actually taken from the draft.
    - Adopted PythonAnalysisContext (from the draft) as a proper reusable
      AST context — pre-collects assignments/functions/classes/imports,
      captures parse errors instead of silently swallowing them. Replaces
      the bare SourceUnit.ast_tree from 2.4.1; foundation for the
      structural/loop rules still on Chatty's roadmap.
    - Added a new parse-failure rule (from the draft, where it was called
      STRUCT-001) flagging a file that fails to parse as Python at all,
      since every AST-based rule was silently no-oping on such files with
      no visible finding. Registered here as PARSE-001, not STRUCT-001:
      Chatty's spec reserves STRUCT-001 for "function too long", so
      reusing it now would silently collide (last register_rule() call
      for an ID wins) once that real rule gets built. For the same
      reason, this whole release is versioned 2.4.2, not 2.5.0 — it
      doesn't implement Chatty's actual "Version 2.5 — Structure and loop
      analysis" spec (function length, parameter count, nesting,
      complexity, loop analysis — none of that exists yet), so 2.5.0
      would misrepresent what's here and collide with that version
      number once it's really built.
    - Upgraded DOC-001 (from the draft, adapted): now validates field
      *values*, not just presence — PRODUCT must equal "Collabware",
      MODULE_VERSION must be semver, LAST_MODIFIED must be ISO-8601, and
      no mandatory field may be empty. Deliberately does NOT duplicate
      CRD-001's "does COLLABORATION credit the whole team" check — the
      draft had folded that into DOC-001 as fixable=False, which is how
      CRD-001's --fix went missing; kept the two rules separate instead.
    - Broadened CONFIG-001's IP check (from the draft) to any IPv4
      address, not just RFC1918 private ranges, excluding loopback/
      unspecified addresses.
    - Added private_key/token to SEC-001's secret patterns (from the
      draft).
    - Fixed a genuine bug in the draft's CONFIG-002 rewrite: it excluded
      magic numbers by VALUE equality against any module-level constant
      ("5 is fine anywhere because MAX_RETRIES = 5 exists somewhere"),
      which silently stops flagging every unrelated use of that number
      elsewhere in the file. Rewritten to exclude by AST node identity,
      scoped to true top-level assignments only.

CHANGES IN 2.4.1:
    - CONFIG-001's path_pattern regex had an invalid character-class range
      ("bad character range _-/") that crashed the entire run on any file
      containing a hard-coded Unix path — found and fixed during 2.4.0's
      own test pass, but that fix was folded silently into the 2.4.0
      delivery instead of being versioned as its own patch. Corrected here
      per standing convention: every change gets its own version bump,
      not bundled into the release it was caught during.

CHANGES IN 2.4.0:
    - Merged Gemini's parallel v2.4.0 draft with the fixes already shipped
      here — that draft was built off an older base and had silently
      dropped CRD-001, the docstring-duplication-safe fix, the anchored
      coding-line regex, and the human-readable console output. All four
      are restored/kept; see below for what was actually taken from it.
    - Upgraded DOC-001 (from Gemini's draft) to validate all 10 mandatory
      header fields instead of only checking for "PRODUCT: Collabware";
      it now reports exactly which fields are missing.
    - Added CONFIG-001 (from Gemini's draft, extended here): hard-coded
      IPs, URLs, filesystem paths, and now also service ports.
    - Added CONFIG-002: AST-based magic-number detection (new), closing
      out the remaining item from Chatty's v2.4 spec. Heuristic, MINOR
      severity, skips values assigned to an ALL_CAPS named constant.
    - Added AST parsing to SourceUnit (from Gemini's draft) as reusable
      infrastructure for CONFIG-002 and future structural/loop rules;
      re-parses automatically after an auto-fix rewrites a file.

CHANGES IN 2.3.0:
    - Default console output is now a live, human-readable report: prints
      the module currently being processed, its findings with suggested
      fixes, and whether an auto-fix was applied, followed by a summary.
    - Added --json flag to restore the old machine-readable JSON-only
      output for automation/CI use; the JSON report is still built and
      returned internally in both modes.
    - Added CRD-001: verifies the COLLABORATION header line credits the
      full team rather than a stale single-tool placeholder; --fix
      corrects it in place.

CHANGES IN 2.2.0:
    - Fixed insertion-point detection: "coding" in line matched any comment
      containing that substring; now anchored to an actual shebang or
      PEP 263 encoding declaration via regex.
    - Fixed header remediation: previously a second header docstring was
      always inserted ahead of any existing leading docstring, leaving two
      floating triple-quoted blocks and silently changing __doc__. Now an
      existing leading docstring is detected and replaced in place; a
      header is only appended fresh when no leading docstring exists.
    - RuleRegistry.get_rules() now returns a shallow copy so callers can't
      mutate the live registry.
    - Documented that duplicate/reuse/loop-nesting config thresholds are
      reserved for rules not yet implemented (not dead code by accident).

INVOCATION INSTRUCTIONS:
    python3 CollabQualityEnforcer.py --dir <path_to_target_directory> [options]

FLAGS:
    --dir       Path to the target directory containing Python scripts (Required)
    --project   Name of the project for the report (Default: "Collabware Project")
    --min-score Minimum acceptable quality score out of 100 (Default: 80.0)
    --max-majors Maximum allowed major findings before failing (Default: 5)
    --fix       Automatically create a .bak backup and apply safe remediations (e.g., inject missing headers)
    --json      Print the machine-readable JSON report instead of the default live human-readable output

EXAMPLE USAGE:
    python3 CollabQualityEnforcer.py --dir ./src --project "CollabCore" --fix
"""

from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Set
import argparse
import ast
import json
import re
import shutil


# The expected collaboration credit line, per Andrew's standing convention
# that every module credits the full team, regardless of who touched that
# particular file. Used by rule CRD-001 to detect stale/incomplete credit
# lines (e.g. leftover single-author or placeholder text from an older
# template), and referenced directly by RemediationEngine's header template.
# Safe to reference here rather than duplicate literally: check_collaboration_credit
# only inspects the FIRST "COLLABORATION:" line in a file, which for this
# module's own source is its real header above — the template's embedded
# occurrence further down is never reached during a self-scan.
TEAM_CREDIT_LINE = (
    "Andrew.human (Architect & QA Lead), Claude A/B (Lead Programmers), "
    "ChatGPT (Systems Analyst), Gemini (Relief Programmer)"
)


class Severity(Enum):
    """Severity levels for code quality findings."""
    BLOCKER = "BLOCKER"
    CRITICAL = "CRITICAL"
    MAJOR = "MAJOR"
    MINOR = "MINOR"
    INFORMATION = "INFORMATION"


class Category(Enum):
    """Categories for code quality findings."""
    DOCUMENTATION = "DOCUMENTATION"
    STRUCTURE = "STRUCTURE"
    SECURITY = "SECURITY"
    CONFIGURATION = "CONFIGURATION"
    PERFORMANCE = "PERFORMANCE"
    REUSE = "REUSE"
    DEPENDENCY = "DEPENDENCY"


class GateStatus(Enum):
    """Quality Gate evaluation status."""
    PASS = "PASS"
    FAIL = "FAIL"


@dataclass
class PythonAnalysisContext:
    """Reusable AST analysis context: parses once and pre-collects the node
    kinds most rules need (assignments, functions, classes, imports,
    constants), plus a captured parse error instead of a silent None so
    PARSE-001 can report exactly what went wrong.
    """
    file_path: str
    source_text: str
    ast_tree: Optional[ast.AST] = None
    parse_error: Optional[str] = None
    assignments: List[ast.Assign] = field(default_factory=list)
    functions: List[ast.FunctionDef] = field(default_factory=list)
    classes: List[ast.ClassDef] = field(default_factory=list)
    imports: List[Any] = field(default_factory=list)
    constants: List[ast.Constant] = field(default_factory=list)

    @classmethod
    def for_unit(cls, file_path: str, content: str) -> "PythonAnalysisContext":
        """Parse content and build the analysis context in one pass."""
        context = cls(file_path=file_path, source_text=content)
        if not content:
            return context
        try:
            context.ast_tree = ast.parse(content, filename=file_path)
            for node in ast.walk(context.ast_tree):
                if isinstance(node, ast.Assign):
                    context.assignments.append(node)
                elif isinstance(node, ast.FunctionDef):
                    context.functions.append(node)
                elif isinstance(node, ast.ClassDef):
                    context.classes.append(node)
                elif isinstance(node, (ast.Import, ast.ImportFrom)):
                    context.imports.append(node)
                elif isinstance(node, ast.Constant):
                    context.constants.append(node)
        except SyntaxError as exc:
            context.ast_tree = None
            context.parse_error = str(exc)
        return context


@dataclass
class SourceUnit:
    """Represents a discoverable source file unit for analysis, including its
    reusable AST analysis context (see PythonAnalysisContext).

    context.ast_tree is None when the file has a syntax error — AST-based
    rules should check for None and skip rather than crash the whole run.
    PARSE-001 is the rule that actually reports the syntax error itself.
    """
    file_path: str
    content: str
    context: Optional[PythonAnalysisContext] = None

    def __post_init__(self) -> None:
        self.reparse()

    def reparse(self) -> None:
        """(Re)build self.context from self.content. Call after mutating
        content directly (e.g. after an auto-fix rewrites the file) since
        dataclass field assignment doesn't re-trigger __post_init__."""
        self.context = PythonAnalysisContext.for_unit(self.file_path, self.content)


@dataclass
class Finding:
    """Standardized finding model representing a single code quality issue."""
    rule_id: str
    severity: Severity
    category: Category
    message: str
    file: Optional[str] = None
    line: Optional[int] = None
    column: Optional[int] = None
    confidence: float = 1.0
    recommendation: Optional[str] = None
    suppression_status: bool = False
    details: Optional[str] = None
    fixable: bool = False


@dataclass
class QualityConfiguration:
    """Configuration thresholds for the quality engine.

    NOTE: secret_threshold, duplicate_threshold, collabcore_reuse_threshold,
    and max_loop_nesting are reserved for rules not yet implemented
    (similarity/duplication and complexity checks). They are intentionally
    present but unused by the current rule set — not accidental dead code.
    """
    max_major_findings: int = 5
    minimum_quality_score: float = 80.0
    secret_threshold: float = 0.8
    duplicate_threshold: float = 0.85
    collabcore_reuse_threshold: float = 0.90
    max_loop_nesting: int = 3
    suppressed_rules: List[str] = field(default_factory=list)


# ==========================================
# BUILT-IN COLLABWARE QUALITY RULES
# ==========================================

def check_structure_parse_failure(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule PARSE-001: Flags a source file that fails to parse as valid
    Python at all. Every AST-based rule (CONFIG-002, and future structural/
    loop rules) silently no-ops on such a file, so this makes the failure
    visible instead of just quietly skipping analysis."""
    findings = []
    if unit.context and unit.context.parse_error:
        findings.append(
            Finding(
                rule_id="PARSE-001",
                severity=Severity.MAJOR,
                category=Category.STRUCTURE,
                message=f"Python source could not be parsed: {unit.context.parse_error}",
                file=unit.file_path,
                recommendation="Fix the Python syntax error so structural/AST-based analysis can run.",
                fixable=False
            )
        )
    return findings


# The 10 mandatory Collabware header fields, in template order.
_MANDATORY_HEADER_FIELDS = (
    "PRODUCT", "PROJECT", "MODULE", "MODULE_VERSION",
    "PURPOSE", "FUNCTION", "DEPENDENCIES", "STATUS",
    "LAST_MODIFIED", "COLLABORATION"
)

_SEMVER_PATTERN = re.compile(r'^\d+\.\d+\.\d+(-[a-zA-Z0-9.]+)?$')
_ISO_DATE_PATTERN = re.compile(r'^\d{4}-\d{2}-\d{2}$')


def _extract_collabware_header_fields(content: str) -> Dict[str, str]:
    """Extract {FIELD_NAME: value} for whichever mandatory header fields
    appear in the first 30 lines of the file. Only presence/value — the
    caller decides what counts as missing, empty, or malformed."""
    fields: Dict[str, str] = {}
    for line in content.splitlines()[:30]:
        for field_name in _MANDATORY_HEADER_FIELDS:
            if field_name in fields:
                continue
            match = re.match(rf'^\s*(?:\*\s*)?{field_name}\s*:\s*(.*)$', line, re.IGNORECASE)
            if match:
                fields[field_name] = match.group(1).strip()
    return fields


def check_documentation_header(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule DOC-001: Validates the Collabware documentation header — not
    just that all 10 mandatory fields are present, but that their values
    are actually sane: PRODUCT is literally "Collabware", MODULE_VERSION
    is proper semver, LAST_MODIFIED is ISO-8601, and no field is empty.

    COLLABORATION's *content* (does it credit the whole team?) is
    deliberately NOT checked here — that's CRD-001's job, which also owns
    the auto-fix for it. This rule only checks that COLLABORATION exists
    and isn't blank, to avoid two rules fighting over the same line.
    """
    findings = []
    fields = _extract_collabware_header_fields(unit.content)

    if not fields:
        findings.append(
            Finding(
                rule_id="DOC-001",
                severity=Severity.MAJOR,
                category=Category.DOCUMENTATION,
                message="Missing Collabware documentation header block entirely.",
                file=unit.file_path,
                recommendation="Inject a complete standard Collabware documentation header.",
                fixable=True
            )
        )
        return findings

    missing_or_empty = [
        name for name in _MANDATORY_HEADER_FIELDS
        if name not in fields or not fields[name]
    ]
    if missing_or_empty:
        findings.append(
            Finding(
                rule_id="DOC-001",
                severity=Severity.MAJOR,
                category=Category.DOCUMENTATION,
                message=(
                    "Incomplete Collabware documentation header. "
                    f"Missing or empty fields: {', '.join(missing_or_empty)}"
                ),
                file=unit.file_path,
                recommendation="Inject a complete standard Collabware documentation header with all mandatory fields.",
                fixable=True
            )
        )

    if fields.get("PRODUCT") and fields["PRODUCT"].lower() != "collabware":
        findings.append(
            Finding(
                rule_id="DOC-001",
                severity=Severity.MAJOR,
                category=Category.DOCUMENTATION,
                message=f"Invalid PRODUCT value: '{fields['PRODUCT']}'. Expected 'Collabware'.",
                file=unit.file_path,
                recommendation="Set PRODUCT to 'Collabware'.",
                fixable=False
            )
        )

    if fields.get("MODULE_VERSION") and not _SEMVER_PATTERN.match(fields["MODULE_VERSION"]):
        findings.append(
            Finding(
                rule_id="DOC-001",
                severity=Severity.MINOR,
                category=Category.DOCUMENTATION,
                message=f"Invalid MODULE_VERSION format: '{fields['MODULE_VERSION']}'. Expected semantic version (e.g. 2.5.0).",
                file=unit.file_path,
                recommendation="Use standard semantic versioning (MAJOR.MINOR.PATCH).",
                fixable=False
            )
        )

    if fields.get("LAST_MODIFIED") and not _ISO_DATE_PATTERN.match(fields["LAST_MODIFIED"]):
        findings.append(
            Finding(
                rule_id="DOC-001",
                severity=Severity.MINOR,
                category=Category.DOCUMENTATION,
                message=f"Invalid LAST_MODIFIED date format: '{fields['LAST_MODIFIED']}'. Expected YYYY-MM-DD.",
                file=unit.file_path,
                recommendation="Use ISO-8601 date format YYYY-MM-DD.",
                fixable=False
            )
        )

    return findings


def check_hardcoded_configuration(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule CONFIG-001: Detects hard-coded network endpoints, URLs, IP
    addresses, filesystem paths, and service ports."""
    findings = []

    # Broadened to any IPv4-looking literal (not just RFC1918 private
    # ranges) — a public IP hard-coded in source is just as much a
    # configuration smell as a private one. Loopback/unspecified excluded
    # below since those are common, usually-fine defaults.
    ip_pattern = re.compile(r'\b(?:\d{1,3}\.){3}\d{1,3}\b')
    url_pattern = re.compile(r'https?://[^\s\'"]+')
    path_pattern = re.compile(r'([A-Za-z]:\\[\\\w._-]+|/(?:opt|var|etc|home|usr|bin|root)/[\\\w./-]+)')
    # Matches PORT/port-style identifiers assigned a plausible port number,
    # e.g. PORT = 8080, db_port=5432, SERVER_PORT: 443. Bounded to 2-5 digits
    # to avoid flagging unrelated large integers.
    port_pattern = re.compile(r'\b\w*port\w*\s*[:=]\s*\d{2,5}\b', re.IGNORECASE)

    for line_num, line in enumerate(unit.content.splitlines(), start=1):
        stripped = line.strip()
        if stripped.startswith('#') or stripped.startswith('"""') or stripped.startswith("'''"):
            continue

        if ip_pattern.search(line) and "127.0.0.1" not in line and "0.0.0.0" not in line:
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MAJOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded internal IP address detected in configuration assignment or expression.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Externalize IP addresses into configuration files or environment variables.",
                    fixable=False
                )
            )
        elif url_pattern.search(line) and "localhost" not in line and "127.0.0.1" not in line:
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MAJOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded service URL endpoint detected.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Externalize service endpoints into configuration properties.",
                    fixable=False
                )
            )
        elif port_pattern.search(line):
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MINOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded service port number detected.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Externalize port numbers into configuration files or environment variables.",
                    fixable=False
                )
            )
        elif path_pattern.search(line):
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MINOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded absolute filesystem path detected.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Use relative paths or configuration-driven directory roots.",
                    fixable=False
                )
            )

    return findings


# Numeric literals excluded from the magic-number check as universally
# unremarkable (loop starts/steps, booleans-as-ints, sign flips, percentages).
_MAGIC_NUMBER_ALLOWLIST = {0, 1, -1, 100}


def check_magic_numbers(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule CONFIG-002: AST-based heuristic for unnamed numeric literals
    ("magic numbers"), excluding values assigned directly to an ALL_CAPS
    name at true module (top-level) scope, plus a small allowlist.

    Two things worth being explicit about, since a naive version of this
    rule is easy to get subtly wrong:
    - Scope: only a top-level `NAME = value` counts as a "named constant"
      declaration. An ALL_CAPS local inside a function is not exempted —
      that's not the pattern this rule is trying to encourage.
    - Exclusion method: the specific Constant AST node assigned to a named
      constant is excluded by identity, not by value. Excluding by value
      (i.e. "5 is fine anywhere because MAX_RETRIES = 5 exists somewhere")
      would silently stop flagging every unrelated use of that number
      elsewhere in the file — a real bug, not just a style nitpick.

    This is a heuristic, not a proof — it will still flag legitimate cases
    (list indices, HTTP status codes compared inline, etc.). Severity is
    MINOR precisely because of that: a nudge to extract a named constant,
    not a hard failure.
    """
    findings = []
    if not unit.context or not unit.context.ast_tree:
        return findings

    # Identify Constant nodes that are the direct RHS of a module-level
    # `ALL_CAPS = <constant>` assignment, by node identity.
    named_constant_value_ids: Set[int] = set()
    for node in unit.context.ast_tree.body:
        if (
            isinstance(node, ast.Assign)
            and len(node.targets) == 1
            and isinstance(node.targets[0], ast.Name)
            and node.targets[0].id.isupper()
            and isinstance(node.value, ast.Constant)
        ):
            named_constant_value_ids.add(id(node.value))

    for node in ast.walk(unit.context.ast_tree):
        if not isinstance(node, ast.Constant):
            continue
        if isinstance(node.value, bool):
            continue  # bool is a subclass of int; not a magic number
        if not isinstance(node.value, (int, float)):
            continue
        if node.value in _MAGIC_NUMBER_ALLOWLIST:
            continue
        if id(node) in named_constant_value_ids:
            continue

        findings.append(
            Finding(
                rule_id="CONFIG-002",
                severity=Severity.MINOR,
                category=Category.CONFIGURATION,
                message=f"Magic number literal detected: {node.value!r}",
                file=unit.file_path,
                line=getattr(node, "lineno", None),
                confidence=0.6,
                recommendation="Extract into a named ALL_CAPS constant at module scope.",
                fixable=False
            )
        )
    return findings


def check_hardcoded_secrets(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule SEC-001: Heuristic check for potential hardcoded credentials or API tokens."""
    findings = []
    secret_patterns = [
        re.compile(r'(password|passwd|pwd|secret|api_key|access_token|private_key|token)\s*=\s*[\'"][^\'"]+[\'"]', re.IGNORECASE)
    ]

    for line_num, line in enumerate(unit.content.splitlines(), start=1):
        for pattern in secret_patterns:
            if pattern.search(line):
                findings.append(
                    Finding(
                        rule_id="SEC-001",
                        severity=Severity.CRITICAL,
                        category=Category.SECURITY,
                        message="Potential hard-coded secret detected.",
                        file=unit.file_path,
                        line=line_num,
                        confidence=0.85,
                        recommendation="Move sensitive credentials to environment variables or secret vaults.",
                        fixable=False
                    )
                )
    return findings


# Required team identifiers that a valid COLLABORATION line must mention.
# Substring/case-insensitive so reasonable variation (extra detail, version
# notes, punctuation) doesn't trip a false positive — this only flags a
# credit line that is missing one or more team members entirely, e.g. a
# stale single-tool placeholder like "CollabQualityEnforcer Auto-Remediation".
_REQUIRED_CREDIT_TOKENS = ("andrew", "claude", "chatgpt", "gemini")

_COLLABORATION_LINE = re.compile(r'^(?P<indent>\s*)COLLABORATION:\s*(?P<credit>.*)$', re.MULTILINE)


def check_collaboration_credit(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule CRD-001: Verifies a COLLABORATION line, if present, credits the
    full team rather than a stale placeholder or partial credit."""
    findings = []

    for line_num, line in enumerate(unit.content.splitlines(), start=1):
        match = _COLLABORATION_LINE.match(line)
        if not match:
            continue

        credit_text = match.group("credit").strip()
        credit_lower = credit_text.lower()
        missing = [name for name in _REQUIRED_CREDIT_TOKENS if name not in credit_lower]

        if missing:
            findings.append(
                Finding(
                    rule_id="CRD-001",
                    severity=Severity.MAJOR,
                    category=Category.DOCUMENTATION,
                    message=(
                        f"Collaboration credit line does not credit the full team "
                        f"(found: '{credit_text}')."
                    ),
                    file=unit.file_path,
                    line=line_num,
                    recommendation=f"Update COLLABORATION line to: {TEAM_CREDIT_LINE}",
                    fixable=True
                )
            )
        # Only the first COLLABORATION line (the module header) is checked;
        # a second occurrence is typically an embedded template/string, not
        # a second real header.
        break

    return findings


class RuleRegistry:
    """Dynamic rule registry pre-loaded with default Collabware quality rules."""

    def __init__(self) -> None:
        self._rules: Dict[str, Callable[[SourceUnit, QualityConfiguration], List[Finding]]] = {}
        self.register_rule("PARSE-001", check_structure_parse_failure)
        self.register_rule("DOC-001", check_documentation_header)
        self.register_rule("SEC-001", check_hardcoded_secrets)
        self.register_rule("CRD-001", check_collaboration_credit)
        self.register_rule("CONFIG-001", check_hardcoded_configuration)
        self.register_rule("CONFIG-002", check_magic_numbers)

    def register_rule(
        self,
        rule_id: str,
        rule_func: Callable[[SourceUnit, QualityConfiguration], List[Finding]]
    ) -> None:
        """Register a new quality rule function into the engine registry."""
        self._rules[rule_id] = rule_func

    def get_rules(self) -> Dict[str, Callable[[SourceUnit, QualityConfiguration], List[Finding]]]:
        """Retrieve a copy of all currently registered quality rules.

        Returns a shallow copy so external callers can't mutate the live
        registry by modifying the returned dict.
        """
        return dict(self._rules)


class QualityGate:
    """Evaluates findings against deterministic quality gate thresholds."""

    @staticmethod
    def evaluate(findings: List[Finding], configuration: QualityConfiguration) -> GateStatus:
        """Evaluate whether active findings pass or fail the quality gate."""
        active_findings = [f for f in findings if not f.suppression_status]

        for finding in active_findings:
            if finding.severity in (Severity.BLOCKER, Severity.CRITICAL):
                return GateStatus.FAIL

        major_count = sum(1 for f in active_findings if f.severity == Severity.MAJOR)
        if major_count > configuration.max_major_findings:
            return GateStatus.FAIL

        score = QualityGate.calculate_score(active_findings)
        if score < configuration.minimum_quality_score:
            return GateStatus.FAIL

        return GateStatus.PASS

    @staticmethod
    def calculate_score(findings: List[Finding]) -> float:
        """Calculate a normalized quality score out of 100 based on severity penalties."""
        weights = {
            Severity.BLOCKER: 25.0,
            Severity.CRITICAL: 15.0,
            Severity.MAJOR: 5.0,
            Severity.MINOR: 1.0,
            Severity.INFORMATION: 0.0,
        }
        penalty = sum(weights.get(f.severity, 0.0) for f in findings)
        score = max(0.0, 100.0 - penalty)
        return round(score, 2)


class RemediationEngine:
    """Applies automated safe fixes to discovered code issues with mandatory backups."""

    # Matches a shebang line, or a PEP 263 encoding declaration on line 1 or 2
    # (e.g. "# -*- coding: utf-8 -*-" or "# coding=utf-8"). The encoding form
    # requires the comment to START with optional "-*-" framing then
    # "coding[:=]<charset>" per PEP 263, so a comment that merely mentions
    # the word "coding" elsewhere in its text doesn't match. Trailing
    # decoration like the closing "-*-" is allowed after the charset.
    _SHEBANG_OR_ENCODING = re.compile(
        r'^(#!.*|#\s*(-\*-\s*)?coding[:=]\s*[-\w.]+(\s*-\*-)?\s*)$'
    )

    # Matches the start of a leading module docstring, single or triple
    # quoted, single or double quote style.
    _DOCSTRING_START = re.compile(r'^\s*(?P<quote>\'\'\'|""")')

    @classmethod
    def _find_insertion_index(cls, lines: List[str]) -> int:
        """Find the line index just past any shebang/encoding preamble."""
        insert_index = 0
        for idx, line in enumerate(lines[:2]):
            if cls._SHEBANG_OR_ENCODING.match(line):
                insert_index = idx + 1
        return insert_index

    @classmethod
    def _find_existing_docstring_span(cls, lines: List[str], start_index: int):
        """If a module docstring begins at/after start_index (skipping blank
        lines), return (start, end) line indices spanning it, else None.
        `end` is exclusive.
        """
        idx = start_index
        while idx < len(lines) and lines[idx].strip() == "":
            idx += 1
        if idx >= len(lines):
            return None

        match = cls._DOCSTRING_START.match(lines[idx])
        if not match:
            return None

        quote = match.group("quote")
        doc_start = idx
        # Does the docstring open and close on the same line? (len > 6 covers
        # the two quote pairs plus at least something between them, or an
        # empty docstring like """""" which is len 6.)
        first_line = lines[idx]
        after_open = first_line[first_line.find(quote) + 3:]
        if quote in after_open:
            return (doc_start, idx + 1)

        idx += 1
        while idx < len(lines):
            if quote in lines[idx]:
                return (doc_start, idx + 1)
            idx += 1

        # Unterminated docstring — treat as not found rather than guessing.
        return None

    @staticmethod
    def _backup_suffix(index: int) -> str:
        """Suffix for the Nth backup: .bak, .bak2, .bak3, ..."""
        return ".bak" if index == 1 else f".bak{index}"

    @classmethod
    def _rotate_backups(cls, file_path: Path) -> Path:
        """Shift any existing .bak chain up by one slot and return the now-free
        .bak path, so a new backup never overwrites an older one.

        e.g. if .bak and .bak2 already exist: .bak2 -> .bak3, .bak -> .bak2,
        then .bak is returned as the destination for the new backup.
        """
        chain_length = 1
        while file_path.with_name(file_path.name + cls._backup_suffix(chain_length)).exists():
            chain_length += 1
        # chain_length is now the first free slot; shift existing ones up,
        # starting from the oldest (highest index) so nothing is clobbered.
        for index in range(chain_length, 1, -1):
            src = file_path.with_name(file_path.name + cls._backup_suffix(index - 1))
            dst = file_path.with_name(file_path.name + cls._backup_suffix(index))
            src.rename(dst)
        return file_path.with_name(file_path.name + cls._backup_suffix(1))

    @classmethod
    def apply_fixes(cls, source_unit: SourceUnit, findings: List[Finding]) -> bool:
        """Create a .bak backup (chaining any existing ones to .bak2, .bak3, ...)
        and apply automated corrections to a source unit."""
        file_path = Path(source_unit.file_path)
        content = source_unit.content
        modified = False

        for finding in findings:
            if finding.rule_id == "DOC-001" and finding.fixable:
                # Preserve whatever field values already exist (e.g. a real
                # PURPOSE/FUNCTION/DEPENDENCIES someone wrote) rather than
                # clobbering them with generic placeholder text. Only fields
                # this tool can actually determine get a value when missing;
                # everything else is left blank on purpose, so the file
                # keeps failing DOC-001 until a human fills it in for real —
                # a blank field is honest, a fabricated one ("Auto-remediated
                # Module", "Standardized compliance block.") is misleading
                # in someone else's real project file.
                existing_fields = _extract_collabware_header_fields(content)
                module_name = file_path.stem
                current_date = datetime.now(timezone.utc).strftime("%Y-%m-%d")

                def _resolve(field_name: str) -> str:
                    if field_name == "LAST_MODIFIED":
                        # This is a true fact about the fix happening now,
                        # not a guess — always safe to set.
                        return current_date
                    if field_name == "MODULE":
                        # Objectively derivable from the filename itself.
                        return existing_fields.get("MODULE") or module_name
                    if field_name == "PRODUCT":
                        # Known project-wide constant, not a per-file guess.
                        return existing_fields.get("PRODUCT") or "Collabware"
                    if field_name == "COLLABORATION":
                        # Known-correct value per standing convention, not
                        # a per-file guess — same reasoning CRD-001 uses.
                        return existing_fields.get("COLLABORATION") or TEAM_CREDIT_LINE
                    # PROJECT, MODULE_VERSION, PURPOSE, FUNCTION,
                    # DEPENDENCIES, STATUS: no way to know these for an
                    # arbitrary file, so keep whatever's already there and
                    # otherwise leave blank rather than fabricate.
                    return existing_fields.get(field_name, "")

                header_lines = ['"""']
                for field_name in _MANDATORY_HEADER_FIELDS:
                    header_lines.append(f"{field_name}: {_resolve(field_name)}")
                header_lines.append('"""\n')
                header_body = "\n".join(header_lines)

                lines = content.splitlines(keepends=True)
                insert_index = cls._find_insertion_index(lines)
                existing_span = cls._find_existing_docstring_span(lines, insert_index)

                if existing_span:
                    # Replace the existing leading docstring in place instead
                    # of stacking a second one ahead of it.
                    doc_start, doc_end = existing_span
                    lines[doc_start:doc_end] = [header_body]
                else:
                    lines.insert(insert_index, header_body + "\n")

                content = "".join(lines)
                modified = True

            elif finding.rule_id == "CRD-001" and finding.fixable:
                # Replace only the first COLLABORATION line's credit text,
                # preserving indentation, matching what check_collaboration_credit
                # inspected (the first COLLABORATION line in the file).
                new_content, count = _COLLABORATION_LINE.subn(
                    lambda m: f'{m.group("indent")}COLLABORATION: {TEAM_CREDIT_LINE}',
                    content,
                    count=1
                )
                if count:
                    content = new_content
                    modified = True

        if modified:
            # Rotate any existing backup chain (.bak -> .bak2 -> .bak3 ...)
            # then write the current pre-fix content as the new .bak.
            backup_path = cls._rotate_backups(file_path)
            shutil.copy2(file_path, backup_path)

            # Write remediated content
            file_path.write_text(content, encoding="utf-8")
            return True, backup_path

        return False, None


class ReportGenerator:
    """Generates human-readable and machine-readable output reports."""

    @staticmethod
    def generate_json_report(
        project_name: str,
        findings: List[Finding],
        gate_status: GateStatus,
        score: float,
        fixes_applied: int = 0
    ) -> str:
        """Generate a machine-readable JSON report of the quality run."""
        summary = {
            "blocker": sum(1 for f in findings if f.severity == Severity.BLOCKER),
            "critical": sum(1 for f in findings if f.severity == Severity.CRITICAL),
            "major": sum(1 for f in findings if f.severity == Severity.MAJOR),
            "minor": sum(1 for f in findings if f.severity == Severity.MINOR),
            "information": sum(1 for f in findings if f.severity == Severity.INFORMATION),
            "fixes_applied": fixes_applied
        }

        report_data = {
            "product": "Collabware",
            "project": project_name,
            "analysis_timestamp": datetime.now(timezone.utc).isoformat(),
            "quality_score": score,
            "quality_gate": gate_status.value,
            "summary": summary,
            "findings": [
                {
                    "rule_id": f.rule_id,
                    "severity": f.severity.value,
                    "category": f.category.value,
                    "file": f.file,
                    "line": f.line,
                    "message": f.message,
                    "confidence": f.confidence,
                    "recommendation": f.recommendation,
                    "fixable": f.fixable
                }
                for f in findings
            ],
        }
        return json.dumps(report_data, indent=4)


class HumanReportPrinter:
    """Prints live, human-readable progress and a final summary to stdout.

    This is the default console output. It is intentionally separate from
    ReportGenerator (which produces the machine-readable JSON) so the two
    can evolve independently — JSON is for automation/CI, this is for a
    person watching the run happen.
    """

    _SEVERITY_MARKS = {
        Severity.BLOCKER: "✖ BLOCKER ",
        Severity.CRITICAL: "✖ CRITICAL",
        Severity.MAJOR: "⚠ MAJOR   ",
        Severity.MINOR: "· MINOR   ",
        Severity.INFORMATION: "· INFO    ",
    }

    @classmethod
    def print_processing_file(cls, file_path: str) -> None:
        """Announce which module is currently being analyzed."""
        print(f"\nProcessing: {file_path}")

    @classmethod
    def print_finding(cls, finding: Finding, suppressed: bool, auto_fix: bool = False) -> None:
        """Print one finding with its suggested fix, indented under the file.

        When auto_fix is True (i.e. --fix was passed) and this finding
        isn't fixable, say so explicitly — otherwise a run that reports
        findings but applies zero fixes looks indistinguishable from a
        broken --fix flag, when really it just had nothing fixable to do.
        """
        mark = cls._SEVERITY_MARKS.get(finding.severity, finding.severity.value)
        location = f" (line {finding.line})" if finding.line else ""
        suffix = "  [SUPPRESSED]" if suppressed else ""
        if auto_fix and not finding.fixable and not suppressed:
            suffix += "  [not auto-fixable — requires manual judgment]"
        print(f"  {mark} [{finding.rule_id}] {finding.message}{location}{suffix}")
        if finding.recommendation:
            print(f"             -> {finding.recommendation}")

    @classmethod
    def print_no_issues(cls) -> None:
        """Print confirmation that a file had no findings."""
        print("  (no issues found)")

    @classmethod
    def print_fix_applied(cls, backup_path: Path) -> None:
        """Confirm an auto-fix was applied and report the actual backup file written
        (may be .bak, .bak2, .bak3, ... depending on chain rotation)."""
        print(f"  -> auto-fix applied (backup written: {backup_path.name})")

    @classmethod
    def print_summary(
        cls,
        project_name: str,
        files_scanned: int,
        findings: List[Finding],
        gate_status: GateStatus,
        score: float,
        fixes_applied: int
    ) -> None:
        """Print the final run summary once all files have been processed."""
        active = [f for f in findings if not f.suppression_status]
        counts = {
            sev: sum(1 for f in active if f.severity == sev)
            for sev in Severity
        }
        print("\n" + "=" * 60)
        print(f"Project:        {project_name}")
        print(f"Files scanned:  {files_scanned}")
        print(f"Fixes applied:  {fixes_applied}")
        print(
            "Findings:       "
            f"{counts[Severity.BLOCKER]} blocker, "
            f"{counts[Severity.CRITICAL]} critical, "
            f"{counts[Severity.MAJOR]} major, "
            f"{counts[Severity.MINOR]} minor, "
            f"{counts[Severity.INFORMATION]} info"
        )
        print(f"Quality Score:  {score}/100")
        print(f"Quality Gate:   {gate_status.value}")
        print("=" * 60)


class QualityEngine:
    """Main orchestrator for the Collabware Code Quality Program."""

    def __init__(self, registry: RuleRegistry) -> None:
        self.registry = registry

    @staticmethod
    def discover_source_units(target_directory: str) -> List[SourceUnit]:
        """Recursively discover and load all Python scripts in the target directory."""
        source_units: List[SourceUnit] = []
        directory_path = Path(target_directory)

        if not directory_path.exists() or not directory_path.is_dir():
            return source_units

        for file_path in directory_path.rglob("*.py"):
            # Skip backup files during discovery (.bak, .bak2, .bak3, ...)
            if re.search(r'\.bak\d*$', file_path.name):
                continue
            try:
                content = file_path.read_text(encoding="utf-8")
                source_units.append(SourceUnit(file_path=str(file_path), content=content))
            except (IOError, OSError):
                continue

        return source_units

    def run_check(
        self,
        project_name: str,
        configuration: QualityConfiguration,
        target_directory: str,
        auto_fix: bool = False,
        human_output: bool = True
    ) -> str:
        """Execute quality check pipeline with automatic backup and remediation capabilities.

        When human_output is True (the default), progress is printed live as
        each module is processed — which file is being scanned, what was
        found, and what fix (if any) was applied — followed by a short
        summary. The JSON report is still built and returned in all cases
        so callers/automation can consume it; pass human_output=False (or
        use --json on the CLI) to suppress the console narration entirely.
        """
        source_units = self.discover_source_units(target_directory)
        all_findings: List[Finding] = []
        fixes_count = 0

        rules = self.registry.get_rules()
        for unit in source_units:
            if human_output:
                HumanReportPrinter.print_processing_file(unit.file_path)

            unit_findings: List[Finding] = []
            for _, rule_func in rules.items():
                findings = rule_func(unit, configuration)
                for finding in findings:
                    if not finding.file:
                        finding.file = unit.file_path
                unit_findings.extend(findings)

            fix_applied = False
            backup_path = None
            if auto_fix and unit_findings:
                fix_applied, backup_path = RemediationEngine.apply_fixes(unit, unit_findings)
                if fix_applied:
                    fixes_count += sum(1 for f in unit_findings if f.fixable)
                    updated_content = Path(unit.file_path).read_text(encoding="utf-8")
                    unit.content = updated_content
                    unit.reparse()
                    unit_findings = []
                    for _, rule_func in rules.items():
                        new_findings = rule_func(unit, configuration)
                        for finding in new_findings:
                            if not finding.file:
                                finding.file = unit.file_path
                        unit_findings.extend(new_findings)

            if human_output:
                if unit_findings:
                    for finding in unit_findings:
                        suppressed = finding.rule_id in configuration.suppressed_rules
                        HumanReportPrinter.print_finding(finding, suppressed, auto_fix=auto_fix)
                else:
                    HumanReportPrinter.print_no_issues()
                if fix_applied:
                    HumanReportPrinter.print_fix_applied(backup_path)

            all_findings.extend(unit_findings)

        for finding in all_findings:
            if finding.rule_id in configuration.suppressed_rules:
                finding.suppression_status = True

        active_findings = [f for f in all_findings if not f.suppression_status]
        score = QualityGate.calculate_score(active_findings)
        gate_status = QualityGate.evaluate(all_findings, configuration)

        if human_output:
            HumanReportPrinter.print_summary(
                project_name, len(source_units), all_findings, gate_status, score, fixes_count
            )

        return ReportGenerator.generate_json_report(project_name, all_findings, gate_status, score, fixes_count)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="CollabQualityEnforcer Static Analysis & Safe Remediation Engine")
    parser.add_argument("--dir", required=True, help="Target directory containing Python source files")
    parser.add_argument("--project", default="Collabware Project", help="Project name for the report")
    parser.add_argument("--min-score", type=float, default=80.0, help="Minimum acceptable quality score")
    parser.add_argument("--max-majors", type=int, default=5, help="Maximum allowed major findings")
    parser.add_argument("--fix", action="store_true", help="Create .bak backups and automatically apply safe remediations")
    parser.add_argument(
        "--json",
        action="store_true",
        help="Print the machine-readable JSON report instead of the default live human-readable output"
    )

    args = parser.parse_args()

    registry = RuleRegistry()
    engine = QualityEngine(registry)
    config = QualityConfiguration(
        max_major_findings=args.max_majors,
        minimum_quality_score=args.min_score
    )

    report_json = engine.run_check(
        args.project, config, args.dir, auto_fix=args.fix, human_output=not args.json
    )
    if args.json:
        print(report_json)

Gemini:

Code: [Select]
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""
CollabQualityEnforcer

PRODUCT: Collabware
PROJECT: Collabware Code Quality Program
MODULE: collabware_quality_core
MODULE_VERSION: 2.4.2
PURPOSE: Core infrastructure, complete DOC-001 strict header validation, reusable AST analysis context, STRUCT-001 parse failure detection, CONFIG rules, and safe remediation.
FUNCTION: Provides finding models, rule registries, quality gates, file discovery, reusable AST context, safe auto-fixes with backups, and structured JSON/human reports.
DEPENDENCIES: dataclasses, enum, typing, json, datetime, pathlib, argparse, shutil, ast, re
STATUS: ACTIVE
LAST_MODIFIED: 2026-08-31
COLLABORATION: Andrew.human, Gemini AI/API, Claude AI/API, ChatGPT AI/API

INVOCATION INSTRUCTIONS:
    python3 CollabQualityEnforcer.py --dir <path_to_target_directory> [options]

FLAGS:
    --dir       Path to the target directory containing Python scripts (Required)
    --project   Name of the project for the report (Default: "Collabware Project")
    --min-score Minimum acceptable quality score out of 100 (Default: 80.0)
    --max-majors Maximum allowed major findings before failing (Default: 5)
    --fix       Create rotated .bak backups and automatically apply safe remediations (e.g., inject full standard headers)

EXAMPLE USAGE:
    python3 CollabQualityEnforcer.py --dir ./src --project "CollabCore" --fix
"""

from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Set
import argparse
import ast
import json
import re
import shutil


class Severity(Enum):
    """Severity levels for code quality findings."""
    BLOCKER = "BLOCKER"
    CRITICAL = "CRITICAL"
    MAJOR = "MAJOR"
    MINOR = "MINOR"
    INFORMATION = "INFORMATION"


class Category(Enum):
    """Categories for code quality findings."""
    DOCUMENTATION = "DOCUMENTATION"
    STRUCTURE = "STRUCTURE"
    SECURITY = "SECURITY"
    CONFIGURATION = "CONFIGURATION"
    PERFORMANCE = "PERFORMANCE"
    REUSE = "REUSE"
    DEPENDENCY = "DEPENDENCY"


class GateStatus(Enum):
    """Quality Gate evaluation status."""
    PASS = "PASS"
    FAIL = "FAIL"


@dataclass
class PythonAnalysisContext:
    """Reusable AST analysis context containing parsed elements and metadata."""
    file_path: str
    source_text: str
    ast_tree: Optional[ast.AST] = None
    parse_error: Optional[str] = None
    assignments: List[ast.Assign] = field(default_factory=list)
    functions: List[ast.FunctionDef] = field(default_factory=list)
    classes: List[ast.ClassDef] = field(default_factory=list)
    imports: List[Any] = field(default_factory=list)
    constants: List[Any] = field(default_factory=list)

    @classmethod
    def for_unit(cls, file_path: str, content: str) -> "PythonAnalysisContext":
        """Factory method to parse a source unit and build the analysis context."""
        context = cls(file_path=file_path, source_text=content)
        try:
            context.ast_tree = ast.parse(content, filename=file_path)
            for node in ast.walk(context.ast_tree):
                if isinstance(node, ast.Assign):
                    context.assignments.append(node)
                elif isinstance(node, ast.FunctionDef):
                    context.functions.append(node)
                elif isinstance(node, ast.ClassDef):
                    context.classes.append(node)
                elif isinstance(node, (ast.Import, ast.ImportFrom)):
                    context.imports.append(node)
                elif isinstance(node, ast.Constant):
                    context.constants.append(node)
        except SyntaxError as e:
            context.ast_tree = None
            context.parse_error = str(e)
        return context


@dataclass
class SourceUnit:
    """Represents a discoverable source file unit for analysis."""
    file_path: str
    content: str
    context: Optional[PythonAnalysisContext] = None

    def __post_init__(self) -> None:
        if self.content:
            self.context = PythonAnalysisContext.for_unit(self.file_path, self.content)


@dataclass
class Finding:
    """Standardized finding model representing a single code quality issue."""
    rule_id: str
    severity: Severity
    category: Category
    message: str
    file: Optional[str] = None
    line: Optional[int] = None
    column: Optional[int] = None
    confidence: float = 1.0
    recommendation: Optional[str] = None
    suppression_status: bool = False
    details: Optional[str] = None
    fixable: bool = False


@dataclass
class QualityConfiguration:
    """Configuration thresholds for the quality engine."""
    max_major_findings: int = 5
    minimum_quality_score: float = 80.0
    secret_threshold: float = 0.8
    duplicate_threshold: float = 0.85
    collabcore_reuse_threshold: float = 0.90
    max_loop_nesting: int = 3
    suppressed_rules: List[str] = field(default_factory=list)


def extract_collabware_header(content: str) -> Dict[str, Any]:
    """Extracts structured header fields and validation state from module text."""
    mandatory_fields = [
        "PRODUCT", "PROJECT", "MODULE", "MODULE_VERSION",
        "PURPOSE", "FUNCTION", "DEPENDENCIES", "STATUS",
        "LAST_MODIFIED", "COLLABORATION"
    ]
   
    extracted_fields = {}
    field_locations = {}
   
    lines = content.splitlines()
    for idx, line in enumerate(lines[:30], start=1):
        for field_name in mandatory_fields:
            pattern = re.compile(rf'^\s*(?:\*\s*)?{field_name}\s*:\s*(.*)$', re.IGNORECASE)
            match = pattern.match(line)
            if match:
                extracted_fields[field_name] = match.group(1).strip()
                field_locations[field_name] = idx

    exists = len(extracted_fields) > 0
    return {
        "exists": exists,
        "fields": extracted_fields,
        "field_locations": field_locations
    }


def check_structure_parse_failure(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule STRUCT-001: Detects Python source code parse/syntax failures."""
    findings = []
    if unit.context and unit.context.parse_error:
        findings.append(
            Finding(
                rule_id="STRUCT-001",
                severity=Severity.MAJOR,
                category=Category.STRUCTURE,
                message=f"Python source could not be parsed successfully: {unit.context.parse_error}",
                file=unit.file_path,
                recommendation="Fix Python syntax errors so structural analysis can proceed.",
                fixable=False
            )
        )
    return findings


def check_strict_documentation_header(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule DOC-001: Comprehensive validation of complete Collabware header requirements."""
    findings = []
    header_data = extract_collabware_header(unit.content)
   
    if not header_data["exists"]:
        findings.append(
            Finding(
                rule_id="DOC-001",
                severity=Severity.MAJOR,
                category=Category.DOCUMENTATION,
                message="Missing complete Collabware documentation header block.",
                file=unit.file_path,
                recommendation="Inject standard Collabware multi-line documentation header.",
                fixable=True
            )
        )
        return findings

    fields = header_data["fields"]
    mandatory_fields = [
        "PRODUCT", "PROJECT", "MODULE", "MODULE_VERSION",
        "PURPOSE", "FUNCTION", "DEPENDENCIES", "STATUS",
        "LAST_MODIFIED", "COLLABORATION"
    ]

    for field_name in mandatory_fields:
        if field_name not in fields:
            findings.append(
                Finding(
                    rule_id="DOC-001",
                    severity=Severity.MAJOR,
                    category=Category.DOCUMENTATION,
                    message=f"Missing required header field: {field_name}",
                    file=unit.file_path,
                    recommendation=f"Add missing {field_name} field to the documentation header.",
                    fixable=False
                )
            )
        elif not fields[field_name]:
            findings.append(
                Finding(
                    rule_id="DOC-001",
                    severity=Severity.MAJOR,
                    category=Category.DOCUMENTATION,
                    message=f"Required header field is empty: {field_name}",
                    file=unit.file_path,
                    recommendation=f"Provide a valid value for {field_name}.",
                    fixable=False
                )
            )

    if "PRODUCT" in fields and fields["PRODUCT"].lower() != "collabware":
        findings.append(
            Finding(
                rule_id="DOC-001",
                severity=Severity.MAJOR,
                category=Category.DOCUMENTATION,
                message=f"Invalid PRODUCT value: '{fields['PRODUCT']}'. Expected 'Collabware'.",
                file=unit.file_path,
                recommendation="Set PRODUCT to 'Collabware'.",
                fixable=False
            )
        )

    if "MODULE_VERSION" in fields:
        version_pattern = re.compile(r'^\d+\.\d+\.\d+(-[a-zA-Z0-9.]+)?$')
        if not version_pattern.match(fields["MODULE_VERSION"]):
            findings.append(
                Finding(
                    rule_id="DOC-001",
                    severity=Severity.MINOR,
                    category=Category.DOCUMENTATION,
                    message=f"Invalid MODULE_VERSION format: '{fields['MODULE_VERSION']}'. Expected semantic version (e.g. 2.4.2).",
                    file=unit.file_path,
                    recommendation="Use standard semantic versioning format.",
                    fixable=False
                )
            )

    if "LAST_MODIFIED" in fields:
        date_pattern = re.compile(r'^\d{4}-\d{2}-\d{2}$')
        if not date_pattern.match(fields["LAST_MODIFIED"]):
            findings.append(
                Finding(
                    rule_id="DOC-001",
                    severity=Severity.MINOR,
                    category=Category.DOCUMENTATION,
                    message=f"Invalid LAST_MODIFIED date format: '{fields['LAST_MODIFIED']}'. Expected YYYY-MM-DD.",
                    file=unit.file_path,
                    recommendation="Use ISO-8601 date format YYYY-MM-DD.",
                    fixable=False
                )
            )

    if "COLLABORATION" in fields:
        collab_text = fields["COLLABORATION"]
        required_names = ["Andrew", "Claude", "ChatGPT", "Gemini"]
        found_names = [name for name in required_names if name.lower() in collab_text.lower()]
        if not found_names:
            findings.append(
                Finding(
                    rule_id="CRD-001",
                    severity=Severity.MAJOR,
                    category=Category.DOCUMENTATION,
                    message="Collaboration field does not acknowledge standard core contributors.",
                    file=unit.file_path,
                    recommendation="Include standard contributor acknowledgements in COLLABORATION.",
                    fixable=False
                )
            )

    return findings


def check_hardcoded_secrets(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule SEC-001: Heuristic check for potential hardcoded credentials or API tokens."""
    findings = []
    secret_patterns = [
        re.compile(r'(password|passwd|pwd|secret|api_key|access_token|private_key|token)\s*=\s*[\'"][^\'"]+[\'"]', re.IGNORECASE)
    ]
   
    for line_num, line in enumerate(unit.content.splitlines(), start=1):
        for pattern in secret_patterns:
            if pattern.search(line):
                findings.append(
                    Finding(
                        rule_id="SEC-001",
                        severity=Severity.CRITICAL,
                        category=Category.SECURITY,
                        message="Potential hard-coded secret detected.",
                        file=unit.file_path,
                        line=line_num,
                        confidence=0.85,
                        recommendation="Move sensitive credentials to environment variables or secret vaults.",
                        fixable=False
                    )
                )
    return findings


def check_hardcoded_configuration(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule CONFIG-001: AST-assisted and regex fallback check for hardcoded network endpoints, IPs, paths, and ports."""
    findings = []
    ip_pattern = re.compile(r'\b(?:\d{1,3}\.){3}\d{1,3}\b')
    url_pattern = re.compile(r'https?://[^\s\'"]+')
    path_pattern = re.compile(r'([A-Za-z]:\\[\\\w._-]+|/(?:opt|var|etc|home|usr|bin|root)/[\\\w._-/]+)')
    port_pattern = re.compile(r'\b\w*port\w*\s*[:=]\s*(\d{2,5})\b', re.IGNORECASE)

    for line_num, line in enumerate(unit.content.splitlines(), start=1):
        stripped = line.strip()
        if stripped.startswith('#') or stripped.startswith('"""') or stripped.startswith("'''"):
            continue

        if ip_pattern.search(line) and "127.0.0.1" not in line and "0.0.0.0" not in line:
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MAJOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded IPv4 address detected.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Externalize IP addresses into configuration files or environment variables.",
                    fixable=False
                )
            )
        elif url_pattern.search(line) and "localhost" not in line and "127.0.0.1" not in line:
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MAJOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded service URL endpoint detected.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Externalize service endpoints into configuration properties.",
                    fixable=False
                )
            )
        elif path_pattern.search(line):
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MINOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded absolute filesystem path detected.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Use relative paths or configuration-driven directory roots.",
                    fixable=False
                )
            )
        elif port_pattern.search(line):
            findings.append(
                Finding(
                    rule_id="CONFIG-001",
                    severity=Severity.MINOR,
                    category=Category.CONFIGURATION,
                    message="Hard-coded service port assignment detected.",
                    file=unit.file_path,
                    line=line_num,
                    recommendation="Externalize port configurations into environment variables.",
                    fixable=False
                )
            )
           
    return findings


def check_magic_numbers(unit: SourceUnit, config: QualityConfiguration) -> List[Finding]:
    """Rule CONFIG-002: Detects magic numbers, ensuring module-level ALL_CAPS constants are correctly scoped."""
    findings = []
    if not unit.context or not unit.context.ast_tree:
        return findings

    allowlist = {0, 1, -1}
    module_constants = set()
    for node in unit.context.ast_tree.body:
        if isinstance(node, ast.Assign):
            for target in node.targets:
                if isinstance(target, ast.Name) and target.id.isupper():
                    if isinstance(node.value, ast.Constant):
                        module_constants.add(node.value.value)

    for node in ast.walk(unit.context.ast_tree):
        if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
            val = node.value
            if val in allowlist or val in module_constants:
                continue
            line_no = getattr(node, 'lineno', 0)
            findings.append(
                Finding(
                    rule_id="CONFIG-002",
                    severity=Severity.MINOR,
                    category=Category.CONFIGURATION,
                    message=f"Potential magic number detected: {val}.",
                    file=unit.file_path,
                    line=line_no,
                    recommendation="Define magic numbers as named module-level constants.",
                    fixable=False
                )
            )

    return findings


class RuleRegistry:
    def __init__(self) -> None:
        self._rules: Dict[str, Callable[[SourceUnit, QualityConfiguration], List[Finding]]] = {}
        self.register_rule("STRUCT-001", check_structure_parse_failure)
        self.register_rule("DOC-001", check_strict_documentation_header)
        self.register_rule("SEC-001", check_hardcoded_secrets)
        self.register_rule("CONFIG-001", check_hardcoded_configuration)
        self.register_rule("CONFIG-002", check_magic_numbers)

    def register_rule(
        self,
        rule_id: str,
        rule_func: Callable[[SourceUnit, QualityConfiguration], List[Finding]]
    ) -> None:
        self._rules[rule_id] = rule_func

    def get_rules(self) -> Dict[str, Callable[[SourceUnit, QualityConfiguration], List[Finding]]]:
        return self._rules


class QualityGate:
    @staticmethod
    def evaluate(findings: List[Finding], configuration: QualityConfiguration) -> GateStatus:
        active_findings = [f for f in findings if not f.suppression_status]

        for finding in active_findings:
            if finding.severity in (Severity.BLOCKER, Severity.CRITICAL):
                return GateStatus.FAIL

        major_count = sum(1 for f in active_findings if f.severity == Severity.MAJOR)
        if major_count > configuration.max_major_findings:
            return GateStatus.FAIL

        score = QualityGate.calculate_score(active_findings)
        if score < configuration.minimum_quality_score:
            return GateStatus.FAIL

        return GateStatus.PASS

    @staticmethod
    def calculate_score(findings: List[Finding]) -> float:
        weights = {
            Severity.BLOCKER: 25.0,
            Severity.CRITICAL: 15.0,
            Severity.MAJOR: 5.0,
            Severity.MINOR: 1.0,
            Severity.INFORMATION: 0.0,
        }
        penalty = sum(weights.get(f.severity, 0.0) for f in findings)
        score = max(0.0, 100.0 - penalty)
        return round(score, 2)


class RemediationEngine:
    @staticmethod
    def apply_fixes(source_unit: SourceUnit, findings: List[Finding]) -> bool:
        file_path = Path(source_unit.file_path)
        content = source_unit.content
        modified = False

        for finding in findings:
            if finding.rule_id == "DOC-001" and finding.fixable:
                module_name = file_path.stem
                current_date = datetime.now(timezone.utc).strftime("%Y-%m-%d")
                header = f'''"""
PRODUCT: Collabware
PROJECT: Collabware Code Quality Program
MODULE: {module_name}
MODULE_VERSION: 2.4.2
PURPOSE: Automatically generated or updated strict Collabware documentation header.
FUNCTION: Standardized structural compliance block.
DEPENDENCIES: None
STATUS: ACTIVE
LAST_MODIFIED: {current_date}
COLLABORATION: Andrew.human, Gemini AI/API, Claude AI/API, ChatGPT AI/API
"""

'''
                lines = content.splitlines(keepends=True)
                insert_index = 0
                for idx, line in enumerate(lines[:3]):
                    if line.startswith("#!") or "coding" in line:
                        insert_index = idx + 1

                lines.insert(insert_index, header)
                content = "".join(lines)
                modified = True

        if modified:
            backup_path = file_path.with_name(file_path.name + ".bak")
            if backup_path.exists():
                counter = 2
                while True:
                    alt_backup = file_path.with_name(f"{file_path.name}.bak{counter}")
                    if not alt_backup.exists():
                        shutil.copy2(backup_path, alt_backup)
                        break
                    counter += 1

            shutil.copy2(file_path, backup_path)
            file_path.write_text(content, encoding="utf-8")
            return True

        return False


class ReportGenerator:
    @staticmethod
    def generate_json_report(
        project_name: str,
        findings: List[Finding],
        gate_status: GateStatus,
        score: float,
        fixes_applied: int = 0
    ) -> str:
        summary = {
            "blocker": sum(1 for f in findings if f.severity == Severity.BLOCKER),
            "critical": sum(1 for f in findings if f.severity == Severity.CRITICAL),
            "major": sum(1 for f in findings if f.severity == Severity.MAJOR),
            "minor": sum(1 for f in findings if f.severity == Severity.MINOR),
            "information": sum(1 for f in findings if f.severity == Severity.INFORMATION),
            "fixes_applied": fixes_applied
        }

        report_data = {
            "product": "Collabware",
            "project": project_name,
            "analysis_timestamp": datetime.now(timezone.utc).isoformat(),
            "quality_score": score,
            "quality_gate": gate_status.value,
            "summary": summary,
            "findings": [
                {
                    "rule_id": f.rule_id,
                    "severity": f.severity.value,
                    "category": f.category.value,
                    "file": f.file,
                    "line": f.line,
                    "message": f.message,
                    "confidence": f.confidence,
                    "recommendation": f.recommendation,
                    "fixable": f.fixable
                }
                for f in findings
            ],
        }
        return json.dumps(report_data, indent=4)


class QualityEngine:
    def __init__(self, registry: RuleRegistry) -> None:
        self.registry = registry

    @staticmethod
    def discover_source_units(target_directory: str) -> List[SourceUnit]:
        source_units: List[SourceUnit] = []
        directory_path = Path(target_directory)

        if not directory_path.exists() or not directory_path.is_dir():
            return source_units

        for file_path in directory_path.rglob("*.py"):
            if ".bak" in file_path.name:
                continue
            try:
                content = file_path.read_text(encoding="utf-8")
                source_units.append(SourceUnit(file_path=str(file_path), content=content))
            except (IOError, OSError):
                continue

        return source_units

    def run_check(
        self,
        project_name: str,
        configuration: QualityConfiguration,
        target_directory: str,
        auto_fix: bool = False
    ) -> str:
        source_units = self.discover_source_units(target_directory)
        all_findings: List[Finding] = []
        fixes_count = 0

        rules = self.registry.get_rules()
        for unit in source_units:
            unit_findings: List[Finding] = []
           
            for _, rule_func in rules.items():
                findings = rule_func(unit, configuration)
                for finding in findings:
                    if not finding.file:
                        finding.file = unit.file_path
                unit_findings.extend(findings)

            if auto_fix and unit_findings:
                fixed = RemediationEngine.apply_fixes(unit, unit_findings)
                if fixed:
                    fixes_count += sum(1 for f in unit_findings if f.fixable)
                    updated_content = Path(unit.file_path).read_text(encoding="utf-8")
                    unit.content = updated_content
                    unit.context = PythonAnalysisContext.for_unit(unit.file_path, updated_content)
                    unit_findings = []
                    for _, rule_func in rules.items():
                        new_findings = rule_func(unit, configuration)
                        for finding in new_findings:
                            if not finding.file:
                                finding.file = unit.file_path
                        unit_findings.extend(new_findings)

            all_findings.extend(unit_findings)

        for finding in all_findings:
            if finding.rule_id in configuration.suppressed_rules:
                finding.suppression_status = True

        active_findings = [f for f in all_findings if not f.suppression_status]
        score = QualityGate.calculate_score(active_findings)
        gate_status = QualityGate.evaluate(all_findings, configuration)

        return ReportGenerator.generate_json_report(project_name, all_findings, gate_status, score, fixes_count)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="CollabQualityEnforcer v2.4.2 Static Analysis & Remediation Engine")
    parser.add_argument("--dir", required=True, help="Target directory containing Python source files")
    parser.add_argument("--project", default="Collabware Project", help="Project name for the report")
    parser.add_argument("--min-score", type=float, default=80.0, help="Minimum acceptable quality score")
    parser.add_argument("--max-majors", type=int, default=5, help="Maximum allowed major findings")
    parser.add_argument("--fix", action="store_true", help="Create rotated .bak backups and inject full standard headers")

    args = parser.parse_args()

    registry = RuleRegistry()
    engine = QualityEngine(registry)
    config = QualityConfiguration(
        max_major_findings=args.max_majors,
        minimum_quality_score=args.min_score
    )

    report_json = engine.run_check(args.project, config, args.dir, auto_fix=args.fix)
    print(report_json)

CollabQualityEnforcer 2.4 Roadmap

Document: CollabQualityEnforcer_2.4_Roadmap.md  Product: Collabware  Project: Collabware Code Quality Program  Target Module: CollabQualityEnforcer.py  Current Version: 2.3.0  Target Version: 2.4.0  Status: IMPLEMENTATION SPECIFICATION 

1. Audience and Instruction

This document is addressed specifically to the programming AIs responsible for implementing and reviewing CollabQualityEnforcer 2.4.

Primary implementation and review participants:

  • Andrew.human — Architect and QA Lead
  • Claude AI/API — Lead Programmer and Code Reviewer
  • Gemini AI/API — Programmer and Independent Code Reviewer
  • ChatGPT AI/API — Systems Analyst and Independent Reviewer

All contributors are subject to the same quality standards enforced by CollabQualityEnforcer.

No contributor, human or AI, is exempt from quality analysis.

The origin of code must not affect:

  • Severity
  • Quality score
  • Quality gate result
  • Rule application
  • Review requirements

2. Current State

CollabQualityEnforcer 2.3.0 provides a functioning quality enforcement framework.

The existing architecture includes:

Source Discovery
       |
       v
Rule Registry
       |
       v
Independent Quality Rules
       |
       v
Finding Model
       |
       v
Suppression Processing
       |
       v
Quality Score
       |
       v
Quality Gate
       |
       v
Human Report + JSON Report

The current implementation also provides:

  • Python source discovery
  • Dynamic rule registration
  • Standardized findings
  • Severity levels
  • Categories
  • Confidence values
  • Recommendations
  • Suppression support
  • Quality scoring
  • Quality gate evaluation
  • Human-readable reporting
  • JSON reporting
  • Safe auto-remediation
  • Rotating backups

Existing backup behaviour:

file.py
file.py.bak
file.py.bak2
file.py.bak3
...

Existing implemented rules:

DOC-001
Collabware documentation header detection

SEC-001
Potential hard-coded secret detection

CRD-001
Collaboration credit validation

3. Primary Rule for Version 2.4

DO NOT REWRITE COLLABQUALITYENFORCER

CollabQualityEnforcer 2.3.0 already has the required core framework.

Version 2.4 must extend the existing architecture.

Do not replace the existing:

  • Finding model
  • QualityConfiguration
  • RuleRegistry
  • QualityGate
  • RemediationEngine
  • ReportGenerator
  • HumanReportPrinter
  • QualityEngine

unless modification is necessary to support the requirements in this specification.

The preferred design principle is:

Extend the framework. Do not redesign the framework.

4. Version 2.4 Objectives

Version 2.4 has three primary objectives.

Objective A — Complete Documentation Validation

Upgrade DOC-001.

The current implementation only verifies the presence of:

PRODUCT: Collabware

This is insufficient.

Version 2.4 must validate the complete Collabware documentation header.

Objective B — Add Hard-Coded Configuration Detection

The current SEC-001 rule detects only a limited class of hard-coded secrets.

Version 2.4 must distinguish between:

SECRET
ENVIRONMENT-SPECIFIC VALUE
CONFIGURATION CANDIDATE
MAGIC NUMBER
LEGITIMATE CONSTANT

Not all literals are defects.

Objective C — Introduce Reusable Python AST Analysis

Version 2.4 must introduce a reusable Python AST analysis layer.

The AST layer is intended to become a foundation for later versions.

It should support future analysis of:

  • Functions
  • Classes
  • Imports
  • Dependencies
  • Assignments
  • Constants
  • Loops
  • Calls
  • Expressions

Do not implement the entire future rule set in 2.4.

Build the AST foundation cleanly.

5. Required Architecture

The target architecture after Version 2.4 should resemble:

CollabQualityEnforcer
        |
        +-- Source Discovery
        |
        +-- Language Analysis
        |        |
        |        +-- Python AST Adapter
        |
        +-- Rule Registry
        |        |
        |        +-- Documentation Rules
        |        |
        |        +-- Security Rules
        |        |
        |        +-- Configuration Rules
        |
        +-- Finding Manager
        |
        +-- Remediation Engine
        |
        +-- Quality Gate
        |
        +-- Report Generator

The AST infrastructure should be reusable by future rule modules.

Do not embed AST parsing logic independently inside every rule.

6. DOC-001 — Complete Documentation Header Validation

Current Problem

The existing rule only verifies that a source file contains:

PRODUCT: Collabware

This permits incomplete headers to pass.

Required Header Fields

Version 2.4 must validate:

PRODUCT
PROJECT
MODULE
MODULE_VERSION
PURPOSE
FUNCTION
DEPENDENCIES
STATUS
LAST_MODIFIED
COLLABORATION

Required Rule Behaviour

The rule must:

  1. Detect the Collabware documentation header.
  2. Identify all header fields.
  3. Detect missing required fields.
  4. Detect empty required fields.
  5. Validate selected fields.
  6. Report findings independently where practical.
  7. Avoid unnecessary duplicate findings.

PRODUCT Validation

Expected value:

Collabware

Failure should generate:

Rule: DOC-001
Severity: MAJOR
Category: DOCUMENTATION

PROJECT Validation

The field must exist and must not be empty.

Example:

PROJECT: Collabware Code Quality Program

MODULE Validation

The field must exist and must not be empty.

Where practical, the system may compare the declared module name with the source filename.

Example:

MODULE: collabware_quality_core

A filename mismatch should initially be reported as:

Severity: MINOR

Do not automatically change the module name unless explicitly supported by safe remediation rules.

MODULE_VERSION Validation

The field must exist and must not be empty.

Version 2.4 should support basic semantic version format validation:

MAJOR.MINOR.PATCH

Example:

2.4.0

Invalid version format should produce a documentation finding.

PURPOSE Validation

The field must:

  • Exist
  • Not be empty
  • Contain meaningful descriptive text

Do not attempt excessive semantic judgement in Version 2.4.

FUNCTION Validation

The field must:

  • Exist
  • Not be empty

The purpose is to describe the primary function or responsibility of the module.

DEPENDENCIES Validation

The field must exist.

Accepted initial values may include:

None

or a dependency list.

Actual dependency comparison is not required for Version 2.4 unless implementation can be cleanly achieved using the AST infrastructure.

Dependency comparison should not delay Version 2.4.

STATUS Validation

The field must exist and must not be empty.

Recommended values:

DEVELOPMENT
TESTING
ACTIVE
DEPRECATED
RETIRED

Unknown values may initially generate:

Severity: INFORMATION

rather than failure.

LAST_MODIFIED Validation

The field must exist.

The expected date format is:

YYYY-MM-DD

Example:

2026-08-31

Invalid date format should generate a documentation finding.

Do not attempt to verify whether the date accurately corresponds to source control history in Version 2.4.

COLLABORATION Validation

The existing CRD-001 rule must remain compatible.

The documentation rule should verify that the field exists.

CRD-001 should continue to verify that the collaboration line includes:

Andrew
Claude
ChatGPT
Gemini

Do not require an exact literal match.

Reasonable variations in role names, API naming, version notation, and punctuation must be tolerated.

7. Documentation Header Parsing

Do not create a large collection of unrelated regular expressions scattered across rules.

Create a reusable header extraction mechanism.

Conceptually:

Source File
     |
     v
Locate Module Documentation Block
     |
     v
Extract Header Fields
     |
     v
{
    PRODUCT: ...,
    PROJECT: ...,
    MODULE: ...,
    ...
}
     |
     v
Documentation Rules

Recommended conceptual API:

header = extract_collabware_header(source_unit)

The returned structure should support:

header.exists
header.fields
header.field_locations
header.raw_content

Example conceptual structure:

{
    "exists": True,
    "fields": {
        "PRODUCT": "Collabware",
        "PROJECT": "Collabware Code Quality Program",
        "MODULE": "collabware_quality_core",
        "MODULE_VERSION": "2.4.0",
        "PURPOSE": "...",
        "FUNCTION": "...",
        "DEPENDENCIES": "...",
        "STATUS": "ACTIVE",
        "LAST_MODIFIED": "2026-08-31",
        "COLLABORATION": "..."
    },
    "field_locations": {
        "PRODUCT": 7,
        "PROJECT": 8
    }
}

The actual implementation may differ.

The objective is reusable structured extraction.

8. CONFIG Rule Family

Version 2.4 must introduce a configuration analysis category.

Recommended category:

CONFIGURATION

The new rules should distinguish between:

Hard-coded secret
        |
        v
SECURITY finding

Hard-coded environment value
        |
        v
CONFIGURATION finding

Hard-coded configuration candidate
        |
        v
CONFIGURATION finding

Magic number
        |
        v
CONFIGURATION or STRUCTURE finding

Legitimate constant
        |
        v
No finding

9. Hard-Coded Environment Values

The system should identify likely environment-specific values such as:

  • URLs
  • Hostnames
  • IPv4 addresses
  • Ports
  • Absolute filesystem paths
  • Database hostnames
  • Service endpoints

Examples:

DATABASE_HOST = "192.168.1.10"

API_URL = "https://api.example.com"

SERVICE_PORT = 8080

DATA_PATH = "/opt/collabware/data"

These should generally generate:

Severity: MINOR or MAJOR
Category: CONFIGURATION

Severity should depend on confidence and context.

Do not automatically assume every URL or path is wrong.

Some values are legitimately fixed constants.

10. Magic Number Detection

Version 2.4 should implement an initial conservative magic-number rule.

Potential candidates:

if retries > 7:
timeout = 600
for index in range(42):

Do not flag universally accepted values such as:

0
1
-1

The initial implementation must be conservative.

False positives are worse than failing to identify every possible magic number.

Recommended rule:

CONFIG-002
Potential magic number or configuration constant

Recommended initial severity:

MINOR

11. AST Infrastructure

Purpose

Python AST analysis must become the reusable analysis foundation.

Conceptually:

Python Source
     |
     v
ast.parse()
     |
     +----------------+
     |                |
     v                v
Parse Success     Parse Failure
     |                |
     v                v
Python AST      STRUCT finding
     |
     v
Reusable AST Context

Parse Failure

If Python source cannot be parsed:

Rule: STRUCT-001
Category: STRUCTURE
Severity: MAJOR
Message:
Python source could not be parsed successfully.

The finding should include parser error information where safe and useful.

Do not expose unnecessary internal tracebacks.

AST Context

Recommended conceptual structure:

class PythonAnalysisContext:

    source_unit
    source_text
    ast_tree

    assignments
    functions
    classes
    imports
    constants

The exact design is flexible.

The objective is to parse once and reuse the result.

12. Rule Execution Context

The existing rule signature is approximately:

rule(unit, config)

Version 2.4 may evolve this if necessary.

A preferred future-oriented approach is:

rule(unit, analysis_context, config)

However:

Do not introduce unnecessary breaking changes.

If modifying the rule interface would require significant rewrites, introduce a context mechanism compatible with existing rules.

For example:

analysis_context = AnalysisContext.for_unit(unit)

The implementation must ensure that AST parsing is not unnecessarily repeated for every rule.

13. Initial AST-Assisted Configuration Analysis

The new configuration rules should prefer AST analysis over regex-only analysis where possible.

For example:

API_URL = "https://example.com"

should be understood as:

Assignment
    Variable: API_URL
    Value: String Constant

rather than merely matching text.

This should allow future rules to consider:

  • Variable name
  • Value type
  • Scope
  • Assignment context

14. Rule IDs

The following IDs are recommended for Version 2.4.

DOC-001
Complete Collabware documentation header validation

CRD-001
Collaboration credit validation

SEC-001
Potential hard-coded secret detection

CONFIG-001
Potential hard-coded environment-specific configuration

CONFIG-002
Potential magic number or configuration constant

STRUCT-001
Python source could not be parsed

Do not renumber existing rules.

Existing rule IDs are stable identifiers.

15. Remediation Requirements

Auto-remediation must remain conservative.

DOC-001

The existing header injection/remediation behaviour may be extended.

If a header is missing, automatic insertion is permitted.

If individual fields are missing, automatic insertion may be permitted only when a safe value is known.

Examples of potentially safe defaults:

PRODUCT: Collabware
MODULE: <derived from filename>
MODULE_VERSION: 1.0.0
STATUS: ACTIVE
LAST_MODIFIED: <current UTC date>
COLLABORATION: <standard team credit>

Examples that should not be guessed without explicit instruction:

PROJECT
PURPOSE
FUNCTION
DEPENDENCIES

Do not automatically invent meaningful project metadata.

CONFIG Rules

Version 2.4 should not automatically rewrite hard-coded configuration values.

Find and report them.

Do not attempt automatic configuration refactoring in Version 2.4.

Secrets

Never:

  • Print the full secret
  • Include the full secret in JSON reports
  • Include the full secret in recommendations
  • Copy secrets into logs

Mask sensitive values.

16. Quality Gate

The existing quality gate should remain intact.

The system should continue to fail when active findings include:

BLOCKER
CRITICAL

Major findings remain governed by:

max_major_findings

Quality score remains governed by:

minimum_quality_score

Do not change scoring weights without explicit approval.

17. Reporting Requirements

All new findings must integrate with both existing report formats.

Human Report

The live report must show:

Processing: module.py

  WARNING MAJOR [DOC-001]
  Missing required field: PURPOSE
      -> Add a concise description of the module's purpose.

  MINOR [CONFIG-001]
  Possible hard-coded environment-specific URL.
      -> Consider moving this value to project configuration.

JSON Report

New rules must automatically appear in the existing JSON finding structure.

Example:

{
    "rule_id": "CONFIG-001",
    "severity": "MINOR",
    "category": "CONFIGURATION",
    "file": "module.py",
    "line": 42,
    "message": "Potential hard-coded environment-specific configuration.",
    "confidence": 0.87,
    "recommendation": "Consider moving this value to configuration.",
    "fixable": false
}

18. Testing Requirements

Version 2.4 must include tests.

At minimum, test the following.

Documentation Header Tests

Valid complete header

Expected:

No DOC-001 findings

Missing header

Expected:

DOC-001 finding

Missing individual field

Expected:

Documentation finding identifying the missing field

Empty field

Expected:

Documentation finding

Invalid module version

Expected:

Documentation finding

Invalid date format

Expected:

Documentation finding

AST Tests

Valid Python

Expected:

Successful AST analysis

Invalid Python

Expected:

STRUCT-001 finding

Configuration Tests

Test:

URL
IPv4 address
Hostname
Absolute path
Port
Magic number
0
1
-1

The accepted constants:

0
1
-1

should not generate magic-number findings by default.

Regression Tests

The following existing behaviour must continue to work:

  • Rule registration
  • Rule suppression
  • Quality scoring
  • Quality gate
  • JSON reporting
  • Human reporting
  • Header auto-fix
  • Collaboration credit auto-fix
  • Backup rotation

19. Non-Goals for Version 2.4

The following are explicitly outside the required scope of Version 2.4.

Do not delay Version 2.4 attempting to implement them.

Duplicate code detection

Near-duplicate code detection

CollabCore reuse matching

Semantic similarity analysis

Loop termination analysis

Infinite-loop proof

Cyclomatic complexity analysis

Deep structural analysis

SQL injection analysis

Command injection analysis

Path traversal analysis

Unsafe deserialisation analysis

Authentication analysis

Authorisation analysis

Automatic configuration refactoring

Multi-language parsing

These are planned future rule families.

20. Version 2.4 Acceptance Criteria

Version 2.4 is complete when all of the following are true.

Architecture

  • Existing quality engine remains intact.
  • Existing public functionality remains operational.
  • AST parsing is reusable.
  • AST parsing is not unnecessarily repeated by every rule.

Documentation

  • Complete required header fields are validated.
  • Missing fields are identified.
  • Empty fields are identified.
  • Module version format is validated.
  • Last-modified date format is validated.
  • Existing collaboration validation remains operational.

Configuration

  • Hard-coded URLs can be detected.
  • Hard-coded IP addresses can be detected.
  • Hard-coded absolute paths can be detected.
  • Hard-coded ports can be detected.
  • Initial magic-number detection exists.
  • Common legitimate constants do not generate findings by default.

Security

  • Existing secret detection remains operational.
  • Secrets are not exposed in reports.

Reporting

  • New findings appear in human reports.
  • New findings appear in JSON reports.
  • Suppression continues to work.

Remediation

  • Existing safe remediation remains operational.
  • Existing backups remain protected.
  • Unsafe automatic changes are not introduced.

Testing

  • New functionality has automated tests.
  • Existing functionality has regression coverage.
  • All tests pass.

21. Collaboration Workflow

Gemini

Primary responsibilities:

  • Implement Version 2.4 requirements.
  • Preserve existing architecture.
  • Add tests.
  • Avoid unnecessary rewrites.
  • Clearly document implementation decisions.

Gemini must not assume existing code is correct merely because it already exists.

Existing code remains subject to review.

Claude

Primary responsibilities:

  • Independently review the Version 2.4 implementation.
  • Check compliance against this specification.
  • Look specifically for:

  - Regression risks   - False positives   - AST parsing duplication   - Unsafe remediation   - Documentation rule weaknesses   - Configuration rule overreach   - Security issues

Claude should report findings against the specification rather than rewriting the implementation unnecessarily.

ChatGPT

Primary responsibilities:

  • Independent architectural and specification review.
  • Review rule consistency.
  • Identify missing requirements.
  • Identify architectural drift.
  • Review future extensibility.

Andrew.human

Final authority for:

  • Architecture
  • Requirements
  • Acceptance
  • Quality exceptions
  • Release approval

22. Mandatory Implementation Principles

All implementation work must follow these principles.

Preserve the Framework

Do not replace working architecture unnecessarily.

Prefer Structured Analysis

Prefer AST analysis over regex when AST analysis is appropriate.

Regex remains acceptable for:

  • Header extraction
  • Simple textual metadata
  • Certain secret signatures

Do not force AST analysis where it adds no value.

Be Conservative

The system must not generate enormous numbers of low-value findings.

Prefer:

High-confidence findings

over:

Large volumes of speculative warnings

Evidence-Based Findings

Every finding should be explainable.

The report should make clear:

What was found
Where it was found
Why it was flagged
How confident the checker is
What should be considered next

Do Not Pretend Heuristics Are Proof

Use confidence and appropriate wording.

Examples:

Potential hard-coded configuration

rather than:

Incorrect configuration

unless the problem can actually be proven.

23. Final Instruction

Implement Version 2.4 as an incremental extension of CollabQualityEnforcer 2.3.0.

The implementation priority is:

1. Preserve existing architecture
        |
        v
2. Complete DOC-001
        |
        v
3. Build reusable header extraction
        |
        v
4. Introduce reusable Python AST analysis
        |
        v
5. Add CONFIG-001
        |
        v
6. Add CONFIG-002
        |
        v
7. Add STRUCT-001 parse failure handling
        |
        v
8. Add tests
        |
        v
9. Regression review
        |
        v
10. Independent review

Do not attempt to implement future versions prematurely.

Version 2.4 should establish a clean foundation for:

2.5
Structure and loop analysis

2.6
Duplicate detection and CollabCore reuse analysis

2.7
Expanded security analysis

The objective is not to produce the largest possible number of checks.

The objective is to produce a reliable, extensible, evidence-based code quality enforcement system that can progressively analyse all Collabware code — regardless of whether that code was written by Andrew.human, Claude, ChatGPT, Gemini, or any future contributor.

« Last Edit: Today at 03:47:30 AM by Chip »
friendly
0
funny
0
informative
0
agree
0
disagree
0
like
0
dislike
0
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
measure twice, cut once
Pages: 1   Go Up
« previous next »
Tags:
 

Related Topics

  Subject / Started by Replies Last post
21 Replies
63268 Views 		Last post July 10, 2016, 02:37:35 AM
by Chip
31 Replies
107815 Views 		Last post September 13, 2016, 02:40:41 PM
by DiacetylKineval
16 Replies
70653 Views 		Last post November 17, 2016, 10:30:25 AM
by DreamerOnTheRun
0 Replies
21574 Views 		Last post May 28, 2019, 07:28:18 AM
by Chip
0 Replies
726 Views 		Last post May 28, 2026, 08:55:40 PM
by smfadmin


dopetalk does not endorse any advertised product nor does it accept any liability for it's use or misuse




TERMS AND CONDITIONS

In no event will d&u or any person involved in creating, producing, or distributing site information be liable for any direct, indirect, incidental, punitive, special or consequential damages arising out of the use of or inability to use d&u. You agree to indemnify and hold harmless d&u, its domain founders, sponsors, maintainers, server administrators, volunteers and contributors from and against all liability, claims, damages, costs and expenses, including legal fees, that arise directly or indirectly from the use of any part of the d&u site.

TO USE THIS WEBSITE YOU MUST AGREE TO THE TERMS AND CONDITIONS ABOVE

Founded December 2014
SimplePortal 2.3.6 © 2008-2014, SimplePortal