Smart Linter Pipeline Reference

Script: routertl_core/smart_linter.py (backward-compat proxy at tools/project-manager/smart_linter.py) Shell Backend: tools/sh/linting.sh

1. Overview

The Smart Linter is a multi-phase HDL analysis pipeline that compiles and checks VHDL files using GHDL. It integrates with the Dependency Resolver for topological ordering and with the Project Manager for library configuration.

2. Three-Phase Flow

sequenceDiagram
    participant SL as smart_linter.py
    participant DR as dependency_resolver.py
    participant LS as linting.sh
    participant Cache as .routertl_cache/libraries.json

    Note over SL: Phase 1 — PRECOMPILE
    SL->>DR: --sort-pkgs (absolute paths)
    SL->>DR: --discover-libs (auto-detect libraries)
    SL->>Cache: Load manual library overrides
    SL->>SL: Merge auto + manual, exclude from pkg list
    SL->>LS: PRECOMPILE (xpm, utils, filtered pkgs)

    Note over SL: Phase 1.5 — COMPILE_LIB
    SL->>DR: --top <entity> --list (per entity, full topo sort)
    SL->>LS: COMPILE_LIB (lib_name, topo-sorted files)

    Note over SL: Phase 2 — LINT
    SL->>DR: --forest or --top --list
    SL->>LS: LINT (unit files, -P search paths)

Phase 1 — PRECOMPILE

Compiles global libraries into persistent GHDL work directories:

• XPM files → sim/work/xpm/
• Utils files → sim/work/utils/
• Project packages → sim/work/work/

Custom library files are excluded from this phase because they use cross-references within their own library (e.g. use tich.system_config_pkg) which would fail when compiled into the default work library.

Phase 1.5 — COMPILE_LIB

Compiles each custom library into its own GHDL work directory (sim/work/<lib_name>/). Library membership is determined by:

1. Auto-discovery (default) — --discover-libs scans source for entity lib.X instantiations and use lib.pkg.all clauses
2. Manual overrides — libraries.json from project.yml extends or overrides auto-discovered entries

Files are topologically sorted using --top --list per entity to ensure correct compilation order (packages and entities with inter-dependencies are resolved, not just packages).

Output shows auto/manual breakdown:

📚 Compiling Custom Libraries...
  - tich (20 auto + 6 manual) ... PASS

Phase 2 — LINT

Resolves project hierarchies via --forest (auto mode) or --top --list (single-top mode). Each hierarchy's files are compiled into work with -P search paths pointing to all custom libraries compiled in Phase 1.5.

3. Path Normalization Contract

The dependency resolver emits absolute paths. The library cache (libraries.json) stores relative paths (relative to project root). The smart linter normalizes all cache paths to absolute via Path(f).resolve() before comparison.

This contract is critical for the PRECOMPILE exclusion set. Without normalization, the same file would appear as two different entries:

# --sort-pkgs output (absolute)
/home/user/project/hw/rtl/pkg/system_config_pkg.vhd

# libraries.json entry (relative)
hw/rtl/pkg/system_config_pkg.vhd

The linter resolves both to the same absolute path before building the exclusion set.

4. .routertl_cache/libraries.json

Format

{
    "tich": [
        "hw/rtl/pkg/arbiter_pkg.vhd",
        "hw/rtl/pkg/axi_pkg.vhd",
        "hw/rtl/pkg/system_config_pkg.vhd"
    ]
}

Keys are VHDL library names. Values are lists of file paths (relative to project root).

Lifecycle

1. Auto-discovered by smart_linter.py via --discover-libs (source code scanning, no configuration needed)
2. Merged with manual entries from libraries.json (advanced override for edge cases)
3. Merged with IP manifest library mappings (from ip_resolver.py)
4. Location: .routertl_cache/libraries.json (gitignored)

Library membership is fully automatic via --discover-libs. Manual libraries.json entries can be used as overrides for edge cases (e.g. generated files not in the source scan path).

Data Flow

VHDL source → dependency_resolver.py → auto_libs
  entity tich.X   --discover-libs        {tich: [a, b, c]}
  use tich.Y.all

project.yml → project_manager.py → libraries.json → manual_libs
  sources:        generate_mk_file()      {tich: [x, y]}    (optional)
    libraries:
      tich: [...]

auto_libs ∪ manual_libs → smart_linter.py → COMPILE_LIB
                           merge + topo-sort   ghdl -a --work=tich ...

5. linting.sh Modes

All modes write GHDL output to a temporary file, pipe it through ghdl_output_parser.py for formatted display, and exit with GHDL's return code.

GHDL Work Directories

sim/work/
├── xpm/          # XPM library artifacts
├── utils/        # Utility library artifacts
├── work/         # Project packages + lint artifacts
├── temp_lint/    # (created but unused in current flow)
└── <lib_name>/   # Custom library artifacts (one per library)

6. project.yml Libraries Section

sources:
  libraries:
    tich:
      - hw/rtl/pkg/arbiter_pkg.vhd
      - hw/rtl/pkg/axi_pkg.vhd
      - hw/rtl/pkg/system_config_pkg.vhd

Files listed under a library name are:

1. Serialized to libraries.json by project_manager.py
2. Excluded from generic PRECOMPILE by smart_linter.py
3. Compiled under the library name in Phase 1.5
4. Added to -P search paths for the LINT phase

7. Known Issues (Resolved)

Path Mismatch (fixed 9fcee52)

Before the fix, smart_linter.py compared --sort-pkgs absolute paths directly against libraries.json relative paths. Custom library files were never excluded from PRECOMPILE, causing cannot find resource library errors.

Arbitrary Compilation Order (fixed 9fcee52)

Before the fix, custom library files were compiled in their libraries.json insertion order, not dependency order. Files with internal cross-references (e.g. sniffer_pkg depending on system_config_pkg) could fail if compiled out of order.

GHDL Error Swallowing (fixed 0acf7a4)

ghdl_output_parser.py captured raw_errors via GHDL_ERROR_PATTERN but never rendered them. File-not-found errors showed as ✓ No issues found while the exit code was non-zero.

Entity Topological Sort (fixed 0acf7a4)

Phase 1.5 used --sort-pkgs (package-only) for ordering. Entity files with inter-dependencies (e.g. mux_cdc_er → event_synchronizer) retained project.yml insertion order. Now uses --top --list per entity for full dependency resolution.

Manual Library Listing Required (fixed 0acf7a4)

Consumers had to manually list every file in library configuration. Auto-discovery via --discover-libs now handles this automatically.

8. Pipeline Output and Exit Codes

The smart_linter.py script intentionally exits with 0 (success) even when linting errors occur, and writes its actual exit status to sim/lint.status. This is an architectural decision to prevent make from halting mid-pipeline and dropping test summaries.

When make linting executes the smart linter, halting the make process via a non-zero Python exit code cascades into abrupt CI pipeline halts.

To verify linter success, downstream integrations (like pre-commit hooks or CI runners) must explicitly read sim/lint.status instead of relying on the shell exit code ($?).

Correct integration pattern:

rm -f sim/lint.status
make linting

if [ -f "sim/lint.status" ]; then
    LINT_STATUS=$(cat sim/lint.status)
    if [ "$LINT_STATUS" != "0" ]; then
        echo "❌ Linting failed ($LINT_STATUS errors). Commit aborted."
        exit 1
    fi
else
    echo "❌ Linting skipped or did not generate status file."
    exit 1
fi

This pattern is strictly enforced in RouteRTL's global and project-level pre-commit hooks.

9. Gitignore-Aware File Discovery

The dependency resolver automatically excludes files matched by the repository's .gitignore rules. This prevents generated HDL outputs (synthesis wrappers, vendor IP stubs, code-gen artifacts) from polluting the linting pipeline.

How It Works

On initialization, DependencyResolver walks up from the first source path to locate .git/. When found, it reads .gitignore and compiles the patterns using the pathspec library (the same gitignore engine used by ruff and black). Each rglob() hit is checked against these patterns before parsing.

Graceful Fallback

When not inside a git repository (e.g. SDK installed via pip install with no .git/ directory), gitignore filtering is silently disabled and all files are discovered — matching the pre-v3.1 behavior.

Override

To include gitignored files in discovery, pass --no-respect-gitignore to the dependency resolver CLI:

python3 dependency_resolver.py --src src --forest --no-respect-gitignore

Or construct the resolver programmatically:

resolver = DependencyResolver([Path("src")], respect_gitignore=False)

The exclude_lint list in project.yml remains available as an orthogonal mechanism. Gitignore filtering removes files at the discovery layer (before dependency resolution), while exclude_lint operates at the linting layer (after resolution).