Assert Quality Reference

rr sim coverage-map flags a @requires("REQ-N")-tagged test as vacuous when the decorated function body contains zero substantive assertions. Trivial assertions (assert True, assert x == x, …) are stripped before counting — they pass the simulator but prove nothing about the DUT, so they don't earn the tag.

This page is the authoritative catalogue: what's flagged, what's not, and what a substantive assertion looks like.

Why this exists. The @requires(REQ-N) decorator is a promise that the decorated test proves the corresponding requirement. A test that imports cocotb, decorates itself, and then assert Trues technically passes — but silently lowers the coverage bar while the next agent or contributor files a clean report. The assert-quality guard is how the contract survives human (and AI) shortcuts.

Flagged as trivial (won't count toward coverage)

All of these evaluate truthy at compile time — the simulator can't fail them no matter what the DUT does, so they don't prove anything.

Patterns not flagged (bugs, not dodges)

Some patterns look trivial but are in fact broken assertions that coverage-map deliberately leaves alone — the simulator will fail them and surface the bug. Fix them in the test, don't work around the gate.

What a substantive assertion looks like

A substantive assertion compares a DUT observable against a hard-coded expected value. The contrast is the whole point: if the expected value is derived from the same inputs at runtime, the test is tautological and proves nothing.

# Substantive — compares dut.signal.value to a literal 42.
assert int(dut.out_value.value) == 42

# Substantive — compares dut.flag to a captured prior snapshot.
before = int(dut.counter.value)
await ClockCycles(dut.clk, 5)
after = int(dut.counter.value)
assert after - before == 5

# Substantive — asserts inertness of an output across a window.
for _ in range(100):
    await RisingEdge(dut.clk)
    assert dut.m_axis_tvalid.value == 0

# Substantive — pytest.approx for float DUTs.
assert float(dut.angle.value) == pytest.approx(0.785, rel=1e-3)

The rule of thumb from the feedback_adversarial_testing stance:

Setup → Drive → Advance clock → Assert observed == hard-coded expected — not a value derived from the same inputs at runtime. A test that can't fail isn't a test.

When coverage-map flags your test

[FAIL] demo_ip: covered=0/1, vacuous=1
  ∅ REQ-1: test_lying (no assertions — vacuous tag)

The ∅ marker means the tag exists but every assertion inside it was classified as trivial. To clear the gate:

1. Remove the trivial assertion(s).
2. Add a substantive one that compares a DUT observable to a hard-coded expected value.
3. Re-run rr sim coverage-map — the tag should now appear with a ✓.

If you genuinely believe your assertion is substantive and coverage-map disagrees, it's a classifier bug. File a backlog item with the exact snippet so the rule set can be tightened without regressing false positives on the cases above.

How the guard is layered

The guard has three layers of increasing sophistication. Layers 1 and 1.5 run on every rr sim coverage-map; layer 2 is a separate spike.

Layer 1.5 — signal-reference lint

Layer 1 stops assert True-style dodges, but it can't stop a test that has substantive assertions aimed at the wrong signals:

# REQ-5: "csr_slv_readdata matches the last value written"
@requires("REQ-5")
async def test_csr_readback(dut):
    # Substantive assertion — passes layer 1 cleanly.
    # But no reference to csr_slv_readdata anywhere in the body.
    assert int(dut.unrelated_flag.value) == 0

Layer 1.5 catches this by cross-checking two things:

1. REQ-side identifiers extracted from the desc string:
   • Snake_case tokens with at least one underscore (csr_slv_readdata, m_axis_tvalid, route_sel_video).
   • Backtick-wrapped tokens (`chipselect`, `ready`) — the explicit opt-in for single-word signal names that don't match the snake_case pattern.
2. Test-side references — every Attribute.attr reached from the @requires-tagged function body (same-file helpers chased per layer 1's RTL-P2.443). dut.csr_slv_readdata.value contributes both csr_slv_readdata and value.

If the REQ desc names at least one identifier and none of the tagged test's substantive-assert paths reference it, the REQ is classified unreferenced (marker ⊘). A desc that names zero identifiers (plain English only) can't be cross-checked and the lint stays silent — back-compat for pre-P3.194 requirements.yml files.

Limitations (document, don't work around)

• Driver-based tests. If your test's assertions go through a driver wrapper (driver.write() / driver.read()), the raw dut.<signal> access may never appear in the function body. Two escape hatches:
  • Wrap the signal name in backticks in the REQ desc and also name it in a comment or local variable in the test body (_ = dut.csr_slv_readdata.value # sampled by driver).
  • Capture the driver's state field into a local: sel = driver.state['route_sel_video'].
• Aliasing via getattr. getattr(dut, "route_sel_video") hides the signal name from the AST walker — the lint won't see it. Use direct attribute access whenever possible.
• Cross-module helpers. Bare-name same-file helpers are chased, but from .helpers import verify; verify(dut) is not. Inline the assertion or move the helper into the test file.

The lint is imperfect by design — it catches the lazy form without pretending to catch every adversarial evasion. When it fires spuriously, prefer a backtick hint in the desc over modifying the test to satisfy it.

Same-file helper functions

The classifier follows bare-name calls into same-file helpers (RTL-P2.443). A test whose body is _verify_mux(dut) still counts the asserts inside _verify_mux — there's no dodging the gate by moving substantive assertions into a helper.

@requires("REQ-1")
async def test_mux_routing(dut):
    # Looks assertion-free to a naive walker, but the classifier chases
    # _verify_channel and counts the real asserts inside.  Tag stands.
    await _verify_channel(dut, channel=0, expected=0xAA)
    await _verify_channel(dut, channel=1, expected=0xBB)


async def _verify_channel(dut, channel, expected):
    dut.sel.value = channel
    await RisingEdge(dut.clk)
    assert int(dut.m_axis_tdata.value) == expected

Cross-module calls (from .helpers import verify) are not chased — walking arbitrary imports is a different class of tool. Keep helper functions in the same file as the test that uses them, or inline the assertion at the call site.