-
test(hygiene): a guard against suites that are green because they run nothing · 70209824
Four suites in this workspace were found hollow in one week, each hollow in a different way, and every one of them had read as fine for months. The pattern is not a bug in any of them — it is that a test suite reports on the code and nothing reports on the suite. So: tools/tests/test_suite_hygiene.py walks every test_*.py by AST (no imports, so it works with or without any project venv) and asserts three things. * No file collects zero tests. A test_*.py with no collectable function is green and hollow. * No test requests a fixture nothing declares. `def test_x(suite)` makes pytest error on a missing fixture, and twelve such errors read as a config nit for months while twelve real audio cases went unrun. * No module-level sys.exit(). It fires during COLLECTION and takes the whole directory down instead of failing one test — which is how `pytest tools/tests/` once ran zero tests while 273 passed individually. It also guards itself: if the glob ever stops finding files, every rule above passes vacuously, so the file count has a floor of its own. Verified by mutation, three planted defects, one per rule — a hollow file, a `def test_thing(suite)`, and a trailing sys.exit(0). Each is caught by name and the tree comes back clean. Also splits tools/tests/test_setlist.py, whose 22 checks reported as one aggregate assertion that named the file and not the fault. check() now records each outcome and pytest replays them one named test per check; the aggregate stays as the belt to those braces, and the file still runs as a script. Every count is pinned, because the failure being guarded against is not a wrong answer but an empty one, and an empty one is invisible without a number to compare against. That guard earned itself immediately: I had counted 28 checks by eye and the truth was 22.PLN (Algolia) authored70209824
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| test-sc-watchdog.sh | Loading commit data... | |
| test_button_roles.py | Loading commit data... | |
| test_gig_log.py | Loading commit data... | |
| test_lcxl3_display.py | Loading commit data... | |
| test_lcxl_grid.py | Loading commit data... | |
| test_orphan_orbits.py | Loading commit data... | |
| test_setlist.py | Loading commit data... | |
| test_silent_eval.py | Loading commit data... | |
| test_suite_hygiene.py | Loading commit data... |