Files
solidcraft/skills/solidcraft-tests/SKILL.md
T
bherbruckandClaude Opus 5.5 3a504f5347 Agents: AGENTS.md becomes a router to focused skills, with a staleness check (Refs #6439) (#6444)
AGENTS.md drops from 16.8k to 3.7k characters: the hard rules, each naming the
check that enforces it, and a table of what to read for each kind of work.
The rest moves into eleven new skills; the five existing ones stay.

cargo xtask skills fails when AGENTS.md or a SKILL.md cites a path, an xtask
command or a skill that does not exist, when the router table and skills/
disagree, or when the router or a new skill grows past its size cap. It runs
in cargo xtask ci and the repository checks job.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 11:51:19 +00:00

3.0 KiB

name, description
name description
solidcraft-tests Use when writing, placing or running tests in SolidCraft: a regression test for a bug, an engine test for a command, a UI scenario, a reference (oracle) test, goldens, or when CI fails a test you did not touch.

Where tests go and how to run them

The long form is docs/book/src/dev/testing.md.

Placement

  • A regression test goes into the existing test file for its area (booleans, blends, shell, sweep, STEP, naming, ...), never a new file named after the bug (#3226).
  • Engine tests sit next to the code or in *_tests.rs files; they measure the result (volume, area, face/edge/vertex counts) against a value you can derive.
  • UI scenarios are JSON in crates/ui-egui/tests/scenarios/*.json; a new file is picked up automatically. Put {"pending": "why"} in a scenario that records a bug before it is fixed, and remove it in the fixing PR.
  • Reference tests are tests/oracle/<case>/recipe.json with an expected.json beside it, run by cargo xtask oracle. Expected values are worked out in closed form (tools/oracle-analytic/) or computed by FreeCAD run as a separate program (tools/freecad-oracle/). A case that fails on a known bug names its issue ("known_failure": "#123"). Never change an expected value to match SolidCraft's output, and never take one from a commercial CAD product.
  • Crash fixes: a regression test, started from the crash report's design.solidcraft and replay.json (skills/solidcraft-crash-safety/SKILL.md).

A failing test needs its fix

A failing test without its fix cannot merge: it turns main red for everyone. Include the fix in the same PR, or mark the test #[ignore = "fails until #123 is fixed"] and remove that line in the PR that fixes it.

Running

  • cargo xtask test builds with the ci profile and runs every test binary at once; arguments after -- go to each binary (cargo xtask test -- shell). Use the ci profile: it builds several times faster than --release, and debug assertions are off because truck's reject valid geometry. If a test fails only in a plain debug build, say so in the PR.
  • SOLIDCRAFT_TEST_JOBS=N bounds concurrent commands on a shared machine.
  • cargo xtask ci is the gate before a commit: fmt, assets, icons, licences, lists, layers, ui-containers, clippy -D warnings, tests and the wasm check.
  • Every test runs in its own process: do not rely on state left by another test, or on wall-clock timing.

Goldens, matrices and skipped tests

  • A geometry change that moves the robustness goldens re-records them in the same PR (ROBUSTNESS_GOLDENS_BLESS="why it changed", docs/robustness.md).
  • The action matrix is a ratchet: record a newly passing action with cargo xtask action-matrix --bless (docs/action-matrix.md).
  • Pull requests skip a few heavy tests, which run in the merge queue, nightly and on main. Check that validation after merging. The list is in .config/nextest.toml and the testing page.
  • Mutation testing finds behaviour the suites miss: docs/mutation-testing.md.