kernel-cases: shells as a graded count, no SMLib verdict where it can't take the record, $OUT

- `"shells": N` in expected.json grades the total number of shells: a body with a sealed void
  holds two, and some joins keep several lumps in one body. Those cases (#3175, #3455, #3526,
  #3730) are graded on validity again instead of being marked not valid.
- When SMLib's record conversion says "not supported yet" (revolved surfaces, traced intersection
  curves), its check gives no verdict. That's shown but not graded, since it isn't a fault of the
  body. solidcraft_kernel::SmlibCheck carries the three outcomes.
- `$OUT` in a recipe is a scratch directory outside the repository, for files a case writes. The
  two STEP-export crash cases write there instead of /tmp.
- cargo xtask kernel-cases --results FILE keeps every result as JSON.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
bherbruckandClaude Opus 5.5 committed 2026-10-21 06:22:03 -04:00
1 parent 6dd104babb
commit 457e63acf3
12 files changed
+102 -52

No files matched your search

+7 -1
View File
@@ -26,6 +26,8 @@ struct Options {
area: Option<String>,
timeout: Duration,
jobs: usize,
/// Where to keep every case's full result as JSON.
results: Option<String>,
}
fn options(rest: &[&str]) -> Result<Options, String> {
@@ -38,7 +40,7 @@ fn options(rest: &[&str]) -> Result<Options, String> {
};
let secs = arg("--timeout").map(|t| t.parse::<u64>().map_err(|e| format!("--timeout: {e}"))).transpose()?.unwrap_or(DEFAULT_TIMEOUT_S);
let jobs = arg("--jobs").map(|j| j.parse::<usize>().map_err(|e| format!("--jobs: {e}"))).transpose()?.unwrap_or(4).clamp(1, 32);
Ok(Options { kernels, cli: arg("--cli"), area: arg("--area"), timeout: Duration::from_secs(secs), jobs })
Ok(Options { kernels, cli: arg("--cli"), area: arg("--area"), timeout: Duration::from_secs(secs), jobs, results: arg("--results") })
}
fn read(path: &Path) -> Value {
@@ -323,6 +325,10 @@ pub fn run(rest: &[&str]) -> Result<(), String> {
return Err(format!("no cases (recipe.json + expected.json) under {}", dir.display()));
}
let results = run_all(&o, &list);
if let Some(path) = &o.results {
let text = serde_json::to_string_pretty(&results).map_err(|e| e.to_string())?;
std::fs::write(path, text).map_err(|e| format!("{path}: {e}"))?;
}
println!("{}", write_report(&o, &dir, &list, &results)?);
Ok(())
}
+3 -2
View File
@@ -39,10 +39,11 @@ commands:
replay the reference tests (tests/oracle/*/recipe.json), compare with each
case's expected.json and write docs/oracle.md (--cli: a solidcraft-cli
already built, instead of cargo run --release)
kernel-cases [--kernel default|smlib|both] [--area AREA] [--cli BIN] [--timeout S] [--jobs N]
kernel-cases [--kernel default|smlib|both] [--area AREA] [--cli BIN] [--timeout S] [--jobs N] [--results FILE]
the kernel regression rig: run every case in tests/kernel-cases/<area>/<id>/ (one
process each, killed after --timeout, default 180 s) with each kernel asked for,
grade it against the case's own answer and write docs/kernel-cases.md
grade it against the case's own answer and write docs/kernel-cases.md (--results:
every result as JSON too)
action-matrix [--bless] [--results FILE] [--from DIR]
try every ordinary action on every face, edge, vertex and body of the design
corpus and compare with tests/action_matrix/baseline.json (--bless rewrites it;