kernel-cases: shells as a graded count, no SMLib verdict where it can't take the record, $OUT
- `"shells": N` in expected.json grades the total number of shells: a body with a sealed void holds two, and some joins keep several lumps in one body. Those cases (#3175, #3455, #3526, #3730) are graded on validity again instead of being marked not valid. - When SMLib's record conversion says "not supported yet" (revolved surfaces, traced intersection curves), its check gives no verdict. That's shown but not graded, since it isn't a fault of the body. solidcraft_kernel::SmlibCheck carries the three outcomes. - `$OUT` in a recipe is a scratch directory outside the repository, for files a case writes. The two STEP-export crash cases write there instead of /tmp. - cargo xtask kernel-cases --results FILE keeps every result as JSON. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6dd104babb
commit
457e63acf3
12 files changed
+102
-52
No files matched your search
@@ -26,6 +26,8 @@ struct Options {
|
||||
area: Option<String>,
|
||||
timeout: Duration,
|
||||
jobs: usize,
|
||||
/// Where to keep every case's full result as JSON.
|
||||
results: Option<String>,
|
||||
}
|
||||
|
||||
fn options(rest: &[&str]) -> Result<Options, String> {
|
||||
@@ -38,7 +40,7 @@ fn options(rest: &[&str]) -> Result<Options, String> {
|
||||
};
|
||||
let secs = arg("--timeout").map(|t| t.parse::<u64>().map_err(|e| format!("--timeout: {e}"))).transpose()?.unwrap_or(DEFAULT_TIMEOUT_S);
|
||||
let jobs = arg("--jobs").map(|j| j.parse::<usize>().map_err(|e| format!("--jobs: {e}"))).transpose()?.unwrap_or(4).clamp(1, 32);
|
||||
Ok(Options { kernels, cli: arg("--cli"), area: arg("--area"), timeout: Duration::from_secs(secs), jobs })
|
||||
Ok(Options { kernels, cli: arg("--cli"), area: arg("--area"), timeout: Duration::from_secs(secs), jobs, results: arg("--results") })
|
||||
}
|
||||
|
||||
fn read(path: &Path) -> Value {
|
||||
@@ -323,6 +325,10 @@ pub fn run(rest: &[&str]) -> Result<(), String> {
|
||||
return Err(format!("no cases (recipe.json + expected.json) under {}", dir.display()));
|
||||
}
|
||||
let results = run_all(&o, &list);
|
||||
if let Some(path) = &o.results {
|
||||
let text = serde_json::to_string_pretty(&results).map_err(|e| e.to_string())?;
|
||||
std::fs::write(path, text).map_err(|e| format!("{path}: {e}"))?;
|
||||
}
|
||||
println!("{}", write_report(&o, &dir, &list, &results)?);
|
||||
Ok(())
|
||||
}
|
||||
+3
-2
@@ -39,10 +39,11 @@ commands:
|
||||
replay the reference tests (tests/oracle/*/recipe.json), compare with each
|
||||
case's expected.json and write docs/oracle.md (--cli: a solidcraft-cli
|
||||
already built, instead of cargo run --release)
|
||||
kernel-cases [--kernel default|smlib|both] [--area AREA] [--cli BIN] [--timeout S] [--jobs N]
|
||||
kernel-cases [--kernel default|smlib|both] [--area AREA] [--cli BIN] [--timeout S] [--jobs N] [--results FILE]
|
||||
the kernel regression rig: run every case in tests/kernel-cases/<area>/<id>/ (one
|
||||
process each, killed after --timeout, default 180 s) with each kernel asked for,
|
||||
grade it against the case's own answer and write docs/kernel-cases.md
|
||||
grade it against the case's own answer and write docs/kernel-cases.md (--results:
|
||||
every result as JSON too)
|
||||
action-matrix [--bless] [--results FILE] [--from DIR]
|
||||
try every ordinary action on every face, edge, vertex and body of the design
|
||||
corpus and compare with tests/action_matrix/baseline.json (--bless rewrites it;
|
||||
|
||||
Reference in new issue
Block a user