Files
solidcraft/.github/workflows/ci.yml
T
bherbruckandClaude Opus 5.5 9c411859bc CI: self-hosted test shards build in their warm directories, no 3 GB download (#2833)
On main and in the merge queue every test shard ran on the self-hosted machine but waited for
the GitHub-hosted Build tests job and then spent about 12.5 minutes downloading its 3.2 GB
archive (four shards at once over one connection). There, each shard now builds the tests in
the runner's warm build directory (the existing build-tests one) and runs its slice directly;
Build tests runs only where the shards run on GitHub's or the untrusted runners.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 20:35:03 -04:00

760 lines
40 KiB
YAML

# Fast checks always run; native builds run for code/CI changes.
# PRs test what they touch; main and nightly test everything. On a pull request the "Select
# checks" job maps the changed files to workspace packages (scripts/ci/affected.py), adds the
# packages that depend on them, and Clippy, the test archive and the test shards cover only
# those. Changes to the lockfile, root manifest, toolchain/cargo/nextest config, .github/,
# vendor/, tests/, scripts/ or any file no package owns select the whole workspace, as do
# pushes to main, the nightly run and Run workflow.
# Tests are built once and run in 4 shards. A few heavy tests (see .config/nextest.toml) are
# skipped on pull requests and run on pushes to main and nightly. The Windows jobs (D3D12 shaders
# and app startup) run on pull requests only when what they test changes (the GPU crate, the app,
# the viewport's GPU glue, the graphics dependencies, their scripts or this file); main and
# nightly always run them. Only main saves the build caches; pull requests restore them.
# The merge queue runs the full suite on exactly the commit it then puts on main, so a push to
# main that a passing merge-queue run already tested runs only the repository checks; a push
# that skipped the queue still runs everything. Pull requests run the reference tests when the
# command line (or anything it depends on) changes, and doc tests for the affected packages.
# Cross-compilation is opt-in through Run workflow; release builds remain in release.yml.
# Trusted runs use both pools: pushes to main, merge-queue groups and pull requests from a branch
# of this repository (people with write access; fork pull requests never run on this machine).
# There, Select checks and ci run on the self-hosted light runner (`self-hosted-light`, quick
# jobs, so they never wait behind a build), and the repository checks, Clippy, the doc tests,
# the reference tests and all four test shards on the self-hosted runners (`self-hosted-fast`, warm
# build directories via scripts/ci/warm-target.sh). Fork PRs' Linux jobs run on the isolated, ephemeral
# `self-hosted-untrusted` runners (no warm caches) when the PR number ends in 7, 8 or 9 (the rest of
# the fork PRs, and everything else, run on GitHub's runners, which have far more free slots).
# Where the test shards run on GitHub's runners, the test archive is built there too: the
# self-hosted machine has a newer glibc, so its binaries wouldn't start on ubuntu-24.04, while the
# archive runs there unchanged (same /home/runner/work checkout path). The self-hosted shards
# (main and the merge queue) skip the archive and build the tests in their warm build directories.
# Fork pull requests, the nightly run and Run workflow use GitHub's runners only; the self-hosted
# runners refuse any other job.
name: CI
on:
# Draft pull requests run no CI (see `changes`, `checks` and `ci`): it starts when the PR is
# marked ready for review, and on every push after that.
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
# The merge queue: each queued group runs the full suite before it lands on main.
merge_group:
push:
branches: [main]
schedule:
- cron: "17 6 * * *"
workflow_dispatch:
inputs:
cross_compile:
description: "Also check WebAssembly and RISC-V (before a release)"
type: boolean
default: false
permissions:
contents: read
concurrency:
# A new PR push replaces the old run. Only one run per group may wait, so a busy main would
# cancel the waiting nightly (or a Run workflow) before it starts: they get groups of their own.
group: ci-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
CARGO_TERM_COLOR: always
jobs:
dependency-advisories:
name: Dependency advisories
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@stable
- name: Install advisory scanner
run: cargo install cargo-deny --version 0.20.2 --locked
- name: Check locked dependencies
run: cargo deny --locked check --warn unmaintained advisories bans sources
changes:
name: Select checks
# Not on a draft: every other check needs this one, so they skip too.
if: github.event_name != 'pull_request' || !github.event.pull_request.draft
# Trusted runs (see the top of this file) use the self-hosted runners.
runs-on: ${{ (github.event_name == 'merge_group' || (github.event_name == 'push' && github.ref == 'refs/heads/main') || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'self-hosted-light' || 'ubuntu-24.04' }}
permissions:
contents: read
# To ask whether the merge queue already tested a commit pushed to main.
actions: read
outputs:
native: ${{ steps.paths.outputs.native }}
kernel: ${{ steps.paths.outputs.kernel }}
goldens: ${{ steps.paths.outputs.goldens }}
windows: ${{ steps.paths.outputs.windows }}
full: ${{ steps.paths.outputs.full }}
packages: ${{ steps.paths.outputs.packages }}
oracle: ${{ steps.paths.outputs.oracle }}
# Whether the other Linux jobs use the self-hosted runners too.
# Trusted self-hosted runs only: a fork PR's checks run on the untrusted pool and must never
# route the jobs after them to the trusted runners.
# The fast self-hosted runners are kept for main and the merge queue (the pre-merge gate);
# pull requests, same-repo ones included, take the GitHub-hosted or untrusted paths.
local: ${{ runner.environment == 'self-hosted' && github.event_name != 'pull_request' }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- id: paths
name: Detect code and CI changes
env:
EVENT_NAME: ${{ github.event_name }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
PUSH_SHA: ${{ github.sha }}
REPO: ${{ github.repository }}
GH_TOKEN: ${{ github.token }}
run: |
native=false
kernel=false
goldens=false
windows=false
full=true
packages=
if [ "$EVENT_NAME" = push ]; then
# The queue lands the very commit its merge_group run tested. If that run passed,
# testing the push again proves nothing new; otherwise (a merge that skipped the
# queue, or the API cannot answer) test everything.
# curl and grep only: Select checks of a push runs on the self-hosted runner.
tested=$(curl -fsS -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" \
"https://api.github.com/repos/$REPO/actions/workflows/ci.yml/runs?event=merge_group&head_sha=$PUSH_SHA&status=success&per_page=1" \
| grep -o '"total_count": *[0-9]*' | grep -o '[0-9]*$' || echo 0)
if [ "${tested:-0}" -gt 0 ]; then
echo "The merge queue already tested $PUSH_SHA: repository checks only."
else
native=true
windows=true
fi
elif [ "$EVENT_NAME" != pull_request ]; then
native=true
windows=true
else
# Compare both old and new paths of renames; unknown paths run native checks.
git diff --no-renames --name-only -z "$BASE_SHA...$HEAD_SHA" > "$RUNNER_TEMP/ci-paths"
while IFS= read -r -d '' path; do
case "$path" in crates/kernel/*|vendor/truck-*) kernel=true ;; esac
# What can change a robustness golden: the kernel and the crates a design is built with.
case "$path" in
crates/kernel/*|crates/engine/*|crates/doc/*|crates/sketch/*|crates/geom/*|vendor/truck-*) goldens=true ;;
esac
# What the Windows D3D12 jobs exercise: the GPU crate and its shaders, the app and
# its startup, the viewport's wgpu glue, the vendored egui-wgpu, their scripts.
case "$path" in
crates/gpu/*|apps/solidcraft/*|crates/ui-egui/src/gpu.rs|vendor/egui-wgpu/*) windows=true ;;
tools/windows-*.ps1|.github/workflows/ci.yml|rust-toolchain*|.cargo/*) windows=true ;;
esac
case "$path" in
.github/workflows/ci.yml) native=true ;;
*.rs|Cargo.toml|Cargo.lock|*/Cargo.toml|.cargo/*|rust-toolchain*) native=true ;;
docs/*|*.md|.github/*|packaging/*|LICENSE*|NOTICE|THIRD-PARTY-LICENSES.txt) ;;
# The GitHub bots' scripts: no Rust, tested in this job (Claim and triage bot tests).
scripts/claim/*|scripts/triage/*) ;;
*) native=true ;;
esac
done < "$RUNNER_TEMP/ci-paths"
# A lockfile change that moves the graphics stack (wgpu, naga, egui, winit, windows).
if [ "$windows" = false ] && grep -qxz 'Cargo.lock' "$RUNNER_TEMP/ci-paths" \
&& git diff "$BASE_SHA...$HEAD_SHA" -- Cargo.lock \
| grep -Eq '^[ +-]name = "(wgpu[a-z_-]*|naga|egui[a-z_-]*|eframe|winit|windows[a-z_-]*|d3d12)"'; then
windows=true
fi
# Pull requests build and test the touched packages and their dependents.
python3 -I scripts/ci/affected.py --paths "$RUNNER_TEMP/ci-paths" > "$RUNNER_TEMP/ci-affected"
cat "$RUNNER_TEMP/ci-affected"
full=$(sed -n 's/^full=//p' "$RUNNER_TEMP/ci-affected")
packages=$(sed -n 's/^packages=//p' "$RUNNER_TEMP/ci-affected")
# Code changed but nothing was selected: be safe and test everything.
if [ "$native" = true ] && [ "$full" != true ] && [ -z "$packages" ]; then full=true; fi
# The goldens are solidcraft-engine tests: its tests must be in the archive.
if [ "$goldens" = true ] && [ "$full" != true ]; then
case " $packages " in *" solidcraft-engine "*) ;; *) packages="$packages solidcraft-engine" ;; esac
fi
fi
# The reference tests replay recipes through solidcraft-cli: needed when it, or anything
# it depends on, is selected.
oracle=$native
if [ "$EVENT_NAME" = pull_request ] && [ "$full" != true ]; then
case " $packages " in *" solidcraft-cli "*) ;; *) oracle=false ;; esac
fi
# The pull request fast lane: Windows and the reference tests run on main (and the nightly),
# not on every pull request push; a break there is fixed forward on main.
if [ "$EVENT_NAME" = pull_request ]; then windows=false; oracle=false; fi
{
echo "native=$native"
echo "kernel=$kernel"
echo "goldens=$goldens"
echo "windows=$windows"
echo "oracle=$oracle"
echo "full=$full"
echo "packages=$packages"
} >> "$GITHUB_OUTPUT"
checks:
name: Format and repository checks
if: github.event_name != 'pull_request' || !github.event.pull_request.draft
runs-on: ${{ (github.event_name == 'merge_group' || (github.event_name == 'push' && github.ref == 'refs/heads/main') || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)) && 'self-hosted-fast' || 'ubuntu-24.04' }}
timeout-minutes: 15
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Warm build directory (self-hosted runner)
if: runner.environment == 'self-hosted' && !(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
run: scripts/ci/warm-target.sh checks
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
# The trusted self-hosted runners keep their build directory and Cargo home instead; the
# untrusted ones (fork pull requests) start empty and restore main's cache, read-only.
- if: runner.environment == 'github-hosted' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: repo-checks
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Format
run: cargo fmt --all -- --check
- name: Asset attribution
run: cargo xtask assets
- name: UI icon standards and review tool
run: cargo xtask icons
- name: Licences
run: cargo xtask licences --check
- name: Dependency layers
run: cargo xtask layers
- name: Sorted lists
run: cargo xtask lists
- name: Design format pending changes
run: bash scripts/test-format-pending.sh
- name: Web app offline use and updates (headless Chrome)
run: cargo xtask web-offline-test
- name: Affected-package selection self-test
run: python3 -I scripts/ci/test_affected.py
- name: Claim and triage bot tests
run: node --test scripts/claim/*.test.js scripts/triage/*.test.js
clippy:
name: Native Linux Clippy
needs: changes
if: needs.changes.outputs.native == 'true'
runs-on: ${{ needs.changes.outputs.local == 'true' && 'self-hosted-fast' || 'ubuntu-24.04' }}
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Warm build directory (self-hosted runner)
if: runner.environment == 'self-hosted' && !(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
run: scripts/ci/warm-target.sh clippy
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
# The trusted self-hosted runners keep their build directory and Cargo home instead; the
# untrusted ones (fork pull requests) start empty and restore main's cache, read-only.
- if: runner.environment == 'github-hosted' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: native-clippy
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Clippy
env:
FULL: ${{ needs.changes.outputs.full }}
PACKAGES: ${{ needs.changes.outputs.packages }}
run: |
# Pull requests lint the affected packages (scripts/ci/affected.py); everything else, all.
scope=(--workspace)
if [ "$FULL" != true ]; then scope=(); for p in $PACKAGES; do scope+=(-p "$p"); done; fi
cargo clippy "${scope[@]}" --all-targets --profile ci --locked -- -D warnings
build-tests:
name: Build tests
needs: changes
# The self-hosted shards (main and the merge queue) build from their warm build directories
# instead: downloading the 3 GB archive took them longer than building.
if: needs.changes.outputs.native == 'true' && needs.changes.outputs.local != 'true'
runs-on: ${{ (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && (endsWith(format('{0}', github.event.pull_request.number), '7') || endsWith(format('{0}', github.event.pull_request.number), '8') || endsWith(format('{0}', github.event.pull_request.number), '9'))) && 'self-hosted-untrusted' || 'ubuntu-24.04' }}
# Fork builds compile inside gVisor on shared boxes; give them room to finish and upload.
timeout-minutes: ${{ (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && (endsWith(format('{0}', github.event.pull_request.number), '7') || endsWith(format('{0}', github.event.pull_request.number), '8') || endsWith(format('{0}', github.event.pull_request.number), '9'))) && 90 || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository) && 60 || 45 }}
env:
CARGO_INCREMENTAL: "0"
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_IGNORE_SERVER_IO_ERROR: "1"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Size parallel compiles to the job's memory
run: scripts/ci/build-jobs.sh
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: native-tests
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Set up compiler cache
uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11
with:
version: v0.16.0
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-nextest
- name: Compile tests into a nextest archive
env:
FULL: ${{ needs.changes.outputs.full }}
PACKAGES: ${{ needs.changes.outputs.packages }}
run: |
# Pull requests archive only the affected packages (scripts/ci/affected.py).
scope=(--workspace)
if [ "$FULL" != true ]; then scope=(); for p in $PACKAGES; do scope+=(-p "$p"); done; fi
cargo nextest archive "${scope[@]}" --cargo-profile ci --locked --archive-file tests.tar.zst
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: nextest-archive
path: tests.tar.zst
retention-days: 1
compression-level: 0
- name: Compiler cache statistics
if: always()
run: sccache --show-stats
tests:
name: Native Linux tests (${{ matrix.shard }}/4)
needs: [changes, build-tests]
if: ${{ !cancelled() && needs.changes.outputs.native == 'true' && (needs.changes.outputs.local == 'true' || needs.build-tests.result == 'success') }}
runs-on: ${{ needs.changes.outputs.local == 'true' && 'self-hosted-fast' || ((github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && (endsWith(format('{0}', github.event.pull_request.number), '7') || endsWith(format('{0}', github.event.pull_request.number), '8') || endsWith(format('{0}', github.event.pull_request.number), '9'))) && 'self-hosted-untrusted' || 'ubuntu-24.04') }}
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
# Pull requests test their affected crates in 2 shards; main and the nightly use 4.
shard: ${{ fromJSON(github.event_name == 'pull_request' && '[1, 2]' || '[1, 2, 3, 4]') }}
env:
# Pictures (snapshots, screenshots, ui.render, the viewport's offscreen tests) are drawn on
# the GPU. The runner has none: Mesa's lavapipe (software Vulkan) draws them, and a test that
# would skip for want of an adapter fails instead.
SOLIDCRAFT_GPU: software
SOLIDCRAFT_REQUIRE_GPU: "1"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install a software Vulkan driver (lavapipe)
# Preinstalled on the self-hosted runner, which has no sudo.
if: runner.environment == 'github-hosted'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends mesa-vulkan-drivers libvulkan1
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-nextest
# On the self-hosted runners each shard builds the tests in its warm build directory (a few
# minutes for what changed); elsewhere it runs the archive the Build tests job made.
- name: Size parallel compiles to the job's memory
if: needs.changes.outputs.local == 'true'
run: scripts/ci/build-jobs.sh
- name: Warm build directory (self-hosted runner)
if: needs.changes.outputs.local == 'true'
run: scripts/ci/warm-target.sh build-tests
- if: needs.changes.outputs.local == 'true'
uses: dtolnay/rust-toolchain@stable
- if: needs.changes.outputs.local != 'true'
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: nextest-archive
- name: Run tests
env:
EVENT_NAME: ${{ github.event_name }}
LOCAL: ${{ needs.changes.outputs.local }}
FULL: ${{ needs.changes.outputs.full }}
PACKAGES: ${{ needs.changes.outputs.packages }}
SHARD: ${{ matrix.shard }}
SHARDS: ${{ github.event_name == 'pull_request' && 2 || 4 }}
run: |
# Pull requests skip the heavy tests (default-filter in .config/nextest.toml).
all=()
# The boolean property tests draw new random inputs every run; they run only in the
# nightly property-tests job, so a known kernel bug cannot turn main red at random.
if [ "$EVENT_NAME" != pull_request ]; then
all=(--ignore-default-filter -E 'not (package(solidcraft-kernel) & binary(boolean_props))')
fi
if [ "$LOCAL" = true ]; then
scope=(--workspace)
if [ "$FULL" != true ]; then scope=(); for p in $PACKAGES; do scope+=(-p "$p"); done; fi
# The warm build directory keeps the last run's junit.xml: never upload that one.
rm -f "$CARGO_TARGET_DIR/nextest/ci/junit.xml"
cargo nextest run "${scope[@]}" --cargo-profile ci --locked \
--profile ci --partition "slice:$SHARD/$SHARDS" --no-tests=pass "${all[@]}"
else
# Extract into ./target: tests that run a built binary (env!("CARGO_BIN_EXE_*")) have
# the build job's path, which is the same checkout path here.
cargo nextest run --archive-file tests.tar.zst --workspace-remap . --extract-to . --extract-overwrite \
--profile ci --partition "slice:$SHARD/$SHARDS" --no-tests=pass "${all[@]}"
fi
- name: Robustness goldens (pull requests that change geometry)
# Every seed of tests/robustness/goldens.json (each seed is a test), split over the shards;
# pushes to main and the nightly run them in the step above.
if: github.event_name == 'pull_request' && needs.changes.outputs.goldens == 'true'
env:
SHARD: ${{ matrix.shard }}
SHARDS: ${{ github.event_name == 'pull_request' && 2 || 4 }}
run: |
if ! cargo nextest run --archive-file tests.tar.zst --workspace-remap . --extract-to . --extract-overwrite \
--profile ci --ignore-default-filter --partition "slice:$SHARD/$SHARDS" --no-tests=pass \
-E 'package(solidcraft-engine) & test(/^tests::robustness_goldens::/)'; then
echo "::error::A robustness golden changed. If the new result is intended, re-record it in this pull request and say why:"
echo "::error::ROBUSTNESS_GOLDENS_BLESS=\"why it changed\" cargo test --profile ci -p solidcraft-engine --lib robustness_goldens:: (see docs/robustness.md)"
exit 1
fi
- name: Upload test timings
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: junit-${{ matrix.shard }}
path: ${{ env.CARGO_TARGET_DIR || 'target' }}/nextest/ci/junit.xml
retention-days: 7
action-matrix:
# Every ordinary action on every face, edge, vertex and body of the design corpus, against
# tests/action_matrix/baseline.json (docs/action-matrix.md). Every test run checks a sample;
# the whole matrix runs nightly (or by hand) in shards of about equal cost, cut by
# tests/action_matrix/costs.json. Keep SHARDS in step with the matrix below.
name: Action matrix (${{ matrix.shard }}/16)
needs: build-tests
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
timeout-minutes: 55
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-nextest
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: nextest-archive
- name: Run one shard of the matrix
# A step limit below the job's, so the outcomes so far are still uploaded.
timeout-minutes: 45
env:
SHARDS: 16
SHARD: ${{ matrix.shard }}
OUT: ${{ github.workspace }}/action-matrix
run: |
mkdir -p "$OUT"
export ACTION_MATRIX_SHARD="$SHARD/$SHARDS" ACTION_MATRIX_RESULTS="$OUT/results-$SHARD.json" \
ACTION_MATRIX_TIMINGS="$OUT/timings-$SHARD.jsonl" ACTION_MATRIX_WORKDIR="$OUT/raw-$SHARD"
cargo nextest run --archive-file tests.tar.zst --workspace-remap . --extract-to . --extract-overwrite \
--profile ci --success-output final --run-ignored only -E "binary(action_matrix) & test(=action_matrix_shard)"
- name: Upload the outcomes and timings
# `cargo xtask action-matrix --from DIR --bless` (with every shard's files in DIR)
# records them in the baseline and costs.json.
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: action-matrix-${{ matrix.shard }}
path: action-matrix
retention-days: 14
action-matrix-determinism:
# A sample of the matrix run twice on one runner: every action must come out the same.
name: Action matrix determinism
needs: build-tests
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-nextest
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: nextest-archive
- name: Run a sample twice
run: |
cargo nextest run --archive-file tests.tar.zst --workspace-remap . --extract-to . --extract-overwrite \
--profile ci --success-output final --run-ignored only -E "binary(action_matrix) & test(=action_matrix_determinism)"
property-tests:
# The boolean property tests and the atomic command tests with many cases. Pull requests skip
# them (.config/nextest.toml); pushes to main run atomic_commands with small defaults (seeds
# 0..4) and skip boolean_props, whose random inputs would fail main at random.
name: Property tests (many cases)
needs: build-tests
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
timeout-minutes: 60
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28
with:
tool: cargo-nextest
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: nextest-archive
- name: Boolean properties and atomic commands
# The defaults (8 cases, seeds 0..4) take 159 s and 802 s on a 2-core VM; four times the
# cases and twice the seeds are estimated at about 40 minutes there (not yet timed on CI).
env:
PROPTEST_CASES: "32"
ATOMIC_SEEDS: "0..8"
run: |
cargo nextest run --archive-file tests.tar.zst --workspace-remap . --extract-to . --extract-overwrite \
--profile ci --ignore-default-filter --success-output final -E "binary(boolean_props) | binary(atomic_commands)"
doc-tests:
name: Doc tests
needs: changes
if: needs.changes.outputs.native == 'true'
runs-on: ${{ needs.changes.outputs.local == 'true' && 'self-hosted-fast' || 'ubuntu-24.04' }}
timeout-minutes: 45
env:
CARGO_INCREMENTAL: "0"
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_IGNORE_SERVER_IO_ERROR: "1"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Size parallel compiles to the job's memory
run: scripts/ci/build-jobs.sh
- name: Warm build directory (self-hosted runner)
if: runner.environment == 'self-hosted' && !(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
run: scripts/ci/warm-target.sh doc-tests
- uses: dtolnay/rust-toolchain@stable
# The trusted self-hosted runners keep their build directory and Cargo home instead; the
# untrusted ones (fork pull requests) start empty and restore main's cache, read-only.
- if: runner.environment == 'github-hosted' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: doc-tests
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Set up compiler cache
uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11
with:
version: v0.16.0
# nextest does not run doc tests.
- name: Doc tests
env:
FULL: ${{ needs.changes.outputs.full }}
PACKAGES: ${{ needs.changes.outputs.packages }}
run: |
# Pull requests test the affected packages that have a library (only libraries have doc
# tests); everything else, the whole workspace.
scope=(--workspace)
if [ "$FULL" != true ]; then
scope=()
libs=$(cargo metadata --no-deps --format-version 1 --locked \
| python3 -I -c 'import json, sys; print(" ".join(p["name"] for p in json.load(sys.stdin)["packages"] if any("lib" in t["kind"] for t in p["targets"])))')
for p in $PACKAGES; do
case " $libs " in *" $p "*) scope+=(-p "$p") ;; esac
done
fi
if [ ${#scope[@]} -eq 0 ]; then
echo "No affected package has a library: no doc tests to run."
exit 0
fi
cargo test "${scope[@]}" --profile ci --locked --doc
oracle:
name: Reference tests
needs: changes
if: needs.changes.outputs.oracle == 'true'
runs-on: ${{ needs.changes.outputs.local == 'true' && 'self-hosted-fast' || ((github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && (endsWith(format('{0}', github.event.pull_request.number), '7') || endsWith(format('{0}', github.event.pull_request.number), '8') || endsWith(format('{0}', github.event.pull_request.number), '9'))) && 'self-hosted-untrusted' || 'ubuntu-24.04') }}
timeout-minutes: 60
env:
CARGO_INCREMENTAL: "0"
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
SCCACHE_IGNORE_SERVER_IO_ERROR: "1"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Size parallel compiles to the job's memory
run: scripts/ci/build-jobs.sh
- name: Warm build directory (self-hosted runner)
if: runner.environment == 'self-hosted' && !(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
run: scripts/ci/warm-target.sh oracle
- uses: dtolnay/rust-toolchain@stable
# The trusted self-hosted runners keep their build directory and Cargo home instead; the
# untrusted ones (fork pull requests) start empty and restore main's cache, read-only.
- if: runner.environment == 'github-hosted' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository)
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: oracle
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Set up compiler cache
uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11
with:
version: v0.16.0
- name: Build the command line
run: cargo build -p solidcraft-cli --profile ci --locked
- name: Replay tests/oracle and compare with expected.json
run: cargo xtask oracle --cli "${CARGO_TARGET_DIR:-target}/ci/solidcraft-cli"
- name: Upload the report
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: oracle-report
path: docs/oracle.md
# D3D12's older shader compiler (FXC, used where dxcompiler.dll is absent) is stricter than
# DXC, Vulkan and Metal: a shader it cannot compile once crashed the app at startup. Build the
# GPU tessellation pipeline with FXC on the runner's WARP software adapter and compare its
# results with the CPU, and draw headless pictures with the viewport's pipelines there
# (crates/gpu/tests/capture.rs).
windows-fxc:
name: Windows D3D12 FXC shaders
needs: changes
if: needs.changes.outputs.windows == 'true'
runs-on: windows-latest
timeout-minutes: 45
env:
WGPU_BACKEND: dx12
WGPU_DX12_COMPILER: fxc
# WARP is a software (Cpu) adapter.
SOLIDCRAFT_GPU: software
SOLIDCRAFT_REQUIRE_GPU: "1"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: windows-fxc
save-if: ${{ github.ref == 'refs/heads/main' }}
# One test at a time: WARP's shader JIT crashes now and then (a null read in
# d3d10warp.dll's JIT optimizer, about 1 run in 60) when several devices compile compute
# pipelines at once, which the parallel tests do. Minidumps showed it in WARP itself, not
# in SolidCraft or FXC; serial runs never hit it.
- run: cargo test -p solidcraft-gpu --profile ci --locked -- --test-threads=1
# The Windows app starts D3D12 first and only the backend it uses (a Vulkan driver crashed the
# app at startup on Intel). Start the real app on the runner (its D3D12 adapter is WARP) with
# several saved settings and safe mode, and check each draws a frame and quits cleanly.
windows-startup:
name: Windows app startup
needs: changes
if: needs.changes.outputs.windows == 'true'
runs-on: windows-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: windows-startup
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Backend order and probing on Windows
run: cargo test -p solidcraft --profile ci --locked -- startup::tests native_crash::tests native_frame::tests
- name: Build the app
run: cargo build -p solidcraft --profile ci --locked
- name: Start the app under different settings
shell: pwsh
run: tools/windows-startup-smoke.ps1 -Exe target/ci/solidcraft.exe
- name: Resize border from the first frame
shell: pwsh
run: tools/windows-resize-smoke.ps1 -Exe target/ci/solidcraft.exe
# Preserve the existing check name for branch protection, including docs-only PRs.
ci:
needs: [changes, checks, clippy, build-tests, tests, doc-tests, oracle, windows-fxc, windows-startup]
if: always() && (github.event_name != 'pull_request' || !github.event.pull_request.draft)
runs-on: ${{ needs.changes.outputs.local == 'true' && 'self-hosted-light' || 'ubuntu-24.04' }}
steps:
- name: Verify required checks
env:
CHANGES: ${{ needs.changes.result }}
CHECKS: ${{ needs.checks.result }}
NATIVE: ${{ needs.changes.outputs.native }}
LOCAL: ${{ needs.changes.outputs.local }}
WINDOWS: ${{ needs.changes.outputs.windows }}
ORACLE_RUN: ${{ needs.changes.outputs.oracle }}
CLIPPY: ${{ needs.clippy.result }}
BUILD: ${{ needs.build-tests.result }}
TESTS: ${{ needs.tests.result }}
DOCS: ${{ needs.doc-tests.result }}
ORACLE: ${{ needs.oracle.result }}
WINDOWS_FXC: ${{ needs.windows-fxc.result }}
WINDOWS_STARTUP: ${{ needs.windows-startup.result }}
run: |
test "$CHANGES" = success
test "$CHECKS" = success
if [ "$NATIVE" = true ]; then
test "$CLIPPY" = success
# The self-hosted test shards build their own tests (no Build tests job there).
[ "$BUILD" = success ] || { [ "$LOCAL" = true ] && [ "$BUILD" = skipped ]; }
test "$TESTS" = success
test "$DOCS" = success
fi
if [ "$ORACLE_RUN" = true ]; then
test "$ORACLE" = success
fi
if [ "$WINDOWS" = true ]; then
test "$WINDOWS_FXC" = success
test "$WINDOWS_STARTUP" = success
fi
wasm:
name: Compile WebAssembly (manual)
if: github.event_name == 'workflow_dispatch' && inputs.cross_compile
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: wasm
save-if: ${{ github.ref == 'refs/heads/main' }}
- run: cargo xtask wasm
riscv64-gates:
name: Compile RISC-V (manual)
if: github.event_name == 'workflow_dispatch' && inputs.cross_compile
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Check out craft-fonts
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: ${{ env.CRAFT_FONTS_REF }}
path: craft-fonts
- uses: dtolnay/rust-toolchain@stable
with:
targets: riscv64gc-unknown-linux-gnu
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: riscv64
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Install cross-compilation dependencies
run: |
# Ubuntu's regular archive serves amd64; RISC-V packages live in ports.
# The 24.04 runner uses deb822 sources, so scope every Ubuntu stanza.
sudo sed -i '/^Architectures:/d; /^Types:/a Architectures: amd64' /etc/apt/sources.list.d/ubuntu.sources
sudo tee /etc/apt/sources.list.d/riscv64.sources >/dev/null <<'EOF'
Types: deb
URIs: http://ports.ubuntu.com/ubuntu-ports
Suites: noble noble-updates noble-backports noble-security
Components: main restricted universe multiverse
Architectures: riscv64
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
EOF
sudo dpkg --add-architecture riscv64
sudo apt-get update
sudo apt-get install -y gcc-riscv64-linux-gnu libasound2-dev:riscv64 libxkbcommon-dev:riscv64 \
libwayland-dev:riscv64 libx11-dev:riscv64 libxrandr-dev:riscv64 libxi-dev:riscv64 libgl1-mesa-dev:riscv64
- name: Cargo check on RISC-V
env:
CARGO_TARGET_RISCV64GC_UNKNOWN_LINUX_GNU_LINKER: riscv64-linux-gnu-gcc
PKG_CONFIG_SYSROOT_DIR: /
PKG_CONFIG_PATH: /usr/lib/riscv64-linux-gnu/pkgconfig:/usr/share/pkgconfig
PKG_CONFIG_ALLOW_CROSS: 1
run: |
. "$HOME/.cargo/env"
export CRAFT_FONTS_DIR="$PWD/craft-fonts"
cargo check --all-targets --locked --target riscv64gc-unknown-linux-gnu