CI: run frametests on the Linux test job and upload the report

Extend the existing 'test' job (Linux leg) with the frametests run after
the pspautotests, plus an artifact upload of the output so failures can
be examined. Both steps are gated on the test set being present in the
checkout, since dumps, references and config are maintained separately
from the runner.
This commit is contained in:
Henrik Rydgård committed 2026-08-09 14:00:37 +02:00
1 parent 0d47851e92
commit 1fd9d3b24b
2 files changed
+29 -6

No files matched your search

+15
View File
@@ -458,6 +458,21 @@ jobs:
working-directory: ${{ env.GITHUB_WORKSPACE }}
run: python test.py -g --graphics=software
# Runs only when the frametests test set is present in the checkout
# (dumps + references + frametests.json are maintained separately from
# the runner and may not be committed yet).
- name: Execute frametests
if: runner.os == 'Linux' && hashFiles('frametests/dumps/**') != '' && hashFiles('frametests/frametests.json') != ''
working-directory: ${{ env.GITHUB_WORKSPACE }}
run: python3 frametests.py frametests/frametests.json --out-mode=failures
- name: Upload frametest report
uses: actions/upload-artifact@v8
if: runner.os == 'Linux' && hashFiles('frametests/dumps/**') != '' && hashFiles('frametests/frametests.json') != '' && always()
with:
name: frametest-report
path: frametests/out/
test-headless-alpine:
runs-on: ubuntu-latest
container:
+14 -6
View File
@@ -162,15 +162,23 @@ The headless binary is located, in order of preference:
In CI (`GITHUB_ACTIONS` is set) the runner behaves as if `--strict` was
passed and prints `::error` annotations for each failing test, so failures
show up inline in the GitHub Actions log. A typical CI job:
show up inline in the GitHub Actions log.
The CI workflow extends the existing `test` job (Linux leg) in
`.github/workflows/build.yml`: after the pspautotests run it executes the
frametests and uploads the report as an artifact. The steps only run when
the test set (`frametests/dumps/`) is present in the checkout - the test
data and config are maintained separately from the runner and may not be
committed. Once committed, CI looks like:
```yaml
- name: Build headless
run: ./b.sh --headless
- name: Run frametests
- name: Execute frametests
if: runner.os == 'Linux' && hashFiles('frametests/dumps/**') != ''
run: python3 frametests.py frametests/frametests.json --out-mode=failures
- name: Upload report
uses: actions/upload-artifact@v4
- name: Upload frametest report
uses: actions/upload-artifact@v8
if: runner.os == 'Linux' && hashFiles('frametests/dumps/**') != '' && always()
with:
name: frametest-report
path: frametests/out/