The API's coverage is measured, and every suite's is in Grafana (#394)

The cargo tests put web.rs at 17% and daemon.rs at 10%, because the
browser suite, which drives the API, ran a daemon nobody measured.
`sh scripts/coverage.sh api` builds an instrumented ipx and
ipx-directory in the capped coverage container and runs the browser
suite here against them (IPX_API_COVERAGE): web.rs is at 80% and
daemon.rs at 77%. What it does not reach: access.rs (Cloudflare Access,
0%; its unit tests 73%) and the directory's public pages (site.rs, 4%).

The coverage image is Ubuntu 24.04 now, as code-server is: built on
Debian trixie, the binaries needed glibc 2.41 and code-server has 2.39.
In that mode the suite starts those binaries rather than building its
own, its tests' `ipx` commands too (global-setup's IPX_BIN), passes
LLVM_PROFILE_FILE to both servers, and stops them with SIGTERM, since
an instrumented binary writes its profile as it exits and Playwright
otherwise kills them outright.

Reports go to target/coverage/cargo, api and page. Each run ends with
scripts/coverage-metrics.ts, which writes every suite's lines covered
and in all, by crate, to node-exporter's textfile directory; the
monitoring project's node-exporter now reads it (one flag, its compose
file backed up beside it), so Prometheus keeps them as
ipx_coverage_lines, and the iPX dashboard's Coverage row draws them:
each suite now, over time, and by part. The cargo tests' totals hold
their own code, which runs whole, so they read higher than the code
alone; the panels say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-07 03:33:49 +00:00
parent 352e2b77a5
commit 194a9250e7
10 changed files with 226 additions and 47 deletions

View File

@@ -4,6 +4,12 @@ import * as setup from './tests/ui/global-setup.ts';
// Before anything else, including the servers below.
setup.prepare();
// Measuring the API (#394): an instrumented binary writes its coverage only as it exits, and
// Playwright otherwise kills its servers outright. Asked to stop, both shut down on their own.
// Where they write it, as coverage.sh sets it; passed on to each server whatever Playwright passes.
const PROFILE: Record<string, string> = process.env.LLVM_PROFILE_FILE ? { LLVM_PROFILE_FILE: process.env.LLVM_PROFILE_FILE } : {};
const STOP = setup.API_COVERAGE ? { gracefulShutdown: { signal: 'SIGTERM' as const, timeout: 15_000 } } : {};
// Real browser against a real daemon. The stub-DOM smoke test catches a script that
// fails to load; it cannot catch a wrong selector, a handler that runs but does nothing,
// or a page that renders empty -- which is exactly what has slipped through before.
@@ -34,12 +40,13 @@ export default defineConfig({
},
{
// The directory the Directory shows (#205), on SQLite in the scratch directory.
command: `cargo build -q -p ipx-directory && exec ./target/debug/ipx-directory --config ${setup.root}/directory.toml`,
command: `${setup.API_COVERAGE ? '' : 'cargo build -q -p ipx-directory && '}exec ${setup.DIRECTORY_BIN} --config ${setup.root}/directory.toml`,
url: `${setup.DIRECTORY}/healthz`,
reuseExistingServer: false,
timeout: 180_000,
stdout: 'ignore',
env: { DIRECTORY_DATABASE_URL: `sqlite://${setup.root}/directory.db?mode=rwc` },
env: { DIRECTORY_DATABASE_URL: `sqlite://${setup.root}/directory.db?mode=rwc`, ...PROFILE },
...STOP,
},
{
// The frontend as production serves it (#203): built from web/, served by Caddy on the
@@ -52,8 +59,9 @@ export default defineConfig({
env: { IPX_COVERAGE: process.env.IPX_COVERAGE ?? '', WEB_BIND: '127.0.0.1:8791', WEB_ROOT: `${setup.root}/web`, IPX_UPSTREAM: '127.0.0.1:8796', HOME: setup.root, XDG_DATA_HOME: setup.root, XDG_CONFIG_HOME: setup.root },
},
{
// Build first so the tests always run against current source.
command: 'cargo build -q && exec ./target/debug/ipx daemon',
// Build first so the tests always run against current source, unless the instrumented
// build is what is measured (#394).
command: `${setup.API_COVERAGE ? '' : 'cargo build -q && '}exec ${setup.IPX_BIN} daemon`,
port: 8796,
reuseExistingServer: false,
timeout: 180_000,
@@ -70,7 +78,9 @@ export default defineConfig({
// Without one, the tests of the server paying for Signals see that it cannot.
TYPESAFE_KEY: process.env.TYPESAFE_KEY ?? '',
IPX_TYPESAFE_URL: 'http://127.0.0.1:8792/jev',
...PROFILE,
},
...STOP,
},
],
});