AI skill: SKILL.md, AI contract, scrubbed references, eval fixture (d)

- rpa-dashboard-ai/SKILL.md: trigger description and the shared question flow;
  points at the no-AI frontend.md, themes and palette instead of copying them.
- templates/contract.md: runs, review rows, endpoints and the server approve gate.
- Superbill extracts: patient names and the client name removed, every file
  carries an EXAMPLE ONLY banner. tests/test_reference_scan.py checks both skills
  for patient data, internal IPs, credentials and keys.
- tests/fixtures/d-ai-extraction and its eval entry, which runs page_check_ai
  on the built folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
VasanthandClaude Opus 5.5 committed 2026-10-16 15:13:55 +05:30
1 parent 91495935b9
commit 8951ac05a5
30 files changed
+372 -26

No files matched your search

+10
View File
@@ -22,6 +22,14 @@ uv run --with pytest --with playwright pytest tests
`python dashboard/bot_runner.py` prints) instead of the built-in run.
- To add a case, write a `test_*.py` file in `page_check/` that uses the `page`
and `push(event, data)` fixtures from `conftest.py`.
- `page_check_ai/`: **Seam 1, AI web page.** Builds the React starter with npm
(filled with test values, in a temp folder that keeps `node_modules`), serves it with
its own `server_stub.py`, and checks bands, band filters, drafts, the approve gate
and "page X of Y", plus every no-AI case. Tests set run and review state on the stub
directly (the `stub` fixture). To check a generated dashboard folder instead, set
`PAGE_CHECK_AI_DIR=path/to/dashboard`. Needs npm on PATH, else it skips.
- `test_reference_scan.py`: no patient data, internal IPs, credentials or keys in any
bundled file, and every Superbill extract carries an `EXAMPLE ONLY` banner.
- `evals/`: **Seam 2.** Runs the skills on a fixture project with `claude -p`,
then grades the reply and runs the page check on the page it built:
@@ -33,6 +41,8 @@ uv run --with pytest --with playwright pytest tests
`FIXTURES` in `run_eval.py`. An entry with a `"fixture"` key reuses another
fixture folder with different answers: `c-keep` and `c-restyle` both run on
`c-web-existing-frontend`, and also check what changed in the files it already had.
An `"ai": True` entry (`d-ai-extraction`) expects the AI skill's React starter and
runs `page_check_ai/` on the folder it built.
For a bot fixture (`b-pyqt-bot`), the eval also dry-runs the generated
`bot_runner.py`, saves its events to `evals/results/<fixture>.run.jsonl`, and
replays them in the page check.