Files
Rpa_Skills/tests
VasanthandClaude Opus 5.5 162edd41df No-AI desktop path: PyQt bot wired to the page
- templates/bot_runner.py (new, no Qt): runs the bot as a child process
  (unbuffered, UTF-8), tails its output, keeps raw lines in bot_raw.log,
  and turns lines into step/log/status events. Step markers and friendly
  rules use specific substrings, first match wins; unmatched error lines
  are always shown, other unmatched lines hidden. Dry run:
  python dashboard/bot_runner.py prints the events the page would get.
- bridge.py: delegates to bot_runner, one run at a time, restores the
  saved dark mode on every page load (fixes sample-data mode in PyQt),
  adds load_latest_report / download_report slots the page calls.
- SKILL.md: skip the stack question for a PyQt project; desktop wiring
  steps from the bot's real output; verify steps for the host.
- tests: assert-based self-check for the line logic and run loop; Seam 1
  run replay (status dot, Run button, steps, log) and wired-page mode;
  fixture b-pyqt-bot and its Seam 2 eval (all 14 checks pass).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:28:51 +05:30
..

Tests

Run everything from the repo root:

uv run --with pytest --with playwright pytest tests
  • test_*.py: assert-based self-checks for template logic (lint of unfilled Python templates, the custom palette, presets in step with the template).

  • page_check/: Seam 1. Loads a dashboard page in headless Chromium with no backend, pushes contract envelopes through window.__pyMsg, and asserts on what is visible. Every case runs in light, dark and narrow (800px) mode.

    • By default it checks the no-AI template, filled with test values. To check a generated page instead, set PAGE_CHECK_HTML=path/to/dashboard.html.
    • The page sees a fake Qt bridge. Every action the page sends is recorded in window.__calls as [slot, ...args].
    • PAGE_CHECK_WIRED=1: the page is wired to a bot (sample data off). Sample-data cases skip, and a case checks that the page's actions go to the bot. With no PAGE_CHECK_HTML, the template is checked with SAMPLE_DATA switched off.
    • PAGE_CHECK_REPLAY=run.jsonl: replay a real bot run (the JSON lines that python dashboard/bot_runner.py prints) instead of the built-in run.
    • To add a case, write a test_*.py file in page_check/ that uses the page and push(event, data) fixtures from conftest.py.
  • evals/: Seam 2. Runs the skills on a fixture project with claude -p, then grades the reply and runs the page check on the page it built:

    uv run --with pytest --with playwright python tests/evals/run_eval.py a-empty
    

    Results go to evals/results/<fixture>.json. To add a fixture, add an entry to FIXTURES in run_eval.py. For a bot fixture (b-pyqt-bot), the eval also dry-runs the generated bot_runner.py, saves its events to evals/results/<fixture>.run.jsonl, and replays them in the page check.

  • fixtures/<letter>-<name>/: small test projects for the evals.