diff --git a/rpa-dashboard-ai/SKILL.md b/rpa-dashboard-ai/SKILL.md new file mode 100644 index 0000000..1de3601 --- /dev/null +++ b/rpa-dashboard-ai/SKILL.md @@ -0,0 +1,126 @@ +--- +name: rpa-dashboard-ai +description: Builds a working dashboard for an RPA bot or service that uses AI (an LLM or OCR extraction call). It has everything from rpa-dashboard-no-ai (sidebar, top bar with dark toggle and status dot, KPI cards, batch donut, run panel with steps and live log, report table with search and export) plus confidence bands, a review split pane (source document beside the extracted fields, band filters, draft edits, Verify) and an approve gate the server enforces. Web path is React + Vite + Tailwind with a sample-data server stub. Use when the user asks for an "AI RPA dashboard", an "extraction dashboard", an "LLM/OCR review dashboard", or any RPA/bot dashboard in a project that calls an LLM or OCR to extract fields. Needs rpa-dashboard-no-ai installed next to it. +--- + +# RPA dashboard (AI) + +Everything this skill needs is bundled. Never read the original source projects. +Shared files live in the no-AI skill, installed next to this one: `` below +means `../rpa-dashboard-no-ai/templates/`. + +| File | What it is | +|---|---| +| `templates/web/` | The React + Vite + Tailwind starter: shell, KPI, donut, run panel, report, **band legend, review split pane, approve gate**, sample data. | +| `templates/web/server_stub.py` | Sample-data backend for runs, review rows and the approve gate. Standard library only. | +| `templates/web/bands.py` | The band cut points and rules. The one source of truth: the backend imports it. | +| `templates/contract.md` | The AI contract: runs, review rows, endpoints and the gate. Extends `contract.md`. | +| `templates/reference/` | Superbill Review extracts, marked EXAMPLE ONLY. Use them to port extra polish, never copy their project names. | +| `frontend.md`, `detect_frontend.py` | Existing frontend: detect, report, keep or restyle. | +| `themes.md`, `palette.py` | Theme presets and the custom-colour steps. | +| `contract.md` | The no-AI contract the AI one extends. | + +The questions are the same as the no-AI skill's, in the same order. Ask one at a +time and wait for the answer. If the user's request already answers a question, +don't ask it again. + +## 1. Scan the project + +Before asking anything: +- **Frontend:** run `python detect_frontend.py ` and read + `frontend.md` section 1. +- **AI call:** find the LLM or OCR extraction (for example `genai`, `openai`, + `anthropic`, Textract or Tesseract calls): which fields it returns, which are + mandatory, and whether it returns per-word scores. +- **Bot or backend:** the entry point, what it prints, and any web framework + (FastAPI, Flask, Django). + +If the project has no AI call at all, say so and suggest `rpa-dashboard-no-ai`. + +## 2. Report the frontend, then ask keep or restyle + +Follow `frontend.md` sections 2 to 5. +- **No frontend:** say "No existing frontend found." and go to step 3. +- **Frontend found:** say "Frontend found at ``, stack ``, colours + `
`." Then ask: **keep its look**, or **restyle** it? + +## 3. Ask for colours + +Skip this if the user chose keep. Offer: +1. **Tebra Sky**: blue glass cards on a sky gradient. +2. **Superbill Ember**: flat surfaces with a red-orange brand gradient. +3. **Custom**: the user gives one brand hex colour. + +## 4. Ask desktop or web + +Ask only if there is no frontend. If there is one, use its stack. If the project +uses PyQt, choose desktop and say so. +- **Web:** the React starter in `templates/web/`, served by the backend. +- **Desktop:** the plain-HTML AI sections for the PyQt page aren't bundled yet. + Tell the user, and offer the web path or the no-AI desktop page + (`rpa-dashboard-no-ai`) without the AI sections. + +## 5. Ask which optional sections to build + +Always built: the no-AI shell and sections, plus the band legend, the review split +pane and the approve gate. Ask which extras to add (choose any, or none): +**checkpoint tracker**, **admin/roles screen**, **login page**. Build only the +chosen ones at their `OPTIONAL:` markers, and delete the markers. + +## 6. Build (web) + +1. Copy `templates/web/` into the project as `dashboard/` (an existing frontend: + see `frontend.md` section 3; in an existing React app, port the + `src/` components into it). Don't copy `node_modules/` or `dist/`. +2. Fill every `{{...}}` in `dashboard/index.html`'s CONFIG block: + + | Placeholder | Value | + |---|---| + | `APP_NAME`, `APP_SUBTITLE` | The project's name and a short tagline. | + | `LOGO_SRC` | The logo's path, or empty. | + | `STEPS_JSON` | The run steps as a JSON list, for example `["Initializing","Extract","Review","Completed"]`. | + | `REPORT_COLUMNS_JSON` | `[{"key":..., "label":...}]`, with a `status` column. | + | `API_BASE` | `/api` unless the backend mounts the dashboard API elsewhere. Keep it equal to `API` in `server_stub.py`. | + +3. Apply the theme in `dashboard/src/index.css` only: replace the `body{...}` and + `body.dark{...}` token blocks with the preset from `themes.md` (for Ember, + add its font `` to `index.html`'s ``), or with + `python palette.py ""` tokens for a custom colour, showing the user + its contrast table. For keep, fill them from the project's tokens + (`frontend.md` section 4). Never change the band colour block. +4. Search the output for `{{`. None may be left. + +## 7. Wire the data + +The contracts: `contract.md` for the shell, `templates/contract.md` for runs, +review rows and the gate. +- **Sample data (no backend for these endpoints yet):** keep `SAMPLE_DATA = true` in + `src/store.ts` and use `server_stub.py`. Replace its `SAMPLE_FIELDS` with the + fields the project's extraction really returns (name, a sample value, word scores, + mandatory). Tell the user which endpoints their backend must serve later. +- **Backend exists:** add the endpoints from `templates/contract.md` to it. Copy + `bands.py` next to it and band every field with `field_band(word_scores, value, + mandatory)`. The approve endpoint must refuse (409 with `detail`) while any flagged + field is unverified: the page never decides that. For the shell, set + `SAMPLE_DATA = false` and serve the no-AI endpoints too. +- Never invent fields, model names or prices the project doesn't have. + +## 8. Verify + +``` +cd dashboard +npm install +npm run build +python server_stub.py # page + sample API on http://127.0.0.1:8000 +``` + +Open the page and check, in light, dark (the toggle) and a window 820px wide or less: +- the dashboard, Run and Report tabs show sample data; +- on Review: the band legend counts and the caveat, band filters, a typed edit marked + unsaved until Save, a refused Approve while flagged fields are unverified, and + "page X of Y" on a running document; +- no `{{` is left and the console shows no script errors. + +If you can't run npm here, say so and give the user these commands. Finish by +telling the user where the files are, which theme and sections were built, and +whether the review data is sample data or the backend's. diff --git a/rpa-dashboard-ai/templates/contract.md b/rpa-dashboard-ai/templates/contract.md new file mode 100644 index 0000000..2674ca1 --- /dev/null +++ b/rpa-dashboard-ai/templates/contract.md @@ -0,0 +1,52 @@ +# Dashboard data contract (AI) + +Everything in the no-AI contract (`../rpa-dashboard-no-ai/templates/contract.md`) +applies unchanged: the `{event, data}` envelope, its events, its actions and its rules. +The AI dashboard adds extraction runs, review rows and the approve gate. These shapes +come from the Superbill prototype, trimmed. + +## Shapes + +``` +Run { job_id, document, status: pending|queued|running|done|failed, model, + pages_done, pages_total, summary: { green, amber, red, blank, unscored } | null, + approved, error } +ReviewRow { row_id, field, value, band: green|amber|red|blank|unscored, + score, reason, verified, corrected_value } +``` + +- `summary` counts the run's rows per band. It is `null` until the run is `done`. +- `score` is the field's worst word score, or `null` when it has none. +- `corrected_value` is `null` until a reviewer saves a value different from `value`. + +## Endpoints (web: `{API_BASE}` + path, JSON in and out) + +| Method and path | What it does | +|---|---| +| `GET /runs` | `{runs: [Run]}`. The page opens the first one. | +| `GET /runs/{job_id}` | One `Run`. | +| `GET /runs/{job_id}/review` | `{rows: [ReviewRow]}`. Empty until the run is `done`. | +| `GET /runs/{job_id}/document` | The source document (PDF or image) shown beside the fields. | +| `PATCH /review/{row_id}` | Body `{corrected_value?, verified}`. Patches one row, returns it. | +| `POST /runs/{job_id}/verify-remaining` | Verifies every flagged row not yet verified. Returns `{verified: n}`. | +| `POST /runs/{job_id}/approve` | **The approve gate.** `409 {detail}` while the run isn't done or any flagged row is unverified, else `200 {ok: true}`. | + +Errors answer with a non-2xx status and `{detail: ""}`. The page shows `detail`. + +## Rules + +- **Bands** come from `templates/web/bands.py` and nowhere else: green ≥0.95, amber + 0.90–0.95, red <0.90. A field takes its worst word score. A blank mandatory field + is `blank`, whatever its score. No scores is `unscored`. +- **Flagged** = every band but green. Each flagged row must be verified before approval. +- **The server enforces the gate.** The page disables nothing that matters for safety: + it asks, and shows the server's refusal. +- **Drafts:** the page sends nothing while a reviewer types. Save sends one PATCH per + edited row (with `verified: true`). Leaving with unsaved edits asks first. +- **Polling:** while a run is `queued` or `running`, the page polls `GET /runs/{job_id}` + every 1.5s and shows "page X of Y". It stops when the run settles. +- **Caveat:** the legend always shows that colour reflects print clarity, not + correctness. + +`templates/web/server_stub.py` implements all of this on sample data, standard library +only. A real backend serves the same endpoints with the same rules. diff --git a/rpa-dashboard-ai/templates/reference/backend/gemini_cost.py b/rpa-dashboard-ai/templates/reference/backend/gemini_cost.py index 7780a4a..e7c5b6b 100644 --- a/rpa-dashboard-ai/templates/reference/backend/gemini_cost.py +++ b/rpa-dashboard-ai/templates/reference/backend/gemini_cost.py @@ -1,3 +1,4 @@ +# EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. INCLUDE_THINKING_IN_COST = True # thinking tokens bill at output rate # ═════════════════════════════════════════════════════════════════════════════ diff --git a/rpa-dashboard-ai/templates/reference/web/DESIGN.md b/rpa-dashboard-ai/templates/reference/web/DESIGN.md index a20252d..1541088 100644 --- a/rpa-dashboard-ai/templates/reference/web/DESIGN.md +++ b/rpa-dashboard-ai/templates/reference/web/DESIGN.md @@ -1,3 +1,4 @@ + --- name: Superbill Extraction and Automation Tool description: Extract billing fields from a scan, verify them against the page, post to eClinicalWorks. diff --git a/rpa-dashboard-ai/templates/reference/web/api.ts b/rpa-dashboard-ai/templates/reference/web/api.ts index 737a20c..cd1caec 100644 --- a/rpa-dashboard-ai/templates/reference/web/api.ts +++ b/rpa-dashboard-ai/templates/reference/web/api.ts @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import type { AuditEntry, Batch, BatchStage, DemographyRecord, DemographyRun, InsuranceMasterRow, LastRun, ManagedUser, @@ -84,7 +85,7 @@ export const api = { for (const f of files) body.append('files', f) // `project` is which board received the drop. Nothing has detected a // template yet, and the board a batch appears on is derived from that, so - // without it a Gulf Coast upload surfaces under ECW and reads as a failed + // without it a Example Client upload surfaces under ECW and reads as a failed // upload on the Tebra board. return req(`/api/batches?${new URLSearchParams({ model, project })}`, { method: 'POST', diff --git a/rpa-dashboard-ai/templates/reference/web/bands.ts b/rpa-dashboard-ai/templates/reference/web/bands.ts index 0208684..16735ae 100644 --- a/rpa-dashboard-ai/templates/reference/web/bands.ts +++ b/rpa-dashboard-ai/templates/reference/web/bands.ts @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ /** * The confidence bands, and the exact colours the workbook paints. * diff --git a/rpa-dashboard-ai/templates/reference/web/components/BrandHeader.tsx b/rpa-dashboard-ai/templates/reference/web/components/BrandHeader.tsx index a8459e6..44c5277 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/BrandHeader.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/BrandHeader.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { ThemeSwitcher } from '../theme' import { ROLE_LABEL } from '../auth' diff --git a/rpa-dashboard-ai/templates/reference/web/components/Donut.tsx b/rpa-dashboard-ai/templates/reference/web/components/Donut.tsx index 51f8e8e..d8ef52c 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/Donut.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/Donut.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ /** * A proportional ring. * diff --git a/rpa-dashboard-ai/templates/reference/web/components/PdfPane.tsx b/rpa-dashboard-ai/templates/reference/web/components/PdfPane.tsx index ea6373e..7e7a611 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/PdfPane.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/PdfPane.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { useCallback, useEffect, useRef, useState } from 'react' import { useQuery } from '@tanstack/react-query' import { api } from '../api' diff --git a/rpa-dashboard-ai/templates/reference/web/components/ReviewPane.tsx b/rpa-dashboard-ai/templates/reference/web/components/ReviewPane.tsx index 2a7bcc6..ab1dfa1 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/ReviewPane.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/ReviewPane.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { useEffect, useMemo, useRef, useState } from 'react' import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import { api } from '../api' @@ -12,7 +13,7 @@ const isOfficeLocation = (t: string | null) => // One review card per record on a page. An ECW roster can list the same // person twice (two Sleep Study encounters for one patient) — line_index -// splits those into separate cards. A Gulf Coast ticket spreads its charges +// splits those into separate cards. A Example Client ticket spreads its charges // over several line_index values and must stay ONE card, so it is excluded. // The name stays in the key so runs stored before line_index existed (every // row LineIndex 0) still group exactly as before. @@ -100,7 +101,7 @@ type Patient = { type SaveArgs = { row: ReviewRow; draft: string; verified: boolean } // ── Office Location ──────────────────────────────────────────────────────── -// A Gulf Coast ticket whose yellow note the Office Location rules read. The +// A Example Client ticket whose yellow note the Office Location rules read. The // backend stores the parsed note on the ticket's first line under this field // (src/superbill/extraction/office_location.py, ParsedNote); a ticket without // it is a run from before the rules existed and keeps the flat layout. @@ -191,11 +192,11 @@ export function ReviewPane({ }) { const [index, setIndex] = useState(0) // Drafts live here, not in FieldRow — a correction is no longer committed on - // blur (that is exactly how four ARMSTRONG fields got saved mid-keystroke: - // "ARMSTRONG, CHARLES T" as "\ONG, CHARLES T", "99291" as "992"). Text typed + // blur (that is exactly how four fields of one record got saved mid-keystroke: + // "Doe, Jane A" as "\e, Jane A", "99291" as "992"). Text typed // into a field is held until an explicit Save, keyed by row_id rather than // bare field name — a bare name repeats once a page can carry more than one - // row per field (Gulf Coast's repeated CPT lines), and DemographyTab.tsx + // row per field (Example Client's repeated CPT lines), and DemographyTab.tsx // hit exactly this before switching its own drafts to row_id. const [drafts, setDrafts] = useState>({}) const qc = useQueryClient() @@ -238,7 +239,7 @@ export function ReviewPane({ // PDF order: page, then where the patient sits on it. line_index is each // record's position on its page, set by build_field_rows. Sorting by name - // here used to put ANDERSON above the first patient on the sheet, so the + // here used to put a later surname above the first patient on the sheet, so the // list no longer matched the document the reviewer had open beside it. // Name is only the tie-break, for runs stored before line_index existed. return [...byKey.values()].sort( @@ -254,7 +255,7 @@ export function ReviewPane({ map.get(key)!.push(row) } for (const rows of map.values()) { - // Gulf Coast (Template 7) records keep the order the API sent — the flat + // Example Client (Template 7) records keep the order the API sent — the flat // 20-column superbill layout, Ticket# through Units. Re-sorting by band // would scatter it (that is what put ICD11 above ICD2 and CPT at the // bottom). Every other template keeps worst-band-last, which is what the @@ -280,7 +281,7 @@ export function ReviewPane({ // A field that was never queued (needs_review === false) has no review // row until this moment on most templates — build_field_rows only emits // a row when a value exists or the field is billing-critical, so a - // genuinely blank optional field (e.g. Gulf Coast's From DOS / To DOS / + // genuinely blank optional field (e.g. Example Client's From DOS / To DOS / // Units, always null today) is addressed by the cell it occupies, // line_index included, rather than by row_id. if (row.needs_review === false) { @@ -363,7 +364,7 @@ export function ReviewPane({ const discardAll = () => setDrafts(settledMap) // Leaving a patient with typed-but-unsaved edits used to be silent — the old - // blur handler saved automatically, which is what produced the ARMSTRONG + // blur handler saved automatically, which is what produced the truncated-name // corruption. Now nothing commits without Save, so leaving instead has to // ask, or the edit is just lost with no signal either way. const guardLeave = (): boolean => { @@ -711,7 +712,7 @@ function PatientView({
{/* Keyed on row_id — a real database identity from /all-fields (SPB_Extraction.RowId), unique across every field on every page. - A page-and-field key would collide on Gulf Coast's repeated CPT + A page-and-field key would collide on Example Client's repeated CPT lines, where two rows share the same (page, field). */} {/* No entry animation. Rows are keyed by row_id, so every Next remounts them all, and the old staggered fade (opacity 0 over @@ -802,7 +803,7 @@ function PatientView({ * lost by making it editable. * * Nothing here commits on its own. An earlier version saved on blur, which is - * exactly how "ARMSTRONG, CHARLES T" got recorded as "\ONG, CHARLES T" — the + * exactly how "Doe, Jane A" got recorded as "\e, Jane A" — the * moment focus left the box mid-keystroke, whatever half-typed text was there * became the correction. Typing only updates the shared draft in ReviewPane; * the patient-level Save bar (here and in PdfPane) is the one place an edit is @@ -835,7 +836,7 @@ function FieldRow({ const corrected = row.corrected_value != null && row.corrected_value !== row.value // While being edited the value takes its own full-width line. A DX field is // five comma-separated codes and a provider name runs past twenty - // characters; in the shared row they were truncated to "ARMSTRONG, C", so a + // characters; in the shared row they were truncated to "Doe, J", so a // reviewer was correcting a value they could not fully see. Expanding only // on focus keeps the scan-many-fields-fast layout for reading. const expanded = focused || dirty @@ -938,7 +939,7 @@ function FieldRow({ /** - * A Gulf Coast ticket laid out the way its yellow note is: the note itself, + * A Example Client ticket laid out the way its yellow note is: the note itself, * the page-level rules it fired, the visit, then one tab per encounter. * * Hovering a field lights the note line it came from. The page preview is an diff --git a/rpa-dashboard-ai/templates/reference/web/components/RoiTab.tsx b/rpa-dashboard-ai/templates/reference/web/components/RoiTab.tsx index e21a72d..bffa1c2 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/RoiTab.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/RoiTab.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { useMemo, useState } from 'react' import { useQuery } from '@tanstack/react-query' import { api } from '../api' @@ -95,7 +96,7 @@ type Filter = 'all' | 'priced' | 'unmeasured' | 'failed' /** * api.roi() has no project filter — it totals every document. The ROI board - * is per-project (Gulf Coast's spend is not eClinicalWorks' spend), so the + * is per-project (Example Client's spend is not eClinicalWorks' spend), so the * totals are recomputed here from the already-fetched runs instead of a new * query param. Every field is a plain sum or count over RoiRun — nothing * server-side is being second-guessed, just re-scoped to fewer rows. @@ -138,7 +139,7 @@ export function RoiTab({ }: { /** Absent on the demography board: its runs have no charge review to open. */ onOpenRun?: (jobId: string) => void - /** Which project's spend this board reports — Gulf Coast tickets and + /** Which project's spend this board reports — Example Client tickets and * eClinicalWorks documents never share a total. */ project: Project /** The Demography ROI (ticket 05): its own payload, one Gemini call per PDF diff --git a/rpa-dashboard-ai/templates/reference/web/components/RunPanel.tsx b/rpa-dashboard-ai/templates/reference/web/components/RunPanel.tsx index 1c885a2..5474c97 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/RunPanel.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/RunPanel.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import { api } from '../api' import { IconDownload } from './icons' diff --git a/rpa-dashboard-ai/templates/reference/web/components/Sidebar.tsx b/rpa-dashboard-ai/templates/reference/web/components/Sidebar.tsx index d5df1bd..7700b6b 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/Sidebar.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/Sidebar.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { useState } from 'react' import type { ReactElement } from 'react' import type { BatchStage, BoardTab } from '../types' @@ -14,7 +15,7 @@ import { IconSpend, IconPage, IconLoop, IconIdCard, IconPatients } from './icons * icons are" step from every navigation. * * Two projects, one rail: ECW's boards (templates 1-6) and Tebra's (template 7 - * / Gulf Coast), each under its own group header. See project.ts for how a + * / Example Client), each under its own group header. See project.ts for how a * batch or run is assigned to one or the other. * * Selection and drag-and-drop-to-move-stage behaviour are unchanged — only the @@ -136,7 +137,7 @@ export function Sidebar({ tab="tebra-roi" icon={IconSpend} label="ROI Dashboard" - hint="What the Gulf Coast pipeline has cost, and what that bought" + hint="What the Example Client pipeline has cost, and what that bought" active={tab === 'tebra-roi'} onSelect={onTab} /> @@ -146,8 +147,8 @@ export function Sidebar({ dropStage="extraction" dropTab="tebra-extraction" icon={IconPage} - label="Gulf Coast Charge Extraction" - hint="Gulf Coast tickets: extract and clear the flagged fields" + label="Example Client Charge Extraction" + hint="Example Client tickets: extract and clear the flagged fields" active={tab === 'tebra-extraction'} n={counts.tebra.extraction} onSelect={onTab} @@ -159,8 +160,8 @@ export function Sidebar({ dropStage="automation" dropTab="tebra-automation" icon={IconLoop} - label="Gulf Coast Charge Automation" - hint="Gulf Coast batches that finished extraction and are ready to post" + label="Example Client Charge Automation" + hint="Example Client batches that finished extraction and are ready to post" active={tab === 'tebra-automation'} n={counts.tebra.automation} onSelect={onTab} diff --git a/rpa-dashboard-ai/templates/reference/web/components/SplitPane.tsx b/rpa-dashboard-ai/templates/reference/web/components/SplitPane.tsx index 8f0ecdf..3f10dbf 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/SplitPane.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/SplitPane.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { useCallback, useEffect, useRef, useState } from 'react' const KEY = 'superbill.split' diff --git a/rpa-dashboard-ai/templates/reference/web/components/StatsBar.tsx b/rpa-dashboard-ai/templates/reference/web/components/StatsBar.tsx index cd23f21..daf2873 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/StatsBar.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/StatsBar.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { BAND_VAR } from '../bands' import type { Run } from '../types' diff --git a/rpa-dashboard-ai/templates/reference/web/components/Tooltip.tsx b/rpa-dashboard-ai/templates/reference/web/components/Tooltip.tsx index dff3fab..270d9ea 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/Tooltip.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/Tooltip.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { useEffect, useLayoutEffect, useRef, useState } from 'react' import { createPortal } from 'react-dom' diff --git a/rpa-dashboard-ai/templates/reference/web/components/icons.tsx b/rpa-dashboard-ai/templates/reference/web/components/icons.tsx index 0911718..7ac3df5 100644 --- a/rpa-dashboard-ai/templates/reference/web/components/icons.tsx +++ b/rpa-dashboard-ai/templates/reference/web/components/icons.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ /** * The icon set. * diff --git a/rpa-dashboard-ai/templates/reference/web/index.css b/rpa-dashboard-ai/templates/reference/web/index.css index 9e83a7f..a79076d 100644 --- a/rpa-dashboard-ai/templates/reference/web/index.css +++ b/rpa-dashboard-ai/templates/reference/web/index.css @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ @import 'tailwindcss'; /* Manual dark mode: Tailwind v4 defaults `dark:` to the OS preference, but this @@ -1662,7 +1663,7 @@ textarea, .sb-brand-name { background-position: 0 0; } } -/* ── Office Location (a Gulf Coast ticket's yellow note) ────────────────────── +/* ── Office Location (a Example Client ticket's yellow note) ────────────────────── The note is amber because it IS amber on the page; everything else stays on the neutral surfaces. No brand ramp here -- it is reserved for its six things (DESIGN.md, "The One Loud Colour Rule"). */ diff --git a/rpa-dashboard-ai/templates/reference/web/index.html b/rpa-dashboard-ai/templates/reference/web/index.html index a78f364..c21e562 100644 --- a/rpa-dashboard-ai/templates/reference/web/index.html +++ b/rpa-dashboard-ai/templates/reference/web/index.html @@ -1,4 +1,5 @@ + diff --git a/rpa-dashboard-ai/templates/reference/web/theme.tsx b/rpa-dashboard-ai/templates/reference/web/theme.tsx index 859ef4a..2f1f026 100644 --- a/rpa-dashboard-ai/templates/reference/web/theme.tsx +++ b/rpa-dashboard-ai/templates/reference/web/theme.tsx @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import { createContext, useContext, useEffect, useState } from 'react' /** diff --git a/rpa-dashboard-ai/templates/reference/web/types.ts b/rpa-dashboard-ai/templates/reference/web/types.ts index e459668..36b492a 100644 --- a/rpa-dashboard-ai/templates/reference/web/types.ts +++ b/rpa-dashboard-ai/templates/reference/web/types.ts @@ -1,3 +1,4 @@ +/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */ import type { Band } from './bands' /** @@ -21,7 +22,7 @@ export type Project = 'ecw' | 'tebra' * stage a card can be dropped into. * * Two projects live in one rail: the ECW boards (templates 1-6) and the Tebra - * boards (template 7 / Gulf Coast). Tebra ids are prefixed because a bare + * boards (template 7 / Example Client). Tebra ids are prefixed because a bare * 'extraction' can no longer identify a board on its own — see project.ts for * the tab -> (project, stage) mapping. */ @@ -118,7 +119,7 @@ export interface ReviewRow { patient: string | null /** * 0 for every template except one that can carry more than one row per - * (page, patient, field) — Gulf Coast's repeated CPT lines, one per coder + * (page, patient, field) — Example Client's repeated CPT lines, one per coder * annotation on the same ticket. */ line_index: number diff --git a/tests/README.md b/tests/README.md index 72f7736..68aaf0d 100644 --- a/tests/README.md +++ b/tests/README.md @@ -22,6 +22,14 @@ uv run --with pytest --with playwright pytest tests `python dashboard/bot_runner.py` prints) instead of the built-in run. - To add a case, write a `test_*.py` file in `page_check/` that uses the `page` and `push(event, data)` fixtures from `conftest.py`. +- `page_check_ai/`: **Seam 1, AI web page.** Builds the React starter with npm + (filled with test values, in a temp folder that keeps `node_modules`), serves it with + its own `server_stub.py`, and checks bands, band filters, drafts, the approve gate + and "page X of Y", plus every no-AI case. Tests set run and review state on the stub + directly (the `stub` fixture). To check a generated dashboard folder instead, set + `PAGE_CHECK_AI_DIR=path/to/dashboard`. Needs npm on PATH, else it skips. +- `test_reference_scan.py`: no patient data, internal IPs, credentials or keys in any + bundled file, and every Superbill extract carries an `EXAMPLE ONLY` banner. - `evals/`: **Seam 2.** Runs the skills on a fixture project with `claude -p`, then grades the reply and runs the page check on the page it built: @@ -33,6 +41,8 @@ uv run --with pytest --with playwright pytest tests `FIXTURES` in `run_eval.py`. An entry with a `"fixture"` key reuses another fixture folder with different answers: `c-keep` and `c-restyle` both run on `c-web-existing-frontend`, and also check what changed in the files it already had. + An `"ai": True` entry (`d-ai-extraction`) expects the AI skill's React starter and + runs `page_check_ai/` on the folder it built. For a bot fixture (`b-pyqt-bot`), the eval also dry-runs the generated `bot_runner.py`, saves its events to `evals/results/.run.jsonl`, and replays them in the page check. diff --git a/tests/evals/run_eval.py b/tests/evals/run_eval.py index bd35850..6510c18 100644 --- a/tests/evals/run_eval.py +++ b/tests/evals/run_eval.py @@ -69,6 +69,17 @@ FIXTURES = { "page_has": ["--accent:#ff3049"], "existing": "#ff3049", # only token lines may change, and the theme now uses this colour }, + # (d) web service with an LLM extraction call and no UI: the AI skill's web path. + "d-ai-extraction": { + "prompt": "Build me an RPA dashboard for this project. I can't answer questions while you " + "work, so here are my answers to anything the skill asks. Colours: Superbill Ember. " + "Desktop or web: web. Optional sections: none.", + "frontend": r"^\s*(none|no\b)", + "asks": [r"colou?r", r"desktop|web", r"section"], + "never_asks": [r"\bkeep\b|restyle"], + "page_has": ["--accent:#ff3049"], + "ai": True, + }, } # A line a restyle may change: a custom property, a comment, a blank, or a web-font link. TOKEN_LINE = r"^\s*(--[\w-]+\s*:.*|/\*.*\*/|]*fonts\.googleapis\.com.*|@import.*fonts\.googleapis\.com.*)?\s*$" @@ -155,6 +166,11 @@ def main(fixture): results.mkdir(exist_ok=True) if pages: html = pages[0].read_text(encoding="utf-8") + if cfg.get("ai"): # the React starter: its theme lives in src/index.css + css = pages[0].parent / "src" / "index.css" + html += css.read_text(encoding="utf-8") if css.exists() else "" + checks["AI path: review starter and server stub built"] = all( + (pages[0].parent / f).exists() for f in ["server_stub.py", "bands.py", "src/components/Review.tsx"]) shutil.copy(pages[0], results / f"{fixture}.dashboard.html") checks["no placeholders left"] = "{{" not in html if cfg.get("brand"): @@ -168,7 +184,10 @@ def main(fixture): env = {**os.environ, "PAGE_CHECK_HTML": str(pages[0])} if cfg.get("bot"): env.update(bot_checks(project, html, checks, results / f"{fixture}.run.jsonl")) - pc = subprocess.run([sys.executable, "-m", "pytest", str(ROOT / "tests" / "page_check"), "-q"], + if cfg.get("ai"): + env["PAGE_CHECK_AI_DIR"] = str(pages[0].parent) + check_dir = ROOT / "tests" / ("page_check_ai" if cfg.get("ai") else "page_check") + pc = subprocess.run([sys.executable, "-m", "pytest", str(check_dir), "-q"], env=env, capture_output=True, text=True) checks["page check passes"] = pc.returncode == 0 page_check = pc.stdout.strip().splitlines()[-1:] if pc.stdout else [pc.stderr[-500:]] diff --git a/tests/fixtures/d-ai-extraction/README.md b/tests/fixtures/d-ai-extraction/README.md new file mode 100644 index 0000000..665e2d6 --- /dev/null +++ b/tests/fixtures/d-ai-extraction/README.md @@ -0,0 +1,8 @@ +# Invoice extraction service + +A small web service. It sends a scanned invoice to an LLM and gets back the +invoice fields, each with a confidence score per word. There is no UI yet. + +``` +uvicorn app:app --port 8000 +``` diff --git a/tests/fixtures/d-ai-extraction/app.py b/tests/fixtures/d-ai-extraction/app.py new file mode 100644 index 0000000..d793328 --- /dev/null +++ b/tests/fixtures/d-ai-extraction/app.py @@ -0,0 +1,11 @@ +"""Upload a scanned invoice, get its fields back.""" +from fastapi import FastAPI, UploadFile + +from extractor import extract_invoice + +app = FastAPI() + + +@app.post("/extract") +async def extract(file: UploadFile): + return extract_invoice(await file.read(), file.content_type or "application/pdf") diff --git a/tests/fixtures/d-ai-extraction/extractor.py b/tests/fixtures/d-ai-extraction/extractor.py new file mode 100644 index 0000000..19e4d09 --- /dev/null +++ b/tests/fixtures/d-ai-extraction/extractor.py @@ -0,0 +1,34 @@ +"""The LLM extraction call: one scanned invoice in, its fields out, each with word scores.""" +import json + +from google import genai +from google.genai import types + +MODEL = "gemini-2.5-flash" +FIELDS = { # field name -> mandatory + "Invoice number": True, + "Invoice date": True, + "Vendor": True, + "PO number": False, + "Due date": True, + "Total": True, +} +PROMPT = ( + "Read this invoice. Return JSON: a list of {field, value, word_scores}, one per field in " + + ", ".join(FIELDS) + + ". word_scores is your 0-1 legibility score for each word of the value. Use an empty " + "value and no scores when a field isn't on the page." +) + + +def extract_invoice(document: bytes, mime_type: str) -> list[dict]: + client = genai.Client() # credentials come from the environment + response = client.models.generate_content( + model=MODEL, + contents=[types.Part.from_bytes(data=document, mime_type=mime_type), PROMPT], + config=types.GenerateContentConfig(response_mime_type="application/json"), + ) + rows = json.loads(response.text) + for row in rows: + row["mandatory"] = FIELDS.get(row["field"], False) + return rows diff --git a/tests/fixtures/d-ai-extraction/requirements.txt b/tests/fixtures/d-ai-extraction/requirements.txt new file mode 100644 index 0000000..1ead6f6 --- /dev/null +++ b/tests/fixtures/d-ai-extraction/requirements.txt @@ -0,0 +1,4 @@ +fastapi +uvicorn +google-genai +python-multipart diff --git a/tests/page_check_ai/conftest.py b/tests/page_check_ai/conftest.py index ad0e484..148acf2 100644 --- a/tests/page_check_ai/conftest.py +++ b/tests/page_check_ai/conftest.py @@ -47,6 +47,13 @@ MODES = {"light": (1280, 820, False), "dark": (1280, 820, True), "narrow": (800, SKIP_COPY = shutil.ignore_patterns("node_modules", "dist", "__pycache__") +@pytest.fixture +def sample_data(): + """Shared no-AI cases that need the page's own sample data skip on a bot-wired page.""" + if os.environ.get("PAGE_CHECK_WIRED") == "1": + pytest.skip("page is wired to a bot, so it has no sample data") + + @pytest.fixture(scope="session") def site(): """The dashboard folder under test: server_stub.py, index.html and the sources.""" diff --git a/tests/test_reference_scan.py b/tests/test_reference_scan.py new file mode 100644 index 0000000..1927e22 --- /dev/null +++ b/tests/test_reference_scan.py @@ -0,0 +1,45 @@ +"""The skills are shared outside the source projects: no patient data, internal IPs, +credentials or keys in any bundled file, and the Superbill extracts are marked as examples.""" +import re +from pathlib import Path + +import pytest + +ROOT = Path(__file__).resolve().parents[1] +SKILLS = [ROOT / "rpa-dashboard-ai", ROOT / "rpa-dashboard-no-ai"] +AI_REFERENCE = ROOT / "rpa-dashboard-ai" / "templates" / "reference" +FORBIDDEN = { + "internal IP": r"\b(10\.\d{1,3}|192\.168|172\.(1[6-9]|2\d|3[01]))\.\d{1,3}\.\d{1,3}\b", + "Google API key": r"AIza[0-9A-Za-z_-]{30,}", + "OpenAI/Anthropic key": r"\bsk-[A-Za-z0-9_-]{20,}", + "AWS key": r"\bAKIA[0-9A-Z]{16}\b", + "private key": r"-----BEGIN [A-Z ]*PRIVATE KEY-----", + "hard-coded secret": r"""(?i)\b(password|passwd|api_?key|secret|client_secret)\b\s*[:=]\s*['"][^'"{}\s]{6,}['"]""", + "SSN": r"\b\d{3}-\d{2}-\d{4}\b", + "quoted roster name ('SURNAME, FIRST')": r"""["'][A-Z]{2,}, [A-Z]""", + "client name": r"Gulf Coast", +} +MARKER = "EXAMPLE ONLY" + + +def text_files(): + for skill in SKILLS: + for f in skill.rglob("*"): + if f.is_file() and "node_modules" not in f.parts and f.name != "package-lock.json": + yield f + + +@pytest.mark.parametrize("what", FORBIDDEN) +def test_no_sensitive_data_in_bundled_files(what): + hits = [f"{f.relative_to(ROOT)}:{n}" for f in text_files() + for n, line in enumerate(f.read_text(encoding="utf-8", errors="ignore").splitlines(), 1) + if re.search(FORBIDDEN[what], line)] + assert not hits, f"{what} found: {hits}" + + +def test_superbill_extracts_are_marked_as_examples(): + files = [f for f in AI_REFERENCE.rglob("*") if f.is_file()] + assert files + unmarked = [str(f.relative_to(ROOT)) for f in files + if MARKER not in "".join(f.read_text(encoding="utf-8").splitlines(keepends=True)[:5])] + assert not unmarked, f"no '{MARKER}' banner in the first lines: {unmarked}" diff --git a/tests/test_templates.py b/tests/test_templates.py index 89174ea..d337dd2 100644 --- a/tests/test_templates.py +++ b/tests/test_templates.py @@ -1,4 +1,5 @@ """Python templates must stay valid Python before their placeholders are filled.""" +import re import shutil import subprocess from pathlib import Path @@ -23,3 +24,13 @@ def test_tebra_sky_preset_is_the_template_default(): themes = (templates / "themes.md").read_text(encoding="utf-8") sky = themes.split("## 1. Tebra Sky")[1].split("```css\n")[1].split("```")[0] assert sky in (templates / "dashboard.html").read_text(encoding="utf-8") + + +def test_ai_skill_references_files_that_exist(): + skill = ROOT / "rpa-dashboard-ai" + text = (skill / "SKILL.md").read_text(encoding="utf-8") + paths = re.findall(r"`((?:templates/|)[\w./-]+)`", text) + assert len(paths) > 5 + missing = [p for p in paths + if not (skill / p.replace("", "../rpa-dashboard-no-ai/templates/")).exists()] + assert not missing