Merge integration/rpa-dashboard-skills (d89dce3) into ticket 04
Brings in tickets 05-07 (AI skill, cost page, AI desktop sections). tests/README.md keeps both the optional-section notes and the AI page-check notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
VasanthandClaude Opus 5.5 committed
commit
73adc74637
67 files changed
+4826
-100
No files matched your search
@@ -0,0 +1,171 @@
|
||||
---
|
||||
name: rpa-dashboard-ai
|
||||
description: Builds a working dashboard for an RPA bot or service that uses AI (an LLM or OCR extraction call). It has everything from rpa-dashboard-no-ai (sidebar, top bar with dark toggle and status dot, KPI cards, batch donut, run panel with steps and live log, report table with search and export) plus confidence bands, a review split pane (source document beside the extracted fields, band filters, draft edits, Verify), an approve gate the server enforces, and a cost/ROI page (tokens and cost per run, total and monthly spend, priced coverage, an against-doing-it-by-hand panel). Web path is React + Vite + Tailwind with a sample-data server stub; desktop path (PyQt) is the no-AI page plus plain-HTML AI sections over the Qt bridge, gate enforced in Python. Use when the user asks for an "AI RPA dashboard", an "extraction dashboard", an "LLM/OCR review dashboard", or any RPA/bot dashboard in a project that calls an LLM or OCR to extract fields. Needs rpa-dashboard-no-ai installed next to it.
|
||||
---
|
||||
|
||||
# RPA dashboard (AI)
|
||||
|
||||
Everything this skill needs is bundled. Never read the original source projects.
|
||||
Shared files live in the no-AI skill, installed next to this one: `<no-ai>` below
|
||||
means `../rpa-dashboard-no-ai/templates/`.
|
||||
|
||||
| File | What it is |
|
||||
|---|---|
|
||||
| `templates/web/` | The React + Vite + Tailwind starter: shell, KPI, donut, run panel, report, **band legend, review split pane, approve gate, cost page**, sample data. |
|
||||
| `templates/web/pricing.py` | The pricing table (with its source and as-of date) and `cost_for_call`. The backend imports it. |
|
||||
| `templates/web/server_stub.py` | Sample-data backend for runs, review rows and the approve gate. Standard library only. |
|
||||
| `templates/web/bands.py` | The band cut points and rules. The one source of truth: the backend imports it. |
|
||||
| `templates/desktop/ai_sections.html` | Desktop: the Review tab (band legend, review split pane, drafts, Verify, approve gate) and the Cost tab in plain HTML, pasted into `<no-ai>dashboard.html`. |
|
||||
| `templates/desktop/ai_bridge.py` | Desktop: the no-AI `Bridge` plus the `ai_request` slot that hands every AI request to the Python review backend. |
|
||||
| `templates/contract.md` | The AI contract: runs, review rows, endpoints and the gate, over HTTP or the Qt bridge. Extends `<no-ai>contract.md`. |
|
||||
| `templates/reference/` | Superbill Review extracts, marked EXAMPLE ONLY. Use them to port extra polish, never copy their project names. |
|
||||
| `<no-ai>frontend.md`, `<no-ai>detect_frontend.py` | Existing frontend: detect, report, keep or restyle. |
|
||||
| `<no-ai>themes.md`, `<no-ai>palette.py` | Theme presets and the custom-colour steps. |
|
||||
| `<no-ai>contract.md` | The no-AI contract the AI one extends. |
|
||||
|
||||
The questions are the same as the no-AI skill's, in the same order. Ask one at a
|
||||
time and wait for the answer. If the user's request already answers a question,
|
||||
don't ask it again.
|
||||
|
||||
## 1. Scan the project
|
||||
|
||||
Before asking anything:
|
||||
- **Frontend:** run `python <no-ai>detect_frontend.py <project root>` and read
|
||||
`<no-ai>frontend.md` section 1.
|
||||
- **AI call:** find the LLM or OCR extraction (for example `genai`, `openai`,
|
||||
`anthropic`, Textract or Tesseract calls): which fields it returns, which are
|
||||
mandatory, and whether it returns per-word scores. Note the provider, the model
|
||||
names, and whether the response's token usage (input, output, thinking) is kept.
|
||||
- **Bot or backend:** the entry point, what it prints, and any web framework
|
||||
(FastAPI, Flask, Django).
|
||||
|
||||
If the project has no AI call at all, say so and suggest `rpa-dashboard-no-ai`.
|
||||
|
||||
## 2. Report the frontend, then ask keep or restyle
|
||||
|
||||
Follow `<no-ai>frontend.md` sections 2 to 5.
|
||||
- **No frontend:** say "No existing frontend found." and go to step 3.
|
||||
- **Frontend found:** say "Frontend found at `<path>`, stack `<stack>`, colours
|
||||
`<main tokens>`." Then ask: **keep its look**, or **restyle** it?
|
||||
|
||||
## 3. Ask for colours
|
||||
|
||||
Skip this if the user chose keep. Offer:
|
||||
1. **Tebra Sky**: blue glass cards on a sky gradient.
|
||||
2. **Superbill Ember**: flat surfaces with a red-orange brand gradient.
|
||||
3. **Custom**: the user gives one brand hex colour.
|
||||
|
||||
## 4. Ask desktop or web
|
||||
|
||||
Ask only if there is no frontend. If there is one, use its stack. If the project
|
||||
uses PyQt, choose desktop and say so.
|
||||
- **Web:** the React starter in `templates/web/`, served by the backend.
|
||||
- **Desktop:** the no-AI PyQt page with `templates/desktop/ai_sections.html` pasted
|
||||
in, hosted by `templates/desktop/ai_bridge.py`. No web server, no npm.
|
||||
|
||||
## 5. Ask which optional sections to build
|
||||
|
||||
Always built: the no-AI shell and sections, plus the band legend, the review split
|
||||
pane, the approve gate and the cost page. Ask which extras to add (choose any, or none):
|
||||
**checkpoint tracker**, **admin/roles screen**, **login page**, **model picker** (choose
|
||||
the model per batch), **retry on another model** (for failed runs). Build only the
|
||||
chosen ones at their `OPTIONAL:` markers, and delete the markers.
|
||||
- **Model picker:** a `<select>` filled from `GET /models` (the pricing table's keys), at
|
||||
the marker in `Cost.tsx` (desktop: `ai_sections.html`) or next to the Run button; the chosen model goes with the
|
||||
batch's start action. Remember the last choice per user.
|
||||
- **Retry on another model:** on a `failed` run, at the marker in `Review.tsx` (desktop: `ai_sections.html`): a model
|
||||
`<select>` (from `GET /models`, defaulting to the run's own model) and a Retry button
|
||||
that sends `POST /runs/{job_id}/retry {model}`, then polls the run as usual.
|
||||
Both need the endpoints in `templates/contract.md`; add them to the backend (and the stub).
|
||||
|
||||
## 6. Build (web)
|
||||
|
||||
1. Copy `templates/web/` into the project as `dashboard/` (an existing frontend:
|
||||
see `<no-ai>frontend.md` section 3; in an existing React app, port the
|
||||
`src/` components into it). Don't copy `node_modules/` or `dist/`.
|
||||
2. Fill every `{{...}}` in `dashboard/index.html`'s CONFIG block:
|
||||
|
||||
| Placeholder | Value |
|
||||
|---|---|
|
||||
| `APP_NAME`, `APP_SUBTITLE` | The project's name and a short tagline. |
|
||||
| `LOGO_SRC` | The logo's path, or empty. |
|
||||
| `STEPS_JSON` | The run steps as a JSON list, for example `["Initializing","Extract","Review","Completed"]`. |
|
||||
| `REPORT_COLUMNS_JSON` | `[{"key":..., "label":...}]`, with a `status` column. |
|
||||
| `API_BASE` | `/api` unless the backend mounts the dashboard API elsewhere. Keep it equal to `API` in `server_stub.py`. |
|
||||
|
||||
3. Apply the theme in `dashboard/src/index.css` only: replace the `body{...}` and
|
||||
`body.dark{...}` token blocks with the preset from `<no-ai>themes.md` (for Ember,
|
||||
add its font `<link>` to `index.html`'s `<head>`), or with
|
||||
`python <no-ai>palette.py "<hex>"` tokens for a custom colour, showing the user
|
||||
its contrast table. For keep, fill them from the project's tokens
|
||||
(`<no-ai>frontend.md` section 4). Never change the band colour block.
|
||||
4. Search the output for `{{`. None may be left.
|
||||
|
||||
## 6. Build (desktop)
|
||||
|
||||
1. Build the no-AI desktop page and host first, exactly as steps 6 and 7 of
|
||||
`../rpa-dashboard-no-ai/SKILL.md` say: `dashboard/dashboard.html` (theme, placeholders,
|
||||
`SAMPLE_DATA = false` once the bot is wired), `bridge.py` and `bot_runner.py`.
|
||||
2. Paste the whole of `templates/desktop/ai_sections.html` (Review and Cost tabs) into `dashboard/dashboard.html`,
|
||||
just before `</body>`. Change nothing in it: it uses the page's theme tokens, and its
|
||||
band colours are fixed.
|
||||
3. Copy `templates/desktop/ai_bridge.py`, `templates/web/server_stub.py`,
|
||||
`templates/web/bands.py` and `templates/web/pricing.py` into `dashboard/`. Fill `{{APP_NAME}}` in `ai_bridge.py`.
|
||||
4. The app now starts with `python dashboard/ai_bridge.py` (not `bridge.py`).
|
||||
5. Search the output for `{{`. None may be left.
|
||||
|
||||
## 7. Wire the data
|
||||
|
||||
The contracts: `<no-ai>contract.md` for the shell, `templates/contract.md` for runs,
|
||||
review rows and the gate.
|
||||
- **Sample data (no backend for these endpoints yet):** keep `SAMPLE_DATA = true` in
|
||||
`src/store.ts` and use `server_stub.py`. Replace its `SAMPLE_FIELDS` with the
|
||||
fields the project's extraction really returns (name, a sample value, word scores,
|
||||
mandatory). Tell the user which endpoints their backend must serve later.
|
||||
- **Backend exists:** add the endpoints from `templates/contract.md` to it. Copy
|
||||
`bands.py` next to it and band every field with `field_band(word_scores, value,
|
||||
mandatory)`. The approve endpoint must refuse (409 with `detail`) while any flagged
|
||||
field is unverified: the page never decides that. For the shell, set
|
||||
`SAMPLE_DATA = false` and serve the no-AI endpoints too.
|
||||
- **Cost page:** copy `pricing.py` next to the backend. Fill `PRICING` with the rates of
|
||||
the models the project really calls, from the provider's pricing page, and set
|
||||
`PRICING_SOURCE` and `PRICING_AS_OF`. If you can't read the rates, ask the user for
|
||||
them. The backend stores each run's token usage and serves `GET /roi`, pricing each run
|
||||
with `cost_for_call`. A run with no recorded usage gets `null`, never 0. If the
|
||||
extraction doesn't keep token usage yet, wire it to (for example Gemini's
|
||||
`usage_metadata`, OpenAI's and Anthropic's `usage`). In sample mode, replace the stub's
|
||||
`ROI_RUNS` models with the project's models.
|
||||
- **Desktop:** the same rules, over the bridge (`templates/contract.md`, "Desktop").
|
||||
The shell is wired in `bot_runner.py` as the no-AI skill says. For review data, keep
|
||||
`server_stub.py` (replace its `SAMPLE_FIELDS` with the real fields, as above) until
|
||||
the project stores its extraction results. Then write a module with the same
|
||||
`dispatch(method, path, body)` over the project's store (same routes, bands from
|
||||
`bands.py`, approve refused while flagged fields are unverified) and import it in
|
||||
`ai_bridge.py` instead of `server_stub`. The page never decides approval.
|
||||
- Never invent fields, model names or prices the project doesn't have.
|
||||
|
||||
## 8. Verify
|
||||
|
||||
```
|
||||
cd dashboard
|
||||
npm install
|
||||
npm run build
|
||||
python server_stub.py # page + sample API on http://127.0.0.1:8000
|
||||
```
|
||||
|
||||
Open the page and check, in light, dark (the toggle) and a window 820px wide or less:
|
||||
- the dashboard, Run and Report tabs show sample data;
|
||||
- on Review: the band legend counts and the caveat, band filters, a typed edit marked
|
||||
unsaved until Save, a refused Approve while flagged fields are unverified, and
|
||||
"page X of Y" on a running document;
|
||||
- on Cost: the coverage pill, total and this month's spend, the month table, a run
|
||||
marked "not priced", the pricing source and as-of date, and the by-hand figure
|
||||
changing when an assumption is edited;
|
||||
- no `{{` is left and the console shows no script errors.
|
||||
|
||||
Desktop: `python dashboard/bot_runner.py` (dry run: real steps, ending at 100), then
|
||||
`python dashboard/ai_bridge.py`, and check the same list in the window.
|
||||
|
||||
If you can't run npm (or open a window) here, say so and give the user these commands. Finish by
|
||||
telling the user where the files are, which theme and sections were built, and
|
||||
whether the review data is sample data or the backend's.
|
||||
@@ -0,0 +1,89 @@
|
||||
# Dashboard data contract (AI)
|
||||
|
||||
Everything in the no-AI contract (`../rpa-dashboard-no-ai/templates/contract.md`)
|
||||
applies unchanged: the `{event, data}` envelope, its events, its actions and its rules.
|
||||
The AI dashboard adds extraction runs, review rows and the approve gate. These shapes
|
||||
come from the Superbill prototype, trimmed.
|
||||
|
||||
## Shapes
|
||||
|
||||
```
|
||||
Run { job_id, document, status: pending|queued|running|done|failed, model,
|
||||
pages_done, pages_total, summary: { green, amber, red, blank, unscored } | null,
|
||||
approved, error }
|
||||
ReviewRow { row_id, field, value, band: green|amber|red|blank|unscored,
|
||||
score, reason, verified, corrected_value }
|
||||
RoiRun { job_id, document, created_at, model, input_tokens, output_tokens,
|
||||
thinking_tokens, total_cost }
|
||||
```
|
||||
|
||||
- `summary` counts the run's rows per band. It is `null` until the run is `done`.
|
||||
- `score` is the field's worst word score, or `null` when it has none.
|
||||
- `corrected_value` is `null` until a reviewer saves a value different from `value`.
|
||||
- `created_at` is ISO 8601 in UTC. Months on the cost page are UTC months.
|
||||
- `RoiRun` token counts and `total_cost` (USD) are `null` when the run recorded no token
|
||||
usage. The page shows it as "not priced" and counts it in the coverage pill, never as $0.
|
||||
|
||||
## Endpoints (web: `{API_BASE}` + path, JSON in and out)
|
||||
|
||||
| Method and path | What it does |
|
||||
|---|---|
|
||||
| `GET /runs` | `{runs: [Run]}`. The page opens the first one. |
|
||||
| `GET /runs/{job_id}` | One `Run`. |
|
||||
| `GET /runs/{job_id}/review` | `{rows: [ReviewRow]}`. Empty until the run is `done`. |
|
||||
| `GET /runs/{job_id}/document` | The source document (PDF or image) shown beside the fields. |
|
||||
| `PATCH /review/{row_id}` | Body `{corrected_value?, verified}`. Patches one row, returns it. |
|
||||
| `POST /runs/{job_id}/verify-remaining` | Verifies every flagged row not yet verified. Returns `{verified: n}`. |
|
||||
| `GET /roi` | **Fetch ROI.** `{runs: [RoiRun], pricing: {source, as_of}}`, every run, newest first. The page sums spend, groups months and computes the by-hand panel itself. |
|
||||
| `POST /runs/{job_id}/approve` | **The approve gate.** `409 {detail}` while the run isn't done or any flagged row is unverified, else `200 {ok: true}`. |
|
||||
|
||||
Optional, only when the user chose those sections:
|
||||
|
||||
| Method and path | What it does |
|
||||
|---|---|
|
||||
| `GET /models` | `{models: [name]}`: the keys of the pricing table, so only priced models can be picked. |
|
||||
| `POST /runs/{job_id}/retry` | Body `{model}`. Re-queues a `failed` run on that model, returns the `Run`. |
|
||||
|
||||
Errors answer with a non-2xx status and `{detail: "<why>"}`. The page shows `detail`.
|
||||
|
||||
## Rules
|
||||
|
||||
- **Bands** come from `templates/web/bands.py` and nowhere else: green ≥0.95, amber
|
||||
0.90–0.95, red <0.90. A field takes its worst word score. A blank mandatory field
|
||||
is `blank`, whatever its score. No scores is `unscored`.
|
||||
- **Flagged** = every band but green. Each flagged row must be verified before approval.
|
||||
- **The server enforces the gate.** The page disables nothing that matters for safety:
|
||||
it asks, and shows the server's refusal.
|
||||
- **Drafts:** the page sends nothing while a reviewer types. Save sends one PATCH per
|
||||
edited row (with `verified: true`). Leaving with unsaved edits asks first.
|
||||
- **Polling:** while a run is `queued` or `running`, the page polls `GET /runs/{job_id}`
|
||||
every 1.5s and shows "page X of Y". It stops when the run settles.
|
||||
- **Cost** comes from `templates/web/pricing.py` and nowhere else: per million tokens,
|
||||
input × input rate + (output + thinking) × output rate, long-context rates above the
|
||||
model's threshold. An unknown model is an error naming the known models, never $0.
|
||||
The table records its source URL and as-of date; `GET /roi` returns both.
|
||||
- **Assumptions** (minutes per document, hourly rate) live only on the page, typed by the
|
||||
reader and labelled as assumptions. The backend never stores or guesses them.
|
||||
- **Caveat:** the legend always shows that colour reflects print clarity, not
|
||||
correctness.
|
||||
|
||||
`templates/web/server_stub.py` implements all of this on sample data, standard library
|
||||
only. A real backend serves the same endpoints with the same rules.
|
||||
|
||||
## Desktop (PyQt, no web server)
|
||||
|
||||
The same endpoints and rules travel over the Qt bridge instead of HTTP:
|
||||
|
||||
- Page to Python: one slot, `ai_request(id, method, path, body)`. `path` is the endpoint
|
||||
path above without `{API_BASE}`, `body` is a JSON string (`"{}"` when there is none).
|
||||
- Python to page: one envelope, `{event: "ai_response", data: {id, status, body}}`, with the
|
||||
same `status` and JSON `body` the web endpoint would answer.
|
||||
- Every endpoint above uses this pair, the cost page's `GET /roi` included (the page sends
|
||||
`ai_request(id, "GET", "/roi", "{}")` and gets `{runs: [RoiRun], pricing: {source, as_of}}`
|
||||
back as the `body`). No other AI events or slots exist.
|
||||
- `GET /runs/{job_id}/document` answers an SVG string or a URL the page can load (for
|
||||
example a `file:///` path to the PDF).
|
||||
|
||||
`templates/desktop/ai_bridge.py` adds that slot to the no-AI `Bridge`. It hands each
|
||||
request to the review backend's `dispatch(method, path, body)` (`server_stub.py` until the
|
||||
project has its own), so the approve gate is enforced in Python, never in the page.
|
||||
@@ -0,0 +1,37 @@
|
||||
"""PyQt6 host for the AI desktop dashboard: the no-AI Bridge plus one slot for the AI contract.
|
||||
|
||||
Copy next to the no-AI bridge.py and bot_runner.py, with server_stub.py and bands.py.
|
||||
Run it with: python dashboard/ai_bridge.py
|
||||
|
||||
The page sends each AI request (templates/contract.md: runs, review rows, the approve gate)
|
||||
as ai_request(id, method, path, body) and gets one `ai_response` envelope back. The review
|
||||
backend answers it in Python, so the approve gate holds whatever the page does.
|
||||
"""
|
||||
import json, sys, threading
|
||||
from PyQt6.QtCore import QUrl, pyqtSlot
|
||||
from PyQt6.QtWidgets import QApplication, QMainWindow
|
||||
from PyQt6.QtWebEngineWidgets import QWebEngineView
|
||||
from PyQt6.QtWebChannel import QWebChannel
|
||||
|
||||
import bridge
|
||||
import server_stub as review # sample data. Once the project has its own review backend, import that (same dispatch()).
|
||||
|
||||
|
||||
class AiBridge(bridge.Bridge):
|
||||
@pyqtSlot(int, str, str, str)
|
||||
def ai_request(self, req_id, method, path, body):
|
||||
# ponytail: answered on the GUI thread; move to a worker if the backend gets slow.
|
||||
status, payload = review.dispatch(method, path, json.loads(body or "{}"))
|
||||
self.send_to_js("ai_response", {"id": req_id, "status": status, "body": payload})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
threading.Thread(target=review.simulate_extraction, daemon=True).start() # sample data only: delete with server_stub
|
||||
app = QApplication(sys.argv)
|
||||
win, view = QMainWindow(), QWebEngineView()
|
||||
host, channel = AiBridge(view), QWebChannel()
|
||||
channel.registerObject("pyBridge", host)
|
||||
view.page().setWebChannel(channel)
|
||||
view.setUrl(QUrl.fromLocalFile(str(bridge.HTML_PATH)))
|
||||
win.setCentralWidget(view); win.resize(1280, 820); win.setWindowTitle("{{APP_NAME}}"); win.show()
|
||||
sys.exit(app.exec())
|
||||
@@ -0,0 +1,264 @@
|
||||
<!-- ===== AI SECTIONS (desktop) =====
|
||||
Paste this whole block into the no-AI dashboard.html, just before </body>. It adds a Review
|
||||
tab: band legend, band filters, the source document beside its extracted fields, drafts,
|
||||
Verify, and Approve. Plain HTML, the page's own theme tokens, no build step.
|
||||
Every request goes over the Qt bridge as one slot call, ai_request(id, method, path, body),
|
||||
and comes back as an `ai_response` envelope (see ../contract.md, "Desktop"). The approve
|
||||
gate is the Python backend's: this page only asks, and shows the refusal.
|
||||
-->
|
||||
<style>
|
||||
/* BAND COLOURS: semantic only (swatches, the rule down a field). The same in every preset and both modes. */
|
||||
body{--clean:#51a68f;--verify:#e39b34;--review:#de5243;--absent:#6b6580}
|
||||
.rv-bar{display:flex;flex-wrap:wrap;align-items:center;gap:10px 14px;padding:14px 18px}
|
||||
.rv-bar label{display:flex;align-items:center;gap:8px;font-size:12.5px;font-weight:600}
|
||||
.rv-bar select{padding:7px 10px;border:1px solid var(--border);border-radius:10px;background:var(--glass-strong);color:var(--ink);font:inherit;font-size:12.5px}
|
||||
.rv-muted{font-size:12px;color:var(--muted)}
|
||||
.rv-right{margin-left:auto;display:flex;align-items:center;gap:8px}
|
||||
.rv-alert{width:100%;border-radius:10px;padding:7px 12px;background:var(--bad-bg);color:var(--bad);font-size:12px;font-weight:700}
|
||||
.rv-legend{display:flex;flex-wrap:wrap;gap:4px 16px}
|
||||
.rv-legend li{display:flex;align-items:center;gap:6px;font-size:12px;font-weight:600}
|
||||
.rv-sw{width:10px;height:10px;border-radius:3px;display:inline-block}
|
||||
.rv-filters{margin-left:auto;display:flex;gap:4px}
|
||||
.rv-filters .btn,.rv-bar .btn,.rv-fields .btn,.rv-save .btn{padding:6px 11px;font-size:12px}
|
||||
.rv-split{display:flex;gap:14px;min-height:460px}
|
||||
.rv-doc{flex:1;min-height:420px;border:1px solid var(--border);border-radius:16px;background:#fff}
|
||||
.rv-pane{flex:1;display:flex;flex-direction:column;padding:0;overflow:hidden;margin-bottom:0}
|
||||
.rv-fields{flex:1;overflow-y:auto;padding:16px}
|
||||
.rv-fields li{display:grid;grid-template-columns:minmax(80px,130px) 1fr auto auto;align-items:center;gap:8px;margin-bottom:9px}
|
||||
.rv-fields li.done{opacity:.7}
|
||||
.rv-name{font-size:12px;font-weight:600;color:var(--muted);overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
|
||||
.rv-fields input{min-width:0;padding:7px 10px;border:1px solid var(--border);border-left-width:4px;border-radius:10px;background:var(--glass-strong);color:var(--ink);font:inherit;font-size:12.5px}
|
||||
.rv-fields input.dirty{outline:2px solid var(--warn)}
|
||||
.rv-score{width:36px;text-align:right;font-size:11px;color:var(--muted);font-variant-numeric:tabular-nums}
|
||||
.rv-note{grid-column:1/-1;margin-top:-4px;font-size:11px;color:var(--muted)}
|
||||
.rv-fields .btn{width:92px}
|
||||
.rv-save{display:flex;align-items:center;gap:10px;border-top:1px solid var(--border);padding:10px 16px}
|
||||
.rv-save .rv-unsaved{font-size:12px;font-weight:700;color:var(--warn)}
|
||||
.co-tile{padding:16px 18px;border-radius:16px;background:var(--glass-strong);border:1px solid var(--border)}
|
||||
.co-tile .kpi-val{margin-top:4px}
|
||||
.co-r{text-align:right}
|
||||
@media (max-width:820px){.rv-split{flex-direction:column}.rv-filters{margin-left:0}}
|
||||
</style>
|
||||
|
||||
<template id="ai-review-tab">
|
||||
<div id="tab-review" class="hidden">
|
||||
<div class="card rv-bar" id="rv-head">
|
||||
<label>Document <select id="rv-doc-pick" aria-label="Document"></select></label>
|
||||
<span id="rv-state"></span>
|
||||
<div class="rv-right" id="rv-actions"></div>
|
||||
<!-- OPTIONAL: retry on another model (failed runs) goes here -->
|
||||
|
||||
<p class="rv-alert hidden" id="rv-alert" role="alert"></p>
|
||||
</div>
|
||||
<div class="card rv-bar">
|
||||
<ul class="rv-legend" id="rv-legend" aria-label="Confidence bands"></ul>
|
||||
<span class="rv-muted">Colour reflects print clarity, not correctness. A cleanly printed value can still be misread.</span>
|
||||
<div class="rv-filters" id="rv-filters" role="group" aria-label="Filter by band"></div>
|
||||
</div>
|
||||
<div class="rv-split">
|
||||
<iframe class="rv-doc" id="rv-doc" title="Source document"></iframe>
|
||||
<div class="card rv-pane">
|
||||
<p class="rv-muted hidden" id="rv-wait" style="padding:18px">Fields appear when extraction finishes.</p>
|
||||
<ul class="rv-fields" id="rv-fields" aria-label="Extracted fields"></ul>
|
||||
<div class="rv-save hidden" id="rv-save"></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<template id="ai-cost-tab">
|
||||
<div id="tab-cost" class="hidden">
|
||||
<div class="toolbar"><span class="pill" id="co-cover"></span><span class="rv-muted" id="co-source"></span></div>
|
||||
<div class="kpi-row" id="co-tiles"></div>
|
||||
<div class="card"><div class="c-title">Month by month</div>
|
||||
<div class="tbl-wrap"><table aria-label="Spend by month"><thead><tr><th>Month</th><th>Documents priced</th><th class="co-r">Spend</th></tr></thead><tbody id="co-months"></tbody></table></div>
|
||||
</div>
|
||||
<section class="card" aria-labelledby="co-by-hand">
|
||||
<div class="c-title" id="co-by-hand">Against doing it by hand</div>
|
||||
<p class="rv-muted" style="margin-bottom:12px">These are your assumptions, not a measurement. Change them to match the team.</p>
|
||||
<div class="toolbar">
|
||||
<label class="rv-muted">Minutes per document <input id="co-min" type="number" min="0" value="5" style="width:80px"></label>
|
||||
<label class="rv-muted">Hourly rate <input id="co-rate" type="number" min="0" value="20" style="width:80px"></label>
|
||||
</div>
|
||||
<p id="co-hand" style="font-size:13px"></p>
|
||||
</section>
|
||||
<!-- OPTIONAL: model picker (per batch) goes here -->
|
||||
<div class="card"><div class="c-title">Cost per run</div>
|
||||
<div class="tbl-wrap"><table aria-label="Cost per run"><thead><tr><th>Document</th><th>Date</th><th>Model</th><th class="co-r">Input</th><th class="co-r">Output</th><th class="co-r">Thinking</th><th class="co-r">Cost</th></tr></thead><tbody id="co-runs"></tbody></table></div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script>
|
||||
(()=>{
|
||||
// Legend order, label and colour token. Cut points come from bands.py on the Python side.
|
||||
const BANDS=[['green','Clean ≥0.95','var(--clean)'],['amber','Check 0.90–0.95','var(--verify)'],
|
||||
['red','Review <0.90','var(--review)'],['blank','Missing (required)','var(--absent)'],['unscored','Unscored','var(--absent)']];
|
||||
const FILTERS=[['All','all'],['Red','red'],['Amber','amber'],['Missing','blank'],['Unscored','unscored']];
|
||||
const BUSY=['queued','running']; // polled every 1.5s while in one of these
|
||||
const colourOf=b=>BANDS.find(x=>x[0]===b)[2];
|
||||
const settled=r=>r.corrected_value??r.value??'';
|
||||
|
||||
// ----- the tabs: nav buttons after Report, sections in the content area -----
|
||||
TABS.review=['Review','Check extracted fields, then approve'];
|
||||
TABS.cost=['Cost','What the AI cost, run by run'];
|
||||
document.querySelector('.nav-item[data-tab="report"]').insertAdjacentHTML('afterend',
|
||||
'<button class="nav-item" data-tab="review" title="Review"><svg viewBox="0 0 24 24"><path d="M9 11l3 3 8-8"/><path d="M20 12v7a1 1 0 0 1-1 1H5a1 1 0 0 1-1-1V5a1 1 0 0 1 1-1h11"/></svg><span>Review</span></button>'
|
||||
+'<button class="nav-item" data-tab="cost" title="Cost"><svg viewBox="0 0 24 24"><path d="M12 2v20M17 6H9.5a3.5 3.5 0 0 0 0 7h5a3.5 3.5 0 0 1 0 7H6"/></svg><span>Cost</span></button>');
|
||||
['review','cost'].forEach(t=>{ document.querySelector(`.nav-item[data-tab="${t}"]`).onclick=()=>showTab(t);
|
||||
document.querySelector('.content').append($(`ai-${t}-tab`).content.cloneNode(true)); });
|
||||
const baseShow=showTab;
|
||||
showTab=t=>{
|
||||
const onReview=!$('tab-review').classList.contains('hidden');
|
||||
if(onReview && t!=='review' && !leaveOk()) return;
|
||||
baseShow(t);
|
||||
if(t==='review') loadRuns();
|
||||
if(t==='cost') loadRoi();
|
||||
};
|
||||
window.addEventListener('beforeunload',e=>{ if(dirty().length) e.preventDefault(); });
|
||||
|
||||
// ----- transport: one bridge slot out, one `ai_response` envelope back -----
|
||||
const pending={}; let seq=0;
|
||||
function ai(method,path,body){
|
||||
if(!bridge) return Promise.reject(new Error('Review needs the desktop app (no Qt bridge here).'));
|
||||
return new Promise((ok,fail)=>{ const id=++seq; pending[id]=[ok,fail];
|
||||
bridge.ai_request(id,method,path,JSON.stringify(body||{})); });
|
||||
}
|
||||
const baseHandle=handle;
|
||||
handle=(event,d)=>{
|
||||
if(event!=='ai_response') return baseHandle(event,d);
|
||||
const p=pending[d.id]; delete pending[d.id];
|
||||
if(p) d.status<300 ? p[0](d.body) : p[1](new Error((d.body&&d.body.detail)||`Error ${d.status}`));
|
||||
};
|
||||
|
||||
// ----- state -----
|
||||
let runs=[], jobId=null, run=null, rows=[], drafts={}, filter='all', refusal=null, timer=null;
|
||||
const dirty=()=>rows.filter(r=>r.row_id in drafts && drafts[r.row_id]!==settled(r));
|
||||
const leaveOk=()=>!dirty().length || confirm(`${dirty().length} unsaved change(s) on this document. Discard them and leave?`);
|
||||
const fail=e=>{ refusal=e.message; renderHead(); };
|
||||
|
||||
async function loadRuns(){
|
||||
try{ runs=(await ai('GET','/runs')).runs; }catch(e){ return fail(e); }
|
||||
$('rv-doc-pick').innerHTML=runs.map(r=>`<option value="${esc(r.job_id)}">${esc(r.document)}</option>`).join('');
|
||||
if(!runs.length){ refusal=null; $('rv-state').innerHTML='<span class="rv-muted">No documents extracted yet.</span>'; return; }
|
||||
openRun(runs.some(r=>r.job_id===jobId) ? jobId : runs[0].job_id);
|
||||
}
|
||||
$('rv-doc-pick').onchange=e=>{ if(leaveOk()) openRun(e.target.value); else e.target.value=jobId; };
|
||||
|
||||
function openRun(id){
|
||||
clearTimeout(timer);
|
||||
jobId=id; run=null; rows=[]; drafts={}; refusal=null;
|
||||
$('rv-doc-pick').value=id;
|
||||
ai('GET',`/runs/${id}/document`).then(doc=>{ if(id===jobId) $('rv-doc').src=
|
||||
String(doc).startsWith('<') ? 'data:image/svg+xml;charset=utf-8,'+encodeURIComponent(doc) : doc; }).catch(()=>{});
|
||||
render(); tick(id);
|
||||
}
|
||||
async function tick(id){ // load the run, and poll it while it is still extracting
|
||||
try{
|
||||
const r=await ai('GET',`/runs/${id}`); if(id!==jobId) return;
|
||||
run=r;
|
||||
if(BUSY.includes(r.status)) timer=setTimeout(()=>tick(id),1500);
|
||||
else{ const d=await ai('GET',`/runs/${id}/review`); if(id!==jobId) return; rows=d.rows; }
|
||||
render();
|
||||
}catch(e){ if(id===jobId) fail(e); }
|
||||
}
|
||||
async function reload(){
|
||||
const id=jobId, [r,d]=await Promise.all([ai('GET',`/runs/${id}`),ai('GET',`/runs/${id}/review`)]);
|
||||
if(id===jobId){ run=r; rows=d.rows; render(); }
|
||||
}
|
||||
|
||||
// ----- actions -----
|
||||
async function act(fn){ refusal=null; try{ await fn(); }catch(e){ refusal=e.message; } await reload().catch(fail); }
|
||||
const save=()=>act(()=>Promise.all(dirty().map(r=>{ const text=drafts[r.row_id].trim();
|
||||
// Same as extracted (or emptied) means "the extracted value was right", not an edit.
|
||||
return ai('PATCH',`/review/${r.row_id}`,{corrected_value: text===''||text===(r.value??'') ? null : text, verified:true});
|
||||
})).then(()=>{ drafts={}; }));
|
||||
const verify=r=>act(()=>ai('PATCH',`/review/${r.row_id}`,{verified:!r.verified}));
|
||||
const verifyRemaining=()=>act(()=>ai('POST',`/runs/${jobId}/verify-remaining`));
|
||||
const approve=()=>act(()=>ai('POST',`/runs/${jobId}/approve`).catch(e=>{ throw new Error('Not approved: '+e.message); }));
|
||||
|
||||
// ----- render -----
|
||||
function render(){ renderHead(); renderLegend(); renderFields(); renderSave(); }
|
||||
function renderHead(){
|
||||
const busy=run && BUSY.includes(run.status), pill=!run?'':run.status==='done'?'success':run.status==='failed'?'failed':'attention';
|
||||
$('rv-state').innerHTML = run ? `<span class="pill ${pill}">${esc(run.status)}</span>`
|
||||
+ (busy ? ` <span class="rv-muted">page ${esc(run.pages_done)} of ${esc(run.pages_total||'?')}</span>` : '')
|
||||
+ (run.error ? ` <span class="rv-muted" style="color:var(--bad)">${esc(run.error)}</span>` : '') : '';
|
||||
const left=rows.filter(r=>r.band!=='green' && !r.verified).length, n=dirty().length, acts=$('rv-actions');
|
||||
acts.innerHTML = run && run.status==='done' ? `<span class="rv-muted">${left} to verify</span>
|
||||
<button class="btn" id="rv-vr"${!left||run.approved?' disabled':''}>Verify remaining</button>`
|
||||
+ (run.approved ? '<span class="pill success">✓ Approved</span>'
|
||||
: `<button class="btn primary" id="rv-ap"${n?' disabled title="Save or discard your edits first"':''}>Approve</button>`) : '';
|
||||
if($('rv-vr')) $('rv-vr').onclick=verifyRemaining;
|
||||
if($('rv-ap')) $('rv-ap').onclick=approve;
|
||||
$('rv-alert').textContent=refusal||''; $('rv-alert').classList.toggle('hidden',!refusal);
|
||||
}
|
||||
function renderLegend(){
|
||||
const s=(run&&run.summary)||{};
|
||||
$('rv-legend').innerHTML=BANDS.map(([b,label,c])=>`<li><i class="rv-sw" style="background:${c}"></i>${label} · ${s[b]||0}</li>`).join('');
|
||||
$('rv-filters').innerHTML=FILTERS.map(([label,b])=>`<button class="btn${filter===b?' primary':''}" aria-pressed="${filter===b}" data-band="${b}">${label}</button>`).join('');
|
||||
$('rv-filters').querySelectorAll('button').forEach(btn=>btn.onclick=()=>{ filter=btn.dataset.band; renderLegend(); renderFields(); });
|
||||
}
|
||||
function renderFields(){
|
||||
$('rv-wait').classList.toggle('hidden',!run||run.status==='done');
|
||||
const shown=filter==='all'?rows:rows.filter(r=>r.band===filter);
|
||||
$('rv-fields').innerHTML=shown.map(r=>{ const draft=drafts[r.row_id]??settled(r), isDirty=draft!==settled(r);
|
||||
const note=[r.reason, r.corrected_value!=null ? `extracted <s>${esc(r.value||'empty')}</s>` : ''].filter(Boolean);
|
||||
return `<li data-row="${r.row_id}" class="${r.verified&&!isDirty?'done':''}">
|
||||
<span class="rv-name" title="${esc(r.field)}">${esc(r.field)}</span>
|
||||
<input aria-label="${esc(r.field)}" value="${esc(draft)}" placeholder="empty" title="${esc(r.reason||'')}" class="${isDirty?'dirty':''}" style="border-left-color:${colourOf(r.band)}">
|
||||
<span class="rv-score" title="Legibility score: how clearly it was read, not whether it is right">${r.score==null?'—':Number(r.score).toFixed(2)}</span>
|
||||
<button class="btn" aria-label="Verify ${esc(r.field)}" aria-pressed="${!!r.verified}"${isDirty?' disabled':''}>${r.verified?'✓ Verified':'Verify'}</button>
|
||||
${note.length?`<span class="rv-note">${note.map(x=>x===r.reason?esc(x):x).join(' · ')}</span>`:''}</li>`; }).join('');
|
||||
$('rv-fields').querySelectorAll('li').forEach(li=>{ const r=rows.find(x=>x.row_id===+li.dataset.row), box=li.querySelector('input');
|
||||
box.oninput=()=>{ drafts[r.row_id]=box.value; const d=box.value!==settled(r); // a draft: nothing is sent until Save
|
||||
box.classList.toggle('dirty',d); li.querySelector('button').disabled=d; renderSave(); renderHead(); };
|
||||
box.onkeydown=e=>{ if(e.key==='Escape'){ box.value=settled(r); box.oninput(); } };
|
||||
li.querySelector('button').onclick=()=>verify(r); });
|
||||
}
|
||||
function renderSave(){
|
||||
const n=dirty().length;
|
||||
$('rv-save').classList.toggle('hidden',!n);
|
||||
$('rv-save').innerHTML = n ? `<span class="rv-unsaved">${n} unsaved change${n===1?'':'s'}</span>
|
||||
<button class="btn" id="rv-discard" style="margin-left:auto">Discard</button><button class="btn primary" id="rv-save-btn">Save</button>` : '';
|
||||
if(n){ $('rv-discard').onclick=()=>{ drafts={}; render(); }; $('rv-save-btn').onclick=save; }
|
||||
}
|
||||
|
||||
// ----- cost: measured (tokens and cost per run, priced in Python) kept apart from assumed (typed here) -----
|
||||
// A run with no recorded token usage is "not priced", never $0, and the coverage pill says how many.
|
||||
let roi=[];
|
||||
const money=n=>`$${n.toFixed(n>0&&n<0.01?4:2)}`, num=n=>n==null?'—':n.toLocaleString('en-US');
|
||||
const monthOf=iso=>iso.slice(0,7); // 'YYYY-MM', UTC
|
||||
const monthLabel=ym=>new Date(`${ym}-01T00:00:00Z`).toLocaleString('en-US',{month:'short',year:'numeric',timeZone:'UTC'});
|
||||
const sum=rs=>rs.reduce((n,r)=>n+(r.total_cost??0),0);
|
||||
const tileHtml=(name,value,note)=>`<div class="co-tile" role="group" aria-label="${name}"><div class="kpi-lbl">${name}</div><div class="kpi-val">${value}</div><div class="kpi-lbl">${esc(note)}</div></div>`;
|
||||
async function loadRoi(){
|
||||
let d; try{ d=await ai('GET','/roi'); }catch(e){ $('co-source').textContent=`Can't load costs: ${e.message}`; return; }
|
||||
roi=d.runs; const p=d.pricing||{};
|
||||
$('co-source').innerHTML=`Rates from ${esc(p.source)}, as of ${esc(p.as_of)}`;
|
||||
renderCost();
|
||||
}
|
||||
function renderCost(){
|
||||
const priced=roi.filter(r=>r.total_cost!=null), thisMonth=new Date().toISOString().slice(0,7);
|
||||
const tok=k=>priced.reduce((n,r)=>n+(r[k]??0),0);
|
||||
$('co-cover').textContent=`${priced.length} of ${roi.length} documents priced`;
|
||||
$('co-cover').className='pill '+(priced.length===roi.length?'success':'attention');
|
||||
$('co-tiles').innerHTML=tileHtml('Total spend',money(sum(priced)),`${priced.length} priced runs`)
|
||||
+tileHtml('This month',money(sum(roi.filter(r=>monthOf(r.created_at)===thisMonth))),monthLabel(thisMonth))
|
||||
+tileHtml('Tokens',num(tok('input_tokens')+tok('output_tokens')+tok('thinking_tokens')),`in ${num(tok('input_tokens'))} · out ${num(tok('output_tokens'))} · thinking ${num(tok('thinking_tokens'))}`)
|
||||
+tileHtml('Per document',priced.length?money(sum(priced)/priced.length):'—','average, priced runs');
|
||||
$('co-months').innerHTML=[...new Set(roi.map(r=>monthOf(r.created_at)))].sort().reverse().map(m=>{
|
||||
const rs=roi.filter(r=>monthOf(r.created_at)===m), n=rs.filter(r=>r.total_cost!=null).length;
|
||||
return `<tr><td>${monthLabel(m)}${m===thisMonth?' <span class="rv-muted">this month</span>':''}</td><td>${n} of ${rs.length}</td><td class="co-r">${n?money(sum(rs)):'—'}</td></tr>`; }).join('');
|
||||
$('co-runs').innerHTML=roi.map(r=>`<tr><td>${esc(r.document)}</td><td>${esc(r.created_at.slice(0,10))}</td><td>${esc(r.model)}</td>
|
||||
<td class="co-r">${num(r.input_tokens)}</td><td class="co-r">${num(r.output_tokens)}</td><td class="co-r">${num(r.thinking_tokens)}</td>
|
||||
<td class="co-r">${r.total_cost==null?'<span class="rv-muted">not priced</span>':money(r.total_cost)}</td></tr>`).join('');
|
||||
renderByHand();
|
||||
}
|
||||
function renderByHand(){
|
||||
const n=roi.filter(r=>r.total_cost!=null).length, min=+$('co-min').value||0, rate=+$('co-rate').value||0;
|
||||
$('co-hand').innerHTML=`By hand, ${n} documents would take ${num(Math.round(n*min/6)/10)} h, about <b>${money(n*min/60*rate)}</b>. The AI spent <b>${money(sum(roi))}</b> on them.`;
|
||||
}
|
||||
$('co-min').oninput=$('co-rate').oninput=renderByHand;
|
||||
})();
|
||||
</script>
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Confidence-band classification — single source of truth for the cut points.
|
||||
|
||||
The workbook colours and the review queue MUST agree about what counts as red.
|
||||
If each applied its own threshold they would drift the first time one is tuned,
|
||||
and a reviewer would see a red cell the queue never asked them to check.
|
||||
Everything that needs a band imports from here.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Optional
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cut points
|
||||
# ---------------------------------------------------------------------------
|
||||
# Calibrated against the measured distribution over all 225 extracted rows of
|
||||
# the six sample documents:
|
||||
#
|
||||
# 1.00 69.8% crisp print
|
||||
# 0.90-0.99 18.2% crisp print
|
||||
# 0.80-0.89 11.0% almost entirely handwritten CPT and modifiers
|
||||
# 0.70-0.79 0.0% <- empty; the natural gap in this data
|
||||
# <0.70 1.1% handwritten modifiers
|
||||
#
|
||||
# 0.80 left only 1.1% of cells red so nearly everything went green and the
|
||||
# colour carried no information. 0.90 sits in the empty band and separates
|
||||
# the handwriting cluster from printed text.
|
||||
CONFIDENCE_THRESHOLD: float = 0.90
|
||||
|
||||
# Above CONFIDENCE_THRESHOLD the 0.90-0.99 band is still 18.2% of cells, and
|
||||
# routing treats it differently from a clean 1.00: reviewers sample it rather
|
||||
# than either trusting it outright or checking every one. A second cut at
|
||||
# 0.95 names that band so the sheet shows what the routing rule already assumes.
|
||||
#
|
||||
# GREEN >= 0.95 clean print, straight through
|
||||
# AMBER 0.90-0.95 slightly unclear, sampled
|
||||
# RED < 0.90 handwriting cluster, always reviewed
|
||||
GREEN_THRESHOLD: float = 0.95
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Band name constants
|
||||
# ---------------------------------------------------------------------------
|
||||
BAND_GREEN = "green"
|
||||
BAND_AMBER = "amber"
|
||||
BAND_RED = "red"
|
||||
|
||||
|
||||
def band_of(score: Optional[float]) -> Optional[str]:
|
||||
"""Return the confidence band name, or ``None`` when the value was never scored.
|
||||
|
||||
WHAT THE BAND CLAIMS: green means the source text this value came from was
|
||||
transcribed cleanly. It does NOT mean the value is correct. The score is
|
||||
the model's own legibility rating, so a confident misread comes back scored
|
||||
high. Verifying correctness needs a second, independent read. Red is the
|
||||
useful direction: it reliably surfaces handwriting.
|
||||
|
||||
Amber does not mean "probably fine". It means the reading was good enough to
|
||||
escape the handwriting cluster but not clean enough to trust unsampled, which
|
||||
is a routing decision, not a quality claim.
|
||||
"""
|
||||
if score is None:
|
||||
return None
|
||||
if score >= GREEN_THRESHOLD:
|
||||
return BAND_GREEN
|
||||
return BAND_AMBER if score >= CONFIDENCE_THRESHOLD else BAND_RED
|
||||
@@ -1,3 +1,4 @@
|
||||
# EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project.
|
||||
INCLUDE_THINKING_IN_COST = True # thinking tokens bill at output rate
|
||||
|
||||
# ═════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
<!-- EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. -->
|
||||
---
|
||||
name: Superbill Extraction and Automation Tool
|
||||
description: Extract billing fields from a scan, verify them against the page, post to eClinicalWorks.
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import type {
|
||||
AuditEntry, Batch, BatchStage, DemographyRecord, DemographyRun, InsuranceMasterRow, LastRun,
|
||||
ManagedUser,
|
||||
@@ -84,7 +85,7 @@ export const api = {
|
||||
for (const f of files) body.append('files', f)
|
||||
// `project` is which board received the drop. Nothing has detected a
|
||||
// template yet, and the board a batch appears on is derived from that, so
|
||||
// without it a Gulf Coast upload surfaces under ECW and reads as a failed
|
||||
// without it a Example Client upload surfaces under ECW and reads as a failed
|
||||
// upload on the Tebra board.
|
||||
return req<Batch>(`/api/batches?${new URLSearchParams({ model, project })}`, {
|
||||
method: 'POST',
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
/**
|
||||
* The confidence bands, and the exact colours the workbook paints.
|
||||
*
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { ThemeSwitcher } from '../theme'
|
||||
import { ROLE_LABEL } from '../auth'
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
/**
|
||||
* A proportional ring.
|
||||
*
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import { api } from '../api'
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
|
||||
import { api } from '../api'
|
||||
@@ -12,7 +13,7 @@ const isOfficeLocation = (t: string | null) =>
|
||||
|
||||
// One review card per record on a page. An ECW roster can list the same
|
||||
// person twice (two Sleep Study encounters for one patient) — line_index
|
||||
// splits those into separate cards. A Gulf Coast ticket spreads its charges
|
||||
// splits those into separate cards. A Example Client ticket spreads its charges
|
||||
// over several line_index values and must stay ONE card, so it is excluded.
|
||||
// The name stays in the key so runs stored before line_index existed (every
|
||||
// row LineIndex 0) still group exactly as before.
|
||||
@@ -100,7 +101,7 @@ type Patient = {
|
||||
type SaveArgs = { row: ReviewRow; draft: string; verified: boolean }
|
||||
|
||||
// ── Office Location ────────────────────────────────────────────────────────
|
||||
// A Gulf Coast ticket whose yellow note the Office Location rules read. The
|
||||
// A Example Client ticket whose yellow note the Office Location rules read. The
|
||||
// backend stores the parsed note on the ticket's first line under this field
|
||||
// (src/superbill/extraction/office_location.py, ParsedNote); a ticket without
|
||||
// it is a run from before the rules existed and keeps the flat layout.
|
||||
@@ -191,11 +192,11 @@ export function ReviewPane({
|
||||
}) {
|
||||
const [index, setIndex] = useState(0)
|
||||
// Drafts live here, not in FieldRow — a correction is no longer committed on
|
||||
// blur (that is exactly how four ARMSTRONG fields got saved mid-keystroke:
|
||||
// "ARMSTRONG, CHARLES T" as "\ONG, CHARLES T", "99291" as "992"). Text typed
|
||||
// blur (that is exactly how four fields of one record got saved mid-keystroke:
|
||||
// "Doe, Jane A" as "\e, Jane A", "99291" as "992"). Text typed
|
||||
// into a field is held until an explicit Save, keyed by row_id rather than
|
||||
// bare field name — a bare name repeats once a page can carry more than one
|
||||
// row per field (Gulf Coast's repeated CPT lines), and DemographyTab.tsx
|
||||
// row per field (Example Client's repeated CPT lines), and DemographyTab.tsx
|
||||
// hit exactly this before switching its own drafts to row_id.
|
||||
const [drafts, setDrafts] = useState<Record<number, string>>({})
|
||||
const qc = useQueryClient()
|
||||
@@ -238,7 +239,7 @@ export function ReviewPane({
|
||||
|
||||
// PDF order: page, then where the patient sits on it. line_index is each
|
||||
// record's position on its page, set by build_field_rows. Sorting by name
|
||||
// here used to put ANDERSON above the first patient on the sheet, so the
|
||||
// here used to put a later surname above the first patient on the sheet, so the
|
||||
// list no longer matched the document the reviewer had open beside it.
|
||||
// Name is only the tie-break, for runs stored before line_index existed.
|
||||
return [...byKey.values()].sort(
|
||||
@@ -254,7 +255,7 @@ export function ReviewPane({
|
||||
map.get(key)!.push(row)
|
||||
}
|
||||
for (const rows of map.values()) {
|
||||
// Gulf Coast (Template 7) records keep the order the API sent — the flat
|
||||
// Example Client (Template 7) records keep the order the API sent — the flat
|
||||
// 20-column superbill layout, Ticket# through Units. Re-sorting by band
|
||||
// would scatter it (that is what put ICD11 above ICD2 and CPT at the
|
||||
// bottom). Every other template keeps worst-band-last, which is what the
|
||||
@@ -280,7 +281,7 @@ export function ReviewPane({
|
||||
// A field that was never queued (needs_review === false) has no review
|
||||
// row until this moment on most templates — build_field_rows only emits
|
||||
// a row when a value exists or the field is billing-critical, so a
|
||||
// genuinely blank optional field (e.g. Gulf Coast's From DOS / To DOS /
|
||||
// genuinely blank optional field (e.g. Example Client's From DOS / To DOS /
|
||||
// Units, always null today) is addressed by the cell it occupies,
|
||||
// line_index included, rather than by row_id.
|
||||
if (row.needs_review === false) {
|
||||
@@ -363,7 +364,7 @@ export function ReviewPane({
|
||||
const discardAll = () => setDrafts(settledMap)
|
||||
|
||||
// Leaving a patient with typed-but-unsaved edits used to be silent — the old
|
||||
// blur handler saved automatically, which is what produced the ARMSTRONG
|
||||
// blur handler saved automatically, which is what produced the truncated-name
|
||||
// corruption. Now nothing commits without Save, so leaving instead has to
|
||||
// ask, or the edit is just lost with no signal either way.
|
||||
const guardLeave = (): boolean => {
|
||||
@@ -711,7 +712,7 @@ function PatientView({
|
||||
<div className="flex flex-col gap-[9px]">
|
||||
{/* Keyed on row_id — a real database identity from /all-fields
|
||||
(SPB_Extraction.RowId), unique across every field on every page.
|
||||
A page-and-field key would collide on Gulf Coast's repeated CPT
|
||||
A page-and-field key would collide on Example Client's repeated CPT
|
||||
lines, where two rows share the same (page, field). */}
|
||||
{/* No entry animation. Rows are keyed by row_id, so every Next
|
||||
remounts them all, and the old staggered fade (opacity 0 over
|
||||
@@ -802,7 +803,7 @@ function PatientView({
|
||||
* lost by making it editable.
|
||||
*
|
||||
* Nothing here commits on its own. An earlier version saved on blur, which is
|
||||
* exactly how "ARMSTRONG, CHARLES T" got recorded as "\ONG, CHARLES T" — the
|
||||
* exactly how "Doe, Jane A" got recorded as "\e, Jane A" — the
|
||||
* moment focus left the box mid-keystroke, whatever half-typed text was there
|
||||
* became the correction. Typing only updates the shared draft in ReviewPane;
|
||||
* the patient-level Save bar (here and in PdfPane) is the one place an edit is
|
||||
@@ -835,7 +836,7 @@ function FieldRow({
|
||||
const corrected = row.corrected_value != null && row.corrected_value !== row.value
|
||||
// While being edited the value takes its own full-width line. A DX field is
|
||||
// five comma-separated codes and a provider name runs past twenty
|
||||
// characters; in the shared row they were truncated to "ARMSTRONG, C", so a
|
||||
// characters; in the shared row they were truncated to "Doe, J", so a
|
||||
// reviewer was correcting a value they could not fully see. Expanding only
|
||||
// on focus keeps the scan-many-fields-fast layout for reading.
|
||||
const expanded = focused || dirty
|
||||
@@ -938,7 +939,7 @@ function FieldRow({
|
||||
|
||||
|
||||
/**
|
||||
* A Gulf Coast ticket laid out the way its yellow note is: the note itself,
|
||||
* A Example Client ticket laid out the way its yellow note is: the note itself,
|
||||
* the page-level rules it fired, the visit, then one tab per encounter.
|
||||
*
|
||||
* Hovering a field lights the note line it came from. The page preview is an
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { useMemo, useState } from 'react'
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import { api } from '../api'
|
||||
@@ -95,7 +96,7 @@ type Filter = 'all' | 'priced' | 'unmeasured' | 'failed'
|
||||
|
||||
/**
|
||||
* api.roi() has no project filter — it totals every document. The ROI board
|
||||
* is per-project (Gulf Coast's spend is not eClinicalWorks' spend), so the
|
||||
* is per-project (Example Client's spend is not eClinicalWorks' spend), so the
|
||||
* totals are recomputed here from the already-fetched runs instead of a new
|
||||
* query param. Every field is a plain sum or count over RoiRun — nothing
|
||||
* server-side is being second-guessed, just re-scoped to fewer rows.
|
||||
@@ -138,7 +139,7 @@ export function RoiTab({
|
||||
}: {
|
||||
/** Absent on the demography board: its runs have no charge review to open. */
|
||||
onOpenRun?: (jobId: string) => void
|
||||
/** Which project's spend this board reports — Gulf Coast tickets and
|
||||
/** Which project's spend this board reports — Example Client tickets and
|
||||
* eClinicalWorks documents never share a total. */
|
||||
project: Project
|
||||
/** The Demography ROI (ticket 05): its own payload, one Gemini call per PDF
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
|
||||
import { api } from '../api'
|
||||
import { IconDownload } from './icons'
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { useState } from 'react'
|
||||
import type { ReactElement } from 'react'
|
||||
import type { BatchStage, BoardTab } from '../types'
|
||||
@@ -14,7 +15,7 @@ import { IconSpend, IconPage, IconLoop, IconIdCard, IconPatients } from './icons
|
||||
* icons are" step from every navigation.
|
||||
*
|
||||
* Two projects, one rail: ECW's boards (templates 1-6) and Tebra's (template 7
|
||||
* / Gulf Coast), each under its own group header. See project.ts for how a
|
||||
* / Example Client), each under its own group header. See project.ts for how a
|
||||
* batch or run is assigned to one or the other.
|
||||
*
|
||||
* Selection and drag-and-drop-to-move-stage behaviour are unchanged — only the
|
||||
@@ -136,7 +137,7 @@ export function Sidebar({
|
||||
tab="tebra-roi"
|
||||
icon={IconSpend}
|
||||
label="ROI Dashboard"
|
||||
hint="What the Gulf Coast pipeline has cost, and what that bought"
|
||||
hint="What the Example Client pipeline has cost, and what that bought"
|
||||
active={tab === 'tebra-roi'}
|
||||
onSelect={onTab}
|
||||
/>
|
||||
@@ -146,8 +147,8 @@ export function Sidebar({
|
||||
dropStage="extraction"
|
||||
dropTab="tebra-extraction"
|
||||
icon={IconPage}
|
||||
label="Gulf Coast Charge Extraction"
|
||||
hint="Gulf Coast tickets: extract and clear the flagged fields"
|
||||
label="Example Client Charge Extraction"
|
||||
hint="Example Client tickets: extract and clear the flagged fields"
|
||||
active={tab === 'tebra-extraction'}
|
||||
n={counts.tebra.extraction}
|
||||
onSelect={onTab}
|
||||
@@ -159,8 +160,8 @@ export function Sidebar({
|
||||
dropStage="automation"
|
||||
dropTab="tebra-automation"
|
||||
icon={IconLoop}
|
||||
label="Gulf Coast Charge Automation"
|
||||
hint="Gulf Coast batches that finished extraction and are ready to post"
|
||||
label="Example Client Charge Automation"
|
||||
hint="Example Client batches that finished extraction and are ready to post"
|
||||
active={tab === 'tebra-automation'}
|
||||
n={counts.tebra.automation}
|
||||
onSelect={onTab}
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
|
||||
const KEY = 'superbill.split'
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { BAND_VAR } from '../bands'
|
||||
import type { Run } from '../types'
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { useEffect, useLayoutEffect, useRef, useState } from 'react'
|
||||
import { createPortal } from 'react-dom'
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
/**
|
||||
* The icon set.
|
||||
*
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
@import 'tailwindcss';
|
||||
|
||||
/* Manual dark mode: Tailwind v4 defaults `dark:` to the OS preference, but this
|
||||
@@ -1662,7 +1663,7 @@ textarea,
|
||||
.sb-brand-name { background-position: 0 0; }
|
||||
}
|
||||
|
||||
/* ── Office Location (a Gulf Coast ticket's yellow note) ──────────────────────
|
||||
/* ── Office Location (a Example Client ticket's yellow note) ──────────────────────
|
||||
The note is amber because it IS amber on the page; everything else stays on
|
||||
the neutral surfaces. No brand ramp here -- it is reserved for its six
|
||||
things (DESIGN.md, "The One Loud Colour Rule"). */
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
<!doctype html>
|
||||
<!-- EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. -->
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import { createContext, useContext, useEffect, useState } from 'react'
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
/* EXAMPLE ONLY: an extract from the Superbill Review prototype, kept as reference for porting. ECW/eClinicalWorks, Tebra, 'Example Client', SPB_* tables and patient/ticket wording are that project's examples. Don't copy them into a new project. */
|
||||
import type { Band } from './bands'
|
||||
|
||||
/**
|
||||
@@ -21,7 +22,7 @@ export type Project = 'ecw' | 'tebra'
|
||||
* stage a card can be dropped into.
|
||||
*
|
||||
* Two projects live in one rail: the ECW boards (templates 1-6) and the Tebra
|
||||
* boards (template 7 / Gulf Coast). Tebra ids are prefixed because a bare
|
||||
* boards (template 7 / Example Client). Tebra ids are prefixed because a bare
|
||||
* 'extraction' can no longer identify a board on its own — see project.ts for
|
||||
* the tab -> (project, stage) mapping.
|
||||
*/
|
||||
@@ -118,7 +119,7 @@ export interface ReviewRow {
|
||||
patient: string | null
|
||||
/**
|
||||
* 0 for every template except one that can carry more than one row per
|
||||
* (page, patient, field) — Gulf Coast's repeated CPT lines, one per coder
|
||||
* (page, patient, field) — Example Client's repeated CPT lines, one per coder
|
||||
* annotation on the same ticket.
|
||||
*/
|
||||
line_index: number
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Confidence bands: the single source of truth for the cut points.
|
||||
|
||||
Copy this file into the backend. Everything that needs a band (the review queue,
|
||||
an exported sheet, the approve gate) imports from here, so they never disagree
|
||||
about what counts as red.
|
||||
|
||||
GREEN >= 0.95 clean print, straight through
|
||||
AMBER 0.90-0.95 slightly unclear, sampled
|
||||
RED < 0.90 handwriting or a messy scan, always reviewed
|
||||
BLANK a mandatory field with no value, always flagged
|
||||
UNSCORED no word scores came back for the value
|
||||
|
||||
Calibration (from the prototype these rules came from): crisp print scored 0.90 and
|
||||
up, handwriting clustered in 0.80-0.89, and 0.70-0.79 was empty. 0.90 sits in that
|
||||
gap. 0.95 splits off the 0.90-0.99 band that reviewers sample rather than trust.
|
||||
|
||||
What a band claims: the score is the model's own legibility rating. Green means the
|
||||
text was read cleanly, NOT that the value is correct; a confident misread scores
|
||||
green. Red is the useful direction: it reliably surfaces handwriting.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Iterable, Optional
|
||||
|
||||
GREEN_THRESHOLD = 0.95
|
||||
CONFIDENCE_THRESHOLD = 0.90
|
||||
BANDS = ("green", "amber", "red", "blank", "unscored")
|
||||
|
||||
|
||||
def band_of(score: float) -> str:
|
||||
if score >= GREEN_THRESHOLD:
|
||||
return "green"
|
||||
return "amber" if score >= CONFIDENCE_THRESHOLD else "red"
|
||||
|
||||
|
||||
def field_band(word_scores: Optional[Iterable[float]], value: Optional[str], mandatory: bool = False) -> str:
|
||||
"""A field's band. It takes its WORST word score, so one shaky word can't hide in
|
||||
a clean-looking field. A blank mandatory field is flagged whatever its score."""
|
||||
if mandatory and not (value or "").strip():
|
||||
return "blank"
|
||||
scores = list(word_scores or [])
|
||||
return band_of(min(scores)) if scores else "unscored"
|
||||
|
||||
|
||||
def needs_review(band: str) -> bool:
|
||||
"""Every band but green must be verified by a person before the run can be approved."""
|
||||
return band != "green"
|
||||
@@ -0,0 +1,27 @@
|
||||
<!doctype html>
|
||||
<!--
|
||||
AI RPA dashboard (React + Vite + Tailwind). Everything on the no-AI page, plus
|
||||
confidence bands, the review split pane and the approve gate.
|
||||
Fill every {{...}} placeholder in the CONFIG block; they mean the same as on the
|
||||
no-AI page. Theme = the body / body.dark token blocks in src/index.css only.
|
||||
-->
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>{{APP_NAME}}</title>
|
||||
<script>
|
||||
// ===== CONFIG (fill per project) =====
|
||||
const APP_NAME = "{{APP_NAME}}";
|
||||
const APP_SUBTITLE = "{{APP_SUBTITLE}}";
|
||||
const LOGO_SRC = "{{LOGO_SRC}}";
|
||||
const STEPS = {{STEPS_JSON}}; // e.g. ["Initializing","Extract","Review","Completed"]
|
||||
const REPORT_COLUMNS = {{REPORT_COLUMNS_JSON}}; // must include {"key":"status",...}
|
||||
const API_BASE = "{{API_BASE}}"; // e.g. "/api"
|
||||
</script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="/src/main.tsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
+2030
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "rpa-ai-dashboard",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "tsc --noEmit && vite build"
|
||||
},
|
||||
"dependencies": {
|
||||
"react": "^19.3.1",
|
||||
"react-dom": "^19.3.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@tailwindcss/vite": "^4.3.3",
|
||||
"@types/react": "^19.2.0",
|
||||
"@types/react-dom": "^19.2.0",
|
||||
"@vitejs/plugin-react": "^6.1.2",
|
||||
"tailwindcss": "^4.3.3",
|
||||
"typescript": "^7.0.2",
|
||||
"vite": "^8.3.4"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Pricing table and cost per call for the cost page. The backend imports it.
|
||||
|
||||
Rates are USD per million tokens. Thinking tokens bill at the output rate. A model
|
||||
with "long_threshold" switches both rates to its *_long rates when a call's input
|
||||
tokens go above that threshold.
|
||||
|
||||
Replace the table with the rates of the provider and models the project really uses,
|
||||
copied from the provider's pricing page, and update PRICING_SOURCE and PRICING_AS_OF.
|
||||
Never guess a rate: a model missing here is an error, never a cost of zero.
|
||||
"""
|
||||
|
||||
PRICING_SOURCE = "https://ai.google.dev/gemini-api/docs/pricing"
|
||||
PRICING_AS_OF = "2026-08-11"
|
||||
|
||||
PRICING = {
|
||||
"gemini-2.5-pro": {"input": 1.25, "output": 10.00,
|
||||
"long_threshold": 200_000, "input_long": 2.50, "output_long": 15.00},
|
||||
"gemini-2.5-flash": {"input": 0.30, "output": 2.50},
|
||||
"gemini-2.5-flash-lite": {"input": 0.10, "output": 0.40},
|
||||
}
|
||||
|
||||
|
||||
def cost_for_call(model, input_tokens, output_tokens, thinking_tokens=0):
|
||||
"""USD for one call. Raises ValueError, naming the known models, for a model not in PRICING."""
|
||||
if model not in PRICING:
|
||||
raise ValueError(f"No pricing for model '{model}'. Add it to PRICING from {PRICING_SOURCE}. "
|
||||
f"Known models: {', '.join(PRICING)}")
|
||||
p = PRICING[model]
|
||||
in_rate, out_rate = p["input"], p["output"]
|
||||
if "long_threshold" in p and input_tokens > p["long_threshold"]:
|
||||
in_rate, out_rate = p["input_long"], p["output_long"]
|
||||
return (input_tokens * in_rate + (output_tokens + thinking_tokens) * out_rate) / 1_000_000
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Sample-data backend for the AI review dashboard: runs, review rows and the approve gate.
|
||||
|
||||
npm run build && python server_stub.py # page + API on http://127.0.0.1:8000
|
||||
python server_stub.py & npm run dev # or Vite on :5173, proxying /api here
|
||||
|
||||
It stands in for the project's backend until that exists. A real backend serves the
|
||||
same endpoints (templates/contract.md) with the same rules: bands from bands.py, and
|
||||
approval refused while any flagged field is unverified. The page never decides that.
|
||||
Standard library only.
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from html import escape
|
||||
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from bands import BANDS, field_band, needs_review
|
||||
from pricing import PRICING_AS_OF, PRICING_SOURCE, cost_for_call
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
API = "/api" # same as API_BASE in index.html
|
||||
REASON = {"green": None, "amber": "Slightly unclear, sample check", "red": "Unclear print, check every word",
|
||||
"blank": "Required field is empty", "unscored": "Not found in the transcription"}
|
||||
# (field, value, word scores, mandatory). Swap in the fields the project's extraction returns.
|
||||
SAMPLE_FIELDS = [
|
||||
("Invoice number", "INV-1001", [1.0], True),
|
||||
("Invoice date", "2026-10-01", [0.99, 0.97], True),
|
||||
("Vendor", "Contoso Supplies", [0.98, 0.92], True),
|
||||
("PO number", "PO-4471", [0.86], False),
|
||||
("Due date", "", [], True),
|
||||
("Total", "2,450.00", [0.99], True),
|
||||
("Tax", "196.00", [], False),
|
||||
]
|
||||
RUNS, ROWS = {}, {}
|
||||
ROI_RUNS = [] # [{job_id, document, created_at, model, input_tokens, output_tokens, thinking_tokens}]
|
||||
_next_row = [1]
|
||||
|
||||
|
||||
def put_run(job_id, document, fields, status="done", pages_done=1, pages_total=1, model="sample-model"):
|
||||
"""Add or replace a run. fields: [(field, value, word scores, mandatory)]."""
|
||||
RUNS[job_id] = {"job_id": job_id, "document": document, "status": status, "model": model,
|
||||
"pages_done": pages_done, "pages_total": pages_total, "approved": False, "error": None}
|
||||
ROWS[job_id] = []
|
||||
for field, value, words, mandatory in fields:
|
||||
band = field_band(words, value, mandatory)
|
||||
ROWS[job_id].append({"row_id": _next_row[0], "field": field, "value": value, "band": band,
|
||||
"score": min(words) if words else None, "reason": REASON[band],
|
||||
"verified": False, "corrected_value": None})
|
||||
_next_row[0] += 1
|
||||
return RUNS[job_id]
|
||||
|
||||
|
||||
def reset():
|
||||
RUNS.clear()
|
||||
ROWS.clear()
|
||||
put_run("job-1002", "invoice-1002.pdf", SAMPLE_FIELDS, status="running", pages_done=1, pages_total=3)
|
||||
put_run("job-1001", "invoice-1001.pdf", SAMPLE_FIELDS, pages_total=2, pages_done=2)
|
||||
ROI_RUNS.clear()
|
||||
now = time.time()
|
||||
for i, (model, tokens) in enumerate([("gemini-2.5-flash", (21_000, 1_800, 900)),
|
||||
("gemini-2.5-flash", (18_500, 1_600, 0)),
|
||||
("gemini-2.5-pro", (240_000, 3_000, 2_500)),
|
||||
("gemini-2.5-flash-lite", (22_000, 1_700, 0)),
|
||||
("gemini-2.5-flash", (None, None, None))]): # usage not recorded
|
||||
ROI_RUNS.append({"job_id": f"job-{900 + i}", "document": f"invoice-{900 + i}.pdf",
|
||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(now - i * 12 * 864e2)),
|
||||
"model": model, **dict(zip(("input_tokens", "output_tokens", "thinking_tokens"), tokens))})
|
||||
|
||||
|
||||
reset() # sample data from import on: the desktop bridge imports this module without a server
|
||||
|
||||
|
||||
def run_view(job_id):
|
||||
run, rows = RUNS[job_id], ROWS[job_id]
|
||||
done = run["status"] == "done"
|
||||
return {**run, "summary": {b: sum(r["band"] == b for r in rows) for b in BANDS} if done else None}
|
||||
|
||||
|
||||
def unverified(job_id):
|
||||
return [r for r in ROWS[job_id] if needs_review(r["band"]) and not r["verified"]]
|
||||
|
||||
|
||||
# ---- routes: (method, path regex, handler(match groups..., body) -> (status, payload)) ----
|
||||
def list_runs(body):
|
||||
return 200, {"runs": [run_view(j) for j in RUNS]}
|
||||
|
||||
|
||||
def get_run(job_id, body):
|
||||
return 200, run_view(job_id)
|
||||
|
||||
|
||||
def review(job_id, body):
|
||||
return 200, {"rows": ROWS[job_id] if RUNS[job_id]["status"] == "done" else []}
|
||||
|
||||
|
||||
def patch_row(row_id, body):
|
||||
row = next(r for rows in ROWS.values() for r in rows if r["row_id"] == int(row_id))
|
||||
row.update({k: body[k] for k in ("corrected_value", "verified") if k in body})
|
||||
return 200, row
|
||||
|
||||
|
||||
def verify_remaining(job_id, body):
|
||||
left = unverified(job_id)
|
||||
for r in left:
|
||||
r["verified"] = True
|
||||
return 200, {"verified": len(left)}
|
||||
|
||||
|
||||
def approve(job_id, body):
|
||||
left = unverified(job_id)
|
||||
if RUNS[job_id]["status"] != "done":
|
||||
return 409, {"detail": "The run hasn't finished extracting."}
|
||||
if left: # the approve gate
|
||||
return 409, {"detail": f"{len(left)} flagged field(s) still need verifying."}
|
||||
RUNS[job_id]["approved"] = True
|
||||
return 200, {"ok": True}
|
||||
|
||||
|
||||
def roi(body):
|
||||
"""Cost per run. total_cost is null when the run recorded no token usage: unpriced, never zero.
|
||||
An unknown model is an error (500 naming the known models), never a cost of zero."""
|
||||
try:
|
||||
runs = [{**r, "total_cost": None if r["input_tokens"] is None else
|
||||
cost_for_call(r["model"], r["input_tokens"], r["output_tokens"], r["thinking_tokens"] or 0)}
|
||||
for r in ROI_RUNS]
|
||||
except ValueError as e:
|
||||
return 500, {"detail": str(e)}
|
||||
return 200, {"runs": runs, "pricing": {"source": PRICING_SOURCE, "as_of": PRICING_AS_OF}}
|
||||
|
||||
|
||||
def document(job_id, body):
|
||||
"""A stand-in for the source document. A real backend returns the PDF or page image."""
|
||||
lines = "".join(f'<text x="48" y="{150 + i * 46}" font-size="20">{escape(f)}: {escape(v)}</text>'
|
||||
for i, (f, v, *_) in enumerate(SAMPLE_FIELDS))
|
||||
svg = (f'<svg xmlns="http://www.w3.org/2000/svg" width="640" height="820" font-family="Georgia,serif">'
|
||||
f'<rect width="100%" height="100%" fill="#fff"/><text x="48" y="80" font-size="28">'
|
||||
f'{escape(RUNS[job_id]["document"])} (sample)</text>{lines}</svg>')
|
||||
return 200, svg
|
||||
|
||||
|
||||
ROUTES = [
|
||||
("GET", r"/runs", list_runs),
|
||||
("GET", r"/runs/([\w-]+)", get_run),
|
||||
("GET", r"/runs/([\w-]+)/review", review),
|
||||
("GET", r"/runs/([\w-]+)/document", document),
|
||||
("PATCH", r"/review/(\d+)", patch_row),
|
||||
("POST", r"/runs/([\w-]+)/verify-remaining", verify_remaining),
|
||||
("POST", r"/runs/([\w-]+)/approve", approve),
|
||||
("GET", r"/roi", roi),
|
||||
("POST", r"/set-dark", lambda body: (200, {})),
|
||||
]
|
||||
|
||||
|
||||
def dispatch(method, path, body):
|
||||
"""One API request -> (status, payload). The web handler and the desktop bridge both call this."""
|
||||
for m, pattern, fn in ROUTES:
|
||||
match = re.fullmatch(pattern, path)
|
||||
if m == method and match:
|
||||
try:
|
||||
return fn(*match.groups(), body)
|
||||
except (KeyError, StopIteration):
|
||||
return 404, {"detail": "Not found"}
|
||||
return 404, {"detail": f"No route for {method} {path}"}
|
||||
|
||||
|
||||
class Handler(SimpleHTTPRequestHandler):
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, directory=str(HERE / "dist"), **kwargs)
|
||||
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
def do_GET(self):
|
||||
self.api("GET") if self.path.startswith(API + "/") else super().do_GET()
|
||||
|
||||
def do_POST(self):
|
||||
self.api("POST")
|
||||
|
||||
def do_PATCH(self):
|
||||
self.api("PATCH")
|
||||
|
||||
def api(self, method):
|
||||
path = urlparse(self.path).path[len(API):]
|
||||
size = int(self.headers.get("Content-Length") or 0)
|
||||
body = json.loads(self.rfile.read(size) or b"{}") if size else {}
|
||||
status, payload = dispatch(method, path, body)
|
||||
svg = isinstance(payload, str)
|
||||
data = payload.encode() if svg else json.dumps(payload).encode()
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "image/svg+xml" if svg else "application/json")
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
|
||||
def make_server(host="127.0.0.1", port=8000):
|
||||
reset()
|
||||
return ThreadingHTTPServer((host, port), Handler)
|
||||
|
||||
|
||||
def simulate_extraction():
|
||||
"""Demo only: the running sample run reads one page every 3 seconds, then finishes."""
|
||||
while True:
|
||||
time.sleep(3)
|
||||
for run in RUNS.values():
|
||||
if run["status"] == "running":
|
||||
run["pages_done"] += 1
|
||||
if run["pages_done"] >= run["pages_total"]:
|
||||
run["status"] = "done"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
server = make_server()
|
||||
threading.Thread(target=simulate_extraction, daemon=True).start()
|
||||
print("Sample backend on http://127.0.0.1:8000 (Ctrl+C to stop)")
|
||||
server.serve_forever()
|
||||
@@ -0,0 +1,108 @@
|
||||
import { useState } from 'react'
|
||||
import { act, clearRun, toggleDark, useStore } from './store'
|
||||
import { Overview } from './components/Overview'
|
||||
import { RunPanel } from './components/RunPanel'
|
||||
import { Report } from './components/Report'
|
||||
import { Review } from './components/Review'
|
||||
import { Cost } from './components/Cost'
|
||||
|
||||
const TABS = {
|
||||
dashboard: ['Dashboard', 'Overview of every batch', <><rect x="3" y="3" width="7" height="7" /><rect x="14" y="3" width="7" height="7" /><rect x="3" y="14" width="7" height="7" /><rect x="14" y="14" width="7" height="7" /></>],
|
||||
run: ['Run', 'Live progress and log', <polygon points="6 4 20 12 6 20 6 4" />],
|
||||
review: ['Review', 'Check extracted fields against the document', <><path d="M4 4h10l6 6v10H4z" /><path d="m8 14 3 3 5-6" /></>],
|
||||
report: ['Report', 'Result of every processed item', <><path d="M4 4h16v16H4z" /><path d="M8 9h8M8 13h8M8 17h5" /></>],
|
||||
cost: ['Cost', 'Tokens, spend and the case against doing it by hand', <><path d="M12 2v20" /><path d="M17 6H9.5a3.5 3.5 0 0 0 0 7h5a3.5 3.5 0 0 1 0 7H6" /></>],
|
||||
// OPTIONAL: admin/roles tab goes here
|
||||
} as const
|
||||
type Tab = keyof typeof TABS
|
||||
|
||||
const DOT = { ready: 'bg-ok', done: 'bg-ok', running: 'bg-warn animate-pulse', error: 'bg-bad' }
|
||||
|
||||
export function App() {
|
||||
const { status, user, dark } = useStore()
|
||||
const [tab, setTab] = useState<Tab>('dashboard')
|
||||
const [unsaved, setUnsaved] = useState(0) // review edits not saved yet
|
||||
|
||||
const show = (t: Tab) => {
|
||||
if (t === tab) return
|
||||
if (unsaved && !window.confirm(`${unsaved} unsaved change(s) on this document. Discard them and leave?`)) return
|
||||
setUnsaved(0)
|
||||
setTab(t)
|
||||
if (t === 'dashboard') act('load_dashboard')
|
||||
if (t === 'report') act('load_latest_report')
|
||||
}
|
||||
const run = () => {
|
||||
show('run')
|
||||
clearRun()
|
||||
act('start_automation')
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex h-screen">
|
||||
<aside className="flex w-[230px] min-w-[230px] flex-col border-r border-line bg-glass max-[820px]:w-[66px] max-[820px]:min-w-[66px]" style={{ backdropFilter: 'var(--blur)' }}>
|
||||
<div className="flex items-center gap-3 border-b border-line px-5 py-5">
|
||||
<div className="flex h-10 w-10 shrink-0 items-center justify-center rounded-xl border border-line bg-glass-strong">
|
||||
{LOGO_SRC && <img src={LOGO_SRC} alt="" className="h-7 w-7 object-contain" onError={(e) => e.currentTarget.remove()} />}
|
||||
</div>
|
||||
<div className="max-[820px]:hidden">
|
||||
<div className="font-head text-[15px] font-bold">{APP_NAME}</div>
|
||||
<div className="mt-0.5 text-[10px] font-semibold uppercase tracking-wide text-muted">{APP_SUBTITLE}</div>
|
||||
</div>
|
||||
</div>
|
||||
<nav className="flex-1 px-3 py-4">
|
||||
{(Object.keys(TABS) as Tab[]).map((t) => (
|
||||
<button
|
||||
key={t}
|
||||
title={TABS[t][0]}
|
||||
onClick={() => show(t)}
|
||||
aria-current={tab === t ? 'page' : undefined}
|
||||
className={`mb-1 flex w-full cursor-pointer items-center gap-3 rounded-[10px] px-3 py-2.5 text-left text-[13.5px] font-semibold ${tab === t ? 'text-white shadow-md' : 'text-muted hover:bg-glass-strong hover:text-ink'}`}
|
||||
style={tab === t ? { background: 'var(--nav-active-bg)' } : undefined}
|
||||
>
|
||||
<svg viewBox="0 0 24 24" className="h-[18px] w-[18px] shrink-0 fill-none stroke-current stroke-2">{TABS[t][2]}</svg>
|
||||
<span className="max-[820px]:hidden">{TABS[t][0]}</span>
|
||||
</button>
|
||||
))}
|
||||
</nav>
|
||||
<div className="flex items-center gap-2.5 border-t border-line p-3.5">
|
||||
<div className="flex h-[34px] w-[34px] shrink-0 items-center justify-center rounded-full text-xs font-bold text-white" style={{ background: 'var(--nav-active-bg)' }}>
|
||||
{(user.username || 'U')[0].toUpperCase()}
|
||||
</div>
|
||||
<div className="max-[820px]:hidden">
|
||||
<div className="text-[13px] font-bold">{user.username}</div>
|
||||
<div className="text-[10.5px] text-muted">{user.role}</div>
|
||||
</div>
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<main className="flex flex-1 flex-col overflow-hidden">
|
||||
<header className="flex items-center gap-3.5 border-b border-line px-7 py-5 shadow-[0_6px_18px_rgba(20,50,70,.10)]"
|
||||
style={{ background: 'var(--header-bg)', color: 'var(--header-ink)', backdropFilter: 'var(--blur)' }}>
|
||||
<div>
|
||||
<div className="font-head text-[21px] font-extrabold">{TABS[tab][0]}</div>
|
||||
<div className="mt-0.5 text-[12.5px]" style={{ color: 'var(--header-muted)' }}>{TABS[tab][1]}</div>
|
||||
</div>
|
||||
<div className="ml-auto flex items-center gap-3">
|
||||
<button className="btn btn-primary" onClick={run} disabled={status.state === 'running'}>▶ Run</button>
|
||||
<button className="h-[38px] w-[38px] cursor-pointer rounded-[11px] border border-line bg-glass-strong text-base text-ink"
|
||||
title="Dark mode" aria-label="Dark mode" aria-pressed={dark} onClick={toggleDark}>
|
||||
{dark ? '☀' : '🌙'}
|
||||
</button>
|
||||
<div className="flex items-center gap-2 rounded-full border border-line bg-glass-strong px-3 py-1.5 text-xs font-bold text-ink">
|
||||
<span className={`h-2 w-2 rounded-full ${DOT[status.state] ?? 'bg-muted'}`} />
|
||||
<span>{status.msg || status.state}</span>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<section className="flex min-h-0 flex-1 flex-col overflow-y-auto px-7 py-6 max-[820px]:px-4">
|
||||
{tab === 'dashboard' && <Overview />}
|
||||
{tab === 'run' && <RunPanel />}
|
||||
{tab === 'review' && <Review onUnsaved={setUnsaved} />}
|
||||
{tab === 'report' && <Report />}
|
||||
{tab === 'cost' && <Cost />}
|
||||
</section>
|
||||
</main>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
/** The AI contract additions (see contract.md): runs, review rows, the approve gate. */
|
||||
export type Band = 'green' | 'amber' | 'red' | 'blank' | 'unscored'
|
||||
export type RunStatus = 'pending' | 'queued' | 'running' | 'done' | 'failed'
|
||||
export type Run = {
|
||||
job_id: string
|
||||
document: string
|
||||
status: RunStatus
|
||||
model: string
|
||||
pages_done: number
|
||||
pages_total: number
|
||||
summary: Record<Band, number> | null // null until the run is done
|
||||
approved: boolean
|
||||
error: string | null
|
||||
}
|
||||
export type ReviewRow = {
|
||||
row_id: number
|
||||
field: string
|
||||
value: string | null
|
||||
band: Band
|
||||
score: number | null
|
||||
reason: string | null
|
||||
verified: boolean
|
||||
corrected_value: string | null
|
||||
}
|
||||
|
||||
/** One run on the cost page. total_cost is null when the run recorded no token usage (unpriced). */
|
||||
export type RoiRun = {
|
||||
job_id: string
|
||||
document: string
|
||||
created_at: string // ISO, UTC
|
||||
model: string
|
||||
input_tokens: number | null
|
||||
output_tokens: number | null
|
||||
thinking_tokens: number | null
|
||||
total_cost: number | null // USD
|
||||
}
|
||||
export type Pricing = { source: string; as_of: string }
|
||||
|
||||
export const BUSY: RunStatus[] = ['queued', 'running'] // polled every 1.5s while in one of these
|
||||
|
||||
/** Legend order, label, and the colour token. Band colours are semantic: never on buttons or headings. */
|
||||
export const BANDS: { band: Band; label: string; colour: string }[] = [
|
||||
{ band: 'green', label: 'Clean ≥0.95', colour: 'var(--clean)' },
|
||||
{ band: 'amber', label: 'Check 0.90–0.95', colour: 'var(--verify)' },
|
||||
{ band: 'red', label: 'Review <0.90', colour: 'var(--review)' },
|
||||
{ band: 'blank', label: 'Missing (required)', colour: 'var(--absent)' },
|
||||
{ band: 'unscored', label: 'Unscored', colour: 'var(--absent)' },
|
||||
]
|
||||
export const CAVEAT = 'Colour reflects print clarity, not correctness. A cleanly printed value can still be misread.'
|
||||
|
||||
async function req<T>(path: string, init?: RequestInit): Promise<T> {
|
||||
const res = await fetch(API_BASE + path, { ...init, headers: { 'Content-Type': 'application/json' } })
|
||||
if (!res.ok) {
|
||||
// The server says why it refused (the approve gate answers 409 with the count).
|
||||
const body = await res.json().catch(() => null)
|
||||
throw new Error(body?.detail ?? `${res.status} ${res.statusText}`)
|
||||
}
|
||||
return res.json()
|
||||
}
|
||||
|
||||
export const ai = {
|
||||
runs: () => req<{ runs: Run[] }>('/runs'),
|
||||
run: (id: string) => req<Run>(`/runs/${id}`),
|
||||
rows: (id: string) => req<{ rows: ReviewRow[] }>(`/runs/${id}/review`),
|
||||
patchRow: (rowId: number, body: { corrected_value?: string | null; verified: boolean }) =>
|
||||
req<ReviewRow>(`/review/${rowId}`, { method: 'PATCH', body: JSON.stringify(body) }),
|
||||
verifyRemaining: (id: string) => req<{ verified: number }>(`/runs/${id}/verify-remaining`, { method: 'POST' }),
|
||||
approve: (id: string) => req<{ ok: boolean }>(`/runs/${id}/approve`, { method: 'POST' }),
|
||||
roi: () => req<{ runs: RoiRun[]; pricing: Pricing }>('/roi'),
|
||||
documentUrl: (id: string) => `${API_BASE}/runs/${id}/document`,
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { ai, type Pricing, type RoiRun } from '../ai'
|
||||
|
||||
const money = (n: number) => `$${n.toFixed(n > 0 && n < 0.01 ? 4 : 2)}`
|
||||
const num = (n: number | null) => (n == null ? '—' : n.toLocaleString('en-US'))
|
||||
const monthOf = (iso: string) => iso.slice(0, 7) // 'YYYY-MM', UTC
|
||||
const monthLabel = (ym: string) =>
|
||||
new Date(`${ym}-01T00:00:00Z`).toLocaleString('en-US', { month: 'short', year: 'numeric', timeZone: 'UTC' })
|
||||
|
||||
/**
|
||||
* The cost page. Two kinds of number, kept apart:
|
||||
* measured: tokens and cost per run, priced by the backend's pricing table;
|
||||
* assumed: minutes per document and an hourly rate, typed by the reader and labelled so.
|
||||
* A run with no recorded token usage is "not priced", never $0, and the coverage pill says how many.
|
||||
*/
|
||||
export function Cost() {
|
||||
const [runs, setRuns] = useState<RoiRun[]>([])
|
||||
const [pricing, setPricing] = useState<Pricing>()
|
||||
const [error, setError] = useState<string>()
|
||||
const [minutes, setMinutes] = useState(5) // assumption: minutes to key one document by hand
|
||||
const [rate, setRate] = useState(20) // assumption: USD per hour of that person's time
|
||||
|
||||
useEffect(() => {
|
||||
ai.roi().then((d) => { setRuns(d.runs); setPricing(d.pricing) }).catch((e) => setError(e.message))
|
||||
}, [])
|
||||
|
||||
const priced = runs.filter((r) => r.total_cost != null)
|
||||
const sum = (rs: RoiRun[]) => rs.reduce((n, r) => n + (r.total_cost ?? 0), 0)
|
||||
const thisMonth = new Date().toISOString().slice(0, 7)
|
||||
const months = [...new Set(runs.map((r) => monthOf(r.created_at)))].sort().reverse()
|
||||
.map((m) => {
|
||||
const rs = runs.filter((r) => monthOf(r.created_at) === m)
|
||||
return { m, docs: rs.length, priced: rs.filter((r) => r.total_cost != null).length, cost: sum(rs) }
|
||||
})
|
||||
const tokens = (k: 'input_tokens' | 'output_tokens' | 'thinking_tokens') => priced.reduce((n, r) => n + (r[k] ?? 0), 0)
|
||||
const byHand = (priced.length * minutes / 60) * rate
|
||||
|
||||
if (error) return <div className="card text-bad">Can't load costs from {API_BASE}/roi: {error}</div>
|
||||
|
||||
return (
|
||||
<>
|
||||
<div className="mb-4 flex flex-wrap items-center gap-3">
|
||||
<span className={`pill ${priced.length === runs.length ? 'bg-ok' : 'bg-warn'}`}>
|
||||
{priced.length} of {runs.length} documents priced
|
||||
</span>
|
||||
{pricing && (
|
||||
<span className="text-[11.5px] text-muted">
|
||||
Rates from <a className="underline" href={pricing.source} target="_blank" rel="noreferrer">{pricing.source}</a>, as of {pricing.as_of}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="mb-4 grid grid-cols-[repeat(auto-fit,minmax(170px,1fr))] gap-4">
|
||||
<Tile name="Total spend" value={money(sum(priced))} note={`${priced.length} priced runs`} />
|
||||
<Tile name="This month" value={money(sum(runs.filter((r) => monthOf(r.created_at) === thisMonth)))} note={monthLabel(thisMonth)} />
|
||||
<Tile name="Tokens" value={num(tokens('input_tokens') + tokens('output_tokens') + tokens('thinking_tokens'))}
|
||||
note={`in ${num(tokens('input_tokens'))} · out ${num(tokens('output_tokens'))} · thinking ${num(tokens('thinking_tokens'))}`} />
|
||||
<Tile name="Per document" value={priced.length ? money(sum(priced) / priced.length) : '—'} note="average, priced runs" />
|
||||
</div>
|
||||
|
||||
<div className="card">
|
||||
<div className="card-title">Month by month</div>
|
||||
<div className="overflow-auto">
|
||||
<table aria-label="Spend by month" className="w-full border-collapse whitespace-nowrap text-xs">
|
||||
<thead><tr><Th>Month</Th><Th>Documents priced</Th><Th right>Spend</Th></tr></thead>
|
||||
<tbody>
|
||||
{months.map((x) => (
|
||||
<tr key={x.m} className="border-t border-line">
|
||||
<Td>{monthLabel(x.m)}{x.m === thisMonth && <span className="ml-2 text-muted">this month</span>}</Td>
|
||||
<Td>{x.priced} of {x.docs}</Td>
|
||||
<Td right>{x.priced ? money(x.cost) : '—'}</Td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section className="card" aria-labelledby="by-hand">
|
||||
<div id="by-hand" className="card-title">Against doing it by hand</div>
|
||||
<p className="mb-3 text-xs text-muted">These are your assumptions, not a measurement. Change them to match the team.</p>
|
||||
<div className="mb-3 flex flex-wrap gap-4 text-xs">
|
||||
<label className="flex items-center gap-2">Minutes per document
|
||||
<input className="input w-20" type="number" min="0" value={minutes} onChange={(e) => setMinutes(+e.target.value)} />
|
||||
</label>
|
||||
<label className="flex items-center gap-2">Hourly rate
|
||||
<input className="input w-20" type="number" min="0" value={rate} onChange={(e) => setRate(+e.target.value)} />
|
||||
</label>
|
||||
</div>
|
||||
<p className="text-[13px]">
|
||||
By hand, {priced.length} documents would take {num(Math.round(priced.length * minutes / 6) / 10)} h, about <b>{money(byHand)}</b>.
|
||||
The AI spent <b>{money(sum(priced))}</b> on them.
|
||||
</p>
|
||||
</section>
|
||||
|
||||
{/* OPTIONAL: model picker (per batch) goes here */}
|
||||
|
||||
<div className="card">
|
||||
<div className="card-title">Cost per run</div>
|
||||
<div className="max-h-[50vh] overflow-auto">
|
||||
<table aria-label="Cost per run" className="w-full border-collapse whitespace-nowrap text-xs">
|
||||
<thead>
|
||||
<tr><Th>Document</Th><Th>Date</Th><Th>Model</Th><Th right>Input</Th><Th right>Output</Th><Th right>Thinking</Th><Th right>Cost</Th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{runs.map((r) => (
|
||||
<tr key={r.job_id} className="border-t border-line">
|
||||
<Td>{r.document}</Td><Td>{r.created_at.slice(0, 10)}</Td><Td>{r.model}</Td>
|
||||
<Td right>{num(r.input_tokens)}</Td><Td right>{num(r.output_tokens)}</Td><Td right>{num(r.thinking_tokens)}</Td>
|
||||
<Td right>{r.total_cost == null ? <span className="text-muted">not priced</span> : money(r.total_cost)}</Td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
function Tile({ name, value, note }: { name: string; value: string; note: string }) {
|
||||
return (
|
||||
<div role="group" aria-label={name} className="card mb-0">
|
||||
<div className="text-[11px] font-bold uppercase tracking-wide text-muted">{name}</div>
|
||||
<div className="mt-1 font-head text-[26px] font-extrabold">{value}</div>
|
||||
<div className="mt-1 text-[11px] text-muted">{note}</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
const Th = ({ children, right }: { children: React.ReactNode; right?: boolean }) =>
|
||||
<th className={`px-3 py-2 font-bold ${right ? 'text-right' : 'text-left'}`}>{children}</th>
|
||||
const Td = ({ children, right }: { children: React.ReactNode; right?: boolean }) =>
|
||||
<td className={`px-3 py-2 ${right ? 'text-right' : ''}`}>{children}</td>
|
||||
@@ -0,0 +1,81 @@
|
||||
import { useStore } from '../store'
|
||||
|
||||
// batch state -> [label, colour token, pill colour]
|
||||
export const STATES: Record<string, [string, string, string]> = {
|
||||
completed: ['Completed', 'var(--ok)', 'bg-ok'],
|
||||
inprogress: ['In progress', 'var(--warn)', 'bg-warn'],
|
||||
notcompleted: ['Not completed', 'var(--bad)', 'bg-bad'],
|
||||
cleared: ['Cleared', 'var(--muted)', 'bg-muted'],
|
||||
}
|
||||
|
||||
function Kpi({ label, n, colour, live }: { label: string; n: number; colour: string; live?: boolean }) {
|
||||
return (
|
||||
<div role="group" aria-label={label} className="flex items-center gap-3 rounded-2xl border border-line bg-glass-strong px-4 py-4 shadow-sm">
|
||||
<div className={`h-[42px] w-[42px] shrink-0 rounded-xl ${live ? 'animate-pulse' : ''}`} style={{ background: colour }} />
|
||||
<div>
|
||||
<div className="font-head text-[26px] leading-tight font-extrabold tabular-nums">{n}</div>
|
||||
<div className="mt-0.5 text-[11.5px] font-semibold text-muted">{label}</div>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export function Overview() {
|
||||
const { batches } = useStore()
|
||||
const count = Object.fromEntries(Object.keys(STATES).map((k) => [k, batches.filter((b) => b.state === k).length]))
|
||||
const total = Object.values(count).reduce((a, x) => a + x, 0)
|
||||
let acc = 0
|
||||
const stops = Object.keys(STATES).flatMap((k) => {
|
||||
if (!count[k]) return []
|
||||
const from = (acc / total) * 100
|
||||
acc += count[k]
|
||||
return [`${STATES[k][1]} ${from}% ${(acc / total) * 100}%`]
|
||||
})
|
||||
|
||||
return (
|
||||
<div>
|
||||
<div className="mb-4 grid grid-cols-4 gap-3.5 max-[820px]:grid-cols-2">
|
||||
<Kpi label="Total batches" n={batches.length} colour="var(--accent)" />
|
||||
<Kpi label="Completed" n={count.completed} colour="var(--ok)" />
|
||||
<Kpi label="In progress" n={count.inprogress} colour="var(--warn)" live={count.inprogress > 0} />
|
||||
<Kpi label="Not completed" n={count.notcompleted} colour="var(--bad)" />
|
||||
</div>
|
||||
<div className="grid grid-cols-2 gap-3.5 max-[820px]:grid-cols-1">
|
||||
<div className="card">
|
||||
<div className="card-title">Batch status split</div>
|
||||
<div className="flex flex-wrap items-center gap-6">
|
||||
<div role="img" className="relative h-[150px] w-[150px] rounded-full"
|
||||
aria-label={'Batch status: ' + (total ? Object.keys(STATES).map((k) => `${STATES[k][0]} ${count[k]}`).join(', ') : 'no batches yet')}
|
||||
style={{ background: total ? `conic-gradient(${stops.join(',')})` : 'conic-gradient(var(--border) 0 100%)' }}>
|
||||
<div className="absolute inset-[22px] flex flex-col items-center justify-center rounded-full" style={{ background: 'var(--bg-bottom)' }}>
|
||||
<div className="font-head text-[26px] font-extrabold">{batches.length}</div>
|
||||
<div className="text-[10px] font-bold uppercase text-muted">batches</div>
|
||||
</div>
|
||||
</div>
|
||||
<ul aria-label="Batch status">
|
||||
{Object.keys(STATES).map((k) => (
|
||||
<li key={k} className="mb-2 flex items-center gap-2 text-[12.5px] font-semibold">
|
||||
<span className="h-2.5 w-2.5 rounded-[3px]" style={{ background: STATES[k][1] }} />
|
||||
{STATES[k][0]} · {count[k]}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<div className="card">
|
||||
<div className="card-title">Recent batches</div>
|
||||
{batches.length === 0 && <div className="py-10 text-center text-[13px] text-muted">No batches yet</div>}
|
||||
{batches.slice(0, 6).map((b) => {
|
||||
const s = STATES[b.state] ?? [b.state, '', 'bg-warn']
|
||||
return (
|
||||
<div key={b.batch_id} className="mb-2 flex items-center gap-2 text-[12.5px] font-semibold">
|
||||
#{b.batch_id} · {b.date}
|
||||
<span className={`pill ml-auto ${s[2]}`}>{s[0]}</span>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
import { useState } from 'react'
|
||||
import { useStore, type ReportRow } from '../store'
|
||||
|
||||
const PILL: Record<string, string> = { success: 'bg-ok', attention: 'bg-warn', failed: 'bg-bad' }
|
||||
|
||||
// ponytail: a CSV that Excel opens. Have the backend send a real .xlsx if the sheet needs formatting.
|
||||
function exportCsv(rows: ReportRow[]) {
|
||||
const cell = (v: unknown) => `"${String(v ?? '').replace(/"/g, '""')}"`
|
||||
const csv = [REPORT_COLUMNS.map((c) => cell(c.label)), ...rows.map((r) => REPORT_COLUMNS.map((c) => cell(r[c.key])))]
|
||||
.map((line) => line.join(',')).join('\r\n')
|
||||
const a = document.createElement('a')
|
||||
a.href = URL.createObjectURL(new Blob(['' + csv], { type: 'text/csv' }))
|
||||
a.download = 'report.csv'
|
||||
a.click()
|
||||
URL.revokeObjectURL(a.href)
|
||||
}
|
||||
|
||||
export function Report() {
|
||||
const { reportRows } = useStore()
|
||||
const [q, setQ] = useState('')
|
||||
const rows = reportRows.filter((r) => !q || Object.values(r).join(' ').toLowerCase().includes(q.toLowerCase()))
|
||||
|
||||
return (
|
||||
<div className="card">
|
||||
<div className="mb-3.5 flex flex-wrap items-center gap-2.5">
|
||||
<input type="search" aria-label="Search report" placeholder="Search…" className="input" value={q} onChange={(e) => setQ(e.target.value)} />
|
||||
<button className="btn btn-primary ml-auto" onClick={() => exportCsv(rows)}>⬇ Export to Excel</button>
|
||||
</div>
|
||||
<div className="max-h-[60vh] overflow-auto rounded-2xl border border-line bg-glass">
|
||||
<table className="w-full border-collapse whitespace-nowrap text-xs">
|
||||
<thead>
|
||||
<tr>{REPORT_COLUMNS.map((c) => <th key={c.key} className="sticky top-0 bg-glass-strong px-3.5 py-3 text-left font-bold">{c.label}</th>)}</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{rows.length === 0 && <tr><td colSpan={REPORT_COLUMNS.length} className="p-10 text-center text-[13px] text-muted">No rows yet</td></tr>}
|
||||
{rows.map((r, i) => (
|
||||
<tr key={i}>
|
||||
{REPORT_COLUMNS.map((c) => (
|
||||
<td key={c.key} className="border-b border-line px-3.5 py-2.5">
|
||||
{c.key === 'status' ? <span className={`pill ${PILL[r.bucket ?? ''] ?? PILL.attention}`}>{r.status}</span> : String(r[c.key] ?? '')}
|
||||
</td>
|
||||
))}
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { ai, BANDS, BUSY, CAVEAT, type Band, type ReviewRow, type Run } from '../ai'
|
||||
|
||||
const FILTERS: { label: string; band: Band | 'all' }[] = [
|
||||
{ label: 'All', band: 'all' }, { label: 'Red', band: 'red' }, { label: 'Amber', band: 'amber' },
|
||||
{ label: 'Missing', band: 'blank' }, { label: 'Unscored', band: 'unscored' },
|
||||
]
|
||||
const colourOf = (b: Band) => BANDS.find((x) => x.band === b)!.colour
|
||||
const settled = (r: ReviewRow) => r.corrected_value ?? r.value ?? ''
|
||||
|
||||
/**
|
||||
* The review split pane: the source document beside its extracted fields.
|
||||
* Typing makes a draft; nothing is written until Save. Verify signs off one field
|
||||
* as it reads. Approve asks the server, which refuses while flagged fields are unverified.
|
||||
*/
|
||||
export function Review({ onUnsaved }: { onUnsaved: (n: number) => void }) {
|
||||
const [runs, setRuns] = useState<Run[]>()
|
||||
const [jobId, setJobId] = useState<string>()
|
||||
const [run, setRun] = useState<Run>()
|
||||
const [rows, setRows] = useState<ReviewRow[]>([])
|
||||
const [drafts, setDrafts] = useState<Record<number, string>>({})
|
||||
const [filter, setFilter] = useState<Band | 'all'>('all')
|
||||
const [refusal, setRefusal] = useState<string>()
|
||||
const [error, setError] = useState<string>()
|
||||
const [saving, setSaving] = useState(false)
|
||||
|
||||
const dirty = rows.filter((r) => r.row_id in drafts && drafts[r.row_id] !== settled(r))
|
||||
useEffect(() => onUnsaved(dirty.length), [dirty.length, onUnsaved])
|
||||
useEffect(() => {
|
||||
if (!dirty.length) return
|
||||
const warn = (e: BeforeUnloadEvent) => e.preventDefault()
|
||||
window.addEventListener('beforeunload', warn)
|
||||
return () => window.removeEventListener('beforeunload', warn)
|
||||
}, [dirty.length])
|
||||
|
||||
useEffect(() => {
|
||||
ai.runs().then((d) => { setRuns(d.runs); setJobId(d.runs[0]?.job_id) })
|
||||
.catch((e) => setError(`Can't load runs from ${API_BASE}: ${e.message}. Is the backend (or server_stub.py) running?`))
|
||||
}, [])
|
||||
|
||||
// Load the chosen run, and poll it every 1.5s while it is still extracting.
|
||||
useEffect(() => {
|
||||
if (!jobId) return
|
||||
let alive = true
|
||||
let timer: ReturnType<typeof setTimeout>
|
||||
setRun(undefined); setRows([]); setDrafts({}); setRefusal(undefined)
|
||||
const tick = async () => {
|
||||
try {
|
||||
const r = await ai.run(jobId)
|
||||
if (!alive) return
|
||||
setRun(r)
|
||||
if (BUSY.includes(r.status)) timer = setTimeout(tick, 1500)
|
||||
else { const d = await ai.rows(jobId); if (alive) setRows(d.rows) }
|
||||
} catch (e) { if (alive) setError((e as Error).message) }
|
||||
}
|
||||
tick()
|
||||
return () => { alive = false; clearTimeout(timer) }
|
||||
}, [jobId])
|
||||
|
||||
const reload = async () => {
|
||||
if (!jobId) return
|
||||
const [r, d] = await Promise.all([ai.run(jobId), ai.rows(jobId)])
|
||||
setRun(r); setRows(d.rows)
|
||||
}
|
||||
const pick = (id: string) => {
|
||||
if (dirty.length && !window.confirm(`${dirty.length} unsaved change(s) on this document. Discard them and leave?`)) return
|
||||
setJobId(id)
|
||||
}
|
||||
const save = async () => {
|
||||
setSaving(true)
|
||||
try {
|
||||
await Promise.all(dirty.map((r) => {
|
||||
const text = drafts[r.row_id].trim()
|
||||
// Same as extracted (or emptied) means "the extracted value was right", not an edit.
|
||||
return ai.patchRow(r.row_id, { corrected_value: text === '' || text === (r.value ?? '') ? null : text, verified: true })
|
||||
}))
|
||||
setDrafts({})
|
||||
await reload()
|
||||
} finally { setSaving(false) }
|
||||
}
|
||||
const verify = async (r: ReviewRow) => { await ai.patchRow(r.row_id, { verified: !r.verified }); await reload() }
|
||||
const verifyRemaining = async () => { if (jobId) { await ai.verifyRemaining(jobId); await reload() } }
|
||||
const approve = async () => {
|
||||
if (!jobId) return
|
||||
setRefusal(undefined)
|
||||
try { await ai.approve(jobId); await reload() } catch (e) { setRefusal((e as Error).message) }
|
||||
}
|
||||
|
||||
if (error) return <div className="card text-[13px] text-bad">{error}</div>
|
||||
if (!runs) return <div className="card text-[13px] text-muted">Loading runs…</div>
|
||||
if (!runs.length) return <div className="card text-[13px] text-muted">No documents extracted yet.</div>
|
||||
|
||||
const shown = filter === 'all' ? rows : rows.filter((r) => r.band === filter)
|
||||
const left = rows.filter((r) => r.band !== 'green' && !r.verified).length
|
||||
|
||||
return (
|
||||
<div className="flex min-h-[560px] flex-1 flex-col gap-3">
|
||||
<div className="card mb-0 flex flex-wrap items-center gap-3 py-3">
|
||||
<label className="flex items-center gap-2 text-[12.5px] font-semibold">
|
||||
Document
|
||||
<select aria-label="Document" className="input" value={jobId} onChange={(e) => pick(e.target.value)}>
|
||||
{runs.map((r) => <option key={r.job_id} value={r.job_id}>{r.document}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
{run && <span className={`pill ${run.status === 'done' ? 'bg-ok' : run.status === 'failed' ? 'bg-bad' : 'bg-warn'}`}>{run.status}</span>}
|
||||
{run && BUSY.includes(run.status) && <span className="text-[12.5px] text-muted tabular-nums">page {run.pages_done} of {run.pages_total || '?'}</span>}
|
||||
{run?.error && <span className="text-[12px] text-bad">{run.error}</span>}
|
||||
{/* OPTIONAL: retry on another model (failed runs) goes here */}
|
||||
{run?.status === 'done' && (
|
||||
<div className="ml-auto flex items-center gap-2">
|
||||
<span className="text-[12px] text-muted">{left} to verify</span>
|
||||
<button className="btn btn-sm" onClick={verifyRemaining} disabled={!left || run.approved}>Verify remaining</button>
|
||||
{run.approved
|
||||
? <span className="pill bg-ok">✓ Approved</span>
|
||||
: <button className="btn btn-primary btn-sm" onClick={approve} disabled={dirty.length > 0}
|
||||
title={dirty.length ? 'Save or discard your edits first' : undefined}>Approve</button>}
|
||||
</div>
|
||||
)}
|
||||
{refusal && <p role="alert" className="w-full rounded-lg bg-bad-bg px-3 py-1.5 text-[12px] font-semibold text-bad">Not approved: {refusal}</p>}
|
||||
</div>
|
||||
|
||||
<div className="card mb-0 flex flex-wrap items-center gap-x-5 gap-y-2 py-3">
|
||||
<ul aria-label="Confidence bands" className="flex flex-wrap gap-x-4 gap-y-1">
|
||||
{BANDS.map((b) => (
|
||||
<li key={b.band} className="flex items-center gap-1.5 text-[12px] font-semibold">
|
||||
<i className="h-2.5 w-2.5 rounded-[3px]" style={{ background: b.colour }} />
|
||||
{b.label} · {run?.summary?.[b.band] ?? 0}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
<span className="text-[11.5px] text-muted">{CAVEAT}</span>
|
||||
<div role="group" aria-label="Filter by band" className="ml-auto flex gap-1">
|
||||
{FILTERS.map((f) => (
|
||||
<button key={f.band} aria-pressed={filter === f.band} onClick={() => setFilter(f.band)}
|
||||
className={`btn btn-sm ${filter === f.band ? 'btn-primary' : ''}`}>{f.label}</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex min-h-0 flex-1 gap-3 max-[820px]:flex-col">
|
||||
<iframe title="Source document" src={jobId && ai.documentUrl(jobId)}
|
||||
className="min-h-[420px] flex-1 rounded-2xl border border-line bg-white" />
|
||||
<div className="card mb-0 flex min-h-0 flex-1 flex-col overflow-hidden p-0">
|
||||
{run && run.status !== 'done' && <p className="p-5 text-[13px] text-muted">Fields appear when extraction finishes.</p>}
|
||||
<ul aria-label="Extracted fields" className="flex-1 overflow-y-auto p-4">
|
||||
{shown.map((r) => {
|
||||
const draft = drafts[r.row_id] ?? settled(r)
|
||||
const isDirty = draft !== settled(r)
|
||||
return (
|
||||
<li key={r.row_id} className={`mb-2 grid grid-cols-[minmax(80px,130px)_1fr_auto_auto] items-center gap-2 ${r.verified && !isDirty ? 'opacity-70' : ''}`}>
|
||||
<span className="truncate text-[12px] font-semibold text-muted" title={r.field}>{r.field}</span>
|
||||
<input aria-label={r.field} value={draft} placeholder="empty" title={r.reason ?? undefined}
|
||||
onChange={(e) => setDrafts((d) => ({ ...d, [r.row_id]: e.target.value }))}
|
||||
onKeyDown={(e) => e.key === 'Escape' && setDrafts((d) => ({ ...d, [r.row_id]: settled(r) }))}
|
||||
className={`input min-w-0 border-l-4 ${isDirty ? 'outline outline-warn' : ''}`} style={{ borderLeftColor: colourOf(r.band) }} />
|
||||
<span className="w-9 text-right text-[11px] text-muted tabular-nums" title="Legibility score: how clearly it was read, not whether it is right">
|
||||
{r.score?.toFixed(2) ?? '—'}
|
||||
</span>
|
||||
<button className="btn btn-sm w-[86px]" aria-label={`Verify ${r.field}`} aria-pressed={r.verified}
|
||||
disabled={isDirty} title={isDirty ? 'Save or discard the edit first' : undefined} onClick={() => verify(r)}>
|
||||
{r.verified ? '✓ Verified' : 'Verify'}
|
||||
</button>
|
||||
{(r.reason || r.corrected_value != null) && (
|
||||
<span className="col-span-4 -mt-1 text-[11px] text-muted">
|
||||
{r.reason}{r.reason && r.corrected_value != null && ' · '}
|
||||
{r.corrected_value != null && <>extracted <s>{r.value || 'empty'}</s></>}
|
||||
</span>
|
||||
)}
|
||||
</li>
|
||||
)
|
||||
})}
|
||||
</ul>
|
||||
{dirty.length > 0 && (
|
||||
<div className="flex items-center gap-2.5 border-t border-line px-4 py-2.5">
|
||||
<span className="text-[12px] font-semibold text-warn">{dirty.length} unsaved change{dirty.length === 1 ? '' : 's'}</span>
|
||||
<button className="btn btn-sm ml-auto" onClick={() => setDrafts({})} disabled={saving}>Discard</button>
|
||||
<button className="btn btn-primary btn-sm" onClick={save} disabled={saving}>{saving ? 'Saving…' : 'Save'}</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import { useEffect, useRef } from 'react'
|
||||
import { useStore } from '../store'
|
||||
|
||||
const LOG_COLOUR = { info: 'text-[#7fc4f0]', success: 'text-[#5fe0a0]', warn: 'text-[#f5c563]', error: 'text-[#f68c82]' }
|
||||
|
||||
export function RunPanel() {
|
||||
const { step, log } = useStore()
|
||||
const active = !step ? -1 : step.pct >= 100 ? STEPS.length : STEPS.indexOf(step.label)
|
||||
const logBox = useRef<HTMLDivElement>(null)
|
||||
useEffect(() => { logBox.current?.scrollTo(0, logBox.current.scrollHeight) }, [log.length])
|
||||
|
||||
return (
|
||||
<div className="card">
|
||||
<div className="card-title">Run automation</div>
|
||||
<div className="mb-2 flex gap-3 text-[13px] font-bold">
|
||||
<span>{step?.label ?? 'Waiting to start'}</span>
|
||||
<span className="ml-auto text-accent">{step?.pct ?? 0}%</span>
|
||||
</div>
|
||||
<div className="mb-4 h-2 overflow-hidden rounded-lg bg-glass-strong">
|
||||
<div className="h-full transition-[width] duration-500" style={{ width: `${step?.pct ?? 0}%`, background: 'var(--progress-bg)' }} />
|
||||
</div>
|
||||
<ol aria-label="Steps">
|
||||
{STEPS.map((s, i) => {
|
||||
const c = i < active ? 'done' : i === active ? 'active' : 'pending'
|
||||
return (
|
||||
<li key={s} aria-current={c === 'active' ? 'step' : undefined}
|
||||
className={`mb-1.5 flex items-center gap-3 rounded-[10px] px-3 py-2.5 text-[13px] font-semibold ${c === 'done' ? 'bg-ok-bg text-ok' : c === 'active' ? 'bg-warn-bg text-warn' : 'opacity-50'}`}>
|
||||
<span className={`flex h-[22px] w-[22px] items-center justify-center rounded-full text-[11px] ${c === 'done' ? 'bg-ok text-white' : c === 'active' ? 'bg-warn text-white' : 'bg-glass-strong'}`}>
|
||||
{c === 'done' ? '✓' : i + 1}
|
||||
</span>
|
||||
{s}
|
||||
</li>
|
||||
)
|
||||
})}
|
||||
</ol>
|
||||
{/* OPTIONAL: checkpoint tracker goes here */}
|
||||
<div ref={logBox} role="log" aria-label="Run log"
|
||||
className="mt-3 max-h-[220px] overflow-y-auto rounded-xl bg-accent-dark px-3.5 py-3 font-mono text-[13px] leading-[1.85] text-[#9fb6c4]">
|
||||
{log.map((l, i) => <div key={i} className={LOG_COLOUR[l.type] ?? LOG_COLOUR.info}>{l.msg}</div>)}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
// Globals from the CONFIG block in index.html.
|
||||
declare const APP_NAME: string
|
||||
declare const APP_SUBTITLE: string
|
||||
declare const LOGO_SRC: string
|
||||
declare const STEPS: string[]
|
||||
declare const REPORT_COLUMNS: { key: string; label: string }[]
|
||||
declare const API_BASE: string
|
||||
|
||||
interface Window {
|
||||
/** Contract entry point: one {event, data} envelope as JSON (desktop hosts, tests). */
|
||||
__pyMsg: (json: string) => void
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
@import 'tailwindcss';
|
||||
@custom-variant dark (&:where(.dark, .dark *));
|
||||
|
||||
/* ===== THEME TOKENS (swap these for a preset/custom palette) =====
|
||||
The same two blocks as the no-AI page, so every preset in
|
||||
rpa-dashboard-no-ai/templates/themes.md applies here unchanged. */
|
||||
body{
|
||||
--bg-top:#8fc9e8; --bg-mid:#bfe3f2; --bg-bottom:#eaf6fb;
|
||||
--glass:rgba(255,255,255,.60); --glass-strong:rgba(255,255,255,.78); --border:rgba(255,255,255,.9);
|
||||
--ink:#1c2b33; --muted:#5c7382; --accent-dark:#16202a;
|
||||
--accent:#3d9bd0; --accent-2:#5cb2e2; --accent-soft:#d4edfa; --accent-soft-2:#a9d8ee;
|
||||
--ok:#1f9d57; --ok-bg:#e6f6ec; --warn:#b7791f; --warn-bg:#fbf1dd; --bad:#c0392b; --bad-bg:#fdecea;
|
||||
--font:'Segoe UI',-apple-system,BlinkMacSystemFont,sans-serif; --font-display:var(--font);
|
||||
--header-bg:var(--glass-strong); --header-ink:var(--ink); --header-muted:var(--muted);
|
||||
--primary-bg:linear-gradient(180deg,var(--accent-2),var(--accent));
|
||||
--nav-active-bg:linear-gradient(135deg,var(--accent),var(--accent-dark));
|
||||
--progress-bg:var(--ok); --blur:blur(14px);
|
||||
}
|
||||
body.dark{
|
||||
--bg-top:#0f1b26; --bg-mid:#13212e; --bg-bottom:#182a38;
|
||||
--glass:rgba(22,34,44,.72); --glass-strong:rgba(30,45,58,.85); --border:rgba(120,160,190,.22);
|
||||
--ink:#e6eef4; --muted:#8fa6b4; --accent-dark:#2e5d80;
|
||||
--accent-soft:#2e5d90; --accent-soft-2:#1d3d5e;
|
||||
--ok:#2fbf72; --ok-bg:rgba(31,157,87,.16); --warn:#d9a13c; --warn-bg:rgba(183,121,31,.16);
|
||||
--bad:#e05a4b; --bad-bg:rgba(192,57,43,.16);
|
||||
}
|
||||
|
||||
/* ===== BAND COLOURS =====
|
||||
Semantic only: swatches, the rule down a field, legend dots. Never on a button
|
||||
or a heading. They say how clearly a value was printed, not whether it is right.
|
||||
The same in every preset and both modes. */
|
||||
body{ --clean:#51a68f; --verify:#e39b34; --review:#de5243; --absent:#6b6580; }
|
||||
|
||||
@theme inline {
|
||||
--color-glass: var(--glass);
|
||||
--color-glass-strong: var(--glass-strong);
|
||||
--color-line: var(--border);
|
||||
--color-ink: var(--ink);
|
||||
--color-muted: var(--muted);
|
||||
--color-accent: var(--accent);
|
||||
--color-accent-dark: var(--accent-dark);
|
||||
--color-accent-soft: var(--accent-soft);
|
||||
--color-accent-soft-2: var(--accent-soft-2);
|
||||
--color-ok: var(--ok);
|
||||
--color-ok-bg: var(--ok-bg);
|
||||
--color-warn: var(--warn);
|
||||
--color-warn-bg: var(--warn-bg);
|
||||
--color-bad: var(--bad);
|
||||
--color-bad-bg: var(--bad-bg);
|
||||
--color-clean: var(--clean);
|
||||
--color-verify: var(--verify);
|
||||
--color-review: var(--review);
|
||||
--color-absent: var(--absent);
|
||||
--font-body: var(--font);
|
||||
--font-head: var(--font-display);
|
||||
}
|
||||
|
||||
html, body { height: 100%; overflow: hidden; }
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: var(--font);
|
||||
color: var(--ink);
|
||||
background: linear-gradient(180deg, var(--bg-top) 0%, var(--bg-mid) 45%, var(--bg-bottom) 100%);
|
||||
}
|
||||
|
||||
/* Shared pieces, kept as classes because every screen uses them. */
|
||||
@layer components {
|
||||
.card { @apply mb-4 rounded-2xl border border-line bg-glass p-5 shadow-[0_10px_30px_rgba(20,50,70,.08)]; backdrop-filter: var(--blur); }
|
||||
.card-title { @apply mb-3.5 font-head text-[15px] font-bold; }
|
||||
.btn { @apply cursor-pointer rounded-[11px] border border-accent-soft-2 px-4 py-2 font-head text-[12.5px] font-semibold text-ink disabled:cursor-not-allowed disabled:opacity-45; background: linear-gradient(180deg, var(--accent-soft), var(--accent-soft-2)); }
|
||||
.btn-primary { @apply border-accent text-white; background: var(--primary-bg); }
|
||||
.btn-sm { @apply px-2.5 py-1 text-[11.5px]; }
|
||||
.pill { @apply inline-block rounded-full px-3 py-1 text-[11px] font-bold text-white; }
|
||||
.input { @apply rounded-[10px] border border-line bg-glass-strong px-2.5 py-2 text-[12.5px] text-ink; }
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
import { createRoot } from 'react-dom/client'
|
||||
import './index.css'
|
||||
import { App } from './App'
|
||||
import { startTransport } from './store'
|
||||
|
||||
startTransport()
|
||||
createRoot(document.getElementById('root')!).render(<App />)
|
||||
@@ -0,0 +1,139 @@
|
||||
import { useSyncExternalStore } from 'react'
|
||||
|
||||
/**
|
||||
* The no-AI contract, as on the no-AI page: envelopes in, actions out.
|
||||
* Events reach handle() from window.__pyMsg, from polling, or from the sample data.
|
||||
* The AI review screen talks to its own endpoints (see ai.ts).
|
||||
*/
|
||||
export const SAMPLE_DATA = true // no bot yet: the page answers its own data actions. Set false once wired.
|
||||
|
||||
export type LogType = 'info' | 'success' | 'warn' | 'error'
|
||||
export type Batch = { batch_id: string | number; date: string; state: string; items_total: number; items_done: number }
|
||||
export type ReportRow = Record<string, unknown> & { status?: string; bucket?: string }
|
||||
export type State = {
|
||||
status: { state: 'ready' | 'running' | 'done' | 'error'; msg?: string }
|
||||
step: { label: string; pct: number } | null
|
||||
log: { msg: string; type: LogType }[]
|
||||
batches: Batch[]
|
||||
reportRows: ReportRow[]
|
||||
user: { username: string; role: string }
|
||||
dark: boolean
|
||||
}
|
||||
|
||||
const DARK_KEY = 'rpa-dashboard.dark'
|
||||
let state: State = {
|
||||
status: { state: 'ready', msg: 'System ready' },
|
||||
step: null,
|
||||
log: [],
|
||||
batches: [],
|
||||
reportRows: [],
|
||||
user: { username: 'User', role: 'Operator' },
|
||||
dark: false,
|
||||
}
|
||||
const subs = new Set<() => void>()
|
||||
const set = (patch: Partial<State>) => {
|
||||
state = { ...state, ...patch }
|
||||
subs.forEach((f) => f())
|
||||
}
|
||||
export const useStore = () =>
|
||||
useSyncExternalStore((cb) => (subs.add(cb), () => void subs.delete(cb)), () => state)
|
||||
|
||||
// ===== EVENT DISPATCH (see contract.md) =====
|
||||
export function handle(event: string, d: any) { // eslint-disable-line @typescript-eslint/no-explicit-any
|
||||
switch (event) {
|
||||
case 'status': set({ status: d }); break
|
||||
case 'step': set({ step: d }); break
|
||||
case 'log': set({ log: [...state.log, d] }); break
|
||||
case 'dashboard_data': set({ batches: d.batches ?? [] }); break
|
||||
case 'report_data': set({ reportRows: d.rows ?? [] }); break
|
||||
case 'set_user': set({ user: d }); break
|
||||
case 'apply_dark': setDark(!!d.on); break
|
||||
}
|
||||
}
|
||||
window.__pyMsg = (json) => {
|
||||
const m = JSON.parse(json)
|
||||
handle(m.event, m.data)
|
||||
}
|
||||
|
||||
/** Flip the theme now (not on the next render), so the page never shows a half-switched theme. */
|
||||
export function setDark(on: boolean) {
|
||||
document.body.classList.toggle('dark', on)
|
||||
try { localStorage.setItem(DARK_KEY, on ? '1' : '') } catch { /* private window */ }
|
||||
set({ dark: on })
|
||||
}
|
||||
export function toggleDark() {
|
||||
const on = !state.dark
|
||||
setDark(on)
|
||||
act('set_dark', on)
|
||||
}
|
||||
export function clearRun() {
|
||||
set({ log: [], step: STEPS.length ? { label: STEPS[0], pct: 0 } : null })
|
||||
}
|
||||
|
||||
// ===== TRANSPORT: sample data, else HTTP polling =====
|
||||
/** Page -> bot action: POST {API_BASE}/<slot-with-dashes>, body = the args. */
|
||||
export function act(slot: string, ...args: unknown[]) {
|
||||
if (SAMPLE_DATA && slot in SAMPLE) return SAMPLE[slot]()
|
||||
fetch(`${API_BASE}/${slot.replace(/_/g, '-')}`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(args),
|
||||
})
|
||||
.then(() => SAMPLE_DATA || poll()) // the answer arrives as events: fetch them now
|
||||
.catch(() => {})
|
||||
}
|
||||
|
||||
let timer: ReturnType<typeof setTimeout> | undefined
|
||||
async function poll() {
|
||||
clearTimeout(timer)
|
||||
try {
|
||||
handle('dashboard_data', await (await fetch(`${API_BASE}/dashboard`)).json())
|
||||
const s = await (await fetch(`${API_BASE}/run/status`)).json()
|
||||
for (const e of s.events ?? []) handle(e.event, e.data)
|
||||
timer = setTimeout(poll, s.running ? 2000 : 10000) // poll fast only while busy
|
||||
} catch {
|
||||
timer = setTimeout(poll, 10000)
|
||||
}
|
||||
}
|
||||
|
||||
export function startTransport() {
|
||||
try { setDark(!!localStorage.getItem(DARK_KEY)) } catch { /* private window */ }
|
||||
if (SAMPLE_DATA) act('load_dashboard')
|
||||
else poll()
|
||||
}
|
||||
|
||||
// ===== SAMPLE DATA: delete this block and set SAMPLE_DATA=false once the bot is wired =====
|
||||
const SAMPLE: Record<string, () => void> = {
|
||||
load_dashboard() {
|
||||
const day = (n: number) => new Date(Date.now() - n * 864e5).toISOString().slice(0, 10)
|
||||
handle('dashboard_data', { batches: [
|
||||
{ batch_id: 1006, date: day(0), state: 'inprogress', items_total: 40, items_done: 18 },
|
||||
{ batch_id: 1005, date: day(1), state: 'completed', items_total: 35, items_done: 35 },
|
||||
{ batch_id: 1004, date: day(2), state: 'completed', items_total: 52, items_done: 52 },
|
||||
{ batch_id: 1003, date: day(3), state: 'notcompleted', items_total: 20, items_done: 7 },
|
||||
{ batch_id: 1002, date: day(4), state: 'completed', items_total: 28, items_done: 28 },
|
||||
{ batch_id: 1001, date: day(6), state: 'cleared', items_total: 12, items_done: 0 },
|
||||
] })
|
||||
},
|
||||
load_latest_report() {
|
||||
const rows = [['Processed successfully', 'success'], ['Posted', 'success'], ['Needs review', 'attention'], ['Error', 'failed']]
|
||||
.map(([status, bucket], i) => ({
|
||||
...Object.fromEntries(REPORT_COLUMNS.map((c) => [c.key, `${c.label} ${i + 1}`])), status, bucket,
|
||||
}))
|
||||
handle('report_data', { rows })
|
||||
},
|
||||
start_automation() {
|
||||
let i = 0
|
||||
handle('status', { state: 'running', msg: 'Running (sample)' })
|
||||
const t = setInterval(() => {
|
||||
if (i < STEPS.length) {
|
||||
handle('step', { label: STEPS[i], pct: Math.round(((i + 1) / STEPS.length) * 100) })
|
||||
handle('log', { msg: `Sample step: ${STEPS[i++]}`, type: 'info' })
|
||||
} else {
|
||||
clearInterval(t)
|
||||
handle('log', { msg: 'Sample run finished.', type: 'success' })
|
||||
handle('status', { state: 'done', msg: 'Completed' })
|
||||
}
|
||||
}, 400)
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"lib": ["ES2022", "DOM", "DOM.Iterable"],
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"jsx": "react-jsx",
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"skipLibCheck": true,
|
||||
"types": ["vite/client"]
|
||||
},
|
||||
"include": ["src"]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
import { defineConfig } from 'vite'
|
||||
import react from '@vitejs/plugin-react'
|
||||
import tailwindcss from '@tailwindcss/vite'
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [react(), tailwindcss()],
|
||||
base: './', // relative asset paths, so the built page works under any URL prefix
|
||||
// `npm run dev`: the backend (or server_stub.py) answers /api on port 8000.
|
||||
server: { proxy: { '/api': 'http://127.0.0.1:8000' } },
|
||||
})
|
||||
@@ -27,6 +27,23 @@ uv run --with pytest --with playwright pytest tests
|
||||
the login section is signed in (`login_result ok`) before each case.
|
||||
- `test_sections.py`: the optional-section strip tool keeps only the chosen
|
||||
sections, and a page with none of them still passes the page check.
|
||||
- `page_check_ai/`: **Seam 1, AI web page.** Builds the React starter with npm
|
||||
(filled with test values, in a temp folder that keeps `node_modules`), serves it with
|
||||
its own `server_stub.py`, and checks bands, band filters, drafts, the approve gate
|
||||
"page X of Y" and the cost page (spend tiles, coverage pill, month breakdown,
|
||||
by-hand assumptions), plus every no-AI case. Tests set run and review state on the stub
|
||||
directly (the `stub` fixture). To check a generated dashboard folder instead, set
|
||||
`PAGE_CHECK_AI_DIR=path/to/dashboard`. Needs npm on PATH, else it skips.
|
||||
- `page_check_ai_desktop/`: **Seam 1, AI desktop page.** The no-AI `dashboard.html` with
|
||||
`ai_sections.html` pasted in, loaded from `file://` with a fake Qt bridge whose
|
||||
`ai_request` slot goes to `server_stub.dispatch()` in this process (what `ai_bridge.py`
|
||||
does). Runs the AI web page's review and cost cases and the no-AI cases unchanged, plus
|
||||
a tampered page calling the slot directly (still refused). To check a generated folder,
|
||||
set `PAGE_CHECK_AI_DESKTOP_DIR=path/to/dashboard`; `PAGE_CHECK_WIRED=1` as above.
|
||||
- `test_pricing.py`: the AI cost function (normal, long-context, thinking tokens,
|
||||
unknown model).
|
||||
- `test_reference_scan.py`: no patient data, internal IPs, credentials or keys in any
|
||||
bundled file, and every Superbill extract carries an `EXAMPLE ONLY` banner.
|
||||
- `evals/`: **Seam 2.** Runs the skills on a fixture project with `claude -p`,
|
||||
then grades the reply and runs the page check on the page it built:
|
||||
|
||||
@@ -38,6 +55,10 @@ uv run --with pytest --with playwright pytest tests
|
||||
`FIXTURES` in `run_eval.py`. An entry with a `"fixture"` key reuses another
|
||||
fixture folder with different answers: `c-keep` and `c-restyle` both run on
|
||||
`c-web-existing-frontend`, and also check what changed in the files it already had.
|
||||
An `"ai": True` entry (`d-ai-extraction`) expects the AI skill's React starter and
|
||||
runs `page_check_ai/` on the folder it built. An `"ai_desktop": True` entry
|
||||
(`e-pyqt-ai`) expects the desktop AI page and runs `page_check/` (with the bot replay)
|
||||
and `page_check_ai_desktop/` on it.
|
||||
For a bot fixture (`b-pyqt-bot`), the eval also dry-runs the generated
|
||||
`bot_runner.py`, saves its events to `evals/results/<fixture>.run.jsonl`, and
|
||||
replays them in the page check.
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Shared by both page checks: one headless Chromium for the whole run
|
||||
(Playwright's sync API allows only one instance per process)."""
|
||||
import pytest
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def browser():
|
||||
with sync_playwright() as p:
|
||||
b = p.chromium.launch()
|
||||
yield b
|
||||
b.close()
|
||||
+39
-1
@@ -81,6 +81,28 @@ FIXTURES = {
|
||||
"page_has": ["--accent:#ff3049"],
|
||||
"existing": "#ff3049", # only token lines may change, and the theme now uses this colour
|
||||
},
|
||||
# (d) web service with an LLM extraction call and no UI: the AI skill's web path.
|
||||
"d-ai-extraction": {
|
||||
"prompt": "Build me an RPA dashboard for this project. I can't answer questions while you "
|
||||
"work, so here are my answers to anything the skill asks. Colours: Superbill Ember. "
|
||||
"Desktop or web: web. Optional sections: none.",
|
||||
"frontend": r"^\s*(none|no\b)",
|
||||
"asks": [r"colou?r", r"desktop|web", r"section"],
|
||||
"never_asks": [r"\bkeep\b|restyle"],
|
||||
"page_has": ["--accent:#ff3049"],
|
||||
"ai": True,
|
||||
},
|
||||
# (e) PyQt bot with an LLM extraction call: the AI skill's desktop path, no web server.
|
||||
"e-pyqt-ai": {
|
||||
"prompt": "Build me an RPA dashboard for this project. I can't answer questions while you "
|
||||
"work, so here are my answers to anything the skill asks. Colours: Tebra Sky. "
|
||||
"Optional sections: none.",
|
||||
"frontend": r"^\s*(none|no\b)",
|
||||
"asks": [r"colou?r", r"section"],
|
||||
"never_asks": [r"\bkeep\b|restyle", r"desktop|web"],
|
||||
"bot": True,
|
||||
"ai_desktop": True,
|
||||
},
|
||||
}
|
||||
# A line a restyle may change: a custom property, a comment, a blank, or a web-font link.
|
||||
TOKEN_LINE = r"^\s*(--[\w-]+\s*:.*|/\*.*\*/|<link[^>]*fonts\.googleapis\.com.*|@import.*fonts\.googleapis\.com.*)?\s*$"
|
||||
@@ -167,6 +189,16 @@ def main(fixture):
|
||||
results.mkdir(exist_ok=True)
|
||||
if pages:
|
||||
html = pages[0].read_text(encoding="utf-8")
|
||||
if cfg.get("ai"): # the React starter: its theme lives in src/index.css
|
||||
css = pages[0].parent / "src" / "index.css"
|
||||
html += css.read_text(encoding="utf-8") if css.exists() else ""
|
||||
checks["AI path: review starter and server stub built"] = all(
|
||||
(pages[0].parent / f).exists() for f in ["server_stub.py", "bands.py", "src/components/Review.tsx"])
|
||||
if cfg.get("ai_desktop"): # the no-AI page with the plain-HTML AI sections, hosted by ai_bridge.py
|
||||
host = pages[0].with_name("ai_bridge.py")
|
||||
checks["AI desktop path: sections, ai_bridge and review backend built"] = 'id="ai-review-tab"' in html and all(
|
||||
pages[0].with_name(f).exists() for f in ["ai_bridge.py", "server_stub.py", "bands.py", "pricing.py"])
|
||||
checks["no placeholders left in ai_bridge.py"] = host.exists() and "{{" not in host.read_text(encoding="utf-8")
|
||||
shutil.copy(pages[0], results / f"{fixture}.dashboard.html")
|
||||
checks["no placeholders left"] = "{{" not in html
|
||||
if cfg.get("brand"):
|
||||
@@ -187,7 +219,13 @@ def main(fixture):
|
||||
env = {**os.environ, "PAGE_CHECK_HTML": str(pages[0])}
|
||||
if cfg.get("bot"):
|
||||
env.update(bot_checks(project, html, checks, results / f"{fixture}.run.jsonl"))
|
||||
pc = subprocess.run([sys.executable, "-m", "pytest", str(ROOT / "tests" / "page_check"), "-q"],
|
||||
if cfg.get("ai"):
|
||||
env["PAGE_CHECK_AI_DIR"] = str(pages[0].parent)
|
||||
check_dirs = [ROOT / "tests" / ("page_check_ai" if cfg.get("ai") else "page_check")]
|
||||
if cfg.get("ai_desktop"):
|
||||
env["PAGE_CHECK_AI_DESKTOP_DIR"] = str(pages[0].parent)
|
||||
check_dirs.append(ROOT / "tests" / "page_check_ai_desktop")
|
||||
pc = subprocess.run([sys.executable, "-m", "pytest", *map(str, check_dirs), "-q"],
|
||||
env=env, capture_output=True, text=True)
|
||||
checks["page check passes"] = pc.returncode == 0
|
||||
page_check = pc.stdout.strip().splitlines()[-1:] if pc.stdout else [pc.stderr[-500:]]
|
||||
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
# Invoice extraction service
|
||||
|
||||
A small web service. It sends a scanned invoice to an LLM and gets back the
|
||||
invoice fields, each with a confidence score per word. There is no UI yet.
|
||||
|
||||
```
|
||||
uvicorn app:app --port 8000
|
||||
```
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
"""Upload a scanned invoice, get its fields back."""
|
||||
from fastapi import FastAPI, UploadFile
|
||||
|
||||
from extractor import extract_invoice
|
||||
|
||||
app = FastAPI()
|
||||
|
||||
|
||||
@app.post("/extract")
|
||||
async def extract(file: UploadFile):
|
||||
return extract_invoice(await file.read(), file.content_type or "application/pdf")
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
"""The LLM extraction call: one scanned invoice in, its fields out, each with word scores."""
|
||||
import json
|
||||
|
||||
from google import genai
|
||||
from google.genai import types
|
||||
|
||||
MODEL = "gemini-2.5-flash"
|
||||
FIELDS = { # field name -> mandatory
|
||||
"Invoice number": True,
|
||||
"Invoice date": True,
|
||||
"Vendor": True,
|
||||
"PO number": False,
|
||||
"Due date": True,
|
||||
"Total": True,
|
||||
}
|
||||
PROMPT = (
|
||||
"Read this invoice. Return JSON: a list of {field, value, word_scores}, one per field in "
|
||||
+ ", ".join(FIELDS)
|
||||
+ ". word_scores is your 0-1 legibility score for each word of the value. Use an empty "
|
||||
"value and no scores when a field isn't on the page."
|
||||
)
|
||||
|
||||
|
||||
def extract_invoice(document: bytes, mime_type: str) -> list[dict]:
|
||||
client = genai.Client() # credentials come from the environment
|
||||
response = client.models.generate_content(
|
||||
model=MODEL,
|
||||
contents=[types.Part.from_bytes(data=document, mime_type=mime_type), PROMPT],
|
||||
config=types.GenerateContentConfig(response_mime_type="application/json"),
|
||||
)
|
||||
rows = json.loads(response.text)
|
||||
for row in rows:
|
||||
row["mandatory"] = FIELDS.get(row["field"], False)
|
||||
return rows
|
||||
@@ -0,0 +1,4 @@
|
||||
fastapi
|
||||
uvicorn
|
||||
google-genai
|
||||
python-multipart
|
||||
Vendored
+11
@@ -0,0 +1,11 @@
|
||||
# Invoice intake bot
|
||||
|
||||
A desktop RPA bot. It reads each scanned invoice in `inbox/` with an LLM
|
||||
(`extractor.py`) and queues the fields for a person to review.
|
||||
The operator window will be a PyQt6 app.
|
||||
|
||||
Run the bot on its own:
|
||||
|
||||
```
|
||||
python -m bot
|
||||
```
|
||||
Vendored
+31
@@ -0,0 +1,31 @@
|
||||
"""Invoice intake bot: picks up scanned invoices and extracts their fields with an LLM.
|
||||
Start it with: python -m bot"""
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
INBOX = Path(__file__).with_name("inbox")
|
||||
|
||||
|
||||
def say(line):
|
||||
print(line, flush=True)
|
||||
time.sleep(0.2)
|
||||
|
||||
|
||||
def main():
|
||||
say("[step 1] Initializing")
|
||||
docs = sorted(INBOX.glob("*.pdf"))
|
||||
say(f"[inbox] {len(docs)} invoices waiting")
|
||||
say("[step 2] Extracting fields")
|
||||
for i, doc in enumerate(docs, 1):
|
||||
try:
|
||||
from extractor import extract_invoice # the LLM call
|
||||
rows = extract_invoice(doc.read_bytes(), "application/pdf")
|
||||
say(f"[extract] {doc.name}: {len(rows)} fields ({i}/{len(docs)})")
|
||||
except Exception as e: # one bad document must not stop the batch
|
||||
say(f"ERROR: extraction failed for {doc.name}: {type(e).__name__}")
|
||||
say("[step 3] Queueing for review")
|
||||
say(f"[all done] {len(docs)} invoices processed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
"""The LLM extraction call: one scanned invoice in, its fields out, each with word scores."""
|
||||
import json
|
||||
|
||||
from google import genai
|
||||
from google.genai import types
|
||||
|
||||
MODEL = "gemini-2.5-flash"
|
||||
FIELDS = { # field name -> mandatory
|
||||
"Invoice number": True,
|
||||
"Invoice date": True,
|
||||
"Vendor": True,
|
||||
"PO number": False,
|
||||
"Due date": True,
|
||||
"Total": True,
|
||||
}
|
||||
PROMPT = (
|
||||
"Read this invoice. Return JSON: a list of {field, value, word_scores}, one per field in "
|
||||
+ ", ".join(FIELDS)
|
||||
+ ". word_scores is your 0-1 legibility score for each word of the value. Use an empty "
|
||||
"value and no scores when a field isn't on the page."
|
||||
)
|
||||
|
||||
|
||||
def extract_invoice(document: bytes, mime_type: str) -> list[dict]:
|
||||
client = genai.Client() # credentials come from the environment
|
||||
response = client.models.generate_content(
|
||||
model=MODEL,
|
||||
contents=[types.Part.from_bytes(data=document, mime_type=mime_type), PROMPT],
|
||||
config=types.GenerateContentConfig(response_mime_type="application/json"),
|
||||
)
|
||||
rows = json.loads(response.text)
|
||||
for row in rows:
|
||||
row["mandatory"] = FIELDS.get(row["field"], False)
|
||||
return rows
|
||||
@@ -0,0 +1,2 @@
|
||||
%PDF-1.4
|
||||
% sample invoice placeholder
|
||||
@@ -0,0 +1,2 @@
|
||||
%PDF-1.4
|
||||
% sample invoice placeholder
|
||||
+3
@@ -0,0 +1,3 @@
|
||||
PyQt6
|
||||
PyQt6-WebEngine
|
||||
google-genai
|
||||
@@ -13,7 +13,6 @@ import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
TEMPLATE = ROOT / "rpa-dashboard-no-ai" / "templates" / "dashboard.html"
|
||||
@@ -57,14 +56,6 @@ def page_url(tmp_path_factory):
|
||||
return out.as_uri()
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def browser():
|
||||
with sync_playwright() as p:
|
||||
b = p.chromium.launch()
|
||||
yield b
|
||||
b.close()
|
||||
|
||||
|
||||
@pytest.fixture(params=list(MODES))
|
||||
def page(request, browser, page_url):
|
||||
width, height, dark = MODES[request.param]
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
"""Seam 1 page check for the AI web dashboard (React + Vite), served by its server stub.
|
||||
|
||||
Page under test: set PAGE_CHECK_AI_DIR to a generated dashboard folder (the one with
|
||||
package.json, index.html and server_stub.py). Without it, the AI web starter template
|
||||
is filled with TEST_FILL and built in a cache folder. Either way the folder is built
|
||||
with npm, and its server_stub.py runs in this process, so tests can set run and
|
||||
review state directly through the `stub` fixture.
|
||||
|
||||
Contract envelopes go in through window.__pyMsg, as in the no-AI check. Each action
|
||||
the page POSTs to {API_BASE}/<slot> is recorded in window.__calls as [slot, ...args].
|
||||
Every page test runs in light, dark and narrow mode.
|
||||
"""
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
TEMPLATE = ROOT / "rpa-dashboard-ai" / "templates" / "web"
|
||||
TEST_FILL = {
|
||||
"APP_NAME": "Test Extractor",
|
||||
"APP_SUBTITLE": "Page check",
|
||||
"LOGO_SRC": "",
|
||||
"API_BASE": "/api",
|
||||
"STEPS_JSON": json.dumps(["Initializing", "Login", "Process items", "Completed"]),
|
||||
"REPORT_COLUMNS_JSON": json.dumps([{"key": "item", "label": "Item"}, {"key": "status", "label": "Status"}]),
|
||||
}
|
||||
RECORD_ACTIONS = """
|
||||
window.__calls = [];
|
||||
const realFetch = window.fetch;
|
||||
window.fetch = (url, opts = {}) => {
|
||||
const m = String(url).match(/\\/api\\/([a-z-]+)$/);
|
||||
if (m && opts.method === 'POST') window.__calls.push([m[1].replace(/-/g, '_'), ...JSON.parse(opts.body || '[]')]);
|
||||
return realFetch(url, opts);
|
||||
};
|
||||
"""
|
||||
MODES = {"light": (1280, 820, False), "dark": (1280, 820, True), "narrow": (800, 900, False)}
|
||||
SKIP_COPY = shutil.ignore_patterns("node_modules", "dist", "__pycache__")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sample_data():
|
||||
"""Shared no-AI cases that need the page's own sample data skip on a bot-wired page."""
|
||||
if os.environ.get("PAGE_CHECK_WIRED") == "1":
|
||||
pytest.skip("page is wired to a bot, so it has no sample data")
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def site():
|
||||
"""The dashboard folder under test: server_stub.py, index.html and the sources."""
|
||||
if os.environ.get("PAGE_CHECK_AI_DIR"):
|
||||
return Path(os.environ["PAGE_CHECK_AI_DIR"]).resolve()
|
||||
pkg = (TEMPLATE / "package.json").read_bytes() + (TEMPLATE / "package-lock.json").read_bytes()
|
||||
out = Path(tempfile.gettempdir()) / f"rpa-dashboard-ai-web-{hashlib.sha1(pkg).hexdigest()[:10]}"
|
||||
shutil.rmtree(out / "src", ignore_errors=True) # node_modules is kept between runs
|
||||
shutil.copytree(TEMPLATE, out, dirs_exist_ok=True, ignore=SKIP_COPY)
|
||||
index = out / "index.html"
|
||||
index.write_text(re.sub(r"\{\{(\w+)\}\}", lambda m: TEST_FILL[m[1]], index.read_text(encoding="utf-8")),
|
||||
encoding="utf-8")
|
||||
return out
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def built(site):
|
||||
npm = shutil.which("npm") or pytest.skip("npm not on PATH")
|
||||
if not (site / "node_modules").exists():
|
||||
subprocess.run([npm, "ci" if (site / "package-lock.json").exists() else "install"], cwd=site, check=True)
|
||||
r = subprocess.run([npm, "run", "build"], cwd=site, capture_output=True, text=True)
|
||||
assert r.returncode == 0, r.stdout + r.stderr
|
||||
return site
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def stub_module(site):
|
||||
sys.path.insert(0, str(site)) # the stub imports bands.py from its own folder
|
||||
spec = importlib.util.spec_from_file_location("server_stub", site / "server_stub.py")
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def server(stub_module):
|
||||
srv = stub_module.make_server("127.0.0.1", 0)
|
||||
threading.Thread(target=srv.serve_forever, daemon=True).start()
|
||||
yield f"http://127.0.0.1:{srv.server_address[1]}"
|
||||
srv.shutdown()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def stub(stub_module, server):
|
||||
"""The running stub, reset to its sample data. Tests change stub.RUNS / stub.ROWS directly."""
|
||||
stub_module.reset()
|
||||
return stub_module
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def page_url(built, server):
|
||||
return server + "/"
|
||||
|
||||
|
||||
@pytest.fixture(params=list(MODES))
|
||||
def page(request, browser, page_url, stub):
|
||||
width, height, dark = MODES[request.param]
|
||||
ctx = browser.new_context(viewport={"width": width, "height": height})
|
||||
ctx.add_init_script(RECORD_ACTIONS)
|
||||
pg = ctx.new_page()
|
||||
pg.set_default_timeout(5000)
|
||||
errors = []
|
||||
pg.on("pageerror", lambda e: errors.append(e))
|
||||
pg.goto(page_url)
|
||||
assert not errors, f"page script errors: {errors}"
|
||||
if dark:
|
||||
push_to(pg, "apply_dark", {"on": True})
|
||||
yield pg
|
||||
ctx.close()
|
||||
|
||||
|
||||
def push_to(pg, event, data):
|
||||
pg.evaluate("([event, data]) => window.__pyMsg(JSON.stringify({event, data}))", [event, data])
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def push(page):
|
||||
"""push(event, data): send one contract envelope to the page, as the bot would."""
|
||||
return lambda event, data: push_to(page, event, data)
|
||||
@@ -0,0 +1,85 @@
|
||||
"""AI cost page: spend tiles, coverage pill, month breakdown and the by-hand assumptions.
|
||||
|
||||
ROI runs are set on the server stub (stub.ROI_RUNS); the stub prices them with pricing.py.
|
||||
The test pins its own pricing table, so it holds on a generated dashboard whose table
|
||||
lists only the project's models. See rpa-dashboard-ai/templates/contract.md.
|
||||
"""
|
||||
import sys
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
from playwright.sync_api import expect
|
||||
|
||||
NOW = datetime.now(timezone.utc)
|
||||
LAST = NOW.replace(day=1) - timedelta(days=1) # a day in last month
|
||||
|
||||
|
||||
def roi_run(job_id, when, model, inp=None, out=None, thinking=0):
|
||||
return {"job_id": job_id, "document": f"{job_id}.pdf", "created_at": when.strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"model": model, "input_tokens": inp, "output_tokens": out, "thinking_tokens": thinking}
|
||||
|
||||
|
||||
TABLE = {
|
||||
"test-flash": {"input": 0.30, "output": 2.50},
|
||||
"test-pro": {"input": 1.25, "output": 10.00, "long_threshold": 200_000, "input_long": 2.50, "output_long": 15.00},
|
||||
"test-lite": {"input": 0.10, "output": 0.40},
|
||||
}
|
||||
AS_OF = "2030-01-31"
|
||||
# Costs worked by hand from TABLE:
|
||||
RUNS = [
|
||||
roi_run("r1", NOW, "test-flash", 1_000_000, 200_000), # 0.30 + 0.50 = 0.80
|
||||
roi_run("r2", NOW, "test-pro", 400_000, 60_000, 40_000), # long: 1.00 + 1.50 = 2.50
|
||||
roi_run("r3", LAST, "test-lite", 1_000_000, 1_000_000), # 0.10 + 0.40 = 0.50
|
||||
roi_run("r4", LAST, "test-flash"), # no token usage recorded
|
||||
]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def cost_page(page, stub, monkeypatch):
|
||||
monkeypatch.setattr(sys.modules["pricing"], "PRICING", TABLE) # the stub's own pricing module
|
||||
monkeypatch.setattr(stub, "PRICING_AS_OF", AS_OF)
|
||||
stub.ROI_RUNS[:] = RUNS
|
||||
page.get_by_role("navigation").get_by_role("button", name="Cost", exact=True).click()
|
||||
return page
|
||||
|
||||
|
||||
def tile(page, name):
|
||||
return page.get_by_role("group", name=name, exact=True)
|
||||
|
||||
|
||||
def test_spend_tiles_sum_the_priced_runs(cost_page):
|
||||
expect(tile(cost_page, "Total spend")).to_contain_text("$3.80")
|
||||
expect(tile(cost_page, "This month")).to_contain_text("$3.30")
|
||||
|
||||
|
||||
def test_coverage_pill_counts_priced_of_all_documents(cost_page):
|
||||
expect(cost_page.get_by_text("3 of 4 documents priced")).to_be_visible()
|
||||
|
||||
|
||||
def test_month_breakdown_groups_runs_by_month(cost_page):
|
||||
rows = cost_page.get_by_role("table", name="Spend by month").get_by_role("row")
|
||||
expect(rows).to_have_count(3) # header + two months, newest first
|
||||
expect(rows.nth(1)).to_contain_text(NOW.strftime("%b %Y"))
|
||||
expect(rows.nth(1)).to_contain_text("2 of 2")
|
||||
expect(rows.nth(1)).to_contain_text("$3.30")
|
||||
expect(rows.nth(2)).to_contain_text(LAST.strftime("%b %Y"))
|
||||
expect(rows.nth(2)).to_contain_text("1 of 2")
|
||||
expect(rows.nth(2)).to_contain_text("$0.50")
|
||||
|
||||
|
||||
def test_runs_show_tokens_and_cost_and_unpriced_is_not_zero(cost_page):
|
||||
runs = cost_page.get_by_role("table", name="Cost per run")
|
||||
r2 = runs.get_by_role("row").filter(has_text="r2.pdf")
|
||||
for text in ("test-pro", "400,000", "60,000", "40,000", "$2.50"):
|
||||
expect(r2).to_contain_text(text)
|
||||
expect(runs.get_by_role("row").filter(has_text="r4.pdf")).to_contain_text("not priced")
|
||||
|
||||
|
||||
def test_by_hand_panel_is_labelled_assumptions_and_recomputes(cost_page):
|
||||
panel = cost_page.get_by_role("region", name="Against doing it by hand")
|
||||
expect(panel).to_contain_text("assumptions")
|
||||
# 3 priced documents x 10 min = 0.5 h at $30/h = $15.00
|
||||
panel.get_by_label("Minutes per document").fill("10")
|
||||
panel.get_by_label("Hourly rate").fill("30")
|
||||
expect(panel).to_contain_text("$15.00")
|
||||
expect(cost_page.get_by_text(AS_OF)).to_be_visible() # pricing as-of date shown
|
||||
@@ -0,0 +1,140 @@
|
||||
"""AI contract cases: confidence bands, review drafts, the approve gate and run progress.
|
||||
|
||||
Run and review state is set on the server stub (the page's backend); what the page
|
||||
shows is asserted through the browser. See rpa-dashboard-ai/templates/contract.md.
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from playwright.sync_api import expect
|
||||
|
||||
# (field, value, word scores, mandatory) -> the band each one must land in
|
||||
FIELDS = [
|
||||
("Invoice number", "INV-2207", [1.0], True), # green
|
||||
("Vendor", "Northwind Traders", [0.99, 0.97], True), # green
|
||||
("Invoice date", "2026-10-02", [0.93], True), # amber
|
||||
("Total", "1,284.50", [0.98, 0.71], True), # red: worst word wins
|
||||
("Due date", "", [1.0], True), # blank mandatory
|
||||
("PO number", "PO-118", [], False), # unscored
|
||||
]
|
||||
EXPECTED = {"green": ["Invoice number", "Vendor"], "amber": ["Invoice date"], "red": ["Total"],
|
||||
"blank": ["Due date"], "unscored": ["PO number"]}
|
||||
FLAGGED = 4 # everything but green
|
||||
|
||||
|
||||
def api(server, method, path, body=None):
|
||||
req = urllib.request.Request(server + "/api" + path, method=method,
|
||||
data=None if body is None else json.dumps(body).encode(),
|
||||
headers={"Content-Type": "application/json"})
|
||||
try:
|
||||
with urllib.request.urlopen(req) as r:
|
||||
return r.status, json.loads(r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, json.loads(e.read())
|
||||
|
||||
|
||||
def test_server_refuses_approval_while_flagged_fields_are_unverified(stub, server):
|
||||
"""The gate lives on the server: no request from any page can skip it."""
|
||||
stub.put_run("job-gate", "gate.pdf", FIELDS)
|
||||
status, body = api(server, "POST", "/runs/job-gate/approve")
|
||||
assert status == 409 and str(FLAGGED) in body["detail"]
|
||||
|
||||
_, rows = api(server, "GET", "/runs/job-gate/review")
|
||||
first_flagged = next(r for r in rows["rows"] if r["band"] != "green")
|
||||
api(server, "PATCH", f"/review/{first_flagged['row_id']}", {"verified": True})
|
||||
status, body = api(server, "POST", "/runs/job-gate/approve")
|
||||
assert status == 409 and str(FLAGGED - 1) in body["detail"]
|
||||
|
||||
api(server, "POST", "/runs/job-gate/verify-remaining")
|
||||
status, _ = api(server, "POST", "/runs/job-gate/approve")
|
||||
assert status == 200
|
||||
assert api(server, "GET", "/runs/job-gate")[1]["approved"] is True
|
||||
|
||||
|
||||
def open_review(page, job_id):
|
||||
page.get_by_role("navigation").get_by_role("button", name="Review", exact=True).click()
|
||||
page.get_by_label("Document", exact=True).select_option(job_id)
|
||||
|
||||
|
||||
def fields_list(page):
|
||||
return page.get_by_role("list", name="Extracted fields").get_by_role("listitem")
|
||||
|
||||
|
||||
LEGEND = {"green": "Clean", "amber": "Check", "red": "Review", "blank": "Missing", "unscored": "Unscored"}
|
||||
|
||||
|
||||
def test_band_counts_and_legend_match(page, stub):
|
||||
stub.put_run("job-t", "bands.pdf", FIELDS)
|
||||
open_review(page, "job-t")
|
||||
legend = page.get_by_role("list", name="Confidence bands").get_by_role("listitem")
|
||||
expect(legend).to_have_count(len(LEGEND))
|
||||
for i, (band, word) in enumerate(LEGEND.items()):
|
||||
expect(legend.nth(i)).to_have_text(re.compile(rf"^\s*{word}\b.*· {len(EXPECTED[band])}\s*$"))
|
||||
expect(page.get_by_text(re.compile("print clarity, not correctness", re.I))).to_be_visible()
|
||||
|
||||
|
||||
def test_band_filters_show_the_right_rows(page, stub):
|
||||
stub.put_run("job-t", "bands.pdf", FIELDS)
|
||||
open_review(page, "job-t")
|
||||
expect(fields_list(page)).to_have_count(len(FIELDS))
|
||||
filters = page.get_by_role("group", name="Filter by band")
|
||||
for name, band in [("Red", "red"), ("Amber", "amber"), ("Missing", "blank"), ("Unscored", "unscored")]:
|
||||
filters.get_by_role("button", name=name, exact=True).click()
|
||||
expect(fields_list(page)).to_have_count(len(EXPECTED[band]))
|
||||
for field in EXPECTED[band]:
|
||||
expect(page.get_by_role("textbox", name=field, exact=True)).to_be_visible()
|
||||
filters.get_by_role("button", name="All", exact=True).click()
|
||||
expect(fields_list(page)).to_have_count(len(FIELDS))
|
||||
|
||||
|
||||
def row(stub, job_id, field):
|
||||
return next(r for r in stub.ROWS[job_id] if r["field"] == field)
|
||||
|
||||
|
||||
def test_draft_edit_is_not_saved_until_save(page, stub):
|
||||
stub.put_run("job-t", "bands.pdf", FIELDS)
|
||||
open_review(page, "job-t")
|
||||
box = page.get_by_role("textbox", name="Vendor", exact=True)
|
||||
box.fill("Northwind Traders Ltd")
|
||||
expect(page.get_by_text("1 unsaved change", exact=True)).to_be_visible()
|
||||
assert row(stub, "job-t", "Vendor")["corrected_value"] is None
|
||||
|
||||
asked = []
|
||||
page.once("dialog", lambda d: (asked.append(d.message), d.dismiss()))
|
||||
page.get_by_role("navigation").get_by_role("button", name="Report", exact=True).click()
|
||||
expect(box).to_have_value("Northwind Traders Ltd") # leaving was refused, the draft is still there
|
||||
assert asked and "unsaved" in asked[0].lower()
|
||||
|
||||
page.get_by_role("button", name="Save", exact=True).click()
|
||||
expect(page.get_by_text("1 unsaved change", exact=True)).to_be_hidden()
|
||||
saved = row(stub, "job-t", "Vendor")
|
||||
assert saved["corrected_value"] == "Northwind Traders Ltd" and saved["verified"] is True
|
||||
|
||||
|
||||
def test_approve_stays_blocked_while_rows_are_unverified(page, stub):
|
||||
stub.put_run("job-t", "bands.pdf", FIELDS)
|
||||
open_review(page, "job-t")
|
||||
page.get_by_role("button", name="Verify Total", exact=True).click()
|
||||
expect(page.get_by_role("button", name="Verify Total", exact=True)).to_have_attribute("aria-pressed", "true")
|
||||
page.get_by_role("button", name="Approve", exact=True).click()
|
||||
expect(page.get_by_role("alert")).to_contain_text(f"{FLAGGED - 1} flagged")
|
||||
assert stub.RUNS["job-t"]["approved"] is False
|
||||
|
||||
page.get_by_role("button", name="Verify remaining", exact=True).click()
|
||||
page.get_by_role("button", name="Approve", exact=True).click()
|
||||
expect(page.get_by_text("✓ Approved", exact=True)).to_be_visible()
|
||||
expect(page.get_by_role("alert")).to_be_hidden()
|
||||
assert stub.RUNS["job-t"]["approved"] is True
|
||||
|
||||
|
||||
def test_page_x_of_y_updates_while_running(page, stub):
|
||||
stub.put_run("job-busy", "busy.pdf", FIELDS, status="running", pages_done=1, pages_total=4)
|
||||
open_review(page, "job-busy")
|
||||
expect(page.get_by_text("page 1 of 4", exact=True)).to_be_visible()
|
||||
stub.RUNS["job-busy"]["pages_done"] = 3
|
||||
expect(page.get_by_text("page 3 of 4", exact=True)).to_be_visible()
|
||||
stub.RUNS["job-busy"].update(status="done", pages_done=4)
|
||||
expect(fields_list(page)).to_have_count(len(FIELDS))
|
||||
expect(page.get_by_text(re.compile(r"^page \d+ of \d+$"))).to_be_hidden()
|
||||
@@ -0,0 +1,19 @@
|
||||
"""Everything from the no-AI page: the no-AI contract cases, run unchanged against the AI page.
|
||||
|
||||
They use this folder's `page` and `push` fixtures, so they check the React page.
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "page_check"))
|
||||
|
||||
from test_contract import ( # noqa: E402,F401
|
||||
test_dark_toggle_flips_theme,
|
||||
test_donut_and_legend_match_batch_states,
|
||||
test_kpi_counts_match_batch_states,
|
||||
test_log_lines_show_their_type,
|
||||
test_no_bot_page_runs_on_sample_data,
|
||||
test_plain_browser_first_open_shows_sample_data,
|
||||
test_report_search_filters_rows_and_shows_bucket_pill,
|
||||
test_step_list_follows_step_label,
|
||||
)
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Seam 1 page check for the AI desktop page: the no-AI dashboard.html with the plain-HTML
|
||||
AI sections (rpa-dashboard-ai/templates/desktop/ai_sections.html) pasted in before </body>.
|
||||
|
||||
Page under test: set PAGE_CHECK_AI_DESKTOP_DIR to a generated dashboard folder (the one with
|
||||
dashboard.html and server_stub.py). Without it, the no-AI template is filled with the no-AI
|
||||
check's TEST_FILL and the AI sections are pasted in, the way the skill does it.
|
||||
|
||||
The page sees a fake Qt bridge, as in the no-AI check: each slot call lands in window.__calls.
|
||||
The AI slot, ai_request(id, method, path, body), goes to the review backend's dispatch() in
|
||||
this process, and the reply comes back as an `ai_response` envelope, exactly what
|
||||
ai_bridge.py's slot does. So the approve gate under test is the Python one. Tests set run and
|
||||
review state on the backend directly (the `stub` fixture). Light, dark and narrow for every case.
|
||||
"""
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
NO_AI = ROOT / "rpa-dashboard-no-ai" / "templates" / "dashboard.html"
|
||||
AI = ROOT / "rpa-dashboard-ai" / "templates"
|
||||
TEST_FILL = {
|
||||
"APP_NAME": "Test Extractor",
|
||||
"APP_SUBTITLE": "Page check",
|
||||
"LOGO_SRC": "",
|
||||
"API_BASE": "",
|
||||
"STEPS_JSON": json.dumps(["Initializing", "Login", "Process items", "Completed"]),
|
||||
"REPORT_COLUMNS_JSON": json.dumps([{"key": "item", "label": "Item"}, {"key": "status", "label": "Status"}]),
|
||||
}
|
||||
FAKE_BRIDGE = """
|
||||
window.__calls = [];
|
||||
window.qt = {webChannelTransport: {}};
|
||||
window.QWebChannel = function (_transport, ready) {
|
||||
const pyBridge = new Proxy({}, {get: (_o, slot) => (...args) => {
|
||||
window.__calls.push([slot, ...args]);
|
||||
if (slot === 'ai_request') window.__aiRequest(...args).then(window.__pyMsg);
|
||||
}});
|
||||
ready({objects: {pyBridge}});
|
||||
};
|
||||
"""
|
||||
MODES = {"light": (1280, 820, False), "dark": (1280, 820, True), "narrow": (800, 900, False)}
|
||||
WIRED = os.environ.get("PAGE_CHECK_WIRED") == "1"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sample_data():
|
||||
if WIRED:
|
||||
pytest.skip("page is wired to a bot, so it has no sample data")
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def site(tmp_path_factory):
|
||||
"""The dashboard folder under test: dashboard.html, server_stub.py and bands.py."""
|
||||
if os.environ.get("PAGE_CHECK_AI_DESKTOP_DIR"):
|
||||
return Path(os.environ["PAGE_CHECK_AI_DESKTOP_DIR"]).resolve()
|
||||
html = re.sub(r"\{\{(\w+)\}\}", lambda m: TEST_FILL[m[1]], NO_AI.read_text(encoding="utf-8"))
|
||||
if WIRED:
|
||||
html = html.replace("const SAMPLE_DATA = true", "const SAMPLE_DATA = false")
|
||||
sections = (AI / "desktop" / "ai_sections.html").read_text(encoding="utf-8")
|
||||
out = tmp_path_factory.mktemp("desktop")
|
||||
(out / "dashboard.html").write_text(html.replace("</body>", sections + "\n</body>"), encoding="utf-8")
|
||||
for f in ["server_stub.py", "bands.py", "pricing.py"]:
|
||||
(out / f).write_bytes((AI / "web" / f).read_bytes())
|
||||
return out
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def page_url(site):
|
||||
return (site / "dashboard.html").as_uri()
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def stub_module(site):
|
||||
sys.path.insert(0, str(site)) # the stub imports bands.py from its own folder
|
||||
spec = importlib.util.spec_from_file_location("desktop_server_stub", site / "server_stub.py")
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def stub(stub_module):
|
||||
"""The review backend the bridge talks to, reset to its sample data."""
|
||||
stub_module.reset()
|
||||
return stub_module
|
||||
|
||||
|
||||
def ai_request(stub, req_id, method, path, body):
|
||||
"""What ai_bridge.py's ai_request slot does: dispatch, then answer with one envelope."""
|
||||
status, payload = stub.dispatch(method, path, json.loads(body or "{}"))
|
||||
return json.dumps({"event": "ai_response", "data": {"id": req_id, "status": status, "body": payload}})
|
||||
|
||||
|
||||
@pytest.fixture(params=list(MODES))
|
||||
def page(request, browser, page_url, stub):
|
||||
width, height, dark = MODES[request.param]
|
||||
ctx = browser.new_context(viewport={"width": width, "height": height})
|
||||
ctx.expose_function("__aiRequest", lambda i, method, path, body: ai_request(stub, i, method, path, body))
|
||||
ctx.add_init_script(FAKE_BRIDGE)
|
||||
pg = ctx.new_page()
|
||||
pg.set_default_timeout(5000)
|
||||
errors = []
|
||||
pg.on("pageerror", lambda e: errors.append(e))
|
||||
pg.goto(page_url)
|
||||
assert not errors, f"page script errors: {errors}"
|
||||
if dark:
|
||||
push_to(pg, "apply_dark", {"on": True})
|
||||
yield pg
|
||||
assert not errors, f"page script errors: {errors}"
|
||||
ctx.close()
|
||||
|
||||
|
||||
def push_to(pg, event, data):
|
||||
pg.evaluate("([event, data]) => window.__pyMsg(JSON.stringify({event, data}))", [event, data])
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def push(page):
|
||||
return lambda event, data: push_to(page, event, data)
|
||||
@@ -0,0 +1,54 @@
|
||||
"""The AI web page's review cases, run unchanged against the desktop page.
|
||||
|
||||
They use this folder's `page` and `stub` fixtures: the page talks to the review backend
|
||||
over the fake Qt bridge instead of HTTP.
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "page_check_ai"))
|
||||
|
||||
from test_ai_review import ( # noqa: E402,F401
|
||||
test_approve_stays_blocked_while_rows_are_unverified,
|
||||
test_band_counts_and_legend_match,
|
||||
test_band_filters_show_the_right_rows,
|
||||
test_draft_edit_is_not_saved_until_save,
|
||||
test_page_x_of_y_updates_while_running,
|
||||
)
|
||||
from test_ai_review import FIELDS, FLAGGED, open_review # noqa: E402
|
||||
from test_ai_cost import ( # noqa: E402,F401
|
||||
cost_page,
|
||||
test_by_hand_panel_is_labelled_assumptions_and_recomputes,
|
||||
test_coverage_pill_counts_priced_of_all_documents,
|
||||
test_month_breakdown_groups_runs_by_month,
|
||||
test_runs_show_tokens_and_cost_and_unpriced_is_not_zero,
|
||||
test_spend_tiles_sum_the_priced_runs,
|
||||
)
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "page_check"))
|
||||
from test_contract import ( # noqa: E402,F401
|
||||
test_dark_toggle_flips_theme,
|
||||
test_donut_and_legend_match_batch_states,
|
||||
test_kpi_counts_match_batch_states,
|
||||
test_log_lines_show_their_type,
|
||||
test_no_bot_page_runs_on_sample_data,
|
||||
test_plain_browser_first_open_shows_sample_data,
|
||||
test_report_search_filters_rows_and_shows_bucket_pill,
|
||||
test_step_list_follows_step_label,
|
||||
)
|
||||
|
||||
TAMPER = """async (path) => {
|
||||
const reply = new Promise(done => { const base = window.__pyMsg;
|
||||
window.__pyMsg = m => { base(m); if (JSON.parse(m).event === 'ai_response') done(JSON.parse(m).data); }; });
|
||||
bridge.ai_request(999, 'POST', path, '{}');
|
||||
return await reply;
|
||||
}"""
|
||||
|
||||
|
||||
def test_a_changed_page_cannot_approve_over_the_bridge(page, stub):
|
||||
"""The gate is Python's: a page that skips its own buttons and calls the slot is still refused."""
|
||||
stub.put_run("job-t", "bands.pdf", FIELDS)
|
||||
open_review(page, "job-t")
|
||||
reply = page.evaluate(TAMPER, "/runs/job-t/approve")
|
||||
assert reply["id"] == 999 and reply["status"] == 409 and str(FLAGGED) in reply["body"]["detail"]
|
||||
assert stub.RUNS["job-t"]["approved"] is False
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Confidence bands: thresholds, the worst-word rule and the blank-mandatory flag (spec: Banding)."""
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
BANDS = Path(__file__).resolve().parents[1] / "rpa-dashboard-ai" / "templates" / "web" / "bands.py"
|
||||
spec = importlib.util.spec_from_file_location("bands", BANDS)
|
||||
bands = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(bands)
|
||||
|
||||
|
||||
def test_thresholds():
|
||||
assert bands.band_of(1.0) == "green"
|
||||
assert bands.band_of(0.95) == "green"
|
||||
assert bands.band_of(0.9499) == "amber"
|
||||
assert bands.band_of(0.90) == "amber"
|
||||
assert bands.band_of(0.8999) == "red"
|
||||
assert bands.band_of(0.0) == "red"
|
||||
|
||||
|
||||
def test_field_takes_its_worst_word():
|
||||
assert bands.field_band([0.99, 0.97, 0.98], "ACME Office Supply") == "green"
|
||||
assert bands.field_band([0.99, 0.93, 0.98], "ACME Office Supply") == "amber"
|
||||
assert bands.field_band([0.99, 0.97, 0.62], "ACME Office Supply") == "red"
|
||||
|
||||
|
||||
def test_blank_mandatory_field_is_always_flagged():
|
||||
assert bands.field_band([1.0], "", mandatory=True) == "blank"
|
||||
assert bands.field_band([], " ", mandatory=True) == "blank"
|
||||
assert bands.field_band(None, None, mandatory=True) == "blank"
|
||||
assert bands.needs_review("blank")
|
||||
# An optional blank is graded like any other field.
|
||||
assert bands.field_band([1.0], "", mandatory=False) == "green"
|
||||
|
||||
|
||||
def test_no_scores_means_unscored():
|
||||
assert bands.field_band([], "INV-1001") == "unscored"
|
||||
|
||||
|
||||
def test_only_green_skips_review():
|
||||
assert not bands.needs_review("green")
|
||||
assert all(bands.needs_review(b) for b in ["amber", "red", "blank", "unscored"])
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Self-check for the AI cost function: rates per million tokens, thinking bills as output,
|
||||
long-context rates above the threshold, and an unknown model is an error, never zero."""
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
SRC = Path(__file__).resolve().parents[1] / "rpa-dashboard-ai" / "templates" / "web" / "pricing.py"
|
||||
spec = importlib.util.spec_from_file_location("pricing", SRC)
|
||||
pricing = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(pricing)
|
||||
|
||||
TABLE = {
|
||||
"flat-model": {"input": 0.30, "output": 2.50},
|
||||
"long-model": {"input": 1.25, "output": 10.00, "long_threshold": 200_000, "input_long": 2.50, "output_long": 15.00},
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def table(monkeypatch):
|
||||
monkeypatch.setattr(pricing, "PRICING", TABLE)
|
||||
|
||||
|
||||
def test_normal_call_prices_input_and_output_per_million():
|
||||
# 1M in at 0.30 + 0.2M out at 2.50 = 0.30 + 0.50
|
||||
assert pricing.cost_for_call("flat-model", 1_000_000, 200_000) == pytest.approx(0.80)
|
||||
|
||||
|
||||
def test_thinking_tokens_bill_at_the_output_rate():
|
||||
# 0.1M in at 0.30 + (0.1M out + 0.3M thinking) at 2.50 = 0.03 + 1.00
|
||||
assert pricing.cost_for_call("flat-model", 100_000, 100_000, 300_000) == pytest.approx(1.03)
|
||||
|
||||
|
||||
def test_long_context_rates_apply_above_the_threshold_only():
|
||||
# at the threshold: 0.2M at 1.25 + 0.1M at 10 = 0.25 + 1.00
|
||||
assert pricing.cost_for_call("long-model", 200_000, 100_000) == pytest.approx(1.25)
|
||||
# above it: 0.4M at 2.50 + 0.1M at 15 = 1.00 + 1.50
|
||||
assert pricing.cost_for_call("long-model", 400_000, 100_000) == pytest.approx(2.50)
|
||||
|
||||
|
||||
def test_unknown_model_raises_and_names_the_known_models():
|
||||
with pytest.raises(ValueError) as e:
|
||||
pricing.cost_for_call("mystery-model", 1000, 1000)
|
||||
assert "mystery-model" in str(e.value)
|
||||
assert "flat-model" in str(e.value) and "long-model" in str(e.value)
|
||||
|
||||
|
||||
def test_bundled_table_records_its_source_and_as_of_date():
|
||||
assert pricing.PRICING_SOURCE.startswith("https://")
|
||||
assert len(pricing.PRICING_AS_OF) == 10 and pricing.PRICING_AS_OF[4] == "-"
|
||||
@@ -0,0 +1,45 @@
|
||||
"""The skills are shared outside the source projects: no patient data, internal IPs,
|
||||
credentials or keys in any bundled file, and the Superbill extracts are marked as examples."""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SKILLS = [ROOT / "rpa-dashboard-ai", ROOT / "rpa-dashboard-no-ai"]
|
||||
AI_REFERENCE = ROOT / "rpa-dashboard-ai" / "templates" / "reference"
|
||||
FORBIDDEN = {
|
||||
"internal IP": r"\b(10\.\d{1,3}|192\.168|172\.(1[6-9]|2\d|3[01]))\.\d{1,3}\.\d{1,3}\b",
|
||||
"Google API key": r"AIza[0-9A-Za-z_-]{30,}",
|
||||
"OpenAI/Anthropic key": r"\bsk-[A-Za-z0-9_-]{20,}",
|
||||
"AWS key": r"\bAKIA[0-9A-Z]{16}\b",
|
||||
"private key": r"-----BEGIN [A-Z ]*PRIVATE KEY-----",
|
||||
"hard-coded secret": r"""(?i)\b(password|passwd|api_?key|secret|client_secret)\b\s*[:=]\s*['"][^'"{}\s]{6,}['"]""",
|
||||
"SSN": r"\b\d{3}-\d{2}-\d{4}\b",
|
||||
"quoted roster name ('SURNAME, FIRST')": r"""["'][A-Z]{2,}, [A-Z]""",
|
||||
"client name": r"Gulf Coast",
|
||||
}
|
||||
MARKER = "EXAMPLE ONLY"
|
||||
|
||||
|
||||
def text_files():
|
||||
for skill in SKILLS:
|
||||
for f in skill.rglob("*"):
|
||||
if f.is_file() and "node_modules" not in f.parts and f.name != "package-lock.json":
|
||||
yield f
|
||||
|
||||
|
||||
@pytest.mark.parametrize("what", FORBIDDEN)
|
||||
def test_no_sensitive_data_in_bundled_files(what):
|
||||
hits = [f"{f.relative_to(ROOT)}:{n}" for f in text_files()
|
||||
for n, line in enumerate(f.read_text(encoding="utf-8", errors="ignore").splitlines(), 1)
|
||||
if re.search(FORBIDDEN[what], line)]
|
||||
assert not hits, f"{what} found: {hits}"
|
||||
|
||||
|
||||
def test_superbill_extracts_are_marked_as_examples():
|
||||
files = [f for f in AI_REFERENCE.rglob("*") if f.is_file()]
|
||||
assert files
|
||||
unmarked = [str(f.relative_to(ROOT)) for f in files
|
||||
if MARKER not in "".join(f.read_text(encoding="utf-8").splitlines(keepends=True)[:5])]
|
||||
assert not unmarked, f"no '{MARKER}' banner in the first lines: {unmarked}"
|
||||
+13
-1
@@ -1,4 +1,5 @@
|
||||
"""Python templates must stay valid Python before their placeholders are filled."""
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
@@ -10,7 +11,8 @@ HOOK_RULES = "E9,F63,F7,F82,F811" # same rule set as the PostToolUse lint hook
|
||||
|
||||
|
||||
@pytest.mark.parametrize("template", ["rpa-dashboard-no-ai/templates/bridge.py",
|
||||
"rpa-dashboard-no-ai/templates/bot_runner.py"])
|
||||
"rpa-dashboard-no-ai/templates/bot_runner.py",
|
||||
"rpa-dashboard-ai/templates/desktop/ai_bridge.py"])
|
||||
def test_template_passes_lint_hook_unfilled(template):
|
||||
ruff = shutil.which("ruff") or pytest.skip("ruff not on PATH")
|
||||
r = subprocess.run([ruff, "check", "--select", HOOK_RULES, "--no-cache", str(ROOT / template)],
|
||||
@@ -23,3 +25,13 @@ def test_tebra_sky_preset_is_the_template_default():
|
||||
themes = (templates / "themes.md").read_text(encoding="utf-8")
|
||||
sky = themes.split("## 1. Tebra Sky")[1].split("```css\n")[1].split("```")[0]
|
||||
assert sky in (templates / "dashboard.html").read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def test_ai_skill_references_files_that_exist():
|
||||
skill = ROOT / "rpa-dashboard-ai"
|
||||
text = (skill / "SKILL.md").read_text(encoding="utf-8")
|
||||
paths = re.findall(r"`((?:templates/|<no-ai>)[\w./-]+)`", text)
|
||||
assert len(paths) > 5
|
||||
missing = [p for p in paths
|
||||
if not (skill / p.replace("<no-ai>", "../rpa-dashboard-no-ai/templates/")).exists()]
|
||||
assert not missing
|
||||
Reference in new issue
Block a user