ManikandanandClaude Opus 5.5
ba15a574ab
Snapshot of the working project as of 2026-10-23 (local 2d4d4fc), one commit with no earlier history: the tracked files were de-identified first (synthetic claim ids and patient data only; see Context.md section 7). Credentials, run logs, screenshots, claim reports and client workbooks are gitignored and not included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
109 lines
9.6 KiB
Markdown
109 lines
9.6 KiB
Markdown
# Results DB + Web Page Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Store every run's Charge Summary rows in SQL Server (`Charge_Validation.dbo.ChargeSummary`) and show them on a local FastAPI page.
|
|
|
|
**Architecture:** A pure module `core/results_db.py` turns Charge Summary rows (the same rows the Excel file gets) into typed DB records and inserts them with one parameterized statement; the flow calls it after the Excel write, behind `RESULTS_DB`, and any DB error is a log line only. A separate `web/` FastAPI app reads the table through a small repository class and renders one Jinja2 page plus two JSON endpoints, bound to 127.0.0.1.
|
|
|
|
**Tech Stack:** Python 3.13, pyodbc + ODBC Driver 18 for SQL Server, FastAPI 0.141 + Jinja2 + uvicorn, Starlette TestClient (httpx). All already installed.
|
|
|
|
**Spec:** `docs/superpowers/specs/2026-10-08-results-db-and-web-page-design.md`
|
|
|
|
## Global Constraints
|
|
|
|
- Server `MECARE331\MECARE331`, database `Charge_Validation`, table `dbo.ChargeSummary`.
|
|
- Default connection: `DRIVER={ODBC Driver 18 for SQL Server};SERVER=MECARE331\MECARE331;DATABASE=Charge_Validation;Trusted_Connection=yes;TrustServerCertificate=yes;`
|
|
- `RESULTS_DB` default **off**. The bot never creates/alters tables; the operator runs `tools/sql/create_tables.sql` in SSMS.
|
|
- A DB failure never stops the run and never touches the Excel summary.
|
|
- Only parameterized SQL; web is read-only (`SELECT` only).
|
|
- Web bound to `127.0.0.1` only. No external scripts/CDNs.
|
|
- No new entries in `requirements.txt`. Tests are standalone scripts (`python tests\test_x.py`, prints `N/M passed`), no pytest, no real DB.
|
|
- Never commit unless the operator asks; never commit `run.py`, `Input/`, `Output/`, `Logs/`, `Screenshots/`, `Dom.txt`, client `.xlsx`.
|
|
|
|
## Rulings (plan vs spec)
|
|
|
|
- Ruling: the flow builds the sheet once more for the DB (`charge_summary.build`) instead of threading one sheet through `write()` — keeps `write()` and its tests unchanged; both builds read the same report and outcomes on the same day, so the rows are identical — cost if wrong: a row differing between Excel and DB, visible on the page.
|
|
- Ruling: only rows the run filled in (non-blank Category) are saved — unprocessed / not-found rows are blank in the Excel too and would only clutter the page — cost if wrong: the page lacks blank rows the Excel has.
|
|
|
|
## File map
|
|
|
|
| File | Responsibility |
|
|
|---|---|
|
|
| `tools/sql/create_tables.sql` | Table + indexes, run by the operator |
|
|
| `src/ecw_bot/settings.py` | `RESULTS_DB`, `RESULTS_DB_CONN` |
|
|
| `src/ecw_bot/core/results_db.py` | Column map, value conversion, `details_for`, `insert_rows`, `save_summary` |
|
|
| `src/ecw_bot/core/charge_summary.py` | `build()` split out of `write()` |
|
|
| `src/ecw_bot/flows/export_medcor_ready.py` | `changes` in `_conditions`, `details` dict, `_run_mode()`, `_save_results_db()` |
|
|
| `web/__init__.py`, `web/repo.py`, `web/app.py`, `web/templates/index.html` | Read-only page + JSON API |
|
|
| `run_web.bat` | Start the page on 127.0.0.1:8000 |
|
|
| `tests/test_results_db.py`, `tests/test_web_app.py` | Tests |
|
|
|
|
---
|
|
|
|
### Task 1: SQL script, settings, `results_db` module
|
|
|
|
**Files:**
|
|
- Create: `tools/sql/create_tables.sql`, `src/ecw_bot/core/results_db.py`, `tests/test_results_db.py`
|
|
- Modify: `src/ecw_bot/settings.py` (after `SUMMARY_ASSIGNED_TO`)
|
|
|
|
**Interfaces:**
|
|
- Produces: `results_db.COLUMNS: list[tuple[str, str]]` (header, db column), `DB_COLUMNS: list[str]`, `TABLE = "dbo.ChargeSummary"`, `to_money(v) -> Decimal|None`, `to_date(v) -> date|None`, `to_record(row: dict, *, run_at, run_mode, details) -> dict`, `details_for(data: dict) -> dict`, `insert_rows(conn, records) -> int`, `save_summary(sheet, details, *, run_at, run_mode, conn_str, connect=None) -> int|None`; settings `RESULTS_DB: bool`, `RESULTS_DB_CONN: str`.
|
|
|
|
- [ ] **Step 1: Write the failing tests** — `tests/test_results_db.py` with cases: every HEADER mapped (24) and `DB_COLUMNS` = run_at, run_mode, 24 columns, details_json; `to_money("$1,234.50") == Decimal("1234.50")`, `"(12.00)"` → -12.00, blank/"abc"/"NaN"/"1e20" → None, `365` → 365.00; `to_date("7/31/2026") == date(2026,7,31)`, ISO and datetime accepted, blank/junk → None; `to_record` converts types, blank text → None, `details_json` is JSON or None; `details_for` copies `_conditions` keys, Dental results, error keys; `insert_rows` sends one `INSERT INTO dbo.ChargeSummary (...) VALUES (?, ...)` with values in `DB_COLUMNS` order and commits once; `save_summary` skips blank-Category rows, closes the connection, returns the count; a raising `connect` returns None and prints `[RESULTS-DB]`. (Full code: see the test file written in Step 1 of execution.)
|
|
|
|
- [ ] **Step 2: Run** `python tests\test_results_db.py` — Expected: ImportError (no `results_db`).
|
|
|
|
- [ ] **Step 3: Implement** `results_db.py` per the Interfaces above (money: strip `$ ,`, `(x)` negative, finite and `< 10**10`, quantize 0.01; dates: `mm/dd/yyyy`, `yyyy-mm-dd`, `date`/`datetime`; `INSERT` built only from `DB_COLUMNS` names and `?` markers; `save_summary` wraps everything in `try/except Exception` → `[RESULTS-DB] Could not save ...` + None). Settings:
|
|
|
|
```python
|
|
# ── Results database (SQL Server) ─────────────────────────────────────────────
|
|
# Also save the Charge Summary rows to SQL Server (core/results_db.py) for the
|
|
# results web page (web/). Off by default; a DB error is a log line only.
|
|
RESULTS_DB = _env_bool("RESULTS_DB", False)
|
|
RESULTS_DB_CONN = _env(
|
|
"RESULTS_DB_CONN",
|
|
"DRIVER={ODBC Driver 18 for SQL Server};SERVER=MECARE331\\MECARE331;"
|
|
"DATABASE=Charge_Validation;Trusted_Connection=yes;"
|
|
"TrustServerCertificate=yes;")
|
|
```
|
|
|
|
and `tools/sql/create_tables.sql` = the spec's `CREATE TABLE` wrapped in `IF OBJECT_ID('dbo.ChargeSummary','U') IS NULL BEGIN ... END`.
|
|
|
|
- [ ] **Step 4: Run** `python tests\test_results_db.py` — Expected: all passed.
|
|
|
|
### Task 2: `charge_summary.build()` and the flow hook
|
|
|
|
**Files:**
|
|
- Modify: `src/ecw_bot/core/charge_summary.py` (`write()`), `src/ecw_bot/flows/export_medcor_ready.py` (`_read_claim_form` `_conditions`, claim loop, after `_write_charge_summary`)
|
|
- Test: `tests/test_results_db.py` (flow cases), `tests/test_charge_summary.py` (unchanged, must stay green)
|
|
|
|
**Interfaces:**
|
|
- Consumes: `results_db.details_for`, `results_db.save_summary`, `cfg.RESULTS_DB`, `cfg.RESULTS_DB_CONN`.
|
|
- Produces: `charge_summary.build(report_path, outcomes, assigned_to) -> list` (header + rows); flow `_run_mode() -> str` ("TEST" / "LIVE-WRITE" / "VERIFY"), `_save_results_db(info, outcomes, details) -> None`.
|
|
|
|
- [ ] **Step 1: Failing tests** in `tests/test_results_db.py`: `_run_mode()` for the three settings combinations; `_save_results_db` with `RESULTS_DB` off calls nothing; on, it calls `charge_summary.build(info["file"], outcomes, cfg.SUMMARY_ASSIGNED_TO)` and `results_db.save_summary(sheet, details, run_at=<datetime>, run_mode=..., conn_str=cfg.RESULTS_DB_CONN)`; a raising `build` prints `[RESULTS-DB]` and does not raise.
|
|
- [ ] **Step 2: Run** — Expected: AttributeError (`_run_mode` missing).
|
|
- [ ] **Step 3: Implement**: move `write()`'s read+`build_rows` into `build()`, `write()` calls `build()`; in `_read_claim_form` add `"changes": {r.condition_name: r.changes for r in results}` to `data["_conditions"]`; in the claim loop `details = {}` and `details[claim_id] = results_db.details_for(data)`; after `_write_charge_summary(info, outcomes)` call `_save_results_db(info, outcomes, details)`.
|
|
- [ ] **Step 4: Run** `tests\test_results_db.py`, `tests\test_charge_summary.py`, `tests\test_claim_boxes.py` — Expected: all passed.
|
|
|
|
### Task 3: Web page
|
|
|
|
**Files:**
|
|
- Create: `web/__init__.py`, `web/repo.py`, `web/app.py`, `web/templates/index.html`, `run_web.bat`, `tests/test_web_app.py`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `results_db.COLUMNS`, `results_db.TABLE`, `cfg.RESULTS_DB_CONN`.
|
|
- Produces: `web.repo.SqlResultsRepo(conn_str, connect=None)` with `runs() -> list[dict]` (`run_at, run_mode, claims, pass_count, fail_count, review_count`, newest first) and `results(run_at, category=None, claim_no=None) -> list[dict]`; `web.app.create_app(repo) -> FastAPI`; module-level `web.app.app`.
|
|
|
|
- [ ] **Step 1: Failing tests** `tests/test_web_app.py` with a fake repo: `/` lists rows of the latest run, shows Pass/Fail/Review counts and the Charge Summary headers; `?category=Review` passes the filter to the repo; `?category=Bad` → 400; `?claim_no=55-51` searches `5551`… digits only; `?run_at=not-a-date` → 400; `/api/runs` and `/api/results` JSON shapes (dates ISO, money numbers); repo raising → 503 with "Results database not reachable"; no runs → page says no results yet; `SqlResultsRepo.results` builds a parameterized query (fake connection) with `category = ?` and `claim_no LIKE ?`.
|
|
- [ ] **Step 2: Run** — Expected: ImportError.
|
|
- [ ] **Step 3: Implement** repo, app (`docs_url=None`), template (form: run select, category select, claim search; tiles; table in Charge Summary order with Category colour; `<details>` per row with pretty JSON), `run_web.bat` (`python -m uvicorn web.app:app --host 127.0.0.1 --port 8000`).
|
|
- [ ] **Step 4: Run** `python tests\test_web_app.py` — Expected: all passed.
|
|
|
|
### Task 4: Docs and full suite
|
|
|
|
- [ ] Context.md: a "Results DB + web page" section (setup: run `tools/sql/create_tables.sql`, grant the Windows login read/write, `RESULTS_DB=1`, `run_web.bat`).
|
|
- [ ] Full suite: `bash run_tests.sh` → TOTAL all passed.
|
|
- [ ] Report to the operator; commit only when asked.
|