ecw-charge-validation/docs/superpowers/plans/2026-10-08-results-db-and-web-page.md

Results DB + Web Page Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Store every run's Charge Summary rows in SQL Server (Charge_Validation.dbo.ChargeSummary) and show them on a local FastAPI page.

Architecture: A pure module core/results_db.py turns Charge Summary rows (the same rows the Excel file gets) into typed DB records and inserts them with one parameterized statement; the flow calls it after the Excel write, behind RESULTS_DB, and any DB error is a log line only. A separate web/ FastAPI app reads the table through a small repository class and renders one Jinja2 page plus two JSON endpoints, bound to 127.0.0.1.

Tech Stack: Python 3.13, pyodbc + ODBC Driver 18 for SQL Server, FastAPI 0.141 + Jinja2 + uvicorn, Starlette TestClient (httpx). All already installed.

Spec: docs/superpowers/specs/2026-10-08-results-db-and-web-page-design.md

Global Constraints

• Server MECARE331\MECARE331, database Charge_Validation, table dbo.ChargeSummary.
• Default connection: DRIVER={ODBC Driver 18 for SQL Server};SERVER=MECARE331\MECARE331;DATABASE=Charge_Validation;Trusted_Connection=yes;TrustServerCertificate=yes;
• RESULTS_DB default off. The bot never creates/alters tables; the operator runs tools/sql/create_tables.sql in SSMS.
• A DB failure never stops the run and never touches the Excel summary.
• Only parameterized SQL; web is read-only (SELECT only).
• Web bound to 127.0.0.1 only. No external scripts/CDNs.
• No new entries in requirements.txt. Tests are standalone scripts (python tests\test_x.py, prints N/M passed), no pytest, no real DB.
• Never commit unless the operator asks; never commit run.py, Input/, Output/, Logs/, Screenshots/, Dom.txt, client .xlsx.

Rulings (plan vs spec)

• Ruling: the flow builds the sheet once more for the DB (charge_summary.build) instead of threading one sheet through write() — keeps write() and its tests unchanged; both builds read the same report and outcomes on the same day, so the rows are identical — cost if wrong: a row differing between Excel and DB, visible on the page.
• Ruling: only rows the run filled in (non-blank Category) are saved — unprocessed / not-found rows are blank in the Excel too and would only clutter the page — cost if wrong: the page lacks blank rows the Excel has.

File map

File	Responsibility
tools/sql/create_tables.sql	Table + indexes, run by the operator
src/ecw_bot/settings.py	RESULTS_DB, RESULTS_DB_CONN
src/ecw_bot/core/results_db.py	Column map, value conversion, details_for, insert_rows, save_summary
src/ecw_bot/core/charge_summary.py	build() split out of write()
src/ecw_bot/flows/export_medcor_ready.py	changes in _conditions, details dict, _run_mode(), _save_results_db()
web/__init__.py, web/repo.py, web/app.py, web/templates/index.html	Read-only page + JSON API
run_web.bat	Start the page on 127.0.0.1:8000
tests/test_results_db.py, tests/test_web_app.py	Tests

---

Task 1: SQL script, settings, results_db module

Files:

• Create: tools/sql/create_tables.sql, src/ecw_bot/core/results_db.py, tests/test_results_db.py
• Modify: src/ecw_bot/settings.py (after SUMMARY_ASSIGNED_TO)

Interfaces:

• Produces: results_db.COLUMNS: list[tuple[str, str]] (header, db column), DB_COLUMNS: list[str], TABLE = "dbo.ChargeSummary", to_money(v) -> Decimal|None, to_date(v) -> date|None, to_record(row: dict, *, run_at, run_mode, details) -> dict, details_for(data: dict) -> dict, insert_rows(conn, records) -> int, save_summary(sheet, details, *, run_at, run_mode, conn_str, connect=None) -> int|None; settings RESULTS_DB: bool, RESULTS_DB_CONN: str.

• Step 1: Write the failing tests — tests/test_results_db.py with cases: every HEADER mapped (24) and DB_COLUMNS = run_at, run_mode, 24 columns, details_json; to_money("$1,234.50") == Decimal("1234.50"), "(12.00)" → -12.00, blank/"abc"/"NaN"/"1e20" → None, 365 → 365.00; to_date("7/31/2026") == date(2026,7,31), ISO and datetime accepted, blank/junk → None; to_record converts types, blank text → None, details_json is JSON or None; details_for copies _conditions keys, Dental results, error keys; insert_rows sends one INSERT INTO dbo.ChargeSummary (...) VALUES (?, ...) with values in DB_COLUMNS order and commits once; save_summary skips blank-Category rows, closes the connection, returns the count; a raising connect returns None and prints [RESULTS-DB]. (Full code: see the test file written in Step 1 of execution.)

• Step 2: Run python tests\test_results_db.py — Expected: ImportError (no results_db).

• Step 3: Implement results_db.py per the Interfaces above (money: strip $ ,, (x) negative, finite and < 10**10, quantize 0.01; dates: mm/dd/yyyy, yyyy-mm-dd, date/datetime; INSERT built only from DB_COLUMNS names and ? markers; save_summary wraps everything in try/except Exception → [RESULTS-DB] Could not save ... + None). Settings:

# ── Results database (SQL Server) ─────────────────────────────────────────────
# Also save the Charge Summary rows to SQL Server (core/results_db.py) for the
# results web page (web/). Off by default; a DB error is a log line only.
RESULTS_DB = _env_bool("RESULTS_DB", False)
RESULTS_DB_CONN = _env(
    "RESULTS_DB_CONN",
    "DRIVER={ODBC Driver 18 for SQL Server};SERVER=MECARE331\\MECARE331;"
    "DATABASE=Charge_Validation;Trusted_Connection=yes;"
    "TrustServerCertificate=yes;")

and tools/sql/create_tables.sql = the spec's CREATE TABLE wrapped in IF OBJECT_ID('dbo.ChargeSummary','U') IS NULL BEGIN ... END.

• Step 4: Run python tests\test_results_db.py — Expected: all passed.

Task 2: charge_summary.build() and the flow hook

Files:

• Modify: src/ecw_bot/core/charge_summary.py (write()), src/ecw_bot/flows/export_medcor_ready.py (_read_claim_form _conditions, claim loop, after _write_charge_summary)
• Test: tests/test_results_db.py (flow cases), tests/test_charge_summary.py (unchanged, must stay green)

Interfaces:

• Consumes: results_db.details_for, results_db.save_summary, cfg.RESULTS_DB, cfg.RESULTS_DB_CONN.

• Produces: charge_summary.build(report_path, outcomes, assigned_to) -> list (header + rows); flow _run_mode() -> str ("TEST" / "LIVE-WRITE" / "VERIFY"), _save_results_db(info, outcomes, details) -> None.

• Step 1: Failing tests in tests/test_results_db.py: _run_mode() for the three settings combinations; _save_results_db with RESULTS_DB off calls nothing; on, it calls charge_summary.build(info["file"], outcomes, cfg.SUMMARY_ASSIGNED_TO) and results_db.save_summary(sheet, details, run_at=<datetime>, run_mode=..., conn_str=cfg.RESULTS_DB_CONN); a raising build prints [RESULTS-DB] and does not raise.

• Step 2: Run — Expected: AttributeError (_run_mode missing).

• Step 3: Implement: move write()'s read+build_rows into build(), write() calls build(); in _read_claim_form add "changes": {r.condition_name: r.changes for r in results} to data["_conditions"]; in the claim loop details = {} and details[claim_id] = results_db.details_for(data); after _write_charge_summary(info, outcomes) call _save_results_db(info, outcomes, details).

• Step 4: Run tests\test_results_db.py, tests\test_charge_summary.py, tests\test_claim_boxes.py — Expected: all passed.

Task 3: Web page

Files:

• Create: web/__init__.py, web/repo.py, web/app.py, web/templates/index.html, run_web.bat, tests/test_web_app.py

Interfaces:

• Consumes: results_db.COLUMNS, results_db.TABLE, cfg.RESULTS_DB_CONN.

• Produces: web.repo.SqlResultsRepo(conn_str, connect=None) with runs() -> list[dict] (run_at, run_mode, claims, pass_count, fail_count, review_count, newest first) and results(run_at, category=None, claim_no=None) -> list[dict]; web.app.create_app(repo) -> FastAPI; module-level web.app.app.

• Step 1: Failing tests tests/test_web_app.py with a fake repo: / lists rows of the latest run, shows Pass/Fail/Review counts and the Charge Summary headers; ?category=Review passes the filter to the repo; ?category=Bad → 400; ?claim_no=55-51 searches 5551… digits only; ?run_at=not-a-date → 400; /api/runs and /api/results JSON shapes (dates ISO, money numbers); repo raising → 503 with "Results database not reachable"; no runs → page says no results yet; SqlResultsRepo.results builds a parameterized query (fake connection) with category = ? and claim_no LIKE ?.

• Step 2: Run — Expected: ImportError.

• Step 3: Implement repo, app (docs_url=None), template (form: run select, category select, claim search; tiles; table in Charge Summary order with Category colour; <details> per row with pretty JSON), run_web.bat (python -m uvicorn web.app:app --host 127.0.0.1 --port 8000).

• Step 4: Run python tests\test_web_app.py — Expected: all passed.

Task 4: Docs and full suite

• Context.md: a "Results DB + web page" section (setup: run tools/sql/create_tables.sql, grant the Windows login read/write, RESULTS_DB=1, run_web.bat).
• Full suite: bash run_tests.sh → TOTAL all passed.
• Report to the operator; commit only when asked.