Initial import: eCW Charge Validation bot and results Portal
Snapshot of the working project as of 2026-10-23 (local 2d4d4fc), one commit with no earlier history: the tracked files were de-identified first (synthetic claim ids and patient data only; see Context.md section 7). Credentials, run logs, screenshots, claim reports and client workbooks are gitignored and not included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ManikandanandClaude Opus 5.5 committed
commit
ba15a574ab
205 files changed
+56772
No files matched your search
@@ -0,0 +1,176 @@
|
||||
# Start the bot from the results web page — design
|
||||
|
||||
Date: 2026-10-14. Approved section by section by the operator in chat
|
||||
(approach A, sections 1–3).
|
||||
|
||||
## Goal
|
||||
|
||||
The operator starts a Charge Validation run from the local results page
|
||||
(`run_web.bat` → http://127.0.0.1:8000) instead of the terminal. While it
|
||||
runs, the page shows live status; it can stop it; when it ends, the page
|
||||
shows that run's results.
|
||||
|
||||
## Decisions (operator, 2026-10-14)
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| Controls | Start + live status + Stop (no settings editing on the page) |
|
||||
| Save mode | **VERIFY only.** A page-started run is always `SAVE_CORRECTIONS=0`, whatever `.env` says. LIVE-WRITE stays terminal-only. |
|
||||
| Message boxes | **Off** for the run's information boxes (per-claim, review-before-close, "Run the Lookup?", summaries). Boxes that need a person to act in the browser — CAPTCHA, 2FA manual step, security image — **stay on**. |
|
||||
| Approach | A: the web server starts `run.py` as a separate child process (not a thread, not Task Scheduler). |
|
||||
|
||||
Out of scope (YAGNI): editing settings from the page, scheduling, more than
|
||||
one run at a time, LIVE-WRITE from the page.
|
||||
|
||||
## Components
|
||||
|
||||
### New: `web/bot_runner.py` — process control only
|
||||
|
||||
- `start() -> dict` — starts `python run.py` (the project's own `run.py`,
|
||||
`sys.executable`, cwd = project root) as a child process with no console
|
||||
window of its own (`CREATE_NO_WINDOW`); the Chrome window still shows.
|
||||
The child's stdout/stderr go to `Logs\web_run_<ts>.out`, which catches a
|
||||
crash before the bot opens its own log; the bot's `Logs\run_<ts>.log`
|
||||
(named in the lock file) is the status source.
|
||||
Refuses (raises `AlreadyRunning`) when the lock file names a live pid.
|
||||
- `stop() -> dict` — `taskkill /PID <pid> /T /F`: the bot and its Chrome
|
||||
close together. Clears the lock file. Status becomes "Stopped by
|
||||
operator".
|
||||
- `status() -> dict` — `running` (bool), `pid`, `started_at`, `log_path`,
|
||||
`exit_code` (when this web process started it and it has ended),
|
||||
`stopped_by_operator`.
|
||||
- The command is fixed. Nothing from the request reaches the command line
|
||||
or the environment.
|
||||
|
||||
**Child environment** — set on the child process, so it wins over `.env`
|
||||
(`settings.py` loads `.env` with `os.environ.setdefault`):
|
||||
|
||||
```
|
||||
SAVE_CORRECTIONS=0
|
||||
ECW_KEEP_OPEN=0 # no "Press Enter to close" at the end
|
||||
QUIET_BOXES=1 # run-flow information boxes go to the log
|
||||
CONFIRM_BEFORE_LOOKUP=0
|
||||
```
|
||||
|
||||
Everything else follows `.env` (claim file, `CLAIM_PROCESS_LIMIT`,
|
||||
`RESULTS_DB`, proxy, ...).
|
||||
|
||||
### Lock file: `Logs\bot_run.json`
|
||||
|
||||
`{"pid", "started_at", "log_path", "source": "web" | "terminal"}`.
|
||||
|
||||
- Written by the **bot itself** at the start of `main()` (after
|
||||
`runlog.start()`, so `log_path` is known) and removed when `main()`
|
||||
returns — so a terminal run is seen by the page too.
|
||||
- Before writing it, `main()` refuses to start (exit 2, before login) when
|
||||
the file names a pid that is alive: two runs would share one eCW login
|
||||
guard and one Charge Summary.
|
||||
- A file naming a dead pid is stale: removed and ignored (a killed run
|
||||
cannot remove its own file).
|
||||
- `bot_runner.start()` checks it too, so the page answers 409 at once.
|
||||
|
||||
### New: `web/run_status.py` — read the run log
|
||||
|
||||
Pure functions over log text (no I/O except reading the file tail):
|
||||
|
||||
- **Phase** from the last matching line:
|
||||
`[LOGIN]` / `[CAPTCHA]` → "Logging in"; `[OTP]` / `[2FA]` → "Waiting for
|
||||
the 2FA approval mail"; `[OK] Logged in` → "Logged in"; `[CLAIMS]` →
|
||||
"Checking claims"; process ended → "Finished" or a failure text by exit
|
||||
code.
|
||||
- **Progress** from `[CLAIMS] --- claim n/N: <id> ---` → `n`, `N`, current
|
||||
claim id.
|
||||
- **Counts** from the new `[RESULT]` lines (below) → Pass / Fail / Review.
|
||||
- **Tail**: the last 30 lines.
|
||||
- **Exit codes** (from `flows/export_medcor_ready.main`): 0 finished; 2
|
||||
config / mode gate; 3 login guard refused (its `[GUARD]` line shown); 5
|
||||
claims phase failed; 6 2FA timed out; other → "ended with code N".
|
||||
|
||||
### Changed: `flows/export_medcor_ready.py`
|
||||
|
||||
- After each claim's outcome is known in `_process_claims`, one
|
||||
machine-readable line (terminal runs print it too):
|
||||
|
||||
```
|
||||
[RESULT] claim 3/7 9126000: Review
|
||||
```
|
||||
|
||||
The word is the Category the Charge Summary gets for that claim; a claim
|
||||
that gets no Charge Summary row (not found in the grid) prints
|
||||
`Not found`.
|
||||
- `main()` writes / removes the lock file as above.
|
||||
|
||||
### Changed: `core/msgbox.py`
|
||||
|
||||
- New setting `QUIET_BOXES` (default off; `settings.py`).
|
||||
- `info_box(title, msg, needs_person=False)`: with `QUIET_BOXES=1` and
|
||||
`needs_person=False`, no box — one log line instead:
|
||||
`[BOX] <title>: <first line of msg>`.
|
||||
- `ok_cancel` / `confirm` are unchanged (every remaining caller is a
|
||||
person-needed step: CAPTCHA, 2FA manual, LIVE-WRITE confirm — the last
|
||||
is never reached in VERIFY). The security-image `info_box` in
|
||||
`auth/login.py` passes `needs_person=True`.
|
||||
|
||||
### Changed: `web/app.py` + `web/templates/index.html`
|
||||
|
||||
Endpoints:
|
||||
|
||||
| Method | Path | Answer |
|
||||
|---|---|---|
|
||||
| POST | `/api/bot/start` | 200 `{status}`; 409 already running; 403 bad Origin/token |
|
||||
| POST | `/api/bot/stop` | 200 `{status}`; 409 nothing running; 403 bad Origin/token |
|
||||
| GET | `/api/bot/status` | `{running, phase, progress, counts, tail, started_at, exit_code, message}` |
|
||||
|
||||
Page: a "Bot run" card above the results.
|
||||
- Idle: **[Start run]** and "VERIFY mode — nothing is saved to eCW".
|
||||
- Running (polls `/api/bot/status` every 2 s): phase, progress bar
|
||||
"3 / 7", current claim, Pass/Fail/Review chips, collapsible "Live log"
|
||||
(last 30 lines), **[Stop]** (with a confirm dialog).
|
||||
- Ended: "Finished in 4m 12s — exit 0" (or the failure text); the results
|
||||
table reloads on the newest run.
|
||||
|
||||
## Security
|
||||
|
||||
The page was read-only; Start/Stop are its first actions. The existing
|
||||
TrustedHost check stops DNS rebinding but not a cross-site POST from
|
||||
another tab, which could burn eCW login attempts. So every POST needs both:
|
||||
|
||||
1. `Origin` (or, when absent, `Referer`) is `http://127.0.0.1:8000` or
|
||||
`http://localhost:8000` — else 403.
|
||||
2. Header `X-Run-Token` equals a token made at server start
|
||||
(`secrets.token_urlsafe(32)`) and embedded in the page — else 403.
|
||||
Compared with `secrets.compare_digest`.
|
||||
|
||||
No password on the page; `.env` credentials are read only by the bot.
|
||||
Responses keep `no-store`. The page stays bound to 127.0.0.1. The live log
|
||||
tail can carry PHI, as the results table already does — local only.
|
||||
|
||||
## Error handling
|
||||
|
||||
- Already running (page or terminal) → 409 "A run is already going
|
||||
(started HH:MM)".
|
||||
- Guard refused (exit 3) → the `[GUARD]` line shown ("next attempt
|
||||
allowed at ...").
|
||||
- Stop mid-claim: the claim form may stay open in eCW. VERIFY saves
|
||||
nothing; the next run's `clear_screen` closes it.
|
||||
- Web server restarted or page closed: the bot keeps running; on reload
|
||||
the page finds it through the lock file. `exit_code` is unknown then
|
||||
(the run is shown "Ended" when its pid is gone).
|
||||
- Log file not there yet → phase "Starting".
|
||||
|
||||
## Testing (standalone "N/M passed" scripts)
|
||||
|
||||
- `tests/test_bot_runner.py` — a fake child script in place of `run.py`
|
||||
(injectable command): start → running → stop → stopped; second start →
|
||||
`AlreadyRunning`; stale lock (dead pid) cleared; the child sees the forced
|
||||
env (`SAVE_CORRECTIONS=0`, `ECW_KEEP_OPEN=0`, `QUIET_BOXES=1`,
|
||||
`CONFIRM_BEFORE_LOOKUP=0`) even when the parent env says otherwise.
|
||||
- `tests/test_run_status.py` — sample log text → phase, progress, counts,
|
||||
tail; each exit code's text.
|
||||
- `tests/test_web_app.py` (added cases) — POST without/with wrong Origin or
|
||||
token → 403; right Origin + token → 200 (runner stubbed); status JSON
|
||||
shape; the token is in the page.
|
||||
- `tests/test_msgbox.py` (added cases) — `QUIET_BOXES=1`: info box logged,
|
||||
not shown; `needs_person=True` still shown.
|
||||
- Flow test — `[RESULT]` line per claim; `main()` lock file written,
|
||||
refused on a live one, removed at the end.
|
||||
Reference in new issue
Block a user