From d500f750da37cd7e9884393a4d41f31b6c2f9233 Mon Sep 17 00:00:00 2001 From: ManikandanC Date: Fri, 30 Oct 2026 10:58:35 +0530 Subject: [PATCH] Self-healing agent (supervisor, recovery, login fixes, operator mail) and Portal sage redesign Snapshot of D:\Ecw Process a11c9ff (2026-10-30). Co-Authored-By: Claude Opus 5.5 --- Context.md | 92 ++++++++- agent.py | 17 ++ src/ecw_bot/auth/login.py | 41 +++- src/ecw_bot/auth/twofactor.py | 72 +++++-- src/ecw_bot/core/notify.py | 86 ++++++++ src/ecw_bot/flows/export_medcor_ready.py | 94 ++++++--- src/ecw_bot/flows/recovery.py | 96 +++++++++ src/ecw_bot/flows/supervisor.py | 209 ++++++++++++++++++++ src/ecw_bot/pages/sliding_fee.py | 4 + src/ecw_bot/settings.py | 20 ++ tests/test_claim_boxes.py | 9 +- tests/test_clean_between_claims.py | 9 +- tests/test_login_username_step.py | 28 +++ tests/test_msgbox.py | 3 + tests/test_notify.py | 130 ++++++++++++ tests/test_portal_signin.py | 3 +- tests/test_recovery.py | 168 ++++++++++++++++ tests/test_save_corrections_gate.py | 9 +- tests/test_supervisor.py | 140 +++++++++++++ tests/test_twofa_late_approval.py | 134 +++++++++++++ tests/test_watchdog_flow.py | 4 +- tests/test_web_app.py | 34 +++- web/app.py | 28 ++- web/templates/base.html | 214 +++++++++++--------- web/templates/dashboard.html | 242 +++++++++++++---------- web/templates/signin.html | 175 ++++++++-------- 26 files changed, 1704 insertions(+), 357 deletions(-) create mode 100644 agent.py create mode 100644 src/ecw_bot/core/notify.py create mode 100644 src/ecw_bot/flows/recovery.py create mode 100644 src/ecw_bot/flows/supervisor.py create mode 100644 tests/test_notify.py create mode 100644 tests/test_recovery.py create mode 100644 tests/test_supervisor.py create mode 100644 tests/test_twofa_late_approval.py diff --git a/Context.md b/Context.md index a40e485..add0d49 100644 --- a/Context.md +++ b/Context.md @@ -40,11 +40,50 @@ Repo: `D:\Ecw Process`, git branch `master`, **no remote** (never pushed). ```bash python run.py # the bot (logs into LIVE eCW) +python agent.py # the agent: the whole day's queue, self-healing (below) python tests/test_.py # one test file, prints "N/M passed" run_web.bat # results portal, http://127.0.0.1:8000 start_ecw_chrome.bat # Chrome the bot attaches to (BROWSER_CDP_URL) ``` +**The agent (2026-10-30, operator: "we are an agent, not a bot — observe, +fix, continue, and mail me what happened").** `python agent.py` +(`flows/supervisor.py`) runs the bot as a child process, pass after pass, +until every claim in the day's Medcor Ready queue has a result. Pass 1 +downloads the live queue; a later pass takes only the claims still +pending — the queue file minus every `CV_ChargeSummary` row since the +agent started, a "Bot could not ..." Review not counting — through +`RESUME_CLAIM_IDS_FILE` (live mode, live Charge Summary, no TEST label, +no second export). After each pass, by exit code (`next_action`): +login failure (1, 6) → retry after 10 then 20 min, the 3rd in a row stops +the agent (the client's account must not be restricted); restricted (4), +screen changed (7), config (2) → stop at once; stuck page (9), stall (8), +claims error (5) → fresh Chrome + login in 1 min; guard wait (3) → wait; +a clean pass leaving the same claims pending → stop ("left for a +person"); 15 passes a day at most. VERIFY only: refuses to start with +`SAVE_CORRECTIONS=1`; forces `QUIET_BOXES=1`, `SHOW_EACH_CLAIM=0`, +`CLAIM_PROCESS_LIMIT=0`. Every failure and the end are mailed +(`core/notify.py`, from the 2FA mailbox's account over SMTP — `SMTP_*` +default to `IMAP_*`, port 465; **`NOTIFY_EMAIL_TO` blank = no mail, and it +is blank on this PC until the operator gives the address**). A mail +carries no claim number (6+ digit runs masked) and only login / error +page PNGs (`notify._EMAILABLE`) — never a claim screen or page HTML. +Inside a pass, `flows/recovery.py`: a claim the bot could not search or +read → login page on screen = session over (exit 9); else Escape, close +Patient Information, clear the screen, reopen the claim search if still +blocked, and try the claim once more; 2 claims in a row still failing +after recovery → exit 9 (live 2026-10-29: a Dental claim's sliding fee +check left Patient Information open and 18 claims went "Bot could not +search" one after another). Login: the security image page's Continue is +clicked by the bot (`login.continue_security_image`, a person only when no +Continue/OK is found; the page is saved as `security_image_page` the first +time); a 2FA link approved but not picked up in `OTP_PICKUP_SECONDS` (60) +→ Resend OTP + approve the new mail, `OTP_PICKUP_RESENDS` (2) times +(live 2026-10-29 night: three logins lost to mail 6 to 8.5 min late with +Resend still on screen). The "Chrome open" and security image boxes are +quiet with `QUIET_BOXES=1` (a `[BOX]` log line); CAPTCHA and manual 2FA +boxes still show — they need a person. **None of this has run live yet.** + **Watching a run live (2026-10-22, `8af5c14`):** `start_ecw_chrome.bat` opens Google Chrome with its own profile `D:\EcwChrome` (`CDP_PROFILE_DIR`), debug port 9222 on 127.0.0.1, the eCW proxy (claude.ai / anthropic.com go @@ -305,6 +344,9 @@ that box and on the per-claim save box means Cancel. | `core/page_utils.py` | `shot()` with a FAILURE tag (`is_failure_tag`: failed / missing / not_ / no_ / unconfirmed ...) also saves the page's DOM + frames beside the png as `_.html` (`SAVE_DOM_ON_FAILURE=1`, at most `SAVE_DOM_LIMIT=30` a run; 2026-10-21) — no more hand-copying DevTools into Dom.txt. `visible()` = instant check; `appears()` = real wait (Playwright 1.62's `is_visible(timeout)` never waits); `click_topmost()` clicks the copy on top when dialogs stack (never `.first`) | | `core/fallback.py` + `selectors/fallbacks.py` | Fallback selectors (2026-10-21): the run-critical selectors are CHAINS — primary first, then alternatives taken from the live DOM (ng-click / ng-model / type+value, plain CSS) — joined as a CSS list, so every call site falls back with no change: password field, Log In, Claim No, Lookup, UB Cancel (`closeUBClaimScreen()`, never the X or OK), Dental Cancel (`closeDntl()`), Prog. Notes (`#UBClaimProgressNoteBtn`, `claimProgressNoteBtn`). `report()` logs `[SELECTOR] fallback: — not found, matched ` once a run (pre-flight and login call it) | | `pages/preflight.py` | Pre-flight selector check (2026-10-21): the claims search page before the first claim, the claim form on the first claim of each screen type (UB / HCFA / Dental: Cancel, CPT tab, rows with code / fee / POS / M1). Missing → `[PREFLIGHT] FAIL `, page shot + DOM saved, run stops with **exit 7** (`PREFLIGHT_STOP=0` only warns). No CPT row on a claim → its row checks wait for the next; 3 in a row = rows missing | +| `flows/supervisor.py` + root `agent.py` | The agent (2026-10-30, §2): passes of the bot as a child process until nothing is pending; `next_action(code, pending, state)` is the whole retry policy, `pending(queue, rows)` the resume list | +| `flows/recovery.py` | A stuck screen inside a pass (§2): `recover()` (logout check, Escape + Patient Information close + clear, reopen search) and `run_claim()` (one retry per claim, 2 in a row → `PageStuck`, exit 9) | +| `core/notify.py` | The agent's mail (§2): `incident()` / `summary()`; claim numbers masked, login / error PNGs only; never fatal | | `core/watchdog.py` | Claim watchdog (2026-10-22): `watching(limit)` around the claim loop, `beat(where)` at each claim step and every message box, `check()` raises `Stalled` → exit 8; on a stall it closes this process's Playwright driver (`node.exe`) so the frozen call returns, and exits 8 itself if the main thread never comes back | | `core/run_events.py` | A Run's timed events → `Logs\events_.jsonl` (`start`, `emit`, `warning`, `claim_error`, `run_error` with traceback, `login_attempt_start/end`, `login_step`); never raises, never writes an OTP code / password / username | | `core/browser.py` | `launch()`: Playwright Chromium with the eCW proxy, or (`BROWSER_CDP_URL`) attach to the operator's Chrome — `clear_ecw_data()` at attach and close, `cdp_ready()` checked before login | @@ -583,6 +625,22 @@ the Portal (restarted 2026-10-26, [portal] terminal tab). header `X-Run-Token` must match the token in the page; else 403), `GET /api/bot/status`. Spec / plan: `docs/superpowers/specs/2026-10-14-start-bot-from-web-page-design.md`. +- **Sage redesign (2026-10-30, the operator-approved mockup, live on + Dokploy that day from `portal-20261030-ui.zip`).** `base.html`: the navy + header became a sage side menu (`