# Start the Bot from the Web Page — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** The operator starts a VERIFY run of the eCW bot from the local results page, watches it live, can stop it, and sees its results when it ends. **Architecture:** The bot records each run in `Logs\bot_run.json` (pid, log path, exit code) and refuses a second live run. The web server starts `run.py` as a child process with forced VERIFY / quiet-box settings, kills its process tree on Stop, and reports status by parsing the bot's own run log. Start/Stop are POSTs guarded by an Origin check and a per-server token. **Tech Stack:** Python 3.13 (`C:\Python313\python.exe`, no venv), FastAPI + Jinja2 + Starlette TestClient, `subprocess`, `ctypes` (kernel32), plain HTML/JS in `web/templates/index.html`. **Spec:** `docs/superpowers/specs/2026-10-14-start-bot-from-web-page-design.md` ## Global Constraints - A page-started run is ALWAYS VERIFY: child env `SAVE_CORRECTIONS=0`, whatever `.env` says. LIVE-WRITE stays terminal-only. - Child env forced exactly: `SAVE_CORRECTIONS=0`, `ECW_KEEP_OPEN=0`, `QUIET_BOXES=1`, `CONFIRM_BEFORE_LOOKUP=0`, `ECW_RUN_SOURCE=web`. Nothing from an HTTP request reaches the command line or the environment. - Boxes that need a person (CAPTCHA, 2FA manual step, security image, LIVE-WRITE confirm) are never quieted. - Every POST: `Origin` (or `Referer` when Origin is absent) must be `http://127.0.0.1:8000` or `http://localhost:8000`, AND header `X-Run-Token` must equal the server's token (`secrets.compare_digest`). Else 403. - The page stays bound to 127.0.0.1; responses keep `no-store`; page text from the log is put in the DOM with `textContent`, never `innerHTML`. - Never `os.kill(pid, 0)` on Windows — it terminates the process. Liveness is `OpenProcess` + `GetExitCodeProcess`. - Tests are standalone scripts printing `N/M passed` (`python tests\.py`); a test never writes into the real `Logs\` folder. - **Commits:** the operator's rule is "never commit unless asked at that moment". Steps below say *stage*; the final task asks the operator before committing. - `run.py` and `tests/test_guard.py` have the operator's own uncommitted edits — never stage them. ## File Structure | File | Responsibility | |---|---| | `src/ecw_bot/core/run_lock.py` (new) | The run record `Logs\bot_run.json`: read / acquire / finish / mark_stopped, `pid_alive()` | | `src/ecw_bot/flows/export_medcor_ready.py` (modify) | `main()` takes and finishes the run record; `[RESULT]` line per claim | | `src/ecw_bot/settings.py` (modify) | `QUIET_BOXES` setting | | `src/ecw_bot/core/msgbox.py` (modify) | `info(..., needs_person=False)` / `info_box(..., needs_person=False)` honour `QUIET_BOXES` | | `src/ecw_bot/auth/login.py` (modify) | security-image `info_box` passes `needs_person=True` | | `web/run_status.py` (new) | Parse run-log text → phase, progress, counts, tail, message | | `web/bot_runner.py` (new) | Start / stop / status of the child bot process | | `web/app.py` (modify) | `/api/bot/start`, `/api/bot/stop`, `/api/bot/status`, token + Origin guard | | `web/templates/index.html` (modify) | "Bot run" card + polling JS | | tests (new) `tests/test_run_lock.py`, `tests/test_run_status.py`, `tests/test_bot_runner.py` | | | tests (modify) `tests/test_msgbox.py`, `tests/test_web_app.py`, `tests/test_mode_banner.py`, `tests/test_test_claim_ids_file.py`, `tests/test_clean_between_claims.py` | | --- ### Task 1: The run record (`core/run_lock.py`) **Files:** - Create: `src/ecw_bot/core/run_lock.py` - Test: `tests/test_run_lock.py` **Interfaces:** - Produces: `run_lock.FILE_NAME = "bot_run.json"`, `run_lock.path() -> Path` (`cfg.LOGS / FILE_NAME`, read at call time), `class Busy(Exception)`, `pid_alive(pid) -> bool`, `read() -> dict | None`, `is_live(rec) -> bool`, `acquire(log_path, source=None) -> dict`, `finish(exit_code) -> None` (never raises), `mark_stopped(pid) -> None`. - Record keys: `pid` (int), `started_at` (ISO seconds str), `log_path` (str), `source` (`"web"`/`"terminal"`), `ended_at` (ISO str or None), `exit_code` (int or None), `stopped_by_operator` (bool). - [ ] **Step 1: Write the failing test** — `tests/test_run_lock.py`: ```python r""" core/run_lock.py — the record of the latest bot run, Logs\bot_run.json (spec 2026-10-14-start-bot-from-web-page-design.md). python tests\test_run_lock.py Every case runs with cfg.LOGS pointed at a temp folder. """ import os import sys import json import pathlib import tempfile import subprocess ROOT = pathlib.Path(__file__).resolve().parent.parent sys.path.insert(0, str(ROOT / "src")) from ecw_bot import settings as cfg # noqa: E402 from ecw_bot.core import run_lock # noqa: E402 CASES = [] def case(fn): CASES.append(fn) return fn def _dead_pid() -> int: p = subprocess.Popen([sys.executable, "-c", "pass"]) p.wait() return p.pid @case def pid_alive_is_true_for_this_python_process(): assert run_lock.pid_alive(os.getpid()) is True @case def pid_alive_is_false_for_junk_and_ended_processes(): for junk in (None, "", "x", 0, -5): assert run_lock.pid_alive(junk) is False, junk assert run_lock.pid_alive(_dead_pid()) is False @case def pid_alive_is_false_for_a_live_process_that_is_not_python(): """A stale record's pid reused by another program must not read as a live bot run.""" p = subprocess.Popen(["ping", "-n", "6", "127.0.0.1"], stdout=subprocess.DEVNULL) try: assert run_lock.pid_alive(p.pid) is False finally: p.kill() p.wait() @case def acquire_writes_this_process_record(): rec = run_lock.acquire(pathlib.Path("Logs/run_x.log"), source="web") assert rec["pid"] == os.getpid() and rec["source"] == "web", rec assert rec["ended_at"] is None and rec["exit_code"] is None, rec on_disk = json.loads(run_lock.path().read_text(encoding="utf-8")) assert on_disk == rec, on_disk @case def acquire_source_defaults_to_the_env_then_terminal(): os.environ.pop("ECW_RUN_SOURCE", None) assert run_lock.acquire("a.log")["source"] == "terminal" run_lock.finish(0) os.environ["ECW_RUN_SOURCE"] = "web" try: assert run_lock.acquire("a.log")["source"] == "web" finally: os.environ.pop("ECW_RUN_SOURCE", None) @case def a_live_record_refuses_a_second_run(): run_lock.acquire("a.log") try: run_lock.acquire("b.log") except run_lock.Busy as exc: assert "already going" in str(exc), exc else: raise AssertionError("second acquire was allowed") @case def a_stale_record_of_a_dead_pid_is_replaced(): run_lock.path().parent.mkdir(parents=True, exist_ok=True) run_lock.path().write_text(json.dumps( {"pid": _dead_pid(), "started_at": "2026-10-14T09:00:00", "log_path": "old.log", "ended_at": None}), encoding="utf-8") rec = run_lock.acquire("new.log") assert rec["log_path"] == "new.log", rec @case def an_ended_record_does_not_block(): run_lock.acquire("a.log") run_lock.finish(0) assert run_lock.acquire("b.log")["log_path"] == "b.log" @case def finish_records_the_exit_code_and_end_time(): run_lock.acquire("a.log") run_lock.finish(5) rec = run_lock.read() assert rec["exit_code"] == 5 and rec["ended_at"], rec assert run_lock.is_live(rec) is False @case def finish_leaves_another_process_record_alone(): run_lock.path().parent.mkdir(parents=True, exist_ok=True) other = {"pid": os.getpid() + 100000, "ended_at": None, "exit_code": None} run_lock.path().write_text(json.dumps(other), encoding="utf-8") run_lock.finish(0) assert run_lock.read() == other @case def finish_never_raises_without_a_record(): run_lock.finish(0) # no file at all run_lock.path().parent.mkdir(parents=True, exist_ok=True) run_lock.path().write_text("{not json", encoding="utf-8") run_lock.finish(0) assert run_lock.read() is None @case def mark_stopped_ends_the_named_run(): rec = run_lock.acquire("a.log") run_lock.mark_stopped(rec["pid"]) rec = run_lock.read() assert rec["stopped_by_operator"] is True and rec["ended_at"], rec assert rec["exit_code"] is None, rec def main() -> int: failures = 0 orig = cfg.LOGS for fn in CASES: name = fn.__name__.replace("_", " ") with tempfile.TemporaryDirectory() as tmp: cfg.LOGS = pathlib.Path(tmp) / "Logs" try: fn() print(f" ok {name}") except AssertionError as exc: print(f" FAIL {name}: {exc}") failures += 1 except Exception as exc: print(f" ERROR {name}: {type(exc).__name__}: {exc}") failures += 1 finally: cfg.LOGS = orig print(f"\n{len(CASES) - failures}/{len(CASES)} passed") return 1 if failures else 0 if __name__ == "__main__": sys.exit(main()) ``` - [ ] **Step 2: Run it to see it fail** Run: `python tests\test_run_lock.py` Expected: `ModuleNotFoundError: No module named 'ecw_bot.core.run_lock'` - [ ] **Step 3: Write `src/ecw_bot/core/run_lock.py`** ```python r""" The run record — `Logs\bot_run.json`, the latest bot run (spec docs/superpowers/specs/2026-10-14-start-bot-from-web-page-design.md). Written by the bot itself at the start of `main()` and finished when `main()` returns, so the results page sees a run started in the terminal as well as one it started. While a run is live a second `main()` refuses to start: two runs would share one eCW login guard and one Charge Summary. {"pid", "started_at", "log_path", "source": "web" | "terminal", "ended_at", "exit_code", "stopped_by_operator"} Live = no `ended_at` and the pid is a running python process. A killed run cannot finish its own record; its pid is gone, so it reads as ended. """ import os import json import ctypes from ctypes import wintypes from datetime import datetime from .. import settings as cfg FILE_NAME = "bot_run.json" _PROCESS_QUERY_LIMITED_INFORMATION = 0x1000 _STILL_ACTIVE = 259 class Busy(Exception): """Another bot run is live.""" def path(): """Read at call time: tests point cfg.LOGS elsewhere.""" return cfg.LOGS / FILE_NAME def _now() -> str: return datetime.now().isoformat(timespec="seconds") def pid_alive(pid) -> bool: """ A running process with this pid whose program is python. Never `os.kill(pid, 0)`: on Windows that TERMINATES the process. The program-name check keeps a stale record whose pid Windows gave to some other program from reading as a live bot run. """ try: pid = int(pid) except (TypeError, ValueError): return False if pid <= 0: return False if os.name != "nt": try: os.kill(pid, 0) return True except OSError: return False k32 = ctypes.WinDLL("kernel32", use_last_error=True) k32.OpenProcess.restype = wintypes.HANDLE k32.OpenProcess.argtypes = [wintypes.DWORD, wintypes.BOOL, wintypes.DWORD] k32.GetExitCodeProcess.argtypes = [wintypes.HANDLE, ctypes.POINTER(wintypes.DWORD)] k32.QueryFullProcessImageNameW.argtypes = [ wintypes.HANDLE, wintypes.DWORD, wintypes.LPWSTR, ctypes.POINTER(wintypes.DWORD)] k32.CloseHandle.argtypes = [wintypes.HANDLE] handle = k32.OpenProcess(_PROCESS_QUERY_LIMITED_INFORMATION, False, pid) if not handle: return False try: code = wintypes.DWORD() if not k32.GetExitCodeProcess(handle, ctypes.byref(code)): return False if code.value != _STILL_ACTIVE: return False buf = ctypes.create_unicode_buffer(1024) size = wintypes.DWORD(len(buf)) if not k32.QueryFullProcessImageNameW(handle, 0, buf, ctypes.byref(size)): return True return "python" in os.path.basename(buf.value).lower() finally: k32.CloseHandle(handle) def read(): """The record, or None when there is none or it cannot be read.""" try: data = json.loads(path().read_text(encoding="utf-8")) except (OSError, ValueError): return None return data if isinstance(data, dict) else None def is_live(rec) -> bool: return bool(rec) and not rec.get("ended_at") and pid_alive(rec.get("pid")) def _write(rec: dict) -> None: target = path() target.parent.mkdir(parents=True, exist_ok=True) tmp = target.with_suffix(".tmp") tmp.write_text(json.dumps(rec, indent=2), encoding="utf-8") os.replace(tmp, target) def acquire(log_path, source=None) -> dict: """This process's record, written; raises Busy over a live one.""" rec = read() if is_live(rec): raise Busy(f"A bot run is already going (pid {rec.get('pid')}, " f"started {rec.get('started_at') or '?'}) — not starting " f"a second one") rec = {"pid": os.getpid(), "started_at": _now(), "log_path": str(log_path), "source": source or os.environ.get("ECW_RUN_SOURCE") or "terminal", "ended_at": None, "exit_code": None, "stopped_by_operator": False} _write(rec) return rec def finish(exit_code) -> None: """End this process's own record. Never raises.""" try: rec = read() if not rec or rec.get("pid") != os.getpid(): return rec.update(ended_at=_now(), exit_code=exit_code) _write(rec) except Exception: pass def mark_stopped(pid) -> None: """The page killed run `pid`: say so on its record.""" rec = read() if not rec or rec.get("pid") != pid or rec.get("ended_at"): return rec.update(ended_at=_now(), exit_code=None, stopped_by_operator=True) _write(rec) ``` - [ ] **Step 4: Run it to see it pass** Run: `python tests\test_run_lock.py` Expected: `12/12 passed` - [ ] **Step 5: Stage** `git add src/ecw_bot/core/run_lock.py tests/test_run_lock.py` --- ### Task 2: The bot takes the run record; one `[RESULT]` line per claim **Files:** - Modify: `src/ecw_bot/flows/export_medcor_ready.py` (imports ~line 49; `main()` ~line 1315; `_process_claims` after `outcomes[claim_id] = outcome` ~line 957) - Modify: `tests/test_mode_banner.py` (`_run_main`), `tests/test_test_claim_ids_file.py` (`_PATCHED` + the main() case), `tests/test_clean_between_claims.py` (`_run`) - Test: `tests/test_run_lock.py` (add cases) **Interfaces:** - Consumes: `run_lock.acquire`, `run_lock.finish`, `run_lock.Busy` (Task 1). - Produces: `main() -> int` unchanged signature; new `_main(log_path) -> int` holds the old body. Log line format consumed by Task 4: `[RESULT] claim {n}/{N} {claim_id}: {Category or "Not found"}`. A busy record → `[RUN] ` and exit code 2. - [ ] **Step 1: Write the failing tests** — append to `tests/test_run_lock.py` before `def main()`: ```python # ── the flow: main() and the [RESULT] line ─────────────────────────────────── import io # noqa: E402 import contextlib # noqa: E402 from ecw_bot.flows import export_medcor_ready as flow # noqa: E402 def _main_with(inner): """flow.main() with runlog stubbed and `_main` replaced by `inner`.""" orig = (flow.runlog.start, flow._main) flow.runlog.start = lambda: pathlib.Path("Logs/run_test.log") flow._main = inner out = io.StringIO() try: with contextlib.redirect_stdout(out): return flow.main(), out.getvalue() finally: flow.runlog.start, flow._main = orig @case def main_records_the_run_and_its_exit_code(): seen = {} def inner(log_path): seen["live"] = run_lock.is_live(run_lock.read()) seen["log"] = str(log_path) return 5 code, _ = _main_with(inner) assert code == 5 assert seen["live"] is True and seen["log"].endswith("run_test.log"), seen rec = run_lock.read() assert rec["exit_code"] == 5 and rec["ended_at"], rec @case def main_finishes_the_record_when_the_run_raises(): def inner(log_path): raise RuntimeError("boom") try: _main_with(inner) except RuntimeError: pass rec = run_lock.read() assert rec["ended_at"] and rec["exit_code"] == 1, rec @case def main_refuses_a_second_run_before_login(): run_lock.acquire("other.log") # this process: live python called = [] code, out = _main_with(lambda log_path: called.append(1) or 0) assert code == 2 and called == [], (code, out) assert "[RUN] A bot run is already going" in out, out @case def each_claim_prints_its_result_line(): orig = (cfg.SHOW_EACH_CLAIM, cfg.SHOW_CLAIM_IDS, cfg.CLAIM_PROCESS_LIMIT, flow.claim_results.search_claim, flow._read_claim_form, flow.claim_filters.clear_claim_no, flow._write_charge_summary, flow._save_results_db, flow.claim_form.clear_screen) cfg.SHOW_EACH_CLAIM = False cfg.SHOW_CLAIM_IDS = False cfg.CLAIM_PROCESS_LIMIT = 0 def search(page, cid): return {"id": cid, "count": 1, "found": cid != "3", "seconds": 1.0} flow.claim_results.search_claim = search flow._read_claim_form = lambda page, cid, head: { "claim_no": cid, "cpt": [{"code": "99213", "checked": True, "fee": "100.00"}], "icd": [{"code": "Z00.00"}], "progress_notes": {"opened": "modal", "text": "Office visit."}, "_conditions": {"fired": [], "saved": False, "failed": [], "review": [], "comments": {}}} flow.claim_filters.clear_claim_no = lambda page: None flow._write_charge_summary = lambda info, o: None flow._save_results_db = lambda info, o, d: None flow.claim_form.clear_screen = lambda page, cid, keep=(): [] out = io.StringIO() try: with contextlib.redirect_stdout(out): flow._process_claims(None, {"ids": ["1", "3"], "file": None}) finally: (cfg.SHOW_EACH_CLAIM, cfg.SHOW_CLAIM_IDS, cfg.CLAIM_PROCESS_LIMIT, flow.claim_results.search_claim, flow._read_claim_form, flow.claim_filters.clear_claim_no, flow._write_charge_summary, flow._save_results_db, flow.claim_form.clear_screen) = orig text = out.getvalue() assert "[RESULT] claim 1/2 1: Pass" in text, text assert "[RESULT] claim 2/2 3: Not found" in text, text ``` - [ ] **Step 2: Run to see the new cases fail** Run: `python tests\test_run_lock.py` Expected: the four new cases FAIL/ERROR (`flow` has no `_main`; no `[RESULT]` line); the first 12 pass. - [ ] **Step 3: Implement in `src/ecw_bot/flows/export_medcor_ready.py`** Add `run_lock` to the `from ..core import (guard, runlog, msgbox, ...)` import list (keep alphabetical-ish placement next to `runlog`). Replace the top of `main()`: ```python def main() -> int: log_path = runlog.start() ``` with: ```python def main() -> int: """ The run record first (`core/run_lock.py`, 2026-10-14): a second live run refuses before login — exit 2, nothing spent — and every way out of `_main()`, a raise included, ends the record with its exit code, so the results page can say how the run ended. """ log_path = runlog.start() try: run_lock.acquire(log_path) except run_lock.Busy as exc: print(f"[RUN] {exc}") return 2 code = 1 try: code = _main(log_path) return code finally: run_lock.finish(code) def _main(log_path) -> int: ``` (The rest of the old body — from `problems = cfg.validate()` to the end — stays exactly as it is, now inside `_main`.) In `_process_claims`, right after `outcomes[claim_id] = outcome`: ```python # One fixed-format line per claim for the results page's live # status (web/run_status.py, 2026-10-14). {} = no Charge Summary # row: the claim was not found in the grid. print(f"[RESULT] claim {n}/{len(todo)} {claim_id}: " f"{(outcome or {}).get('Category') or 'Not found'}") ``` - [ ] **Step 4: Keep the existing main() tests out of the real `Logs\`** `tests/test_mode_banner.py`, in `_run_main`, extend the saved/stubbed tuple: ```python orig = (flow.runlog.start, flow.sync_playwright, flow._login_with_retry, msgbox.ok_cancel, flow.run_lock.acquire, flow.run_lock.finish) flow.runlog.start = lambda: pathlib.Path("unused.log") flow.sync_playwright = _reached_login flow._login_with_retry = _reached_login msgbox.ok_cancel = _box flow.run_lock.acquire = lambda log_path, source=None: {} flow.run_lock.finish = lambda code: None ``` and the restore line: ```python (flow.runlog.start, flow.sync_playwright, flow._login_with_retry, msgbox.ok_cancel, flow.run_lock.acquire, flow.run_lock.finish) = orig ``` `tests/test_test_claim_ids_file.py`: add `(flow.run_lock, "acquire"), (flow.run_lock, "finish"),` to `_PATCHED`, and in the case that calls `flow.main()` (next to `flow.runlog.start = lambda: pathlib.Path("unused.log")`) add: ```python flow.run_lock.acquire = lambda log_path, source=None: {} flow.run_lock.finish = lambda code: None ``` `tests/test_clean_between_claims.py` needs nothing (it calls `_process_claims`, not `main`). - [ ] **Step 5: Run the affected tests** Run: `python tests\test_run_lock.py` → `16/16 passed` Run: `python tests\test_mode_banner.py` → all passed Run: `python tests\test_test_claim_ids_file.py` → all passed Run: `python tests\test_clean_between_claims.py` → `20/20 passed` Then: `dir Logs\bot_run.json` must NOT exist (tests did not write the real Logs). If it exists from a test, find which test wrote it and stub it there. - [ ] **Step 6: Stage** `git add src/ecw_bot/flows/export_medcor_ready.py tests/test_run_lock.py tests/test_mode_banner.py tests/test_test_claim_ids_file.py` --- ### Task 3: `QUIET_BOXES` — information boxes to the log **Files:** - Modify: `src/ecw_bot/settings.py` (next to `SHOW_EACH_CLAIM`, ~line 480) - Modify: `src/ecw_bot/core/msgbox.py` (`info`, `info_box`) - Modify: `src/ecw_bot/auth/login.py` (security-image `msgbox.info_box`, ~line 93) - Test: `tests/test_msgbox.py` (add cases) **Interfaces:** - Produces: `cfg.QUIET_BOXES: bool` (env `QUIET_BOXES`, default False); `msgbox.info(title, msg, timeout_s, needs_person=False) -> str`; `msgbox.info_box(title, msg, needs_person=False) -> None`. Quiet log line: `[BOX] {title}: {first line of msg}`; `info()` returns `OK` when quiet. - [ ] **Step 1: Write the failing tests** — add to `tests/test_msgbox.py` before its `main()`: ```python @case def quiet_boxes_send_an_info_box_to_the_log_not_the_screen(): import io import contextlib shown = [] orig = (msgbox._raw_call, msgbox.cfg.QUIET_BOXES) msgbox._raw_call = lambda *a: shown.append(a) or 1 msgbox.cfg.QUIET_BOXES = True out = io.StringIO() try: with contextlib.redirect_stdout(out): msgbox.info_box("eCW Bot — Claim 1", "Summary : Pass\nmore") rc = msgbox.info("eCW Bot — CSV report", "Saved.", 5) finally: msgbox._raw_call, msgbox.cfg.QUIET_BOXES = orig assert shown == [], shown assert rc == msgbox.OK text = out.getvalue() assert "[BOX] eCW Bot — Claim 1: Summary : Pass" in text, text assert "more" not in text, text @case def quiet_boxes_still_show_a_box_that_needs_a_person(): shown = [] orig = (msgbox._raw_call, msgbox.cfg.QUIET_BOXES) msgbox._raw_call = lambda *a: shown.append(a) or 1 msgbox.cfg.QUIET_BOXES = True try: msgbox.info_box("eCW Bot — Security Image", "Click Continue", needs_person=True) msgbox.confirm("eCW Bot — CAPTCHA", "Type it", 5) finally: msgbox._raw_call, msgbox.cfg.QUIET_BOXES = orig assert len(shown) == 2, shown @case def without_quiet_boxes_an_info_box_is_shown(): shown = [] orig = (msgbox._raw_call, msgbox.cfg.QUIET_BOXES) msgbox._raw_call = lambda *a: shown.append(a) or 1 msgbox.cfg.QUIET_BOXES = False try: msgbox.info_box("eCW Bot — Claim 1", "Summary : Pass") finally: msgbox._raw_call, msgbox.cfg.QUIET_BOXES = orig assert len(shown) == 1, shown ``` - [ ] **Step 2: Run to see them fail** Run: `python tests\test_msgbox.py` Expected: the new cases ERROR with `AttributeError: ... 'QUIET_BOXES'` / unexpected keyword `needs_person`. - [ ] **Step 3: Implement** `src/ecw_bot/settings.py`, after the `SHOW_EACH_CLAIM = ...` line: ```python # Information boxes go to the log as one "[BOX] title: first line" line # instead of the screen. The results page sets it for the runs it starts # (2026-10-14). Boxes a person must act on — CAPTCHA, 2FA manual step, # security image, LIVE-WRITE confirm — are never quieted. QUIET_BOXES = _env_bool("QUIET_BOXES", False) ``` `src/ecw_bot/core/msgbox.py` — replace `info()`: ```python def info(title: str, msg: str, timeout_s: int, needs_person: bool = False) -> str: """Informational box. Returns OK or TIMEOUT. With QUIET_BOXES on (a run the results page started), a box that does not need a person is one log line instead, and answers OK.""" if cfg.QUIET_BOXES and not needs_person: first = (msg or "").strip().splitlines()[0] if (msg or "").strip() \ else "" print(f"[BOX] {titled(title)}: {first}") return OK try: rc = _raw_call(msg, titled(title), _MB_OK | _MB_ICONINFO | _MB_TOPMOST, timeout_s) except Exception: return TIMEOUT return TIMEOUT if rc == _IDTIMEOUT else OK ``` and `info_box()`: ```python def info_box(title: str, msg: str, needs_person: bool = False) -> None: """Informational box on the configured timeout. Never blocks the run. `needs_person=True` for a box a person must act on (it is never quieted by QUIET_BOXES).""" title = titled(title) if info(title, msg, cfg.MSGBOX_TIMEOUT_SECONDS, needs_person=needs_person) == TIMEOUT: print(f"[UI] '{title}' timed out with nobody at the desk — continuing") ``` (`info()` references `cfg` and `titled`, defined lower in the module — fine at call time.) `src/ecw_bot/auth/login.py`, the security-image box: add `needs_person=True` as the last argument of that `msgbox.info_box(...)` call. - [ ] **Step 4: Run** Run: `python tests\test_msgbox.py` → all passed. Run: `python tests\test_test_claim_ids_file.py` → all passed (it stubs `msgbox.info_box` with `lambda t, m:` — flow calls never pass `needs_person`, so that still works). - [ ] **Step 5: Stage** `git add src/ecw_bot/settings.py src/ecw_bot/core/msgbox.py src/ecw_bot/auth/login.py tests/test_msgbox.py` --- ### Task 4: Read the run log (`web/run_status.py`) **Files:** - Create: `web/run_status.py` - Test: `tests/test_run_status.py` **Interfaces:** - Consumes: log line formats — `[LOGIN]`, `[CAPTCHA]`, `[OTP]`, `[2FA]`, `[OK] Logged in`, `[CLAIMS] --- claim n/N: ---`, `[RESULT] claim n/N : ` (Task 2), `[GUARD] ...`; exit codes 0/2/3/5/6. - Produces: `TAIL_LINES = 30`; `parse(text) -> {"phase", "progress": [n, N] | None, "current": str | None, "counts": {"Pass","Fail","Review","Not found": int}, "guard": str | None, "tail": [str]}`; `exit_text(code, stopped=False) -> str`; `read_tail(path, max_bytes=256_000) -> str`; `summarize(runner_status: dict) -> dict` (runner status keys from Task 5 plus `phase`, `progress`, `current`, `counts`, `tail`, `message`, `elapsed_s`). - [ ] **Step 1: Write the failing test** — `tests/test_run_status.py`: ```python r""" web/run_status.py — the bot's run log read into the page's live status. python tests\test_run_status.py """ import sys import pathlib import tempfile ROOT = pathlib.Path(__file__).resolve().parent.parent sys.path.insert(0, str(ROOT)) sys.path.insert(0, str(ROOT / "src")) from web import run_status as rs # noqa: E402 CASES = [] def case(fn): CASES.append(fn) return fn LOG = """\ ====================================================================== 2026-10-14 10:00:00 — run start [CONFIG] user=x otp=imap [LOGIN] Opening the login page [OTP] Waiting for the approval mail [OK] Logged in in 95s [CLAIMS] --- claim 1/3: 9124000 --- [RESULT] claim 1/3 9124000: Pass [CLAIMS] --- claim 2/3: 9119000 --- [CLAIMS] --- claim 2/3 done, moving to next --- [RESULT] claim 2/3 9119000: Fail [CLAIMS] --- claim 3/3: 9126000 --- """ @case def phase_follows_the_last_tagged_line(): assert rs.parse("[LOGIN] x")["phase"] == "Logging in" assert rs.parse("[LOGIN] x\n[2FA] y")["phase"] == \ "Waiting for the 2FA approval mail" assert rs.parse("[OTP] y\n[OK] Logged in in 3s")["phase"] == "Logged in" assert rs.parse(LOG)["phase"] == "Checking claims" assert rs.parse("")["phase"] == "Starting" @case def progress_and_current_claim_come_from_the_claim_header(): p = rs.parse(LOG) assert p["progress"] == [3, 3] and p["current"] == "9126000", p @case def a_done_line_is_not_a_claim_header(): p = rs.parse("[CLAIMS] --- claim 2/3 done, moving to next ---") assert p["progress"] is None, p @case def counts_come_from_result_lines(): c = rs.parse(LOG + "[RESULT] claim 3/3 9126000: Review\n")["counts"] assert c == {"Pass": 1, "Fail": 1, "Review": 1, "Not found": 0}, c @case def tail_is_the_last_lines(): text = "\n".join(f"line {i}" for i in range(100)) tail = rs.parse(text)["tail"] assert len(tail) == rs.TAIL_LINES and tail[-1] == "line 99", tail @case def guard_line_is_kept(): p = rs.parse("[GUARD] Refused: next attempt allowed at 10:40") assert p["guard"] == "[GUARD] Refused: next attempt allowed at 10:40" @case def exit_codes_read_as_plain_words(): assert rs.exit_text(0) == "Finished" assert "setting" in rs.exit_text(2) assert "guard" in rs.exit_text(3) assert "claims step failed" in rs.exit_text(5) assert "2FA" in rs.exit_text(6) assert rs.exit_text(9) == "Ended with code 9" assert rs.exit_text(None, stopped=True) == "Stopped by operator" assert "no exit code" in rs.exit_text(None) @case def read_tail_reads_the_end_of_a_file_and_missing_is_empty(): with tempfile.TemporaryDirectory() as tmp: p = pathlib.Path(tmp) / "run.log" p.write_text("a\n" * 1000 + "LAST\n", encoding="utf-8") assert rs.read_tail(p, max_bytes=100).endswith("LAST\n") assert rs.read_tail(pathlib.Path(tmp) / "none.log") == "" assert rs.read_tail(None) == "" @case def summarize_running_uses_the_phase_as_the_message(): with tempfile.TemporaryDirectory() as tmp: p = pathlib.Path(tmp) / "run.log" p.write_text(LOG, encoding="utf-8") s = rs.summarize({"running": True, "starting": False, "log_path": str(p), "out_path": None, "exit_code": None, "stopped_by_operator": False, "started_at": "2026-10-14T10:00:00", "ended_at": None}) assert s["message"] == "Checking claims", s assert s["progress"] == [3, 3] and "log_path" not in s, s @case def summarize_ended_says_how_and_how_long(): with tempfile.TemporaryDirectory() as tmp: p = pathlib.Path(tmp) / "run.log" p.write_text("[GUARD] Refused: next attempt allowed at 10:40\n", encoding="utf-8") s = rs.summarize({"running": False, "starting": False, "log_path": str(p), "out_path": None, "exit_code": 3, "stopped_by_operator": False, "started_at": "2026-10-14T10:00:00", "ended_at": "2026-10-14T10:04:12"}) assert s["message"].startswith("Login guard refused"), s assert "next attempt allowed at 10:40" in s["message"], s assert s["elapsed_s"] == 252, s @case def summarize_starting_reads_the_child_output_file(): with tempfile.TemporaryDirectory() as tmp: out = pathlib.Path(tmp) / "web_run.out" out.write_text("Traceback: boom\n", encoding="utf-8") s = rs.summarize({"running": True, "starting": True, "log_path": None, "out_path": str(out), "exit_code": None, "stopped_by_operator": False, "started_at": None, "ended_at": None}) assert s["message"] == "Starting" and s["tail"] == ["Traceback: boom"], s def main() -> int: failures = 0 for fn in CASES: name = fn.__name__.replace("_", " ") try: fn() print(f" ok {name}") except AssertionError as exc: print(f" FAIL {name}: {exc}") failures += 1 except Exception as exc: print(f" ERROR {name}: {type(exc).__name__}: {exc}") failures += 1 print(f"\n{len(CASES) - failures}/{len(CASES)} passed") return 1 if failures else 0 if __name__ == "__main__": sys.exit(main()) ``` - [ ] **Step 2: Run to see it fail** Run: `python tests\test_run_status.py` Expected: `ImportError: cannot import name 'run_status' from 'web'` - [ ] **Step 3: Write `web/run_status.py`** ```python r""" The bot's run log, read into the results page's live status (spec docs/superpowers/specs/2026-10-14-start-bot-from-web-page-design.md). Everything comes from lines the bot already prints, plus its one fixed-format line per claim, `[RESULT] claim n/N : `. The log can hold patient data — the page shows it on 127.0.0.1 only, as it shows the results table. """ import re from datetime import datetime TAIL_LINES = 30 _CLAIM = re.compile(r"^\[CLAIMS\] --- claim (\d+)/(\d+): (\S+) ---$") _RESULT = re.compile(r"^\[RESULT\] claim (\d+)/(\d+) (\S+): (.+)$") _PHASES = ( (("[LOGIN]", "[CAPTCHA]"), "Logging in"), (("[OTP]", "[2FA]"), "Waiting for the 2FA approval mail"), (("[OK] Logged in",), "Logged in"), (("[CLAIMS]", "[RESULT]"), "Checking claims"), ) _EXIT = { 0: "Finished", 2: "Stopped before login — a setting needs fixing, or another run is " "going (see the log)", 3: "Login guard refused the login — too many attempts this hour", 5: "The claims step failed (see the log)", 6: "The 2FA approval mail did not arrive in time", } def parse(text: str) -> dict: phase, progress, current, guard = "Starting", None, None, None counts = {"Pass": 0, "Fail": 0, "Review": 0, "Not found": 0} lines = (text or "").splitlines() for raw in lines: line = raw.strip() for prefixes, name in _PHASES: if line.startswith(prefixes): phase = name m = _CLAIM.match(line) if m: progress = [int(m.group(1)), int(m.group(2))] current = m.group(3) m = _RESULT.match(line) if m: cat = m.group(4).strip() counts[cat] = counts.get(cat, 0) + 1 if line.startswith("[GUARD]"): guard = line return {"phase": phase, "progress": progress, "current": current, "counts": counts, "guard": guard, "tail": [ln.rstrip() for ln in lines[-TAIL_LINES:]]} def exit_text(code, stopped: bool = False) -> str: if stopped: return "Stopped by operator" if code is None: return "Ended (no exit code — the run was stopped or crashed)" return _EXIT.get(code, f"Ended with code {code}") def read_tail(path, max_bytes: int = 256_000) -> str: """The end of a text file; '' when there is none.""" if not path: return "" try: with open(path, "rb") as fh: fh.seek(0, 2) size = fh.tell() fh.seek(max(0, size - max_bytes)) return fh.read().decode("utf-8", errors="replace") except OSError: return "" def _elapsed(started, ended): try: a = datetime.fromisoformat(started) b = datetime.fromisoformat(ended) if ended else datetime.now() return int((b - a).total_seconds()) except (TypeError, ValueError): return None def summarize(st: dict) -> dict: """The runner's status (web/bot_runner.py) plus what the log says. File paths are not sent to the page.""" text = read_tail(st.get("log_path")) or read_tail(st.get("out_path")) p = parse(text) if st.get("running"): message = p["phase"] else: message = exit_text(st.get("exit_code"), st.get("stopped_by_operator", False)) if st.get("exit_code") == 3 and p["guard"]: message += f" — {p['guard']}" out = {k: v for k, v in st.items() if k not in ("log_path", "out_path")} out.update(phase=p["phase"], progress=p["progress"], current=p["current"], counts=p["counts"], tail=p["tail"], message=message, elapsed_s=_elapsed(st.get("started_at"), st.get("ended_at"))) return out ``` - [ ] **Step 4: Run** Run: `python tests\test_run_status.py` → `11/11 passed` - [ ] **Step 5: Stage** `git add web/run_status.py tests/test_run_status.py` --- ### Task 5: Start / stop the child process (`web/bot_runner.py`) **Files:** - Create: `web/bot_runner.py` - Test: `tests/test_bot_runner.py` **Interfaces:** - Consumes: `run_lock.read`, `run_lock.is_live`, `run_lock.mark_stopped` (Task 1). - Produces: `FORCED_ENV: dict`; `class AlreadyRunning(Exception)`; `class NotRunning(Exception)`; `class BotRunner(command=None, root=ROOT)` with `start() -> dict`, `stop() -> dict`, `status() -> dict`. Status keys: `running, starting, pid, started_at, ended_at, exit_code, stopped_by_operator, source, log_path, out_path`. - [ ] **Step 1: Write the failing test** — `tests/test_bot_runner.py`: ```python r""" web/bot_runner.py — start, watch and stop the bot as a child process. A fake child script stands in for run.py: it takes the run record the way the bot does, writes the env it was given, and waits. python tests\test_bot_runner.py """ import os import sys import json import time import pathlib import tempfile ROOT = pathlib.Path(__file__).resolve().parent.parent sys.path.insert(0, str(ROOT)) sys.path.insert(0, str(ROOT / "src")) from ecw_bot import settings as cfg # noqa: E402 from ecw_bot.core import run_lock # noqa: E402 from web import bot_runner as br # noqa: E402 CASES = [] def case(fn): CASES.append(fn) return fn # The child: like run.py, it takes the record (cfg.LOGS from argv), dumps # the env keys under test, then sleeps until killed or `seconds` pass. CHILD = r''' import os, sys, json, time, pathlib sys.path.insert(0, sys.argv[1]) from ecw_bot import settings as cfg from ecw_bot.core import run_lock cfg.LOGS = pathlib.Path(sys.argv[2]) log = cfg.LOGS / "run_child.log" log.write_text("[LOGIN] fake\n", encoding="utf-8") run_lock.acquire(log) keys = ["SAVE_CORRECTIONS", "ECW_KEEP_OPEN", "QUIET_BOXES", "CONFIRM_BEFORE_LOOKUP", "ECW_RUN_SOURCE"] (cfg.LOGS / "env.json").write_text( json.dumps({k: os.environ.get(k) for k in keys}), encoding="utf-8") time.sleep(float(sys.argv[3])) run_lock.finish(0) ''' def _runner(tmp, seconds=30): script = pathlib.Path(tmp) / "child.py" script.write_text(CHILD, encoding="utf-8") cmd = [sys.executable, str(script), str(ROOT / "src"), str(cfg.LOGS), str(seconds)] return br.BotRunner(command=cmd, root=ROOT) def _wait(pred, timeout=15): end = time.monotonic() + timeout while time.monotonic() < end: if pred(): return True time.sleep(0.2) return False @case def start_runs_the_child_with_the_forced_env(tmp): os.environ["SAVE_CORRECTIONS"] = "1" # the parent says LIVE-WRITE try: r = _runner(tmp) st = r.start() assert st["running"] is True, st assert _wait(lambda: (cfg.LOGS / "env.json").exists()) env = json.loads((cfg.LOGS / "env.json").read_text(encoding="utf-8")) assert env == {"SAVE_CORRECTIONS": "0", "ECW_KEEP_OPEN": "0", "QUIET_BOXES": "1", "CONFIRM_BEFORE_LOOKUP": "0", "ECW_RUN_SOURCE": "web"}, env assert _wait(lambda: r.status()["starting"] is False) st = r.status() assert st["running"] is True and st["source"] == "web", st assert st["log_path"].endswith("run_child.log"), st finally: os.environ.pop("SAVE_CORRECTIONS", None) try: r.stop() except br.NotRunning: pass @case def a_second_start_is_refused(tmp): r = _runner(tmp) r.start() try: assert _wait(lambda: run_lock.is_live(run_lock.read())) for other in (r, _runner(tmp)): try: other.start() except br.AlreadyRunning as exc: assert "already going" in str(exc), exc else: raise AssertionError("second start allowed") finally: r.stop() @case def stop_kills_the_run_and_marks_it_stopped(tmp): r = _runner(tmp) r.start() assert _wait(lambda: run_lock.is_live(run_lock.read())) pid = run_lock.read()["pid"] st = r.stop() assert st["running"] is False, st assert st["stopped_by_operator"] is True, st assert run_lock.pid_alive(pid) is False @case def stop_with_nothing_running_says_so(tmp): try: _runner(tmp).stop() except br.NotRunning: pass else: raise AssertionError("stop of nothing did not raise") @case def a_run_that_ends_reports_its_exit_code(tmp): r = _runner(tmp, seconds=0.5) r.start() assert _wait(lambda: r.status()["running"] is False) st = r.status() assert st["exit_code"] == 0 and st["stopped_by_operator"] is False, st @case def a_terminal_run_is_seen_and_blocks_start(tmp): """The record of a live run this runner did not start (this test process stands in for a terminal run).""" run_lock.acquire("terminal.log", source="terminal") r = _runner(tmp) st = r.status() assert st["running"] is True and st["source"] == "terminal", st try: r.start() except br.AlreadyRunning: pass else: raise AssertionError("started over a terminal run") run_lock.finish(0) def main() -> int: failures = 0 orig = cfg.LOGS for fn in CASES: name = fn.__name__.replace("_", " ") with tempfile.TemporaryDirectory() as tmp: cfg.LOGS = pathlib.Path(tmp) / "Logs" cfg.LOGS.mkdir() try: fn(tmp) print(f" ok {name}") except AssertionError as exc: print(f" FAIL {name}: {exc}") failures += 1 except Exception as exc: print(f" ERROR {name}: {type(exc).__name__}: {exc}") failures += 1 finally: cfg.LOGS = orig print(f"\n{len(CASES) - failures}/{len(CASES)} passed") return 1 if failures else 0 if __name__ == "__main__": sys.exit(main()) ``` - [ ] **Step 2: Run to see it fail** Run: `python tests\test_bot_runner.py` Expected: `ImportError: cannot import name 'bot_runner' from 'web'` - [ ] **Step 3: Write `web/bot_runner.py`** ```python r""" Start, watch and stop the bot from the results page (spec docs/superpowers/specs/2026-10-14-start-bot-from-web-page-design.md). The bot runs as its own process — `python run.py`, the same entry point as the terminal — so a hung or crashed run never takes the web server with it, and closing the page does not stop the run. The command and its environment are fixed here; nothing from a request reaches them. The run itself says whether it is live: `Logs\bot_run.json` (`core/run_lock.py`), which the bot writes at start and finishes at the end, terminal runs included. """ import os import sys import pathlib import subprocess from datetime import datetime from ecw_bot import settings as cfg from ecw_bot.core import run_lock ROOT = pathlib.Path(__file__).resolve().parent.parent # A page-started run: VERIFY whatever .env says, no "Press Enter", no # information boxes (the page shows the run), no "Run the Lookup?" box. FORCED_ENV = { "SAVE_CORRECTIONS": "0", "ECW_KEEP_OPEN": "0", "QUIET_BOXES": "1", "CONFIRM_BEFORE_LOOKUP": "0", "ECW_RUN_SOURCE": "web", } _NO_WINDOW = getattr(subprocess, "CREATE_NO_WINDOW", 0) class AlreadyRunning(Exception): pass class NotRunning(Exception): pass class BotRunner: def __init__(self, command=None, root=ROOT): self.command = list(command or [sys.executable, str(root / "run.py")]) self.root = pathlib.Path(root) self._proc = None self._out_path = None def _child_running(self) -> bool: return self._proc is not None and self._proc.poll() is None def start(self) -> dict: rec = run_lock.read() if run_lock.is_live(rec): raise AlreadyRunning( f"A run is already going (started " f"{(rec or {}).get('started_at') or '?'})") if self._child_running(): raise AlreadyRunning("A run is already going (still starting)") env = dict(os.environ) env.update(FORCED_ENV) cfg.LOGS.mkdir(parents=True, exist_ok=True) self._out_path = cfg.LOGS / f"web_run_{datetime.now():%Y%m%d_%H%M%S}.out" with open(self._out_path, "ab") as out: self._proc = subprocess.Popen( self.command, cwd=str(self.root), env=env, stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, creationflags=_NO_WINDOW) print(f"[WEB] Bot run started from the page (pid {self._proc.pid})") return self.status() def stop(self) -> dict: rec = run_lock.read() pids = [] if run_lock.is_live(rec): pids.append(int(rec["pid"])) if self._child_running(): pids.append(self._proc.pid) if not pids: raise NotRunning("No run is going") for pid in dict.fromkeys(pids): # /T: the bot's Chrome goes with it. subprocess.run(["taskkill", "/PID", str(pid), "/T", "/F"], capture_output=True, creationflags=_NO_WINDOW) if self._proc is not None: try: self._proc.wait(timeout=15) except subprocess.TimeoutExpired: pass if rec and rec.get("pid") in pids: run_lock.mark_stopped(rec["pid"]) print(f"[WEB] Bot run stopped from the page (pid " f"{', '.join(str(p) for p in dict.fromkeys(pids))})") return self.status() def status(self) -> dict: rec = run_lock.read() or {} ours = self._proc is not None and rec.get("pid") == self._proc.pid if self._proc is not None and not ours: # The child has not written its record yet — or ended before # it could (a crash at import, a refused second run). if self._child_running(): return {"running": True, "starting": True, "pid": None, "started_at": None, "ended_at": None, "exit_code": None, "stopped_by_operator": False, "source": "web", "log_path": None, "out_path": str(self._out_path)} if not run_lock.is_live(rec): return {"running": False, "starting": False, "pid": None, "started_at": None, "ended_at": None, "exit_code": self._proc.returncode, "stopped_by_operator": False, "source": "web", "log_path": None, "out_path": str(self._out_path)} exit_code = rec.get("exit_code") if (ours and exit_code is None and not self._child_running() and not rec.get("stopped_by_operator")): exit_code = self._proc.returncode return {"running": run_lock.is_live(rec), "starting": False, "pid": rec.get("pid"), "started_at": rec.get("started_at"), "ended_at": rec.get("ended_at"), "exit_code": exit_code, "stopped_by_operator": bool(rec.get("stopped_by_operator")), "source": rec.get("source"), "log_path": rec.get("log_path"), "out_path": str(self._out_path) if ours and self._out_path else None} ``` - [ ] **Step 4: Run** Run: `python tests\test_bot_runner.py` → `6/6 passed` - [ ] **Step 5: Stage** `git add web/bot_runner.py tests/test_bot_runner.py` --- ### Task 6: Endpoints, guard and the "Bot run" card **Files:** - Modify: `web/app.py` (imports; `create_app`; module docstring) - Modify: `web/templates/index.html` - Test: `tests/test_web_app.py` (add cases + `FakeRunner`) **Interfaces:** - Consumes: `BotRunner`, `AlreadyRunning`, `NotRunning` (Task 5); `run_status.summarize` (Task 4). - Produces: `create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None, allowed_origins=ALLOWED_ORIGINS)`; `app.state.run_token: str`; `ALLOWED_ORIGINS = ("http://127.0.0.1:8000", "http://localhost:8000")`; routes `POST /api/bot/start`, `POST /api/bot/stop`, `GET /api/bot/status`; template variable `run_token`. - [ ] **Step 1: Write the failing tests** — in `tests/test_web_app.py`, after `_client`: ```python class FakeRunner: def __init__(self, running=False): self.running = running self.calls = [] def _st(self): return {"running": self.running, "starting": False, "pid": 1, "started_at": "2026-10-14T10:00:00", "ended_at": None, "exit_code": None, "stopped_by_operator": False, "source": "web", "log_path": None, "out_path": None} def start(self): self.calls.append("start") if self.running: raise web_app.AlreadyRunning("A run is already going (started x)") self.running = True return self._st() def stop(self): self.calls.append("stop") if not self.running: raise web_app.NotRunning("No run is going") self.running = False return self._st() def status(self): return self._st() def _bot_client(runner=None): runner = runner or FakeRunner() app = web_app.create_app(FakeRepo(), runner=runner) return (TestClient(app, base_url="http://127.0.0.1:8000"), runner, app.state.run_token) _GOOD_ORIGIN = {"Origin": "http://127.0.0.1:8000"} ``` and the cases (before `def main()`): ```python @case def the_page_carries_the_run_token_and_the_start_button(): client, _, token = _bot_client() page = client.get("/").text assert f'content="{token}"' in page assert 'id="br-start"' in page and "VERIFY mode" in page @case def start_without_the_token_or_origin_is_forbidden(): client, runner, token = _bot_client() for headers in ({}, {"X-Run-Token": token}, {**_GOOD_ORIGIN}, {**_GOOD_ORIGIN, "X-Run-Token": "wrong"}, {"Origin": "http://evil.example", "X-Run-Token": token}, {"Referer": "http://evil.example/x", "X-Run-Token": token}): r = client.post("/api/bot/start", headers=headers) assert r.status_code == 403, (headers, r.status_code) assert runner.calls == [], runner.calls @case def start_and_stop_with_origin_and_token(): client, runner, token = _bot_client() h = {**_GOOD_ORIGIN, "X-Run-Token": token} r = client.post("/api/bot/start", headers=h) assert r.status_code == 200 and r.json()["running"] is True, r.text r = client.post("/api/bot/stop", headers=h) assert r.status_code == 200 and r.json()["running"] is False, r.text assert runner.calls == ["start", "stop"] @case def a_referer_from_the_page_counts_when_origin_is_absent(): client, runner, token = _bot_client() r = client.post("/api/bot/start", headers={ "Referer": "http://localhost:8000/?run_at=x", "X-Run-Token": token}) assert r.status_code == 200, r.text @case def start_while_running_is_409_and_stop_of_nothing_is_409(): client, runner, token = _bot_client(FakeRunner(running=True)) h = {**_GOOD_ORIGIN, "X-Run-Token": token} r = client.post("/api/bot/start", headers=h) assert r.status_code == 409 and "already going" in r.json()["detail"] client.post("/api/bot/stop", headers=h) r = client.post("/api/bot/stop", headers=h) assert r.status_code == 409 and "No run" in r.json()["detail"] @case def status_has_the_page_fields_and_no_file_paths(): client, _, _ = _bot_client(FakeRunner(running=True)) s = client.get("/api/bot/status").json() for key in ("running", "phase", "progress", "counts", "tail", "message", "started_at", "exit_code"): assert key in s, (key, s) assert "log_path" not in s and "out_path" not in s, s ``` - [ ] **Step 2: Run to see them fail** Run: `python tests\test_web_app.py` Expected: new cases ERROR (`create_app() got an unexpected keyword argument 'runner'`). - [ ] **Step 3: Implement `web/app.py`** Imports (after `from fastapi.templating import Jinja2Templates`): ```python import secrets # noqa: E402 from urllib.parse import urlsplit # noqa: E402 ``` and after `from .repo import SqlResultsRepo`: ```python from . import run_status # noqa: E402 from .bot_runner import (AlreadyRunning, BotRunner, # noqa: E402 NotRunning) ``` After `ALLOWED_HOSTS = ...`: ```python # The only pages allowed to Start / Stop a run: this one (2026-10-14). A # page on another site in the same browser could otherwise POST here. ALLOWED_ORIGINS = ("http://127.0.0.1:8000", "http://localhost:8000") ``` Add near the other exception classes: ```python class Forbidden(Exception): pass ``` Change the signature and add, at the top of `create_app` after `app = FastAPI(...)`: ```python def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None, allowed_origins=ALLOWED_ORIGINS) -> FastAPI: app = FastAPI(title="eCW Charge Validation — Results", docs_url=None, redoc_url=None, openapi_url=None) runner = runner if runner is not None else BotRunner() # Made per server start and put in the page: a page on another site # cannot read it, so it cannot send it. app.state.run_token = secrets.token_urlsafe(32) def _check_writer(request: Request) -> None: origin = request.headers.get("origin") if not origin: ref = urlsplit(request.headers.get("referer") or "") origin = f"{ref.scheme}://{ref.netloc}" if ref.netloc else "" if origin not in allowed_origins: raise Forbidden() sent = request.headers.get("x-run-token") or "" if not secrets.compare_digest(sent, app.state.run_token): raise Forbidden() ``` Add the handler next to the other `exception_handler`s: ```python @app.exception_handler(Forbidden) def _forbidden(request: Request, exc: Forbidden): return JSONResponse({"detail": "Not allowed from here"}, status_code=403) ``` In `page()`, add `"run_token": app.state.run_token,` to the template context dict. Before `return app`: ```python @app.post("/api/bot/start") def bot_start(request: Request): _check_writer(request) try: return run_status.summarize(runner.start()) except AlreadyRunning as exc: return JSONResponse({"detail": str(exc)}, status_code=409) @app.post("/api/bot/stop") def bot_stop(request: Request): _check_writer(request) try: return run_status.summarize(runner.stop()) except NotRunning as exc: return JSONResponse({"detail": str(exc)}, status_code=409) @app.get("/api/bot/status") def bot_status(): return run_status.summarize(runner.status()) ``` Module docstring: replace "Read-only." with "Read-only for the results; it can also start a VERIFY run of the bot and stop it (2026-10-14, `bot_runner.py`), each POST checked for this page's Origin and token." - [ ] **Step 4: Implement the card in `web/templates/index.html`** In ``, after the viewport meta: ```html ``` In ``: ```css .botrun { background: var(--panel); border: 1px solid var(--line); border-radius: 8px; padding: 12px 16px; margin-bottom: 16px; } .botrun .row { display: flex; gap: 12px; align-items: center; flex-wrap: wrap; } .botrun button.stop { background: var(--fail); border-color: var(--fail); } .botrun button[disabled] { opacity: .5; cursor: default; } .botrun progress { width: 220px; } .botrun pre { background: var(--head); padding: 8px; border-radius: 6px; max-height: 260px; overflow: auto; font-size: 12px; white-space: pre-wrap; } .chip { padding: 2px 8px; border-radius: 10px; font-size: 12px; font-weight: 600; } ``` As the first child of `
` (before `{% if not runs %}`): ```html
Bot run …

VERIFY mode — nothing is saved to eCW. CAPTCHA and 2FA boxes still come up on this PC when the login needs a person.

``` Before ``: ```html ``` - [ ] **Step 5: Run** Run: `python tests\test_web_app.py` → all passed (old cases + 6 new). - [ ] **Step 6: Stage** `git add web/app.py web/templates/index.html tests/test_web_app.py` --- ### Task 7: Full suite, docs, live check, commit (with the operator) **Files:** - Modify: `Context.md` (CRLF — patch through a Python script, not an editor that changes line endings): §2 test count and "How to run" (Start from the page), §3 code layout (`core/run_lock.py`, `web/bot_runner.py`, `web/run_status.py`), §5 web page bullet (Start/Stop/status, VERIFY only, token), §6 rulings (2026-10-14: page runs VERIFY only; info boxes off for page runs; one live run at a time), §8 last commits, §9 EEWA-17 note. - Modify: `.env.example` — document `QUIET_BOXES` (default 0; set by the page for its runs). - [ ] **Step 1: Full suite** Run every `tests\test_*.py`; total must be all passed. Confirm `Logs\bot_run.json` was not created by the suite (`dir Logs\bot_run.json` → not found, unless a real run happened). - [ ] **Step 2: Docs** — update Context.md and `.env.example` as listed. - [ ] **Step 3: Live check — only when the operator says to run** (4/hour login guard; mailed 2FA) 1. `.\run_web.bat`, open http://127.0.0.1:8000. 2. Start run → phase moves Logging in → Waiting for the 2FA approval mail → Checking claims; progress and Pass/Fail/Review chips move; no per-claim boxes on the desktop; the log has `[BOX]` and `[RESULT]` lines and `[MODE]` says VERIFY. 3. Start again while running → "already going". 4. Let it finish → card says "Finished in …", results table shows the new run. 5. A second short run: Stop mid-way → "Stopped by operator", Chrome gone. - [ ] **Step 4: Ask the operator to commit** — then `git add src/ecw_bot/core/run_lock.py src/ecw_bot/flows/export_medcor_ready.py src/ecw_bot/settings.py src/ecw_bot/core/msgbox.py src/ecw_bot/auth/login.py web/run_status.py web/bot_runner.py web/app.py web/templates/index.html tests/test_run_lock.py tests/test_run_status.py tests/test_bot_runner.py tests/test_msgbox.py tests/test_web_app.py tests/test_mode_banner.py tests/test_test_claim_ids_file.py docs/superpowers/specs/2026-10-14-start-bot-from-web-page-design.md docs/superpowers/plans/2026-10-14-start-bot-from-web-page.md .env.example` and commit `feat: start a VERIFY bot run from the results page`; Context.md in its own `docs:` commit. Never `run.py`, `tests/test_guard.py`, `tools/sql/Create_CV_Table.rpt`. --- ## Rulings made while planning (spec refinements) - **Record kept, not deleted:** the spec said the lock file is removed when `main()` returns. The plan keeps `Logs\bot_run.json` as the latest run's record and adds `ended_at` / `exit_code` / `stopped_by_operator`, so the page can say how the run ended even after a web restart. "Live" = no `ended_at` and the pid is a running python process. Cost if wrong: one JSON file left in `Logs\`. - **Quieting in `info()`, not only `info_box()`:** the export-report box (`pages/claim_report.py`) calls `msgbox.info()` directly; quieting at `info()` covers it. `confirm()` is never quieted. - **The page needs the results database to render:** if SQL Server is down the page shows its error and no Start button (unchanged behaviour). Start from the page then waits for the database; the terminal still works.