Run events in the database: Runs page and Agent run card on the Dokploy Portal

Snapshot of D:\Ecw Process 65c92c5 (2026-10-30).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Context.md

@@ -348,7 +348,7 @@ that box and on the per-claim save box means Cancel.
 | `flows/recovery.py` | A stuck screen inside a pass (§2): `recover()` (logout check, Escape + Patient Information close + clear, reopen search) and `run_claim()` (one retry per claim, 2 in a row → `PageStuck`, exit 9) |
 | `core/notify.py` | The agent's mail (§2): `incident()` / `summary()`; claim numbers masked, login / error PNGs only; never fatal |
 | `core/watchdog.py` | Claim watchdog (2026-10-22): `watching(limit)` around the claim loop, `beat(where)` at each claim step and every message box, `check()` raises `Stalled` → exit 8; on a stall it closes this process's Playwright driver (`node.exe`) so the frozen call returns, and exits 8 itself if the main thread never comes back |
-| `core/run_events.py` | A Run's timed events → `Logs\events_<run>.jsonl` (`start`, `emit`, `warning`, `claim_error`, `run_error` with traceback, `login_attempt_start/end`, `login_step`); never raises, never writes an OTP code / password / username |
+| `core/run_events.py` | A Run's timed events → `Logs\events_<run>.jsonl` (`start`, `emit`, `warning`, `claim_error`, `run_error` with traceback, `login_attempt_start/end`, `login_step`); never raises, never writes an OTP code / password / username. Since `c9a43ac` (2026-10-30) each line is also copied to `dbo.CV_RunEvent` (§5) — only for a Run in the real `Logs` folder with `RESULTS_DB=1`; the first database error turns the copy off for that Run |
 | `core/browser.py` | `launch()`: Playwright Chromium with the eCW proxy, or (`BROWSER_CDP_URL`) attach to the operator's Chrome — `clear_ecw_data()` at attach and close, `cdp_ready()` checked before login |
 | `core/msgbox.py` | Timed Windows message boxes; `default_cancel=True` makes Enter mean Cancel |
 | `core/text.py` | `first_line()` shared by core / conditions / flows |
@@ -641,6 +641,24 @@ the Portal (restarted 2026-10-26, [portal] terminal tab).
   queue file — the agent would have to write the queue size to the DB).
   The mockup canvas: claude.ai artifact "Charge Validation Portal Mockups"
   (private to the operator).
+- **Runs on Dokploy (2026-10-30, `c9a43ac`).** The Dokploy Portal cannot
+  read the bot PC's `Logs` folder, so its Runs page showed "No runs
+  match" for every date. Now every Run event is also a row in
+  `dbo.CV_RunEvent` (run_id, at, kind, body = the same JSON line;
+  created in SSMS by the operator that day, script in
+  `tools/sql/create_tables.sql`; `cv_portal` reads it through
+  `db_datareader`). `web/runs.py` reads Runs through an events source:
+  `FileEvents` (the PC's local Portal, as before) or `DbEvents` (a
+  `PORTAL_SERVER` Portal; one query per page). On Dokploy the Runs list,
+  filter, run detail and last-run banner come from the table, and the
+  Dashboard's Agent run card from `runs.live_status()` (the newest Run;
+  "running" = no end and an event in the last 10 minutes). Failure
+  screenshots stay on the PC (PHI): a server shows their names only.
+  `tools/import_run_events.py` loaded the 34 older Runs (2,043 events,
+  22–29/10) once; a second run adds nothing. Checked live: 28–29/10 lists
+  15 Runs; the 21:10 Run's detail shows the 7 m 7 s 2FA mail wait. ADR
+  0001 (run events in files) still holds: the file is the record, the row
+  a copy.
 
 ---
 
@@ -753,7 +771,9 @@ the Portal (restarted 2026-10-26, [portal] terminal tab).
 
 History: `git log --oneline`.
 
-Last code commits: `78409f0` (2026-10-30, Portal sage redesign — §5;
+Last code commits: `c9a43ac` (2026-10-30, run events in the database —
+Runs page and Agent run card on Dokploy, §5; deployed from
+`portal-20261030-runs.zip`), `78409f0` (2026-10-30, Portal sage redesign — §5;
 deployed to Dokploy from the working tree before this commit, same
 files), `24c9ce4` (2026-10-30, the self-healing agent: supervisor,
 recovery, security image Continue, Resend pickup, operator mail — §2,
@@ -898,10 +918,19 @@ test claim ids — new posts must not (§7).
   `.env` (the operator's address) and one test mail; a first `python
   agent.py` to see the security image Continue click on the real page
   (its HTML is saved as `security_image_page` — tighten the selector from
-  it) and a late-mail Resend pickup. Not built: the agent's status in the
+  it) and a late-mail Resend pickup. The Dokploy Dashboard's Agent run
-  Portal (it runs on the bot PC; the Dokploy Portal sees only the
+  card shows the newest bot Run from `CV_RunEvent` (`c9a43ac`), but not the
-  database — a status / queue-size row would carry it), Queue / Pending
+  agent's own passes or waits (the supervisor writes no events yet), and
-  tiles.
+  there are no Queue / Pending tiles (no queue size in the database).
+- **Dokploy wording (2026-10-30, asked, not done):** the Agent run card
+  says "see the log" and "CAPTCHA and 2FA boxes still come up on this PC"
+  — true for the PC's local Portal, misleading on Dokploy; reword for a
+  server Portal.
+- **A public IP in the Dokploy logs (2026-10-30):** `/api/bot/status`
+  polls from `43.254.111.194` beside the internal `10.200.1.15`. The
+  Portal is meant to be Ecare network / VPN only — IT to confirm the
+  address is Ecare's own; if not, the domain is reachable from the
+  internet (PHI).
 - **29/10 live queue (690 claims).** 208 got a result in two runs (17:18
   126, 19:29 82); the rest were not reached: Ctrl-C would not stop a
   run blocked in a Playwright call (the tab was closed instead), a stuck

src/ecw_bot/core/run_events.py

@@ -8,6 +8,11 @@ writes `Logs\\events_<run>.jsonl` beside `run_<run>.log`: one JSON object a
 line, each with its time to the millisecond, flushed at once so a killed Run
 keeps what it wrote. The Portal's Runs page reads these files.
 
+Every event is also copied to `dbo.CV_RunEvent` (2026-10-30, with
+`RESULTS_DB=1`): the Portal on Dokploy cannot read this PC's Logs folder.
+The file stays the record; the row is the same JSON line. The first
+database error switches the copy off for the rest of the Run.
+
 An event never stops a Run: anything that goes wrong writing one drops that
 event and nothing else. Nothing secret goes in — no OTP code, no password,
 no eCW username: the callers pass fixed fields, never a whole log line.
@@ -22,6 +27,9 @@ from pathlib import Path
 
 _handle = None
 _attempt = 0
+_run_id = ""
+_db_on = True
+_conn = None
 
 
 def close() -> None:
@@ -31,13 +39,62 @@ def close() -> None:
 
 def _reset() -> None:
     """Close the file and forget the attempt count (tests; a new Run)."""
-    global _handle, _attempt
+    global _handle, _attempt, _run_id, _db_on, _conn
     try:
         if _handle:
             _handle.close()
     except Exception:
         pass
-    _handle, _attempt = None, 0
+    try:
+        if _conn is not None:
+            _conn.close()
+    except Exception:
+        pass
+    _handle, _attempt, _run_id, _db_on, _conn = None, 0, "", True, None
+
+
+def _sql_insert(run_id, at, kind, body) -> None:
+    """The real copy: one INSERT on a connection kept for the Run."""
+    global _conn
+    from .. import settings as cfg
+    # A test that points LOGS at a temp folder never writes the real table.
+    if not cfg.RESULTS_DB or \
+            Path(cfg.LOGS).resolve() != (Path(cfg.ROOT) / "Logs").resolve():
+        return
+    if _conn is None:
+        import pyodbc
+        _conn = pyodbc.connect(cfg.RESULTS_DB_CONN, timeout=10,
+                               autocommit=True)
+    _conn.cursor().execute(
+        "INSERT INTO dbo.CV_RunEvent (run_id, at, kind, body) "
+        "VALUES (?, ?, ?, ?)", [run_id, at, kind, body])
+
+
+_db_writer = _sql_insert
+
+
+def _real_run(log_path) -> bool:
+    """A Run in the real Logs folder: a test's log in a temp folder never
+    reaches the database, whatever this PC's `.env` says."""
+    try:
+        from .. import settings as cfg
+        return Path(log_path).resolve().parent == Path(cfg.LOGS).resolve()
+    except Exception:
+        return False
+
+
+def _copy(run_id: str, kind: str, line: str) -> None:
+    """The line to the database too. Never raises."""
+    global _db_on
+    if not _db_on or not run_id:
+        return
+    try:
+        _db_writer(run_id, datetime.now(), kind, line)
+    except Exception as exc:
+        _db_on = False
+        why = str(exc).splitlines()[0] if str(exc) else type(exc).__name__
+        print(f"[EVENTS] Not copied to the database for the rest of this "
+              f"run: {why}")
 
 
 def run_id_of(log_path) -> str:
@@ -77,8 +134,9 @@ def _line(kind: str, fields: dict) -> str:
 
 def start(log_path, mode: str, claim_limit=None, test_sheet: str = "") -> None:
     """Open this Run's events file and write `run_start`."""
-    global _handle
+    global _handle, _run_id
     _reset()
+    _run_id = run_id_of(log_path) if _real_run(log_path) else ""
     try:
         _handle = open(path_for(log_path), "a", encoding="utf-8",
                        buffering=1)
@@ -98,20 +156,25 @@ def emit(kind: str, **fields) -> None:
     """Write one event. Never raises."""
     if _handle is None:
         return
+    line = _line(kind, fields)
     try:
-        _handle.write(_line(kind, fields) + "\n")
+        _handle.write(line + "\n")
         _handle.flush()
     except Exception:
         pass
+    _copy(_run_id, kind, line)
 
 
 def append_to(log_path, kind: str, **fields) -> None:
     """Add one event to another Run's file — the Portal, after a Stop."""
+    line = _line(kind, fields)
     try:
         with open(path_for(log_path), "a", encoding="utf-8") as fh:
-            fh.write(_line(kind, fields) + "\n")
+            fh.write(line + "\n")
     except Exception:
         pass
+    if _real_run(log_path):
+        _copy(run_id_of(log_path), kind, line)
 
 
 def warning(code: str, message: str, **fields) -> None:

tests/test_run_events_db.py

@@ -0,0 +1,129 @@
+r"""
+core/run_events.py — every event also goes to `dbo.CV_RunEvent` (operator,
+2026-10-30): the Portal on Dokploy cannot read the bot PC's Logs folder, so
+its Runs page showed "No runs match" for every date.
+
+The file stays the record of a Run; the database row is a copy. A database
+that is off, missing the table or down never stops a Run — after the first
+failure the copy is switched off for the rest of the Run, said once in the
+log.
+
+    python tests\test_run_events_db.py
+
+No database: the writer is a stand-in.
+"""
+
+import io
+import sys
+import json
+import pathlib
+import tempfile
+import contextlib
+
+sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent.parent / "src"))
+
+from ecw_bot import settings as cfg                   # noqa: E402
+from ecw_bot.core import run_events                   # noqa: E402
+
+CASES = []
+
+
+def case(fn):
+    CASES.append(fn)
+    return fn
+
+
+def _start(tmp, writer):
+    run_events._db_writer = writer
+    log = pathlib.Path(tmp) / "run_20261030_101500.log"
+    run_events.start(log, "VERIFY")
+    return log
+
+
+@case
+def every_event_is_copied_with_its_run_id_and_kind(tmp):
+    rows = []
+    _start(tmp, lambda run_id, at, kind, body: rows.append(
+        (run_id, kind, json.loads(body))))
+    run_events.emit("claim_start", n=1, total=2, claim="9103000")
+    run_events.close()
+    assert [k for _, k, _ in rows] == ["run_start", "claim_start"], rows
+    assert {r for r, _, _ in rows} == {"20261030_101500"}
+    assert rows[1][2]["claim"] == "9103000" and rows[1][2]["t"], rows
+
+
+@case
+def the_file_and_the_row_carry_the_same_line(tmp):
+    bodies = []
+    log = _start(tmp, lambda run_id, at, kind, body: bodies.append(body))
+    run_events.emit("warning", code="x", message="y")
+    run_events.close()
+    lines = run_events.path_for(log).read_text(encoding="utf-8").splitlines()
+    assert lines == bodies, (lines, bodies)
+
+
+@case
+def a_database_error_switches_the_copy_off_and_the_run_goes_on(tmp):
+    calls = []
+
+    def boom(*a):
+        calls.append(a)
+        raise OSError("no such table")
+    out = io.StringIO()
+    with contextlib.redirect_stdout(out):
+        log = _start(tmp, boom)
+        run_events.emit("claim_start", n=1)
+        run_events.emit("claim_start", n=2)
+        run_events.close()
+    assert len(calls) == 1, "tried once, then off"
+    assert out.getvalue().count("[EVENTS]") == 1, out.getvalue()
+    assert len(run_events.path_for(log).read_text().splitlines()) == 3
+
+
+@case
+def an_event_added_after_a_stop_is_copied_too(tmp):
+    rows = []
+    run_events._db_writer = lambda run_id, at, kind, body: rows.append(kind)
+    log = pathlib.Path(tmp) / "run_20261030_101500.log"
+    run_events.append_to(log, "stopped", by="sam")
+    assert rows == ["stopped"], rows
+
+
+@case
+def a_run_outside_the_logs_folder_is_never_copied(tmp):
+    rows = []
+    run_events._db_writer = lambda *a: rows.append(a)
+    other = pathlib.Path(tmp) / "elsewhere"
+    other.mkdir()
+    run_events.start(other / "run_20261030_101500.log", "VERIFY")
+    run_events.emit("claim_start", n=1)
+    run_events.close()
+    run_events.append_to(other / "run_20261030_101500.log", "stopped")
+    assert rows == [], "a test's run never reaches the database"
+
+
+def main() -> int:
+    failures = 0
+    real = run_events._db_writer
+    for fn in CASES:
+        name = fn.__name__.replace("_", " ")
+        with tempfile.TemporaryDirectory() as tmp:
+            cfg.LOGS = pathlib.Path(tmp)     # a real-looking Logs folder
+            try:
+                fn(tmp)
+                print(f"  ok    {name}")
+            except AssertionError as exc:
+                print(f"  FAIL  {name}: {exc}")
+                failures += 1
+            except Exception as exc:
+                print(f"  ERROR {name}: {type(exc).__name__}: {exc}")
+                failures += 1
+            finally:
+                run_events._reset()
+                run_events._db_writer = real
+    print(f"\n{len(CASES) - failures}/{len(CASES)} passed")
+    return 1 if failures else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tests/test_runs_from_db.py

@@ -0,0 +1,177 @@
+r"""
+web/runs.py from the database (operator, 2026-10-30): the Portal on Dokploy
+cannot read the bot PC's Logs folder, so its Runs page said "No runs match"
+for every date. With `DbEvents` the same summaries come from
+`dbo.CV_RunEvent` — the same JSON lines the files hold.
+
+    python tests\test_runs_from_db.py
+
+No database: the repo is a stand-in.
+"""
+
+import sys
+import json
+import pathlib
+
+ROOT = pathlib.Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(ROOT / "src"))
+sys.path.insert(0, str(ROOT / "tests"))
+sys.path.insert(0, str(ROOT))
+
+from web import runs                                  # noqa: E402
+
+CASES = []
+
+
+def case(fn):
+    CASES.append(fn)
+    return fn
+
+
+def _line(day, t, kind, **f):
+    return json.dumps({**f, "t": f"2026-10-{day}T{t}.000", "kind": kind})
+
+
+def _run(day, hhmmss, end_code=0):
+    t = f"{hhmmss[:2]}:{hhmmss[2:4]}:{hhmmss[4:]}"
+    run_id = f"202610{day}_{hhmmss}"
+    lines = [_line(day, t, "run_start", source="terminal", started_by="ops",
+                   mode="VERIFY", claim_limit=0, test_sheet="", pid=1,
+                   log="x"),
+             _line(day, t, "claim_start", n=1, total=1, claim="9103000"),
+             _line(day, t, "run_end", exit_code=end_code)]
+    return run_id, lines
+
+
+class FakeEventsRepo:
+    def __init__(self, *runs_):
+        self.rows = [(rid, line) for rid, lines in runs_ for line in lines]
+        self.calls = []
+
+    def run_ids(self):
+        self.calls.append(("run_ids",))
+        return sorted({r for r, _ in self.rows}, reverse=True)
+
+    def run_event_lines(self, run_ids):
+        self.calls.append(("run_event_lines", tuple(run_ids)))
+        return [(r, b) for r, b in self.rows if r in set(run_ids)]
+
+
+@case
+def runs_come_from_the_database_newest_first_and_filtered_by_day():
+    repo = FakeEventsRepo(_run("28", "161611"), _run("29", "163000"),
+                          _run("29", "170620", end_code=9))
+    src = runs.DbEvents(repo)
+    got = runs.list_runs(src, day_from="2026-10-29", day_to="2026-10-29")
+    assert [r["run_id"] for r in got] == ["20261029_170620",
+                                         "20261029_163000"], got
+    assert got[0]["status"] == "Run Error" and got[1]["status"] == "Done"
+    assert repo.calls[-1] == ("run_event_lines",
+                              ("20261029_170620", "20261029_163000")), \
+        "one query for the day's runs, not one per run"
+
+
+@case
+def one_run_and_the_latest_run_come_from_the_database():
+    repo = FakeEventsRepo(_run("28", "161611"), _run("29", "163000"))
+    src = runs.DbEvents(repo)
+    assert runs.load_run(src, "20261028_161611")["status"] == "Done"
+    assert runs.load_run(src, "20261030_000000") is None
+    assert runs.latest_run(src)["run_id"] == "20261029_163000"
+
+
+@case
+def a_screenshot_is_never_served_from_the_database():
+    repo = FakeEventsRepo(("20261029_163000", [
+        _line("29", "16:30:00", "screenshot", path="C:/x/101500_shot.png")]))
+    assert runs.screenshot_path(runs.DbEvents(repo), "20261029_163000", 0,
+                                "C:/x") is None
+
+
+@case
+def the_portal_runs_page_reads_its_events_source():
+    import test_web_app as t
+    repo = FakeEventsRepo(_run("29", "163000"))
+    client, _ = t.signed_in(t.FakeRepo(), t.FakeRunner(),
+                            events=runs.DbEvents(repo))
+    html = client.get("/runs", params={"day_from": "2026-10-29",
+                                       "day_to": "2026-10-29"}).text
+    assert "No runs match" not in html, html[-800:]
+    assert "/runs/20261029_163000" in html
+
+
+def _live_lines(day, hhmm, ss_last, end=False):
+    """A run whose last event is at hh:mm:ss_last; ended or not."""
+    run_id = f"202610{day}_{hhmm}00"
+    t0 = f"{hhmm[:2]}:{hhmm[2:]}:00"
+    lines = [_line(day, t0, "run_start", source="terminal", started_by="ops",
+                   mode="VERIFY", claim_limit=0, test_sheet="", pid=1,
+                   log="x"),
+             _line(day, t0, "queue", claims=["9103000", "9104000"]),
+             _line(day, t0, "claim_start", n=1, total=2, claim="9103000"),
+             _line(day, f"{hhmm[:2]}:{hhmm[2:]}:{ss_last}", "claim_result",
+                   n=1, claim="9103000", category="Fail")]
+    if end:
+        lines.append(_line(day, f"{hhmm[:2]}:{hhmm[2:]}:{ss_last}",
+                           "run_end", exit_code=0))
+    return run_id, lines
+
+
+@case
+def the_server_dashboard_shows_a_run_still_writing_events_as_running():
+    from datetime import datetime
+    repo = FakeEventsRepo(_live_lines("30", "1015", "40"))
+    st = runs.live_status(runs.DbEvents(repo),
+                          now=datetime(2026, 10, 30, 10, 17, 0))
+    assert st["running"] is True and st["progress"] == [1, 2], st
+    assert st["counts"] == {"Pass": 0, "Fail": 1, "Review": 0}, st
+    assert st["source"] == "terminal" and st["tail"] == [], st
+
+
+@case
+def the_server_dashboard_shows_an_ended_or_silent_run_as_ended():
+    from datetime import datetime
+    repo = FakeEventsRepo(_live_lines("30", "1015", "40", end=True))
+    st = runs.live_status(runs.DbEvents(repo),
+                          now=datetime(2026, 10, 30, 10, 17, 0))
+    assert st["running"] is False and st["exit_code"] == 0, st
+    assert "Done" in st["message"] or "exit 0" in st["message"].lower() \
+        or st["message"], st
+    silent = FakeEventsRepo(_live_lines("30", "1015", "40"))
+    st = runs.live_status(runs.DbEvents(silent),
+                          now=datetime(2026, 10, 30, 11, 30, 0))
+    assert st["running"] is False, "no event for over 10 minutes"
+    assert runs.live_status(runs.DbEvents(FakeEventsRepo()))["running"] \
+        is False
+
+
+@case
+def the_server_portals_bot_status_comes_from_the_database():
+    import test_web_app as t
+    repo = FakeEventsRepo(_live_lines("30", "1015", "40", end=True))
+    client, _ = t.signed_in(t.FakeRepo(), t.FakeRunner(),
+                            events=runs.DbEvents(repo))
+    st = client.get("/api/bot/status").json()
+    assert st["running"] is False and st["exit_code"] == 0, st
+    assert st["counts"]["Fail"] == 1 and st["progress"] == [1, 2], st
+
+
+def main() -> int:
+    failures = 0
+    for fn in CASES:
+        name = fn.__name__.replace("_", " ")
+        try:
+            fn()
+            print(f"  ok    {name}")
+        except AssertionError as exc:
+            print(f"  FAIL  {name}: {exc}")
+            failures += 1
+        except Exception as exc:
+            print(f"  ERROR {name}: {type(exc).__name__}: {exc}")
+            failures += 1
+    print(f"\n{len(CASES) - failures}/{len(CASES)} passed")
+    return 1 if failures else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/import_run_events.py

@@ -0,0 +1,71 @@
+r"""
+Load this PC's older Run events into `dbo.CV_RunEvent` (2026-10-30), once,
+so the Portal on Dokploy lists the Runs from before the bot copied its
+events to the database itself. A Run already in the table is skipped, so a
+second run of this tool adds nothing.
+
+    python tools\import_run_events.py            # import
+    python tools\import_run_events.py --dry-run  # only say what it would do
+
+Run it from PowerShell on the bot PC (Windows login to the database).
+"""
+
+import sys
+import json
+from datetime import datetime
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(ROOT / "src"))
+
+from ecw_bot import settings as cfg                   # noqa: E402
+
+RUN_PREFIX = "events_"
+
+
+def rows_of(run_id: str, text: str) -> list:
+    """(run_id, at, kind, line) per timed JSON line; a bad line skipped."""
+    out = []
+    for line in text.splitlines():
+        try:
+            e = json.loads(line)
+            at = datetime.fromisoformat(e["t"])
+        except (ValueError, KeyError, TypeError):
+            continue
+        out.append((run_id, at, str(e.get("kind", ""))[:40], line))
+    return out
+
+
+def main(argv) -> int:
+    dry = "--dry-run" in argv
+    import pyodbc
+    conn = pyodbc.connect(cfg.RESULTS_DB_CONN, timeout=15)
+    cur = conn.cursor()
+    have = {r[0] for r in cur.execute(
+        "SELECT DISTINCT run_id FROM dbo.CV_RunEvent").fetchall()}
+    added_runs = added_rows = 0
+    for path in sorted(Path(cfg.LOGS).glob(f"{RUN_PREFIX}*.jsonl")):
+        run_id = path.stem[len(RUN_PREFIX):]
+        if run_id in have:
+            continue
+        rows = rows_of(run_id, path.read_text(encoding="utf-8",
+                                               errors="replace"))
+        if not rows:
+            continue
+        added_runs += 1
+        added_rows += len(rows)
+        if not dry:
+            cur.executemany(
+                "INSERT INTO dbo.CV_RunEvent (run_id, at, kind, body) "
+                "VALUES (?, ?, ?, ?)", rows)
+    if not dry:
+        conn.commit()
+    conn.close()
+    verb = "Would add" if dry else "Added"
+    print(f"{verb} {added_runs} run(s), {added_rows} event(s); "
+          f"{len(have)} run(s) were already in the table.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

tools/sql/create_tables.sql

@@ -104,6 +104,23 @@ BEGIN
 END
 GO
 
+-- Every Run's events (2026-10-30): the same JSON lines as the bot PC's
+-- Logs\events_<run>.jsonl, so the Portal on Dokploy has a Runs page and an
+-- Agent run card. Written by the bot (db_datawriter), read by cv_portal
+-- (db_datareader). tools/import_run_events.py loads the older files once.
+IF OBJECT_ID('dbo.CV_RunEvent', 'U') IS NULL
+BEGIN
+    CREATE TABLE dbo.CV_RunEvent (
+        id      BIGINT IDENTITY(1,1) PRIMARY KEY,
+        run_id  VARCHAR(15)    NOT NULL,  -- 20261030_101500, the run log's stamp
+        at      DATETIME2(3)   NOT NULL,
+        kind    NVARCHAR(40)   NOT NULL,  -- run_start, claim_start, run_end, ...
+        body    NVARCHAR(MAX)  NOT NULL   -- the event's JSON line
+    );
+    CREATE INDEX IX_CV_RunEvent_run ON dbo.CV_RunEvent (run_id, id);
+END
+GO
+
 -- So the bot can connect with your Windows login (no password), give that
 -- login read/write here. Uncomment and run once, as an admin (e.g. sa):
 --

web/app.py

@@ -325,12 +325,15 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
                allowed_origins=ALLOWED_ORIGINS, logs_dir=None,
                screenshots_root=None, users_store=None,
                sessions=None, can_start=True,
-               secure_cookie=False, ecare=None) -> FastAPI:
+               secure_cookie=False, ecare=None, events=None) -> FastAPI:
     app = FastAPI(title="eCW Charge Validation — Results",
                   docs_url=None, redoc_url=None, openapi_url=None)
     if runner is None:
         runner = BotRunner() if can_start else NoRunner()
     logs_dir = logs_dir if logs_dir is not None else cfg.LOGS
+    # Where the Runs page and the last-run banner read Runs from: this PC's
+    # Logs folder, or (the Portal on Dokploy, 2026-10-30) the database.
+    events = events if events is not None else run_view.FileEvents(logs_dir)
     screenshots_root = (screenshots_root if screenshots_root is not None
                         else cfg.ROOT / "Screenshots")
     # Made per server start and put in the page: a page on another site
@@ -554,7 +557,7 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
         """The latest Run, if it ended badly — the Dashboard's red banner
         (EEWA-33). A Run Error, or a Run that ended without a record
         (killed); never a Stop, a finished Run or one still going."""
-        last = run_view.latest_run(logs_dir, _live_run_id())
+        last = run_view.latest_run(events, _live_run_id())
         if last and last["status"] in ("Run Error", "Ended without a record"):
             return last
         return None
@@ -580,7 +583,7 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
                     raise BadFilter(f"Not a date: {value}")
         if status and status not in run_view.STATUSES:
             raise BadFilter(f"Not a result: {status}")
-        rows = run_view.list_runs(logs_dir, _live_run_id(),
+        rows = run_view.list_runs(events, _live_run_id(),
                                   day_from=day_from or None,
                                   day_to=day_to or None,
                                   status=status or None)
@@ -592,7 +595,7 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
 
     @app.get("/runs/{run_id}", response_class=HTMLResponse)
     def run_page(request: Request, run_id: str):
-        run = run_view.load_run(logs_dir, run_id, _live_run_id())
+        run = run_view.load_run(events, run_id, _live_run_id())
         if run is None:
             return HTMLResponse("<h1>No such run</h1>", status_code=404)
         return _page(request, "run.html",
@@ -602,7 +605,7 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
 
     @app.get("/runs/{run_id}/shot/{n}")
     def run_shot(run_id: str, n: int):
-        path = run_view.screenshot_path(logs_dir, run_id, n, screenshots_root)
+        path = run_view.screenshot_path(events, run_id, n, screenshots_root)
         if path is None:
             return HTMLResponse("<h1>No such screenshot</h1>",
                                 status_code=404)
@@ -652,6 +655,10 @@ def create_app(repo, allowed_hosts=ALLOWED_HOSTS, runner=None,
 
     @app.get("/api/bot/status")
     def bot_status():
+        # A server Portal cannot see the bot PC's process or log: the
+        # newest Run in the database instead (2026-10-30).
+        if isinstance(events, run_view.DbEvents):
+            return jsonable_encoder(run_view.live_status(events))
         return run_status.summarize(runner.status())
 
     # ── Sign-in (Stage 2, EEWA-32) ───────────────────────────────────────
@@ -873,8 +880,11 @@ def _safe_next(target) -> str:
     return target
 
 
-app = create_app(SqlResultsRepo(cfg.RESULTS_DB_CONN),
+_repo = SqlResultsRepo(cfg.RESULTS_DB_CONN)
+app = create_app(_repo,
                  users_store=SqlUserStore(cfg.RESULTS_DB_CONN),
+                 # A server Portal has no Logs folder: Runs from the database.
+                 events=run_view.DbEvents(_repo) if cfg.PORTAL_SERVER else None,
                  ecare=(EcareSignIn(cfg.AUTH_TOKEN_API_URL)
                         if cfg.AUTH_TOKEN_API_URL else None),
                  **portal_settings(cfg.PORTAL_URLS, cfg.PORTAL_SERVER))

web/repo.py

@@ -136,3 +136,21 @@ class SqlResultsRepo:
             f"SELECT TOP 200 {_SELECT_COLUMNS} FROM {results_db.TABLE} "
             f"WHERE claim_no LIKE ? ORDER BY run_at DESC, s_no, id",
             [f"%{claim_no}%"])
+
+    # ── Run events (2026-10-30): the Runs page on a server Portal ─────────
+
+    def run_ids(self) -> list:
+        """Every Run with events, newest first (at most 500)."""
+        return [r["run_id"] for r in self._query(
+            "SELECT TOP 500 run_id FROM dbo.CV_RunEvent "
+            "GROUP BY run_id ORDER BY run_id DESC")]
+
+    def run_event_lines(self, run_ids) -> list:
+        """(run_id, JSON line) for those Runs, in the order written."""
+        run_ids = list(run_ids)[:500]
+        if not run_ids:
+            return []
+        marks = ", ".join("?" * len(run_ids))
+        return [(r["run_id"], r["body"]) for r in self._query(
+            f"SELECT run_id, body FROM dbo.CV_RunEvent "
+            f"WHERE run_id IN ({marks}) ORDER BY run_id, id", run_ids)]

web/runs.py

@@ -64,12 +64,17 @@ def read_events(logs_dir, run_id) -> list:
     if not valid_run_id(run_id):
         return []
     path = Path(logs_dir) / f"events_{run_id}.jsonl"
-    out = []
     try:
         text = path.read_text(encoding="utf-8", errors="replace")
     except OSError:
         return []
-    for line in text.splitlines():
+    return _parse(text.splitlines())
+
+
+def _parse(lines) -> list:
+    """Event dicts from JSON lines, a bad or untimed line skipped."""
+    out = []
+    for line in lines:
         try:
             e = json.loads(line)
         except ValueError:
@@ -224,34 +229,79 @@ def _time_of_id(run_id):
         return None
 
 
+class FileEvents:
+    """Runs from this PC's `Logs/events_<run>.jsonl` files."""
+
+    def __init__(self, logs_dir):
+        self.dir = Path(logs_dir)
+
+    def run_ids(self) -> list:
+        try:
+            files = sorted(self.dir.glob("events_*.jsonl"), reverse=True)
+        except OSError:
+            return []
+        return [p.stem[len("events_"):] for p in files
+                if valid_run_id(p.stem[len("events_"):])]
+
+    def events(self, run_ids) -> dict:
+        return {r: read_events(self.dir, r) for r in run_ids
+                if (self.dir / f"events_{r}.jsonl").is_file()}
+
+
+class DbEvents:
+    """Runs from `dbo.CV_RunEvent` (2026-10-30): the Portal on Dokploy,
+    which cannot read the bot PC's Logs folder. One query per page."""
+
+    def __init__(self, repo):
+        self.repo = repo
+
+    def run_ids(self) -> list:
+        return [r for r in self.repo.run_ids() if valid_run_id(r)]
+
+    def events(self, run_ids) -> dict:
+        run_ids = list(run_ids)
+        if not run_ids:
+            return {}
+        lines = {}
+        for run_id, body in self.repo.run_event_lines(run_ids):
+            lines.setdefault(run_id, []).append(body)
+        return {r: _parse(lines[r]) for r in run_ids if r in lines}
+
+
+def _source(src):
+    """A path (the old way, and the tests) or an events source."""
+    return src if hasattr(src, "run_ids") else FileEvents(src)
+
+
 def load_run(logs_dir, run_id, live_run_id=None, now=None):
-    """One Run's detail, or None (bad id, no events file)."""
+    """One Run's detail, or None (bad id, no events)."""
     if not valid_run_id(run_id):
         return None
-    if not (Path(logs_dir) / f"events_{run_id}.jsonl").is_file():
+    found = _source(logs_dir).events([run_id])
+    if run_id not in found:
         return None
-    return summarize(run_id, read_events(logs_dir, run_id), live_run_id, now)
+    return summarize(run_id, found[run_id], live_run_id, now)
 
 
 def list_runs(logs_dir, live_run_id=None, now=None, day_from=None,
               day_to=None, status=None) -> list:
-    """Every Run with an events file, newest first, filtered. `day_from`
+    """Every Run with events, newest first, filtered. `day_from` /
-    / `day_to` are ISO dates (inclusive), `status` one of STATUSES."""
+    `day_to` are ISO dates (inclusive), `status` one of STATUSES."""
     d_from = date.fromisoformat(day_from) if day_from else None
     d_to = date.fromisoformat(day_to) if day_to else None
-    out = []
+    src = _source(logs_dir)
-    try:
+    ids = []
-        files = sorted(Path(logs_dir).glob("events_*.jsonl"), reverse=True)
+    for run_id in src.run_ids():
-    except OSError:
-        files = []
-    for path in files:
-        run_id = path.stem[len("events_"):]
-        if not valid_run_id(run_id):
-            continue
         day = _time_of_id(run_id).date()
         if (d_from and day < d_from) or (d_to and day > d_to):
             continue
-        r = summarize(run_id, read_events(logs_dir, run_id), live_run_id, now)
+        ids.append(run_id)
+    found = src.events(ids)
+    out = []
+    for run_id in ids:
+        if run_id not in found:
+            continue
+        r = summarize(run_id, found[run_id], live_run_id, now)
         if status and r["status"] != status:
             continue
         out.append(r)
@@ -259,24 +309,22 @@ def list_runs(logs_dir, live_run_id=None, now=None, day_from=None,
 
 
 def latest_run(logs_dir, live_run_id=None, now=None):
-    """The newest Run's summary, or None — reads one file, not all."""
+    """The newest Run's summary, or None — reads one Run, not all."""
-    try:
+    src = _source(logs_dir)
-        files = sorted(Path(logs_dir).glob("events_*.jsonl"), reverse=True)
+    for run_id in src.run_ids():
-    except OSError:
+        found = src.events([run_id])
-        return None
+        if run_id in found:
-    for path in files:
+            return summarize(run_id, found[run_id], live_run_id, now)
-        run_id = path.stem[len("events_"):]
-        if valid_run_id(run_id):
-            return summarize(run_id, read_events(logs_dir, run_id),
-                             live_run_id, now)
     return None
 
 
 def screenshot_path(logs_dir, run_id, n, screenshots_root):
     """The n-th screenshot event's .png, if it is inside `screenshots_root`
     and exists; else None. Only paths a Run itself recorded are served."""
-    shots = [e for e in read_events(logs_dir, run_id)
+    if not valid_run_id(run_id):
-             if e["kind"] == "screenshot"]
+        return None
+    events = _source(logs_dir).events([run_id]).get(run_id, [])
+    shots = [e for e in events if e["kind"] == "screenshot"]
     if not 0 <= n < len(shots):
         return None
     try:
@@ -290,3 +338,50 @@ def screenshot_path(logs_dir, run_id, n, screenshots_root):
             or not path.is_file()):
         return None
     return path
+
+
+LIVE_STALE_S = 600
+
+
+def live_status(source, now=None) -> dict:
+    """The Dashboard's Agent run card on a server Portal (2026-10-30): the
+    newest Run in the events source, in `/api/bot/status`'s shape. Running
+    = no end yet and an event in the last LIVE_STALE_S seconds (the server
+    cannot see the bot PC's process). No log tail: that is the PC's log."""
+    from .run_status import exit_text
+    now = now or datetime.now()
+    out = {"running": False, "starting": False, "started_at": None,
+           "ended_at": None, "exit_code": None, "stopped_by_operator": False,
+           "source": None, "started_by": None, "phase": "", "progress": None,
+           "current": None, "counts": {"Pass": 0, "Fail": 0, "Review": 0},
+           "tail": [], "claims": [], "results_run_at": None,
+           "elapsed_s": None, "message": "No run yet"}
+    try:
+        ids = source.run_ids()
+    except Exception:
+        return out
+    for run_id in ids:
+        found = source.events([run_id])
+        if run_id not in found or not found[run_id]:
+            continue
+        events = found[run_id]
+        r = summarize(run_id, events, None, now)
+        ended = any(e["kind"] in ("run_end", "stopped") for e in events)
+        fresh = (now - events[-1]["_t"]).total_seconds() < LIVE_STALE_S
+        running = not ended and fresh
+        stopped = r["status"] == "Stopped"
+        out.update(
+            running=running, source=r["source"], started_by=r["started_by"],
+            started_at=r["started"].isoformat() if r["started"] else None,
+            ended_at=r["ended"].isoformat() if r["ended"] else None,
+            exit_code=r["exit_code"], stopped_by_operator=stopped,
+            progress=([r["claims_done"], r["claims_total"]]
+                      if r["claims_total"] else None),
+            counts={"Pass": r["pass"], "Fail": r["fail"],
+                    "Review": r["review"]},
+            elapsed_s=r["duration_s"],
+            message=(f"Running on the bot PC · {r['claims_done']} of "
+                     f"{r['claims_total']} claims" if running
+                     else exit_text(r["exit_code"], stopped)))
+        return out
+    return out