Add rpa-bot-handoff skill: ops guide for bot handover

The skill scans a bot with subagents (never runs it) and writes
HANDOVER.md, an 11-section ops guide in the operator's order: where is
what, how to run, how long to wait, after the run, how to see an
exception, exception list, how you will know, how to tell us, don'ts,
issues seen before. Dev material (check against 12 failure classes from
Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md.

Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval
runner (--resume, --regrade) with results from 47 runs of the earlier
13-section version. The grader still expects that structure and must be
updated before the next eval run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

tests/evals/results/handoff/h-web-portal-full-r1.md

@@ -0,0 +1,192 @@
+# Payer portal downloader - ops handover
+
+Version: not versioned (the project has no version string) · Written 2026-10-23 · Escalation: RPA dev team, Teams channel **#rpa-support**
+
+## 1. What the bot does
+
+Every day at 07:00 the bot signs in to the payer web portal and downloads each provider's remittance report as a CSV file. It reads the input workbook `data/input/providers.xlsx`. The **Login** sheet holds one portal username and password. The **Providers** sheet has one provider per row: NPI, name, and an active flag. For each active provider it opens that provider's remittance page and clicks the CSV export. The CSVs go to `data/downloads/`. At the end it writes one report, `data/output/Download_Report_<YYYYMMDD>.xlsx`, with one row per provider: Provider, NPI, Status, Remarks. Windows Task Scheduler starts it. It drives one website, the payer portal, in its own Chrome window.
+
+## 2. How it works
+
+```
+Windows Task Scheduler (07:00 daily)
+   └─ starts → Downloader script (no window of its own)
+                    │  ──drives──> Chrome window → payer web portal
+                    │               (do not touch this window while it runs)
+                    ├─ reads  → data/input/providers.xlsx
+                    ├─ saves  → data/downloads/*.csv        (Chrome saves the files)
+                    ├─ saves  → data/screens/<NPI>.png      (only when a page did not load)
+                    ├─ writes → data/output/Download_Report_<YYYYMMDD>.xlsx  (at the end)
+                    └─ writes → logs/bot.log                (one line per provider)
+```
+
+- **Task Scheduler** starts the run. The bot has no screen or Stop button of its own.
+- **Downloader script**: the bot. It works through the provider list in order, one at a time.
+- **Chrome window**: opened and closed by the bot. Everything it does on the portal happens in this window.
+- **data/input/providers.xlsx**: the input. Whoever maintains it controls which providers are downloaded.
+- **data/downloads/**: the CSVs. The portal chooses the file names. The bot does **not** rename them, so a file name does not tell you which provider it belongs to.
+- **data/screens/**: a screenshot of the portal page, saved only when a provider's report page did not load.
+- **Download report**: the main place to check results. It is written only when the run ends, whether it finished or crashed.
+- **logs/bot.log**: one line per provider with its status (not the remark), plus a warning line for each re-login.
+
+**One run, step by step:**
+1. Read the input workbook: the login from the Login sheet, the provider list from the Providers sheet.
+2. Open Chrome and sign in to the portal. If the portal's home page doesn't appear within 30 s, every provider gets **Login Failed** and the run ends.
+3. For each provider:
+   1. If it is marked inactive, write Skipped.
+   2. Otherwise open its remittance page.
+   3. If the portal has logged the bot out, sign in again and retry. It signs in again at most 2 times per provider.
+   4. Wait up to 30 s for the report grid.
+   5. If the portal says there are no remittances, write No Data.
+   6. Otherwise click Export CSV and wait up to 60 s for a new CSV in `data/downloads/`.
+4. Close Chrome and save the download report.
+
+## 3. Before a run
+
+- [ ] Google Chrome is installed on the bot machine and can open the portal.
+- [ ] These folders exist. The bot does **not** create them:
+  - `logs/`: without it, the bot stops before doing anything and writes no log.
+  - `data/output/`: without it, the report is not saved, even after a good run.
+  - `data/downloads/`
+  - `data/screens/`: without it, failure screenshots are silently not saved.
+- [ ] `config.yaml` is next to the bot.
+- [ ] `data/input/providers.xlsx` exists and has both sheets, spelled exactly **Login** and **Providers**.
+- [ ] **Login** sheet: the username is in cell A2 and the password in B2. Check that this login works by signing in to the portal by hand, especially after a password change.
+- [ ] **Providers** sheet, from row 2 down:
+  - **One provider per row.** Column A = NPI, column B = provider name, column C = active flag.
+  - Rows with an empty column A are skipped without any message. Don't leave the NPI blank.
+  - **To make a provider inactive, type a capital `N` in column C**, with nothing else in the cell.
+    - Good: `N`
+    - Bad: `n`, `No`, ` N` (leading space). The bot treats these as **active** and downloads that provider anyway.
+  - NPI as a whole number or text.
+    - Good: `1234567890`
+    - Bad: `1234567890.0`. Decimal formatting puts the wrong NPI into the portal address.
+- [ ] If today's report already exists in `data/output/`, **copy it somewhere else before a second run today**. A second run on the same day overwrites it.
+
+## 4. During a run
+
+- **Do not click, type in, close or navigate the Chrome window the bot opened.** Any of these ends the run straight away, and the providers after that point are left out of the report (E08).
+- Don't save or download anything into `data/downloads/` while the bot runs. **Any** new CSV in that folder counts as success for whichever provider the bot is on (E10).
+- Other programs and other Chrome windows on the machine are fine. The bot does not need the mouse or keyboard.
+- **To stop the bot:** there is no Stop button. The only clean way is to end the task in Windows Task Scheduler, and ops cannot open it. **Ask in #rpa-support.** Closing the bot's Chrome window also stops it, but it counts as a crash: the report keeps only the providers finished so far.
+- **A run looks stuck when** `logs/bot.log` has had no new line for **10 minutes**. One provider normally takes under 2 minutes. The worst case, with all re-logins and waits, is about 4 minutes. **The bot has no stuck-detection or overall time limit of its own**, so it will not give up by itself. After 10 minutes with no new line, escalate (E09).
+
+## 5. Where to look when something goes wrong
+
+| Question | Where to look |
+|---|---|
+| Did the run finish? | `data/output/` should have `Download_Report_<today's date>.xlsx`, with **one row per provider in the input** (inactive ones included). No file today means the run never started or failed before saving (E09). Fewer rows than the input means it crashed partway (E08). |
+| Which providers failed? | The download report, **Status** column. Filter for `Error` and `Login Failed`. **Also filter `Downloaded` and read Remarks**: a maintenance row is green but has no file (E05). |
+| Why did one provider fail? | The download report, **Remarks** column on that row. If the remark is "Report grid did not load", open `data/screens/<NPI>.png` to see what the portal showed. |
+| Did the bot already retry? | `logs/bot.log`: search for "session expired, re-login" just before that provider's line. That is the only retry the bot does. Grid, download and first-login timeouts are **never** retried. |
+| When did it run, and how far did it get? | `logs/bot.log`: each line has a date and time. The last line of the day is the last provider finished. |
+| Is the CSV really there? | `data/downloads/`, sorted by date modified. The number of new CSVs from today should match the number of `Downloaded - Report saved` rows. |
+| Which version is running? | There is no version to check. See section 10. |
+
+## 6. Statuses
+
+The bot writes these values in the **Status** column of the download report. The colour is the fill of the Status cell.
+
+| Status | Remarks | Colour | Meaning | What ops does |
+|---|---|---|---|---|
+| Downloaded | Report saved | green | A new CSV appeared in `data/downloads/` after the export click. The bot does not check that the file is really this provider's. | Nothing. If the CSV count doesn't match, see E10. |
+| Downloaded | portal maintenance - no file | **green, but needs follow-up** | **No file was downloaded.** The portal showed a maintenance banner. | Download it again once maintenance is over (E05). |
+| No Data | No remittances in the date range | green | The portal says this provider has no remittances. No file is expected. | Nothing. |
+| Skipped - inactive provider | Marked N in the input | green | Column C of the input is `N`. Not attempted. | Nothing, unless the provider should be active. Then fix column C. |
+| Login Failed | Check the portal username/password | **no colour (white)** | The first sign-in failed, so **no provider was attempted.** Every row in the report has this status. | See E01. |
+| Error | Report grid did not load | red | The report page didn't load within 30 s. A screenshot was saved. | See E03. |
+| Error | Download did not finish | red | Export was clicked, but no new CSV arrived within 60 s. | See E04. |
+| Error | Session expired | red | The portal logged the bot out 3 times on this provider. | See E02. |
+| *(row missing)* | - | - | The provider is in the input but not in the report. The run crashed before reaching it. | See E08. |
+
+## 7. Exception catalogue
+
+| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
+|---|---|---|---|---|---|---|---|
+| E01 | **Every** row is `Login Failed` - "Check the portal username/password" (cells not coloured) | After signing in, it waits 30 s for the portal home page. If the page doesn't appear, it gives up. A wrong password and a slow portal look the same. | Download report, all rows. Nothing in `logs/bot.log` for that run. | System (or wrong login in the input) | Ops | 1. Sign in to the portal by hand with the Login sheet username and password. 2. If that fails, correct the login in cells A2/B2. 3. If it works by hand, the portal was probably slow. Ask in #rpa-support for a re-run. | Manual login works but the next run is all Login Failed again. Send: the report file, the run date and time. |
+| E02 | One row: `Error` - "Session expired" | When it opens a provider's page and lands on the login page instead, it signs in again and retries, up to 3 attempts in total. | Download report. The log has "session expired, re-login 1" and "2" lines before that provider's line. | System | Bot, then Ops | The bot already retried twice. Download this provider's report by hand from the portal. | It happens on several providers in one run, or on two days running. Send: the report file, `logs/bot.log`, the run date. |
+| E03 | One row: `Error` - "Report grid did not load" | It waits 30 s for the report grid on the provider's page. If the grid doesn't appear, it takes a screenshot and moves on. No retry. | Download report. Screenshot `data/screens/<NPI>.png` (replaced on the next failure for the same provider). | System | Ops | 1. Open the screenshot. 2. Wrong NPI or "provider not found" in the portal: check column A of the input. 3. Blank or slow page: download this provider by hand from the portal. | Many providers fail in one run, or the screenshot shows a new portal layout. Send: the report file, the screenshots, the run date and time. |
+| E04 | One row: `Error` - "Download did not finish" | After clicking Export CSV, it checks `data/downloads/` every second for 60 s. If no new CSV appears, it moves on. No retry. | Download report only. No screenshot. | System | Ops | 1. Check `data/downloads/` for a CSV that arrived late, close to that time. **A late file may have been counted as the next provider's download** (see E10). 2. Download this provider by hand from the portal. | It happens to several providers in one run. Send: the report file, a listing of `data/downloads/` with times, the run date. |
+| E05 | Green row: `Downloaded` - "**portal maintenance - no file**" | The portal page shows a maintenance banner. The bot doesn't wait or retry. | Download report, Remarks column. | System (portal side) | Ops | **Treat it as not downloaded.** Download it by hand once maintenance is over. | Every provider shows it on several days in a row. |
+| E06 | Row: `No Data` - "No remittances in the date range" | The portal shows its "no results" message on the provider's page. | Download report. | Business | Ops (nothing to do) | Nothing. Confirm in the portal only if remittances were expected. | Never, unless the portal shows remittances the bot called No Data. |
+| E07 | Row: `Skipped - inactive provider` - "Marked N in the input" / or an inactive provider was downloaded anyway | Column C is exactly `N`. Anything else (`n`, `No`, ` N`) counts as active. | Download report. | Business | Ops | To make a provider inactive, type a capital `N` alone in column C. To reactivate it, clear the cell. | - |
+| E08 | **Report has fewer rows than the input.** The last providers are missing. No error row explains it. | Not detected. Something the bot doesn't handle stopped the run mid-list. Possible causes: a re-login failed, a portal button was missing, or the Chrome window was closed or touched. | Partial download report. The last line in `logs/bot.log` is the last provider finished. The error itself is **not** in the log. | System | Dev | Note the last provider in the log. Download the missing providers by hand if they are needed today. | Always. Send: the report file, `logs/bot.log`, the run date and time, the last provider in the log, and whether anyone touched the bot's Chrome window. |
+| E09 | **No report for today** in `data/output/`, or the log has no lines today, or no new log line for 10+ minutes | Not detected. The run never started, failed before saving, or is hanging. Possible causes: a missing folder (section 3), a missing input file, a Chrome problem, the portal hanging. | Nothing ops can see. Only Task Scheduler (dev access) has the result. | System | Dev | 1. Check the section 3 list: folders, input file name, sheet names. 2. Post in #rpa-support. | Always. Send: the date, the time you checked, the last line of `logs/bot.log`, and which section 3 items you checked. |
+| E10 | More `Downloaded - Report saved` rows than new CSVs, or a CSV that looks like the wrong provider's | The bot counts **any** new CSV in `data/downloads/` as this provider's file. A late file from the previous provider, or a file someone else saved there, counts. | Download report vs the contents of `data/downloads/`. | System | Ops, then Dev | 1. Count today's new CSVs. 2. Open any suspicious one and check its provider. 3. Download the doubtful providers by hand. | The counts don't match on more than one day. Send: the report file, a listing of `data/downloads/` with times. |
+| E11 | This morning's report is gone, replaced by a later one | A second run on the same day writes a report with the same name. | `data/output/` - only one report per date. | Business (process) | Ops | Before any second run on a day, copy the existing report somewhere else. | - |
+
+## 8. What the bot recovers by itself
+
+| Problem | What the bot does | Limit | After the limit |
+|---|---|---|---|
+| Portal logged the bot out (detected when opening a provider's page) | Signs in again and retries the same provider | 2 re-logins, so 3 attempts per provider | Row = `Error` - "Session expired", next provider. **If a re-login itself fails, the run stops** (E08). |
+| Report grid slow | Waits | 30 s, **no retry** | Row = `Error` - "Report grid did not load", screenshot, next provider |
+| CSV download slow | Waits, checking every second | 60 s, **no retry** | Row = `Error` - "Download did not finish", next provider |
+| First sign-in slow or failed | Waits | 30 s, **no retry** | Every row `Login Failed`, run ends |
+| Run hangs or stuck | **Nothing.** No watchdog, no restart. | None | Runs until someone ends it in Task Scheduler (dev). Escalate after 10 min with no new log line. |
+| Run crashes | Closes Chrome and saves a partial report | - | Missing providers are not marked. No automatic re-run. A re-run starts from the first provider again and re-downloads everything. |
+
+## 9. Known incidents
+
+The log on the bot machine covers only one run (2026-09-02). Older history was not available.
+
+| When / version | What ops saw | Cause | What to do now | Ops could handle? |
+|---|---|---|---|---|
+| 2026-09-02 (unversioned) | One provider red `Error` in the day's report. In the log, a "session expired, re-login 1" warning, then that provider as Error about 80 s later. | The portal logged the bot out about a minute into the run. The re-login worked, but the retry then hit either the 30 s grid wait or the 60 s download wait. The log doesn't say which, because remarks are not logged. | Read that row's Remarks. If it is "Report grid did not load", check `data/screens/`. Download the provider by hand (E03 / E04). | Yes |
+
+## 10. Releases and version check
+
+- **How an update arrives:** there is no update mechanism and no version number. The dev team changes the bot's files on the bot machine directly.
+- **How to confirm it applied:** ops cannot check this. Ask in #rpa-support for confirmation, with the date the change went live.
+- **If behaviour did not change after a promised fix:** post in #rpa-support with the run date and the report file.
+- There is only one bot machine. No shared resource is rewritten on start, so version drift between machines does not apply.
+
+## 11. Escalating to dev
+
+Post in Teams **#rpa-support**. Send:
+- the exception ID from section 7 (E01-E11)
+- the run date and the time you noticed
+- the download report `data/output/Download_Report_<YYYYMMDD>.xlsx`
+- `logs/bot.log`
+- any screenshot from `data/screens/`
+- the last provider line in the log, for a crash
+
+**Never send:** the portal password, the input workbook (it holds the login), `config.yaml`, or provider or patient data pasted into chat.
+
+## 12. Failure-class check
+
+- F1 (missing prerequisite) covered in section 3 (Chrome, folders, config, input sheets) and E09.
+- F2 (wrong window or focus) covered in section 4: don't touch the bot's Chrome window, don't save into `data/downloads/`. The bot uses its own browser session, not window titles or focus.
+- F3 (target slower or different) covered in E01, E03 and E04, with limits in section 8. Portal layout changes lead to E08.
+- F4 (input read differently) covered in section 3 input rules (the `N` flag, NPI format, blank rows, sheet names) and E07.
+- F5 (marked done when nothing was done) covered in E05 and E10, and Fix before handover items 1 and 2.
+- F6 (action happened but not confirmed) covered in E04: a late CSV can be credited to the next provider, so check `data/downloads/` before re-downloading. Fix item 2.
+- F7 (business exception shown as success) covered in section 6 (maintenance row green, needs follow-up), E05 and Fix item 1.
+- F8 (endless re-runs or watchdog): no auto-rerun loop exists. The re-login loop is capped at 2. The missing watchdog is covered in section 4, section 8 and Fix item 11.
+- F9 (stopping the wrong way) covered in section 4 (no Stop button; ending the task needs dev) and Fix item 11.
+- F10 not applicable: there is no batch or queue gate. Each daily run is independent, and a failed provider blocks nothing. The nearest risk is a crash dropping the remaining providers, covered in E08.
+- F11 (cause hidden from ops) covered in Fix items 4, 5 and 6: crash errors and remarks are not in the log, and the run result is only in Task Scheduler. Section 5 points only at the report, `data/screens/`, `data/downloads/` and `logs/bot.log`.
+- F12 (update didn't arrive) covered in section 10 and Fix item 12: no version string and no release process.
+
+## 13. Fix before handover (dev team)
+
+| # | Gap | Why it hurts ops | Where (file:line) |
+|---|---|---|---|
+| 1 | Portal maintenance returns status `Downloaded` (coloured green) with no file. | Ops reads green as done, and the report is never downloaded. | `bot.py:54-55`, colour map `bot.py:27` |
+| 2 | Any new `*.csv` in the shared downloads folder counts as success. Files are never renamed to the provider. | A late file is credited to the wrong provider. A CSV can't be tied to a provider. | `bot.py:63-68` |
+| 3 | A failed re-login inside the provider loop raises an exception that nothing catches. Missing `username`/`password`/`signin`/`export-csv` elements and any browser error do the same. | The run crashes. The remaining providers are silently absent from the report, with no "not attempted" row. | `bot.py:102`, `bot.py:41-43`, `bot.py:64`, loop `bot.py:92-107` |
+| 4 | No traceback, run-start or run-end line is ever logged. | A crash or a run that never started looks like a quiet day. Ops can't tell a finished run from a dead one. | `bot.py:30-32`, `bot.py:85-110` |
+| 5 | The log line has the status but not the remark. | The log alone can't say which of three `Error` reasons happened (seen on 2026-09-02). | `bot.py:105` |
+| 6 | A `Login Failed` run still exits normally. Task Scheduler shows success, and no alert is sent. | A whole day of downloads can be lost without anyone noticing. | `bot.py:88-91` |
+| 7 | `Login Failed` rows get no colour. | They don't stand out in the report. | `bot.py:90` (no `fill`), `bot.py:107` |
+| 8 | The first login has no retry. A 30 s timeout means a slow portal and a wrong password give the same message. | Ops changes a correct password, or re-runs a wrong one. | `bot.py:44-47`, `bot.py:86-91` |
+| 9 | Required folders are never created. A missing `logs/` crashes at import. A missing `data/output/` loses the report. A missing `data/screens/` silently drops screenshots. | Silent or total failure on a fresh machine. | `bot.py:20-21`, `bot.py:30`, `bot.py:59`, `bot.py:110` |
+| 10 | The report name has the date only, so a same-day re-run overwrites it. There is no resume or single-provider re-run, so a re-run re-downloads every provider (duplicate CSVs). Screenshots are overwritten per NPI. | Evidence is lost. Ops can't re-do just the failed providers. | `bot.py:110`, `bot.py:92`, `bot.py:59` |
+| 11 | No Stop mechanism, no overall run timeout, no stuck-detection. | Ops can't stop a hung run and has to wait on dev. | `bot.py:73-110` |
+| 12 | No version string and no documented release process. | Ops can't confirm a fix is live. | `README.md`, `bot.py` (none) |
+| 13 | Input not validated. The active flag is an exact `!= "N"` test. The NPI becomes `str()` of the cell, so a float cell gives `"123.0"`. Sheets and credentials are not checked. | A provider is downloaded when it should be skipped, or the wrong NPI is used, with no message. | `bot.py:76-78` |
+| 14 | Session expiry is only detected when a page is opened. An expiry during the grid or download wait shows up as `Report grid did not load` or `Download did not finish` instead. | Misleading cause on the report. | `bot.py:52-53` |
+| 15 | If `driver.quit()` raises in `finally`, the report is never saved. | The whole run's results are lost. | `bot.py:108-110` |
+| 16 | Security: `config.yaml` holds a plaintext token that the code never reads, so **rotate and remove it**. The portal password is stored in plain cells of the input workbook. Provider names are written to every log line. | Exposure risk. Ops can't safely share the input or the log. | `config.yaml:2`, `bot.py:76`, `bot.py:105` |
+| 17 | Undocumented marker file `BOT_WAS_RUN` is written to the project root on every run. | Confusing, with no stated purpose. | `bot.py:74` |
+| 18 | `logs/bot.log` holds only one run (2026-09-02), although the bot appends daily. History was cleared or not provided. | Past incidents can't be reviewed. Please supply the older logs, past reports and Task Scheduler history. | `logs/bot.log`, `bot.py:30` |