Add rpa-bot-handoff skill: ops guide for bot handover

The skill scans a bot with subagents (never runs it) and writes
HANDOVER.md, an 11-section ops guide in the operator's order: where is
what, how to run, how long to wait, after the run, how to see an
exception, exception list, how you will know, how to tell us, don'ts,
issues seen before. Dev material (check against 12 failure classes from
Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md.

Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval
runner (--resume, --regrade) with results from 47 runs of the earlier
13-section version. The grader still expects that structure and must be
updated before the next eval run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

tests/evals/results/handoff/h-api-sync-screens-r5.md

@@ -0,0 +1,158 @@
+# Invoice sync - ops handover
+
+Version 0.9.1 · Written 2026-10-23 · Escalation: raise a ticket in the **RPA Support** queue
+
+## 1. What the bot does
+
+Every hour, the bot reads the finance export of approved invoices (one invoice per row) and sends each invoice to the billing system over its web connection (API). It does not drive any desktop app or website screen. For every run it writes one report file in the output folder that lists each invoice with a **Status** and a **Message**. That report is the only place ops can see the result.
+
+## 2. How it works
+
+```
+Windows Task Scheduler (every hour)
+   └─ starts → Invoice sync bot ──sends each invoice──> Billing system (web API)
+                     │
+                     ├─ reads  → finance export file (input, one invoice per row)
+                     ├─ writes → data/output/sync_report_<date>_<time>.csv   (ops can open)
+                     └─ writes → a log file in the bot folder               (dev only)
+```
+
+- **Task Scheduler** starts the bot once an hour. There is no app window and nothing to click.
+- **The bot** checks each invoice, sends it to billing, and retries by itself when billing is busy or down.
+- **The billing system** is where the invoice is created. It is the final truth: if in doubt, check there.
+- **The report file** gets one row per invoice: invoice number, customer, Status, Message.
+- **The log file** is for developers only. Ops cannot open it.
+
+**One run, step by step:**
+1. Create a new report file, named with the date and time the run started (for example `sync_report_20260903_0900.csv` = 3 Sep 2026, 09:00).
+2. Read the input file, top to bottom.
+3. For each invoice: check that the required fields are filled and the amount is a positive number. If not, mark it **Invalid** and move on, without sending it.
+4. Send it to billing. If billing says yes, mark it **Synced**. If billing says it already exists, mark it **Duplicate**. If billing rejects the data, mark it **Invalid** with billing's reason.
+5. If billing is busy, down or does not answer: check whether the invoice was created anyway, wait, and try again, up to 5 times. Then mark it **Failed**.
+6. After the last invoice, the run ends. The report does not show an "end of run" line (see section 13).
+
+## 3. Before a run
+
+- [ ] The `data/output` folder exists in the bot folder. **The bot does not create it.** Without it the run stops at once and no report appears at all (E09).
+- [ ] The bot's settings file (billing address and access key) is in the bot folder. Without it, the run stops at the first invoice or every invoice ends **Failed** (E07, E09). Only dev can check this file.
+- [ ] Input file: every row has **invoice_no, customer_id, amount and currency** filled. customer_name is optional for the bot but appears in the report.
+- [ ] Amounts are plain numbers with a dot for decimals, no thousands separator and no currency sign.
+  - Bad: `1,200.00` · `$1200` · `-50` · `0` → Invalid
+  - Good: `1200.00`
+- [ ] Save the input as plain CSV. If Excel saves it as "CSV UTF-8", every row can come out **Invalid - "missing invoice_no"** (E02).
+- [ ] The version is 0.9.1 (see section 10 - ops cannot check this yourselves today).
+
+## 4. During a run
+
+- There is nothing to watch on screen. The bot has no window and does not use the mouse or keyboard, so you can use the machine as normal.
+- To stop the bot: end the "Invoice sync" task in Windows Task Scheduler (or ask whoever manages that machine). Do not delete or move the report file while a run is going. Invoices already sent stay in billing. Invoices not yet reached are tried again in the next hourly run.
+- A run looks slow when billing is having problems. Each invoice that fails can take **about 1 minute, and up to about 6 minutes**, before the bot gives up and marks it Failed. A file with many failing invoices can take longer than an hour. Wait for that before you act.
+- The same invoice can show up in more than one hourly report. An invoice that was **Synced** in an earlier run shows **Duplicate** the next time. That is normal.
+
+## 5. Where to look when something goes wrong
+
+Ops can open only the **output folder** (`data/output`) and the report files in it.
+
+| Question | Where to look |
+|---|---|
+| Did the run happen? | Output folder: there is a `sync_report_<date>_<time>.csv` for that hour. No file for an hour means the run did not start or stopped at once (E09). |
+| Did the run finish? | **This cannot be seen in the report today.** A report that has fewer rows than invoices in the input, or that is empty below the header, means the run stopped part-way (E08). Raise a ticket. |
+| Which invoices failed? | The report → filter the **Status** column: anything other than Synced. Also check Synced rows whose Message is "Created (some lines rejected)". |
+| Why did one invoice fail? | The report → the **Message** column on that row. Then use section 7. |
+| Did the bot already retry? | Message "No success after 5 tries" = yes, 5 times. Message "Created (confirmed after retry)" = it retried and the invoice is in billing. Any other message = no retry was needed. |
+| Which version is running? | Ops cannot see it. Ask in the ticket (section 13). |
+
+## 6. Statuses
+
+The report is a plain CSV file, so there are **no colours**. Read the Status and Message columns together.
+
+| Status | Message | Meaning | What ops does |
+|---|---|---|---|
+| Synced | Created | Billing created the invoice | nothing |
+| Synced | Created (confirmed after retry) | Billing did not answer at first, but the bot then confirmed the invoice exists in billing | nothing |
+| Synced | Created (some lines rejected) | **Needs follow-up.** The invoice was created, but billing refused some of its lines. Shown as Synced. | Check the invoice in billing and fix the missing lines (E05) |
+| Duplicate | Invoice already exists in billing | Billing already has an invoice with this number | nothing, if it was Synced in an earlier run. Otherwise check it (E04) |
+| Invalid | missing <field names> | A required field is empty in the input. Not sent. | Fix the input row (E01, E02) |
+| Invalid | amount must be positive | Amount is 0 or negative. Not sent. | Fix the input row (E03) |
+| Invalid | amount '<value>' is not a number | Amount has commas, a currency sign or text. Not sent. | Fix the input row (E03) |
+| Invalid | <billing's reason>, or "Rejected by billing" | The bot sent it and billing refused the data | Read billing's reason and fix the input row (E06) |
+| Failed | No success after 5 tries | Billing was busy, down, unreachable or refused access 5 times in a row | See E07 |
+
+## 7. Exception catalogue
+
+Quote the ID (E01, E02...) in your ticket.
+
+| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
+|---|---|---|---|---|---|---|---|
+| E01 | Invalid - "missing customer_id" (or currency, amount, invoice_no) on some rows | Before sending, it checks the four required fields are not empty | Report, Status + Message | Business | Ops | Fill the field in the finance export. The next hourly run picks it up. | Never - this is an input fix |
+| E02 | **Every** row Invalid - "missing invoice_no", although the numbers are in the file | Same check. The first column name in the file is not read as "invoice_no", usually because Excel saved it as "CSV UTF-8" | Report, Status + Message | Business | Ops, then Dev | Re-save the input as plain "CSV (comma delimited)" and wait for the next run | Still every row Invalid after re-saving. Send the report file name and the time. |
+| E03 | Invalid - "amount must be positive" or "amount '...' is not a number" | Before sending, it reads the amount as a number and checks it is above 0 | Report, Status + Message | Business | Ops | Write the amount as a plain number, e.g. `1200.00`. Credit notes / negative amounts cannot go through this bot. | A real negative amount must be billed - dev must decide how |
+| E04 | Duplicate - "Invoice already exists in billing" | Billing answers that the invoice number already exists | Report, Status + Message | Business | Ops | If the invoice was Synced in an earlier hourly report, nothing to do. If it never synced before, open it in billing and check it is the right invoice (same customer, same amount). | The invoice in billing belongs to a different customer or amount |
+| E05 | **Synced** - "Created (some lines rejected)" | Billing answers "partly accepted" | Report, Status + Message. Which lines were refused is **not** recorded anywhere ops can see | Business | Ops | Treat as **not done**. Open the invoice in billing, compare its lines with the export, and fix the missing lines in billing. | You cannot tell which lines are missing. Send the invoice number and report file name. |
+| E06 | Invalid - with a reason from billing, or "Rejected by billing" | Billing refuses the data as wrong (e.g. unknown customer, bad currency) | Report, Status + Message | Business | Ops | Read the reason, fix the input row, wait for the next run | The reason is "Rejected by billing" with no detail, or the data looks correct |
+| E07 | Failed - "No success after 5 tries" | Billing is busy, down or does not answer within 30 s. Before each new try the bot checks whether billing created the invoice anyway. After 5 tries it gives up. | Report, Status + Message | System | Bot, then Ops / Dev | One or a few rows: leave them. The next hourly run tries again, and if the invoice was created after all it will show Duplicate (safe). | **Every** row in a report is Failed (billing down, or the bot's access key is wrong or expired), or the same invoice is Failed in 3 hourly runs in a row. Send the report file name(s) and the time. |
+| E08 | The report stops part-way: fewer rows than the input, or only the header row | The bot stopped unexpectedly (for example, it lost the network connection while checking billing, or billing sent an answer the bot could not read). The invoices after that point were not tried. | Report is shorter than the input. The reason is only in the developer log. | System | Dev | Raise a ticket. The next hourly run starts from the top again, so missing invoices may still go through. Check the next report. | Always - send the report file name, the time, and the last invoice number in the report |
+| E09 | No report file at all for an hour (or for every hour) | The run stopped before writing anything: the output folder is missing, the settings file is missing, or the task did not start | Nothing ops can see - the absence of the file is the sign | System | Dev | Raise a ticket | Always - send the hour(s) with no report |
+| E10 | An invoice appears twice in billing | Before 0.9.1, a timeout after billing had already created the invoice made the bot create it again. Fixed in 0.9.1, but two runs running at the same time can still send the same invoice. | Billing only | System | Ops, then Dev | Void the extra copy in billing following the normal finance process | It happens on 0.9.1. Send both invoice numbers/IDs from billing and the report file names. |
+
+## 8. What the bot recovers by itself
+
+| Problem | What the bot does | Limit | After the limit |
+|---|---|---|---|
+| Billing is busy ("too many requests") | Waits and tries again: 2, 4, 8, 16, 32 seconds | 5 tries per invoice, about 1 minute | Row = Failed, next invoice |
+| Billing is down or errors | Checks whether the invoice was created anyway; if yes, marks Synced. Otherwise waits and tries again. | 5 tries per invoice, about 1 to 6 minutes | Row = Failed, next invoice |
+| Billing does not answer | Waits up to 30 s per attempt, then same as above | 5 tries per invoice, up to about 6 minutes | Row = Failed, next invoice |
+| Whole run stopped (E08, E09) | No restart within the hour. The next hourly run starts from the top of the input. | 1 hour | Next hourly run |
+
+The bot never retries Invalid or Duplicate rows by itself within a run. They come back in the next hourly run only if the row is still in the input.
+
+## 9. Known incidents
+
+| When / version | What ops saw | Cause | What to do now | Ops could handle? |
+|---|---|---|---|---|
+| Before 0.9.1 | Some invoices created twice in billing | Billing timed out after it had already created the invoice; the bot sent it again | Fixed in 0.9.1: the bot checks billing before each retry. If it still happens, see E10. | Partly - ops could spot and void duplicates, not prevent them |
+| Before 0.9.1 | Hourly run crashed when billing was busy | Billing's "too many requests" answer was not handled | Fixed in 0.9.1 (retries with waits). | No |
+
+## 10. Releases and version check
+
+- How an update arrives: a developer replaces the bot files on the machine. There is no automatic update.
+- How to confirm it applied: ops cannot see the version today (no screen, not in the report). Ask dev to confirm the version in the ticket.
+- If the version did not change: raise a ticket in the RPA Support queue.
+
+## 11. Escalating to dev
+
+Raise a ticket in the **RPA Support** queue.
+
+Send: the exception ID from section 7, the report file name (e.g. `sync_report_20260903_0900.csv`), the time you noticed it, and the invoice number(s) involved.
+Never send: passwords or access keys, the bot's settings file, or customer data in chat. Attach the report file to the ticket instead of pasting rows.
+
+## 12. Failure-class check
+
+- F1 covered in section 3 (output folder, settings file) and E09, E07.
+- F2 not applicable: the bot drives no windows; it talks to billing over its web API.
+- F3 covered in E07 and section 8 (30 s wait per attempt, 5 tries). No screens, so no screen-size rules.
+- F4 covered in section 3 input rules, E01, E02, E03; gaps in section 13 #3.
+- F5 not applicable: Synced is only written after billing itself confirms the invoice exists.
+- F6 covered in E07 (bot checks billing before retrying; re-run is safe, shows Duplicate) and section 13 #6.
+- F7 covered in section 6 and E05; section 13 #1.
+- F8 covered in section 8 and section 4 (long runs); overlap risk in section 13 #7.
+- F9 covered in section 4 (end the task in Task Scheduler).
+- F10 covered in E08 (one bad answer stops the rest of the run) and section 13 #4.
+- F11 covered in section 5 and section 13 #2, #5.
+- F12 covered in section 10 and section 13 #8.
+
+## 13. Fix before handover (dev team)
+
+| # | Gap | Why it hurts ops | Where (file:line) |
+|---|---|---|---|
+| 1 | Partly accepted invoices (billing 207) are written as Synced | Ops reads them as done; refused lines are never followed up. Which lines were refused is not recorded. | sync.py:54-55 |
+| 2 | Network errors and crash reasons go to the console only (`print`), which Task Scheduler throws away; the log keeps only invoice + status, not the message | The cause of E07/E08/E09 is lost, even for dev | sync.py:52, sync.py:75, sync.py:84 |
+| 3 | Input read as `utf-8`, not `utf-8-sig`; amount parsing rejects `1,200.00` | Excel "CSV UTF-8" files make every row Invalid (E02); common amount formats are rejected | sync.py:69, sync.py:34 |
+| 4 | `exists()` has no error handling, and `r.json()` on a 422 is not guarded | One network blip or non-JSON reply stops the whole run; later invoices are skipped (E08) | sync.py:41-43, sync.py:59-60 |
+| 5 | No end-of-run marker or summary in the report | Ops cannot tell a finished run from one that stopped part-way | sync.py:77 (log only) |
+| 6 | 401/403/404 and other 4xx are retried like outages and end as "Failed - No success after 5 tries" | A wrong/expired access key looks like a billing outage, and each invoice burns ~1-6 min | sync.py:54-62 |
+| 7 | No guard against two hourly runs overlapping; report named only to the minute | Long runs can overlap and send the same invoice twice (E10); two runs in the same minute overwrite one report | sync.py:68, Task Scheduler settings |
+| 8 | Version is only in README; not shown in the report or log | Ops cannot confirm an update applied | README.md:5 |
+| 9 | `data/output` is not created by the code | Fresh install or a cleared folder = no report, every hour (E09) | sync.py:68-69 |
+| 10 | The `.env` settings file (billing access key) is committed to git | Credential exposed to anyone with repo access; rotate the key and remove it from history | .env (in git since 9fa3ae1) |
+| 11 | After the 5th failed try the bot still waits 32 s before giving up | Slows runs with no benefit | sync.py:62 |