Skills/Rpa_Skills
8
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add rpa-bot-handoff skill: ops guide for bot handover

Browse Source 
The skill scans a bot with subagents (never runs it) and writes
HANDOVER.md, an 11-section ops guide in the operator's order: where is
what, how to run, how long to wait, after the run, how to see an
exception, exception list, how you will know, how to tell us, don'ts,
issues seen before. Dev material (check against 12 failure classes from
Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md.

Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval
runner (--resume, --regrade) with results from 47 runs of the earlier
13-section version. The grader still expects that structure and must be
updated before the next eval run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
VasanthandClaude Opus 5.5 committed 2026-10-28 15:01:42 +05:30
1 parent 3d2db6aca6
commit 3942fad6cd
143 files changed
+11815 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/evals/results/handoff/h-api-sync-screens-r2.md
+179
View File
@@ -0,0 +1,179 @@
# Invoice sync - ops handover
Version 0.9.1 · Written 2026-10-23 · Escalation: raise a ticket in the **RPA Support** queue
## 1. What the bot does
Every hour, the bot sends approved invoices from the finance export file (`data/input/invoices.csv`) into the billing system. It goes through the file one invoice at a time. First it checks that the invoice has an invoice number, customer, amount and currency, and that the amount is a positive number. Then it sends the invoice to billing. For every invoice it writes one row in a report file, `data/output/sync_report_<date>_<time>.csv`, giving the result and the reason. The bot has no window and no screens. It runs in the background, started by Windows Task Scheduler. It does not drive any desktop app or website. It talks to the billing system directly over the network.
## 2. How it works
```
Windows Task Scheduler (every hour)
└─ starts → Invoice sync (background, no window)
│
├─ reads → data/input/invoices.csv (finance export)
├─ sends → Billing system (over the network)
├─ writes → data/output/sync_report_<date>_<time>.csv ← ops reads this
└─ writes → logs/sync.log (dev only)
```
- **Task Scheduler** starts the bot at the top of every hour.
- **Invoice sync** is the bot itself. It has no window, so you never see it running.
- **The finance export** is the bot's only input. Each run reads the **whole** file again from the top.
- **The billing system** is where invoices end up. You can look up an invoice there to see whether it really arrived.
- **The report file** gets one new file per run, with one row per invoice. This is the only place ops can see results.
- **The log** holds one short line per invoice with no reason. Ops cannot open it; dev uses it.
**One run, step by step:**
1. The bot starts and creates an empty report for this hour.
2. It reads the next invoice from the finance export.
3. It checks the invoice. If a field is missing or the amount is wrong, the row is **Invalid** and it moves on.
4. It sends the invoice to billing.
5. If billing gives no clear answer, the bot first checks whether billing already has the invoice. If billing does not have it, the bot waits and tries again, up to 5 times.
6. It writes the result (Synced, Duplicate, Invalid or Failed) and a message into the report.
7. It repeats steps 2-6 until the file ends, then the run is over.
## 3. Before a run
These are mostly for dev or whoever sets up the machine and prepares the export. Ops should know them, because each one has a symptom in section 7.
- [ ] The settings file (`.env`) is in the bot's folder and holds the billing address and a **valid, current** API key. If the key is missing or expired, the bot does not warn. Every invoice ends as **Failed** (E10).
- [ ] The folders `data/output` and `logs` exist in the bot's folder. If either is missing, the run stops at once and **no report file appears** (E01).
- [ ] Python and the bot's two add-on packages are installed on the machine. If not, no report file appears (E01).
- [ ] Only **one** machine has the hourly task. Two machines running it would send the same invoices twice.
- [ ] Input rules for the finance export:
- **Export the file straight from the finance system. Do not open it and save it again in Excel.** A file re-saved by Excel can make every row show "missing invoice_no" (E03).
- **Amounts are plain numbers, no thousands separator and no currency sign.** Bad: `1,250.00`, `₹500`. Good: `1250.00`, `500`.
- **Amounts must be above zero.** Credit notes and zero invoices are rejected as Invalid (E04). The bot does not handle them.
- Every row needs invoice_no, customer_id, amount and currency filled in. customer_name is copied into the report but is not checked.
## 4. During a run
- **Ops does not need to do anything while a run is going on.** There is no window to watch and nothing to click.
- The current hour's report is written while the bot works. If you open it during the run you may see only part of it. **Judge an hour's report only once the next hour's report has appeared.**
- **To stop the bot:** ops cannot stop it. If it must stop (for example, it is sending wrong invoices), raise an **urgent** ticket in RPA Support. Dev disables the hourly task. Closing a report file or the billing system does **not** stop it.
- **How long a run takes:** usually a few minutes. Each invoice that billing does not answer can take **up to about 6 minutes** before it is marked Failed. A file with many failures can run past the hour.
- **When to worry:** no new report file in `data/output` for **2 hours in a row**. See E01.
## 5. Where to look when something goes wrong
Ops can open only the output folder `data/output` and the report files in it. Everything else is dev-only.
| Question | Where to look |
|---|---|
| Did the run happen? | `data/output`: is there a `sync_report_<date>_<time>.csv` for that hour? No file means the run did not start, or it stopped at once (E01). |
| Did the run finish? | **You cannot tell for sure from the report.** If a report has far fewer rows than the hour before, the run probably stopped partway (E11). See Fix before handover #7. |
| Which invoices failed? | Open the report and filter the **Status** column for `Failed` and `Invalid`. |
| Why did one invoice fail? | The **Message** column in the same row. |
| Did the bot already retry? | Every `Failed` row has **already** been tried 5 times. A `Synced` row with "confirmed after retry" was also retried. Every invoice still in the export is tried again in the next hourly run. |
| Did the invoice really reach billing? | Search for the invoice number in the billing system. |
| Which version is running? | Ops cannot see this. Ask in the ticket (Fix before handover #12). |
## 6. Statuses
The report is a plain CSV file, so there are no colours. Status is the **Status** column and the message is the **Message** column.
| Status - Message | Colour | Meaning | What ops does |
|---|---|---|---|
| Synced - "Created" | none | Billing created the invoice | nothing |
| Synced - "Created (confirmed after retry)" | none | The first send got no clear answer. The bot then checked and found the invoice in billing | nothing (see E09) |
| **Synced - "Created (some lines rejected)"** | none | **Needs follow-up.** Billing created the invoice but **dropped some of its lines**. The report does not say which ones | Check the invoice in billing against the export (E08) |
| Duplicate - "Invoice already exists in billing" | none | Billing already has this invoice number. Normal for invoices sent in earlier hours, because each run re-sends the whole export | nothing, unless the invoice is new (E07) |
| Invalid - "missing <field names>" | none | A required field is empty in the export. Not sent | Fix the export (E02 / E03) |
| Invalid - "amount must be positive" | none | Amount is zero or negative. Not sent | Handle by hand (E04) |
| Invalid - "amount '<value>' is not a number" | none | Amount has commas, a currency sign or text. Not sent | Fix the export (E05) |
| Invalid - "Rejected by billing", or billing's own reason text | none | Billing refused the invoice and said why. Not sent | Fix per the message (E06) |
| Failed - "No success after 5 tries" | none | Billing never gave a usable answer after 5 tries. **Not sent** | See E10 |
## 7. Exception catalogue
| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
|---|---|---|---|---|---|---|---|
| E01 | No report file in `data/output` for an hour | The bot stops before writing anything. Causes: the output or log folder is missing, the export file is missing, Python is not set up, or the hourly task did not run | Nowhere ops can see (dev: console output only) | System | Dev | Wait for the next hour. A missing hour is caught up by the next run, because each run re-reads the whole export | No report for **2 hours in a row**. Send: the hours with no report and a screenshot of the `data/output` folder |
| E02 | Invalid - "missing customer_id" (or other field names) on some rows | Before sending, the bot checks that invoice_no, customer_id, amount and currency are filled in | Report: Status, Message | Business | Ops | Ask finance to fill the field in the source system. The next hourly run picks the row up | The field is filled in the finance system but still reported missing |
| E03 | **Every** row is Invalid - "missing invoice_no" (and the invoice_no column in the report is empty) | The bot cannot read the first column heading, usually because the export was re-saved in Excel | Report: Status, Message | Business | Ops, then Dev | Ask finance to re-export the file straight from the finance system, not through Excel | It still happens after a clean export. Send: the report file name |
| E04 | Invalid - "amount must be positive" | The amount is zero or negative | Report: Status, Message | Business | Ops | The bot does not send credit notes or zero invoices. Handle them by hand in billing per your process, or ask finance to remove them from the export | Never (by design) |
| E05 | Invalid - "amount '<value>' is not a number" | The amount cannot be read as a number (commas, currency sign, text) | Report: Status, Message | Business | Ops | Ask finance to export the amount as a plain number, e.g. `1250.00` | The amount already looks like a plain number |
| E06 | Invalid - billing's own reason, or "Rejected by billing" | Billing refused the invoice and sent a reason | Report: Status, Message | Business | Ops | Read the message, then fix the data in finance (e.g. unknown customer, wrong currency) | The message is unclear or the data looks right. Send: report file name and invoice number |
| E07 | Duplicate - "Invoice already exists in billing" for an invoice that is **new** this hour | Billing says the invoice number is already in use | Report: Status, Message | Business | Ops | Search billing for the number. If it is our own invoice from a moment ago, nothing to do. If it is a **different** invoice with the same number, ask finance to renumber | Billing shows no invoice with that number |
| E08 | Synced - "Created (some lines rejected)" | Billing accepted the invoice but said some lines were rejected | Report: Status, Message. **Which lines is not recorded** | Business | Ops | Open the invoice in billing, compare its lines with the export, add or fix the missing lines by hand | You cannot work out which lines are missing. Send: report file name and invoice number |
| E09 | Synced - "Created (confirmed after retry)" | The first send timed out or got an error. Before trying again, the bot asked billing and found the invoice | Report: Status, Message | System | Bot | Nothing. If you want to be sure, check that billing has only **one** copy of the invoice (see E13) | Billing has two copies |
| E10 | Failed - "No success after 5 tries" | Billing gave no usable answer 5 times in a row (down, too busy, or the API key is wrong or expired) | Report: Status, Message | System | Bot, then Ops / Dev | **First check billing for the invoice.** If it is there, nothing to do. If not, wait: the next hourly run tries it again | **Every** row in a report is Failed (likely the API key), or the same invoice is Failed in **2 hourly reports in a row**. Send: report file names and invoice numbers |
| E11 | Report stops partway: far fewer rows than the hour before, and invoices you expected are missing from it | The run stopped on an error it does not handle. Typical cause: billing timed out while the bot was checking whether an invoice exists, or billing sent back an unreadable rejection | Nowhere ops can see. The report just ends | System | Bot, then Dev | Wait for the next hour. The next run re-reads the whole export and continues; invoices already sent show Duplicate | The report stops at the **same** invoice 2 hours in a row. Send: report file names and the last invoice number in each |
| E12 | The same invoice appears in every hourly report, never Synced | The export still contains it, and each run sends the whole file again | Report: one row per hour | Business / System | Ops, then Dev | For Invalid rows, fix the data (E02-E06). For Failed rows, see E10 | Failed for 2 hours or more. Send: the report file names |
| E13 | An invoice appears **twice** in billing | Billing created it, but its answer was lost, and billing's lookup did not show it yet, so the bot sent it again. The bot cannot catch this case | Not recorded. Seen only in billing | System | Ops, then Dev | Void the extra copy in billing per your billing process | Always. Send: invoice number, both report file names where it appears |
## 8. What the bot recovers by itself
| Problem | What the bot does | Limit | After the limit |
|---|---|---|---|
| Billing does not answer, is busy, or answers with an error | Before each new try, checks whether billing already has the invoice (if yes: Synced, "confirmed after retry"). Otherwise waits and sends again | 5 tries, waiting 2, 4, 8, 16, 32 s between them. **Up to about 6 minutes per invoice** | Row = Failed, bot moves to the next invoice |
| An invoice was Failed, Invalid or skipped by a stopped run | Nothing during the run. The **next hourly run** reads the whole export again and retries every invoice in it | No limit. It is retried every hour for as long as it stays in the export | Same result every hour until the data or billing is fixed (E12) |
| The run stops partway (E11) | No restart within the hour | n/a | The next hourly run starts again from the top |
There is no watchdog, no automatic restart and no "resume". The hourly schedule is the only recovery.
## 9. Known incidents
Both incidents happened before v0.9.1. Git history does not hold real dates.
| When / version | What ops saw | Cause | What to do now | Ops could handle? |
|---|---|---|---|---|
| Before 0.9.1 | Invoices appeared **twice** in billing | Billing timed out after it had already created the invoice, and the bot sent it again | Fixed: the bot now checks billing before each retry. It can still happen in rare cases (E13). Void the extra copy | Partly. Ops can find and void duplicates. Only code could prevent them |
| Before 0.9.1 | Hourly run stopped partway when billing said "too many requests"; later invoices not sent that hour | No retry for a busy billing system | Fixed: retries with waiting (section 8). A similar stop can still happen if billing times out during the check (E11) | Partly. Waiting for the next hour is safe |
## 10. Releases and version check
- How an update arrives: dev replaces the bot files on the machine by hand. There is no launcher and no automatic update.
- How to confirm it applied: ops cannot see the version. Ask dev to confirm the version in the ticket after any update.
- If behaviour did not change after an update: raise a ticket. Only one machine should run the bot, so there is only one place to update.
## 11. Escalating to dev
Raise a ticket in the **RPA Support** queue.
Send:
- the exception ID from section 7 (e.g. E10)
- the report file name(s), e.g. `sync_report_<date>_<time>.csv`
- the invoice number(s)
- the time you noticed it
- for E01, a screenshot of the `data/output` folder
Never send: the `.env` settings file, API keys, or customer names and amounts in chat. Attach the report file to the ticket instead.
## 12. Failure-class check
- F1 covered in section 3 (settings file, folders, Python), E01, E10, and Fix #1, #6, #8.
- F2 not applicable: the bot drives no windows. It talks to billing over the network.
- F3 covered in E10, E11 and section 8. A slow billing system is handled by the 30 s wait and 5 retries; a timeout during the existence check is Fix #3.
- F4 covered in section 3 input rules, E03 and E05, and Fix #9.
- F5 covered in E08, and Fix #2 and #4.
- F6 covered in E09, E10 ("check billing first") and E13, and Fix #4.
- F7 covered in section 6 ("Created (some lines rejected)" needs follow-up) and E08, and Fix #2.
- F8 covered in E12 and section 8 (every invoice in the export is retried every hour, without limit), and Fix #10.
- F9 covered in section 4: ops raises an urgent ticket and dev disables the task. Fix #11.
- F10 covered in section 4 and E10: failing invoices slow the run (up to 6 min each) but never block other invoices. Fix #11.
- F11 covered in section 5 and E01 / E11: the real cause is often only in console output or the dev-only log. Fix #7.
- F12 covered in section 10. The version is not visible to ops. Fix #12.
## 13. Fix before handover (dev team)
| # | Gap | Why it hurts ops | Where (file:line) |
|---|---|---|---|
| 1 | The settings file with the live API key was committed to git in the first commit, and there is no `.gitignore` | Credential exposure. Rotate the key and remove the file from history | `.env`, commit `88b2057` |
| 2 | A partial reject from billing (207) is stored as Synced, and the rejected lines are not recorded | Ops sees "Synced" and assumes it is done (F7) | `sync.py:54-55` |
| 3 | The existence check has no error handling. A timeout there stops the whole run | Report stops partway with no reason (E11), the same symptom as the 429 incident | `sync.py:41-43`, `sync.py:60` |
| 4 | The existence check treats any non-200 (429, 5xx, 401) as "not there" and sends again. There is no idempotency key | Duplicates in billing are still possible (E13) | `sync.py:43`, `sync.py:50` |
| 5 | A 422 with a non-JSON body crashes the run | Report stops partway (E11) | `sync.py:59` |
| 6 | Every non-success code is retried, including 400/401/403. A missing API key silently becomes empty | An expired key shows as a generic "Failed" on every row, about a minute per row, with no hint | `sync.py:19`, `sync.py:60-63` |
| 7 | Network errors and crashes go to the console only. The log holds no reason, and the report has no "run finished" row. Ops cannot open the log | Ops cannot tell a finished run from a stopped one, or see why it stopped (F11) | `sync.py:52`, `sync.py:75`, `sync.py:77`, `sync.py:84` |
| 8 | `data/output` is not in the repo and not created by the code. `logs/` is also assumed to exist | On a fresh machine the first run stops with no report (E01) | `sync.py:24`, `sync.py:68-69` |
| 9 | The input is read as `utf-8`, not `utf-8-sig`. `nan` and `inf` amounts pass validation | An Excel-saved export makes every row Invalid (E03); bad amounts reach billing | `sync.py:69`, `sync.py:34` |
| 10 | The whole export is re-sent every hour, with no archive or processed marker | Hourly "Duplicate" noise; Failed rows retried forever (E12) | `sync.py:72` |
| 11 | No run lock and no time budget, and the report name only goes down to the minute | A long run can overlap the next one; two runs in the same minute overwrite each other's report | `sync.py:66-68` |
| 12 | The version exists only in the README (which also has a stray commit line) and is not shown anywhere ops can see | Ops cannot confirm an update applied (F12) | `README.md:5-6` |
| 13 | The two "fix" commits only added comment lines. The retry and existence-check code was already in the initial import | History does not show when the fixes actually landed | `sync.py:86-87`, commits `83a380f`, `b2dcd70` |
| 14 | A `BOT_WAS_RUN` marker file is written to the repo root on every run | Unexplained file in the bot folder | `sync.py:67` |
| 15 | The bot sleeps after the 5th failed try and ignores `Retry-After` | Adds 32 s to every Failed row and may keep hitting the rate limit | `sync.py:62` |
| 16 | The task definition for the hourly schedule is not in the repo | Rebuilding the machine depends on memory | n/a (missing file) |