Skills/Rpa_Skills
8
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add rpa-bot-handoff skill: ops guide for bot handover

Browse Source 
The skill scans a bot with subagents (never runs it) and writes
HANDOVER.md, an 11-section ops guide in the operator's order: where is
what, how to run, how long to wait, after the run, how to see an
exception, exception list, how you will know, how to tell us, don'ts,
issues seen before. Dev material (check against 12 failure classes from
Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md.

Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval
runner (--resume, --regrade) with results from 47 runs of the earlier
13-section version. The grader still expects that structure and must be
updated before the next eval run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
VasanthandClaude Opus 5.5 committed 2026-10-28 15:01:42 +05:30
1 parent 3d2db6aca6
commit 3942fad6cd
143 files changed
+11815 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/evals/results/handoff/h-packaged-desktop-full-r4.md
+172
View File
@@ -0,0 +1,172 @@
# LedgerBot - ops handover
Version 3.2.0 · Written 2026-10-23 · Escalation: RPA dev team, Teams channel **#rpa-support**
## 1. What the bot does
LedgerBot reconciles the daily bank statement Excel with **LedgerPro** (the desktop accounting app). It exports LedgerPro's entries to Excel, compares them with the statement line by line (by reference and amount), and **posts every statement line that LedgerPro doesn't have yet** by clicking LedgerPro's buttons and typing the entry. Lines where the amounts differ are flagged, never posted. It runs once per statement (normally daily). Input: the bank statement Excel. Output: one result Excel per run in the `data\output` folder, plus a log in the `logs` folder. It drives one app: **LedgerPro**.
## 2. How it works
```
launcher.exe (what you start)
└─ checks for an update, then starts → LedgerBot.exe (black console window, no screens)
│
├──drives──> LedgerPro (do not touch while it runs)
│ └─ export → data\ledger_export.xlsx
├─ reads → the bank statement Excel you gave it
├─ writes → data\output\Reconcile_<date>_<time>.xlsx (at the very end)
└─ writes → logs\ledger_<date>.log and logs\notfound_<button>.png
```
- **launcher.exe** looks for a newer build on the update server, installs it if there is one, starts LedgerBot and then closes. It does not watch the bot.
- **LedgerBot.exe** is the bot itself. It has no windows of its own, only a black console window. Error details appear **only in that console window**.
- **LedgerPro** is opened by the bot from its Desktop shortcut. The bot finds LedgerPro's buttons by **matching pictures of them on the screen**, then types into whatever window is in front.
- **The result Excel** has one row per statement line: Reference, Amount, Result, Note. It is written **only when the run finishes**.
- **The log** has one line per statement line, the same as the result Excel. A run that crashes leaves only the lines up to the crash.
**One run, step by step:**
1. Opens LedgerPro from the Desktop shortcut and waits 15 seconds.
2. Clicks LedgerPro's "Export to Excel" button and waits 10 seconds for `data\ledger_export.xlsx`.
3. Reads that export and your bank statement (column A = reference, column B = amount, from row 2).
4. For each statement line: same reference and same amount (within 0.01) → OK. Amount differs → Mismatch, not posted. Reference not in LedgerPro → the bot clicks "New entry", types the reference, Tab, the amount, and clicks "Save".
5. Saves the result Excel in `data\output` and ends.
## 3. Before a run
- [ ] **Windows display scaling is 100% and the screen resolution is 1920 x 1080.** At any other setting the bot can't find LedgerPro's buttons (E03, E04).
- [ ] The **LedgerPro shortcut** (named exactly `LedgerPro`) is on your Desktop or the Public Desktop (E01).
- [ ] **LedgerPro opens straight to its main screen**: already signed in, no login box, no "what's new" or licence popup. The bot can't sign in or close popups.
- [ ] LedgerPro's Excel export saves to the bot's `data` folder as `ledger_export.xlsx`. **Delete yesterday's `ledger_export.xlsx` before the run**, so an export that fails can't be mistaken for today's (E06).
- [ ] The bot folder has a `logs` folder and a `data\output` folder. Without `logs` the bot won't start; without `data\output` it posts everything and then loses the result file (E09).
- [ ] Start the bot **with the statement file**: drag the bank statement Excel onto `launcher.exe`. Starting it with no file closes it at once (E08).
- [ ] Bank statement rules:
- Column A = reference, column B = amount, row 1 = headers, data from row 2.
- **Every amount is a number.** A blank or text amount stops the whole run (E08). Bad: `1,250.00 CR` or empty. Good: `1250.00`.
- **Each reference appears once.** If a reference appears twice, the bot keeps only the last row and silently ignores the other one. Bad: two rows `INV-1001`. Good: combine or give them distinct references.
- **Store references as text, the same way LedgerPro shows them.** A reference typed as a plain number (for example `5512`) may be read as `5512.0`, not match LedgerPro, and get **posted twice**. Bad: `5512` as a number. Good: `5512` formatted as Text.
- [ ] Check the version: see section 10. Today there is **no place on screen that shows it**.
## 4. During a run
- **Hands off the machine.** Don't touch the mouse or keyboard, don't open other windows, don't lock the screen; turn off the screensaver and sleep. The bot types into whatever window is in front: if you click another window, the reference and amount get typed there.
- Keep LedgerPro maximised and in front. Don't move or resize it.
- **To stop the bot:** close the black LedgerBot console window. (Moving the mouse hard into a screen corner also stops it.) Closing LedgerPro does **not** stop it: the bot keeps looking for buttons and marks the remaining lines Error.
- After stopping it early: **no result Excel is written.** Check in LedgerPro which entries were already posted before you re-run (see E07).
- A run looks stuck when nothing happens on screen for more than about a minute. The bot never gives up by itself on the whole run: each button gets 20 seconds, then it moves on. There is no watchdog. A run with many Missing lines where LedgerPro has frozen takes about 40 seconds per line; stop it.
## 5. Where to look when something goes wrong
| Question | Where to look |
|---|---|
| Did the run finish? | `data\output` → is there a `Reconcile_<date>_<time>.xlsx` from this run? **No file = the run crashed** (or was stopped). |
| Which lines failed? | The result Excel → filter **Result** = `Error` or `Mismatch` |
| Why did one line fail? | The result Excel → **Note** column. Then `logs\ledger_<date>.log` → search for the reference. |
| Why did the whole run stop? | The black console window, if still open: the last lines say why. The log only shows the last line processed before the crash. |
| What did the screen look like when a button wasn't found? | `logs\notfound_<button>.png` (only the most recent one per button; each failure overwrites the last) |
| Did the bot already retry? | It keeps looking for each button for 20 seconds; it never re-tries a line or a run. |
| Which version is running? | Nowhere on screen today. Ask dev on #rpa-support (see section 10). |
The log file is named by the date the run **started**. A run that crosses midnight keeps writing to the previous day's log.
## 6. Statuses
These are the values in the **Result** and **Note** columns of the result Excel (and the log). The Excel has no colours.
| Result | Note | Meaning | What ops does |
|---|---|---|---|
| OK | (empty) | LedgerPro already has this reference with the same amount | nothing |
| Missing | Posted | **The bot typed the entry and clicked Save. It did not check that LedgerPro saved it.** Result stays "Missing" even though it was posted. | Spot-check a few in LedgerPro. Any LedgerPro error popup during the run means check them all (E07). |
| Mismatch | LedgerPro has <amount> | Reference exists in LedgerPro with a different amount. **Not posted.** | Needs follow-up: compare the statement with LedgerPro and correct by hand (E02) |
| Error | Button new_entry.png not found on screen | Couldn't find LedgerPro's "New entry" button. **Not posted.** | E03 |
| Error | Button save_btn.png not found on screen | The entry was typed but "Save" wasn't found. **Probably not saved; a half-filled entry may be left open.** | E04 |
| (no result file) | - | The run crashed or was stopped | E01, E05, E06, E08, E09, E10, E11 |
The bot also writes a file called `BOT_WAS_RUN` in its folder on every start. It means nothing to ops; ignore it.
## 7. Exception catalogue
| ID | What you see | How the bot identifies it | Where it is recorded | Type | Who acts | What to do | Escalate when |
|---|---|---|---|---|---|---|---|
| E01 | LedgerPro never opens; console shows "LedgerPro shortcut not found on the Desktop"; no result file | It looks for a shortcut called `LedgerPro` on your Desktop and the Public Desktop; if neither has it, it stops | Console window only | System | Ops | Put the LedgerPro shortcut on the Desktop with exactly that name, re-run | The shortcut is there and it still fails: send the console text and time |
| E02 | Result `Mismatch`, Note "LedgerPro has <amount>" | Reference found in LedgerPro but the amounts differ by more than 0.01 | Result Excel, log | Business | Ops | Compare the statement line with the LedgerPro entry, correct whichever is wrong by hand. The bot never fixes these. | Never (business decision) |
| E03 | Result `Error`, Note "Button new_entry.png not found on screen" | It looks for the "New entry" button picture for 20 s; if it never matches, it saves a screenshot and moves to the next line | Result Excel, log, `logs\notfound_new_entry.png` | System | Ops, then Dev | Check display scaling 100% and 1920 x 1080; open the screenshot (popup over LedgerPro? wrong window?). Close any popup, re-run. Lines already posted show as OK on the re-run. | **Every** Missing line is Error (likely a bad build, see the first incident in section 9), or it fails again with correct settings: send the log, the screenshot, run time |
| E04 | Result `Error`, Note "Button save_btn.png not found on screen" | Same picture search for "Save", after the reference and amount were already typed | Result Excel, log, `logs\notfound_save_btn.png` | System | Ops | **Check LedgerPro first.** A half-filled or saved entry may be there. Cancel or finish it by hand, then re-run. The following lines may also fail, because the half-filled form is still open. | Fails again with 100% scaling: send the log, screenshot, run time |
| E05 | No result file; console shows "Button export_excel.png not found on screen" | Picture search for the Export button fails after 20 s; the bot stops | Console; `logs\notfound_export_excel.png` | System | Ops, then Dev | Check scaling, that LedgerPro opened to its main screen (no login/popup), re-run | Fails twice with correct settings: send the screenshot, time |
| E06 | No result file, the console mentions `ledger_export.xlsx` not found; **or** lines that should be OK get posted again | The bot waits a fixed 10 s for the export and doesn't check it. If the export is slow it finds no file (stops) or **yesterday's** file (reconciles against old data) | Console only; or duplicates in LedgerPro | System | Ops | Delete the old `ledger_export.xlsx` before each run (section 3). If duplicates were posted, remove them in LedgerPro by hand. | It happens on most runs (the wait is too short for this laptop): send the time, log |
| E07 | An entry marked "Posted" is not in LedgerPro, or LedgerPro showed an error popup during the run | Nothing: the bot writes "Posted" as soon as it clicks Save | Result Excel shows "Posted" anyway | System | **Ops** | Check the line in LedgerPro. If not saved, post it by hand, or re-run (a fresh run posts what's still missing). **Never re-run before checking** if the bot was stopped mid-run: lines it already posted might be posted again if the export is stale. | LedgerPro rejects the bot's entries regularly: send the reference, time, log |
| E08 | Console flashes and closes at once, or shows an error about a number / "could not convert"; no result file | The bot stops on a statement amount that isn't a number, or when started without a statement file | Console only | Business | Ops | Start it by dragging the statement onto `launcher.exe`. Fix blank or text amounts in column B (section 3), re-run | Statement is clean and it still stops: send the statement's layout (not the data), time |
| E09 | Run seemed to finish but no result file in `data\output`; entries **are** in LedgerPro | The `data\output` folder is missing, so the final save fails **after** all posting | Console only; log has every line | System | Ops | Create the `data\output` folder. **Don't re-run blind**: use the log lines (reference + Posted) to see what was done; re-run only after deleting the old export, so posted lines show OK | Folder exists and it still happens: send console text, log |
| E10 | Nothing starts at all; or bot started from `launcher.exe` shows an error and closes | `logs` folder missing (bot and launcher fail on start), or the update download/unzip failed | Console only | System | Ops, then Dev | Create the `logs` folder in the bot folder, re-run | `logs` exists and it still fails: send console text, time. Possibly a broken update (section 10) |
| E11 | Bot stops in the middle with a "fail-safe" message in the console | The mouse was moved into a screen corner (this is also the manual stop) | Console only; no result file | System | Ops | Hands off the mouse (section 4). Check LedgerPro for lines already posted, delete the old export, re-run | Never |
| E12 | One laptop behaves differently from the others after a release (old behaviour, old errors) | The launcher hides any update failure (network or certificate) and runs the old build | Nowhere | System | Dev | Report it | Always: send laptop name, date of the release, time you started the bot |
## 8. What the bot recovers by itself
| Problem | What the bot does | Limit | After the limit |
|---|---|---|---|
| A LedgerPro button isn't on screen yet | Looks for it again every second | 20 s per button | "New entry" / "Save": line = Error, next line. "Export": the run stops |
| LedgerPro slow to open | Nothing: fixed 15 s wait | 15 s | Export button not found (E05) |
| Export slow | Nothing: fixed 10 s wait | 10 s | Missing or stale export (E06) |
| Update server unreachable | Runs the current build silently | 10 s | Old version keeps running (E12) |
Nothing else is automatic: no re-try of a line, no restart of LedgerPro, no resume. A re-run always starts from the top; lines posted earlier show as OK **only if the new export is fresh**.
## 9. Known incidents
No dates were recorded; newest first by commit order.
| When / version | What ops saw | Cause | What to do now | Ops could handle? |
|---|---|---|---|---|
| 3.2.0 (latest fix) | Probably every posting failed with "not found" in the packaged exe | The button pictures weren't packed into the exe build | Treat "every Missing line is Error" as a bad build: escalate (E03). The fix is not visible in the repo. | No |
| 3.2.0 | Lines Error "Button save_btn.png not found on screen", entries not posted (seen on 2026-09-05) | Laptops at 125% display scaling; buttons were captured at 100% | **Keep 100% scaling and 1920 x 1080** on every bot laptop (section 3). Not fixed in code: it's a rule. | Yes |
| 3.2.0 | Two laptops never got the update, with no error | Update server's certificate not trusted; the launcher hides the error | Report version mismatches (E12). Not fixed in code. | Partly |
## 10. Releases and version check
- How an update arrives: `launcher.exe` checks the update server on every start, and if the server's version differs, downloads it and unzips it over the bot folder, then starts the bot.
- How to confirm it applied: **there is no on-screen version today** (the bot has no window or title bar). Ask dev on #rpa-support to confirm the build after each release; until dev adds a version display, compare the date of `LedgerBot.exe` in the bot folder with the other laptops.
- If the version did not change: post in #rpa-support with the laptop name. Don't copy files from another laptop by hand unless dev tells you to.
- Every bot laptop should run the same version.
## 11. Escalating to dev
Post in **#rpa-support** (Teams). Send: the exception ID from section 7, the time the run started and failed, the laptop name, the log file `logs\ledger_<date>.log`, any `logs\notfound_<button>.png`, a photo or copy of the black console window's last lines, and the result Excel name if there is one.
Never send: passwords, the `.env` file, or bank statement or customer data in chat. Refer to lines by reference only if dev asks.
## 12. Failure-class check
- F1 covered in section 3 (shortcut, `logs` and `data\output` folders, statement file) and E01, E08, E09, E10; packaged-exe-only bug in section 9 (exe-assets incident) and Fix #5.
- F2 covered in section 4 (hands off, LedgerPro in front) and E11; no window check is Fix #3.
- F3 covered in E03, E04, E05, E06, section 3 (100% scaling, 1920 x 1080) and section 8; fixed waits are Fix #4.
- F4 covered in section 3 input rules (numeric amounts, unique references, references as text) and E08; Fix #8.
- F5 covered in E09 (posted but no result file); missing positive check is Fix #1.
- F6 covered in E04 and E07 (check LedgerPro before re-running); Fix #1.
- F7 covered in section 6: "Missing - Posted" stays "Missing"; Mismatch needs follow-up; Fix #2.
- F8 not applicable: the bot has no auto-rerun loop and no watchdog; a frozen LedgerPro just makes each line fail after 20-40 s (section 4).
- F9 covered in section 4 (close the console window; closing LedgerPro doesn't stop it) and E11.
- F10 not applicable: there are no batches, queues or locks; each run is independent. The nearest risk (stale export causing duplicates) is E06.
- F11 covered in section 5; causes that only reach the console are Fix #6, overwritten screenshots Fix #7.
- F12 covered in section 10 and E12; Fix #5 and #9.
## 13. Fix before handover (dev team)
| # | Gap | Why it hurts ops | Where (file:line) |
|---|---|---|---|
| 1 | "Posted" is written right after clicking Save; nothing checks LedgerPro saved it | Unsaved entries look done; a crash mid-loop leaves no record of what was posted | `app.py:65-70`, `app.py:89-90` |
| 2 | Posted lines keep Result = "Missing"; the summary count "Missing" mixes posted and not posted | Ops can't tell done from not done at a glance | `app.py:86-93` |
| 3 | No check that LedgerPro is the active window before typing; no popup/login handling; failed Save leaves the form open and cascades into following lines | Keystrokes land in other windows; one failure spoils the rest of the run | `app.py:65-70`, `app.py:75-77` |
| 4 | Fixed 15 s / 10 s waits; export file's age is never checked | Slow laptop → crash, or reconcile against yesterday's export and post duplicates | `app.py:76-79` |
| 5 | The three "fix:" commits only added a comment; no code changed. `ledgerbot.spec` is three comment lines (assets `datas` commented out), so the real build spec isn't in the repo; no `assets/` change for the 100% recapture | The update and exe-assets incidents (section 9) can recur; builds aren't reproducible | `launcher.py:19-20,35`, `ledgerbot.spec:1-3`, `app.py:102` |
| 6 | Crashes (missing shortcut, Export not found, bad amount, missing folders, no argument) only go to the console; nothing logs errors or tracebacks; result Excel saved only at the end | After a crash ops has no file to look at; all row results lost | `app.py:24,34,79-81,96,101` |
| 7 | `notfound_<button>.png` has a fixed name, overwritten each time, not tied to a reference or time | Screenshot evidence is lost | `app.py:45` |
| 8 | Statement and export read into dicts: duplicate references silently dropped; numeric references become "5512.0" and may not match; non-numeric amount crashes the run | Lines skipped or posted twice without any message | `app.py:52`, `app.py:81` |
| 9 | Launcher swallows every update error (incl. self-signed certificate), installs on any version difference (can downgrade), no checksum, unzips over the live folder, doesn't log; launcher imports `app`, so a missing `logs` folder also breaks the launcher | Silent stale builds; a bad zip breaks the install | `launcher.py:9,15-29` |
| 10 | No version shown anywhere (the comment mentions a window title that doesn't exist) | Ops can't confirm an update applied | `app.py:15` |
| 11 | `logs` and `data\output` folders must pre-exist; neither is in the repo nor created by the code | Bot won't start, or loses the result file after posting | `app.py:24`, `app.py:96` |
| 12 | Exit code is 0 even when lines errored; summary printed to stdout only, and the launcher doesn't wait for or capture it | No way to alert on a failed run | `app.py:97`, `launcher.py:30` |
| 13 | Output name only to the minute | Two runs in the same minute overwrite each other | `app.py:96` |
| 14 | `.env` and `logs\` are committed to git (the log holds a customer name; `.env` is read by no code) | Data and secrets in repo history | `.env`, `logs/ledger_20260905.log` |
| 15 | Committed log line doesn't match the current code's Mismatch note, so it came from a different build | History in the repo can mislead | `logs/ledger_20260905.log:3` |
| 16 | pyautogui/pyscreeze versions are unpinned; newer versions raise an exception on "not found" instead of returning nothing, which would skip the 20 s retry and crash the run on the first miss (not tested) | Run could die on any slow screen | `requirements.txt`, `app.py:40` |
| 17 | Mouse-corner fail-safe stop isn't caught; no clean stop that saves partial results | Every stop loses the result file | `app.py:86-96` |