Files
Rpa_Skills/tests/evals/results/handoff/h-packaged-desktop-full-r1.json
VasanthandClaude Opus 5.5 3942fad6cd Add rpa-bot-handoff skill: ops guide for bot handover
The skill scans a bot with subagents (never runs it) and writes
HANDOVER.md, an 11-section ops guide in the operator's order: where is
what, how to run, how long to wait, after the run, how to see an
exception, exception list, how you will know, how to tell us, don'ts,
issues seen before. Dev material (check against 12 failure classes from
Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md.

Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval
runner (--resume, --regrade) with results from 47 runs of the earlier
13-section version. The grader still expects that structure and must be
updated before the next eval run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 15:01:42 +05:30

32 lines
3.2 KiB
JSON

{
"case": "h-packaged-desktop/full",
"rep": 1,
"passed": false,
"checks": {
"HANDOVER.md written": true,
"no other file changed": true,
"bot never run": true,
"no placeholders left": true,
"13 sections in order": true,
"used subagents": true,
"every status listed": true,
"catalogue: 4+ complete rows": true,
"F1-F12 all accounted for": true,
"no code refs in ops sections": true,
"fix-before-handover has file refs": true,
"no secrets or personal data": true,
"where-to-look respects ops access": true,
"asks escalation": true,
"asks ops access": true,
"known incidents from history": true,
"display scaling rule": true,
"version check": false,
"shortcut prerequisite": true,
"silent update failure flagged": true
},
"cost_usd": 1.3562872000000001,
"subagents": 2,
"work_dir": "C:\\Users\\ec10866\\AppData\\Local\\Temp\\handoff-h-packaged-desktop-full-hy65g4yp\\project",
"reply": "I wrote `HANDOVER.md` in the repo root for **LedgerBot v3.2.0**, and changed no code. I only read code, history and logs; I never ran the bot.\n\n| Check | Result |\n|---|---|\n| Sections | 13, same order as the template |\n| Exceptions catalogued | 11 (E01\u2013E11); every Dev row says what to send in #rpa-support |\n| Failure classes | 10 covered, 2 not applicable (F8: no auto-rerun or watchdog; F10: no batches or locks) |\n| Statuses | All 6 Result + Note combinations listed; the file has no colours |\n| Sensitive data | No customer data, passwords, internal addresses or `.env` values; code references only in section 13 |\n\n**Fix before handover (14 items for dev):**\n1. The launcher still hides update errors, so the 30cb604 \"fix\" only added a comment.\n2. \"Posted\" is written without confirming the save worked.\n3. Crash causes are never logged.\n4. All results are lost if a run stops part-way.\n5. Screenshots overwrite each other, and the run summary is never saved.\n6. No check that LedgerPro has focus; a failed Save leaves a half-filled form.\n7. Fixed waits instead of real checks, and nothing checks that the LedgerPro export is fresh, which can post entries twice.\n8. Duplicate or blank references silently drop out of the results.\n9. \"Missing\" still shows on rows that were posted.\n10. The version is not visible anywhere, and the build spec is a commented-out stub.\n11. No display-scaling check at startup.\n12. `.env` with credentials is committed to git.\n13. An old log contains a customer name.\n14. pyautogui is not pinned.\n\n**Where I used my own reading of the code:**\n- I couldn't find who saves `ledger_export.xlsx` into `data/`. The doc tells ops to confirm it with dev.\n- The bot has no window, so the doc tells ops to check the version by the Date modified of `LedgerBot.exe`.\n- Ops can't open the `data` folder, so they can't check the export themselves before a run.\n\nQUESTIONS: Escalation: who does ops call for a dev issue (team or role), and how (ticket, chat channel)? | Ops access: which of the bot's places can ops open - the app's screens, the output folder, the log folder, the database? | Unknowns: what do the statuses or exceptions mean where the code doesn't make it clear?",
"stderr": ""
}