VasanthandClaude Opus 5.5
3942fad6cd
The skill scans a bot with subagents (never runs it) and writes HANDOVER.md, an 11-section ops guide in the operator's order: where is what, how to run, how long to wait, after the run, how to see an exception, exception list, how you will know, how to tell us, don'ts, issues seen before. Dev material (check against 12 failure classes from Tebra Auto Post, gaps with file:line, incidents) goes to HANDOVER-DEV.md. Also adds ADR-0004, glossary terms, five fixture bots and a handoff eval runner (--resume, --regrade) with results from 47 runs of the earlier 13-section version. The grader still expects that structure and must be updated before the next eval run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
64 lines
6.1 KiB
Markdown
64 lines
6.1 KiB
Markdown
# Failure classes
|
|
|
|
Twelve kinds of problem Tebra Auto Post hit in its first months with ops. Check every new bot against each one. For each class: what to look for in the code, and where the ops guide (`HANDOVER.md`) or the dev notes (`HANDOVER-DEV.md`) must cover it. Section numbers are the ops guide's.
|
|
|
|
## F1. Missing prerequisite on the ops machine
|
|
**Tebra:** a missing database driver gave a cryptic error only on some laptops; a missing settings file made the duplicate check fail silently; a helper app's shortcut wasn't where the bot searched; some bugs appeared only in the packaged exe, never under `uv run`.
|
|
**Look for:** drivers, `.env` / settings files, shortcuts or install paths the bot searches, `sys.frozen` branches.
|
|
**Doc:** the "Before you start" checklist in section 3, and the exact message each missing item produces in section 7.
|
|
|
|
## F2. The bot grabs the wrong window, or can't take focus
|
|
**Tebra:** it matched its own window, and a database tool whose title held the target app's name, then closed the real app. Windows' foreground lock meant clicks landed elsewhere when someone used the machine during a run.
|
|
**Look for:** window matching by title substring, focus / foreground calls, exclude lists.
|
|
**Doc:** section 10: which windows must not be open, and hands off the machine.
|
|
|
|
## F3. The target app is slower, or different, than the bot expects
|
|
**Tebra:** a screen took about 98 s under load against a 90 s timeout; two screens shared a header and were confused; controls drawn without accessibility elements; a long list put the target off-screen; a wrapped name was clicked between its lines on a small screen.
|
|
**Look for:** every timeout value, screen-identification checks, "element not found" paths, screen-size assumptions.
|
|
**Doc:** a section 7 row for each "not found / did not load" message, and the wait from section 4. Screen-resolution rules go in section 3.
|
|
|
|
## F4. Input data the bot reads differently than ops writes it
|
|
**Tebra:** names containing commas were split into pieces and silently skipped; a `&` in a name broke matching; Excel dates arrived as serial numbers; a new client with no mapping entry got the wrong file prefix.
|
|
**Look for:** split / separator logic, name matching, date parsing, mapping or config files ops must maintain.
|
|
**Doc:** input rules in section 3, each with one wrong and one right example. Point at the sample input file if the bot has one.
|
|
|
|
## F5. Marked done when nothing was done
|
|
**Tebra:** a location was marked Success with 0 items when the bot couldn't confirm it was on the right page; a company was marked Success after a crash.
|
|
**Look for:** where success / complete is set, and whether it checks a positive count or a confirmation first.
|
|
**Doc:** if a guard exists, a section 7 row for the "left Pending" message. If not, a gap in `HANDOVER-DEV.md`.
|
|
|
|
## F6. The action happened but the bot couldn't confirm it
|
|
**Tebra:** "Payment not confirmed after Save" - the save took longer than the wait, so the row showed Error while the payment really had posted. Re-running risked posting twice.
|
|
**Look for:** waits after a commit action (save, submit, send), and what the bot writes when the confirmation times out.
|
|
**Doc:** a section 7 row that tells ops, in bold, to **check in the target app before re-running**, and how to correct the record afterwards.
|
|
|
|
## F7. A business exception shown as success
|
|
**Tebra:** "ERA Errors on posting" and "negative balance - not posted" rows showed green (Completed) although a person had to follow up.
|
|
**Look for:** status mapping tables; any comment that means "not done" stored under a success status.
|
|
**Doc:** mark these "not done, needs follow-up" in the section 5 status table, give each a section 7 row, and add the mapping to the gaps in `HANDOVER-DEV.md`.
|
|
|
|
## F8. Endless re-runs, or killed by the watchdog
|
|
**Tebra:** practices with no items re-ran the same company forever; already-processed rows were re-walked slowly after a restart until the watchdog killed the run at exactly 300 s, again and again.
|
|
**Look for:** auto-rerun loops and their limits, watchdog timeouts, "0 items" handling, heartbeats.
|
|
**Doc:** every limit in section 4, and in section 7 the signs of a loop ops should report ("the same company restarts more than N times").
|
|
|
|
## F9. Stopping the bot the wrong way
|
|
**Tebra:** closing the app window didn't stop the bot, which kept driving the target app; closing the target app doesn't stop it either.
|
|
**Look for:** how the child process ends (process-tree kill, parent-death watch), and any Stop button.
|
|
**Doc:** the one right way to stop it, in section 10.
|
|
|
|
## F10. One stuck item blocks the queue
|
|
**Tebra:** one Error row kept a whole batch Pending for weeks; a run that died in its first seconds locked the upload all day. Ops didn't know which row blocked it, or that yesterday's Pending batch never blocks today's upload.
|
|
**Look for:** completion gates (what must be true for a batch / company to complete), locks on new uploads, resume logic.
|
|
**Doc:** in section 7: what blocks what, how to find the blocking item (section 6), and how to release it (edit, discard, clear).
|
|
|
|
## F11. The cause is hidden from ops
|
|
**Tebra:** the real cause was only in a database column the Report tab didn't show, or in the raw log; the friendly on-screen log hid errors; some app messages went to the screen only, never to a file.
|
|
**Look for:** columns written but never displayed, log filters, messages printed only to the UI.
|
|
**Doc:** sections 2 and 6 point only at places ops can open (question 2). Every hidden cause becomes a gap in `HANDOVER-DEV.md`.
|
|
|
|
## F12. The update didn't arrive, or an old build undid a new one
|
|
**Tebra:** the launcher failed silently on a self-signed certificate, so users stayed on an old version; the server reported a deploy as done without swapping the container; a stale zip was shipped; each start re-deployed database procedures, so a machine on an old build overwrote newer ones.
|
|
**Look for:** update / launcher code, version display, anything the app writes to shared resources on start.
|
|
**Doc:** section 3 (check the version before a run) and section 11. Add "every machine must run the same version" when shared resources are rewritten on start.
|